Assessment rounds

An assessment round sends a grant round's submitted applications to assessors to score. Assessment Round, Assessment, Assessor and Application are your organisation's own terms if you've renamed them.

Create an assessment round

Select Assess > Assessment rounds, then New Assessment Round.
Choose what to set up:
- Assessment Round: choose which submitted applications to assess, and which assessors assess each.
- Singular Assessment: assess one application by one or more assessors.
Work through the steps:
- Details: the Title, an optional Description assessors see, and the Grant the round is for. Changing the grant later clears every selected application and assignment.
- Application: select the submitted applications to assess.
- Assessors: include administrators and assessors, then Assign each one their applications, or share them out with Assign by rule. See Assignments.
- Template: choose the assessment form. See Assessment templates.
- Workflow: turn on Assessors declare any Conflict of Interest before assessing and choose what they see when they declare, and turn on Decision to Assess workflow so assessors accept or decline each assessment first (choose its Decision form). These cannot be changed once the round is published. Turn on AI assessor aids so assessors can ask for an AI summary, evidence quotes, consistency, eligibility and budget checks, and possible duplicates. It's off unless you turn it on, it can't be changed once the round is published, and the aids are advisory: they never change a score. The switch is disabled until assessor aids are switched on in your organisation's AI assistance settings. Under Concealed information, turn on Blind review to hide what identifies the applicant from assessors. See Blind review. Under Assignment acceptance, turn on Require assignment acceptance so each assessor accepts or declines an assignment before they can open it. Set Accept within (days) (1 to 30, 5 unless you change it), counted from the day the assessors are notified, and turn on Reassign automatically to give a declined or lapsed assignment to the next eligible assessor in the pool. Accept within (days) is disabled while acceptance is off. See Assignments. Under After submission, set Allow changes for N minutes after submission (0 to 60) so an assessor can unsubmit and change their assessment for a while, turn on Assessors may amend until Close to let them amend it with a note until the close date, and turn on Allow bulk download.
- Dates: optional Open, Close and Due dates. After the close date assessors can no longer edit or submit.
- Publish: check the Summary and select Publish Assessment Round.

If something must be fixed first, Backed lists it under Fix these before publishing. Each assessor is told once the round is published.

Follow a round

Open a round from Assess > Assessment rounds with View. Its View tab shows the details, the workflow settings fixed at publishing and the next steps; its Assessments tab lists every assessment with its status. Under Assignment acceptance, the View shows the round's acceptance settings and, for each assessor, how many of their assignments are Awaiting, Accepted, Declined and Lapsed. The Assessments tab can show an Acceptance column and filter by it. AI assessor aids shows how many assessments used AI aids, such as 3 Assessments used AI aids, and whether the round offers them.

The header of a published round shows how many of its assessments are assessed, such as 3 of 8 Assessed. Assessments that were declined or have a conflict of interest don't count. The number updates when you come back to the tab, so you don't need to refresh the page after an assessor submits.

Open an assessment to see its Assessor, Status, Due date, Total Score and the scores per question with their weights, and the Conflict of Interest and Decision to assess answers.

Turn on Blind review under Concealed information on the Workflow step of a round that isn't published yet. Each change saves straight away.

Backed adds the applicant's registration details and every field it detects as identifying (an entity name or address field, or a field tagged as applicant identity). These are marked Detected.
To add any other field, choose it under Add a field and select Add. To remove a field you added, select Remove. To remove a Detected one, select Remove and type REMOVE to confirm; the removal is recorded in the audit log.
Choose how assessors see uploads: Include attachments unchanged, Hide attachments, or Anonymise attachments, which is chosen when you turn blind review on. Anonymised uploads are named Attachment 1, Attachment 2 and so on, and author details are removed from PDF, Word, Excel and PowerPoint files. Other files are renamed only. Your own downloads are unchanged.
Under Anonymised Applications, blind review covers Every Application in the round unless you choose Only the Applications I choose and tick the applications to anonymise. Every other application reaches its assessors un-anonymised. At least one stays ticked; to cover them all again, choose Every Application in the round.
Under Conflict of Interest declarations, choose what assessors see when they declare. Keep the applicant anonymised (the default) shows the application's title or reference number, and you cross-reference who the applicant is. Show who the applicant is shows the applicant's name and the list of collaborators, where the round's Conflict of Interest settings show them. Once an assessor declares no conflict, everything else stays anonymised.
Select Preview anonymised Application to see a selected application exactly as an assessor of the round will, then Next and Previous to move between them. An application outside the ones you chose says it reaches its assessors un-anonymised.

Once the round is published its concealed information, the applications it covers and the Conflict of Interest approach can't change, and what's concealed stays hidden after the close date. To show it to assessors, open the round's View and select Lift concealment, then give a reason. Lifting can't be undone, is recorded in the audit log, and doesn't email anyone. The View then says who lifted it and when.

Review panels

A panel is a group of the round's assessors with one lead assessor. The lead sees the members' scores and comments on an application once the lead has submitted their own, and in a consensus panel records the panel's Pass or Fail for it. Open the round's Panels tab.

Select Create panel. Enter a Panel name and tick its Members from the round's assessors.
Choose the Lead assessor. Only ticked members are offered; untick the lead and you choose again.
Choose which applications it reviews: every application in the round, by Funding Category, or by application tag.
Choose the Mode: Scored, where members score as usual, or Consensus pass or fail, where the lead records one outcome per application once every member without a conflict has submitted. Tick Share scores within a panel after submission to let members, not only the lead, see each other's scores after submitting their own.
Set the panel's Assignment rule, then select Create panel.

A panel's rule assigns only its members to its applications. Open the panel from the list and select Apply rule: Backed shows the assignments first and makes them when you select Apply. Then notify the assessors from Change Assignments.

The panel's page shows, for each application, every member's status and Total Score, the mean, the range and the outcome. A row is highlighted and marked Divergent when the members' scores on any scored question differ by more than the Divergence threshold (2 unless you change it); change the threshold to see the table again at a new one. If the lead has a conflict on an application, select Name deputy lead on its row and choose one of the panel's members to record its outcome instead. Only members who are assessing that application are offered, because the deputy records the outcome from their own Panel tab once their own assessment is submitted.

Edit panel changes any of the panel's settings. Delete panel keeps every member's assessments, archives the panel's rule so it can no longer be applied, and deletes the panel's recorded outcomes after recording each one in the audit trail.

Rank and shortlist

A round's Leaderboard tab ranks its applications by assessment score, highest first, with a panel's Pass above Fail. Each row shows the rank, the score, how many assessments are submitted of those assigned, the score range, Requested Funding, the cumulative requested funding in rank order, the application's tags and where it sits against the cut line. Choose a Funding Category to rank only its applications.

Under Cut line, choose Cut by: Rank (top number), Minimum score or Budget (Requested Funding), and enter the number, score or amount. The table shows where the line falls as you type; the line is a preview until you select Save cut line.
Applications that share a rank at the line are marked Tie and listed together at the line. Ties are never broken: they are not shortlisted.
Applications with assessments still pending are marked Incomplete and listed below the line. Tick Include incomplete to rank them with the rest.
Name the Shortlist tag (shortlisted unless you change it) and save the line.
Select Shortlist above the line. Backed shows how many applications are above the saved line and the tag they get, and tags them when you select Shortlist.

Use Remove from shortlist on a row to take the tag off one application. Export downloads the ranked list as a CSV, with the cut line in its first row. The shortlist tag can then be used to choose applications in a later stage's Select Applications and a decision's Applications step.

Draw an equal-merit band

When the cut line is a budget and applications tied on score straddle it, the Leaderboard tab shows Equal-merit band. Applications ranked above the band are funded first; a recorded random draw decides which applications in the band get what the budget leaves.

The band's Band floor (score) and Band ceiling (score) are filled in with the tie that crosses the budget line. Change them to widen the band; the panel lists the applications in the band, their combined Requested Funding and what the budget leaves.
Save the budget cut line, then select Draw. Backed shows the band, the remaining amount and the seed. Leave Seed (optional) blank for Backed to generate one, or enter a seed announced in advance. Tick the confirmation and select Draw.
Backed orders the band by the SHA-256 of the seed and each application's number, and selects, in that order, each application whose Requested Funding fits what remains. Selected applications get the shortlist tag and are marked Lottery on the leaderboard; the rest are marked Not drawn.

The standing draw shows the band, the budget, the seed, who drew it and when, and the band in draw order with each result. Export draw downloads it as a CSV from which anyone can recompute the order. If an application in the band is rescored after the draw, the draw is marked Stale and names it; the draw is never run again on its own. Void draw, with a reason, is the only way to draw again: it removes the lottery's tags and keeps the voided draw on record.

Draw and Void draw are shown only to administrators who can also tag the round's applications.

Export and Export draw also need Can generate reports or Can generate document reports, so someone who can only view assessment rounds can't take a copy of the ranking or the draw.

Compare assessors' scores

A round's Scoring matrix tab has one row for each application and one column for each assessor. Each cell shows that assessor's Total Score once they've submitted their assessment. A cell is blank until then. A cell shows COI when the assessor declared a conflict of interest, or the conflict was confirmed. The application's Assessment Score is at the end of the row, and it never counts an assessment with a conflict.

Select Show criterion scores on a row to see each submitted assessment's score for each scored question, with the best possible score under the question. Only each assessor's current scores are shown.

An application is marked Divergent when two assessors' scores for the same question differ by more than the Divergence threshold. The sentence under it names the questions, and each of those scores is marked Flagged. The threshold starts at 2 points. Change it and the matrix updates. Choose a Funding Category to show only its applications.

Select Export CSV to download the matrix as it is on screen: one row for each application, one column for each assessor, and the Assessment Score.

See a round's summary

A round's Summary tab counts its assessments by status, from Pending Assessment to Conflict Confirmed, with the total. Below that, each application shows its status (Assessed, In progress or Not started), how many of its assessments are assessed, and its Assessment Score. Select Export CSV to download the summary.

Export CSV on both tabs also needs Can generate reports or Can generate document reports. The Scoring matrix also needs Can access Assessments.

Compare scores across a grant

Assess > Assessments compares the scores of a grant round's applications. Choose a Grant, select Search, then sort by number or by score. Show an application's assessments to see each assessor's score. Choose an Acceptance state to list only the applications with an assessment in that state — Declined and Lapsed include assignments that have since been removed.