Skip to main content
One run tells you how an agent did once. Ranking, history and the score ledger tell you whether it is getting better, and which missions it keeps failing. Time: about 5 minutes · You need: at least two finished runs · Interface: CLI or web UI

​
What gets aggregated

Every run that has reached terminal and carries a recorded result is rolled up per agent and per mission. Six figures come out of that: the number of runs, how many were scored, the average and best overall score, the completion and partial rates, and when the agent last ran. One rule shapes everything you read here: partial runs are excluded from the score averages but still counted in the totals. A partial run is one that did not end on the agent’s own terms — it hit its budget, or you stopped it. Those runs are real and they count against the completion rate, but scoring an agent on a run that was cut short would punish it for the clock rather than for its work. An agent with a high average and a low completion rate is not doing well; it is finishing rarely and being scored only on the runs it finished.

​
1. Rank your agents

You should see one row per agent that has finished at least one run.
Output
With no finished runs the CLI prints no finished runs yet — nothing to rank, and --json returns [] rather than prose, so a script can branch on an empty list.
The XORCISE Performance page showing per-agent cards ranked best first, each with runs, average, best, completion and partial rates

The Performance page. Average and Best exclude partial runs; Completion and Partial tell you how often the agent finished on its own terms.

​
2. Read one agent’s history

Ranking tells you where an agent sits. History tells you which way it is moving.
You should see that agent’s results in time order, so a trend is visible at a glance. An agent record survives being updated and renamed — its id, versions and runs are kept — so history spans the whole life of the agent rather than restarting when you change its declaration.
The one exception is xorcise agent rm, which deletes the agent’s runs and results along with it.

​
3. Compare by mission instead of by agent

The same results grouped the other way answer a different question: not “which agent is best” but “which mission is beating everyone”. The Missions tab on the Results page carries the identical tiles, grouped by mission. A mission where every agent’s average sits far below its best is usually a mission with one hard gate rather than a uniformly difficult one. This is not available from the CLI.

​
4. Export the ledger

The All Runs tab is a dense table of every run: agent, mission, status, overall, deterministic, judge, and date. Ungraded and still-running runs show — rather than a zero, so an unfinished run never reads as a failure. For the same data as JSON:

​
Comparing scores without fooling yourself

Three things make two scores incomparable, and all three are easy to miss. A run with no judge configured is capped at 50%. The judge half scores 0.0 and is still averaged in. If you configured a judge partway through a series, every run before that point is on a different scale — the earlier ones are not worse, they are half-scored. Re-evaluate the older runs against the sealed evidence rather than re-running them, and the series becomes comparable. Deterministic and judge halves move for different reasons. A rising overall score built entirely on the deterministic half means the agent is passing more checks; one built on the judge half means a model’s assessment of its method changed. The ledger shows both columns separately for exactly this reason. Read them, not just the overall. Averages exclude partial runs, totals do not. An agent whose average climbs while its completion rate falls is being scored on a shrinking, self-selected sample of its own best runs.

How grading works

Where each half of a score comes from, and how to re-evaluate a sealed run.

CLI reference

Every flag on leaderboard, agent history, and the JSON output they emit.

Evaluate Agents with Playbooks

Comparing models rather than tracking one agent — many models over the same missions, under a fixed harness. Experimental.