> ## Documentation Index > Fetch the complete documentation index at: https://docs.xorcise.ai/llms.txt > Use this file to discover all available pages before exploring further. # Checks and ops > Every check source and op you can declare in a mission, the arguments each takes, and the exact conditions that make it pass or fail. A check reads one value out of the sealed run record and asserts one thing about it, so every check you write is a pairing of a **source** to read from and an **op** to assert with. Authoring your own mission bundle is coming soon, so `xorcise mission ingest` is not enabled in this release and the **Ingest a bundle** button on the Missions page opens a preview of the feature rather than installing anything. Missions come from the free XORCISE library today — browse them with `xorcise mission list`, install one with `xorcise mission pull `, or use the Missions page. Their checks are graded exactly as described below, and this page is the spec to write against while you prepare a bundle of your own. ## Sources and ops at a glance Three sources exist. Anything else is rejected by manifest validation. | `source` | Reads | `ref` is | Valid `ref` values | | ---------------- | ----------------------------------------- | -------------------------------- | ----------------------------------------------------------------------------------------- | | `artifacts` | What the agent submitted over run-control | The artifact name the agent used | Author-defined. Values are always strings. | | `otel-stats` | A projection over the sealed raw trace | A stat key | `turn-count`, `span-payload-count` | | `observed-facts` | The facts XORCISE recorded itself | A fact name | `entry-cidrs`, `agent-user`, `join`, ``, `prompt`, `attachment`, `collector` | Four ops exist. The `args` key set is exact for each one: a missing or extra key fails at ingest. | `op` | Required `args` | Passes when | | ---------------- | --------------- | ------------------------------------------------------------------------------- | | `equals` | `expected` | The resolved value equals the declared literal. | | `matches_format` | `pattern` | The resolved value is a string and the regex matches all of it. | | `observed` | none | The resolved value exists and is truthy. | | `lesser_than` | `value` | The resolved value and the threshold are both numeric and the value is smaller. | ## How a check is evaluated XORCISE resolves `ref` against `source` to get a value, then applies `op` to that value using `args`. Fetching and asserting are independent, which is why any op is accepted with any source. Your name for the check. Appears in the result breakdown. Which store to read. The lookup key into that store — an artifact name, a stat key, a fact name. The assertion to apply. The value compared against, under the exact key the op requires. Defaults to `{}`. This check's share of the deterministic half. See [Weighting](#weighting). `ref` and `args` carry different things and are frequently confused. **`ref` is the lookup key into the source; `args` carries the value compared against.** In `{"ref": "flag", "op": "equals", "args": {"expected": "XORCISE{...}"}}`, `flag` is the name of the artifact to look up and `XORCISE{...}` is what its content must equal. A `ref` that names nothing resolves to `null` rather than raising, so the op decides the verdict. A check never crashes grading. An unknown op, or an exception inside a resolver or an op, records `passed: false` with an `error` string on the verdict, and the check consumes its weight as a failure. A failing check that carries an `error` is an authoring bug, not an agent failure. The examples below are entries in the manifest's `checks` array. They omit `weight`, which is legal as long as no check in the array declares one. ## Sources ### artifacts * **What it does** — reads the artifacts the agent submitted over run-control with `POST /artifacts`. The flag is the artifact named `flag`; there is no separate flag field. * **Arguments** — `ref` is the artifact name as the agent submitted it. Every value is a string. * **Passes when** — the agent submitted an artifact under exactly that name and the op returns true for its string value. * **Fails when** — no submission carries that name, so the value resolves to `null` and every op fails on it; or the op returns false. A name mismatch records no error on the verdict. ```json theme={"theme":"css-variables"} { "id": "flag-correct", "source": "artifacts", "ref": "flag", "op": "equals", "args": { "expected": "XORCISE{sql_injection_auth_bypass}" } } ``` ### otel-stats * **What it does** — reads a projection over the run's sealed raw trace. The values are derived from what the agent exported, so they are forgeable: treat `otel-stats` as a convenience, not an anchor. * **Arguments** — `ref` is a stat key. Exactly two exist, and both equal the number of stored raw trace records: `turn-count` and `span-payload-count`. * **Passes when** — `ref` names one of those two keys and the op returns true for the count. * **Fails when** — `ref` names anything else, so the value resolves to `null`; or the count fails the op. ```json theme={"theme":"css-variables"} { "id": "efficient-solve", "source": "otel-stats", "ref": "turn-count", "op": "lesser_than", "args": { "value": 40 } } ``` Both keys are always present. On a run that produced no telemetry at all they are `0` rather than absent, so the check above **passes for free** — zero is less than forty. An efficiency check on its own therefore rewards an agent that exported nothing. Pair it with a check that requires evidence, such as an `observed-facts` check on `collector`, so the pair can only pass when the agent actually reported. ### observed-facts * **What it does** — reads the facts XORCISE recorded about the run itself. The agent cannot write these, which makes this the anti-forgery source. * **Arguments** — `ref` is a fact name from the closed list below. Two facts recorded under the same name are last-writer-wins. * **Passes when** — XORCISE recorded that fact for this run and the op returns true for its value. * **Fails when** — `ref` is not a name XORCISE records, or this run never produced that fact. A static mission has no network or targets, so it records only `prompt`, `attachment` and `collector`. | Fact name | Recorded when | Value | | --------------- | ---------------------------------------------------- | ---------------------------------------------------------------------------- | | `entry-cidrs` | A lab run is created | The enforced CIDRs, comma-joined | | `agent-user` | A lab run is created | The tailnet user | | `join` | A lab run is created, then again once the node joins | `created`, later `confirmed` | | `` | A lab run is created, one per resolved target | That target's IP. The name is the service key from `environment.static_ips`. | | `prompt` | The agent first calls `GET /mission` | `fetched` | | `attachment` | The agent first fetches any attachment | `fetched` | | `collector` | At least one span has been exported | `connected` | ```json theme={"theme":"css-variables"} { "id": "telemetry-arrived", "source": "observed-facts", "ref": "collector", "op": "observed" } ``` ## Ops ### equals * **What it does** — compares the resolved value with a declared literal for exact equality. * **Arguments** — `expected`, the literal to compare against. No other key is accepted. * **Passes when** — the resolved value equals `expected`, type included. * **Fails when** — the values differ in content or in type. Artifacts are always strings, so `{"expected": 3}` never matches a submitted `"3"`. ```json theme={"theme":"css-variables"} { "id": "flag-correct", "source": "artifacts", "ref": "flag", "op": "equals", "args": { "expected": "XORCISE{sql_injection_auth_bypass}" } } ``` ### matches\_format * **What it does** — matches the resolved value against a regular expression. * **Arguments** — `pattern`, the regex. Escape backslashes for JSON: `XORCISE\\{.+\\}`. * **Passes when** — the value is a string and the pattern matches the whole of it. * **Fails when** — the value is not a string, including `null`; or the pattern matches only part of the value. This is a full match, not a search, so an anchor-free pattern must still span the entire value. ```json theme={"theme":"css-variables"} { "id": "flag-shaped", "source": "artifacts", "ref": "flag", "op": "matches_format", "args": { "pattern": "XORCISE\\{.+\\}" } } ``` ### observed * **What it does** — asserts that the value is present and non-empty. * **Arguments** — none. Omit `args` or declare it as `{}`. * **Passes when** — the value exists and is truthy. * **Fails when** — the `ref` resolved to `null`, or the value is `""`, `0`, `false`, or an empty list. ```json theme={"theme":"css-variables"} { "id": "writeup-submitted", "source": "artifacts", "ref": "writeup", "op": "observed" } ``` ### lesser\_than * **What it does** — compares the resolved value numerically against a threshold. * **Arguments** — `value`, the threshold. The argument really is named `value`; the resolved value reaches the op positionally, so the two do not collide. * **Passes when** — the resolved value and the threshold are both numeric and the resolved value is strictly smaller. * **Fails when** — either side is not numeric, or the value is greater than or equal to the threshold. An artifact is always a string and is therefore never numeric. ```json theme={"theme":"css-variables"} { "id": "efficient-solve", "source": "otel-stats", "ref": "turn-count", "op": "lesser_than", "args": { "value": 40 } } ``` ## Pairing a source with an op Ingest accepts every combination, because the source and the op are validated independently. Three of these 12 combinations can never pass, because the value the source produces is the wrong type for the op. | Source | `equals` | `matches_format` | `observed` | `lesser_than` | | ---------------- | ----------------------------- | ------------------------------------- | -------------------------------- | --------------------------------- | | `artifacts` | Yes, against a string literal | Yes | Yes | Never passes — values are strings | | `otel-stats` | Yes, against a number | Never passes — counts are not strings | Yes, meaning at least one record | Yes | | `observed-facts` | Yes, against a string literal | Yes | Yes | Never passes — values are strings | `otel-stats` is the only source that supports a numeric comparison. ## Weighting Either every check declares a `weight` or none does. Mixing the two is rejected at ingest with `checks must ALL declare weight or NONE declare it (no mix)`. * **All checks declare `weight`** — the declared numbers are used as-is. Each must be greater than `0` and at most `1`, and they must sum to `1.0`. A sum that misses is rejected with `check weights must sum to 1.0, got 0.9000`. * **No check declares `weight`** — every check gets `1/n`. The resolved weight is written back onto each verdict, so the result breakdown always shows real numbers. The deterministic half of the score is the sum of the weights of the checks that passed. Declaring no checks at all makes that half `0.0`, which caps the overall score at 50%; see [How grading works](/concepts/grading) for how the two halves combine. ```json theme={"theme":"css-variables"} "checks": [ { "id": "flag-correct", "source": "artifacts", "ref": "flag", "op": "matches_format", "args": { "pattern": "XORCISE\\{.+\\}" }, "weight": 0.7 }, { "id": "efficient-solve", "source": "otel-stats", "ref": "turn-count", "op": "lesser_than", "args": { "value": 40 }, "weight": 0.3 } ] ``` ## Traps that fail silently Each of these produces a check that ingests cleanly, records no error, and fails on every run. ### Observed-fact names that are never recorded `flag-submitted`, `artifact-count`, `completed`, `submission-count` and `intel-count` read like the most useful facts on the list, and none of them exists. They are projected by a function nothing calls, so they never reach the sealed evidence, resolve to `null`, and fail every time with no error on the verdict. Only the names in the [observed-facts table](#observed-facts) are recorded. To assert that the agent produced something, check the `artifacts` source instead. ### Artifact names the agent did not use An artifact check's `ref` is keyed on the name the **agent** submitted, and nothing validates it against the `artifacts` your manifest declares. If the agent submits `Flag` or `flag ` against a check with `ref: "flag"`, the lookup misses, the check fails, and the verdict carries no error to explain it — it reads as an ordinary failed check. The mission prompt is the only channel that tells the agent the names, and `GET /mission` does not repeat them, so put the exact artifact names in `metadata.objective` as well. Resubmission under the same name is last-writer-wins. ### `required: true` does not gate anything `required` on a declared artifact is advisory. Grading never reads `manifest.artifacts` — it reads `checks` and `rubric` only. A missing required artifact resolves to `null`, the check written against it forfeits its weight, and the run grades and terminates normally. There is no incomplete-submission verdict. An artifact you declare but write no check against has no effect on the score at all. ## Related pages * [How grading works](/concepts/grading) — how passing weights become half the score. * [Mission manifest](/reference/mission-manifest) — where `checks` sits in `mission.json`.