> ## Documentation Index
> Fetch the complete documentation index at: https://docs.xorcise.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Troubleshooting
> Symptom-first fixes for the failures XORCISE actually produces at start, during a run, while grading, and when pulling missions.
Find the line you are looking at in the table below, then read its entry. Two commands answer most triage questions before you get that far: `xorcise doctor` checks this host's prerequisites and prints an exact fix per failure, and `xorcise status` probes the REST plane, the OTLP plane, and the Docker daemon.
## Find your problem
| Symptom | Likely cause | Section |
| --------------------------------------------------------- | --------------------------------------------------- | ------------------------------------------------- |
| `Cannot connect to the Docker daemon` | Docker is not running | [Startup](#startup-problems) |
| `permission denied` on the Docker socket | Your user is not in the `docker` group yet | [Startup](#startup-problems) |
| `docker compose` reported missing | Only the legacy v1 `docker-compose` is installed | [Startup](#startup-problems) |
| `openssl` reported missing | XORCISE generates the local TLS certificate with it | [Startup](#startup-problems) |
| `port 3001 (rest) is busy — using 3002 instead` | Nothing. XORCISE moved and told you | [Startup](#startup-problems) |
| `already running — UI at …` | Nothing. `up` is idempotent | [Startup](#startup-problems) |
| `database is behind the schema` | The database has data from an older build | [Startup](#startup-problems) |
| `migration history mismatch` | The database was stamped by a different build | [Startup](#startup-problems) |
| `XORCISE did not become healthy in 10s` | The brain crashed during boot | [Startup](#startup-problems) |
| The web UI loads blank or unstyled | A stale static export, in a source checkout | [Startup](#startup-problems) |
| The run sits on **Awaiting agent** | No telemetry has arrived yet | [Runs and telemetry](#run-and-telemetry-problems) |
| No traces arrive at all | Launch mode, or the env was not exported | [Runs and telemetry](#run-and-telemetry-problems) |
| `partialSuccess.rejectedSpans` in the exporter's response | The batch named no run, or the run is sealed | [Runs and telemetry](#run-and-telemetry-problems) |
| Spans arrived but the replay is empty | The logs exporter is not configured | [Runs and telemetry](#run-and-telemetry-problems) |
| The agent joined but cannot reach a target | No DNS on the tailnet, or userspace mode | [Runs and telemetry](#run-and-telemetry-problems) |
| The environment never becomes **Ready** | It failed to come up inside the readiness window | [Runs and telemetry](#run-and-telemetry-problems) |
| The result says the judge did not run | No judge model is configured | [Grading](#grading-problems) |
| The judge failed on a long run | The prompt exceeded the model's context window | [Grading](#grading-problems) |
| The result reads `grading` | Grading is still running | [Grading](#grading-problems) |
| Library missions do not appear | The library is disconnected or unreachable | [Missions](#mission-and-library-problems) |
| A pull sits at 0 bytes | It is in a phase that reports no bytes | [Missions](#mission-and-library-problems) |
| `Ingesting your own mission bundle is coming soon.` | Installing from a local bundle has not shipped yet | [Missions](#mission-and-library-problems) |
| A CI gate calls a healthy run a failure | Exit code `3` means still in progress | [Exit codes](#exit-codes-in-scripts) |
## Startup problems
**Cause.** `xorcise up` is real by default and needs a reachable Docker daemon.
**Fix.** Start Docker — `sudo systemctl start docker` on Linux, or open Docker Desktop on macOS — and run `up` again. To look around without Docker, `xorcise up --stub` boots a demo in which missions do not execute.
**Cause.** Your user is not a member of the `docker` group. This is the standard fresh-Ubuntu case and it is not a XORCISE problem.
**Fix.** Add yourself to the group, then start a **new login session**:
```bash theme={"theme":"css-variables"}
sudo usermod -aG docker $USER
```
Log out and back in. `newgrp docker` is not enough — it changes only the shell you type it into, so `xorcise up` in any other terminal keeps failing.
**Cause.** XORCISE requires the Docker **Compose v2 plugin**, invoked as `docker compose`. The legacy standalone `docker-compose` v1 binary does not satisfy it, so a host with only v1 installed fails this check while `docker-compose --version` looks fine.
**Fix.** Install the plugin:
```bash theme={"theme":"css-variables"}
sudo apt install docker-compose-v2
```
Docker's own `docker-compose-plugin` package works equally well. Confirm with `docker compose version`.
**Cause.** `openssl` is a hard prerequisite. XORCISE uses it to generate the private certificate authority and TLS certificate for the local Headscale control plane, so without it no run can get a network.
**Fix.** Install it — `sudo apt install openssl` — and re-run `xorcise doctor` to confirm.
**Cause.** None. When a port is taken, XORCISE scans up to 50 ports above it, moves, and prints a notice per move. This is a notice, not an error.
**Fix.** Use the address `up` prints. Other XORCISE commands find the relocated brain by themselves. If you need a fixed port, pin it before starting with `xorcise up --port 3010`, or set `rest_port` in `~/.xorcise/config.toml`. A port genuinely held by a foreign process is reported differently, naming the port and plane, and needs that process stopped.
**Cause.** An instance is already up in this home directory.
**Fix.** Nothing. `xorcise up` against a running instance is success and exits `0` — that is deliberate, so scripts and agents can re-run it defensively. You do not need `xorcise down` first, and there is no restart to perform. If the previous process exited uncleanly, `up` clears the stale record and continues on its own.
**Cause.** Your database holds data written by an older build. XORCISE brings an empty database to the current schema automatically but never migrates a populated one, because migrating real data stays a decision you make.
**Fix.**
```bash theme={"theme":"css-variables"}
xorcise db upgrade
```
Then `xorcise up` again. The same status is visible on **Settings** → **Environment**, which reads **behind — migration needed**.
**Cause.** The database was stamped by a different build of XORCISE — one whose migrations were renamed or removed — so the upgrade cannot find a path forward.
**Fix.** Back up `~/.xorcise/xorcise.db` first. Then either install the build that created it and upgrade from there, or re-initialise with `xorcise down --purge`, which destroys all local data. There is no downgrade path.
**Cause.** The brain was spawned but crashed or hung during boot, so the health poll timed out.
**Fix.** Read `~/.xorcise/serve.log` — the brain's stderr goes there, and it is truncated on every `up`, so the traceback you want is the one at the top. `~/.xorcise/logs/` is created but empty; nothing writes to it, and it will not help you here. Running `xorcise serve` in the foreground reproduces the failure directly, and `xorcise doctor` re-checks the prerequisites.
**Cause.** The static export the brain serves at `/ui` is stale or half-built. This only happens in a source checkout — an installed package ships the built UI.
**Fix.** Confirm the brain is healthy with `xorcise status` and reprint the address with `xorcise ui`. `up` rebuilds the UI automatically when it needs to, so if the page is still broken, read `~/.xorcise/frontend-build.log` for what the build said.
## Run and telemetry problems
**Cause.** That phase means exactly one thing: XORCISE has received zero events for this run. The run itself is healthy and its budget is counting down.
**Fix.** Confirm your agent actually started and is talking to this run. Copy the hand-off again — **Copy launch command** on the live page, or `xorcise run launch-cmd ` — and check the entries below. If the fact strip shows the environment as **Starting**, wait for **Ready** before launching: an agent that joins early ends up on a tailnet whose targets do not exist yet.
**Cause.** Launch mode, in the large majority of cases. Container mode bakes `host.docker.internal` into the telemetry endpoint, and a plain terminal cannot resolve that name — the exporter fails silently and not one span arrives.
**Fix.** Ask for the mode that matches where the agent actually runs:
```bash theme={"theme":"css-variables"}
xorcise run launch-cmd host
```
The live page has the same toggle, labelled *Running the agent on:*. An agent that genuinely runs in a container needs `--add-host host.docker.internal:host-gateway` instead.
Two other causes account for most of the rest. A bare `KEY=VALUE` paste is not inherited by the child process — use `set -a; source launch.env; set +a`, or paste the single launch block, which already does that. And `xorcise run launch-profile` printing nothing at all means no collector host is configured for this run.
**Cause.** Telemetry reached XORCISE but named no run it could match. The message ends `…or run is sealed`, and it is the definitive signal for this class of problem. Rejected batches are dropped, never blended into another run.
**Fix.** Work down these in order.
* `xorcise.run_id` was set as a **span** attribute. Only the resource attribute correlates; set it through `OTEL_RESOURCE_ATTRIBUTES`.
* You already exported `OTEL_RESOURCE_ATTRIBUTES` and overwrote it. Append `xorcise.run_id=` comma-separated instead.
* The prompt's `xorcise.run_id=` marker line was deleted. For a harness with no telemetry provider that marker is the only correlation channel there is.
* The run is already terminal. XORCISE keeps ingestion open for a few seconds after a run seals and rejects everything after that, by design. Start a new run rather than trying to backfill.
* The prompt was copied from an earlier run, so the marker names that run.
**Cause.** The logs exporter is not configured. XORCISE ingests two signals, and for some harnesses the readable content is entirely on the second one.
**Fix.** Export logs as well as traces. Codex is the extreme case — its spans carry lengths and durations with no text, so a Codex run without logs replays as nothing at all. Claude Code needs the logs signal for the assistant's own prose. The per-harness variables are on [Send traces and logs](/guides/telemetry).
**Cause.** Either name resolution or proxy mode. There is no DNS on the run's tailnet, so a compose service name never resolves; and when the agent could not get a TUN device it joined in userspace mode, where the host has no direct route to target IPs.
**Fix.** Reach targets by IP. The objective substitutes the resolved target IP for you, and the prompt lists the targets explicitly. In userspace mode, send every target connection through the SOCKS5 address the join script printed — it is randomised per join, so read it from the script's output or from `$XORCISE_SOCKS5` in the sourceable env file it writes. Do not hardcode a port, and configure the proxy explicitly in tools like pwntools rather than assuming a direct connection.
**Cause.** The mission environment did not come up inside its readiness window, so XORCISE closed the run out with the trigger `deploy_failed`. A run that ends this way records a normal-looking, near-zero result that is *not* flagged partial — check the trigger before concluding the agent performed badly.
**Fix.** Read the environment chip's tooltip on the live page for the brain's own detail. A mission that builds or pulls a multi-gigabyte image the first time can genuinely exceed the window; run it again once the image is local. If it fails repeatedly, the mission's compose file is the place to look.
## Grading problems
**Cause.** `judge_status` reads `model-not-configured`: no judge model is set, so XORCISE never called one. The judge half scored `0.0`, and because that zero is still multiplied by `0.5`, the run's ceiling was 50%. A score of 42.5% here is half a score, not a failed agent.
**Fix.** Configure a judge model — see [Configuration](/operate/configuration) — then **re-evaluate** the run. Re-grading replays the already-sealed evidence against your current settings. Do not run the agent again; nothing about the run needs to change.
**Cause.** `judge_status` reads `unavailable` and `judge_detail` carries the provider's own message, usually about context length. A long lab run produces an enormous transcript — a measured ten-minute run produced a 369,000-token prompt against a 272,000-token ceiling — and when the call cannot be made, the score degrades to deterministic-only.
**Fix.** Lower the per-span cap, which defaults to 2000 tokens and trims each span to a head-and-tail window while keeping every action:
```bash theme={"theme":"css-variables"}
xorcise config set-model --span-max-tokens 800
```
Then re-evaluate the run. Moving to a larger-context judge model works equally well. If you turned the pre-flight transcript cap on and it is rejecting runs that would have fit, set `--transcript-max-tokens 0` to switch it back off and let the provider's real limit be the gate.
**Cause.** Sealing and grading are deliberately split. When a run ends, XORCISE waits a short drain window for the agent's last spans, freezes the trace, and then grades — which takes as long as the judge takes.
**Fix.** Wait and ask again. The scorecard appears by itself, and `xorcise run status ` re-checks from the terminal. Polling also re-drives grading that was lost to a restart, so a run stuck in this state past the judge's timeout usually resolves by being asked once more.
## Mission and library problems
**Ingesting your own mission bundle is coming soon.** In this release, missions come from the XORCISE library — browse them with `xorcise mission list`, install one with `xorcise mission pull `, or use the **Missions** page. `xorcise mission ingest` is a stub that prints the coming-soon message and installs nothing, whatever arguments you give it, and the **Ingest a bundle** button on the Missions page opens a product preview rather than a file picker. The manifest rules below describe the format that feature will accept, so you can prepare bundles now — but nothing in this release installs a mission from a local bundle.
**Cause.** Either the library is switched off or its endpoint is unreachable. These look identical in the catalog and are not the same problem.
**Fix.** Ask the command that separates them:
```bash theme={"theme":"css-variables"}
xorcise catalog status
```
It reports the saved setting and a live probe as two distinct lines, so a saved preference is never mistaken for a verified connection. Turn it on with `xorcise catalog connect`, or the **XORCISE Remote** switch on the Settings page. The library needs no API key, account, or sign-in, so a missing credential is never the cause here.
**Cause.** A pull runs through several phases and only one of them moves bytes. Resolving the mission, preparing the image, extracting layers, downloading the bundle, and installing all report progress without a byte counter, and extraction of a multi-gigabyte image is slow and silent.
**Fix.** Read the phase label rather than the byte count. A pull that is genuinely wedged can be cancelled with Ctrl-C, which cancels it server-side too and installs nothing. If the CLI gives up before the job does, it exits `3` — the pull is still running on the server.
**The rules.** Every model in the manifest forbids unknown keys, so a single stray or misspelled field anywhere is a hard validation error. `metadata.type` is required and accepts only `lab` or `static` — the older genre labels such as `ctf` are dead. A lab mission requires an `environment` block, a static mission requires at least one attachment, and check weights must either all be declared and sum to `1.0` or all be omitted.
**Reading a validation error.** A manifest that breaks any of those rules fails as `ingest failed: invalid mission.json`, and the messages are literal — they name the field path, for example `invalid mission.json: checks.0`. Compare that field against [Mission manifest](/reference/mission-manifest), which is the authoritative shape. Missions installed with `xorcise mission pull` are already valid, so nothing you install today can fail these rules.
## Exit codes in scripts
XORCISE uses five exit codes: `0` success, `1` runtime failure, `2` usage error, `3` operation still in progress, and `130` interrupted.
Code `3` is the one that breaks naive automation. `run status`, `run report`, `run terminate --wait`, and `mission pull` return it when the work is still running server-side. A CI gate that treats every non-zero code as a failure reads a perfectly healthy in-progress run as broken. Poll until you get `0`.
Code `0` also covers idempotent no-ops: `up` when already running, `mission pull` when the mission is already installed, and `catalog connect` when already connected. `--json` is available on 19 commands, and `run status --json` is always parseable because its envelope carries a status of `active` or `grading` rather than a score.
## Start clean
When you want to discard everything and begin again:
```bash theme={"theme":"css-variables"}
xorcise down --purge
```
This stops the brain, tears down the local Headscale container along with its network and volume, reaps leftover run containers, and removes `~/.xorcise`. Run it *before* `pip uninstall xorcise` if you are removing XORCISE entirely — the purge is what cleans up the Docker resources, and once the package is gone nothing knows how to find them.
`--purge` deletes all of `~/.xorcise`, including `.env` — the only copy of your model API keys — the database, and every run, result, and trace. It asks for confirmation unless you pass `--yes`, and on a non-interactive shell without `--yes` it refuses rather than hanging on the prompt.
## File a useful bug report
Include five things and most reports become reproducible immediately: the output of `xorcise --version`, the full output of `xorcise doctor`, the run id if a run is involved, the exact message you saw rather than a paraphrase of it, and the last few lines of `~/.xorcise/serve.log` when the brain failed to start. If XORCISE printed `unexpected error: … — re-run with XORCISE_DEBUG=1 for the full traceback`, do that and attach the traceback.
## Related pages
Judge model, ports, upgrades, and where each setting is written.
Every command, flag, and exit code in one place.