Architecture

​

1. Shape of the product

• All internal code lives under xorcise.core.*. The bare xorcise.* namespace is reserved for future products, and scripts/check_single_distribution.py fails CI if a second distribution appears. Do not add top-level modules.
• Imports must have no side effects. Importing any package must not read config, open a database, touch the network, or start a thread. Settings load only when get_settings() is called.
• Bring Your Own Model. XORCISE bundles no LLM and no key. Every feature that needs a model — the judge, terrain attribution — uses the operator’s own OpenAI-compatible endpoint. Never add a bundled or default model, and never ship a credential.
• Vendor-neutral about the agent under test. XORCISE never launches the agent and ships no agent templates. The agent is given a connect prompt and reaches the run over the network on its own terms.

​

2. Layering and imports

cli  >  roles  >  application  >  parts  >  kernel  >  contracts

• Delivery surfaces sit above domain modules. rest and frontend may import a domain module; a domain module must never import a delivery surface.
• Part-islands are mutually non-importing. The runner, headscale and otel parts must not import each other. They communicate through typed clients in orchestration/clients/. When a part needs a fact owned by another module, it receives it as an injected callback, not an import — the headscale part learns which runs are active this way, and stays pure.
• The kernel splits: config sits below db and observability. config is shared-kernel and must not import an application-layer module — this is why it re-derives the home path locally instead of importing xorcise.core.home.
• Only the runner part touches Docker. Ingestion drives image builds through a builder port that it stubs in tests.

​

3. Persistence

• Migrations are hand-written. Autogenerate stays off. The Alembic environment must never import an application-layer model, so there is no target_metadata to diff against. Write the migration by hand.
• Never add a model-level ForeignKey across modules. The foreign key lives in the migration DDL only. SQLite does not enforce foreign keys by default, and a model-level FK across modules would break part-island independence.
• Cascades are application-layer and orchestrated by the delivery surface. Deleting a run, or an agent with its runs, deletes everything recorded for those runs. rest/run_purge.py holds the one ordered list of run-scoped stores. Both deletes walk it, and so does the boot sweep that clears records whose run row is gone. Each owning module lists and deletes its own tables. tests/unit/test_run_purge.py seeds every table with a run_id column, so a run-keyed table added later fails it until its module deletes it too. The result goes only after the run row, because a run row without its result is graded again at boot from whatever evidence is left. Do not “simplify” this to ON DELETE CASCADE — it would not fire on SQLite and would reintroduce the cross-module FK. This is the canonical case of an obvious simplification being wrong.
• Never migrate populated data on boot. A database is classified fresh (scaffold it to head), ready (boot), or stale (refuse loudly and defer to xorcise db upgrade). Bootstrap happens on up; migration is always an explicit operator action.

​

4. Run lifecycle and durable state

• Persist within the operation that changes the state. In-memory maps are caches and must always have a durable fallback. A fact that outlives a request, or crosses a process or crash boundary, is written through the runs domain layer at the moment it changes.
• Address external resources by run-derived identity — the router node is {run_id}-router, the container is named for the run — so a reconciler that never saw the creating process can still find and reap them.
• Make create, teardown, terminal transition and result-recording idempotent and replayable. Reconcile-on-startup re-runs them to convergence after a crash; anything that is not replayable will corrupt state instead of healing it.
• Release a run’s environment before deleting its row. The run row is the release sweeper’s only handle on the run: deleting it while the container or tailnet nodes are still up leaves the container until the next boot and the nodes forever. rest/run_teardown.py check_deletable refuses a delete for a run that is unfinished, still grading, or unreleased, and releases a finished one synchronously first.
• The server never calls its own REST API to persist its own state. Go through the domain layer.
• Terminal is first-wins and grading is asynchronous. Sealing happens synchronously so the agent gets an immediate answer; grading runs on a background task. The safety comes from a pair of guards — result-recording is idempotent first-write-wins on run_id, and the grade-and-record path is a guarded no-op if it has already run. Do not remove either guard, and do not move grading back onto the request thread. The budget watchdog runs on its own thread.
• Versions bump only on an explicit update. Registering a duplicate name is a 409, never a silent overwrite. A mission version is stamped only through the shared atomic-install helper; a new install path that bypasses it silently breaks cross-run comparability.

​

5. Network isolation

• Mission containers run on a Docker network nested inside a fused container, so no route exists from the host to a mission address.
• Every network the mission declares is created internal — not only the carved entry networks — so a multi-homed service cannot keep a route off box through its second attachment. The run’s own Tailscale router sits on the single non-internal network and is the run’s only egress. environment.allow_egress in the manifest opts a mission out of all of it.
• Refuse rather than deploy a run that cannot be confined. A mission declaring the reserved network name xorcise-egress, or an external: true network, is rejected at build time with the collision named. Both would have reported themselves confined while one network silently was not: the reserved name gets overwritten, and internal is a no-op on a network Compose did not create. A compose file that cannot be read at all is the same refusal — fail closed, never narrow confinement silently.
• A private tailnet carries the agent in. The access policy is fail-closed: exactly one rule per run, no default-allow. Refuse to apply a policy that does not match that shape.
• Each run’s router carries a per-run ACL tag, not just the shared base tag. The inbound rule that lets a target open connections back to the agent pins that tag as its only source, so it names exactly one router. With a shared tag every run’s router matched every run’s inbound rule, and a mission that pivoted through its router had a compiled path to a concurrent run’s agent on any port. The policy gate pairs each router tag with its single permitted agent and rejects any policy that does not.
• Render subnet allocation and the access policy from persisted non-terminal runs, unioned with the in-process in-flight set, and reconcile the policy after the row persists. Deriving them from in-process state alone lets concurrent runs collide and lets one process clobber another’s policy.
• Delete the run’s router node by name once the run is graded. A completed run that leaves an online node behind leaks into the next run’s allocation.
• Agents reach targets by IP; there is no DNS inside the fence. The tailnet login server is never host.docker.internal.
• The per-run join key is read through its narrow getter only and is never surfaced on the general run entity.

​

6. Missions

• The mission and its runner are fused into one self-contained image per mission, with the inner stack baked in and loaded on boot so a deploy needs no inner pull. The cost is that a change to runner control logic requires rebuilding every mission image. That is an accepted trade, not a bug.
• The runner is DinD-only. A mission’s containers always run on the nested daemon inside the run’s own container, never as siblings on the operator’s daemon. The sibling topology is gone, not optional: it put every mission’s containers on one daemon, where parallel runs collided on fixed container_name values and published ports. nested_container_check=skip bypasses the precondition check for hosts where the probe cannot run; it cannot restore the sibling topology.
• A fused image records the base generation it was built on, and this build refuses an artifact older than the generation it runs, in both the CLI and the console, with a named refusal rather than a mid-run failure. One verdict serves both the run-create gate and the catalog browse surface, so a card can never say runnable for something a run would refuse.
• The two acquisition paths never cross. A library mission is pulled as a prebuilt fused image; a local mission is ingested and built locally. Both write their installed record through the same shared helper.
• Attachment bytes never ride run-control. The agent asks for a companion file by name and receives a short-lived HMAC-signed URL it fetches separately. Signing secrets are generated per process unless explicitly pinned.

​

7. Telemetry and the evidence pipeline

• The live view and the grader read natively off the persisted raw store. The OTLP receiver is ingress-only — it never becomes a routing, correlation or persistence layer.
• The agent-event projection is a rebuildable cache, never a source of truth. It is versioned by adapter name and version so a changed adapter invalidates and re-derives it.
• The grader must never read the agent-event projection. It consumes the sealed raw record only. This one is enforced by name in .importlinter.
• Adapters must be total. An unrecognised span becomes a generic event, an adapter exception becomes exactly one error event, and neither ever produces a 5xx. Supporting a new agent must require no frontend change and no core change.
• Adapter selection is authoritative from the run’s recorded agent, not from agent-emitted resource attributes, which are forgeable.
• Admit a batch and seal a run under the run’s admission lock. Both start their transaction by updating the run’s row in otel_admissions, so whichever comes second waits for the first to commit. In that one transaction the receiver checks the seal, numbers the batch MAX(seq) + 1 and inserts it (TraceStore.append_next), so a batch either commits before the seal and is hashed into the evidence digest, or sees the seal and is refused. Do not split the seal check from the append, and do not number a batch by counting the run’s rows. Ingest runs on the receiver’s own worker threads and the seal on another thread, so nothing else orders them: a split check lets a batch land while the run is being hashed, and an untouched run then reads mismatch; a count lets two batches share a seq. A unique (run_id, seq) index on traces and logs backs this up. On SQLite the seal’s half of the lock looks like dead code, because the database’s single write lock already orders the two; on an engine that locks rows, only this lock keeps a batch out of the hash. tests/unit/test_otel_admission.py races real threads against a real database, and test_seal_takes_the_admission_lock in tests/adapters/test_otel_seal_store_sqlite.py pins the seal’s half.
• A run is sealed on terminal; spans arriving afterwards are rejected. Replay never affects a score.
• The evidence digest covers every evidence stream grading reads: raw traces, raw logs, run-control submissions and observed facts. A new evidence stream goes inside it, and any change to what it covers or how it is framed bumps the scheme tag, so an old digest reads as unverifiable rather than as tampered. A recorded digest is never overwritten, and a digest XORCISE cannot check is never reported as a mismatch.
• Telemetry is optional. A run with no traces is valid: deterministic checks read artifacts and observed facts, and criteria the judge cannot evidence are left unscored: they count as zero, and the result carries an upper bound that grants their weight. Never make a trace a precondition for grading.

​

8. Grading

• Deterministic checks assert a named operation over a value resolved from a named source. Fetching is decoupled from asserting so a new check op needs no new resolver.
• Agent-controlled evidence rides the user role, fenced as untrusted and stripped of fence markers; instructions ride the system role. Never interpolate agent output into the system prompt.
• The transcript budget is counted with an explicitly configured tokenizer. Tokenizers are not interchangeable, so the choice is recorded rather than inferred. Going over budget must fail loudly; never silently truncate evidence.
• A degraded judge produces a recorded, re-gradeable failure — never a silent zero.

​

9. Agent control surface

• The agent’s verbs are /mission, /intel, /artifacts, /complete, plus the connect and join helpers. Authentication is a per-run bearer token.
• There is one submission verb. The flag is not special: it is the artifact named flag, credited by a deterministic check. Never reintroduce a dedicated flag endpoint.
• Post-terminal run-control is uniformly 409 — no per-verb special cases, no “late submission” path.
• Note that the agent-event vocabulary retains MCP call and result kinds. Those describe the agent under test invoking its own MCP tools, which is a different thing that happens to share a name.

​

10. Configuration

• The config API is a fixed, judge-shaped contract, never a generic settings passthrough — a generic one could brick a running server.
• Secrets are write-only. A stored key reads back only as its last few characters.
• All writers go through the shared upsert helper and then clear the settings cache. Writing the environment file by any other path leaves the running server serving stale configuration.

​

11. Deferred, and deliberately absent

• Remote and cloud-hosted mission planes are not implemented. XORCISE runs on one machine. No Terraform ships in this repo, and the remote and cloud auth commands are placeholders.
• There is no streaming trace endpoint. The live view polls with an incremental cursor. This is a deliberate simplification.
• Telemetry mirroring to an external service is a reserved, default-off seam. Enabling it fails fast rather than silently exporting: nothing leaves the operator’s machine by default.

​

12. Contributor ground rules

​

Coding standards

• ruff (lint+format), mypy --strict. No # type: ignore without a reason comment.
• Small, focused files (one responsibility). Typed DTOs (xorcise.core.contracts) over loose dicts at boundaries.
• TDD: failing test → minimal impl → green → commit.
• CI is GitHub Actions (.github/workflows/ci.yml); any CI lane, dependency setup, guard, or test-marker change must be reflected there.

​

Security

• The application layer never holds the Docker socket (only runner). Nothing leaves by default (egress is opt-in).
• Secrets live in ~/.xorcise/.env (0600), never in the repo. BYOM keys are the user’s.