ait-server: Remote Authority and Recovery

Why run ait-server#

A local AIT repository works without a server. Add ait-server when one or more of these outcomes are required:

• Remote workflow management: keep Repository, Plan, Task, Change, Patchset, evidence, policy, review, Land, and Worker Job state under one remote authority.
• Offsite preservation and disaster recovery: retain recorded Snapshot content and Line heads on a different machine or failure domain, then use them to rebuild local AIT authority and materialize source files.
• Repository-owned CI: admit typed jobs that a compatible ait-runner executes through the repository's declared ci/run.sh or ci/run.ps1.

These outcomes share the same server but remain distinct. A reachable server does not make every local file recoverable, and an offsite copy does not by itself provide automatic high availability or failover.

Authority owned by the server#

One server data root owns a global append-only Repository registry. Every registered Repository receives a numeric repository_index; that index, not the display name, selects its authority. Repository names may repeat.

Each numeric Repository authority owns its remote:

• Snapshot metadata, content packs, ancestry, and Line heads;
• Plan, Task, Change, Patchset, attestation, review, policy, and Land records;
• repository-scoped Worker Jobs and CI result state; and
• protocol state needed to validate and sequence those operations.

The server is authoritative for those remote records. It does not own the developer's current working directory, choose the repository's test framework, or infer files that were never recorded.

Preserve the exact repository_index in the repository recovery inventory. It is routing metadata rather than a secret, but losing it removes the safe way to identify an existing authority when rebuilding a client. Never guess an index or substitute one from another Repository.

Register and verify a Repository#

From an initialized repository, add the server as a remote:

ait remote add origin <server-url> --default
ait remote list --json
ait config show --json
ait repo show --remote origin --json

When no Repository index is configured, remote add registers a new numeric authority and persists the returned index. When an exact index is already configured, the command verifies that authority at the supplied server and fails closed on a mismatch.

Registration transfers no Snapshot or workspace content. It establishes the address and authority required by later workflow, push, pull, and recovery operations.

Remote workflow management#

For a remote-scoped Task, the local Task worktree remains the authoring boundary. After a Snapshot is created, readiness publishes the selected Patchset and the Snapshot evidence required by the remote Change:

ait workflow ready <change-id> --apply
ait task land <task-or-change-id>

The server sequences CI, attestation, review, policy, and Land state. Final Task land consumes the ready selected Patchset and advances the remote target Line. workflow ready is a text-only decision surface; follow the exact next action it prints when a gate is pending.

Operators can inspect the registered authority and its execution state without changing it:

ait repo show --remote origin --json
ait repo jobs --remote origin --json
ait repo ci-capabilities --remote origin --json

Remote workflow publication is also an offsite copy of its recorded Snapshot content. A published but unlanded Patchset remains workflow evidence; it does not mean the remote target Line has advanced.

Preserve a Line explicitly#

ait push is the direct synchronization path when the goal is to preserve an already-recorded Line independently of a governed remote Task:

ait status --json
ait snapshot create --message "Record the recovery point"
ait push --remote origin --line main

Push uploads the selected Line head and required Snapshot ancestry, then advances that remote Line. It does not capture dirty workspace files, create a Snapshot, publish a Patchset, merge divergent history, or perform Task land. Push every Line that needs its own recovery point; registration does not start an automatic synchronization schedule.

The latest successfully landed or pushed remote Line head defines the recoverable point for that Line. Set the publication cadence from the maximum acceptable data-loss window, then verify the server and practice restoration instead of treating a successful registration as backup evidence.

Protection boundary#

The server can preserve:

• Snapshot content and ancestry that were uploaded by push or remote workflow publication;
• remote Line heads that were successfully advanced;
• published remote workflow records and their admitted evidence; and
• repository-scoped CI and Worker Job state stored by that authority.

The server cannot recover:

• dirty, untracked, ignored, or un-Snapshotted workspace content;
• Snapshots or Line movement that remained local and were never pushed or published;
• files removed before the recorded recovery point;
• secrets, credentials, editor state, external services, or dependencies that are not part of the recorded Snapshot; or
• the server data root after the server itself is lost unless the operator also protected that data root elsewhere.

This boundary is why offsite preservation is deliberate: create the Snapshot, transfer it, confirm the remote state, and retain the Repository index needed to address it.

Restore when local authority is healthy#

When .ait and the remote configuration are intact, pull the remote Line and materialize its head into a clean workspace:

ait status --json
ait diff --stat
ait pull --remote origin --line main --restore

Pull imports the remote Snapshot ancestry before classifying the local and remote heads. It fast-forwards when the remote is ahead, leaves the Line unchanged when local history is ahead, and refuses unrequested divergence. --restore replaces the workspace with the selected pulled head. A dirty workspace is rejected unless --force is explicitly supplied; preserve any surviving local-only files before making that choice.

Recover missing or damaged local authority#

Use a separate clean recovery directory when the original .ait authority is missing or untrusted. Preserve the damaged directory for diagnosis instead of deleting it. Restore the trusted saved root routing fields as part of the local authority first: the exact repository_index, default_remote, and remotes map. RC.8 has no public repository_index setter, and ait remote add is a registration operation rather than a safe substitute for that recovery inventory.

# Run after the exact saved routing fields are restored.
ait config show --json
ait repo show --remote origin --json

ait remote recover-head --remote origin
ait remote recover-head --remote origin --apply
ait pull --remote origin --line main --restore

The first recover-head command is a preview. --apply downloads the verified Snapshot ancestry of the server's logical main head and reconstructs it inside a validated local Binary authority generation. RC.8 recover-head has no --line or multi-Line option. The final pull materializes main as source files. Recovery does not clone the server's remote workflow ledger into a separate local workflow ledger; remote Task, Change, Patchset, and job records remain queryable from their server authority.

Stop if the authority response does not match the saved index, the preview reports an unexpected head, validation fails, or the workspace contains files that must be preserved. If the exact saved routing fields cannot be restored, stop. Do not register replacement authority merely to bypass a mismatch. A complete retirement archive follows a different lifecycle: ait repo restore --remote origin creates restored authority with a new Repository index.

Runner and CI boundary#

ait-runner is an executor, not durable Repository authority. It claims one compatible typed Worker Job, materializes exact Snapshot content into isolated attempt storage, runs the repository-authored entrypoint, and returns bounded results. The server owns admission, leases, job state, and result acceptance.

The runner does not select a language-specific build system, become a second copy of the Repository authority, or preserve arbitrary files from its attempt workspace. Keep attempt roots isolated, monitor failed result delivery, and clean attempt-owned data after terminal jobs.

See ait-runner: Native CI Execution Plane for the exact RC.8 commands, job lifecycle, request and result contracts, materialization limits, lease behavior, and deployment checks.

Protect the server data root#

Using an offsite server protects the client only while the server authority survives. The container example stores it in the persistent volume mounted at /var/lib/ait; installed user and service modes use their configured data root. That complete root is the unit the operator must protect.

ait-server does not schedule a second replica, copy its own data root, elect a standby, or fail over automatically. Use an independent backup or storage-snapshot system in another failure domain. Capture an application-consistent copy by stopping writes or using storage that provides an equivalent consistency guarantee; do not treat an arbitrary live copy of individual authority files as recoverable.

Retain the server version or immutable image digest, data-root backup, exact Repository index inventory, remote URL, and separately managed access material needed for a recovery drill. Validate a restored copy in isolation before it serves traffic:

ait-server probe --data <restored-data-root> --defer-ci-admission

Probe failure means the restored root must not be activated. A successful /healthz response proves that the running process is healthy; it does not prove backup recency, completeness, or recoverability.

Operational checklist#

• Bind the service to loopback or a reviewed private ingress by default.
• Terminate TLS and enforce caller verification, network controls, monitoring, and capacity limits at the trusted ingress and host boundary.
• Pin compatible RC.8 client, server, and runner artifacts.
• Record every Repository's numeric index outside the client data root.
• Define and verify a Snapshot publication cadence for each protected Line.
• Back up the complete server data root to another failure domain.
• Rehearse probe, recover-head, and pull --restore with an isolated copy.
• Monitor server health, failed jobs, runner compatibility, and storage growth.

See the complete `ait-server` REST API Reference for every RC.8 HTTP route, request contract, response, and error boundary. Remote Infrastructure has the container boundary, ait-runner: Native CI Execution Plane has the full runner operations contract, and Troubleshooting has the general evidence-preservation rules.