ait-runner: Native CI Execution Plane

What ait-runner does#

ait-runner is the native execution plane for CI work admitted by ait-server. The server owns the Repository authority, Worker Job record, lease, state transition, and accepted result. The runner owns one bounded execution attempt: it claims compatible work, reconstructs the exact recorded source, invokes the Repository's CI entrypoint, returns evidence, and removes its attempt directory.

The runner is useful when CI must execute away from the developer worktree, when different machines serve different Repository sets, or when an operator must replay one known Worker Job pair. It is not a Repository backup, a policy engine, a scheduler independent of ait-server, or a language-specific build system.

Only two Binary Worker Job kinds are externally claimable in RC.8:

The durable job identity is the pair (repository_index, worker_job_index). Never substitute a Repository name for that pair when diagnosing or replaying a Binary Worker Job.

Four operating situations#

Persistent execution service#

Run serve under a service manager when one worker should poll continuously, maintain leases, execute compatible jobs, and deliver terminal results:

ait-runner serve \
  --server https://ait.example.internal \
  --worker-id runner-taipei-01 \
  --attempt-root /srv/ait/runner-attempts

--worker-id is required operational identity, not an environment setting. The server URL, source root, and attempt root are also explicit command arguments so the launched process describes its operating boundary.

Repository-scoped worker pool#

Repeat --repository-index to restrict one service instance to an exact set of Repository authorities. Duplicate indexes are normalized:

ait-runner serve \
  --server https://ait.example.internal \
  --worker-id runner-linux-arm64-02 \
  --repository-index 3 \
  --repository-index 8 \
  --attempt-root /srv/ait/runner-attempts

Omitting the filter lets the compatible queue select across registered Repositories. Use filters to express an intentional fleet boundary, not to guess which Repository owns a failing job.

One exact Worker Job#

Use run-job for a known Binary pair during controlled diagnosis or recovery. It claims, executes, heartbeats, and finishes only that pair:

ait-runner run-job \
  --server https://ait.example.internal \
  --repository-index 3 \
  --worker-job-index 42 \
  --attempt-root /srv/ait/runner-attempts

The server must advertise the current ait.runner.native-job.v3 contract. The command fails if the pair is not claimable, its lease proof is invalid, or the request does not match the claimed Repository.

Local request validation#

Use execute to exercise one typed local-directory request without claiming a server job. This is useful for validating the runner boundary itself; it does not create or update a Worker Job:

{
  "contract": "ait.runner.native-job.v3",
  "label": "local-repository-ci",
  "source": {"kind": "local_directory", "path": "."},
  "command": {
    "argv": ["./ci/run"],
    "working_directory": ".",
    "environment": {}
  },
  "timeout_ms": 900000,
  "suite_id": "repository-ci"
}

ait-runner execute \
  --request request.json \
  --source-root /srv/ait/sources \
  --attempt-root /srv/ait/runner-attempts

--request - reads the same JSON from standard input and is the default. A server-provided remote_snapshot request requires the server materialization provider used by run-job or serve; standalone execute is the local source path.

Complete RC.8 command surface#

ait-runner doctor --server <SERVER>

ait-runner execute \
  [--request <REQUEST>] \
  [--source-root <SOURCE_ROOT>] \
  [--attempt-root <ATTEMPT_ROOT>]

ait-runner run-job \
  --server <SERVER> \
  --repository-index <REPOSITORY_INDEX> \
  --worker-job-index <WORKER_JOB_INDEX> \
  [--source-root <SOURCE_ROOT>] \
  [--attempt-root <ATTEMPT_ROOT>]

ait-runner serve \
  --server <SERVER> \
  --worker-id <WORKER_ID> \
  [--repository-index <REPOSITORY_INDEX>]... \
  [--once] \
  [--poll-interval-ms <MILLISECONDS>] \
  [--heartbeat-interval-ms <MILLISECONDS>] \
  [--source-root <SOURCE_ROOT>] \
  [--attempt-root <ATTEMPT_ROOT>]

execute, run-job, and serve default --source-root to .. An omitted --attempt-root uses the platform temporary directory below ait-runner/attempts. serve defaults to a 1,000 ms poll interval and a 30,000 ms requested heartbeat interval. Both intervals must be nonzero; the effective Binary heartbeat is further bounded by the active lease.

End-to-end job lifecycle#

1. A remote workflow or Repository CI request causes ait-server to commit a typed Worker Job for one Repository.
2. The runner checks server health and selects the current Binary runner contract. doctor stops here and claims nothing.
3. serve claims the next compatible job, or run-job claims the exact (repository_index, worker_job_index) pair. The server returns a lease proof, attempt count, fixed job kind, and typed runtime request.
4. The runner validates the proof, job kind, Repository index, request size, paths, arguments, environment, platform, and CI entrypoint before execution.
5. For remote_snapshot, it downloads and verifies the exact Snapshot Tree, Blob packs, and locked external Repository Snapshots into a new attempt.
6. The logical argv selector ./ci/run resolves to ci/run.sh on Unix or ci/run.ps1 on Windows. The runner starts it with direct argv, not a concatenated shell command.
7. A separate heartbeat maintains the server lease while the process runs. A timeout terminates the owned process group before evidence is finalized.
8. The runner reduces stdout and stderr to bounded evidence, records exit and materialization facts, removes the attempt directory, and validates the terminal result size.
9. The runner submits complete or fail with the original lease proof. The server alone decides whether that transition is still admissible and stores the accepted terminal state.

The selected Snapshot remains immutable throughout the attempt. A runner never executes dirty developer-worktree content for a remote Binary job.

Request contract#

The current request schema is ait.runner.native-job.v3. Unknown fields are rejected. Its top-level fields are:

For a server-delivered source, remote_snapshot carries exact repository_index, repository_name, snapshot_id, and a map of external Repository names to their numeric indexes. The request's Repository index must match the claimed Worker Job pair. The command object requires argv, while working_directory defaults to . and environment defaults to an empty map.

The first argv value is always the logical ./ci/run selector. Additional values are passed directly to the platform entrypoint. The working directory must remain relative and confined to the materialized workspace.

Materialization and attempt isolation#

Every execution uses a uniquely named attempt-* directory containing the workspace, downloaded packs, and temporary logs. serve takes an exclusive lock on its configured attempt parent and reclaims only exact runner-owned stale attempt directories. Cleanup handles read-only files and does not follow symlinks outside the attempt.

Remote materialization fails closed on malformed inventory, checksum mismatch, unknown or excessive records, Tree cycles or depth, path escape, symlink source or entrypoint, and content that exceeds the declared bounds. Locked external Repositories are materialized from their own exact Snapshots, not from an ambient checkout.

The child CI process receives these runner-owned values:

These are injected process boundaries, not settings for launching the runner. The only supported runner credential environment variable is the optional AIT_SERVER_TOKEN. Server URL and worker identity remain --server and --worker-id; AIT_SERVER_URL and AIT_RUNNER_WORKER_ID are not inputs.

Bounded execution and evidence#

Each stream record contains the total byte count, SHA-256 of all captured bytes, base64 tail, tail byte count, and a truncation flag. The full temporary log is removed with the attempt, so terminal evidence is intentionally bounded rather than a hidden log archive.

Remote Snapshot import also enforces a 64 MiB manifest limit, 10,000,000 materialized files, 512 GiB materialized bytes, a 16 GiB per-pack download limit, and eight concurrent pack download/decode operations. These are safety ceilings, not recommended job sizes; operators should set substantially lower capacity expectations for a real worker host.

Result and diagnostic contracts#

Successful command execution writes one ait.runner.native-result.v1 object. Its terminal status is succeeded, command_failed, or timed_out, and tests_status is pass only for succeeded. The result contains suite records, exit code or signal, duration, materialized file and byte counts, bounded stdout and stderr evidence, and cleanup evidence.

Server-facing commands wrap an accepted terminal transition in ait.runner.delivery.v1. Other public output contracts are:

--once is fail-fast and returns after its single outcome. Persistent serve continues after a bounded per-job failure event. When polling cannot reach the server, persistent service mode uses bounded reconnect backoff up to about 30 seconds. Once the current Binary contract has been selected, the process will not silently downgrade. A heartbeat failure fails the claimed job; the runner does not overlap ambiguous heartbeat retries.

Deployment#

RC.8 publishes native runner executables for macOS, Linux/glibc, and Windows on both arm64 and x86_64. The Linux OCI index covers amd64 and arm64:

ghcr.io/weita2026/ait-runner:1.0.0-rc.8

The image entrypoint is ait-runner, its working directory is /workspace, and it runs as numeric UID/GID 65532. Give the attempt path a writable volume owned for that identity; a remote Snapshot worker does not need a mutable local Repository checkout:

docker volume create ait-runner-attempts
docker run --rm \
  --network ait-native-rc \
  --volume ait-runner-attempts:/var/lib/ait-runner \
  ghcr.io/weita2026/ait-runner:1.0.0-rc.8 \
  serve \
  --server http://ait-server:8088 \
  --worker-id runner-container-01 \
  --repository-index 0 \
  --attempt-root /var/lib/ait-runner \
  --once

Use a service or container secret for AIT_SERVER_TOKEN; do not embed the token in the image, command, Repository, or retained logs. Pin the immutable RC.8 image digest in production after verifying the published index.

Security and operating checklist#

• Run each worker under a dedicated OS identity or container with least filesystem and network access.
• Restrict server ingress, use authenticated TLS at the trusted boundary, and keep AIT_SERVER_TOKEN out of argv and files under the attempt root.
• Put attempt storage outside Repository authority and outside any source root.
• Do not share one attempt root between concurrently running serve processes; the exclusive lease will reject it.
• Author ci/run.sh and ci/run.ps1 as the Repository's reviewed CI boundary. Do not expect the runner to infer a package manager or test command.
• Monitor server Worker Job state, runner failure events, lease loss, disk capacity, cleanup evidence, and version-contract mismatches.
• Treat command_failed as repository CI evidence, timed_out as an execution limit, and ait.runner.error.v1 as a runner or request-boundary failure.

Start diagnosis without claiming work:

ait-runner --version
ait-runner doctor --server https://ait.example.internal
ait repo jobs --remote origin --json
ait repo ci-capabilities --remote origin --json

Then compare the exact Repository and Worker Job indexes, selected runner contract, attempt count, lease timing, terminal operation, stream digests, and cleanup evidence. Preserve server-side job evidence; do not preserve or reuse a partially cleaned attempt directory as if it were authoritative source.

For the HTTP claim, heartbeat, complete, fail, Snapshot import, and pack routes, see the ait-server REST API Reference. For server authority and offsite recovery, see ait-server: Remote Authority and Recovery. The complete public environment registry is in Appendix: Environment Variables.