Files
PaperClipAI/doc/architecture/paperclip-runner.md
Dotta 41bf5cafa1 docs(runner): define architecture and compatibility (#12084)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agent execution currently uses direct adapters inside the server
process.
> - The proposed Paperclip Runner adds a separate process and a new
protocol boundary.
> - This boundary needs clear trust, recovery, rollout, and
compatibility rules before code lands.
> - Large runner changes are difficult to review as one pull request.
> - This pull request defines the first small boundary for the runner
series.
> - The benefit is a stable design contract for later implementation
pull requests.

## Linked Issues or Issue Description

**Issue type**

Missing documentation.

**Where is the issue?**

The repository does not have a concise architecture decision or
compatibility contract for Paperclip Runner.

**What's wrong?**

The available runner design material is too large for normal review. It
mixes architecture, implementation history, test evidence, and deferred
work. Reviewers need a short statement of the process boundary, trust
model, rollout behavior, and direct-adapter compatibility rules.

**Suggested fix**

Add one architecture decision record and one compatibility document.
Keep implementation details and campaign evidence out of this pull
request.

Related public work: Refs #11041, #11297, #11634, #11639, #11640, and
#11962. This pull request is the first small replacement in the new
review series for #11962.

## What Changed

- Added an architecture decision for the runner process, PRP v1
transport, semantic tools, durable recovery, and additive server
integration.
- Added a compatibility and rollout contract for the default-off
adapter, existing direct adapters, persisted native runs, and the task
page.
- Defined the initial package and provider limits. The first production
provider is Codex only.
- Defined acceptance checks for later implementation pull requests.

## Verification

- `pnpm install --frozen-lockfile` passed with Node 24.19.0 and pnpm
9.15.4.
- `pnpm check:node-version` passed.
- `pnpm -r typecheck` passed.
- `pnpm build` passed.
- `git diff --check origin/master...HEAD` passed.
- The pull request changes 2 files.
- `pnpm test:run` completed with 4,686 passing tests and 30 failures in
unchanged master paths. The failures reproduce macOS path aliases,
invalid generated port values, and local listener behavior. This
documentation-only change does not touch those paths. Linux CI must pass
before this pull request is ready.
- All applicable GitHub Actions and security scans passed. The Storybook
visual job skipped because this documentation-only change does not match
its paths.
- Greptile completed at 5/5 with no actionable comments.

## Risks

Low implementation risk. This pull request changes documentation only. A
later implementation can still diverge from the contract. Each later
pull request must prove its behavior against these compatibility rules.

I checked `ROADMAP.md`. This design supports the governed tool and
control-plane direction. It does not add an overlapping user feature.

## Model Used

OpenAI Codex with GPT-5 was used. The exact serving model ID and context
size were not exposed. The model used high reasoning, repository tools,
GitHub tools, and local code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [ ] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-24 09:30:47 -05:00

11 KiB

ADR: Paperclip Runner architecture

Context

Paperclip is a control plane. It owns companies, agents, issues, budgets, approvals, and durable workflow state. Agent providers remain execution services. Existing adapters invoke those services directly from the Paperclip server.

Paperclip Runner introduces a separate execution process for provider sessions. This process needs durable delivery, restart recovery, and governed access to Paperclip actions. It must not become a second control plane. It must also land without changing the behavior of existing adapters.

The initial implementation is intentionally narrow. It supports Codex through an explicit, experimental adapter. Other providers and developer tools remain outside this decision.

Decision

Add a standalone package named @paperclipai/paperclip-runner. The package owns the language-neutral Paperclip Runner Protocol (PRP), the Rust runner process, provider drivers, deterministic replay, and semantic action dispatch contracts.

Add one explicit adapter named paperclip_runner. The adapter is available only when an instance-level, default-off rollout flag is enabled. Its first supported provider is Codex.

Do not route existing adapters through Paperclip Runner. A direct adapter keeps its current invocation, transcript, interaction, cancellation, and finalization paths.

Goals

  • Keep runner process ownership outside the Paperclip server process.
  • Preserve Paperclip as the authority for identity, policy, and workflow state.
  • Recover a run after runner or network interruption without duplicate effects.
  • Expose only actions that the current run is allowed to use.
  • Make protocol behavior deterministic across TypeScript and Rust.
  • Keep existing adapter behavior unchanged while the runner is experimental.

Non-goals

  • Replace existing direct adapters.
  • Move business authorization or issue status policy into Rust.
  • Give runnerd a broad Paperclip API credential.
  • Support OpenCode, ACPX, Claude Managed, AWS AgentCore, or remote sandboxes in the first production slice.
  • Expose browser SDK, React SDK, eval, lab, or scenario-explorer package entry points in the initial release.
  • Commit recorded screenshots, stress logs, or construction history as product architecture.

Topology

The initial local topology is:

Paperclip server
  |  authenticated PRP v1 WebSocket
  v
paperclip-runnerd
  |  Codex app-server protocol
  v
Codex

The server opens a native run and launches a verified runnerd artifact in the realized execution environment. Runnerd opens the outbound PRP connection. It then owns the provider process group and the durable transport state for that run.

The browser does not connect to runnerd. It reads projections from the existing Paperclip APIs and task-thread models.

Dependency direction

The runner package must build and test without importing Paperclip server, UI, CLI, database, or other private workspace implementation modules.

JSON Schema and fixtures
          |
          +------------------+
          v                  v
 TypeScript contracts   Rust runner core
          |                  |
          +--------+---------+
                   v
         deterministic parity

Paperclip server ----implements----> runner public ports

The dependency points from an implementation to a contract. The Paperclip server may implement a public runner port. The runner package must not import the server implementation.

The initial public package surfaces are:

  • @paperclipai/paperclip-runner for runtime contracts and clients.
  • @paperclipai/paperclip-runner/testing for deterministic fakes and conformance helpers.

Every export must have an implementation and a clean-consumer test before it is published. Later SDK, eval, and lab surfaces require separate decisions.

Protocol boundary

PRP v1 uses a WebSocket at:

/api/runner/v1/connect/:runId

JSON Schema is the language-neutral protocol authority. TypeScript and Rust use the same canonical fixtures and must produce the same replay result.

Protocol rules are fail closed:

  • Unknown required protocol or schema versions are rejected.
  • Unknown required discriminators and enum values are rejected.
  • Additive optional object fields may be accepted when v1 consumers can ignore them safely.
  • Frames, headers, durable state, diagnostic tails, and replay windows are bounded.
  • Event identity and ordering remain stable across reconnect and restart.

Provider-native messages do not cross this boundary. Drivers translate them to provider-neutral PRP events, results, usage, cancellations, and structured input.

Trust boundary

The Paperclip server is authoritative for:

  • company, agent, issue, run, session, and user attribution;
  • rollout and runtime selection;
  • action discovery and authorization;
  • approval, budget, secret, revision, and workspace policy;
  • durable application records and activity history;
  • result acceptance, finalization, and issue status.

Runnerd is authoritative only for its local responsibilities:

  • provider process supervision;
  • provider session and turn transport;
  • durable PRP outbox and command receipts;
  • stable runner-side event identity;
  • bounded process diagnostics.

Runnerd receives a short-lived, one-use bootstrap ticket. The ticket is bound to the company, agent, issue, run, runner, session, turn, and verified artifact. The server exchanges it for a short-lived connection lease. Raw tickets are never stored. Runnerd never receives a broad Paperclip API key.

The server rejects expired, replayed, revoked, cross-company, mismatched, malformed, oversized, or protocol-incompatible connections. Cancellation, timeout, supersession, and environment-lease loss revoke runner authority.

Durable delivery and recovery

Runnerd persists an event before it sends the event. The server acknowledges a cumulative source cursor. Runnerd removes acknowledged data only after it validates that cursor.

A reconnect reports the last processed command, the next source sequence, the last acknowledged source sequence, and the unacknowledged range. The server may then replay one pending command and accept byte-equivalent event retries.

Command identity is idempotent. Reusing a command ID with the same canonical input returns the stored result. Reusing the ID with different input fails closed. A repeated semantic tool call cannot repeat an application effect.

Recovery is bounded. Exhausted storage, reconnect, command, or time limits end the run with a classified failure instead of an unbounded loop.

Semantic actions

PRP carries provider-neutral semantic operation IDs. The package may define an operation catalog, but catalog presence does not grant authority.

For each run, the server projects only operations that have a production binding and that the current actor may discover. An unbound or unauthorized operation is absent from discovery. The server validates inputs, executes the existing application authority, and returns a redacted receipt.

This keeps these invariants in one place:

  • company scoping;
  • actor and run attribution;
  • authorization and approval;
  • revision and idempotency checks;
  • budget and secret policy;
  • activity logging and safe error details.

Runnerd cannot forge identity through tool input. Identity and scope come from the authenticated connection binding.

Structured input

Questions use paperclip.question_set.v1. Responses use the matching canonical response contract. Drivers translate between these provider-neutral records and provider-native input APIs.

The server validates a response at the untrusted API edge and again against the persisted question set before delivery. A process loss may materialize a durable task-thread interaction, but cancellation and an already resolved request must not create a second interaction or continuation.

Persistence and finalization

Native run records are additive. Existing heartbeat and issue records remain readable. A runner result is an untrusted claim. It does not directly update an issue status.

The server validates the result, classifies durable evidence, applies status policy, commits workflow effects, and records finalization. Each phase is idempotent and recoverable. Existing direct adapters keep their existing finalization paths and do not invoke native status arbitration.

Disabling the rollout flag blocks fresh runner starts. It does not make an already persisted native run unreadable or prevent bounded recovery and finalization of that run.

Rollout

The rollout has three gates:

  1. The instance flag is enabled.
  2. The agent explicitly selects paperclip_runner.
  3. The adapter selects a supported provider. The initial provider is codex.

The adapter is hidden from creation and selection surfaces while the flag is off. Server validation also rejects a fresh runner selection or start while the flag is off. UI hiding is not the security boundary.

Runtime selection is persisted before launch. Later setting changes cannot silently move an in-flight run between the direct and native execution paths.

The detailed compatibility rules are in Paperclip Runner compatibility and rollout.

Observability

The server records structured runner, provider, semantic action, interaction, usage, and terminal events. Logs and receipts must not contain bootstrap tickets, leases, provider credentials, secret values, complete environments, or private host paths.

The task page may project these records through existing thread components. A runner-only control must depend on persisted runtime facts. It must not depend only on the agent adapter profile.

Consequences

This design adds process, protocol, and recovery complexity. In return, it gives provider sessions a durable and testable execution boundary without moving Paperclip governance into the runner.

The default-off, explicit-adapter rollout duplicates some provider configuration during the experiment. This is intentional. It keeps comparison and rollback simple and prevents a global migration of existing agents.

The standalone package boundary requires generated artifacts, clean-consumer tests, and cross-language parity gates. These checks add build cost, but they prevent server implementation details from becoming accidental public API.

Required proof before general availability

  • TypeScript and Rust accept and reject the same protocol fixtures.
  • Lost acknowledgements, reconnects, runner restarts, and duplicate commands do not duplicate events or effects.
  • Bootstrap replay, binding mismatch, and cross-company access fail closed.
  • Only authorized and production-bound actions appear in discovery.
  • Cancellation settles provider, runner, transport, and application state.
  • Flag-off direct-adapter runs create no native records and start no runner process.
  • Persisted native runs remain readable and recoverable after the flag changes.
  • Codex completes the server to PRP to runnerd to provider to server path.
  • Existing adapter compatibility tests remain byte stable where specified.