Nicky LeachandClaude Fable 5.2 ddfbbfff20 fix(runner): recover provider-started turns after a controller restart (#21612)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Core runs Claude Code through Paperclip Runner. Claude can start a
shell command in the background and end its turn. When the command
finishes, Claude Code starts a cycle of its own to continue.
> - The Runner can keep Claude while a background shell is running and
give the cycle of Claude to the controller as a turn that the provider
started. Core can wait for that turn. #21607 lets a caller ask the
transport for it.
> - A controller can restart at any time, for example at a deploy. A
wait can last 10 minutes and a provider-started turn can last longer, so
a restart during one of them is a normal event.
> - Tests with the real `paperclip-runnerd` showed that such a restart
failed the session. The checkpoint that Core had was the checkpoint of
the turn before, and the recovery of the driver refused or mislabelled
the turns that it did not know.
> - This pull request makes that restart safe. It changes no production
run, because no caller asks for provider-started turns before part 3.
> - The benefit is that part 3 can turn the behavior on without a failed
run at each deploy.

## Linked Issues or Issue Description

No public issue exists. The issue is described here.

#21456 turned the behavior on and had grown too large to review. It is
now three stacked pull requests:

1. #21607 (merged): the transport can ask the Runner for
provider-started turns. No caller asks.
2. **This pull request:** the recovery after a controller restart during
a wait or during a provider-started turn.
3. #21456: Core sets the option for native ACPX Claude runs. Only that
part changes what a production run does.

This is part 2. Its base branch is `master`. Refs #19912 (design),
#21456.

**What existing behavior does this improve?**

None in production. It makes a controller restart safe for a run that
asked for provider-started turns.

**Subsystem affected**

Paperclip Runner package: the consumer
(`src/native-session-runtime.ts`), the session backend
(`src/backends/harness-driver-backend.ts`), the driver
(`src/drivers/codex/`) and the transport (`src/live/`). Only for a run
that asked for provider-started turns.

**Current behavior**

- The consumer saves no checkpoint when a wait starts or when a
provider-started turn starts. During a wait and during a
provider-started turn, the checkpoint that Core has is the checkpoint
from the start of the turn before. It names that turn as active.
- A restart from that checkpoint fails: the driver has the old turn as
active, gets the events of a provider-started turn, and stops the
session with `turn_binding_mismatch`.
- The recovery of the driver adopts one turn that it does not know as a
disposition recovery turn, and refuses two or more. With two or more, it
closes the transport, and that stops the Runner and Claude.
- The Runner does not send again an event that the controller
acknowledged. If the start of a provider-started turn was acknowledged
before the restart, the new driver gets the other events of that turn
with no start.
- A turn that ends waiting for a provider-started turn sets the terminal
of the session, as a turn that ends the run does. After a restart, if
the provider-started turn already has its result when recovery
completes, the consumer finds a result and a terminal, settles the run,
and does not read the events of that turn. The start, the items, the
usage event and the terminal of that turn do not reach Core.
- The recovered driver reads notifications from its construction, and
`reconcile()` maps the terminal of the saved turn only after its reads.
If the saved checkpoint is still the checkpoint of the controller turn
and the start of the provider-started turn is already queued, the driver
judges that start too early and stops the session with
`turn_binding_mismatch`.
- A checkpoint that waits can also have a consumed disposition marker (a
recovery turn after an earlier restart ended waiting). The driver
restores that session as closed and reads no notifications, and the
consumer settles the run on the earlier terminal with no result.

**Proposed behavior**

- **Checkpoints.** The consumer saves a checkpoint when a wait starts
and when a provider-started turn starts, with the function that it
already uses for a governed wait. A checkpoint that fails stops the
turn, as for a governed wait.
- **A checkpoint as of the event that Core stored.** The driver gives
each event its sequence when it reads it from the provider, before the
consumer stores it. A snapshot that is taken later can be ahead of Core.
For these two checkpoints, the driver captures its snapshot when it has
handled the notification that made the event (the `turn.completed` that
waits, or the `turn.started` of the provider), with the sequence of that
event. The consumer asks for the snapshot as of the event that it has
just appended. The saved state and cursor are then those of an event
that Core has.
- **Session terminal.** A turn that ends waiting for a provider-started
turn does not set the terminal of the session (`HarnessNativeSession`).
The terminal of the session is the end of the run, and that turn does
not end the run. A checkpoint that is saved during a wait or during a
provider-started turn has no terminal, so a recovered session settles
only from the events that it reads, in order, to the terminal of the
turn that has the result.
- **Driver, a checkpoint that waits.** Recovery keeps the wait. It does
not adopt and does not refuse the turns after the last terminal that it
knows. The Runner sends again, in order, each event that was not
acknowledged, and the driver admits each provider-started turn from that
stream, as it does with no restart.
- **Driver, a checkpoint with an active turn.** `reconcile()` accepts
one turn in progress that comes after the saved active turn, when the
saved turn ended with the wait marker. It refuses each other unexpected
active turn as before, and it refuses this one too when the reconciled
turn settled a result.
- **Driver, a start that is queued before recovery reconciled the saved
turn.** A `turn/started` of the provider that arrives while the session
does not wait for one first waits until `reconcile()` has settled, and
is then judged as before. `reconcile()` judges the live state after its
reads; if the stream already settled the saved turn into a wait, it
accepts exactly one turn in progress after the last terminal turn and
does not adopt it. The driver admits that turn at its start.
- **A wait with a consumed disposition marker.** The driver keeps
reading notifications for a restored session that waits, as it does for
a wait with no restart. It still refuses a second controller-started
turn. The consumer does not take the consumed-marker shortcut for a
recovered snapshot that waits, so the wait is resumed.
- **Transport, the start comes first.** For each driver, the transport
gives the `turn/started` of a provider-started turn before each other
notification and before a tool call of that turn. If it did not give
that start to this driver, it builds it from the `turn.started` event in
its stored events. It does not do this for the turn that the recovered
driver already has as active (`resumeActiveTurnId`). It binds the events
and the tool call of a provider-started turn to that turn, not to the
turn that the Runner reports as active at the time of the recovery read.
- **Transport, which start owns an event.** The stored events can have
gaps: the window removes old events but keeps the latest of each anchor
type, and the controller checkpoint drops streamed deltas. The owner of
an event is the last stored `turn.started` before it, if it is a
provider start, and if the first turn terminal that is stored after that
start is not before the event and is the terminal of the same provider
turn (`providerTurnId`). If the first stored terminal after the start is
the terminal of a different turn, the end of the provider turn is not
known, and the transport builds no start.

**Reason and benefit**

- The checkpoint is the cause. With a true checkpoint, the two states of
a restart are states that recovery can handle: a wait, or an active
turn.
- The Runner already sends again each event that was not acknowledged.
Recovery uses that stream and does not build the turns again from a
read.
- The window removes a terminal only while a later terminal of the same
type is stored. Thus the first stored terminal after a start shows if a
terminal of that turn was removed. A rule that only ignores the gaps is
not safe: the Runner can make a `turn.failed` for a turn that did not
emit its start (`provider_turn_lost_on_restore`).

## What Changed

Production code: 113 lines added, 19 removed.

- `src/native-session-runtime.ts` (15 added, 4 removed): the two
checkpoints in `consumeTurn`, with the sequence of the appended event,
and the resumed wait with a consumed disposition marker.
- `src/drivers/codex/codex-session-state.ts` (24 added, 1 removed),
`codex-session-terminal.ts` (2 added), `codex-session-notifications.ts`
(2 added), `codex-harness-session.ts` (24 added, 7 removed),
`codex-app-server-driver-impl.ts` (4 added, 1 removed): the snapshot at
the event, and the recovery of the driver.
- `src/contracts/harness-driver.ts` and
`src/contracts/native-session-backend.ts` (1 added each): an optional
`boundarySnapshot` on a driver session, and an optional sequence in the
snapshot options. Other drivers do not implement it and are not changed.
- `src/backends/harness-driver-backend.ts` (7 added, 5 removed): the
session terminal, and the snapshot as of a sequence.
- `src/live/runnerd-codex-transport.ts` (15 added, 1 removed) and
`src/live/provider-turn-tool-order.ts` (18 added): the start-first rule
and the owner of an event.
- Tests: `src/live/runnerd-acpx-provider-turn.test.ts` (572 added, 5
removed),
`src/drivers/codex/codex-app-server-driver.provider-turn.test.ts` (348
added), `src/live/runnerd-codex-transport.test.ts` (137 added),
`src/control-plane/durable-prp-control-plane.test.ts` (84 added),
`src/live/provider-turn-tool-order.test.ts` (new, 61),
`src/native-session-runtime.provider-turn.test.ts` (92 added, 6
removed), `src/backends/harness-driver-backend.test.ts` (56 added).
- `runner/crates/runner-core/src/bin/fake-acpx-sidecar.rs` (test binary
only, 123 added, 14 removed): a delay before a provider-started turn, a
goal state at its terminal, a list of provider-started turns for one
session, a gap between them, a turn that stays open until the new
controller reads the snapshot, and a turn that starts after that read.
The modes that existed are not changed, and the fake reads its input on
one thread as before.

## State saved by the previous release

- No new persisted field. The driver session already has
`awaitingProviderTurn` (#21453).
- **More checkpoints:** one when a wait starts and one when a
provider-started turn starts. An ordinary turn saves the same number of
checkpoints as before (4 in the unit test, measured before and after).
- **No terminal in a checkpoint of a wait.** A checkpoint that is saved
during a wait or during a provider-started turn has `terminal: null`.
When a wait ends with no provider-started turn (the limit, a comment of
a person, or the Runner ends the run), the last checkpoint of the run
also has `terminal: null`; before, it had the terminal of the turn that
waited. One reader uses that field with no result: the text of a cleanup
failure (`nativeCheckpointRecordsCompletedTurn`), which then does not
say that the turn completed. In the tests with the real daemon (the
limit, the provider process lost, the sidecar exits), the run still ends
on the path of today (`native_finalization_missing`).
- **A checkpoint of the previous release** has no waiting state.
Recovery takes the path of today.
- **Rollback of the Runner package during a wait:** the previous code
reads the checkpoint, ignores `awaitingProviderTurn`, and takes its
recovery path of today.

## Verification

With nothing from part 3. The Rust checks and the build checks in the
first item were run before the last seven commits; these commits change
no Rust. The other items were run at the head of this branch
(`36d5e5e587`), or at an earlier commit of this branch for a suite whose
files the later commits do not change.

- Before the last seven commits: `pnpm run typecheck:typescript`, `pnpm
run build:typescript` and `node --test
test/acpx-sidecar-contract.test.mjs` (18 passed) in
`packages/paperclip-runner`; `node
scripts/verify-runner-vendor-dependencies.mjs` in `server`; `node --test
scripts/__tests__/runner-release-crate-cache.test.mjs`; `cargo fmt
--check`, `cargo build`, the 17 `acpx_*` suites (241 passed) and the
`acpx` unit tests (102 passed): all passed.
- `npx tsc --noEmit -p tsconfig.json` and `npx tsc --noEmit -p
tsconfig.test.json` in `packages/paperclip-runner`; `pnpm typecheck` in
`server`: all passed.
- `src/live/runnerd-acpx-provider-turn.test.ts`, 4 runs with the
production code of this head, 26 of 26 passed in each. The Core of these
tests follows the production append (`appendNativeSourceEvent`): it
stores an event that is out of order, keeps its contiguous cursor below
a gap, and reports a duplicate or a conflict for an event that it has.
It does not reject a gap in the sequence; production does not. The
restart cases, all with the real `paperclip-runnerd`:
- **From the checkpoint that the real consumer saved.** The first
`executeNativeSession` is fenced and detached as the server does for a
restart, and a second one recovers from the last checkpoint that the
first one saved. (a) Restart during the wait; the provider-started turn
starts after the new controller is attached. (b) Restart while a
provider-started turn is in progress. In each: the test asserts the
shape of that checkpoint, the run settles with the result of the
provider-started turn, each event is appended one time and in order, the
turn has one `turn.started` with `origin: "provider"`, and the session
closes with no error.
- **Without the two checkpoints, each of these cases fails:** the saved
checkpoint is the checkpoint of the turn before, and the second
controller fails with `turn_binding_mismatch` or with a replay conflict.
- **A slow `appendEvent`, with events that arrive behind the stored
event** (3 cases, each run 3 times with the fix and 3 times without it,
with the same result in each run). R7: the append of the
`turn.completed` that waits is slow, and the provider-started turn
starts behind it. With the snapshot at the event, the checkpoint is the
waiting state at the sequence that Core has, and Core gets the
`turn.started` of the turn one time. Without it, the checkpoint names
the provider-started turn as active, the transport builds no start, and
Core never gets that `turn.started`. R8: the first controller answers
the `paperclip_finish` call while the append of the provider
`turn.started` is slow. With the snapshot at the event, the checkpoint
has no result and the run ends with `native_finalization_missing` (see
Risks). Without it, the checkpoint has a result that Core does not have,
and recovery stops with "session cannot start another turn". R6: the
append of the provider `turn.started` is slow and a delta is queued
behind it. With the snapshot at the event, the new controller continues
the sequence that Core has. Without it, the only difference is one
sequence number that Core never gets.
- **The start was acknowledged before the restart** (3 cases: one turn
open across the restart; two turns; a turn that waits for the answer to
its `paperclip_finish` call). The new driver gets the start of each turn
one time before its other events, the finish call is answered exactly
one time, and the session closes with no error.
- **The turn ran with no controller attached** (2 cases, with and with
no result) and **two turns in one outage** (2 cases): the turns arrive
in order after the recovery.
- **A provider-started turn that is active in the checkpoint** gets no
second start, and the next provider-started turn arrives after it.
- **The turn already has its result when recovery completes** (3 cases:
the turn made its `paperclip_finish` call with no controller attached;
the turn waited for the answer to that call at the restart; two turns
across the restart). The test asserts the exact list of events that Core
gets: each provider-started turn has one start, its usage item, its
message item and its terminal, and then `run.result.accepted` and
`run.terminal`. `completeRun` is called one time with the turn that has
the result. Before the change to the session terminal, the second
controller appended only `run.result.accepted` and `run.terminal` in
each of these cases.
- **The wait checkpoint is not saved, then a restart** (R10). The last
checkpoint is the checkpoint of the controller turn, Core has the
terminal of that turn, and the start of the provider-started turn is
queued. The run settles with the result of the provider-started turn,
Core has one `turn.started` for it, and the finish call is answered one
time. Without the wait in the driver, recovery stops with
`turn_binding_mismatch`.
- **An ordinary turn**, with no option. One that waits for the answer to
its finish call at a restart: Core gets its result, its terminal and the
run terminal one time. One whose checkpoint after `startTurn` is ahead
of Core at a restart (R9): the run settles, and Core does not have only
the one event that the first controller did not store. These cases pass
with and without this pull request.
- **No checkpoint with a terminal and no result** is saved by the two
controllers in the cases above.
- `src/live/provider-turn-tool-order.test.ts` and
`src/live/runnerd-codex-transport.test.ts`: 345 passed. The helper tests
cover a stored start with removed events after it, a dropped delta, the
terminal of the turn before and after the event, a terminal of a
different turn after a removed terminal, and a controller start. With no
option, with a different agent or provider, and for a turn that the
controller started, the notifications of the transport are the same as
before this pull request.
- `src/control-plane/durable-prp-control-plane.test.ts`: 230 passed.
Three new tests use the real commit path: a full window of 4,096 events
removes the events after a provider start and the start is found; the
window removes the `turn.failed` of a provider turn after a later
`turn.failed` with no start, and no start is found; a restored
checkpoint that dropped the deltas of the turn, and the start is found.
- `src/drivers/codex` and `src/backends/harness-driver-backend.test.ts`:
491 passed. The driver tests cover a provider start that is queued
before `recoverSession()` is called, the wait state with one turn in
progress at `reconcile()` (and the two cases that it must still refuse),
a restored wait with a consumed disposition marker (the provider turn is
admitted and `startTurn` is refused), the snapshot at the two events
(also after the driver moved on, and for a restart continuation that
ends waiting), one later turn in progress, one completed and a second in
progress, two completed with a result, and the cases that `reconcile()`
must still refuse. The backend tests cover the session terminal and the
snapshot as of a sequence.
- `src/native-session-runtime.provider-turn.test.ts` and
`src/native-session-runtime.test.ts`: 333 passed. Two new cases resume a
restored wait with a consumed disposition marker: the provider-started
turn arrives and settles the run, or the wait ends at its limit and the
run takes the path of today with no new turn.
- `server`: `native-session-executor.test.ts`,
`native-provider-turn-wait.test.ts` and
`native-runner-restart-recovery.integration.test.ts`: 1,743 passed, 21
skipped.
- Mutation checks, each made named tests fail: no checkpoint at the
start of a wait; no checkpoint at the start of a provider-started turn;
the consumer does not pass the sequence (R6, R7 and R8 fail: a hole in
the sequence, no `turn.started` in Core, a different failure); the
backend ignores the sequence; the driver has no snapshot at the event;
no start-first rule; no exclusion of the active turn of the recovered
driver; the turn binding only at the first delivery; the contiguous
sequence required again; no check of the first terminal after the start;
the adoption of later turns put back; no exception in `reconcile()`; no
check after the settlement in `reconcile()`; the terminal of the session
set again by a turn that ends waiting; no wait of the provider start for
`reconcile()`; no exception in `reconcile()` for a session that waits;
the pump deferred for a restored wait; the consumed-marker shortcut
taken for a snapshot that waits.

Not run: the full suite. CI supplies that coverage.

## Risks

- **No production effect.** Each change is behind the waiting state, a
provider `turn.started`, or the option of the transport. No caller sets
the option before part 3.
- **An event that the first controller received but Core did not store
is not sent again after a restart.** This is the same on `master` and is
not changed here. Only the terminal of a turn is built again. If the
lost event is the result of a provider-started turn (the first
controller answered the `paperclip_finish` call and stopped before Core
stored the proposal), the run ends on the path of today for a turn with
no result. R8 shows this. R9 shows the same loss for one event of an
ordinary turn.
- **A second `turn.started` for one provider-started turn can reach
Core.** This occurs if the restart comes after Core stored the start and
before the checkpoint of that start is saved (the time of one checkpoint
write). The last checkpoint then waits, and the transport builds the
start again for the new driver. Seen with the real daemon when the
checkpoint write is held; not in a committed test. The usage accounting
of the server uses the turn id as its key, so there is no second count,
but the second start resets the "restored" mark of that turn. The
transcript does not read `turn.started`.
- **The older checkpoints can also save a cursor that is ahead of Core**
(measured for the checkpoint after `startTurn` of an ordinary turn).
Production accepts the gap in the sequence that follows, so this is not
a failure there. It is on `master` and is not changed here.
- **When the terminal of a provider-started turn was removed from the
stored events,** the transport builds no start for an event of that
turn, and the session stops on that event, as before. This needs a later
terminal of the same type with no start of its own (a turn that was lost
while it started) and a full window.
- **The last checkpoint of a wait that ends with no provider-started
turn has no terminal,** as the section above says. The change is only in
a run that waits, and no run waits before part 3.
- **A wait with a consumed disposition marker has no test with the real
daemon.** The state needs two restarts in one run, and the test helper
does one. Unit tests of the driver and of the consumer cover it.
- **A provider start that waits for `reconcile()`** waits as long as the
reads of `reconcile()` take. Each read ends at the command deadline of
the Runner (30 seconds by default) or when the transport closes.
- **`item.delta` events across a restart are not asserted.** A streamed
delta that the first controller received is not sent again. The
completed items are asserted.
- **The driver keeps a maximum of 8 snapshots at events** and drops each
one when the consumer asks for it or for a later one. If the consumer is
more than 8 such events behind, the checkpoint uses the current
snapshot, as before this change.
- **A second provider-started turn that also ends during the same
outage** is not tested with the real daemon: the fake gives each
terminal the same goal revision, and the Runner then holds the second
terminal until a controller is attached. The end of the reconnect time
of the Runner is not tested here.

## Model Used

- Claude Fable 5.2 (`claude-fable-5-2`) in Claude Code planned the
change and the split, decided the designs, reviewed the diffs, and ran a
part of the verification again.
- Claude Opus 5.5 (`claude-opus-5-5`), as Claude Code subagents with
tool use (shell, file edits, local test runs), wrote the change and the
tests, reproduced the review findings with the real daemon, and verified
this slice alone.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-14 12:51:52 -08:00
…
…
…
…
…

Paperclip is the app people use to manage AI agents for work.

Quickstart · Docs · GitHub · Discord · Twitter · Website

MIT License Stars Discord Security posture verified by Superagent

Star History Rank


Sign up for the Paperclip Cloud waitlist →


Paperclip is the app people use to manage AI agents for work.

Open-source orchestration for teams of AI agents.

If OpenClaw is an employee, Paperclip is the company.

Paperclip is a Node.js server and React UI that orchestrates a team of AI agents to run a business. Bring your own agents, assign goals, and track work and costs from one dashboard. Choose models and harnesses per agent while keeping your team's tasks, skills, permissions, and history in one place.

It looks like a task manager. Under the hood: org charts, budgets, governance, goal alignment, and agent coordination.

Manage business goals, not pull requests.

Step Example
01 Define the goal "Build the #1 AI note-taking app to $1M MRR."
02 Hire the team CEO, CTO, engineers, designers, marketers — any bot, any provider.
03 Approve and run Review strategy. Set budgets. Hit go. Monitor from the dashboard.

Works
with
OpenClaw
OpenClaw
Claude Code
Claude Code
Codex
Codex
Cursor and Cursor Cloud
Cursor
+ Cloud
Gemini CLI
Gemini CLI
OpenCode
OpenCode
Pi
Pi
Hermes and Hermes Gateway
Hermes
+ Gateway
Grok Build
Grok Build
Kimi Code
Kimi Code

If it can receive a heartbeat, it's hired.

Custom processes, HTTP endpoints, and external adapter packages extend the roster. See the adapter overview for setup and capabilities.


Paperclip is right for you if

  • ✅ You want to build autonomous AI organizations
  • ✅ You coordinate many different agents (OpenClaw, Codex, Claude, Cursor) toward a common goal
  • ✅ You have 20 simultaneous Claude Code terminals open and lose track of what everyone is doing
  • ✅ You want agents running autonomously 24/7, but still want to audit work and chime in when needed
  • ✅ You want to monitor costs and enforce budgets
  • ✅ You want a process for managing agents that feels like using a task manager
  • ✅ You want to manage your autonomous businesses from your phone

The four pillars

Four things have to work for an organization of AI agents to actually produce: the tasks, the org, the training, and the infrastructure. Paperclip is built around exactly those four pillars.

The four pillars of Paperclip
Pillar Built for What it covers
Agentic Task Manager — Declare intent. Agents work. You verify the output. Everyone, daily Tasks, approvals & review gates · proactive agent coworkers · auditable routines & workflows · verify from diffs, screenshots & tests
Org Chart for Agents — Roles, permissions & boundaries for humans and agents. Managers Mixed human + agent org chart · responsibilities, delegation, specialization · governance: who can do what · scoped secrets & company boundaries · connection permissions & responsible-user identities
Agent Employee Training — Design, train & evaluate your AI employees. Enablers Skill Studio & shared org-wide skills · evals & saved test runs · active learning loops & quality metrics · performance reviews for agents · saved test inputs · skill version history & restore · reusable team templates
Agentic OS — The infrastructure that makes the work run. IT & platform Cross-provider runtime: any model, any agent · sandboxing, integrations & MCP servers · SSO, GRC, RBAC & cost controls · data privacy, internal trace collection, compounding data value · personal & shared app connections · run history & opt-in tracing

Features

🔌 Bring Your Own Agent

Any agent, any runtime, one org chart. If it can receive a heartbeat, it's hired.

🎯 Goal Alignment

Link tasks and projects to your organization goals. Agents receive the goal context behind their work.

💓 Heartbeats

Agents wake for assigned work, follow-up messages, or configured schedules. Delegation flows up and down the org chart.

💰 Cost Control

Company, agent, and project budgets. Track reported spend, get threshold alerts, and pause work at configured limits.

🏢 Multi-Organization

One deployment, many organizations. Separate tasks, agents, permissions, and activity histories for each.

🎫 Task Threads

Keep conversations, plans, blockers, files, and run history attached to the work. Assign tasks to agents or people.

🛡️ Governance

Configure review and approval stages, approve hires, and pause, reassign, or stop work when needed.

📊 Org Chart

Hierarchies, roles, reporting lines. Your agents have a boss, a title, and a job description.

📱 Mobile Ready

Monitor and manage your autonomous businesses from anywhere.

🔗 Apps & Connections

Connect services such as GitHub, Notion, and Railway, or your own MCP server. Set gateway actions to Allowed, Ask first, or Off.

👥 Shared Agents, Personal Accounts

Choose who can use a connection and which agents can access it. Managed GitHub operations can use the account of the person directing the work.

🧠 Skills & Skill Studio

Install or write shared skills, test them with saved inputs, inspect results, and restore earlier versions.

📅 Scheduled Routines

Run recurring work on a schedule or trigger it through an API or webhook. Each run has a task, an owner, and a history.

📎 Artifacts & Feedback

Find the files and documents agents produce. Preview supported formats and leave comments on specific passages in documents.

📦 Ready-Made Teams

Preview and install teams with roles, skills, projects, and routines. Choose their runtimes and make the setup your own.

Experimental Agent Chat and chat/email connectors add conversations with agents in Paperclip and through configured services such as Slack, Discord, Telegram, and AgentMail. Enable the relevant instance settings to try them.


Problems Paperclip solves

Without Paperclip With Paperclip
❌ You have 20 Claude Code tabs open and can't track which one does what. On reboot you lose everything. ✅ Tasks are ticket-based, conversations are threaded, sessions persist across reboots.
❌ You manually gather context from several places to remind your bot what you're actually doing. ✅ Context flows from the task up through the project and company goals — your agent always knows what to do and why.
❌ Folders of agent configs are disorganized and you're re-inventing task management, communication, and coordination between agents. ✅ Paperclip gives you org charts, ticketing, delegation, and governance out of the box — so you run a company, not a pile of scripts.
❌ Runaway loops waste hundreds of dollars of tokens and max your quota before you even know what happened. ✅ Spend tracking, budget alerts, and automatic pauses help you control the cost of ongoing work.
❌ You have recurring jobs (customer support, social, reports) and have to remember to manually kick them off. ✅ Routines create assigned tasks on a schedule, with outputs and run history you can inspect.
❌ You have an idea, you have to find your repo, fire up Claude Code, keep a tab open, and babysit it. ✅ Add a task in Paperclip. Your coding agent works on it until it's done. Management reviews their work.

Why Paperclip is special

Paperclip handles the hard orchestration details correctly.

Atomic task checkout. A single assignee and execution locks prevent competing runs from claiming the same task.
Persistent work context. Tasks, comments, and documents stay in Paperclip. Supporting adapters resume saved sessions across runs.
Runtime skill injection. Agents can learn Paperclip workflows and project context at runtime, without retraining.
Governance with rollback. Approval gates are enforced, config changes are revisioned, and bad changes can be rolled back safely.
Accountable connections. Human access, agent eligibility, and gateway action permissions are separate controls. Approve a call once or save a revocable rule.
Goal-aware execution. Linked tasks and projects carry goal ancestry so agents see the "why," not just a title.
Portable company templates. Export/import orgs, agents, and skills with secret scrubbing and collision handling.
Organization boundaries. Company-scoped access checks keep each organization's work, agents, and activity separate within one deployment.

What's Under the Hood

Paperclip is a full control plane, not a wrapper. Before you build any of this yourself, know that it already exists:

┌──────────────────────────────────────────────────────────────┐
│                       PAPERCLIP SERVER                       │
│                                                              │
│  ┌───────────┐  ┌───────────┐  ┌───────────┐  ┌───────────┐  │
│  │Identity & │  │  Work &   │  │ Heartbeat │  │Governance │  │
│  │  Access   │  │   Tasks   │  │ Execution │  │& Approvals│  │
│  └───────────┘  └───────────┘  └───────────┘  └───────────┘  │
│                                                              │
│  ┌───────────┐  ┌───────────┐  ┌───────────┐  ┌───────────┐  │
│  │ Org Chart │  │Workspaces │  │  Plugins  │  │  Budget   │  │
│  │ & Agents  │  │ & Runtime │  │           │  │ & Costs   │  │
│  └───────────┘  └───────────┘  └───────────┘  └───────────┘  │
│                                                              │
│  ┌───────────┐  ┌───────────┐  ┌───────────┐  ┌───────────┐  │
│  │ Routines  │  │ Secrets & │  │ Activity  │  │  Company  │  │
│  │& Schedules│  │  Storage  │  │ & Events  │  │Portability│  │
│  └───────────┘  └───────────┘  └───────────┘  └───────────┘  │
└──────────────────────────────────────────────────────────────┘
         ▲              ▲              ▲              ▲
   ┌─────┴─────┐  ┌─────┴─────┐  ┌─────┴─────┐  ┌─────┴─────┐
   │  Claude   │  │   Codex   │  │   CLI     │  │ HTTP/web  │
   │   Code    │  │           │  │  agents   │  │   bots    │
   └───────────┘  └───────────┘  └───────────┘  └───────────┘

The Systems

Identity & Access — Two deployment modes (trusted local or authenticated), human roles and permissions, agent API keys, short-lived run JWTs, company memberships, and invite flows. Responsible-user attribution follows work through delegation and supported managed connections.

Org Chart & Agents — Agents have roles, titles, reporting lines, permissions, and budgets. Adapter examples match the diagram: Claude Code, Codex, CLI agents such as Cursor/Gemini/bash, HTTP/webhook bots such as OpenClaw, and external adapter plugins. If it can receive a heartbeat, it's hired.

Work & Task System — Issues carry company/project/goal/parent links, atomic checkout with execution locks, first-class blocker dependencies, comments, documents, attachments, work products, labels, and inbox state. Search across work, review document revisions, and leave anchored feedback.

Heartbeat Execution — DB-backed wakeup queue with coalescing, budget checks, workspace resolution, secret injection, skill loading, and adapter invocation. Runs produce logs, usage records, and adapter-specific session state. Bounded recovery handles supported failures and surfaces cases that need human action.

Workspaces & Runtime — Project repositories and workspaces, optional isolated execution workspaces (git worktrees, operator branches), and runtime services (dev servers, preview URLs). Sandbox providers extend execution beyond the local host; availability depends on the configured environment and adapter.

Governance & Approvals — Board approval workflows, execution policies with review/approval stages, decision tracking, budget hard-stops, and agent pause/resume/terminate. Configured task reviews govern completion; connection action approvals govern calls through the tool gateway.

Budget & Cost Control — Token and cost tracking by company, agent, project, goal, issue, provider, and model. Scoped budget policies with warning thresholds and hard stops. Enforcement uses recorded spend; usage reporting and in-flight work can delay a stop.

Routines & Schedules — Recurring tasks with cron, webhook, and API triggers. Concurrency and catch-up policies. Each routine execution creates a tracked issue and wakes the assigned agent — no manual kick-offs needed.

Plugins — Instance-wide plugin system with out-of-process workers, capability-gated host services, job scheduling, tool exposure, and UI contributions. Extend Paperclip without forking it.

Secrets & Storage — Company secrets and per-person secret values, encrypted credential storage, local or S3-compatible file storage, attachments, and work products. Secret references supply credentials to authorized runs without copying values into ordinary agent configuration.

Activity & Events — Mutating actions, heartbeat state changes, cost events, approvals, comments, and work products are recorded as durable activity so operators can audit what happened and why.

Company Portability — Preview, export, and import organization packages with agents, skills, and optional projects, routines, tasks, and attachments. Referenced secret values are omitted; review packages before sharing because plain environment values and local paths can remain. Packages share an operating setup; full-instance recovery uses backups.


What Paperclip is not

Not just a chatbot. Conversations stay attached to tasks, plans, decisions, and outputs. Experimental Agent Chat can hand work off to assigned tasks.
Not an agent framework. We don't tell you how to build agents. We tell you how to run a company made of them.
Not just a workflow builder. Routines operate within an organization, with roles, goals, budgets, and governance.
Not a prompt manager. Agents bring their own prompts, models, and runtimes. Paperclip manages the organization they work in.
Not limited to one agent. Start with one agent and grow into a team with shared skills, delegation, and review.
Not only for code review. Coding and PR review fit alongside research, operations, content, and other work.

Quickstart

Open source. Self-hosted. No Paperclip account required. Follow the guided quickstart to set up your first agent.

Just ask your agent to install Paperclip

Share the installation guide with your agent.

Or install it yourself

With Node.js 24.11 or newer installed:

npx paperclipai@latest onboard --yes

The CLI runs from npm's cache; your instance configuration and data persist locally.

See the installation guide for managed installs, pinned versions, canary and git-ref installs, updates, rollback, service management, and uninstalling.

For an isolated manual test instance that is already initialized with a CEO agent, use test-drive. It stays in the foreground, never installs a service or creates a first task, and opens the browser only after setup succeeds:

ANTHROPIC_API_KEY=... npx paperclipai test-drive
OPENAI_API_KEY=... npx paperclipai test-drive --harness codex
OPENROUTER_API_KEY=... npx paperclipai test-drive \
  --harness opencode \
  --model openrouter/anthropic/claude-sonnet-4.5

Each run without --data-dir gets a unique, retained temporary directory; its absolute path is printed at startup. Pass --data-dir to reuse one, or --no-browser to leave the initialized instance unopened. When invoked from a linked Git worktree, test-drive also enables task execution in that worktree. See the test-drive guide for credential and reuse behavior.

Troubleshooting: private npm registry .npmrc

If this fails with an E404 for paperclipai (or similar) and you use a private npm registry (for example GitHub Packages) via a global ~/.npmrc, npx may be resolving paperclipai against that private registry instead of the public npm registry.

Diagnostic:

npm config get registry

Workaround (cross-platform; force the public npm registry for this command):

npx --registry https://registry.npmjs.org paperclipai@latest onboard --yes

That quickstart path now defaults to trusted local loopback mode for the fastest first run. To start in authenticated/private mode instead, choose a bind preset explicitly:

npx paperclipai@latest onboard --yes --bind lan
# or:
npx paperclipai@latest onboard --yes --bind tailnet

If you already have Paperclip configured, rerunning onboard keeps the existing config in place. Use npx paperclipai configure to edit settings.

Or manually:

git clone https://github.com/paperclipai/paperclip.git
cd paperclip
pnpm install
pnpm dev

This starts the UI and API at http://localhost:3100. An embedded PostgreSQL database is created automatically — no setup required.

Requirements: Node.js 24.11+, pnpm 9.15+

Local Claude and Codex subscription sign-in also needs Python 3 and the corresponding provider CLI on the Paperclip host. The Docker image includes them.

Source development also builds the native Paperclip Runner when enabled (the self-hosted default). Install a Rust toolchain, or set PAPERCLIP_RUNNER_BINARY to a compatible prebuilt runner.


FAQ

Q: Is this project maintained or just slop?

A: Paperclip is maintained by the Paperclip team. We've merged over 2,700 pull requests.


Q: What does a typical setup look like?

A: Locally, a single Node.js process manages an embedded Postgres and local file storage. For production, point it at your own Postgres and deploy however you like. Configure projects, agents, and goals — the agents take care of the rest.

For remote access, use authenticated mode with a private-network bind such as Tailscale, or deploy the persistent server with Docker. See deployment modes and the Docker guide.


Q: Can I run multiple companies?

A: Yes. A single deployment can host multiple organizations with company-scoped data and access checks.


Q: How is Paperclip different from agents like OpenClaw or Claude Code?

A: Paperclip uses those agents. It orchestrates them into a company — with org charts, budgets, goals, governance, and accountability.


Q: Why should I use Paperclip instead of just pointing my OpenClaw to Asana or Trello?

A: Agent orchestration has subtleties in how you coordinate who has work checked out, how to maintain sessions, monitoring costs, establishing governance - Paperclip does this for you.

(Bring-your-own-ticket-system is on the Roadmap)


Q: Do agents run continuously?

A: Agents wake for assigned work and follow-up messages. Optional timer heartbeats let them check for work periodically; routines create recurring tasks on their own schedules. You can also connect externally running agents such as OpenClaw. A mention alone does not assign work or wake another agent.


Development

pnpm dev              # Full dev (API + UI, watch mode)
pnpm dev:once         # Full dev without file watching
pnpm dev:server       # Server only
pnpm dev:mobile       # Serve prebuilt UI on :3101 for phones/tablets (proxies /api → :3100)
pnpm dev:both         # Run `pnpm dev` and `pnpm dev:mobile` together
pnpm build            # Build all
pnpm typecheck        # Type checking
pnpm test             # Cheap default test run (Vitest only)
pnpm test:watch       # Vitest watch mode
pnpm test:e2e         # Playwright browser suite
pnpm db:generate      # Generate DB migration
pnpm db:migrate       # Apply migrations

pnpm test does not run Playwright. Browser suites stay separate and are typically run only when working on those flows or in CI.

See doc/DEVELOPING.md for the full development guide.


Roadmap

  • ✅ Plugin system (e.g. add a knowledge base, custom tracing, queues, etc)
  • ✅ Get OpenClaw / claw-style agent employees
  • ✅ companies.sh - import and export entire organizations
  • ✅ Easy AGENTS.md configurations
  • ✅ Skills Manager, Skill Studio & Skills Store
  • ✅ Scheduled Routines
  • ✅ Better Budgeting
  • ✅ Agent Reviews and Approvals
  • ✅ Multiple Human Users
  • ✅ Cloud / Sandbox agents (e2b, Cloudflare, Daytona, Modal, Novita, boxd, self-hosted Kubernetes)
  • ✅ Artifacts & Work Products
  • ✅ Deep Planning (planning mode, revisioned plans, plan approvals)
  • ✅ Enforced Outcomes (watchdogs, recovery actions, review gates)
  • ✅ MCP Tool Gateway & Apps (governed tool access)
  • ✅ Secrets Manager with per-agent access
  • ✅ Activity log & action attribution
  • ✅ Self-healing runs & automatic recovery
  • ✅ Agent evals & feedback
  • ✅ Connected Apps
  • ✅ Personal & Shared AI Accounts
  • ✅ Shared Agents Use Personal GitHub Identities
  • ✅ Skill Version History & Restore
  • ✅ Document Comments & Revision History
  • ✅ Company-Wide Search
  • ✅ Multi-Model & Multi-Harness Teams
  • 🟡 Memory / Knowledge
  • ⚪ MAXIMIZER MODE
  • ⚪ Work Queues
  • ⚪ Self-Organization
  • ⚪ Automatic Organizational Learning
  • 🟡 Agent Chat
  • 🟡 Cloud deployments
  • ⚪ Desktop App
  • ⚪ Bring-your-own-ticket-system (Asana / Linear / Jira as on-ramps)

This is the short roadmap preview. See the full roadmap in ROADMAP.md.


Community & Plugins

Find Plugins and more at awesome-paperclip

Observability

Paperclip ships with opt-in OpenTelemetry auto-instrumentation for the server (traces only). It activates when OTEL_EXPORTER_OTLP_ENDPOINT is set and supports grpc, http/protobuf, and http/json via the standard OTEL_EXPORTER_OTLP_PROTOCOL env var. @opentelemetry/api is a normal server dependency; the SDK, auto-instrumentation, and exporter packages are optional peer dependencies — install them only if you want tracing. See doc/observability.md for install commands and the full env-var reference.

Paperclip also ships with opt-in Sentry error monitoring for the server and the browser. Set SENTRY_DSN_FRONTEND to activate it for the browser and SENTRY_DSN_BACKEND to activate it for the server — each variable is optional, and the legacy SENTRY_DSN variable still works as a fallback for either component. The supported server SDK version is @sentry/node@10.71.0; it is an optional peer dependency for the server, so install it only if you want error monitoring. The browser SDK, @sentry/browser, is pinned to the same exact version. See doc/observability.md for the install command, the privacy settings, and the full default capture set.

Telemetry

Paperclip collects anonymous usage telemetry to help us understand how the product is used and improve it. No personal information, issue content, prompts, file paths, or secrets are ever collected. Private repository references are hashed with a per-install salt before being sent.

Contributors changing emitted telemetry events should follow the Telemetry Data Contract. For proposed first-party events that are not in the generated contract yet, follow Telemetry Workflow.

Telemetry is enabled by default and can be disabled with any of the following:

Method How
Environment variable PAPERCLIP_TELEMETRY_DISABLED=1
Standard convention DO_NOT_TRACK=1
CI environments Automatically disabled when CI=true
Config file Set telemetry.enabled: false in your Paperclip config

Contributing

We welcome contributions. See the contributing guide for details.

We're hiring


Community


License

MIT © 2026 Paperclip Labs, Inc

Star History

Star History Chart

Open source under MIT. Built for people who want to get work done, not babysit agents.

S
Description
No description provided
Readme MIT
3.1 GiB
0 Stars 1 Watchers 0 Forks
Languages
TypeScript 92.5%
JavaScript 3.8%
Rust 2.7%
Python 0.3%
CSS 0.3%
Other 0.3%