Dotta 7b094724e6 fix(runner): recover native sessions across restarts (#12845)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The Paperclip Runner keeps durable run and provider state outside
one server process.
> - A server restart can leave that runner alive or can interrupt it
after a provider checkpoint.
> - The old startup path used handoff intent and PID evidence, but it
did not reconstruct native ownership.
> - That gap could block the issue, create a replacement run, or start
duplicate provider work.
> - This pull request adds durable same-run recovery for coordinated and
uncoordinated restarts.
> - The benefit is exact recovery of the run, runner, session, provider,
steering, and finalization state.

## Linked Issues or Issue Description

Refs #9628. That pull request added earlier local-adapter hot-restart
work. This change adds native PRP authority reconstruction and same-run
provider resume.

Refs #10935. That pull request handles missing hot-restart snapshots.
This change also supports hard restarts with no snapshot.

Refs #11624. That pull request prevents unsafe retry after an adopted
legacy process exits. This change reconciles native terminal evidence
before provider recovery.

Refs #12070. That pull request improves process liveness checks. This
change also binds recovery to a process-start fingerprint and fails
closed on ambiguity.

**What happened?**

The server could record hot-restart intent, but startup did not rebuild
native runner ownership. A live runner could not re-register its PRP
authority. A dead runner could not resume the exact native and provider
session on the same heartbeat run. Generic recovery could then block the
issue or create replacement work.

**Expected behavior**

A live native runner must reconnect with the same PID and logical
identities. A dead runner must resume the same durable session and
heartbeat run with only a new operating-system PID. A proposed or
terminal result must finalize once before any provider turn starts.
Ambiguous process or session evidence must stay blocked without a signal
or duplicate spawn.

**Steps to reproduce**

1. Start a Paperclip Runner heartbeat and wait for an active provider
turn.
2. Restart only the Paperclip server, with or without a hot-restart
marker.
3. Observe that the old startup path does not reconstruct the native
control-plane authority.
4. Kill both the server and runner after a provider checkpoint.
5. Observe that the old path cannot resume the exact native session on
the original heartbeat run.

**Paperclip version or commit**

The defect was reproduced from commit
`1991f31fd53e7f7794d5c2e4b93be384ade2b41d`. This branch is rebased onto
the current `master`.

**Deployment mode**

Local development and self-hosted server deployments that use the local
Paperclip Runner.

## What Changed

- Added correlated hot-restart requests and version-compatible native
handoff fields.
- Added controller boot identity, process-start identity, controller
generation, recovery state, request id, and bounded history to the
native finalization ledger.
- Added transactional recovery claims for live-runner reattach,
dead-runner resume, and incomplete bootstrap.
- Added fail-closed ownership takeover rules and process identity
validation.
- Added live runner adoption to the local runner transport without a
duplicate spawn.
- Added same-run provider checkpoint resume and legacy retry-row
compatibility.
- Reconciled proposed and terminal results before runner or provider
recovery.
- Bound the HTTP and PRP listener before startup recovery and delayed
scheduling and generic reapers until classification completes.
- Added restart-aware health diagnostics, run-log recovery transitions,
durable runner diagnostics, and bounded shutdown finalizer draining.
- Moved restart-survivable diagnostics into runner-owned, pre-redacted
bounded writes; raw stdout and stderr are never persisted.
- Added process-start fencing for controller, runner, and provider PIDs;
startup classifies every candidate without an implicit cap.
- Added crash-recoverable, contention-safe development restart-request
coordination and failed-startup listener cleanup.
- Added a credential-free real-process restart suite for eight restart,
scale, and identity scenarios.
- Documented native restart operation, persistence, diagnostics, and
verification.

## Verification

- The documented native restart commands passed. They ran eight
real-process/database recovery scenarios and the live runner adoption
transport test.
- Native executor tests passed: 111 tests.
- Heartbeat recovery tests passed: 124 tests.
- Hot restart, health, and shutdown tests passed: 52 tests.
- The broader affected server suite passed: 350 tests.
- Focused native recovery and startup tests passed: 49 tests.
- Runner transport and control-plane tests passed: 63 tests.
- Runner-owned diagnostic tests passed for write-time bounding,
credential redaction, private file modes, and raw stream
non-persistence.
- Development restart coordination tests passed: 11 tests.
- Database migration checks and the partial-application/replay
regression test passed.
- Server, database, and Paperclip Runner typechecks passed.
- `git diff --check` passed.
- Full Paperclip PR CI passed, including build, canary, all five general
server shards, all five serialized server shards, all three browser E2E
shards, workspace suites, and release-registry verification.
- Greptile completed at 5/5 with no outstanding findings,
recommendations, follow-ups, or open review threads.

## Risks

- Moderate risk. This changes startup ordering and ownership transfer
for active native runs.
- The migration adds nullable columns and does not rewrite existing
rows.
- Recovery fails closed when process or durable session identity is
incomplete or contradictory.
- The first implementation supports the local Paperclip Runner. Remote
targets keep their existing behavior.
- The real-process suite covers cleanup and asserts that no runner or
provider process survives each test.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex with GPT-5. The runtime did not expose a more specific
model revision or context-window size. Repository editing, shell
execution, database tests, and real-process test execution were enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-04 15:03:53 -05:00
2026-03-07 02:57:28 +09:00

Paperclip is the app people use to manage AI agents for work.

Quickstart · Docs · GitHub · Discord · Twitter · Website

MIT License Stars Star History Rank Discord



Paperclip is the app people use to manage AI agents for work.

Open-source orchestration for teams of AI agents.

If OpenClaw is an employee, Paperclip is the company.

Paperclip is a Node.js server and React UI that orchestrates a team of AI agents to run a business. Bring your own agents, assign goals, and track work and costs from one dashboard.

It looks like a task manager. Under the hood: org charts, budgets, governance, goal alignment, and agent coordination.

Manage business goals, not pull requests.

Step Example
01 Define the goal "Build the #1 AI note-taking app to $1M MRR."
02 Hire the team CEO, CTO, engineers, designers, marketers — any bot, any provider.
03 Approve and run Review strategy. Set budgets. Hit go. Monitor from the dashboard.

Works
with
OpenClaw
OpenClaw
Claude
Claude Code
Codex
Codex
Cursor
Cursor
Bash
Bash
HTTP
HTTP

If it can receive a heartbeat, it's hired.


Paperclip is right for you if

  • ✅ You want to build autonomous AI organizations
  • ✅ You coordinate many different agents (OpenClaw, Codex, Claude, Cursor) toward a common goal
  • ✅ You have 20 simultaneous Claude Code terminals open and lose track of what everyone is doing
  • ✅ You want agents running autonomously 24/7, but still want to audit work and chime in when needed
  • ✅ You want to monitor costs and enforce budgets
  • ✅ You want a process for managing agents that feels like using a task manager
  • ✅ You want to manage your autonomous businesses from your phone

The four pillars

Four things have to work for an organization of AI agents to actually produce: the tasks, the org, the training, and the infrastructure. Paperclip is built around exactly those four pillars.

The four pillars of Paperclip
Pillar Built for What it covers
Agentic Task Manager — Declare intent. Agents work. You verify the output. Everyone, daily Tasks, approvals & review gates · proactive agent coworkers · auditable routines & workflows · verify from diffs, screenshots & tests
Org Chart for Agents — Roles, permissions & boundaries for humans and agents. Managers Mixed human + agent org chart · responsibilities, delegation, specialization · governance: who can do what · scoped secrets & company boundaries
Agent Employee Training — Design, train & evaluate your AI employees. Enablers Skill Studio & shared org-wide skills · evals & saved test runs · active learning loops & quality metrics · performance reviews for agents
Agentic OS — The infrastructure that makes the work run. IT & platform Cross-provider runtime: any model, any agent · sandboxing, integrations & MCP servers · SSO, GRC, RBAC & cost controls · data privacy, internal trace collection, compounding data value

Features

🔌 Bring Your Own Agent

Any agent, any runtime, one org chart. If it can receive a heartbeat, it's hired.

🎯 Goal Alignment

Every task traces back to the organization mission. Agents know what to do and why.

💓 Heartbeats

Agents wake on a schedule, check work, and act. Delegation flows up and down the org chart.

💰 Cost Control

Monthly budgets per agent. When they hit the limit, they stop. No runaway costs.

🏢 Multi-Organization

One deployment, many organizations. Complete data isolation. One control plane for your portfolio.

🎫 Ticket System

Every conversation traced. Every decision explained. Full tool-call tracing and immutable audit log.

🛡️ Governance

Approve hires, override strategy, pause or terminate any agent — at any time.

📊 Org Chart

Hierarchies, roles, reporting lines. Your agents have a boss, a title, and a job description.

📱 Mobile Ready

Monitor and manage your autonomous businesses from anywhere.

Problems Paperclip solves

Without Paperclip With Paperclip
❌ You have 20 Claude Code tabs open and can't track which one does what. On reboot you lose everything. ✅ Tasks are ticket-based, conversations are threaded, sessions persist across reboots.
❌ You manually gather context from several places to remind your bot what you're actually doing. ✅ Context flows from the task up through the project and company goals — your agent always knows what to do and why.
❌ Folders of agent configs are disorganized and you're re-inventing task management, communication, and coordination between agents. ✅ Paperclip gives you org charts, ticketing, delegation, and governance out of the box — so you run a company, not a pile of scripts.
❌ Runaway loops waste hundreds of dollars of tokens and max your quota before you even know what happened. ✅ Cost tracking surfaces token budgets and throttles agents when they're out. Management prioritizes with budgets.
❌ You have recurring jobs (customer support, social, reports) and have to remember to manually kick them off. ✅ Heartbeats handle regular work on a schedule. Management supervises.
❌ You have an idea, you have to find your repo, fire up Claude Code, keep a tab open, and babysit it. ✅ Add a task in Paperclip. Your coding agent works on it until it's done. Management reviews their work.

Why Paperclip is special

Paperclip handles the hard orchestration details correctly.

Atomic execution. Task checkout and budget enforcement are atomic, so no double-work and no runaway spend.
Persistent agent state. Agents resume the same task context across heartbeats instead of restarting from scratch.
Runtime skill injection. Agents can learn Paperclip workflows and project context at runtime, without retraining.
Governance with rollback. Approval gates are enforced, config changes are revisioned, and bad changes can be rolled back safely.
Goal-aware execution. Tasks carry full goal ancestry so agents consistently see the "why," not just a title.
Portable company templates. Export/import orgs, agents, and skills with secret scrubbing and collision handling.
True multi-organization isolation. Every entity is company-scoped, so one deployment can run many companies with separate data and audit trails.

What's Under the Hood

Paperclip is a full control plane, not a wrapper. Before you build any of this yourself, know that it already exists:

┌──────────────────────────────────────────────────────────────┐
│                       PAPERCLIP SERVER                       │
│                                                              │
│  ┌───────────┐  ┌───────────┐  ┌───────────┐  ┌───────────┐  │
│  │Identity & │  │  Work &   │  │ Heartbeat │  │Governance │  │
│  │  Access   │  │   Tasks   │  │ Execution │  │& Approvals│  │
│  └───────────┘  └───────────┘  └───────────┘  └───────────┘  │
│                                                              │
│  ┌───────────┐  ┌───────────┐  ┌───────────┐  ┌───────────┐  │
│  │ Org Chart │  │Workspaces │  │  Plugins  │  │  Budget   │  │
│  │ & Agents  │  │ & Runtime │  │           │  │ & Costs   │  │
│  └───────────┘  └───────────┘  └───────────┘  └───────────┘  │
│                                                              │
│  ┌───────────┐  ┌───────────┐  ┌───────────┐  ┌───────────┐  │
│  │ Routines  │  │ Secrets & │  │ Activity  │  │  Company  │  │
│  │& Schedules│  │  Storage  │  │ & Events  │  │Portability│  │
│  └───────────┘  └───────────┘  └───────────┘  └───────────┘  │
└──────────────────────────────────────────────────────────────┘
         ▲              ▲              ▲              ▲
   ┌─────┴─────┐  ┌─────┴─────┐  ┌─────┴─────┐  ┌─────┴─────┐
   │  Claude   │  │   Codex   │  │   CLI     │  │ HTTP/web  │
   │   Code    │  │           │  │  agents   │  │   bots    │
   └───────────┘  └───────────┘  └───────────┘  └───────────┘

The Systems

Identity & Access — Two deployment modes (trusted local or authenticated), board users, agent API keys, short-lived run JWTs, company memberships, invite flows, and OpenClaw onboarding. Every mutating request is traced to an actor.

Org Chart & Agents — Agents have roles, titles, reporting lines, permissions, and budgets. Adapter examples match the diagram: Claude Code, Codex, CLI agents such as Cursor/Gemini/bash, HTTP/webhook bots such as OpenClaw, and external adapter plugins. If it can receive a heartbeat, it's hired.

Work & Task System — Issues carry company/project/goal/parent links, atomic checkout with execution locks, first-class blocker dependencies, comments, documents, attachments, work products, labels, and inbox state. No double-work, no lost context.

Heartbeat Execution — DB-backed wakeup queue with coalescing, budget checks, workspace resolution, secret injection, skill loading, and adapter invocation. Runs produce structured logs, cost events, session state, and audit trails. Recovery handles orphaned runs automatically.

Workspaces & Runtime — Project workspaces, isolated execution workspaces (git worktrees, operator branches), and runtime services (dev servers, preview URLs). Agents work in the right directory with the right context every time.

Governance & Approvals — Board approval workflows, execution policies with review/approval stages, decision tracking, budget hard-stops, agent pause/resume/terminate, and full audit logging. Nothing ships without your sign-off.

Budget & Cost Control — Token and cost tracking by company, agent, project, goal, issue, provider, and model. Scoped budget policies with warning thresholds and hard stops. Overspend pauses agents and cancels queued work automatically.

Routines & Schedules — Recurring tasks with cron, webhook, and API triggers. Concurrency and catch-up policies. Each routine execution creates a tracked issue and wakes the assigned agent — no manual kick-offs needed.

Plugins — Instance-wide plugin system with out-of-process workers, capability-gated host services, job scheduling, tool exposure, and UI contributions. Extend Paperclip without forking it.

Secrets & Storage — Instance and company secrets, encrypted local storage, provider-backed object storage, attachments, and work products. Sensitive values stay out of prompts unless a scoped run explicitly needs them.

Activity & Events — Mutating actions, heartbeat state changes, cost events, approvals, comments, and work products are recorded as durable activity so operators can audit what happened and why.

Company Portability — Export and import entire organizations — agents, skills, projects, routines, and issues — with secret scrubbing and collision handling. One deployment, many companies, complete data isolation.


What Paperclip is not

Not a chatbot. Agents have jobs, not chat windows.
Not an agent framework. We don't tell you how to build agents. We tell you how to run a company made of them.
Not a workflow builder. No drag-and-drop pipelines. Paperclip models companies — with org charts, goals, budgets, and governance.
Not a prompt manager. Agents bring their own prompts, models, and runtimes. Paperclip manages the organization they work in.
Not a single-agent tool. This is for teams. If you have one agent, you probably don't need Paperclip. If you have twenty — you definitely do.
Not a code review tool. Paperclip orchestrates work, not pull requests. Bring your own review process.

Quickstart

Open source. Self-hosted. No Paperclip account required.

curl -fsSLO https://paperclip.ing/install.sh
curl -fsSLO https://paperclip.ing/install.sh.sha256
if command -v sha256sum >/dev/null 2>&1; then
  sha256sum -c install.sh.sha256
else
  shasum -a 256 -c install.sh.sha256
fi
bash install.sh

The installer ensures Node.js 24.11 or newer is available, installs a managed Paperclip CLI under ~/.paperclip/cli, and starts interactive onboarding. It can also install Paperclip as a background service on supported Linux and macOS systems. The checksum detects transfer or publishing mistakes, but it is served from the same origin as the script; use a release-tag or commit-pinned GitHub copy when you need an independently hosted source.

For a non-interactive managed install:

curl -fsSL https://paperclip.ing/install.sh | bash -s -- --no-prompt --no-onboard
paperclipai onboard --yes

The piped form requires supported Node.js, npm, and npx to already be present. If Node.js bootstrap is required, download and review install.sh before running it so no privileged dependency-install command is accepted through a pipe.

To try Paperclip without installing anything permanently:

npx --registry https://registry.npmjs.org paperclipai onboard --yes

Troubleshooting: private npm registry .npmrc

If this fails with an E404 for paperclipai (or similar) and you use a private npm registry (for example GitHub Packages) via a global ~/.npmrc, npx may be resolving paperclipai against that private registry instead of the public npm registry.

Diagnostic:

npm config get registry

Workaround (cross-platform; force the public npm registry for this command):

npx --registry https://registry.npmjs.org paperclipai onboard --yes

That quickstart path now defaults to trusted local loopback mode for the fastest first run. To start in authenticated/private mode instead, choose a bind preset explicitly:

paperclipai onboard --yes --bind lan
# or:
paperclipai onboard --yes --bind tailnet

If you already have Paperclip configured, rerunning onboard keeps the existing config in place. Use paperclipai configure to edit settings.

See doc/INSTALLING.md for pinned versions, canary and git-ref installs, updates, rollback, service management, and uninstalling.

Or manually:

git clone https://github.com/paperclipai/paperclip.git
cd paperclip
pnpm install
pnpm dev

This starts the API server at http://localhost:3100. An embedded PostgreSQL database is created automatically — no setup required.

Requirements: Node.js 24.11+, pnpm 9.15+


FAQ

What does a typical setup look like? Locally, a single Node.js process manages an embedded Postgres and local file storage. For production, point it at your own Postgres and deploy however you like. Configure projects, agents, and goals — the agents take care of the rest.

If you're a solo entrepreneur you can use Tailscale to access Paperclip on the go. Then later you can deploy to e.g. Vercel when you need it.

Can I run multiple companies? Yes. A single deployment can run an unlimited number of companies with complete data isolation.

How is Paperclip different from agents like OpenClaw or Claude Code? Paperclip uses those agents. It orchestrates them into a company — with org charts, budgets, goals, governance, and accountability.

Why should I use Paperclip instead of just pointing my OpenClaw to Asana or Trello? Agent orchestration has subtleties in how you coordinate who has work checked out, how to maintain sessions, monitoring costs, establishing governance - Paperclip does this for you.

(Bring-your-own-ticket-system is on the Roadmap)

Do agents run continuously? By default, agents run on scheduled heartbeats and event-based triggers (task assignment, @-mentions). You can also hook in continuous agents like OpenClaw. You bring your agent and Paperclip coordinates.


Development

pnpm dev              # Full dev (API + UI, watch mode)
pnpm dev:once         # Full dev without file watching
pnpm dev:server       # Server only
pnpm dev:mobile       # Serve prebuilt UI on :3101 for phones/tablets (proxies /api → :3100)
pnpm dev:both         # Run `pnpm dev` and `pnpm dev:mobile` together
pnpm build            # Build all
pnpm typecheck        # Type checking
pnpm test             # Cheap default test run (Vitest only)
pnpm test:watch       # Vitest watch mode
pnpm test:e2e         # Playwright browser suite
pnpm db:generate      # Generate DB migration
pnpm db:migrate       # Apply migrations

pnpm test does not run Playwright. Browser suites stay separate and are typically run only when working on those flows or in CI.

See doc/DEVELOPING.md for the full development guide.


Roadmap

  • ✅ Plugin system (e.g. add a knowledge base, custom tracing, queues, etc)
  • ✅ Get OpenClaw / claw-style agent employees
  • ✅ companies.sh - import and export entire organizations
  • ✅ Easy AGENTS.md configurations
  • ✅ Skills Manager, Skill Studio & Skills Store
  • ✅ Scheduled Routines
  • ✅ Better Budgeting
  • ✅ Agent Reviews and Approvals
  • ✅ Multiple Human Users
  • ✅ Cloud / Sandbox agents (e2b, Cloudflare, Daytona, Modal, Novita, self-hosted Kubernetes)
  • ✅ Artifacts & Work Products
  • ✅ Deep Planning (planning mode, revisioned plans, plan approvals)
  • ✅ Enforced Outcomes (watchdogs, recovery actions, review gates)
  • ✅ MCP Tool Gateway & Apps (governed tool access)
  • ✅ Secrets Manager with per-agent access
  • ✅ Activity log & action attribution
  • ✅ Self-healing runs & automatic recovery
  • ✅ Agent evals & feedback
  • ⚪ Memory / Knowledge
  • ⚪ MAXIMIZER MODE
  • ⚪ Work Queues
  • ⚪ Self-Organization
  • ⚪ Automatic Organizational Learning
  • ⚪ CEO Chat
  • 🟡 Cloud deployments (multi-tenant isolation & company Import/Export shipped)
  • ⚪ Desktop App
  • ⚪ Bring-your-own-ticket-system (Asana / Linear / Jira as on-ramps)
  • ⚪ Connected Apps (one-click integrations, e.g. Vercel)

This is the short roadmap preview. See the full roadmap in ROADMAP.md.


Community & Plugins

Find Plugins and more at awesome-paperclip

Observability

Paperclip ships with opt-in OpenTelemetry auto-instrumentation for the server (traces only). It activates when OTEL_EXPORTER_OTLP_ENDPOINT is set and supports grpc, http/protobuf, and http/json via the standard OTEL_EXPORTER_OTLP_PROTOCOL env var. @opentelemetry/api is a normal server dependency; the SDK, auto-instrumentation, and exporter packages are optional peer dependencies — install them only if you want tracing. See doc/observability.md for install commands and the full env-var reference.

Paperclip also ships with opt-in Sentry error monitoring for the server and the browser. Set SENTRY_DSN_FRONTEND to activate it for the browser and SENTRY_DSN_BACKEND to activate it for the server — each variable is optional, and the legacy SENTRY_DSN variable still works as a fallback for either component. The supported server SDK version is @sentry/node@10.71.0; it is an optional peer dependency for the server, so install it only if you want error monitoring. The browser SDK, @sentry/browser, is pinned to the same exact version. See doc/observability.md for the install command, the privacy settings, and the full default capture set.

Telemetry

Paperclip collects anonymous usage telemetry to help us understand how the product is used and improve it. No personal information, issue content, prompts, file paths, or secrets are ever collected. Private repository references are hashed with a per-install salt before being sent.

Contributors changing emitted telemetry events should follow the Telemetry Data Contract. For proposed first-party events that are not in the generated contract yet, follow Telemetry Workflow.

Telemetry is enabled by default and can be disabled with any of the following:

Method How
Environment variable PAPERCLIP_TELEMETRY_DISABLED=1
Standard convention DO_NOT_TRACK=1
CI environments Automatically disabled when CI=true
Config file Set telemetry.enabled: false in your Paperclip config

Contributing

We welcome contributions. See the contributing guide for details.


Community


License

MIT © 2026 Paperclip Labs, Inc

Star History

Star History Chart

Open source under MIT. Built for people who want to get work done, not babysit agents.

S
Description
No description provided
Readme MIT
2 GiB
0 Stars 1 Watchers 0 Forks
Languages
TypeScript 92.3%
JavaScript 3.9%
Rust 2.7%
Python 0.4%
CSS 0.3%
Other 0.3%