DottaandPaperclip c65ab09d9f fix(recovery): wait for provider quota resets (#9635)
## Thinking Path

> - Paperclip is the open source control plane people use to manage
AI-agent companies and keep assigned work moving safely.
> - The recovery subsystem decides whether a failed agent run should
retry, wait, block for configuration, or escalate to another owner.
> - Provider usage-limit failures currently arrive as generic
`adapter_failed` results, so stranded-work reconciliation can create
takeover recovery even when the provider states that capacity will reset
later.
> - Credential and model lookup failures are also configuration
problems, not evidence that another agent should take over the task.
> - This pull request classifies those failure families at recovery time
and persists the classification on the run.
> - Quota failures now schedule a monitor for the original assignee at
the parsed reset time, or after a bounded default backoff when no reset
time is available.
> - The benefit is that transient provider capacity waits no longer wake
recovery owners, while configuration failures stop with an actionable
classification.

## Linked Issues or Issue Description

No public GitHub issue exists for this exact change.

**What happened?** When an assigned issue's latest run failed with a
provider usage-limit message such as "try again at 12:00 AM (UTC),"
recovery treated the run as generic `adapter_failed` work and could
create a takeover action. Missing credentials and `model_not_found`
failures followed the same generic path.

**Expected behavior:** Provider quota failures should keep the original
assignee and schedule a monitor for the reset time, without creating
recovery work or immediately waking another owner. Missing credentials
and model lookup failures should be classified as
`configuration_incomplete` and blocked with the configuration fix
recorded.

**Steps to reproduce:**
1. Assign and start an issue for an agent.
2. Record a failed heartbeat run with `errorCode: adapter_failed` and a
provider quota/reset message.
3. Run stranded assigned-issue reconciliation.
4. Observe that the old behavior routes the issue through generic
recovery instead of waiting for provider capacity.

Reproduced on `master` at `9af96461d`. This is a core recovery bug, not
adapter-specific, and applies to built-from-source deployments with
either embedded PGlite or Postgres.

Related work checked: #9288 adds adapter-side Claude provider-limit
classification; #5392 suppresses some recovery creation for quota-class
errors; #9634 is a broader recovery-routing change with overlapping
provider-quota behavior. This PR is the narrow recovery-service fix with
focused parsed-reset, fallback-backoff, zero-takeover, and
configuration-failure coverage.

## What Changed

- Added conservative recovery-time classification for provider quota,
missing-credential, and model-not-found adapter failures.
- Parsed provider reset timestamps with a default one-hour backoff when
no usable reset time is present.
- Persisted `provider_quota` or `configuration_incomplete` metadata on
the failed heartbeat run.
- Scheduled quota monitors for the active issue owner, including the
current review participant, without creating recovery actions or
enqueueing takeover wakes.
- Routed configuration failures to blocked recovery with actionable
evidence instead of a takeover.
- Added unit and embedded-database regression coverage for
parsed/fallback quota timing, zero CTO/recovery wake behavior, and
configuration classification.

## Verification

- `pnpm exec vitest run
server/src/services/recovery/provider-failure-classification.test.ts
server/src/__tests__/issue-recovery-actions.test.ts
server/src/__tests__/issue-monitor-scheduler.test.ts
server/src/__tests__/heartbeat-stale-queue-invalidation.test.ts` — 4
files passed, 66 tests passed.
- `pnpm --filter @paperclipai/shared typecheck` — passed.
- `pnpm --filter @paperclipai/server typecheck` — passed.

## Risks

- Recovery behavior changes for text-matched adapter failures; matching
is intentionally conservative, and unmatched failures retain the
existing generic recovery path.
- Provider reset strings do not always include a date or timezone;
parsing chooses the next future matching time and falls back to a
one-hour wait when the timestamp is unusable.
- This overlaps the provider-quota portion of broader recovery-routing
PR #9634, so only one implementation should land if both remain open.
- No schema, migration, API contract, or UI changes are included. No
documentation update is needed because this corrects internal recovery
behavior without changing operator commands or configuration.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex with model `gpt-5.4`, medium reasoning, tool use, and
code execution. The runtime does not expose its configured
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-16 09:24:38 -05:00
…
…
…

Paperclip is the app people use to manage AI agents for work.

Quickstart · Docs · GitHub · Discord · Twitter · Website

MIT License Stars Discord



Paperclip is the app people use to manage AI agents for work.

Open-source orchestration for teams of AI agents.

If OpenClaw is an employee, Paperclip is the company.

Paperclip is a Node.js server and React UI that orchestrates a team of AI agents to run a business. Bring your own agents, assign goals, and track work and costs from one dashboard.

It looks like a task manager. Under the hood: org charts, budgets, governance, goal alignment, and agent coordination.

Manage business goals, not pull requests.

Step Example
01 Define the goal "Build the #1 AI note-taking app to $1M MRR."
02 Hire the team CEO, CTO, engineers, designers, marketers — any bot, any provider.
03 Approve and run Review strategy. Set budgets. Hit go. Monitor from the dashboard.

Works
with
OpenClaw
OpenClaw
Claude
Claude Code
Codex
Codex
Cursor
Cursor
Bash
Bash
HTTP
HTTP

If it can receive a heartbeat, it's hired.


Paperclip is right for you if

  • ✅ You want to build autonomous AI companies
  • ✅ You coordinate many different agents (OpenClaw, Codex, Claude, Cursor) toward a common goal
  • ✅ You have 20 simultaneous Claude Code terminals open and lose track of what everyone is doing
  • ✅ You want agents running autonomously 24/7, but still want to audit work and chime in when needed
  • ✅ You want to monitor costs and enforce budgets
  • ✅ You want a process for managing agents that feels like using a task manager
  • ✅ You want to manage your autonomous businesses from your phone

Features

🔌 Bring Your Own Agent

Any agent, any runtime, one org chart. If it can receive a heartbeat, it's hired.

🎯 Goal Alignment

Every task traces back to the company mission. Agents know what to do and why.

💓 Heartbeats

Agents wake on a schedule, check work, and act. Delegation flows up and down the org chart.

💰 Cost Control

Monthly budgets per agent. When they hit the limit, they stop. No runaway costs.

🏢 Multi-Company

One deployment, many companies. Complete data isolation. One control plane for your portfolio.

🎫 Ticket System

Every conversation traced. Every decision explained. Full tool-call tracing and immutable audit log.

🛡️ Governance

Approve hires, override strategy, pause or terminate any agent — at any time.

📊 Org Chart

Hierarchies, roles, reporting lines. Your agents have a boss, a title, and a job description.

📱 Mobile Ready

Monitor and manage your autonomous businesses from anywhere.

Problems Paperclip solves

Without Paperclip With Paperclip
❌ You have 20 Claude Code tabs open and can't track which one does what. On reboot you lose everything. ✅ Tasks are ticket-based, conversations are threaded, sessions persist across reboots.
❌ You manually gather context from several places to remind your bot what you're actually doing. ✅ Context flows from the task up through the project and company goals — your agent always knows what to do and why.
❌ Folders of agent configs are disorganized and you're re-inventing task management, communication, and coordination between agents. ✅ Paperclip gives you org charts, ticketing, delegation, and governance out of the box — so you run a company, not a pile of scripts.
❌ Runaway loops waste hundreds of dollars of tokens and max your quota before you even know what happened. ✅ Cost tracking surfaces token budgets and throttles agents when they're out. Management prioritizes with budgets.
❌ You have recurring jobs (customer support, social, reports) and have to remember to manually kick them off. ✅ Heartbeats handle regular work on a schedule. Management supervises.
❌ You have an idea, you have to find your repo, fire up Claude Code, keep a tab open, and babysit it. ✅ Add a task in Paperclip. Your coding agent works on it until it's done. Management reviews their work.

Why Paperclip is special

Paperclip handles the hard orchestration details correctly.

Atomic execution. Task checkout and budget enforcement are atomic, so no double-work and no runaway spend.
Persistent agent state. Agents resume the same task context across heartbeats instead of restarting from scratch.
Runtime skill injection. Agents can learn Paperclip workflows and project context at runtime, without retraining.
Governance with rollback. Approval gates are enforced, config changes are revisioned, and bad changes can be rolled back safely.
Goal-aware execution. Tasks carry full goal ancestry so agents consistently see the "why," not just a title.
Portable company templates. Export/import orgs, agents, and skills with secret scrubbing and collision handling.
True multi-company isolation. Every entity is company-scoped, so one deployment can run many companies with separate data and audit trails.

What's Under the Hood

Paperclip is a full control plane, not a wrapper. Before you build any of this yourself, know that it already exists:

┌──────────────────────────────────────────────────────────────┐
│                       PAPERCLIP SERVER                       │
│                                                              │
│  ┌───────────┐  ┌───────────┐  ┌───────────┐  ┌───────────┐  │
│  │Identity & │  │  Work &   │  │ Heartbeat │  │Governance │  │
│  │  Access   │  │   Tasks   │  │ Execution │  │& Approvals│  │
│  └───────────┘  └───────────┘  └───────────┘  └───────────┘  │
│                                                              │
│  ┌───────────┐  ┌───────────┐  ┌───────────┐  ┌───────────┐  │
│  │ Org Chart │  │Workspaces │  │  Plugins  │  │  Budget   │  │
│  │ & Agents  │  │ & Runtime │  │           │  │ & Costs   │  │
│  └───────────┘  └───────────┘  └───────────┘  └───────────┘  │
│                                                              │
│  ┌───────────┐  ┌───────────┐  ┌───────────┐  ┌───────────┐  │
│  │ Routines  │  │ Secrets & │  │ Activity  │  │  Company  │  │
│  │& Schedules│  │  Storage  │  │ & Events  │  │Portability│  │
│  └───────────┘  └───────────┘  └───────────┘  └───────────┘  │
└──────────────────────────────────────────────────────────────┘
         ▲              ▲              ▲              ▲
   ┌─────┴─────┐  ┌─────┴─────┐  ┌─────┴─────┐  ┌─────┴─────┐
   │  Claude   │  │   Codex   │  │   CLI     │  │ HTTP/web  │
   │   Code    │  │           │  │  agents   │  │   bots    │
   └───────────┘  └───────────┘  └───────────┘  └───────────┘

The Systems

Identity & Access — Two deployment modes (trusted local or authenticated), board users, agent API keys, short-lived run JWTs, company memberships, invite flows, and OpenClaw onboarding. Every mutating request is traced to an actor.

Org Chart & Agents — Agents have roles, titles, reporting lines, permissions, and budgets. Adapter examples match the diagram: Claude Code, Codex, CLI agents such as Cursor/Gemini/bash, HTTP/webhook bots such as OpenClaw, and external adapter plugins. If it can receive a heartbeat, it's hired.

Work & Task System — Issues carry company/project/goal/parent links, atomic checkout with execution locks, first-class blocker dependencies, comments, documents, attachments, work products, labels, and inbox state. No double-work, no lost context.

Heartbeat Execution — DB-backed wakeup queue with coalescing, budget checks, workspace resolution, secret injection, skill loading, and adapter invocation. Runs produce structured logs, cost events, session state, and audit trails. Recovery handles orphaned runs automatically.

Workspaces & Runtime — Project workspaces, isolated execution workspaces (git worktrees, operator branches), and runtime services (dev servers, preview URLs). Agents work in the right directory with the right context every time.

Governance & Approvals — Board approval workflows, execution policies with review/approval stages, decision tracking, budget hard-stops, agent pause/resume/terminate, and full audit logging. Nothing ships without your sign-off.

Budget & Cost Control — Token and cost tracking by company, agent, project, goal, issue, provider, and model. Scoped budget policies with warning thresholds and hard stops. Overspend pauses agents and cancels queued work automatically.

Routines & Schedules — Recurring tasks with cron, webhook, and API triggers. Concurrency and catch-up policies. Each routine execution creates a tracked issue and wakes the assigned agent — no manual kick-offs needed.

Plugins — Instance-wide plugin system with out-of-process workers, capability-gated host services, job scheduling, tool exposure, and UI contributions. Extend Paperclip without forking it.

Secrets & Storage — Instance and company secrets, encrypted local storage, provider-backed object storage, attachments, and work products. Sensitive values stay out of prompts unless a scoped run explicitly needs them.

Activity & Events — Mutating actions, heartbeat state changes, cost events, approvals, comments, and work products are recorded as durable activity so operators can audit what happened and why.

Company Portability — Export and import entire organizations — agents, skills, projects, routines, and issues — with secret scrubbing and collision handling. One deployment, many companies, complete data isolation.


What Paperclip is not

Not a chatbot. Agents have jobs, not chat windows.
Not an agent framework. We don't tell you how to build agents. We tell you how to run a company made of them.
Not a workflow builder. No drag-and-drop pipelines. Paperclip models companies — with org charts, goals, budgets, and governance.
Not a prompt manager. Agents bring their own prompts, models, and runtimes. Paperclip manages the organization they work in.
Not a single-agent tool. This is for teams. If you have one agent, you probably don't need Paperclip. If you have twenty — you definitely do.
Not a code review tool. Paperclip orchestrates work, not pull requests. Bring your own review process.

Quickstart

Open source. Self-hosted. No Paperclip account required.

npx paperclipai onboard --yes

Troubleshooting: private npm registry .npmrc

If this fails with an E404 for paperclipai (or similar) and you use a private npm registry (for example GitHub Packages) via a global ~/.npmrc, npx may be resolving paperclipai against that private registry instead of the public npm registry.

Diagnostic:

npm config get registry

Workaround (cross-platform; force the public npm registry for this command):

npx --registry https://registry.npmjs.org paperclipai onboard --yes

That quickstart path now defaults to trusted local loopback mode for the fastest first run. To start in authenticated/private mode instead, choose a bind preset explicitly:

npx paperclipai onboard --yes --bind lan
# or:
npx paperclipai onboard --yes --bind tailnet

If you already have Paperclip configured, rerunning onboard keeps the existing config in place. Use paperclipai configure to edit settings.

Or manually:

git clone https://github.com/paperclipai/paperclip.git
cd paperclip
pnpm install
pnpm dev

This starts the API server at http://localhost:3100. An embedded PostgreSQL database is created automatically — no setup required.

Requirements: Node.js 20+, pnpm 9.15+


FAQ

What does a typical setup look like? Locally, a single Node.js process manages an embedded Postgres and local file storage. For production, point it at your own Postgres and deploy however you like. Configure projects, agents, and goals — the agents take care of the rest.

If you're a solo entrepreneur you can use Tailscale to access Paperclip on the go. Then later you can deploy to e.g. Vercel when you need it.

Can I run multiple companies? Yes. A single deployment can run an unlimited number of companies with complete data isolation.

How is Paperclip different from agents like OpenClaw or Claude Code? Paperclip uses those agents. It orchestrates them into a company — with org charts, budgets, goals, governance, and accountability.

Why should I use Paperclip instead of just pointing my OpenClaw to Asana or Trello? Agent orchestration has subtleties in how you coordinate who has work checked out, how to maintain sessions, monitoring costs, establishing governance - Paperclip does this for you.

(Bring-your-own-ticket-system is on the Roadmap)

Do agents run continuously? By default, agents run on scheduled heartbeats and event-based triggers (task assignment, @-mentions). You can also hook in continuous agents like OpenClaw. You bring your agent and Paperclip coordinates.


Development

pnpm dev              # Full dev (API + UI, watch mode)
pnpm dev:once         # Full dev without file watching
pnpm dev:server       # Server only
pnpm build            # Build all
pnpm typecheck        # Type checking
pnpm test             # Cheap default test run (Vitest only)
pnpm test:watch       # Vitest watch mode
pnpm test:e2e         # Playwright browser suite
pnpm db:generate      # Generate DB migration
pnpm db:migrate       # Apply migrations

pnpm test does not run Playwright. Browser suites stay separate and are typically run only when working on those flows or in CI.

See doc/DEVELOPING.md for the full development guide.


Roadmap

  • ✅ Plugin system (e.g. add a knowledge base, custom tracing, queues, etc)
  • ✅ Get OpenClaw / claw-style agent employees
  • ✅ companies.sh - import and export entire organizations
  • ✅ Easy AGENTS.md configurations
  • ✅ Skills Manager
  • ✅ Scheduled Routines
  • ✅ Better Budgeting
  • ✅ Agent Reviews and Approvals
  • ✅ Multiple Human Users
  • ⚪ Cloud / Sandbox agents (e.g. Cursor / e2b / Novita agents)
  • ⚪ Artifacts & Work Products
  • ⚪ Memory / Knowledge
  • ⚪ Enforced Outcomes
  • ⚪ MAXIMIZER MODE
  • ⚪ Deep Planning
  • ⚪ Work Queues
  • ⚪ Self-Organization
  • ⚪ Automatic Organizational Learning
  • ⚪ CEO Chat
  • ⚪ Cloud deployments
  • ⚪ Desktop App

This is the short roadmap preview. See the full roadmap in ROADMAP.md.


Community & Plugins

Find Plugins and more at awesome-paperclip

Observability

Paperclip ships with opt-in OpenTelemetry auto-instrumentation for the server (traces only). It activates when OTEL_EXPORTER_OTLP_ENDPOINT is set and supports grpc, http/protobuf, and http/json via the standard OTEL_EXPORTER_OTLP_PROTOCOL env var. The @opentelemetry/* packages are optional peer dependencies — install them only if you want tracing. See doc/observability.md for install commands and the full env-var reference.

Telemetry

Paperclip collects anonymous usage telemetry to help us understand how the product is used and improve it. No personal information, issue content, prompts, file paths, or secrets are ever collected. Private repository references are hashed with a per-install salt before being sent.

Contributors changing emitted telemetry events should follow the Telemetry Data Contract. For proposed first-party events that are not in the generated contract yet, follow Telemetry Workflow.

Telemetry is enabled by default and can be disabled with any of the following:

Method How
Environment variable PAPERCLIP_TELEMETRY_DISABLED=1
Standard convention DO_NOT_TRACK=1
CI environments Automatically disabled when CI=true
Config file Set telemetry.enabled: false in your Paperclip config

Contributing

We welcome contributions. See the contributing guide for details.


Community


License

MIT © 2026 Paperclip Labs, Inc

Star History

Star History Chart



Open source under MIT. Built for people who want to get work done, not babysit agents.

S
Description
No description provided
Readme MIT
1.6 GiB
0 Stars 1 Watchers 0 Forks
Languages
TypeScript 93.2%
Rust 3.1%
JavaScript 2.8%
Shell 0.4%
CSS 0.3%