mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-15 12:28:10 +02:00
DottaandPaperclip
fd8c6b920a
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native tasks can pause while a human decides whether to allow a connection action. > - The next turn needs both the saved decision and complete accounting for the previous turn. > - Cancellation could discard final usage, and complete direct Claude API receipts could remain unpriced. > - A stale blocked or review report could also request approval again after the original card was declined. > - This pull request retains shutdown accounting and rejects approval waits bound to an already resolved action. > - The benefit is reliable continuation with the existing budget and approval controls. ## Linked Issues or Issue Description Refs #15420. Related: #15312 addresses requester ownership during dispatch. This change addresses receipt capture and final-response validation. ## What Changed - Retain usage events after native cancellation. Continue to reject late provider messages and work. - Drain same-turn accounting and terminal events for at most fifteen seconds after a durable governed wait. Keep incomplete accounting blocked. - Estimate complete, unpriced, direct Anthropic API receipts for the exact `claude-sonnet-5` model. Record the rate version and assumptions. Use the one-hour cache-write rate when the receipt lacks cache TTL. - Bind stale approval reports to exact interaction, action-request, or invocation IDs in the same company, task, agent, and run. Cover blocked, review, and response-wake reports. Keep independent reviews valid. - Fence checkpoint and result writes after a controller detaches for restart, including operations waiting for a database lock. Reject stale successful returns before certifying accounting. - Allow bounded subscription teardown only after retaining an actual provider terminal. - Journal the exact governed-wait trigger and disposition before provider interruption. Recover that wait independently of a later saved answer, replay retained accounting, and reject mismatched or unproven terminal evidence. - Give settling governed turns a bounded window before shutdown detaches their controller. - Keep fuzzy external app matches alongside installed capability matches instead of forcing an unrelated provider question for a generic query. - Return up to twenty exact active catalog tool names after an invalid request, after eligibility checks; still reject the request without granting access or creating an approval. - Require retained provider terminal proof before settling a governed wait, including when complete usage arrives before stream closure/error/timeout. Retain harmless numbered cancellation events so restart replay stays contiguous. - Isolate accounting-test OpenCode config from the host plugin directory. - Add regression coverage and document the accounting, restart and connection-search behavior. ## Verification Current PR source: `0cf08efd75f8fb23f7989beda6dbda92587088bf`. The live matrix below measured frozen `09a776bcb1f77422156f8be11e13f8c29f43e7f7`; later review fixes are verified separately and do not relabel those runs. - Repository typecheck and build pass. - Current runner runtime and cancellation suites: 191 tests pass. Six new regressions cover stream end/error/timeout without provider-stop proof and contiguous cancellation acknowledgement/request replay; all six failed before the fix. The existing bounded cleanup case now explicitly supplies terminal proof. All 31 adapter accounting tests pass with isolated fixture config. - New checkpoint-rebinding and approval-criterion suites: 65 tests pass; runner HTTP integration: 29 tests pass. Unchanged executor/control-plane suites: 640 tests pass; database-backed connection suites: 68 tests pass. - Current retained evidence verification covers 222 file hashes across all fifteen original result artifacts. The three-case [published report](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-37651896525-1/index.html) and all eight screenshot hashes verify. The [three-case recovery report](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-37662587356-1/index.html) and all six screenshots also verify; the nine-result campaign did not publish. - Evaluation support: 1,805 Vitest tests pass, one skipped; 128 Node tests pass. Eval typecheck and catalog discovery pass. - Full local repository unit run was interrupted before the follow-up edits after three tool-access failures and one runner HTTP failure. Those failures pass in isolation; the 09a full run was interrupted after one rapid Slack callback-ordering failure and seven skill-service failures. All eight pass both isolated and with full-runner environment settings, and all 74 skill-service tests pass together; the subsequent full run reported two 15-second OpenCode accounting timeouts and was stopped with exit 130 to apply review fixes. The timeouts reproduce while copying this host’s 61 MB OpenCode config. All 31 tests pass after isolating config inside each fixture without increasing timeouts or changing assertions. A complete local full-suite pass is not claimed. Repository-wide CI also passes on the final review-fix head in [run 37669185952](https://github.com/paperclipai/paperclip/actions/runs/37669185952). Earlier database-skipped diagnostics and the older ENFILE run are retained and are not full-suite passing evidence. - Three-case live campaign [37651896525](https://github.com/paperclipai/paperclip/actions/runs/37651896525) passes all three original grades on frozen source 09a: 55/55 checks, eight succeeded run records, complete accounting receipts, matching checkpoint identities and no pending approvals. Campaign [37653533353](https://github.com/paperclipai/paperclip/actions/runs/37653533353) adds nine original passes (173/173 checks, eighteen succeeded records) on the identical source. Its other three jobs failed before runner assignment or any step while GitHub could not load the paid environment; those original infrastructure failures are retained. Campaign [37662587356](https://github.com/paperclipai/paperclip/actions/runs/37662587356) completes only those unstarted cells: all three original grades pass (57/57 checks, six succeeded run records). All fifteen exact cases now pass on source 09a: eight FAIL → PASS, seven PASS → PASS, zero new overall failures and zero pending pairs. Total current evidence: 285/285 checks and thirty-two succeeded run records, complete accounting receipts, matching checkpoint identities, no pending approvals or retry records. Earlier campaigns retain forty-seven additional run records and two known same-run recovery attempts; actual provider-call counts and invoices remain unknown. The nine-result campaign skipped publication and its public URL returns 403; original artifacts remain retained. Previous ba9 campaign [37645656840](https://github.com/paperclipai/paperclip/actions/runs/37645656840) completed 2 PASS / 1 FAIL: Claude restart/approval and Codex decline pass, while OpenCode resumes but times out searching for exact tool names and never creates the access card. That failure and incomplete cancelled-run accounting remain preserved; the new catalog error guidance targets this observed dead end. Campaign [37642957312](https://github.com/paperclipai/paperclip/actions/runs/37642957312) remains 0 PASS / 3 FAIL and exposed the now-corrected cross-run marker leak and unknown-criterion approval gap. The original baseline remains 7 PASS / 8 FAIL, first repair 2 PASS / 3 FAIL, and second repair 0 PASS / 3 FAIL. All fifteen selected cases are qualified by their original grades in this bounded trial. These live grades belong to 09a. Its twelve governed-wait checkpoints retain matching same-turn terminal fingerprints, but passing artifacts omit detailed event journals; the later six adversarial regressions qualify the new terminal-proof and replay guards separately. Final-head repository CI passes. [Fresh Greptile review](https://github.com/paperclipai/paperclip/pull/15471#issuecomment-6044369780) is 5/5, confirms both findings are fixed, and reports no new actionable issues. All review threads are resolved and the PR has no merge conflicts. ## Risks - Governed cancellation drains accounting for up to fifteen seconds. Restart detachment gives a settling batch up to twenty seconds to finish. An incomplete receipt or unproven provider terminal still prevents successful qualification. - Claude prices are estimates, not invoices. The estimate assumes standard global API pricing and uses a conservative cache-write rate. Unsupported models, billers, and billing modes remain unpriced. - Approval identity matching must remain scoped to the current run and the requested approval. It does not authorize execution of a declined call. - Original baseline and final candidate have different merged master context. Exact-case outcomes are before/after observations, not isolated causal attribution to this repair. - No schema migration, fixture, oracle or grader change. Search-result guidance now treats fuzzy external matches as suggestions. Existing app authorization and provider-consent checks remain required. ## Model Used - OpenAI GPT-6 through Codex, with code editing, terminal tools, and test execution. The exact deployment identifier and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (191 current runtime tests and 31 accounting tests; interrupted full-suite history and CI coverage are disclosed above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
Apps, Connections, and Integrations
Audience: internal engineers and product contributors working on integrations.
Start here when adding a provider: Connection authoring runbook. It is the canonical agent tutorial from provider research and protocol classification through manifest generation, branding, secrets, deterministic tests, real-account proof, and PR submission.
Runtime authentication: AI Connections.
Long-term memory: Experimental memory connectors.
Provider notes: Google Workspace, Gmail, Asana, PostHog, Neon, Superagent, AgentMail, Telem.AI, iMessage Photon, and Enterpret. Optional credential custody: Vercel Connect.
Post-read action: classify a new integration request, pick the right Paperclip layer to change, and avoid creating a parallel connection framework.
Decision Record
Board decisions from PAP-13211 make the Apps v2 substrate on the PAP-10341 branch canonical:
- D1: Apps v2 is the substrate. The active model is
tool_applications,tool_connections, catalog entries, profiles, policy rules, action requests, gateway sessions, audit events, and runtime slots. Connections v1 is retired as an implementation path. - D2: one credential authority per connection, brokered projections. The
default is the Paperclip instance vault: durable third-party credentials live
in
company_secretsas secret refs. A reviewed remote MCP method may instead opt in to Vercel Connect, in which case Vercel is the durable credential authority and Paperclip stores only the connector reference and redacted grant metadata. A connection must never mix those two sources. Adapter config, plugin config, harness credential files, and run environments may receive only brokered or projected credentials. - D3: the vocabulary and three-door IA are product law. The default product doors are Apps, Connections, and Review. Protocol and operator-depth concepts live behind Developer or Advanced surfaces.
- D4: unification lands on PAP-10341. Pages, CircleBack-style harness MCP OAuth, provider gallery work, and plugin-provided integrations converge on this branch instead of spawning new integration substrates.
- D5: inbound stays thin. External clients that call Paperclip use scoped Paperclip tokens and existing profiles/rules. They do not get a separate permission model.
Canonical Object Model
Use connection as the unifying noun. A connection is four things:
- A stored credential reference.
- A capability catalog.
- A governance layer.
- An audit trail.
Everything else is an axis on that object:
| Axis | Values | It answers |
|---|---|---|
| Direction | outbound, inbound | Who is the client? |
| Transport | MCP, native REST/OpenAPI, OAuth app install, webhook | How do bytes move? |
| Auth mode | OAuth, API key/PAT, app installation, none | What does the secret represent? |
| Credential owner | company, user, run | Whose identity acts? |
| Packaging | catalog entry, plugin, skill | How does it ship? |
MCP is a transport, not a product category. "Install the Discord app", "connect Google Drive", and "add an MCP endpoint" all produce governed connections with different transport/auth values.
Layer Stack
When you are unsure where a change belongs, place it on the narrowest layer that solves the problem:
| Layer | Owns | Examples |
|---|---|---|
| Surface | user-facing Apps, Connections, Review, Developer/Advanced screens | gallery cards, setup wizard, review queue |
| Governance | profiles, bindings, allow/ask-first/block rules, quarantine, audit | read-only profile, ask-first write policy |
| Capability | action catalogs, schemas, risk classes, changed-tool review | search_issues, create_comment, schema hash |
| Credential | company_secrets, OAuth broker, credential resolver, token broker |
Slack bot token ref, Google OAuth refresh token ref |
| Identity | actor attribution and token exchange | board user, agent run, first-party service identity |
| Transport | how the external system is reached | remote HTTP MCP, local stdio, REST/OpenAPI, webhook |
The agent should not hold a durable provider credential. It should hold a Paperclip run/session token; the server or broker resolves the connection, checks governance, invokes the provider, and writes audit.
Identity vs. connections
Signing a user in and connecting a resource are different planes with different owners, different token profiles, and different homes. Do not merge them. This section is the public, connections-side statement of the identity model so connector implementers inherit the rule without depending on private identity-service documentation or re-deriving it.
| Plane | Question | Lives where | Token profile |
|---|---|---|---|
| P1. Sign-in methods | Who are you? | paperclip-id (id.paperclip.ing → Account) |
Minimal-scope provider tokens (openid email profile), used once to authenticate, encrypted at rest, never exported |
| P2. Connections (Apps) | What may your agents touch? | Paperclip App instances (tool_connections), acquired via the connect broker for hosted + self-hosted |
Rich-scope, long-lived resource tokens in the instance's encrypted vault; per-agent grants; risk-tier policy defaults |
| P3. Login with Paperclip | Who may authenticate against us? | paperclip-id OIDC provider + DB-backed client registry |
Our ES256 ID/access tokens issued by us to registered RPs (instances, the broker, future third parties) |
Everything in doc/connections/ — the First-30 matrix,
the connection authoring runbook, and the connect-broker work —
lives on plane P2. It never acquires, stores, or brokers a P1 sign-in token.
The standing rule (D7)
Adopted as a standing rule (decision D7) with the identity-model plan. State it verbatim in any P2 design so the app-store work cannot drift into merging the planes:
Sign-in tokens are never reused as resource tokens; id.paperclip.ing never stores resource tokens; no connections hub on the ID service.
P2 tokens flow broker → instance vault as pass-through only; the id.paperclip.ing Account page therefore must not grow a "Connections" hub. The reasons to hold the planes apart (from the plan §3):
- Scope discipline. Sign-in wants the narrowest grant; connections want deliberately broad ones. One button that does both is how you grant repo access just to log in.
- Blast radius. id.paperclip.ing holding every customer's Vercel/Slack/GitHub resource tokens would make it the single juiciest target in the fleet; the broker is intentionally pass-through.
- Self-hosted symmetry. Instances own their vaults, so self-hosters don't depend on our uptime to use their own connections.
- Legibility. Sign-in and connections answer different user questions, and every product we benchmarked (Vercel, Railway, GitHub, Google) keeps them on separate pages with separate names.
The explicit Vercel Connect exception does not change D7 or merge P1 and P2.
The operator chooses Vercel as the P2 credential authority for an individual
connection. id.paperclip.ing is not involved, and neither sign-in tokens nor
provider tokens pass through it. The deployment's Vercel access token or
workload OIDC identity is bootstrap authority for that external vault, not a
provider resource credential.
Naming alignment
Use the surface-correct name for each plane; they intentionally differ:
| Surface | Plane | Name to use |
|---|---|---|
| Paperclip App instances | P2 | "Connections" |
| id.paperclip.ing Account | P1 | "Ways to sign in" |
| id.paperclip.ing admin | P3 | "OIDC clients" (until the app store productizes it) |
Packaging Rule
Default to a catalog entry when an integration can be described as metadata: manifest, auth config, action catalog, resource filters, and policy defaults.
Use a plugin only when the integration needs product code such as custom UI pages, its own tables, workers, migrations, routines, or specialized ingestion. A plugin may bundle catalog entries, but it must not bypass the connection, profile, policy, credential, and audit model.
Use a skill for agent instructions. Skills may use connections; they must not own durable tokens.
Canonical Docs
- Glossary defines product and internal terms.
- Identity vs. connections is the public statement of the P1/P2/P3 boundary and the D7 standing rule for connections work.
- Security threat model harvests the keeper from PAP-2359 and maps it onto Apps v2.
- First-30 matrix harvests the keeper from PAP-2432 and is the source matrix for connector playbook work.
- Connecting any remote MCP server is the baseline: how an operator connects a standards-compliant remote MCP endpoint with no Paperclip code change, and how sign-in resolves a client.
- Connection authoring runbook is the one end-to-end, agent-executable guide for adding a vendor as a catalog entry on Apps v2: research, connection-type selection, OAuth/API-key/generated-URL setup, encrypted credential handling, branding, implementation, browser and live-provider testing, verification, and PR submission.
- Vercel Connect operator guide documents the optional external credential source, deployment flags, runtime resolution, recovery, and smoke requirements.
- MCP access governance remains the operator runbook for the current gateway, profile, policy, approval, runtime, and audit APIs.
Migration Notes
Connections v1 contributed useful policy, UX, and rollout thinking, but its
implementation branch is no longer the target. When you see old tickets or code
using connections, connection_grants, or a provider-directory mental model,
translate the intent into Apps v2:
| Connections v1 intent | Apps v2 home |
|---|---|
| Provider directory | Apps gallery / tool_applications |
| Configured provider instance | Connection / tool_connections |
| Grant allowlist | Profiles, profile bindings, policies |
| Resource filters | Policy/profile conditions plus provider config |
| Tool broker | Tool gateway and runtime supervisor |
| Connection UX tail | Apps, Connections, Review, Developer/Advanced IA |
Do not add new work to the retired v1 branch. If an old ticket still describes a valid product gap, retarget it to an active Apps v2 issue or close it as superseded with a link to the replacement.