DottaandPaperclip
f912ecaacf
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents hire other agents and assign tasks to them. > - Managed AI connections must follow those hires across legacy and native runners. > - Missing accounts should pause task execution and let the user connect from the task. > - Subscription contention must wait without asking for new credentials. > - This pull request fixes these paths and the native tool and Daytona staging failures found during live tests. > - The result is a working hire, subtask, and connection setup flow on local and remote runners. ## Linked Issues or Issue Description **What happened?** A managed Claude or Codex agent could hire a teammate without a usable AI binding. Cross-provider hiring could fail before the user had a chance to connect the new provider. First-time task setup did not show the existing AI credential form inline. A busy subscription could request a new connection. Native API replies could stop the parent after a hire had already committed. Fresh Daytona sandboxes could fail to extract read-only skill directories created on macOS. **Expected behavior** Compatible hires inherit the managed connection choice. A hire for another provider uses the responsible user's default. If that account is missing, the hire succeeds and the task asks for a connection. Completing setup in the task resumes work automatically. Explicit child auth settings and existing unmanaged login paths keep precedence. Shared-account access checks remain in force. **Steps to reproduce** 1. Connect a Claude or Codex parent with a managed AI account. 2. Ask it to hire one agent of each provider and create a self-assigned subtask. 3. Assign work to both hires without connecting the second provider first. 4. Connect the missing provider from its task card. 5. Check that all tasks finish and same-provider work uses the original account. 6. Repeat with native runners and fresh Daytona sandboxes. The opt-in browser suite in `tests/hiring-ai-connections/README.md` performs these steps. **Paperclip version or commit** The live failures were reproduced from `f2c5e54dc`. The branch is rebased onto `5282cabde`. **Deployment mode** Isolated local development instance. Legacy CLI and native runners. Local execution and ephemeral Daytona sandboxes. Related work: Refs #13247 for managed AI connections. Refs #13268 for legacy credential-reference inheritance, which this branch preserves. Refs #13432 for a concurrent managed-inheritance fix. This PR also covers cross-provider task setup, subscription waits, native API replies, and Daytona extraction. It permits missing responsible-user defaults at hire time; restricted shared selections still fail. ## What Changed - Apply managed connection defaults to both agent creation routes. Preserve explicit auth choices and legacy credential-reference inheritance. - Allow hires before their responsible user connects the provider. Keep approval gates, company boundaries, and shared-account access checks. - Reuse the production AI credential form inside the pending task card. Resume the task after setup. - Retry subscription lease contention without consuming the provider-failure allowance or creating a connection request. - Require task execution-lock ownership when scheduling, promoting, and dispatching subscription retries. Recheck ownership under the issue row lock. - Rename the HTTP operation identity at the native tool boundary so it cannot override the runner's operation identity. - Delay directory permission restoration during Daytona extraction. Preserve the final read-only modes. - Add database-backed regressions, real browser acceptance tests, and Storybook states. Document setup and run-log behavior. ## Verification - Six real browser scenarios passed: both parent providers on legacy local and legacy Daytona; native Codex locally; native Claude on Daytona. Each scenario hires both providers, completes a self-subtask and assigned work, and connects the missing provider inline with automatic continuation. - Successful runs verify the account, responsible user, runner mode, and Daytona lease. All 18 test sandboxes were deleted. - Live authentication used API keys. Subscription inheritance, lease contention, and retry have integration coverage. Fresh subscription OAuth sign-in was not automated. - Red/green tests reproduced missing bindings, missing inline forms, subscription contention, native API reply failure, and GNU tar permission failure. - Seven Storybook browser checks passed. They cover both providers, method selection, narrow layout, completion, cancellation, and invalid credentials. - Full local suite coverage completed before rebase. Initial timing and fixture startup failures passed unchanged on isolated reruns. The first full command did not exit cleanly; the remaining workspace and serialized groups were completed separately. - After rebase, 107 hiring/auth/retry tests and 59 native API, task-card, and Daytona tests passed. The full workspace typecheck, production build, and token gates passed again. Storybook build passed before rebase. - Review fixes: 169 hiring/retry/dispatch tests, 37 adjacent tests, and four explicit cancellation-race cases passed. Eight cross-provider cases cover stale auth keys on both creation routes and both runner types. Server typecheck and build passed. - Final CI on `ee4890837a8a4913e07453392b9a75969580dae1`: 32 checks passed. Two optional Storybook jobs were skipped. The full server, workspace, browser, native runner, build, typecheck, and release checks passed. - Three unchanged tests initially failed on a busy port, a chat row-lock race, and preview-server readiness. Each affected job passed after one CI rerun. Isolated local checks also passed: 41 credential tests, the chat-concurrency case, and 25 preview-runtime tests. - Greptile reviewed the final commit at 5/5. Both review threads are resolved. GitHub reports no merge conflicts. ## Risks - A missing personal account now defers authentication to the first task. Explicit incompatible bindings and restricted shared accounts still fail at hire time. - An inherited personal default uses the responsible user's existing authorization to install access for the new agent. It never copies credentials or another user's identity. - Subscription contention retries after a delay and rechecks task eligibility. It does not consume the provider-failure budget. - Native hiring uses the existing managed API-tools opt-in. Remote native runners require a matching Linux binary and provider pack, as documented in the acceptance README. - No schema changes. Live tests make paid provider calls and remain opt-in. ## Model Used OpenAI GPT-6 (`gpt-6-astra`) in Codex, with reasoning, repository inspection, code execution, browser automation, and API tools. The context-window size is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
Apps, Connections, and Integrations
Audience: internal engineers and product contributors working on integrations.
Start here when adding a provider: Connection authoring runbook. It is the canonical agent tutorial from provider research and protocol classification through manifest generation, branding, secrets, deterministic tests, real-account proof, and PR submission.
Runtime authentication: AI Connections.
Provider notes: Google Workspace, Gmail, PostHog, AgentMail, and iMessage Photon. Optional credential custody: Vercel Connect.
Post-read action: classify a new integration request, pick the right Paperclip layer to change, and avoid creating a parallel connection framework.
Decision Record
Board decisions from PAP-13211 make the Apps v2 substrate on the PAP-10341 branch canonical:
- D1: Apps v2 is the substrate. The active model is
tool_applications,tool_connections, catalog entries, profiles, policy rules, action requests, gateway sessions, audit events, and runtime slots. Connections v1 is retired as an implementation path. - D2: one credential authority per connection, brokered projections. The
default is the Paperclip instance vault: durable third-party credentials live
in
company_secretsas secret refs. A reviewed remote MCP method may instead opt in to Vercel Connect, in which case Vercel is the durable credential authority and Paperclip stores only the connector reference and redacted grant metadata. A connection must never mix those two sources. Adapter config, plugin config, harness credential files, and run environments may receive only brokered or projected credentials. - D3: the vocabulary and three-door IA are product law. The default product doors are Apps, Connections, and Review. Protocol and operator-depth concepts live behind Developer or Advanced surfaces.
- D4: unification lands on PAP-10341. Pages, CircleBack-style harness MCP OAuth, provider gallery work, and plugin-provided integrations converge on this branch instead of spawning new integration substrates.
- D5: inbound stays thin. External clients that call Paperclip use scoped Paperclip tokens and existing profiles/rules. They do not get a separate permission model.
Canonical Object Model
Use connection as the unifying noun. A connection is four things:
- A stored credential reference.
- A capability catalog.
- A governance layer.
- An audit trail.
Everything else is an axis on that object:
| Axis | Values | It answers |
|---|---|---|
| Direction | outbound, inbound | Who is the client? |
| Transport | MCP, native REST/OpenAPI, OAuth app install, webhook | How do bytes move? |
| Auth mode | OAuth, API key/PAT, app installation, none | What does the secret represent? |
| Credential owner | company, user, run | Whose identity acts? |
| Packaging | catalog entry, plugin, skill | How does it ship? |
MCP is a transport, not a product category. "Install the Discord app", "connect Google Drive", and "add an MCP endpoint" all produce governed connections with different transport/auth values.
Layer Stack
When you are unsure where a change belongs, place it on the narrowest layer that solves the problem:
| Layer | Owns | Examples |
|---|---|---|
| Surface | user-facing Apps, Connections, Review, Developer/Advanced screens | gallery cards, setup wizard, review queue |
| Governance | profiles, bindings, allow/ask-first/block rules, quarantine, audit | read-only profile, ask-first write policy |
| Capability | action catalogs, schemas, risk classes, changed-tool review | search_issues, create_comment, schema hash |
| Credential | company_secrets, OAuth broker, credential resolver, token broker |
Slack bot token ref, Google OAuth refresh token ref |
| Identity | actor attribution and token exchange | board user, agent run, first-party service identity |
| Transport | how the external system is reached | remote HTTP MCP, local stdio, REST/OpenAPI, webhook |
The agent should not hold a durable provider credential. It should hold a Paperclip run/session token; the server or broker resolves the connection, checks governance, invokes the provider, and writes audit.
Identity vs. connections
Signing a user in and connecting a resource are different planes with different owners, different token profiles, and different homes. Do not merge them. This section is the public, connections-side statement of the identity model so connector implementers inherit the rule without depending on private identity-service documentation or re-deriving it.
| Plane | Question | Lives where | Token profile |
|---|---|---|---|
| P1. Sign-in methods | Who are you? | paperclip-id (id.paperclip.ing → Account) |
Minimal-scope provider tokens (openid email profile), used once to authenticate, encrypted at rest, never exported |
| P2. Connections (Apps) | What may your agents touch? | Paperclip App instances (tool_connections), acquired via the connect broker for hosted + self-hosted |
Rich-scope, long-lived resource tokens in the instance's encrypted vault; per-agent grants; risk-tier policy defaults |
| P3. Login with Paperclip | Who may authenticate against us? | paperclip-id OIDC provider + DB-backed client registry |
Our ES256 ID/access tokens issued by us to registered RPs (instances, the broker, future third parties) |
Everything in doc/connections/ — the First-30 matrix,
the connection authoring runbook, and the connect-broker work —
lives on plane P2. It never acquires, stores, or brokers a P1 sign-in token.
The standing rule (D7)
Adopted as a standing rule (decision D7) with the identity-model plan. State it verbatim in any P2 design so the app-store work cannot drift into merging the planes:
Sign-in tokens are never reused as resource tokens; id.paperclip.ing never stores resource tokens; no connections hub on the ID service.
P2 tokens flow broker → instance vault as pass-through only; the id.paperclip.ing Account page therefore must not grow a "Connections" hub. The reasons to hold the planes apart (from the plan §3):
- Scope discipline. Sign-in wants the narrowest grant; connections want deliberately broad ones. One button that does both is how you grant repo access just to log in.
- Blast radius. id.paperclip.ing holding every customer's Vercel/Slack/GitHub resource tokens would make it the single juiciest target in the fleet; the broker is intentionally pass-through.
- Self-hosted symmetry. Instances own their vaults, so self-hosters don't depend on our uptime to use their own connections.
- Legibility. Sign-in and connections answer different user questions, and every product we benchmarked (Vercel, Railway, GitHub, Google) keeps them on separate pages with separate names.
The explicit Vercel Connect exception does not change D7 or merge P1 and P2.
The operator chooses Vercel as the P2 credential authority for an individual
connection. id.paperclip.ing is not involved, and neither sign-in tokens nor
provider tokens pass through it. The deployment's Vercel access token or
workload OIDC identity is bootstrap authority for that external vault, not a
provider resource credential.
Naming alignment
Use the surface-correct name for each plane; they intentionally differ:
| Surface | Plane | Name to use |
|---|---|---|
| Paperclip App instances | P2 | "Connections" |
| id.paperclip.ing Account | P1 | "Ways to sign in" |
| id.paperclip.ing admin | P3 | "OIDC clients" (until the app store productizes it) |
Packaging Rule
Default to a catalog entry when an integration can be described as metadata: manifest, auth config, action catalog, resource filters, and policy defaults.
Use a plugin only when the integration needs product code such as custom UI pages, its own tables, workers, migrations, routines, or specialized ingestion. A plugin may bundle catalog entries, but it must not bypass the connection, profile, policy, credential, and audit model.
Use a skill for agent instructions. Skills may use connections; they must not own durable tokens.
Canonical Docs
- Glossary defines product and internal terms.
- Identity vs. connections is the public statement of the P1/P2/P3 boundary and the D7 standing rule for connections work.
- Security threat model harvests the keeper from PAP-2359 and maps it onto Apps v2.
- First-30 matrix harvests the keeper from PAP-2432 and is the source matrix for connector playbook work.
- Connecting any remote MCP server is the baseline: how an operator connects a standards-compliant remote MCP endpoint with no Paperclip code change, and how sign-in resolves a client.
- Connection authoring runbook is the one end-to-end, agent-executable guide for adding a vendor as a catalog entry on Apps v2: research, connection-type selection, OAuth/API-key/generated-URL setup, encrypted credential handling, branding, implementation, browser and live-provider testing, verification, and PR submission.
- Vercel Connect operator guide documents the optional external credential source, deployment flags, runtime resolution, recovery, and smoke requirements.
- MCP access governance remains the operator runbook for the current gateway, profile, policy, approval, runtime, and audit APIs.
Migration Notes
Connections v1 contributed useful policy, UX, and rollout thinking, but its
implementation branch is no longer the target. When you see old tickets or code
using connections, connection_grants, or a provider-directory mental model,
translate the intent into Apps v2:
| Connections v1 intent | Apps v2 home |
|---|---|
| Provider directory | Apps gallery / tool_applications |
| Configured provider instance | Connection / tool_connections |
| Grant allowlist | Profiles, profile bindings, policies |
| Resource filters | Policy/profile conditions plus provider config |
| Tool broker | Tool gateway and runtime supervisor |
| Connection UX tail | Apps, Connections, Review, Developer/Advanced IA |
Do not add new work to the retired v1 branch. If an old ticket still describes a valid product gap, retarget it to an active Apps v2 issue or close it as superseded with a link to the replacement.