Commit Graph
122 Commits
Author SHA1 Message Date
deacaf727c feat(connections): guided AWS role setup with a per-connection CloudFormation template (#22233)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The AWS connector (#22195) lets agents use AWS with credentials
minted per connection. Its role method stores no AWS keys
> - That method needs the customer to create an IAM role that trusts
exactly one Paperclip connection, but setup only offered the generic
form and a doc
> - The connect API also refused the draft step this method needs, so
the role method could not be set up through the UI at all
> - This pull request adds a guided setup screen. Paperclip saves a
draft, hands the user a prompt for their own agent that creates the
role, takes back the role ARN the agent reports, and tests the same
draft. CloudFormation and IAM steps are there under "Or do it by hand"
> - The benefit is that a person can connect AWS without stored keys and
without typing account details, using a template AWS has validated

## Linked Issues or Issue Description

Refs #22195.

**Subsystem affected**
Connections (AWS connector setup), UI, tool-access API.

**Problem or motivation**
The AWS role method must save a connection before AWS can trust it. The
API rejected `saveDraft` for every method except custom MCP servers, so
the UI could not complete this flow. There was also no screen that
showed the issuer, the subject, the trust policy or a template.

**Proposed solution**
- The connect API accepts `saveDraft` for methods that declare
`credentialMinter`.
- `GET /tool-connections/:id/aws-setup` returns the setup values and a
CloudFormation template generated for that connection.
- A new `AwsRoleSetup` step runs between saving the draft and testing
it. The role is named after the connection and CloudFormation fills in
the account, so the first form no longer asks for a role ARN; the user
pastes it back once the role exists.

**Alternatives considered**
- A one-click CloudFormation quick-create link. Rejected: AWS accepts
quick-create templates only from Amazon S3, so Paperclip would need to
host a public template bucket. The screen offers a download and a
console link instead.
- Instructions in a doc only. Rejected: too many hand-copied values.
- Asking for the role ARN up front. Rejected: the role does not exist
yet, so users had to invent an ARN and then match it exactly.
- Letting the user's agent report the role ARN to Paperclip itself,
through a short-lived link. Not done here: it needs a new
unauthenticated endpoint (and a Paperclip Cloud front-door change). The
user pastes the ARN instead.

## What Changed

- `packages/shared`:
  - The `saveDraft` validation allows methods with minted credentials.
  - New `AwsConnectionSetup` type.
- `server/src/services/connectors/aws/setup.ts` builds the setup values:
  - The issuer, audience and connection subject.
  - The provider host and the trust policy.
- A CloudFormation template with an optional identity provider
(`CreateIdentityProvider`), a role named `paperclip-<connection id>`,
`AccessPolicyArn` (`ReadOnlyAccess` by default for read-only
connections) and `AWSMCPSignInOAuthAccessPolicy`. The account comes from
CloudFormation, so the template needs no edits.
- The service method `getAwsSetup` and the route `GET
/api/tool-connections/{connectionId}/aws-setup`:
  - The route requires `connections:configure`.
  - It returns 422 for a connection that is not an AWS role connection.
- It reports whether the issuer key is published and any role ARN
already saved.
- It returns `available: false` (`address_changed`) when the instance
address changed after the draft was saved, since a role set up for the
old address could never pass the test.
  - It is registered in OpenAPI.
- UI:
- The AWS form no longer shows a Role ARN field (it is hidden with an
empty default).
- `AwsRoleSetup.tsx` shows, by default, only "Set it up with your agent"
(a copyable prompt that checks the AWS account, deploys the template
with `aws cloudformation deploy`, and reports the role ARN) and a field
to paste that ARN. "Or do it by hand" reveals the CloudFormation steps
(download, console link, parameter notes, IAM acknowledgement) and the
IAM console / AWS CLI steps with copyable values; the CLI commands read
the account ID themselves.
- It shows an access-key message when AWS cannot reach the instance, and
asks the user to save again when the instance address changed.
- Failed tests appear in place. AWS error codes show the server's own
plain message, because the shared error copy replaces every 502 with
"temporarily unavailable".
- `useAwsRoleSetupStep` owns the AWS sequence; `ConnectionSetupFlow`
only asks it for request options, hands back results and renders its
step:
- The first Connect for `aws-role` saves a draft and opens the AWS step.
  - "Test connection" resumes the same draft with the pasted role ARN.
- Back keeps the draft, so a later Connect updates it and does not
create a second one. Returning to an unfinished draft reopens the step.
- Docs: the `doc/connections/AWS.md` setup section and the plan's phase
table. The SDK public input pins are refreshed.

## Verification

- `server/src/__tests__/aws-mcp-connector.test.ts` covers a new
API-level case through the real route validation:
- Saving an `aws-role` draft over HTTP works without a role ARN, and
stores the keys the screen uses to reopen it.
- `aws-setup` returns the issuer, subject and the connection's role
name.
- The template's trust condition keys equal `<issuer host>:aud` and
`<issuer host>:sub`.
  - The template provider URL equals the issuer.
- After an address change it returns `address_changed`, and saving the
draft again records the new address.
  - An access-key connection returns 422.
  - No response field looks like a secret.
- `ui/src/features/connections/AwsRoleSetup.test.tsx` covers:
- Only the agent setup shows by default; Test connection stays off until
a valid role ARN is pasted, then sends it.
  - A saved role ARN is filled in.
- The by-hand steps and CLI commands; the agent prompt for read-only and
full access.
  - The template download (filename and object URL cleanup).
  - The unreachable-instance message.
  - The in-place error and the address-changed message.
  - The AWS message mapping.
- `ui/src/pages/apps/AppsConnect.test.tsx` covers the full two-step
flow: the form has no Role ARN field, the first Connect sends
`saveDraft: true` with no role ARN and does not finish, the AWS step
loads the setup, and "Test connection" resumes `aws-draft` with the
pasted role ARN and finishes. A second test reopens the step for a
returning draft.
- Red/green:
  - Removing the API change fails the API test.
  - Removing the draft-first wiring fails the flow test.
- Totals: server AWS, OpenAPI and connector tests (75), shared
`app-definitions` (42), UI connections and apps pages (1108). Typecheck
is clean for shared, server, UI (except Storybook's generated catalog,
which this worktree has not built) and all four server test-tree layers.
- Browser, on a real dev server with an HTTPS public address:
  - AWS appears in Connectors with its logo.
- The form has only region and access level. Connect saves the draft
(reusing an earlier one) and shows only the agent prompt and the role
ARN field; "Or do it by hand" reveals the CloudFormation and IAM steps.
- The copied prompt names the connection's role and stack, and the
template.
- Test connection with a pasted role ARN showed AWS's refusal in place
(no such role, placeholder issuer), as expected. An earlier run found
the generic 502 copy problem, which is fixed and re-checked.
- AWS CloudFormation `validate-template` (dev account, read-only)
accepts the current generated template (role named after the connection)
and reports `CAPABILITY_NAMED_IAM`. The screen tells the user to
acknowledge it.
- Not run: an agent actually running the prompt and creating a stack in
AWS, and Paperclip Cloud staging.

## Risks

- The feature is still behind `enableAwsConnector` (off by default). No
migration.
- The template creates IAM resources in the customer's account, and the
customer must acknowledge that. The role trusts only this connection's
subject.
- If the customer already has an identity provider for the issuer and
leaves `CreateIdentityProvider` at true, AWS fails the stack with a
clear error. The screen explains the parameter.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, the in-app browser against a local dev server, AWS
CLI `validate-template` in the dev account, an exploration subagent).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-19 10:53:53 -06:00
651749bdb4 feat(connections): AWS connector with credentials minted per connection (#22195)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents use outside services through connections, and the control
plane holds each connection's credentials
> - Companies want agents to "do things on AWS" with exactly the
permissions they choose, without Paperclip storing long-lived AWS keys
or rebuilding the AWS API
> - AWS hosts an MCP server that runs any AWS API call in its own
sandbox, and it accepts a one-hour bearer token that can be minted from
IAM credentials
> - This pull request adds an AWS connection that mints that token for
each connection at call time. The role method proves the connection with
the company's workload identity issuer (#22083), so no AWS key is stored
> - The benefit is that each AWS connection carries its own role and
permissions, AWS enforces them, and agents never see any AWS credential

## Linked Issues or Issue Description

Refs #22083 (workload identity issuer) and #22191 (spike findings).

**Subsystem affected**
Connections (tool access), tool gateway, shared app definitions,
instance settings.

**Problem or motivation**
Paperclip has no way to connect AWS. AWS's hosted MCP server needs a
bearer token from AWS Sign-in, minted from IAM credentials. No existing
credential path mints a header at call time from a role or key.

**Proposed solution**
Methods can declare `credentialMinter: "aws_mcp"`. The gateway and
connection health checks mint the `Authorization` header at call time,
and they share one in-memory cache per connection and grant (about 55
minutes). There are two methods:
- Role (`AssumeRoleWithWebIdentity` with a Paperclip-signed connection
token).
- Access key (the key directly, `GetFederationToken`, or `AssumeRole`).

Read-only connections add `ReadOnlyAccess` and
`AWSMCPSignInOAuthAccessPolicy` as session policies, so AWS refuses
changes. The feature is behind the `enableAwsConnector` experimental
setting.

**Alternatives considered**
- Run AWS's `mcp-proxy-for-aws` process on the control plane. Rejected:
it adds a Python runtime and processes, and gives nothing the token
exchange does not.
- Put credentials in the agent sandbox and use the AWS CLI. Deferred:
there is no per-call governance, and every sandbox would need the CLI.
- Add a new `credentialSource`. Rejected: it needs a migration, and the
`vercel_connect` database rules do not fit stored access keys.

## What Changed

- `packages/shared`:
- `credentialMinter` on connection methods. The validator requires
`mcp_remote` and `api_key`, and does not allow `keyPlacement` or another
credential source.
- The `aws` app definition (two methods, reviewed tools,
`mcpSessionRequired`, experimental key).
- The `enableAwsConnector` setting in the catalog, types and validators.
  - `aws` added to the connectable apps.
- `server/src/services/connectors/aws/mcp-credentials.ts`:
- Settings parsing and checks. The URL must be AWS's endpoint for the
saved region, and the role ARN must be valid.
- The minter: role or key session, read-only session policies, Sign-in
exchange, cache keyed on settings, key references and status, in-flight
de-duplication, eviction.
- A plain-language error mapping that never shows AWS's own text,
because that text can contain account IDs and ARNs.
- Gateway (`resolveCredentialHeadersUnrecorded`), credential access
(`resolveCredentialHeaders`), catalog discovery and health-failure
mapping:
  - The AWS branch.
- After a 401, the gateway evicts the token and mints again, then
requires an explicit retry (no replay). Discovery lists again.
- Setup:
- Minted methods may save a draft without contacting the provider. The
role must trust a connection that already exists.
- Saving an AWS role draft publishes the company's issuer, so AWS can
validate it when the customer registers it.
- `workload-identity.ts`: a draft connection can get a token, so setup
can test it. Disabled, archived or switched-off connections still get
none.
- `server/package.json`: `@aws-sdk/client-sts` and
`@aws-sdk/client-signin`. The lockfile is left to the refresh workflow.
- Artwork: AWS light and dark marks from LobeHub's MIT icon set (already
licensed in the repo). The manifest entry passes the brand checks.
- `scripts/ingest-app-definitions.mjs`: minted methods are exempt from
the `keyPlacement` requirement. The override source is
`scripts/app-definition-overrides/aws.json`. Permission reviews are
added.
- UI: the experimental settings toggle, in alphabetical position.
- Docs: `doc/connections/AWS.md`, the connections index, the issuer API
doc (draft rule), and the plan's phase table.

## Verification

- `server/src/services/connectors/aws/mcp-credentials.test.ts` (7 tests)
covers these items:
  - Both methods and the read-only policies.
  - Caching, revision, forced refresh, expiry and eviction.
  - Concurrent de-duplication.
  - The error mapping, with no AWS text leaking.
- Settings validation, including the URL and region check and the rule
that an unknown access level means read only.
- `server/src/__tests__/aws-mcp-connector.test.ts` (3 tests, real setup,
catalog and gateway code with a fake AWS and a session-requiring fake
MCP server) covers these items:
  - The setting gates setup.
  - The access key is never sent as a header or stored in config.
  - A cached token is reused.
  - After a 401, a fresh token is minted and the call is not replayed.
  - Role draft-then-resume publishes the issuer.
  - The signed token has the right `iss`, `sub` and `aud`.
  - An AWS refusal shows a plain health message without the account ID.
- Red/green:
  - Removing the gateway hook fails the access-key test.
  - Turning off `mcpSessionRequired` fails both integration tests.
- Removing draft support for minted methods fails both integration
tests.
  - Restoring each change passes again.
- Nearby suites: `tool-gateway-service`, `tool-access-service`,
`remote-mcp-connectors`, `instance-settings-service`,
`workload-identity`, plus the new files. 775 of 775 pass.
- Shared `app-definitions.test.ts` passes (42), including new
`credentialMinter` schema tests. UI `InstanceExperimentalSettings` (63)
and the brand asset tests pass. `check-app-brand-assets` validates 112
identities.
- Typecheck is clean for `packages/shared`, `server` and `ui`. The only
UI errors are in Storybook's generated catalog files, which this
worktree has not built.
- **Real AWS** (Paperclip dev account, integrated code path, issuer
behind a temporary public tunnel):
- Access key, read only: 8 tools listed. `run_script` `ListBuckets`
succeeded in 8.9 s on a cold sandbox. AWS refused `CreateBucket` ("no
session policy allows").
- Role: the draft was saved without contacting AWS, and the issuer was
published. After an IAM provider and a role trusting only that
connection were created, resuming setup listed 8 tools. `ListBuckets`
succeeded in 12.8 s. The role's own permissions refused `CreateBucket`.
- This run found two bugs the fakes did not: AWS needs MCP sessions for
calls, and the role method needs draft-first setup with the issuer
published. Both are fixed and both now have tests.
- All IAM users, keys, roles, providers and buckets were deleted, and a
listing confirmed it.
- Not run: Paperclip Cloud staging (the setting is off by default), and
`AssumeRole` from an access key with a role ARN against real AWS (unit
tests only).

## Risks

- The feature is off by default, behind `enableAwsConnector`. No
migration.
- New dependencies `@aws-sdk/client-sts` and `@aws-sdk/client-signin`
are in the existing `@aws-sdk` family (the server already uses
`client-s3`).
- Minted tokens are held only in process memory. A restart mints again.
A cached token can outlive a deactivated key by up to an hour, unless
AWS returns 401, which evicts it.
- `aws___run_script` is classified as a write, so the normal approval
and ask-first policy applies. Read-only connections rely on AWS session
policies.
- The guided setup screen and the CloudFormation template are not in
this PR. Setup uses the generic form and the documented draft and resume
steps.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, AWS SDK for JavaScript against the Paperclip dev AWS
account, a Cloudflare quick tunnel, an exploration subagent).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-18 12:48:49 -06:00
DottaandClaude Opus 5.5 adb2bcf896 fix(connectors): activate plugin connectors with agent grants and no sign-in (#22201)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Plugins can now add connector types (#22004) and serve their tools
with the `plugin` transport (#22019)
> - A plugin connector method can declare `grantKinds: ["agent"]` with
`auth: "none"` or `"api_key"`
> - Such a connector cannot reach its agent: the connect form cannot
name the agent, connect creates no agent grant, and Paperclip Runner
runs never list plugin-transport tools
> - The product rule is that a plugin connector behaves exactly like any
existing connector
> - This pull request fixes the connect form, creates the agent grant at
connect, and lets runner runs list plugin-transport tools
> - The benefit is that an operator can activate a per-agent plugin
connector, and only the chosen agent gets its tools

## Linked Issues or Issue Description

Refs #22004
Refs #22019

**Subsystem affected**
Connectors (connect wizard, tool access service), Paperclip Runner
assigned-tool gateway.

**Problem or motivation**
A plugin connector with agent grants and no sign-in cannot be activated:
1. Connect form: it says "No sign-in needed, available to all agents".
"Change" shows "No identity required" and "Which agent owns this GitHub
account?", with no agent picker. Connect sends `grantKind: "agent"`
without `subjectAgentId`, and the server rejects it with a validation
error. The "Finish setup" resume form has the same problem.
2. Grant: connect with `subjectAgentId` saves a `per_agent` connection
with zero grants. Finish adds the agent install, but there is still no
grant. Only the OAuth callback writes agent grants, and these methods
have no OAuth step. For `api_key` methods connect fails, because the
health check finds no agent grant, and the key has nowhere to go.
3. Runner: a Paperclip Runner run builds its assigned-tool gateway from
an allowlist of transports (`mcp_remote`, `local_stdio`, REST, Browser
Use, GitHub bot, voice). `plugin` is not in it, so the run sees no
plugin-transport tools, even when the install, profile and grant allow
them.

**Proposed solution**
Create the agent grant at connect for curated methods without sign-in.
Add `plugin` to the runner assignment and its digest. Show a one-agent
picker with connector-neutral copy.

**Expected behavior**
Connecting alone grants no tools. Finish with agent A gives only agent A
the tools, in gateway discovery and in a runner run. The connection has
exactly one active agent grant, for A.

## What Changed

- `server/src/services/tool-access.ts`: `connectGalleryApp` writes an
active agent grant for `subjectAgentId` when a curated method with
`auth` `none` or `api_key` uses `grantKind: "agent"`. The API key goes
on that grant, not on the connection. A resume without `subjectAgentId`
keeps the agent of the existing grant and keeps its saved key. A second
agent on the same connection is a 409 conflict. The grant write is
audited and joins the existing revive rollback.
- Finish does not create agent grants. Finish carries no credential, and
it can list many agents while a per-agent connection has one identity. A
credential-less grant would also claim a connected OAuth identity that
does not exist. The grant comes from connect (or the OAuth callback).
Finish creates only the install.
- `server/src/services/heartbeat/run-preparation.ts` and
`native-runtime/runtime-context.ts`: include the `plugin` transport in
the runner assignment and in its digest (the two lists must match).
- `ui/src/features/connections/ConnectionSetupFlow.tsx`: for an agent
grant, show the agent picker (exactly one agent), keep the install
choice on "specific", open the "Change" panel and disable Connect until
one agent is chosen, and send `subjectAgentId`. Copy is neutral ("Which
agent uses this connection?"); the GitHub copy stays for GitHub. The
summary no longer says "available to all agents" for an agent grant.
Finish sends the chosen agent, never `all_agents`, for an agent grant.
- `doc/connections/CONNECTOR-PLAYBOOK.md`: document agent grants for
methods without sign-in.

## Verification

Each new test failed before its fix and passes after it:
- `server/src/__tests__/plugin-connector-agent-grant.test.ts` (embedded
Postgres, real routes and gateway):
- `auth: "none"` + agent grants: after connect, no agent has tools;
after finish with agent A, A's discovery lists the tools and B's lists
none; exactly one active agent grant, for A. (Red: zero grants.)
- `auth: "api_key"` + agent grants: same, the key is on A's grant, and a
tool call receives it. (Red: connect returned 422.)
  - A second agent on the same connection is a 409. (Red: 201.)
- Resume of an API-key draft without the key keeps the key on the grant.
- Paperclip Runner session: `buildPaperclipRuntimeMcpServers` +
`createAssignedMcpTools` list the installed plugin tools for A, and
search by the manifest tool name (`update_record`) finds the gateway
tool. B gets none. (Red: no tools.)
- Regression: an organization-grant method keeps one default
organization grant, and both agents get the tools with `all_agents`.
- `ui/src/pages/apps/AppsConnect.test.tsx`: for `none` and `api_key`
agent-grant plugin methods, the form shows the picker, has no GitHub or
"available to all agents" copy, blocks Connect until an agent is chosen,
and sends `subjectAgentId`. (Red: "available to all agents" shown.)
- Regression suites: `plugin-connectors-plugin-transport`,
`plugin-connectors-circleback-parity`, `heartbeat-runtime-mcp-servers`,
`run-preparation`, `tool-access-service`,
`tool-access-oauth-protocol.integration`, `generic-mcp-connection`
(OAuth, GitHub per-agent and organization flows) pass. UI:
`src/features/connections`, `src/pages/apps`, `App.deep-link-routing`
pass (66 files, 1103 tests).
- `tsc --noEmit` passes for `server` and `ui` source. `pnpm
check:module-boundaries` passes.

## Risks

- Low. The grant change applies only to curated methods with agent
grants and no OAuth. Organization, personal and OAuth flows keep their
code paths.
- The runner change makes plugin-transport tools visible to runner runs.
Access still needs an install, a profile and a grant, and every call is
re-authorized by the gateway.
- Connections created before this fix have no agent grant. Connecting
again (or "Finish setup") with the agent creates it.
- No migration.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edits, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 11:25:25 -06:00
Nicky LeachandPaperclip f05b90508a refactor(core): remove retired compatibility paths (#22055)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Core manages chat authorization, AI connection defaults, navigation,
and the plugin host.
> - Core still writes old default rows and checks an Agent Chat flag
that always reads true.
> - It also carries a desktop rail that cannot open and private aliases
with canonical equivalents.
> - This pull request removes those paths and moves their tests onto the
paths Core uses.
> - The result is fewer Core database operations and less code to
maintain.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Core AI default writes, Agent Chat execution and delivery, private
plugin host and Runner setup, and board navigation.

**Subsystem affected**

Cross-cutting. Core server, board UI, private Runner helpers, and a
repository guard script.

**Current behavior**

The server writes provider-scoped and per-method AI defaults, then reads
only the provider-scoped rows. It reads settings to decide whether to
block Agent Chat, although every settings view enables it. The desktop
sidebar renders expanded while still tracking requests to open and close
a retired icon rail. The agent client retries conflicts that the server
already resolves when there is exactly one active match.

**Proposed behavior**

Write provider-scoped defaults directly. Use the existing Agent Chat
authorization and retry checks without the redundant settings reads.
Render the current sidebar directly and let the API report true agent
reference conflicts.

**Reason and benefit**

A personal non-custom connection save or explicit default selection
sends one fewer SQL statement. It also avoids the provider-default write
that the legacy trigger performed when the old statement inserted or
updated a row. A successful task-to-chat send or receipt replay avoids
two transaction settings SELECTs when the singleton row already exists.
Conversation comment, wake release, and retry paths also avoid a
transaction settings read when they previously entered the gate. Savings
in other settings paths depend on cache hits. Core-to-plugin RPC and
plugin-to-Launcher HTTP traffic stay the same.

**Breaking changes**

The public plugin SDK, host navigation, and organization-switcher fields
retain their existing interfaces and behavior. The settings API still
accepts and reports the Agent Chat compatibility key. This change needs
no schema migration or rollout ordering.

**Additional context**

Refs #10610 for prior reporting on the retired sidebar rail. Refs #21756
for adjacent Coordinator plugin slots, which remain a separate change.

## What Changed

- Stop writing per-method AI defaults. Preserve the provider-scope
upsert and first-account behavior. Keep the existing tables and
migration trigger.
- Remove Agent Chat checks that read a constant true value. Keep
conversation ownership, task scope, session, and request-retry checks.
- Remove unused lock-owner fields and private plugin lifecycle, secret
extraction, Gmail connector, PRP setup, heartbeat resolver, and ACPX
aliases.
- Use a single API request for agent references. Remove retired sidebar
rail state, hooks, and branches across the active and production
snapshot layouts. Preserve desktop resizing, mobile drawer controls,
contextual navigation, and public plugin slot inputs.
- Remove unused FeedCard and AgentDetail inputs and the test-only Git
push matcher alias.
- Keep the cleanup that erases historical plaintext chat retries from
browser storage.

Production source in the touched files is 106,800 → 105,633 nonblank
lines, down 1,167. The production Git diff adds 251 physical lines and
removes 1,480, down 1,229. Those figures exclude tests, fixtures, docs,
and Storybook. The entire Git diff adds 482 physical lines and removes
2,325, down 1,843.

## Verification

- Merged master at `b56f240f45` into the PR branch as `f73fea4009`. The
five sidebar files retain the expanded navigation cleanup and master’s
explicit session-replay masking contract. The merge also preserves
master’s account-repair recovery guard and GitHub star-prompt behavior.
- Focused UI Vitest run passed 361 tests across 11 files. This covers
analytics privacy and replay masking, sidebar and layout, GitHub star
prompt, and agent detail.
- Focused server Vitest run passed 45 tests across three files. This
covers restored configuration handling, conversation recovery, and Agent
Chat authority and receipt replay.
- `pnpm --filter @paperclipai/ui typecheck`, the Git push guard and its
14 tests, and `git diff --check origin/master...HEAD` passed.
- `pnpm verify --base origin/master` classified the Git-push guard as a
global CI input and returned exit 2. GitHub CI supplied the broader
coverage.
- GitHub CI and Runner image checks passed on `f73fea4009`. Both
required checks, `ci / verify` and `ci / e2e`, passed. [CI
workflow](https://github.com/paperclipai/paperclip/actions/runs/44181261752)
· [Runner image
workflow](https://github.com/paperclipai/paperclip/actions/runs/44181261413)
- Standard Greptile reported 5/5 with no actionable findings or open
review threads. [Greptile
review](https://github.com/paperclipai/paperclip/pull/22055#issuecomment-6959078925)
- The optional Superagent Security Scan finished neutral with an
inconclusive workflow result and zero annotations. The final rollup had
61 successes, five skips, and one neutral, with no failed or pending
checks. [PR
checks](https://github.com/paperclipai/paperclip/pull/22055/checks)

## Risks

Agent Chat authorization and retry are sensitive to small query changes.
The final diff keeps the conversation agent field and all existing
authorization and idempotency checks. The tests cover revoked defaults,
legacy migration reads, bound-listener authority, and mobile and desktop
navigation. Installations that still run software old enough to read the
per-method AI table will no longer see new defaults in that table.
Current Core reads the provider-scoped table.

## Model Used

OpenAI Codex, GPT-6.1 Sol (`gpt-6.1-sol`), with source inspection and
shell-based tests.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-17 14:00:14 -08:00
DottaandClaude Opus 5.5 23ce68a8dc feat(connections): end Google access when a Paperclip-managed Google connection is removed (#22065)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Connectors let agents use Google Workspace through a Google sign-in
that Paperclip Cloud manages for the user
> - When a user removes a Paperclip-managed Google connection, Paperclip
deletes the tokens locally, but the grant stays live at Google
> - Google's OAuth review expects that a disconnect ends the app's
access, and users expect it too
> - Google's revoke ends the user's whole grant to the shared client, so
only the Paperclip Cloud broker can decide when it is safe to revoke
> - This pull request asks the broker to revoke each removed grant,
keeps local removal unconditional, and tells the user the outcome in
plain language
> - The benefit is that removal ends Google access when no other
Paperclip connection needs it, and the user knows what to do when it did
not

## Linked Issues or Issue Description

**Subsystem affected**
Connectors: Paperclip-managed Google Workspace and the legacy
per-service Google connections (server removal, shared contracts,
connection UI).

**Problem or motivation**
Removing a Paperclip-managed Google connection or grant was local only.
The refresh token was deleted in Paperclip, but Google still listed
Paperclip as having access to the account. The docs said so ("Removal is
local for managed grants").

**Proposed solution**
On removal, send each removed grant's refresh token to the broker's
signed `POST /v1/connector/revoke`. The broker revokes at Google only
when no other live binding could share the grant. Paperclip records the
broker's outcome code and shows one of three plain messages. Local
removal never waits on, or fails because of, that call.

**Alternatives considered**
Call Google's revoke endpoint from the app. This ends the user's whole
grant to the shared client and can break their other Google connections,
so the broker owns the decision.

**Additional context**
Older brokers answer 409 `REVOCATION_NOT_AVAILABLE`. The app treats
that, a timeout, and a network error as "could not end access at Google"
and still removes the connection.

## What Changed

- `paperclip-cloud-connector.ts`: `revoke()` takes an optional
`timeoutMs`, returns the broker's `outcome` from an allowlist, and keeps
`REVOCATION_NOT_AVAILABLE` as an allowed rejection reason.
- New `tool-access/connections/google-access-removal.ts`. For a managed
Google connection, `prepare()` reads each removed grant's refresh token
and returns a `send()` step. `send()` calls the broker for each distinct
token, one after another, under one 5-second deadline. Every result and
every error (including a Paperclip Cloud client that cannot be created)
becomes an outcome code. It never rejects and never logs a token. An
unexpected error is logged with the error name only and reported as
`unexpected_error`.
- Order for removing a whole connection: read the tokens, run the whole
local teardown (grants, installs, app profile, gateway tokens, runtime
slots, catalog, OAuth state, archive the connection), call the broker,
delete the sent refresh-token secrets whatever the outcome, then delete
the remaining secrets. A failure anywhere in the teardown makes no
broker call. Order for removing one grant: read the token, commit the
revocation, call the broker, delete the grant's refresh-token secret.
`send()` owns the call and the deletion through an injected
secret-removal function.
- Owed revocations: a revoked managed Google grant whose refresh-token
secret still exists (and that nothing live uses) is sent by every later
removal, then the secret is deleted. This covers a process stop after
the teardown and before the call, with no new column. Once the secret is
deleted, nothing is sent again.
- Shared tokens: the shared (organization) sign-in writes the same
secret refs to the organization grant and the connection. So revoking an
organization grant alone neither calls the broker nor deletes the
secret, because the connection still uses that token; removing the
connection does both. The same rule applies to any refresh token still
used by another live grant or connection.
- Audit history: once the broker call settles, a
`connection.google_access_removal` audit event records `reasonCode` =
result and `providerAccess` (plus `grantId` for a single grant). If that
write fails, the removal still succeeds and the failure is logged.
- Revoke signs with the subject that consented to the grant
(`providerTenant.externalId`, saved by the managed callback from the
claim). Grants saved without it fall back to the refresh rule. Refresh
is unchanged; see Risks.
- `removal.ts` and `grants.ts`: `providerAccess: { provider, result,
outcomes }` goes to the responses and to the `tool_connection.archived`
and `tool_connection.grant_revoked` activity entries. The tool access
audit field `providerRevocation` keeps its Vercel meaning only.
- `packages/shared`: `ManagedGoogleAccessRemoval`,
`ManagedGoogleRevocationOutcome`, `ConnectionGrantRevocationResult`.
- UI: one helper, `connectionRemovalToast`, builds the removal toast
(base sentence plus the Google notice, tone and action) for Connectors,
Connections and the app page. No codes are shown.
- API: `docs/api/tool-connections.md` documents both DELETE responses
and `providerAccess`. The OpenAPI registration describes the optional
field and its four results.
- Docs: `GOOGLE-WORKSPACE.md` (order, owed revocations, decision table),
`GMAIL.md`, the Google connector memory note, and the help page on
deleting connection data.

Decision table, judged over the distinct tokens sent in this removal:

| Outcomes | `result` | User message |
| --- | --- | --- |
| Every token `revoked` | `ended` | "Paperclip's access to this Google
account was also ended at Google." |
| Any failure-type code: `provider_revocation_failed`,
`binding_not_known` (the broker had no record, so Google was not
called), `provider_revocation_disabled`, `revocation_in_progress`,
`revocation_not_available`, `broker_unavailable`, `broker_error`,
`credential_unavailable`, `unexpected_error` | `not_ended` | "Paperclip
removed the connection but couldn't end its access at Google. You can
remove Paperclip from your Google account's third-party connections." +
"Open Google account connections" |
| Only `other_connections_remain` | `other_connections_remain` | "Other
Paperclip connections still use this Google account, so Google keeps
Paperclip's access until you remove those too." |
| At least one `revoked`, the rest `other_connections_remain` |
`ended_unconfirmed` | "Paperclip's access was ended at Google. If more
than one Google account was connected here, check your Google account's
third-party connections to be sure." + "Open Google account connections"
|

Grants record no Google account identity: the managed callback saves
`externalId` (the Paperclip subject), `name` (the provider name), and
OAuth timing and scopes, and none of the 14 scopes returns an account
identifier. So the last row is always `ended_unconfirmed`. This includes
two tokens of one user and several agent grants on one Google account.

## Verification

- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/google-managed-connection-removal.test.ts
src/services/tool-access
src/services/paperclip-cloud-connector-revoke.test.ts
src/services/paperclip-cloud-connector.test.ts
src/__tests__/tool-connection-removal.test.ts
src/__tests__/tool-access-service.test.ts
src/__tests__/openapi-routes.test.ts` (714 passed)
- `pnpm --filter @paperclipai/ui exec vitest run
src/features/connections/googleAccessRemoval.test.ts src/pages/apps`
(passed)
- Server test typecheck (`node server/scripts/typecheck-layers.mjs`)
passes.
- Red then green: the server tests failed on `master` sources (21), then
each review round's new tests failed on the previous head and pass now.
They cover: the order on both paths, local failure before the grant
commit making no call and the retry sending, cleanup failure after the
call making no second call, a process stop between commit and call being
sent on retry with the secret deleted after, a Paperclip Cloud client
that cannot be created, the consenting subject for an organization
grant, sequential sends under one deadline, each row of the decision
table (per-agent and same-user cases give `ended_unconfirmed`, a mixed
failure gives `not_ended`), the unexpected-error log without a token, no
token in any log call, the delete-stored-data option, and a toast test
per surface.
- Added time, measured locally with a stubbed broker that answers at
once: median 22.1 ms against 20.3 ms without the broker step (12 runs
each), about 2 ms. With a real broker, one round trip per removed token,
capped at 5 seconds in total.
- `pnpm verify` exits 2 (a global CI input changed), so required CI
supplies the broad coverage.
- Not verified against a live broker or a real Google account.

## Risks

- A removal of a managed Google connection or grant can take up to 5
seconds longer when the broker is slow.
- Refresh subject is not changed in this PR. In the broker
(`recordProviderGrant`), a ledger row's subject is fixed when the row is
first written, and a refresh under another subject leaves that row
untouched. So moving refresh to a new subject could stop updating the
row's last-seen time. After 200 days the row would then count as not
live, and a later removal elsewhere could revoke at Google while this
connection still works. That risk cannot be ruled out from code for rows
written before this release, so both refresh paths keep their old rules.
The two refresh paths already use different subject orders today: the
gateway prefers `connectorSubjectAgentId` and then
`connectorSubjectUserId`, while background refresh prefers the grant's
own subject. That is pre-existing, and this PR leaves it unchanged.
Revoke uses the consenting subject, which matches the claim-time ledger
row. When the row was first written by a refresh under a different
subject, the broker answers `binding_not_known` and the user gets the
warning.
- Repeated removals can send an already-removed token again; the broker
returns its recorded outcome and changes nothing at Google.
- Against an older broker, users see the "could not end access at
Google" message, which is accurate.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and
code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 15:52:20 -06:00
DottaandClaude Opus 5.5 f4cb69205e Let plugin workers serve connector tools (plugin transport) (#22019)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Plugins can now add connector types to the Connectors catalog
(#22004), but only for transports the host already runs (remote MCP,
local stdio)
> - A plugin cannot yet serve a connector's tools itself, and legacy
plugin tools (top-level manifest `tools`) still reach every agent
without a connection or install
> - A plugin should be able to ship a full connector: settings,
credentials, tools, grants, policies and audit, like a built-in one
> - This pull request adds the `plugin` transport, the SDK API to serve
it, and moves legacy plugin tools under the same activation rules
without changing today's access
> - The benefit is that any connector can live in a plugin, and agents
get plugin tools only when an operator enables them

## Linked Issues or Issue Description

Refs #22004

**Subsystem affected**
Plugins (SDK, worker protocol, manifest), Connectors (tool access,
catalog discovery), tool gateway, database.

**Problem or motivation**
A plugin connector can only describe an existing transport. There is no
way for a plugin worker to serve a connector's tools with per-connection
settings and credentials. Legacy plugin tools are visible to every agent
as soon as the plugin is enabled, which is the opposite of the connector
activation rules.

**Proposed solution**
Add a `plugin` transport. The gateway keeps every decision (grant,
policy, approval, rate limit, audit) and calls the plugin worker with
the connection, its settings and a per-call credential. Legacy top-level
tools become one implicit connector per plugin. Existing installations
keep exactly their current access through a one-time migration.

**Alternatives considered**
Move legacy tools to connection tool names (`app.<connection>:<tool>`).
Rejected: names change, so existing tool policies and agent instructions
that name `plugin:tool` would stop matching (a possible widening or
loss). Legacy tools keep their names and handlers; only their visibility
is now gated by an install of the implicit connector.

## What Changed

- `packages/db`: migration `0491` widens
`tool_connections_transport_check` with `plugin` (`NOT VALID`, annotated
`online_compatible`), and adds `plugin_connector_legacy_access`, seeded
with one row per installed legacy-tools plugin and company.
- `packages/shared`: method schema gains `settingsSchema` and `tools`
(`{ kind: "static", list: [{ name, displayName?, description?, risk,
parametersSchema }] } | { kind: "dynamic" }`), only for `transport:
"plugin"`; tool names must match `[a-z0-9_]`; plugin-served methods use
`none` or `api_key`. The connect API accepts `settings`.
- `packages/plugins/sdk`: `ToolRunContext.connection` (`id`, `appSlug`,
`methodKey`, `grantId`, `settings`, `credential?`);
`ctx.connectors.register(appSlug, toolName, fn)` and
`ctx.connectors.listTools(appSlug, fn)`; new `listConnectorTools` RPC;
`executeTool` routes connector calls by `appSlug`. A worker refuses to
start when a declared static tool (or dynamic lister) has no handler.
The test harness gains `executeConnectorTool` and `listConnectorTools`.
- `server/src/services/plugin-connector-runtime.ts`: routes one call to
the plugin's worker; a stopped worker is a
`PluginWorkerUnavailableError`.
- Connect flow (`tool-access.ts`): a `plugin` method creates a
`paperclip_plugin` application and a `plugin` connection with `config =
{ sourceTemplateKey, connectionMethodKey, settings }`, validated against
`settingsSchema`. Credentials, grants, credential policy, installs and
profiles use the existing code.
- Catalog: static tools come from the definition, dynamic tools from
`listConnectorTools`, with the existing version hashing and retirement.
Tool risk comes from the declaration.
- Gateway: plugin connections join connected-tool discovery and both
execution paths (direct and approved actions). For each call the gateway
resolves the grant and, for `api_key`, the credential for that call
only. The run context comes only from the session. A stopped worker
returns `503 plugin_connector_unavailable` with `retryable: true`. The
token broker refuses to export plugin connector credentials.
- Legacy tools: a plugin with top-level `tools` and no `apps` gets an
implicit connector (slug `default`, named after the plugin, no
authentication). Its tools keep their names, handlers and today's access
decision, and an agent sees them only when a connection of that
connector is installed for it or for the company. The connector's saved
action rules (Off, Ask first) can only restrict them.
`completeLegacyPluginToolAccess` turns each migration row into such a
connection with a company install, at boot and when a disabled plugin is
enabled again; it is single-flight across servers, resumes a draft left
by an interrupted attempt, and never overrides an operator's later
changes.
- Catalog: tools a dynamic connector stops offering are retired from
discovery; `listConnectorTools` carries the connection's `companyId`,
and uses the acting person's grant for personal connections.
- Board: the setup flow renders `settingsSchema` with the shared JSON
Schema form, sends `settings`, and restores saved settings on resume and
reconnect.
- `create-paperclip-plugin --template connector` now scaffolds a plugin
connector (`apps` + `ctx.connectors.register` + a harness test). SDK
README and `PLUGIN_SPEC.md` describe connectors; top-level `tools` stay
documented as legacy.
- `inferToolRisk` moves to `tool-risk-inference.ts` so the catalog can
reuse it.

Design notes where the spec left room:
- OAuth for `plugin`-transport methods is not supported in this PR
(`none` and `api_key` only); OAuth plugin connectors can use
`mcp_remote` today. Noted as follow-up.
- Board "Test" calls on a plugin connection pass an empty
`agentId`/`runId`, because there is no agent run.
- Companies created after the upgrade get no automatic legacy
connection; they follow the normal activation rules.

## Verification

- `server`: `npx vitest run
src/__tests__/plugin-connectors-plugin-transport.test.ts` (new):
- rules 1-3: a ready plugin and configured accounts give no tools; an
install for agent A gives only A the tools; agent B is denied on
invocation;
- the worker receives `runContext.connection` with the right id, method,
grant, settings and the call's credential; spoofed arguments do not
change it; each account calls with its own key; results do not contain
the key;
- invalid settings are rejected (422); a stopped worker is `503
plugin_connector_unavailable` (retryable); a disabled plugin hides the
tools;
- dynamic tools are listed through the worker and a tool it stops
offering is retired;
- rule 5: a new installation of a legacy-tools plugin gives no tools; an
installation recorded by the migration keeps the same tool names for
every agent, through one company-wide connection (two concurrent
carry-overs make one; an interrupted draft is finished; a retry keeps an
operator's restriction), and is not also exposed as connection tools;
Ask first and Off on the implicit connector restrict the legacy tools;
  - the token broker refuses to export a plugin connection's credential.
- Existing gateway plugin-tool tests (`tool-gateway`,
`tool-gateway-service`, `prompt-injection-gateway`) now set up the
legacy grant an existing installation has, and pass unchanged otherwise.
- Regression: `plugin-connectors-circleback-parity`, `tool-gateway`,
`tool-gateway-service`, `prompt-injection-gateway`,
`tool-access-service` (846 passed), plus `connector-tool-policy`,
`plugin-lifecycle-restart`, `remote-mcp-connectors`,
`ai-connection-router` in an earlier run.
- UI: `AppsConnect.test.tsx` covers sending plugin settings from the
settings schema.
- SDK: `tests/worker-rpc-host.test.ts` (start refusal without handlers,
connector routing, tool listing). Scaffold: `entrypoints.test.ts`.
- `packages/db`: `pnpm typecheck` including `check:migrations`; server
layered typecheck (sources and tests); `tsc --noEmit` for shared, SDK,
scaffold and UI.

## Risks

- Behavior change for new installations: legacy plugin tools no longer
reach agents until an operator connects the plugin's connector.
Installations that exist at upgrade time keep their tools through the
migration rows.
- The migration widens a check constraint (`NOT VALID`) and copies
plugin × company rows into a new table. Both are fast; the previous
release never writes `plugin`.
- Calls depend on the plugin worker; when it is down the agent gets a
retryable error.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file editing, test execution).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 18:09:42 +00:00
DottaandClaude Opus 5.5 b1efe32c60 feat(connections): name and guide connections to another Paperclip instance (#22018)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Each Paperclip instance serves an assistant connection (MCP) at
`/mcp/paperclip`, and any instance can connect a remote MCP server by
its address
> - So one instance's agents can file and follow tasks on another
instance, for example a colleague's
> - But the other instance's tools look like the agent's own Paperclip
tools: same product, similar names, and the connection prefix was a
generic internal link key
> - An agent can then file local work on someone else's instance, or the
reverse
> - This pull request recognizes a Paperclip address, names the
connection and tool prefix after that instance, adds agent instructions,
and refuses a connection to the same instance
> - The benefit is that agents can tell their own Paperclip from a
connected one

## Linked Issues or Issue Description

**Subsystem affected**

Connections: "Connect your own MCP server" (generic remote MCP) and the
tool gateway names that agents see.

**Problem or motivation**

When instance A connects instance B through `/mcp/paperclip`, A's agents
get B's `paperclip_create_task` and other tools. The tool prefix came
from an internal key (`app-gallery🔗<uuid>`), so native runners saw
`app_mcp_app_gallery_link_paperclip_create_task_…`. Nothing told the
agent that the tools act on a different instance. An agent could send
internal work to another organization by mistake.

**Proposed solution**

Recognize an address whose path is `/mcp/paperclip`. Use only the typed
address, never data from the remote server. Name the connection after
the host (`acme Paperclip`). Use that name as the tool prefix for these
connections only. Add default agent instructions. Refuse the instance's
own address with `paperclip_self_connection` (422).

**Alternatives considered**

Read the company name from the remote `paperclip_connection` tool. This
needs an extra authenticated call during setup, and it puts
remote-controlled text in names and instructions. The host is enough to
identify the stack. Require approval for each write. We rejected this:
actions start on, the same as other connections.

**Roadmap alignment**

Supports instance-to-instance work through existing MCP connections. It
adds no new transport.

## What Changed

- `packages/shared/src/paperclip-peer.ts`: detect a Paperclip MCP
address, derive a label and name, compare with own origins (loopback
hosts on the same port are equal), and build default agent instructions.
- `server/src/services/tool-access.ts`: default name and default agent
instructions for a new Paperclip connection. Instructions that the
person supplies are kept.
- `server/src/routes/tool-access.ts`: refuse the instance's own address
before any network request.
- `server/src/services/tool-gateway.ts`: use the prefix recorded on a
Paperclip connection (`paperclipPeerToolNamespace`, derived from its
address at creation). Renames never change it. Other connections, and
Paperclip connections made before this change, keep their current tool
names.
- `ui/src/pages/apps/generic-mcp-connect.ts`: same default name in the
setup flow.
- `doc/connections/GENERIC-REMOTE-MCP.md`: new section "Connecting
another Paperclip instance".

## Verification

- `packages/shared`: `npx vitest run src/paperclip-peer.test.ts`, 20
passed.
- `server`: `npx vitest run
src/__tests__/generic-mcp-connection.test.ts`, 98 passed. The naming,
instructions, self-connection and tool-prefix tests fail on the base
revision and pass with this change (red/green).
- `ui`: `npx vitest run src/pages/apps/generic-mcp-connect.test.ts`, 29
passed.
- `tsc --noEmit` passes for `packages/shared` and `server`.
- Live check with two local instances. Instance B was in authenticated
mode on port 3201 with assistant connections on. Instance A was on port
3202.
- A connected to `http://localhost:3201/mcp/paperclip`. The name was
`localhost-3201 Paperclip`. A registered with B by dynamic client
registration and asked for read, write, configure and offline access.
- B's consent page showed "Connect Paperclip localhost-3202 to
Paperclip". After approval, A had all 37 tools active, writes included.
- From A, `paperclip_connection` returned B's company.
`paperclip_create_task` created `ACM-1` on B.
- An agent session on A listed
`mcp.localhost-3201-paperclip-…:paperclip-create-task` (runner name
`app_mcp_localhost_3201_p_paperclip_create_task_…`). Before the gateway
change, the name was `app_mcp_app_gallery_link_paperclip_connection_…`.
The run received the connection instructions.
- Connecting A to `http://localhost:3202/mcp/paperclip` returned 422
`paperclip_self_connection`.

## Risks

- Low risk. Only connections whose address path is exactly
`/mcp/paperclip` change. Other connections keep their names,
instructions and tool names.
- A Paperclip connection created before this change keeps its original
tool names, because only connections that record
`paperclipPeerToolNamespace` at creation get the new prefix. Saved exact
tool-name rules and pending approvals for those connections keep
matching. They also get no default instructions; people can add them.
- Native runners keep only 20 characters of the namespace (`mcp_` uses
4), so long host labels are cut. The tool part stays complete.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, browser pane).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 11:41:13 -06:00
DottaandClaude Opus 5.5 01451fae5b Let plugins add connector types to the Connectors catalog (#22004)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents use external services through connectors: a catalog of
connector types, connections (accounts) per type, and per-agent
installs, policies and audit in the tool gateway
> - Today only core can add a connector type. Every type is a JSON
definition compiled into `@paperclipai/shared`
> - Plugins can add agent tools, but those tools skip connections,
grants, per-agent installs and the Connectors page
> - This pull request lets a plugin add connector types with the same
definition schema, and makes them behave exactly like built-in
connectors
> - The benefit is that a connector can ship as a plugin, and an
operator controls it with the same Connectors page and the same access
rules

## Linked Issues or Issue Description

**Subsystem affected**
Plugins, Connectors (tool access), tool gateway

**Problem or motivation**
A plugin cannot add a connector type to the Connectors catalog. Plugin
tools (`tools` in the manifest) do not use connections, credentials per
account, agent installs, policies or the Connectors page.

**Proposed solution**
Add a manifest field `apps` (gated by a new `connectors.provide`
capability). Each entry uses the built-in app-definition schema. The
host namespaces the slug by plugin key and runs setup, OAuth, discovery,
grants, policies and audit through the existing code. This is the first
of two pull requests. The second adds a `plugin` transport so a plugin
worker can serve the tools itself.

**Activation rules (product requirements)**
1. Installing or enabling a plugin grants no tools. It only adds
connector types to the catalog.
2. Connecting an account grants no tools.
3. An agent gets the tools only from an agent install or a company
install of an active connection. Discovery and invocation both enforce
this.
4. A disabled or uninstalled plugin (or an upgrade that removes the app)
hides its connector types and connections from the Connectors APIs and
they reach no agent. Saved setup survives disable and comes back on
re-enable. Uninstall keeps rows archived for audit and revokes
credentials.


**Alternatives considered**
Keep plugin tools (`tools`) as the only plugin path. Rejected: they
bypass connections, per-account credentials, per-agent installs and the
Connectors page.

## What Changed

- `packages/shared`: new capability `connectors.provide`; manifest
`apps` validated with `appDefinitionSchema`. The validator rejects apps
without the capability, duplicate slugs, host-derived fields (`plugin`,
`aiConnectionRouter`, `experimentalFeatureKey`, `availability`), and
methods that need provider code in the host (REST adapters, chat
channels, AI runtimes).
- `packages/shared/src/plugin-apps.ts`: reversible namespaced slugs
(`plugin-<hex plugin key>-<slug>`), the catalog entry builder, and
`reviewedToolRisk`.
- `packages/shared/src/app-definitions.ts`: catalog lookups
(`getConnectableAppDefinition`, `getAppStoreDefinition`, ...) also
resolve the current plugin apps. The set is replaced on each read of
plugin state, so a built-in app is never shadowed.
- App definitions can declare `reviewedTools` (reviewed
read/write/destructive names and the risk for unlisted tools).
Circleback's reviewed risk moves from
`server/src/services/connectors/circleback/risk.ts` into
`circleback.json`, with the same classification. A copy of the
definition now classifies tools the same way.
- `validators/json-schema.ts`: `jsonSchemaSchema` moves to its own
module to break an import cycle between the plugin and app-definition
validators (still re-exported from `validators/plugin.ts`).
- `server/src/services/plugin-app-catalog.ts`: reads ready plugins from
the `plugins` table and answers "is this connection's connector
available". It does not trust in-memory lifecycle events, because
several lifecycle managers and processes can share one database.
- Gateway: connected-tool discovery and `resolveConnectionGrant` drop
connections from an unavailable plugin connector. Built-in connections
do not read the plugins table.
- Connectors APIs: the gallery lists ready plugin apps (with the same
Cloud connector profile filter as core); `listConnections` and
`listApplications` hide unavailable plugin connections; setup entry
points refresh plugin apps first.
- Plugin uninstall archives the plugin's connections through the
standard connection removal (grants revoked, owned credentials deleted,
rows kept).
- UI: the gallery response registers plugin apps for the setup screens;
`/apps/connect?source=plugin-…` loads the gallery first and shows an
unavailable state when the plugin is disabled.
- Docs: `doc/plugins/PLUGIN_SPEC.md` section 11A and the capability
list.

Design choices where the spec left room (picked to match "a plugin
connector behaves like a built-in connector"):
- Only `ready` plugins provide connectors. `error` and `upgrade_pending`
hide them, the same as the AI connection router plugins.
- `rest_api` is rejected for plugin apps in this PR. Core REST
connectors need reviewed adapter code in the host, so a definition alone
cannot work. The `plugin` transport in the next PR covers that case.
- Disabled plugin connections are hidden from list endpoints. Reading
one connection by id still works, so audit links keep resolving.

## Verification

- `packages/shared`: `npx vitest run src/plugin-apps.test.ts
src/validators/ src/app-definitions.test.ts` (416 passed). New tests:
manifest validation, slug namespacing, no shadowing of built-in apps,
reviewed risk.
- `server`: `npx vitest run
src/__tests__/plugin-connectors-circleback-parity.test.ts` (new). A
test-only plugin
(`server/src/__tests__/fixtures/plugin-connectors/circleback-parity/`)
ships an unmodified copy of `circleback.json`. The same journey runs
against built-in Circleback and the plugin copy with a recorded DCR
OAuth + MCP fixture, and must give identical catalog risk/status,
suggested defaults and per-agent tool sets. It covers each rule:
- Rule 1: the gallery adds only the namespaced type; no connections
exist; no agent tool changes.
  - Rule 2: two signed-in accounts, no agent sees a tool.
- Rule 3: an install for agent A gives only A the tools of that
connection; agent B is denied on invocation; each account calls the
provider with its own credential.
- Rule 4: disable hides the type, both connections and applications,
removes tools from discovery and invocation (404) and blocks new setup,
while built-in Circleback is unchanged; installs and grants are
byte-identical; re-enable restores tools with the same names; an upgrade
that removes the app hides it; uninstall archives both connections,
revokes grants and deletes owned secrets.
- Red/green: removing the gateway gate or the declared review makes the
test fail.
- An env-gated live check (`PAPERCLIP_CIRCLEBACK_LIVE=1`) starts the
real Circleback sign-in for both connectors and compares the requests.
It never runs in CI. I did not run it for this PR.
- Regression: `ai-connection-router`, `connector-tool-policy`,
`mcp-apps`, `plugin-lifecycle-restart`, `tool-access-service`,
`braintrust-connection`, `remote-mcp-connectors`,
`connection-intents-service`, `tool-gateway` (796 passed).
- `ui`: `app-connect-policy.test.ts`, `AppsConnect.test.tsx` (new plugin
cases) pass.
- `tsc --noEmit` for shared, server, ui (only pre-existing ungenerated
Storybook catalog errors), plugin SDK and CLI.
- `pnpm verify --explain` selects a broad plan because a doc under
`doc/plugins` changed; CI supplies that coverage.

## Risks

- No migration. Built-in connectors only gain one string check per
discovery; the plugins table is read only when a plugin connection is
present.
- Circleback's reviewed risk is now data. The behavior is the same, and
the existing policy tests and new tests cover it.
- Connection intents (agent connection requests in chat) still search
only built-in apps. Plugin connectors can be set up from the Connectors
page. This is a follow-up.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file editing, test execution).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 15:51:39 +00:00
DottaandClaude Opus 5.5 f6c5986b61 feat(connections): machine sign-in (MCP OAuth client credentials) for organization connections (#21473)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents use apps through connections to remote MCP servers
> - Today an organization connection must sign in as a person (browser
sign-in) or use a static key
> - Service accounts and CI-style integrations have no person who can
sign in
> - The MCP OAuth Client Credentials extension (2026-07-28) defines
machine-to-machine sign-in for this case
> - This pull request adds "Machine sign-in" for organization
connections, with a Paperclip-held signing key (recommended) or a client
secret
> - The benefit is that organizations can connect automation-owned MCP
servers safely, with credentials only in the vault and bound to one
sign-in service

## Linked Issues or Issue Description

**Subsystem affected**
Connections: remote MCP OAuth (server `tool-access/oauth`, tool gateway
token refresh) and the custom MCP server setup flow in the UI.

**Problem or motivation**
Organization-owned automation (service accounts, CI integrations) cannot
connect an MCP server that requires OAuth, because no person can
complete a browser sign-in. The MCP OAuth Client Credentials extension
(`io.modelcontextprotocol/oauth-client-credentials`) solves this with
`grant_type=client_credentials` and RFC 7523 signed client assertions.

**Proposed solution**
Add machine sign-in for organization connections only. Paperclip creates
a keypair per connection, keeps the private key in the company vault,
and shows the public key (PEM and JWK) for the admin to register. A
client secret is also supported. The sign-in service (issuer and token
endpoint) is pinned at setup. Tokens use the existing organization grant
refresh path, so caching, concurrent dedupe and 401 re-request reuse the
existing lease and single flight.

**Alternatives considered**
Reuse the deployment-wide `PAPERCLIP_TOOL_OAUTH_*` client env path that
already sent `client_credentials`. Rejected: it is not per connection,
it keeps secrets in env, and it has no issuer pin. Route machine sign-in
through `connectGalleryApp`. Rejected: its rollback deletes the draft on
a failed test, which would lose a key the admin already registered.

**Roadmap alignment**
Phase 1 of the enterprise MCP plan (no enterprise single sign-on
needed). OIDC login and enterprise-managed authorization (ID-JAG) are
later phases.

## What Changed

- `oauth/client-credentials.ts`: keypair generation (RS256, or ES256
when the service requires it), RFC 7638 key IDs, and RFC 7523 assertions
(`iss` = `sub` = client ID, `aud` = token endpoint, 60 s `exp`, unique
`jti`) with Node `crypto` only.
- `oauth/client.ts`: the token exchange accepts a signed assertion and
then sends only `client_assertion_type` and `client_assertion` (no
`client_id` or secret), as the extension specifies.
- `oauth/client-credentials-service.ts`: discovery of the sign-in
service (protected-resource metadata, then RFC 8414/OIDC metadata with
issuer check, or a token URL entered directly), issuer pinning,
vault-only key and secret handling, draft creation, test-and-activate,
and no-downtime rotation (test the new secret first; stage a new key and
switch only after the provider accepts it).
- `oauth/grant-refresh.ts` and `oauth/connection-refresh.ts`: machine
sign-in requests a new access token with the connection's own
credentials instead of needing a refresh token. It never falls back to
deployment-wide OAuth clients.
- `catalog/discovery.ts`: a rejected token on a machine sign-in
connection never re-runs discovery or rewrites the pinned endpoints.
- `mcp-http.ts` and the tool gateway: declare
`io.modelcontextprotocol/oauth-client-credentials` in client
capabilities only for connections that use it.
- Routes (with OpenAPI and activity logging): `POST
/companies/{companyId}/tools/machine-sign-in`, `GET|POST
/tool-connections/{id}/machine-sign-in[/test|/rotate|/rotate/complete]`.
Removal activity now records `machineSignIn: true`. The runner API
treats these routes as sensitive.
- Shared validators and types for machine sign-in.
- UI: "Machine sign-in" under Advanced authentication in the custom MCP
server flow (organization connections only), `MachineSignInSetup`
(choose Paperclip key or client secret, register the public key, test),
and Storybook stories for each setup state.
- Docs: `doc/connections/MCP-CLIENT-CREDENTIALS.md`, linked from
`doc/connections/README.md` and `GENERIC-REMOTE-MCP.md`.

## Verification

- `server/src/__tests__/tool-access-client-credentials.test.ts`: 16/16
pass. It uses a fake sign-in service that verifies real signatures
against registered keys, and a fake MCP server. It covers RS256 and
ES256 assertion claims and signatures, assertion-only client
authentication, the secret flow (`client_secret_basic`), discovery and
direct token URL, issuer pinning (refuses a different sign-in service
and sends it nothing), the pinned endpoint surviving a hostile
challenge, the token cache (0 token requests while live), concurrent
dedupe (5 callers on 2 service instances produce 1 token request), 401
re-request, key and secret rotation, a refused personal connection,
secret redaction in responses, rows and activity, and vault cleanup on
removal.
- Red/green: with the grant-refresh machine sign-in branch and the
discovery pin guard disabled, 5 runtime tests fail (pin, cache/dedupe,
401 re-request, both rotations). Restored: 16/16 pass. The other tests
cover new routes and services that do not exist on `master`.
- `ui/src/features/connections/MachineSignInSetup.test.tsx`: 3/3.
`ConnectionSetupFlow.architecture` and `.oauth` tests: 19/19.
- `openapi-routes.test.ts`: 51/51. `runner-api.test.ts`: 2153/2153.
`mcp-http.test.ts`: 9/9. Existing `tool-access-service.test.ts` refresh
and legacy client-credentials tests: 54/54 (filtered).
- Server package `tsc --noEmit`: clean. UI `tsc`: clean except the
pre-existing generated Storybook catalog imports.
- Not run locally (machine under heavy load): full suites, repo-wide
typecheck, browser check of the live app. CI covers breadth.

## Risks

- No migration. New config lives in `config.oauth` (`grantType:
"client_credentials"` plus `clientAuth`). Older env-configured
client-credentials connections have no `clientAuth` and keep their
original path.
- The capability declaration adds one extension key only for machine
sign-in connections. Other requests are unchanged.
- Rotation has no settings-page control yet; it is available through the
API. This is a follow-up.
- Low risk to agent boot and time to first token: a live token is read
exactly like a browser sign-in token.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edits, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 03:10:32 +00:00
Devin FoleyandPaperclip dc018f9bc0 Keep agent-owned connections healthy during catalog refresh (#21539)
Use dedicated agent grants for catalog refresh, GitHub health checks, and reconnect status. Preserve shared connection health when a different agent lacks authorization.

Add PostgreSQL and HTTP regressions for credential selection, stale health recovery, caller isolation, and actual provider rejection. Correct the MCP demo approval fixture to submit a valid saved-card option through normal answer validation.

Validation: 629 focused tests, server typecheck, all CI checks, and Greptile 5/5 with no unresolved review comments.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-13 18:45:20 -08:00
DottaandClaude Fable 5.2 0199c963e2 feat(apps): ship the redesigned sign-in step for a pasted MCP server (#21469)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Apps lets an operator connect a remote MCP server by pasting its
address ("Connect your own MCP server")
> - Step 2 of that flow put every sign-in choice in an "Advanced
authentication" box with wrapped pill buttons and a hint paragraph
> - The box hid the choice behind a vague label, and its shape did not
match the access line ("Connects for everyone… Change") right below it
> - The owner approved a redesign: a stated sign-in default with a
Change link, and five options with their fields directly below each
option
> - This pull request wires that redesign into the real screen, removes
the old step, and updates copy that named the removed box
> - The benefit is a screen that reads as two stated answers and one
action, with the same default path (paste the address, press Check link)

## Linked Issues or Issue Description

No public issue exists. Enhancement:

**What existing behavior does this improve?**
The sign-in section of the "Connect your own MCP server" step in Apps.

**Subsystem affected**
UI: Apps connection setup (`ui/src/features/connections`). Small copy
changes in the server OAuth client registration message and docs.

**Current behavior**
Step 2 shows an "Advanced authentication" collapsible with five wrapped
pill buttons and a hint paragraph below them. The fields for the chosen
mode appear further down. Guidance banners tell the operator to look
"under Advanced authentication". The page also shows the "Unverified
server" badge twice (header and body). The stepper reads "Pick app · Add
your key", even when the server needs no key.

**Proposed behavior**
Step 2 states the sign-in choice in one line ("Paperclip asks the server
how to sign in.") with a Change link. Change shows five radio cards: Let
Paperclip check, Key or token, Browser sign-in, Custom headers, No
sign-in needed. The fields for the selected option are directly below
it. Done collapses the list to a one-line summary (for example "Signs in
with 1 custom header."). The list stays open while the chosen option
still needs a value. Client ID and secret are behind "Use your own
client ID" unless they have values or the server asked for them. The
stepper reads "Pick app · Connect".

**Reason and benefit**
The choice is visible as a stated default in the same shape as the
access line. Each option keeps its fields next to it. No copy points at
a box that does not exist.

**Breaking changes**
None. The five auth modes and their request payloads do not change.

**Additional context**
Related open PRs change server behavior for the same flow. They are not
duplicates. Refs #15192, Refs #15450.

## What Changed

- `ui/src/features/connections/GenericMcpConnectStep.tsx`: new step
component (same props as the old `LinkConnectStep`). It uses stable
field ids `generic-mcp-key`, `generic-mcp-client-id`,
`generic-mcp-client-secret` and a `data-testid` on the sign-in
Change/Done link.
- `ui/src/features/connections/ConnectionSetupFlow.tsx`: render
`GenericMcpConnectStep`; delete `LinkConnectStep`,
`GENERIC_AUTH_MODE_OPTIONS`, `SegmentedOption` and unused imports;
change the pasted-link stepper label from "Add your key" to "Connect"
(`STEP_LABELS` is used only by the pasted-link flow; curated apps keep
their own `["Add your key"]`).
- `ui/src/pages/apps/generic-mcp-connect.ts`: `oauth_challenge` and
`oauth_manual_client_required` guidance now say "below" instead of
"under Advanced authentication".
- `server/src/services/tool-access/oauth/client-registration.ts`: the
manual-client message no longer names the removed box.
- `doc/connections/GENERIC-REMOTE-MCP.md`: describe the sign-in line,
Change, and the five options.
- Comments in `packages/shared` and `server/src/services/tool-access.ts`
reworded. `tests/runner-e2e/source-sdk-public-inputs.json` regenerated
for the two shared files (blob hashes only).
- `ui/storybook/stories/connection-setup/byo-mcp-sign-in.stories.tsx`:
14 stories for the shipped component; annotation is now `current` /
`production`.
- Tests: `AppsConnect.test.tsx` generic MCP block uses the sign-in
toggle and radio options; new tests for the collapsed summary, the
open-while-empty-key rule, and a single "Unverified server" label. New
`GenericMcpConnectStep.test.ts` unit-tests `genericMcpAuthSummary`.

Before: "Advanced authentication" box with pill buttons and a hint;
fields below the pills; two "Unverified server" badges; stepper "Pick
app · Add your key".
After: one sign-in line with Change; five radio cards with fields under
the selected card; one "Unverified server" badge in the header; stepper
"Pick app · Connect". Storybook: `byo-mcp-sign-in--choosing`,
`byo-mcp-sign-in--server-asked-for-key`.

## Verification

- `cd ui && pnpm exec vitest run src/pages/apps/AppsConnect.test.tsx
src/pages/apps/generic-mcp-connect.test.ts
src/features/connections/GenericMcpConnectStep.test.ts
src/features/connections/ConnectionSetupFlow.architecture.test.ts
src/features/connections/ConnectionSetupFlow.oauth.test.ts`: 5 files,
300 tests pass.
- `cd ui && node ../scripts/run-tsc.mjs -b`: pass.
- `pnpm -s typecheck` in `packages/shared` and `server`: pass.
- `pnpm check:token-gates`: all gates clean.
- `node scripts/storybook-catalog.mjs`: pass (generated indexes not
committed).
- `pnpm verify`: exit 2 (broader verification required, because
`doc/connections/GENERIC-REMOTE-MCP.md` is a global CI input). Required
CI supplies that coverage. The focused files above were rerun on the
final revision after the rebase.
- Headless Playwright against the Storybook dev server: the Default,
Choosing and Server-asked-for-key stories render the new step, the new
guidance copy, and the "Pick app · Connect" stepper.

## Risks

- Low risk. UI change on one screen; request payloads
(`genericConnectPayload`) do not change.
- Tests that click the first collapsed "Change" button now skip the
sign-in toggle, because the access line also has a "Change" button.
- The server message for `oauth_manual_client_required` changes text. No
code matches on the text; the client uses the error code.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, headless Playwright).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-13 14:35:49 -06:00
DottaandPaperclip 5859e7cfa6 perf(apps): avoid repeated writes during permission saves (#21033)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Apps give agents access to tools through saved profiles and approval
rules.
> - A save replaces the full set of action entries and access bindings.
> - It also updates each Ask first rule and each retired rule in a
separate query.
> - Adding one agent can therefore cause hundreds of database round
trips.
> - This pull request keeps unchanged rows and batches the required
changes.
> - The benefit is faster saves with the same action permissions.

## Linked Issues or Issue Description

**What happened?**

An app permission save repeats database work for actions and agents that
did not change. With 100 actions and 500 retired rules, one agent
addition makes 636 queries. Of these, 599 update approval policies.
Related boundary coverage: Refs #19973. This change keeps that issue's
Off and agent-selection protections.

**Expected behavior**

Adding an agent applies the saved Allowed, Ask first, and Off choices.
The save changes only the rows that need a change. Independent approval
rules remain in force.

**Steps to reproduce**

1. Create an app with 100 active actions and 99 Ask first rules.
2. Add 500 disabled app approval rules.
3. Save access for three agents. Then add a fourth agent.
4. Count database queries during the save.

**Paperclip version or commit**

Baseline: `51d77648bc62`.

**Deployment mode**

Built from source. The reproduction uses disposable local PostgreSQL.

## What Changed

- Keep action entries and access bindings when their saved values match.
- Insert new bindings in one query. Remove only bindings and entries
that no longer apply.
- Read managed approval rules for the selected app. Skip unchanged rules
and rules that are already disabled.
- Batch new or changed Ask first rules and disable stale enabled rules
in one query.
- Keep existing policy conditions, priority, author, and creation time
when a rule changes.
- Add real PostgreSQL regression tests and a repeatable benchmark.
Document independent approval rules.

## Verification

- Red: the original regression made 2,995 policy writes across five
saves and failed the zero-write check.
- Green: 636 focused service and runtime tests passed after the rebase.
CI supplies the broad checks required by `pnpm verify` (exit 2).
- Benchmark: `node --import ./server/node_modules/tsx/dist/loader.mjs
scripts/benchmark-app-permission-save.ts --baseline-ref=51d77648bc62
--output=/tmp/app-permission-save.json`.
- Local benchmark: two warmups and six samples in alternating order.
Restore three agents outside each timed sample, then add the fourth
agent. Median save time fell from 458.51 ms to 38.05 ms (12.0 times
faster). Query count fell from 636 to 32. Unchanged policy writes fell
from 599 to zero.
- Compare all saved profile, entry, binding, and policy values after
mapping fixture IDs and excluding generated row IDs and write times.
Check Allowed and Ask first decisions for the added agent. Tests also
cover Off, an unassigned agent, additive access, removal, independent
policies, company boundaries, and re-enabled rules.
- CI passes on `069e2c76252a19ec3c11a061de0506eccb0463d1`: [PR
workflow](https://github.com/paperclipai/paperclip/actions/runs/43162600023).
This includes typecheck, build, server tests, and all eight browser
shards.
-
[Greptile](https://github.com/paperclipai/paperclip/pull/21033#issuecomment-6815608829)
gives the final head 5/5. Its one finding is resolved. The separate
Paperclip Review check is still queued. Its review task stopped after a
Runner failure.
- Hosted request timing and the specific reported approval prompt are
not verified. No live permissions changed.

## Risks

- The save now retains unchanged row IDs and timestamps. Consumers must
use permission values to determine access.
- Existing transaction locks and permission checks remain in place. The
runtime concurrency tests exercise these paths.
- The benchmark measures a local service with synthetic data. It does
not establish hosted endpoint latency.
- No migration or startup change is required.

## Model Used

OpenAI Codex, GPT-6. Exact provider model ID is not exposed in this
session. Used reasoning, tool calls, and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-13 12:57:03 -06:00
DottaandClaude Opus 5.5 2d1c2db64a feat(mcp): host the MCP Skills extension for connected apps (#21219)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents use connected apps through MCP. Paperclip already hosts MCP
Apps views and speaks stateless MCP 2026-07-28.
> - The MCP Skills extension (SEP-2640, Final) lets an app publish
skills (instructions and files) next to its tools. Hugging Face already
serves 25.
> - Paperclip did not read them, so agents missed each app's own
workflow guidance.
> - Skills are a prompt-injection surface. The spec makes manifest
binding, verification, origin tagging and approval revocation host
requirements.
> - This pull request makes Paperclip a Skills host: discover, review,
verify, and deliver app skills to the agents that use the app.
> - The benefit is that connected apps can teach agents their workflows,
under board control.

## Linked Issues or Issue Description

**Problem**
Connected MCP servers can publish skills with the
`io.modelcontextprotocol/skills` extension. Paperclip ignored them. A
board could not see, review or approve them, and agents never received
them.

**Proposed solution**
Implement the host side of SEP-2640. List skills during catalog refresh,
record them per connection and company, bind board approval to the full
file manifest, and load approved skills into agent runs with verified,
lazily fetched content.

**Why it matters**
Apps ship workflow guidance with their tools. Without the extension,
each team must copy that guidance into company skills by hand, and it
goes stale.

Refs #21119, Refs #21140 (the MCP Apps and stateless MCP client work
this builds on).

## What Changed

- **Client capability.** Every stateless request declares
`io.modelcontextprotocol/skills`. Legacy handshake sessions do not. The
capabilities that `server/discover` advertises are remembered beside the
protocol era.
- **Discovery.** Catalog refresh calls `skills/list` (paged) only when
the server declared the extension. A listing inside its `ttlMs` is
reused. A skills failure never fails the tool refresh. Nothing runs
while a run starts.
- **Schema.** New `tool_mcp_skills` (company, connection, skill URI;
frontmatter, sorted manifest, manifest digest, review state) and
`tool_mcp_skill_files` (verified bytes keyed by connection, URI and
digest). Both cascade with the connection.
- **Governance.** New skills wait for review. Approval binds to the
order-free digest of every `{uri, digest, size}`; a request for any
other digest gets 409. A changed manifest or frontmatter revokes
approval and logs `tool_mcp_skill.approval_revoked`. Dynamic, oversized
and invalid skills are declined with a reason. Approve and decline use
the company skill policy (`skills.install`, `external_package`,
`mcp-connection:<id>`).
- **Run time.** Approved skills of connections the agent can use load in
parallel with other skill preparation, inside a 3 s budget. `SKILL.md`
is read with the agent's grant and checked for size, SHA-256 and
frontmatter. A failure calls `skills/get` and takes the skill out of
use. Each skill becomes a per-run bundle on the connector skill path,
named in its app's namespace, tagged with its origin. Paperclip writes
the frontmatter: `allowed-tools` only if the board approved it.
- **Supporting files.** `GET
/api/agents/me/mcp-skills/:skillId/files?path=` serves manifest-listed
files to an agent in an active run, through the skill's own connection.
The sandbox bridge allows the route.
- **Board API and UI.** `GET/POST /api/companies/:companyId/mcp-skills…`
(list, read instructions, approve, decline). The Skills page shows
"Skills from connected apps" with app, state, size, requested tools and
plain-language notes. It renders nothing when no app publishes skills.
- **Docs.** `doc/connections/MCP-SKILLS.md`, linked from
`doc/connections/README.md`.
- **Shared frontmatter parser.** It now decodes YAML single-quoted
scalars. Before, 5 of Hugging Face's 25 skills kept their quotes and
would fail the frontmatter check; now all 25 match. The generated runner
copy is regenerated.

File size notes: `server/src/services/tool-gateway.ts` grows by about
200 lines. The run-time reads must use its grant resolution and
`callAssignedConnectionProtocol`, like MCP Apps view reads. Splitting
the gateway is a separate refactor. `CompanySkills.tsx` and
`CompanySkills.production.tsx` gain 2–4 lines each; the section lives in
its own component.

## Verification

- `server/src/__tests__/mcp-skills.test.ts` (15 tests) uses a fake
stateless skills server and embedded Postgres. It covers: discovery only
after the server declares the extension; the capability in `_meta`;
`ttlMs` reuse; lazy fetch (no reads at listing or approval; `SKILL.md`
at load; supporting files on read); a stale-digest approval refused; a
reorder that keeps approval; a changed or added file that revokes it;
dynamic skills declined; a same-size tampered `SKILL.md` rejected,
refreshed with `skills/get`, and taken out of use; mismatched
frontmatter rejected; reads outside the manifest refused; the same URI
and name from two servers kept apart; company scoping; a slow app
skipped within the budget and cached later; HTTP authorization (board
only, agent needs an active run, run header must match) and activity
logging.
- Red/green: with the digest check disabled, the tamper test fails; with
the manifest comparison disabled, the revocation test fails. Both pass
with the code.
- `server/src/__tests__/mcp-protocol-versions.test.ts`,
`mcp-apps.test.ts`, `connection-instructions.test.ts`,
`tool-gateway-discovery-http.test.ts`: 47 pass (two capability
assertions updated for the new extension).
- `packages/shared/src/frontmatter.test.ts` adds a single-quoted scalar
case; the runner contract test passes (23 tests).
- `ui/src/components/skills/ConnectedAppSkills.test.tsx` (3) and
`ui/src/pages/CompanySkills.test.tsx` (44) pass.
`packages/adapter-utils/src/sandbox-callback-bridge.test.ts` (89)
passes.
- `pnpm --filter @paperclipai/server typecheck` passes. UI `tsc` passes
except the known missing generated Storybook catalog files.
- `pnpm verify` returned exit 2 (doc change is a global CI input);
required CI supplies broad coverage.
- Live check against `https://huggingface.co/mcp` without credentials,
through Paperclip's own services and the real egress guard: 25 skills
discovered as pending; three approved; cold load 135 ms, cached load 13
ms; a supporting file read through the manifest; the bundle tagged with
its origin.
- Storybook: `Agents and reusable capabilities/Skills and Skill
Studio/Connected app skills` (needs review, all states, tool request,
name collision, reading instructions).

## Risks

- **Migration:** two new tables only. No changes to existing tables.
Safe with the previous release running.
- **Run start:** companies without an approved app skill pay one indexed
read. With approved skills the wait is capped at 3 s; late or failing
skills are left out with a `skills.mcp_unavailable` run warning and
never fail the run.
- **Agent harness enforcement:** Paperclip cannot gate each
code-execution tool call an agent makes while following an app's skill.
Approval is explicit and per skill, the approval note says agents may
run commands the skill describes, and `allowed-tools` is never granted
without a separate tick.
- **Shared-home adapters** receive app skills like company skills. A
revoked skill leaves on the next skill sync.
- Not in this PR: per-connection opt-in for dynamic skills; serving
Paperclip's own skills from its public MCP endpoint;
`resources/directory/read`; app skills on an agent's skills tab.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edits, local test runs, live HTTP checks).
Extended reasoning enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 19:18:06 -06:00
DottaandPaperclip 5bad074baa feat: add individual agent interaction permissions (#20411)
Add eleven sparse incoming and outgoing restrictions, human CLI/API administration, shared mutation enforcement, and pending-delivery checks. Preserve existing defaults, agent settings, and existing authorization boundaries.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-09 20:34:04 -06:00
DottaandPaperclip b94963f91e fix: retry temporary repository catalog failures before dispatch (#19690)
## Thinking Path

> - Paperclip manages AI agents and their work.
> - A task must check repository access before it starts agent work.
> - Repository discovery can fail while the user's existing grant
remains valid.
> - Discovery previously replaced a known temporary provider refusal
with an empty catalog.
> - The task then reported lost access and did not schedule recovery.
> - This change keeps temporary discovery failures distinct and uses the
existing bounded retry owner.
> - Each retry must pass a fresh access check before it can use the
saved repository.

## Linked Issues or Issue Description

**What happened?**

A task with a saved catalog repository stops with an access denial when
GitHub returns an exhausted-quota response or HTTP 503. The task does
not start provider work, but the discovery layer discards the temporary
failure and returns an empty catalog.

**Expected behavior**

Known temporary failures before provider dispatch use the existing
bounded infrastructure retry. The successor checks the same responsible
user's current grants and reads the saved files only after fresh
discovery succeeds. A completed denial remains terminal.

**Steps to reproduce**

1. Bind a task to an authorized catalog repository with saved local
changes.
2. Make its authorized GitHub discovery return HTTP 403 with zero
remaining quota, or HTTP 503.
3. Let discovery recover before the existing retry deadline.
4. Observe one successful successor using the original repository bytes.

**Paperclip version or commit**

Reproduced on master at `b8d94b3ad1`. A search found no existing PR for
this temporary catalog failure.

## What Changed

- Keep a safe typed failure for known HTTP availability and quota
responses. Honor the later provider deadline and bound it to one day.
- Propagate that failure only when fresh discovery cannot find a
required repository. Preserve successful partial catalogs and the
existing connection setup error contract.
- Classify the failure only before provider dispatch. Reuse existing
bounded retry, interaction ownership, and bootstrap settlement rules.
- Keep current grant checks, saved repository selection, normal Stop,
and provider-started failure policy.
- Keep provider deadlines when accepted interactions use their own retry
owner. Recognize secondary HTTP 403 rate limits with valid retry advice
while keeping ordinary permission denials terminal. This matches [GitHub
rate-limit
guidance](https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api).
- Keep direct repository request responses outside the automatic
bootstrap retry contract. Test successful partial discovery and
missing-required-repository recovery with two independent eligible
connections.
- Document task repository discovery recovery.

## Verification

- Behavioral RED: real heartbeat and PostgreSQL fixtures reached actual
HTTP 403 and503 discovery responses, started no adapter, and found zero
successors instead of one. Both cases failed before the production
change.
- Behavioral GREEN: both cases waited for the real retry deadline, used
the normal queue worker, ran the controlled adapter once after fresh
discovery, and read the original saved bytes. Two cases passed in 74.73
seconds. No external model or provider request was made.
- Deadline RED/GREEN: a combined Retry-After and quota-reset response
initially selected the earlier deadline. The regression now selects the
later deadline.
- Initial repair at `d8d4714a18` (same production source as
`018e8c52d345d79972d7907468080dc9908c299f`): seven affected files
passed, 69 cases in 185.16 seconds. That test-only cleanup then passed
all five real-database acceptance cases in 134.76 seconds.
- Commands: `pnpm exec vitest run` with
`heartbeat-repository-catalog-recovery.test.ts`,
`github-catalog-timing.test.ts`, `github-grant-metadata.test.ts`,
`project-repositories-persistence.test.ts`,
`task-workspace-repositories-persistence.test.ts`,
`github-repository-catalog-failure.test.ts`, and
`services/heartbeat/retries.test.ts` from the server source tree. The
acceptance file also ran fresh at `018e8c52d3`. A fresh five-case run
after the P1 repairs passed at `f490ca6b78` in 122.06 seconds. The final
eight-case file passed fresh at
`e5b2efab11919467850f55fd6345317132b9b672` in 124.62 seconds, using real
owned embedded PostgreSQL. It covers automatic recovery with saved
bytes, genuine denial, provider-started classification, normal Stop, the
direct HTTP route, and both partial-catalog outcomes.
- Review repairs at `f490ca6b787cf77df96d3d6edfb2ab9115eb19a6`: both
reported P1s reproduced RED. The real database interaction scheduler
chose a retry after 24 seconds before the provider deadline of one hour.
A secondary HTTP 403 with valid Retry-After lost its retry
classification. The narrow fixes then passed 44 cases across scheduler,
header, and metadata-contract files in 37.66 seconds.
- A real direct request through the task workspace router and normal
error middleware reproduced a 500 response instead of the existing 403.
Required-ID retry mode now applies only to admission and its rechecks.
The route and both two-connection partial-catalog cases then passed in
29.09 seconds. Both partial cases use actual owned grants and credential
bindings.
- All seven server type layers passed again at final head
`e5b2efab11919467850f55fd6345317132b9b672`. The affected test type layer
passed again after the test-only cleanup. Environment: macOS arm64, Node
26.4.0, pinned pnpm 9.15.4, Vitest 5.0.3, owned disposable PostgreSQL 18
on loopback. The current checkout's shared packages, plugin SDK, and
Runner were built first.
- One local acceptance run under concurrent owned test/typecheck load
hit the unchanged 15-second Stop-case timeout. Its four other cases
passed. The unchanged complete file then passed serially. The timeout
receipt remains recorded; no assertion or timeout was weakened.
- A provider-started failure retains the existing generic adapter retry
policy. It does not receive the catalog no-provider certificate or a
provider retry deadline. Fixture cleanup cancels its scheduled retry
through the normal owner.
- Final head `e5b2efab11919467850f55fd6345317132b9b672` has 59
successful checks, 3 allowed skips, no pending or failed check, and
Greptile 5/5 with no open findings. [Completed CI
workflow](https://github.com/paperclipai/paperclip/actions/runs/42322202977).
- Additional local diagnostic for the late recovery-owner advisory: on
unchanged `e5b2efab`, the dedicated scheduler returned `retry_exhausted`
with attempt 3/max 2. The normal stranded sweeper then escalated once,
created no extra run, and started no adapter, even after catalog access
recovered. One case passed in 17.32 seconds. The existing
failed-predecessor path delegates to that bounded scheduler and returns;
it does not enter a new generic retry budget. No production registry
change was made. This diagnostic was not a CI case. Its local harness is
preserved separately at `74e3854eea3f7c4121fdcd9c8a8ad522cd8c7477` on
`codex/catalog-exhaustion-acceptance`; its production source is
identical to `e5b2efab`.
- Module boundary and whitespace checks passed.

## Risks

- A temporary failure from another eligible connection can delay a truly
missing repository for the existing limit of two attempts. It cannot
grant access.
- Unknown network errors and genuine access denials retain their
existing behavior. This change handles known HTTP failures only.
- Historical provider quota causality is unproven. This repair has a
separate local producer regression. It does not rewrite past failures or
queued user intents.
- Acceptance uses real local database and heartbeat owners with
controlled HTTP and adapter fixtures. It does not claim a live
native-provider recovery.

## Model Used

OpenAI GPT-6.1 Codex (`gpt-6.1-sol`), with high reasoning, repository
tools, code execution, and controlled local PostgreSQL tests. The host
did not expose a context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-06 19:29:40 -06:00
Nicky LeachandClaude Opus 5.5 ff1d1dea5d fix(tools): reject non-tool connections before any write (#19492)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Apps and tools connect to Paperclip through tool connections
> - `POST /api/companies/:companyId/tools/connections` creates a tool
connection through `createConnection`
> - The request schema accepts a channel purpose and the chat and voice
transports, but `createConnection` cannot store them
> - The database then rejects the row, so the caller gets a 500 and a
stray tool application stays behind
> - This pull request rejects those requests with a 422 before any
write, and stores `config` as `transportConfig` when the caller leaves
`transportConfig` out
> - The benefit is a clear error, no leftover rows, and consistent
connection rows

## Linked Issues or Issue Description

No issue exists. The new test typecheck in #18739 found this problem.
#18739 listed it as a possible product problem and did not change it.

**What happened?**

1. `createConnection` never writes `input.connectionPurpose`, so every
row gets the database default `tool`.
2. The schema allows `transport: "chat_sdk"` and `transport: "voice"`.
The database check `tool_connections_channel_transport_check` does not
allow these transports on a `tool` row.
3. No error mapping exists for a check violation, so the route returns
500.
4. When the request has no `applicationId`, the service inserts a
`tool_applications` row first, outside the connection transaction. That
row stays after the failure.
5. The schema gave `transportConfig` a default of `{}`. The service
stores `input.transportConfig ?? config`, so it stored `{}` instead of a
copy of `config`.

**Expected behavior**

The route returns 422 for a purpose or transport that it cannot create,
and writes no rows. When the request leaves `transportConfig` out, the
row stores `config` there.

**Steps to reproduce**

1. Send `POST /api/companies/:companyId/tools/connections` with `{
"name": "x", "transport": "chat_sdk" }`.
2. See the 500 response.
3. See the new `tool_applications` row for the company.

## What Changed

- `server/src/services/tool-access.ts`: `createConnection` rejects a
purpose other than `tool`, and the `chat_sdk` and `voice` transports,
with 422. The check runs before any lookup or write. Channel connections
still come from their own services.
- `packages/shared/src/validators/tool-access.ts`: `transportConfig` is
optional and has no default. The update schema already removed defaults,
so it does not change.
- `server/src/__tests__/tool-access-service.test.ts`: added tests for
the three rejected inputs (no rows written), and for a request that
sends only `config`.

## Verification

- `pnpm exec vitest run server/src/__tests__/tool-access-service.test.ts
server/src/__tests__/generic-mcp-connection.test.ts
server/src/__tests__/tool-connection-health-work.test.ts
server/src/__tests__/openapi-routes.test.ts
server/src/__tests__/tool-access-routes.test.ts
packages/shared/src/validators`: 27 files, 995 tests pass.
- `cd server && pnpm typecheck`: passes. This includes the source layers
and the three test-tree shards.
- Not run locally: the full `pnpm test:run`, the root `pnpm build`, and
the UI typecheck. Required CI covers them.

## Risks

- Low risk. A client that sends a channel purpose or a chat or voice
transport to this route now gets 422 instead of 500. The request failed
before this change too.
- A client that sends only `config` now gets the same value in
`transportConfig`. Every reader checks `config` first, so this changes
no behavior.
- The OpenAPI output for this body no longer shows a default for
`transportConfig`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits) and extended thinking.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 17:30:24 +00:00
Devin FoleyandPaperclip 8f79a32898 fix(connections): keep agent access consistent through setup and permission changes (#19352)
Task-hosted tool setup kept requester-only defaults, and audience edits could leave runtime installations restricted after permission bindings were widened. Large provider catalogs could also fail before creating an access card.

Use the normal agent audience defaults for new tool setup, preserve explicit selections through verified OAuth and draft resumption, and synchronize permission bindings and runtime installations atomically. Enforce agent-edit authority and preserve existing access during additive account reuse.

Validation: 915 focused regression tests, typechecks, and builds passed. All 54 CI checks passed and Greptile Apex scored the reviewed head 5/5 with no unresolved threads. Broad local test failures, recoveries, and incomplete heartbeat coverage are documented in the PR.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-05 17:38:32 -08:00
Michael NguyenandPaperclip 58d1b21098 fix: separate fresh setup from archived personal connections (#18902)
## Thinking Path

> - Paperclip helps people manage AI agents and their service access.
> - Each personal connection belongs to one user.
> - Setup can recover an archived connection by its name and provider.
> - A fresh setup can therefore select another user's old identity and
fail its owner check.
> - This change creates a separate connection for fresh setup after
personal archival.
> - Explicit reconnect still requires the existing owner and preserves
its history.

## Linked Issues or Issue Description

**What happened?**

A fresh GitHub setup can return 403 after a personal connection with the
same name was archived. The request does not include a reconnect target.
Setup still selects the archived identity by name and provider.

**Expected behavior**

Fresh setup creates a separate personal identity. It leaves the archived
connection, credentials, grants, and activity unchanged. Explicit
reconnect rejects a different user.

**Steps to reproduce**

1. Create a personal GitHub connection in a test company.
2. Archive that connection and its application.
3. As another authenticated member, start fresh personal setup with the
same name and provider.
4. Observe the owner-check failure before this change.

**Paperclip version or commit**

Base: db9be4c5a. Tests use isolated database and HTTP fixtures.

**Deployment mode**

Authenticated. The legacy owner can be a local board identity.

## What Changed

- Exclude implicitly recovered archived personal connections from fresh
setup.
- Use the existing company-scoped name allocator to create a separate
application and connection.
- Test three owners: a legacy board identity, another user, and the
current user.
- Check credential isolation, unchanged archived records and activity,
missing-key rejection, and explicit reconnect denial.
- Keep managed revival and generic MCP rollback/reconnect fixtures on
explicit reconnect targets.
- Verify generic personal URL setup creates an independent identity and
preserves the archived records.
- Document fresh setup and reconnect behavior.

## Verification

- Final head: `96184e3179aab2eeb88df87d442691741a4875a2`. Full CI
recovered and completed successfully: [run 41752164958, attempt
2](https://github.com/paperclipai/paperclip/actions/runs/41752164958/attempts/2).
PR rollup: 53 successful checks, 3 skipped, none failing or pending.
Typecheck, build, all server shards, Runner lanes, and E2E gates pass.
- Exact-head Greptile: 5/5, zero inline comments or review threads.
Independent engineering/security reviews approved the production
implementation; engineering approved the test-only follow-up. Browser
acceptance review approved with the timing qualification below.
- Final-head local generic MCP suite: 84/84 passed (23.26 s); server
typecheck passed. Reviewer independently reran the three changed
scenarios: 3/3 passed (15.74 s).
- Production head `587631230`: local tool-access suite 504/504,
recursive typecheck, and full build passed. Broad local checks were not
repeated for the test-only delta. Full local `pnpm test:run` remains
incomplete after interruption during the first chat shard; no
full-local-suite pass is claimed.
- Initial full CI at `587631230` failed three generic MCP fixtures that
still relied on implicit archived personal recovery. All three
reproduced locally (15.01 s). The test-only follow-up gives owner
reconnect/rollback scenarios explicit targets and verifies independent
fresh URL setup without relaxing their assertions. Corrected server
shard 12/12 passes on final head. The unfinished earlier run was
cancelled after the follow-up push.
- Initial current-head E2E shard 3/8 failed a 5000 ms task-heading wait
at `tests/e2e/cloud-auth.spec.ts:46` (40 passed, one failed). Its
screenshot/accessibility snapshot contains the expected heading. The
single authorized no-code rerun passed 41/41 with zero test retries,
including all URL, handoff, and login-form checks. The first-heading
assertion took 3774 ms on rerun. Both attempts tested the same merge
commit for this PR head. These observations support a timing concern;
the original cause remains unknown. The unchanged Cloud-auth path does
not invoke archived-connection setup. Recovery is established; an
infrastructure cause is not. If it recurs, capture an instrumented
first-navigation trace and compare an unchanged baseline under
comparable conditions.
- PR remains draft. Merge and deployment are outside this request.
Synthetic provider/browser fixtures do not prove live GitHub
authorization or a deployed fix.

## Risks

- Fresh setup after personal archival now creates a new ID and a
suffixed name, including for the same owner. Clients must supply an
explicit target to reconnect the old identity.
- The guard applies to all personal gallery connections, not only
GitHub.
- No schema migration is required. Explicit owner checks remain in
place.
- Security review approved the unchanged production implementation.
Merge and deployment are outside this change request.

## Model Used

OpenAI GPT-6 through Codex. The runtime does not expose a more specific
model snapshot or context window. Used reasoning, code editing, and
shell tools.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-03 17:51:29 -08:00
DottaandPaperclip 58308a3e63 perf(apps): scope sidebar review counts to one connection (#18688)
## Thinking Path

> - Paperclip helps people manage AI agents and their governed app
access.
> - The Apps sidebar displays a Review badge for one connection.
> - It polls the full company attention service every 30 seconds to get
two counts.
> - That service also loads every profile, rule and active catalog
entry.
> - This pull request adds an authorized connection read for only those
counts.
> - The same badge now avoids unrelated profile evaluation and database
result decoding.

## Linked Issues or Issue Description

**Current behavior**

The contextual sidebar polls company-wide attention. It needs only one
connection's signed pending action count and whether any catalog entries
are quarantined.

**Proposed behavior**

Both sidebar variants poll a connection-scoped count endpoint with
existing connection read authority. The full attention API stays
unchanged.

**Reason and benefit**

The retained 200-profile fixture performs seven service queries and
returns 80,115 rows for this badge. The new reader performs one query
and returns one row. This reduces local service CPU and avoids
unnecessary decoded profile/rule data. Related merged evaluator fixes
are already in the baseline: Refs #18673, Refs #18681. A search found no
existing PR for this scoped count reader. ROADMAP.md contains no
duplicate planned work.

## What Changed

- Add the narrow shared DTO, typed client, guarded REST/OpenAPI route
and catalog-owned reader.
- Count independent aggregates with both company boundaries and exact
signed-argument semantics. Keep archived counts at zero.
- Update both contextual sidebar variants. Keep the exact badge, 30
second interval and existing attention-prefix invalidation.
- Add PostgreSQL differential tests, authenticated HTTP tests, sidebar
tests, a repeatable benchmark and access documentation.

## Verification

- Baseline: `3f8ebfff8aa9febd0733c6f65b07485dcadfd0e2`, including both
merged evaluator fixes. Service treatment: `0827f7d0b3`; route/fixture
refinement: `230c42ee15`; final local existence gate: `390a218254`.
- Passed: focused PostgreSQL differential test; assembled HTTP test with
board session and agent JWT; 14 sidebar component tests; token gates;
map checks; diff check.
- Fixture: 200 profiles, 76,000 rules, 1,900 active tools, 695 removed
references, 12 agents, a sibling connection and 2,000 unrelated
historical invocations. Complete normalized two-count results are equal
on both paths.
- Quiet local measurement: seven alternating warm pairs after two warmup
pairs, Node 24.22.2, macOS, real disposable PostgreSQL. Legacy/new
service CPU median: 213.891/1.680 ms; ranges:
209.329–228.552/0.515–6.137 ms. Wall median: 156.867/1.248 ms; ranges:
151.739–162.289/1.115–1.386 ms. Queries: 7/1. Returned rows: 80,115/1.
- Run `cd server && node --expose-gc --import
./node_modules/tsx/dist/loader.mjs
scripts/benchmark-connection-review-counts.ts`. See
`doc/connection-review-counts.md` for scope and failure history.
Byte-volume accounting runs outside timing; no HTTP payload reduction is
claimed.
- Initial raw scalar SQL failed with ambiguous columns. Typed subqueries
fixed it and passed real PostgreSQL equality. Initial fixture agents
lacked permissions and correctly received 403. The final fixture grants
explicit authority and tests revocation. An initial benchmark included
byte serialization in timing; the reported run excludes it.
- Passed required local `pnpm -r typecheck` and `pnpm build`. The
initial sandbox typecheck could not open the tsx IPC socket; the
escalated rerun passed. Full monolithic `pnpm test:run` was not run
locally: repeating a 15–25 minute suite serially across both performance
PRs adds no distinct evidence. Broad suite acceptance uses final-head
CI, which runs the same general/serialized test partitions and the
native Runner lane. Final-head CI is in progress.
- Passed built-static Chromium journey on an owned local-trusted server
and disposable PostgreSQL: Review 3 appears, only scoped counts are
requested by this sidebar, resolving requests/clearing quarantines then
reloading removes the badge, and archived counts are zero. The first ad
hoc launcher omitted required decision-service options and failed at
startup; using the normal no-op wake callback fixed that harness.
- Greptile identified an invalid resolved-request fixture label.
Corrected `denied` to the declared `rejected`, then reran the real
PostgreSQL differential test. The count SQL and non-pending exclusion
are unchanged.
- The first final-head CI failed the static cross-tenant
existence-oracle guard because the new explicit resource company
assertion had no nearby `hasCompanyAccess` gate. Existing management
admission already rejects foreign resources with 404. The route now also
follows the standard local two-step gate. The unchanged guard plus
authenticated HTTP negative cases passed (3 tests). No allowlist or test
was weakened.
- These local service measurements do not prove hosted performance. No
deployment is included.

## Risks

Low risk. New read admission reuses existing current-authority checks.
The old attention API is unchanged. Counts still require database work
over relevant pending/catalog rows. No cache, schema migration or
telemetry is added. Roll back this PR if the badge differs or scoped
reads deny previously permitted users.

## Model Used

OpenAI `gpt-6.1-sol` (Sol 6.1), with reasoning, tool use and code
execution. The host does not expose the configured context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-02 17:12:22 -06:00
DottaandPaperclip eb12cfc96b perf(github): activate connection work from events and deadlines (#18395)
Reviewed and verified at f77620b9a4.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-31 15:38:43 -05:00
DottaandPaperclip 100b364264 Preserve selected Google services when setup resumes (#18048)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Connections control which external services an agent can use.
> - Google Workspace setup lets a person choose services and read or
write access.
> - A draft lost that choice after a declined or expired OAuth handoff.
> - Finish setup could then show all services with write access instead
of the saved narrow choice.
> - This pull request saves the exact draft choice without treating it
as granted access.
> - The benefit is safe setup recovery that preserves the person's
choice and existing account limits.

## Linked Issues or Issue Description

Refs #15545. The duplicate search found no open PR for Workspace draft
consent recovery. Related #18000 concerns satisfied gateway access
requests, not saved OAuth setup intent.

**What happened?**

A cold Workspace draft resumed with the new-account service defaults.
The selected request lived only in short-lived OAuth state. Denial
consumed that state. The picker ignored draft connections.

**Expected behavior**

Finish setup restores the same person's saved service choice. It does
not use another person's choice, a broad profile hint, or a credential
grant as draft intent. An old draft with no reliable choice asks the
person to choose services.

**Steps to reproduce**

1. Start a new Google Workspace connection. Select Gmail read-only. Turn
off Drive and Calendar.
2. Continue to Google, then decline authorization.
3. Close the setup page. Return through Finish setup.
4. Open the service picker. Before this fix, all three services can be
enabled with write access.

**Paperclip version or commit**

Reproduced in tests on master
`820c458465fa40137c81fe510f3e6dc573b4f895`.

**Deployment mode**

Authenticated and local source builds. The source regression uses real
PostgreSQL fixtures and the production React setup components. Live
provider acceptance of this repair is still pending deployment.

## What Changed

- Save validated, non-secret draft intent before managed or
customer-owned OAuth. Bind it to company, connection, actor, grant kind,
and subject.
- Restore that intent after denial, state expiry, and cold Finish setup.
Keep active reconnect bound to its saved grant. Do not change
new-account defaults.
- Preserve agent-to-human setup handoffs. Consume only the addressed
person's reviewed proposal. Keep other people's proposals.
- Lock and merge the current draft before saving intent. Return 409 if
its credential identity or method changed during setup.
- Preserve the latest saved choice across overlapping customer-owned
starts. Merge both current configuration records under short locks.
Reject stale client-binding changes. Keep each OAuth state's own
requested scopes.
- Add shared, service, handoff, picker, and cold-page regression tests.
Update the connector contract and existing feature plan.

- Route the default ungrouped test/release paths through six sequential
exact-cover chat partitions. Preserve remaining server/native coverage,
explicit grouped CI, selection-first execution and shared-database
paths.
- Correct the retention test driver so pending owned real I/O does not
advance logical deadlines. Pump zero-duration connection timers and add
a held-I/O regression; keep production scheduling and original timing
assertions unchanged.

The shared helper is needed by the server and picker to validate the
same narrow binding. It does not resolve credentials or grant access. No
production schema migration, new route, provider scope or deployment
configuration is added. The test-harness changes do not change
production timers.

## Verification

### Current candidate: source qualification passed; live acceptance
pending

- Current published candidate:
`82d13d4d698a5579378b5d506cb7a1f9cb64b5c8`, tree
`78d6a821ce53f481fe3e9991edce56f0acffb22f`. It combines the Workspace
repair with launcher commit `bf29c9` and retention-test commit `e25987`,
then integrates master `493e229d93ba7798e80152f4549e2e9177ce50ae`. The
integration brings 40 upstream commits across 713 files. One launcher
conflict keeps upstream validated CI selection and its selection-first
path, then applies six default chat partitions only without a selection.
The authored PR diff has 16 files, 833 additions and 100 deletions; it
adds no authored lockfile or `.github` changes. The public head is
confirmed at this revision. Fresh CI and review passed as recorded
below. Source qualification is complete using exact-tree full CI and the
local results below. The separate local whole command was interrupted by
host disk protection and is not called a pass. Live repaired-draft
acceptance remains pending deployment.
- Exact clean `82d13d4` passed one Node 24.19.0 builtin-only run: 41/41
cases (37 launcher and four CI-workflow cases), zero failures/skips.
HEAD, tree and relevant file hashes stayed unchanged and clean before,
at start and after. This proves launcher fixtures, not real chat
execution, current typecheck/build, the dependency-backed selection
graph, or full feature acceptance.
- The new default ungrouped whole command uses six sequential
exact-cover chat partitions, then the remaining server suites,
workspaces and serialized suites: 28 expected invocations. It preserves
all native suites, including upstream usage coverage. The plain
ungrouped release launch path also changes. Explicit grouped CI,
validated selection and shared-database execution stay unchanged.
Launcher hard RED had three expected body failures and two unchanged
passes; the repaired focused five and final 37 passed. A later 41-case
staged-merge pass is retained separately; the clean 82 run above is the
frozen proof.
- Fresh [CI run
41153113944](https://github.com/paperclipai/paperclip/actions/runs/41153113944)
completed successfully on October 31 at 01:14:53 UTC. Current head has
52 successful check runs, four conditional skips and no pending/failing
checks; legacy Snyk also passed. The skips are test-file selection, the
optional paired benchmark and two Storybook checks. The full
server/chat/workspace/serialized matrix, all eight e2e shards, normal
build, build-gap typecheck, Runner checks and final aggregate gates
passed. Build and all three chat job logs identify checkout
`e5e831fd3a8534d0d0f1e892118c0f8fbe12dd92`, merging this head into
`493e229d93`. Its Git tree is exactly
`78d6a821ce53f481fe3e9991edce56f0acffb22f`, identical to the clean
candidate. The six real chat partitions passed 314, 358, 273, 239, 321
and 311 cases: 1,816 total. Other cases shown as skipped in each
partition were selection exclusions, not extra passed cases. This is
current tree-equivalent CI proof, not a pass of the separate default
ungrouped local whole command.
- Fresh [Greptile
summary](https://github.com/paperclipai/paperclip/pull/18048#issuecomment-6473856992),
edited October 31 at 01:05:04 UTC, explicitly reviews exact head
`82d13d4` at 5/5 with no actionable code issue. Its exact-head check
passed. The prior overlapping-start discussion is resolved. This review
is source-only, not provider or live acceptance.
- Exact clean `82d13d4` passed full recursive typecheck and normal `pnpm
build` on the independent darwin/arm64 snapshot. Typecheck finished on
October 31 at 01:27:33 UTC; build finished at 01:28:33 UTC. Both exited
0. Normal native preparation and code signing passed. Source HEAD, tree
and tracked/untracked state stayed unchanged and clean. `pnpm
check:token-gates` also passed all token, palette and ClipLab checks on
this candidate. The earlier forced-clone `ENOSYS` preflight remains
recorded as setup failure, not an assertion result.
- The unchanged default `pnpm test:run` started at 01:51:33.796 UTC on
clean exact `82d13d4`. All six real chat partitions passed: 358, 314,
273, 239, 311 and 321 cases, or 1,816 actual cases. The original FIFO
and ambiguous-retry cases both passed under their unchanged 15-second
limits. General-server then reported 37 completed file rows with 5,438
passed cases and five default remote-continuity skips, including all 498
tool-access service cases and all 83 connection-route cases. At 02:39:44
UTC, the unchanged 5-GiB host-disk guard terminated the owned run. The
child ended with SIGTERM and the container/harness exited 143, with no
OOM or recorded assertion failure. Only seven of 28 RUN invocations
started. Retention had no completed local row. Later invocations did not
run. This whole local command is interrupted/incomplete, not passed.
- A coverage check found no default suite missing from current full PR
CI. The maintained launcher tests prove exact coverage for all 12 PR
server shards, chat and five native suites; all configured workspace
projects; and all nine serialized shards. The final Runner CI shard
executes the five native server suites. The 41 local launcher/workflow
cases cover six sequential chat children, per-child isolation, native
retention, fail-fast propagation and unchanged explicit
selection/shared-database paths. The actual six-chat run above exercises
that changed default path. The remaining unqualified local difference is
uninterrupted sequential whole-command composition and resource
accumulation, not omitted test selection. Current exact-tree full CI
supplies broad source-test qualification; no local whole-pass claim or
provider acceptance is inferred.
- A task-owned remote runner was considered to complete that separate
composition check. Its provider rejected the requested allocation with
per-sandbox limits of 8 GiB RAM and 10 GiB disk. No sandbox was admitted
and no remote tests ran. No tests, timeouts, assertions, database
synchronization, source or provider settings were changed to avoid the
capacity limit. Historical a0 failures remain below. No deployment or
live repaired-draft acceptance has run.

### Bounded follow-up on the prior source

- On a0, one finite phase-traced chat diagnostic executed exactly the
two failed cases. Both passed (1,374 and 1,767 ms); 1,814 other cases
were filtered, not passed. This did not reproduce a blocked await or
establish the original monolithic timeout cause. No assertion or
15-second limit changed.
- The retention diagnosis preserved an initial zero-body selector
failure, then corrected only the selector: one original case passed in
783 ms, with 42 filtered. Trace showed the old settling helper advancing
logical time while real I/O was active. A new held-I/O regression then
failed hard because the old helper advanced 1,000 ms. This proves a
test-driver defect, not the precise cause of the old 5,340 ms full-run
gap.
- The first manual-clock repair failed the complete 44-case file: 43
passed, one body hit 15 seconds and cleanup hit 30 seconds. Its planned
mutant did not run. Source then identified a fresh PostgreSQL
connection's zero-duration timer. The corrected driver pumps
zero-duration timers while initial readiness or tracked I/O is pending,
without advancing logical Date. That pre-integration file passed 44/44
with no skips. A separate scratch-only 5,000 ms normal-continuation
mutant failed the original hard `5000 < 3000` assertion; its expected
failure is not a test pass. The original page-count, fairness, expiry
and quiet assertions stayed intact. The authored repair does not change
the production worker. The merged master changes an older retention case
and dependencies; these pre-integration results are not a complete 82
retention or whole-suite pass.

### Historical qualification: SHA-bound, not 82 acceptance

- Historical a0 candidate: `a0f9089fc1ff466bf4e1eeddc5b1a910d09e380b`,
tree `b9d73713319dcd52b9eaa6053008b5190e43fdf1`. It adds the
overlapping-start repair to parent `6c18`, which already merged pinned
master `1511c92965325249ed6b52dd5be52eb6d736fbfa`. That candidate
changed 12 files. Its CI and review passed; its local full qualification
remained failed/incomplete.
- Greptile's previous `6c18` review found P1 [Latest service choice gets
lost](https://github.com/paperclipai/paperclip/pull/18048#discussion_r4425965828),
confidence 4/5. Two hard-assert regressions reproduced it on untouched
`6c18`: the already-bound path overwrote the later choice, and real
first-use client stamping also restored the old choice. Existing OAuth
states/grants were not widened, but a later setup could restore an older
request. The new repair covers both write boundaries without holding
locks over external work or copying another attempt's scopes.
- The complete affected service file passed 498 tests with no failures
or skips. All 31 required controls ran, including both unchanged RED
cases, 14 new client-drift cases, seven earlier identity guards,
reconnect/cold-resume cases and upstream transaction tests. The tested
three-file patch is byte-identical to this commit; the connector
contract update is separate. Independent scoped review found no
additional actionable issue. Fresh Greptile reviewed exact `a0f` at 5/5
with no actionable findings. The previous P1 thread is resolved.
- On exact `a0f`, the five shared/UI files passed 312 tests. An initial
complete connection-management run then failed: 74 passed, eight timed
out at the unchanged 15-second limit, and one AI-account verification
returned 422 instead of 201. The Google addressed-user handoff passed.
The container exited 1 without OOM and stopped before chat/token/whole
tests. This material failure remains retained and its cause
unclassified; the later controlled recovery below is not a causal
explanation.
- One diagnostic executed the first failed gateway case and the two AI
cases with temporary phase-only tracing. All three passed at 8.516,
5.511 and 3.523 seconds; 80 other cases were filtered. Original hard
assertions and timeouts stayed unchanged. No timeout or suspected late
provider cleanup reproduced, so the complete-file failure remains
unresolved. Vitest exited 0; the harness exited 1 because its trace
parser missed ANSI prefixes. All 203 events were recovered from retained
logs without rerunning tests. Instrumentation was removed. This is not
acceptance.
- Historical a0 CI
[41097596364](https://github.com/paperclipai/paperclip/actions/runs/41097596364)
completed successfully: 52 successful checks, two expected Storybook
skips and no failed/pending checks. Legacy Snyk passed. All 48 CI jobs
passed, including all 1,816 chat cases and explicit passing lines for
the original 20 local failures. [Server
11/12](https://github.com/paperclipai/paperclip/actions/runs/41097596364/job/124207469276)
explicitly passed the complete 83-case connection file in 129.49
seconds, including all nine local failure titles. The actual checkout
was synthetic merge `06851da6eb969d63e1c7dc9f7c95f08d09fa4ff1`,
combining `a0f` with `aa58fa107389e8b9ae5949f7ad815943dfd94292`. This is
not standalone local `a0f` qualification and does not erase the nine
local route failures.
- Linux admission uses Node 24.22.2, pnpm 9.15.4 and Rust 1.97.1. Frozen
offline install, exact native hashes and real migrated PostgreSQL 18.1
passed. Native/protocol/build/lock inputs are identical between `6c18`
and `a0f`; validated cache reuse retains the actual `6c18` native-build
provenance. The selected packaged release binary is not called a fresh
`a0f` build. Database synchronization settings stayed on. This admission
alone does not explain the route failures.
- A fresh Mac snapshot of exact `a0f` passed frozen install, workspace
prerequisites, full sequential recursive typecheck and normal `pnpm
build`, both on their first attempt. Typecheck completed at 17:43:05
UTC, build at 17:44:43 UTC; both exited 0 with clean tracked/untracked
source. Pinned Node 24.19, pnpm 9.15.4, Rust 1.97.1, package concurrency
one, Cargo jobs one and a 12 GiB V8 heap ceiling were retained.
Largest-process maximum RSS was 7,668,154,368 bytes for typecheck and
10,038,837,248 bytes for build, not aggregate tree peaks. The first
build correctly withheld its freshness stamp because 125 ignored, pinned
Copilot assets appeared during preparation; exact path/hash analysis
explained this. The normal build then recorded a genuine stamp and
reused Cargo outputs in 1.18 seconds. No manual stamp, source workaround
or gate retry. This is darwin/arm64 static/build proof, not Linux
packaging/runtime or full tests.
- The historical a0 candidate's token checker passed all four gates
across 1,410 files, run directly with pinned Node 24 against the script
behind `pnpm check:token-gates`.
- The single controlled complete-file comparison passed: baseline `6c18`
83/83 in 287.07 seconds, then candidate `a0f` 83/83 in 293.03 seconds,
with zero skips. All nine earlier failure titles passed on both. Both
child/container exits were 0, no OOM/interruption; source and
native/resolver/PostgreSQL guards matched. This ran only after all owned
Mac compilation ended, with fresh processes/databases and identical
four-CPU-quota/6-GiB/overlay inputs. No filtering, instrumentation,
assertion/timeout/source changes or retry loop. This establishes
standalone complete-file recovery, not the cause of the earlier eight
timeouts/AI 422.
- The first unchanged full `pnpm test:run` on exact clean `a0f` ran from
October 30 at 18:01:43 UTC to 21:06:19 UTC. It failed naturally with
test and container exit 1, without OOM. General-server reported 1,257
passed, one failed and seven skipped files; 28,633 passed, one failed
and 103 skipped tests. Only the first of 22 invocations ran. The later
four groups and 21 invocations did not run. The sole failure was the
unchanged chat-shard CLI meta-test: its nested projectless pnpm lookup
hit the offline network guard. This was a failed full gate, not Google
provider acceptance.
- A finite environment pair reproduced that one meta-test failure, then
passed the same unchanged case after supported `corepack prepare
pnpm@9.15.4 --activate` as the existing non-root node user. The cached
pnpm payload was identical. Only the missing node-user last-known-good
record changed. Core source, dependencies, lockfile, fixtures,
assertions, timers and offline guards stayed unchanged. This isolates
the nested collection defect; it does not explain the earlier route
failures or complete the full gate.
- The corrected unchanged full command ran from October 30 at
21:29:03.500 UTC to October 31 at 00:02:44.207 UTC on the same exact
clean `a0f`. It failed naturally with exit 1, signal null and no OOM.
Source stayed clean before, at start and after. It retained two visible
CPUs/quota, 6 GiB, private IPC, normal overlay, matching native inputs,
synchronized real PostgreSQL 18.1, and the default five groups/22
invocations. Only the first general-server invocation ran; the other 21
did not run. General-server reported 1,265 files: 1,256 passed, two
failed and seven skipped. It reported 28,737 tests: 28,631 passed, three
failed and 103 skipped. The two chat cases emitted `Test timed out in
15000ms` at `chat-channels.integration.test.ts:77310` and `:81790`, with
durations of 15,045 and 15,058 ms. These were the durable pre-run FIFO
notice and ambiguous GitHub text restart/retry cases. The retention case
emitted `expected 5340 to be less than 3000` at
`skill-version-retention.test.ts:593`. Its real duration was 1,727 ms;
5,340 ms was the recorded fake-time directory-page gap, not a deadline
timeout. The timeout stacks do not identify the blocked internal await.
The cause of these failures remains unclassified. Inside this same
failed run, the complete 83-case connection file and 498-case service
file passed. The former Corepack meta-case also passed in 6,396 ms.
These component passes do not waive the failed/incomplete full gate. No
rerun, cancellation, timeout/assertion change or full-pass claim. A
later fresh two-case phase trace passed both chat cases; it did not
reproduce or establish the original timeout cause. At that checkpoint,
the PR stayed draft.
- Prior `6c18` targeted checks passed 930 tests across eight files and
token gates. Its selected queue group passed all 53 cases, including the
original 20 failures and 12 new named controls. Backlog cases took 10.49
and 8.45 seconds under the unchanged 15-second limit. Another 1,763 chat
cases were intentionally filtered. This is historical targeted recovery,
not a full pass or proof of the new repair. No timeout or assertion was
relaxed.
- Prior `6c18` full local qualification failed during serial typecheck:
server TypeScript was killed with code 137. Docker confirmed OOM in the
6 GiB container; full tests and build never started. The failure is
retained. The new Mac resource plan avoids repeating that undersized run
and does not change the shared Docker VM.
- Prior CI
[41089793558](https://github.com/paperclipai/paperclip/actions/runs/41089793558)
completed successfully for synthetic merge
`a73bffcfa3ede051961bcabcfa576c703d10c932` (`6c18` plus
`abc270ee4c03e13b4d6eaad156a41b4ebed4bb0a`). All 48 CI jobs passed; all
1,816 chat cases passed, including explicit log lines for the original
20. Check runs were 51 success, two expected skips and one Greptile
failure; legacy Snyk passed. These results do not qualify `a0f` or erase
its local route failures.
- Red proof on the unmodified master baseline: cold picker and Finish
setup restore broad defaults; managed and customer-owned drafts lack
saved intent. The stale setup tests also fail before the lock guard.
- Live staging baseline `b25f3e2af8e77d882cd22d36ca55640e6468e46f` also
reproduces the ordinary failure: select Gmail read-only only, cancel at
Google, reload the connector catalog, then Finish setup. All three
groups return enabled at read/write. Authorization was declined; no new
Google access was granted. This is baseline failure evidence, not a
deployed-fix pass.
- Historical affected suites: 800 tests passed before the final lock
guard. The final focused service and handoff cases passed 126 tests
after that guard. These are not results for the integrated candidate.
- Historical `pnpm -r typecheck` and final lock-guard server typecheck
passed. The historical a0 candidate passed the full recursive gate
separately, as recorded above.
- `pnpm build`: passed again on the final lock-guard source in commit
`1c31cde0ed199fbf4e41b7649590554f7886dfc2`.
- Historical `pnpm check:token-gates` passed; the historical a0
candidate's direct checker pass is recorded above.
- `pnpm test:run` on macOS failed on the frozen commit. It exited 1
after 9,432.41 seconds. General server results: 1,195 files passed, 10
failed, eight skipped; 26,706 tests passed, one timed out, 157 skipped.
Nine setup hooks timed out while starting embedded PostgreSQL. The
retained log records host-wide semaphore exhaustion (`semget`), not disk
exhaustion. One unchanged publication-text performance case also timed
out. Its cause is not established. Later workspace and serialized groups
did not run. This is a failed and incomplete local gate.
- The independent full-source Linux run also failed and was incomplete
on the same frozen commit. Native and real-database admission passed.
Chat integration reported 20 failures in 1,593 tests. Tool-access passed
all 479 tests. Native-session-executor passed all 965 tests. Docker
interrupted the run after about 48 minutes on October 29. The container
exited 143 and was not OOM-killed. The run produced no final group
totals or detailed assertion summary. Shell trap metadata reports zero,
but this is not a valid test-pass result. Source and evidence remain
unchanged. The focused diagnosis below retains those failures; their
cause is not assumed from the interruption. Neither historical local
full run passed.
- A fresh focused replay on the frozen source then executed exactly
those 20 case bodies. All 20 failed; this was not a setup-hook failure.
The two backlog cases reached the unchanged 15-second timeout. The 18
FIFO cases failed on missing notices or receipt rows. All 18 FIFO cases
passed when run alone in a fresh process and database. This supports an
order interaction, not a complete cause or baseline conclusion. A
separate two-backlog control used RAM-backed temporary storage and real
PostgreSQL with synchronization settings on. One backlog case passed in
10.65 seconds; the other still timed out at 15 seconds. No source,
assertion or timeout was changed. The full local gate remains open.
- Earlier mixed-revision run: it began before the final guard and
imported older source before collecting later regression cases. It
failed those new cases with that older cached module. This run is
retained, not counted as candidate acceptance. The fresh focused run
passed all seven stale-setup cases. No case was skipped to hide a
failure.
- Historical remote gates for `1c31` were terminal and clean: 53
successful contexts and two expected Storybook skips. Main CI
[40720536514](https://github.com/paperclipai/paperclip/actions/runs/40720536514)
passed. Greptile reviewed exact head
`1c31cde0ed199fbf4e41b7649590554f7886dfc2` at 5/5 with no actionable
findings; the separate review job also passed, and there were no inline
review comments. These results do not qualify `6c18`.
- CI source qualification: the chat jobs checked out synthetic merge
`ef36746b3d2ec52ba5727dcbe4da265a03421fe4`, which merges the PR into
`7d75ae0569e41786640cf09647ae8849411d2f22`. All 20 locally failing case
names have explicit passing CI log lines. These passes do not establish
standalone local acceptance of `1c31`. The merge adds two upstream Teams
tests, so it collects 1,595 chat cases instead of 1,593. Neither
backlog/FIFO assertions nor their timeouts changed.
- Independent source review found an earlier credential-identity race.
The guard and seven deterministic cases resolve that finding. A later
integration review missed the separate overlapping-start race. Both RED
failures and that review miss are retained. The narrow follow-up repair
passed 498 service cases, independent source review and fresh remote
CI/5-of-5 review; full local qualification remains incomplete.
- Source qualification is ready for merge. Live repaired-draft recovery,
delivery and Google approval are not established by these results. The
local whole command remains explicitly incomplete.

## Risks

- Historical drafts without a matching intent now need an explicit
service choice. This is deliberate and prevents silent broadening.
- A concurrent credential or method edit now returns 409. Reloading lets
the person review the changed setup.
- Overlapping starts keep separate requested scope sets. Updating draft
intent does not relabel an earlier OAuth state's consent request.
- Saved intent is not consent. Provider-reported scopes and current
action policies still control discovery and dispatch.
- No migration is needed. No existing token or grant is rewritten by
saving draft intent.

- Default ungrouped launch paths now execute six chat children in
sequence and 28 whole-command invocations. Exact coverage, local
launcher contracts and all six real default chat partitions passed. A
complete uninterrupted local whole-command run remains unqualified
because of host capacity.
- The retention test-clock repair depends on owned-I/O and zero-duration
connection-timer progress. The controlled regression, sensitivity mutant
and current exact-tree CI protect the original assertion. It did not
reach a completed row in the interrupted local whole run.

## Model Used

OpenAI Codex, described by the runtime as GPT-5 based. The exact runtime
model ID and context-window size are not exposed to this task. Used
reasoning, code execution, browser verification, and independent
delegated review.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used, including the runtime metadata
limit
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have described the issue in-PR following the bug report template
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
ticket or instance identifier
- [x] I have run tests locally and they pass — exact clean 82 passed 41
launcher/workflow cases and all 1,816 real default chat cases, full
recursive typecheck, normal build and token gates. Full exact-tree PR CI
passed. The separate local whole `pnpm test:run` was interrupted by disk
protection and remains incomplete; it is not claimed as a local whole
pass.
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented risks above
- [x] All Paperclip CI gates are green — current exact-head checks
passed; the verified CI checkout tree equals clean 82. The separate
interrupted local whole command is disclosed above.
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups —
fresh exact-82 summary and check passed; the earlier finding is
resolved. This is source review, not acceptance.
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-30 22:02:58 -05:00
DottaandPaperclip 5f0da562db fix(server): wake chat lifecycle milestones from committed changes (#17606)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Connected chats show safe notices when a run starts, waits,
finishes, or fails.
> - A recurring scan projected these notices even when no eligible work
existed.
> - Eligibility depends on committed run state, conversation lineage,
and current authority.
> - Existing database rows can reconstruct that work after restart.
> - This PR wakes a scoped worker from those writes and removes
lifecycle polling.
> - Native progress events keep their separate existing owner until the
next migration.

## Linked Issues or Issue Description

**What happened?**

The chat maintenance timer repeatedly scanned run lifecycle state. Empty
and settled instances still queried the database. A run-status event
alone cannot replace the scan: prompt receipts, continuation receipts,
identity changes, and restored authority can make a run eligible later.

**What did you expect?**

Committed changes wake projection. Startup reconstructs eligible work. A
completed pass stops until another relevant change arrives. Failed work
retains recovery without scanning settled history forever.

**Steps to reproduce**

Configure an automatic chat endpoint. Finish its work, or leave it
unused. Observe recurring lifecycle projection queries from the chat
maintenance callback.

Stacked on #17592. Related foundation: #15625. This PR owns lifecycle
milestones only.

## What Changed

- Preserve upstream coalesced Cloud membership writes together with
locked membership restoration and its committed chat notification.

- Signal milestone eligibility from the real Slack account-link
transaction.

- Register the lifecycle worker with the existing delivery coordinator.
Remove lifecycle projection from the periodic chat callback. Keep native
progress in a separately named lane and preserve the explicit combined
API.
- Reconstruct company/issue scopes from the existing selector. Drain
bounded batches with company fairness. Retry failures, retain writes
through drain/resume, and disarm settled work.
- Signal from supported run, presentation, interaction, receipt,
lineage, conversation, recovery, and authority writers. Preserve their
transaction ownership, including nested savepoints and uncertain
settlement. No new table is added.
- Preserve provider authorization, authored-final suppression,
responsible-user continuity, deduplication, the 24-hour selection
window, and raw-output restrictions. Keep provider interpretation in its
owner.
- Preserve PostgreSQL timestamp precision in lifecycle pagination.
Rejected submillisecond candidates must not repeat the same page and
starve later notices.
- Document producer coverage and alternate-writer dispositions in
`doc/plans/2026-10-25-chat-run-milestone-sleep-audit.md`.

## Verification

Published head: `1d056f68c5b1005933b9e12de8745bf1c4cbeeca`, targeting
master after #17592 merged at
`7d75ae0569e41786640cf09647ae8849411d2f22`. A normal merge integrates
the actual landed parent. The entire 77-path milestone patch is
binary-identical to the locally verified preparation across that
ancestry reconciliation; both unrelated master paths are preserved. The
four earlier producer/test integration conflicts retain scoped milestone
and voice notices, current onboarding completion admission, and expanded
identity query/read-scope coverage.

- **330 distinct selected cases pass on the verified unchanged patch:**
305 across 14 complete suites (all five milestone worker suites, run
identity, board claim, Cloud provisioning, auth session routes, Cloud
actor authentication, stale-lock recovery, GitHub credentials, project
Git authorization, and app admission wiring), plus 25 selected real chat
producer cases. The remaining 1,590 chat integration cases were skipped,
not passed.
- The real producer cases cover saved milestone discovery, GitHub and
Slack identity changes, account-link rollback/reconfirmation, ordinary
ingress reactivation, post-close action/delivery receipt settlement, all
ten authority-restoration paths including Cloud synchronization, and
changed native review metadata.
- Worker coverage verifies outer commit, rollback, uncertain-write
recovery, restart, bounded batching/fairness, timestamp precision,
retries, drain/resume, and settled SQL quiet. Authorization and existing
publication deduplication remain with their owning services.
- Server production typechecking, build, and module-boundary checks
pass. The tests above were executed separately. Production typechecking
excludes `src/__tests__` and does not replace those runs. Each database
owner uses its own newly created and app-migrated PostgreSQL fixture.
- [Current-parent acceptance and preserved failure
ledger](https://github.com/paperclipai/paperclip/blob/1d056f68c5b1005933b9e12de8745bf1c4cbeeca/doc/plans/2026-10-25-chat-run-milestone-sleep-audit.md#current-parent-integration-and-acceptance-october-28)
records commands, counts, producer scope, conflict dispositions, and
limits. Earlier combined-stack counts and failures keep their original
revision scope; local acceptance does not erase old CI failures.
- No new migration, queue table, dependency, workflow, or shared
scheduler contract is introduced. Parent migration 0397, bounded consent
recovery, voice cleanup/retention repairs, and runner checkpoint
restoration remain present.
- **Exact-head Greptile is 5/5 and all 55 current checks are clean.** No
Ponytail review is present, which is an explicitly authorized
nonblocker; no failing review is waived. Older reviews and parent checks
are not current passes. Actual failing Ponytail findings must be fixed;
an absent/stopped/paused turn without a verdict is an authorized
nonblocker. Active unfinished Ponytail uses its original request/start
timer, with a 25-minute monitor cutoff respecting the user's 30-minute
maximum.
- No complete local repository matrix, live provider tenant, external
alarm, physical host/database suspension, or fleet savings claim is
made. Native progress remains a separate following slice.

## Risks

- A future eligibility writer must notify the existing projection
boundary. Independent SQL writers or another active server process are
outside this process-local signal contract.
- Authority and lineage remain sensitive. The worker uses the existing
selectors and publication policy; it does not grant execution or
disclosure authority.
- Native progress and other publication maintenance still have recurring
work. This PR removes one reason, not the entire chat maintenance timer.
- Worker quiet does not prove database suspension, host sleep, or
external alarm coverage. Cloud sleep blockers remain unchanged.
- Merge eligibility requires passing current-head CI and Greptile 5/5
after this actual-master reconciliation. Subsequent stack slices retain
their own gates.

## Model Used

OpenAI Codex, GPT-6 family. The exact serving model ID and
context-window size are not exposed. Used repository inspection,
reasoning, shell execution, PostgreSQL tests, browser E2Es and GitHub
tools.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details available in this session)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run the documented focused tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 11:07:54 -05:00
DottaandPaperclip e93732c3d4 fix(server): deliver voice replies from commits and saved deadlines (#17578)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Phone calls receive approved answers from the existing chat
publication queue.
> - The chat maintenance timer checks for voice replies every second,
including when no reply is ready.
> - A reply can be saved before its publication commits. An early wake
alone can miss that final transition.
> - Existing session, reply, publication, and action rows already record
delivery and recovery work.
> - This pull request uses committed changes and saved deadlines to
drive phone reply delivery.
> - Empty reply queues become quiet while authorization, ordering, and
uncertain-send protection remain in place.

## Linked Issues or Issue Description

Stacked on #17576. Related delivery infrastructure: #15625. Related
voice connection work: #11418.

**What existing behavior does this improve?**

Background phone reply delivery and recovery of abandoned dispatch
intents.

**Current behavior**

Chat maintenance calls voice reply delivery every second. It also scans
old dispatch intents. Publication transport starts a direct push before
the final publication commit. The timer masks that timing gap.

**Proposed behavior**

Committed publication, reply, session, and configuration changes wake
the shared coordinator. Ready replies drain in bounded passes. Saved
rate-limit and dispatch-expiry deadlines retain recovery. An empty queue
or a permanent delivery barrier has no recurring reply scan.

**Reason and benefit**

Remove recurring SQL from idle phone reply delivery without dropping
replies, replaying uncertain sends, or adding a queue table.

**Breaking changes**

No API or schema change. Explicit manual recovery remains available.
Inbound admission cleanup, history retention, and reports remain on
their current paths for separate migrations.

## What Changed

- Extract ordered reply state and multipart policy within the voice
owner. Retain current authorization and atomic dispatch claims.
- Register a scoped reply worker with startup reconstruction, fair work
batches, failure backoff, drain/resume handling and shutdown ownership.
- Extend the existing shared dispatcher with normalized partial-page
results to preserve the bounded parent recovery contract. Existing
complete-array consumers retain their calls.
- Signal before relevant writes in their existing transactions. Both
normal publication completion and operator resolution wake the actual
conversation after commit.
- Bound reply inventory to 100 total headers per refresh. Preserve
partial scans, concurrent changes and newer denied-call restoration
obligations across reconstruction.
- Track abandoned dispatch actions independently of live calls.
Quarantine expired intents without resending them.
- Use existing active-call authorization checks to reactivate replies
that were previously denied. Healthy call inspections do not rescan
empty reply queues.
- Remove the periodic reply scan and the early direct publication push.
Keep provider requests outside database transactions.

## Verification

Published head: `1435cab10be3f95c38ab6672d2a890ae49e9d300`. Voice
recovery #17576 has landed at master
`e1ba77f06771a27114b3999004b0ca5e3c172dd4`. A normal merge reconciles
squash ancestry while preserving the complete reviewed 31-file reply
patch byte for byte and all unrelated master changes. The PR now targets
master; fresh exact-head CI and Greptile are required. The prior
449-case/typecheck/build results below cover the identical owned source,
not a new full-suite run.

- **449 distinct targeted cases pass**: 111 complete voice-store/reply;
19 complete parent recovery; and 319 complete tool-gateway,
secret-proposal, shared dispatcher/coordinator/batching/startup/hostname
cases. Production server typecheck and build pass. The policy checker
and its Node tests pass. Later edits record evidence only.
- Two real normal-publication/operator-resolution tests seed 105
unrelated calls and 105 future dispatches. Five publications selected
**1,065/1,060 inventory rows before repair**; both paths now pass the
same **125-row total budget** and deliver every answer once. This counts
selected SQL results, not physical tuple visits or host savings.
- Publication commits now signal their actual conversation. The voice
reader returns at most 100 total session/action headers per refresh,
preserves phase/cursor state and retries, and resets on worker
registration. Current-state checks and dispatch claims still own
provider effects. The 105-action case measures the page cap, restarts
the same owner during reconstruction, waits at saved future deadlines
and quarantines every abandoned action without resending.
- A real 403 fixture proves that an older broad scan cannot erase a
denied call first observed behind its cursor. Its owner callback still
emits the committed restoration wake after scan completion. The existing
actual-service authority-restoration and final-publication timing tests
remain in the full suite. They are not replaced by direct helper calls.
- Guarded removal of conversation scoping, cursor reset or the denial
revision fence fails the intended budget, 100-versus-5 restart or
missing committed-wake assertion; source is restored. An initial
shadowed fixture name and an inadequate due-action reset probe are
retained as failed attempts in the [current
audit](https://github.com/paperclipai/paperclip/blob/d2c0a7de8c3e9cac8382b1268f3f5d883a2067b3/doc/plans/2026-10-25-voice-reply-delivery-sleep-audit.md#publication-inventory-and-paging-acceptance-october-28).
- Parent recovery still passes paging, restart, behind-cursor insertion,
feature changes, actual disablement, quiet health observations, current
authority, exact-scope failure fences and wildcard-failure isolation
through the shared worker. Four existing array-based worker consumers
retain their API and pass their full suites.
- The two voice fixtures self-migrate separate empty databases. Legacy
tool-review/secrets suites use separate migrated clones. Earlier
full-local attempts remain limited by embedded PostgreSQL setup and are
not passing full-matrix claims. Older counts apply only to their stated
revisions. Fresh exact-head Greptile, Ponytail if present, security and
all CI remain merge gates. No live-provider, multi-writer,
external-wake, deployed-sleep or savings claim is made.

## Risks

- Missing a supported producer can strand work. Tests cover real
enqueue, retrieval, normal publication, and operator publication paths.
Startup reconstructs existing records.
- Signals cover the process-local database owner. Independent SQL
writers or other replicas need their own wake contract.
- Authority restoration relies on the existing live-call recovery check.
This dependency must be preserved if that owner changes later.
- A provider acceptance receipt does not prove speech. Unknown outcomes
remain blocked from automatic resend.
- This change does not extend Cloud alarms, allow sleep during live
calls, or prove deployed suspension or cost savings.

## Model Used

OpenAI GPT-6 via Codex, with repository inspection, reasoning, code
execution and test tools. The runtime does not expose a more specific
deployment identifier or context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used with the available version and
capability details
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
ticket identifier
- [x] I have run the selected tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 07:50:27 -05:00
DottaandPaperclip e1ba77f067 fix(server): schedule live voice recovery from session changes and deadlines (#17576)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Voice calls retain durable session state and provider identities.
> - The chat maintenance timer scans live voice sessions even when no
call exists.
> - Session writes, saved expiry and active-call inspection deadlines
identify real work.
> - This pull request moves live-session recovery to the shared
coordinator.
> - Idle session recovery becomes quiet without losing exact-call
cleanup or current authorization checks.

## Linked Issues or Issue Description

Refs #17572. This PR is stacked on receiver discovery. It owns only live
voice session recovery. Reply delivery, inbound admission cleanup,
inbound history retention and call reports remain separate migration
slices.

**What happened?**
The one-second chat maintenance lane reads voice settings and scans live
sessions even when an instance has no calls or only terminal history.

**What did you expect?**
Reconstruct live calls on startup. Wake on committed session or
configuration changes. Inspect active calls and retry cleanup at actual
deadlines. Stop querying when no live obligation remains.

**Steps to reproduce**
Start an instance with no live voice sessions. Let startup settle. The
old maintenance method continues to read settings and session inventory
every chat maintenance pass.

## What Changed

- Add a voice-owned bounded deadline reader and scoped worker on the
existing coordinator. Retain concurrent writes across pages and reset
partial scans on restart. No tables or migrations are added.
- Separate generic connection health observations from configuration
changes. Existing receiver and command consumers retain their
notifications. Voice eligibility and provider operations remain in the
voice owner.
- Signal existing session transactions and configuration/feature
changes. Keep provider calls outside database transactions. Standalone
session writes use one small transaction helper.
- Preserve saved expiry, the two-minute unknown-creation grace,
active-call authorization and missed-hangup inspection, and
exact-provider cleanup. Recovery never creates a provider call.
- Retry transient authority lookup failures instead of treating them as
confirmed revocation.
- Fence expiration against a provider identity bound after discovery. A
late known call remains eligible for cleanup.
- Preserve work through rollback, drain, restart, concurrent refresh,
backoff and shutdown. Page inventory and rotate bounded batches across
companies.
- Leave the four other voice maintenance duties in the periodic lane.
- Document producer coverage and the supported writer contract in
`doc/idle-voice-session-recovery.md` and the dated audit.

## Verification

Published head: `f0b2e3cc22a18e7d08b10d4fc02aae4a19a52f30`. Receiver
#17572 has landed in master at
`dce004259394486f6361d0c0dd5a33a5272acc0a`. This normal merge reconciles
squash ancestry; the complete 22-file voice recovery patch is
byte-for-byte identical to the reviewed and tested `ad79c7e805` patch,
while all unrelated master changes are preserved. The PR now targets
master. Fresh exact-head CI and Greptile are running; interrupted
Ponytail turns are not completed verdicts under the maintainer’s
clarified merge policy, and an actual failing verdict still blocks.

- **210 distinct targeted cases pass on the current repair**: 19
complete recovery lifecycle cases and 191 complete voice-store,
coordinator, batching, startup and hostname cases. Production server
typecheck and build pass. Later edits record evidence only.
- CI `40633667141` on prior head `1b6323c164` failed the feature-change
assertion with `34050 < 25000`; its verification aggregate also failed.
The test now disables automatically advancing fake time, drives 20
controlled scheduler turns and requires all 205 calls to close before an
unrelated inspection deadline can mask a lost wake. Removing generation
replay fails with missing closures. The repair does not benchmark
database/host speed or raise the elapsed-time bound.
- A real committed exact-call regression reproduces wildcard-backoff
starvation. The worker keeps the broad retry but processes independent
exact hints and cached due calls. Each check reloads current state and
authority. Failed exact scopes remain fenced. Removing wildcard
isolation fails the intended assertion; a successful broad retry then
leaves the worker SQL-quiet. Full-row and lean queries now share the
state/scope/cursor predicate.
- Earlier acceptance on `712f3862b5` passed 725 distinct targeted cases.
Its measurements remain useful history: five real metadata writes and
one health check over 205 future calls selected 1,230 rows before the
inventory repair and zero afterward, without changing the saved
deadline. Those older test counts are not added to the current 210-case
count.
- Both external voice fixtures self-migrate; the current full
voice-store run uses a separate initially empty database. Earlier
fixture/method/observer failures, source mutations and partial edit
attempts remain in the [current
audit](https://github.com/paperclipai/paperclip/blob/ad79c7e805367fc7e2df3ea53c942ca6c7d27365/doc/plans/2026-10-25-voice-maintenance-sleep-audit.md#wildcard-failure-and-deterministic-generation-acceptance-october-28).
Source is restored after guarded mutations.
- Older CI `40632057899` failed because its trusted policy script was
absent from the checkout, while build separately received a runner
shutdown signal. E2E shards succeeded but their aggregate failed on
policy. The current checkout includes the landed policy tooling and its
checker/tests pass locally. The shutdown cause is unproven; neither old
failed run is counted as passing. Earlier full-local attempts stopped on
embedded PostgreSQL setup failures and did not pass.
- Fresh exact-head Greptile, security and all CI must pass before merge.
A completed failing Ponytail verdict blocks; an interrupted turn without
a verdict does not. No full current local repository matrix, live
provider acceptance, multi-writer support, Cloud wake, host suspension
or savings is claimed.

## Risks

- Process-local hints cover the existing active writer and shared
clients. Independent SQL writers or another replica need explicit wake
integration.
- Membership, task binding, phone preferences and other authority inputs
remain checked on the existing 30-second active-call inspection deadline
and on use. This is not an instant cleanup guarantee for every authority
mutation.
- Unknown creation retains its existing maximum-duration grace. Cleanup
failures retain retries. A provider request is never replayed merely
because its creation response is unknown.
- Genuine configuration changes retain bounded reconstruction. Metadata
writes can probe parent eligibility, while health-only observations
leave voice recovery quiet. Paging state is process-local and
reconstructs from durable rows after restart.
- Active calls, the other voice maintenance owners and connector
transports remain real obligations. This does not expand Cloud alarm
coverage, relax sleep eligibility or prove deployed suspension.
- The receiver and Discord parents have landed. Current-master
compatibility and exact-head CI and Greptile remain merge gates.

## Model Used

OpenAI GPT-6 via Codex, with repository inspection, reasoning, code
execution and test tools. The runtime does not expose a more specific
deployment identifier or context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either linked existing issues or described the issue in-PR
following the relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change and contains no internal
ticket identifier
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 07:38:48 -05:00
DottaandPaperclip 94a780286f fix(server): schedule tool health from committed changes and deadlines (#17554)
## Thinking Path

> - Paperclip manages AI-agent work and its connected tools.
> - Idle hosted instances need their database and network activity to
stop.
> - The maintenance tick queries tool connections even when no health
check is due.
> - Existing connection rows already contain eligibility and the last
health observation.
> - This pull request reconstructs health deadlines and refreshes them
after committed changes.
> - The shared coordinator preserves retries, drain, and shutdown
without another private timer.

## Linked Issues or Issue Description

**What happened?**

The aggregate maintenance tick scans connection health every 30 seconds.
Empty instances and connections with fresh health observations still
cause SQL activity.

**Expected behavior**

No eligible connection means no recurring health SQL. Eligible
connections retain the existing 15-minute proactive check. Configuration
changes and health observations update the schedule after commit.

**Steps to reproduce**

Start a server with heartbeat scheduling enabled and no tool
connections. Let startup finish and observe two maintenance ticks. The
old code repeats the health inventory query. The new startup regression
verifies that those ticks do not invoke the health inventory.

**Paperclip version or commit**

This branch includes merged parent #17518 and master
`96727b3cfefb61f85ffe1fbc29a9b6dd6837efe8`. The 47 changed paths belong
to the single tool-connection health duty. No database file differs from
master.

Related: #14638 covers runtime-auth exclusions. #15448, #15325, and
#11910 concern MCP failure classification. This PR preserves those
provider policies and does not replace their work.

## What Changed

- Keep the GitHub error-projection unit fixture at the health-write
boundary and verify the real transaction path in the PostgreSQL
health-worker suite.

- Add a health schedule owned by tool access. Read lean eligibility
fields and saved observation timestamps.
- Register health work with the existing coordinator. Refresh only dirty
connection/application/company scopes using bounded inventory pages,
check due connections in fair company batches, continue concrete due
backlog after one second, and retain failures with backoff.
- Notify before supported eligibility and observation writes in their
owning transactions. Cover API/gallery setup, OAuth, catalog/gateway
observations, email, voice, access removal, deletion, and settings.
- Commit connection creation with grant, binding, and runtime-slot SQL
before its wake. Keep provider calls outside the new transactions.
- Keep provider-to-transport mapping in the chat catalog. SDK-only chat
changes do not wake health work.
- Replace aggregate health scans, retain the public manual sweep, and
document the producer audit and remaining sleep limits.
- Keep metadata-only example and Smoke Lab installs scoped. Lock and
re-read application type before publishing a real eligibility change.
- Keep failed inventory reads on their own retry deadline while healthy
due batches continue.
- Apply fixture migrations before using external disposable databases in
all six producer suites.

## Verification

Published head: `8c1d293d1ae2b7db753ab20f5a2a2edec1fb0a62`. Fresh CI and
reviews are required for this exact head.

- **623 distinct targeted cases pass.** The final four complete health,
fair-batching, GitHub health recovery, and startup suites pass 80 cases
after the retry fix. Five complete producer suites pass 543 cases: tool
access 469, access 18, email 45, GitHub events four, and Smoke Lab
seven. The earlier 26 health cases are part of this total, not
additional cases.
- The six producer suites first passed against separate, initially empty
PostgreSQL 17 databases at `acbf262843`. They apply their own
migrations. The subsequent repair changes only the health worker and its
regression; its complete final consumer suites pass on that integrated
parent. Server typecheck and build pass with the final repair.
- **Installer amplification:** five real installs with 205 unrelated
connections reproduced 1,035 example rows and 1,045 Smoke Lab rows.
Repeated metadata installs now load only changed fixture connections.
The same cases restore an excluded application through each installer
and require all 205 unrelated IDs to re-enter the schedule.
- **Concurrent eligibility:** a real PostgreSQL lock-wait case requires
the installer to notify after a concurrent type change. A mutation that
retains the lock but uses the old lookup fails the settled-hint
assertion; original source was restored byte for byte.
- **Retry isolation:** the failing baseline read one unavailable company
three times while another company's 60 due checks drained. Final tests
require one initial read, at least 30 seconds between failed reads,
prompt healthy progress, eventual recovery, and SQL quiet after
settlement.
- Real-service producer coverage includes AgentMail activation/pause,
GitHub installation changes, and member access removal. Lifecycle
coverage includes outer commit/rollback, startup reconstruction,
drain/resume, failure, and shutdown.
- Failure history and exact evidence scope: [health
acceptance](https://github.com/paperclipai/paperclip/blob/8c1d293d1ae2b7db753ab20f5a2a2edec1fb0a62/doc/idle-tool-connection-health.md#acceptance-follow-up-after-the-plugin-parent-merged-october-27),
[producer
audit](https://github.com/paperclipai/paperclip/blob/8c1d293d1ae2b7db753ab20f5a2a2edec1fb0a62/doc/plans/2026-10-24-tool-connection-health-sleep-audit.md).
Prior remote failures and earlier 706-case evidence remain historical.
Startup tests emitted listener-count warnings; all assertions passed. No
full local repository-suite, deployed sleep, fleet savings, or live
provider qualification claim is made.

## Risks

- Supported live writers must use the transaction signals. Independent
SQL writers and other replicas need their own wake integration.
- Configured eligible connections still have a real proactive health
obligation. This does not prove Cloud alarm coverage or host/database
suspension.
- Existing manual probes and catalog operations can overlap. This
migration does not introduce a credential-version CAS or change provider
failure policy.
- Voice remains eligible under the previous health policy. Its generic
probe can still report an unsupported transport; changing that behavior
is separate work.
- No database migration, new table, workflow, or lockfile change. This
PR includes its merged parent and current inspected master; later stack
PRs and all current-head acceptance gates remain separate.

## Model Used

OpenAI Codex, GPT-6 family, with repository tools, code execution, and
reasoning. The exact serving version and context-window size are not
exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-27 21:37:50 -05:00
DottaandPaperclip 13709b4c7a feat(connections): let agents configure connections within delegated authority (#17737)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Connections provide governed tools, chat channels, model accounts,
and gateways.
> - Several management routes require a human board actor even when an
agent has the required grant.
> - Default agents must configure these resources within the responsible
user's authority.
> - This pull request adds a shared authorization path to the existing
REST and native API surface.
> - Human authentication, execution approvals, and local host boundaries
remain enforced.
> - The benefit is that a standard agent can complete authorized setup
with a bounded human handoff.

## Linked Issues or Issue Description

**Subsystem affected**

Server authorization, connection services, shared contracts, native API
discovery, database schema.

**Problem or motivation**

Connection administration is available in the web app but often
unavailable to agents. Agent grants, responsible-user authority,
credential ownership, and actor attribution need separate checks.
Opening gateway-token creation also requires preserving the issuing
agent's delegated authority during token use.

**Proposed solution**

Use the existing authorization evaluator for every connection family.
Enforce current responsible-user authority, effective trust, grant
scope, resource visibility, and target-agent access. Keep existing agent
grants unchanged. Expose authorized configuration through default
`search_api` and `call_api`. Save human authentication steps and let
agents continue after completion.

**Roadmap alignment**

This change completes the API access path for the existing Connected
Apps, Personal & Shared AI Accounts, and governed MCP Gateway
capabilities in `ROADMAP.md`. It follows the maintainer's requested
design. Related work: #11899 enforces connection execution access and
creator/manager rules; this PR extends agent management across
connection families. It does not replace that execution-policy work.

## What Changed

- Add shared connection management context and mandatory
responsible-user, trust, scope, and target checks.
- Allow authorized agents to configure tool, chat/email, GitHub/Slack,
AI, profile, and gateway resources through existing routes.
- Preserve human credential ownership and record the actual agent,
responsible user, and run or key in audit data.
- Preserve saved browser authentication handoffs and callback session
checks.
- Bind agent-issued durable gateway tokens to their responsible user and
recheck delegated authority at use.
- Add explicit REST transport and management capability metadata,
built-in guidance, and sensitive-response handling.
- Intersect caller and endpoint-agent Slack test/invitation policies,
including independent and shared rates. Require the addressed human’s
current endpoint identity link before completing identity handoffs.
- Add production-authenticated integration scenarios. Companion #17739
contains the saved setup UI (including the required human
import/rotation step), product and permission documentation,
catalog/Vercel credential and callback ownership, and operation maps to
keep each review below 100 files. Both PRs must ship together; merge
this implementation first, then its companion.

## Verification

- Current implementation head:
`2ecd6451d0d0c8d82121a4e3fa4ce94c5befbbd6`, including master
`ea727dc3c2`, 99 files. The latest follow-up fixes the two valid
findings from the preceding f9 review. Exact-head Greptile has completed
at5/5 with no findings and Superagent passed. Full exact-head Linux CI
workflow40446606056 completed SUCCESS; only the queued external
Paperclip Review remains pending; earlier Linux passes do not qualify
this successor.
- Latest preserved-stack verification passed **2,165 tests** in fresh
PostgreSQL: **70 actual-auth scenarios**, 27 OpenAPI contracts, 2,022
native API tests and 46 saved GitHub wizard cases. **141 existing
gateway/policy acceptance tests** also passed. Workspace
typecheck/build, maps and token gates passed. The first workspace
typecheck was blocked by sandbox tsx IPC permissions; its authorized
rerun passed. This is focused local evidence; the full Linux CI matrix
is still required.
- Issuer/target gateway policy regressions were red before the fix. Both
decisions now intersect, including Block/Ask first/rates, same
credential identity and current token/user/agent authority after
approval. Invocations retain their token and actual requesting agent;
deleted tokens fail closed. Thirteen actual-auth cases cover revocation,
narrowed scope, approvals, shared rates and board compatibility.
Intermediate missing-token attribution, review-context and
implicit-session actor failures remain in the qualification record; all
final cases pass.
- Activity pagination now retains a bounded cursor across private-only
windows. The actual-JWT regression reaches three older readable events
after 27 private ones without exposing private details. A native
device-login regression was red for plaintext receipts; local check and
test-call results now use existing protected-response handling. The
passing actual-auth case proves encrypted persistence, exact replay,
redacted history and no response asset.

- Previous-head `5b55195593` Linux CI initially failed e2e shard 4/8 in
the existing connection-intent continuation test: periodic recovery
queued a repair at 01:21:47, the scripted journey completed the task at
01:21:48, and the extra queued run was cancelled at 01:21:49 because the
task was terminal. The assertion selected that latest cancelled run.
GitHub refused the first attempt while the workflow was nonterminal.
After completion, the single evidence-supported failed-job rerun was
accepted as **attempt 2** on the same head (workflow 40431202466). The
failed shard and final E2E aggregate passed on attempt 2, and the
complete Linux workflow for that head is successful. The original
failure remains part of qualification. No additional unchanged-code
rerun will be used for this failure.

- Companion CI also has an unchanged login-cleanup idleness timeout,
retained in its qualification record. That same 16-case suite passed in
this parent Linux run and a fresh local database. Both preceding
workflows passed after their single failed-job reruns as attempt 2; the
original failures remain recorded. No additional unchanged-code rerun
was used.

- Migration `0380_broken_aqueduct.sql` was regenerated through the
repository workflow after master took 0379 for project Git
authorization. It adds the responsible-user column with `IF NOT EXISTS`,
preserves upstream migrations, and satisfies snapshot retention and
migration safety checks.
- Upstream published Copilot v62 and v63 identities remain byte-for-byte
unchanged. The combined guidance is now v64, digest
`sha256:22e31f95ef972bcead98a4a707c9f33433558fd111e390e027a2bbb4346e112a`.
The normal shared TypeScript/Rust identity check and **61 Copilot
installation tests passed**. The old source-identity assertion first
failed against unchanged published v63; regenerating a new v64 preserves
the published release.
- Workspace `pnpm -r typecheck` and `pnpm build`, map validation, and UI
token gates passed on the final combined stack after the upstream merge.
Node 24.22.1 / pnpm 9.15.4 were used.
- **Earlier 82 API/OpenAPI tests passed** in fresh PostgreSQL 18: 55
assembled-server scenarios and 27 contracts. They use real persistent
agent keys, run JWTs, human sessions, production defaults, and
deterministic provider fixtures. No actor injection establishes access.
- The new email regression was red before per-account scope filtering.
The Arcade regression was red before human ownership and inventory-key
fixes; it also verifies actual-agent audit identity and a human-session
inventory read. The activity regression measured **200 authorization
checks before the fix and one afterward** for the same one-item page.
- **82 GitHub UI tests passed**, including all three new
failed-replacement regressions that were red before their fix. The UI
preserves an earlier copy only after confirming its committed
generation. Changed or unreadable results block copying and reconnecting
until reconciled. Human-only secret responses now include their
nonsecret committed generation.
- Private test status now rechecks the originating principal and linked
task, including mandatory agent/responsible-user privacy. Cross-agent
tests require the full caller tool access decision and the same
credential identity, regardless of auth label; caller and target
policies/approvals and rate limits apply before execution and after
human approval. A profile-only precondition was found to reject valid
allow-policy and scoped-grant access and has been removed. Both
permitted-access cases and a no-auth-label identity regression were red
before the correction; caller-specific Block still prevents provider
execution. The private-status regression was also red before its fix.
- The tool-activity regression measured 100 capability decisions and 102
run captures on a 100-event page before the fix. It now uses two task
decisions and at most two captures while retaining the same 50 visible
events, private-event filtering, and 403 after audit-grant revocation.
The regression was red before request-context reuse and task-decision
sharing; final server typecheck/build also pass.
- Generic chat configure/reconnect now hands agent-supplied verification
credentials to the addressed human before provider work or vault
changes. Actual key/JWT regressions were red for Slack signing-key
replacement and generic channel secret-reference rebinding. They now
prove unchanged refs and vaulted signing value, human setup, permitted
agent bot-token repair, and governed test messages. All 82 API/OpenAPI
cases pass in fresh PostgreSQL. Eight native receipt protections and
final server typecheck/build also pass.
- The preceding review found that old healthy state could retire a
credential-entry card before a human changed anything. The actual-JWT
regression was red. Saved field generations now change only with
protected values and commit with refs plus a verification fence; retries
retain the baseline. Old health, bot-token repair and unchanged entry
stay pending. New signing verification and current authority permit
continuation. A separate red race proved that user revocation during
provider authentication could return 403 after refs changed; authority
is now checked before persistence and inside the credential transaction.
The final 82 API/OpenAPI cases, four neighboring real-database
Slack/GitHub lifecycle cases, and final server typecheck/build pass.
- A preceding full run crossed a rate-window boundary. Database rows
prove one consumed slot at 23:35 and another at 23:36 for the same
policy/bucket. Only Date is frozen in the rate test cases; transport and
asynchronous timers remain real. Production rate behavior is unchanged,
and the final full run passes. An intermediate fixture failure that
overwrote hidden generation metadata with an API projection is retained
in the qualification record.
- Linux CI on an earlier head found two rate-admission assertions that
omitted the newly persisted bucket key. The failures were reproduced
locally; both assertions now verify the exact actor/tool bucket and all
49 policy-service tests pass. The companion qualification record retains
the initial and intermediate failures. Catalog creator attribution is
preserved in the companion to keep this implementation review at 99
files.
- The latest review reproduced three regressions before repair: complete
human replacements failed on unavailable old signing secrets, fresh
GitHub links incorrectly selected reconnect, and different
credential-entry requests shared an idempotency key. Each old protected
ref now has a narrow known-unavailable fallback; unexpected resolver
failures and company/scope violations still abort. Fresh imports keep
their credential form, configured endpoints keep reconnect, and
canonical field sets/completion modes distinguish handoffs while
matching retries retain their baseline. All 55 actual-auth scenarios and
27 separately run OpenAPI contracts pass, as do server typecheck/build.
Companion #17739 also passes all 41 wizard tests including actual
fresh-form submission, UI typecheck, maps and token gates. Earlier red
results and a mistyped OpenAPI filename are retained in qualification.
- The configured GitHub handoff regression was red in the
browser-component tests and actual-auth link/persistence tests. Saved
links now name requested fields and all/any completion mode; the
companion form submits those supplied fields through reconnect and
retains other saved credentials. Webhook/client-secret reconnect remains
human-only; first-setup secret restrictions remain. All 46 wizard tests
pass, including combined modes and legacy links. A duplicate review
referenced the older companion bf760912; the current form is already
fixed, linked in the reply, and its thread is resolved.
- Linux CI on bf760912 failed automatic Slack provisioning because vault
persistence cleared an already verified manifest-update challenge. The
same regression was reproduced locally. The provider-owned helper now
preserves proof only for the current signing fingerprint and callback
URL. Eight existing compatibility cases pass, including this case and
initial GitHub secret rejection. The first uncommitted GitHub-field
variant also failed that initial boundary and was narrowed to reconnect;
both failures remain in qualification.
- The upstream merge conflicted only in generated migration metadata.
Upstream 0379 is preserved, normal generation/pruning produced 0380, and
two repeated applications of the idempotent additive SQL passed. The
final combined stack passed **128 tests** (55 actual-auth, 27 OpenAPI
and 46 wizard cases) in fresh PostgreSQL. Eight Google UI cases also
pass; the two protocol cases could not initialize embedded PostgreSQL in
this fresh local attempt. Their earlier passing evidence remains
historical; fresh Linux CI must execute them. Vercel personal credential
attribution is preserved in companion #17739 with its other
ownership/consent follow-ups, keeping this PR at 99 files.
- The suite covers fresh defaults, restricted users/agents, scopes,
personal isolation, token revocation, human handoffs, callback replay,
execution approvals, authorized self-targeted policy changes,
self-approval denial, and current authority before commit. Reproduce API
checks with `pnpm exec vitest run
server/src/__tests__/connection-management.integration.test.ts
server/src/__tests__/openapi-routes.test.ts`, using a fresh database
through `PAPERCLIP_CONNECTION_MANAGEMENT_TEST_DATABASE_URL` or the
default embedded test database.
- Companion #17739 also records consent separately from token refresh so
saved Google choices survive refresh. Its eight service-picker and two
local-HTTP OAuth tests pass; both new UI cases were red before the fix.
Normal migration generation confirms no SQL change is needed for the
additive JSON metadata.
- The integration module shares one assembled server, authentication
factory, and provider lifecycle to preserve the
HTTP-to-vault-to-task-continuation proof. A reused external database
caused a Slack fixture identity conflict; the fresh final run passed.
Earlier fixture/transport and contract-update failures remain in the
companion qualification record.
- The broad local `pnpm test:run` attempt remains blocked by macOS
embedded PostgreSQL's host semaphore limit. It is not a clean pass. The
fresh Linux CI matrix must complete. Provider fixtures do not prove live
setup or an empirical LLM success rate.
- A current-head completed Paperclip assessment is not yet recorded. Its
earlier external recovery boundary has not been bypassed, and no
incomplete assessment is reported as passing. The user authorized both
merges when checks pass; merge this implementation first, then replay,
retarget, and requalify companion #17739 against the squash-merged
master.

- After the master refresh, local workspace typecheck/build, **97
actual-auth/OpenAPI cases** and **141 gateway/policy cases** pass. The
original fixed database port was occupied; Docker assigned a free
localhost port. The first neighbor setup command could not resolve root
`tsx`, leaving its schema uninitialized; the built database package
migrated the separate fixture and all unchanged cases passed. Both setup
failures are retained in qualification. Fresh Linux CI remains required;
no full local test-sweep pass is claimed.

- The f9 Greptile review scored **2/5** for two reproduced findings:
Slack test sends skipped the caller’s tool policy, and healthy App
credentials completed an unconfirmed identity handoff. Both are fixed in
this head and have resolved evidence threads
([Slack](https://github.com/paperclipai/paperclip/pull/17737#discussion_r4384927765),
[identity](https://github.com/paperclipai/paperclip/pull/17737#discussion_r4384927895)).
Actual key/JWT cases cover twelve Slack caller/target/recipient
combinations; the human-session GitHub OAuth callback alone remains
pending until explicit confirmation. All **99 API/OpenAPI tests** (72
actual-auth + 27 contracts), workspace typecheck and build pass. On the
combined companion, all four focused cases (including the twelve Slack
combinations), typecheck/build and maps also pass. The continuation
suite’s 50 embedded-PG cases were skipped because its database could not
start locally; fresh Linux CI must run them. Initial fixture
credential-collision and audit-projection assertion failures are
retained with the red reproductions in the companion qualification
record. No full local sweep or live consent is claimed.

## Risks

- The maintainer explicitly confirmed authorized policy administration,
including self-targeted changes that loosen controls. These writes
require management grants and the responsible user’s current authority.
Agents cannot approve their own governed action requests or edit their
own principal permission grants. The actual-auth regression exercises
both permitted administration and those denials.
- This is a broad authorization change. Incorrect scope or owner
resolution could deny permitted setup or expose another user's
resources. Integration cases cover both permitted and denied requests
with actual authentication.
- The additive nullable gateway-token column does not infer identities
for legacy clients or backfill agent grants. Legacy agent tokens without
a responsible user fail closed.
- Provider fixtures test supported setup and callback contracts. They do
not prove live provider availability.

## Model Used

- OpenAI Codex, GPT-6 family, with reasoning, repository tools, and code
execution. The runtime does not expose a more specific deployment ID or
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass — focused suites above;
full local sweep remains blocked as documented
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green — exact `2ecd6451d0` Linux CI
completed SUCCESS; external Paperclip Review is still queued
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups —
exact `2ecd6451d0` review completed; both f9 findings have resolved
evidence replies
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-27 08:20:24 -05:00
DottaandPaperclip 9c442cf48e feat(connections): deploy and configure Supabase and Vercel (#17217)
Add governed Supabase and Vercel deployment/configuration connectors with vaulted credentials, agent access policies, approvals, audit, and bounded provider operations.

Deliver reviewed REST actions to native agents and guard Vercel preview intent on empty projects. Preserve secret-safe request identity and durable retry behavior.

Verified by current-head CI, fresh Greptile 5/5 review, and real-agent deployment/configuration walkthroughs on disposable provider resources. Test resources and temporary provider tokens were removed after verification.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-23 11:37:29 -07:00
Devin Foley 059ae49cab feat(connections): add Unikraft Cloud connection (#17139)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Apps connections give agents governed access to external services
through vault credentials, grants, action policies, the tool gateway,
and audit.
> - Unikraft Cloud runs unikernel micro-VMs, services, and volumes
across regional metros, and it offers a sandbox plugin for commands and
files.
> - Unikraft is not in the catalog. It has no hosted MCP server, so the
generic "Connect your own MCP server" flow cannot reach it.
> - Daytona (#16483) added a native REST pattern for this case: a
catalog generated from the provider's published OpenAPI specifications
and executed through the gateway.
> - This pull request adds Unikraft on that pattern and moves the shared
native-provider code into one registry, so the shared connection code
does not name each provider.
> - The benefit is governed agent access to all published Unikraft
actions, with the token kept in the vault and sent only to published
Unikraft hosts.

## Linked Issues or Issue Description

Refs #16483

**Problem or motivation**

Teams that run workloads on Unikraft Cloud want their agents to list
instances, read logs, deploy and scale services, and run commands in
sandboxes. Unikraft is not in the Apps catalog. Unikraft has no hosted
MCP server, so operators cannot connect it through the remote MCP flow.

**Proposed solution**

Add a native Unikraft connection. Generate its actions from Unikraft's
official OpenAPI specifications for the platform API and the sandbox
plugin. Pin the specifications to a reviewed commit. The operator
connects an API token. The token stays in the vault and goes only to the
five published metro hosts. Every action takes an optional metro.
Actions without a metro use the connection's default metro, which the
operator can change under Advanced.

**Alternatives considered**

A catalog-only AppDefinition was not possible because no hosted MCP
endpoint exists. Wrapping the `unikraft` CLI was rejected because it
would expose host filesystem behavior and CLI login state. A second copy
of the Daytona bridge was rejected; the shared transport and catalog
contract are now one module with a provider registry.

**Roadmap alignment**

This extends the existing Apps and Connections provider coverage. It
uses the existing credential, grant, policy, gateway, and audit model.
It does not overlap planned core work.

## What Changed

- Added `scripts/generate-unikraft-tools.mjs` and its tests. The
generator reads the platform and sandbox plugin specifications at commit
`5dd107ea1cd8716b83cb86a83ed2a6ee46f99b5b`, resolves shared parameters,
records source SHA-256 hashes, and writes
`server/src/services/unikraft/operations.generated.json`.
- The catalog has 105 actions: 89 platform operations and 16 sandbox
plugin operations. The server-sent audit event stream is recorded in the
inventory and is not advertised as an action.
- Classification: GET operations and sandbox reads and waits are reads.
Delete, stop, suspend, detach, unpin, and sandbox command execution,
signals, and stdin are destructive. All other operations are writes.
- Added `server/src/services/connectors/unikraft/client.ts` and its
generated catalog. It validates arguments against the generated schemas,
accepts only published metros, rejects path traversal, sends JSON bodies
on GET and DELETE as the specification requires, and verifies the token
with `GET /v1/users/quotas`. A sandbox wait that times out (documented
HTTP 408) returns a still-running result instead of a failure.
- Moved the bounded, credential-redacting HTTP sender and the catalog
contract helpers from `daytona.ts` to `native-rest-client.ts`. Added
`native-provider-catalogs.ts`. `tool-access.ts`, the
`tool-access/catalog` and `token-broker` modules,
`connectors/tool-policy.ts`, and `tool-gateway.ts` now use this registry
instead of Daytona-specific checks. Daytona behavior does not change.
- Added the Unikraft definition source, the generated definition, the
brand mark and manifest entry, and the permission review (`liveProof:
not-run`).
- Added `doc/connections/UNIKRAFT.md` with a sequence diagram,
endpoints, setup, actions, and limits. Updated the connections README,
the permission audit, and both capability maps.
- Tests: definition shape, database-backed gateway reads, writes,
deletes, and sandbox commands for personal and shared grants, default
and per-call metros, rejection of unpublished metros and traversal, Off
policies, revocation, stale-contract blocking, credential export denial,
schema validation for all actions, pinned-transport tests for GET and
DELETE bodies, and a browser setup check.

## Verification

- `pnpm exec vitest run packages/shared/src
server/src/__tests__/unikraft-connection.test.ts
server/src/__tests__/unikraft-http-boundary.test.ts
server/src/__tests__/daytona-connection.test.ts
server/src/__tests__/daytona-http-boundary.test.ts
server/src/__tests__/tool-access-service.test.ts
server/src/__tests__/tool-gateway-service.test.ts
server/src/__tests__/tool-gateway.test.ts
ui/src/pages/apps/AppsConnect.test.tsx ui/src/pages/apps/Browse.test.tsx
ui/src/lib/app-brand-assets.test.ts` plus `connector-tool-policy` and
`tool-catalog-publication`: all passed after the rebase onto the split
tool-access modules.
- `node --test scripts/generate-unikraft-tools.test.mjs
scripts/generate-daytona-tools.test.mjs
scripts/app-brand-validation.test.mjs`: 20 passed.
- `node scripts/check-app-brand-assets.mjs`, `node
scripts/check-maps.mjs`, and `git diff --check`: passed.
- `pnpm --filter @paperclipai/shared typecheck`, `pnpm --filter
@paperclipai/server typecheck`, and `pnpm --filter @paperclipai/ui
typecheck`: clean.
- `npx playwright test --config tests/e2e/playwright.config.ts
tests/e2e/unikraft-connector.spec.ts
tests/e2e/daytona-connector.spec.ts`: 2 passed.
- `node scripts/generate-unikraft-tools.mjs` regenerates the committed
catalog with the recorded hashes.
- Manual: on a local instance, open Connectors, search for Unikraft, and
select **Add key**. Confirm the icon, the password field, the key
guidance, and the default metro select under **Change**.
- Not verified: a real Unikraft account. Live provider read and write
proof is pending.

## Risks

- A Unikraft token acts for the whole organization in every metro.
Unikraft offers no narrower token permissions. Assign only trusted
agents and review action permissions.
- Sandbox actions run arbitrary commands inside the selected instance.
They are classified as destructive.
- A timed-out write can still complete at the provider. The bridge does
not retry writes automatically.
- New metros fail closed until the specification is regenerated and
reviewed. Legacy `*.kraft.cloud` hosts are not reachable.
- The Daytona refactor moves code without behavior changes. All Daytona
tests pass without edits.
- The generated JSON snapshot is large because it holds the provider
input contracts. No migration or lockfile change is required.

## Model Used

- Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, 1M context
window, used through Claude Code with tool use (shell, file editing, web
research, browser tests).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-10-22 15:14:26 -07:00
DottaandPaperclip 4dfcb4c9b8 refactor(tools): isolate connection access and removal (#17167)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Connection grants and installations control which identities and
agents can reach tools.
> - Removal must close those access paths and stop runtimes before
cleaning up owned secrets.
> - These responsibilities still shared the setup coordinator after
earlier extractions.
> - This pull request gives grants, installations, credential validation
and removal explicit owners.
> - The benefit is a smaller coordinator and focused access-lifecycle
modules with existing security ordering intact.

## Linked Issues or Issue Description

Depends on #17165. Earlier stack: #17144 → #17146 → #17147 → #17155 →
#17157 → #17159 → #17161 → #17162 → #17165 → this PR.

**What existing behavior does this improve?**

Maintenance of connection audiences, delegation, installation and
removal.

**Current behavior**

The 6,419-line service still owns grant mutation, installation reach and
resumable credential cleanup alongside app setup.

**Proposed behavior**

A grant owner manages audiences and delegation. An installation owner
manages reach and its profile bindings. A removal owner keeps access
revocation, runtime shutdown, secret classification and cleanup
together. The coordinator delegates through typed APIs.

**Reason and benefit**

The entry point falls to 4,821 lines / 198,065 bytes, meeting the
requested under-5,000-line target. Across the full stack it falls from
21,525 lines / 829,519 bytes. App setup remains together in the
coordinator.

## What Changed

- Extract credential validation (75 lines), grants (603), installations
(350) and removal (658).
- Preserve company boundaries, audience advisory locks, membership
locks, assignment locks and audit order.
- Keep the existing runtime supervisor and secret client. Retain live
HTTP options for provider shutdown.
- Keep Slack credential-mutation leasing and resumable owned-secret
cleanup with removal.

## Verification

- Removal, grant-resolution and the stronger real HTTP OAuth fixture
pass before extraction (13 cases); the existing service and publication
baselines also pass.
- The original combined targeted run passed 525 cases across seven
suites. After replaying onto master, 601 cases pass across nine suites,
including Braintrust lifecycle and connector-policy coverage, with one
database worker.
- Full repository typecheck and build passed before merge integration.
After integration, server typecheck, module-boundary and whitespace
gates pass; fresh CI covers the full repository gates.
- All 94 pre-existing function/method bodies match by normalized syntax
comparison. A full-stack TypeScript symbol audit preserves all 18
exports and 89 service methods.
- The earlier local full-suite run was interrupted after 79 minutes and
remains unverified. All current-head CI gates pass on
`d5307fcea2365e51a75d49db82deeed236d19951`, including the full sharded
test matrix, browser suites, typecheck, build and canary dry run.
Greptile is 5/5 with no open review threads after merge integration.

- GitHub canceled duplicate CI run
[39716841632](https://github.com/paperclipai/paperclip/actions/runs/39716841632)
during the atomic stack update. Replacement run
[39716843243](https://github.com/paperclipai/paperclip/actions/runs/39716843243)
passed on the same head; no failing test was retried.

## Risks

- Audience replacement must not erase an inactive member marker and
widen authority. Its advisory and membership locks remain in the grant
owner.
- Uninstall must remove only its own profile bindings. Existing
operator-authored bindings keep their current behavior.
- Removal must remain fail-closed and retryable after provider failures.
Its order, shared-secret retention, supervisor and cleanup references
are unchanged and covered by removal tests.
- No schema, public API, policy or scheduling change is intended. The
first eight PRs have landed; this PR follows the updated catalog
extraction.

## Model Used

OpenAI Codex, GPT-6 family, with repository tools and code execution.
The exact serving model ID and context-window size are not exposed in
this session.

## Checklist


- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-21 22:39:00 -05:00
DottaandPaperclip f7210cd31f refactor(tools): isolate catalog ownership (#17165)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents receive reviewed actions discovered from their connections.
> - Transport, discovery, risk classification, publication and account
inventory lived in the same service.
> - Their ownership differs: transport guards requests, discovery reads
providers, and publication changes the catalog under locks.
> - This pull request gives those phases separate modules with explicit
capabilities.
> - The benefit is a visible discovery-to-publication path that
preserves permission and quarantine behavior.

## Linked Issues or Issue Description

Builds on merged #17162 and preserves the connector policy from #17141.
Earlier stack: #17144 → #17146 → #17147 → #17155 → #17157 → #17159 →
#17161 → #17162 → this PR.

**What existing behavior does this improve?**

Maintenance of tool discovery, catalog publication, health and upstream
account inventory.

**Current behavior**

The 8,988-line service handles HTTP guards, MCP sessions, provider
probes, risk classification, quarantine, cached reads and account sync.

**Proposed behavior**

Discovery returns descriptors through the existing credential and
transport owners. Publication owns catalog writes and default access. A
reader owns refresh deduplication. Health and aggregator inventory have
separate owners.

**Reason and benefit**

The entry point falls to 6,419 lines, a reduction of 2,569. Provider
discovery no longer shares a closure with connection setup and teardown.

## What Changed

- Extract guarded remote HTTP, descriptor policy, discovery,
publication, default access, cached reads, health, attention and
aggregator account sync.
- Keep the public descriptor/risk exports and existing service methods.
- Preserve database locks, transaction-scoped default-access writes,
quarantine decisions, current grant checks and failure classification.
- Preserve per-service catalog flights and aggregator jobs, lease
ownership and live remote-request options.
- New modules range from 39 to 492 lines.

## Verification

- Before and after the move, 470 public service/HTTP integration cases
and 44 publication, HTTP rebinding and aggregator sync cases pass.
- Server typecheck, module-boundary gate and whitespace checks pass.
- All 157 pre-existing function/method bodies match by normalized syntax
comparison.
- Before merge integration, the completed stack passed 525 targeted
cases, repository-wide typecheck and build. Its stronger rotating-token
fixture is included.
- After replaying onto master, the integrated stack passes 601 cases
across nine suites, including Braintrust lifecycle and connector-policy
coverage. Server typecheck and module-boundary checks pass. All 157
moved bodies still match the current master versions.
- The earlier local full-suite run was interrupted after 79 minutes and
remains unverified. All current-head CI gates pass on
`a21595f20809a514b7619d8a8748c56f8e546e2f`, including the full sharded
test matrix, browser suites, typecheck, build and canary dry run.
Greptile is 5/5 with no open review threads after merge integration.

## Risks

- Discovery must keep its current grant and session behavior;
publication must retain its locks and quarantine rules. Existing
authority, schema-drift and publication-race tests exercise these paths.
- Aggregator account observations must not become authorization. Lease,
freshness, failure and credential-fingerprint behavior stay unchanged.
- Live transport options must not be snapshotted. The HTTP owner retains
the same options object.
- No schema, public API, access-policy or scheduling change is intended.

## Model Used

OpenAI Codex, GPT-6 family, with repository tools and code execution.
The exact serving model ID and context-window size are not exposed in
this session.

## Checklist


- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-21 22:38:59 -05:00
DottaandPaperclip ba9e6e4bc5 refactor(tools): isolate authorization lifecycle (#17162)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Browser authorization binds a consenting identity to a company
connection.
> - Direct OAuth, managed-cloud claims and Vercel authorization have
different state and persistence rules.
> - Keeping all of them in the general tool service obscures those rules
and their cleanup owners.
> - This pull request separates state, initiation, provider completion
and explicit identity promotion.
> - The benefit is reviewable authorization flows with their existing
authority checks and ordering preserved.

## Linked Issues or Issue Description

Depends on #17161. Earlier stack: #17144 → #17146 → #17147 → #17155 →
#17157 → #17159 → #17161 → this PR.

**What existing behavior does this improve?**

Maintenance of browser authorization and credential-sharing lifecycle.

**Current behavior**

The 12,019-line service combines provider callbacks, actor-bound states,
authorization URLs and personal-to-company credential promotion.

**Proposed behavior**

Separate owners handle authorization state, start, direct/cloud/Vercel
callbacks, catalog completion and explicit access finalization. Shared
setup policy and identity checks have typed APIs used by the coordinator
and authorization owners.

**Reason and benefit**

The entry point falls to 8,997 lines, a reduction of 3,022.
Provider-specific state consumption and durable-write rules are visible
in their own modules.

## What Changed

- Extract authorization flows and their constraints; the largest new
module is 700 lines.
- Preserve direct/Vercel terminal state consumption and managed-cloud
retryability until claim persistence commits.
- Keep membership and grant locks, secret transaction clients,
interaction resolution and catalog ordering unchanged.
- Keep partial secret acquisition and cleanup together in access
finalization.
- Give shared organization-grant persistence, personal identity checks,
setup policy and response projections named owners.

## Verification

- All 470 public service and real local HTTP/PostgreSQL OAuth cases pass
before and after extraction.
- Coverage includes same-actor state binding, replay/denial, scope
selection, membership revocation, provider failures, refresh concurrency
and persisted usable credentials.
- Server typecheck, module-boundary gate and whitespace checks pass.
- All 182 pre-existing function/method bodies match by normalized syntax
comparison.
- The completed stack passes 525 targeted cases, full repository
typecheck and build. The earlier interrupted local full suite remains
unverified; all current-head CI gates pass; Greptile is 5/5 with no open
review threads.

- Original CI failure retained: [server shard
1/12](https://github.com/paperclipai/paperclip/actions/runs/39710536789/job/119600621627),
`workspace-runtime.test.ts` case “replaces a reused Paperclip dev
runtime whose 2xx health payload is unhealthy”. The fixture failed its
initial health readiness with `ECONNREFUSED` after 97 probes / 10
seconds. The fixture and runtime implementation are unchanged and do not
import this service. The isolated case passes without changing its
timeout or assertions. One retry of only that shard passed at the same
commit. All CI gates are now green; the original failure remains
recorded here.

## Risks

- Provider flows intentionally consume authorization state at different
points. This extraction preserves each flow rather than sharing its
completion algorithm.
- Promotion must remove partially created secrets on failure and retire
source secrets only after persistence. Both phases remain in the same
owner with unchanged ordering.
- Local HTTP fixtures prove assembled application behavior, not live
third-party qualification.
- No schema, public API or authorization policy change is intended.

## Model Used

OpenAI Codex, GPT-6 family, with repository tools and code execution.
The exact serving model ID and context-window size are not exposed in
this session.

## Checklist


- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-21 22:21:43 -05:00
DottaandPaperclip 4fdef9457f refactor(tools): isolate credential ownership (#17161)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Connections need durable credentials that survive authorization and
token rotation.
> - Storage, registration, refresh coordination and credential selection
lived in the general tool service.
> - Each has a different ownership boundary: secrets, process-wide
registration, database leases and per-service refresh work.
> - This pull request gives those responsibilities named modules and
passes their narrow capabilities to consumers.
> - The benefit is visible credential ownership while retaining current
concurrency and failure behavior.

## Linked Issues or Issue Description

Depends on #17159. Earlier stack: #17144 → #17146 → #17147 → #17155 →
#17157 → #17159 → this PR.

**What existing behavior does this improve?**

Maintenance of OAuth credential persistence, refresh and provider
credential access.

**Current behavior**

The 14,877-line service owns registration flights, refresh leases,
secret writes and provider selection alongside app setup.

**Proposed behavior**

Dedicated owners handle credential storage, client registration, refresh
leases, connection/grant refresh, GitHub continuity and Vercel
credential access. Credential selection composes those owners.
Authorization callbacks remain in the coordinator for the next
extraction.

**Reason and benefit**

The entry point falls to 12,019 lines, a reduction of 2,858. Lease
acquisition and cleanup can be reviewed together, independently of
callback completion.

## What Changed

- Extract credential owners and shared single-flight/connection-link
primitives. The largest new module is 650 lines.
- Preserve process-wide client-registration flights, per-service refresh
maps and GitHub sweep state.
- Preserve live cloud-connector lookup, options and environment reads;
use the existing secret client and transaction-scoped clients.
- Extend the real local HTTP/PostgreSQL OAuth fixture with concurrent
refresh across two service instances and authenticated use of rotated
credentials.

## Verification

- The two extended HTTP integration cases pass before extraction for
public and client-secret-basic clients.
- After extraction, all 470 public service and HTTP integration cases
pass, including refresh failure, abandoned leases, secret redaction and
membership revocation cases. The fixture was then strengthened after
review: a second expiry must use the persisted rotated refresh token,
and the provider rejects the old token. Both cases pass before and after
extraction; server typecheck also passes.
- Server typecheck, module-boundary gate and diff whitespace checks
pass.
- All 233 pre-existing function/method bodies match by normalized syntax
comparison.
- Full typecheck/build will run at the stack handoff. The earlier
interrupted local full suite remains unverified; all current-head CI
gates pass, with Greptile 5/5 and no open review findings.

- Prior-head CI failure retained: run 39708550066, general server shard
2/12, `native-session-executor.test.ts` case “preserves a real listening
process and its marker when a health RPC times out” raised
`runner_remote_process_identity_unavailable` /
`process_identity_changed`. Both the fixture and runner implementation
are unchanged. The isolated case passes without changing assertions or
timeouts. The fresh full CI run passes for the test-only review fix; the
prior failure remains recorded here.

## Risks

- Changing state lifetime could permit duplicate refreshes or
registrations. Existing concurrency cases and the real HTTP fixture
exercise these paths.
- A transaction-scoped secret write must stay inside its authority lock.
Function bodies, await ordering and client arguments are preserved.
- The fixture proves assembled local HTTP/database behavior, not live
third-party qualification.
- No schema, public API, policy or scheduling change is intended.

## Model Used

OpenAI Codex, GPT-6 family, with repository tools and code execution.
The exact serving model ID and context-window size are not exposed in
this session.

## Checklist


- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-21 22:21:43 -05:00
DottaandPaperclip ffcd113761 refactor(tools): isolate runtime and activity ownership (#17159)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Operators inspect tool execution, review pending actions and control
local runtimes.
> - These paths shared a large service with token exchange and app
authorization.
> - Runtime state and execution history need explicit owners and stable
public responses.
> - This pull request separates runtime controls, health summaries,
activity reads and the review queue.
> - The benefit is focused modules that preserve the current supervisor
and approval behavior.

## Linked Issues or Issue Description

Depends on #17157. Earlier stack: #17144 → #17146 → #17147 → #17155 →
#17157 → this PR.

**What existing behavior does this improve?**

Maintenance of tool runtime administration and execution activity views.

**Current behavior**

The 16,100-line service contains runtime slot controls, health
calculations, usage history, invocation decisions and action review
queries.

**Proposed behavior**

Runtime modules own slot persistence, operator controls and health
summaries. Activity modules own history, response projections and the
review queue. The public service delegates to these owners.

**Reason and benefit**

The entry point falls to 14,877 lines, a reduction of 1,223 lines. The
coordinator no longer needs execution-history joins or runtime alert
calculations.

## What Changed

- Extract five modules totaling 1,282 lines. The largest is 426 lines.
- Pass the existing supervisor to slot controls. Pass the same live
deployment options, clock and host getter to runtime health.
- Keep activity reads scoped as before. Keep unsigned-request grace and
invalid-signature cancellation in the review-queue owner.
- Preserve public methods, response projections, query order and audit
writes.

## Verification

- The public service and unsigned review-queue tests pass before and
after the move. The combined post-move run passes 469 cases with one
database worker.
- Existing cases exercise actual local stdio stop/restart, route
permissions, stale-slot and audit-failure health, activity attribution,
run decisions and unsigned pending requests.
- Server TypeScript check, module-boundary gate and `git diff --check`
pass.
- Normalized syntax comparison finds all 256 pre-existing
function/method bodies unchanged.
- Full typecheck/build will run at the stack handoff. The earlier
interrupted local full suite remains unverified; all current-head CI
gates pass, with Greptile 5/5 and no open review threads.

## Risks

- Creating another supervisor would split process ownership. The new
module receives the same instance; it does not construct one.
- Copying deployment settings could freeze live values. The health
reader holds the existing options object and getter.
- Review-queue reads can cancel invalid requests. Those writes and the
unsigned-request grace period are preserved and covered by the existing
regression test.
- No schema, public API, authorization policy or background scheduling
change is intended.

## Model Used

OpenAI Codex, GPT-6 family, with repository tools and code execution.
The exact serving model ID and context-window size are not exposed in
this session.

## Checklist


- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-21 22:21:42 -05:00
DottaandPaperclip a42f213244 refactor(tools): isolate connection token broker (#17157)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents can request short-lived tokens for connections that allow
token exchange.
> - Admission, scope checks, exchange protocols and issuance records
lived inside the general tool service.
> - The security flow needs one visible owner while provider payloads
need a separate boundary.
> - This pull request extracts the broker into admission, configuration,
endpoint, exchange and issuance modules.
> - The benefit is a focused issuance flow with its locks, authority
checks and audit order intact.

## Linked Issues or Issue Description

Depends on #17155. Earlier stack: #17144 → #17146 → #17147 → #17155 →
this PR.

**What existing behavior does this improve?**

Maintenance and verification of connection token issuance for agent
runs.

**Current behavior**

A 17,752-line service contains broker configuration, endpoint policy,
credential exchange, admission and issuance.

**Proposed behavior**

A dedicated broker coordinates the same admission and transaction flow.
The exchange owner returns normalized token material. It receives the
existing secret client, including the transaction-scoped client at
issuance time.

**Reason and benefit**

The entry point falls to 16,100 lines, a reduction of 1,652 lines.
Reviewers can see the responsible-user lock and hash-only issuance write
without navigating app setup or OAuth callbacks.

## What Changed

- Extract six broker modules. The largest is 755 lines; together they
contain 1,721 lines.
- Give tool-access audit persistence a named owner shared by the broker
and coordinator.
- Keep current-state checks, policy decisions, rate limits, secret
resolution and transaction order unchanged. Private-host configuration
is still read live.
- Add two public-route integration cases with a real local HTTP server
and disposable PostgreSQL. Cover Pages and RFC 8693 success and upstream
failure.

## Verification

- Baseline: all 466 existing public service tests passed before this
move. The two new real HTTP cases passed on the original implementation
before extraction.
- After extraction: 495 tests pass across the public service and HTTP
rebinding suites, with one database worker. This includes the two new
cases, membership revocation ordering, scope denials, disabled broker
behavior and hash-only persistence.
- Server TypeScript checking, the module-boundary gate and `git diff
--check` pass.
- Normalized syntax comparison finds all 294 pre-existing
function/method bodies unchanged.
- Full typecheck and build passed for the preceding batch. Its
interrupted local full-suite result remains unverified. All current-head
CI gates pass; Greptile is 5/5 with no open review threads.

## Risks

- A new closure could use the wrong secret client or move exchange
outside the membership lock. The existing transaction still passes
`secretService(tx)` to the exchange owner after taking that lock.
- Endpoint validation must preserve live configuration and fail before
credential disclosure. Existing rebinding and private-host tests cover
this path.
- The local fixture proves real HTTP and persisted application behavior.
It does not qualify a live third-party broker.
- No schema, API or access policy change is intended.

## Model Used

OpenAI Codex, GPT-6 family, with repository tools and code execution.
The exact serving model ID and context-window size are not exposed in
this session.

## Checklist


- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-21 22:21:42 -05:00
DottaandPaperclip 233753e577 refactor(tools): isolate profile ownership (#17155)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Tool profiles select the actions each agent can use.
> - Profile reads, review, mutation and assignment resolution shared a
large OAuth and connection service.
> - Those paths have their own company checks, lock order and database
projections.
> - This pull request gives profiles explicit owners while preserving
the existing transactions and public API.
> - The benefit is a smaller coordinator and a reviewable access
boundary.

## Linked Issues or Issue Description

Depends on #17147. Stack: #17144 → #17146 → #17147 → this PR. These PRs
remain open.

**What existing behavior does this improve?**

Maintenance of tool profile management and effective agent assignments.

**Current behavior**

The service entry point contains 19,293 lines. Profile behavior is
spread across its closure and returned object.

**Proposed behavior**

Profile management accepts only a database and clock. Assignment
resolution accepts three named capabilities. Projection and shared lock
functions have explicit homes. Public methods and responses remain
unchanged.

**Reason and benefit**

Reviewers can follow profile transactions, lightweight reads and
precedence without navigating OAuth setup. The entry point falls to
17,752 lines, a reduction of 1,541 lines in this PR.

## What Changed

- Extract profile management, projections and effective assignment
resolution into three modules. Their sizes are 886, 268 and 216 lines.
- Move shared actor/transaction types, identifiers and assignment locks
into named modules. Preserve the public lock export.
- Keep all database reads, transaction boundaries, lock order and method
signatures. Add no cache or background work.
- Record the next ownership cuts toward an entry point below 5,000
lines.

## Verification

- Before and after: 466 cases in `tool-access-service.test.ts` and 33
cases across profile entry validation, effective assignments and catalog
publication. All 499 cases pass with one database worker.
- Server TypeScript check and `node scripts/check-module-boundaries.mjs`
pass. `git diff --check` passes.
- Normalized syntax comparison finds all 329 pre-existing
function/method bodies unchanged. Wiring passes the unchanged public
service tests.
- The previous stack passed full typecheck and build. Its full local
test run was interrupted and remains unverified; its complete CI matrix
passed. This PR will record its own current-head CI result below.

- Current-head CI is green. Current review: Greptile is 5/5 at
`0f5c85905e`, with no inline review threads.
- CI failure preserved: [general server shard
1](https://github.com/paperclipai/paperclip/actions/runs/39704932041/job/119582710174)
failed the unchanged workspace-runtime test that replaces an unhealthy
local process. The replacement refused connections for its original
10-second readiness budget. The fixture and runtime service are
unchanged and do not import tool-access. One isolated run passed without
changes or timeout increases. The failure log is preserved; the cause is
not reproduced. Only that failed job was retried; it passed at the same
head without code changes. The original failure remains recorded above.

## Risks

- Moving profile mutations could change transaction clients or lock
order. Both are unchanged and the existing database tests cover
publication and assignment behavior.
- Effective assignment resolution still uses the same
installed-connection loader and full catalog projection. The lightweight
path still omits large schema and metadata fields.
- No schema, public API or access policy change is intended. Existing
tests provide characterization coverage; no new test mirrors the
extraction.

## Model Used

OpenAI Codex, GPT-6 family, with repository tools and code execution.
The exact serving model ID and context-window size are not exposed in
this session.

## Checklist


- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-21 22:21:42 -05:00
DottaandPaperclip 4413bb262f refactor(tools): isolate OAuth protocol boundaries (#17147)
## Thinking Path

> - Paperclip manages AI agents and their access to external tools.
> - Tool connections use OAuth to obtain credentials from remote
providers.
> - The tool-access service mixed provider protocol parsing with
database and secret writes.
> - This made protocol changes hard to review without reading the full
coordinator.
> - This PR gives discovery, endpoint policy, and client exchange
explicit owners.
> - The coordinator keeps authorization, leases, persistence, and audit
decisions.
> - A real local provider fixture checks the boundary through public
routes.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The ownership of OAuth discovery, registration, token exchange, and
client metadata.

**Current behavior**

The tool-access service contains over 20,000 lines. Provider responses,
endpoint policy, and stateful operations share the same closure.

**Proposed behavior**

Small protocol modules accept settings and named request capabilities.
They return normalized results. The service owns durable state and
existing public exports. This is part 3 of the stack: #17144, then
#17146, then this PR. It targets #17146's branch. Review only this PR's
diff.

## What Changed

- Extract endpoint policy, metadata parsing/discovery,
registration/token exchange, provider error normalization, and the
public client metadata document.
- Validate remote registration responses in the protocol module. Return
client material to the coordinator for secret and connection writes.
- Remove the unused raw provider response from the internal token
result. Keep every consumed field.
- Preserve process-wide registration single-flight, service-local maps,
refresh leases, membership checks, callback checks, and transaction
ownership.
- Reuse connection-method URL normalization. Keep the endpoint setting
and clock callbacks live.
- Add a real local HTTP OAuth/MCP route fixture for public and
Basic-authenticated clients.
- Reduce `tool-access.ts` from 20,917 lines / 808,777 bytes to 19,293
lines / 749,158 bytes. The seven OAuth modules total 1,697 lines; the
largest has 582 lines. The stack removes 2,232 lines from the original
service.
- Record the next profile-resolution and token-broker boundaries in the
dated plan.

## Verification

- Baseline: 466 service cases and 80 protocol/provider-policy cases
pass. The new two-case HTTP fixture passes before moving production
code.
- After: `pnpm exec vitest run
server/src/__tests__/tool-access-oauth-protocol.integration.test.ts
server/src/__tests__/tool-access-service.test.ts
server/src/__tests__/remote-mcp-connectors.test.ts
server/src/__tests__/remote-http-rebinding.test.ts
server/src/__tests__/oauth-client-metadata.test.ts` — 508 tests pass
with one worker and disposable PostgreSQL.
- The fixture drives connect, metadata discovery, DCR, PKCE code
exchange, callback, catalog review, and finish. It checks persisted
active state, secret references, issuer/auth-method binding, and
wire-level authentication.
- `pnpm -r --workspace-concurrency=2 typecheck` passes. The sandbox-only
attempt could not open a local IPC socket; the same command passed with
local process permissions.
- `node scripts/check-module-boundaries.mjs` and `git diff --check`
pass.
- AST comparison confirms 37 moved bodies are unchanged. Registration
protocol and persistence statements match their original halves. Token
exchange only drops its unused raw result. Preflight uses explicit
defaults and a clock callback.
- Initial fixture authoring used a nonexistent catalog enabled field;
the corrected assertion checks persisted status and reviewer identity.
An initial typecheck found optional defaults at the new boundary; the
caller now passes its already-validated defaults.
- Final-stack local verification: `pnpm -r --workspace-concurrency=2
typecheck` and `pnpm build` pass. Remote CI is green at
`3dc46c132408556ecb31038a9b9621ac20aacefc`. Greptile is 5/5 with no
actionable findings or open inline threads.

- Full local-suite limitation: `pnpm test:run` was interrupted after 79
minutes during the serial general-server phase, after CI had completed
the full test matrix. Before interruption, it flagged the unchanged
`company-search-service.test.ts` suite (26 database cases skipped). One
isolated rerun passed all 28 cases without code changes. The original
log and a pre-interruption copy are preserved. The original failure
cause was not reproduced. The full local suite and its later groups are
not claimed as passed; the complete CI matrix is green on this stack.

## Risks

- OAuth endpoint checks and issuer/resource binding are security
boundaries. Existing denial, redirect, scope, DCR, secret-redaction, and
refresh tests cover the preserved paths.
- A moved closure could capture stale policy or split
registration/refresh ownership. The new modules hold no mutable state.
Policy is read through the existing callback, and state stays in the
service.
- The local fixture proves real HTTP and database behavior. It does not
claim a live third-party provider sign-in.
- No schema, API, authorization policy, or persisted-data migration
changes are intended.

## Model Used

OpenAI Codex, GPT-6 family, with code execution and repository tools.
The exact serving model ID and context-window size are not exposed in
this session.

## Checklist


- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-21 22:21:41 -05:00
DottaandPaperclip 8e95caf9ae refactor(tools): isolate connection method projection (#17146)
## Thinking Path

> - Paperclip manages AI agents and their access to external tools.
> - Connection methods define operator settings and managed tool
arguments.
> - Setup, catalog discovery, and gateway invocation must use the same
projection rules.
> - These rules were embedded in a large database service.
> - This PR gives them one module with a narrow settings contract.
> - Existing provider policy modules still decide their
provider-specific restrictions.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Connection method configuration and tool input projection.

**Current behavior**

The tool-access database service owns settings validation and pure
input/schema transforms. Callers import these through the service.

**Proposed behavior**

A connection-method module owns these transforms and accepts only
connection settings. Existing imports remain compatible. This PR depends
on #17144 and targets its branch. Review only this PR's diff. It is part
2 of the three-PR stack. #17147 follows this PR.

## What Changed

- Move method selection, settings validation, and header/schema/argument
projection into `tool-access/connection-methods.ts`.
- Replace database row parameter types with required config and
transport settings. No new validation is added inside typed calls.
- Keep Honcho, Google Chat, and Gmail policy in their existing modules.
Preserve their call order.
- Retain the service's public exports for routes, gateway consumers, and
tests.
- Reduce `tool-access.ts` from 21,237 lines / 820,116 bytes to 20,917
lines / 808,777 bytes. The new module is 359 lines / 12,665 bytes.

## Verification

- Baseline: all 466 tool-access service tests and 80
protocol/provider-policy cases pass before the move.
- After: `pnpm exec vitest run
server/src/__tests__/tool-access-service.test.ts
server/src/__tests__/memory-connectors.test.ts
server/src/services/google-chat-tool-policy.test.ts
server/src/services/gmail-tool-governance.test.ts` — 506 tests pass with
one worker and disposable PostgreSQL.
- `pnpm --filter @paperclipai/server exec tsc --noEmit` passes after the
existing workspace prerequisites were built.
- AST body comparison confirms the 11 moved functions and private record
parser are unchanged. `git diff --check` passes.
- The first typecheck found optional config in the public DTO. The new
input type now states the existing required-config invariant directly.
- Final-stack local verification: `pnpm -r --workspace-concurrency=2
typecheck` and `pnpm build` pass. Greptile is 5/5 at
`2d0bb86b5830c0306580e3ddc8637c4994b2a75a` with no actionable findings.
- CI passed all non-browser gates. [The first browser
attempt](https://github.com/paperclipai/paperclip/actions/runs/39675461620/job/119484152569)
failed in the unchanged Cases read-only test: the status filter remained
All statuses while the announcement-dismiss locator handler interrupted
the assertion. The same test passed on #17144. The failure log is
preserved. One retry of only the failed jobs passed without code
changes; all CI gates are now green on the same head.

- Full local-suite limitation: `pnpm test:run` was interrupted after 79
minutes during the serial general-server phase, after CI had completed
the full test matrix. Before interruption, it flagged the unchanged
`company-search-service.test.ts` suite (26 database cases skipped). One
isolated rerun passed all 28 cases without code changes. The original
log and a pre-interruption copy are preserved. The original failure
cause was not reproduced. The full local suite and its later groups are
not claimed as passed; the complete CI matrix is green on this stack.

## Risks

- A projection-order change could alter managed arguments or advertised
schemas. Function bodies and call order are unchanged, and tests
exercise setup, discovery, and gateway consumers.
- Legacy Gmail method aliases, GitHub toolset headers, missing unrelated
settings, and disabled method filtering retain their previous behavior.
- The module has no database dependency, new network work, or mutable
state.

## Model Used

OpenAI Codex, GPT-6 family, with code execution and repository tools.
The exact serving model ID and context-window size are not exposed in
this session.

## Checklist


- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-21 22:21:41 -05:00
DottaandPaperclip 9db3a5726e refactor(tools): isolate GitHub access discovery (#17144)
## Thinking Path

> - Paperclip manages AI agents and their access to external tools.
> - Tool connections discover repositories through GitHub account
grants.
> - The tool-access service mixes GitHub response parsing with grant
authorization and persistence.
> - These responsibilities have different inputs and failure rules.
> - This PR puts GitHub discovery in one provider module.
> - The service keeps current authority checks and stored grant updates.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The ownership and review of GitHub access discovery in the tool-access
service.

**Current behavior**

The 21,525-line service contains GitHub pagination, account metadata
validation, and grant lifecycle code.

**Proposed behavior**

A GitHub module returns normalized access metadata. Existing service
exports remain compatible. This is the first PR in a three-part refactor
stack. No behavior change is intended. A search found no existing
tool-access refactor PR or issue.

Stack order: this PR, #17146, then #17147. Each PR targets its
predecessor.

## What Changed

- Move six functions into `tool-access/github-access-discovery.ts`.
Their bodies are unchanged.
- Keep grant selection, credential resolution, refresh retry, membership
locks, revision checks, and webhook updates in the service.
- Add public-entry characterization tests for HTTP error classification
and invalid pagination.
- Record five candidate boundaries and the initial three-PR scope in the
dated plan.
- Reduce `tool-access.ts` from 21,525 lines / 829,519 bytes to 21,237
lines / 820,116 bytes. The new module is 294 lines / 9,722 bytes.

## Verification

- Before and after: `pnpm exec vitest run
server/src/__tests__/github-grant-metadata.test.ts
server/src/__tests__/project-repositories.test.ts` — 13 tests pass.
- Before and after: `pnpm exec vitest run
server/src/__tests__/tool-access-service.test.ts -t 'GitHub|github'` —
11 selected tests pass with disposable PostgreSQL; 455 cases are outside
this selection.
- `pnpm --filter @paperclipai/server typecheck` passes.
- Exact text comparison confirms all six moved bodies match the base.
`git diff --check` passes.
- The initial baseline attempt failed because the plugin SDK had not
been built. The normal workspace prerequisite build resolved it before
behavioral validation.
- Final-stack local verification: `pnpm -r --workspace-concurrency=2
typecheck` and `pnpm build` pass. CI is green at
`07e78dd1b1c26e0bd4234006766444c71e234451`. Greptile is 5/5 with no
actionable findings.

- Full local-suite limitation: `pnpm test:run` was interrupted after 79
minutes during the serial general-server phase, after CI had completed
the full test matrix. Before interruption, it flagged the unchanged
`company-search-service.test.ts` suite (26 database cases skipped). One
isolated rerun passed all 28 cases without code changes. The original
log and a pre-interruption copy are preserved. The original failure
cause was not reproduced. The full local suite and its later groups are
not claimed as passed; the complete CI matrix is green on this stack.

## Risks

- Import wiring could break existing consumers. Existing imports remain
supported and all four production call sites are unchanged.
- Provider requests retain their ordering, timeout, fixed GitHub origin,
and response redaction. The new module has no database dependency or
shared mutable state.
- Tests simulate GitHub responses. They do not qualify live GitHub
access.

## Model Used

OpenAI Codex, GPT-6 family, with code execution and repository tools.
The exact serving model ID and context-window size are not exposed in
this session.

## Checklist


- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-21 22:21:40 -05:00
DottaandPaperclip 92cfd701c3 feat: add governed Braintrust app connections (#17141)
Add hosted US and EU Braintrust OAuth and API-key connections through the existing governed MCP system.

Keep reviewed tool risks and catalog review policy in connector modules and a narrow registry. Preserve credential ownership, operator decisions, regional setup and changed-tool quarantine. Include official artwork, setup guidance, focused regressions and sanitized qualification evidence.

US OAuth has bounded live read/write, gateway, reconnect and cleanup proof. Remaining account-method and token-lifecycle qualification gaps are documented.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-21 16:19:58 -05:00
Devin Foley a1e2173ceb feat(apps): add Ditto connection (#16472)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents reach external services through the Apps catalog. Each
catalog entry is a reviewed `AppDefinition` that connects a provider's
hosted MCP server to Paperclip's shared vault, grants, policies,
gateway, and audit trail.
> - Ditto (`dittowords.com`) is a single source of truth for product
copy. Its hosted MCP server lets agents read style guide rules, search
and reuse approved text, and create, edit, translate and delete text,
components, variants and style guides.
> - Ditto is not in the catalog. Operators must use the generic "Connect
your own MCP server" flow. That flow has no branding, no guidance about
Ditto's `token` API-key scheme, and no warning about permanent deletes.
> - The generic risk classifier also reads several Ditto mutations as
reads: `translate_*` replaces variant text, and `link_*` and `unlink_*`
replace developer IDs.
> - This pull request adds the Ditto definition, its artwork, the
research and permission-review ledger rows, documentation, deterministic
tests, and one reviewed risk-classification rule.
> - The benefit is a branded, governed Ditto connection with browser
sign-in, an API-key option that uses the correct header, and correct
risk levels for every documented tool.

## Linked Issues or Issue Description

**Problem or motivation**

Product and engineering teams use Ditto to keep UI copy consistent. They
want their Paperclip agents to follow the Ditto style guide and reuse
approved strings when they write code. Ditto is not in the Apps catalog.
Operators must paste the MCP URL into the generic remote-MCP flow. That
flow gives no branding and no method guidance. It also does not tell the
operator that Ditto API keys use `Authorization: token <key>`, not
`Bearer`, and it does not warn that delete tools are permanent. The
generic classifier also marks five Ditto mutations as reads, so the
Permissions screen shows them with the wrong risk level.

**Proposed solution**

Add a catalog-only Ditto connection built from the connector playbook.
Browser sign-in uses dynamic client registration on Ditto's Auth0
authorization server and requests the reviewed scopes `openid`,
`profile`, `email` and `offline_access`. An optional method sends a
customer-created workspace API key with the `token` scheme. Both methods
share one `write` capability profile, so the delete and link warning
shows before sign-in. Add a reviewed `ditto` rule to `classifyRisk`:
`list_*`, `get_*` and `search_*` tools, `suggest_edit`, and tools that
Ditto marks read-only are reads; `delete_*` tools are destructive; all
other tools are writes.

**Alternatives considered**

A plugin was not needed because no custom UI, tables, workers, or
webhooks are involved. Requesting no scopes was not chosen, because
Auth0 issues a refresh token only for `offline_access`, so the
connection would expire with its first access token. Ditto's separate
documentation MCP server (`developer.dittowords.com/mcp`) was not added,
because it only searches Ditto's public docs.

**Roadmap alignment**

This extends the existing self-serve remote-MCP connection catalog and
does not overlap planned core work.

## What Changed

- Added the `ditto` provider to `scripts/ingest-app-definitions.mjs`
(category `content`, `token` API-key placement, guidance, warnings,
shared capability profile, description) and regenerated
`packages/shared/src/app-definitions/ditto.json` and the generated
registry.
- Added the Ditto row to the self-serve MCP research ledger with
`dcr_or_api_key` auth and risk tier S4, because the normal tool catalog
includes permanent deletes. Widened `SelfServeMcpResearchEntry.wave` to
include `4`, which existing entries already use.
- Added permission reviews for `ditto/mcp-oauth` (explicit scopes
`openid`, `profile`, `email`, `offline_access`) and `ditto/mcp-api-key`
(provider key), with evidence links.
- Added a reviewed `ditto` rule to `classifyRisk` in
`server/src/services/tool-access.ts`.
- Added Ditto's mark (`ui/public/brands/apps/ditto.png`, the avatar of
Ditto's official GitHub organization) and the brand manifest entry.
- Added gallery copy for the Ditto card.
- Added `doc/connections/DITTO.md` (sign-in sequence diagram, provider
and Paperclip endpoints including `/api/tools/oauth/callback`, scopes,
administrator setup, capabilities and policy, manifest, brand
provenance, validation hook) and linked it from the connections README
and the permission audit.
- Tests: Ditto definition shape, store visibility and artwork, URL
recognition, reviewed scopes, `token` header projection with the key
kept out of connection config, risk classification of every read-looking
Ditto mutation, the connect form's sign-in default, the warning before
sign-in, API-key gating, and the pinned catalog counts.

## Verification

- `pnpm exec vitest run packages/shared/src/app-definitions.test.ts
packages/shared/src/app-definitions-url.test.ts
ui/src/pages/apps/AppsConnect.test.tsx ui/src/pages/apps/Browse.test.tsx
ui/src/lib/app-brand-assets.test.ts
ui/src/pages/apps/AppLogo.brand-assets.test.tsx
server/src/__tests__/tool-access-service.test.ts` — 765 passed.
- `node scripts/check-app-brand-assets.mjs` and `node --test
scripts/app-brand-validation.test.mjs` — passed.
- `pnpm --filter @paperclipai/shared typecheck`, `pnpm --filter
@paperclipai/server typecheck`, `pnpm --filter @paperclipai/ui
typecheck` — clean.
- Manual: on a local instance, open Apps → Browse and confirm the Ditto
card and icon. Open `/apps/connect?source=ditto` and confirm that
sign-in is the default, that the warning shows before sign-in, and that
the API-key method is under Advanced.
- Live metadata probe: an unauthenticated `initialize` on
`https://api.dittowords.com/v2/mcp` returns 401 with
`resource_metadata`. The protected-resource and authorization-server
documents return the recorded endpoints. Dynamic client registration
succeeds, and the authorize endpoint redirects to Ditto's sign-in page
with the reviewed scopes and the `resource` parameter.
- Each of the 41 documented tool names was run through `classifyRisk`
with and without the new rule. Without the rule, `translate_text_items`,
`translate_library_components`, `link_figma_nodes_to_text_item`,
`link_text_items_to_component` and `unlink_text_items_from_component`
were reads.

## Risks

- Low risk to existing providers: the catalog change is additive, and
the new `classifyRisk` branch runs only for the `ditto` source template.
- Ditto delete tools are permanent, and linking a text item to a
component replaces its developer ID. Both methods warn the operator
before connecting, and the API-key helper text tells operators to set
delete and link actions to Ask first.
- Ditto API keys are not scoped, reach the whole workspace, and act as
the user who created them. Paperclip cannot narrow an issued key.
- The permission-review ledger records live proof for both methods as
not run. The lifecycle checklist in `doc/connections/DITTO.md` still
needs a documented pass with a Ditto workspace, including confirmation
that Auth0 issues a refresh token that `/v2/mcp` accepts.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`, 1M context) in Claude Code, with
tool use (shell, file editing, web research).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-10-21 11:29:48 -07:00
DottaandPaperclip 3ce8d976a9 Add experimental Paperclip Notebook MCP connection (#16984)
## Thinking Path

> - Paperclip connects agents to external tools through saved accounts
and per-action permissions.
> - Notebook exposes artifact management through an ordinary remote MCP
server with OAuth.
> - A private pilot needs a hidden, default-off catalog and execution
gate.
> - Local file publishing belongs to Notebook, which can accept bounded
file bytes through ordinary MCP tools.
> - Paperclip therefore needs the connector registration and generic
availability checks, with no file-transfer architecture.

## Linked Issues or Issue Description

**Subsystem affected**

Apps catalog, experimental feature metadata, connection setup, and
connector availability.

**Problem or motivation**

Users need to connect Notebook as their Cloud account, assign its tools
to agents, and retain Paperclip's normal Allowed / Ask first / Off
rules. The private pilot must remain hidden until an operator enables
it. Notebook should own its upload protocol and local publishing script.

**Proposed solution**

Register a personal remote MCP OAuth connection at Notebook's canonical
endpoint. Reuse existing OAuth discovery, credential storage, refresh,
reconnect, and removal. Enforce availability through generic catalog
feature metadata and saved application identity. Keep Notebook's
standalone uploader in its own service repository; it calls the ordinary
gateway and receives no saved OAuth credential.

**Alternatives considered**

A gallery-only filter permits retained tools and deep links while
disabled. A shared workspace publisher and private upload-ticket
transport add runtime behavior that a normal MCP connector does not
require. Notebook can receive file bytes directly through its own tools
instead.

**Roadmap alignment**

This extends Connected Apps and supports Artifacts and Work Products.
ROADMAP.md and related PRs were checked. This PR is standalone on
master. The separate OAuth lifecycle and file-transfer proposals (#17085
and #17086) are closed and are not dependencies.

## What Changed

- Register Notebook's personal MCP OAuth method, canonical endpoint,
scopes, catalog metadata, and branding.
- Declare managed `enablePaperclipNotebook` with false defaults and no
user-facing settings toggle or staging opt-in.
- Use generic availability metadata for gallery/setup, retained
connection identity, OAuth completion, health, tool/context discovery,
and dispatch. Resources/prompts recheck before credentials, including
retained handles. Removal remains available while disabled.
- Resolve catalog application identity from the saved application key,
so editable connection labels cannot bypass the hidden gate.
- Select a method's sole supported credential kind when no kind is
supplied.
- Keep UI changes to metadata-driven availability and existing
setup/reconnect controls.
- Document Notebook's service-owned upload script, inline publication,
ordinary action policies, and credential behavior.
- Add ordinary MCP OAuth, assignment, action-policy, disabled-state,
settings, and catalog tests.

No workspace contribution registry, private transfer endpoint, native
publisher, Runner file-reader change, or Paperclip CLI upload command is
included. The entire native-runtime and CLI trees match master. Upload
code lives in `paperclip-cloud/notebook/`.

## Verification

Source head: `65ef741e4f476c67425b57d2fb2d6d4c00e99b06`, directly on
master `53732c811bb69655eb43d3d12f2063ae4ca5ad67`. The diff is 42 files.
Shared execution changes only enforce generic catalog availability; tool
access also resolves saved application identity and the sole supported
credential kind. Neither module contains Notebook upload logic.

- Catalog/shared/settings/UI suites: 592 tests passed across 12 files.
Shared/server/UI typechecks passed using installed compilers.
- Fresh real PostgreSQL generic MCP suite: 84/84 passed at final source;
server typecheck and module boundaries pass. Coverage includes personal
OAuth, ordinary refresh/reconnect, assignments, Allowed / Ask first /
Off, hidden feature enforcement, and local credential removal. Four new
resources/prompts cases failed against the prior code and pass after
filtering saved application identity and rechecking before credentials.
They prove missing/relabelled setup markers and stale eligibility cannot
dispatch or read secrets while disabled; an ordinary provider remains
available with the flag off.
- The full tool-access service suite passed 463/464 cases. One unchanged
Google Workspace `slides.write` case timed out at 15 seconds; it passed
on one isolated retry without source or timeout changes. Its exact test
block matches master. The cause is not established; final-head CI must
qualify the complete suite.
- Maps, module boundaries, token gates, and whitespace checks passed.
Byte comparisons confirm the former transfer/runtime/CLI surfaces match
master.
- Fresh exact-head Greptile is 5/5 with zero unresolved findings after
an exhaustive feedback audit. Current-head CI workflow
[39625691137](https://github.com/paperclipai/paperclip/actions/runs/39625691137)
succeeded, including both required checks (`ci / verify`, `ci / e2e`),
Build, and browser E2E. Supplementary security/advisory scan status is
recorded in the PR Checks panel and qualification receipt; prior-head
scans are not counted. The superseded `4bd7af9b2e` completed CI
successfully, but Greptile found the valid resources/prompts
availability gap described above. It was fixed, replied to, and
resolved; earlier-head results do not qualify this revision.

Preserved environment failures: the first socket-restricted UI/route run
had 93 listener setup failures, and the same 93 cases passed with
local-socket access. The package-manager launcher could not fetch its
signature metadata; installed compilers passed without dependency or
configuration changes. The full service timeout and isolated retry
remain recorded separately. The superseded `585e870e73` CI also failed
an unchanged agent-chat project-plan revision assertion after task
creation; the complete spec and relevant runtime sources match master,
and a same-master prior run passed. Its cause is not established. Logs
are preserved; no test or timeout was weakened and no old-head rerun was
requested. The subsequent heads run full CI independently; no prior-head
pass substitutes for current qualification.

The Notebook uploader requires a separate service release. This public
PR adds no uploader runtime to Paperclip and claims no newly deployed
browser/agent acceptance. The Notebook-owned upload path is qualified
separately. Code-owner review remains required before merge.

## Risks

- The feature flag gates the curated connector; it does not create a
Notebook service-side team allowlist or block arbitrary standard MCP
URLs.
- Publishing, sharing, and deletion retain the existing assignment and
action-policy boundaries.
- The hidden feature must be checked for retained tools and saved
identity, including when connection labels change.
- Disconnect removes local credentials. It does not revoke the remote
Notebook authorization; the owner can revoke it in Notebook.
- Ordinary refresh uses the existing client. A lost response after
refresh-token rotation can require reconnect.
- The standalone Notebook script and its bounded upload tools must ship
together in the Notebook service before that documented convenience path
is available.
- No database schema, workflow, lockfile, shared transfer, or Runner
change is included. No merge, deployment, or rollout is performed by
this update.

## Model Used

OpenAI GPT-6 in Codex, with reasoning, repository inspection, code
execution, and tool use. The runtime did not expose a more specific
model ID or context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-21 11:07:18 -05:00
DottaandPaperclip 6c9b87d092 trace(server): expose repository admission waits (#17040)
## Thinking Path

> - Paperclip manages work performed by AI agents.
> - The controller prepares a task workspace before it starts a provider
turn.
> - Repository admission also runs when a checkout already exists.
> - One observed reuse took 16.877 seconds. Its child spans explained
only 3.556 seconds.
> - The missing detail covered repeated authority checks, catalog work,
and publication waits.
> - This change exposes those operations under the existing execution
attempt.
> - Operators can expand the slow phase before choosing a performance
fix.

## Linked Issues or Issue Description

Refs #16963, #16330, and #16515. The provider-routing work in #17019 has
a different scope.

**What existing behavior does this improve?**

Repository-admission traces for new and reused checkouts.

**Subsystem affected**

server/ orchestration and operator-configured OpenTelemetry tracing.

**Current behavior**

The admission span can contain a long interval without child spans.
Existing project checks, HTTP requests, and checkout lookup do not
explain current-task checks, database awaits, credential refresh, or
publication locks.

**Proposed behavior**

Expose nested spans for current binding and task access, individual
identity reads, catalog construction, credentials, grant transactions,
filesystem validation, Git inspection, and directory-lock acquisition,
work, and release.

**Reason and benefit**

The trace can distinguish repeated checks, remote calls, database
awaits, and directory contention. A database-await span includes driver,
network, pool, and server time. It is not isolated SQL execution time.

**Breaking changes**

None. No API, schema, permission, retry, lock, or repository behavior
changes.

## What Changed

- Add fixed admission phases beneath the existing execution attempt.
Keep the existing project, checkout, and HTTP detail.
- Trace current-task authorization stages only under workspace access.
Unrelated decisions cannot consume that detail budget before admission.
- Trace GitHub catalog reads, credentials, provider metadata, grant row
locking and writes, transaction settlement, and webhook work.
- Separate directory-lock acquisition, the protected operation, and
release without changing the adapter lock contract.
- Document the hierarchy and timing limits. Keep endpoint gating, span
caps, and hashed run correlation. Add no span attributes or exception
text.

## Verification

- Nine new tests passed with real PostgreSQL, real Git, and SDK-exported
spans. They cover directory contention, grant-row blocking, all three
binding checks, reuse without cloning, dirty files, private-task denial,
receipt repair, conflict retry, endpoint-off behavior, and parent IDs.
- Twenty earlier unrelated authorization decisions do not exhaust the
admission detail budget.
- Existing authorization and trust suites: 105 tests passed. Existing
timing/context suites: 26 tests passed. Existing repository-access
tests: 6 tests passed.
- `pnpm -r typecheck` and `pnpm build` passed on commit
`5a986d8c86644dfb7ee39b3b832e2d7c86532598`.
- `pnpm test:run` could not complete locally. The temporary
five-connection cluster returned `sorry, too many clients already` in
chat fixtures, and the run was interrupted. It is not a pass. The
complete standard CI workflow passed on the same commit, including all
12 server shards and all three chat shards.
- Final current-head checks: 54 success, two neutral, two skipped, zero
pending or failing. The [CI
aggregate](https://github.com/paperclipai/paperclip/actions/runs/39522187940/job/118971261973)
passed.
[Greptile](https://github.com/paperclipai/paperclip/pull/17040#issuecomment-6305066520)
and [Paperclip
Review](https://github.com/paperclipai/paperclip/pull/17040#issuecomment-6305017182)
both assessed this exact commit at 5/5 with no actionable findings or
unresolved threads.
- `node scripts/check-module-boundaries.mjs` and `git diff --check`
passed.
- This macOS host exhausted its SysV semaphore quota. Database tests
used a temporary command-scoped initdb preload with five connections,
fewer workers, and a larger lock table. No repository test
infrastructure or shared host settings changed. Initial skipped tests,
focused-fixture failures, and clock-order assertion failures were
corrected and rerun; they are not counted as passes. The complete local
run remained connection-limited and was interrupted.
- Local SDK export proves parentage and privacy. Hosted Honeycomb
ingestion remains unverified until deployment. No deployment occurred.

## Risks

Span recording requires an operator-configured OTLP endpoint. Each fixed
detail name remains capped at 16 occurrences per attempt. Later
operations still execute under their enclosing phase. Timing wrappers
preserve the existing operations and their errors. No migration is
required. The change does not itself reduce admission latency.

## Model Used

OpenAI Codex, based on GPT-6, with tool use, code execution, and
delegated agents. The exact hosted model identifier and context-window
size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-20 19:50:51 -05:00
Devin FoleyandPaperclip 24758b9da1 Avoid loading tool schemas for effective assignments (#16969)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Tool profiles define which tools an agent can use.
> - Run preparation and connection checks resolve these assignments from
the database.
> - These callers need tool IDs and names, but the existing lookup also
reads complete schemas and provider metadata.
> - Repeated lookups transfer data that these callers discard.
> - This pull request adds a small assignment projection through the
same permission resolver.
> - The benefit is less database transfer and JSON work with current
permission checks and the same public response.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Effective tool assignment reads during run preparation, native runtime
setup, and connection policy checks.

**Subsystem affected**

server/ — tool access and orchestration services.

**Current behavior**

Internal callers use the public effective-profile lookup. Its catalog
query reads complete tool rows, including input schemas, output schemas,
annotations, and provider metadata. These callers only use catalog IDs
and names.

**Proposed behavior**

Internal callers use an assignment lookup that selects catalog identity
and selector fields. Both lookups share one resolver. The public
effective-profile endpoint keeps its complete definitions.

**Reason and benefit**

Large catalogs increase database transfer and JSON work for every
assignment lookup. The smaller query avoids that payload without caching
permissions or changing query frequency.

**Breaking changes**

None. No route, schema, migration, permission rule, or public response
changes.

Related: Refs #16761, Refs #16723. The former narrows profile summary
reads. The latter changes how the gateway presents tool definitions.
Neither narrows this internal effective-assignment lookup.

## What Changed

- Share effective-profile resolution between full definitions and
assignment records.
- Select the existing catalog identity and selector projection for
internal assignments.
- Use the assignment lookup in run preparation, native runtime setup,
and connection policy checks.
- Add real PostgreSQL tests for selectors, exclusions, current profile
and binding state, company boundaries, and returned payload size.
- Document the database read contract.

## Verification

- Five focused test files passed: 521 tests. They cover tool access,
connection intents, runtime preparation, native runtime context, and the
new assignment lookup.
- `pnpm -r typecheck` passed.
- `pnpm check:module-boundaries` passed.
- The PostgreSQL fixture compares the full authorization result with the
assignment result. It checks all selector types, excludes removed tools,
and observes catalog, profile, and binding changes on later calls.
- The payload test measures rows returned by PostgreSQL before
application mapping. With large schema and annotation fields, the
assignment result is less than 1% of the full catalog payload. This is a
synthetic fixture result, not a fleet savings estimate.
- `pnpm build` passed.
- The full local suite is in progress. It found two Git fixture failures
that also reproduce on unchanged master with `init.defaultBranch=main`.
Both pass with a process-only `init.defaultBranch=master` setting. The
global Git configuration was not changed. This earlier failure remains
recorded; it is not counted as a clean first pass.
- Apex review: 5/5, no actionable comments. All 54 CI checks passed; two
conditional jobs were skipped.
- The broader local run also reached two UI test timeouts and one
adapter test timeout. All three passed when rerun in isolation with
unchanged source. The exact cause of these timeouts is not established.
No UI or adapter implementation is changed by this PR. The CLI
import/export journey passes its assertions but its local teardown fails
with `ENOTEMPTY` while a runtime context asset is written. Its CI check
passed. The complete local suite is not reported as a clean pass.

## Risks

- Both lookups share permission resolution. Existing tool access tests
and result parity checks cover that extraction.
- Future assignment callers must use the fields in the smaller result.
Typechecking rejects attempts to read full schemas from it.
- The lookup still scans the active company catalog and resolves current
connections on every call. This change reduces returned bytes; it does
not reduce query count or introduce a cache.
- No database migration or operational configuration change is required.

## Model Used

- OpenAI GPT-6 through Codex, with tool-assisted source inspection,
implementation, and test execution. The exact model variant and context
window are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-20 08:42:59 -07:00
DottaandPaperclip 92b238137b Connect and configure GitHub Copilot through Apps (#15705)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Apps and AI Connections store credentials and govern access to agent
runtimes.
> - Users need a supported saved-token connection and config values that
survive save and reopen.
> - The release must preserve saved-token access and task lifecycle
guarantees.
> - This pull request will connect Copilot through the normal Apps and
AI Connections path.
> - The benefit is a reviewable part of the Copilot production release.

## Linked Issues or Issue Description

Refs #15701 and #15703. Supersedes setup work in #15562. Base: #15703.
Next: #15706.

## What Changed

- Require configuration update authority before Copilot
credential-bearing model discovery; default to the saved environment and
preserve explicit selection for editors.

- Keep Copilot discovery passive during reconnect reconciliation;
refresh on explicit opening, account/environment changes and reconnect.

- Add GitHub AI token connections with encrypted storage, ownership,
grants and audit logging.
- Keep repository access separate from Copilot runtime authentication.
- Probe the selected environment without sending a model prompt and
preserve authenticated model selection through save/reopen.
- Keep fresh execution admission consistent with normal Copilot setup;
persist task model choices without changing provider or permission
policy.
- Refresh authenticated models after reconnecting the same saved Copilot
account, including recovery from a previous authentication error.
- Apply the additive provider-default constraint migration through the
normal generated migration workflow.

## Verification

Runtime #15701 is merged at `d855e550ad8d2a267312983f26457aabf5393c3a`.
History #15703 is merged at `df80616d98119cbf5abd40549794d9980a5fabf6`.
Remaining source
[`42bcbd61299655f7e37aabe6bb5f19bba770ad1d`](https://github.com/paperclipai/paperclip/commit/42bcbd61299655f7e37aabe6bb5f19bba770ad1d)
contains the four-file authorization repair. The remaining PR trees
preserve six nonoverlapping files from the newer public mainline
managed-settings fix and the CI-owned lockfile. No unrelated provider is
enabled.

The security finding is repaired: existing-agent discovery requires
`agent_config:update` before connection selection or token resolution.
The saved environment is the default. Authorized editors can select a
prospective environment. Both denied board-viewer and agent-reader
regressions fail on the original source. All49 route tests and server
typecheck pass with the fix. Denied requests perform no connection
selection, credential resolution or probe. Current-head security scans
pass on both remaining PRs; the repaired thread is resolved.

Copilot CLI1.0.88 and profile47 retain digest
`sha256:55044aa235e88b3fe36ec8a556ee1e3382590870aab907e4462ef7d64af49cea`.
The [qualification
record](https://github.com/paperclipai/paperclip/blob/42bcbd61299655f7e37aabe6bb5f19bba770ad1d/doc/plans/2026-10-20-copilot-qualification-v47.json)
reports20/20 Product cells,154 assertions and7/7 protocol cases,46
assertions against frozen normal installed source
`c2616370800f5f6df45246b0890ba35a7cd2faab`. The exact qualification
model is `gpt-5.6-luna`. Every passing Product case confirms cleanup.
Original failures remain recorded; automatic behavior retries are
disabled. The route-only authorization repair changes no bound
runtime/profile bytes and does not relabel the frozen campaign.

The ordinary installed remote journey passed saved-token selection,
model discovery, normal UI save/reopen, metadata-only Test, completed
task, accepted receipt and exact19-byte browser download. An open model
catalog caused no additional discovery requests during117seconds. Actual
installed runtime smokes passed on macOS ARM64, macOS x64 under Rosetta
and native Linux x64 on Daytona. Raw token browser entry was not
exercised; encrypted connection creation used the normal API.

The first Daytona file attempt timed out during delayed semantic
dispatch; its precise stalled layer remains unresolved. Independent
ordinary remote delivery and the other19 Product cells passed before one
instrumented recheck passed8/8 with unchanged source,prompt,assertions
and deadlines. No causal latency repair is claimed.

Before the authorization repair, all four final PR heads passed required
verify/E2E and fresh Greptile5/5 reviews with zero unresolved threads.
Current-head CI/review is running for the two repaired heads. Local
focused checks retain129 connection,59 runner,72 shared-eval and88
recovery/workspace passes, plus migration safety and token gates. The
additive Copilot migration is361; released master history is preserved.

[Cloud normal-package
run39414239704](https://github.com/paperclipai/paperclip/actions/runs/39414239704)
builds the pre-security source4f3f. All four native producers and Linux
package assembly pass; the final clean installed consumers are
finishing. All18 retained archives match their hashes;37,863 regular
files contain zero selected credentials. That package does not contain
the later route repair. The actual final published package must include
it and pass the local/Daytona live canaries. The frozen Daytona image
remains published and independently verified. No separate full-image
scan is claimed. Docker builds run in the cloud.

**Remaining:** current-head CI/review, setup and qualification merges,
final publication, and ordinary local/Daytona canaries against the
actual shipped package. The user explicitly authorizes the maintainer
review override; CI remains a merge gate. No production deployment is
claimed. Provider USD remains unknown and GitHub-attributed.

## Risks

Bad credentials, missing entitlement, runtime installation and
unavailable models must remain distinct failures. A connection must not
adopt ambient credentials. The constraint migration changes accepted
provider values, with no credential-storage migration.

Current-master NewAgent fixture extraction is retained. Copilot cases
use that shared support and test both passive discovery and explicit
reconnect under the same setup lifecycle.

The onboarding retry regression shares the component suite’s
saved-connection, authenticated-model, draft, agent-list, hire and
render lifecycle fixtures. Keeping it beside the existing onboarding
failures makes adoption versus duplicate hire observable under the same
lifecycle; this is the concrete cohesion reason for the existing large
test file.

## Model Used

OpenAI Codex, based on GPT-6. The exact serving model ID and context
window are not exposed in this task. Used for code changes, review and
tool-assisted verification.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-20 07:02:22 -05:00
Nicky LeachandPaperclip 313fb265db feat(plugins): invalidate runtimes when connector access changes (#16605)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Runtime preparation depends on access and credential state as well
as tool definitions.
> - Grant, membership, secret, and endpoint changes need committed
invalidation.
> - Their writers must acquire locks before journal fanout can lock
agent rows.
> - This pull request adds access hooks with the corresponding
transaction guards.
> - Plugins can refresh preparation immediately while task-time
authorization remains authoritative.

## Linked Issues or Issue Description

**What happened?**

Grant changes, credential rotation and revocation, company access
changes, and connector endpoint configuration do not request runtime
synchronization. Journal fanout exposes lock inversions between
credential persistence, access cleanup, and connection assignment
writers.

**Expected behavior**

Committed access changes request runtime preparation without polling or
credential values in the journal. Agent-specific secret bindings and new
personal delegations affect their target agent. Company access changes
remain conservative. Provider bookkeeping does not create preparation
work.

**Steps to reproduce**

1. Change an audience, delegation, membership permission, referenced
credential, vault configuration, or endpoint instruction.
2. Expect a committed runtime event in the correct company and scope.
3. Run reconnect, credential refresh, SSH setup, or membership cleanup
concurrently with related writes.
4. Expect both operations to preserve access checks and finish without a
lock-order cycle.

**Additional context**

Final feature extraction from #16503. The earlier slices, #16604, and
the separate delivery-test correction #16871 are merged. This PR is now
the active change against master. The rebase preserves the merged
assignment and definition recipient rules, Unicode selector semantics,
and profile/reference transaction guards.

## What Changed

- Add migration 0356 for grants, audiences, delegation,
membership/permissions, secret bindings, referenced credentials and
vaults, and chat/email endpoint configuration.
- Keep events content-free. Ignore health, delivery, publication, Vercel
verification, and GitHub webhook bookkeeping while retaining credential
and repository-access changes.
- Target agent secret bindings and all personal delegation changes to
their old and new target agents. Coalesce delegation changes per
transaction.
- Lock existing grants before connections and credential persistence in
OAuth and cleanup writers.
- Reject superseded refresh leases before rotating stored credentials.
Ignore lease acquisition and release when access metadata is unchanged.
- Serialize audience replacement, validate membership, and preserve
unchanged audiences.
- Lock membership before permission-only saves and grant before Railway
SSH setup. Individual delegation revocation locks its grant and target.
Whole-grant and membership cleanup lock company recipients before
deleting scoped delegations.
- Lock grants and connection inputs before static-key reconnect writes.
Lock company agents before cascading secret deletion; use a nonblocking
secret lock and bounded retries to avoid a new cycle with credential
writers.
- Add PostgreSQL concurrency coverage for access writers and endpoint
publication/configuration behavior.
- Preserve the owner lock for plugin instance-secret removal; these
secrets cannot use ordinary bindings.
- Align Slack search refresh, callbacks, and disconnect with
grant-before-connection locking. Recheck newly discovered grants without
waiting before provider work.
- Retry complete membership transactions when a credential deletion
candidate is busy. Cover all four cleanup entry points and process
company access in ID order.
- Document final invalidation coverage and task-time authorization
boundaries.

## Verification

- Current head: `78aace55bef7f52bc5742b131e0b3961eb5c0f11`. All 1,061
tests across 23 affected suites pass. The named Slack authorization
integration scenario also passes; it exercises authorization, refresh,
disconnect, and rejection of a late callback. Server/database
typechecks, migration safety and numbering, five-snapshot retention,
feature maps, and diff checks pass. The complete PR diff was reviewed
for secrets and personal data before push. Final acceptance: [CI run
39308303837, attempt
2](https://github.com/paperclipai/paperclip/actions/runs/39308303837) is
green, with 55 successful checks and two optional Storybook skips. The
current-head Greptile reassessment returned **Apex 5/5** at 19:16 UTC on
2026-10-19. Its full summary reports no actionable issues. Security
passed, all threads are resolved, and GitHub reports no merge conflicts.
The reassessment was requested as a standard review after the fixes;
Greptile retained the Apex tier.
- On this head, [CI run
39308303837](https://github.com/paperclipai/paperclip/actions/runs/39308303837)
failed the unchanged adapter-utils HTTP/2 fixture with `EADDRINUSE`
before READY. Its closed probe left a port handoff race. That test and
its gateway are byte-identical on master. No upstream fix was found. The
deterministic fixture correction is isolated in #16888; no adapter test
changes are included here. The failed jobs recovered on retry without a
feature source change. The original failure remains documented here, and
#16888 carries the fixture correction separately. That retry
demonstrates recovery; the held-probe negative control establishes the
port handoff race.
- Rebased onto master `194fc11406`, including merged #16604 and #16871.
Replayed only the access-hook commit. Resolved OAuth, test, and
documentation conflicts while retaining both sets of tests. Migration
0356 preserves the merged assignment/definition recipient rules, Unicode
selector semantics, and profile/reference guards.
- Apex returned 3/5 on `0fb65bc27`. Negative controls reproduce its
Slack delegation and member-cleanup deadlocks (`40P01`) and unwanted
Vercel/GitHub bookkeeping events. Slack grant writers now take their
locks before the connection. A separate late-grant control reproduces
`40P01` without the post-lock recheck; the complete guard passes. All
four membership cleanup commands retry from a fresh transaction when the
credential is busy. Tests verify successful cleanup and no journal rows
from failed attempts. Credential IDs and repository-access changes still
notify runtimes.
- Earlier review corrections remain covered: static-key reconnect and
cascading secret removal each reproduce `40P01` without their guards.
Grant lease acquisition/release is ignored. Secret removal releases
agent locks when a credential writer is busy. Instance-secret removal
retains the owner lock and passes its public plugin API contention
regression. Delegation updates and revocations target old/new agents;
unrelated-peer, coalescing, unchanged-update, and rollback cases pass.
Three negative controls verify the required whole-grant and membership
cleanup guards after narrowing delegation scope.
- Preserve prior acceptance failures: the first rebased 20-suite run
passed 1,025/1,026; its only failure was a draft uninstall fixture
blocking a random peer. Blocking the assigned agent corrected that
assumption, and all 44 access/assignment tests passed. Later, all 1,044
tests across 21 suites, [CI run
39302185527](https://github.com/paperclipai/paperclip/actions/runs/39302185527),
standard Greptile 5/5, and security passed on `0fb65bc27`. That did not
clear the later Apex findings described above. Draft run 38975428081
failed a server shard; its cause was not re-investigated. A later green
run does not establish that earlier cause.
- Local fixture corrections are recorded separately from product
failures: an assignment filename filter initially matched no file; the
actual suite was included in full runs. A delegation fixture omitted its
required creator, and a guard run omitted a query import. The new
global-access cleanup fixture now uses its own user to avoid earlier
fixtures' memberships. All were corrected before publishing their final
iterations.
- Local checks target affected services and real PostgreSQL boundaries.
A full local build and repository-wide test run were not performed.
Hosted CI supplies broader verification. Provider responses in tests are
controlled; live provider behavior was not qualified.

## Risks

Migration 0356 adds triggers without changing table schema. Apply it
through the normal migration flow. Most access changes retain
synchronous company-wide invalidation. Company fanout takes O(eligible
company agents) locks and journal rows, without a fanout ceiling or
mutation quota. Agent secret bindings and individual delegation changes
retain narrower recipients. Events contain no credentials.

Physical removal of non-instance secrets takes ordered locks on all
company agents before cascading into bindings. This costs O(company
agents) locks even for an otherwise unreferenced secret. Instance
secrets cannot use ordinary bindings and retain their owner lock. A busy
secret lock releases the transaction guards and retries up to three
times. Continued contention returns a retryable conflict. Provider
deletion stays outside that transaction; a retry resumes removal of an
already disabled secret.

Membership cleanup uses nonblocking credential guards and retries the
full membership transaction. Slack search adds two grant queries, scoped
to the subject and connection. Its retries occur before provider work or
credential persistence. Each retry loop is limited to three attempts.
Invalidation requests preparation; task-time authorization and provider
session activation remain authoritative. Provider calls use fixtures; no
live provider qualification is claimed.

## Model Used

OpenAI Codex, GPT-6, with reasoning, repository inspection, code
execution, and testing. The exact deployment model ID and context window
are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-19 12:49:11 -07:00
Nicky LeachandPaperclip 194fc11406 feat(plugins): invalidate runtimes when tool definitions change (#16604)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Lifecycle plugins must refresh runtime preparation when tool
definitions change.
> - Standing assignments alone do not cover catalogs, profile entries,
or policies.
> - New journal hooks must preserve the order used by setup
finalization.
> - This pull request adds tool-definition invalidation and its required
writer guards.
> - It leaves credentials and provider access changes for the next
independent capability.

## Linked Issues or Issue Description

**What happened?**

Catalog, profile-entry, policy, gateway, and connection-configuration
changes do not request agent runtime synchronization. A new
profile-entry trigger also exposes a deletion/finalization deadlock if
the child row is changed before its parent is locked.

**Expected behavior**

Committed tool-definition changes advance runtime revisions without
polling. Bookkeeping does not. Entry changes and profile replacement use
a consistent parent/reference lock order and commit atomically.

**Steps to reproduce**

1. Change a catalog descriptor, profile entry, policy, or connection
configuration.
2. Expect committed runtime events for the affected agents.
3. Run profile-entry deletion concurrently with setup finalization while
both wait for an agent row.
4. Expect both operations to complete without PostgreSQL deadlock 40P01.

**Additional context**

Extracted from #16503. Catalog publication #16602 and standing
assignments #16603 are merged. This PR is rebased onto master and is the
active review. It addresses the original Apex profile-entry deletion
finding. Credential/access hooks #16605 remain a draft without review
requests.

## What Changed

- Add migration 0355 for application/connection configuration, catalog,
profile-entry, policy, and MCP gateway invalidation.
- Preserve transaction coalescing and ignore health bookkeeping and
trust-rule usage counters.
- Keep credential-only fields outside this migration's coverage.
- Preserve merged recipient scoping for bindings and installations.
Profile-entry edits resolve old/new standing profile bindings. Policy
edits resolve old/new agent selectors with their intersection/list
semantics.
- Lock old and new profile-entry references before parent profiles and
child mutations. Recheck the reference set under the parent lock, retry
changed snapshots up to three times, and merge unspecified fields from
the locked profile.
- Make entry add/edit/delete and profile replacement transactional,
including validation and profile reads on the reserved connection.
- Lock policy rows in ID order and change priorities in one statement so
scoped agent locks are ordered across the full set.
- Lock the managed profile before connection removal deletes its entries
in the same transaction.
- Update assignment tests whose old absence assertions are superseded by
expanded configuration coverage.
- Document the supported capability and the remaining credential/access
boundary.

## Verification

- Final head: `809fcb11794bfb12c545af0a2f96ab9bc2b570a2`. It is rebased
onto master with merged #16623. GitHub reports no merge conflicts. [CI
run
39293925136](https://github.com/paperclipai/paperclip/actions/runs/39293925136)
has 55 successful checks and two optional Storybook skips.
- Standard Greptile returned an independent 5/5. The final Apex review
returned 5/5 on this exact head at 17:47 UTC on 2026-10-19. Its summary
explicitly clears this PR for merge independently of draft #16605. The
security scan passed. All review threads are resolved.
- Two real-PostgreSQL negative controls reproduce `40P01` without the
old-reference guard or the concurrent-snapshot check. Both regression
tests pass with the final correction. Server typecheck, migration safety
and numbering, five-snapshot retention, journal history, feature maps,
and diff checks pass. The actual PR diff was scanned for secrets and
personal data before push.
- The final 15-suite local run passed 821/822 tests. The one failure was
in the unchanged runtime-delivery test: a startup callback changed its
global callback count during an independent transaction. The test-only
correction is isolated in #16871. That PR passes all 17 runtime-hook
tests, server types, hosted CI, and standard Greptile 5/5. Its
notification-disabled control fails at the independent event-delivery
assertion. No changes from #16871 are included here; this feature head
passes hosted CI independently.
- Earlier affected-suite runs passed 814 tests after rebase and 820
tests after recipient and whitespace corrections. Four affected suites
passed 514 tests before the second reference-contention regression was
added. Single-connection tests cover entry add, edit, delete, and
replacement. Validation and profile reads use the reserved transaction
connection.
- Review fixes have negative controls: the old code hangs with a
one-connection pool; parent-lock omissions deadlock entry deletion or
app removal against setup finalization; per-row policy reorder deadlocks
under contention; unscoped triggers journal unrelated peers; default SQL
trimming misses JavaScript whitespace. Corrected cases pass. Selector
tests cover all 25 ECMAScript whitespace characters. Recipient tests
cover repeated edits, old/new targets, ignored bindings, rollback,
coalescing, and company scope. The definition suite now has 28 cases.
- Earlier standard scores that depended on #16605 were not accepted as
independent clearance. The first Apex review returned 4/5 for
replacement versus application deletion. The final correction locks old
and new references, checks the reference set again under the parent
lock, and retries before any mutation. The final independent reviews
clear that correction.
- Preserve prior failures for acceptance: draft run 38975432017 had
failures whose causes were not re-investigated. Run 38990400765 exposed
the unrelated durable-resume watchdog race, fixed in merged #16623. In
run 39287657148, two jobs reported cancellation without a failed-test
summary; the cause remains unknown. Both passed on retry without source
changes. Initial local fixtures and SQL bindings also needed correction:
an internal helper name was replaced with public `archiveConnection`,
unchanged priorities were changed to exercise reorder, a driver
timestamp was bound as ISO text, and a superseded company-fanout
assertion was updated. These recovered checks do not establish a cause
for the unexplained CI failures.
- Local checks target the affected services and real database
boundaries. A full local build and repository-wide test run were not
performed. Hosted CI supplies the broader checks. Provider responses in
tests are controlled; live provider behavior was not qualified.

## Risks

Migration 0355 adds statement triggers without changing table schema.
Apply it through the normal migration flow. A tool-definition event can
advance a runtime revision without changing the bounded assignment
snapshot. Consumers still need authorized readers for definitions.
Credential-only changes, grants, secrets, and endpoints remain for the
access-hook migration. Profile replacement now rolls back as a unit if
any entry fails. Binding and install notifications retain scoped
recipients. Profile-entry changes use standing bindings, and policies
use old/new agent selectors. Unscoped policies, company-bound profiles,
and other conservative definition changes retain synchronous
transaction-atomic company-wide fanout: O(eligible company agents) locks
and journal storage, without a fanout ceiling or mutation quota. This
retains the accepted tradeoff from #16603 for company-wide changes.

## Model Used

OpenAI Codex, GPT-6, with reasoning, repository inspection, code
execution, and testing. The exact deployment model ID and context window
are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run local checks; results and the isolated test failure are
recorded above
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-19 10:56:08 -07:00
DottaandPaperclip 23b2ceb0d1 feat(connections): add unified Google Workspace connector (#15545)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Connected Apps supply governed tools to those agents.
> - Google services currently require separate connections or one broad
request.
> - Users need one account and control over which services it can
access.
> - A picker must change the OAuth request, not just the screen.
> - This pull request binds selected services to consent and tool
access.
> - The benefit is simpler setup without enabling unselected tools.

## Linked Issues or Issue Description

**Subsystem affected**

Connections catalog, OAuth grants, and the MCP tool gateway.

**Current behavior**

Google setup uses separate service profiles. A combined request alone
cannot express the user's choice before Google consent. Accumulated
Google grants can be broader than a new request.

**Proposed behavior**

Offer one Workspace connection. Let users choose Gmail, Drive (including
Docs, Sheets, and Slides), and Calendar. Start all three with read and
write access. Let users disable groups or choose read-only access. Do
not offer Chat, Contacts/directory, or cross-service Search in new
setup. Bind selected scopes to durable OAuth state. Cap effective tool
access to both the saved request and provider-reported consent.

**Reason and benefit**

Users can choose services without managing several accounts. Existing
company, identity, agent, and action-policy controls remain in force.

**Breaking changes**

No database migration. Existing service connections and grants remain
supported. New Workspace requests require the companion broker change in
https://github.com/paperclipai/paperclip-cloud/pull/823. Old brokers
reject narrowed requests safely.

Related work: #14740 reduced Google scopes; #14774 introduced the
earlier catalog hold. #15412 proposes a separate Gmail REST
implementation. This maintainer-requested change extends the existing
Connections and governed tool gateway paths. It does not replace them.

## What Changed

- Add one Workspace catalog entry and a namespaced action catalog.
Retain legacy definitions and saved accounts.
- Use the approved production service picker in Apps and task-hosted
setup. Keep dropdowns between names and toggles. Block empty selection.
- Derive exact scopes from a closed registry. Default selection requests
nine scopes. New requests exclude Chat, Contacts/directory, and Search
scopes.
- Bind selected scopes through managed and customer-owned OAuth state,
callback, claim redemption, and refresh.
- Cap accumulated Google grants to the saved selection. Recheck
grant-specific permission at discovery, listing, and dispatch, including
cached actions.
- Restore the current subject's saved request on reconnect. Block on
grant-load failure. Preserve draft selection on cancel and retry. Do not
create an account on picker cancellation.
- Preserve healthy services during partial outages, without combining
grants across identities. Keep writes from being automatically replayed.
- Preserve the original 21-scope legacy Workspace envelope contract as
individual profiles evolve.
- Add production-component Storybook stories, request validators,
consent regression tests, and rollout guidance. Integrate the stories
with the current feature catalog while preserving their IDs.

## Verification

- Integrated branch merges current master and has no merge conflicts.
- Full workspace build passed on the integrated checkout.
- Token gates passed. Storybook catalog structure check passed: 283
files and 2,750 stories.
- Final affected app suites: 914 passed. This includes 502
database-backed service/gateway tests and 412
UI/shared/validator/broker-handoff/governance tests. The production
picker tests cover defaults, identity-specific reconnect, and
failed-load recovery.
- Full workspace typecheck passed after adding required current-master
Storybook annotations. The final UI build also passed.
- The full repository suite is still running and includes the two
pre-fix retry assertions. Both were updated for the current gateway's
explicit-retry contract and pass in the clean 502-test database run. No
complete final-head full-suite pass is claimed.
- The initial local database startup failure came from missing
native-library symlinks in the fresh dependency install. The package's
postinstall repair fixed it; the clean database-backed run has no
skipped tests.
- Broker focused suite: 47 passed, including real signed session →
callback → claim redemption and wrong-scope rejection. Full broker suite
before the final test-only follow-up: 3,562 passed, 103 skipped.
- Real integrated-app walkthrough: Connectors → Workspace →
customer-owned app → service picker. Verified all three defaults,
empty-selection blocking, cancel/reopen with the selected read-only
access preserved, preserved form values, and zero saved connections
after cancel. Credentials were synthetic; no Google grant was made.
- Current-head CI and Greptile review remain pending. Previous-head
results are not treated as current.

## Risks

Ship the companion broker before app instances send narrowed requests.
Enable the intended Workspace client/profile separately on the target
broker. This pull request does not deploy a tenant app, start a fleet
rollout, alter Google Console, or upgrade the pinned reviewer stack.

The three-group picker is not proof of live provider access. Before
production, verify a real selected-service grant, reconnect, refresh,
allowed reads/writes, and denied unselected actions on the isolated
staging reviewer app. Google verification and preview enrollment remain
separate requirements.

Legacy service profiles stay compatible. New grant metadata keeps
provider scope evidence separate from requested scope evidence. Missing
or unsupported evidence fails closed. No scope union across user or
organization grants is allowed.

## Model Used

OpenAI Codex, GPT-5-based coding agent, with repository editing, code
execution, and browser tools. The exact serving model ID and
context-window size were not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change and contains no internal issue
ID
- [ ] I have run all final-head tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-19 12:07:30 -05:00
Devin FoleyandPaperclip 5dced3c766 perf(tools): narrow profile and attention catalog reads (#16761)
Read only the catalog, connection, and invocation fields needed by profile summaries, reviews, and attention counts. Preserve company scope and response contents while reducing query payloads.

Verified with 445 affected service tests, workspace typecheck/build, full CI, and Greptile Apex 5/5 on the reviewed head. Local full-suite limitations and isolated rechecks are documented in the PR.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-18 13:56:04 -07:00
Nicky LeachandPaperclip 5f5602bba7 feat(plugins): synchronize standing connection assignments (#16603)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Lifecycle plugins prepare agent runtimes outside the control plane.
> - They need committed connection assignment changes without polling.
> - A reader and its revision hooks must cover the same fields.
> - This pull request adds a bounded standing-assignment snapshot and
its matching hooks.
> - Plugins can react to assignment changes while execution keeps its
existing access checks.

## Linked Issues or Issue Description

**What happened?**

The agent runtime reader does not expose standing connection
assignments. Changes to connection installs, active profiles, and
connector feature switches do not request synchronization. Adding
company-wide invalidation also exposes lock inversions in assignment
writers and AI adoption.

**Expected behavior**

The owning lifecycle plugin can read a bounded assignment snapshot.
Every committed change to its stored fields advances the runtime
revision. Rollback removes both the change and its event. Assignment
writers finish without a lock-order cycle.

**Steps to reproduce**

1. Install a connection or bind an active profile to an agent.
2. Read its runtime inputs through the owning lifecycle plugin.
3. Rename or disable the connection, revoke the assignment, or switch
connector features.
4. Expect a committed runtime event and an updated snapshot without
periodic polling.

**Additional context**

Extracted from #16503. Catalog publication #16602 and the separate test
maintenance PRs #16601 and #16610 are merged. This PR now targets master
and is the active review. Tool-definition hooks #16604 and
access/credential hooks #16605 remain drafts without review requests.

## What Changed

- Add SDK types and an owning-plugin runtime snapshot for installed
connections, active profile IDs, and effective connector feature
switches.
- Reject more than 512 connections or 128 active profiles instead of
returning partial data.
- Add migration 0354 for the exact stored fields used by the snapshot,
with transaction-coalesced invalidation. Agent-targeted bindings and
installs notify only their old/new target agents;
project/routine/issue/gateway bindings are outside the snapshot and
produce no notification.
- Use ordered agent locks that are compatible with foreign-key
references and exclude runtime readers. Prelock complete old/new target
sets before replacing assignments or restoring managed profiles.
- Record one company baseline at delivery startup for process-managed
feature overlays.
- Lock assignment inputs before install saves, task completion, and
email assignment. Setup finalization reads the locked connection state
and prelocks company agents before activating, re-enabling, or reviving
a profile.
- Make profile binding and reassignment writes transactional and lock
parent profiles first. Keep target validation and profile/detail reads
on the reserved transaction connection, including with a one-connection
pool.
- Lock company agents in order before validated AI adoption. Keep
adoption and shared configuration writes at `FOR NO KEY UPDATE`,
avoiding an upgrade that can deadlock foreign-key inserts during gateway
creation. Configuration fields cannot change agent identity; lifecycle
transitions and deletion retain their existing locks.
- Document supported coverage and the inputs reserved for later PRs.

## Verification

- Thirteen affected suites passed: 757 tests covering runtime
hooks/assignments/skills, concurrent skill publication, tool
access/catalog, task connection intents, lifecycle delivery, email, AI
accounts, migration histories, snapshot drift, and the agent lifecycle
module. This includes the merged catalog PR's OAuth and
overlapping-refresh regressions.
- The assignment suite passes all 24 cases, including bounded snapshots,
company/agent isolation, rollback, health-field no-ops, process-start
feature overlays, and real lock contention.
- Four one-connection PostgreSQL regressions reproduce the earlier
helper-read hangs before the correction; bind, unbind, delete, and
reassign complete with it. The earlier adoption/gateway probe exposed
both lock upgrades: changing only validation still deadlocks in shared
configuration. Current scoped coverage verifies adoption with
agent-targeted binding writes; gateway-only bindings no longer
invalidate this snapshot.
- The original ordered-adoption regression reproduces PostgreSQL `40P01`
without ordered company-agent locking. Scoped-recipient tests reproduce
unrelated-peer journaling before the trigger correction and verify
repeated bind/unbind operations leave peers untouched, ignored binding
types produce no events, both reassignment recipients are reached, and
coalescing remains intact.
- Six replacement contention cases cover installs, existing setup,
activation, re-enabling, profile revival, and profile reassignment.
Negative controls reproduce `40P01` without the corresponding
target/company prelocks; all cases pass with the guards. Activation
decisions use the connection row reread under lock.
- An initial contention fixture failed because its settings connection
lacked an MCP URL. Supplying the fixture URL and controlled lookup
corrected that test setup error.
- Server typecheck, database migration numbering/safety, five-snapshot
retention, feature-map structural validation, and `git diff --check`
passed. Migration metadata preserves historical journal entries and adds
no schema change. The complete diff was reviewed for secrets and
personal data before push.
- Earlier hosted PR CI runs 38983440861 and 38985530069 passed; the
superseded promotion run 38983440470 was cancelled. Superagent
acknowledged the recipient fix and accepted the documented company-wide
cost at `359b5b2f30a8a0248177367e458663d5379810d1`. Its two standard
Greptile summaries were conditional on a later setup correction and were
not counted as independent acceptance; the setup correction is now
included here.
- At `7da246aab46274db1a120a0546abbb404c5427c7`, hosted PR run
38986958988 passed: all 55 checks succeeded, with two optional Storybook
checks skipped. Superagent passed, all review threads are resolved, and
standard Greptile independently returned 5/5 with no blocking findings
(review completed 2026-10-17 00:40 UTC). Apex independently returned 5/5
with no actionable findings (completed 2026-10-17 00:50 UTC). CI was
rechecked after Apex and remains green; the branch is mergeable against
master. Full local build and repository-wide tests were not run.

## Risks

Migration 0354 adds database functions and triggers without changing
table schema. Apply it through the normal migration flow. Agent-targeted
bindings and installs only notify their affected agents. Explicit
company-wide assignments, connection metadata, profile
activation/removal, and connector feature switches retain synchronous,
transaction-atomic company fanout. Each such transaction takes ordered
row locks and inserts up to one connector runtime event per eligible
company agent: O(number of eligible agents) work and journal storage,
with no company-size cap or cross-transaction mutation quota. This
remains an intentional scope tradeoff for this PR; deferred fanout is
not implemented. Many of these conservative changes can request
preparation even if an individual snapshot stays unchanged. Snapshot
limits fail closed. This PR provides desired assignments, not
credentials or task authorization. Later PRs must add mutation coverage
before consumers can rely on immediate synchronization of tool
definitions or credentials.

## Model Used

OpenAI Codex, GPT-6, with reasoning, repository inspection, code
execution, and testing. The exact deployment model ID and context window
are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-16 17:55:47 -07:00
Nicky LeachandPaperclip 190080642e perf(tools): publish catalog refreshes in bounded chunks (#16602)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - MCP discovery publishes tools that setup and access profiles use.
> - Large catalogs currently write one row at a time.
> - Repeated page entries can produce duplicate results, and profile
writes can partly commit.
> - This pull request publishes tools in bounded chunks and makes
profile synchronization atomic.
> - The result supports large catalogs before connector runtime hooks
are enabled.

## Linked Issues or Issue Description

**What happened?**

Catalog refresh performs one database write per tool. Repeated tool
names across discovery pages appear more than once in the result.
Default-profile synchronization uses separate statements, so failure can
leave a profile without its binding. OAuth callbacks also acquire
connection/application locks in the reverse order from publication and
setup, allowing a concurrent refresh to deadlock.

**Expected behavior**

Publish at most 500 tools per catalog statement. Keep one result per
tool name, using its last descriptor. Preserve existing IDs, first-seen
timestamps, disabled actions, and unchanged review decisions. Commit or
roll back default-profile changes together. OAuth persistence must
acquire application then connection locks before secret or connection
writes.

**Steps to reproduce**

1. Refresh a synthetic MCP catalog with 2,500 tools.
2. Return the same tool on two discovery pages with different
descriptions.
3. Fail the profile-binding insert during default-profile creation.
4. Overlap personal, organization, or managed OAuth persistence with
setup finalization or discovery of a new tool.

**Additional context**

This is the first feature extracted from #16503. The separate test-only
maintenance PRs #16601 and #16610 are merged. This PR now targets master
and is the active review. Standing assignments #16603, tool definitions
#16604, and access/credential hooks #16605 remain stacked drafts without
review requests.

## What Changed

- Deduplicate discovery results by tool name and keep the last
descriptor.
- Upsert catalog rows in transactions of at most 500 tools.
- Preserve row identity and operator-disabled status on conflict.
Determine default-access candidates from the rows read under each
publication lock, so an overlapping refresh cannot restore an action
turned Off after another refresh finishes.
- Lock the application before the connection during publication, setup
finalization, removal, and all three OAuth persistence paths. The OAuth
ordering fix is included here so this PR is independently safe; it does
not depend on #16605.
- Synchronize the managed default profile, its entries, bindings, and
managed policies in one transaction.
- Add PostgreSQL publication tests and document partial-failure
recovery.
- Target the existing callback failure injection at profile creation
instead of a transaction ordinal.

## Verification

- `pnpm exec vitest run
server/src/__tests__/tool-catalog-publication.test.ts
server/src/__tests__/tool-access-service.test.ts`: 436 tests passed with
the OAuth and overlapping-refresh corrections on top of the merged
#16610 fix.
- The new tests exercise real PostgreSQL with 250 and 2,500 tools,
repeated discovery pages, preserved choices, lock contention, and
failures between chunks and during profile creation.
- Four selected regression cases fail with the production source
restored to the parent commit. The implementation was then restored and
rechecked.
- Six additional real-PostgreSQL regressions cover personal,
organization, and managed OAuth persistence against both setup
finalization and new-tool discovery. All six reproduce PostgreSQL
deadlock `40P01` with the OAuth ordering fix removed; all six pass with
it restored. Discovery uses synthetic credential headers to isolate
publication locking from credential rotation.
- Apex initially returned 3/5 on
`0d6013a9649388041975406fe4b0247cd0e695aa`, identifying an
overlapping-refresh permission race. Two new PostgreSQL regressions
cover both absent and quarantined tools; both prove an agent goes from
deny back to allow without the correction, and remains denied with it.
The test pauses one refresh after its initial catalog snapshot,
completes another refresh, turns the action Off through the service, and
resumes the stale refresh.
- `pnpm --filter @paperclipai/server typecheck`: passed.
- `node scripts/check-maps.mjs`: passed (structural validation only).
- `git diff --check`: passed.
- Reviewed the complete diff for secrets and personal data before push.
- Earlier hosted run 38975428064 failed the maximum-size newline
skill-audit watchdog; other interrupted jobs reported runner
shutdown/cancellation. The original watchdog failure did not identify
whether startup or auditing timed out, so its exact cause remains
unknown. The separate merged #16610 isolates the audit deadline from
module startup and preserves the audit hang guard. This branch is
rebased onto that fix.
- Full local build and repository-wide tests were not run. At the
previous head `0d6013a9649388041975406fe4b0247cd0e695aa`, all 55 hosted
checks succeeded (two optional Storybook checks skipped), and standard
Greptile returned 5/5 before Apex identified the permission race above.
At corrected head `dc120d2bc879c54c14347db30e74c753bcff61e6`, all 55
hosted checks succeeded (two optional Storybook checks skipped), and a
fresh standard review returned 5/5 with no open findings. The second
Apex review also returned 5/5 on that exact head (successful Apex check
completed 2026-10-16T23:36:44Z). Both earlier findings are addressed,
there are no open review threads, and the branch is mergeable against
master.

## Risks

Each successful catalog chunk remains committed if a later operation
fails. A retry completes publication. Provider discovery occurs outside
database locks. Default-profile changes commit separately from the
catalog chunks and roll back together on failure. Tests use a controlled
MCP response; they do not qualify a live provider. There is no schema
migration or new plugin API in this PR.

## Model Used

OpenAI Codex, GPT-6, with reasoning, repository inspection, code
execution, and testing. The exact deployment model ID and context window
are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-16 16:40:12 -07:00