Commit Graph
90 Commits
Author SHA1 Message Date
DottaandPaperclip c17ef9249d fix(comments): keep saved authorization cards from blocking replies (#21744)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - People use task comments to ask for help with pending requests.
> - Saving a comment also checks whether it replaces any pending
requests.
> - A stored request that fails today's payload schema can abort that
comment write.
> - The connection gateway could write such a request without validating
its generated target.
> - This change validates new authorization cards, reads older custom
identities, and keeps malformed requests from blocking comments.
> - Comments remain available while authorization and review gates
retain their authority.

## Linked Issues or Issue Description

**What happened?**

A connection authorization card can contain a generated custom target
key longer than 120 characters. The gateway writes the card directly.
Later, the interaction reader rejects it. Both the task interaction list
and comment submission then return a validation error. Other malformed
saved requests can also abort a comment write.

**Expected behavior**

People can ask questions on a task while authorization is pending. New
cards meet the shared schema. Older generated identities remain
readable. A malformed request never becomes approval because a person
posted a comment.

**Steps to reproduce**

1. Use a personal connection with a long UID and no grant for the
current user.
2. Invoke a tool. The gateway saves an authorization request with the
long UID in its target key.
3. Open the task or post a comment. Reading the saved payload fails the
key length limit.

Related PR searches found no existing repair for this defect. This is a
focused bug fix, with no roadmap feature overlap.

## What Changed

- Validate generated connection authorization payloads before either
insert or update. Use a stable SHA-256 target identity from the existing
connection/user idempotency key. Bound its display label.
- Read older confirmation custom keys and labels without the new-input
length limits. Preserve their exact identities. Keep target types, URLs,
document revisions, and resolver checks strict. New requests still use
the strict shared schema.
- During current and historical comment supersession, keep an invalid
saved request pending, report its ID and schema locations, and continue
with valid siblings. Database and other unexpected errors still
propagate.
- Keep this behavior in the gateway writer and interaction owner. The
two existing source files exceed 1,000 lines; the additions extend their
existing write/hydration boundaries rather than introduce another
ownership path.

## Verification

- Reproduced the deployed failure through the browser: comment POST and
interaction GET both returned HTTP 400 for `target.key` exceeding 120
characters. The person's draft was restored.
- Ran the new regressions before the repair: four expected failures and
one short-identity control pass. Failures came from real payload parsing
and real PostgreSQL comment writes, not fixture setup.
- After repair, all 257 tests passed in the complete affected suites:
`issue-thread-interactions-service.test.ts`, `tool-gateway.test.ts`, and
`issue-thread-interactions.test.ts`.
- Safety cases cover fresh and reused authorization cards, repeated
missing-grant calls, refusing another user's credentials, two persisted
follow-ups, strict new-input limits, wrong-company resolution, unsafe
URLs, invalid document revisions, malformed question and confirmation
payloads, and valid sibling supersession.
- Local browser acceptance used an isolated, credential-free source
instance on macOS arm64 with embedded PostgreSQL and Chrome. A card
shaped like an older saved long-key authorization request remained
visible. Two comments saved through Send, including a second after
reload. The original comment persisted and the authorization request
remained pending. No model execution was configured.
- Browser acceptance was repeated on final commit
`58b57dfe104b2ab421c7f24c89db4c29fcfda8ec` after rebase. The server
health and ownership check confirmed that exact serving revision. Both
comments persisted and the authorization stayed pending. Both owned
instances were then stopped. All 257 affected tests passed again after
rebase. The final fixture typing correction passed its six affected
PostgreSQL cases. Greptile reviewed final head
`58b57dfe104b2ab421c7f24c89db4c29fcfda8ec` at 5/5, with zero review
threads or open findings. Required CI is green on that head: aggregate
`ci / verify` and `ci / e2e` both passed. The repository CI run is
https://github.com/paperclipai/paperclip/actions/runs/43710675824 . All
55 completed checks passed; four inapplicable checks were skipped. The
non-required Superagent Security Scan remains in progress at 35%, with
no annotations or reported findings. This external scan is still
unverified; it was not retried or marked as passed.
- `git diff --check` and the module boundary check pass. All seven local
server typecheck layers pass on the final fixture revision. An earlier
run found fixture typing errors; those were corrected and rerun
successfully. Full repo tests, release qualification, and deployment
were not run.

## Risks

- Stored custom keys and labels may exceed current input limits. Only
those length limits change for saved confirmation payloads; unsafe links
and invalid document targets still fail validation.
- A malformed request stays pending and still needs repair. It does not
receive consent, disappear, or block a comment. Its read or resolution
can still return a validation error.
- This PR does not deploy the fix or recover a remote task. The original
environment remains affected until rollout.

## Model Used

OpenAI GPT-6-based Codex, with code editing, shell execution, browser
tools, and PostgreSQL-backed tests. The host does not expose a more
specific model version or context size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
(behavior and limits are recorded in this PR)
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-15 06:10:35 -06:00
DottaandClaude Opus 5.5 f6c5986b61 feat(connections): machine sign-in (MCP OAuth client credentials) for organization connections (#21473)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents use apps through connections to remote MCP servers
> - Today an organization connection must sign in as a person (browser
sign-in) or use a static key
> - Service accounts and CI-style integrations have no person who can
sign in
> - The MCP OAuth Client Credentials extension (2026-07-28) defines
machine-to-machine sign-in for this case
> - This pull request adds "Machine sign-in" for organization
connections, with a Paperclip-held signing key (recommended) or a client
secret
> - The benefit is that organizations can connect automation-owned MCP
servers safely, with credentials only in the vault and bound to one
sign-in service

## Linked Issues or Issue Description

**Subsystem affected**
Connections: remote MCP OAuth (server `tool-access/oauth`, tool gateway
token refresh) and the custom MCP server setup flow in the UI.

**Problem or motivation**
Organization-owned automation (service accounts, CI integrations) cannot
connect an MCP server that requires OAuth, because no person can
complete a browser sign-in. The MCP OAuth Client Credentials extension
(`io.modelcontextprotocol/oauth-client-credentials`) solves this with
`grant_type=client_credentials` and RFC 7523 signed client assertions.

**Proposed solution**
Add machine sign-in for organization connections only. Paperclip creates
a keypair per connection, keeps the private key in the company vault,
and shows the public key (PEM and JWK) for the admin to register. A
client secret is also supported. The sign-in service (issuer and token
endpoint) is pinned at setup. Tokens use the existing organization grant
refresh path, so caching, concurrent dedupe and 401 re-request reuse the
existing lease and single flight.

**Alternatives considered**
Reuse the deployment-wide `PAPERCLIP_TOOL_OAUTH_*` client env path that
already sent `client_credentials`. Rejected: it is not per connection,
it keeps secrets in env, and it has no issuer pin. Route machine sign-in
through `connectGalleryApp`. Rejected: its rollback deletes the draft on
a failed test, which would lose a key the admin already registered.

**Roadmap alignment**
Phase 1 of the enterprise MCP plan (no enterprise single sign-on
needed). OIDC login and enterprise-managed authorization (ID-JAG) are
later phases.

## What Changed

- `oauth/client-credentials.ts`: keypair generation (RS256, or ES256
when the service requires it), RFC 7638 key IDs, and RFC 7523 assertions
(`iss` = `sub` = client ID, `aud` = token endpoint, 60 s `exp`, unique
`jti`) with Node `crypto` only.
- `oauth/client.ts`: the token exchange accepts a signed assertion and
then sends only `client_assertion_type` and `client_assertion` (no
`client_id` or secret), as the extension specifies.
- `oauth/client-credentials-service.ts`: discovery of the sign-in
service (protected-resource metadata, then RFC 8414/OIDC metadata with
issuer check, or a token URL entered directly), issuer pinning,
vault-only key and secret handling, draft creation, test-and-activate,
and no-downtime rotation (test the new secret first; stage a new key and
switch only after the provider accepts it).
- `oauth/grant-refresh.ts` and `oauth/connection-refresh.ts`: machine
sign-in requests a new access token with the connection's own
credentials instead of needing a refresh token. It never falls back to
deployment-wide OAuth clients.
- `catalog/discovery.ts`: a rejected token on a machine sign-in
connection never re-runs discovery or rewrites the pinned endpoints.
- `mcp-http.ts` and the tool gateway: declare
`io.modelcontextprotocol/oauth-client-credentials` in client
capabilities only for connections that use it.
- Routes (with OpenAPI and activity logging): `POST
/companies/{companyId}/tools/machine-sign-in`, `GET|POST
/tool-connections/{id}/machine-sign-in[/test|/rotate|/rotate/complete]`.
Removal activity now records `machineSignIn: true`. The runner API
treats these routes as sensitive.
- Shared validators and types for machine sign-in.
- UI: "Machine sign-in" under Advanced authentication in the custom MCP
server flow (organization connections only), `MachineSignInSetup`
(choose Paperclip key or client secret, register the public key, test),
and Storybook stories for each setup state.
- Docs: `doc/connections/MCP-CLIENT-CREDENTIALS.md`, linked from
`doc/connections/README.md` and `GENERIC-REMOTE-MCP.md`.

## Verification

- `server/src/__tests__/tool-access-client-credentials.test.ts`: 16/16
pass. It uses a fake sign-in service that verifies real signatures
against registered keys, and a fake MCP server. It covers RS256 and
ES256 assertion claims and signatures, assertion-only client
authentication, the secret flow (`client_secret_basic`), discovery and
direct token URL, issuer pinning (refuses a different sign-in service
and sends it nothing), the pinned endpoint surviving a hostile
challenge, the token cache (0 token requests while live), concurrent
dedupe (5 callers on 2 service instances produce 1 token request), 401
re-request, key and secret rotation, a refused personal connection,
secret redaction in responses, rows and activity, and vault cleanup on
removal.
- Red/green: with the grant-refresh machine sign-in branch and the
discovery pin guard disabled, 5 runtime tests fail (pin, cache/dedupe,
401 re-request, both rotations). Restored: 16/16 pass. The other tests
cover new routes and services that do not exist on `master`.
- `ui/src/features/connections/MachineSignInSetup.test.tsx`: 3/3.
`ConnectionSetupFlow.architecture` and `.oauth` tests: 19/19.
- `openapi-routes.test.ts`: 51/51. `runner-api.test.ts`: 2153/2153.
`mcp-http.test.ts`: 9/9. Existing `tool-access-service.test.ts` refresh
and legacy client-credentials tests: 54/54 (filtered).
- Server package `tsc --noEmit`: clean. UI `tsc`: clean except the
pre-existing generated Storybook catalog imports.
- Not run locally (machine under heavy load): full suites, repo-wide
typecheck, browser check of the live app. CI covers breadth.

## Risks

- No migration. New config lives in `config.oauth` (`grantType:
"client_credentials"` plus `clientAuth`). Older env-configured
client-credentials connections have no `clientAuth` and keep their
original path.
- The capability declaration adds one extension key only for machine
sign-in connections. Other requests are unchanged.
- Rotation has no settings-page control yet; it is available through the
API. This is a follow-up.
- Low risk to agent boot and time to first token: a live token is read
exactly like a browser sign-in token.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edits, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 03:10:32 +00:00
DottaandClaude Opus 5.5 a726dce4f0 feat(mcp): show live progress for long connected-app tool calls in the task chat (#21449)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents call tools from connected apps through the Paperclip MCP
gateway
> - Some tool calls take a long time: exports, builds, large searches
> - The MCP progress utility lets an app report how far a call is, but
Paperclip ignored it
> - So the task chat showed only a spinner until the call finished, with
no sign of life
> - This pull request reads the app's progress and shows it on the
tool's row while the call runs
> - The benefit is that people can see that a long call is working, how
far it is, and how long it has run

## Linked Issues or Issue Description

**Subsystem affected**
Connected apps (MCP gateway) and the task chat.

**Problem or motivation**
A long tool call from a connected MCP app shows only a spinner in the
task chat. Paperclip does not send a `progressToken`, and it drops
`notifications/progress` from the app.

**Proposed solution**
Send a `progressToken` on each `tools/call` that an agent run makes.
Read the app's progress notifications from the call's response stream.
Show a progress bar (or an indeterminate sweep), the latest message, and
elapsed time on the running tool row. Keep the latest state so that a
reload mid-call shows it.

**Alternatives considered**
A new transport for progress. The existing company live-events socket
already carries per-run events with run read checks, so this change uses
it.

**Additional context**
Spec:
https://modelcontextprotocol.io/specification/2026-07-28/basic/utilities/progress

## What Changed

- `mcp-http.ts`: `mcpHttpRequestInit` can add `_meta.progressToken`.
`readMcpHttpResponse` gives each notification on the response stream to
an observer before the result. The observer cannot fail or delay the
call. New `mcpProgressNotificationOf` matches a notification to its
token.
- `tool-gateway.ts`: remote HTTP and local stdio `tools/call` from an
agent run send a per-call token and forward progress to a reporter.
Calls without a run send no token.
- New `server/src/services/tool-call-progress.ts`: checks each update
(finite, increasing, plain single-line text of at most 200 characters).
It publishes a `heartbeat.run.tool_progress` live event at most every
250 ms (latest wins) and saves the latest update on the invocation at
most every 1 s. Nothing is awaited by the call. A final `done` event
goes out when the call ends.
- DB: nullable `tool_invocations.progress` jsonb column (migration
`0479`, `ADD COLUMN` only). A save never writes onto a finished call.
- Live delivery: the new event follows its run like transcript output. A
client gets it only for runs it follows, after the run read check, in
the run's company only.
- New route `GET /api/issues/:issueId/tool-call-progress` (task read
check) returns the latest progress of the task's running calls, for a
reload mid-call. Documented in `docs/api/issues.md`.
- UI: `ToolCallProgressProvider` (one per chat) keeps the latest update
per call. A running tool row subscribes and shows a bar or sweep, the
message, the percent and elapsed time (design tokens, reduced motion
honored). Finished rows drop it and never fetch or re-render for
progress. `LiveUpdatesProvider` ignores the event for query
invalidation.
- `packages/kv-demo-mcp-server`: new `slow_with_progress` demo tool
(`steps`, `stepMs`, `withTotal`) in its own file.
- Storybook: `Tasks/Tool progress` stories (running with total, running
without total, done).
- Docs: new `doc/connections/MCP-PROGRESS.md`;
`MCP-PROTOCOL-VERSIONS.md` no longer lists progress as unsupported.

## Verification

- Red/green: on the original source, 5 of the new server tests fail
(token in `_meta`, SSE with interleaved progress and result,
notification matching, run-interest delivery, gateway end-to-end). With
this change they pass.
- `server/src/__tests__/mcp-http.test.ts` (13),
`tool-call-progress.test.ts` (9: throttling, latest wins, bounded text,
out-of-order drop, finish, failures never throw, company/run/tool
scoping), `tool-call-progress-store.test.ts` (2, embedded Postgres:
company and task scoping, no write onto a finished call),
`realtime/live-event-delivery.test.ts` (run interest),
`tool-gateway.test.ts` end-to-end against the real SDK kv demo server
over Streamable HTTP: 6 provider updates over about 1.2 s give 3 to 7
throttled events, all for the right company, run, task and tool, the
last one `done`; no event reaches another company; the invocation keeps
the progress; the result is unchanged.
- UI: `tool-call-progress.test.tsx` (10: name matching, row states,
plain-text rendering, drop on complete, restore after reload, no fetch
for finished rows). All 67 task-chat test files pass.
- `packages/kv-demo-mcp-server`: 13 tests, including progress over HTTP
with and without a total.
- Storybook: checked the three stories in a browser. The bar fills to
40%, the sweep animates, the elapsed time ticks, and the done row shows
no progress.
- `pnpm verify`: see the CI checks for the final revision.

## Risks

- Low. The migration adds one nullable column.
- An SSE response is now bounded per message, not per stream, so handled
notifications (such as progress) do not use up the result's size limit.
Each message is still bounded.
- Progress code runs only while a run's tool call is in flight. It adds
a few bytes to the request and throttled, fire-and-forget work. Agent
boot and time to first token do not change.
- A row finds its call by run and tool name. If one run calls the same
tool twice at the same time, both rows show the newer call's progress.
- Progress from a call that became an MCP Task is not shown.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Claude Code agent with tool use
and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 20:14:22 -06:00
DottaandClaude Opus 5.5 a4c570aede feat(mcp): ask an app's form questions in the task chat and send the answer back (MCP elicitation) (#21466)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents use connected MCP apps through the tool gateway.
> - An MCP app can ask the person a question in the middle of a tool
call (MCP elicitation). There are two kinds: a form, or a link to open.
> - Today a form question is a dead end:
> - A 2026-07-28 server's form `input_required` stops with "needs
information Paperclip can't collect yet".
> - An older server's `elicitation/create` makes a card, but the answer
is never sent back, and the server waits on an open stream.
> - Paperclip does not declare form support, so compliant servers never
ask.
> - This pull request shows the question as a card in the task chat. The
card names the app, and only a person can answer it. When the person
answers, the agent wakes. The agent's repeated call sends the answer to
the app exactly once, for both protocol eras.
> - The benefit: apps that need a choice or a detail from the person now
finish their work. Restarts lose nothing, and no call is left waiting.

## Linked Issues or Issue Description

Refs #21140 (stateless MCP client; this PR closes its "form elicitation"
follow-up). Refs #21263 (MCP Tasks; this PR shares its card code). Refs
#21389 (MCP Tasks follow-ups: the card answer validation item).

**Is your feature request related to a problem?**
When a connected MCP app asks the person a form question during a tool
call, the call stops and cannot continue. A 2026-07-28 server gets a
"needs input" error. An older server gets a card whose answer never
reaches it.

**Describe the solution you'd like**
Show the question in the task chat. Let a person answer, skip, or leave
it. Wake the agent when the question is resolved. Send the answer on the
agent's repeated call, once, with the server's exact state.

**Describe alternatives you've considered**
Holding the tool call open until the person answers. We rejected this:
it holds a stream and a run slot, and it does not survive deploys or
restarts. The plan (`doc/plans/2026-11-13-mcp-elicitation.md`) records
nine design decisions with their alternatives.

## What Changed

- **Plan:** `doc/plans/2026-11-13-mcp-elicitation.md` covers the spec,
the inventory, the decisions, the journey, tests, and risks.
- **New module `server/src/services/mcp-elicitation.ts`.** It is the
single mapping for direct calls and MCP Tasks:
- It turns each spec form field into a real card field: text with
defaults, number, yes/no, single choice (`enum` and `oneOf`), and
multiple choice (`items.enum` and `items.anyOf`).
  - It turns card answers back into typed `content`.
- Answered sends `accept`. Skip sends `decline`. Withdrawn, closed, or
expired sends `cancel`.
- It refuses forms that ask for passwords, keys, tokens, card numbers,
or one-time codes.
  - It keeps app text plain, so links in it are not clickable.
- It seals `requestState` with AES-256-GCM, bound to the company and
connection.
- **New table `tool_mcp_elicitations`** (migration 0479). It is
company-scoped, with one row per question round. It holds a keyed call
identity, the sealed state, the questions and their cards, a claim time,
and a 24-hour expiry.
- **Gateway, stateless servers:**
  - Paperclip declares `elicitation: {form, url}` on task tool calls.
- A form-only `input_required` saves the round, makes the cards, and
stops with a wait message.
- A repeated call before the person answers stops again without
contacting the app.
- After the answer, the repeated call claims the round and sends
`inputResponses` and the exact `requestState`, once.
- A failed send releases the claim. A crash mid-send lets the claim
lapse.
- **Gateway, older servers:**
  - Paperclip declares form support at `initialize`.
  - The first question gets `cancel` at once, so the server never waits.
- When the same question (same words and schema) arrives on the repeated
call, Paperclip answers it from the card.
- **Gateway, mixed rounds and retries:**
  - A form beside a URL step shows the link first.
- A repeated call that stopped for a question runs again in the same
run. It is no longer replayed by the per-run idempotency key.
- **Cards:** the title is "*App* needs your input". The resolver policy
is `human_only`. "Send to *App*" sends the answer. A user comment does
not supersede the card.
- **Activity log:** new events `tool_gateway.elicitation_sent`,
`tool_gateway.elicitation_refused`, and
`tool_gateway.elicitation_expired`. They record the action only, never
the values.
- **MCP Tasks:** task cards use the same mapping and gain the app name,
real fields, and `human_only`. A secret or an unsupported form stays
unsupported.
- **Storybook:** `Tasks and conversations/Questions and approvals/MCP
App Questions` has five states: Asking, Answered, Skipped, Expired, and
TwoQuestions. The fixtures are generated from the server mapper's
output.
- **Docs:** `doc/connections/MCP-ELICITATION.md` (linked from the
README) is new. `MCP-PROTOCOL-VERSIONS.md` and `MCP-TASKS.md` are
updated.

## Verification

- `server/src/__tests__/mcp-elicitation.test.ts` has 16 tests. They pass
on the branch.
- **Red/green:** 9 of the 9 gateway tests fail with master's
`tool-gateway.ts` and `mcp-http.ts`. The 7 pure unit tests cover the new
module.
- The tests use a fake 2026-07-28 server and a fake 2025-06-18 server
that asks over the open stream and waits for the reply.
- Covered: answer, skip (`decline`), withdraw (`cancel`), two questions
at once, a restart while waiting, a crash mid-send, another company with
the same app and arguments, expiry, a refused password field, an agent
that tries to answer (rejected), and capabilities.
- Existing suites: `mcp-tasks`, `mcp-protocol-versions`,
`tool-gateway-service`, `tool-gateway`, and `remote-mcp-connectors`. See
the results comment below.
- Storybook: the five stories render in headless Chromium, and the play
tests pass. The only console errors are the 404s from the backend-less
API calls that every story makes.
- `tsc --noEmit` is clean for `server` and `ui`. `node
scripts/storybook-catalog.mjs` passes.

## Risks

- **Older servers may start asking questions.** Declaring form support
at `initialize` can make older servers ask where they did not before.
Mitigation: the first question always gets `cancel` at once, so no call
hangs, and the agent gets a clear next step.
- **Per-call cost.** Each 2026-07-28 HTTP tool call in a task makes one
indexed lookup for a waiting round. Nothing is added before the first
token or to agent boot.
- **Migration.** One additive table with no backfill, so it is safe to
apply online.
- **Changed Tasks behavior.** A task card that is withdrawn or closed
now sends `cancel` instead of `decline`, as the spec defines. A skipped
card still sends `decline`.
- **Stale cards.** Expiry is checked only when the agent calls again. A
card can sit past 24 hours until the next call. Then it is withdrawn and
the app asks again.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code with tool use
(shell, file edits, headless browser), with extended reasoning.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 19:46:56 -06:00
DottaandClaude Opus 5.5 944b1c64d7 feat(mcp): keep long provider calls durable with the MCP Tasks extension (#21263)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents use connected apps through Paperclip's MCP client in the tool
gateway
> - Some provider actions run for minutes: CI runs, deploys, batch jobs
> - Today such a call is lost if it outlives the tool timeout or if
Paperclip restarts or deploys mid-call, and a retry can start the work a
second time
> - Recovery must just work, and the target rate of runs that need a
manual retry is 0%
> - This pull request implements the MCP Tasks extension
(`io.modelcontextprotocol/tasks`, 2026-07-28) in the MCP client
> - The benefit is that a long provider call becomes a durable task: it
survives timeouts, restarts, and deploys, and it is never started twice

## Linked Issues or Issue Description

**Problem**
A long provider tool call holds an HTTP request open. When it passes the
tool timeout, or when Paperclip restarts or deploys during the call, the
result is lost. The agent then often repeats the call, and the provider
does the work again.

**Proposed solution**
Implement the client side of the MCP Tasks extension. Declare it on
stateless tool calls. Store each returned task durably, wait on it
during the call, keep polling it in the background after the call ends,
and let the same call attach to the stored task instead of sending
`tools/call` again.

**Why it matters**
CI, deploy, and batch actions are where agents lose the most work today.
A durable task handle makes these actions safe across timeouts,
restarts, and deploys, without a person.

Refs #21119, Refs #21140, Refs #21219 (the MCP Apps, stateless MCP
client, and MCP Skills work this builds on).

## What Changed

- **Off by default:** MCP Tasks are behind the new instance experimental
setting `enableMcpTasks` ("Durable app actions (MCP Tasks)"), default
off. When it is off, Paperclip never declares the extension, nothing
attaches to or polls tasks, pending rows do not block idle sleep, and
behavior matches master. A test covers the off state.

- `mcp-http.ts`: the tasks extension is declared per request, only on
HTTP `tools/call` and `tasks/*`. Discovery, listings, and stdio servers
do not declare it. `tasks/*` requests put the task ID in `Mcp-Name`.
- New `mcp-tasks.ts`: parses task results, bounds the server's timing
hints, sends `tasks/get`, `tasks/update`, and `tasks/cancel`, and seals
task IDs at rest (AES-256-GCM, bound to company and connection; a keyed
hash gives uniqueness).
- New company-scoped table `tool_mcp_tasks` (migration `0475`,
online-compatible). It links the task to the call, run, task,
connection, and grant. It stores no provider result.
- `tool-gateway.ts`:
- A returned task is recorded, then polled up to the normal tool
timeout. Its result goes through the same validation, redaction, and
recording as a direct result.
- Past the timeout, or when the caller disconnects, the agent gets
`provider_task_pending` ("call again with the same arguments"). The task
stays durable.
- A call with the same endpoint, tool, full arguments, grant, agent,
task, and actor attaches to the undelivered task. A retry with the same
idempotency key also attaches, and it can never start different work
under that key.
- Input: URL requests show the provider link (Paperclip never answers
them). Form requests become a question card; only the person's answer is
sent with `tasks/update`. Other requests stop with a clear "needs input"
error.
- Stopping the run sends `tasks/cancel` and marks the task cancelled. A
caller that only disconnects keeps the task.
- `failed` becomes a tool error with the provider's message. An unknown
task or a passed TTL becomes `provider_task_expired`. Every task stops
being tracked at its TTL, or after 7 days at most.
- Audit entries: `tool_gateway.task_started`, `task_input_sent`,
`task_cancelled`.
- New `mcp-task-work.ts`, registered at startup with the shared deadline
scheduler: it resumes polling pending tasks after a restart, cancels
tasks whose task closed or run was stopped, and wakes the assignee once
when a task settles after its run ended.
- Docs: `doc/connections/MCP-TASKS.md`, linked from
`doc/connections/README.md`.

Background work (per `rules/sleep-aware-background-work.mdc`): trigger
is a task row write or release (`mcp-tasks` queue) and one startup pass.
The deadline is the earliest `next_poll_at`. Cadence is the server's
`pollIntervalMs` within 5 s – 5 min, with backoff; 5 min while waiting
for a person. It is quiet when no row has a next poll. This is the
narrow external-source exception: providers give no wake Paperclip can
receive, and polling stops with the task.

## Verification

- `server/src/__tests__/mcp-tasks.test.ts`: 20 tests with a fake
2026-07-28 server that returns tasks only to a client that declares the
extension and checks `Mcp-Name`. They cover:
- polling to completion, with the result identical to a synchronous
call;
  - timeout, then an attaching call with no second `tools/call`;
  - a keyed retry, and a reused key with different arguments (refused);
- restart mid-task: a fresh gateway with no in-memory state polls the
row, wakes the agent once, and the recovered run gets the result once
with no second `tools/call`;
  - no wake while the run is still going;
- URL input (link, never answered), and form input (question card, then
`tasks/update` with the answer);
- cancellation by the run, by closing the task, and no cancellation when
only the caller leaves;
- failure, an unknown task, and TTL expiry (ends in under 3 s, no hang);
  - another agent never attaches;
- synchronous calls unchanged and older servers never see the extension.
- Red/green: with attach disabled, the timeout, keyed-retry, and restart
tests fail (3). With `tasks/cancel` disabled, both cancellation tests
fail (2). Before `tools/call` declared the extension, 13 end-to-end
tests failed. All pass with the change.
- Existing suites pass: `mcp-protocol-versions`, `mcp-http`, `mcp-apps`,
`mcp-skills`, `server-startup-feedback-export`, `tool-gateway`,
`tool-gateway-discovery-http`, `remote-mcp-connectors` (342 tests).
- `tsc --noEmit -p server` is clean. `pnpm --filter @paperclipai/db
check:migrations` passes (the migration is classified
online-compatible). Snapshots pruned to five.
- Agent boot and time-to-first-token: no change in the boot or
first-turn path. The only new per-call cost is one indexed lookup on
stateless tool calls.
- Not run: a live provider that returns tasks. No public MCP server that
returns tasks was available without new credentials.

## Risks

- Shipped off by default. Review findings that only affect the
setting-on path after this merge are tracked as follow-ups.

- Migration: a new table with indexes and foreign keys to `companies`,
`tool_invocations`, and `tool_connections`. Online-compatible.
- Behaviour change only for servers that return tasks, which they may do
only to clients that declare the extension. Older servers and stdio
servers are unchanged.
- A task that settles while nobody is waiting stays attachable for 1
hour, so a deliberate repeat of the same call within that hour collects
the earlier result instead of starting new work.
- The wake after a background task settles uses the generic task wake;
the agent must call the tool again to collect the result.
- Follow-ups: `notifications/tasks` via `subscriptions/listen`; sampling
and roots input requests; showing a long action's progress in the task
chat.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with
tool use (shell, file edits, local test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 08:32:06 -06:00
DottaandClaude Opus 5.5 8af889f612 feat(mcp-apps): view messages with consent, app context, full screen and provider tool icons (#21266)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - MCP Apps (#21119) show interactive widgets from MCP servers, such as
Circleback meeting cards, with the agent's reply in the task chat.
> - The first version deferred three parts of the MCP Apps spec. A
widget could not propose a chat message (`ui/message`). It could not
give the agent context (`ui/update-model-context`). It could not go full
screen.
> - Paperclip also ignored the `icons` that MCP servers declare on tools
(SEP-973).
> - The security review of #21119 showed what these features need from
day one: explicit consent, untrusted-data labels and strict icon
sanitizing.
> - This pull request adds the three host features and provider tool
icons, with those protections built in.
> - The benefit is that widgets can now work with the person and the
agent, not only display data.

## Linked Issues or Issue Description

Refs #21119

**Subsystem affected**
MCP Apps host (task chat widgets), tool catalog, run wake prompt.

**Problem or motivation**
The MCP Apps host refused `ui/message` (-32601), dropped
`ui/update-model-context` and kept every view inline. MCP tool `icons`
were not parsed or shown. So widgets could not ask the agent a
follow-up, the agent could not see what the person selected in a widget,
large widgets were cramped, and tool rows used only generic icons.

**Proposed solution**
Add consent-gated view messages, per-view app context delivered to the
task's next run as untrusted data, a full-screen dialog, and sanitized
provider icons. Details are under "What Changed".

**Alternatives considered**
- Send view messages with a new server route that creates the comment
itself. Rejected: comment creation has many rules (agent chat ownership,
paused conversations, queued delivery, wakeups, receipts). The widget's
Send button uses the same send path as the composer. A small route then
records which view the comment came from.
- Keep the iframe in place for full screen with CSS. Rejected: chat
containers can create containing blocks that break `position: fixed`. A
moved iframe always reloads, so the dialog hosts a new frame instead.

**Roadmap alignment**
Continues the MCP feature work after #21119, #21140 and #21219. It does
not touch the MCP Tasks work (gateway `tools/call` polling and
elicitation).

## What Changed

- **`ui/message`.** The host shows "<App> wants to send: …" with
**Send** and **Dismiss** under the widget (and inside the full-screen
dialog). Nothing is sent before the click. Send posts the text as the
viewer's own comment through the chat's normal send path
(`handleThreadAdd`). `POST
/api/issues/:issueId/mcp-app-views/:viewId/messages` then logs
`issue.mcp_app_message_sent`. It accepts only the caller's own comment
on the same task, matched by `clientRequestId`, from the last 10
minutes. One proposal per view can wait at a time. The view gets `{}`
after the send, or -32000 "Message sending denied" after Dismiss. Only
`user` text up to 8,000 characters is accepted.
- **`ui/update-model-context`.** `PUT
/api/issues/:issueId/mcp-app-views/:viewId/model-context` stores the
latest context per view in `tool_mcp_app_views.model_context` (text
blocks and `structuredContent`, 16 KB at most). Updates are applied in
order and deduped in the browser and on the server. A change logs
`issue.mcp_app_context_updated`, and an empty update clears the context.
The task's next run gets the three most recently updated contexts as an
`## App context (untrusted)` wake-prompt section. The section quotes the
data as JSON and says it is third-party data, not instructions. The read
runs in parallel with the other run preparation and never fails the run.
CLI adapters and the native runner render it through the same
`renderPaperclipWakePrompt`.
- **Who may do this.** Messages and context need a signed-in person with
`issue:comment` on the task. Agents and API keys get 403. Other tasks
and companies get 404. `message` and `updateModelContext` are advertised
in `hostCapabilities` only where the viewer can send messages.
- **Full screen.** The host advertises `availableDisplayModes:
["inline","fullscreen"]` and handles `ui/request-display-mode`. It
honors the view's declared modes and answers with the resulting mode. It
sends `host-context-changed` with `displayMode` and the container (fixed
`height`/`width` in full screen), and again when that container resizes.
A new **Open full screen** control sits next to "Widget from …". Full
screen is a Paperclip dialog that closes with Exit, Escape or the view's
own request.
- **Provider tool icons.** `icons` on MCP tool descriptors are sanitized
into the new `tool_catalog_entries.icons` column, outside the version
hash, so a new icon never quarantines a reviewed tool. Only `https:`
URLs and `data:image/*` URIs are kept, and SVG only as `data:`. A
`data:` icon is at most 32 KB, a URL at most 2,048 characters, with at
most 4 icons per tool. `GET /api/companies/:companyId/tool-icons` lists
them keyed by gateway tool name, in one cached request per chat. Task
chat tool rows and the app's actions list show them through
`ProviderToolIcon`. It renders `<img referrerPolicy="no-referrer">` at a
fixed size and never inlines markup. It checks the source again before
rendering and falls back to the taxonomy icon.
- **Schema.** Migration `0477` adds `tool_catalog_entries.icons`,
`tool_mcp_app_views.model_context` and `model_context_updated_at`, all
nullable.
- **Docs and stories.** `doc/connections/MCP-APPS.md` documents the new
behavior, and the finished items are removed from "Limits and
follow-ups". `Tasks/MCP App views` has new stories: "View asks to send a
message", "Full screen" and "Tool rows with provider icons". The fixture
view now uses `ui/message`, `ui/request-display-mode` and
`ui/update-model-context`.

## Verification

- `pnpm exec vitest run server/src/__tests__/mcp-apps.test.ts
server/src/__tests__/tool-catalog-publication.test.ts
server/src/__tests__/openapi-routes.test.ts
packages/shared/src/mcp-icons.test.ts
packages/shared/src/mcp-apps.test.ts`: pass (embedded Postgres). These
tests cover:
- Context: stored, deduped, logged once, 413 when too large, cleared,
scoped to the task and company, refused for agents (403), and rendered
into the wake prompt as untrusted data.
- Messages: attributed only to the viewer's own recent comment on the
same task. A teammate's comment, an unknown request id or an old comment
gets 404. An agent gets 403.
- Icons: stored sanitized, and the version hash and status do not
change. Another company sees nothing.
- `cd ui && pnpm exec vitest run src/components/mcp-app/
src/components/ProviderToolIcon.test.tsx src/pages/apps/app-detail/`: 69
pass. These tests cover:
- The host protocol for each new method: capabilities, consent wait, one
proposal at a time, invalid input, ordered and deduped context, the size
limit, and the display-mode rules.
  - The consent card (nothing sent without Send).
  - The full-screen dialog (one frame at a time).
  - Icon rendering (no-referrer `img`, no inline SVG, fallback).
- `cd ui && pnpm typecheck` (including Storybook) and `tsc --noEmit` for
the server: clean.
- Red/green: the new route and host tests fail on `master`, where the
routes do not exist and the host answers -32601 or drops updates. The
icon descriptor test failed when the publication write was removed.
- Not run locally: the full suite, and
`packages/adapter-utils/src/server-utils.test.ts › runChildProcess ›
does not crash when the child closes stdin before a delayed write`. That
test times out on this Mac with or without this change, so it is a local
platform issue. CI covers both.

## Risks

- Migration risk is low: three nullable columns, no backfill.
- Every run on a task now does one extra indexed read, in parallel with
the other preparation. Errors are logged and the run continues without
app context.
- If the attribution request fails after the comment is sent, the
comment stays without the `mcp_app_message_sent` activity row. This is
documented.
- Switching between inline and full screen restarts the view, so state
inside the widget resets. This is documented as a limit.
- Icon names shortened with a hash by the gateway (very long names) do
not match, and those rows keep the taxonomy icon.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and
code execution. One subagent (the same model) built the provider-icon
part.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 07:32:48 -06:00
DottaandClaude Opus 5.5 2d1c2db64a feat(mcp): host the MCP Skills extension for connected apps (#21219)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents use connected apps through MCP. Paperclip already hosts MCP
Apps views and speaks stateless MCP 2026-07-28.
> - The MCP Skills extension (SEP-2640, Final) lets an app publish
skills (instructions and files) next to its tools. Hugging Face already
serves 25.
> - Paperclip did not read them, so agents missed each app's own
workflow guidance.
> - Skills are a prompt-injection surface. The spec makes manifest
binding, verification, origin tagging and approval revocation host
requirements.
> - This pull request makes Paperclip a Skills host: discover, review,
verify, and deliver app skills to the agents that use the app.
> - The benefit is that connected apps can teach agents their workflows,
under board control.

## Linked Issues or Issue Description

**Problem**
Connected MCP servers can publish skills with the
`io.modelcontextprotocol/skills` extension. Paperclip ignored them. A
board could not see, review or approve them, and agents never received
them.

**Proposed solution**
Implement the host side of SEP-2640. List skills during catalog refresh,
record them per connection and company, bind board approval to the full
file manifest, and load approved skills into agent runs with verified,
lazily fetched content.

**Why it matters**
Apps ship workflow guidance with their tools. Without the extension,
each team must copy that guidance into company skills by hand, and it
goes stale.

Refs #21119, Refs #21140 (the MCP Apps and stateless MCP client work
this builds on).

## What Changed

- **Client capability.** Every stateless request declares
`io.modelcontextprotocol/skills`. Legacy handshake sessions do not. The
capabilities that `server/discover` advertises are remembered beside the
protocol era.
- **Discovery.** Catalog refresh calls `skills/list` (paged) only when
the server declared the extension. A listing inside its `ttlMs` is
reused. A skills failure never fails the tool refresh. Nothing runs
while a run starts.
- **Schema.** New `tool_mcp_skills` (company, connection, skill URI;
frontmatter, sorted manifest, manifest digest, review state) and
`tool_mcp_skill_files` (verified bytes keyed by connection, URI and
digest). Both cascade with the connection.
- **Governance.** New skills wait for review. Approval binds to the
order-free digest of every `{uri, digest, size}`; a request for any
other digest gets 409. A changed manifest or frontmatter revokes
approval and logs `tool_mcp_skill.approval_revoked`. Dynamic, oversized
and invalid skills are declined with a reason. Approve and decline use
the company skill policy (`skills.install`, `external_package`,
`mcp-connection:<id>`).
- **Run time.** Approved skills of connections the agent can use load in
parallel with other skill preparation, inside a 3 s budget. `SKILL.md`
is read with the agent's grant and checked for size, SHA-256 and
frontmatter. A failure calls `skills/get` and takes the skill out of
use. Each skill becomes a per-run bundle on the connector skill path,
named in its app's namespace, tagged with its origin. Paperclip writes
the frontmatter: `allowed-tools` only if the board approved it.
- **Supporting files.** `GET
/api/agents/me/mcp-skills/:skillId/files?path=` serves manifest-listed
files to an agent in an active run, through the skill's own connection.
The sandbox bridge allows the route.
- **Board API and UI.** `GET/POST /api/companies/:companyId/mcp-skills…`
(list, read instructions, approve, decline). The Skills page shows
"Skills from connected apps" with app, state, size, requested tools and
plain-language notes. It renders nothing when no app publishes skills.
- **Docs.** `doc/connections/MCP-SKILLS.md`, linked from
`doc/connections/README.md`.
- **Shared frontmatter parser.** It now decodes YAML single-quoted
scalars. Before, 5 of Hugging Face's 25 skills kept their quotes and
would fail the frontmatter check; now all 25 match. The generated runner
copy is regenerated.

File size notes: `server/src/services/tool-gateway.ts` grows by about
200 lines. The run-time reads must use its grant resolution and
`callAssignedConnectionProtocol`, like MCP Apps view reads. Splitting
the gateway is a separate refactor. `CompanySkills.tsx` and
`CompanySkills.production.tsx` gain 2–4 lines each; the section lives in
its own component.

## Verification

- `server/src/__tests__/mcp-skills.test.ts` (15 tests) uses a fake
stateless skills server and embedded Postgres. It covers: discovery only
after the server declares the extension; the capability in `_meta`;
`ttlMs` reuse; lazy fetch (no reads at listing or approval; `SKILL.md`
at load; supporting files on read); a stale-digest approval refused; a
reorder that keeps approval; a changed or added file that revokes it;
dynamic skills declined; a same-size tampered `SKILL.md` rejected,
refreshed with `skills/get`, and taken out of use; mismatched
frontmatter rejected; reads outside the manifest refused; the same URI
and name from two servers kept apart; company scoping; a slow app
skipped within the budget and cached later; HTTP authorization (board
only, agent needs an active run, run header must match) and activity
logging.
- Red/green: with the digest check disabled, the tamper test fails; with
the manifest comparison disabled, the revocation test fails. Both pass
with the code.
- `server/src/__tests__/mcp-protocol-versions.test.ts`,
`mcp-apps.test.ts`, `connection-instructions.test.ts`,
`tool-gateway-discovery-http.test.ts`: 47 pass (two capability
assertions updated for the new extension).
- `packages/shared/src/frontmatter.test.ts` adds a single-quoted scalar
case; the runner contract test passes (23 tests).
- `ui/src/components/skills/ConnectedAppSkills.test.tsx` (3) and
`ui/src/pages/CompanySkills.test.tsx` (44) pass.
`packages/adapter-utils/src/sandbox-callback-bridge.test.ts` (89)
passes.
- `pnpm --filter @paperclipai/server typecheck` passes. UI `tsc` passes
except the known missing generated Storybook catalog files.
- `pnpm verify` returned exit 2 (doc change is a global CI input);
required CI supplies broad coverage.
- Live check against `https://huggingface.co/mcp` without credentials,
through Paperclip's own services and the real egress guard: 25 skills
discovered as pending; three approved; cold load 135 ms, cached load 13
ms; a supporting file read through the manifest; the bundle tagged with
its origin.
- Storybook: `Agents and reusable capabilities/Skills and Skill
Studio/Connected app skills` (needs review, all states, tool request,
name collision, reading instructions).

## Risks

- **Migration:** two new tables only. No changes to existing tables.
Safe with the previous release running.
- **Run start:** companies without an approved app skill pay one indexed
read. With approved skills the wait is capped at 3 s; late or failing
skills are left out with a `skills.mcp_unavailable` run warning and
never fail the run.
- **Agent harness enforcement:** Paperclip cannot gate each
code-execution tool call an agent makes while following an app's skill.
Approval is explicit and per skill, the approval note says agents may
run commands the skill describes, and `allowed-tools` is never granted
without a separate tick.
- **Shared-home adapters** receive app skills like company skills. A
revoked skill leaves on the next skill sync.
- Not in this PR: per-connection opt-in for dynamic skills; serving
Paperclip's own skills from its public MCP endpoint;
`resources/directory/read`; app skills on an agent's skills tab.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edits, local test runs, live HTTP checks).
Extended reasoning enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 19:18:06 -06:00
DottaandClaude Opus 5.5 267983356c feat(mcp): speak stateless MCP 2026-07-28 to provider servers, keep older servers working (#21140)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents use apps through Paperclip's MCP client. The client calls
each provider's remote (Streamable HTTP) or local (stdio) MCP server.
> - The client spoke only protocol 2025-06-18. It always did the
`initialize` handshake and it sent client capabilities only in that
handshake.
> - MCP 2026-07-28 removes the handshake and sessions. Every request
carries its version, capabilities and client identity in `_meta`.
Servers that only speak 2026-07-28 reject the old requests, and they
never see that Paperclip renders MCP Apps views.
> - Public servers already speak it: DeepWiki and Context7 answer
`server/discover` with 2026-07-28 today.
> - This pull request makes the client negotiate the version per server.
It speaks stateless MCP to 2026-07-28 servers and keeps the handshake
for 2025-11-25 and older servers.
> - The benefit is that new and old providers both work, and a tool call
to a stateless server is one HTTP request with no handshake.

## Linked Issues or Issue Description

Refs #21119 (MCP Apps host). This PR sends its MCP Apps capability to
stateless servers too.

**Subsystem affected**
Server: the MCP client in `server/src/services/mcp-http.ts`, catalog
discovery, the tool gateway (`tools/call`, context reads, stdio), and
the remote MCP OAuth flow.

**Problem or motivation**
The client used protocol 2025-06-18 only. MCP 2026-07-28 makes MCP
stateless: no `initialize`, no `Mcp-Session-Id`, per-request `_meta`,
`server/discover`, `resultType` with multi round-trip requests (MRTR),
cacheable lists, required `Mcp-Method`/`Mcp-Name` headers, and RFC 9207
`iss` validation. Servers that speak only 2026-07-28 cannot be used.
Paperclip advertised MCP Apps support only in `initialize`, so stateless
servers never saw it.

**Proposed solution**
Probe each server once with `server/discover`, pick the newest mutually
supported version, and save it on the connection. Send stateless
requests to modern servers and keep the handshake for legacy servers.
Tool calls use the saved version and never probe. The saved version is
checked again only when a request built from it is refused.

**Alternatives considered**
Inline negotiation (send each request in modern shape first and fall
back on `400`) adds a failed request to every call to a legacy session
server. Probing on every tool call adds a round trip before each tool
runs. Saving the negotiated version keeps tool calls at one request.

**Roadmap alignment**
MCP protocol currency for the Apps and connections substrate. It follows
the MCP Apps host work in #21119.

**Additional context**
Spec:
https://modelcontextprotocol.io/specification/2026-07-28/changelog. Out
of scope (separate work): the Tasks extension, Skills over MCP,
`subscriptions/listen`, progress and cancellation, icons,
enterprise-managed authorization.

## What Changed

- `mcp-http.ts`: `negotiateMcpProtocolVersion` probes with
`server/discover`. A `DiscoverResult` or a modern error
(`-32020`/`-32021`/`-32022`) means a modern server. Anything else means
a legacy server, which gets `initialize`. A `401`/`403` keeps the
challenge for OAuth discovery. The era is remembered per connection and
endpoint.
- `mcpHttpRequestInit` builds every request. It always sends
`Mcp-Method` and `Mcp-Name` (Base64 sentinel when unsafe). For modern
versions it adds `_meta` (`protocolVersion`, `clientCapabilities` with
`extensions["io.modelcontextprotocol/ui"]` and URL elicitation,
`clientInfo`) and `Mcp-Param-*` headers from `x-mcp-header`.
- Legacy `initialize` now requests 2025-11-25 and advertises the MCP
Apps extension.
- Catalog refresh saves `config.mcpProtocolVersion`. It drops tools with
invalid `x-mcp-header` annotations. It reuses a fresh `tools/list`
result (`ttlMs`, `cacheScope`). It probes again when the saved version
is refused, and it negotiates again after a credential refresh.
- `tools/call`: a saved modern version is called directly (no probe, no
handshake). A modern rejection of a legacy-shaped call, or `-32022`,
gets one retry with a negotiated version. The server rejected the first
request before it ran, so the retry cannot repeat a side effect. A
remembered legacy server that refuses `initialize` is probed again.
- MRTR: `input_required` with URL requests becomes the existing provider
link flow (`provider_interaction_required`). The next identical call
sends `requestState` and `inputResponses`. Form, sampling and roots
requests stop with `provider_input_required` and a plain message. A
missing `resultType` means `complete`.
- Context reads (`resources/*`, `prompts/*`) use the saved version,
cache fresh results, and stop on `input_required`. The error code is
kept, so resource not found (`-32602` or `-32002`) is visible.
- stdio: probe with `server/discover` (2 s limit). Restart a server that
exits on the probe. Send `_meta` to modern servers.
- OAuth: refuse a callback without `iss` when the server advertises
`authorization_response_iss_parameter_supported`. Check `iss` before
acting on a provider error. Send `application_type: "native"` for
localhost callbacks and `web` otherwise. Issuer-bound clients and the
CIMD-first order already existed.
- Docs: `doc/connections/MCP-PROTOCOL-VERSIONS.md`, linked from
`doc/connections/README.md`.

## Verification

- New suite `server/src/__tests__/mcp-protocol-versions.test.ts` (19
tests) uses a fake 2026-07-28 server and a fake legacy session server.
It covers discovery, `_meta`, headers, version fallback, `-32022`,
`resultType`, MRTR link and retry, form stop, cache TTL, legacy flow,
and recovery when a saved legacy server moves to 2026-07-28. Embedded
Postgres runs the end-to-end cases.
- New OAuth tests in `generic-mcp-connection.test.ts`: `iss` required
when advertised, `iss` checked on provider errors, `native` for
localhost, and a stateless server listed again after a revoked token.
- Red/green: with the master service code, all 21 new protocol and OAuth
tests fail. With this change they pass. The token-refresh regression
test fails without its fix and passes with it.
- Existing suites: 12 touched files, 1088 tests pass. `tsc --noEmit`
passes for `server`. Updated expectations count the one-time
`server/discover` probe, and the Calendly DCR test now expects `native`
for its localhost callback.
- `pnpm verify` exits 2 (a doc is a global CI input), so CI supplies the
broad coverage.
- Live check with the changed client, without credentials:
- DeepWiki (`mcp.deepwiki.com/mcp`): discover 200 → 2026-07-28, no
session, `tools/list` 200 (3 tools).
- Context7 (`mcp.context7.com/mcp`): discover 200 → 2026-07-28, no
session, `tools/list` 200 (2 tools).
- GitMCP (`gitmcp.io/docs`): discover 400 → `initialize` (2025-03-26) →
session → `tools/list` 200 (5 tools).
- Circleback (`circleback.ai/api/mcp`): discover 401 with its
`WWW-Authenticate` challenge kept for sign-in.

## Risks

- A legacy server that is not saved as session-required gets one extra
request (`server/discover`) the first time it is refreshed. Then the
version is saved. Saved session-required connections are not probed.
- DCR from a localhost callback now registers a `native` client. This is
what the spec asks for. Existing registrations are reused.
- An authorization server that advertises `iss` support but does not
send `iss` now fails sign-in with a clear message. The spec requires
this.
- MRTR answers are kept in memory for 30 minutes. After a restart the
server asks again, and the user sees the link again.
- No schema change. `config.mcpProtocolVersion` is a new key in the
existing JSON config.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used in Claude Code
with tool use (shell, file edits, web fetch of the MCP spec), extended
reasoning.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 14:34:26 -06:00
DottaandClaude Opus 5.5 8ff45c0fa9 feat(connections): render MCP Apps views in the task chat, starting with Circleback (#21119)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents reach external apps through MCP connections. Paperclip
records each call and shows it as a tool row in the task chat.
> - MCP Apps (SEP-1865, extension `io.modelcontextprotocol/ui`, spec
2026-01-26) lets an MCP server attach an interactive `ui://` view to a
tool. Claude, Claude Code and VS Code render these views in chat.
Circleback uses them to show meeting cards.
> - Paperclip dropped the tool's `_meta.ui` at discovery and the
result's `_meta` at call time. Its gateway reported MCP App UI as
"unsupported". Circleback was reachable only through Composio, whose
toolkit has no `Display*` tools.
> - This pull request makes Paperclip an MCP Apps host. It also adds
Circleback as a native catalog connection.
> - The benefit is that a Circleback recap in a task shows the same
meeting cards as in Claude, and any other MCP server with views works
the same way.

## Linked Issues or Issue Description

**Subsystem affected**
Connections and apps (remote MCP gateway) and the task chat.

**Problem or motivation**
An agent that uses Circleback in Paperclip can only return text. The
same tools in Claude Code show interactive meeting cards
(`DisplayMeetings`). Paperclip discards the MCP Apps metadata, so it
cannot show them.

**Proposed solution**
Paperclip implements the host side of MCP Apps end to end. It keeps the
tool's view declaration and records each call that has a view. It
renders the view under the tool row in a sandboxed iframe that uses the
MCP Apps postMessage protocol. Circleback becomes a native catalog
connection, so its `Display*` tools are available.

**Alternatives considered**
Composio's `circleback_mcp` toolkit has 11 read tools and no `Display*`
tools, and its meta-tools return plain JSON, so it cannot carry views.
The spec's double-iframe sandbox proxy on a separate origin needs a
second origin that Paperclip does not have. A single opaque-origin
`srcdoc` frame with an injected CSP gives the same isolation.

**Roadmap alignment**
This extends the existing Apps catalog and MCP gateway. It adds no new
integration substrate (decision D4 in doc/connections/README.md).

**Additional context**
The standard, the design and the testing plan are in
`doc/connections/MCP-APPS.md`.

## What Changed

- **Shared:** `packages/shared/src/mcp-apps.ts` adds MCP Apps types,
`_meta.ui` and resource parsing (current and legacy keys), and a CSP
builder that keeps only well-formed `https` origins.
- **DB:** migration `0471` (`online_compatible`) adds
`tool_catalog_entries.ui_meta` and a new `tool_mcp_app_views` table.
- **Server, catalog:** discovery keeps each tool's `_meta.ui`. It is not
part of the version hash, so a view never quarantines a reviewed tool.
- **Server, gateway:**
- Paperclip advertises the MCP Apps client capability when it
initializes a session.
  - App-only tools (`visibility: ["app"]`) are hidden from agents.
- The result `_meta` is kept for the view and is not sent to the agent.
- Each successful call to a tool with a view records a view row.
Recording is best effort.
- The gateway's `initialize` now reports `paperclip/mcp-app-ui:
host_rendered`.
- The context protocol (`resources/read`) now uses the injected test
transport, like the other remote calls.
- **Server, routes:** `server/src/routes/mcp-apps.ts`
- `GET /api/issues/:issueId/mcp-app-views` lists a task's views. It
needs task read access.
- `GET /api/issues/:issueId/mcp-app-views/:viewId` returns the view
document, read live through the agent's connection.
- `POST /api/issues/:issueId/mcp-app-views/:viewId/tool-calls` runs the
view's `tools/call`. It needs `tools:use` and reaches only tools of the
same connection that allow `app`. It uses the connection test-call path,
so policy, Ask first and audit apply.
- **UI:** `ui/src/components/mcp-app/`
- The host protocol: `ui/initialize` with theme, style variables and
size limits; tool-input; tool-result; size-changed; `tools/call`;
`ui/open-link` (http and https only); teardown.
- An opaque-origin iframe (`allow-scripts`, never `allow-same-origin`)
with the CSP placed before the view's markup.
- The views of a run render with that run's reply, in the live turn and
in settled history. They are keyed by the reply's run, so history needs
no transcript.
- **Circleback:** a native catalog app (`mcp-oauth`, DCR, `user
user:write`) with:
  - the official logo;
- a reviewed risk classifier (Share* tools are writes, Delete* tools are
destructive, Display* tools are reads);
  - warnings in the sign-in step;
  - `doc/connections/CIRCLEBACK.md`.
  
Search now prefers a native app over an aggregator listing named "X
MCP".
- **Storybook:** `Tasks/MCP App views` shows the tool row with its view,
the ready state (Refresh calls a tool), loading, view unavailable, and a
tool call that needs approval.

## Verification

- `pnpm exec vitest run packages/shared/src/mcp-apps.test.ts
ui/src/components/mcp-app/`: 19 passed.
- `pnpm exec vitest run server/src/__tests__/mcp-apps.test.ts`: 3
passed, on embedded Postgres. The test connects Circleback through the
catalog with DCR OAuth against a fake Circleback MCP server. It checks:
  - `ui_meta` is stored;
  - an app-only tool is hidden from the agent;
- an agent call records one view with arguments, structured content and
`_meta`;
  - a call without a view records nothing;
  - the routes serve the view's HTML and CSP;
  - the view's tool call works and a model-only tool is refused;
  - another task, another company and an agent actor are refused.
- Circleback connector: 13 focused files, 645 tests passed. These are
the app definitions, search, aggregator fallback, connection intents,
tool policy, `AppsConnect` and `Browse`.
- Typecheck passed for `@paperclipai/shared`, `@paperclipai/server` and
`@paperclipai/ui`. `node scripts/storybook-catalog.mjs` and the
migration compatibility check also passed.
- Storybook check in the browser:
- In both themes, the view runs the handshake, renders the meeting cards
from `structuredContent`, picks up the theme variables, and resizes to
its content.
  - Refresh sends `tools/call` and re-renders the cards.
- **Live Circleback end to end** on a local instance, with a real
Circleback account:
- Connected through the catalog entry with Circleback's dynamic client
registration and OAuth.
  - Discovery stored the real view declarations:
    - `DisplayMeetings` uses `ui://circleback/meetings.v2.html`.
    - `DisplayActionItems` and `DisplayCalendarEvents` also have views.
- `ListAssigneeOptions` and `SetCalendarEventRecording` are app-only
helpers with no view. They are hidden from agents.
  - A `claude_local` agent called `DisplayMeetings` on two tasks.
- The real 3.5 MB Circleback view rendered the meeting cards with the
agent's reply.
- Clicking a card opened that meeting in Circleback through
`ui/open-link`.
- The live run found two bugs, both fixed in this PR, each with a
regression test:
    - The 1 MB tool-result limit blocked the view document.
    - Views were hidden inside the folded tool history.

## Risks

- **New migration:** a new table and a nullable column. It is
`online_compatible`.
- **Existing connections:** tools get `ui_meta` on the next catalog
refresh. Until then, no view shows and the tool row is unchanged.
- **Untrusted provider HTML:** the view runs as provider HTML in an
opaque-origin sandbox. It cannot reach Paperclip's session, DOM or API.
Its network access is limited to the origins its server declared. A
view's tool calls use the viewer's permissions and the normal policy and
Ask-first path.
- **Agent tool lists:** app-only tools are removed from them. No current
provider marks tools app-only, so no listed tool disappears.
- **Search:** the change that prefers a native app over an "X MCP"
aggregator listing affects every app.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and
subagents (one subagent authored the Circleback catalog connector).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 13:28:05 -06:00
DottaandClaude Opus 5.5 0dfbbdeb31 fix(chat): route company creation, decline retries and question cancellation correctly (#21035)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Chat agents follow the chat directive and discover API routes with
search_api and call_api
> - Full chat cohort 4 (OpenCode with DeepSeek v4.1 flash) showed four
guidance gaps
> - The missing-name rule overrode the board-only company-creation
boundary; a repeated request after a decline did not post a new approval
card; search_api showed the board-only question cancel route as open to
agents (and withdraw as board-only); write calls carried `responseText`
beside or instead of `body`
> - This pull request fixes the guidance, the route authorization
metadata and the pre-send refusal
> - The benefit is that agents take the supported path the first time
instead of asking for the wrong input, refusing a valid retry, or
calling a route that returns 403

## Linked Issues or Issue Description

**What happened?**
- onboarding.new-company: asked to "set up a new company for my website
business", the CEO cited the missing-name rule (#21007) and asked for a
business name with a question card. Creating a company is a board
action, so the agent should send the person to the creation flow.
- app-permissions.ask-before-action: the person declined a write card
and then repeated the same request. The CEO asked the person to tell it
to resubmit instead of posting a new approval card. The gateway already
allows a fresh request after a new human message
(`isNewHumanIntentAfterToolRejection`), but the decline text said "Do
not retry the same call".
- questions-and-approvals.multi-part-question: search_api listed `POST
/api/issues/{id}/interactions/{interactionId}/cancel` as
`board_or_agent`. The route rejects agents. The agent route,
`/withdraw`, was listed as board-only although its route admits the
card's creator agent and the task's assignee.
- Several cases: write calls through call_api carried `responseText`
with no body (for example "Received arguments: operationId, pathParams,
responseText").

**Expected behavior**
The agent links the company-creation flow; it posts a fresh approval
card for a repeated request; search_api shows the true authorization of
cancel and withdraw; a write with `responseText` is refused before it is
sent, with the same guidance #21002 gave `saveResponse`.

**Steps to reproduce**
Run the four chat cases above with OpenCode and DeepSeek v4.1 flash on
master.

Refs #21002, #21007

## What Changed

- `AGENT_CHAT_DIRECTIVE`: the missing-name rule applies to work the
agent creates itself and never comes before a board-only boundary. For a
new company the agent links [Create organization](/onboarding) (no
company prefix) and repeats the business details. A repeated request
after a declined approval card is an explicit retry: make the same call
so a fresh card is posted. A clarification question card puts candidate
links in its description, because labels show as plain text.
- Decline instructions in `tool-action-delivery.ts` and
`tool-gateway.ts` say that a later request from the person for the same
action is an explicit retry.
- OpenAPI: `POST /api/issues/{id}/interactions/{interactionId}/cancel`
is board-only and its summary names the `/withdraw` route for agents.
`/withdraw` is no longer marked board-only, which matches its route.
- `call_api`: a new write that carries `responseText` is refused before
it is sent; the message says `responseText` never carries request fields
and names the allowed body fields. Older saved calls still validate for
recovery. `doc/runner-api-tools.md` is updated.

## Verification

- `server/src/__tests__/agent-conversations.test.ts`: new directive
tests for the board-only boundary order, the decline retry and card
links.
- `server/src/__tests__/openapi-routes.test.ts`: cancel is `board`,
withdraw is `board_or_agent`, the cancel summary names withdraw.
- `server/src/services/native-runtime/runner-api.test.ts`: a write with
`responseText` is refused before sending (three window shapes), the same
write without it is sent once, and saved calls still validate.
- `tool-gateway.test.ts`, `tool-gateway-service.test.ts`,
`issue-thread-interaction-routes.test.ts`,
`tool-action-replay-intent.test.ts`: passed. `server` `tsc --noEmit`:
passed.
- Paid run (8 cells, OpenCode + DeepSeek v4.1 flash) together with the
eval PR: see the PR comment for results.

## Risks

- A model that used to send a write with `responseText` and a valid body
now gets one refusal and must resend without it. Nothing is sent before
the refusal, so there is no duplicate side effect.
- The directive grows by a few sentences, which adds a small number of
prompt tokens per chat turn.
- Agents that read `/withdraw` as board-only before now see that they
can withdraw their own card; the route already enforced who may.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with
tool use, shell execution and code editing.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 00:27:35 -06:00
DottaandClaude Opus 5.5 353945c86d fix(tool-gateway): let agents create gateway tokens without receiving the value (#20613)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Tool gateways let external MCP clients use a company's tools with a
gateway client token
> - Since delegated connection management (#17737), an agent can create
and revoke gateway client tokens for its responsible person
> - #20581 stopped agents from getting the one-time token value, but it
did this by making token creation person-only
> - The product decision is that agents keep their delegated authority
to create and revoke tokens
> - This pull request restores agent token creation, but the agent never
receives the token value: the person reveals the value once on the
gateway's Tokens page
> - The benefit is that an agent can do the rotation work for the
person, and the secret still never enters chat, run events, logs or tool
output

## Linked Issues or Issue Description

Refs #17737
Refs #20581

**What happened**
After #20581, `POST /api/tool-gateway/gateways/{gatewayId}/tokens`
returns 403 `gateway_token_requires_person` to every agent caller.
Agents lost the delegated authority to create gateway client tokens that
#17737 gave them. Before #20581, an agent that created a token received
the plaintext value in its tool result and could repeat it in chat.

**Expected behavior**
An agent with delegated connection management can create and revoke
gateway client tokens. The agent never receives a token value. The
responsible person gets the value once, on the gateway's Tokens page.
Person callers keep today's behavior.

**Steps to reproduce**
1. Give an agent delegated connection management for a responsible
person with `tools:admin`.
2. As the agent, call `POST
/api/tool-gateway/gateways/{gatewayId}/tokens` (for example through
`call_api`).
3. See 403 `gateway_token_requires_person`.

## What Changed

- Agent create: `POST /api/tool-gateway/gateways/:gatewayId/tokens` from
an agent with delegated authority returns 201 with token metadata only
(`id`, `tokenPrefix`, `expiresAt`, `pendingReveal: true`), plus
`tokensPath`, a `reveal_credential` `requiredUserAction` for the
responsible person, and a `nextStep` instruction. The saved row has no
usable value: its stored hash is of a value that is generated and then
discarded. Gateway authentication also rejects `pendingReveal` rows.
- Person reveal: new person-only `POST
/api/tool-gateway/gateway-tokens/:tokenId/reveal`. It generates the
value, stores its hash, clears `pendingReveal` and returns the value
once. A conditional update makes a second or concurrent reveal return
409. Only the token's responsible person can reveal it (403
`gateway_token_reveal_requires_responsible_person`). Agent callers get
403 `gateway_token_requires_person`. A token revoked before reveal
cannot be revealed.
- Why a reveal that generates the value: no stored plaintext existed to
reveal, and storing one (even encrypted) for later display adds a secret
at rest. Generating the value at reveal time keeps the existing one-time
display and stores only a hash.
- Schema: `tool_mcp_gateway_tokens.pending_reveal boolean not null
default false` (migration `0460_gateway_token_pending_reveal`). Shared
`ToolMcpGatewayToken.pendingReveal`, and `ConnectionRequiredUserAction`
gets `reveal_credential` / `gateway_token`.
- UI: the Tokens page shows a "Waiting for you to reveal" section for
pending tokens with a "Reveal token" button. The value appears in the
existing "New token — copy now" one-time block. Pending tokens show the
status "Not revealed".
- Agent gateway create: the 201 for agent callers also includes
`gatewayPath`, `tokensPath` and a `nextStep` that tells the agent to
link the saved gateway.
- The agent create-token `nextStep` and the catalog say not to revoke
the old token in the same turn: its client keeps working on it until the
person installs the new value.
- The reveal update and its `tool_gateway.token_revealed` activity entry
commit in one transaction. The Tokens page offers Reveal only for tokens
that act for the signed-in person.
- API reference: new `docs/api/tool-gateway-tokens.md` (create, reveal,
revoke) in the docs nav.
- Revoke stays allowed for agents with delegated authority (unchanged
route).
- API catalog: create-gateway, create-token, reveal and revoke
descriptions now say who acts, that agents never receive values, the
page paths, and that the old token is revoked only after the client
works with the new one.
- Agent access map: create-token policy note and a new reveal entry.
- Evals `gateways-profiles.create-gateway` and
`gateways-profiles.rotate-token`:
- Remove the keyword grader (`human|operator|admin|manage`) and the
handoff text checks. Grade saved state, rendered links and secret
absence. A plain reply or a question card is accepted.
- Remove the setup probes that called the create routes as the agent.
Under delegated authority, those probes created a real gateway
("Forbidden agent gateway"), which failed create-gateway before the
agent ran.
- create-gateway: exactly one gateway for the intended profile and
company, created by the agent (or by the person after a handoff); the
person opens it from a rendered link; attribution matches the creator;
any agent-created token is still pending reveal; no token value in chat,
cards or run events.
- rotate-token: after the agent's turn, both setup tokens still work and
any replacement the agent created is pending the person's reveal; the
person follows the rendered Tokens page link and reveals (or issues) the
value in a private browser; the new value works before the old token is
revoked; the unrelated client keeps access; no token value appears
anywhere, including a regex scan for any full `pcgw_<id>.<secret>`
value.
- The changed scenario context and outcomes are recorded in
`chat-full-catalog-testops-manifest.json` `changedFields` and pinned in
the manifest test. The inventory digest, descriptors (`oracleVersion`
v2, `sourceFiles`) and SDK public input pins are updated. The unused
`external-gateway-handoff.ts` module is removed.
- Fixture realism: the create-gateway profile now allows a real read
tool (an empty profile made the agent ask which apps to add instead of
finishing), and the rotation tokens get a neutral owner note instead of
"Disposable rotation fixture" (the agent read that note as an eval
hint).
- Shared private-browser helpers (`external-private-browser.ts`,
`external-private-browser-worker.ts`) get one optional `pendingTokenId`
input and one `revealedPending` receipt field.

## Verification

- Red: on `master` route code, the new integration tests `lets agents
create and revoke gateway client tokens without ever receiving a token
value` and `reveals an agent-created gateway token value only to its
responsible person` fail (403 `gateway_token_requires_person`).
- Green: `cd server && npx vitest run
src/__tests__/connection-management.integration.test.ts -t gateway` (25
passed). The test checks that the agent `call_api` result has no `token`
field and no token-shaped value, that the agent cannot reveal, that the
person reveals once (second reveal 409) and the value authenticates,
that agent revoke works and stops the value, that a token revoked before
reveal never gets a value, and that person create still returns the
value.
- `npx vitest run src/__tests__/tool-gateway.test.ts
src/__tests__/openapi-routes.test.ts
src/services/native-runtime/runner-api.test.ts` (2247 passed).
- `cd ui && npx vitest run
src/pages/apps/gateways/panels/TokensPanel.test.tsx
src/pages/apps/gateways/gateway-helpers.test.ts` (16 passed); `pnpm
check:token-gates` clean.
- Eval graders and catalog: `pnpm exec vitest run --config
tests/runner-e2e/vitest.config.ts` on the gateway oracle tests,
`external.test.ts`, catalog consistency/contract, TestOps
export/retained, phase/clock callers and SDK input/type tests (all
pass); `node --test` on the TestOps manifest, adapter, contract,
streaming and reconcile tests (all pass); `node
tests/runner-e2e/source-sdk-public-inputs.mjs <HEAD> --check`
(unchanged).
- `node scripts/check-maps.mjs` passes. `tsc --noEmit` for `server`,
`ui` and `tests/runner-e2e`: no errors in changed files.
- Paid end-to-end run of both eval cases, run
[42872921890](https://github.com/paperclipai/paperclip/actions/runs/42872921890)
on 39f38a7f6c (before the last three commits): both FAILED, for
agent-behavior reasons the later commits address. Not rerun yet.
- rotate-token: the agent created the replacement (pending reveal, no
value), then revoked the old token in the same turn, despite the create
response saying to wait. Its reasoning cited the eval-looking fixture
note. The grader rejected the premature revocation. The reply linked the
Tokens page correctly.
- create-gateway: the agent created the gateway and one pending token,
then asked a question card about which apps to add to the empty profile,
with no link to the gateway. The grader rejected the missing link.
- Token values: a scan of all evidence files of both cells found no full
`pcgw_<id>.<secret>` value and no `pcgw_<uuid>` form. Agent `call_api`
payloads in run events are redacted; the integration test proves the
agent tool result has no value.

## Risks

- Behavior change for agents: token create returns metadata and a Tokens
page path, not a value. Agents that expect `token` in the response must
send the person to the Tokens page.
- Migration adds one boolean column with a default. Existing tokens get
`false` and keep working.
- A pending token that is never revealed stays as an unusable row until
someone revokes it. It cannot authenticate.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 15:13:34 -06:00
DottaandPaperclip 5bad074baa feat: add individual agent interaction permissions (#20411)
Add eleven sparse incoming and outgoing restrictions, human CLI/API administration, shared mutation enforcement, and pending-delivery checks. Preserve existing defaults, agent settings, and existing authorization boundaries.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-09 20:34:04 -06:00
DottaandPaperclip c67909cb0f fix: keep tool approvals valid through OAuth refresh and delayed review (#20340)
## Thinking Path

> - Paperclip lets people manage agent work and approve app actions.
> - An approval must bind the action that the person reviewed.
> - OAuth access tokens can expire while that person is away.
> - The old target snapshot treated each token refresh as a new
authorization. It also imposed a one-hour review deadline.
> - This pull request separates token renewal from authorization
identity and keeps the human decision durable.
> - Current access, the reviewed arguments, meaningful target changes,
task closure, and execution receipts still control dispatch.
> - People can approve later without repeating a decision after routine
connection maintenance.

## Linked Issues or Issue Description

**What happened?**

A pending app action returned `approved_tool_target_changed` after its
OAuth tokens refreshed. The tool and connection settings were unchanged.
The same card also expired after one hour. A sign-in problem could
discard an accepted decision before provider dispatch.

**Expected behavior**

The original card remains reviewable after routine OAuth refresh and a
server restart. A later approval runs the exact reviewed action once. A
connection repair preserves the decision while Paperclip verifies the
same authorization. Meaningful changes require a new review.

**Steps to reproduce**

1. Connect an OAuth MCP app and configure an action to require approval.
2. Request the action, then refresh its access and refresh tokens before
approving.
3. Approve the original card. Before this change, it returns HTTP 409
even when the authorization is unchanged.
4. Wait more than one hour. Before this change, the card expires without
a task-specific deadline.

**Paperclip version or commit**

Reproduced from master at `76254ead2229`. The final branch is rebased
onto current master.

## What Changed

- Add versioned approval target comparison. New snapshots bind
credential identity epochs, consent, permissions, ownership, connection
configuration, tool contracts, and credential references. Authenticated
OAuth refresh preserves identity. Manual credential replacement does
not.
- Remove the implicit one-hour deadline from ordinary app action
reviews. Keep explicit deadlines and terminal-task expiry. Synchronize
the nullable interaction deadline in shared contracts and UI.
- Keep accepted, undispatched actions when sign-in is required. Show a
saved-approval repair state. Resume on committed connection changes
after target and access revalidation, without an idle retry loop.
- Preserve the existing execution claim, result receipt, and
uncertain-outcome no-replay rules.
- Explain meaningful target changes without inviting the person to retry
the stale card.
- Add a production task-thread Storybook example and document
compatibility and recovery behavior.

## Verification

- Red: production-refresher invalid-grant handling and formal-approval
withdrawal during a sign-in wait also reproduced as failing regressions
before their fixes.
- Red: before implementation, the local OAuth HTTP fixture failed for
both public and confidential clients with `approved_tool_target_changed`
after token refresh.
- Green: the extended fixture passes for public and confidential clients
with personal and shared grants. It covers actual local HTTP token
exchanges, multiple refreshes, a rebuilt gateway, an ended originating
run, approval after 25 hours, duplicate approval, preserved sharing
arguments, sign-in waiting, and event-driven repair. The gateways use
the production OAuth refresher; an invalid refresh preserves the
accepted decision before provider dispatch. Omitted-scope responses
cover equivalent scope-reporting changes. The provider is an isolated
local fixture, not a deployed Notebook service.
- Negative controls refuse changed consent, scopes, tool definition,
destination, and manual credential replacement before provider
execution.
- Existing gateway service, connected-tool acceptance, secret owner,
secret service, unsigned-review queue, and connection-health suites
passed. Explicit-deadline fixtures still prove expiry behavior,
including deadlines on accepted sign-in waits. A withdrawn formal
approval cancels a waiting action before connection recovery.
- Approval card, task-chat, error-copy and target-comparison tests
passed. Server and UI typechecks passed. Token gates and Storybook
catalog checks passed.
- Inspected the new saved-approval repair card in local Storybook using
the production task thread. It shows a reconnect link and no second
approval button or false running state.
- `pnpm verify --explain` returned exit 2 and requested broad CI
coverage. [Full
CI](https://github.com/paperclipai/paperclip/actions/runs/42677855507)
passed on `eb4f1c7caa9ee6e802c314ffc5eadec475a5de0c`, including build,
typecheck/release registry, general and serialized tests, and all eight
browser shards.
- Greptile reviewed the same final head at 5/5. Both findings from the
first review are fixed and their threads resolved; no open follow-ups
remain.

## Risks

- No database migration is needed. The new comparison reuses the vault
identity epoch. Only authenticated OAuth refresh code can preserve that
epoch; HTTP credential-write schemas cannot select that behavior.
- Earlier signed requests keep their original exact-version checks and
saved deadlines. Already-expired cards are not revived. A historical
card whose credentials already changed needs a fresh review.
- A new consent, account, permission set, destination, or manually
replaced credential must not inherit the old authorization. Reconnection
continues automatically only when revalidation proves the same saved
authorization.
- The interaction enrichment now permits a null deadline. Existing
string deadlines remain valid. The web client and shared validator
change together.
- This PR does not deploy to staging or perform a live Notebook
publication.

## Model Used

OpenAI Codex, GPT-6 family, with repository tools, code execution, and
browser inspection. The runtime does not expose the exact model ID,
context-window limit, or reasoning-mode label; those values are not
asserted.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-09 13:46:28 -06:00
Pavel-GuseynovandClaude Opus 5.5 4d297b998a fix(server): give remote MCP tool calls a 30 second default deadline and record stream failure evidence (#19768)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The tool gateway calls remote MCP servers over Streamable HTTP, and
a server can answer `tools/call` with a `text/event-stream` body
> - Such a server can send progress notifications before the response,
and a slow tool can take more than 10 seconds
> - The gateway aborts a remote MCP `tools/call` that uses the default
deadline after 10 seconds, the general tool default
> - When the call fails, the call evidence does not show how far the
stream got, or whether the cause was the deadline or a broken transport
> - This pull request gives remote MCP calls a default deadline of 30
seconds, the default response timeout of the remote HTTP client, and
records the failure cause and the stream progress
> - The benefit is that slow remote tools complete, and an operator can
tell a deadline from a peer reset in the call evidence

## Linked Issues or Issue Description

Refs #10147. That issue reports that a named MCP gateway aborts remote
tools after 10 seconds and ignores the connection timeoutMs. This pull
request raises the default deadline of a remote MCP `tools/call` to 30
seconds. It does not make the named gateway read the connection
timeoutMs.

Related: #11910 (open) keeps a connection callable after one timed-out
tools/call. On current master a timeout and a JSON-RPC response error
already keep the connection healthy, and this pull request keeps that
policy.

I searched pull requests and issues for "event stream tools/call", "SSE
MCP tool call hang", "remote MCP timeout stream", "mcp
text/event-stream" and "tool_timeout remote MCP". The only related open
pull request is #11910.

## What Changed

- `server/src/services/remote-http-fetch.ts`: the default response
timeout is exported as `DEFAULT_REMOTE_HTTP_RESPONSE_TIMEOUT_MS` (30,000
ms, unchanged value).
- `server/src/services/tool-gateway.ts`:
- A remote MCP `tools/call` that uses the default deadline now gets
`DEFAULT_REMOTE_HTTP_RESPONSE_TIMEOUT_MS` instead of the general tool
default of 10 seconds. The Railway `run-command` budget and an explicit
`timeoutMs` keep their values.
- The call evidence records the response status, content type and
request id before the body is read. A body that fails partway records
the bytes received so far as its size.
- A failed call records `failureKind` (`invocation_timeout`,
`protocol_failure` or `transport_failure`), the deadline and whether it
was aborted, the stream progress, and the error with its cause chain.
Each string in the chain is redacted with `redactSensitiveText`, and a
cyclic cause chain is cut with a "[Circular cause]" marker.
- A deadline abort is a 504 `tool_timeout` and starts a fresh MCP
session on the next call, as before. The connection health policy of
master does not change.
- A deliberate stop of a call (`elicitation_required`,
`provider_interaction_required`, `oauth_insufficient_scope`,
`oauth_refreshed_retry_required`) keeps its own reason when the deadline
passes while it is recorded, as on master. An error that is already a
gateway error also gets a failure kind: `tool_timeout` is
`invocation_timeout`; a malformed, invalid or oversized response is
`protocol_failure`; and `remote_http_*` (for example a DNS failure) and
`mcp_remote_fetch_failed` are `transport_failure`.
- A non-success response body is read as a stream, so a body that fails
partway records the bytes that arrived. The reader stops at the size
limit and cancels the body, as the `tools/call` reader does. #19785
makes the same reader change for the context path.
- `server/src/services/mcp-http.ts`: `readMcpHttpResponse` takes an
`onProgress` callback. It reports bytes, chunks and events received,
whether the matching response arrived, and whether the stream ended. As
on master, the read returns as soon as the response with the matching id
arrives.
- Tests:
- `server/src/__tests__/mcp-http.test.ts`: a partial event followed by a
peer error keeps the progress that was consumed, and releases the
reader.
- `server/src/__tests__/tool-gateway.test.ts`: six scenarios run against
a remote server that streams one progress event first: slow success
after 12 seconds, the default deadline, an explicit 1,500 ms deadline, a
peer reset, a cyclic error cause, and a primitive error value.
- `server/src/__tests__/tool-gateway.test.ts`: three regression tests: a
tool question sent after the deadline passed keeps
`elicitation_required`; a DNS failure records `transport_failure`; and a
503 body that fails partway records the bytes that arrived. All three
fail on 717e6c77c and pass on this head.
- `server/src/__tests__/tool-gateway-service.test.ts`: when the call
deadline passes during a forced Vercel token refresh, the call keeps the
409 `oauth_refreshed_retry_required` stop. `tool-gateway.ts` adds this
reason to the deliberate stops, so the deadline does not change it to a
504 `tool_timeout`.

## Verification

Head SHA: `25c6e703d1ded895989e9d2fcd6d6352a4f851f5`. It is the previous
head `7836447f2` plus commit `25c6e703d`, which keeps the
`oauth_refreshed_retry_required` reason when the call deadline passes
during a token refresh. At this head, run from `server` with pnpm
9.15.4: `pnpm typecheck`: no errors; `mcp-http.test.ts`,
`tool-gateway.test.ts` and `tool-gateway-service.test.ts`: 307 passed.
Without the one-line change in `tool-gateway.ts`, the new test "keeps
the explicit retry reason when the call deadline passes during the
Vercel token refresh" fails: it receives 504 `tool_timeout`, not 409
`oauth_refreshed_retry_required`.

Previous head: `7836447f24e2e6e9e0c402d4644d1f7f69b75cb8`. It is an
ordinary merge of master `76994c50275dbc005cc248e11189a38baa21719f`,
which includes #19877, into the previous head `00150e59c`. At this head,
run from `server` with pnpm 9.15.4: `pnpm typecheck` and the test
commands below. Results: typecheck: no errors; the 2 test files above:
116 passed.

Previous head: 00150e59c1. It merges
master 18b6b71d8. Run from `server`:

```sh
pnpm exec vitest run src/__tests__/mcp-http.test.ts src/__tests__/tool-gateway.test.ts
```

- With this branch: 116 passed. With tool-gateway-service.test.ts and
remote-mcp-socks-proxy.test.ts added: 315 passed in 4 files.
- With `services/mcp-http.ts`, `services/remote-http-fetch.ts` and
`services/tool-gateway.ts` from master and the test files from this
branch, measured at 717e6c77c: 7 failed, 106 passed. The failing tests
are the six "records remote SSE ... without confusing a call deadline
with a broken transport" scenarios and "retains consumed SSE progress
when a partial event fails and releases the reader".
- The existing master tests "keeps a remote MCP connection healthy and
starts a fresh MCP session after a tool call timeout" and "... after a
response body timeout" pass with this branch.
- Typecheck: `pnpm --filter @paperclipai/server typecheck` reports 29
errors, all "Cannot find module @paperclipai/plugin-sdk" and the
implicit any that follows, in the sandbox provider plugin packages of my
local install. None is in a file that this pull request touches.
- I ran the tests with an embedded PostgreSQL from the repository
dependencies. The full upstream suite runs in CI. I ran only the test
files named above.

## Risks

- Medium risk for slow tools: a remote MCP call that uses the default
deadline can now run for up to 30 seconds instead of 10 before it fails
with `tool_timeout`. An explicit `timeoutMs` still applies.
- The call evidence is larger: it can hold a redacted error chain and
its stack traces. All strings go through `redactSensitiveText`.
- No automatic replay: a timed-out `tools/call` is not sent again,
because it can have changed data in the remote app.

## Model Used

- Original change: the model that wrote it was not recorded. Port to
current master, the adaptation to the connection health policy of
master, and this description: Claude Opus 5.5 (claude-opus-5-5), with
tool use and terminal code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes (no
user-facing documentation change needed)
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 09:19:08 -06:00
Devin FoleyandPaperclip 0e49576fd9 fix(github): default repository owner to the only installed organization (#20141)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents use a managed GitHub connection to create repositories and to
fork repositories
> - The Paperclip GitHub App can be installed only on an organization,
and not on the user's personal account
> - Then `create_repository` and `fork_repository` fail when
`organization` is empty, because GitHub sends them to the personal
account
> - Agents, approval cards, and connection tests all fail on their first
attempt with `403 Resource not accessible by integration`
> - This pull request fills an empty `organization` with the only
organization where the App is installed
> - The benefit is that repository creation works on the first attempt
for these connections

## Linked Issues or Issue Description

No public issue exists. Refs #19655 (GitHub repair guidance for the same
error).

**What happened?**
A user connected GitHub through the managed connection. The Paperclip
GitHub App was installed on one organization and not on the user's
personal account. An agent called `create_repository` without
`organization`. GitHub returned `POST https://api.github.com/user/repos:
403 Resource not accessible by integration`. The test button on the
connection page gave the same error. The agent then stopped and asked
the user to create the repository manually.

**Expected behavior**
When the App can create repositories only in one organization, an empty
`organization` uses that organization. The approval card and the test
result show that owner.

**Steps to reproduce**
1. Install the Paperclip GitHub App on one organization only. Do not
install it on the personal account.
2. Connect GitHub with the managed connection.
3. Run `create_repository` with only `name`, from an agent or from the
connection test dialog.
4. GitHub returns 403.

## What Changed

- Added `server/src/services/tool-access/github-default-owner.ts`. It
finds the default organization from the grant's access snapshot. It
returns an organization only when there is exactly one installation, and
the snapshot marks that owner's repositories as organization
repositories. Older snapshots without account types get no default until
access is refreshed.
- `governedToolArguments` in `tool-gateway.ts` fills an empty
`organization` for `create_repository` and `fork_repository` on managed
GitHub connections. The change occurs before policy, approval signing,
previews, and audit. Thus agent calls, approval cards, and connection
tests use the same owner.
- The grant lookup runs only for these two tools, and only when
`organization` is empty. If the credential is unavailable, the call does
not change and fails with its own error at dispatch.
- An explicit owner does not change. A fork into its own owner does not
change. Token (PAT) connections, personal installations, and multiple
organizations keep GitHub's behavior.
- `projectedConnectionToolInputSchema` describes the default on the
`organization` argument. Agents and the connection test form see where
the repository goes.
- Updated `doc/connections/GITHUB.md`.

## Verification

- `pnpm --filter @paperclipai/server typecheck` passes.
- `npx vitest run
src/services/tool-access/github-default-owner.test.ts`: new unit tests
for the owner rules (single organization, personal installation, several
organizations, another person's account, older snapshots, blank values,
forks into the same owner, schema text).
- New gateway test in `src/__tests__/tool-gateway.test.ts`: "defaults
GitHub repository creation to the only installed organization for agents
and connection tests". It covers an agent call, an explicit owner, a
connection test call, and a personal installation. It fails without the
gateway change.
- Related suites pass, 393 tests in 9 files: `tool-gateway.test.ts`,
`tool-gateway-service.test.ts`, `connection-instructions.test.ts`,
`github-access-errors.test.ts`, `github-default-owner.test.ts`,
`tool-gateway-discovery-http.test.ts`,
`github-operation-credentials.test.ts`,
`project-git-authorization.test.ts`, `prompt-injection-gateway.test.ts`.
- Not tested against a live GitHub App installation.

## Risks

- Low risk. The default applies only when the grant snapshot shows one
organization installation and no personal installation. Without the
change, these calls fail with 403.
- A stale access snapshot can choose the organization after a user
installs the App on the personal account. Installation events and
**Refresh access** update the snapshot.
- An approval that was created before this change, without
`organization`, no longer matches the projected arguments. That approval
cannot run and must be requested again. Without the change, it fails
with 403.
- `gh repo create` and `gh repo fork` in agent shells do not use the
gateway, so they keep GitHub's personal default.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (code reading, shell commands, and test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-08 17:08:13 -08:00
Devin FoleyandPaperclip 2a4a001d37 Preserve GitHub access repair guidance for MCP callers (#19655)
Return GitHub permission denials as failed MCP tool results with trusted,
connection-specific recovery guidance. Explain the personal-account and
organization App installation boundary without replaying writes or bypassing
provider-content checks. Align the repair steps with the Permissions UI.

Cover enabled model checks and malicious provider failures across direct
calls, connection tests and stored approvals. Repair the CI-exposed synthetic
onboarding environment fixture without weakening cancellation assertions.

Verified 311 targeted tests, six onboarding Playwright tests, server
typecheck, module boundaries, full CI and Apex 5/5 with all threads resolved.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-06 16:32:27 -08:00
b929ad43de fix: keep dependency waits neutral and explain GitHub access failures (#19315)
## Thinking Path

> - Paperclip manages agent work and shows users what can move it
forward.
> - Task status and tool approval cards describe different parts of that
work.
> - A queued dependency wait has no provider attempt to retry.
> - GitHub can reject an approved action while other connection tools
still work.
> - These cases showed a false Retry banner and an unhelpful provider
error.
> - This pull request keeps dependency waits neutral and gives GitHub
access failures a direct repair path.

## Linked Issues or Issue Description

Related: #19268 preserves pending Retry preparation and current failure
causes.

**What happened?**

A queued wake cancelled by the dependency gate appeared as a failed
attempt with Retry. A GitHub MCP permission denial appeared as raw HTTP
error text after an approved repository action.

**Expected behavior**

Keep the task's dependency state. Show GitHub access failures with a
clear explanation and a link to the selected connection's Permissions
page. Preserve actual failures, provider details, and approval gates.

**Steps to reproduce**

1. Queue a wake, then leave its task dependent on unfinished work before
dispatch.
2. Open the task after the dependency gate cancels that wake.
3. Separately approve a GitHub action whose MCP result contains `403
Resource not accessible by integration`.
4. Reopen the saved tool approval.

## What Changed

- Recognize both queues' unstarted dependency-wait receipts in
cancellation provenance and task projections. No dispatch or recovery
gate changes.
- Classify GitHub's specific permission error after MCP result
normalization. Keep result validation, the raw diagnostic, and working
tools available.
- Apply the same display code to both historical GitHub error codes.
Identify the provider from company-scoped connection metadata, including
renamed apps, without rewriting stored decisions.
- Show a token-based inline access card with **Manage GitHub access**
and collapsed **Error details**.
- Give agents trusted recovery steps after an approved GitHub action
fails. Keep provider text in the untrusted result and prevent automatic
replay.
- Add regressions, a production-component Storybook case, and
execution/connector documentation.

## Verification

- Red before the fix: cancellation provenance, dependency-wait
projection, the real queue cancellation, approved MCP failure
classification, saved interaction projection, and the error card
assertions failed at the expected behavior.
- Green focused tests: queue cancellation, one ready-task executor,
approval-before-dispatch, one failed GitHub request, connection reuse,
retained provider diagnostics, saved interaction reads, task list/detail
projection and redaction.
- UI: 106 component tests passed. Token gates and Storybook catalog
checks passed.
- `node scripts/typecheck-low-memory.mjs`: passed.
- Local source instance: verified a saved dependency wait stays neutral
after reload; a saved GitHub error shows repair guidance, expands its
original error, and opens the correct connection Permissions page.
PostgreSQL and HTTP APIs were real. The GitHub denial was a controlled
MCP fixture; no account or provider permission changed.
- `pnpm build`: passed for all workspace packages, the Runner binary,
server, CLI and UI. The server build and all server typecheck layers
also passed after the review repair.
- Additional safety checks passed for realistic repository arguments,
active recovery, failed Retry preparation, and bounded provider
diagnostics.
- Review repair: 45 pure tests and 7 PostgreSQL cases passed. These
cover both historical error codes, renamed apps, and
company/provider/application identity boundaries. A second local browser
check verified the renamed app, retained error detail, and correct
Permissions destination.
- Final review regressions: the dispatcher and metadata cases failed
before the repair. Afterward, 25 focused cases and 59
projection/identity cases passed, including connection-key precedence.
The native wait and app-only GitHub source also passed a local browser
check.
- The historical read adds at most one connection query per interaction
read batch. It selects IDs and the provider key only and runs only for
candidate permission failures. It adds no provider requests.
- Full local `pnpm test:run` stopped at the chat suite's `afterAll`
cleanup timeout. All 268 assertions in that group and the three prior
chat groups passed. The failure is retained here; it is not counted as a
full-suite pass. CI passed on `e8a692e1e2` after one workspace-startup
rerun. Fresh CI is pending for the combined head. Its first workspace-a
attempt timed out in the CLI company import/export test's child-server
startup wait; 4,968 UI tests and 287 CLI tests passed in that job. The
same shard passed on the preceding commit. The workspace shard passed on
its one rerun. All other test, build and browser jobs passed. The [first
failure](https://github.com/paperclipai/paperclip/actions/runs/42093823928/job/127478869941)
is retained in the Actions history. Greptile reviewed the preceding head
at 5/5. Fresh review is pending for the combined head.

## Risks

The GitHub classifier matches the documented permission-denial phrase,
not every 403. Rate limits, missing resources and other providers retain
their existing error behavior. Historical normalization affects display
only. A dependency cancellation needs an unstarted run and a supported
queue receipt; current user gates, Retry preparation and recovery remain
authoritative. No schema migration or credential changes.

## Model Used

OpenAI GPT-6, through the Codex agent. The exact runtime model ID and
context-window size are not exposed in this session. Used reasoning,
repository tools, TypeScript, PostgreSQL, MCP fixtures and browser
verification. The parallel agent-recovery commit records Anthropic
Claude Opus 5.5 as co-author.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used with version and capability
details
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have described the issue in-PR using the bug template
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change
- [x] I have run the focused tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation
- [x] I have considered and documented risks
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open findings
- [x] I will address reviewer comments before requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 20:55:34 +00:00
DottaandPaperclip a2a6129ed5 Allow a new human intent to request tool review after a decline (#18742)
## Thinking Path

> - Paperclip manages work for companies of AI agents.
> - App writes can require a human review for each invocation.
> - Decline and Always allow are separate controls.
> - The gateway currently reuses a declined action across later human
requests with the same arguments.
> - A new request can therefore receive the old decline instead of a new
review.
> - This change uses the saved run and human comment to distinguish a
new intent.
> - The new intent still follows the current policy and approval gates.

## Linked Issues or Issue Description

**What happened?**

After a person declines an Ask first app write, a later request from
that person for the same write receives `action_declined`. The gateway
matches company, task, agent, tool and arguments across runs. It does
not distinguish the later human request.

**Expected behavior**

A distinct later request from the same responsible person can open a new
Ask first review. It must not execute the write or reuse approval. A
retry of the declined intent, including its rejection continuation, must
remain declined.

**Steps to reproduce**

1. Ask an agent to perform an app write that requires approval.
2. Decline the proposed action.
3. Send a new request for the same action.
4. Observe that the gateway reuses the old rejection and creates no new
pending review.

**Paperclip version or commit**

The behavior is present on master `7b8aa60b6d` and in a retained native
journey. The focused parent control reproduces the matching error. No
related open fix was found in the scoped PR search.

## What Changed

- Only a rejected match can be released for a later human intent.
- Require the active run, company, task, agent and responsible person to
match the saved direct-human comment. Reject derived, deleted, untrusted
or system-authored input and invalid chronology.
- Preserve all other replay states, signed argument checks, current
policy, unresolved-effect guards, approval and execution owners.
- Document the distinction between decline and a standing policy change.

## Verification

- Actual matching-function control: parent fails; this change passes.
The control uses the unchanged owner body with synthetic database rows.
It does not claim real PostgreSQL or provider verification.
- 52 focused pure controls pass. These cover same-intent retries,
rejection continuations, foreign scope and actor, invalid authorship,
chronology and other action states.
- TypeScript 7.0.2 checks pass for the new helper and its owning tests.
- `node scripts/check-maps.mjs` and `git diff --check` pass.
- The existing gateway service suite has a new signed-action integration
test. It covers ordinary and reopened human requests. It requires the
old rejection plus one new pending request, a stable retry, and no
completed call. Each leaf prints one fixed success label after its
assertions so CI can distinguish execution from a platform skip. No
local PostgreSQL, app, model or provider execution was run. Current-head
normal CI41594912212 completed with48 successful jobs and1 expected
skip. Both real PostgreSQL leaves executed and emitted their
post-assertion markers; the owning job passed. Full typecheck/build
checks passed in CI.

## Risks

This adds one indexed run/comment read only after a rejected matching
action. Missing identity or ambiguous provenance keeps the old
rejection. Legacy and external comments without direct-human provenance
do not reset it. A new human intent is not consent to execute. This core
PR remains unmerged. Candidate native journey qualification is separate;
the original failed journey is retained.

## Model Used

OpenAI `gpt-6.1-sol` with reasoning, source inspection and tool use. The
host did not expose the context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#123` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-03 06:28:36 -06:00
79ce86630a fix(server): schedule governed review response recovery (#17702)
## Thinking Path

> - Paperclip manages agents, tasks and their external chat responses.
> - A completed chat turn can owe a response while an earlier review
gate remains pending.
> - Periodic recovery revisits that response history even when no work
can advance.
> - This change uses the existing delivery coordinator and saved retry
deadlines.
> - Approval, interaction and authority changes wake the affected issue
or run after commit.
> - The existing presentation owner verifies the durable result and
current authority.
> - Settled and dormant history becomes quiet without losing future
eligible responses.

## Linked Issues or Issue Description

Refs #17696 (merged) and #15625. This is slice 49, now based on master.
It owns governed-review response presentation. Accepted responses and
pending decision effects keep their existing owners.

**What happened?**

Native finalization recovery repeatedly inspects committed review
responses. Removing the scan without covering approval, interaction,
task-policy and authority changes would strand responses that become
eligible later.

**Expected behavior**

Recover an owed response once from existing durable facts. Enforce
authority at publication, wake after relevant outer transactions commit,
retain real retry deadlines and restart/drain recovery, and stop
recurring SQL when no response can advance.

**Steps to reproduce**

Complete an external chat turn while an earlier completion review
remains pending. Block its response behind an approval or interaction,
then resolve that blocker. The response must appear once without a
periodic scan or resolving the original review gate.

## What Changed

- Add a scoped governed-review worker using the existing coordinator,
finalizations, decisions and comment receipts. No queue table or
transaction wrapper is added.
- Keep recovery and publication policy in the native finalizer. Preserve
binding, source/session authority, ownership, deadline and
deleted-comment fences; match coordinator-before-issue lock ordering.
- Notify through actual approval, interaction, OAuth, task-policy and
lifecycle producers before their owned writes. Serialize approval
resolution with task linking so a concurrent link retains its wake.
Nested transactions still settle at outer commit.
- Scope ordinary milestones, finalization and coalesced input to their
issue/run. Use the explicit company authority boundary for genuine
global restoration, including membership changes. Both response workers
share this boundary.
- Register under maintenance and drain gates, disable the two periodic
repair entrances, and retain fresh finalization and explicit recovery.
Add the concurrent 0413 comment-receipt index and migration
registration; retain five snapshots.
- Preserve the producer inventory, source qualifications and earlier
failures in the owner audit. Retain a real Photon regression proving
routine ownership assertions stay quiet and concurrent authority
restoration wakes each worker once.

## Verification

- Scoped repair source `2341714720b5c273f0ce011b48345af0ef39227f` passes
**2053 distinct cases**: 1703 in 15 complete producer/authority/recovery
files, including all 273 governed-response cases and 95 wake-queue
adapter cases, plus 350 selected chat cases. Another 1463 chat cases are
outside that selection. Thirteen fresh PostgreSQL 17 fixtures migrate
through unchanged 0413; the one temporary queue-fixture import restores
exactly. All sixteen captured hashes match.
- Two actual repeated coarse-hint SQL regressions fail before repair and
pass afterward. Actual membership restoration passes before and after.
The same assertions and time limits prove dormant work stays quiet and
real authority restoration still publishes once. Real coalesced input
covers both queues after commit.
- Combined source `586f5674cd9798b33cd22437255e0480b5726d20`, including
independent master #18129, passes **1297 distinct cases**: 1093 in five
complete server files, 185 in five runner files, 14 Node protocol cases
and five UI registry cases. This includes the complete 22-case Photon
file for both response workers. The external Photon transport is mocked;
Paperclip callbacks, transaction fences and observed SQL are real. The
earlier and combined case counts are separate, with no cumulative
double-counting.
- A normal merge retains both complete owned and independent master
binary patches. All twelve new-context hashes and sixteen earlier repair
hashes match. Three dedicated fresh 0413 fixtures plus the live-event
helper's own fresh database use PostgreSQL 17 without source
adaptations. Database build, server types/build, boundaries, full-SHA
migration order and generated TS/Rust profiles pass. The new inherited
Runner vendor inputs rebuild successfully.
- Database bytes remain identical to the previously qualified 0413
source. The recorded populated 0412-to0413 upgrade retains all 411 prior
receipts and 38 companies, appends one receipt and none on replay; the
index is valid and ready. The earlier 2828-case acceptance, CLI journey,
setup failures and their repairs retain their recorded source scope in
`doc/plans/2026-10-26-governed-review-response-sleep-audit.md`.
- Parent #17696 merged with 58 clean checks, Greptile 5/5 and Ponytail
5/5. Its Photon warning was disproved by real callback/SQL evidence and
the security check became successful. This PR still requires its own
fresh exact-head CI and review gates. No deployed sleep, external alarm
or fleet savings claim is made.

## Risks

- Notifications use the existing process-local writer contract.
Independent SQL writers and uncoordinated replicas are outside its
coverage.
- Genuine startup and company authority reconstruction remain
proportional to the owed backlog. Queries page 100 rows, but discovery
accumulates the candidates; this is not a constant-cost global
operation.
- Current publication authorization, durable binding, ownership and lock
ordering remain mandatory. The worker does not resolve the original
review gate.
- The concurrent index uses existing migration replay handling. Other
recovery duties and Cloud sleep eligibility remain separate.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, repository tools, shell
execution and PostgreSQL tests. The runtime does not expose a more
specific serving snapshot or context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Codex <noreply@openai.com>
2026-10-29 02:54:46 -05:00
DottaandPaperclip 5f0da562db fix(server): wake chat lifecycle milestones from committed changes (#17606)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Connected chats show safe notices when a run starts, waits,
finishes, or fails.
> - A recurring scan projected these notices even when no eligible work
existed.
> - Eligibility depends on committed run state, conversation lineage,
and current authority.
> - Existing database rows can reconstruct that work after restart.
> - This PR wakes a scoped worker from those writes and removes
lifecycle polling.
> - Native progress events keep their separate existing owner until the
next migration.

## Linked Issues or Issue Description

**What happened?**

The chat maintenance timer repeatedly scanned run lifecycle state. Empty
and settled instances still queried the database. A run-status event
alone cannot replace the scan: prompt receipts, continuation receipts,
identity changes, and restored authority can make a run eligible later.

**What did you expect?**

Committed changes wake projection. Startup reconstructs eligible work. A
completed pass stops until another relevant change arrives. Failed work
retains recovery without scanning settled history forever.

**Steps to reproduce**

Configure an automatic chat endpoint. Finish its work, or leave it
unused. Observe recurring lifecycle projection queries from the chat
maintenance callback.

Stacked on #17592. Related foundation: #15625. This PR owns lifecycle
milestones only.

## What Changed

- Preserve upstream coalesced Cloud membership writes together with
locked membership restoration and its committed chat notification.

- Signal milestone eligibility from the real Slack account-link
transaction.

- Register the lifecycle worker with the existing delivery coordinator.
Remove lifecycle projection from the periodic chat callback. Keep native
progress in a separately named lane and preserve the explicit combined
API.
- Reconstruct company/issue scopes from the existing selector. Drain
bounded batches with company fairness. Retry failures, retain writes
through drain/resume, and disarm settled work.
- Signal from supported run, presentation, interaction, receipt,
lineage, conversation, recovery, and authority writers. Preserve their
transaction ownership, including nested savepoints and uncertain
settlement. No new table is added.
- Preserve provider authorization, authored-final suppression,
responsible-user continuity, deduplication, the 24-hour selection
window, and raw-output restrictions. Keep provider interpretation in its
owner.
- Preserve PostgreSQL timestamp precision in lifecycle pagination.
Rejected submillisecond candidates must not repeat the same page and
starve later notices.
- Document producer coverage and alternate-writer dispositions in
`doc/plans/2026-10-25-chat-run-milestone-sleep-audit.md`.

## Verification

Published head: `1d056f68c5b1005933b9e12de8745bf1c4cbeeca`, targeting
master after #17592 merged at
`7d75ae0569e41786640cf09647ae8849411d2f22`. A normal merge integrates
the actual landed parent. The entire 77-path milestone patch is
binary-identical to the locally verified preparation across that
ancestry reconciliation; both unrelated master paths are preserved. The
four earlier producer/test integration conflicts retain scoped milestone
and voice notices, current onboarding completion admission, and expanded
identity query/read-scope coverage.

- **330 distinct selected cases pass on the verified unchanged patch:**
305 across 14 complete suites (all five milestone worker suites, run
identity, board claim, Cloud provisioning, auth session routes, Cloud
actor authentication, stale-lock recovery, GitHub credentials, project
Git authorization, and app admission wiring), plus 25 selected real chat
producer cases. The remaining 1,590 chat integration cases were skipped,
not passed.
- The real producer cases cover saved milestone discovery, GitHub and
Slack identity changes, account-link rollback/reconfirmation, ordinary
ingress reactivation, post-close action/delivery receipt settlement, all
ten authority-restoration paths including Cloud synchronization, and
changed native review metadata.
- Worker coverage verifies outer commit, rollback, uncertain-write
recovery, restart, bounded batching/fairness, timestamp precision,
retries, drain/resume, and settled SQL quiet. Authorization and existing
publication deduplication remain with their owning services.
- Server production typechecking, build, and module-boundary checks
pass. The tests above were executed separately. Production typechecking
excludes `src/__tests__` and does not replace those runs. Each database
owner uses its own newly created and app-migrated PostgreSQL fixture.
- [Current-parent acceptance and preserved failure
ledger](https://github.com/paperclipai/paperclip/blob/1d056f68c5b1005933b9e12de8745bf1c4cbeeca/doc/plans/2026-10-25-chat-run-milestone-sleep-audit.md#current-parent-integration-and-acceptance-october-28)
records commands, counts, producer scope, conflict dispositions, and
limits. Earlier combined-stack counts and failures keep their original
revision scope; local acceptance does not erase old CI failures.
- No new migration, queue table, dependency, workflow, or shared
scheduler contract is introduced. Parent migration 0397, bounded consent
recovery, voice cleanup/retention repairs, and runner checkpoint
restoration remain present.
- **Exact-head Greptile is 5/5 and all 55 current checks are clean.** No
Ponytail review is present, which is an explicitly authorized
nonblocker; no failing review is waived. Older reviews and parent checks
are not current passes. Actual failing Ponytail findings must be fixed;
an absent/stopped/paused turn without a verdict is an authorized
nonblocker. Active unfinished Ponytail uses its original request/start
timer, with a 25-minute monitor cutoff respecting the user's 30-minute
maximum.
- No complete local repository matrix, live provider tenant, external
alarm, physical host/database suspension, or fleet savings claim is
made. Native progress remains a separate following slice.

## Risks

- A future eligibility writer must notify the existing projection
boundary. Independent SQL writers or another active server process are
outside this process-local signal contract.
- Authority and lineage remain sensitive. The worker uses the existing
selectors and publication policy; it does not grant execution or
disclosure authority.
- Native progress and other publication maintenance still have recurring
work. This PR removes one reason, not the entire chat maintenance timer.
- Worker quiet does not prove database suspension, host sleep, or
external alarm coverage. Cloud sleep blockers remain unchanged.
- Merge eligibility requires passing current-head CI and Greptile 5/5
after this actual-master reconciliation. Subsequent stack slices retain
their own gates.

## Model Used

OpenAI Codex, GPT-6 family. The exact serving model ID and
context-window size are not exposed. Used repository inspection,
reasoning, shell execution, PostgreSQL tests, browser E2Es and GitHub
tools.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details available in this session)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run the documented focused tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 11:07:54 -05:00
DottaandPaperclip 94a780286f fix(server): schedule tool health from committed changes and deadlines (#17554)
## Thinking Path

> - Paperclip manages AI-agent work and its connected tools.
> - Idle hosted instances need their database and network activity to
stop.
> - The maintenance tick queries tool connections even when no health
check is due.
> - Existing connection rows already contain eligibility and the last
health observation.
> - This pull request reconstructs health deadlines and refreshes them
after committed changes.
> - The shared coordinator preserves retries, drain, and shutdown
without another private timer.

## Linked Issues or Issue Description

**What happened?**

The aggregate maintenance tick scans connection health every 30 seconds.
Empty instances and connections with fresh health observations still
cause SQL activity.

**Expected behavior**

No eligible connection means no recurring health SQL. Eligible
connections retain the existing 15-minute proactive check. Configuration
changes and health observations update the schedule after commit.

**Steps to reproduce**

Start a server with heartbeat scheduling enabled and no tool
connections. Let startup finish and observe two maintenance ticks. The
old code repeats the health inventory query. The new startup regression
verifies that those ticks do not invoke the health inventory.

**Paperclip version or commit**

This branch includes merged parent #17518 and master
`96727b3cfefb61f85ffe1fbc29a9b6dd6837efe8`. The 47 changed paths belong
to the single tool-connection health duty. No database file differs from
master.

Related: #14638 covers runtime-auth exclusions. #15448, #15325, and
#11910 concern MCP failure classification. This PR preserves those
provider policies and does not replace their work.

## What Changed

- Keep the GitHub error-projection unit fixture at the health-write
boundary and verify the real transaction path in the PostgreSQL
health-worker suite.

- Add a health schedule owned by tool access. Read lean eligibility
fields and saved observation timestamps.
- Register health work with the existing coordinator. Refresh only dirty
connection/application/company scopes using bounded inventory pages,
check due connections in fair company batches, continue concrete due
backlog after one second, and retain failures with backoff.
- Notify before supported eligibility and observation writes in their
owning transactions. Cover API/gallery setup, OAuth, catalog/gateway
observations, email, voice, access removal, deletion, and settings.
- Commit connection creation with grant, binding, and runtime-slot SQL
before its wake. Keep provider calls outside the new transactions.
- Keep provider-to-transport mapping in the chat catalog. SDK-only chat
changes do not wake health work.
- Replace aggregate health scans, retain the public manual sweep, and
document the producer audit and remaining sleep limits.
- Keep metadata-only example and Smoke Lab installs scoped. Lock and
re-read application type before publishing a real eligibility change.
- Keep failed inventory reads on their own retry deadline while healthy
due batches continue.
- Apply fixture migrations before using external disposable databases in
all six producer suites.

## Verification

Published head: `8c1d293d1ae2b7db753ab20f5a2a2edec1fb0a62`. Fresh CI and
reviews are required for this exact head.

- **623 distinct targeted cases pass.** The final four complete health,
fair-batching, GitHub health recovery, and startup suites pass 80 cases
after the retry fix. Five complete producer suites pass 543 cases: tool
access 469, access 18, email 45, GitHub events four, and Smoke Lab
seven. The earlier 26 health cases are part of this total, not
additional cases.
- The six producer suites first passed against separate, initially empty
PostgreSQL 17 databases at `acbf262843`. They apply their own
migrations. The subsequent repair changes only the health worker and its
regression; its complete final consumer suites pass on that integrated
parent. Server typecheck and build pass with the final repair.
- **Installer amplification:** five real installs with 205 unrelated
connections reproduced 1,035 example rows and 1,045 Smoke Lab rows.
Repeated metadata installs now load only changed fixture connections.
The same cases restore an excluded application through each installer
and require all 205 unrelated IDs to re-enter the schedule.
- **Concurrent eligibility:** a real PostgreSQL lock-wait case requires
the installer to notify after a concurrent type change. A mutation that
retains the lock but uses the old lookup fails the settled-hint
assertion; original source was restored byte for byte.
- **Retry isolation:** the failing baseline read one unavailable company
three times while another company's 60 due checks drained. Final tests
require one initial read, at least 30 seconds between failed reads,
prompt healthy progress, eventual recovery, and SQL quiet after
settlement.
- Real-service producer coverage includes AgentMail activation/pause,
GitHub installation changes, and member access removal. Lifecycle
coverage includes outer commit/rollback, startup reconstruction,
drain/resume, failure, and shutdown.
- Failure history and exact evidence scope: [health
acceptance](https://github.com/paperclipai/paperclip/blob/8c1d293d1ae2b7db753ab20f5a2a2edec1fb0a62/doc/idle-tool-connection-health.md#acceptance-follow-up-after-the-plugin-parent-merged-october-27),
[producer
audit](https://github.com/paperclipai/paperclip/blob/8c1d293d1ae2b7db753ab20f5a2a2edec1fb0a62/doc/plans/2026-10-24-tool-connection-health-sleep-audit.md).
Prior remote failures and earlier 706-case evidence remain historical.
Startup tests emitted listener-count warnings; all assertions passed. No
full local repository-suite, deployed sleep, fleet savings, or live
provider qualification claim is made.

## Risks

- Supported live writers must use the transaction signals. Independent
SQL writers and other replicas need their own wake integration.
- Configured eligible connections still have a real proactive health
obligation. This does not prove Cloud alarm coverage or host/database
suspension.
- Existing manual probes and catalog operations can overlap. This
migration does not introduce a credential-version CAS or change provider
failure policy.
- Voice remains eligible under the previous health policy. Its generic
probe can still report an unsupported transport; changing that behavior
is separate work.
- No database migration, new table, workflow, or lockfile change. This
PR includes its merged parent and current inspected master; later stack
PRs and all current-head acceptance gates remain separate.

## Model Used

OpenAI Codex, GPT-6 family, with repository tools, code execution, and
reasoning. The exact serving version and context-window size are not
exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-27 21:37:50 -05:00
DottaandPaperclip 4e27a82a42 fix(server): wake approval queue seeding from committed changes (#17510)
## Thinking Path

> - Paperclip manages work and human decisions for AI-agent companies.
> - Automatic decision queues store membership for pending attention
sources.
> - The old attention maintenance scan created that membership as a side
effect.
> - Moving approval retention out of the scan also removed approval
queue discovery.
> - Queue API consumers do not build the attention feed first, so they
can miss pending PR approvals.
> - This pull request gives approval queue discovery its own worker and
commit signals.
> - The worker becomes quiet after it saves every missing membership.

## Linked Issues or Issue Description

Refs #17497. Stacked on #17508, following #17505.

This is a correctness dependency for the approval retention migration in
#17497. Do not treat that migration as complete without this
queue-population path. A real PostgreSQL regression found no PR queue
when a pending approval had a linked PR work product and the maintenance
feed excluded approvals.

Related: #13364 proposes pruning stale seeded memberships. This PR
preserves existing membership retention and does not implement pruning.
#15149 documents the decision-queue APIs.

## What Changed

- Add an approval queue worker that selects missing PR-queue memberships
from existing durable records. Bound each pass to 100 rows. Continue
past failed rows and revisit them on retry.
- Reuse the queue service's seed writer through a normalized source
contract. Keep queue rules, uniqueness, and audit writes in that
service. Keep feed readers compatible.
- Wake from committed approval and link changes, issue/company
visibility, PR work-product changes and imports, and queue-rule changes.
Reuse the existing transaction boundaries.
- Recheck approval status under a row lock. Serialize queue-rule changes
with the seed writer. Preserve required recovery during drain and
shutdown.
- Register startup reconstruction under the existing maintenance switch.
Leave other attention-source seeding with its current owner until its
own migration.
- Document the complete producer audit and stack dependency. Add a
validated external PostgreSQL option to the existing queue route suite
so it runs on hosts without embedded database support.

## Verification

Current tested revision: `7948136ef8`. This PR targets master after
#17508 merged as `8406047b4e`. Normal ancestry reconciliation preserved
the complete tested integration tree, before the review repairs below.
Only the original 25 seeding paths differ from master, and database
schema/migration metadata is identical.

- **56 cases pass in the three complete affected suites:** 28 approval
seeding, 6 queue routes and 22 task-watchdog cases. Server typecheck and
production build pass.
- Three real PostgreSQL retry regressions failed then passed: new
matching arrivals cannot postpone a recovered failure indefinitely;
successful partial progress resets old failure penalties; an empty
cursor rollover does not reset outage backoff. Exact one- and two-second
retry assertions, eventual complete recovery and existing
capped-backoff/company isolation tests pass.
- Three genuinely empty external databases failed before setup repair
because tables were missing. Each suite now applies migrations itself.
All 55 cases passed on a second set of fresh databases; adding the final
rollover guard then passed all 56 cases using those now-migrated
fixtures. No production migration or table is added.
- The initial expanded fresh run passed 54 cases but failed a cleanup
expectation that omitted a required cursor rollover pass. The fixture
now permits that explicit continuation while preserving exact deadline
and complete-membership assertions. All failure records remain in
`doc/idle-approval-queue-seeding.md`.
- The prior integrated revision `4e725e2055` passed **522 cases in
fifteen complete producer/consumer suites**, plus types/build. Its
unchanged consumer code and historical deadlock, cross-company
amplification, backoff, gateway integration and setup evidence remain
documented; those counts are not relabeled as current acceptance.

Commands: server Vitest for approval queue seeding, queue routes and
task watchdogs; server typecheck/build. Fresh exact-head
Greptile/Ponytail reviews and all CI checks remain required. No full
local repository-suite, deployed suspension or cost claim is made.

## Risks

- Signals cover the supported in-process service writers. Independent
SQL writers and multiple server replicas need a separate notification
contract.
- The source predicate must stay aligned with unscoped attention
maintenance. The first linked issue, company scope, pending status, and
execution visibility are tested. A pending approval itself keeps its
linked external conversation active.
- Existing enabled seed rules can re-add manually removed matching
memberships. This preserves the old scan's behavior. Completed or
archived membership is not pruned.
- Queue seeding and retention are separate owners. Other attention
sources still need their own event-driven discovery before the aggregate
scan can be retired.
- This PR does not extend Cloud wake coverage, relax sleep admission, or
claim measured host/database suspension.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning and code execution. The exact
serving model identifier and context window were not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-27 20:33:01 -05:00
DottaandPaperclip 553a004926 fix(server): schedule approval retention from durable deadlines (#17497)
## Thinking Path

> - Paperclip manages agent work and should become idle when it owes no
background work.
> - The attention feed archives old pending approvals under the existing
90-day policy.
> - The server rebuilds the feed on each maintenance tick, even when
approval state has not changed.
> - Some approvals have never been read, so retention rows alone cannot
define the work.
> - Approval records and committed eligibility changes define the real
deadline and wake triggers.
> - This PR gives approval retention its own deadline worker and reuses
the existing retention and archive-delivery contracts.
> - The approval lane stops idle scans while other attention sources
remain separate migration work.

## Linked Issues or Issue Description

Stacked on #17490. This PR owns approval retention only. Archive
notification delivery is already separated in #17475. Other
attention-source retention and remaining recurring server duties remain
in the stack inventory. The related-PR search found no overlapping
approval-retention worker migration.

**What existing behavior does this improve?**
Automatic archival of old pending approvals from attention.

**Current behavior**
The shared maintenance tick builds the attention feed for each active
company. That query discovers pending approvals and materializes
retention records before checking their age. An unchanged or empty
approval source is read repeatedly.

**Proposed behavior**
Reconstruct eligible approvals at startup. Schedule the earliest
approval activity timestamp plus 90 days. Wake after committed changes
to approvals, links, issue visibility, company activity, or retention
controls. Stop scheduling when no eligible approval remains. Archive
records through the existing version checks, audit writes, and
notification outbox.

**Reason and benefit**
Idle approval retention should generate no recurring SQL. Unseen
approvals still need eventual archival. A stale candidate must not
archive an approval that was resubmitted while the worker waited for its
source lock.

**Breaking changes**
None intended. The archive age, eligibility, keep controls, audit, and
notification policy remain. Normal attention reads still include
approvals. No Cloud sleep blocker is relaxed.

## What Changed

- Restrict tool-review signals to their existing terminal transition
owners (approve/reject/cancel). Creation/link/revision/resubmit signal
retention only.
- Coalesce retention notifications under one thirty-second aggregate
read budget. Known deadlines still preempt it; no notification or
durable deadline means no scan.
- Add a bounded approval-retention worker. It uses the shared deadline
coordinator and reads pending approvals directly.
- Share first-linked-issue ordering and execution visibility through
`approval-visibility.ts`, used by both the feed and worker. Preserve
actor-specific read filtering and locked archival rechecks.
- Materialize unseen approvals on demand inside manual
keep/archive/revive transactions. Use the saved activity timestamp and
existing versioned state; retain company scoping and rollback on
failure.
- Add awaited pre-write signals to approval, budget, tool, link,
visibility, company, and retention mutation owners. Preserve outer
transaction ownership; keep provider calls outside SQL wrappers.
- Narrow retention inputs to the source fields the existing service
uses. Signal materialization only for inserted or changed state so
unchanged reads converge.
- Exclude approval collection from the maintenance feed before its
query. Other source kinds remain in that sweep.
- Add the pending-only approval deadline index with concurrent,
idempotent creation. Keep the five newest migration snapshots.
- Add real-Postgres lifecycle tests and startup wiring tests. Enable
validated external test databases for existing attention and budget
suites.
- Document producer coverage, quiet conditions, recovery and remaining
scope in `doc/idle-approval-retention.md` and the stack ledger.

## Verification

Current test revision: `dd844f4e8cbbf1db681a774dc31cd552301bef48`,
followed by docs-only `3efda33812`, which received both 5/5 reviews and
passing CI. This candidate normally integrates landed Grok `2adcde162e`.
All runtime and acceptance-test sources are byte-identical to that
tested/reviewed tree; only four independent Runner E2E accounting files
and the ledger differ. Migration 0392 and the parent journal prefix are
unchanged. Local checks are reused for their unchanged source; fresh
final-head CI and reviews are required against master. This remains the
approval-retention duty.

- **315 cases pass** across eight complete suites: approval
retention/service, attention, decision retention, budgets, tool gateway,
server startup and scoped deadlines. All 30 real-PostgreSQL lifecycle
cases pass. Server production typecheck and build pass.
- **Four regressions failed before repair**: unseen young
keep/archive/revive returned 404; the manual keep rollback case could
not reach its update. They now pass. A failed keep update leaves no
retention row or audit; retry succeeds with one audit.
- Both linked-issue orders are checked against the feed: hidden
first/visible later is excluded; visible first/hidden later remains
eligible. The feed and worker use the same owner query, with actor read
filtering preserved.
- Generated concurrent/idempotent migration
`0392_small_frightful_four.sql` adds only the pending approval index.
Every historical parent SQL migration and all 390 parent journal entries
are unchanged. Five generated snapshots remain (0388–0392). Schema
drift, migration safety/numbering and actual fresh apply/replay on five
isolated PostgreSQL databases pass.
- Existing precision, transaction, drain/resume, failure fairness and
shutdown tests remain. Earlier transaction-stub, budget-import,
diagnostic-client and CI failures are recorded in the owning audit and
stack ledger; later passes do not erase them.

- Three additional real regressions failed before the wake repair:
ordinary mutations emitted tool-review signals in both resolution cases,
and sequential tenant writes caused repeated global reads. Tests now
prove terminal approve/reject/cancel signals and one aggregate
reconciliation per thirty-second window with a different company's
pending approval retained. The aggregate budget is stricter than a
separate per-company budget and matches the old default cadence; known
deadlines preempt it.
- The initial coalescing fixture expected the concrete pending-work hold
to clear before reconciliation; it now preserves that hold and measures
SQL. The first broad repair passed 313 cases but failed the existing
parked-review cancel case. The original cancel producer is restored; all
315 now pass without weakening assertions. At old head `5991f257bb`,
security was action-required and Greptile returned service error
`8fb1835c-e564-476b-b802-d5be164ea915`. Fresh repaired-head results are
required, not assumed.

Commands: server Vitest for `approval-retention-work`,
`approvals-service`, `attention-service`, `decision-retention-service`,
`budgets-service`, `tool-gateway-service`,
`server-startup-feedback-export`, `scoped-deadline-work`; server
typecheck/build; DB snapshot drift, migration checks and fresh
apply/replay. Fresh exact-head Greptile/Ponytail and CI remain merge
gates. No full local repository suite or deployed host suspension/cost
claim is made.

Earlier stack verification below is historical evidence at the
explicitly named revisions, not this head's acceptance:

Approved delegated Test-tab calls now evaluate caller and target
policies in the existing claim transaction. Real HTTP regressions first
showed a missing caller charge and shared-limit self-rejection. All 78
connection-management cases and 92 gateway cases now pass on the owning
branch, along with server production typechecking. All 78
connection-management cases also pass at combined tip
`d25f830db89c1623bff3547bfc68015c1c66e0cf`. The three new cases cover
shared charges, separate charges, rollback on denial, duplicate
approval, and rejection of a later authorized call. Intermediate
counter/identity fixture failures and a reused-database Slack collision
are recorded in [the owning
audit](https://github.com/paperclipai/paperclip/blob/20557d2c3dd0949c734eaa8095fd78b9b07d7fd5/doc/idle-approved-tool-recovery.md#delegated-test-tab-admission-after-policy-changes).
The earlier combined checks below remain evidence for their stated
revisions. Current-head CI and review remain required.

Earlier published head: `20557d2c3dd0949c734eaa8095fd78b9b07d7fd5`. This
remains a single-duty PR stacked on #17490.

- Integrated pinned master `8548173fc9efdc0fa07644fb92dcc0b6851a95e5`
with normal merges. Every branch passed generated-schema checks and its
target-branch migration-order gate. Existing SQL is unchanged; the 25
stack migrations follow master’s new migration.
- Duty-specific producer coverage, lifecycle tests, and earlier failure
records:
[2026-10-24-recurring-server-sleep-stack.md](https://github.com/paperclipai/paperclip/blob/76a49810c7ff403ae7720bef0d453747bd844731/doc/plans/2026-10-24-recurring-server-sleep-stack.md).
- Combined tip `51e6cbe67979b55b340ea9f0f434c5b5c682ccd6`: **355
startup/cancellation/recovery/dispatch/preservation tests**, **75
connection-management tests**, and production server typechecking pass.
These combined checks do not mean every intermediate PR independently
ran the full repository suite.
- Real master-to-stack database upgrade passes: **379 unchanged prior
migration receipts plus 25 new migrations**, with the gateway
responsibility fixture preserved.
- Owning-branch checks: 84 runner installation/settlement cases plus
runner typechecking and generated TypeScript/Rust profile agreement; 253
GitHub review workflow cases. Stop-attribution and held-worker readiness
regressions failed before their fixes.
- Failure accounting: the initial combined run passed 349 cases and
failed six idle assertions in a reused preservation database containing
old cancelled runs. Repeating the combined command with a fresh
preservation database passed. Earlier invalid project selection, merge
import/type errors, and the interrupted full local suite are not counted
as passing checks.
- **Latest-head CI and review remain required.** Prior CI findings
include GitHub idle observation, route teardown, attention cleanup,
touch-picker E2E, workspace health, and voice-cleanup timing; local
passes do not erase those failures. No full local repository-suite,
deployed sleep, fleet cost, or live Copilot qualification claim is made.

## Risks

- Signals cover the application database owner and its child pools.
Independent SQL writers and other active processes need an explicit wake
integration. Startup reconstructs committed durable state.
- The source row lock prevents stale approval-activity archival. It does
not add cross-table serializability for concurrent link, visibility, or
company-policy changes.
- Newly changed eligibility can wait up to the thirty-second
notification window, matching the former default maintenance cadence.
Saved deadlines still preempt; no new idle polling is added.
- Failed records retain recovery debt. Bounded cursor progress lets
later rows proceed without dropping failures.
- The shared retention transaction now groups materialization and
activity-version updates. Existing attention and retention consumers are
tested against PostgreSQL.
- Kept, archived, resolved or inactive-company approvals can become
quiet, but their supported mutations must retain their wake
notifications.
- The local scheduler cannot wake stopped compute. This PR neither adds
approval deadlines to Cloud's external alarm contract nor claims
measured host or database suspension.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning and code execution. The exact
serving model identifier and context window were not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-27 19:23:24 -05:00
DottaandPaperclip 8161f2a813 fix(server): wake tool-review card repair from committed changes (#17468)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Tool requests and invocations store the execution outcome. Review
cards show that outcome.
> - The maintenance tick repeatedly joins these tables even when every
card matches.
> - Committed changes can wake repair. Existing rows can restore work
after restart.
> - A large company backlog also needs a bounded pass so other companies
and shutdown can proceed.
> - This PR uses the shared coordinator for scoped repair, continuation
and failure deadlines. Settled history becomes quiet.

## Linked Issues or Issue Description

**What happened?**

The aggregate maintenance tick scans tool-review projections on every
tick. Matching terminal history still causes database work. A company
with many stale cards can also hold one worker in an unlimited drain.

**What did you expect?**

Relevant writes and startup should wake repair. Failed or unresolved
cards should retain recovery. Matching history should cause no recurring
SQL. Each normal worker pass should honor drain and shutdown.

**Steps to reproduce**

Start an instance with completed tool requests and matching cards. Leave
it idle. The old maintenance tick keeps querying for mismatches. A large
mismatch backlog also drains without a row limit or cancellation check.

This is slice 07 of the sleep migration. It now targets master after
#17458 landed. Related owners are pending expiry #17453 and approved
recovery #17454. Their execution duties remain separate from card
repair. The prior open-PR search found no duplicate projection
migration.

## What Changed

- Register company-scoped projection repair with the existing delivery
coordinator. Remove its recurring aggregate-tick scan.
- Signal through the existing review-write transaction owner and linked
interaction mutations. Ordinary unlinked confirmations do not wake
repair.
- Keep interpretation, locks, current-state checks, activity and
continuation callbacks in the gateway. A repair never repeats a provider
tool call.
- Limit each normal worker pass to 500 rows. Check abort and admission
between rows. Save company progress through truncated and failed passes
so blocked early pages cannot hide later cards.
- Schedule truncated work after one second. Back off unresolved cards
from thirty seconds to five minutes; fresh committed hints reset the
delay. Clear obsolete cursors after empty reconstruction.
- Skip repeated company discovery for scoped hints and the extra
candidate query on an empty sweep with no saved cursor. Attempted or
resumed passes still verify remaining mismatches because reflection
returns no settlement result.
- Add generated migration `0386_steady_epoch.sql` for the existing
`(company_id, interaction_id)` lookup. Use the existing concurrent-index
executor. Add no table.

## Verification

Current candidate: `d4706fbf2a04a00ade30181eb1bd9322e8455818`. All 222
cases in the full gateway/projection/interruption, startup and
scoped-deadline suites pass without skips after the query/retry
efficiency repair; server types and build pass. A single committed
company hint with matching retained history and a separate empty direct
sweep each perform one candidate query. The unresolved-card regression
failed before repair and now verifies backoff through the five-minute
cap, prompt producer activation and quiet after settlement. An initial
assertion incorrectly assumed an entire approval emitted one wake; its
separate real writes require isolating the single-hint measurement.

The earlier master integration `fe6e10629b` additionally established the
following evidence before this efficiency follow-up:

- **320 cases pass** across full gateway/projection/interruption,
startup, scoped-deadline, narrowed identity-read and authenticated
gateway HTTP suites. The HTTP selection includes all 81 cases.
- **Six real PostgreSQL regressions failed before their fixes:** pass
bounds, abort, drain, progress past unprojectable pages, progress past
failed pages, and first repair after empty reconstruction. The complete
owning selection passes all 220 cases.
- Server production typechecking and build pass. Generated index
migration, snapshot drift and numbering/safety checks pass. The five
newest snapshots remain; master journal entries are unchanged.
- A real missing-link query on 30,000 history rows changed from a
sequential scan that filtered all 30,000 rows and touched 429 buffers to
the new index with two buffers. Both treatments returned no rows.
- Normal master integration preserves 35 independent performance files
exactly. The gateway also retains master's narrowed identity read.

Commands include `pnpm --filter @paperclipai/server typecheck`, `pnpm
--filter @paperclipai/server build`, and server Vitest selections for
`tool-gateway-service`, `server-startup-feedback-export`,
`scoped-deadline-work`, `run-identity-projections`, and
`connection-management.integration`, with separate PostgreSQL test
fixtures.

[The behavior and failure audit](doc/idle-tool-review-projection.md)
records the producer contract, earlier fixture failures, index
measurement, red/green regressions and integration evidence. Fresh
current-head CI and Greptile/Ponytail remain required before merge.

## Risks

- Signals use the existing single-active-writer database owner. Another
replica or independent SQL writer needs a wake contract.
- An already issued database query or reflection finishes before
cancellation takes effect. Checks between rows bound further dispatch.
- Legacy manual/global and disabled-scheduler one-shot repairs keep
their complete-sweep behavior. The normal coordinator worker always
supplies its row budget and admission controls.
- Unprojectable cards and thrown failures retain recovery with a
five-minute backoff cap. The company mismatch query still grows with
terminal history. Empty history does not retain a recurring scan.
- The index migration uses concurrent creation and the existing
invalid-index recovery path. Prior SQL and journal entries remain
unchanged.
- Cloud sleep eligibility and external-wake coverage do not change.
Local worker quiet is not proof of host or database suspension.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning and code execution. The exact
serving model identifier and context window were not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-27 17:09:31 -05:00
DottaandPaperclip f6bc653f7f fix(server): schedule interrupted tool claims by their deadlines (#17458)
## Thinking Path

> - Paperclip governs tool execution and keeps durable approval records.
> - An interrupted execution may have produced an external effect
without a saved final request state.
> - The maintenance tick currently scans executing requests even when
none exist.
> - Every executing claim already has a ten-minute stale deadline.
> - This PR schedules that deadline from committed claim changes and
clears it after settlement.
> - Recovery preserves saved provider outcomes and quarantines unknown
outcomes without another provider call.

## Linked Issues or Issue Description

Refs #17454. This is the sixth recurring-server sleep slice, stacked on
`codex/sleep-05-approved-tool-recovery`.

**Problem**
The aggregate maintenance tick repeatedly searches for stale executing
tool requests. Its settlement can overwrite a saved successful
invocation with an unknown-outcome failure after a crash between
invocation and request writes.

**Proposed Solution**
Use the existing review notification boundary and shared scheduler. Read
the earliest executing claim deadline through a partial index. At that
deadline, lock invocation then request, recheck claim identity, and
atomically settle from the saved outcome or quarantine the unknown
outcome.

**Alternatives Considered**
A longer polling interval still queries empty work. A new queue would
duplicate existing records. Replaying an interrupted call can duplicate
an external effect. Treating every stale request as unknown discards
already saved outcomes.

## What Changed

- Coalesce sequential review notifications for up to 30 seconds while
preserving already loaded recovery deadlines. Do not schedule scans when
no notification or execution deadline remains.

- Extract interrupted-execution deadline lookup and settlement into the
owning tool-review service.
- Register its worker at startup. Use committed claim/result changes to
update the existing ten-minute deadline; retry failed settlement after
30 seconds.
- Preserve terminal invocation results, reasons, and completion times
when repairing request state.
- Settle uncertain request/invocation state atomically with the existing
unknown-outcome reason, without dispatching a provider call.
- Notify all gateway claim and settlement paths through the existing
review transaction helper. Keep framework wiring and the shared
scheduler free of provider-specific behavior.
- Add an idempotent concurrent partial index for executing deadlines.
Preserve the combined manual sweep and scheduler-disabled startup pass.
- Remove interrupted scanning from the aggregate tick. Interaction
projection repair remains the next separate duty.

## Verification

Current head: `648d8a9a64ded15deeeb331545b92a12d69e277d`. The final
commit changes no files and requests CI after the master retarget. A
preceding run appeared while that trigger was being published; it was
observational delay, not an established CI outage. No test rerun or
bypass was used to turn a failure green. Source remains identical to the
198-test passing integration, apart from the already recorded ledger
update. Fresh checks and both automated reviews must qualify this
current head before merge.


Current master-targeted head:
`d55e44b376555e4926bb2c2a0e4dcfb431324162`. Parent #17454 landed as
`77295faebc8`; a normal merge reconciles that squash. Every conflicted
master file was checked against the already-integrated parent, and all
matched byte-for-byte. Each interruption-only hunk was retained; the
complete integrated source tree matched the 198-test passing
`848f1b87b3` exactly before the sole final documentation update. All
master migration journal entries are unchanged, with only this PR's
executing-deadline index appended. Server types pass. Fresh CI and
current-head Greptile/Ponytail are required before the next authorized
merge.


Latest published integration:
`848f1b87b3a6646ae4e56e8ab142f64889d7a69c`, normally merging
approved-recovery parent `23868c1751`. All 198
gateway/interruption/startup/deadline cases and server production
typechecking pass. Both parent approval fixes and all interruption tests
are retained. The two merge conflicts preserved the interruption import
alongside the parent's claim-bound hash helper and preserved the
parameterized historical test helpers. This remains stacked on #17454;
fresh CI and both automated review gates are required after integration.


- Review documentation correction at
`c985eb9d1b48a481909de6a1c2d89d7af11bfc03`: the focused real-PostgreSQL
interruption-worker selection passes all 16 cases; 124 unrelated cases
were filtered out. The docs now correctly name 16 cases and six saved
terminal statuses. No production behavior changed in the correction.
Fresh remote CI/review is pending after integrating the verified parent.

Approved delegated Test-tab calls now evaluate caller and target
policies in the existing claim transaction. Real HTTP regressions first
showed a missing caller charge and shared-limit self-rejection. All 78
connection-management cases and 92 gateway cases now pass on the owning
branch, along with server production typechecking. All 78
connection-management cases also pass at combined tip
`d25f830db89c1623bff3547bfc68015c1c66e0cf`. The three new cases cover
shared charges, separate charges, rollback on denial, duplicate
approval, and rejection of a later authorized call. Intermediate
counter/identity fixture failures and a reused-database Slack collision
are recorded in [the owning
audit](https://github.com/paperclipai/paperclip/blob/f6a5949fb726f71c247950714adecb275655284f/doc/idle-approved-tool-recovery.md#delegated-test-tab-admission-after-policy-changes).
The earlier combined checks below remain evidence for their stated
revisions. Current-head CI and review remain required.

Current head: `f6a5949fb726f71c247950714adecb275655284f`. This remains a
single-duty PR stacked on #17454.

- Integrated pinned master `8548173fc9efdc0fa07644fb92dcc0b6851a95e5`
with normal merges. Every branch passed generated-schema checks and its
target-branch migration-order gate. Existing SQL is unchanged; the 25
stack migrations follow master’s new migration.
- Duty-specific producer coverage, lifecycle tests, and earlier failure
records:
[2026-10-24-recurring-server-sleep-stack.md](https://github.com/paperclipai/paperclip/blob/eaac8d343e5ef4d2c13cdf7cb7607cb5a498e9a2/doc/plans/2026-10-24-recurring-server-sleep-stack.md).
- Combined tip `51e6cbe67979b55b340ea9f0f434c5b5c682ccd6`: **355
startup/cancellation/recovery/dispatch/preservation tests**, **75
connection-management tests**, and production server typechecking pass.
These combined checks do not mean every intermediate PR independently
ran the full repository suite.
- Real master-to-stack database upgrade passes: **379 unchanged prior
migration receipts plus 25 new migrations**, with the gateway
responsibility fixture preserved.
- Owning-branch checks: 84 runner installation/settlement cases plus
runner typechecking and generated TypeScript/Rust profile agreement; 253
GitHub review workflow cases. Stop-attribution and held-worker readiness
regressions failed before their fixes.
- Failure accounting: the initial combined run passed 349 cases and
failed six idle assertions in a reused preservation database containing
old cancelled runs. Repeating the combined command with a fresh
preservation database passed. Earlier invalid project selection, merge
import/type errors, and the interrupted full local suite are not counted
as passing checks.
- **Latest-head CI and review remain required.** Prior CI findings
include GitHub idle observation, route teardown, attention cleanup,
touch-picker E2E, workspace health, and voice-cleanup timing; local
passes do not erase those failures. No full local repository-suite,
deployed sleep, fleet cost, or live Copilot qualification claim is made.

## Risks

- Signals cover the existing single-active-writer database owner.
Independent writers and replicas need an explicit wake contract.
- The ten-minute policy is unchanged. A provider call that outlives its
claim remains an uncertain outcome; this worker never retries the
external call.
- Interaction projection follows authoritative settlement outside its
transaction. Its separate existing repair duty still handles an
interrupted projection.
- Non-executing review writes can cause one empty reconciliation.
Without an executing claim or failed attempt, no recurring deadline
remains.
- Cloud eligibility and external-alarm coverage remain conservative.
Worker quiet does not prove actual host or database suspension.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning and code execution. The exact
serving model identifier and context window were not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-27 16:33:30 -05:00
DottaandPaperclip c3a9a8e92a perf(server): scope run identity reads to operation needs (#17931)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Tool and connection operations capture the current accepted
requester.
> - Capture locks the task and run, then reconciles pending steering.
> - Most callers use identity or task fields from the returned run.
> - The default query also transfers saved prompts, runner
configuration, results, and logs.
> - This pull request gives each production caller an explicit read
scope.
> - Narrow scopes keep their authority checks and return only selected
columns.
> - The Muse project path still reads its complete saved execution
input.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Operation-time run identity capture for native tools, gateway sessions,
project and workspace access, Slack, voice, connection management, and
connection token admission.

**Current behavior**

These callers use the default full run read. Large saved execution
documents cross the database connection even when the caller only needs
the current accepted requester or task binding.

**Proposed behavior**

Use typed identity, task, broker, and project-execution scopes. Keep
task-before-run locks, company and agent predicates, active status
validation, and pending-steering reconciliation. Return the complete
saved native input only where project admission uses it.

**Reason and benefit**

A synthetic PostgreSQL 17.11 fixture matches a wide run with a 142 KB
context and a 77 KB runner profile. Its locked run DataRow falls from
225,101 bytes to 190–605 bytes for authority reads. The project scope
returns 66,004 bytes because it must retain a complete 64 KB execution
input. These are local DataRow measurements, not a forecast of
production billing or latency.

**Breaking changes**

None to public APIs, saved data, or authority policy. Internal narrow
return types omit columns that were not selected. The default full
compatibility read and the existing GitHub-credential and authentication
scopes keep their current contracts.

**Additional context**

The public issue and PR duplicate searches found no matching open
change. This is one independent query optimization. It does not require
other performance PRs.

## What Changed

- Add typed scopes for accepted identity metadata, task authorization,
token attribution, and saved project execution input.
- Assign explicit scopes at all production `captureRunIdentity` call
sites.
- Keep the full native execution input for Muse project admission while
omitting unrelated profile, context, result, and log data.
- Narrow the broker's initial read and the existing Slack and
connection-management historical reads to the same required fields.
- Extend steering and acknowledgement regressions to all new scopes. Add
real PostgreSQL projection, malformed JSON, tenant and agent,
session-generation, and saved-input coverage.
- Add a service-generated SQL benchmark with exact raw DataRow
accounting, retained-field equality, plans, and alternating transaction
timings.
- Document the scope contracts and reproducible benchmark.

## Verification

- Baseline: `623f1dc289a14a699ebbe3f09b94179d38fc2ae2`.
- `vitest run server/src/__tests__/run-identity-projections.test.ts
server/src/__tests__/run-identity.test.ts`: 54 tests passed on an owned
disposable PostgreSQL 17.11 database.
- The tests retain task-before-run and audit foreign-key lock behavior.
Every scope rejects uncertain steering, accepts its durable
acknowledgement, and preserves requester replay ordering.
- A full-row mutation makes the narrow identity regression fail. The
restored treatment passes.
- `pnpm --filter @paperclipai/server exec tsx
../scripts/benchmark-run-identity-projections.ts
--baseline-ref=623f1dc289a14a699ebbe3f09b94179d38fc2ae2` captures the
real locked SQL for both versions.
- Every retained column and selected JSON field matches the baseline.
Identity contexts match. The complete stored run remains unchanged.
Capture keeps the same five queries.

| Scope | Before DataRow bytes | After DataRow bytes | Reduction |
| --- | ---: | ---: | ---: |
| Identity | 225,101 | 190 | 99.92% |
| Task authorization | 225,101 | 410 | 99.82% |
| Token broker | 225,101 | 605 | 99.73% |
| Project execution input | 225,101 | 66,004 | 70.68% |

- DataRow bytes include row framing and raw text-format column values.
They exclude response metadata, TLS, and TCP.
- Two warmups and seven alternating samples measure the complete capture
transaction. Several timing ranges overlap. The primary claim is byte
reduction; production latency is not verified.
- Initial fixture runs exposed an empty external schema and an
incomplete synthetic conversation binding. The test setup now applies
migrations and supplies the required user/state fields. The final 54
tests passed.
- `pnpm -r typecheck` and `pnpm build` passed locally.
- Four existing skill-setup tests passed. Four assembled
connection-management/native-tool consumer cases passed with real
JWT/API-key authentication. These include per-request identity freshness
and task-scoped activity authority.
- A CI-only stale mock assertion expected two capture arguments; it now
verifies the explicit identity scope. A repository-wide capture mock
audit found no other stale argument assertions.
- Review follow-up: the projection suite uses the existing
embedded-support guard while always enabling an explicitly supplied
external database. The audit-lock regression merges URL query parameters
and retains existing PostgreSQL options.
- `vitest run server/src/__tests__/run-identity-projections.test.ts
server/src/__tests__/run-identity.test.ts
server/src/__tests__/company-skills-routes.test.ts
server/src/__tests__/project-skill-setup-identity.test.ts`: 123 tests
passed. The external PostgreSQL URL included both `application_name` and
`options`; lock tests verified `lock_timeout=1s` and retained
`statement_timeout=5s`.
- Full local tests are not run because the host's embedded PostgreSQL
setup is unavailable. Greptile reviewed
`06e1dd53a3fb24df74d5fbfb2ad397f291f1e30f` at 5/5 with zero unresolved
threads. The full final-head CI matrix passes. All 58 checks are
terminal: 55 success, one neutral, and two optional Storybook skips.
- No migration, lockfile, workflow, new polling, or cache change.

## Risks

A narrow read must retain each caller's actual fields. The caller audit
includes Slack's wake-request fallback, conversation generation, legacy
broker attribution, and the Muse parser's complete native input. JSON
selection retains value types, including malformed legacy values. The
saved-input scope preserves missing-key behavior. The default full read
still returns the complete stored row for compatibility. Locks and live
authority checks are unchanged.

## Model Used

OpenAI GPT-6, accessed through Codex. The exact runtime model ID and
context-window size were not exposed in this session. Used repository
inspection, code execution, and PostgreSQL validation.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-27 16:31:21 -05:00
DottaandPaperclip 77295faebc fix(server): recover approved tool actions from committed changes (#17454)
## Thinking Path

> - Paperclip manages agent work and its governed tool calls.
> - A human approval can persist before the approving process dispatches
the tool.
> - Recovery currently scans approved requests on every maintenance
tick.
> - Completed and deliberately inert approvals do not owe another
execution.
> - Test-tab approval also needs a shared execution claim before
recovery can safely race the foreground path.
> - This PR wakes recovery from committed changes and retries only
unresolved approvals.
> - Idle history becomes quiet while existing approval and execution
checks remain with the gateway.

## Linked Issues or Issue Description

Refs #17453. This is the fifth slice of the recurring-server sleep
migration, stacked on `codex/sleep-04-tool-review-expiry`.

**Problem**
Approved tool requests are scanned periodically even when no executable
work remains. The Test-tab foreground path lacks the request claim
needed to admit concurrent recovery safely.

**Proposed Solution**
Use the existing transaction-aware review notification boundary and
shared deadline worker. Keep signed-payload interpretation, current
policy and target checks, and provider dispatch in the gateway. Share a
claim between Test-tab foreground and recovery, then persist both
authoritative outcome rows atomically.

**Alternatives Considered**
A longer interval still scans idle history. A new durable queue would
duplicate existing request/invocation state. Dispatching Test-tab
approvals without a claim risks duplicate external calls.

## What Changed

- Recheck both principals for delegated Test-tab approvals. Save their
rate receipts in the same claim transaction, share only the same bucket,
and roll back new charges on denial.

- Preserve delegated gateway authorization and both issuer/target rate
receipts inside the existing claimed-invocation transaction; retain the
durable actor during approved replay.

- Expire stale managed connector payloads before consuming approved-tool
admission.

- Persist current rate-policy charges and their invocation receipts
atomically before dispatch, preserving exact approval claim fencing and
read-only preflight.

- Recover approved requests at startup and after committed review
changes; retry only unresolved approvals.
- Leave legacy agent approvals without execute-on-approve intent inert.
Repair completed Test-tab history without another provider call.
- Fence Test-tab execution and outcome settlement with an atomic claim
shared by both callers. Revalidate the approved target and current
policy before dispatch.
- Keep provider calls outside database transactions. Retain quarantine
for interrupted execution and unknown outcomes.
- Add a concurrent, idempotent partial index for approved-request
pagination. Reuse the parent's company-scoped review signals and shared
scoped deadline worker.
- Remove approved scanning from the aggregate tick. Preserve the public
combined sweep and scheduler-disabled startup behavior.
- Document producer ownership, quiet states, lifecycle behavior, and
remaining recovery duties.

## Verification

Latest published head: `23868c1751c100a27f58087f7ce16b93c52d4eba`,
targeting master. The review-found delegated gateway pool deadlock is
repaired by passing the admission transaction through token and
issuer-authority reads. Foreground and recovery each reproduced the
deadlock with a one-connection pool; all three constrained-pool cases,
including revoked authority, now pass. All 81 HTTP integration cases
pass. The historical foreground rate-slot regression also reproduced
before its repair; verified preview hashes are accepted only by
read-only preflight, while consuming admission still requires the
complete identity under the exact claim. Five full suites passed 329
cases; the gateway service then passed all 141 cases after two
additional mixed historical hash controls. Production types/build passed
before the final normal integration of master `35820cf398` (including
independent Grok/Runner changes); integration validation is running.
Initial fixture/assertion errors and the prior E2E
restoration/cancellation deadlock remain recorded in [the owning
audit](doc/idle-approved-tool-recovery.md#gateway-admission-uses-its-transaction-connection).
Fresh-head CI, Greptile 5/5, and passing Ponytail are still required
before merge.


- Master landing reconciliation at
`8dfe3958db6e4e8a0b7499f3f5be0a396ca77361` after #17453 merged:
production and server test files are byte-identical to the accepted
320-case repair. The landed migration journal prefix is unchanged, and
#17928's independent eval reporting changes are preserved. Fresh CI and
reviews on this exact head are required before merge.

- Historical-identity repair published at
`096b7c7198c3ca864aca9377e261cbaa0a8aad6f` (production/test source
identical to integrated acceptance
`81231824d773a79fc50c835eccf7d9b3977e79fc`): all 320 accepted gateway,
policy, startup, preparation-timing and authenticated HTTP management
cases pass. Server types and the production server build pass. Four real
historical sensitive-argument regressions failed before repair;
invalid-signature, foreign-hash and stale-claim controls remain denied
without dispatch or identity changes. Both agent and Test-tab calls
deliver the original signed arguments once, and new rate policies still
apply.
- The first integrated combined run passed 319 cases and hit a Slack bot
ownership collision left in a reused HTTP test database. The guard
correctly returned 409. The full 78-case HTTP suite passed on a fresh
dedicated fixture without product or assertion changes. Earlier
fixture-construction errors are distinguished from the four real
regressions in the audit document. Fresh remote CI and reviews remain
required.

- Parent reconciliation at `736e7f56097055e21ce13b4e45d9033391483af8`:
299 cases pass (221 full gateway/policy/startup cases and 78
assembled-server connection management cases), with no skipped cases.
Server types pass. This includes real delegated Test-tab approval
admission for newly added shared and caller rate rules. Fresh remote CI
and Greptile review remain required.

Approved delegated Test-tab calls now evaluate caller and target
policies in the existing claim transaction. Real HTTP regressions first
showed a missing caller charge and shared-limit self-rejection. All 78
connection-management cases and 92 gateway cases now pass on the owning
branch, along with server production typechecking. All 78
connection-management cases also pass at combined tip
`d25f830db89c1623bff3547bfc68015c1c66e0cf`. The three new cases cover
shared charges, separate charges, rollback on denial, duplicate
approval, and rejection of a later authorized call. Intermediate
counter/identity fixture failures and a reused-database Slack collision
are recorded in [the owning
audit](https://github.com/paperclipai/paperclip/blob/bcca5b006976a471f4afd1f70c4031aa40a2b3f7/doc/idle-approved-tool-recovery.md#delegated-test-tab-admission-after-policy-changes).
The earlier combined checks below remain evidence for their stated
revisions. Current-head CI and review remain required.

Current head: `bcca5b006976a471f4afd1f70c4031aa40a2b3f7`. This remains a
single-duty PR stacked on #17453.

- Integrated pinned master `8548173fc9efdc0fa07644fb92dcc0b6851a95e5`
with normal merges. Every branch passed generated-schema checks and its
target-branch migration-order gate. Existing SQL is unchanged; the 25
stack migrations follow master’s new migration.
- Duty-specific producer coverage, lifecycle tests, and earlier failure
records:
[2026-10-24-recurring-server-sleep-stack.md](https://github.com/paperclipai/paperclip/blob/a021467f04d22be42ee5b0c87b6583b2300a62a9/doc/plans/2026-10-24-recurring-server-sleep-stack.md).
- Combined tip `51e6cbe67979b55b340ea9f0f434c5b5c682ccd6`: **355
startup/cancellation/recovery/dispatch/preservation tests**, **75
connection-management tests**, and production server typechecking pass.
These combined checks do not mean every intermediate PR independently
ran the full repository suite.
- Real master-to-stack database upgrade passes: **379 unchanged prior
migration receipts plus 25 new migrations**, with the gateway
responsibility fixture preserved.
- Owning-branch checks: 84 runner installation/settlement cases plus
runner typechecking and generated TypeScript/Rust profile agreement; 253
GitHub review workflow cases. Stop-attribution and held-worker readiness
regressions failed before their fixes.
- Failure accounting: the initial combined run passed 349 cases and
failed six idle assertions in a reused preservation database containing
old cancelled runs. Repeating the combined command with a fresh
preservation database passed. Earlier invalid project selection, merge
import/type errors, and the interrupted full local suite are not counted
as passing checks.
- **Latest-head CI and review remain required.** Prior CI findings
include GitHub idle observation, route teardown, attention cleanup,
touch-picker E2E, workspace health, and voice-cleanup timing; local
passes do not erase those failures. No full local repository-suite,
deployed sleep, fleet cost, or live Copilot qualification claim is made.

## Risks

- Notifications cover the existing database-owner, single-active-writer
contract. Independent SQL writers and other replicas need an explicit
wake path.
- Unknown provider outcomes cannot be replayed safely. Failed result
persistence retains an executing claim and the existing stale-execution
quarantine.
- Interrupted execution and terminal interaction projection repair
remain periodic until their separate slices. This PR changes only
approved recovery.
- A pending-only review write can trigger one empty approved-recovery
pass. Settled idle work schedules no further scan.
- The partial index uses the existing top-level concurrent migration
executor. Snapshot pruning follows repository policy.
- Cloud sleep eligibility and external alarm coverage remain unchanged.
Worker quiet is not proof of deployed instance suspension.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning and code execution. The exact
serving model identifier and context window were not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-27 16:10:44 -05:00
DottaandPaperclip d108aeeaaf fix(server): schedule pending tool-review expiry from committed changes (#17453)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Governed tools can wait for a human review before execution.
> - Pending reviews already save a deadline and must reject approval
after expiry.
> - The server currently scans for expired reviews on every maintenance
tick.
> - Empty scans keep the database active without advancing work.
> - This pull request schedules pending review expiry from committed
changes and saved deadlines.
> - Configured idle state and resolved history no longer cause this
expiry scan.

## Linked Issues or Issue Description

**What happened?**

The combined tool review sweep repeatedly queries pending expiry even
when no pending request has a deadline.

**What did you expect?**

Signing or resolving a request should update the expiry schedule. A
saved expiry should trigger settlement. Once no expiry remains, this
duty should stop querying.

**Steps to reproduce**

Start a claimed instance with heartbeat scheduling enabled. Leave it
idle with no pending tool reviews. Observe pending-review expiry
discovery in the maintenance sweep.

Stacked on #17450; earlier slices are #17446 and #17443. These all reuse
the delivery and transaction foundations from #15625. This PR owns
pending-review expiry only. Approved-action recovery,
interrupted-execution quarantine, and terminal interaction projection
repair remain separate migrations. No duplicate pending-review expiry
migration was found in the open PR search; credential lifetime changes
in #17221 and #17274 concern the parent credential slice.

## What Changed

- Replace repeated company review-deadline aggregates with an indexed
single-row lookup. Add and replay a concurrent covering-index migration
and generate combined snapshots throughout the stack.

- Extract pending-expiry selection and settlement into its owning
service. Keep provider execution and interaction publication with the
gateway.
- Signal deadline assignment and pending resolution inside the owning
SQL transaction. Preserve outer commit/rollback and central uncertainty
handling.
- Register company expiry deadlines with the shared scoped coordinator.
Reconstruct on startup; normal producer hints reload only their company.
Preserve writes during drain and concurrent reconciliation.
- Settle request expiry and invocation failure atomically. Keep
interaction projection recovery in the existing separate sweep.
- Remove pending-expiry scanning from the main tick. Preserve the
combined manual API and startup-only behavior when scheduling is
disabled.
- Add a partial pending-expiry index through an idempotent concurrent
migration. No queue table is added.
- Document supported producers, quiet states, remaining duties, and
external wake limits.

## Verification

- CI run `40574661575` at `f5c935d1b89877033eb82bf890f867baeb372ef3`
lost the runners for general-server shard 9/12 and E2E shard 2/8 at
20:09:03 UTC. Both job logs explicitly show a runner shutdown signal
followed by cancellation; no test assertion failure was reported by
those jobs. A single infrastructure retry is justified once the run
completes; it has not been issued yet. Both current-head reviews are
5/5, and CI must still be green before merge.

- Current master landing at `f5c935d1b89877033eb82bf890f867baeb372ef3`:
128 complete gateway expiry, startup, scoped deadline and
preparation-timing cases pass with no skips. Server types and the
production server build pass. Landed tracing changes from #17918 are
preserved. The master migration journal prefix remains semantically
identical, with only this PR's two migrations added. Fresh remote
CI/review is required before merge.

- Parent reconciliation at `17539e2a6cf8ac4922b3cbe06af6df361bfe5665`:
116 tests pass across the full tool gateway service, startup and scoped
deadline suites against a dedicated migrated PostgreSQL database. No
cases were skipped. Server typecheck passes. Fresh remote CI and review
are pending.

Current head: `ec28076f825e76a267f05517ff0c44a3e2283ce7`. This remains a
single-duty PR stacked on #17450.

- Integrated pinned master `8548173fc9efdc0fa07644fb92dcc0b6851a95e5`
with normal merges. Every branch passed generated-schema checks and its
target-branch migration-order gate. Existing SQL is unchanged; the 25
stack migrations follow master’s new migration.
- Duty-specific producer coverage, lifecycle tests, and earlier failure
records:
[2026-10-24-recurring-server-sleep-stack.md](https://github.com/paperclipai/paperclip/blob/ec28076f825e76a267f05517ff0c44a3e2283ce7/doc/plans/2026-10-24-recurring-server-sleep-stack.md).
- Combined tip `51e6cbe67979b55b340ea9f0f434c5b5c682ccd6`: **355
startup/cancellation/recovery/dispatch/preservation tests**, **75
connection-management tests**, and production server typechecking pass.
These combined checks do not mean every intermediate PR independently
ran the full repository suite.
- Real master-to-stack database upgrade passes: **379 unchanged prior
migration receipts plus 25 new migrations**, with the gateway
responsibility fixture preserved.
- Owning-branch checks: 84 runner installation/settlement cases plus
runner typechecking and generated TypeScript/Rust profile agreement; 253
GitHub review workflow cases. Stop-attribution and held-worker readiness
regressions failed before their fixes.
- Failure accounting: the initial combined run passed 349 cases and
failed six idle assertions in a reused preservation database containing
old cancelled runs. Repeating the combined command with a fresh
preservation database passed. Earlier invalid project selection, merge
import/type errors, and the interrupted full local suite are not counted
as passing checks.
- **Latest-head CI and review remain required.** Prior CI findings
include GitHub idle observation, route teardown, attention cleanup,
touch-picker E2E, workspace health, and voice-cleanup timing; local
passes do not erase those failures. No full local repository-suite,
deployed sleep, fleet cost, or live Copilot qualification claim is made.

## Risks

- All supported producers must share the database signal owner.
Independent SQL writers and other replicas need a live notification
contract; startup alone does not cover them.
- A crash after authoritative expiry and before interaction projection
still relies on the existing projection-repair duty. That duty,
approved-action recovery, and uncertain execution recovery remain
periodic in this slice.
- Unknown external outcomes remain quarantined. Provider calls stay
outside SQL transactions. Expiry is still checked before approving a
request even if physical cleanup waits during drain.
- The new index uses the existing top-level migration executor because
PostgreSQL concurrent index creation cannot run in a file-wide
transaction. Snapshot pruning follows the repository rule.
- This establishes worker quiet and reactivation, not deployed host
suspension. Existing Cloud idle-safety and external alarm coverage
remain unchanged.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning and code execution. The exact
serving model identifier and context window were not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Latest-head CI is now green after one evidence-backed retry. Greptile is
5/5 and all review threads are resolved.

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-27 15:26:19 -05:00
DottaandPaperclip 5398275d2a fix(server): schedule gateway credential expiry from committed changes (#17450)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The tool gateway stores credentials for sessions and named gateways.
> - Those credentials already contain expiry dates.
> - The server repeatedly scans their tables even when no expiry work
exists.
> - Creation signals and saved deadlines can drive cleanup without idle
scans.
> - This PR registers that work with the existing coordinator.
> - Authentication continues to reject expired credentials independently
of physical cleanup.

## Linked Issues or Issue Description

Refs #17446, #17221, #17274. This is the third sleep-migration slice,
stacked on #17446.

**What problem does this solve?**
Gateway credential cleanup queries PostgreSQL on every maintenance tick,
including empty tables and retained non-expiring tokens.

**What is the proposed solution?**
Signal expiring token/session creation from its transaction. Restore the
earliest saved expiry at startup. Retry only while cleanup or
reconciliation remains outstanding.

**Why is this useful?**
Configured idle gateways no longer create recurring expiry traffic.
Existing credential data remains the durable source of truth.

Related work: #17221 and #17274 change run-token lifetime and cleanup
eligibility. They remain separate. This PR preserves current master
authorization and deletion rules. When those PRs are combined, update
the deadline query and signals to match their new operator-token expiry
predicate; run-token idle markers must not keep this worker scanning.
Retain the separate run-liveness reaper and run both regression suites.
Text mergeability is insufficient evidence for that integration.

## What Changed

- Make the credential test clock follow real scheduled work without
overshooting future-expiry assertions; production behavior is unchanged
by this fixture follow-up.

- Coalesce sequential credential notifications within the former cleanup
cadence, preserve loaded expiry deadlines and immediate authentication
rejection, and bound scheduled deletion to 500 rows per pass.

- Added transaction-aware intent before the two existing credential
inserts. Non-expiring named tokens do not signal expiry work.
- Added earliest-expiry selection in the gateway service and a narrow
registration wrapper for the shared deadline helper.
- Removed gateway credential cleanup from the aggregate maintenance
tick. Startup reconstructs saved deadlines. Heartbeat-disabled instances
retain their startup-only cleanup behavior.
- Preserved expiry/revocation checks, run authorization, audit behavior,
and the existing 500-token cleanup batch limit.
- Added real PostgreSQL tests and a lifecycle/producer document.
Tool-action review recovery remains a separate slice.

- Drain all expired session/token pages in explicit and
scheduler-disabled startup cleanup, while retaining one-page scheduled
passes and authentication enforcement.

## Verification

- At `907098cd3656bac59c60220c73b085d0349d4b5f`, all 290 Runner
transport tests pass locally (including all seven prepared-environment
variants). CI run `40570815360` attempt 1 had one provider-drain
settlement failure in the `close-attachment` case; the identical source
passed the preceding CI lane. A single failed-job rerun is in progress.
The failure remains recorded; no shutdown assertion or production
behavior was changed to suppress it.

- Master landing reconciliation at
`907098cd3656bac59c60220c73b085d0349d4b5f` after #17446 merged: all
production and test files are byte-identical to
`24318d9b99ee7f044a864ff3981fc1929bf3e509`, where the 150-case
acceptance and server types passed. Only commit ancestry and the plan's
merged status changed. Fresh CI/review is pending on this head.

- Parent reconciliation at `24318d9b99ee7f044a864ff3981fc1929bf3e509`:
150 affected tests pass across gateway credential expiry, the complete
tool gateway suite, custom-image worker and startup. The accepted
PostgreSQL runs skip no cases. Server types pass. Existing direct tests
cover sequential-write coalescing and saved-deadline precedence; the
older Ponytail finding needs a fresh review. Remote CI is pending.

Current head: `10c85a5eaa288867aa2997a27aee5f675c2a713d`. This remains a
single-duty PR stacked on #17446.

- Integrated pinned master `8548173fc9efdc0fa07644fb92dcc0b6851a95e5`
with normal merges. Every branch passed generated-schema checks and its
target-branch migration-order gate. Existing SQL is unchanged; the 25
stack migrations follow master’s new migration.
- Duty-specific producer coverage, lifecycle tests, and earlier failure
records:
[2026-10-08-instance-sleep-and-scheduler-inventory.md](https://github.com/paperclipai/paperclip/blob/10c85a5eaa288867aa2997a27aee5f675c2a713d/doc/plans/2026-10-08-instance-sleep-and-scheduler-inventory.md),
[2026-10-24-recurring-server-sleep-stack.md](https://github.com/paperclipai/paperclip/blob/10c85a5eaa288867aa2997a27aee5f675c2a713d/doc/plans/2026-10-24-recurring-server-sleep-stack.md).
- Combined tip `51e6cbe67979b55b340ea9f0f434c5b5c682ccd6`: **355
startup/cancellation/recovery/dispatch/preservation tests**, **75
connection-management tests**, and production server typechecking pass.
These combined checks do not mean every intermediate PR independently
ran the full repository suite.
- Real master-to-stack database upgrade passes: **379 unchanged prior
migration receipts plus 25 new migrations**, with the gateway
responsibility fixture preserved.
- Owning-branch checks: 84 runner installation/settlement cases plus
runner typechecking and generated TypeScript/Rust profile agreement; 253
GitHub review workflow cases. Stop-attribution and held-worker readiness
regressions failed before their fixes.
- Failure accounting: the initial combined run passed 349 cases and
failed six idle assertions in a reused preservation database containing
old cancelled runs. Repeating the combined command with a fresh
preservation database passed. Earlier invalid project selection, merge
import/type errors, and the interrupted full local suite are not counted
as passing checks.
- **Latest-head CI and review remain required.** Prior CI findings
include GitHub idle observation, route teardown, attention cleanup,
touch-picker E2E, workspace health, and voice-cleanup timing; local
passes do not erase those failures. No full local repository-suite,
deployed sleep, fleet cost, or live Copilot qualification claim is made.

## Risks

- Shared-owner signals cover supported local producers. Independent SQL
writers or another replica need a separate ingress contract; startup
recovery is not live discovery.
- This is scheduling only. The open run-token lifetime PRs need semantic
integration as described above before combining their behavior.
- Expiry cleanup can wait during drain; authentication still enforces
expiry on use. Existing Cloud safety gates are preserved. Local
deadlines do not prove external wake coverage or deployed suspension.
- Revocation does not move expiry. Non-expiring tokens remain retained
by design; this PR does not invent a retention deadline.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning and code execution. The exact
serving model identifier and context window were not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Latest-head CI is now green after one evidence-backed retry. Greptile is
5/5 and all review threads are resolved.

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-27 14:58:11 -05:00
DottaandPaperclip 13709b4c7a feat(connections): let agents configure connections within delegated authority (#17737)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Connections provide governed tools, chat channels, model accounts,
and gateways.
> - Several management routes require a human board actor even when an
agent has the required grant.
> - Default agents must configure these resources within the responsible
user's authority.
> - This pull request adds a shared authorization path to the existing
REST and native API surface.
> - Human authentication, execution approvals, and local host boundaries
remain enforced.
> - The benefit is that a standard agent can complete authorized setup
with a bounded human handoff.

## Linked Issues or Issue Description

**Subsystem affected**

Server authorization, connection services, shared contracts, native API
discovery, database schema.

**Problem or motivation**

Connection administration is available in the web app but often
unavailable to agents. Agent grants, responsible-user authority,
credential ownership, and actor attribution need separate checks.
Opening gateway-token creation also requires preserving the issuing
agent's delegated authority during token use.

**Proposed solution**

Use the existing authorization evaluator for every connection family.
Enforce current responsible-user authority, effective trust, grant
scope, resource visibility, and target-agent access. Keep existing agent
grants unchanged. Expose authorized configuration through default
`search_api` and `call_api`. Save human authentication steps and let
agents continue after completion.

**Roadmap alignment**

This change completes the API access path for the existing Connected
Apps, Personal & Shared AI Accounts, and governed MCP Gateway
capabilities in `ROADMAP.md`. It follows the maintainer's requested
design. Related work: #11899 enforces connection execution access and
creator/manager rules; this PR extends agent management across
connection families. It does not replace that execution-policy work.

## What Changed

- Add shared connection management context and mandatory
responsible-user, trust, scope, and target checks.
- Allow authorized agents to configure tool, chat/email, GitHub/Slack,
AI, profile, and gateway resources through existing routes.
- Preserve human credential ownership and record the actual agent,
responsible user, and run or key in audit data.
- Preserve saved browser authentication handoffs and callback session
checks.
- Bind agent-issued durable gateway tokens to their responsible user and
recheck delegated authority at use.
- Add explicit REST transport and management capability metadata,
built-in guidance, and sensitive-response handling.
- Intersect caller and endpoint-agent Slack test/invitation policies,
including independent and shared rates. Require the addressed human’s
current endpoint identity link before completing identity handoffs.
- Add production-authenticated integration scenarios. Companion #17739
contains the saved setup UI (including the required human
import/rotation step), product and permission documentation,
catalog/Vercel credential and callback ownership, and operation maps to
keep each review below 100 files. Both PRs must ship together; merge
this implementation first, then its companion.

## Verification

- Current implementation head:
`2ecd6451d0d0c8d82121a4e3fa4ce94c5befbbd6`, including master
`ea727dc3c2`, 99 files. The latest follow-up fixes the two valid
findings from the preceding f9 review. Exact-head Greptile has completed
at5/5 with no findings and Superagent passed. Full exact-head Linux CI
workflow40446606056 completed SUCCESS; only the queued external
Paperclip Review remains pending; earlier Linux passes do not qualify
this successor.
- Latest preserved-stack verification passed **2,165 tests** in fresh
PostgreSQL: **70 actual-auth scenarios**, 27 OpenAPI contracts, 2,022
native API tests and 46 saved GitHub wizard cases. **141 existing
gateway/policy acceptance tests** also passed. Workspace
typecheck/build, maps and token gates passed. The first workspace
typecheck was blocked by sandbox tsx IPC permissions; its authorized
rerun passed. This is focused local evidence; the full Linux CI matrix
is still required.
- Issuer/target gateway policy regressions were red before the fix. Both
decisions now intersect, including Block/Ask first/rates, same
credential identity and current token/user/agent authority after
approval. Invocations retain their token and actual requesting agent;
deleted tokens fail closed. Thirteen actual-auth cases cover revocation,
narrowed scope, approvals, shared rates and board compatibility.
Intermediate missing-token attribution, review-context and
implicit-session actor failures remain in the qualification record; all
final cases pass.
- Activity pagination now retains a bounded cursor across private-only
windows. The actual-JWT regression reaches three older readable events
after 27 private ones without exposing private details. A native
device-login regression was red for plaintext receipts; local check and
test-call results now use existing protected-response handling. The
passing actual-auth case proves encrypted persistence, exact replay,
redacted history and no response asset.

- Previous-head `5b55195593` Linux CI initially failed e2e shard 4/8 in
the existing connection-intent continuation test: periodic recovery
queued a repair at 01:21:47, the scripted journey completed the task at
01:21:48, and the extra queued run was cancelled at 01:21:49 because the
task was terminal. The assertion selected that latest cancelled run.
GitHub refused the first attempt while the workflow was nonterminal.
After completion, the single evidence-supported failed-job rerun was
accepted as **attempt 2** on the same head (workflow 40431202466). The
failed shard and final E2E aggregate passed on attempt 2, and the
complete Linux workflow for that head is successful. The original
failure remains part of qualification. No additional unchanged-code
rerun will be used for this failure.

- Companion CI also has an unchanged login-cleanup idleness timeout,
retained in its qualification record. That same 16-case suite passed in
this parent Linux run and a fresh local database. Both preceding
workflows passed after their single failed-job reruns as attempt 2; the
original failures remain recorded. No additional unchanged-code rerun
was used.

- Migration `0380_broken_aqueduct.sql` was regenerated through the
repository workflow after master took 0379 for project Git
authorization. It adds the responsible-user column with `IF NOT EXISTS`,
preserves upstream migrations, and satisfies snapshot retention and
migration safety checks.
- Upstream published Copilot v62 and v63 identities remain byte-for-byte
unchanged. The combined guidance is now v64, digest
`sha256:22e31f95ef972bcead98a4a707c9f33433558fd111e390e027a2bbb4346e112a`.
The normal shared TypeScript/Rust identity check and **61 Copilot
installation tests passed**. The old source-identity assertion first
failed against unchanged published v63; regenerating a new v64 preserves
the published release.
- Workspace `pnpm -r typecheck` and `pnpm build`, map validation, and UI
token gates passed on the final combined stack after the upstream merge.
Node 24.22.1 / pnpm 9.15.4 were used.
- **Earlier 82 API/OpenAPI tests passed** in fresh PostgreSQL 18: 55
assembled-server scenarios and 27 contracts. They use real persistent
agent keys, run JWTs, human sessions, production defaults, and
deterministic provider fixtures. No actor injection establishes access.
- The new email regression was red before per-account scope filtering.
The Arcade regression was red before human ownership and inventory-key
fixes; it also verifies actual-agent audit identity and a human-session
inventory read. The activity regression measured **200 authorization
checks before the fix and one afterward** for the same one-item page.
- **82 GitHub UI tests passed**, including all three new
failed-replacement regressions that were red before their fix. The UI
preserves an earlier copy only after confirming its committed
generation. Changed or unreadable results block copying and reconnecting
until reconciled. Human-only secret responses now include their
nonsecret committed generation.
- Private test status now rechecks the originating principal and linked
task, including mandatory agent/responsible-user privacy. Cross-agent
tests require the full caller tool access decision and the same
credential identity, regardless of auth label; caller and target
policies/approvals and rate limits apply before execution and after
human approval. A profile-only precondition was found to reject valid
allow-policy and scoped-grant access and has been removed. Both
permitted-access cases and a no-auth-label identity regression were red
before the correction; caller-specific Block still prevents provider
execution. The private-status regression was also red before its fix.
- The tool-activity regression measured 100 capability decisions and 102
run captures on a 100-event page before the fix. It now uses two task
decisions and at most two captures while retaining the same 50 visible
events, private-event filtering, and 403 after audit-grant revocation.
The regression was red before request-context reuse and task-decision
sharing; final server typecheck/build also pass.
- Generic chat configure/reconnect now hands agent-supplied verification
credentials to the addressed human before provider work or vault
changes. Actual key/JWT regressions were red for Slack signing-key
replacement and generic channel secret-reference rebinding. They now
prove unchanged refs and vaulted signing value, human setup, permitted
agent bot-token repair, and governed test messages. All 82 API/OpenAPI
cases pass in fresh PostgreSQL. Eight native receipt protections and
final server typecheck/build also pass.
- The preceding review found that old healthy state could retire a
credential-entry card before a human changed anything. The actual-JWT
regression was red. Saved field generations now change only with
protected values and commit with refs plus a verification fence; retries
retain the baseline. Old health, bot-token repair and unchanged entry
stay pending. New signing verification and current authority permit
continuation. A separate red race proved that user revocation during
provider authentication could return 403 after refs changed; authority
is now checked before persistence and inside the credential transaction.
The final 82 API/OpenAPI cases, four neighboring real-database
Slack/GitHub lifecycle cases, and final server typecheck/build pass.
- A preceding full run crossed a rate-window boundary. Database rows
prove one consumed slot at 23:35 and another at 23:36 for the same
policy/bucket. Only Date is frozen in the rate test cases; transport and
asynchronous timers remain real. Production rate behavior is unchanged,
and the final full run passes. An intermediate fixture failure that
overwrote hidden generation metadata with an API projection is retained
in the qualification record.
- Linux CI on an earlier head found two rate-admission assertions that
omitted the newly persisted bucket key. The failures were reproduced
locally; both assertions now verify the exact actor/tool bucket and all
49 policy-service tests pass. The companion qualification record retains
the initial and intermediate failures. Catalog creator attribution is
preserved in the companion to keep this implementation review at 99
files.
- The latest review reproduced three regressions before repair: complete
human replacements failed on unavailable old signing secrets, fresh
GitHub links incorrectly selected reconnect, and different
credential-entry requests shared an idempotency key. Each old protected
ref now has a narrow known-unavailable fallback; unexpected resolver
failures and company/scope violations still abort. Fresh imports keep
their credential form, configured endpoints keep reconnect, and
canonical field sets/completion modes distinguish handoffs while
matching retries retain their baseline. All 55 actual-auth scenarios and
27 separately run OpenAPI contracts pass, as do server typecheck/build.
Companion #17739 also passes all 41 wizard tests including actual
fresh-form submission, UI typecheck, maps and token gates. Earlier red
results and a mistyped OpenAPI filename are retained in qualification.
- The configured GitHub handoff regression was red in the
browser-component tests and actual-auth link/persistence tests. Saved
links now name requested fields and all/any completion mode; the
companion form submits those supplied fields through reconnect and
retains other saved credentials. Webhook/client-secret reconnect remains
human-only; first-setup secret restrictions remain. All 46 wizard tests
pass, including combined modes and legacy links. A duplicate review
referenced the older companion bf760912; the current form is already
fixed, linked in the reply, and its thread is resolved.
- Linux CI on bf760912 failed automatic Slack provisioning because vault
persistence cleared an already verified manifest-update challenge. The
same regression was reproduced locally. The provider-owned helper now
preserves proof only for the current signing fingerprint and callback
URL. Eight existing compatibility cases pass, including this case and
initial GitHub secret rejection. The first uncommitted GitHub-field
variant also failed that initial boundary and was narrowed to reconnect;
both failures remain in qualification.
- The upstream merge conflicted only in generated migration metadata.
Upstream 0379 is preserved, normal generation/pruning produced 0380, and
two repeated applications of the idempotent additive SQL passed. The
final combined stack passed **128 tests** (55 actual-auth, 27 OpenAPI
and 46 wizard cases) in fresh PostgreSQL. Eight Google UI cases also
pass; the two protocol cases could not initialize embedded PostgreSQL in
this fresh local attempt. Their earlier passing evidence remains
historical; fresh Linux CI must execute them. Vercel personal credential
attribution is preserved in companion #17739 with its other
ownership/consent follow-ups, keeping this PR at 99 files.
- The suite covers fresh defaults, restricted users/agents, scopes,
personal isolation, token revocation, human handoffs, callback replay,
execution approvals, authorized self-targeted policy changes,
self-approval denial, and current authority before commit. Reproduce API
checks with `pnpm exec vitest run
server/src/__tests__/connection-management.integration.test.ts
server/src/__tests__/openapi-routes.test.ts`, using a fresh database
through `PAPERCLIP_CONNECTION_MANAGEMENT_TEST_DATABASE_URL` or the
default embedded test database.
- Companion #17739 also records consent separately from token refresh so
saved Google choices survive refresh. Its eight service-picker and two
local-HTTP OAuth tests pass; both new UI cases were red before the fix.
Normal migration generation confirms no SQL change is needed for the
additive JSON metadata.
- The integration module shares one assembled server, authentication
factory, and provider lifecycle to preserve the
HTTP-to-vault-to-task-continuation proof. A reused external database
caused a Slack fixture identity conflict; the fresh final run passed.
Earlier fixture/transport and contract-update failures remain in the
companion qualification record.
- The broad local `pnpm test:run` attempt remains blocked by macOS
embedded PostgreSQL's host semaphore limit. It is not a clean pass. The
fresh Linux CI matrix must complete. Provider fixtures do not prove live
setup or an empirical LLM success rate.
- A current-head completed Paperclip assessment is not yet recorded. Its
earlier external recovery boundary has not been bypassed, and no
incomplete assessment is reported as passing. The user authorized both
merges when checks pass; merge this implementation first, then replay,
retarget, and requalify companion #17739 against the squash-merged
master.

- After the master refresh, local workspace typecheck/build, **97
actual-auth/OpenAPI cases** and **141 gateway/policy cases** pass. The
original fixed database port was occupied; Docker assigned a free
localhost port. The first neighbor setup command could not resolve root
`tsx`, leaving its schema uninitialized; the built database package
migrated the separate fixture and all unchanged cases passed. Both setup
failures are retained in qualification. Fresh Linux CI remains required;
no full local test-sweep pass is claimed.

- The f9 Greptile review scored **2/5** for two reproduced findings:
Slack test sends skipped the caller’s tool policy, and healthy App
credentials completed an unconfirmed identity handoff. Both are fixed in
this head and have resolved evidence threads
([Slack](https://github.com/paperclipai/paperclip/pull/17737#discussion_r4384927765),
[identity](https://github.com/paperclipai/paperclip/pull/17737#discussion_r4384927895)).
Actual key/JWT cases cover twelve Slack caller/target/recipient
combinations; the human-session GitHub OAuth callback alone remains
pending until explicit confirmation. All **99 API/OpenAPI tests** (72
actual-auth + 27 contracts), workspace typecheck and build pass. On the
combined companion, all four focused cases (including the twelve Slack
combinations), typecheck/build and maps also pass. The continuation
suite’s 50 embedded-PG cases were skipped because its database could not
start locally; fresh Linux CI must run them. Initial fixture
credential-collision and audit-projection assertion failures are
retained with the red reproductions in the companion qualification
record. No full local sweep or live consent is claimed.

## Risks

- The maintainer explicitly confirmed authorized policy administration,
including self-targeted changes that loosen controls. These writes
require management grants and the responsible user’s current authority.
Agents cannot approve their own governed action requests or edit their
own principal permission grants. The actual-auth regression exercises
both permitted administration and those denials.
- This is a broad authorization change. Incorrect scope or owner
resolution could deny permitted setup or expose another user's
resources. Integration cases cover both permitted and denied requests
with actual authentication.
- The additive nullable gateway-token column does not infer identities
for legacy clients or backfill agent grants. Legacy agent tokens without
a responsible user fail closed.
- Provider fixtures test supported setup and callback contracts. They do
not prove live provider availability.

## Model Used

- OpenAI Codex, GPT-6 family, with reasoning, repository tools, and code
execution. The runtime does not expose a more specific deployment ID or
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass — focused suites above;
full local sweep remains blocked as documented
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green — exact `2ecd6451d0` Linux CI
completed SUCCESS; external Paperclip Review is still queued
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups —
exact `2ecd6451d0` review completed; both f9 findings have resolved
evidence replies
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-27 08:20:24 -05:00
DottaandPaperclip db32bdf98b fix(github): recover review replay and preserve passive summaries (#17726)
<!-- entire-trail-link-start -->
https://entire.com/gh/paperclipai/-/trails/8/changes/paperclip/8
<!-- entire-trail-link-end -->

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - GitHub tools save review assessments and publish their confirmed
receipts.
> - A timeout before publication staging can leave an invocation with no
saved publication.
> - Exact replay must repeat only the authorized preflight and converge
on one publication.
> - PR requests also share an App-owned summary comment. Progress and
maintenance must preserve its final content.
> - A native turn can intentionally wait for an author without sending
another reply.
> - This pull request repairs unstaged replay and prevents queued or
passive-wait notices from replacing a published assessment.

## Linked Issues or Issue Description

Related repairs: Refs #16883, Refs #16284, Refs #16275 recover saved
publication receipts. Refs #17183 covers unavailable approved replay
output. Refs #17714 fixes a separate publication lock-order problem.

**What happened?**

Exact review retries can return null `begin_review` output or
`github_publication_unavailable` after an upstream read times out before
publication staging. Separately, response maintenance can replace a
completed PR assessment with “Review queued…” or a generic
missing-response notice after an authorized no-new-work reply. The
latter run can correctly commit a passive wait while its external
summary is still overwritten.

**Expected behavior**

Exact retries repeat only the authorized preflight. A saved or uncertain
publication keeps its existing recovery owner. Queued acknowledgements
preserve final App-owned summary content. A committed native passive
wait settles its current acknowledgement without writing a response or
treating an old summary as a new-run reply. Success before presentation
remains held until a durable decision exists; exact current-run
confirmed receipts can retire their placeholders. Genuine missing/error
responses retain fallback handling.

**Steps to reproduce**

1. Hold a GitHub metadata read until an authorized gateway call times
out before publication staging. Repeat the exact request.
2. Publish a valid PR assessment. Send an ordinary author reply that
asks for no new work and no duplicate comment.
3. Commit the native `response_wake` with a no-comment server
presentation decision. Run response maintenance twice.
4. The base returns the invalid replay result or replaces the assessment
with progress/fallback text.

**Paperclip version or commit**

Replay regression: `41c42895fa862c586ff841f8c50200d93ffc83a4`.
Maintenance regression: the previous PR head
`c7553d2325df53d98d9febff7023ade4b1e704eb`. Both use real PostgreSQL and
simulated GitHub responses. The maintenance failure was also observed in
normal hosted use.

**Deployment mode**

Source checkout on macOS arm64 with isolated PostgreSQL 18, followed by
a dedicated QA qualification. No production rollout is included.

## What Changed

- Route exact `begin_review` replay through its typed owner. Repeat
unstaged preflight only with the complete request digest and no
successful invocation result.
- Preserve current policy, immutable assessments, unique action keys,
receipt recovery, and provider leases.
- Preserve a verified App/marker-owned final shared PR comment during
acknowledgements. New canonical assessments can still replace it at the
same or a new head.
- Project only a bounded server-committed native wait decision. Settle
the current main-PR acknowledgement without provider writes or a
fabricated current-run reply receipt.
- Hold succeeded native runs while presentation is undecided, including
the committed-before-presentation window. An exact current-run confirmed
receipt can still retire its placeholder.
- Keep missing/error fallback and uncertain publication checks. Update
the bot guide, execution semantics, maps, and real-database regressions.

## Verification

- **Causal RED:** exact begin replay returned null; exact unstaged
submit replay returned unavailable. The new real-database maintenance
test then reproduced the exact generic summary overwrite. Its first
repair also exposed an earlier queued-acknowledgement overwrite. Both
publication boundaries are repaired. Four additional ordering
regressions reproduced the generic PATCH while native presentation was
undecided; the corrected tests hold until the same row settles.
- **Local GREEN:** 196 complete GitHub workflow cases pass on an empty
migrated PostgreSQL fixture. This includes queued/running preservation,
both success-before-presentation windows, null/scalar/invalid decisions,
exact-current versus predecessor receipts, durable acknowledgement
settlement, repeated no-op maintenance, eventual genuine missing/error
fallback, pending writes and same/new-head canonical replacement.
Workspace typecheck, exact-head build, maps and whitespace checks pass.
The real database respects the actual selected-column projection.
- **Command:** `PAPERCLIP_TEST_DATABASE_URL=<owned-postgres-18> pnpm
--filter @paperclipai/server exec vitest run
src/__tests__/chat-channels.integration.test.ts -t 'GitHub agent review
workflow'`.
- **CI/review:** historical heads
`c7553d2325df53d98d9febff7023ade4b1e704eb` and
`4d00d3af6f2ccbda8f38a98f81d392d82dd3c59c` had Greptile 5/5. Final
readiness head `2d267f1e4aa0b835a11aede97798fbd800d9cd96` passed
independent source review with no actionable finding and fresh Greptile
5/5 with zero unresolved threads; all 54 check runs pass with 2 intended
skips; Snyk is successful and the merge state is CLEAN. The dedicated QA
destination acceptance also passed.
- **Full-suite limitation:** the earlier `pnpm test:run` attempt
produced no final summary. Its retained log reports failures in
unchanged workspace-runtime tests and 15-second gateway-audit and
stale-question timeouts. Their cause remains unknown. Do not treat the
targeted results as a full local-suite pass.
- **Live evidence and limits:** normal native OpenCode/DeepSeek review
delivery published a useful assessment, inline finding and App check at
the exact reviewed head. A separate ordinary author reply delivered a
useful answer and settled. Those are ordinary delivery proofs, not live
injected-timeout proofs. A later no-new-work reply exposed the
maintenance overwrite. The final combined source
`070e9c4ead9341cd497b23f533a91bf194475e79` includes the exact
publication repair and passed normal review → ordinary author reply →
quiet destination acceptance. A normal current-head review restored the
legitimate assessment; the ordinary no-new-work reply then committed a
passive wait without new GitHub tools. The same 1,436-character summary
stayed identical while queued/running, after terminal settlement, and
more than 3 minutes 50 seconds after successful cleanup. Review rows and
receipts stayed unchanged, with no owner, live run or queued input.

The timeout fault fixtures prove replay and single-publication safety.
The historical first upstream timeout producer remains unknown. The live
board reads do not expose the raw acknowledgement action row: its
terminal cancellation and next-pass no-op are real-PostgreSQL fixture
proofs, while the hosted checks prove actual destination stability. The
hosted journey did not inject a timeout or pause the
terminal/presentation writer window. Private destinations, raw receipts
and internal task identifiers remain outside this PR.

## Risks

- Original and repeated metadata reads can overlap. The immutable
assessment, action uniqueness and publication lease must remain
authoritative.
- Missing complete request identity or a deleted completed receipt
remains held. The gateway does not reconstruct receipts or relax
approval.
- Passive suppression requires actual native mode/committed phase,
current source ownership and the server's no-comment decision. It
applies only to the main PR review surface. Uncertain current writes
stay held; other surfaces keep normal placeholder retirement.
- There is no migration, credential change, dependency or new retry
scheduler.

## Model Used

OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) in Codex assisted with diagnosis,
implementation, tests, and source review. It used reasoning, repository
tools and code execution. The context window size is not exposed in this
session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-26 22:22:57 -05:00
DottaandPaperclip 9c442cf48e feat(connections): deploy and configure Supabase and Vercel (#17217)
Add governed Supabase and Vercel deployment/configuration connectors with vaulted credentials, agent access policies, approvals, audit, and bounded provider operations.

Deliver reviewed REST actions to native agents and guard Vercel preview intent on empty projects. Preserve secret-safe request identity and durable retry behavior.

Verified by current-head CI, fresh Greptile 5/5 review, and real-agent deployment/configuration walkthroughs on disposable provider resources. Test resources and temporary provider tokens were removed after verification.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-23 11:37:29 -07:00
Devin Foley 059ae49cab feat(connections): add Unikraft Cloud connection (#17139)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Apps connections give agents governed access to external services
through vault credentials, grants, action policies, the tool gateway,
and audit.
> - Unikraft Cloud runs unikernel micro-VMs, services, and volumes
across regional metros, and it offers a sandbox plugin for commands and
files.
> - Unikraft is not in the catalog. It has no hosted MCP server, so the
generic "Connect your own MCP server" flow cannot reach it.
> - Daytona (#16483) added a native REST pattern for this case: a
catalog generated from the provider's published OpenAPI specifications
and executed through the gateway.
> - This pull request adds Unikraft on that pattern and moves the shared
native-provider code into one registry, so the shared connection code
does not name each provider.
> - The benefit is governed agent access to all published Unikraft
actions, with the token kept in the vault and sent only to published
Unikraft hosts.

## Linked Issues or Issue Description

Refs #16483

**Problem or motivation**

Teams that run workloads on Unikraft Cloud want their agents to list
instances, read logs, deploy and scale services, and run commands in
sandboxes. Unikraft is not in the Apps catalog. Unikraft has no hosted
MCP server, so operators cannot connect it through the remote MCP flow.

**Proposed solution**

Add a native Unikraft connection. Generate its actions from Unikraft's
official OpenAPI specifications for the platform API and the sandbox
plugin. Pin the specifications to a reviewed commit. The operator
connects an API token. The token stays in the vault and goes only to the
five published metro hosts. Every action takes an optional metro.
Actions without a metro use the connection's default metro, which the
operator can change under Advanced.

**Alternatives considered**

A catalog-only AppDefinition was not possible because no hosted MCP
endpoint exists. Wrapping the `unikraft` CLI was rejected because it
would expose host filesystem behavior and CLI login state. A second copy
of the Daytona bridge was rejected; the shared transport and catalog
contract are now one module with a provider registry.

**Roadmap alignment**

This extends the existing Apps and Connections provider coverage. It
uses the existing credential, grant, policy, gateway, and audit model.
It does not overlap planned core work.

## What Changed

- Added `scripts/generate-unikraft-tools.mjs` and its tests. The
generator reads the platform and sandbox plugin specifications at commit
`5dd107ea1cd8716b83cb86a83ed2a6ee46f99b5b`, resolves shared parameters,
records source SHA-256 hashes, and writes
`server/src/services/unikraft/operations.generated.json`.
- The catalog has 105 actions: 89 platform operations and 16 sandbox
plugin operations. The server-sent audit event stream is recorded in the
inventory and is not advertised as an action.
- Classification: GET operations and sandbox reads and waits are reads.
Delete, stop, suspend, detach, unpin, and sandbox command execution,
signals, and stdin are destructive. All other operations are writes.
- Added `server/src/services/connectors/unikraft/client.ts` and its
generated catalog. It validates arguments against the generated schemas,
accepts only published metros, rejects path traversal, sends JSON bodies
on GET and DELETE as the specification requires, and verifies the token
with `GET /v1/users/quotas`. A sandbox wait that times out (documented
HTTP 408) returns a still-running result instead of a failure.
- Moved the bounded, credential-redacting HTTP sender and the catalog
contract helpers from `daytona.ts` to `native-rest-client.ts`. Added
`native-provider-catalogs.ts`. `tool-access.ts`, the
`tool-access/catalog` and `token-broker` modules,
`connectors/tool-policy.ts`, and `tool-gateway.ts` now use this registry
instead of Daytona-specific checks. Daytona behavior does not change.
- Added the Unikraft definition source, the generated definition, the
brand mark and manifest entry, and the permission review (`liveProof:
not-run`).
- Added `doc/connections/UNIKRAFT.md` with a sequence diagram,
endpoints, setup, actions, and limits. Updated the connections README,
the permission audit, and both capability maps.
- Tests: definition shape, database-backed gateway reads, writes,
deletes, and sandbox commands for personal and shared grants, default
and per-call metros, rejection of unpublished metros and traversal, Off
policies, revocation, stale-contract blocking, credential export denial,
schema validation for all actions, pinned-transport tests for GET and
DELETE bodies, and a browser setup check.

## Verification

- `pnpm exec vitest run packages/shared/src
server/src/__tests__/unikraft-connection.test.ts
server/src/__tests__/unikraft-http-boundary.test.ts
server/src/__tests__/daytona-connection.test.ts
server/src/__tests__/daytona-http-boundary.test.ts
server/src/__tests__/tool-access-service.test.ts
server/src/__tests__/tool-gateway-service.test.ts
server/src/__tests__/tool-gateway.test.ts
ui/src/pages/apps/AppsConnect.test.tsx ui/src/pages/apps/Browse.test.tsx
ui/src/lib/app-brand-assets.test.ts` plus `connector-tool-policy` and
`tool-catalog-publication`: all passed after the rebase onto the split
tool-access modules.
- `node --test scripts/generate-unikraft-tools.test.mjs
scripts/generate-daytona-tools.test.mjs
scripts/app-brand-validation.test.mjs`: 20 passed.
- `node scripts/check-app-brand-assets.mjs`, `node
scripts/check-maps.mjs`, and `git diff --check`: passed.
- `pnpm --filter @paperclipai/shared typecheck`, `pnpm --filter
@paperclipai/server typecheck`, and `pnpm --filter @paperclipai/ui
typecheck`: clean.
- `npx playwright test --config tests/e2e/playwright.config.ts
tests/e2e/unikraft-connector.spec.ts
tests/e2e/daytona-connector.spec.ts`: 2 passed.
- `node scripts/generate-unikraft-tools.mjs` regenerates the committed
catalog with the recorded hashes.
- Manual: on a local instance, open Connectors, search for Unikraft, and
select **Add key**. Confirm the icon, the password field, the key
guidance, and the default metro select under **Change**.
- Not verified: a real Unikraft account. Live provider read and write
proof is pending.

## Risks

- A Unikraft token acts for the whole organization in every metro.
Unikraft offers no narrower token permissions. Assign only trusted
agents and review action permissions.
- Sandbox actions run arbitrary commands inside the selected instance.
They are classified as destructive.
- A timed-out write can still complete at the provider. The bridge does
not retry writes automatically.
- New metros fail closed until the specification is regenerated and
reviewed. Legacy `*.kraft.cloud` hosts are not reachable.
- The Daytona refactor moves code without behavior changes. All Daytona
tests pass without edits.
- The generated JSON snapshot is large because it holds the provider
input contracts. No migration or lockfile change is required.

## Model Used

- Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, 1M context
window, used through Claude Code with tool use (shell, file editing, web
research, browser tests).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-10-22 15:14:26 -07:00
DottaandPaperclip 3ce8d976a9 Add experimental Paperclip Notebook MCP connection (#16984)
## Thinking Path

> - Paperclip connects agents to external tools through saved accounts
and per-action permissions.
> - Notebook exposes artifact management through an ordinary remote MCP
server with OAuth.
> - A private pilot needs a hidden, default-off catalog and execution
gate.
> - Local file publishing belongs to Notebook, which can accept bounded
file bytes through ordinary MCP tools.
> - Paperclip therefore needs the connector registration and generic
availability checks, with no file-transfer architecture.

## Linked Issues or Issue Description

**Subsystem affected**

Apps catalog, experimental feature metadata, connection setup, and
connector availability.

**Problem or motivation**

Users need to connect Notebook as their Cloud account, assign its tools
to agents, and retain Paperclip's normal Allowed / Ask first / Off
rules. The private pilot must remain hidden until an operator enables
it. Notebook should own its upload protocol and local publishing script.

**Proposed solution**

Register a personal remote MCP OAuth connection at Notebook's canonical
endpoint. Reuse existing OAuth discovery, credential storage, refresh,
reconnect, and removal. Enforce availability through generic catalog
feature metadata and saved application identity. Keep Notebook's
standalone uploader in its own service repository; it calls the ordinary
gateway and receives no saved OAuth credential.

**Alternatives considered**

A gallery-only filter permits retained tools and deep links while
disabled. A shared workspace publisher and private upload-ticket
transport add runtime behavior that a normal MCP connector does not
require. Notebook can receive file bytes directly through its own tools
instead.

**Roadmap alignment**

This extends Connected Apps and supports Artifacts and Work Products.
ROADMAP.md and related PRs were checked. This PR is standalone on
master. The separate OAuth lifecycle and file-transfer proposals (#17085
and #17086) are closed and are not dependencies.

## What Changed

- Register Notebook's personal MCP OAuth method, canonical endpoint,
scopes, catalog metadata, and branding.
- Declare managed `enablePaperclipNotebook` with false defaults and no
user-facing settings toggle or staging opt-in.
- Use generic availability metadata for gallery/setup, retained
connection identity, OAuth completion, health, tool/context discovery,
and dispatch. Resources/prompts recheck before credentials, including
retained handles. Removal remains available while disabled.
- Resolve catalog application identity from the saved application key,
so editable connection labels cannot bypass the hidden gate.
- Select a method's sole supported credential kind when no kind is
supplied.
- Keep UI changes to metadata-driven availability and existing
setup/reconnect controls.
- Document Notebook's service-owned upload script, inline publication,
ordinary action policies, and credential behavior.
- Add ordinary MCP OAuth, assignment, action-policy, disabled-state,
settings, and catalog tests.

No workspace contribution registry, private transfer endpoint, native
publisher, Runner file-reader change, or Paperclip CLI upload command is
included. The entire native-runtime and CLI trees match master. Upload
code lives in `paperclip-cloud/notebook/`.

## Verification

Source head: `65ef741e4f476c67425b57d2fb2d6d4c00e99b06`, directly on
master `53732c811bb69655eb43d3d12f2063ae4ca5ad67`. The diff is 42 files.
Shared execution changes only enforce generic catalog availability; tool
access also resolves saved application identity and the sole supported
credential kind. Neither module contains Notebook upload logic.

- Catalog/shared/settings/UI suites: 592 tests passed across 12 files.
Shared/server/UI typechecks passed using installed compilers.
- Fresh real PostgreSQL generic MCP suite: 84/84 passed at final source;
server typecheck and module boundaries pass. Coverage includes personal
OAuth, ordinary refresh/reconnect, assignments, Allowed / Ask first /
Off, hidden feature enforcement, and local credential removal. Four new
resources/prompts cases failed against the prior code and pass after
filtering saved application identity and rechecking before credentials.
They prove missing/relabelled setup markers and stale eligibility cannot
dispatch or read secrets while disabled; an ordinary provider remains
available with the flag off.
- The full tool-access service suite passed 463/464 cases. One unchanged
Google Workspace `slides.write` case timed out at 15 seconds; it passed
on one isolated retry without source or timeout changes. Its exact test
block matches master. The cause is not established; final-head CI must
qualify the complete suite.
- Maps, module boundaries, token gates, and whitespace checks passed.
Byte comparisons confirm the former transfer/runtime/CLI surfaces match
master.
- Fresh exact-head Greptile is 5/5 with zero unresolved findings after
an exhaustive feedback audit. Current-head CI workflow
[39625691137](https://github.com/paperclipai/paperclip/actions/runs/39625691137)
succeeded, including both required checks (`ci / verify`, `ci / e2e`),
Build, and browser E2E. Supplementary security/advisory scan status is
recorded in the PR Checks panel and qualification receipt; prior-head
scans are not counted. The superseded `4bd7af9b2e` completed CI
successfully, but Greptile found the valid resources/prompts
availability gap described above. It was fixed, replied to, and
resolved; earlier-head results do not qualify this revision.

Preserved environment failures: the first socket-restricted UI/route run
had 93 listener setup failures, and the same 93 cases passed with
local-socket access. The package-manager launcher could not fetch its
signature metadata; installed compilers passed without dependency or
configuration changes. The full service timeout and isolated retry
remain recorded separately. The superseded `585e870e73` CI also failed
an unchanged agent-chat project-plan revision assertion after task
creation; the complete spec and relevant runtime sources match master,
and a same-master prior run passed. Its cause is not established. Logs
are preserved; no test or timeout was weakened and no old-head rerun was
requested. The subsequent heads run full CI independently; no prior-head
pass substitutes for current qualification.

The Notebook uploader requires a separate service release. This public
PR adds no uploader runtime to Paperclip and claims no newly deployed
browser/agent acceptance. The Notebook-owned upload path is qualified
separately. Code-owner review remains required before merge.

## Risks

- The feature flag gates the curated connector; it does not create a
Notebook service-side team allowlist or block arbitrary standard MCP
URLs.
- Publishing, sharing, and deletion retain the existing assignment and
action-policy boundaries.
- The hidden feature must be checked for retained tools and saved
identity, including when connection labels change.
- Disconnect removes local credentials. It does not revoke the remote
Notebook authorization; the owner can revoke it in Notebook.
- Ordinary refresh uses the existing client. A lost response after
refresh-token rotation can require reconnect.
- The standalone Notebook script and its bounded upload tools must ship
together in the Notebook service before that documented convenience path
is available.
- No database schema, workflow, lockfile, shared transfer, or Runner
change is included. No merge, deployment, or rollout is performed by
this update.

## Model Used

OpenAI GPT-6 in Codex, with reasoning, repository inspection, code
execution, and tool use. The runtime did not expose a more specific
model ID or context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-21 11:07:18 -05:00
DottaandPaperclip d33b40a5c3 fix: recover saved work and provide reliable task Retry (#16965)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The control plane must retain work when an execution or workspace
export fails.
> - Retained-workspace holds can leave saved input waiting with no
usable Retry action.
> - Users cannot verify process ownership or repair provider
allocations.
> - This pull request gives system recovery a durable budget and gives
each deliberate Retry a durable intent.
> - Safe recovery can preserve files, settle an accepted result, or
prepare an independent workspace.
> - One task state keeps infrastructure diagnostics out of the ordinary
conversation while preserving real permission and approval gates.

## Linked Issues or Issue Description

**What happened?**

A task with a retained workspace can show recovery, failed-run and
saved-message warnings at the same time. Historical runs without native
workspace descriptors cannot use the native restore path. Old recovery
metadata can prevent a new attempt even when an independent workspace
would be safe.

**Expected behavior**

Paperclip owns bounded infrastructure retries and file preservation. A
user can request another attempt with ordinary Retry. Thinking appears
immediately. The service preserves completed effects and current user
controls. It does not claim that starting fresh recovered the old files.

**Steps to reproduce**

1. Retain a failed run and its workspace after export or process-stop
failure.
2. Save a follow-up while the recovery hold is active.
3. Open the task and try to continue without infrastructure credentials
or operator certification.

Refs #16906, #16835, #16883. This extends their recovery and receipt
contracts. Related work: #16962 and #16963.

## What Changed

- Add task-level Retry with company-scoped authorization, all-status
idempotency and durable admission before physical work. Keep exact-run
Retry compatible.
- Add typed recovery phases, finite persisted backoff, historical
adoption and durable local failure diagnostics.
- Preserve native workspaces or verified historical snapshots. Admit an
independent fresh workspace only after execution containment. Keep
old-source preservation and cleanup obligations separate.
- Carry completed publication and API receipts forward. Fence uncertain
mutation retries across replacement runs. Preserve connected-chat
identity and authorization.
- Re-enter normal dispatch for saved input. Reuse accepted results
through the existing finalizer.
- Use one task interruption projection across the thread, composer and
task list. Show optimistic Thinking, ordinary Retry and genuine user
gates.
- Add PostgreSQL, filesystem, API, browser-fixture and controlled
provider coverage. Document the behavior and update the feature/access
maps.

## Verification

- All GitHub Actions CI jobs pass on
`6d8c6d27d50586d8548b31d03b47f6d73a0ee11d`: typecheck, all
server/chat/workspace and serialized suites, Runner checks, eight
browser shards, canary and build. The required merge gate passes.
- Greptile reviewed the final commit at 5/5, with no actionable findings
or unresolved review threads. Its two earlier P1 findings are fixed.
- Full recursive typecheck and build pass locally on the final commit.
All 16 task Retry cases pass against isolated real PostgreSQL. The
unrelated-success case and both stale Retry cases fail on the previous
code and pass after the fix.
- The corrected gateway fixture passes its uncertain-mutation and
independent-read case. Four real HTTP API cases pass for definite
403/422 rejection, completed-effect reuse, and uncertain-effect fences
across deliberate and automatic retries. These focused gateway/API
checks used a temporary database lifecycle harness on an existing
healthy local PostgreSQL process. The standard repository helper is
unchanged.
- Earlier qualification covers 291 task UI cases, saved-file copyback
and ancestry, current Agent Chat ownership races, nine chat Retry API
browser cases, and six assembled browser recovery fixtures. A real
API/PostgreSQL/native-runner journey showed Thinking in 25 ms despite a
delayed response, admitted one successor in a separate workspace,
retained original files, and survived reload.
- The controlled Daytona test interrupts export, joins the controller,
restarts it in a separate process, and verifies one agent start, one
external sentinel operation, retained file contents and no premature
cleanup. It does not establish abrupt-kill or live LLM/GitHub
acceptance.
- The full local `pnpm test:run` was attempted and was not green: an
OAuth row-lock timing assertion failed, and a later suite exhausted
macOS PostgreSQL semaphores. The ten OAuth timing cases passed on rerun
against a separate database. Later embedded database initialization
failures remain recorded; the final CI matrix passes.
- Security scanning passed on production-code commit
`63953edf069b111c6becc3144dffa6deb52f68b5`. Its only successor changes
one test-fixture identity line. The final scan is neutral/inconclusive
because its pinned merge-base checkout failed or timed out. Both rerun
API requests returned 404, so no fresh successful scan is claimed.
Socket and Snyk checks pass on the final head.
- The optional Paperclip Review remains queued behind an earlier run in
workspace finalization. Its prior completed assessment was incomplete
because a comparison tool was blocked by the platform filter. It is not
counted as a completed code review. GitHub does not require this check
for merge.
- Hosted acceptance remains pending deployment. Local and CI results do
not establish recovery of a hosted task or its retained files.

## Risks

- Recovery changes span admission, physical ownership and dispatch.
Incorrect evidence could permit overlapping work. Existing physical
locks and authoritative stop requirements remain in place.
- The migration adds concurrent indexes. It requires the existing
top-level migration runner and retries incomplete index creation safely.
- A fresh attempt can complete while old-source preservation remains
unresolved. The service retains that obligation and does not report
those files as recovered.
- Unknown tool mutations remain blocked until their authoritative
receipt owner resolves them. Generic upstream tools have no universal
result-query protocol.
- Recovery and diagnostics stay inside the stack. No Cloud incident
endpoint, notification worker or new outbound reporting channel is
included.

## Model Used

OpenAI Codex, GPT-6 family. The runtime does not expose a more specific
model version or context-window size. The model used reasoning, shell
execution, source editing and browser tools. No model subagents were
used for this implementation.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (focused suites; the full
local run limitation is documented above)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-20 19:58:53 -05:00
DottaandPaperclip 2c96f76780 feat(connections): add model-backed prompt injection prevention
Add opt-in OpenRouter checks at existing tool-result boundaries with regex fallback, service accounting, and settings on Connections. Make migration 0362 safe to replay over existing or partially applied schema state.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-20 11:32:25 -05:00
DottaandPaperclip 370fb50ad4 fix(mcp): keep sibling tools available after transient calls (#16890)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents use remote MCP tools through a governed gateway.
> - The gateway lists tools only while their shared connection is
healthy.
> - One failed call currently marks the whole connection as unhealthy.
> - Later calls to healthy sibling tools then fail with
`tool_not_found`.
> - This pull request keeps non-authentication call failures on their
invocation.
> - Agents can continue their work while credential and access gates
remain in force.

## Linked Issues or Issue Description

Refs #10147.

Related work: #15462 fixed oversized and malformed response handling.
#15448 covers timeout and JSON-RPC health handling together with
separate SSE and catalog discovery changes. Its patch still marks fetch
failures and HTTP failures as connection errors. #15325 proposed a
transport failure counter; it has merge conflicts and has not changed
since October 6. #11910 is an older timeout fix with merge conflicts.
This PR covers the remaining fetch and HTTP failure paths on current
master, with the same narrow call-health rule for timeouts and JSON-RPC
failures. It does not include the SSE or discovery changes from #15448.
The timeout session-reset check follows the recovery case discussed in
#15448.

**What happened?**

A shared remote MCP connection works until one call has a network
failure, timeout, HTTP 503, or JSON-RPC error. Every later call on that
connection returns `tool_not_found`, including healthy sibling tools.
Connection setup then asks for user attention even though the failure
did not prove that credentials need repair.

**Expected behavior**

The original invocation retains its failure. Authorized sibling tools
remain callable. The gateway does not replay an ambiguous action. HTTP
401/403, missing credentials, and explicit insufficient scope keep their
access gates.

**Steps to reproduce**

1. Give an agent access to two read tools on one remote MCP connection.
2. Make one tool fail with a network error or HTTP 503.
3. Call the other tool in the same gateway session.
4. Before this fix, the second call fails with `tool_not_found` before
provider dispatch.

**Paperclip version or commit**

`master` at `e6dcd1e8e9`. The regression runs against source with
synthetic data.

**Deployment mode**

Any deployment that uses a shared remote MCP connection. This is a core
gateway issue.

## What Changed

- Preserve connection health after a network failure, timeout,
non-authentication HTTP error, or JSON-RPC error from one tool call.
- Retain HTTP 401/403 and insufficient-scope health gates.
- Retain HTTP 401/403 gates during MCP initialization and its
initialized notification, with HTTP 503 controls for both stages.
- Discard only the upstream session used by a network failure or
timeout. Let the next explicit call initialize again.
- Add sibling-tool regression cases, authentication controls, and real
HTTP header/body timeout checks.
- Update runtime recovery guidance and the feature and access maps.

## Verification

- Red: the four new sibling-tool cases failed on the original
implementation. Each second call returned HTTP 404 with
`tool_not_found`. HTTP 401/403 control cases passed.
- Green: the same six-case selection passed after the production change:
`pnpm --filter @paperclipai/server exec vitest run
src/__tests__/tool-gateway-service.test.ts -t 'keeps sibling MCP tools
usable|keeps the connection blocked' --maxWorkers=1`.
- The regression checks the actual gateway and embedded PostgreSQL. The
injected provider fault preserves the call boundary. It checks failure
recording, no automatic replay, catalog retention, and successful
sibling dispatch.
- The real local HTTP fixture checks cancellation before headers and
during a body, then a successful explicit call with a fresh MCP session.
- `node scripts/check-maps.mjs` and `git diff --check` pass.
- Complete nearby suites pass: 166 tests across
`tool-gateway-service.test.ts`, `tool-gateway.test.ts`,
`remote-mcp-protocol.test.ts`, and `mcp-http.test.ts`, with one test
worker.
- `pnpm --filter @paperclipai/server exec tsc --noEmit` passes after
refreshing Runner TypeScript declarations. The first package typecheck
used declarations from before the rebase and failed two unchanged
native-session call sites; rebuilding that dependency resolved both
errors.
- The rebased head also passes all 11 selected
sibling/authentication/session regressions.
- Review regression red: setup-stage HTTP 401/403 controls failed
because connection health stayed `ok`; the two setup-stage HTTP 503
controls passed. Green: all six setup controls and the complete 166-test
nearby run pass. The fix separately handles typed initialization
authentication failures and preserves the existing caller error. The
server compiler also passes after this change.
- Full repository CI passes on
`256a65ccc302c1c31a80e87f1f91411d955028de`: typecheck, build, test
shards, Runner checks, browser checks, Canary dry run, and aggregate
gates. All checks are terminal: 55 successful, 2 expected skips, and the
separate Paperclip Review check is neutral. Greptile is 5/5 on this
exact head, with no open findings.
- Full repository checks were run in CI. Local verification in this
worktree used the focused suites and server compiler to keep parallel
work bounded.
- This change has no live provider or staging deployment proof.

## Risks

- A provider that remains unavailable can stay listed until a health
check detects the outage. Each call still fails with its original
controlled reason code.
- No automatic retry occurs because a failed remote action may already
have run.
- Already unhealthy connections still need a successful health check
after an upgrade.
- No schema, credential policy, or permission contract changes.

## Model Used

- OpenAI Codex, GPT-6 family, with tool use and code execution. The
runtime does not expose the exact serving model ID or context window.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-19 16:11:15 -05:00
DottaandPaperclip 9e8a9f28d5 fix(github): recover long publications from durable receipts (#16883)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents publish GitHub reviews through governed tools.
> - GitHub can accept a publication before the caller receives its
result.
> - Exact retries must return the existing publication receipt.
> - The gateway compared a shortened audit hash with a complete request
hash.
> - Long reviews therefore failed the retry check even when their
content was unchanged.
> - This pull request checks the complete saved publication and returns
its current receipt.
> - Agents can confirm their work without another provider write.

## Linked Issues or Issue Description

Refs #16284, which added durable publication recovery. Refs #16275,
which extended the GitHub tool deadline.

**What happened?**

An exact review retry returned an idempotency conflict after GitHub had
published the review. The invocation hash uses an audit preview that
shortens strings to 500 characters and arrays to 50 entries. The gateway
compared that hash with the complete request hash. The earlier retry
tests used short assessments.

**Expected behavior**

An unchanged request returns its durable receipt. Changed content,
changed authority, or a missing publication must not cause another
write.

**Steps to reproduce**

1. Submit a review with a summary and rationale longer than 500
characters.
2. Let publication finish after a short caller deadline.
3. Repeat the exact request. Before this fix, the gateway returns an
idempotency conflict.

**Paperclip version or commit**

Reproduced on master at `c5226daf7a37559ddc007446681717791d80a223`.

## What Changed

- Use the durable publication's complete request check for GitHub
retries. Keep run, agent, task, connection, tool, and current authority
checks.
- Return the saved receipt through normal gateway validation and
bookkeeping. Leave pending provider work to the durable worker.
- Refuse retries without a saved publication. Keep existing invocation
keys and audit previews unchanged.
- Cover long reviews, comments, progress updates, text suffix changes,
nested coverage changes, pending receipts, missing publications, and
revoked access.
- Document the recovery contract and verification recipes.

## Verification

- Red: both long-review receipt cases failed on unchanged production
code with the expected idempotency conflict.
- Green: all 144 GitHub review workflow tests passed on the final
source. This includes long reviews, comments, progress updates, pending
and missing receipts, revoked policy, and the native assigned-tool entry
point.
- Server and workspace typechecks passed. The production build passed.
- Map checks passed. These are structural checks.
- The first broad run found an incorrect new fixture assumption: its
discovery query returned two review tools. The fixture now selects the
exact submission tool. An existing description-edit polling test also
timed out; it passed separately without a production change, then passed
in the complete 144-test rerun. Both earlier failures remain recorded.
- Final commit `7cdcdc3a1ac79cb4ed77510a32d795228e3315e8` passed all 55
CI checks. Two optional Storybook jobs were skipped. [Complete CI
run](https://github.com/paperclipai/paperclip/actions/runs/39304187397).
Greptile is 5/5 on this exact commit, with zero unresolved review
threads.
- The earlier local full-suite attempt was stopped after the fixture
correction. The final local full-suite attempt was stopped after
complete final-head CI passed. No complete local full-suite pass is
claimed; full-suite acceptance uses the current-head CI matrix.
- Provider responses are controlled fixtures. Native assigned-tool
coverage uses the production gateway and disposable PostgreSQL. No live
GitHub fault injection, deployment, or historical task recovery is
claimed.

## Risks

A pending receipt remains pending. It does not complete the task. Caller
retries no longer dispatch provider work; the durable worker retains
that responsibility. A missing durable publication returns a conflict
because the gateway cannot establish the original write's outcome.
Existing approval and revocation checks remain effective. No database
migration or audit hash change is required.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, tool use, and code
execution. The session does not expose a more specific serving model ID
or context-window limit.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-19 13:54:42 -05:00
DottaandPaperclip 23b2ceb0d1 feat(connections): add unified Google Workspace connector (#15545)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Connected Apps supply governed tools to those agents.
> - Google services currently require separate connections or one broad
request.
> - Users need one account and control over which services it can
access.
> - A picker must change the OAuth request, not just the screen.
> - This pull request binds selected services to consent and tool
access.
> - The benefit is simpler setup without enabling unselected tools.

## Linked Issues or Issue Description

**Subsystem affected**

Connections catalog, OAuth grants, and the MCP tool gateway.

**Current behavior**

Google setup uses separate service profiles. A combined request alone
cannot express the user's choice before Google consent. Accumulated
Google grants can be broader than a new request.

**Proposed behavior**

Offer one Workspace connection. Let users choose Gmail, Drive (including
Docs, Sheets, and Slides), and Calendar. Start all three with read and
write access. Let users disable groups or choose read-only access. Do
not offer Chat, Contacts/directory, or cross-service Search in new
setup. Bind selected scopes to durable OAuth state. Cap effective tool
access to both the saved request and provider-reported consent.

**Reason and benefit**

Users can choose services without managing several accounts. Existing
company, identity, agent, and action-policy controls remain in force.

**Breaking changes**

No database migration. Existing service connections and grants remain
supported. New Workspace requests require the companion broker change in
https://github.com/paperclipai/paperclip-cloud/pull/823. Old brokers
reject narrowed requests safely.

Related work: #14740 reduced Google scopes; #14774 introduced the
earlier catalog hold. #15412 proposes a separate Gmail REST
implementation. This maintainer-requested change extends the existing
Connections and governed tool gateway paths. It does not replace them.

## What Changed

- Add one Workspace catalog entry and a namespaced action catalog.
Retain legacy definitions and saved accounts.
- Use the approved production service picker in Apps and task-hosted
setup. Keep dropdowns between names and toggles. Block empty selection.
- Derive exact scopes from a closed registry. Default selection requests
nine scopes. New requests exclude Chat, Contacts/directory, and Search
scopes.
- Bind selected scopes through managed and customer-owned OAuth state,
callback, claim redemption, and refresh.
- Cap accumulated Google grants to the saved selection. Recheck
grant-specific permission at discovery, listing, and dispatch, including
cached actions.
- Restore the current subject's saved request on reconnect. Block on
grant-load failure. Preserve draft selection on cancel and retry. Do not
create an account on picker cancellation.
- Preserve healthy services during partial outages, without combining
grants across identities. Keep writes from being automatically replayed.
- Preserve the original 21-scope legacy Workspace envelope contract as
individual profiles evolve.
- Add production-component Storybook stories, request validators,
consent regression tests, and rollout guidance. Integrate the stories
with the current feature catalog while preserving their IDs.

## Verification

- Integrated branch merges current master and has no merge conflicts.
- Full workspace build passed on the integrated checkout.
- Token gates passed. Storybook catalog structure check passed: 283
files and 2,750 stories.
- Final affected app suites: 914 passed. This includes 502
database-backed service/gateway tests and 412
UI/shared/validator/broker-handoff/governance tests. The production
picker tests cover defaults, identity-specific reconnect, and
failed-load recovery.
- Full workspace typecheck passed after adding required current-master
Storybook annotations. The final UI build also passed.
- The full repository suite is still running and includes the two
pre-fix retry assertions. Both were updated for the current gateway's
explicit-retry contract and pass in the clean 502-test database run. No
complete final-head full-suite pass is claimed.
- The initial local database startup failure came from missing
native-library symlinks in the fresh dependency install. The package's
postinstall repair fixed it; the clean database-backed run has no
skipped tests.
- Broker focused suite: 47 passed, including real signed session →
callback → claim redemption and wrong-scope rejection. Full broker suite
before the final test-only follow-up: 3,562 passed, 103 skipped.
- Real integrated-app walkthrough: Connectors → Workspace →
customer-owned app → service picker. Verified all three defaults,
empty-selection blocking, cancel/reopen with the selected read-only
access preserved, preserved form values, and zero saved connections
after cancel. Credentials were synthetic; no Google grant was made.
- Current-head CI and Greptile review remain pending. Previous-head
results are not treated as current.

## Risks

Ship the companion broker before app instances send narrowed requests.
Enable the intended Workspace client/profile separately on the target
broker. This pull request does not deploy a tenant app, start a fleet
rollout, alter Google Console, or upgrade the pinned reviewer stack.

The three-group picker is not proof of live provider access. Before
production, verify a real selected-service grant, reconnect, refresh,
allowed reads/writes, and denied unselected actions on the isolated
staging reviewer app. Google verification and preview enrollment remain
separate requirements.

Legacy service profiles stay compatible. New grant metadata keeps
provider scope evidence separate from requested scope evidence. Missing
or unsupported evidence fails closed. No scope union across user or
organization grants is allowed.

## Model Used

OpenAI Codex, GPT-5-based coding agent, with repository editing, code
execution, and browser tools. The exact serving model ID and
context-window size were not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change and contains no internal issue
ID
- [ ] I have run all final-head tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-19 12:07:30 -05:00
DottaandPaperclip 2cbca4ceb0 fix: retain channel tools across task continuations (#16806)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Chat channels give an agent tools for its assigned task.
> - Each later run must resolve the task and its accepted requester
again.
> - Some native runs store the task in a database column without the
older JSON fields.
> - Tool discovery read only the JSON fields. GitHub calls also required
a new inbound comment.
> - This pull request uses the durable task binding and accepted request
history.
> - Later runs retain their channel tools while current access checks
still apply.

## Linked Issues or Issue Description

**What happened?**

A GitHub task can find `begin_review` on its first run, then receive an
empty tool list on a later native run. If discovery is repaired alone, a
call can still fail because the later run has no new inbound comment.
Slack and voice tool access also read the older JSON task fields.

**Expected behavior**

A later run for the same task can find and call its assigned channel
tools. The current requester, task, company, agent, endpoint, and access
rules must still allow the call. Revoked access must fail.

**Steps to reproduce**

1. Start a GitHub task and accept its linked requester.
2. Confirm that the runtime can find `begin_review`.
3. Start a later native run with `nativeIssueId`, accepted parent
identity, and no JSON task ID or new wake comment.
4. Search for `begin_review` again. Before this change, the search
returns an empty list.
5. Repair discovery alone and call `read_pull_request`. Before the
request-history fix, the call fails with no active conversation
capability.

**Paperclip version or commit**

Reproduced in source tests before this patch. The branch includes master
at `fa50cab5685b9cdbb26ad9c610cc5b6c373d3ff4`.

**Additional context**

This is a core run-binding bug. The tests use PostgreSQL and synthetic
provider responses. They do not send live reviews, Slack messages, or
phone calls.

## What Changed

- Resolve run task IDs from durable columns, with legacy JSON fields as
a fallback.
- Use the same binding in the gateway, access policy, identity service,
and channel runtime preparation.
- Recover accepted channel request history across later runs. Keep
company, task, agent, and requester boundaries.
- Recover older wake sources through verified parent runs, including
after same-user board follow-ups. Clear the board message’s own legacy
wake fallback. Keep cycle checks without a fixed limit on valid task
history.
- Retain the GitHub origin after a board comment from the same accepted
user. Skip GitHub origin lookup results from other providers.
- Keep ordinary board tools available when the requester changes. Accept
legacy user comments without an author-type field. Still reject explicit
malformed task IDs.
- Prevent a later failure reply from replacing an earlier final GitHub
answer. Recheck this rule under the publication lease.
- Rotate the native tool contract to replace retained sessions with an
older tool list.
- Add continuation, publication, revocation, and boundary tests. Update
the access and feature maps.

## Verification

- Red: the later GitHub assigned-tool search returned an empty list
before the binding fix.
- Red: the later GitHub read failed with no active conversation
capability before the request-history fix.
- Green: runtime MCP discovery, GitHub read/begin/submit review, and
response publication pass for monitor and same-user board continuations.
- Green: Slack history, retained access controls, revocation, and voice
gateway continuation pass.
- Green: 35 channel, failure-publication, legacy-origin, long-history,
and provenance tests pass. Both review findings had failing tests before
their fixes.
- Green: 107 gateway, operation-credential, identity, and provenance
tests pass. These cover both first-run CI failures without changing
their tests.
- Green: 122 runtime, discovery-policy, and native-session tests pass; 2
unrelated tests are skipped.
- Map checks and whitespace checks pass.
- Updated to master in `cfcaba22628772d7fe1fb32b57ff7031c7072781`. The
sole conflict was two tests added at the same location. Both the
continuation test and the publication 403/401 test are retained.
- Current local verification: all 179 gateway, credential,
requester-identity, access-policy, runtime-context, and session-resume
tests passed. Repository typecheck and build passed. Map and whitespace
checks passed.
- The new review regression first returned no source for a same-user
board follow-up over legacy wake metadata. After the repair, all 27
provenance and channel cases passed. The combined GitHub
discovery/read/begin/submit path and revoked access are covered.
- The broader local channel run passed 176 cases and failed four
timing/setup cases: a 15-second Slack authorization timeout,
webhook-verification polling, a Slack acknowledgement deadline, and
Discord setup polling. All four passed in isolation without source,
assertion, or deadline changes. The earlier failure record is retained;
the exact load/order cause is not established. Full CI must pass on the
latest commit.
- Current head: `7b9a08dcbc7b8140b589742f136944270b0496bb`. Repository
typecheck and build pass on this commit. Greptile completed on this
exact commit with 5/5; all four review threads are resolved. The
final-head [CI
run](https://github.com/paperclipai/paperclip/actions/runs/39278921653)
passed: 55 checks passed and 2 intentionally skipped, with no failures
or pending checks. This includes the full sharded test matrix, browser
tests, typecheck, build, and Runner verification. Results were delayed
during the [GitHub Actions
incident](https://www.githubstatus.com/incidents/8tzbpcnl75jp). An
additional unsharded local full test run has not completed; the
completed full-suite evidence is CI.
- Earlier verification: the first PR commit failed two server suites.
Those failures were fixed and verified before this update. The original
4 GiB sandbox could not complete full local typechecking or tests; the
current checkout completed typechecking and build. The full test suite
runs through the CI shards. No full local test-suite pass or live
provider delivery is claimed.

## Risks

- Request history can restore the wrong authority if its scope is too
broad. The resolver checks each accepted parent against the task,
company, agent, and requester. Tests cover invalid and cyclic history.
- The native contract change starts a new provider session for older
contracts.
- No database migration or public API schema changes are required. Each
tool call still checks live access.

## Model Used

- OpenAI Codex, GPT-6 family. The runtime does not expose the exact
deployment model ID or context-window size. The agent used reasoning,
repository tools, code edits, and local test execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-19 11:20:39 -05:00
Devin FoleyandPaperclip 0b62c736a0 feat(connections): add generic Daytona API and Toolbox access (#16483)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Apps connections give agents controlled access to external services.
> - Daytona provides sandbox, file, command, Git, and account APIs.
> - Operators need to assign this access without exposing a server
credential.
> - A small task-specific tool list cannot serve general Daytona work.
> - This pull request adds the published HTTP actions through the
existing connection gateway.
> - Operators keep control of identity, agent access, and each action.

## Linked Issues or Issue Description

**Problem or motivation**

Paperclip has a Daytona sandbox provider for its own execution
infrastructure. It lacks a normal Apps connection for agents to manage
an operator's Daytona account. Reusing the server's provider credential
would bypass normal connection ownership and action permissions.

**Proposed solution**

Add a Daytona connection with personal or shared vault credentials.
Generate its action catalog from the official platform and Toolbox API
specifications. Use the existing run-bound gateway, grants, action
policies, approvals, revocation, and audit paths. Sandbox IDs come from
Daytona; tools need no Paperclip task or lease ID.

**Alternatives considered**

A four-action diagnostic subset cannot support general provider work.
Launching the provider CLI would expose host filesystem behavior. Native
OAuth is not included because a supported third-party registration path
was not verified. The native method uses an organization API key.
Existing Composio setup is a separate possible OAuth route and is not
qualified by this change.

**Roadmap alignment**

This extends the existing Apps and Connections provider coverage in
`ROADMAP.md`. It uses that system's credential and permission model.
GitHub searches found no existing Daytona Apps connector PR or issue.

## What Changed

- Add the Daytona catalog entry, provider source override, official
theme logos, permission review, and setup documentation.
- Generate 274 HTTP actions: 169 platform and 105 Toolbox operations.
Record two WebSocket upgrades separately; do not advertise them as HTTP
tools. Record source URLs and hashes.
- Add provider-focused execution for sandboxes, snapshots, volumes,
files, Git, commands, computer use, and account resources. Provider
authority still limits which endpoints accept an organization key.
- Validate action arguments. Resolve Toolbox targets through the fixed
platform API. Check sandbox identity and organization before using an
approved Daytona proxy. Keep normal guarded network requests and vault
resolution.
- Transfer file bytes without reading or writing host files. Share one
caller deadline across prerequisite reads, dispatch, and response
reading. Bound requests and responses. Do not automatically retry writes
with unknown outcomes. Redact the selected credential from results and
errors.
- Classify credential access, access changes, interactive input, command
execution, file permissions, and deletion (including TTL and
auto-delete) as destructive actions. Keep new and changed actions under
catalog review.
- Upgrade saved catalogs and retire the earlier diagnostic names. Hide
retired actions from the UI and agent discovery even when provider
refresh fails. Preserve existing action choices and historical records.
Reject stale same-name contracts during agent discovery and dispatch
before a board refresh.
- Add database-backed gateway and catalog-upgrade regressions, schema
generator tests, a browser setup check, and both capability map updates.

## Verification

- Passed on the review-fix head: 57 focused Vitest cases (18 Daytona
database/client cases, five HTTP boundary cases, and 34 shared catalog
cases), four generator tests, `pnpm -r typecheck`, and `pnpm build`. The
five HTTP cases use a real local server through the pinned production
transport. The normal browser setup check and 17 generator/artwork cases
passed on the initial PR head; those UI and artwork paths are unchanged.
- Passed on this branch: brand asset validation, token gates, map
structure checks, and `git diff --check`. All latest-head GitHub checks
pass on `ea979ccb3a64dc9508807823540f935c4316ea35`. Apex is 5/5, with
all five findings addressed and zero unresolved threads. The branch has
no merge conflicts.
- The initial catalog-generation command could not find the external
research corpus. The supported `--definitions-only` mode regenerated the
definitions successfully. Unrelated generated drift was excluded.
- An earlier full local Vitest attempt on the preview branch failed when
bundled PostgreSQL could not start on macOS. After replay onto current
master, all three unchanged event-sequencing cases pass with bundled
PostgreSQL; the earlier startup cause remains uncertain.
- The first full-suite command on this branch used my external test
database override. Chat fixtures require a current migrated schema and
failed because `agents.lifecycle_state` was absent in that disposable
database. A focused comparison reproduces failure with the override and
passes with normal setup. That attempt was stopped, and the full suite
restarted without the override. Focused Daytona cases use their own
migrated PostgreSQL 16 databases.
- The clean local full-suite attempt recorded embedded PostgreSQL
startup failures in two unchanged native-runtime cases and two failures
in unchanged Git fixtures. It was stopped after these failures were
isolated; no full local suite pass is claimed. The 15 matching
native-runtime cases pass separately. The Git cases fail with the local
`init.defaultBranch=main` setting and pass with `GIT_CONFIG_COUNT=1
GIT_CONFIG_KEY_0=init.defaultBranch GIT_CONFIG_VALUE_0=master` limited
to the test command. No global Git setting was changed. Full Linux CI is
green.
- Fixtures use real database persistence and run-bound gateway sessions
with provider responses replaced by fixtures. They cover personal/shared
vault ownership, reads and writes, file bytes, Off policies, revocation,
organization mismatch, untrusted proxy rejection, output bounds, saved
catalog upgrades, failed refresh, and permission preservation.
- The first Apex review found five valid issues: multipart transport
compatibility, destructive classification, caller deadlines, audit
filters, and stale saved contracts. All five were fixed with regressions
before the second review. A typecheck then found the nullable saved
schema hash; admission now accepts that stored type and rejects null as
stale.
- The browser check covers setup controls without submitting a provider
key. No real Daytona account, provider write, native OAuth flow, or
agent model run is qualified. Live provider read/write proof remains
pending a test account.

## Risks

- Daytona keys have broad Toolbox authority, including keys with no
selected resource scopes. Only assign this connection to trusted agents
and review its action permissions.
- Remote writes or commands can continue after a timeout. Calls do not
automatically retry an uncertain outcome.
- The bridge targets Daytona Cloud. Proxy hosts are restricted; an
unrecognized future regional host fails closed.
- Files and responses have an 8 MiB bound. Each action has one caller
deadline, capped at 30 seconds across all provider requests and response
reading. Interactive WebSocket sessions are not exposed as HTTP actions.
- The generated JSON snapshot exceeds 1,000 lines because it holds the
provider input contracts. Generator logic and execution code remain in
separate small modules. Updates require explicit regeneration and
catalog review.
- No database migration or lockfile change is required.

## Model Used

OpenAI Codex, GPT-6. The runtime identifies the model family but does
not expose the exact backend model ID or context window. Used reasoning,
repository inspection, code editing, shell tools, database verification,
and browser tests.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-15 14:05:26 -07:00
DottaandPaperclip 5ec93dae01 fix(connectors): avoid redundant run preparation discovery (#16461)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Run preparation selects connector skills and tools under current
permissions.
> - An assignment check can build an unrelated company tool catalog and
wait for a shared discovery slot.
> - Native startup repeats assignment discovery to obtain static tool
definitions.
> - Slack also locks and reconciles an unchanged catalog, then repeats
task authorization.
> - This pull request removes that redundant startup work and preserves
live checks at tool execution.
> - The benefit is less preparation work, less contention, and unchanged
permission and approval enforcement.

## Linked Issues or Issue Description

Related: #16454 (detailed preparation tracing), #16425 (workspace
preparation performance).

**What happened?**

Connector configuration added avoidable work to run preparation. Browser
Use assignment authorized the whole company catalog, including other
connector families. Four connector resolvers ran in series. Native
controller startup repeated all assignment resolution when any connector
skill was pinned. Slack reconciled an unchanged catalog under an
advisory lock and checked task authority twice.

**Expected behavior**

Prepare only the connector data needed by the run. Reuse static tool
definitions from pinned skills. Preserve current company, assignment,
identity, profile, policy, grant, and approval checks when an operation
executes.

**Steps to reproduce**

1. Configure Browser Use and an unrelated MCP catalog with 500 tools for
one company.
2. Resolve Browser Use resources for an authorized task. The old path
reads and authorizes the unrelated catalog.
3. Occupy both full discovery slots. The old assignment check waits for
those slots.
4. Resolve a linked Slack task repeatedly with an existing current
catalog. The old path still acquires a catalog lock and repeats
authority checks.

**Paperclip version or commit**

Base: `68a09dcb5f50e3fb6ceaaeac2706d96db97fc9b3`.

**Deployment mode**

Observed in hosted native runs. The local reproduction uses a source
checkout and real embedded PostgreSQL.

## What Changed

- Restrict Browser Use catalog reads and policy decisions to active
Browser Use connections. Preserve guest restrictions, reviewed catalog
filters, live grants, and deterministic resource order. Keep assignment
checks out of the full discovery queue.
- Resolve the four independent connector families concurrently. Wait for
outstanding work to settle before propagating a failure, and retain
registration order.
- Register native direct connector tool definitions from pinned
connector skills. Keep live assignment resolution and governed execution
on every tool call.
- Compare Slack catalog hashes before reconciliation. Current catalogs
skip the catalog transaction and lock. Check task authority once after
current profile checks. Initial setup and upgrades retain locked
reconciliation and checks before and after it.
- Precompute immutable bundled Slack descriptor hashes. Preserve
disabled entries, profile state, removed bindings, and review metadata.
- Document the startup paths and add focused contention, revocation,
catalog upgrade, and failure-settlement regressions.
- Repair a pre-existing CI guard false positive: the filename
`docker-compose-secrets.test.mjs` matched `secrets.`. Check GitHub
secret expressions instead, with dot, bracket, and whole-context
controls. No workflow changes.

## Verification

- Conflict resolution: merge `b1566ca494f4af6e8d4418fa37393c5caee02582`
from master into head `55140b1609ae7bd21a038eb75a48c2683299e4b4`.
Combine both secret-expression guards and regression cases. Preserve
current workspace, native recovery, and Daytona changes. All 467 policy
tests, 787 focused native/connector/Browser Use tests, and server
typecheck passed locally after integration. Fresh full CI passed all 55
checks on this head, with two optional Storybook jobs skipped. Greptile
reviewed this exact head at 5/5 with no unresolved threads.

- `pnpm -r typecheck` — passed.
- `pnpm build` — passed.
- Focused native startup, connector concurrency, and AgentMail suites —
763 tests passed.
- Complete Browser Use suite — 33 tests passed. Focused Slack catalog,
linked task, retained provenance/revocation, and pinned native tool
tests passed. The complete runner tool authority suite passed (26
tests). The 1,337-case chat suite passed 1,335 tests and had two
failures: the original new transaction-count assertion and a 15-second
timeout in the publication-batch case. Both cases passed an isolated
rerun. The publication timeout occurred during concurrent local checks;
the passing rerun does not erase it. CI must still pass the chat shards.
- The first focused run found two incorrect new assertions: policy
evaluation performs a second catalog lookup, and an explicit allow
policy overrides a profile default. The tests now distinguish the scoped
catalog query and revoke the governing policy. The focused Browser Use
rerun passed all 33 cases after these assertion fixes.
- Full-suite test verification used the CI shards of `pnpm test:run`.
Local runs focused on the changed boundaries; the complete repository
test command was not duplicated locally.
- `node --test '.github/scripts/tests/*.test.mjs'` — all 467 policy
tests passed. The CI failure reproduced on the unchanged base guard
before this correction. Actual GitHub secret expressions still match the
guard.
- CI on the final head passed all chat shards, typecheck, build, Runner
checks, browser shards, and other test lanes. One existing
workspace-runtime case hit `EADDRINUSE` for a randomly selected test
listener in server shard 1/12 (1,162 tests passed in that shard). The
shard passed its single retry on the same SHA with no source changes.
The first failure remains recorded here. Previous reviewed head
`7191ac40691ddb1ef38b86665c95cc14c57ea390`: all 55 checks passed, two
optional Storybook jobs skipped, and no pending or failed checks.
Greptile is 5/5 on this head with no unresolved review threads. The
external security scan passed.

Local before/after measurements use the old and new implementations
against the same fixture. Both paths are warmed. Seven samples alternate
execution order. The database is real embedded PostgreSQL. No latency is
injected. These measurements do not establish hosted speed.

| Local path | Before median (range) | After median (range) | Work
reduction |
| --- | --- | --- | --- |
| Browser Use assignment with 500 unrelated MCP tools | 44.020 ms
(33.994–107.328) | 26.502 ms (20.385–99.037) | 26 → 22 statements;
unrelated catalog and discovery queue removed |
| Slack assignment with current catalog and linked responsible user |
49.723 ms (43.624–80.653) | 22.141 ms (19.781–30.818) | 5 → 2 database
transactions; no catalog lock on the current path |

The contention regression fails against the old implementation after a
bounded five-second wait, then passes against the optimized
implementation. It occupies the real global discovery scheduler and
verifies that Browser Use assignment completes without releasing those
slots. It records catalog SQL to verify connection scope. Revocation
cases remove policy, catalog availability, and grants between calls.
Existing Slack integration cases exercise retained provenance, removed
profile bindings, identity revocation, approved writes, and current
responsible-user restrictions.

## Risks

- Pinned native tool definitions can remain visible after access is
revoked. They confer no authority. Invocation rechecks current
assignments and fails closed before an operation executes. Newly
available connector families still require a newly admitted skill
selection.
- Concurrent resolution can increase instantaneous database reads to
four connector families per configuration call. It is a fixed bound.
Outstanding work settles before a failed preparation returns.
- Slack skips reconciliation only when all bundled version hashes and an
existing profile match. Missing or changed definitions still use the
existing locked path. Operator revocations remain effective.
- This PR addresses connector preparation. Workspace transfer, provider
startup, copyback, and queue delay can still keep total latency above
the user's goal. Hosted improvement needs a deployed before/after trace.
- No schema, lockfile, workflow, telemetry event, or authorization cache
changes.

## Model Used

OpenAI GPT-6 (Codex), with reasoning, repository inspection, code
execution, and GitHub tools. The runtime does not expose the exact
serving model ID or context window.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-15 14:08:01 -05:00
DottaandPaperclip 25b16e65fc fix(observability): detail workspace and connector preparation (#16454)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The server prepares a repository, sandbox workspace, and connector
access before an agent can reply.
> - Existing traces measure large preparation phases but leave important
work inside those phases unclear.
> - Repository source materialization and Browser Use discovery can do
more work than their callers suggest.
> - New detail must preserve access checks, run ownership, and the
existing work order.
> - This pull request adds fixed OpenTelemetry spans inside those
operations.
> - Operators can use one deployed trace to select the next performance
fix.

## Linked Issues or Issue Description

Refs: #16419, #16425, #16428.

**What existing behavior does this improve?**

Startup traces for server workspace preparation, native sandbox
synchronization, and connector assignment resolution.

**Current behavior**

Workspace and connector aggregate spans can take many seconds. The trace
does not separate private source cloning, seed reuse, archive work,
progress persistence, or each connector resolver. Browser Use resource
selection builds the full authorized tool catalog. The configuration and
native backend can each resolve connector assignments.

**Proposed behavior**

Show the awaited work as fixed child spans. Separate discovery queue
wait from catalog construction. Preserve the initiating run context when
another run releases a scheduler slot. Add no new lifecycle event writes
for workspace detail callbacks.

**Reason and benefit**

Operators need evidence about the hosted critical path before another
performance change. These spans distinguish filesystem work, provider
transfer, authorization, transaction entry, locks, and reporting
overhead.

**Breaking changes**

None. Authorization, query order, transaction ownership,
synchronization, existing progress events, and task results retain their
contracts. The change adds opt-in Observability spans under the existing
OTLP endpoint gate.

## What Changed

- Trace private source directory setup, Git credentials, clone,
requested ref, source identity, and checkout publication.
- Trace sandbox source capture, seed cache/replay, shallow clone and
archive, overlay copy/archive, verification, upload, warm stamps, and
descriptor persistence.
- Separate existing awaited progress reporting from useful workspace
work. Keep new detail callbacks trace-only.
- Trace each connector resolver and Browser Use catalog/grant work.
Split discovery queueing, descriptor sources, and policy evaluation.
- Trace Slack authority, credential reads, catalog synchronization,
transaction entry/settlement, and advisory locks.
- Update the real filesystem staging trace expectations and the
Honeycomb investigation guide.
- Add a real scheduler regression for a queued job released by another
run. Assert run ownership, parent spans, and exactly one completed queue
span.

## Verification

- Passed: server and adapter-utils TypeScript checks (`pnpm --filter
@paperclipai/server exec tsc --noEmit` and `pnpm --filter
@paperclipai/adapter-utils exec tsc --noEmit`).
- Passed: 252 checks across 9 focused files. These cover real filesystem
staging, native workspace synchronization, trace ownership, endpoint
gating, and workspace preparation.
- Passed after review: 26 trace-context, tracing-failure, and
discovery-scheduler checks across 3 files (these overlap the first run).
The new queued-discovery regression fails when context binding is
removed and passes after restoration. An initial submillisecond
timestamp assertion was replaced with explicit span-completion order to
avoid SDK clock precision assumptions.
- Passed on implementation commit
`0e68968a4bd61001893d833f281f12bb110edbe1`: `pnpm -r typecheck` and
`pnpm build`. The later commit updates only an engine test expectation.
- The first CI workspace lane found an old engine trace-tree
expectation. It expected the earlier span set. The updated assertion now
checks the new detail spans, their parents, and single completion. Its
targeted executor check passes.
- The full test suite, build, release registry, and end-to-end gates
pass in CI on final head `40e37d89ef679109d60a96bb5731a0616efeaa43`: 55
passing checks and 2 optional Storybook skips. The full suite was not
duplicated locally after the focused run.
- Greptile reviewed the final head at 5/5. No unresolved review threads
remain.
- After deployment, filter Honeycomb by the hashed run ID and verify
`service.version`. Inspect `heartbeat.prepare.workspace`,
`runtime_selection.workspace_sync`, and the connector assignment
parents. The guide in `doc/observability.md` explains each new family
and the nested intervals.
- No hosted run has used this new instrumentation yet. Local checks
cannot establish hosted latency improvement.

## Risks

- Tracing adds spans when an operator configures an OTLP endpoint.
Detail remains capped at 16 spans per phase per execution attempt.
- Parent and child durations overlap. The documentation explains how to
avoid counting them twice.
- A discovery job can start in another run's scheduler continuation. The
bound callback preserves the initiating async context.
- New spans export fixed names and the existing hashed run ID. They add
no paths, repository URLs, connector resource IDs, credentials, tool
schemas, or exception text.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, tool use, and code
execution. The exact served model ID and context-window size are not
exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-15 12:21:32 -05:00
Leia MarshallandLeia Marshall e4ce989099 Add opt-in Gmail archive capability (#16331)
## Thinking Path

> - Paperclip separates Gmail capabilities by reviewed OAuth scope and
tool policy.
> - Existing Gmail connections support reading and draft creation but
not bounded archiving.
> - Gmail archive requires `gmail.modify`, so it must not widen existing
read or draft grants.
> - This change adds an opt-in archive profile and limits it to removing
only the `INBOX` label.
> - A final pre-dispatch check preserves that boundary for approved and
replayed calls.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

This improves the existing Gmail connection and tool-governance
behavior.

**Subsystem affected**

Cross-cutting: `packages/shared` and `server`.

**Current behavior**

Paperclip can read Gmail and create drafts. It does not provide an
opt-in, bounded action for archiving a thread.

**Proposed behavior**

Paperclip offers a separate Read & archive connection. It exposes
`unlabel_thread` only for that connection and accepts only `threadId`
with `labelIds: ["INBOX"]`.

**Reason and benefit**

Users can archive processed Gmail threads while sending, trashing, and
other label changes remain disabled.

**Breaking changes**

None. Existing read and draft methods keep their current scopes and
defaults.

Related work: #15545 adds a unified Google Workspace connector and keeps
Gmail draft-only. #15412 adds a separate Gmail REST server without label
mutations. Neither PR adds this bounded archive capability.

## What Changed

- Added the `gmail.archive` connector profile.
- Added managed and customer-owned archive connection methods.
- Added an Inbox-only schema and final pre-dispatch argument guard.
- Classified archive as destructive and added Ask-first defaults for the
archive methods.
- Added permission-review records and focused tests.

## Verification

- `pnpm --filter @paperclipai/server typecheck`
- `pnpm --filter @paperclipai/server build`
- `pnpm --filter @paperclipai/shared typecheck`
- Focused Vitest suites: 42 tests passed.
- App-definition ingestion: 103 definitions and 0 review flags.
- `git diff --check`

## Risks

The main risk is accidental scope or operation expansion. The new
profile is opt-in. Existing profiles retain their scopes. The gateway
validates the exact Inbox-only payload immediately before dispatch.
Production rollout requires separate OAuth configuration and fresh
consent.

## Model Used

Unverified — reliable runtime model metadata was not available. Confirm
the model attribution before publishing.

## Checklist

- [x] I have included a concise design rationale
- [x] I have specified that model attribution is unverified
- [x] I have checked ROADMAP.md and found no Gmail archive entry
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have described the issue in this PR
- [x] I have not referenced internal or instance-local issues
- [x] My branch name describes the change
- [x] Local tests and builds pass
- [x] I have added or updated tests
- [x] I have updated relevant documentation and permission records
- [x] I have documented risks
- [ ] All hosted CI gates are green
- [ ] Greptile and human review are complete

---------

Co-authored-by: Leia Marshall <leia@paperclip.ing>
2026-10-15 09:24:31 -07:00
DottaandPaperclip 66424362a8 perf(runtime): skip discovery without a Browser Use connection (#16345)
Skip full tool discovery when no enabled, active Browser Use connection exists for the run company. Configured connections retain existing authorization and discovery.

Verified exact head 7fdf51eae8: full CI, E2E, Superagent and Apex 5/5; 29 affected lifecycle tests and full repository typecheck/build pass.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-14 18:18:00 -05:00
DottaandPaperclip 21ee5c9c85 feat(sentry): add opt-in run failure reporting with saved run references (#16300)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agent runs retain execution results and output in the instance.
> - Operators can report run failures to Sentry for investigation.
> - A backend Sentry DSN currently enables those reports with no
separate choice.
> - A repair bot also needs the exact instance, company, agent, and
saved run.
> - This pull request adds an explicit reporting mode and authenticated
run references.
> - The benefit is that operators can investigate every failure on
selected deployments while keeping other deployments quiet.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Optional server reporting of terminal agent run failures to Sentry.

**Current behavior**

A backend DSN enables run failure reports. Known account, quota, and
connection failures are excluded. Reports omit company and agent
references. Runs without a task are skipped. Tool-name collision errors
omit the conflicting bindings.

**Proposed behavior**

Set `PAPERCLIP_SENTRY_RUN_FAILURES=unexpected` to retain the existing
filters. Set it to `all` to report every failed or timed-out run,
including known setup and quota failures. The default is `off`. Expected
stops stay excluded. Each report supplies identifiers and API paths that
a bot can use with its own authorized Paperclip connection.

**Reason and benefit**

Operators can enable broad failure reporting on one staging deployment.
A bot can fetch the saved run, events, and logs instead of relying on a
short error label or copying complete transcripts into Sentry.

**Breaking changes**

A backend DSN alone no longer enables run failure reports. Existing
operators must set `PAPERCLIP_SENTRY_RUN_FAILURES=unexpected` to retain
the previous policy. Ordinary server exception capture remains enabled
by the DSN. Failure grouping gains the phase and original cause code
when available.

Refs #16281, #16297, #16298. These cover related tool discovery and
failure diagnostics. This change retains their existing boundaries and
does not depend on their open changes.

## What Changed

- Add `off`, `unexpected`, and `all` modes. Missing and invalid values
keep reporting off before diagnostic reads.
- Keep the Sentry backend DSN and audited optional SDK requirements.
- Include company, agent, run, and optional task identifiers. Include
versioned paths for the saved run, events, log, and task. Add absolute
run URLs only for a valid public HTTP(S) origin.
- Report company runs without a task. Prefer the first-class issue
binding when selecting the task reference.
- Select saved native provider/model settings, timestamps, and original
failure codes. Preserve gateway reason codes through redacted exception
projection.
- Group by error code, adapter, and available phase/cause. Keep run IDs
and error text out of grouping.
- Include the tool names and connection bindings in collision errors
without copying tool schemas or provider metadata.
- Document configuration, upgrade behavior, bot access, redaction,
retention, and best-effort delivery.

## Verification

- All 56 checks passed for `cccb47a721eaa1cc36e63278fc177b9101477541`.
Two unrelated Storybook jobs were skipped by their path filters. No
checks remain pending.
- Complete [Paperclip CI
run](https://github.com/paperclipai/paperclip/actions/runs/38591422284):
passed. This covers general and serialized tests, Runner checks, browser
E2E shards, typecheck, build, release-registry checks, canary dry run,
and Docker checks.
- Greptile reviewed that exact head: 5/5, with no unresolved threads or
actionable findings. Superagent, Socket, and Snyk checks passed.
- Local focused reporting, diagnostics, collision, and real SDK contract
tests: 410 passed across five suites. This includes the external-wake
filter in both reporting modes.
- The audited `@sentry/node@10.71.0` tests use an in-memory transport.
No events were sent to Sentry.
- `pnpm -r typecheck` and `pnpm build`: passed locally after the final
rebase.
- `node scripts/check-module-boundaries.mjs` and `git diff --check`:
passed.
- The full local `pnpm test:run` command was stopped to avoid
duplicating the complete CI shards. It did not finish locally. The full
suite passed in CI.

## Risks

- The new default disables existing run failure exports until the
operator selects a mode. The upgrade setting is documented.
- `all` can increase event volume and include known user-action
failures. Secret redaction and diagnostic limits still apply.
- Reporting remains asynchronous and best effort. This PR adds no
durable delivery queue and does not install a bot or Sentry alert rule.
- API references grant no authority. Existing company, private-task, and
run-output checks still apply. Complete transcripts stay in Paperclip.
- This changes optional Sentry error reporting. It changes neither
first-party Telemetry nor OpenTelemetry span exports. It adds no
database migration.

## Model Used

- OpenAI GPT-6 through Codex, with reasoning, shell, file editing, and
test execution. The exact backend deployment ID and context window are
not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-14 10:56:40 -05:00
DottaandPaperclip f69c6a8a35 fix(github): retry and recover review publication (#16284)
## Thinking Path

> - Paperclip manages agent work and its published results.
> - GitHub review bots publish task-bound assessments through governed
tools.
> - A review can survive a caller timeout because its publication intent
is durable.
> - A transient provider failure still returned a failed receipt before
recovery.
> - Gateway replay could return an empty cached result, and publication
retries had no dedicated scheduler deadline.
> - This pull request adds bounded retries, receipt recovery, and an
independent durable worker.
> - A temporary GitHub failure can recover without another review or
duplicate publication.

## Linked Issues or Issue Description

Refs #16275. That change increased the GitHub tool deadline. This change
repairs retry and recovery behavior.

**What happened?**

A one-off GitHub HTTP 503 or 429 left the caller with a failed
publication receipt. The saved retry did not have its own scheduler
worker. An identical gateway call after a timeout could return an empty
cached result instead of the saved publication receipt.

**Expected behavior**

Temporary failures should retry under current authority. Pending work
should resume at its saved deadline after restart. Exact retries should
return the current receipt. Lost responses must not create duplicate
comments, findings, or checks.

**Steps to reproduce**

1. Admit an advisory PR review request and begin its assessment.
2. Return HTTP 503 or 429 for the first summary-comment PATCH, then let
GitHub succeed.
3. Submit the assessment through the run-scoped tool gateway.
4. On master, the result says publication failed. With this change, the
same call returns the processed receipt and neutral check.
5. For receipt recovery, use a shorter caller deadline and repeat the
exact call after publication finishes.

**Paperclip version or commit**

Reproduced on master at `6cb6ac3b8e40cb90d734ba115a2f22f850ff9477`.

## What Changed

- Classify transient transport, server, and rate-limit failures. Honor
Retry-After and primary reset deadlines. Stop retries for permanent
permission and validation errors.
- Retry safe reads and installation-token issuance within a bounded call
window. Retry publication writes through the existing outbox and
provider receipt checks.
- Wait for up to three publication attempts within a 45-second retry
window. Keep the durable eight-failure cap and enforce saved deadlines
on explicit retries.
- Register an independent GitHub publication worker with
transaction-owned wake notifications, startup recovery, and per-PR lease
deadlines.
- Recover exact comment, progress, and assessment gateway calls from
their stored publication. Require the same company, agent, task, run,
connection, tool, arguments, and current policy. Clear stale timeout
errors after recovery. Approved exact retries read the current receipt
through the existing signed replay path; the worker retains the verified
approval for deferred writes and still rejects revocation.
- Add regression coverage and update the connector guide and
feature/access maps.

## Verification

- Red: both one-off HTTP 503 and 429 regressions failed on master with a
failed publication receipt.
- Green: the same tests passed after the change.
- GitHub review workflows: 133 covered against isolated PostgreSQL 18.
The final combined run passed 132 and hit one fifteen-second timeout
under concurrent build load. All three variants of that workflow passed
in isolation. The earlier combined run passed all 131 workflows before
the two approval regressions were added.
- Retry policy and delivery coordinator tests: 35 passed.
- Additional focused checks passed for the actual eight-failure cap,
exact-argument conflicts, policy revocation, and timeout receipt
recovery. The approved deferred-publication and revocation regressions
failed before the review repair and passed after it, with one approval
request preserved.
- `pnpm -r typecheck`: passed. `pnpm build`: passed. `node
scripts/check-maps.mjs`: passed.
- The broad local `pnpm test:run` path remains blocked by the host's
exhausted PostgreSQL shared-memory slots. The changed database-backed
workflows use an isolated Docker database instead. All Paperclip CI jobs
passed at a6d81b67cb, including the required verify and E2E gates.
Greptile is 5/5 at the same head, with the approval-recovery finding
resolved. The external Superagent Security Scan also passed.
- Live acceptance passed on a dedicated single-repository local test App
and native Paperclip Runner. A test-only preload returned one HTTP 503
before the final summary PATCH. One submit_review call completed in
17.305 seconds; the bot published its 3/5 advisory review and neutral
check, then automatically finished the task. [Published
review](https://github.com/paperclipai/paperclip-permissions-smoke-20260926-pap57-fee7428e/pull/9#issuecomment-6168809007),
[neutral
check](https://github.com/paperclipai/paperclip-permissions-smoke-20260926-pap57-fee7428e/runs/115863564014).
This live test ran at de6862072c; approval-specific recovery is covered
by the later regressions. The preload is excluded from the repository
and removed from the running local instance.

## Risks

Retries make additional bounded provider requests. A provider write with
a lost response is recovered through its existing App-owned identity
before another write is permitted. The worker rechecks authority and the
PR head; revocation and superseded work stop publication. Calls can
still exceed their deadline, and repeated failures eventually exhaust
the retry cap. A pending publication is not a completed task. The agent
still owns task completion after receiving a confirmed receipt. No
schema migration is needed.

## Model Used

OpenAI Codex (GPT-6 family), with code inspection, editing, and test
execution. The exact deployment model ID and context window are not
exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-14 08:40:32 -05:00
DottaandPaperclip 1e731520bb fix(mcp): deduplicate discovered tool descriptors (#16281)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents receive governed app tools through the MCP gateway.
> - GitHub review tasks can contain a PR discussion and inline finding
threads.
> - Discovery repeated each tool for every conversation linked to the
task.
> - Native tool registration treated these repeated descriptors as a
fatal collision.
> - This pull request lists GitHub tools once and deduplicates identical
descriptors across providers.
> - Agents can continue work while conflicting definitions and normal
tool permissions remain enforced.

## Linked Issues or Issue Description

**What happened?**

A follow-up run on a GitHub review task failed with
`assigned_mcp_tool_name_collision` before the provider started. The task
had a main conversation and an inline finding thread. Discovery returned
the same seven tools twice. Repeated descriptors from other providers
could cause the same failure.

**Expected behavior**

Discovery exposes each identical tool once. Separate connection names
remain separate tools. Conflicting definitions remain an error. Every
call still passes the gateway's policy, approval, revocation, and audit
checks.

**Steps to reproduce**

1. Run a GitHub review and publish an inline finding.
2. Send a follow-up request on the same PR task.
3. Inspect the task-bound bot catalog and the next native startup.
4. Separately, return two identical descriptors from an assigned app or
plugin catalog.

Related: #13717 added the GitHub review workflow.

## What Changed

- Use a distinct GitHub discovery query across conversations that share
an endpoint and task. Keep both thread mappings.
- Deduplicate identical gateway descriptors by their full gateway name
across providers. Use this rule for MCP listings, plugin listings,
assigned-tool startup, and fresh assigned-tool discovery.
- Reject conflicting descriptors with `tool_name_collision`. Keep the
projected-name collision guard.
- Cover GitHub inline-thread discovery, repeated plugin/app tools,
direct and on-demand catalogs, revocation, conflicting bindings, and
separate connection names.
- Update the execution contract, GitHub documentation, and agent access
map.

## Verification

- Regression baseline: six new assertions failed on the original
implementation. GitHub returned fourteen entries, plugin discovery
repeated a tool, and native startup failed on repeated app tools.
- Focused verification passed: 272 GitHub cases, all 29 assigned-tool
cases, and all 88 gateway acceptance cases.
- `pnpm -r typecheck` passed.
- `pnpm build` passed.
- `node scripts/check-maps.mjs` passed.
- Current-head CI passed: 55 checks passed, two inapplicable checks
skipped. Greptile scored the current head 5/5 with no unresolved
threads.
- Full local `pnpm test:run` encountered a 15-second timeout in the
existing bearer-auth throttling test, before tool discovery. That test
passed in the separate 88-case gateway run and on one isolated rerun.
The duplicate broad local run was then stopped; the complete
current-head CI matrix passed.

## Risks

- Deduplication keeps the first identical descriptor and preserves
listing order. Same-name descriptors with different definitions fail
with HTTP 409 rather than selecting an ambiguous binding.
- The GitHub query changes result cardinality. It retains the existing
task/company/agent filters and leaves conversation rows intact.
- No schema migration or credential change. Live staging recovery
requires deployment and a new authorized run.

## Model Used

OpenAI Codex, based on GPT-6, with code inspection, editing, browser
investigation, and test execution. The exact serving model ID and
context window are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-14 07:39:54 -05:00
DottaandPaperclip 6cb6ac3b8e fix(github): allow review publication to finish before tool timeout (#16275)
## Thinking Path

> - Paperclip manages agent work and its published results.
> - GitHub review bots publish task-scoped assessments through the tool
gateway.
> - A submission saves its assessment and publication intent before it
waits for GitHub.
> - The gateway's ten-second default can expire while those provider
requests continue.
> - GitHub can then show a completed review while the agent receives a
timeout and blocks its task.
> - This pull request gives GitHub bot tools the existing sixty-second
maximum by default.
> - The caller can receive the confirmed publication result without
manual task repair for ordinary slower responses.

## Linked Issues or Issue Description

**What happened?**

A GitHub bot's `submit_review` call returned `504 tool_timeout`. Its
saved assessment continued to publish a summary and completed check
after the caller stopped waiting. The agent treated publication as
unconfirmed and blocked the task.

**Expected behavior**

A valid GitHub response that takes longer than ten seconds should return
its confirmed publication receipt within the gateway's supported
deadline.

**Steps to reproduce**

1. Configure a report-only GitHub review bot and admit a PR review
request.
2. Delay the summary-comment PATCH response by eleven seconds.
3. Submit a complete assessment through a real run-scoped tool gateway.
4. Before this fix, the caller gets a timeout while the durable action
publishes one summary and a neutral check.
5. With this fix, the caller gets `completed`, `processed`, and the
assessment score. Retrying does not add another comment.

**Paperclip version or commit**

Reproduced from master at `2c17d3ca52a7840f280f1dd87975a2ad77de3b35`.

Related changes: Refs #16259 and #16263. Those changes repaired
connection recovery. This change addresses the subsequent submission
deadline.

## What Changed

- Use a sixty-second default for `paperclip_github_chat` tools in run
execution and board tool tests.
- Preserve explicit shorter deadlines and the existing maximum.
- Add PostgreSQL-backed regressions through the real gateway for delayed
publication, explicit deadlines, neutral checks, and duplicate
prevention.
- Document the deadline and the durable publication behavior.

## Verification

- Red: the eleven-second publication regression failed with
`ToolGatewayHttpError(504, tool_timeout)` after the summary and neutral
check were published.
- Green: the same regression passed after the repair.
- GitHub review workflow suite: 122 tests passed.
- `pnpm -r typecheck`: passed.
- `pnpm build`: passed.
- `pnpm test:run`: running; results will be recorded before handoff.
- Live local verification is running against a dedicated test App on
[the disposable test
PR](https://github.com/paperclipai/paperclip-permissions-smoke-20260926-pap57-fee7428e/pull/9).
A test-only preload delays one real GitHub PATCH response by eleven
seconds. It is not part of this change.

## Risks

GitHub bot calls can occupy their execution slot for up to sixty seconds
by default. Calls that exceed that limit can still time out. Durable
publication may continue after any caller timeout, so a timeout alone is
not proof of a failed write. This change does not add automatic task
reconciliation for arbitrarily slow or unavailable providers. No schema
or credential changes are required.

## Model Used

OpenAI Codex (GPT-6), with code inspection, editing, test execution, and
browser verification. The exact deployment model ID and context window
are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-14 06:52:33 -05:00
Devin FoleyandPaperclip 5847f047c7 fix(connections): retire requests for already available tools (#15856)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Connection cards let users provide the access an agent needs.
> - Users can complete setup in Apps while an earlier card stays
pending.
> - A stale card can then cancel recovery despite valid existing access.
> - This change verifies existing access and expires the obsolete
request.
> - It records a system outcome and sends one durable continuation.
> - Approval, identity, permission, and workspace recovery gates remain
in place.

## Linked Issues or Issue Description

**What happened?**

An ordinary tool connection request stayed pending after the addressed
user connected the service in Apps. A later conversation retry was
cancelled with `issue_waiting_for_response`.

**Expected behavior**

A request expires when the same user and requesting agent already have
usable access. The agent gets a durable tool refresh. The system records
no new human approval.

**Steps to reproduce**

1. Have an agent request a tool connection on an active task.
2. Connect the service through Apps with the addressed user's existing
grant, agent install, and allowed tools.
3. Let a conversation run stop and retry. The pending card still blocks
the retry on the base branch.

Related: #12345 introduced the connection request flow. This fix uses
its existing access checks and delivery queue.

## What Changed

- Verify company, task ownership, user identity, membership, health,
grants, installs, and task tool policy before expiring an ordinary
request.
- Preserve AI account, channel, additional permission, and upstream
provider requests on their existing paths.
- Save the system outcome, audit event, and continuation in one
transaction. Notify an idle delivery worker when a request commits.
- Recheck access and other questions or approvals before delivery,
retry, final dispatch, and native projection. Refresh system
observations for an authorized replacement account.
- Rearm delivery when access changes before provider work starts. Verify
the host's admission receipt against execution evidence before allowing
retry without a repair incident.
- Require the current delivery claim when marking an attempt complete.
An old completion cannot overwrite a rearmed continuation.
- Add embedded-Postgres regressions for permissions, policy,
concurrency, restart, replacement accounts, both dispatch gates, and the
delivery race. Keep genuine workspace repair holds blocking.

## Verification

- Latest-head connection, wake-context, dispatch, and legacy recovery
suites: 124 tests passed. The connection suite includes 44 real-Postgres
cases.
- Server typecheck passed. Full `pnpm -r typecheck` and `pnpm build`
passed on the latest head.
- `pnpm test:run` was run locally. Its general-server group passed
18,583 tests and failed 20. That long run started before the review
fixes, then loaded new test cases against modules transformed before
those fixes. The affected latest-head suites pass in a fresh process.
- One full-run failure also reproduces on unchanged base commit
de9ab8e7e0: native-session-resume's damaged-prior-epoch replacement
case. It is outside this change. Full-run groups after general-server
did not run because that command stopped on failure.
- Apex reviewed commit 96b96297f4 at 5/5. All review threads are
resolved. All latest-head CI gates passed. The branch has no merge
conflicts.

## Risks

A false readiness result could retire a request too early. The fix
reuses the runtime access selection rules, makes no new grants, and
rechecks access at delivery. Ordinary expired requests do not authorize
continuation. A failed workspace restore still requires repair through
the existing gate. No schema migration is needed.

## Model Used

OpenAI GPT-6 through Codex, with repository inspection, code editing,
shell execution, and test tools. The session identifies the model family
as GPT-6. It does not expose a more specific runtime model ID or context
window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (focused suites; full local
limitations are listed above)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-11 09:30:45 -07:00