Commit Graph
8241 Commits
Author SHA1 Message Date
11a0ff0205 feat(connections): per-company workload identity issuer with public signing keys (#22083)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents use outside services through connections. The control plane
holds each connection's credentials and makes the calls.
> - Some clouds, such as AWS, do not need stored keys. They trust a
signed token from an identity provider that they can verify against its
public keys.
> - Paperclip has no signing key whose public half it can publish. Agent
identity keys are Ed25519, which AWS does not accept, and the agent's
run receives the private key. Agent run tokens use a shared secret.
> - This pull request gives each company its own signing key and
publishes the public half as a standard OIDC issuer.
> - The benefit is that a connection, such as the planned AWS connector,
can get its own short-lived cloud credentials with no stored cloud keys.
The connection's role in the customer's account sets its permissions.

## Linked Issues or Issue Description

Refs #19414 and #10411. Those pull requests add OIDC *sign-in* to
Paperclip. This pull request is the other direction: Paperclip *issues*
tokens that outside clouds verify.

**Subsystem affected**
Connections (tool access) and the database.

**Problem or motivation**
An AWS connector should not store a long-lived AWS access key. AWS
supports OIDC federation (`AssumeRoleWithWebIdentity`). AWS fetches the
issuer's public keys over HTTPS and checks each token. Paperclip has no
key that it can publish for this, so this method is not available.

**Proposed solution**
Each company gets an ES256 key that Paperclip creates on first use. The
vault master key encrypts the private half. The issuer is
`https://{instance}/oidc/companies/{companyId}`. Paperclip publishes a
discovery document and a JWKS there. Tokens last 5 minutes and have `sub
= connection:{id}`. Only the control plane signs them. The board can see
key state and run planned rotation or emergency rotation. The design is
in `doc/plans/2026-11-17-workload-identity-issuer.md`. The AWS connector
plan that uses it is in
`doc/plans/2026-11-17-aws-connector-and-minted-credentials.md`.

**Alternatives considered**
- Reuse agent identity keys. Rejected: AWS does not accept Ed25519, and
the agent's run receives the private key.
- Derive the key from the master key, with no table. Rejected: the
master key alone would then sign as production. Worktree environments
copy the master key on purpose. A master key rotation would also change
the signing key and break every customer trust.
- Use one issuer for the instance. Rejected: if a customer's role checks
the issuer but not `sub`, any company on a shared instance could assume
that role.

## What Changed

- `packages/db`: new table `company_workload_identity_keys` and
migration `0498`. The migration only creates a new table, so the
previous release keeps working. Partial unique indexes allow one
`active` key and one `next` key per company.
- `server/src/services/workload-identity.ts`: lazy key creation with
concurrent convergence, a private-key cache with a fail-closed key match
check, the discovery document, the JWKS, `signConnectionToken`, and
rotation steps. A key that is no longer active stays published for 24
hours. Paperclip calculates this when it reads the keys, so no
background job is necessary.
- `server/src/routes/workload-identity.ts`: public `GET
/oidc/companies/{companyId}/.well-known/openid-configuration` and `GET
/oidc/companies/{companyId}/jwks.json` (404 until a key exists). Board
`GET /api/companies/{companyId}/workload-identity` and `POST
.../workload-identity/rotation`, with an activity log entry for each
rotation.
- Token issuance writes a tool-access audit event. The event never
contains the token.
- `cli`: worktree seeds exclude the new table in the minimal mode and
the full mode. This is the same as for `agent_identity_keys`, so a
development copy never holds production's signing key.
- Docs: `docs/api/workload-identity.md`, an entry in the API overview,
and the two plan docs.
- No connector uses `signConnectionToken` yet. The AWS connector pull
request will use it.

## Verification

- `cd server && npx vitest run src/__tests__/workload-identity.test.ts`:
8 tests pass. The tests cover these items:
  - The issuer requires https with no port and no loopback host.
  - The endpoints return 404 before a key exists.
  - Six concurrent first uses create one key.
  - The JWKS contains only `kty crv x y kid alg use`.
- A token verifies against the published key and has the expected
claims.
- A company cannot sign for another company's connection, and its keys
do not verify that company's tokens.
  - Signing stops when the public address is not usable or has changed.
  - Two-step rotation and emergency rotation work.
- Board-only access is enforced, and each rotation writes an activity
log entry.
- Red/green: I put back a JWK that leaks `d` and removed the company
check on the connection. Three tests failed. I restored the code, and
all 8 tests passed.
- `cd cli && npx vitest run src/__tests__/worktree.test.ts`: 63 tests
pass.
- `npx tsc --noEmit` passes for `packages/db`, `server` and `cli`.
- I started this branch on a real dev server. Before a key existed, the
public routes returned JSON 404 (not the UI page). After an emergency
rotation from the board route, the JWKS returned one public P-256 key.
On the http dev instance, discovery stayed 404 and the board route
reported `not_https`.
- `pnpm verify` exited 2 because of the migration. The required CI gates
supply the broad coverage.
- Not run: a real AWS `AssumeRoleWithWebIdentity` call. That needs a
public HTTPS instance. It is the first step of the AWS connector work
and will run against staging.

## Risks

- The migration is low risk. It creates a new table with no backfill,
and the compatibility checker classifies it as compatible with the
running release.
- The public routes are unauthenticated. They show only public keys,
return 404 for companies without a key, and use random company UUIDs.
Each request is one small indexed read.
- The issuer URL shape is permanent after customers trust it. A change
to the instance's public address breaks those trusts until the customer
updates the identity provider. `signConnectionToken` stops on that
mismatch when the caller supplies the recorded issuer.
- Anyone who holds the production database and the master key could sign
for that instance's companies. That is the same exposure as every other
vault secret.
- Edge routing on Paperclip Cloud must pass `/oidc/...` to the stack. I
will confirm this on staging with the AWS connector work.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, web fetch of AWS documentation, a local dev server).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Paperclip <noreply@paperclip.ing>
canary/v2026.1118.0-canary.824100
2026-11-17 23:33:47 -06:00
43b2d8be22 feat(dashboard): add an experimental areas and responsibilities dashboard (#22069)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The dashboard is the first page a person sees in a company
> - The classic dashboard shows agents, runs, costs and recent tasks
> - It does not show what the person is responsible for, or if that work
goes well
> - People think in areas, such as Fundraising or Customers, and in the
things they own in each area
> - This pull request adds a dashboard built on areas and
responsibilities, behind an experimental setting that is off by default
> - The benefit is that a person sees what they own, how it is going,
and what agents do for it, on one page

## Linked Issues or Issue Description

No public issue exists for this work. The description follows the
feature request template.

**Subsystem affected**

Dashboard, navigation, the task conversation, routines, and the instance
experimental settings.

**Problem or motivation**

The classic dashboard counts agents and tasks. It does not say what a
person owns or if it goes well. People who run a company with agents
want to see their areas, the responsibilities in each area, the bar each
one must meet, and the agents and routines that serve it.

**Proposed solution**

A person owns responsibilities. Each responsibility belongs to an area
and has a bar, a number to reach. Projects, tasks and routines link to a
responsibility, and the page shows its status from those links. An agent
can interview the person and draft the dashboard. The person accepts or
declines each change, and any change can be undone from a saved version.

**Alternatives considered**

Add the same data to the classic dashboard. That page is a fixed set of
cards for company activity, so a second, person-centred page is clearer.
The setting lets a person pick one of the two.

**Roadmap alignment**

This is an experimental feature behind `enableResponsibilities`. It is
off by default and changes nothing for a person who leaves it off.

## What Changed

- Add the setting "Areas & Responsibilities dashboard"
(`enableResponsibilities`, off by default). On, Dashboard in the sidebar
opens the new page, and the classic dashboard link is not shown in the
sidebars or the command palette. Off, nothing changes.
- Add tables and migrations 0492 to 0497 for areas, responsibilities,
links to work, metrics, status reports, saved dashboard versions, and
model-call records for summaries.
- Add the API: areas, responsibilities, links, bars, metrics, status
reports, summaries, runs, automation, and the dashboard conversation
with versions and undo. Each route checks company access and writes to
the activity log.
- Add four task conversations with an agent: about a bar, about an
automation, about a routine, and about the whole dashboard. The agent
proposes a change. Nothing applies until a person accepts it.
- Add standing checks: routines that an agent carries out on its
heartbeat and reports.
- Add the Dashboard page: search, counts, areas that open and close, a
primary New area button, and a message box at the bottom of the page.
Add the responsibility page with its bar, metrics, work and routines.
- Add the Paperclip skill sections and runner API entries that let an
agent use these routes. The runner skill inventory is regenerated (166
to 174 rows).
- Keep the new runner API tests in their own file so
`runner-api.test.ts` stays at 985 lines.
- Onboarding is not changed. The onboarding changes that build on this
work are a separate pull request.

## Verification

- Type checks pass for `packages/db`, `packages/shared`, `server` (all
layers) and `ui`.
- `pnpm --filter @paperclipai/db check:migrations` passes. A new
`drizzle-kit generate` run produces no further change.
- Tests for every changed file pass: 26 server test files (3,009 tests),
116 UI test files (1,991 tests) and 28 shared test files (432 tests).
- The runner capability inventory checks pass.
- I started the app on a local database that already held earlier data.
Master's new migrations and these migrations applied without error. With
the setting on, the Dashboard and a responsibility page loaded with no
console or server errors. The More menu and the command palette show one
Dashboard.
- I did not run `pnpm verify`, the full local suite, or the browser
suites. Required CI covers them.
- The agent conversations are tested with simulated agents. No real
model has run them.

## Risks

- Medium. The change is large, and the setting is off by default. With
it off, the new routes refuse requests and the UI is unchanged.
- Migrations only add tables and columns. Two replace a CHECK
constraint, and one builds an index `CONCURRENTLY`. These two have the
`online_compatible` override with a reason in the file, and the second
one is listed in `MIGRATIONS_WITH_TOP_LEVEL_STATEMENTS`.
- Master can add migrations before this merges. The migrations then need
new numbers.
- `responsibilities.ts` (1,663 lines), `responsibility-summaries.ts`
(1,239 lines) and one Storybook fixture file (1,305 lines) are new files
over 1,000 lines. They each keep one cohesive unit together: the service
for one table family, the model-call path for summaries, and the
fixtures that the stories share. No existing file crosses 1,000 lines.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Anthropic Claude: Claude Fable 5.2 (`claude-fable-5-2`) wrote most of
the change, with Claude Sonnet 5.5 (`claude-sonnet-5-5`) for the final
steps. Both used tool use, code execution and browser checks.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Scott Tong <scott@scottsmbpm5max.lan>
Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-17 20:45:44 -08:00
DottaandClaude Opus 5.5 4afcbfc0c7 fix(native): retry transient provider refusals while reattaching a handed-off run (#22118)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A release swap hands live native runs from the outgoing server to
the incoming server. The incoming server reattaches each run to its
original sandbox.
> - Reattachment first resumes the exact sandbox lease through the
sandbox provider plugin.
> - In a release-swap wave, one reattach resume got an HTTP 502 from the
provider gateway. The host did not retry it. The run failed with the
generic `setup_failed` code after 13 minutes of agent work.
> - The task then stayed blocked with a Retry banner for about 10
minutes, until the fresh-continuation fallback reached its bound and
started a new run.
> - This pull request retries a transient provider refusal during
reattachment, with a bounded budget. It also reads the provider status
from the plugin RPC error envelope, so the host sees the 502.
> - The benefit is that a brief gateway failure during a swap is
invisible to the user. The same run continues and no Retry banner shows.

## Linked Issues or Issue Description

**What happened?**
During a graceful release swap, the incoming server reattached a live
native run to its retained sandbox. The `environmentResumeLease` plugin
call failed with an HTML "502 Bad Gateway" body 280 ms after the
reattach started. The other 18 handed-off runs in the same swap
reattached in about 8 seconds. The host treated the 502 as an unknown
error, so the run failed with `setup_failed`. The task stayed `blocked`
with `canRetry: true` until the fresh-continuation fallback created a
new run 10 minutes later.

**Expected behavior**
A transient provider refusal (HTTP 408/425/429/5xx, a connection errno,
or a plugin RPC deadline) during reattachment is retried within a
bounded budget. The same run continues hands-off. A definite refusal
(for example HTTP 409) or a confirmed missing sandbox still fails at
once with its own named reason.

**Steps to reproduce**
1. Start a remote native run on a reusable sandbox.
2. Send SIGTERM to the server so it hands the run to the next server.
3. Make the first `environmentResumeLease` call on the incoming server
fail with HTTP 502.
4. See that the run fails with `setup_failed` and the task shows Retry.

**Paperclip version or commit**
`master` at ca34aca777 (release-swap wave) and ea154ce55e (this branch
base).

Root causes:
- Why setup failed: `environmentRunOrchestrator.acquireForRun` called
`resumeRunLease` once for a reattachment. Acquisition already retries
transient resume refusals, but reattachment did not. Also,
`transientSandboxResumeFailure` read `status` and `statusCode` only from
the thrown error. A plugin refusal arrives as `JsonRpcCallError`, which
carries the provider status only in the validated
`paperclip/environment-resume-error/v1` data envelope. So a plugin 502
was not transient even on the acquisition path.
- Why recovery took 10 minutes: `setup_failed` on a native run with
provider work is not a restart-class code, so the same-run reattach lane
does not take it. Failed-workspace preservation recorded a restore hold.
The fresh-continuation fallback waits for that hold up to
`FRESH_CONTINUATION_PENDING_BOUND_MS` (10 minutes) when it sees no
progress. The continuation started at 23:23:40, which is 10 minutes and
2 seconds after the run finished at 23:13:38.
- Why a Retry banner showed: no fresh-continuation check row existed yet
and no system recovery was active, so the presentation fell through to
`failed` with `canRetry: true`. The banner did not stay after the task
was done. The issue rests at `done` with `canRetry: false`. The grader
reported 10482 s because it closed the banner at its final read hours
later. Its last sample that saw the banner was 272 s after the first.

## What Changed

- `environment-run-orchestrator.ts`: the exact-lease resume during
reattachment goes through `retryTransientResumeCall`. The first call
keeps the provider's configured start deadline. A first call that used
up that deadline (plugin RPC TIMEOUT) is not repeated. After a transient
refusal, a 90 s budget (`REUSABLE_SANDBOX_RESUME_BUDGET_MS`) bounds each
later wait and passes the remaining budget as `timeoutMs`. A confirmed
missing sandbox and a definite refusal are not retried.
- `packages/plugins/sdk/src/environment-resume-error.ts`: owns one retry
policy for provider failures: `isRetryableEnvironmentHttpStatus` (408,
425, 429, 500, 502-504 and gateway/edge 520-527, 529, 598, 599; never
501 or 505) and `isRetryableEnvironmentErrorCode(code, scope)`.
ABORT_ERR is never retried. ENOTFOUND is retried only in the
`known_host` scope (an exact reattach to a host this run already
reached, the DNS-outage case). Lease acquisition keeps its canonical
semantics in the `first_acquisition` scope.
- `environment-provider-unavailable-error.ts`: lease acquisition uses
the SDK policy instead of its own sets.
- `reusable-sandbox-resume-replacement.ts`: owns the resume retry helper
and `transientSandboxResumeFailure(error, scope)`, which reads only
structured evidence (status, envelope errno, plugin RPC deadline,
sandbox state transition) and a few transport phrases. The acquisition
resume loop uses the same classifier and backoff helpers, and
`resumeFailureTimedOut` (label or envelope ETIMEDOUT) for its idle-start
extension.
- `environment-runtime.ts`: `EnvironmentDriverResumeInput.timeoutMs`
caps the sandbox plugin driver's resume RPC. Other drivers keep their
own deadline.
- Cloudflare bridge client: a bridge request deadline throws
`CloudflareBridgeTimeoutError` with code `ETIMEDOUT`. Daytona:
`SandboxLivenessTimeoutError` carries code `ETIMEDOUT`. Both reach the
host as a structured errno in the resume envelope.
- `tests/restart-chaos/staging-mix.ts`: `bannerShownMs` counts a visible
Retry banner at most 2 minutes past the last sample that saw it. Before,
it counted the whole unsampled gap until the final read.

Previous-release story: this changes no persisted state, launch
configuration, or pack identity. A run saved by the previous release
reattaches the same way. Only the in-process retry around the resume
call is new.

## Verification

- `pnpm verify` (selected Vitest files), and focused runs of the four
changed server test files and the two restart-chaos test files.
- Red/green: `server/src/__tests__/environment-run-orchestrator.test.ts`
- "retries a provider gateway refusal during reattachment and keeps the
original allocation" fails on master (1 call, rejects with the 502) and
passes with the fix.
- "treats a provider HTTP %i refusal during reattachment as transient"
(500, 502, 520, 599).
- "gives the first reattach resume the provider deadline and bounds
every retry by the budget".
- "does not repeat a reattach resume that used up the provider's start
deadline", "does not retry a definite provider refusal during
reattachment", and "preserves a retained sandbox startup failure without
acquiring a replacement" (default budget, one call).
- `server/src/services/reusable-sandbox-resume-retry.test.ts`: a table
over every classifier branch, and the helper's deadline rules.
- `server/src/__tests__/environment-runtime.test.ts`: "caps the exact
resume call at the caller's remaining budget".
- `server/src/services/environment-provider-unavailable-error.test.ts`:
scope table and the extended status lists. Orchestrator reattach cases:
ENOTFOUND retried, ABORT_ERR and 501 not.
-
`server/src/__tests__/environment-runtime-reusable-resume-replacement.test.ts`:
"applies the idle-start extension to a provider liveness deadline named
only by its errno" (red/green).
- Cloudflare `bridge-client.test.ts` and Daytona `plugin.test.ts`: the
deadline errors carry ETIMEDOUT in the resume envelope.
- Red/green integration:
`server/src/__tests__/heartbeat-native-remote-failure-retention.test.ts`
"keeps reattaching through a gateway refusal during a graceful hand-off
and continues the same run". The run's controller is lost, restart
recovery claims it to reattach, and the first resume fails with the 502
envelope. On master the run fails before the runner step (1 resume
call). With the fix the resume is retried, the same run reaches the
runner step at controller generation 2, no replacement allocation is
acquired, and no second run is created.
- `tests/restart-chaos/staging-mix.test.ts`: banner tests for a late
hide sample and for a banner still open at report time.

## Risks

- The acquisition resume loop no longer retries a refusal whose only
evidence is "timeout" or "timed out" text. Cloudflare and Daytona now
name their own deadlines ETIMEDOUT, and the plugin RPC deadline is
structured, so those stay retried. A provider that reports a timeout
only as text fails at once.
- Lease acquisition now also retries gateway statuses 520-527, 529, 598
and 599.
- A reattach resume can now take the provider deadline plus up to 90 s
when it gets transient refusals. Each retry is the same exact-lease
lifecycle resume. It cannot acquire or replace a sandbox, so a retry
cannot create a second executor.
- The aggregate recovery brake is unchanged. This change adds no new run
starts and no work before `turn.start`.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic. Agentic coding with
tool use (shell, file edits, test runs) in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
canary/v2026.1118.0-canary.823900
2026-11-17 22:11:04 -06:00
DottaandClaude Opus 5.5 6902c3f193 fix(native): keep recovery workers off a run its live heartbeat is finishing (#22101)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A native run ends in two steps: the live heartbeat exports the
workspace, then it commits the accepted result and records the run's
exit code, log summary and usage
> - Two recovery workers can also do these steps: the workspace
finalization worker and the result commit worker
> - The runner posts its accepted result before its process returns.
That post wakes both workers at once
> - The workspace finalization worker then takes the export lock while
the heartbeat still waits for the runner. The heartbeat gets "busy",
stops, and leaves the run to the workers
> - On staging, 45 of 135 recent succeeded native runs had no exit code
and no log size because of this
> - This pull request makes the workers leave a run alone while its live
heartbeat in the same process still runs
> - The benefit is that the heartbeat finishes its own run again, with
exit code, logs and usage. The workers still recover a run that the
heartbeat leaves unfinished

## Linked Issues or Issue Description

Refs #22043

**What happened?**
About a third of succeeded native runs had a null `exit_code` and a null
`log_bytes`. Their traces show a `task.workspace_finalization` span
(only the workspace finalization worker makes it) and no
`execution.native_finalize` or `completion.*` spans. The live heartbeat
lost the workspace export lock to the worker, threw
`NativeWorkspaceFinalizationBusyError` and stopped without completing
the run.

**Expected behavior**
The live heartbeat exports the workspace, commits the result, and
records the exit code, log summary and usage. The workers act only when
the heartbeat cannot finish the run.

**Steps to reproduce**
1. Start a native run that syncs its workspace.
2. Let the runner post its accepted result before its process returns.
The result post wakes the workspace finalization worker at once.
3. The worker exports the workspace first. The heartbeat gets `acquired:
false` and stops. The result commit worker then commits the run without
the heartbeat's completion data.

**Relevant logs or output**
Staging run `cb1cfc23-79eb-4528-bcdc-dcf639430d31`: the worker restored
the workspace in about 51s. The worker then committed the status
decision and projected the run as succeeded. The heartbeat wrote no
completion data.

## What Changed

- Add `server/src/services/native-runtime/live-native-finalization.ts`.
It keeps an in-process reservation for each run whose live heartbeat is
still running.
- The heartbeat (`runtime-execution.ts`) reserves a native run before it
dispatches. It releases the reservation when its execution ends. The
release wakes both worker queues for that run.
- `withNativeWorkspaceFinalizationOwnership` returns `owner_active` to
every caller except the run's own heartbeat while the run is reserved.
The workspace finalization worker already waits for a release
notification on `owner_active`.
- The result commit worker skips a reserved run. The release
notification wakes it again.

## Verification

- New regression tests, red before the fix and green after:
- `native-workspace-finalization-work.test.ts`: "leaves a result posted
mid-run to the live heartbeat, then resumes after its release". Without
the fix the worker exported the workspace while the heartbeat held the
run.
- `native-result-commit-work.test.ts`: "leaves a live heartbeat's run to
it and commits only what the heartbeat leaves behind". Without the fix
the worker committed the run while the heartbeat held it.
- Both tests also prove that the workers still finish the run after the
heartbeat releases it without finishing.
- `pnpm verify` (affected tests) passes. Server typecheck passes.
- After deploy: measure the share of succeeded native runs on staging
with a null `exit_code`. It should go down toward 0%.

## Risks

- The reservation is in memory. A worker in another server process (for
example during a deploy overlap) can still race the heartbeat. That case
behaves as before this change.
- If the heartbeat stops in an unusual way, its `finally` block releases
the reservation and wakes the workers. If the process crashes, the
reservation goes away, and startup discovery recovers the run as before.
- While a heartbeat is still running after the runner posted its result,
the workers wait. Before, they could start the export at once. The
heartbeat does the same work, so the result reaches the user at about
the same time.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and
code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
canary/v2026.1118.0-canary.823800
2026-11-17 21:21:32 -06:00
Devin FoleyandPaperclip ea154ce55e Add bulk keyed issue document reads for native helpers (#22105)
## Thinking Path

> - Paperclip manages work for AI agents.
> - Native helpers read task documents through authorized API tools.
> - A scan of 146 cases needs one saved response and one download per
case.
> - The document service can read the same key across authorized tasks
with bounded queries.
> - This pull request adds a bounded bulk read with the existing task
access policy.
> - Helpers can transfer one batch and check current revisions before
fetching changed bodies.

## Linked Issues or Issue Description

**Subsystem affected**

Task document REST API, shared query contracts, and native API
discovery.

**Problem or motivation**

Per-task document reads make native sync scans cost hundreds of tool
calls. Task list timestamps do not prove that document content is
unchanged.

**Proposed solution**

Add `GET
/api/companies/{companyId}/issue-documents?key=notes&issueIds=...`. Read
up to 200 task UUIDs under current company and task permissions. Return
current bodies and revision metadata, or metadata alone with
`includeBody=false`.

**Alternatives considered**

Generic request batching would add tool execution and partial failure
rules. One bulk resource read removes the per-case relay cost with the
existing tools. Do not skip documents based only on task-row timestamps.
Keep managed skill files immutable; cleanup of copied skill test
fixtures belongs to those tests.

**Roadmap alignment**

This improves existing task documents and native API access. It adds no
new workflow or UI. The roadmap has no duplicate bulk document read
work. No duplicate open pull request was found.

## What Changed

- Add a bulk keyed document route and a company-scoped document reader.
Reuse the current batched task-read policy and completed-task read
receipts.
- Preserve request order and deduplicate IDs. Return `document: null`
for readable tasks without the key. Omit unreadable, hidden, foreign and
missing tasks without revealing which exist.
- Bound requests to 200 IDs and responses to 8 MiB of JSON bytes. Check
encoded body sizes in SQL before fetching bodies in the same read
snapshot. Build JSON within the byte budget. Oversized responses fail
with 413 and are never saved as partial artifacts.
- Admit at most two active reads per actor and eight per server
instance. Extra requests receive 429 with Retry-After. Keep the slot
until both the database work and response settle.
- Batch completed-handoff lookups and group child receipts by parent.
Extend the existing event batch writer for ordinary non-waking receipts
with atomic sequence allocation. Stored receipt shapes stay unchanged
for existing consumers.
- Expose revision-only reads without selecting document bodies. Document
native discovery, one save plus one download, and complete-inventory
checks.
- Add real HTTP, database and artifact tests for 146 open and completed
cases, 200 maximum-size bodies, metadata overhead, access failures,
query limits and JSON expansion. Cover admission limits, failure
cleanup, and concurrent event writers. Refresh the shared SDK source
pins.

## Verification

Final revision: `93dc8b029600211d79a050650f3dff82e5dad52f`.

- [CI run 44212220917, attempt
3](https://github.com/paperclipai/paperclip/actions/runs/44212220917)
passed. This includes typecheck, build, general tests, Runner checks,
serialized server suites, all e2e shards, and the final verify gate.
-
[Build](https://github.com/paperclipai/paperclip/actions/runs/44212220917/job/134506305267),
[e2e](https://github.com/paperclipai/paperclip/actions/runs/44212220917/job/134506309117),
and
[verify](https://github.com/paperclipai/paperclip/actions/runs/44212220917/job/134510062263)
are green.
- [Linux CI job
134506309291](https://github.com/paperclipai/paperclip/actions/runs/44212220917/job/134506309291)
passed all 16 bulk document integration tests. This includes the native
transfer of all 146 documents with the saved size and hash,
completed-case receipt batching, and size-only rejection of 200
maximum-size documents.
- [Greptile
review](https://github.com/paperclipai/paperclip/pull/22105#issuecomment-6965449032)
is 5/5 on the final revision. All review threads are resolved. Security
checks pass. GitHub reports CLEAN and MERGEABLE.
- Local server typecheck passed on macOS with Node 25.6.1. The focused
run below passed 177 tests; the Linux-only transfer test was skipped
locally and then passed in CI.

```sh
pnpm --filter @paperclipai/server typecheck
cd server
pnpm exec vitest run \
  src/routes/issue-document-batch.test.ts \
  src/services/native-runtime/issue-document-batch.integration.test.ts \
  src/__tests__/documents-service.test.ts \
  src/__tests__/openapi-routes.test.ts \
  src/__tests__/heartbeat-run-event-atomic-append.test.ts \
  src/__tests__/chat-completion-delivery.test.ts \
  src/__tests__/native-child-task-read-routes.test.ts
```

- Shared typecheck and build passed. From `packages/shared/`, `pnpm exec
vitest run src/validators/issue.test.ts` passed 61 tests before the
rebase; the batch query contract did not change in the rebase.
- `node tests/runner-e2e/source-sdk-public-inputs.mjs <HEAD> --check`
passed. Runtime and lockfile pins are unchanged.
- `pnpm verify --explain` returned exit 2 because the API docs and
shared contracts require broader CI. The successful CI run above
supplies that coverage.

Failure and recovery record:

- Both performance findings reproduced before the fixes. Scale tests now
verify a size-only query for 200 large documents and one handoff lookup
plus two receipt-write statements for 146 completed cases. Concurrent
writers retain contiguous unique sequences and decoded payloads.
- Initial local runs found missing shared exports and OpenAPI route
coverage. These are fixed. Fixtures now respect Linux-only installation,
thrown cross-company denials, and the ES2022 TypeScript target. Rebase
conflicts preserved the onboarding coverage from master.
- CI attempt 1 interrupted Build, e2e shard 7 and six general-test
shards with runner shutdown signals. Attempt 2 recovered the tests but
its [Build
job](https://github.com/paperclipai/paperclip/actions/runs/44212220917/job/134499680425)
also received a shutdown signal and exited 143. No compiler or test
assertion failure was found in those interrupted job logs. The reason
the runners stopped remains unknown. Attempt 3 recovered Build and its
dependent verify gate on the same revision. Earlier interruptions are
not counted as passes.

## Risks

- Additive API only. No migration or existing endpoint response change.
- Helpers must adopt the batch response to gain the tool-call reduction.
They must verify that every requested task is present before treating
the scan as complete. Unreadable tasks are intentionally omitted.
- Split large document batches when the route returns 413. Metadata
remains available without bodies. Concurrency limits are per server
instance and can refuse simultaneous reads with 429; callers retry after
an active read finishes.
- This does not change company-owned helper inventory expiry or cleanup
tests, installed skill permissions, or generic tool batching. It does
not deploy to a tenant.

## Model Used

OpenAI `gpt-6.1-sol` through Codex in T3 Code, with xhigh reasoning,
tool use, and code execution. Context window size was not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
canary/v2026.1118.0-canary.823700
2026-11-17 17:42:18 -08:00
Devin FoleyandPaperclip bf55063567 fix(server): keep chat titles current and space verification retries (#22102)
## Thinking Path

> - Paperclip manages AI agents and their work.
> - Agent chats use persistent tasks, and remote runs verify their
provider files before execution.
> - A chat keeps the name that its agent had when the chat was created.
> - A fast gateway failure can consume both verification attempts before
the gateway recovers.
> - This change updates generated chat titles on an agent rename and
waits before a verification retry.
> - Chats show the current name, and a short gateway fault has time to
clear.

## Linked Issues or Issue Description

**What happened?**

Renaming an agent leaves existing generated chat titles unchanged.
Remote provider verification also retries a transport failure
immediately. Two fast HTTP 502 responses can fail startup before the
model starts.

**Expected behavior**

Generated chat titles follow an agent rename. A read-only verification
retry gives the remote gateway time to recover. Verification still
rejects incompatible files and stops after two attempts.

**Steps to reproduce**

1. Open a chat with an agent, then rename that agent. Inspect the task
linked from a run: its title still has the old agent name.
2. Make the remote verification transport return HTTP 502 for one
second. Start verification. Both immediate attempts fail before the
transport recovers.

Related: #22087 covers a provider connection failure during lease
acquisition. This change covers a later provider-file verification
request.

## What Changed

- Update matching generated chat titles in the agent configuration
transaction. Preserve custom chat titles, ordinary task titles, and
other agents' chats.
- Record each changed chat as `issue.updated` and publish its event
after commit so open pages refresh. Roll back the rename and queued
events if audit persistence fails.
- Derive new chat titles from the current agent name under the existing
company transaction lock. Both user-opened and proactive chats use this
path, including a create that read the old name before a rename
committed.
- Acquire that company lock before proactive chat delivery takes chat or
agent row locks, so delivery and renames use a consistent lock order.
- Wait one second before the second remote verification attempt. Keep
the existing attempt limit, transport timeouts, artifact checks, and
failure codes.
- Add route and startup-verification regressions. Document the retry
delay.
- Wait for the connection-review E2E fixture to save its Ask first
permission before it creates a task or navigates away.

## Verification

- Final acceptance on `3fb74ffaf6fa0ee2ecbe5dba3d01d252cb0c986f`: [full
CI
run](https://github.com/paperclipai/paperclip/actions/runs/44208144377)
is green, all 61 reported checks are complete with no failures, Greptile
is 5/5 on this exact commit, all review threads are resolved, and GitHub
reports the branch clean and mergeable.
- Red/green: the rename test retained the old titles before the fix. The
gateway recovery test consumed both attempts before the one-second
recovery point. Both pass with the changes.
- The live-update and concurrent-create tests failed on the first PR
revision: no title events were published, and a paused chat creation
retained the old name. The fixes queue events for publication after
commit and re-read the agent name under the company lock. Both
reproductions now pass.
- `pnpm exec vitest run
server/src/__tests__/agent-configuration-save.test.ts
server/src/__tests__/agent-chat-messages.test.ts
server/src/__tests__/agent-conversations.test.ts
server/src/services/native-runtime/remote-provider-pack.test.ts
server/src/services/native-runtime/remote-provider-pack-verification.test.ts`:
191 passed on macOS, including the concurrent rename/message regression.
- `pnpm --filter @paperclipai/server typecheck`: passed, including
source and test layers.
- Earlier CI on `de2a0a4` failed in the release-registry watchdog
self-test ([job
log](https://github.com/paperclipai/paperclip/actions/runs/44202511888/job/134460224014)).
Its 1.5-second watchdog fired during `starting the file's tests`; the
assertion expected `test "waits forever"`. Both reporter files are
unchanged from the base and current head. `node --test
scripts/ci/stalled-file-reporter.test.mjs` passed all 7 tests on macOS
with Node v25.6.1. The current revision also passed the Linux Typecheck
+ Release Registry CI check; no reporter code was changed.
- CI on `c83fcd9` failed the connection-review failure journey ([shard
log](https://github.com/paperclipai/paperclip/actions/runs/44204043415/job/134465938135)).
Navigation aborted the pending permission save, so the fixture tool ran
without creating the expected approval card. A temporary two-second
delay on that save reproduced the failure locally. With the response
wait, the same delayed-save reproduction passed. The temporary delay is
removed from the diff.
- `node scripts/typecheck-test-projects.mjs --reuse-declarations
tests/e2e`: passed using the server declarations from the same
application source revision.
- Before the rebase, the complete connection-review spec passed 4
journeys and skipped the controlled-restart journey under the default
configuration. On `95fb2aa`, rebased onto `9032f55d83e`, the
provider-failure journey, all 190 focused tests, server typecheck, and
E2E test-project typecheck passed again. The dedicated
controlled-restart suite was not run.
- A concurrent rename/message regression reproduced PostgreSQL `40P01:
deadlock detected` on `95fb2aa`. Delivery now takes the shared company
lock before the chat and agent locks. The regression uses observed
PostgreSQL blockers to stage the collision, and both operations commit
with the fix. Concurrent creation also waits behind the same lock and
retains the renamed title.
- The application tree for `3fb74ff` passed all 191 focused tests and
server typecheck. The subsequent rebase onto `96907f5518c` changed only
`.github/CODEOWNERS` compared with that tested tree. The full CI run for
this revision is now green.
- CI jobs on `95fb2aa` were interrupted after their runners received
shutdown signals: [server
shard](https://github.com/paperclipai/paperclip/actions/runs/44206614135/job/134474311777),
[workspace
shard](https://github.com/paperclipai/paperclip/actions/runs/44206614135/job/134474311921),
[typecheck/release
registry](https://github.com/paperclipai/paperclip/actions/runs/44206614135/job/134474310521),
and [serialized server
shard](https://github.com/paperclipai/paperclip/actions/runs/44206614135/job/134474311690).
The typecheck job is labelled failed, but its active step was cancelled.
These interruptions did not supply passing test evidence. The fresh CI
run on `3fb74ff` completed without failures.
- `pnpm verify --explain`: exit 2. The changed recovery document
requires broad CI coverage. Required CI supplies the full test and build
coverage; no full local suite or build was run.
- `git diff --check` and a Gitleaks scan of the diff passed. The diff
was also checked for personal data and private identifiers.

## Risks

- An inconclusive verification takes one extra second before its last
attempt. A persistent outage still fails with the existing named error.
No provider turn or artifact mutation is replayed by this retry.
- Only chat titles that match the previous generated title change.
Earlier stale titles and custom titles need an explicit correction.
- Proactive chat delivery now shares the existing company writer lock
with agent configuration. Concurrent writes in that company wait for the
active transaction to finish.
- No schema, session, checkpoint, launch identity, or pack identity
changes. State saved by the previous release carries forward with the
same verification checks and attempt limit.
- Revert this commit to restore the previous behavior. Existing title
updates are metadata and need no schema rollback.

## Model Used

- OpenAI Codex, `gpt-6.1-sol`, xhigh reasoning, with source inspection,
browser investigation, and local code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
canary/v2026.1118.0-canary.823600
2026-11-17 16:59:05 -08:00
ce27dde855 Start onboarding in chat and defer execution until the user chooses work (#22077)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Onboarding connects an agent and gives the user a place to discuss
work.
> - The previous flow created an execution task and project before the
user chose a scope.
> - A standing chat can gather the goal and clear exit criteria first.
> - This pull request opens chat after onboarding and starts a live
welcome after the chat is visible.
> - The agent proposes work for acceptance, or creates a task when the
user explicitly requests one.
> - Subscription setup and shared question state must work for this
first conversation to succeed.

## Linked Issues or Issue Description

**Subsystem affected**
Company onboarding, agent chat, local subscription setup, task
numbering, and shared interaction views.

**Problem or motivation**
The first session creates work before the user has agreed to a goal.
Standing chats consume task numbers. An answered chat question can
remain pending in the inbox. Claude code entry requires an extra click,
and switching from Claude to Codex can leave an incompatible model
selected.

**Proposed solution**
Open an owner-bound chat for every new company. Start a live welcome
after the chat paints in a visible tab. Gather scope in conversation and
use the existing task acceptance flow. Keep chat records out of the task
number sequence. Publish question responses to shared caches and cancel
stale reads. Submit pasted Claude codes once and wait for verified
sign-in before connecting.

**Alternatives considered**
A fixed welcome would not use the onboarding context. A hidden execution
task would still create work and consume a task number. Keep the
existing task proposal and acceptance contract instead.

**Roadmap alignment**
This improves the shipped Agent Chat and Personal & Shared AI Accounts
flows in ROADMAP.md. It does not add a separate execution or approval
system.

Related prior work: Refs #19643 and #19512. Related route handoff work:
Refs #12557. This change replaces the automatic first task with chat; it
also repairs the subscription and shared question state paths used
during that journey.

## What Changed

- Finish onboarding in agent chat for default and team-template
companies. Create no execution task or project.
- Queue one live welcome after visible chat content paints. Pass company
goals and the initial idea as context. Preserve access, budget, and
hiring controls.
- Instruct the agent to gather a goal and exit criteria, offer an
interview, and propose tasks for acceptance. Allow direct creation on an
explicit user request.
- Leave new standing chats unnumbered. The first execution task receives
number 1. Preserve existing identifiers.
- Synchronize answered question cards across chat and inbox, including
concurrent requests and inactive cached views.
- Submit a pasted Claude authorization code once. Show connection
progress and hide the restart link while sign-in is still preparing.
Detect rejected OAuth codes promptly. Clear incompatible models when the
provider changes.
- Use `Work with [agent name]` as the default chat composer placeholder.
- Add regression coverage and update onboarding documentation and API
notes.

## Verification

- Completed a real Claude subscription journey on macOS with an isolated
local database. Authorized the account, pasted the real code, and
reached the review step without clicking Submit code or Connect. Get
started opened chat and the live welcome run succeeded. The company had
zero execution tasks, zero projects, and task counter 0.
- On rebased commit c664949858, 6 browser cases passed for
paste-to-connect-to-chat, task-free onboarding, existing-company
routing, the native connection probe gate, desktop/mobile planning, and
two-tab chat/inbox question synchronization.
- On the rebased head, 419 focused UI tests passed. The focused server
and adapter run passed 160 tests and hit one `socket hang up` in the
existing authenticated chat route test. The full 72-test conversation
suite passed on one repeat. The cause of that first socket failure
remains unconfirmed; the passing repeat does not establish its cause.
- UI typecheck and server declaration typecheck pass after the rebase.
The first server check found a missing local dependency from master; a
frozen-lockfile install repaired it. No dependency manifest or lockfile
change was needed.
- The first Greptile review found two welcome edge cases. Four new tests
reproduced both before the fix. On commit b380f14ef9, all 114 onboarding
and queue tests pass, including timeout retries and queued welcomes
replaced by a user message or chat reset. Server source and focused test
type checks pass on this revision.
- [Greptile
review](https://github.com/paperclipai/paperclip/pull/22077#issuecomment-6961533641)
gives rebased commit 2dbad34445 a score of
5/5. The current-head check passed. Both welcome review threads are
resolved, with no new actionable findings.
- CI serialized shard 8/9 found a missing OpenAPI inventory entry for
the new route file. The exact coverage test reproduced this locally.
Commit 17a72ec8b7 adds explicit mounted paths and documents both
board-only operations. All 65 OpenAPI and onboarding route tests and
server source typecheck pass after the fix. The final CI run passes on
this revision.
- Rebased onto master 23ce68a8dc and regenerated the SDK source registry
from committed inputs. Worktree preflight, SDK source validation, and
all 65 API/onboarding tests pass on final head
2dbad34445.
- On final head 2dbad34445, all 419
affected UI tests pass. UI package typecheck and server declaration
typecheck also pass. Commands: `pnpm --filter @paperclipai/ui
typecheck`; `node server/scripts/typecheck-layers.mjs
--declarations-only`; focused Vitest suites for the wizard, subscription
login, agent chat, shared interaction cache, and OpenAPI/onboarding
routes.
- [Final-head CI
run](https://github.com/paperclipai/paperclip/actions/runs/44208652564)
completed successfully on commit
2dbad34445. All required typecheck, build,
test, Runner, browser, serialized-server, and canary gates pass,
including the [final verify
job](https://github.com/paperclipai/paperclip/actions/runs/44208652564/job/134487749343).
The fresh routing gate selected the trusted AWS fleet after the
protected owner policy in #22098 merged. No source rebase was needed:
the gate reads that policy from protected master.
- The repaired OpenAPI shard passed in the earlier same-head CI run.
General server shard 8/12 hit a 20-second step timeout in the existing
`issue-recovery-live-run-agent-census` suite. This PR does not change
that suite or its fixture. All four tests pass locally on the exact
current commit in 84 seconds. The cause of the CI timeout remains
unconfirmed. Every other test job passed. A single failed-shard retry
was accepted but remained queued on its saved GitHub-hosted selection.
It was cancelled before test execution after #22098 merged. The fresh
full-CI run passed on the unchanged head under the approved AWS policy,
including [server shard
8/12](https://github.com/paperclipai/paperclip/actions/runs/44208652564/job/134481388730).
This passing execution does not establish the cause of the earlier
timeout.
- Worktree preflight, committed SDK source input validation, and diff
whitespace checks pass after the rebase.
- `pnpm verify` reports exit 2 because the new subsystem documentation
requires broader CI coverage. The final successful CI run supplies that
broader coverage. This is not a full local suite pass.

## Risks

- The initial task is now an explicit user decision. Existing onboarding
tasks and numbered records remain readable. No schema migration is
required.
- A welcome still depends on a working provider connection and the
normal run queue. Failed starts can be retried without duplicate
successful welcomes.
- Claude paste acceptance was verified with a real subscription on
macOS. This change does not claim a new live Codex or Windows acceptance
test.

## Model Used

OpenAI Codex, based on GPT-6, with tool-assisted reasoning, code
execution, terminal checks, and browser verification. The client does
not expose the exact routed build or context limit.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Scott Tong <scott@scottsmbpm5max.lan>
Co-authored-by: Paperclip <noreply@paperclip.ing>
canary/v2026.1118.0-canary.823500
2026-11-17 16:52:35 -08:00
Devin FoleyandPaperclip 96907f5518 ci: include team maintainers in trusted runner policy (#22098)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Pull requests must pass CI before merge.
> - The protected CI gate requires each author and triggering user to be
an AWS-allowlisted owner of `.github/**`.
> - Four team maintainers are missing from that owner rule. Their PRs
use the GitHub-hosted queue.
> - This pull request adds those four accounts to the existing CI owner
rule.
> - Their eligible PRs can use the AWS test fleet after this change
reaches protected `master`.

## Linked Issues or Issue Description

Related: #22077 showed this routing result. #21192 introduced the
current CI CODEOWNER check. No duplicate roster-change PR was found.

**What existing behavior does this improve?**
Runner selection for PRs from team maintainers.

**Current behavior**
`tonio-alucema`, `scotttong`, `lucktastic`, and `nguyenm7` are not
owners of `.github/**`. The gate sends their test jobs to GitHub-hosted
runners. Repository admin access alone does not satisfy the gate.

**Proposed behavior**
Add all four users to the existing `.github/**` CODEOWNERS rule. Keep
all existing owners and the current identity, event, and commit checks.

**Reason and benefit**
These maintainers can use the trusted AWS fleet without a separate
routing exception. This avoids the hosted queue for eligible runs.

## What Changed

- Add `@tonio-alucema`, `@scotttong`, `@lucktastic`, and `@nguyenm7` to
the `.github/**` owner rule.
- Prepare the separate repository settings for this policy: add
lucktastic's verified user ID, `13757243`, to `AWS_CI_TRUSTED_USER_IDS`.
Synchronize `AWS_CI_ROUTING_USER_IDS_SOURCE` and
`AWS_CI_ROUTING_USER_IDS`. The other three users were already listed.
This settings change is complete; the protected master owner check still
controls test execution.

## Verification

- Final tested head: `0fc7873beefe73fe53ffbf1363862c865fc77eef`. [CI
attempt 3 completed
successfully](https://github.com/paperclipai/paperclip/actions/runs/44199904434/attempts/3).
All applicable jobs pass, including typecheck, build, all test matrices,
[verify](https://github.com/paperclipai/paperclip/actions/runs/44199904434/job/134472019170),
and
[E2E](https://github.com/paperclipai/paperclip/actions/runs/44199904434/job/134468650593).
Failed-job retries retain earlier passing jobs; this is combined
coverage on the unchanged head, not a new full-suite execution.
- [Greptile: 5/5 with no actionable
issues](https://github.com/paperclipai/paperclip/pull/22098#issuecomment-6964150026).
No unresolved review threads. No merge conflicts with `master`.
- Passed `node --test .github/scripts/tests/ci-routing.test.mjs
.github/scripts/tests/ci-profile.test.mjs`: 163 tests, zero failures.
These cover accepted and rejected identities, non-owner authors and
rerun actors, malformed settings, and trusted-source validation.
- Resolved the proposed CODEOWNERS file through the existing
`trustedOwnerIds` resolver and live GitHub user records. All four
requested users resolve to distinct human accounts with repository admin
access. Existing owners remain present. All four users are in the
numeric AWS allowlist, and both admission variables match it.
- Passed `git diff --check`. Reviewed the full one-line diff for
credentials, private URLs, personal data, and run artifacts; none are
present. No new test implementation was needed for this roster-only
edit. CI supplies the broad application coverage; no full local
application suite was run.
- GitHub requires approval from an existing CI code owner before merge.
The automatic review requests are in place.

<details>
<summary>Earlier CI failures and retry evidence</summary>

Attempt 1 was incomplete. The [Build
runner](https://github.com/paperclipai/paperclip/actions/runs/44199904434/job/134451301493)
lost communication with GitHub; its annotation confirms runner loss, and
its log was unavailable (HTTP 404). [E2E shard
8/8](https://github.com/paperclipai/paperclip/actions/runs/44199904434/job/134451302068),
[server shard
5/12](https://github.com/paperclipai/paperclip/actions/runs/44199904434/job/134451302631),
and [server shard
9/12](https://github.com/paperclipai/paperclip/actions/runs/44199904434/job/134451302800)
each logged a runner shutdown signal.

[E2E shard
7/8](https://github.com/paperclipai/paperclip/actions/runs/44199904434/job/134451302023)
separately failed at `tests/e2e/plan-task-fixture.spec.ts:30`. A
test-company deletion returned HTTP 500 because an activity-log row
still referenced the company
(`activity_log_company_id_companies_id_fk`). The fixture and application
code match the base revision. This test failure prevented the automatic
runner-loss retry. A manual failed-job retry used the unchanged head.

Attempt 2's [Build
retry](https://github.com/paperclipai/paperclip/actions/runs/44199904434/job/134458626180)
passed. [E2E shard
7/8](https://github.com/paperclipai/paperclip/actions/runs/44199904434/job/134458626964)
passed all 32 active tests, including both planning fixtures. The
cleanup error did not repeat without a source change. This establishes
recovery; its original cause remains unresolved. [Server shard
9/12](https://github.com/paperclipai/paperclip/actions/runs/44199904434/job/134458630607)
also passed. [E2E shard
8/8](https://github.com/paperclipai/paperclip/actions/runs/44199904434/job/134458627148)
and [server shard
5/12](https://github.com/paperclipai/paperclip/actions/runs/44199904434/job/134458630550)
each logged another runner shutdown. The E2E and verify aggregates
rejected the interrupted dependencies.

A second manual failed-job retry completed the missing coverage. Attempt
3's [E2E shard
8/8](https://github.com/paperclipai/paperclip/actions/runs/44199904434/job/134465389776)
and [server shard
5/12](https://github.com/paperclipai/paperclip/actions/runs/44199904434/job/134465390839)
passed, and both required aggregate checks passed. No source changes or
check bypasses were used during recovery.

</details>

## Risks

- These users become code-owner reviewers for `.github/**`. They also
become eligible for the CI profiling paths that use this owner rule.
- Runner authorization still requires the numeric allowlist and the
protected master owner rule for every checked identity. The change takes
effect for newly evaluated gates after merge. PR edits cannot grant
themselves access.
- Existing queued jobs retain their runner selection. Start a new PR run
or rerun the routing gate after merge to evaluate the new roster.

## Model Used

OpenAI `gpt-6.1-sol` through the Codex harness, with xhigh reasoning
effort, repository tools, and code execution. Context window size is not
exposed by this runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-17 16:24:40 -08:00
Devin FoleyandPaperclip 9032f55d83 fix(ssh): support all Cloud SSH harnesses in remote task workspaces (#22078)
Support durable in-place SSH task workspaces across all declared SSH-capable
Cloud harnesses. Keep controller filesystem operations off remote paths,
preserve SSH session identity, and stage provider assets on their owning
execution target. Keep managed ACP conversation identity stable across
credential-home rotation and preserve Pi JSONL history after credential cleanup.

Verify with the real SSH heartbeat matrix, adapter and session regression
checks, and typechecking. Final CI passed on 3b629da with 57 successful checks
and three skips. Greptile scored the change 5/5 with no unresolved comments.
Live paid-provider acceptance remains unqualified.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-17 16:06:25 -08:00
Devin FoleyandPaperclip a3f86e5cc8 fix(recovery): continue a recovered preparation that failed with a transient code (#22096)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Before a remote run starts its provider, Paperclip prepares the
sandbox: it restores the saved workspace and stages the agent files
> - When a legacy Runner preparation fails after the sandbox exists,
Paperclip keeps the sandbox, saves its files, and holds the task with
"preserve without replay"
> - A preparation that a restart or a lost process stopped continues by
itself after the files are saved. A preparation that failed with a
transient error does not: it waits for a person to click Retry, even
though its own bounded retry would continue it
> - The agent-file Git exclusion timeout also did not count as
transient. It failed with the generic `adapter_failed` code, so nothing
retried it, and the wake that started the run was lost
> - This pull request continues a recovered preparation that failed with
a transient preparation code, in the same bounded lane as a lost
preparation, and names the exclusion timeout as a transient preparation
timeout
> - The benefit is that a slow sandbox no longer stops a task until a
person notices it

## Linked Issues or Issue Description

Refs #22039, which retries the timed-out Git exclusion in place. This
change covers the case where all three attempts time out.

**What happened?**
A run failed in the preparing stage with "Could not exclude agent files
from task Git staging". No provider work started. The sandbox was kept,
and workspace recovery later saved its files with no changes. The
recovery hold recorded `replay: blocked`, so no new run started and the
wake was lost.

**Steps to reproduce**
1. Start a legacy Runner run on a sandbox that keeps its lease on
failure.
2. Make the agent-file Git exclusion time out on every attempt.
3. Let workspace recovery save the retained sandbox.

**Expected behavior**
After the files are saved, the run continues once in its bounded retry
lane, as a preparation stopped by a restart does.

**Actual behavior**
The run fails as `adapter_failed`. After recovery, the task waits for a
manual Retry.

## What Changed

- `failed-preparation-retry.ts` (new):
`isTransientWorkspacePreparationCode` and
`failedPreparationOwnsBoundedRetry` move here unchanged from
`heartbeat/retries.ts`, which re-exports them. The code list is now an
array, so SQL can use it. The move lets `cancelled-native-startup.ts`
use the predicate without an import cycle, because `retries.ts` already
imports that file.
- `cancelled-native-startup.ts`:
`isInterruptedNativePreparationCandidate` also accepts a failed
preparation that has a required workspace recovery and whose own bounded
retry owns it. All other conditions stay the same: legacy Runner, still
preparing, no provider evidence, released leases with stop receipts, and
a verified workspace repair before replay.
- `workspace-recovery.ts`: the recovery sweep lists the same runs, so
their continuation is delivered after the files are saved.
- `run-completion.ts`: a `RemoteWorkspacePreparationTimeoutError` that a
legacy run raises before it enters the adapter now records
`remote_workspace_prepare_timeout`, the same code the setup path already
records.
- `agent-instruction-files.ts`: after the last timed-out attempt, the
exclusion throws `RemoteWorkspacePreparationTimeoutError`. The message
stays "... (reason=timed_out)". `RemoteWorkspacePreparationTimeoutError`
takes an optional message for this.
- Tests: a recovered preparation that failed with
`remote_workspace_prepare_timeout` is listed by the sweep and continues
once; one that failed with `adapter_failed` still waits for Retry.
`failRun` names the timeout only when the adapter did not start. The
agent-file test checks the error type.

## Verification

- `npx vitest run` on the new and changed cases: the
retained-preparation continuation, the `failRun` classification, and the
agent-file suites. All pass.
- Each new case fails without its source change. With only the sweep
query reverted, the listing assertion fails.
- `npx vitest run` on 62 related server test files (workspace recovery
and its preparation suites, legacy execution recovery, run completion,
retry scheduling, recovery service, remote workspaces, task retry, fresh
continuation): 2361 pass, 16 skipped.
- `npx tsc --noEmit` passes for the server.

## Risks

- Low to medium. A failed preparation that kept its sandbox now
continues once its files are saved, if its failure code is transient.
The continuation uses the interrupted-preparation lane, which is bounded
to two attempts, and the existing failure budget still applies.
- A failure that is not transient (`adapter_failed`, `setup_failed`) is
unchanged: it still waits for Retry.
- A run whose failure is not a pre-provider remote preparation timeout
keeps `adapter_failed`.
- A task that is `done` or `cancelled` is unchanged. A scheduled retry
never runs on a closed task, so a wake on a closed task is still not
replayed after a failed preparation.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution
in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-17 16:04:19 -08:00
Nicky Leach 323504e41e perf(runner): enable TCP_NODELAY for loopback PRP (#22091)
## Thinking Path

> - Paperclip manages AI agents for work.
> - Native Runner exchanges commands and events over WebSocket PRP.
> - Both Runner connection modes use TCP sockets.
> - Several small frames can follow each other during a start.
> - Loopback peers can receive those frames promptly with TCP_NODELAY.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Delivery of PRP command results and events on local TCP connections.

**Subsystem affected**

Native Runner PRP transport.

**Current behavior**

Runner uses the default TCP setting for acknowledged small writes on its
accepted and outbound PRP sockets. In an A/B test with a loopback model
fixture, enabling TCP_NODELAY on the accepted Runner socket reduced
median time to observe turn.started by 29 to 52 ms, depending on the
provider and session state.

**Proposed behavior**

Enable TCP_NODELAY on accepted and outbound PRP sockets before the
WebSocket handshake when the connected peer address is loopback.

**Reason and benefit**

The controller receives turn-start events sooner. The outbound path
shares the same setup rule and has socket-option coverage.

**Breaking changes**

The change is a per-connection socket option. Existing launch arguments,
durable checkpoints, and PRP frames carry forward.

## What Changed

- Share initial stream configuration between the listener and dialer.
- Check the actual peer address before enabling TCP_NODELAY.
- Complete a real WebSocket handshake in one test for each connection
mode and read back the option.

## Verification

- `cargo fmt --manifest-path packages/paperclip-runner/runner/Cargo.toml
--all --check`
- `cargo test --offline --locked -p paperclip-runner-core --lib
durable::transport::tests` passed locally on macOS (27 passed).
- Both new tests failed on the original socket setup and passed with the
change.
- `cargo test --offline --locked -p paperclip-runner-core --lib --bins
--test listener_accept` passed locally on macOS (676 passed, 4 ignored).

## Risks

- TCP can send more small packets during high-volume local output. The
check limits the option to loopback peers. The earlier A/B measured
turn-start timing on the listener path.

## Model Used

- OpenAI GPT-6 (Codex), with source inspection and local Rust tests.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g., `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
canary/v2026.1117.0-canary.823100
2026-11-17 23:14:48 +00:00
DottaandClaude Opus 5.5 f295e496db fix(daytona): retry lease acquisition after a provider connection failure (#22087)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agent runs on Daytona start with a lease acquisition in the Daytona
plugin
> - When the host cannot reach Daytona, the Daytona SDK raises a
connection error that has no HTTP status and no errno
> - The host only retries a lease failure that carries structured
evidence, so this failure became `setup_failed` and the task waited for
a person to press Retry
> - This pull request makes the plugin report the SDK connection error
as a closed transport code that the host already retries
> - The benefit is that a short network blip during sandbox startup no
longer needs a Retry click

## Linked Issues or Issue Description

**What happened?**

In a restart-chaos wave (201 tasks, 6 graceful release swaps), the host
could not resolve or reach Daytona for about 12 minutes. One task needed
a Retry click. Its only run failed with `setup_failed`
(`lease_acquire_failed`) and the message `Failed to acquire lease for
environment "Daytona warm ..." (sandbox): read EADDRNOTAVAIL`. Automatic
recovery did not continue the task because `setup_failed` is not a
transient setup code. The task stayed failed for about 100 minutes until
the grader pressed Retry.

**Expected behavior**

A lease acquisition that fails because the provider could not be reached
is retried automatically with the bounded bootstrap budget, as
`environment_provider_unavailable`. A person never needs to press Retry
for a network blip.

**Steps to reproduce**

1. Use the Daytona provider.
2. Make a Daytona API call during lease setup fail at the transport
level (no HTTP response), for example with `read EADDRNOTAVAIL` or
`getaddrinfo ENOTFOUND`.
3. See that the run fails with `setup_failed` and the task shows Retry.

**Paperclip version or commit**

master `ca34aca777` (wave) and `02ee538be9`.

Root cause: the pinned `@daytonaio/sdk` converts an Axios transport
error into `DaytonaConnectionError` (or
`DaytonaConnectionTimeoutError`). It keeps neither the errno nor the
cause. The worker RPC therefore sent no provider diagnostic, and
`readTransientLeaseAcquisitionFailure` returned `null`.

## What Changed

- The Daytona plugin wraps a `DaytonaConnectionError` from lease
acquisition in an error with code `CONNECTION_FAILED`, and a
`DaytonaConnectionTimeoutError` with code `ETIMEDOUT`. The message is
unchanged. It reads the SDK class name only, never message text.
- The plugin SDK closed diagnostic set accepts `CONNECTION_FAILED`, so
the code crosses the worker RPC.
- The host treats `CONNECTION_FAILED` as a transient connection failure.
The run becomes `environment_provider_unavailable` and uses the existing
bounded bootstrap retry (30 s, 60 s) and the automatic-run brake.
- Refreshed `tests/runner-e2e/source-sdk-public-inputs.json` for the
changed SDK file.

## Verification

- Red/green: with the source change removed, the new plugin tests
(`reports a DaytonaConnectionError during setup as a transient transport
failure`, timeout variant) and the new host tests (`treats errno
CONNECTION_FAILED as transient`, `retries a provider connection failure
reported across the worker RPC`) fail. With the change, they pass.
- `cd packages/plugins/sandbox-providers/daytona && npx vitest run` (637
passed).
- `npx vitest run
server/src/services/environment-provider-unavailable-error.test.ts
packages/plugins/sdk/tests/environment-acquire-error.test.ts` (66
passed).
- `pnpm typecheck` in the Daytona plugin passes.
- `pnpm verify` asks for broad CI coverage because the SDK input
registry changed. CI supplies that coverage.

## Risks

- Low risk. A wrong Daytona API URL also produces
`DaytonaConnectionError`. It now gets two bounded retries (about 90 s)
before the same Retry state as before.
- The retries count toward the automatic-run brake, so a long outage
cannot multiply run starts across the company.
- A create that timed out on the client keeps its existing
creation-cleanup envelope. The retry is a new creation attempt with its
own identity.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 16:55:47 -06:00
Devin FoleyandPaperclip eaf6461cee fix(server): give a slow remote more time to exclude agent files from Git (#22039)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Before a remote run starts, Paperclip runs a short Node program in
the sandbox that keeps the reserved runtime directory out of task Git
staging
> - That program had one 15 second attempt. A freshly created sandbox is
sometimes slow, so the attempt timed out
> - The run then failed before any provider work started, with the
generic `adapter_failed` code. Nothing retried it, and the wake that
started the run was lost
> - This pull request gives each attempt 30 seconds and retries a
timed-out attempt, up to three attempts
> - The benefit is that a slow sandbox no longer stops a task, while a
real failure still fails at once and names its reason

## Linked Issues or Issue Description

Builds on #21499 (merged), which adds the closed `reason=` code to this
error.

**What happened?**
A run failed during preparation with "Could not exclude agent files from
task Git staging". The sandbox lease was acquired about 50 seconds
before the agent-file copy was registered (3 to 7 seconds on the same
stack's earlier runs). The copy failed 20 seconds after it was
registered, which matches the 15 second limit. Earlier failures with
this message all took at least 16 seconds before failing, so they also
match a timeout rather than an error from the program itself.

**Steps to reproduce**
1. Start a remote run on a sandbox that takes more than 15 seconds to
run a short command.
2. The run fails in the preparing stage and is not retried.

**Expected behavior**
A slow sandbox gets more time, and a timed-out attempt is retried.

**Actual behavior**
One 15 second attempt, then the run fails.

## What Changed

- `agent-instruction-files.ts`: add `excludeRuntimeFromRemoteGit`. Each
attempt gets 30 seconds. A timed-out attempt is retried, up to three
attempts. Any other failure throws at once. The error keeps the closed
`reason=` code from #21499.
- `agent-directory-working-copies.ts` and
`agent-instruction-working-copies.ts`: both remote call sites use the
helper, so the private instruction copy also names a reason.
- `execution-target.ts`: `runSshCommand` rejects a command its timeout
killed with `code: null` and `killed: true`, but only `ETIMEDOUT` was
treated as a timeout, so an SSH timeout was thrown instead of returned.
A killed command now returns `timedOut: true`, as the sandbox path
already does.
- Tests: a timeout followed by success stages the agent files; three
timeouts fail with `reason=timed_out`; a non-timeout failure is
attempted once. An adapter-utils test covers the real SSH timeout
rejection.

## Verification

- `npx vitest run` on the agent-directory, instruction working-copy, Git
exclusion, copy retry backoff, instruction-working-copy and remote
checkpoint lock suites: 217/217 pass.
- adapter-utils `execution-target` and sandbox suites: 173/173 pass.
adapter-utils SSH suites: 65 pass, 2 skipped. Server adapter registry
suite: 67/67 pass.
- The new cases fail without this change.
- `npx tsc --noEmit` passes for server and adapter-utils.

## Risks

- Low. A sandbox that never answers now takes up to 90 seconds to fail
instead of 15 seconds.
- An SSH shell command that times out is now returned as `timedOut`
instead of thrown. Every caller that checks the result already treats
`timedOut` as a failure. The home-directory probe and the empty-file
helper ignore the result, so for them an SSH timeout now behaves as a
sandbox timeout already did.
- The exclusion program is idempotent: it writes its file through a
temporary name and a rename, so a retry after a late first attempt
leaves the same result.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution
in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-17 14:28:27 -08:00
Nicky LeachandPaperclip f05b90508a refactor(core): remove retired compatibility paths (#22055)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Core manages chat authorization, AI connection defaults, navigation,
and the plugin host.
> - Core still writes old default rows and checks an Agent Chat flag
that always reads true.
> - It also carries a desktop rail that cannot open and private aliases
with canonical equivalents.
> - This pull request removes those paths and moves their tests onto the
paths Core uses.
> - The result is fewer Core database operations and less code to
maintain.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Core AI default writes, Agent Chat execution and delivery, private
plugin host and Runner setup, and board navigation.

**Subsystem affected**

Cross-cutting. Core server, board UI, private Runner helpers, and a
repository guard script.

**Current behavior**

The server writes provider-scoped and per-method AI defaults, then reads
only the provider-scoped rows. It reads settings to decide whether to
block Agent Chat, although every settings view enables it. The desktop
sidebar renders expanded while still tracking requests to open and close
a retired icon rail. The agent client retries conflicts that the server
already resolves when there is exactly one active match.

**Proposed behavior**

Write provider-scoped defaults directly. Use the existing Agent Chat
authorization and retry checks without the redundant settings reads.
Render the current sidebar directly and let the API report true agent
reference conflicts.

**Reason and benefit**

A personal non-custom connection save or explicit default selection
sends one fewer SQL statement. It also avoids the provider-default write
that the legacy trigger performed when the old statement inserted or
updated a row. A successful task-to-chat send or receipt replay avoids
two transaction settings SELECTs when the singleton row already exists.
Conversation comment, wake release, and retry paths also avoid a
transaction settings read when they previously entered the gate. Savings
in other settings paths depend on cache hits. Core-to-plugin RPC and
plugin-to-Launcher HTTP traffic stay the same.

**Breaking changes**

The public plugin SDK, host navigation, and organization-switcher fields
retain their existing interfaces and behavior. The settings API still
accepts and reports the Agent Chat compatibility key. This change needs
no schema migration or rollout ordering.

**Additional context**

Refs #10610 for prior reporting on the retired sidebar rail. Refs #21756
for adjacent Coordinator plugin slots, which remain a separate change.

## What Changed

- Stop writing per-method AI defaults. Preserve the provider-scope
upsert and first-account behavior. Keep the existing tables and
migration trigger.
- Remove Agent Chat checks that read a constant true value. Keep
conversation ownership, task scope, session, and request-retry checks.
- Remove unused lock-owner fields and private plugin lifecycle, secret
extraction, Gmail connector, PRP setup, heartbeat resolver, and ACPX
aliases.
- Use a single API request for agent references. Remove retired sidebar
rail state, hooks, and branches across the active and production
snapshot layouts. Preserve desktop resizing, mobile drawer controls,
contextual navigation, and public plugin slot inputs.
- Remove unused FeedCard and AgentDetail inputs and the test-only Git
push matcher alias.
- Keep the cleanup that erases historical plaintext chat retries from
browser storage.

Production source in the touched files is 106,800 → 105,633 nonblank
lines, down 1,167. The production Git diff adds 251 physical lines and
removes 1,480, down 1,229. Those figures exclude tests, fixtures, docs,
and Storybook. The entire Git diff adds 482 physical lines and removes
2,325, down 1,843.

## Verification

- Merged master at `b56f240f45` into the PR branch as `f73fea4009`. The
five sidebar files retain the expanded navigation cleanup and master’s
explicit session-replay masking contract. The merge also preserves
master’s account-repair recovery guard and GitHub star-prompt behavior.
- Focused UI Vitest run passed 361 tests across 11 files. This covers
analytics privacy and replay masking, sidebar and layout, GitHub star
prompt, and agent detail.
- Focused server Vitest run passed 45 tests across three files. This
covers restored configuration handling, conversation recovery, and Agent
Chat authority and receipt replay.
- `pnpm --filter @paperclipai/ui typecheck`, the Git push guard and its
14 tests, and `git diff --check origin/master...HEAD` passed.
- `pnpm verify --base origin/master` classified the Git-push guard as a
global CI input and returned exit 2. GitHub CI supplied the broader
coverage.
- GitHub CI and Runner image checks passed on `f73fea4009`. Both
required checks, `ci / verify` and `ci / e2e`, passed. [CI
workflow](https://github.com/paperclipai/paperclip/actions/runs/44181261752)
· [Runner image
workflow](https://github.com/paperclipai/paperclip/actions/runs/44181261413)
- Standard Greptile reported 5/5 with no actionable findings or open
review threads. [Greptile
review](https://github.com/paperclipai/paperclip/pull/22055#issuecomment-6959078925)
- The optional Superagent Security Scan finished neutral with an
inconclusive workflow result and zero annotations. The final rollup had
61 successes, five skips, and one neutral, with no failed or pending
checks. [PR
checks](https://github.com/paperclipai/paperclip/pull/22055/checks)

## Risks

Agent Chat authorization and retry are sensitive to small query changes.
The final diff keeps the conversation agent field and all existing
authorization and idempotency checks. The tests cover revoked defaults,
legacy migration reads, bound-listener authority, and mobile and desktop
navigation. Installations that still run software old enough to read the
per-method AI table will no longer see new defaults in that table.
Current Core reads the provider-scoped table.

## Model Used

OpenAI Codex, GPT-6.1 Sol (`gpt-6.1-sol`), with source inspection and
shell-based tests.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-17 14:00:14 -08:00
Devin FoleyandPaperclip f5f60cfcb2 fix(server): name why agent files could not be excluded from task Git staging (#21499)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Before a remote run starts, Paperclip prepares the agent's files in
the sandbox and keeps the reserved runtime directory out of task Git
staging
> - A short program in the sandbox writes the Git exclusion. When it
failed, the run error said only "Could not exclude agent files from task
Git staging"
> - The error dropped the exit code, the timeout state and the program
output. Many staging runs failed with this message, and none of them
showed the cause
> - This pull request adds a closed reason code to the error
> - The benefit is that the next failures show what to fix, without
leaking sandbox paths or user names

## Linked Issues or Issue Description

**What happened?**
Many staging runs failed in the preparing stage with "Could not exclude
agent files from task Git staging". The error had no cause.

**Steps to reproduce**
1. Start a remote run where the Git exclusion program fails (for
example, it times out, Node is missing, or the exclusion file is
unsafe).
2. Read the run error.

**Expected behavior**
The error names why the exclusion failed.

**Actual behavior**
The error gives no reason, so the failure cannot be diagnosed.

## What Changed

- `agent-instruction-files.ts`: add
`instructionGitExcludeFailureReason`. It maps the result to a closed
reason: `timed_out`, `unsafe_directory`, `unsafe_file`, a Node error
code such as `enoent` or `eacces`, `node_unavailable`, `signal_<name>`,
`exit_<code>` or `unknown`.
- `agent-directory-working-copies.ts`: append `(reason=<code>)` to the
existing error message.
- The reason never copies sandbox output, paths or user names.
- Tests: run the real exclusion program against failing fixtures and
check the reason.

## Verification

- `pnpm vitest run
server/src/__tests__/agent-instruction-git-exclusion.test.ts`: 16/16
pass.
- Working copy suite: 136/136 pass.
- These local results are from the earlier master base. The rebase onto
the current master had no conflicts. CI verifies the current head.

## Risks

- Low. The change adds a suffix to one error message. It does not change
when the error is thrown.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution
in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-17 13:56:09 -08:00
DottaandClaude Opus 5.5 23ce68a8dc feat(connections): end Google access when a Paperclip-managed Google connection is removed (#22065)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Connectors let agents use Google Workspace through a Google sign-in
that Paperclip Cloud manages for the user
> - When a user removes a Paperclip-managed Google connection, Paperclip
deletes the tokens locally, but the grant stays live at Google
> - Google's OAuth review expects that a disconnect ends the app's
access, and users expect it too
> - Google's revoke ends the user's whole grant to the shared client, so
only the Paperclip Cloud broker can decide when it is safe to revoke
> - This pull request asks the broker to revoke each removed grant,
keeps local removal unconditional, and tells the user the outcome in
plain language
> - The benefit is that removal ends Google access when no other
Paperclip connection needs it, and the user knows what to do when it did
not

## Linked Issues or Issue Description

**Subsystem affected**
Connectors: Paperclip-managed Google Workspace and the legacy
per-service Google connections (server removal, shared contracts,
connection UI).

**Problem or motivation**
Removing a Paperclip-managed Google connection or grant was local only.
The refresh token was deleted in Paperclip, but Google still listed
Paperclip as having access to the account. The docs said so ("Removal is
local for managed grants").

**Proposed solution**
On removal, send each removed grant's refresh token to the broker's
signed `POST /v1/connector/revoke`. The broker revokes at Google only
when no other live binding could share the grant. Paperclip records the
broker's outcome code and shows one of three plain messages. Local
removal never waits on, or fails because of, that call.

**Alternatives considered**
Call Google's revoke endpoint from the app. This ends the user's whole
grant to the shared client and can break their other Google connections,
so the broker owns the decision.

**Additional context**
Older brokers answer 409 `REVOCATION_NOT_AVAILABLE`. The app treats
that, a timeout, and a network error as "could not end access at Google"
and still removes the connection.

## What Changed

- `paperclip-cloud-connector.ts`: `revoke()` takes an optional
`timeoutMs`, returns the broker's `outcome` from an allowlist, and keeps
`REVOCATION_NOT_AVAILABLE` as an allowed rejection reason.
- New `tool-access/connections/google-access-removal.ts`. For a managed
Google connection, `prepare()` reads each removed grant's refresh token
and returns a `send()` step. `send()` calls the broker for each distinct
token, one after another, under one 5-second deadline. Every result and
every error (including a Paperclip Cloud client that cannot be created)
becomes an outcome code. It never rejects and never logs a token. An
unexpected error is logged with the error name only and reported as
`unexpected_error`.
- Order for removing a whole connection: read the tokens, run the whole
local teardown (grants, installs, app profile, gateway tokens, runtime
slots, catalog, OAuth state, archive the connection), call the broker,
delete the sent refresh-token secrets whatever the outcome, then delete
the remaining secrets. A failure anywhere in the teardown makes no
broker call. Order for removing one grant: read the token, commit the
revocation, call the broker, delete the grant's refresh-token secret.
`send()` owns the call and the deletion through an injected
secret-removal function.
- Owed revocations: a revoked managed Google grant whose refresh-token
secret still exists (and that nothing live uses) is sent by every later
removal, then the secret is deleted. This covers a process stop after
the teardown and before the call, with no new column. Once the secret is
deleted, nothing is sent again.
- Shared tokens: the shared (organization) sign-in writes the same
secret refs to the organization grant and the connection. So revoking an
organization grant alone neither calls the broker nor deletes the
secret, because the connection still uses that token; removing the
connection does both. The same rule applies to any refresh token still
used by another live grant or connection.
- Audit history: once the broker call settles, a
`connection.google_access_removal` audit event records `reasonCode` =
result and `providerAccess` (plus `grantId` for a single grant). If that
write fails, the removal still succeeds and the failure is logged.
- Revoke signs with the subject that consented to the grant
(`providerTenant.externalId`, saved by the managed callback from the
claim). Grants saved without it fall back to the refresh rule. Refresh
is unchanged; see Risks.
- `removal.ts` and `grants.ts`: `providerAccess: { provider, result,
outcomes }` goes to the responses and to the `tool_connection.archived`
and `tool_connection.grant_revoked` activity entries. The tool access
audit field `providerRevocation` keeps its Vercel meaning only.
- `packages/shared`: `ManagedGoogleAccessRemoval`,
`ManagedGoogleRevocationOutcome`, `ConnectionGrantRevocationResult`.
- UI: one helper, `connectionRemovalToast`, builds the removal toast
(base sentence plus the Google notice, tone and action) for Connectors,
Connections and the app page. No codes are shown.
- API: `docs/api/tool-connections.md` documents both DELETE responses
and `providerAccess`. The OpenAPI registration describes the optional
field and its four results.
- Docs: `GOOGLE-WORKSPACE.md` (order, owed revocations, decision table),
`GMAIL.md`, the Google connector memory note, and the help page on
deleting connection data.

Decision table, judged over the distinct tokens sent in this removal:

| Outcomes | `result` | User message |
| --- | --- | --- |
| Every token `revoked` | `ended` | "Paperclip's access to this Google
account was also ended at Google." |
| Any failure-type code: `provider_revocation_failed`,
`binding_not_known` (the broker had no record, so Google was not
called), `provider_revocation_disabled`, `revocation_in_progress`,
`revocation_not_available`, `broker_unavailable`, `broker_error`,
`credential_unavailable`, `unexpected_error` | `not_ended` | "Paperclip
removed the connection but couldn't end its access at Google. You can
remove Paperclip from your Google account's third-party connections." +
"Open Google account connections" |
| Only `other_connections_remain` | `other_connections_remain` | "Other
Paperclip connections still use this Google account, so Google keeps
Paperclip's access until you remove those too." |
| At least one `revoked`, the rest `other_connections_remain` |
`ended_unconfirmed` | "Paperclip's access was ended at Google. If more
than one Google account was connected here, check your Google account's
third-party connections to be sure." + "Open Google account connections"
|

Grants record no Google account identity: the managed callback saves
`externalId` (the Paperclip subject), `name` (the provider name), and
OAuth timing and scopes, and none of the 14 scopes returns an account
identifier. So the last row is always `ended_unconfirmed`. This includes
two tokens of one user and several agent grants on one Google account.

## Verification

- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/google-managed-connection-removal.test.ts
src/services/tool-access
src/services/paperclip-cloud-connector-revoke.test.ts
src/services/paperclip-cloud-connector.test.ts
src/__tests__/tool-connection-removal.test.ts
src/__tests__/tool-access-service.test.ts
src/__tests__/openapi-routes.test.ts` (714 passed)
- `pnpm --filter @paperclipai/ui exec vitest run
src/features/connections/googleAccessRemoval.test.ts src/pages/apps`
(passed)
- Server test typecheck (`node server/scripts/typecheck-layers.mjs`)
passes.
- Red then green: the server tests failed on `master` sources (21), then
each review round's new tests failed on the previous head and pass now.
They cover: the order on both paths, local failure before the grant
commit making no call and the retry sending, cleanup failure after the
call making no second call, a process stop between commit and call being
sent on retry with the secret deleted after, a Paperclip Cloud client
that cannot be created, the consenting subject for an organization
grant, sequential sends under one deadline, each row of the decision
table (per-agent and same-user cases give `ended_unconfirmed`, a mixed
failure gives `not_ended`), the unexpected-error log without a token, no
token in any log call, the delete-stored-data option, and a toast test
per surface.
- Added time, measured locally with a stubbed broker that answers at
once: median 22.1 ms against 20.3 ms without the broker step (12 runs
each), about 2 ms. With a real broker, one round trip per removed token,
capped at 5 seconds in total.
- `pnpm verify` exits 2 (a global CI input changed), so required CI
supplies the broad coverage.
- Not verified against a live broker or a real Google account.

## Risks

- A removal of a managed Google connection or grant can take up to 5
seconds longer when the broker is slow.
- Refresh subject is not changed in this PR. In the broker
(`recordProviderGrant`), a ledger row's subject is fixed when the row is
first written, and a refresh under another subject leaves that row
untouched. So moving refresh to a new subject could stop updating the
row's last-seen time. After 200 days the row would then count as not
live, and a later removal elsewhere could revoke at Google while this
connection still works. That risk cannot be ruled out from code for rows
written before this release, so both refresh paths keep their old rules.
The two refresh paths already use different subject orders today: the
gateway prefers `connectorSubjectAgentId` and then
`connectorSubjectUserId`, while background refresh prefers the grant's
own subject. That is pre-existing, and this PR leaves it unchanged.
Revoke uses the consenting subject, which matches the claim-time ledger
row. When the row was first written by a refresh under a different
subject, the broker answers `binding_not_known` and the user gets the
warning.
- Repeated removals can send an already-removed token again; the broker
returns its recorded outcome and changes nothing at Google.
- Against an older broker, users see the "could not end access at
Google" message, which is accurate.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and
code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 15:52:20 -06:00
dependabot[bot] 24728076d6 build(deps-dev): bump wrangler from 4.156.2 to 4.158.0 in /scripts/typecheck-sandbox-providers (#20200)
Bumps
[wrangler](https://github.com/cloudflare/workers-sdk/tree/HEAD/packages/wrangler)
from 4.156.2 to 4.158.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/cloudflare/workers-sdk/releases">wrangler's
releases</a>.</em></p>
<blockquote>
<h2>wrangler@4.158.0</h2>
<h3>Minor Changes</h3>
<ul>
<li>
<p><a
href="https://redirect.github.com/cloudflare/workers-sdk/pull/16186">#16186</a>
<a
href="https://github.com/cloudflare/workers-sdk/commit/f1d81a20c8e535ec7cb870143283f4a683fec70d"><code>f1d81a2</code></a>
Thanks <a
href="https://github.com/leostera"><code>@​leostera</code></a>! -
Support local Artifacts across Wrangler, Vite, and Workers Vitest</p>
<p>Set <code>remote: false</code> on an Artifacts binding to opt into
the Miniflare simulator and persistent local repositories during
development. Omitted <code>remote</code> continues to use the deployed
resource, with a warning that changes affect real repositories;
<code>remote: true</code> explicitly selects it. When remote bindings
are disabled (<code>wrangler dev --local</code>, or Vite's
<code>remoteBindings: false</code>), Artifacts bindings use the local
simulator. Workers Vitest uses the local simulator when
<code>remote</code> is omitted, so tests do not change real repositories
by accident; set <code>remote: true</code> to test against the deployed
resource. Local development does not require Cloudflare credentials, but
does require Git 2.32 or newer on the host.</p>
<p>In Miniflare's own options, Artifacts bindings route like every other
binding: only <code>dev.remote: true</code> with a remote proxy
connection uses the deployed resource. Wrangler and Vite resolve the
deployed-by-default rule, and Workers Vitest its local default, before
passing options to Miniflare.</p>
</li>
</ul>
<h3>Patch Changes</h3>
<ul>
<li>
<p><a
href="https://redirect.github.com/cloudflare/workers-sdk/pull/16700">#16700</a>
<a
href="https://github.com/cloudflare/workers-sdk/commit/dc35efbd3cdfe67b3ef972c75b8faae65dfed195"><code>dc35efb</code></a>
Thanks <a href="https://github.com/cjol"><code>@​cjol</code></a>! - Bump
bundled <code>brace-expansion</code> to pick up security fixes</p>
<p>These packages bundle <code>brace-expansion</code> (a dependency of
<code>minimatch</code>), and the bundled versions were affected by
published advisories. The bundles now include
<code>brace-expansion</code> 2.1.7 (from 2.0.1) and 5.0.12 (from 5.0.5).
Normal glob patterns expand as before; only the pathological patterns
targeted by the advisories are handled differently.</p>
</li>
<li>
<p><a
href="https://redirect.github.com/cloudflare/workers-sdk/pull/16700">#16700</a>
<a
href="https://github.com/cloudflare/workers-sdk/commit/dc35efbd3cdfe67b3ef972c75b8faae65dfed195"><code>dc35efb</code></a>
Thanks <a href="https://github.com/cjol"><code>@​cjol</code></a>! - Bump
bundled <code>fast-xml-parser</code> from 4.4.1 to 4.5.5 to pick up
security fixes</p>
<p>Miniflare bundles <code>fast-xml-parser</code> into its local R2 S3
API, and Wrangler bundles it through the AWS SDK used by <code>wrangler
pipelines</code>. Version 4.4.1 is affected by several published
advisories, and 4.5.5 includes their fixes. XML handling does not
change.</p>
</li>
<li>
<p><a
href="https://redirect.github.com/cloudflare/workers-sdk/pull/16700">#16700</a>
<a
href="https://github.com/cloudflare/workers-sdk/commit/dc35efbd3cdfe67b3ef972c75b8faae65dfed195"><code>dc35efb</code></a>
Thanks <a href="https://github.com/cjol"><code>@​cjol</code></a>! - Bump
bundled <code>minimatch</code> from 5.1.6 to 5.1.9 to pick up security
fixes</p>
<p>These packages bundle <code>minimatch</code>, and 5.1.6 is affected
by published ReDoS advisories. Version 5.1.9 includes the fixes. 5.1.8
is skipped because it broke recursive <code>**</code> matching, which
D1's <code>migrations_pattern</code> relies on.</p>
</li>
<li>
<p><a
href="https://redirect.github.com/cloudflare/workers-sdk/pull/16700">#16700</a>
<a
href="https://github.com/cloudflare/workers-sdk/commit/dc35efbd3cdfe67b3ef972c75b8faae65dfed195"><code>dc35efb</code></a>
Thanks <a href="https://github.com/cjol"><code>@​cjol</code></a>! - Bump
bundled <code>shell-quote</code> from 1.10.0 to 1.11.0 to pick up a
security fix</p>
<p><code>shell-quote</code> is bundled into Wrangler, which uses it to
parse and quote shell commands such as custom build and Pages dev
commands. Version 1.10.0 is affected by a published advisory, and 1.11.0
includes the fix. Command parsing does not change.</p>
</li>
<li>
<p><a
href="https://redirect.github.com/cloudflare/workers-sdk/pull/16700">#16700</a>
<a
href="https://github.com/cloudflare/workers-sdk/commit/dc35efbd3cdfe67b3ef972c75b8faae65dfed195"><code>dc35efb</code></a>
Thanks <a href="https://github.com/cjol"><code>@​cjol</code></a>! - Bump
bundled <code>yaml</code> from 2.8.1 to 2.9.0 to pick up a security
fix</p>
<p><code>yaml</code> is bundled into Wrangler, which uses it to show
container application details. Version 2.8.1 is affected by a published
advisory, and versions from 2.8.3 onwards include the fix. Output does
not change.</p>
</li>
<li>
<p><a
href="https://redirect.github.com/cloudflare/workers-sdk/pull/16688">#16688</a>
<a
href="https://github.com/cloudflare/workers-sdk/commit/fa3edb325694670084bc38af02232957315ccdb5"><code>fa3edb3</code></a>
Thanks <a
href="https://github.com/FlorentCollin"><code>@​FlorentCollin</code></a>!
- Use the <code>account_id</code> from the configuration file in
<code>wrangler d1 migrations list --remote</code></p>
<p>Before this change, the command ignored the <code>account_id</code>
in the configuration file. When your login had access to more than one
account, the command failed in non-interactive mode. In interactive
mode, it asked you to select an account. The command now uses the
configured <code>account_id</code>, as <code>wrangler d1 migrations
apply</code> does.</p>
</li>
<li>
<p><a
href="https://redirect.github.com/cloudflare/workers-sdk/pull/16695">#16695</a>
<a
href="https://github.com/cloudflare/workers-sdk/commit/b2061f40f812d8888b461f68506552b58f8e468b"><code>b2061f4</code></a>
Thanks <a
href="https://github.com/apps/dependabot"><code>@​dependabot</code></a>!
- Update <code>DEFAULT_COMPAT_DATE</code> to 2026-11-04</p>
<p>Use the bundled workerd release's compatibility date by default when
scaffolding new projects or when no compatibility date is
configured.</p>
</li>
<li>
<p><a
href="https://redirect.github.com/cloudflare/workers-sdk/pull/16695">#16695</a>
<a
href="https://github.com/cloudflare/workers-sdk/commit/b2061f40f812d8888b461f68506552b58f8e468b"><code>b2061f4</code></a>
Thanks <a
href="https://github.com/apps/dependabot"><code>@​dependabot</code></a>!
- Update dependencies of &quot;miniflare&quot;, &quot;wrangler&quot;</p>
<p>The following dependency versions have been updated:</p>
<table>
<thead>
<tr>
<th>Dependency</th>
<th>From</th>
<th>To</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>@​cloudflare/workers-types</code></td>
<td>^5.20261103.1</td>
<td>^5.20261104.1</td>
</tr>
<tr>
<td>workerd</td>
<td>1.20261103.1</td>
<td>1.20261104.1</td>
</tr>
</tbody>
</table>
</li>
<li>
<p><a
href="https://redirect.github.com/cloudflare/workers-sdk/pull/16306">#16306</a>
<a
href="https://github.com/cloudflare/workers-sdk/commit/36e64f20025cf0bd5e99a585ca3427c8d75fd18b"><code>36e64f2</code></a>
Thanks <a href="https://github.com/efalcao"><code>@​efalcao</code></a>!
- Report Cloudflare Access login redirects on remote bindings instead of
looping until &quot;Too many redirects&quot;</p>
</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/cloudflare/workers-sdk/commit/c579d500d0c5732393f2e85f0cdd085167aee32e"><code>c579d50</code></a>
Version Packages (<a
href="https://github.com/cloudflare/workers-sdk/tree/HEAD/packages/wrangler/issues/16681">#16681</a>)</li>
<li><a
href="https://github.com/cloudflare/workers-sdk/commit/f1d81a20c8e535ec7cb870143283f4a683fec70d"><code>f1d81a2</code></a>
[miniflare] Enable local Artifacts in development tools [3/3] (<a
href="https://github.com/cloudflare/workers-sdk/tree/HEAD/packages/wrangler/issues/16186">#16186</a>)</li>
<li><a
href="https://github.com/cloudflare/workers-sdk/commit/dc35efbd3cdfe67b3ef972c75b8faae65dfed195"><code>dc35efb</code></a>
Bump dependencies to clear 101 Dependabot alerts (<a
href="https://github.com/cloudflare/workers-sdk/tree/HEAD/packages/wrangler/issues/16700">#16700</a>)</li>
<li><a
href="https://github.com/cloudflare/workers-sdk/commit/4d88a45a952542c55ec63bbca671fb1f003a09c7"><code>4d88a45</code></a>
[miniflare] Stabilize Artifacts integration tests on Windows (<a
href="https://github.com/cloudflare/workers-sdk/tree/HEAD/packages/wrangler/issues/16693">#16693</a>)</li>
<li><a
href="https://github.com/cloudflare/workers-sdk/commit/0083f807de557d771f9d2d4f08d6406f5062c295"><code>0083f80</code></a>
[wrangler] Harden the local KV pagination regression test (<a
href="https://github.com/cloudflare/workers-sdk/tree/HEAD/packages/wrangler/issues/16697">#16697</a>)</li>
<li><a
href="https://github.com/cloudflare/workers-sdk/commit/fa3edb325694670084bc38af02232957315ccdb5"><code>fa3edb3</code></a>
[wrangler] Use config account_id in d1 migrations list --remote (<a
href="https://github.com/cloudflare/workers-sdk/tree/HEAD/packages/wrangler/issues/16688">#16688</a>)</li>
<li><a
href="https://github.com/cloudflare/workers-sdk/commit/9a0732c3f4f6cbbc016a7134fc1d20edb98b454e"><code>9a0732c</code></a>
Version Packages (<a
href="https://github.com/cloudflare/workers-sdk/tree/HEAD/packages/wrangler/issues/16654">#16654</a>)</li>
<li><a
href="https://github.com/cloudflare/workers-sdk/commit/44d1ee439c0a077f2fed4d86e35a4edf46b51f74"><code>44d1ee4</code></a>
fix(wrangler): stop <code>wrangler delete</code> orphaning Container
applications (<a
href="https://github.com/cloudflare/workers-sdk/tree/HEAD/packages/wrangler/issues/16011">#16011</a>)</li>
<li><a
href="https://github.com/cloudflare/workers-sdk/commit/8ac43df32667a77cd7e8e267da86e96645e3031b"><code>8ac43df</code></a>
Require explicit route metadata flags when downloading Worker
configuration (...</li>
<li>See full diff in <a
href="https://github.com/cloudflare/workers-sdk/commits/wrangler@4.158.0/packages/wrangler">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=wrangler&package-manager=npm_and_yarn&previous-version=4.156.2&new-version=4.158.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-17 13:51:44 -08:00
Dotta 02ee538be9 feat(github): steer active reviews when new messages arrive (#22066)
Send compatible GitHub review updates to active native turns after normal admission. Preserve durable receipt fallback and recover acknowledgements without consuming unsent edits.

Keep legacy conversation batching and existing identity, source-trust, and attachment boundaries. Add PostgreSQL regressions for concurrency, restart recovery, and edited inputs.
2026-11-17 15:47:13 -06:00
DottaandClaude Opus 5.5 73dc33c620 fix(runner): keep a disk reserve so a full sandbox disk does not strand the runner (#22068)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents in a Paperclip-managed sandbox run under the Paperclip runner
(`runnerd`), which keeps durable state on the sandbox disk
> - An agent can fill that disk with its own work. In one case the agent
installed a Rust toolchain and a second checkout with dependencies on a
10 GiB sandbox
> - `runnerd` then could not write its state and exited. Each relaunch
tried to upload its launch files to the full disk and failed, 83 times
in 7 minutes
> - The run ended as a generic provider failure that asked a person to
inspect it, and Retry repeated the same failure
> - This pull request keeps a small disk reserve that belongs to
Paperclip. Recovery releases it to relaunch the runner, and stops at
once with a named code when nothing is left to free
> - The benefit is that the agent's turn can continue and see the full
disk in its own tool output, instead of the run dying with it

## Linked Issues or Issue Description

**What happened?**
An agent in a managed Daytona sandbox filled the 10 GiB disk. `runnerd`
exited with `failed to persist bounded diagnostic: No space left on
device (os error 28)`. Runner process recovery then retried for the full
420000 ms reconnect grace. Every attempt failed with
`.paperclip-upload-<id>.tar.gz: create: ... no space left on device`.
The run failed as `native_provider_terminal_failed`, and the recovery
asked a board user to verify that the provider stopped.

**Steps to reproduce**
1. Run an agent in a managed sandbox and have it fill the disk (for
example a Rust toolchain plus a full dependency install).
2. `runnerd` exits when it next writes its state.
3. Watch recovery retry the relaunch upload until the grace expires.

**Expected behavior**
Paperclip relaunches its runner on a disk the agent filled, so the turn
continues and the agent sees the full disk in its tools. When the runner
truly cannot start, recovery says so at once with a named cause instead
of retrying for 7 minutes.

**Environment**
Managed Daytona sandbox (fleet snapshot, 4 CPU, 8 GiB memory, 10 GiB
disk), `paperclip_runner` native runtime.

## What Changed

- New `server/src/services/native-runtime/remote-runner-disk-reserve.ts`
owns the reserve: hold/release scripts, the full-disk classifier, and
`createRemoteRunnerDiskReserve`. A release waits for an in-flight hold,
and no hold starts after a release.
- Both remote runner launchers (agent runs and Dot) get a reserve path
in managed sandboxes only:
`<workspace>/.paperclip-runtime/paperclip-runner/disk-reserve`.
Workspace saves and Git already exclude `.paperclip-runtime`. SSH hosts
get no reserve.
- After the first launch's identity is confirmed, the launcher holds 256
MiB in the background, only while 2 GiB more is free. It is off the
startup path.
- A relaunch releases the reserve before it stages. A launch that hits a
full disk while staging releases the reserve and stages once more.
- Release tries twice and reports `released`, `not_held` or `failed`.
Only `not_held` ends the launch with `sandbox_disk_full`. A release that
cannot run rethrows the original error, so recovery still retries.
- Command-based staging (no `syncIn`) keeps a full disk recognizable
with an `ENOSPC` code instead of a bare
`runner_remote_*_staging_failed`.
- `SANDBOX_DISK_FULL_FAILURE_CODE` and `isSandboxDiskFullFailure` live
in the runner package. Runner process recovery checks the error code and
stops at once instead of retrying for the whole grace. Other staging
failures still retry.

## Verification

- Red before the fix, green after:
-
`server/src/services/native-runtime/remote-runner-disk-reserve.test.ts`:
release-and-restage, release before relaunch, `sandbox_disk_full` when
release frees too little, no retry for a failure that is not a full
disk, hold/release scripts in a real shell, no hold below the free-space
floor. Before the fix, the launch rejected with the exact upload error
from the incident.
- `packages/paperclip-runner/src/live/runnerd-codex-transport.test.ts`
"fails at once when a replacement cannot be staged because the sandbox
disk is full": before the fix it was still retrying after 4 s; after, it
fails with `sandbox_disk_full` after one replacement.
- Kept: "retries a replacement whose staging failed before any process
launched" still passes, so a raw transient staging failure still
retries.
- Updated: "confirms no replacement process exists when staging fails
before the launch RPC" now expects `sandbox_disk_full` with the upload
error as cause. Exit is still confirmed and no launch RPC runs.
- Real sandbox from the same fleet snapshot (Daytona, 10 GiB): hold
created a 256 MiB reserve. The disk was filled to 100%. A 5 MB write
then failed. Release returned `true`, and a 50 MB write then succeeded.
A second release returned `false`. Hold on the now 98% full disk did not
create a reserve.
- `pnpm --filter @paperclipai/server exec tsc --noEmit` passes. `pnpm
verify` result is in a comment below.
- Review round: tests for a lost release RPC, hold/release ordering,
command-based staging on a full disk, the hold after a dispatched first
launch (not on relaunch), and no reserve on SSH. Each launcher-level
test failed with its fix removed.
- Focused suites on the review revision: transport 335/335, launcher
130/130 (filtered), launch assets 35/35, reserve 14/14. Server and
runner typecheck pass.
- Not run: a full product run in which an agent fills the disk and the
relaunched `runnerd` resumes the turn end to end.

## Risks

- Each managed sandbox holds 256 MiB more disk while it has more than 2
GiB free. That is 2.5% of a 10 GiB sandbox.
- A relaunch for any reason releases the reserve for the rest of that
run. The next run's launch takes it again.
- If the agent keeps writing after the relaunch, the disk can fill
again. The second time, nothing is left to release, and the run fails at
once with `sandbox_disk_full` instead of after 7 minutes.
- The run's error code is still `native_provider_terminal_failed`. The
message now names `sandbox_disk_full`. A dedicated failure code and
recovery text are a follow-up.
- `runnerd` loses its own error message when it cannot persist its
diagnostic. This PR does not change the Rust binary.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and
code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 15:45:20 -06:00
Nicky LeachandClaude Opus 5.5 960a2b781e perf(workspaces): hold kept plugin workspaces instead of re-inspecting them every sweep (#22054)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Workspace storage plugins hold task workspaces, and Core's terminal
workspace sweep removes them after their tasks finish
> - The sweep inspects each candidate workspace through the provider. A
provider inspection can start a sandbox or call a remote API, so each
one has a real cost
> - When an inspection fails, the sweep backs off. When an inspection
succeeds but keeps the workspace (local data, or work that is not
merged), there is no backoff, so the sweep inspects the same workspace
again on every pass, forever
> - Kept workspaces also kept a past-due sweep deadline, so targeted
sweeps came back to them every few minutes, not only once per hourly
rotation
> - With many closed workspaces, this repeated work grows without limit
and gives the same answer each time
> - This pull request records a hold after an inspection keeps a
provider workspace, and skips the provider inspection while nothing that
could change the answer has changed
> - The benefit is far fewer provider operations for kept workspaces,
with no delay for workspaces that can be removed

## Linked Issues or Issue Description

**Subsystem affected**
Server: the terminal workspace sweep for workspace storage plugins
(`server/src/services/execution-workspaces.ts`).

**Problem or motivation**
The terminal workspace sweep inspects every kept plugin workspace again
on every pass. For a workspace kept for local data, the answer cannot
change until something changes the workspace. For a clean, unmerged
workspace, the answer changes only when the remote changes. Each
repeated inspection is a provider operation, which can start a sandbox
or call a remote API.

**Proposed solution**
After an inspection keeps a plugin workspace, Core records a hold. A
workspace kept for local data is not inspected again until it or its
task tree changes, or a change notice names it. A clean, unmerged
workspace is checked again after 1 hour, then on an interval that
doubles up to 24 hours. Workspaces that can be removed are removed as
before.

**Alternatives considered**
An in-memory hold only. It is lost on restart, and the existing
in-memory inspection cache holds only 256 entries, so a large set of
kept workspaces would be inspected again after every restart.

## What Changed

- `server/src/services/terminal-workspace-retry.ts`: add the
`terminalInspectionHold` workspace metadata record (generation,
fingerprint, reason, HEAD, check count, check time, next check time).
The write is fenced on the lifecycle generation. Add the recheck delay:
1 hour, doubling, capped at 24 hours.
- `server/src/services/execution-workspaces.ts`
(`sweepTerminalWorkspaces`):
- Before it inspects a plugin workspace, the sweep compares a
fingerprint of the inputs that can change the answer: the row's
`updatedAt`, status, provider ref, repository, branch, base ref,
provider binding, delivery candidate, and the task tree's terminal
state. The lifecycle generation scopes the record. While the hold
applies, the sweep skips the inspection and counts the workspace as
`skippedBackoff`.
- After an inspection keeps the workspace, the sweep records a
`local_data` hold (dirty or untracked files, or local-only data; no
recheck time) or an `unmerged` hold (growing recheck interval). A new
inspected HEAD, a new fingerprint, or a change notice restarts the
interval.
- A kept workspace now gets a sweep deadline at its recheck time, or
none. Before, it kept its past-due archive deadline, so targeted sweeps
came back to it every few minutes.
- A provider refresh notice (`plugin-workspaces`: activation, upgrade,
capability change, company re-enablement) ends every hold, because it
can change how any plugin workspace inspects.
- A `workspace:` or `issue:` change notice ends the holds that it names
(the workspace, or a workspace whose task tree or linked tasks contain
the issue). A notice without a scope ends every hold. The next sweep
visit clears an ended hold from the row, so the reset survives a
restart. The in-memory notice entries are pruned after the next full
rotation.
- When an inspection finds the workspace removable, the sweep clears the
hold and continues to removal as before.
- The hold applies only to plugin workspaces. Local Git worktree
inspections are cheap, and a person can change those folders directly,
so they keep the current behavior.
- Core owns the new metadata key. Workspace metadata updates keep it,
like `terminalCleanupRetry`.
- `doc/plugins/PLUGIN_RUNNER_EXECUTION.md`: describe the hold for
provider authors.
- Optional item not done: `removePluginWorkspace` still inspects again
before it removes. That inspection runs right before a destructive
operation, passes the recovery PR hint to the provider, and checks
`remotelyRecoverable`, which the sweep's inspection result does not
carry. Skipping it is not clearly safe.

## Verification

- `npx vitest run
src/__tests__/plugin-workspace-terminal-hold.integration.test.ts` (new,
embedded Postgres):
- Local-data hold: no inspection after 30 days; a workspace notice, an
issue notice, a generation bump, a metadata update, a new task
completion time, and a provider refresh notice each cause exactly one
new inspection; no sweep deadline is left. The provider refresh step
fails without its fix (6 inspections, not 7).
- Unmerged backoff: recheck delays are 1, 2, 4, 8, 16, 24, 24 hours; a
new HEAD resets to 1 hour; a workspace notice resets to 1 hour.
- Removable workspaces: a clean merged workspace is removed on its first
sweep (inspection count unchanged at 2); a held dirty workspace is
removed on the next sweep after a notice, and the hold is cleared; a
closed held workspace is removed after an unscoped notice.
- Failure backoff: delays stay 5 and 10 minutes, and a failed inspection
does not record a hold.
- Red check: the five new tests fail against the unchanged service and
pass with the change.
- Related suites pass locally (10 files, 204 tests):
`execution-workspaces-service`, `plugin-workspaces.integration`,
`terminal-workspace-deadlines`, `terminal-workspace-idle`,
`terminal-workspace-page-batch`, `terminal-workspace-rotation-limits`,
`terminal-workspace-work`, `workspace-delivery-cleanup.integration`,
`plugin-workspace-terminal-retry.integration`, and the new file.
- A scratch check confirmed the old deadline behavior: after a sweep
kept a workspace, the old code left it due with a past deadline. The new
code leaves no deadline.
- `plugin-workspace-terminal-retry.integration.test.ts`: one step now
sends a workspace notice before it checks that a new failure restarts
the backoff at 1, because the preceding successful inspection now holds
the dirty workspace.
- `cd server && npx tsc --noEmit -p .` passes.

## Risks

- A held `local_data` workspace is inspected again only after a change.
If a change notice is lost (for example a restart between the commit and
the notice) and the change does not touch the workspace row, its
generation, binding, status, delivery candidate, or task tree terminal
state, the workspace stays held until one of those changes. Runs end
with an `issue:` notice and new task completion times, so the normal
paths are covered.
- An `unmerged` workspace that becomes merged on the remote waits until
its next recheck (at most 24 hours), not the next hourly rotation.
- Each sweep page now runs one extra query for the linked tasks of held
workspaces. Each kept workspace gets one metadata write per inspection.
- No schema migration. The record is in existing workspace metadata.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 13:40:35 -08:00
Nicky Leach 3a41e54c51 perf(runner): start ACPX once on parked checkpoint relaunch (#22064)
## Thinking Path

> - Paperclip manages work for AI agents.
> - Runner uses ACPX to resume provider sessions from durable
checkpoints.
> - A parked Runner can be gone after a reboot or retention eviction.
The host can then verify the saved park checkpoint and start a
replacement process.
> - The relaunch currently opens a sidecar for source-run readiness and
opens another for the incoming run.
> - The replacement Runner keeps the source-run command fence until
authenticated attachment.
> - Runner can open the sidecar once with the incoming run's bindings
and verify dormancy before it accepts the attachment.

## Linked Issues or Issue Description

Refs #19912 for ACPX run and tool authority design.

**What existing behavior does this improve?**

ACPX follow-up when the parked Runner process is gone and the host
starts a replacement from its verified park checkpoint.

**Subsystem affected**

Native Runner hosting and the ACPX provider executor.

**Current behavior**

The readiness probe and the idle event poll both enter generic recovery.
Recovery opens the source-run sidecar. Attachment then closes it and
opens a new sidecar for the incoming run.

**Proposed behavior**

A host can signal this path through private Runner control after
verifying an earlier ACPX park checkpoint and confirming that the old
execution has stopped. The replacement Runner checks provider
eligibility and checkpoint settlement, serves readiness from saved
state, and fences commands until authenticated `run.attach` starts the
provider with the incoming run’s bindings. A live frozen Runner follows
the existing thaw and claim flow.

**Reason and benefit**

This removes a sidecar startup and shutdown from the time to the next
ACPX turn while preserving the conversation and applying the incoming
run’s tool, instruction, and workspace bindings.

**Breaking changes**

The launch argument is optional and requires private host control in
listener mode. The feature reads checkpoints from the previous release.
No checkpoint or controller message format changes.

## What Changed

- Keep a relaunched park within the existing host claim command fence.
- Load a settled provider checkpoint that qualifies for host parking
without opening the provider during readiness or polling.
- Open the provider with the incoming run's settings. Verify the session
identity and that it has no active work before publishing the new
provider checkpoint.
- Verify one sidecar start, preservation of the saved checkpoint on
failed admission, and rejection of early turn execution.

## Verification

- `cargo fmt --manifest-path packages/paperclip-runner/runner/Cargo.toml
--all --check`
- `cargo test --offline --locked -p paperclip-runner-core --lib --bins`
passed locally (673 passed, 4 ignored).
- The confined sidecar regression uses previous-release checkpoints for
two ACPX providers. It verifies a single start on attachment for both
and preserves the saved checkpoint when the live provider reports an
active goal.
- The authenticated loopback test restarts with a saved incoming
provider checkpoint and a prepared authority receipt. It verifies
receipt replay, rejection of the prior authority after activation, and
command execution under the incoming run.
- `cargo test --offline --locked -p paperclip-runner-core --test
acpx_warm_attachment --test acpx_park_readiness --test durable_recovery`
passed locally (19 passed).
- Hosted CI passed. Greptile scored the current commit 5/5 with no
unresolved review threads.

## Risks

- An incorrect checkpoint could cross a run boundary. The private host
signal requires settled Runner and ACPX provider checkpoints. Runner
verifies live session identity and dormancy before transferring
authority.
- The existing transition receipt handles reconnects after attachment.
The optional launch signal has no effect on older Runner binaries.

## Model Used

- OpenAI GPT-6 (Codex), with source analysis and local Rust process and
protocol tests.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have documented the private launch contract and compatibility
above
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-11-17 13:32:28 -08:00
DottaandClaude Opus 5.5 b56f240f45 feat(analytics): show session replay text by default, mask only customer content (#22057)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Hosted Paperclip Cloud records browser sessions with PostHog session
replay
> - Replay masked all text and showed only an allowlist of chrome, so
most replays showed only stars
> - The team cannot see how people use the board, and the allowlist
needed a review for each new label
> - The owner decided to show text by default and mask only marked
customer content, the way PostHog normally works
> - This pull request flips the default, removes the allowlist, and
marks every surface that can show connected-account content
> - The benefit is readable replays, while Google user data and Slack
data stay masked

## Linked Issues or Issue Description

**Subsystem affected**

Hosted Cloud browser analytics: PostHog session replay in the board UI
(`ui/src/lib/analytics/`).

**Problem or motivation**

Replay masked every text node. Only sidebar items and opt-in components
showed text. Replays were almost unreadable. Each new label needed an
`analyticsChrome` opt-in and an allowlist review.

**Proposed solution**

Record text as it is. Mask text only inside `ph-mask` (customer
content), `ph-no-capture` (secrets, blocked) and editable content. Add
`ANALYTICS_MASK_CLASS` to every surface that can show connected-account
content (Gmail, Calendar, Drive and Docs, Slack) or other free-form text
from agents or customers.

**Alternatives considered**

Keep "mask all" and grow the allowlist. This needs a review for each
label and still hides most of the board.

## What Changed

- `privacy.ts`: `ANALYTICS_MASK_TEXT_SELECTOR` is now `.ph-mask,
.ph-no-capture, [contenteditable]:not([contenteditable="false"])`. rrweb
masks a text node only when its parent or an ancestor matches the
selector. Removed `maskReplayText`, `isReplayTextRevealed`,
`ANALYTICS_UNMASK_ATTRIBUTE` and `ANALYTICS_UNMASK_PROPS`.
- `product-analytics.ts`: removed `maskTextFn`, so rrweb uses its
default star masking. `maskAllInputs`, `blockClass`, `maskAttributeFn`,
`mask_all_text` and `mask_all_element_attributes` do not change.
- Removed the `analyticsChrome` prop from `Button`, `Label`,
`TabsTrigger`, `EmptyState` and `SidebarNavItem`, and from all sidebar
call sites.
- Added `ANALYTICS_MASK_CLASS` to the content element (not the chrome)
of these surfaces:
- Link preview cards, task quicklook description, external object pills
and rows, related-work titles.
  - Email message cards and email send intents.
- Connected account names and email addresses (Browse, Connections, app
detail headers, app sidebar, connection choice list, connection access
and repair cards), GitHub login and repositories, Slack workspace names,
chat channel and linked user labels.
- Inbox and decision rows (subject and detail), obligation evidence,
Needs You notes and receipts, decision options and result summaries, and
the interaction card roots (questions, options, answers, tool action
requests, suggested tasks).
- Approval payloads, action card input and reason, recovery card
headline, evidence and options, run ledger retry detail, interrupt
handoff previews.
- Run transcripts, run failure details and run events on the agent page.
- Search result snippets, toast bodies, voice and call transcripts, case
summary, files and fields, attachment previews, artifact previews (text,
tables, patches and documents).
- Replaced `unmask-allowlist.test.ts` with `replay-masking.test.tsx`. It
checks that unmarked chrome records as it is, that each marker masks,
that a rendered voice transcript and run transcript mask, that each
surface above has the class, and that autocapture records no element
text. Updated `privacy.test.ts`, `shared-chrome.test.tsx` and
`product-analytics.test.ts`.
- Updated `doc/cloud-browser-analytics.md`: new default, marker classes,
masked surfaces, and residual risk.

## Verification

- `pnpm exec vitest run src/lib/analytics` in `ui/`: 83 tests pass.
- Vitest for the 48 test files next to the changed components: 994 tests
pass.
- `tsc --noEmit -p ui/tsconfig.json`: no new errors (only the existing
generated Storybook catalog errors).
- `pnpm verify` exits 2 because a doc is a global CI input. Required CI
supplies the broad coverage.
- Manual: I ran an isolated local instance with a Cloud-like browser
analytics config. `PAPERCLIP_POSTHOG_HOST` pointed to a local capture
server, not PostHog. I opened a task page in headless Chromium, hovered
a GitHub pull request link, and decoded the captured `/s/` replay
payloads:
- Clear text: sidebar (`New task`, `Chat`, `Tasks`, `Inbox`, `More`),
breadcrumb, task title `Quarterly launch checklist`, `REP-1`, the
Connectors promo heading and button.
- Masked (stars only): the markdown task body, the chat comment, the
link preview card (pull request title, author, URL, status), and the
company menu.
- Marker strings in the body, comment and preview never appeared in
clear text in any payload.
- The page `<title>` (task title and company name) is in the replay
head. This is accepted residual risk.

## Risks

- Text that is not marked is now recorded. Task titles, agent names,
project names, company names, routine names, artifact and document
titles, and activity rows are visible in replays. The owner accepted
this. An agent can copy email or Slack text into a task title.
- A new surface that shows connected-account content without
`ANALYTICS_MASK_CLASS` is recorded until somebody marks it.
`replay-masking.test.tsx` and the doc list the marked surfaces.
- The masking options in `posthog.init` take precedence over the PostHog
project masking settings (posthog-js 1.450), so a dashboard change
cannot unmask content. Keep the project settings empty to avoid a
console warning.
- Some masked areas also mask nearby fixed labels (for example the
interaction card buttons). This is safe.
- Analytics is off on self-hosted and local installs, so they see no
change.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (claude-opus-5-5), Anthropic, with tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 15:18:50 -06:00
Devin FoleyandPaperclip 84b75502c0 fix(ssh): support ACP with durable Cloud workspaces (#22063)
## Thinking Path

> - Paperclip coordinates agents that run on local and remote computers.
> - SSH environments use the harness installed on a computer the
operator controls.
> - Cloud SSH tasks now use durable remote workspaces.
> - Claude and Codex default to ACP, but ACP rejects those in-place SSH
workspaces.
> - The shared ACP engine already has SSH process bridges and managed
account delivery.
> - This pull request lets that engine use the remote task directory and
stage runtime assets separately.
> - The benefit is ACP execution with the selected Paperclip account and
the existing SSH environment.

## Linked Issues or Issue Description

**What happened?**

A Claude or Codex agent using the default ACP engine cannot run in a
volume-free Cloud SSH environment. The adapter rejects its in-place
workspace before execution. Setting the engine to CLI works around that
restriction.

**Expected behavior**

ACP runs directly in the authoritative remote task directory. Paperclip
keeps using the selected account, permission settings, process bridges,
and durable workspace owner.

**Steps to reproduce**

1. Create an SSH environment with agent-managed authentication on a
volume-free server.
2. Select a Paperclip AI account for a Claude or Codex agent and leave
its engine at the default.
3. Assign a task to the agent.
4. Observe the ACP in-place workspace rejection.

Related: #21957 adds durable Cloud SSH workspaces. #21248 adds
agent-selected SSH accounts. A search for SSH ACP work found #21248 and
no open duplicate. ROADMAP.md has no overlapping item.

## What Changed

- Admit managed SSH ACP with an in-place workspace in both adapters.
- Keep ACP's session directory and fingerprint bound to the remote task
directory.
- Stage private runtime assets with workspace copying disabled. The
remote workspace owner still saves task files.
- Retain the copy-mode SSH path and existing account, confinement, and
command availability checks. No engine fallback or permission change is
added.
- Extend the existing lifecycle tests for in-place staging, session
reuse, current credentials, account changes, and cleanup after bridge or
turn failures.
- Extend the real OpenSSH/Postgres heartbeat test to the default ACP
engine. Its protocol fixture creates a file and verifies conversation
history on a follow-up.
- Update the SSH setup documentation.

Workspace placement belongs to the shared ACP engine. The adapter
changes remove their explicit SSH admission refusals. The server test
exercises the existing heartbeat path; no server execution logic
changes. Existing large test files retain their shared harnesses instead
of adding a second lifecycle mock graph.

## Verification

- 418 focused tests pass across 7 suites: 411 adapter/ACP lifecycle
tests, all 4 real SSH workspace tests, and 3 real ACP host-spawn smoke
tests.
- Typechecks pass for adapter-utils, Claude, Codex, and the server.
- `pnpm verify` returns 2 because this touches shared runtime code and
deployment documentation. Required CI supplies the broader build,
browser, and release coverage.
- Tested head: `33b17a9a21c0a5929523702ebed9c8faa3b22955`.
- `git diff --check` and the local secret/PII scan pass. Greptile
reviewed the tested head at 5/5 with no actionable findings. Final-head
[CI attempt
2](https://github.com/paperclipai/paperclip/actions/runs/44167483231/attempts/2)
passes on `33b17a9a21c0a5929523702ebed9c8faa3b22955`: 58 checks pass and
6 are skipped, with no failures or pending checks. Both required gates
pass. Four first-attempt jobs were cancelled after their runners
received shutdown signals, confirmed in each job log; automatic recovery
reran those jobs successfully. No source change was needed for that
recovery. Greptile is 5/5 with no unresolved findings, and the branch
has no merge conflicts. GitHub still requires code-owner review before
merge.

```sh
pnpm exec vitest run --maxWorkers=2 \
  packages/adapter-utils/src/acpx-engine/execute.test.ts \
  packages/adapters/claude-local/src/server/execute.remote.test.ts \
  packages/adapters/codex-local/src/server/execute.remote.test.ts \
  packages/adapters/claude-local/src/server/acp.test.ts \
  packages/adapters/codex-local/src/server/acp.test.ts
PAPERCLIP_ENABLE_DARWIN_SSH_ENV_LAB=1 pnpm exec vitest run --maxWorkers=2 \
  server/src/services/ssh-remote-workspace.integration.test.ts
pnpm exec vitest run packages/adapter-utils/src/acpx-engine/remote-spawn-smoke.test.ts
pnpm --filter @paperclipai/adapter-utils typecheck
pnpm --filter @paperclipai/adapter-claude-local typecheck
pnpm --filter @paperclipai/adapter-codex-local typecheck
pnpm --filter @paperclipai/server typecheck
```

The new admission and staging cases failed before the source fix. The
first whole-heartbeat follow-up test also changed an agent setting
between turns. That correctly reset the session. The fixture now keeps
those settings unchanged and verifies the resumed conversation. Running
both engine cases together also exposed duplicate company prefixes and
environment names in the fixture; each case now uses its own names.

The SSH qualification uses a real isolated OpenSSH server, real
Postgres, real ACPX runtime and bridges, and a synthetic ACP provider
with a selected test account. It does not call a live provider or run an
agent on a user's computer.

## Risks

The change affects the shared ACP staging path, restricted to SSH
targets that declare in-place realization. Sandbox staging retains its
prior behavior, including the existing Codex in-place refusal. Copy-mode
SSH sessions retain their prior directory layout. Previously saved
sessions keep their format; this adds no migration or session reset.

ACP still requires its server command on the remote login PATH. Host
authentication and local confinement remain CLI-only. Cold
controller/provider reconnection and live Claude/Codex provider
acceptance are not qualified by this test. No staging deployment or
existing agent configuration is changed by this pull request.

## Model Used

OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) through the Codex harness, with xhigh
reasoning effort and code tools.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
canary/v2026.1117.0-canary.821900
2026-11-17 13:18:26 -08:00
DottaandPaperclip 9911b7052f Expose continuation retry keys and preserve account-repair resumes (#22046)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Runner gives agents a tool to report work and wait for a later
check.
> - That tool requires a stable retry key for each explicit wait.
> - Codex can hide the wait fields when it renders our nested schema.
> - Hidden fields make a correct report harder to construct.
> - This pull request changes the provider schema structure and
preserves its rules.
> - Agents can see the required fields before they submit a report.

## Linked Issues or Issue Description

**What happened?**

The completion tool appears in Codex code mode with `continuation?:
unknown`. An agent can omit the required retry key and receive a
rejected report. The top-level schema was fixed in #21398. The nested
continuation still uses the same problematic structure.

**Expected behavior**

Codex must show the required `kind`, `summary`, and `idempotencyKey`
fields. Invalid reports must still fail validation.

**Steps to reproduce**

Render `PRP_COMPLETION_RESULT_PROVIDER_INPUT_SCHEMA` with the public
Codex code-mode renderer at `b271854568d6c9282e999d6f36e5e85f447d027d`.
Its `render_map` function processes `allOf` before sibling properties.
The conditional-only branch renders as `unknown` and hides those
properties.

Source:
https://github.com/openai/codex/blob/b271854568d6c9282e999d6f36e5e85f447d027d/codex-rs/code-mode-protocol/src/json_schema_types.rs#L127

**Paperclip version or commit**

Baseline: `01451fae5b25f784956e9932f12031dc194ac247`.

The first complete CI browser attempt also exposed an intermittent
account-repair race. Repair restores the task and resolves its recovery
action, then a sweep can reclassify the same old failure and block it
again before the queued reply starts. The repair in this PR honors that
exact persisted resolution while holding the existing task lock. A later
failed run still escalates.

## What Changed

- Express the single provider continuation condition as direct `if` and
`then` fields. This is equivalent to its previous `allOf` wrapper.
- Keep the canonical output schema unchanged. Keep required retry keys
and case handoff IDs.
- Add a discovery guard, a 60-case provider/canonical validation
comparison, and authenticated HTTP bridge checks.

### Hypothesis and thesis for a fix

The nested `allOf` hides fields in the tested Codex renderer. The
equivalent direct condition lets that renderer reach the object fields.
This belongs in the existing provider compatibility schema. It changes
no controller policy, company access, saved state, or retry identity.
The experiment proves field visibility. It does not prove which renderer
served a historical request or measure model omission rates.

- Honor a resolved/restored configuration recovery for the same company,
task and failed run before re-escalating. Dispatch checks and recovery
for new failures remain intact.
- Add a PostgreSQL regression covering repeat sweeps, a new failure
identity, an unresolved recovery, and another task's repair. This
belongs in the existing locked escalation function; no new state or
subsystem is introduced. `service.ts` was already over 1,000 lines.

- Define the handoff condition once and explicitly compose both
canonical and provider schemas from it, removing the implicit
one-element `allOf` assumption raised in review.

## Verification

- Built and ran the exact pinned public Codex renderer in a disposable
Rust crate. The complete baseline schema renders `continuation?:
unknown`. The changed schema renders a nullable object with required
`kind`, `summary`, and `idempotencyKey` fields.
- The new discovery regression fails on the baseline. The same test
passes with the fix.
- Runner: `pnpm exec vitest run src/contracts/completion-result.test.ts
src/contracts/tool-input-variants.test.ts
src/contracts/provider-tool-schema-shape.test.ts
src/drivers/runner-tool-bridge.schemas.test.ts
src/drivers/runner-tool-bridge.continuation.test.ts` — 48 tests pass.
- Runner: `pnpm exec tsc -p tsconfig.json` and `node
scripts/generate-semantic-contracts.mjs --check` pass.
- Server: the selected `task-monitor.integration.test.ts` test for
unfinished monitor waits passes with embedded PostgreSQL. It rejects an
unscheduled wait and accepts the saved monitor.
- Server: the selected `issue-monitor-scheduler.test.ts` recovery test
passes with embedded PostgreSQL. It checks persistence, same-key replay,
and one scheduled wake. The fixture's successor run fails its separate
responsible-user setup. This is not proof of successful successor
execution.
- HTTP bridge handler is simulated. It verifies rejection before
dispatch, valid monitor/response-wake/handoff reports, retry identity,
and authentication.
- Recovery regression: baseline 1 failed / 3 passed (the restored task
was blocked again); unchanged test after the fix 4 passed. Together with
recovery state-read tests: 7 passed. Existing configuration-recovery
selection: 4 passed, 108 deselected.
- Final completion schema and authenticated HTTP bridge selection: 33
passed.
- Local real browser journey with disposable embedded PostgreSQL and
ACP/provider fixtures: baseline repeat5 produced 2 failures / 3 passes,
matching CI. Fixed repeat5: 5 passed in 1.8 minutes. The test
independently checks exactly one successful successor, stored connection
binding and persisted reply after reload.
- No full local suite, deployed recovery, or probabilistic model result
is claimed. Fresh CI and review are required for the updated head.

- Review follow-up: 48 focused schema/discovery/HTTP tests pass; `node
scripts/generate-semantic-contracts.mjs --check` passes. Fresh CI/review
required after the refactor.

## Risks

Low runtime risk. The accepted input rules are unchanged. A provider
that handles direct conditionals differently could expose a
compatibility issue. The canonical validator still enforces the output
contract. No migration or saved-session reset is needed. Previously
saved state keeps the same format. Recovery now honors existing
resolved/restored actions only for the exact failed source run, so a new
configuration failure remains eligible for escalation. Company, task,
ownership and dispatch checks remain in force.

## Model Used

OpenAI GPT-6 Codex agent, with reasoning, tool use, and code execution.
The runtime did not expose a more specific model ID.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
canary/v2026.1117.0-canary.821800
2026-11-17 14:23:19 -06:00
DottaandClaude Opus 5.5 fa8bc78d2a fix(posthog-proxy): forward the visitor IP behind a private proxy hop (#22052)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Paperclip Cloud sends browser analytics to PostHog through a
same-origin proxy at `/_ph`
> - On Cloud, the server runs behind Railway's private proxy hop and
`TRUST_PROXY` is not set
> - So the proxy forwarded the hop address (100.64.x.x) to PostHog
instead of the visitor's IP
> - PostHog could not resolve a location, and every visitor shared one
per-client in-flight bucket
> - This pull request makes the proxy use the first `X-Forwarded-For`
entry when the socket peer is private
> - The benefit is correct visitor location in PostHog web analytics and
per-visitor limits that work on Cloud

## Linked Issues or Issue Description

**What happened?**
On Paperclip Cloud staging, every browser event that reaches PostHog
through the `/_ph` proxy has `$ip` set to a Railway private address
(100.64.0.5 to 100.64.0.21). PostHog shows no country or city. All
visitors also share a few per-client in-flight buckets.

**Expected behavior**
PostHog receives the visitor's own IP, so web analytics shows visitor
location. Each visitor gets its own in-flight bucket.

**Steps to reproduce**
1. Turn on browser analytics on a Cloud stack
(`PAPERCLIP_POSTHOG_BROWSER_ENABLED=1`) without `TRUST_PROXY`.
2. Load the board and navigate a few pages.
3. In PostHog, query `properties.$ip` for those `$pageview` events. The
value is a 100.64.x.x address and there is no geo data.

**Deployment mode**
Paperclip Cloud (authenticated, hosted on Railway).

## What Changed

- Add `posthogVisitorIp()` in `server/src/routes/posthog-proxy.ts`.
- Use it for the forwarded `X-Forwarded-For` header and for the
per-visitor in-flight limit.
- Add tests for the helper and for the forwarded header.

## Verification

- `npx vitest run src/__tests__/cloud-posthog-browser.test.ts` (server):
51 pass.
- Red check: with the old proxy code, the new tests fail (11 failures).
- Reproduced on paperclip.staging.paperclip.app: PostHog `$ip` values
were 100.64.0.5 to 100.64.0.21, with no location.

## Risks

- A client can set its own `X-Forwarded-For`. This only changes its own
analytics location and its per-visitor bucket. The global in-flight
limit still applies.
- The proxy is mounted only on Cloud with browser analytics on, so
self-hosted installs are not affected.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 14:18:45 -06:00
12635af273 fix(ui): skip the GitHub star prompt for established companies (#22031)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The self-hosted board asks people to star GitHub after five
completed tasks.
> - The current prompt checks only whether five completed tasks exist.
> - This also interrupts established companies on their first visit
after an update.
> - This pull request records whether a company was first observed below
five tasks.
> - Eligible companies still receive the prompt when they reach five
tasks.

## Linked Issues or Issue Description

**What happened?**

The GitHub star modal appears for established companies with five or
more completed tasks. The query returns at most five tasks, so a company
with 100 completed tasks also meets the current trigger.

**Expected behavior**

Only companies first observed below five completed tasks should qualify.
Companies already at or above five should stay excluded.

**Steps to reproduce**

1. Use a self-hosted company with at least five completed tasks.
2. Open the board in a browser profile that has not shown the GitHub
star prompt.
3. Open the task list. The modal appears even though the milestone
happened earlier.

**Paperclip version or commit**

Reproduced from `cefef8f11d67`. The fix is rebased on `9c776cdf39`.

**Deployment mode**

Self-hosted server. This is a board UI bug and does not depend on an
agent adapter.

Related merged work: [initial star
prompt](https://github.com/paperclipai/paperclip/pull/20453) and [task
interruption
guards](https://github.com/paperclipai/paperclip/pull/20713). A search
for related issues and PRs found no open duplicate for milestone
eligibility. This small bug fix does not overlap planned roadmap work.

## What Changed

- Record eligibility from the first successful completed-task read.
First reads at five or more do not qualify.
- Save that decision per company in browser storage. Preserve
eligibility across reloads and company switches.
- Keep the existing cloud, onboarding, task-focus, retry, other-dialog,
and one-time display guards.
- Add regression coverage for established companies, milestone
completion, return visits, company switches, and failed reads. Document
the behavior in `doc/PRODUCT.md`.

## Verification

- The new regression tests failed on the old implementation for initial
counts of five and 100, and for a company switch.
- After the rebase, `cd ui && pnpm exec vitest run
src/components/GitHubStarPrompt.test.tsx` passed all 14 tests on
`beaa2f1c86e5c5281f8bc7f3914e54625a3f4767` on macOS with Node.js 26.4.0.
- An earlier `pnpm verify` attempt passed 505 UI test files. CLI
fixtures failed because the sandbox denied loopback listeners (`listen
EPERM`). All five blocked CLI suites passed on a focused rerun with
loopback access: 104 tests.
- Local `pnpm verify` exited 2 because `doc/PRODUCT.md` requires broad
CI coverage. The focused modal suite passed. The final CI run supplies
that broad coverage.
- CI on `beaa2f1c86e5c5281f8bc7f3914e54625a3f4767`: [Typecheck + Release
Registry](https://github.com/paperclipai/paperclip/actions/runs/44132073335/job/134211999908),
[Build](https://github.com/paperclipai/paperclip/actions/runs/44132073335/job/134211999799),
[Canary Dry
Run](https://github.com/paperclipai/paperclip/actions/runs/44132073335/job/134211999904),
all eight browser shards, all nine serialized-server suites, all
general-test shards, and the final [CI verification
gate](https://github.com/paperclipai/paperclip/actions/runs/44132073335/job/134264789640)
passed on GitHub-hosted Ubuntu runners. The PR has 59 passing checks and
three intentional skips, with no pending or failing checks. See the [CI
run](https://github.com/paperclipai/paperclip/actions/runs/44132073335).
- The [first server-shard 7
attempt](https://github.com/paperclipai/paperclip/actions/runs/44132073335/job/134212002338)
failed in the unchanged `issue-recovery-live-run-agent-census.test.ts`.
A background delivery pass did not settle within the fixture's 20-second
bound. All four tests in that file passed in an isolated local run in
67.46 seconds. A [single targeted CI
rerun](https://github.com/paperclipai/paperclip/actions/runs/44132073335/job/134257768504)
then passed without source changes. This proves recovery; the cause of
the earlier timeout is not established. The failed attempt remains part
of this record.
- [Greptile
review](https://github.com/paperclipai/paperclip/pull/22031#issuecomment-6955022258):
5/5 on the same revision, with no inline findings.
- `git diff --check` passed.

## Risks

Low risk. The change affects only an optional prompt. Eligibility
remains local to a browser and company. It does not sync between
devices. If browser storage is unavailable, the prompt stays hidden.
Companies first observed at or above five stay excluded even if tasks
are later reopened.

## Model Used

OpenAI GPT-6 through Codex. The exact deployment ID and context window
were not exposed. Used reasoning, terminal tools, and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Scott Tong <scott@scottsmbpm5max.lan>
Co-authored-by: Paperclip <noreply@paperclip.ing>
canary/v2026.1117.0-canary.821600
2026-11-17 11:38:25 -08:00
DottaandClaude Opus 5.5 56023fe80f feat(google-workspace): gateway sets the REST tool result size limit for all six services; update connector docs (#22050)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents read Gmail, Calendar, Drive, Docs, Sheets and Slides through
the built-in Google Workspace connection
> - Paperclip now has its own REST-backed Google tools in
`server/src/google-workspace-rest-tools/`. No service uses them yet
> - That folder guessed a result size limit of 250,000 bytes. It applied
the limit only to Drive, Docs, Sheets and Slides. Gmail and Calendar had
no limit
> - The guess was wrong in both directions. It refused documents that
Paperclip can deliver, and the folder should not know Paperclip's limits
> - This pull request makes the limit a number that the gateway gives.
The folder applies it to every tool in all six services through one
check. It also brings three Google planning docs up to date
> - The benefit is that a REST result is refused only when the gateway
would also refuse the same result from Google's own MCP server

## Linked Issues or Issue Description

Refs #22027, #22028, #22029, #22026

**Is your feature request related to a problem?**
The REST tools refused results over a size the folder chose itself
(250,000 bytes). Paperclip can deliver larger results: the runner saves
a large result as an artifact and gives the agent a pointer. Gmail and
Calendar results had no limit at all.

**Describe the solution you'd like**
The gateway gives the limit. The folder applies the given number to the
final serialized result of every tool, with one helper. A result over
the limit is an error that says how to ask for less. It is never cut
short.

**Describe alternatives you've considered**
A larger number chosen from the runner's 1 GiB capture limit. Google's
own transfer limit (20 MB per response) and memory use make that a poor
choice. Also, results would then differ between the REST path and
Google's MCP path.

### Limits a tool result passes on the way to an agent

| Point | Limit | What happens over it |
| --- | --- | --- |
| Google response read by `google-http.ts` | 20 MB
(`MAX_RESPONSE_BYTES`) | Tool error. Nothing returned |
| REST folder result (new) | the number the gateway passes: 999,885
bytes | Tool error with "read less" advice. The gateway returns
`tool_error` (502) |
| Gateway read of a remote MCP response (`MAX_REMOTE_MCP_RESPONSE_BYTES`
in `tool-gateway.ts`) | 1,000,000 bytes per JSON body or event-stream
message | `mcp_remote_response_too_large` (502). This is the "1,000,000
bytes are rejected" statement in `doc/prompt-injection-prevention.md`.
It applies to the Google MCP path only; the REST path has no HTTP read |
| Injection screening model check | 256 KiB model request | Not a
delivery limit. A larger result is checked by the built-in patterns only
|
| Gateway HTTP route (`POST /tool-gateway/.../tools/call`) | none in
Paperclip | `res.json(result)` |
| Runner (`captureRunnerToolResult`) | 500,000 bytes
(`RUNNER_TOOL_RESULT_MAX_BYTES`) | Saved as an artifact. The agent gets
a pointer and reads it with `read_tool_result` |
| Runner capture | 1 GiB (`RUNNER_API_RESPONSE_MAX_BYTES`) |
`SemanticToolOutcomeUnknownError` |

The gateway wraps the result as `{ content, data: { content,
structuredContent, ... } }`, so the payload occurs again. Measured: a
Gmail result of 676,459 bytes from the folder became 1,017,386 bytes
delivered (ratio 1.50). The ratio cannot be more than 2.

### The budget

`GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES` in `tool-gateway.ts` =
`MAX_REMOTE_MCP_RESPONSE_BYTES` less the JSON-RPC envelope and
event-stream framing (`event: message\ndata:
{"jsonrpc":"2.0","id":"paperclip-tool-<uuid>-version","result":...}`) =
**999,885 bytes**. A result that Google's MCP server could deliver
through the gateway is also delivered over REST, and the reverse. With
the wrapper, the largest delivered result is less than 2 MB. The runner
saves it as an artifact (far below 1 GiB). So the folder never refuses a
result that the runner can give as a pointer.

Practical limits at that budget (plain ASCII; escaped and non-ASCII
characters count more):

| Tool | Largest content |
| --- | --- |
| Gmail `get_message` | about 500 KB plain body, or about 250 KB each
for plain plus HTML |
| Calendar `get_event` | about 500 KB description |
| Drive `read_file_content` | about 500 KB text |
| Drive `download_file_content` | about 375 KB file bytes |
| Docs `read_doc` | about 240 KB document text |
| Slides `read_presentation` | about 250 KB text |
| Sheets `get_values` | about 500 KB cell text, at most 50,000 cells |

## What Changed

- `tool.ts`: `defineTool` takes `whenTooLarge` advice per read tool
(required for reads, refused for writes) and a `ToolLimits` argument. It
measures the final `CallToolResult` once (`withinResultBudget`) and
replaces an oversized result with a `too_large` tool error.
`FALLBACK_MAX_RESULT_BYTES` (250,000) applies only when a direct caller
gives no limit.
- `callGoogleWorkspaceRestTool` and `createGoogleWorkspaceRestServer`
take `maxResultBytes`. A value that is not a positive integer is refused
before any request.
- All 23 read tools state their advice: a smaller `pageSize` for list
tools, `messageFormat` or `view` for Gmail, a narrower range for Sheets,
`fields` for Slides. Write tools are not checked: a completed write is
never reported as an error, so an agent is not led to repeat it.
- Drive, Docs, Slides and Sheets no longer have their own byte checks
(`boundedStructuredResult`, `boundedResult`, `sizedResult`,
`MAX_CONTENT_BYTES` removed). Drive downloads stop reading at the given
limit. The Sheets 50,000-cell limit stays.
- Two Drive tool descriptions no longer state a fixed number.
- Adapter `google-workspace-rest-dispatch.ts`: `maxResultBytes` is a
required input and is passed through.
- `tool-gateway.ts`: new `GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES`,
passed on the REST branch.
- Folder README: new "Result size" section with the limits and costs;
old 250,000-byte text replaced.
- Docs: `doc/memory/google-connector-memory.md`, the production
checklist and the code-fix plan now list #22026 to #22029, the REST
status (36 of 36, none switched on), the November 17 decisions, the
known differences from Google's tools, open risks and next steps.

## Verification

- `cd server && npx vitest run src/google-workspace-rest-tools` passes
(53 files, 1,361 tests with the adapter test).
- New `result-budget.test.ts`: for Gmail (read, list), Calendar (list,
read), Drive, Docs, Slides and Sheets, a result just under a given
budget is returned and one just over is refused with the tool's advice.
Without a budget, the fallback applies; with a larger budget, the same
result is returned. Drive downloads use the given cap. A Gmail list
tells the caller to lower `pageSize`. A Gmail draft result over the
budget is still returned, not an error. `defineTool` refuses a read tool
without advice and a write tool with it.
- `npx vitest run src/services/google-workspace-rest-dispatch.test.ts`:
the adapter passes the gateway's number, and the folder applies exactly
that number.
- `npx vitest run src/__tests__/google-workspace-rest-transport.test.ts`
(24 tests): with Gmail on REST, a 330 KB email passes the gateway with
screening, is more than 500,000 bytes delivered, and
`captureRunnerToolResult` turns it into a `read_tool_result` pointer. A
520 KB email is refused with `tool_error` and the advice, after one
Google request.
- `npx tsc --noEmit -p server/tsconfig.json`: no errors.
- `pnpm verify --explain`: broader verification required (a doc under
`doc/` is a global CI input). CI supplies that coverage.
- Time per call: the check is one `JSON.stringify` of the result. About
5 µs for a 4 KB result, 18 µs for 42 KB, 0.24 ms for 700 KB. Drive,
Docs, Sheets and Slides had the same pass before, so they add nothing.
Gmail and Calendar add this one pass.

## Risks

- Gmail and Calendar results now have a limit. Direct callers that pass
no limit get the 250,000-byte fallback. The gateway always passes its
number.
- Drive, Docs, Slides and Sheets accept results up to about 4 times
larger than before through the gateway. Large results reach agents as
artifact pointers.
- Error text for oversized results changed to one shared form.
- No service is switched to REST, so users see no change today.

## Model Used

Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used in Claude Code with
tool use (shell, file edits, GitHub CLI) and extended reasoning.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 13:05:13 -06:00
DottaandClaude Opus 5.5 98ff5e002a feat(cloud): read-only durable-data eligibility for online enrollment (#22051)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Hosted Paperclip servers can move from a server with a persistent
volume to a volume-free server, which allows deploys without downtime
> - The online move is allowed only when no durable data still lives on
the local disk; Core checks this in `readEphemeralServerDataEligibility`
> - Cloud can only learn the result by calling the preparation endpoint,
and that call starts a real move when the server is eligible
> - So Cloud cannot safely survey many servers, or watch a background
migration make progress
> - This pull request adds a signed, read-only GET that runs the same
check and returns the blocker names
> - The benefit is that operators can see which data blocks each server,
and move only the servers that are ready

## Linked Issues or Issue Description

**What existing behavior does this improve?**
Cloud can see a server's durable-data eligibility only through `POST
/api/instance/ephemeral-enrollment/prepare`. A refused call returns the
blockers. An accepted call starts the online move.

**Subsystem affected**
Server: Cloud control endpoints and ephemeral enrollment
(`server/src/routes/ephemeral-enrollment.ts`,
`server/src/services/ephemeral-enrollment.ts`,
`server/src/middleware/cloud-control.ts`).

**Current behavior**
There is no way to read eligibility without the side effect of
preparation.

**Proposed behavior**
`GET /api/instance/ephemeral-enrollment/eligibility` accepts only a
Cloud control assertion for the new action
`ephemeral-enrollment:eligibility`. It returns `{ eligible, blockers,
ephemeralServer }` and writes nothing.

**Reason and benefit**
A rollout to many servers needs to know which servers are ready, and
which data owners block the others, before it starts any move.

**Breaking changes**
None. This adds one endpoint and one action. Existing actions are
unchanged.

## What Changed

- `routes/ephemeral-enrollment.ts`: new `GET
/instance/ephemeral-enrollment/eligibility`. It requires a verified
Cloud control claim for `ephemeral-enrollment:eligibility` and sets
`Cache-Control: no-store`.
- `services/ephemeral-enrollment.ts`: new
`readOnlineEphemeralEnrollmentEligibility`. It uses the same check as
preparation (`purpose: "startup"`) and returns only the eligibility
flag, the canonical blocker names, and whether this server is already
volume-free.
- `middleware/cloud-control.ts` and
`services/cloud-runtime-identity.ts`: register the endpoint and the
action. Each assertion stays bound to one action, so a prepare token
cannot read eligibility, and the reverse.
- `__tests__/openapi-routes.test.ts`: list the route with the other
Cloud-control endpoints that are not in the board OpenAPI document.

## Verification

- `npx vitest run src/routes/ephemeral-enrollment.test.ts
src/__tests__/openapi-routes.test.ts
src/services/ephemeral-enrollment.test.ts
src/services/cloud-runtime-identity.test.ts`: 83/83 pass.
- New route tests:
  - a signed read returns the result and does not call preparation;
  - an unsigned caller and a tenant instance admin get 403;
  - a prepare assertion on this path gets 401.
- The three new tests fail without the route (red) and pass with it
(green).
- `tsc --noEmit` reports no errors in the touched files. A full local
typecheck needs the Runner build, so CI covers it.

## Risks

- Low risk. The endpoint is read-only and reachable only with a Cloud
control assertion bound to this stack and this action.
- The check runs a few indexed reads and pages skill installations.
Cloud calls it on request, not on a timer.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 13:02:37 -06:00
github-actions[bot]andlockfile-bot b43a08201c chore(lockfile): refresh pnpm lockfiles (#22020)
## Thinking Path

> - Paperclip uses a dependency lock for reproducible builds.
> - Dependency changes on master need matching lock entries.
> - Paid eval reports also pin a reviewed offline correction.
> - Reports must install before paid model work starts.
> - This automated PR keeps the lock and report proof together.

## Linked Issues or Issue Description

**What happened?**

Dependency manifests on master changed after the last committed lock
refresh.

**Expected behavior**

Frozen installs must match those manifests. Eval reports must retain
matching offline lock proof.

**Steps to reproduce**

Run a frozen install before and after this lock refresh.

## What Changed

- Refresh the dependency lock from the manifests on master.
- Refresh the sandbox provider lockfiles from their manifests on master.
- Include the matching offline report correction and hashes.
- Resync the SDK public source proof's lock pins from the refreshed
lock.

## Verification

- Pinned pnpm resolution completed with lifecycle scripts disabled.
- The report proof refresh accepted the reviewed committed baseline.
- The workflow rejected changes outside the lock and report proof files.

## Risks

The lockfile bot merges this PR without CI or review when every added
package version is at least 3 days old on npm. Otherwise the PR waits
for a maintainer, and the daily refresh tries again. The bot merges only
the exact commit this workflow generated from manifests already reviewed
on master. Unknown report lock baselines remain blocked. This update
changes no permissions or paid enable gates.

## Model Used

None — generated by pnpm and the deterministic report-lock updater.

## Checklist

- [x] Thinking path and problem description included
- [x] Generated lock and matching report proof included
- [x] Release-age gate decides between bot merge and maintainer review

Co-authored-by: lockfile-bot <lockfile-bot@users.noreply.github.com>
canary/v2026.1117.0-canary.821300
2026-11-17 12:56:23 -06:00
DottaandClaude Opus 5.5 3c5b117cca fix(native): post the agent's reply when the result commit worker finishes the run (#22043)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A person sends a message on a task, an agent run starts, and the
agent answers with a task comment
> - A native run normally finishes in its live heartbeat, which commits
the result and posts the reply in one pass
> - The result commit worker also commits native results. It races the
live heartbeat as soon as the workspace save succeeds, and it often wins
> - That worker skipped reply presentation and left the reply to the
presentation worker, which often posted it many minutes later
> - The task showed its new status (for example "blocked") with no
reply, so the person thought the message did not run
> - This pull request makes the result commit worker post the reply in
the same pass
> - The benefit is that the reply appears when the run finishes,
whichever path finishes it

## Linked Issues or Issue Description

**What happened?**

A person commented on a task. A run started one second later, and the
agent wrote its answer in about one minute. The run finished and the
task changed to "blocked", but no reply appeared for 8.5 minutes. The
person checked the task and saw no answer.

**Expected behavior**

The agent's reply appears on the task when the run finishes.

**Steps to reproduce**

1. Run a native agent on a task. The agent reports a result (for example
`blocked`) with a final reply.
2. The live heartbeat saves the workspace. The result commit worker
commits the result before the heartbeat's own finalizer does.
3. The run is "succeeded" and the task status changes, but the task has
no reply.
4. The reply appears only when the presentation worker later picks up
the run.

**Evidence**

On one staging company, 135 recent succeeded runs had a reply comment:

| Path that finished the run | Runs | Reply delay after the run ended |
| --- | --- | --- |
| Live heartbeat (reply posted inline) | 90 | 1–16 s |
| Result commit worker (reply left to the presentation worker) | 45 | 0
s – 3,864 s; 29 of 45 took over 30 s |

All 45 slow-path runs have no live completion write (no exit code, no
log size). Traces confirm that the live `execution.native_finalize` and
`completion.*` steps did not run for these runs.

**Paperclip version or commit**

`master` at 904c01d1a3

**Deployment mode**

Hosted (staging cloud). The defect is in server finalization, so it
applies to all deployment modes that use the native runtime.

## What Changed

- The result commit worker no longer passes
`skipNativeResponsePresentation`. After it commits and projects the run,
it runs the same idempotent reply presentation step that the live
heartbeat and the retry paths use.
- The presentation worker stays the retry owner. If the inline attempt
fails or is refused, the run stays a presentation candidate, as before.
- Review responses are still left to their own worker
(`skipNativeReviewResponse` is unchanged).

## Verification

- Red/green test in
`server/src/services/native-runtime/native-blocked-reply-recovery.test.ts`:
"posts the agent's reply when the result commit worker finishes the run
instead of the heartbeat". It drives the real `nativeResultCommitWork`
worker for a blocked run whose workspace save succeeded.
- Red on `master`: the task is blocked and the run succeeded, but the
task has no comment.
- Green with this change: the agent's final reply is posted once, and
the presentation worker has no remaining candidate.
- `cd server && npx vitest run
src/services/native-runtime/native-blocked-reply-recovery.test.ts
src/__tests__/native-result-commit-work.test.ts`: 26/26 pass.
- `pnpm verify` on the change.

## Risks

- Low risk. The presentation step is idempotent: it locks the run,
records `presentationDecision.commentId`, and never posts twice. The
live heartbeat and the retry paths already call it inline.
- The result commit worker does slightly more work per committed run
(one presentation transaction, typically 30–500 ms on staging).
- Not addressed here: why the result commit worker wins the race against
the live heartbeat in about a third of runs. The reply no longer depends
on that race.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use (shell,
browser, code execution) in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Refs #21169

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 12:31:17 -06:00
DottaandPaperclip d10e250d55 fix(ui): make OpenRouter setup models searchable (#22034)
## Thinking Path

> - Paperclip helps people manage AI agents for work.
> - Agent setup lets people select a provider and a model.
> - OpenRouter has a large model catalog and accepts custom model IDs.
> - The setup page used a native select for OpenCode and Pi.
> - The shared model picker already supports search and manual entry.
> - This pull request uses that picker for multi-provider and routed
models.
> - People can find a model or enter a model ID in the same control.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Model selection in the new agent setup page.

**Current behavior**

OpenCode and Pi use a native model select. Users must scroll through the
catalog. Custom models require a separate select option and input field.

**Proposed behavior**

Use the shared searchable model picker for multi-provider harnesses and
routed connections. Users can filter models or choose `Use manual model`
after they type a custom ID.

**Reason and benefit**

OpenRouter catalogs can contain many models. Search helps users find a
model. Manual entry supports models that are absent from the catalog or
entered while the catalog is unavailable.

**Breaking changes**

None. The existing model ID resolver and save path remain in use.

Related work: #15422 set the qualified runner defaults. This change
addresses the model control on the resulting setup page. A search of
related PRs found no duplicate fix.

## What Changed

- Enable the existing searchable picker for multi-provider and routed
models.
- Add seven setup tests with the real model picker. Cover filtered
selection, saved IDs, custom IDs during catalog errors, the OpenCode
routing prefix, single-provider routing, and a linear work limit for 600
models.

## Verification

- The five regression cases fail before the fix because the searchable
model control is absent.
- Final focused Vitest run: 145 tests pass across setup, model picker,
and connection model suites.
- UI typecheck passes after building the required shared and plugin SDK
packages.
- The 600-model search test and the routed Claude setup test pass.
- The broader selected run passed 576 test files and failed two CLI
permission tests under this workspace's `077` umask. The isolated CLI
suites pass all 24 tests with `umask 022`. Review edits also invalidated
that broad run, so it is not a final-revision pass.
- Greptile gives 5/5 with no actionable findings on commit
`1686422063d426b1a35b0c0c0c9a1340153c9679`. All review threads are
resolved. All applicable CI checks pass on that commit. Paperclip review
also gives 5/5 with no actionable findings. [CI
run](https://github.com/paperclipai/paperclip/actions/runs/44137699265).
- Chromium checks pass for desktop search, selection, custom entry,
OpenRouter ID normalization, and mobile search without horizontal
overflow. Five screenshots were captured at desktop and mobile sizes.
They include the populated dropdown with an empty search field. These
checks use production components with API fixtures. They do not make
provider calls.

## Risks

Low risk. The change selects an existing shared control. Model
resolution, connection selection, validation, and saved configuration
use their existing paths. Multi-provider and routed model lists now use
search during setup.

## Model Used

OpenAI GPT-6 Codex. The agent used reasoning, repository tools, and code
execution. The runtime did not expose a more specific model version.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
canary/v2026.1117.0-canary.821100
2026-11-17 12:20:16 -06:00
DottaandClaude Opus 5.5 f4cb69205e Let plugin workers serve connector tools (plugin transport) (#22019)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Plugins can now add connector types to the Connectors catalog
(#22004), but only for transports the host already runs (remote MCP,
local stdio)
> - A plugin cannot yet serve a connector's tools itself, and legacy
plugin tools (top-level manifest `tools`) still reach every agent
without a connection or install
> - A plugin should be able to ship a full connector: settings,
credentials, tools, grants, policies and audit, like a built-in one
> - This pull request adds the `plugin` transport, the SDK API to serve
it, and moves legacy plugin tools under the same activation rules
without changing today's access
> - The benefit is that any connector can live in a plugin, and agents
get plugin tools only when an operator enables them

## Linked Issues or Issue Description

Refs #22004

**Subsystem affected**
Plugins (SDK, worker protocol, manifest), Connectors (tool access,
catalog discovery), tool gateway, database.

**Problem or motivation**
A plugin connector can only describe an existing transport. There is no
way for a plugin worker to serve a connector's tools with per-connection
settings and credentials. Legacy plugin tools are visible to every agent
as soon as the plugin is enabled, which is the opposite of the connector
activation rules.

**Proposed solution**
Add a `plugin` transport. The gateway keeps every decision (grant,
policy, approval, rate limit, audit) and calls the plugin worker with
the connection, its settings and a per-call credential. Legacy top-level
tools become one implicit connector per plugin. Existing installations
keep exactly their current access through a one-time migration.

**Alternatives considered**
Move legacy tools to connection tool names (`app.<connection>:<tool>`).
Rejected: names change, so existing tool policies and agent instructions
that name `plugin:tool` would stop matching (a possible widening or
loss). Legacy tools keep their names and handlers; only their visibility
is now gated by an install of the implicit connector.

## What Changed

- `packages/db`: migration `0491` widens
`tool_connections_transport_check` with `plugin` (`NOT VALID`, annotated
`online_compatible`), and adds `plugin_connector_legacy_access`, seeded
with one row per installed legacy-tools plugin and company.
- `packages/shared`: method schema gains `settingsSchema` and `tools`
(`{ kind: "static", list: [{ name, displayName?, description?, risk,
parametersSchema }] } | { kind: "dynamic" }`), only for `transport:
"plugin"`; tool names must match `[a-z0-9_]`; plugin-served methods use
`none` or `api_key`. The connect API accepts `settings`.
- `packages/plugins/sdk`: `ToolRunContext.connection` (`id`, `appSlug`,
`methodKey`, `grantId`, `settings`, `credential?`);
`ctx.connectors.register(appSlug, toolName, fn)` and
`ctx.connectors.listTools(appSlug, fn)`; new `listConnectorTools` RPC;
`executeTool` routes connector calls by `appSlug`. A worker refuses to
start when a declared static tool (or dynamic lister) has no handler.
The test harness gains `executeConnectorTool` and `listConnectorTools`.
- `server/src/services/plugin-connector-runtime.ts`: routes one call to
the plugin's worker; a stopped worker is a
`PluginWorkerUnavailableError`.
- Connect flow (`tool-access.ts`): a `plugin` method creates a
`paperclip_plugin` application and a `plugin` connection with `config =
{ sourceTemplateKey, connectionMethodKey, settings }`, validated against
`settingsSchema`. Credentials, grants, credential policy, installs and
profiles use the existing code.
- Catalog: static tools come from the definition, dynamic tools from
`listConnectorTools`, with the existing version hashing and retirement.
Tool risk comes from the declaration.
- Gateway: plugin connections join connected-tool discovery and both
execution paths (direct and approved actions). For each call the gateway
resolves the grant and, for `api_key`, the credential for that call
only. The run context comes only from the session. A stopped worker
returns `503 plugin_connector_unavailable` with `retryable: true`. The
token broker refuses to export plugin connector credentials.
- Legacy tools: a plugin with top-level `tools` and no `apps` gets an
implicit connector (slug `default`, named after the plugin, no
authentication). Its tools keep their names, handlers and today's access
decision, and an agent sees them only when a connection of that
connector is installed for it or for the company. The connector's saved
action rules (Off, Ask first) can only restrict them.
`completeLegacyPluginToolAccess` turns each migration row into such a
connection with a company install, at boot and when a disabled plugin is
enabled again; it is single-flight across servers, resumes a draft left
by an interrupted attempt, and never overrides an operator's later
changes.
- Catalog: tools a dynamic connector stops offering are retired from
discovery; `listConnectorTools` carries the connection's `companyId`,
and uses the acting person's grant for personal connections.
- Board: the setup flow renders `settingsSchema` with the shared JSON
Schema form, sends `settings`, and restores saved settings on resume and
reconnect.
- `create-paperclip-plugin --template connector` now scaffolds a plugin
connector (`apps` + `ctx.connectors.register` + a harness test). SDK
README and `PLUGIN_SPEC.md` describe connectors; top-level `tools` stay
documented as legacy.
- `inferToolRisk` moves to `tool-risk-inference.ts` so the catalog can
reuse it.

Design notes where the spec left room:
- OAuth for `plugin`-transport methods is not supported in this PR
(`none` and `api_key` only); OAuth plugin connectors can use
`mcp_remote` today. Noted as follow-up.
- Board "Test" calls on a plugin connection pass an empty
`agentId`/`runId`, because there is no agent run.
- Companies created after the upgrade get no automatic legacy
connection; they follow the normal activation rules.

## Verification

- `server`: `npx vitest run
src/__tests__/plugin-connectors-plugin-transport.test.ts` (new):
- rules 1-3: a ready plugin and configured accounts give no tools; an
install for agent A gives only A the tools; agent B is denied on
invocation;
- the worker receives `runContext.connection` with the right id, method,
grant, settings and the call's credential; spoofed arguments do not
change it; each account calls with its own key; results do not contain
the key;
- invalid settings are rejected (422); a stopped worker is `503
plugin_connector_unavailable` (retryable); a disabled plugin hides the
tools;
- dynamic tools are listed through the worker and a tool it stops
offering is retired;
- rule 5: a new installation of a legacy-tools plugin gives no tools; an
installation recorded by the migration keeps the same tool names for
every agent, through one company-wide connection (two concurrent
carry-overs make one; an interrupted draft is finished; a retry keeps an
operator's restriction), and is not also exposed as connection tools;
Ask first and Off on the implicit connector restrict the legacy tools;
  - the token broker refuses to export a plugin connection's credential.
- Existing gateway plugin-tool tests (`tool-gateway`,
`tool-gateway-service`, `prompt-injection-gateway`) now set up the
legacy grant an existing installation has, and pass unchanged otherwise.
- Regression: `plugin-connectors-circleback-parity`, `tool-gateway`,
`tool-gateway-service`, `prompt-injection-gateway`,
`tool-access-service` (846 passed), plus `connector-tool-policy`,
`plugin-lifecycle-restart`, `remote-mcp-connectors`,
`ai-connection-router` in an earlier run.
- UI: `AppsConnect.test.tsx` covers sending plugin settings from the
settings schema.
- SDK: `tests/worker-rpc-host.test.ts` (start refusal without handlers,
connector routing, tool listing). Scaffold: `entrypoints.test.ts`.
- `packages/db`: `pnpm typecheck` including `check:migrations`; server
layered typecheck (sources and tests); `tsc --noEmit` for shared, SDK,
scaffold and UI.

## Risks

- Behavior change for new installations: legacy plugin tools no longer
reach agents until an operator connects the plugin's connector.
Installations that exist at upgrade time keep their tools through the
migration rows.
- The migration widens a check constraint (`NOT VALID`) and copies
plugin × company rows into a new table. Both are fast; the previous
release never writes `plugin`.
- Calls depend on the plugin worker; when it is down the agent gets a
retryable error.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file editing, test execution).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 18:09:42 +00:00
DottaandClaude Opus 5.5 ceaf574421 feat(google-workspace): REST-backed Drive tools in server/src/google-workspace-rest-tools (#22028)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents use Google Workspace through the tool gateway, and Paperclip
is replacing Google's preview MCP servers with its own tools over
Google's REST APIs (`server/src/google-workspace-rest-tools/`)
> - Gmail and Calendar are done there. Drive has no REST tools yet, so
Drive cannot switch to the REST transport
> - Drive is the hard service: file content must be exported or
downloaded, kept under a size cap, and never sent to another host;
search needs a safe query; writes run under the narrow `drive.file`
scope
> - This pull request adds the eight allowlisted Drive tools, with the
same names, inputs and result shapes as Google's Drive MCP tools
> - The benefit is that Drive can switch to `paperclip_rest` once the
switch is approved, with no broader scope and no new dependency

## Linked Issues or Issue Description

Refs #22000, #22014, #22016, #22017 (earlier parts of the same REST tool
set). Related open PRs in the same folder: #22027 (Sheets).

**What problem does this solve?**
The REST tool set has no Drive tools.
`googleWorkspaceRestTransportGaps()` blocks a Drive switch while any
allowlisted Drive tool is missing.

**Proposed solution**
Add `drive/` with the eight tools on the allowlist (`toolScopes.drive`),
a Drive probe, and the Drive paths in `google-http.ts`.

**Alternatives considered**
Converting PDFs and Office files to Google Docs to extract text. That is
a write, so the read tool does not do it.

## What Changed

- `drive/`: `search_files`, `list_recent_files`, `get_file_metadata`,
`get_file_permissions`, `read_file_content`, `download_file_content`
(reads, `drive.readonly`), `create_file`, `copy_file` (writes,
`drive.file`). One file per tool, pure request and result functions,
table-driven tests with synthetic fixtures.
- `drive/query.ts`: parses Google's MCP search syntax into a tree,
checks terms and operators, and writes a new Drive `q` string with every
value re-escaped. Adds `trashed = false` unless the query uses
`trashed`. Unsupported terms are errors, not ignored.
- Content: Google Docs are exported as Markdown (read) or plain text
(download), Sheets as CSV of the first sheet (the read text says so),
Slides as plain text. Binary files are downloaded and base64 encoded.
Every Drive read result is bounded after serialization to 250,000 bytes
(the payload is carried twice, in text and structured content); a larger
result is an error that says how to read less, never cut-short content.
`exportMimeType` must be a format Google lists for the file type, and
every request uses the caller's `fileId`. `create_file` and `copy_file`
are annotated as non-idempotent writes; an annotations test checks all
eight tools.
- `google-http.ts`: per-host path prefix lists, required for every host
(`www.googleapis.com` → `/calendar/v3/`, `/drive/v3/`,
`/upload/drive/v3/`; `sheets.googleapis.com` → `/v4/spreadsheets/`;
`gmail.googleapis.com` → `/gmail/v1/users/me/`, which had no prefix
before, so any path was accepted on that host); a bounded `download()`
for raw bytes; raw upload bodies for multipart uploads; `redirect:
"manual"` with any 3xx turned into a tool error; Drive error reasons
`exportSizeLimitExceeded` and `appNotAuthorizedToFile` mapped to clear
messages.
- `google-http.test.ts`: the host and path allowlist test is now one
table with Sheets rows (allows `/v4/spreadsheets/{id}` and
`/v4/spreadsheets/{id}:batchUpdate`; rejects `/v4/spreadsheets`,
`/v4/other`, `/v4/spreadsheets/../other`, and the Sheets path on
`www.googleapis.com`) and a structural check that every host in
`GOOGLE_API_HOSTS` has at least one allowed and one rejected row.
- `errors.ts`: new `too_large` kind. `probe.ts`: Drive probe
(`about.get`, `fields=kind`), allowed by `drive.readonly` or
`drive.file`.
- `allowlist.test.ts`: Drive 8 of 8; only `create_file` and `copy_file`
write; no update, delete, trash, move or share tool.
- README: Drive section with an endpoint-and-scope table (one Google
reference URL per row), content formats, size caps, requests per call,
and differences from Google's tools.
- Outside the folder, one line:
`server/src/services/google-workspace-rest-dispatch.test.ts` used
`drive` as its example of an unimplemented service. It now uses `chat`.
After merging master (#22026), this file and
`google-workspace-rest-transport.test.ts` pass with Drive added; no
other change was needed.

## Verification

- `cd server && npx vitest run src/google-workspace-rest-tools`: 46
files with the dispatch and transport tests (after merging master with
Sheets), 992 tests pass.
- `cd server && npx tsc --noEmit -p .`: no errors in the folder.
- `pnpm verify`: exit 2 (README is a broad input); required CI supplies
the broad checks.
- Not run: live calls to Google. All tests use synthetic responses.

## Risks

- `copy_file` under `drive.file`: Google does not document whether a
token with `drive.readonly` + `drive.file` can copy a file the app never
created or opened. If not, the tool returns a clear error and nothing is
copied. A live check on the test account is needed before the switch.
- `google-http.ts` now uses `redirect: "manual"` for every request. A
redirect from Gmail or Calendar was a retried network error before and
is now a clear tool error. Google's APIs do not redirect these calls.
- Drive stays on `google_mcp`; nothing calls these tools until the
switch is flipped.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edits, web fetch of Google's public docs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 12:06:00 -06:00
DottaandPaperclip cd656330fe fix(tool-gateway): preserve safe remote MCP error messages (#22025)
Preserve bounded, redacted and screened upstream JSON-RPC error messages for remote MCP tools and context reads. Audit blocked instructions, retain received error semantics, and redact URL/session and decoded credential echoes.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
canary/v2026.1117.0-canary.820800
2026-11-17 12:01:23 -06:00
DottaandClaude Opus 5.5 904c01d1a3 feat(mcp): graduate Assistant connections (MCP) out of experimental (#22015)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Assistant connections (MCP) let an outside assistant, or another
Paperclip instance, sign in as a person and use the `/mcp/paperclip`
endpoint
> - The feature was behind the experimental `enablePublicMcp` setting.
The setting was off by default and in practice off on Cloud stacks
> - Instance-to-instance work (one person's agent files a task on a
colleague's instance) fails when the receiving instance has the setting
off
> - This pull request moves the feature out of Experimental. It is now
always on, and a stored or Cloud-managed "off" value is ignored
> - The benefit is that every authenticated instance with a public URL,
including Cloud workspaces, accepts assistant connections with no setup
step

## Linked Issues or Issue Description

**Subsystem affected**
Public MCP (`server/src/services/public-mcp/`), instance settings,
Connectors → Assistant Connection (MCP) setup page, Experimental
settings page.

**Problem or motivation**
A remote MCP connector on one instance cannot use another instance's
`/mcp/paperclip` unless an admin on the other instance finds and turns
on an experimental toggle. On Cloud, the toggle is off and is managed by
the fleet. Instance-to-instance delegation therefore does not work by
default.

**Proposed solution**
Remove the gate. Keep `enablePublicMcp` only as a compatibility key, the
same way `enableApps` was graduated: old stored rows, PATCH bodies and
Cloud managed configs still parse, but the effective value is always
`true`.

**Alternatives considered**
An instance-admin kill switch that defaults ON. I did not keep one. Each
person already controls access: a connection exists only after that
person approves OAuth consent for one company, and the person or an
admin can revoke it. Deployment policy (authenticated mode and a valid
public URL) still decides whether the endpoint exists. A second switch
would add a state that can silently break instance-to-instance work
again. A schema migration was also not needed: the value lives in the
`experimental` JSON column, and the stored key becomes inert.

**Roadmap alignment**
Requested by the maintainer to enable instance-to-instance
communication.

## What Changed

- `packages/shared`: `enablePublicMcp` is a deprecated compatibility
key. The schema default is `true`. The feature catalog entry is
"Assistant connections (compatibility)" with both defaults `true`.
- `server` instance settings: normalization always returns
`enablePublicMcp: true`. The Cloud managed overlay skips the key, so a
fleet value of `false` cannot disable it. Removed the key from the Cloud
staging experimental-key list. Removed the client-mirror policy step
that ran only when the toggle changed.
- `server` public MCP: personal discovery, OAuth, tools, setup and event
delivery no longer read instance settings (`isEnabled()` returns `true`
for the personal endpoint without a DB read). Dot, Grok and Muse agent
connections keep their own experimental settings.
- `server` Cloud paths: the client-mirror sync, the hosted event
authority, the ingress-handoff expiry worker and the idle sleep proof
treat the front door as enabled. The warm-standby `503
workspace_unclaimed` behavior is unchanged.
- `server` Dot/Grok gates: they no longer also require
`enablePublicMcp`. Error messages no longer tell people to enable
Assistant connections.
- `ui`: removed the Experimental toggle, the "Enable assistant
connections / Open Experimental settings" block on the Assistant
Connection page, and the "Enable Assistant connections (MCP)" hint in
Dot setup. Updated the Dot and Grok copy.
- Storybook: deleted the Experimental-setting story and removed the
"enable the experiment" step from the journey and presenter.
- Docs: `doc/public-mcp.md` ("Enable an instance", "Start from
Connections", `GET /api/mcp/setup`), `doc/openai-dot-runner.md`,
`doc/idle-grok-dispatch.md`, and the agent-access map.
- Tests: new red/green tests for "enabled with no setting stored" and "a
stored or managed off value is ignored". Removed the tests for the
"disabled" behavior, which no longer exists.

## Verification

- Red/green: with the old source and the new tests,
`instance-settings-readonly.test.ts`,
`instance-settings-service.test.ts` and `public-mcp.test.ts` fail (4
failures, and the public-MCP suite fails in `beforeAll` because
`isEnabled()` was `false` with no setting stored). With the new source
they pass.
- `server`: `public-mcp.test.ts`, instance-settings (readonly, service,
managed overlay), `managed-config`, `cloud-event-authority`,
`events-work`, `dot-runner`, `dot-oauth-onboarding` and
`agent-runtime-assignments`. 354 passed.
- Cloud client-mirror lifecycle
(`client-mirror-lifecycle.postgres.test.ts`, skipped in CI) passed
against a local Postgres 17 with
`PAPERCLIP_MCP_SLEEP_TEST_DATABASE_URL`. 6/6, plus `events-work` 28/28.
- Idle and sleep: `idle-incoming-wake` (service and route),
`idle-deadline-proof`, `idle-incoming-wake-capability`,
`idle-sleep-proof`, `client-mirror-ack`, `recovery-deadline-acceptance`
and `idle-completed-history` pass. The warm-standby test now also
asserts that discovery after the claim does no SQL read.
- `ui`: `InstanceExperimentalSettings`, `AssistantConnection`,
`NewAgentDialog`, `NewAgent` and `ExternalAgentInviteDialog.muse` pass.
267 passed.
- Typecheck: `packages/shared`, `server` and `ui` pass. `node
scripts/check-maps.mjs` passes.
- `pnpm verify`: see the CI checks on this PR.
- After the deploy: confirm
`/.well-known/oauth-protected-resource/mcp/paperclip` and `/mcp/setup`
return 200 on paperclip.staging without changing any setting.

## Risks

- Behavior change: instances that had the setting off now serve
discovery, OAuth and `/mcp/paperclip`. Security invariants are
unchanged: the endpoint is still mounted only in authenticated mode with
a valid public URL, OAuth still needs a real signed-in user's consent,
local implicit board authority cannot approve, and revocation works the
same way.
- Cloud sleep: on a managed stack, the MCP sleep proof blocks sleep
until the client mirror publishes the tenant front door to Cloud. The
client-mirror sync runs at boot and does this once. Stacks that already
had the flag on use this path today. A stack whose Cloud connector
cannot publish the mirror would stay awake. Watch fleet sleep after the
deploy.
- Personal public-MCP requests now read instance settings one time less.
- Fleet admin: the Cloud fleet catalog can still show and set
`enablePublicMcp`. The value is ignored. Removing it from the fleet
catalog is a separate change in the Cloud repo.
- No migration. The `0354` trigger still compares the stored key. That
causes at most one extra connector-runtime invalidation if an old client
writes the key.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, test execution).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 11:42:36 -06:00
DottaandClaude Opus 5.5 78cf5aa81a feat(public-mcp): let paperclip_create_task assign a person or no one (#22011)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The public MCP endpoint (`/mcp/paperclip`) lets an outside
assistant, or another Paperclip instance, act in a company as a person.
> - The `paperclip_create_task` tool requires `assigneeAgentId`.
> - So every task that an outside caller files starts an agent run on
the receiving instance and spends its budget.
> - A colleague on another instance must be able to file work for a
person, or for nobody, without that cost.
> - This pull request makes the agent assignee optional and accepts
`assigneeUserId`, the same field the REST issues API uses.
> - The benefit is that instance-to-instance task filing does not spend
the receiving company's budget unless the caller picks an agent.

## Linked Issues or Issue Description

**Subsystem affected**
Public MCP endpoint (`/mcp/paperclip`), tool `paperclip_create_task`.

**Problem or motivation**
`paperclip_create_task` requires `assigneeAgentId`. A caller cannot
create a task for a person or an unassigned task. One Paperclip instance
can connect to another through the generic remote MCP connector, and
people use this to file tasks on a colleague's instance. Every such task
wakes an agent on the receiving instance and can spend its budget.

**Proposed solution**
Make `assigneeAgentId` optional and accept `assigneeUserId`, the same
field the REST issues API uses. Reject both at once. Only an agent
assignee starts a run. Add a read-only `paperclip_list_people` tool so
the caller can find a person's user ID.

**Alternatives considered**
A new MCP-only assignee field. We did not use it, because the REST API
already has `assigneeUserId` and its checks.

**Roadmap alignment**
This supports instance-to-instance communication through the public MCP
endpoint. It does not duplicate planned core work in ROADMAP.md.

## What Changed

- `paperclip_create_task`: `assigneeAgentId` is optional. The tool
accepts `assigneeUserId`, with the same shape as the REST issues API.
- `paperclip_create_task`: a schema refinement rejects a call that has
both assignees. The rejection occurs before the server writes an
idempotency receipt or a task.
- `paperclip_create_task`: the tool description says that an agent
assignee may start work and spend budget, and that a person or no
assignee does not start a run. The `scheduling` result text now matches
the assignee type.
- New read-only tool `paperclip_list_people`. It uses the existing `GET
/companies/:companyId/user-directory` route, which only needs company
access. It returns user ID, name and role. It does not return email.
- The task goes through the same REST create route as before. Company
scope, the `tasks:assign` decision (which also checks that a person is
an active member), activity logging and the service checks all still
apply.
- `doc/public-mcp.md`: documents the three assignee shapes and the new
tool.
- `tests/runner-e2e/public-mcp-grading.ts`: adds `paperclip_list_people`
to the read-only tool list.
- The stdio server (`packages/mcp-server`) already uses
`createIssueInputSchema`, which accepts both fields. It does not change.

## Verification

- New test in `server/src/__tests__/public-mcp.test.ts`. It uses the
real issue and access routes. It proves:
- a task for a person and an unassigned task are created as `todo`, and
their `requestId` replays return the same result;
  - neither shape wakes an agent, and the company has no heartbeat runs;
  - a task for an agent still wakes that agent one time;
- both assignees at once are rejected with a clear message, and no
receipt is written;
- a member of another company is rejected (HTTP 403 from the REST
assignment check), with a result identical to a user ID that does not
exist;
- `paperclip_list_people` returns members without email and without
other companies' users.
- Red first: the test failed with "Unknown Paperclip operation" before
the change. Green after the change.
- `cd server && npx vitest run src/__tests__/public-mcp.test.ts`: 85
passed (also after the rebase on `origin/master`).
- `npx vitest run --config tests/runner-e2e/vitest.config.ts
tests/runner-e2e/public-mcp-grading.test.ts
tests/runner-e2e/public-mcp-expanded-flow.test.ts
tests/runner-e2e/public-mcp-invitation-grading.test.ts`: 81 passed.
- `cd server && npx tsc --noEmit -p .`: passed.
- `pnpm verify` exits 2 because a doc changed (a global CI input). CI
supplies the broad coverage.

## Risks

- Low risk. The change only adds optional input. Existing calls with
`assigneeAgentId` behave the same.
- An unassigned task is created as `todo`, the same status as before. No
code path wakes an agent for a task without an agent assignee.
- `paperclip_list_people` shows member names and roles to any connection
with read scope. The same data is already available to any company
member in the UI assignee picker through the same route.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), through Claude Code, with tool
use and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 11:42:31 -06:00
DottaandClaude Opus 5.5 b1efe32c60 feat(connections): name and guide connections to another Paperclip instance (#22018)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Each Paperclip instance serves an assistant connection (MCP) at
`/mcp/paperclip`, and any instance can connect a remote MCP server by
its address
> - So one instance's agents can file and follow tasks on another
instance, for example a colleague's
> - But the other instance's tools look like the agent's own Paperclip
tools: same product, similar names, and the connection prefix was a
generic internal link key
> - An agent can then file local work on someone else's instance, or the
reverse
> - This pull request recognizes a Paperclip address, names the
connection and tool prefix after that instance, adds agent instructions,
and refuses a connection to the same instance
> - The benefit is that agents can tell their own Paperclip from a
connected one

## Linked Issues or Issue Description

**Subsystem affected**

Connections: "Connect your own MCP server" (generic remote MCP) and the
tool gateway names that agents see.

**Problem or motivation**

When instance A connects instance B through `/mcp/paperclip`, A's agents
get B's `paperclip_create_task` and other tools. The tool prefix came
from an internal key (`app-gallery🔗<uuid>`), so native runners saw
`app_mcp_app_gallery_link_paperclip_create_task_…`. Nothing told the
agent that the tools act on a different instance. An agent could send
internal work to another organization by mistake.

**Proposed solution**

Recognize an address whose path is `/mcp/paperclip`. Use only the typed
address, never data from the remote server. Name the connection after
the host (`acme Paperclip`). Use that name as the tool prefix for these
connections only. Add default agent instructions. Refuse the instance's
own address with `paperclip_self_connection` (422).

**Alternatives considered**

Read the company name from the remote `paperclip_connection` tool. This
needs an extra authenticated call during setup, and it puts
remote-controlled text in names and instructions. The host is enough to
identify the stack. Require approval for each write. We rejected this:
actions start on, the same as other connections.

**Roadmap alignment**

Supports instance-to-instance work through existing MCP connections. It
adds no new transport.

## What Changed

- `packages/shared/src/paperclip-peer.ts`: detect a Paperclip MCP
address, derive a label and name, compare with own origins (loopback
hosts on the same port are equal), and build default agent instructions.
- `server/src/services/tool-access.ts`: default name and default agent
instructions for a new Paperclip connection. Instructions that the
person supplies are kept.
- `server/src/routes/tool-access.ts`: refuse the instance's own address
before any network request.
- `server/src/services/tool-gateway.ts`: use the prefix recorded on a
Paperclip connection (`paperclipPeerToolNamespace`, derived from its
address at creation). Renames never change it. Other connections, and
Paperclip connections made before this change, keep their current tool
names.
- `ui/src/pages/apps/generic-mcp-connect.ts`: same default name in the
setup flow.
- `doc/connections/GENERIC-REMOTE-MCP.md`: new section "Connecting
another Paperclip instance".

## Verification

- `packages/shared`: `npx vitest run src/paperclip-peer.test.ts`, 20
passed.
- `server`: `npx vitest run
src/__tests__/generic-mcp-connection.test.ts`, 98 passed. The naming,
instructions, self-connection and tool-prefix tests fail on the base
revision and pass with this change (red/green).
- `ui`: `npx vitest run src/pages/apps/generic-mcp-connect.test.ts`, 29
passed.
- `tsc --noEmit` passes for `packages/shared` and `server`.
- Live check with two local instances. Instance B was in authenticated
mode on port 3201 with assistant connections on. Instance A was on port
3202.
- A connected to `http://localhost:3201/mcp/paperclip`. The name was
`localhost-3201 Paperclip`. A registered with B by dynamic client
registration and asked for read, write, configure and offline access.
- B's consent page showed "Connect Paperclip localhost-3202 to
Paperclip". After approval, A had all 37 tools active, writes included.
- From A, `paperclip_connection` returned B's company.
`paperclip_create_task` created `ACM-1` on B.
- An agent session on A listed
`mcp.localhost-3201-paperclip-…:paperclip-create-task` (runner name
`app_mcp_localhost_3201_p_paperclip_create_task_…`). Before the gateway
change, the name was `app_mcp_app_gallery_link_paperclip_connection_…`.
The run received the connection instructions.
- Connecting A to `http://localhost:3202/mcp/paperclip` returned 422
`paperclip_self_connection`.

## Risks

- Low risk. Only connections whose address path is exactly
`/mcp/paperclip` change. Other connections keep their names,
instructions and tool names.
- A Paperclip connection created before this change keeps its original
tool names, because only connections that record
`paperclipPeerToolNamespace` at creation get the new prefix. Saved exact
tool-name rules and pending approvals for those connections keep
matching. They also get no default instructions; people can add them.
- Native runners keep only 20 characters of the namespace (`mcp_` uses
4), so long host labels are cut. The tool part stays complete.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, browser pane).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 11:41:13 -06:00
Devin FoleyandPaperclip 1d2b669e7a fix(ssh): allow CLI agents on volume-free servers (#21957)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - An SSH environment runs an installed harness on a computer the
operator controls.
> - Volume-free Cloud servers require durable task files outside the
controller.
> - The current admission path accepts native sandbox workspaces and
rejects existing SSH CLI agents before execution.
> - The remote workspace owner already provides saved versions and a
single-writer fence.
> - This pull request gives that owner an SSH archive transport and
admits the existing Claude and Codex CLI adapters under its authority.
> - The benefit is SSH task execution without a controller volume or a
new environment type.

## Linked Issues or Issue Description

**What happened?**

A task assigned to an existing SSH Claude Code agent fails on a
volume-free Cloud server. It reports that execution requires native
Paperclip Runner and remote workspace storage. The failure occurs before
provider work starts, even when SSH and the configured AI account are
valid.

**Expected behavior**

An SSH CLI agent can use the configured Paperclip AI account and an
installed harness. Task files remain durable when the Cloud controller
is replaced. The same SSH behavior applies to direct and Tailscale
connections.

**Steps to reproduce**

1. Configure an SSH environment on a volume-free server with external
Postgres, stable encryption, and object storage.
2. Assign a Claude Code or Codex CLI agent to that environment.
3. Give it a task without a project, such as creating `hello_world.txt`.
4. Observe the workspace admission failure before execution.

Related work: #21248 adds agent-selected SSH authentication. #20617 adds
Tailscale network selection. #15120 and #17291 cover SSH process and
lease cleanup. A search of open and closed PRs found no duplicate of
this workspace admission fix.

## What Changed

- Reuse remote workspace preparation, archives, saved versions, and
single-writer admission for SSH targets.
- Add an explicit file-sync capability to the existing SSH command
runner. It streams archive files with bounded deadlines and private
temporary files. Ordinary SSH runners keep their previous capabilities.
- Run the existing CLI adapter in the remote workspace. Bind both the
target and its SSH command specification to that directory.
- Admit a direct SSH adapter on a volume-free server only when its
durable workspace, environment, lease, and path agree. Local filesystem
authority remains rejected.
- Preserve the server-owned remote workspace receipt when selecting the
legacy runtime.
- Reuse the workspace finalization lock for a legacy run only when its
company, workspace, writer, epoch, and lease receipt match. A claimed
token can finish and clear itself after another task reserves a
successfully saved shared workspace. Storage writes still require the
current workspace authority.
- Transfer recovered copies to SSH through the verified workspace
archive format. Sandbox recovery retains its existing directory
transfer.
- Document Node.js, Git, explicit `engine=cli`, and uncertain-process
limitations. No UI, Cloud control-plane configuration, schema, or
deployment change is included.

The remote workspace service owns this change. The shared SSH utility
adds file transport because its existing authentication and stream
cleanup must remain the only owners of SSH credentials. The heartbeat
changes pass the storage authority through existing preparation,
selection, and finalization boundaries. The finalization owner extension
avoids a separate copy-back lock for SSH. Existing native callers keep
their prior admission contract.

The existing SSH utility and heartbeat test files are already large. The
added code stays beside their existing stream lifecycle and shared
fixtures. A larger extraction would widen this compatibility fix.

## Verification

- Red-green tests reproduce SSH rejection and verify admission only with
remote storage authority.
- A real isolated OpenSSH fixture exercises archive capture, a
replacement database client, a saved-workspace restore to another
allocation, and a failed SSH transfer. Failed saves retain the files and
block a competing writer.
- A whole-heartbeat test uses the real SSH transport and Claude CLI
adapter, a synthetic CLI, a selected Paperclip AI account, and fixture
object storage. A follow-up resumes the same provider session and its
retained history. It makes no provider API calls. The synthetic provider
also supplies its task disposition update.
- Existing Claude and Codex SSH adapter tests, workspace preparation,
and native workspace owner recovery are included in focused checks.
- Before the review fixes, clean-checkout verification passed 610 tests
in 11 files. This includes the full workspace preparation, runtime
selection, runtime execution, environment orchestrator,
ephemeral-server, workspace transport, native owner recovery, SSH
integration, SSH command, and Claude/Codex remote execution suites. The
macOS SSH fixture was enabled with
`PAPERCLIP_ENABLE_DARWIN_SSH_ENV_LAB=1`; Vitest used `--maxWorkers=2`.
- An initial run in the main worktree hit a skill-root lookup outside
that checkout. The full selected set passes in the clean checkout. The
final company-binding type correction changes no executable behavior;
all 8 focused selection regressions pass after that correction.
- At the initial head `dd77a4d3cf0`, server and adapter-utils typechecks
passed, and the clean verification checkout matched its tree
`3f90b0681b1004556d1a720ed45fc8d439b04e96`.
- Review regressions first reproduced both a lost ownership token after
shared-workspace handoff and a rejected recovered-directory transfer.
With the fixes, 223 tests in 5 affected suites pass, including real SSH
recovery and native owner-restart, save-concurrency, archive transport,
and recovery-receipt checks. The handoff test also confirms that a stale
owner cannot save again or claim ownership after a new writer takes
over. Server typecheck passes after these fixes. The tested review-fix
head is `c5182f5d9c56e738e683d9f26b226e9a820bba83`.
- [The initial CI server
shard](https://github.com/paperclipai/paperclip/actions/runs/44009944657/job/133794523181)
failed on an assertion still expecting the old native-sandbox-only
error. Its unsupported sandbox adapter remains rejected; the assertion
now matches the supported-remote-environment error. After updating that
assertion, all 115 tests in the workspace lifecycle suite pass. Together
with the 5 affected suites, this gives 338 passing tests for the review
fixes.
- `pnpm verify` returns 2 and requests broad CI coverage for the
shared/runtime change. The full local suite was not substituted for CI.
Final-head
[CI](https://github.com/paperclipai/paperclip/actions/runs/44012991355)
is green on `c5182f5d9c56e738e683d9f26b226e9a820bba83`: 58 checks pass
and 3 are skipped, with no failures or pending checks. Typecheck, build,
end-to-end, and the aggregate verification gate all pass. Greptile is
5/5 on this head, security scan passes, and there are no unresolved
review threads or merge conflicts.

Review-fix checks:

```sh
PAPERCLIP_ENABLE_DARWIN_SSH_ENV_LAB=1 pnpm exec vitest run --maxWorkers=2 \
  server/src/services/ssh-remote-workspace.integration.test.ts \
  server/src/services/native-runtime/native-workspace-owner-restart-recovery.test.ts \
  server/src/services/remote-workspace-transport.test.ts \
  server/src/services/remote-workspace-recovery-receipt-fallback.test.ts \
  server/src/services/remote-workspace-save-concurrency.test.ts
pnpm exec vitest run --maxWorkers=2 server/src/services/remote-workspaces.integration.test.ts
pnpm --filter @paperclipai/server typecheck
```

## Risks

This extends the remote workspace owner to another transport and allows
qualified SSH CLI execution on volume-free servers. It changes remote
file placement to a task directory under `.paperclip-workspaces`. It
does not import an existing controller-local workspace.

The installed harness uses its existing adapter. Set `engine=cli`
explicitly; ACP archive staging does not support these in-place
workspaces. This does not qualify native macOS provider packs or change
an agent's engine or permission settings automatically.

An unknown remote process after SSH loss remains held until it can be
reconciled. This change does not add automatic reconciliation of that
process. SSH computers are never stopped or deleted. Real provider
acceptance and a hosted staging journey have not been run for this
revision.

## Model Used

OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) through the Codex harness, with xhigh
reasoning effort and code tools.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-17 09:33:11 -08:00
DottaandClaude Opus 5.5 f61b919701 feat(google-workspace): REST-backed Docs and Slides tools; mark Calendar update_event destructive (#22029)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The Google Workspace connector gives agents Google tools through the
tool gateway
> - Google's hosted Workspace MCP servers are a Developer Preview, so
Paperclip launches on its own tools over Google's REST APIs, in
`server/src/google-workspace-rest-tools/`
> - Gmail and Calendar are done there; Docs and Slides have no tools yet
> - This pull request adds `read_doc`, `update_doc`, `read_presentation`
and `update_presentation` with the same names and inputs as Google's MCP
tools
> - The benefit is that the Docs and Slides services can switch to
`paperclip_rest` once their API is enabled, and reviewers see a full
document in Paperclip

## Linked Issues or Issue Description

**Subsystem affected**
Google Workspace REST tool set
(`server/src/google-workspace-rest-tools/`), Docs and Slides services.

**Problem or motivation**
The REST tool set has no Docs or Slides tools. The allowlist
(`toolScopes.docs`, `toolScopes.slides`) names two tools for each
service. A service cannot switch to `paperclip_rest` until all of its
allowlisted tools exist.

**Proposed solution**
Add the four tools in new `docs/` and `slides/` folders. Each tool makes
exactly one REST call. The read tools return the whole file as
Markdown-style text plus an index structure for edits. The update tools
post `batchUpdate` requests in order and require the revision ID that
the caller read.

**Alternatives considered**
Return the raw `documents.get` JSON. It is 10 to 20 times larger than
the text, so ordinary documents go over the result size limit, and it is
hard to compare with Google Docs. Reorder edit requests by index. That
changes what dependent requests do, and Google's tool does not do it.

Refs #22000, #22014, #22017

## What Changed

- `docs/`: `read_doc` (`documents.get` with `includeTabsContent=true`)
and `update_doc` (`documents.batchUpdate`), a renderer for every tab,
child tab, heading, list (real numbering), table, link, header, footer,
footnote, chip, suggestion and comment, and an index structure where
each paragraph's `text.length` equals `endIndex - startIndex`.
- `slides/`: `read_presentation` (`presentations.get`; with `fields` it
returns Google's resource as is) and `update_presentation`
(`presentations.batchUpdate`), a renderer for every slide's text,
tables, groups, media and speaker notes.
- `documents-shared.ts`: the 250,000-byte limit on the final serialized
result (an error, never a cut-short result), the revision-conflict
error, comment rendering, and argument checks.
- Update tools require `writeControl.requiredRevisionId` and refuse
`targetRevisionId`. A revision conflict returns "nothing was changed;
read it again". Writes are never retried.
- `google-http.ts`: allows `docs.googleapis.com` only under
`/v1/documents/` and `slides.googleapis.com` only under
`/v1/presentations/`.
- `tools.ts`, `probe.ts`: register the services. Docs and Slides have no
network probe, because every read needs a file ID.
- Update tools accept only a closed list of batch request kinds
(`docs/request-kinds.ts`, `slides/request-kinds.ts`). Kinds and forms
that read another file are refused by name: `insertRichLink`, the Sheets
chart kinds, `importSlides`, `createAudio`, Drive videos, and images
from Drive or Docs links. Unknown kinds are refused too.
- `read_doc` always requests `SUGGESTIONS_INLINE` and falls back to an
explicit `PREVIEW_WITHOUT_SUGGESTIONS` read for view-only users; the
result says which view it used.
- Tables with merged cells are placed on the full grid; covered cells
read `(merged left)` or `(merged up)`.
- Revision conflicts are recognized from Google's `error.status` and
`error.details` first, with the message as a fallback.
- Annotations: `update_doc` and `update_presentation` are destructive
and not idempotent. **Calendar `update_event` is now also marked
destructive** (it overwrites fields, removes attendees and replaces
reminders). Google's references mark both non-destructive; the README
records this.
- `README.md`: endpoint and scope table with Google's documentation
links, the result format, what cannot be shown, update order and
revision rules, and every difference from Google's tools.

## Verification

- `cd server && npx vitest run src/google-workspace-rest-tools`: 33
files, 573 tests pass (169 new in `docs/` and `slides/`, plus host,
probe and allowlist tests).
- `cd server && pnpm typecheck`: passes.
- `pnpm verify`: exit 2 (README is a broad input), so required CI
supplies broad coverage.
- `src/services/google-workspace-rest-dispatch.test.ts` fails the same
way on `master` (it still expects Gmail to be incomplete after #22014).
This PR does not touch it.

## Risks

- Low risk. Nothing calls these tools until the Docs or Slides transport
is set to `paperclip_rest`; both stay `google_mcp`.
- The inner format of `content` differs from Google's tool, which Google
does not publish. The README lists this.
- Revision-conflict detection matches Google's 400 message on
"revision". Any 400 rejects the whole batch, so nothing changes in
either case.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with
tool use and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 17:28:13 +00:00
DottaandPaperclip 7f9867895c fix(ui): keep automated work out of personal recents (#22024)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The sidebar helps each person return to their own work.
> - A task visit previously added any readable task to personal recents.
> - Automated updates also moved old tasks above newer personal work.
> - The Mine inbox already defines personal participation.
> - This pull request applies that rule to recent task candidates and
preserves their order during background updates.
> - Users can see current task status without a stream of unrelated
automated work.

## Linked Issues or Issue Description

**What happened?**

Routine tasks and imported GitHub review tasks appeared in personal
recents after a read. Later agent updates moved these tasks up the list.
A connected account or stored creator ID could make these tasks look
like personal work.

**Expected behavior**

A task must pass the viewer's existing Mine participation rule before it
appears in personal recents. Routine ownership, connected-account
provenance and reads alone do not qualify. A viewer can qualify through
manual creation, assignment, a direct comment, an allowed user action or
a live request addressed to them. Inbox archive state does not remove
personal history. Passive reads and machine updates must not advance its
order.

**Steps to reproduce**

1. Open an automated routine task or imported review task without
participating in it.
2. Check the sidebar recents.
3. Let an agent update an older task while newer personal work is in the
list.
4. Before this fix, the automated task appears and background updates
can move older tasks up.

## What Changed

- Add a viewer participation field to bounded navigation summaries.
Reuse the canonical Mine query and existing access checks. The field is
a read projection; no database migration is needed.
- Show only confirmed personal tasks. Remove old nonparticipants after a
successful read. Do not trust participation from local storage or full
task caches.
- Keep status and title snapshots current without changing personal
order. Explicit local edits and comments can advance recency.
- Migrate browser history once to a new storage version. Older tabs
cannot restore machine-driven ordering.
- Keep confirmed rows visible if a new batch request fails. Keep these
facts within the current account and company. Hidden candidates do not
bypass the one-second reorder delay.
- Record sidebar renames as local activity. Place a first passive visit
below existing history.
- Use the saved comment time for comments sent with reassignment or
custom run settings. Pending sends do not advance recency.
- Keep readable subtasks as context beneath qualifying parents. Keep
inbox archive independent of recents.
- Update the design rules, UI behavior notes, regression tests and
Runner source contract pin.

## Verification

- Current head: `92ece0e61859a6c54b714b3611f062ff9b5633bb`. The PR
merges cleanly with current master.
- Latest red/green: both configured-comment cases failed before the fix
and now pass. Reassignment and custom run settings advance recency only
after the comment saves, using its saved time.
- Latest local checks: 339 tests passed across IssueDetail and recent
storage, hooks and sidebar. UI typecheck and diff checks passed.
- Earlier local coverage: 592 targeted API/UI tests passed. After the
master conflict resolution, all 29 overlapping Gmail tests and the
548-input source registry check passed.
- Both Greptile and Paperclip Review report 5/5 on the current head. All
four review threads are resolved.
- Current-head [CI attempt
2](https://github.com/paperclipai/paperclip/actions/runs/44132164473)
passed, including the final verify gate, build, typecheck, browser
shards, serialized tests, Runner checks and all general-test shards. The
workspace-a (1/2) retry passed after the first attempt hit an
asynchronous jsdom/Radix event error in NewAgentDialog.
- `pnpm verify` requires broader CI because DESIGN.md changed. No full
local suite or deployed-browser acceptance was claimed. This change is
not deployed.

## Risks

- Unknown eligibility stays hidden until the summary succeeds. Failed
requests preserve saved candidates and previously confirmed state within
the current account and company, including after batch changes.
- This intentionally removes automated tasks from old personal history
when the viewer has no qualifying participation.
- Readable subtasks under a personal parent remain context. They do not
become independent recent entries from machine activity.
- No task ownership, routine definition, inbox membership or permission
rule changes.

## Model Used

OpenAI GPT-6, running as Codex, with reasoning, repository tools and
focused test execution. No additional model workers were used.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-17 11:25:58 -06:00
DottaandClaude Opus 5.5 8172d27089 feat(cloud): browser product analytics through a same-origin PostHog proxy (#22012)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Cloud runs the same app for hosted customers, and the same
UI ships in the npm package.
> - The PostHog project for Cloud gets only server events today. It has
no page views, sessions, replays or people.
> - So the Product and Web analytics pages are empty, and we cannot see
how Cloud users move through the board.
> - This pull request adds browser analytics for Cloud only. It is off
until an operator sets two switches on a Cloud stack.
> - Self-hosted installs do not change: they never download posthog-js
and never serve the PostHog proxy.
> - The benefit is real page views, sessions, heatmaps, Web Vitals and
masked session replays for Cloud, linked to the user and the workspace.

## Linked Issues or Issue Description

**Subsystem affected**
Board UI (analytics bootstrap and privacy markers), server session
payload and a new same-origin proxy route, shared validators.

**Problem or motivation**
Cloud sends only server events to PostHog. There is no browser SDK, so
PostHog has no page views, sessions, replays or identified people for
Cloud. We cannot see where Cloud users get stuck.

**Proposed solution**
The signed-in session gives the UI a PostHog config only on a
Cloud-managed stack with `PAPERCLIP_POSTHOG_PROJECT_TOKEN` and
`PAPERCLIP_POSTHOG_BROWSER_ENABLED=1`. The UI loads posthog-js after the
page settles. Events go through a strict same-origin proxy at `/_ph`, so
ad blockers do not drop them. Replay masks all text and all inputs by
default.

**Alternatives considered**
Bake a key into the UI build: refused, because the same build ships to
self-hosted users. A generic reverse proxy: refused, because it would
forward any path. Mask only content areas: refused, because a missed
surface would leak; masking is deny by default.

**Additional context**
This is the browser part of the Cloud direct PostHog plan. The server
direct sender is already on master (#22006). Fleet configuration is a
separate change. PostHog person deletion on account deletion is a
follow-up.

## What Changed

- Uses the shared Cloud PostHog config (`resolveCloudPostHogConfig` in
`server/src/services/cloud-posthog-config.ts`, already on master from
the server direct-sending change). This PR does not change it.
- `server/src/services/cloud-posthog-browser.ts`:
`resolveBrowserAnalyticsConfig` returns `{ token, apiHost: "/_ph",
uiHost, environment, stackId, replayEnabled }` only when
`PAPERCLIP_POSTHOG_BROWSER_ENABLED=1`. `environment` is always set: the
explicit variable, else `staging` for a `*.staging.paperclip.app` host,
else `production`.
- `GET /api/auth/get-session` carries the new optional
`browserAnalytics` field (the existing signed-in bootstrap request, so
no new request on the cold path). `packages/shared` has
`browserAnalyticsConfigSchema` and the session schema field.
- `server/src/routes/posthog-proxy.ts`: a strict proxy, mounted before
the body parsers and only when browser analytics is on. It forwards only
these requests and answers 404 to all others without contacting PostHog:
- `POST /e/`, `/i/v0/e/`, `/batch/`, `/s/`, `/flags/`, `/decide/` to the
ingest host.
- `GET /array/<token>/config(.js)` and `GET
/static/[<version>/]<name>.js` to the assets host (derived from the
ingest host).
- `stream.pipeline` joins the streams, so an error on either side
(PostHog closes the connection partway through a response, or the
browser goes away) closes both and never goes unhandled. It builds a new
header set from an allowlist plus `X-Forwarded-For`. It never forwards
the inbound Host, Cookie or Authorization headers. It drops `Set-Cookie`
from responses. Bodies stream; nothing is buffered. It refuses bodies
over 64 MiB (413, also for chunked bodies that grow past the limit) and
limits requests in flight to 16 per client address and 256 in total
(429).
- `ui/src/lib/analytics/product-analytics.ts` (the only module that
talks to posthog-js) and `ui/src/components/ProductAnalyticsGate.tsx`:
- posthog-js loads after the page settles (or at a 15 s deadline), only
when the session has a config. It loads as PostHog's browser script
`/_ph/static/array.js` through the proxy, the same way posthog-js itself
loads its extensions (recorder, web vitals). It is not an npm dependency
and is in no bundle chunk.
- Init: page views on history change, page leave, autocapture, Web
Vitals, heatmaps, `person_profiles: "identified_only"`,
`tracing_headers` for our own host only, replay only when
`replayEnabled`. Surveys, product tours, web experiments and
conversations are off.
- Super properties on every event: `deployment_source: "cloud"` and
`environment`.
- After sign-in: `identify(session.userId, { email, name })` (person
properties only). `session.userId` is the authenticated actor's user id
verbatim (for example `paperclip-id:...`): `GET /api/auth/get-session`
copies `req.actor.userId` into it, and server events use that same actor
id as `distinct_id`, so browser and server events land on one person.
Tests: `auth-routes.test.ts` (the session returns the actor id verbatim)
and `ProductAnalyticsGate.test.tsx` (identify uses it) and
`group("workspace", stackId)`. On sign-out: `reset()`, then the super
properties again. `useSignOut` calls it before it leaves the page,
because Cloud sign-out navigates away while the session query is still
populated.
- Privacy (`ui/src/lib/analytics/privacy.ts`), deny by default:
- Replay masks every text node (`maskTextSelector: "*"` with
`maskTextFn`) and every input value (`maskAllInputs`).
- Only elements with `ANALYTICS_UNMASK_PROPS` show their text: sidebar
items whose label is a fixed string (`analyticsChrome`, added at the 99
call sites with literal labels), and a shared `Button`, `Label`,
`TabsTrigger` or `EmptyState` only when its caller passes
`analyticsChrome`. None of them opts in by default, because they also
render customer text (agent names, document titles). Sidebar items with
dynamic labels (recent tasks, agents, projects) stay masked.
- Replay masks attribute values too (`maskAttributeFn`, deny by
default). Only layout attributes (`class`, `style`, `role`,
`data-state`, SVG geometry), rrweb metadata and stylesheet links keep
their values. `title`, `aria-label`, `alt`, `placeholder`, `src`,
`href`, `data-*` and element references (`id`, `for`, `aria-labelledby`,
`aria-describedby`, `aria-controls`, which often embed names or slugs)
are always masked, even on unmasked chrome. So the organization
switcher's `aria-label`, task titles in `title` attributes, image
sources and download links with file paths never reach a replay.
- `before_send` (`scrubCapturedEvent`) drops the `title` property from
every event, because posthog-js copies `document.title` (task and
organization names) into page views. Every URL string in every event,
including replay meta events, keeps only its origin and path: query and
hash values never leave (they can hold tokens, `code`/`state`, file
paths or search text), and the token segment of `/board-claim/<token>`
and `/invite/<token>` is redacted. It recurses only into plain objects
and arrays, so the `Date` timestamp stays intact.
- `ANALYTICS_MASK_CLASS` keeps customer content masked even inside
unmasked chrome: `MarkdownBody` (task and issue bodies, comments, chat
messages, documents and artifact markdown), `MarkdownEditor`, the task
chat thread, the issue chat thread, the comment timeline, the file
viewer (source, markdown and CSV views) and the organization name in the
organization switcher. Text in any `contenteditable` editor is always
masked.
- `ANALYTICS_NO_CAPTURE_CLASS` (`ph-no-capture`, a blank box in replay)
on every password-type `Input` (about 40 secret, token and API key
fields), environment variable values and new secret values.
- Autocapture never records element text or attribute values
(`mask_all_text`, `mask_all_element_attributes`).
- Connected-account content (Google user data and Slack data: email
bodies, calendar events, Slack messages in tasks, chat, tool results,
previews) never appears unmasked in a replay. Replay masks all text by
default; nothing that renders connected-account content is in the unmask
allowlist; autocapture records no element text anywhere
(`mask_all_text`). `ui/src/lib/analytics/unmask-allowlist.test.ts` pins
the allowlist: only 7 files may spread the unmask attribute, sidebar
opt-ins need a literal label, and any `analyticsChrome` opt-in on
`Button`/`Label`/`TabsTrigger`/`EmptyState` fails until it is added to a
reviewed list (empty today). The same test renders Gmail, Calendar and
Slack-style tool results next to unmasked navigation and checks they are
masked.
- Guard test: UI code must never call `capture()` with a
`PAPERCLIP_EVENTS` name, so each event has one sender.
- Docs: `doc/cloud-browser-analytics.md` (switches, per-stack opt-out,
proxy allowlist, identity, privacy) and a link from
`doc/observability.md`.
- No new npm dependency and no lockfile change: posthog-js comes from
PostHog through the proxy at run time. (An earlier revision added
`posthog-js` to `ui/package.json`; the Runner support suite requires
every `ui` dependency in the reviewed lockfile, which a pull request
cannot update, so the script approach replaced it.)

## Verification

- `pnpm verify`: see the result below.
- Focused tests:
- `server/src/__tests__/cloud-posthog-browser.test.ts`: exposure matrix
(self-hosted with every switch set, Cloud without token, without stack
ID, without the browser switch, switch not `1`, enabled, replay off),
environment fallback, proxy mount only when enabled, allowed routes, 404
for disallowed methods and paths (including `//host` and `..` paths)
with no upstream contact, Host/Cookie/Authorization stripped,
`Set-Cookie` dropped, 502 when PostHog is down, and no unhandled error
when PostHog closes a response partway (red on the old `pipe()` code,
green now).
- `server/src/__tests__/auth-routes.test.ts`: the session carries
`browserAnalytics` only on Cloud with the token and the switch.
- `ui/src/lib/analytics/*.test.ts`,
`ui/src/components/ProductAnalyticsGate.test.tsx`: no import of
posthog-js without a config or when signed out; load after the page
settles, once; identify, group, reset with a mocked posthog; email and
name never become event properties; deny-by-default replay masking;
guard test.
- Real run (local, headless Chromium, production UI build, never the
real PostHog project):
- Self-hosted server, no env: 0 requests for PostHog's script and 0
requests to `/_ph` in 9 cold loads and in a 40 s session with
navigation. `browserAnalytics` is `null`. `/_ph/e/` falls through to the
UI.
- Cloud config: a full Cloud boot needs S3, a managed database and the
control plane. So a small local stand-in served the same production UI
build, mounted the real proxy from `server/src/routes/posthog-proxy.ts`,
added the real `resolveBrowserAnalyticsConfig()` output to the session,
and forwarded other API calls to the local server.
`PAPERCLIP_POSTHOG_HOST` pointed at a local capture server.
- Captured through `/_ph` only (0 requests to any posthog.com host):
`$pageview` on each client navigation (5), `$identify` with `$set: {
email, name }`, `$groupidentify`, `$groups: { workspace: <stack id> }`
on events, `environment` and `deployment_source` on every event,
`$web_vitals`, `$autocapture` with no `$el_text`, `$$heatmap`, and 10
`$snapshot` replay events. A search of every decoded `/e/` and `/s/`
payload (events, replay text and replay attributes) found the email only
in `$identify`'s `$set`, and found none of: the task body, the typed
comment, the task title, the organization name. No Cookie header reached
the capture server. 96 of 113 API requests carried
`X-POSTHOG-SESSION-ID` (the others ran before posthog-js loaded).
- Client-side visits to `/invite/INVITESECRET123` and
`/cli-auth/abc?token=CLISECRET456` reached PostHog as `/invite/REDACTED`
and `/cli-auth/abc` (no query); neither secret was in any payload.
- Disallowed proxy requests (`PUT /_ph/e/`, `GET /_ph/api/projects/`,
`POST /_ph/static/x.js`) answered 404.
- Sample replay payload (decoded rrweb text nodes from a captured `/s/`
request). The task body, the typed comment, the task title, the
organization name and the email were not in any replay payload:

```json
{"text_nodes": [
  {"parentTag": "title", "textContent": "****** ** * *********"},
  {"parentTag": "a",     "textContent": "**** ** **** *******"},
  {"parentTag": "span",  "textContent": "****** **"},
  {"parentTag": "span",  "textContent": "*** ****"},
  {"parentTag": "span",  "textContent": "Tasks"},
  {"parentTag": "span",  "textContent": "Inbox"},
  {"parentTag": "h2",    "textContent": "******* *******"},
  {"parentTag": "p",     "textContent": "****** *** * ******* ** ******"}
]}
```

- Cold load, 9 fresh-profile loads each, median, same stand-in with
analytics off and on:

| | DOMContentLoaded | Board content visible |
| --- | --- | --- |
| Analytics off | 128 ms | 336 ms |
| Analytics on | 126 ms | 334 ms |

Final head. The difference is inside run-to-run noise (earlier pairs:
332/335 ms and 335/348 ms). PostHog's script loads only after the
content is visible.

## Risks

- Inert by default. Without a Cloud-managed stack, a project token, a
stack ID and `PAPERCLIP_POSTHOG_BROWSER_ENABLED=1`, the session field is
`null`, the proxy is not mounted and no analytics code runs. The OSS
package does not change behavior.
- Per-stack opt-out: `PAPERCLIP_POSTHOG_REPLAY_ENABLED=0` turns replay
off; unsetting `PAPERCLIP_POSTHOG_BROWSER_ENABLED` turns all browser
analytics off. Both apply on the next restart.
- Replay text shown in the clear comes only from fixed sidebar labels
and from shared components whose caller opts in. Today no `Button`,
`Label`, `TabsTrigger` or `EmptyState` call site opts in, so replays
show layout, clicks and navigation with most copy masked. Opting in
fixed-label call sites is a follow-up. Review sample replays on staging
before production.
- Image `src` values are masked, so images record as empty boxes.
Sandboxed iframes (HTML artifacts, MCP apps) also record as empty boxes.
Replays therefore show layout and product copy, not customer media.
- PostHog's script is not version-pinned: the assets host serves its
current browser build, as for any PostHog snippet install, and
posthog-js already loads its extensions from that host at run time. It
runs only on Cloud stacks with the browser switch on.
- The proxy needs no sign-in, like PostHog's documented proxies:
posthog-js sends some events while a page unloads, and the proxy can
only reach the two PostHog hosts, on the listed paths, with the public
project token. Body size and in-flight limits bound the load one client
can add.
- posthog-js rejects bots (including headless browsers) on its own,
which is the wanted production behavior.
- `SidebarNavItem.production.tsx`, `Sidebar*.tsx` and the shared `ui/*`
components changed by one attribute each; no visual change.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edits, headless browser runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 11:23:03 -06:00
DottaandClaude Opus 5.5 c366a606b1 feat(google-workspace): REST-backed Sheets tools in server/src/google-workspace-rest-tools (#22027)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents reach Google Workspace through the built-in Google connector
and its tool gateway
> - Google's hosted Workspace MCP servers are a Developer Preview, so
Paperclip will launch on its own tools over Google's REST APIs
(`server/src/google-workspace-rest-tools/`)
> - Gmail and Calendar are there; Sheets has no REST tools yet, so a
Sheets grant has no tools when the connector runs on REST
> - This pull request adds the six Sheets tools that `toolScopes.sheets`
allows, with the same names, inputs and result shapes as Google's Sheets
MCP tools
> - The benefit is that Sheets can switch to `paperclip_rest`, and later
back to Google's MCP servers, without a rewrite

## Linked Issues or Issue Description

Refs #22000
Refs #22017

**Subsystem affected**
Google Workspace REST tool set,
`server/src/google-workspace-rest-tools/`.

**Problem or motivation**
The REST tool set has no Sheets tools. Agents with a Sheets grant cannot
read or write spreadsheets when the connector runs on REST.

**Proposed solution**
One file per tool under `sheets/`, with pure argument-to-request and
response-to-result functions. Names, fields and annotations come from
Google's Sheets MCP reference
(https://developers.google.com/workspace/sheets/api/reference/mcp). Each
tool uses only the scopes its allowlist entry permits. The README lists
the endpoint, scope and Google documentation URL for each tool, the
value handling, the bounds, and every difference from Google's tools.

**Alternatives considered**
Reuse `packages/google-sheets-mcp-server`. It has its own tool names and
shapes, and the folder's boundary test forbids the import.

## What Changed

- `sheets/get-values.ts`: `get_values` over `spreadsheets.values.get`,
formatted values, rows first (the REST defaults, sent explicitly).
Google's answer is returned unchanged (`range`, `values`).
- `sheets/get-spreadsheet.ts`: `get_spreadsheet` over `spreadsheets.get`
with `includeGridData`, `ranges` and `fields`. `commentsIncluded: true`
is refused because REST cannot return comments.
- `sheets/update-values.ts` and `sheets/update-formulas.ts`:
`spreadsheets.values.update` with `valueInputOption` `RAW` and
`USER_ENTERED`. Each description names its option.
- `sheets/insert-dimension.ts`: one `insertDimension` request through
`spreadsheets.batchUpdate`.
- `sheets/update-spreadsheet.ts`: `spreadsheets.batchUpdate` with the 64
request types Google documents that REST supports. Comment and data
source requests, unknown request types and `writeControl` are refused
before sending. The tool is marked `destructiveHint: true` because
Google's list includes `deleteSheet`, `deleteDimension`, `deleteRange`
and other removing operations.
- `sheets/range.ts`: A1 and R1C1 range validation, sheet-name quoting,
and range descriptions for write results.
- Bounds: at most 50,000 cells read or written per call and 450,000
characters per read result (under the gateway's 1 MB limit). Over a
limit, the tool returns an error that asks for a narrower range. It
never returns a cut-off result.
- Write results: Google's REST response under `content`, plus a `status`
sentence that names the changed ranges and the number of cells.
- `google-http.ts`: allows `sheets.googleapis.com` only under
`/v4/spreadsheets/`.
- `probe.ts`: Sheets has no probe, because every Sheets read needs a
spreadsheet ID.
- `tools.ts`, `allowlist.test.ts`, `README.md`: register the service,
check Sheets 6 of 6, and document the tools.

## Verification

- `cd server && npx vitest run src/google-workspace-rest-tools`: 34
files, 647 tests pass. The Sheets tests use synthetic fixtures and a
fake fetch. There is no network and no real spreadsheet.
- Table-driven tests cover ranges (accepted and refused, quoting round
trips), request building for each tool, result shapes, cell and size
bounds, the uncertain-outcome warning on failed writes without a retry,
the host path restriction, the absent probe, and each tool's scopes.
- `npx tsc --noEmit -p server`: no errors in
`server/src/google-workspace-rest-tools/`. Other errors in this local
run come from workspace packages that were not built.
- `pnpm verify`: exit 2 (the README is a broad input). Required CI
supplies the broad coverage.

## Risks

- Low risk. Nothing calls these tools until Sheets is set to
`paperclip_rest` in `GOOGLE_WORKSPACE_TOOL_TRANSPORT`. That setting
stays `google_mcp`.
- Google's reference does not name the value input option of
`update_values` or the render option of `get_values`. This PR uses `RAW`
and formatted values. A recorded comparison on Google's servers can
confirm both. The README lists this.
- `update_spreadsheet` accepts the removing operations that Google
documents, such as `deleteSheet`. Approval and audit stay with the
gateway.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code
with tool use (shell, file edits, web fetch).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
canary/v2026.1117.0-canary.820000
2026-11-17 10:59:22 -06:00
DottaandPaperclip 9c776cdf39 fix(apps): show the selected ChatGPT workspace on permissions (#22023)
## Thinking Path

> - Paperclip helps people manage AI agents and their access to external
accounts.
> - Apps permissions shows the provider account behind a saved AI
connection.
> - One OpenAI email can have both a personal workspace and a business
workspace.
> - The current page shows the email but does not show the selected
ChatGPT workspace.
> - This pull request reads the provider name for the saved workspace ID
through the existing account identity endpoint.
> - People can identify the account that their agents will use before
they change its access.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The Connected account section on the Apps permissions page for OpenAI
subscription connections.

**Current behavior**

The page shows the saved email. It cannot distinguish two ChatGPT
workspaces that use that email. The account identity endpoint supports
only Anthropic profiles.

**Proposed behavior**

Show the selected ChatGPT workspace name beside the email. Show Personal
workspace when OpenAI returns the personal type without a name. Show an
unavailable state when the provider cannot supply a name.

**Reason and benefit**

People must know which provider workspace an agent will use. Paperclip
personal or shared access does not identify that workspace.

**Breaking changes**

None. The change uses the existing safe identity response and requires
no database migration.

No related open or closed pull request was found with `OpenAI workspace
in:title`. This change improves the existing Connected Apps and AI
Accounts items in ROADMAP.md.

## What Changed

- Add a provider module for OpenAI's Codex accounts endpoint. Match the
saved workspace ID in both supported response formats. Do not choose the
first or default account.
- Use the existing credential audience, company access, audit, timeout,
and response-size controls. Keep provider IDs, bearer tokens, and other
workspace names out of the response.
- Enable the OpenAI subscription account lookup on Permissions. Keep the
saved email and account controls when the lookup fails. Skip API keys
and routed connections.
- Add provider, UI, and authenticated route tests. Update the AI
Connections documentation with the upstream protocol sources.

## Verification

- Linux x64, Node v25.9.0, source checkout. Provider responses use test
fixtures.
- Red: the new provider and UI regression cases failed before the fix.
The page did not show the workspace and the service returned
unsupported.
- Green: `pnpm exec vitest run
server/src/services/ai-account-identity.test.ts
ui/src/components/ai-connections/AiConnectionAccountDetails.test.tsx` —
41 tests passed.
- Green: `pnpm exec vitest run
server/src/__tests__/ai-connections.test.ts -t 'identity only for an
authorized credential'` — 2 tests passed. These tests check both
providers, company and audience denial, invalid grants, agent denial,
provider denial, and revoked credentials.
- `git diff --check` passed.
- `pnpm verify` returned exit 2 because the documentation change
requires broader CI coverage. No full local suite was run.
Final-revision CI passed, as linked below.
- The first preflight could not install because the existing lockfile
omitted the server's PostHog dependency. A local install with
`--no-frozen-lockfile` prepared dependencies. The lockfile was restored
and is not part of this change. The route suite also needed the existing
plugin SDK and Runner TypeScript builds before it could load; it passed
after those builds.
- No live OpenAI account lookup or staging deployment was performed.

- Final head `47b0e10ee5707e6615ca56d518fbd0d03f0fb341`: [PR CI attempt
2](https://github.com/paperclipai/paperclip/actions/runs/44118848278)
passed, including
[typecheck](https://github.com/paperclipai/paperclip/actions/runs/44118848278/job/134175539941),
[build](https://github.com/paperclipai/paperclip/actions/runs/44118848278/job/134175461467),
[browser
CI](https://github.com/paperclipai/paperclip/actions/runs/44118848278/job/134175467287),
and [final
verification](https://github.com/paperclipai/paperclip/actions/runs/44118848278/job/134185142615).
Four jobs interrupted by runner shutdowns passed on retry.
-
[Greptile](https://github.com/paperclipai/paperclip/pull/22023#issuecomment-6953183751)
and [Paperclip
review](https://github.com/paperclipai/paperclip/pull/22023#issuecomment-6953104317):
5/5 on the same head, no findings or open review threads.

## Risks

- The workspace lookup uses the same private provider endpoint as
upstream Codex. Older credentials or provider changes can make names
unavailable. The page keeps the saved email and account controls in that
case.
- The provider protocol stays in the OpenAI identity module. The
connection service only selects the existing credential, and the UI
consumes the existing normalized labels.

## Model Used

OpenAI GPT-6 in Codex, with tool use and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-17 10:48:06 -06:00
DottaandClaude Opus 5.5 7f1f0eef67 fix(google-workspace): request free/busy with read & write Calendar, and update connector docs (#22026)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The built-in Google Workspace connector gives agents Gmail, Drive
and Calendar through one connection, behind an experimental setting
> - The user picks each service as read only or read & write. Paperclip
then requests only scopes from a fixed list of 14
> - Read & write Calendar requested `calendar.calendarlist.readonly` and
`calendar.events`. Our code assumed `calendar.events` also covers
free/busy lookups
> - Google's REST `freeBusy.query` does not accept `calendar.events`. So
the REST `suggest_time` tool cannot read free/busy on a read & write
grant, and falls back to reading each attendee's events
> - This pull request adds `calendar.events.freebusy` (already one of
the 14) to the read & write Calendar selection of the unified
connection, and stops treating `calendar.events` as implying free/busy
> - The benefit is that `suggest_time` on REST can see coworkers who
share only free/busy, and the stored grant no longer claims a permission
that Google did not give. It also brings three Google connector planning
docs up to date

## Linked Issues or Issue Description

Refs #22016, #22017, #22010, #22014

**What happened?**

The read & write Calendar selection of the unified Google Workspace
connection requests `calendar.calendarlist.readonly` and
`calendar.events`. `toolScopes.calendar.suggest_time` lists
`calendar.events` as sufficient, and `googleWorkspaceEffectiveScopes`
treats `calendar.events` as implying `calendar.events.freebusy`.

Google's two references differ:

- REST
[`freeBusy.query`](https://developers.google.com/workspace/calendar/api/v3/reference/freebusy/query),
checked 2026-11-17: "This request allows authorization with at least one
of the following scopes:" `calendar.readonly`, `calendar`,
`calendar.events.freebusy`, `calendar.freebusy`. `calendar.events` is
not in the list.
- Google's hosted Calendar MCP
[`suggest_time`](https://developers.google.com/workspace/calendar/api/v3/reference/mcp/tools_list/suggest_time),
checked 2026-11-17: "Requires one of the following OAuth scopes:"
`calendar`, `calendar.readonly`, `calendar.events`,
`calendar.events.freebusy`, `calendar.freebusy`. Here `calendar.events`
is accepted.

So `calendar.events` is enough for Google's MCP tool but not for the
REST endpoint that our REST `suggest_time` uses.

**Expected behavior**

A user who picks read & write Calendar gets a grant that allows
`freeBusy.query`. A grant that does not have the free/busy permission is
not recorded as having it. Grants made before this change keep working
as read & write.

**Steps to reproduce**

1. Call `googleWorkspaceSelectedScopes({ calendar: "write" })`. It
returns two scopes, without `calendar.events.freebusy`.
2. Call `googleWorkspaceEffectiveScopes(readOnlyCalendarScopes,
[calendar.events])`. It returns `calendar.events.freebusy`, which Google
did not grant.
3. With Calendar on the REST transport, `suggest_time` on a read & write
grant gets an insufficient-scope answer from `freeBusy.query` and uses
the events fallback.

## What Changed

- `packages/shared/src/google-workspace-connectors.ts`:
- `googleWorkspaceSelectedScopes` adds selection-level scopes from a new
`SELECTION_ADDED_SCOPES` map. Read & write Calendar adds
`calendar.events.freebusy`. The per-service `calendar.write` profile
(`SERVICE_PROFILES["calendar.write"]`) is unchanged, because the Cloud
broker matches legacy profiles by exact scope set.
- `googleWorkspaceEffectiveScopes` no longer expands `calendar.events`
to `calendar.events.freebusy`. `calendar.events` still implies
`calendar.events.readonly`. `calendar.readonly` still implies free/busy,
which Google's reference allows.
- `googleWorkspaceSelectionForScopes` is unchanged. It needs the legacy
write profile's scopes, so a two-scope grant and a three-scope grant
both read back as read & write.
- `toolScopes.calendar.suggest_time` is unchanged. It still accepts
`calendar.events`: Google's MCP tool accepts it, and on REST the tool
has an events-based fallback that names attendees it cannot read.
- Counts: default all-read-and-write selection is now 10 scopes (was 9);
all read-only stays 8; the union of all 27 selections stays exactly the
14.
- Tests (written first and seen failing on `master`):
- `packages/shared/src/google-workspace-connectors.test.ts`: scopes for
read and for read & write Calendar; the legacy `calendar.write` profile
keeps two scopes; default count 10 and read-only count 8; old two-scope
grant, new three-scope grant and partial consent (user declines
free/busy) all round-trip as read & write and keep `list_calendars`,
`list_events`, `suggest_time`, `create_event` and `delete_event`; no
recorded free/busy that Google did not grant. The existing exhaustive
test still checks that no selection requests a scope outside the 14.
- `server/src/__tests__/tool-access-service.test.ts`: customer-owned
Workspace OAuth with read & write Calendar asks Google for all three
scopes. With and without free/busy in Google's answer, the connection is
active, the Calendar tools are listed, the grant stores exactly what
Google granted, and it reads back as read & write.
- Stale tests on `master` fixed: `#22014` made Gmail 9 of 9 in the REST
tool set while `#22016` still expected 3 Gmail REST tools.
`google-workspace-rest-dispatch.test.ts` and
`google-workspace-rest-transport.test.ts` now derive the Gmail list from
the allowlist and use Chat as the service with no REST tools.
- Docs: `doc/connections/GOOGLE-WORKSPACE.md`,
`doc/connections/GOOGLE-WORKSPACE-SERVICE-CONSENT.md` and
`doc/connections/tool-method-permission-reviews.json` say ten scopes and
explain the Calendar change.
- Docs: `doc/memory/google-connector-memory.md` (shipped PRs #22010,
#22014, #22016, #22017; three Nov 17 decisions; "Where Google content is
kept" replaced with #22010's inventory, what deletion keeps and its
limits; Scopes section; REST tools status table; still-open list),
`doc/plans/2026-11-17-google-workspace-production-checklist.md` (help
page and retention audit ticked with #22010; REST tool progress),
`doc/plans/2026-11-17-google-workspace-code-fixes.md` (fix 6 done with
limits; section 9 status table; fix 10 note).
- `tests/runner-e2e/source-sdk-public-inputs.json`: refreshed pins for
the changed shared files.

Not changed:

- `server/src/google-workspace-rest-tools/**`. Another change owns that
folder. Its README open question about this scope is now resolved and
can be removed there.
- The picker disclosure (`googleWorkspaceDisclosure.ts`). Read & write
Calendar already says agents "can see your calendars, events and
free/busy times". With this change that line is accurate on REST too.

Behavior for existing grants: a saved read & write grant without
free/busy still reads back as read & write, keeps every Calendar tool,
and refreshes without change. Nothing prompts the user to reconnect. A
user who chooses read & write Calendar in the picker is asked for the
new scope. An account-page reconnect without a new choice still restores
the exact saved request, as before.

Cloud broker and Google Console: no change needed.
`calendar.events.freebusy` is one of the 14 in `workspace.all`, the
broker accepts any nonempty subset of `workspace.all`
(`doc/connections/GOOGLE-WORKSPACE-SERVICE-CONSENT.md`), and the legacy
`calendar.write` profile's exact scope set did not change.

## Verification

- `cd packages/shared && npx vitest run
src/google-workspace-connectors.test.ts src/app-definitions.test.ts`: 55
passed. On `master` the new shared tests fail (3 failed).
- `cd server && npx vitest run src/__tests__/tool-access-service.test.ts
-t "free/busy"`: 2 passed. With `master`'s
`google-workspace-connectors.ts`: 2 failed.
- `cd server && npx vitest run
src/__tests__/google-workspace-rest-transport.test.ts
src/services/google-workspace-rest-dispatch.test.ts`: 29 passed (4
failed on `master` before this change).
- `cd ui && npx vitest run
src/features/connections/GoogleWorkspaceServices.test.tsx
src/pages/apps/AppDetail.test.tsx`: 115 passed.
- `cd packages/shared && npx tsc --noEmit -p .`: clean.
- `pnpm verify` exits 2 (a global CI input changed), so required CI
supplies the broad coverage.
- Docs facts checked against `gh pr view` for #22010, #22014, #22016,
#22017, the code, and paperclipai/paperclip-website PR 197 (still a
draft; does not mention OpenRouter).

## Risks

- Low risk. The submitted set of 14 does not change, and the broker
contract does not change.
- New read & write Calendar grants record `calendar.events.freebusy`
next to `calendar.events`. Code that reads stored scopes sees one more
scope. All consumers (callback, refresh, discovery, picker pre-fill) are
covered by the tests above.
- A read-only Calendar request on an account that Google reports only
`calendar.events` for no longer records free/busy. That matches what
Google granted; `suggest_time` is then not offered on that grant.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use, web fetch and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
canary/v2026.1117.0-canary.819800
2026-11-17 16:40:58 +00:00
DottaandClaude Opus 5.5 01451fae5b Let plugins add connector types to the Connectors catalog (#22004)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents use external services through connectors: a catalog of
connector types, connections (accounts) per type, and per-agent
installs, policies and audit in the tool gateway
> - Today only core can add a connector type. Every type is a JSON
definition compiled into `@paperclipai/shared`
> - Plugins can add agent tools, but those tools skip connections,
grants, per-agent installs and the Connectors page
> - This pull request lets a plugin add connector types with the same
definition schema, and makes them behave exactly like built-in
connectors
> - The benefit is that a connector can ship as a plugin, and an
operator controls it with the same Connectors page and the same access
rules

## Linked Issues or Issue Description

**Subsystem affected**
Plugins, Connectors (tool access), tool gateway

**Problem or motivation**
A plugin cannot add a connector type to the Connectors catalog. Plugin
tools (`tools` in the manifest) do not use connections, credentials per
account, agent installs, policies or the Connectors page.

**Proposed solution**
Add a manifest field `apps` (gated by a new `connectors.provide`
capability). Each entry uses the built-in app-definition schema. The
host namespaces the slug by plugin key and runs setup, OAuth, discovery,
grants, policies and audit through the existing code. This is the first
of two pull requests. The second adds a `plugin` transport so a plugin
worker can serve the tools itself.

**Activation rules (product requirements)**
1. Installing or enabling a plugin grants no tools. It only adds
connector types to the catalog.
2. Connecting an account grants no tools.
3. An agent gets the tools only from an agent install or a company
install of an active connection. Discovery and invocation both enforce
this.
4. A disabled or uninstalled plugin (or an upgrade that removes the app)
hides its connector types and connections from the Connectors APIs and
they reach no agent. Saved setup survives disable and comes back on
re-enable. Uninstall keeps rows archived for audit and revokes
credentials.


**Alternatives considered**
Keep plugin tools (`tools`) as the only plugin path. Rejected: they
bypass connections, per-account credentials, per-agent installs and the
Connectors page.

## What Changed

- `packages/shared`: new capability `connectors.provide`; manifest
`apps` validated with `appDefinitionSchema`. The validator rejects apps
without the capability, duplicate slugs, host-derived fields (`plugin`,
`aiConnectionRouter`, `experimentalFeatureKey`, `availability`), and
methods that need provider code in the host (REST adapters, chat
channels, AI runtimes).
- `packages/shared/src/plugin-apps.ts`: reversible namespaced slugs
(`plugin-<hex plugin key>-<slug>`), the catalog entry builder, and
`reviewedToolRisk`.
- `packages/shared/src/app-definitions.ts`: catalog lookups
(`getConnectableAppDefinition`, `getAppStoreDefinition`, ...) also
resolve the current plugin apps. The set is replaced on each read of
plugin state, so a built-in app is never shadowed.
- App definitions can declare `reviewedTools` (reviewed
read/write/destructive names and the risk for unlisted tools).
Circleback's reviewed risk moves from
`server/src/services/connectors/circleback/risk.ts` into
`circleback.json`, with the same classification. A copy of the
definition now classifies tools the same way.
- `validators/json-schema.ts`: `jsonSchemaSchema` moves to its own
module to break an import cycle between the plugin and app-definition
validators (still re-exported from `validators/plugin.ts`).
- `server/src/services/plugin-app-catalog.ts`: reads ready plugins from
the `plugins` table and answers "is this connection's connector
available". It does not trust in-memory lifecycle events, because
several lifecycle managers and processes can share one database.
- Gateway: connected-tool discovery and `resolveConnectionGrant` drop
connections from an unavailable plugin connector. Built-in connections
do not read the plugins table.
- Connectors APIs: the gallery lists ready plugin apps (with the same
Cloud connector profile filter as core); `listConnections` and
`listApplications` hide unavailable plugin connections; setup entry
points refresh plugin apps first.
- Plugin uninstall archives the plugin's connections through the
standard connection removal (grants revoked, owned credentials deleted,
rows kept).
- UI: the gallery response registers plugin apps for the setup screens;
`/apps/connect?source=plugin-…` loads the gallery first and shows an
unavailable state when the plugin is disabled.
- Docs: `doc/plugins/PLUGIN_SPEC.md` section 11A and the capability
list.

Design choices where the spec left room (picked to match "a plugin
connector behaves like a built-in connector"):
- Only `ready` plugins provide connectors. `error` and `upgrade_pending`
hide them, the same as the AI connection router plugins.
- `rest_api` is rejected for plugin apps in this PR. Core REST
connectors need reviewed adapter code in the host, so a definition alone
cannot work. The `plugin` transport in the next PR covers that case.
- Disabled plugin connections are hidden from list endpoints. Reading
one connection by id still works, so audit links keep resolving.

## Verification

- `packages/shared`: `npx vitest run src/plugin-apps.test.ts
src/validators/ src/app-definitions.test.ts` (416 passed). New tests:
manifest validation, slug namespacing, no shadowing of built-in apps,
reviewed risk.
- `server`: `npx vitest run
src/__tests__/plugin-connectors-circleback-parity.test.ts` (new). A
test-only plugin
(`server/src/__tests__/fixtures/plugin-connectors/circleback-parity/`)
ships an unmodified copy of `circleback.json`. The same journey runs
against built-in Circleback and the plugin copy with a recorded DCR
OAuth + MCP fixture, and must give identical catalog risk/status,
suggested defaults and per-agent tool sets. It covers each rule:
- Rule 1: the gallery adds only the namespaced type; no connections
exist; no agent tool changes.
  - Rule 2: two signed-in accounts, no agent sees a tool.
- Rule 3: an install for agent A gives only A the tools of that
connection; agent B is denied on invocation; each account calls the
provider with its own credential.
- Rule 4: disable hides the type, both connections and applications,
removes tools from discovery and invocation (404) and blocks new setup,
while built-in Circleback is unchanged; installs and grants are
byte-identical; re-enable restores tools with the same names; an upgrade
that removes the app hides it; uninstall archives both connections,
revokes grants and deletes owned secrets.
- Red/green: removing the gateway gate or the declared review makes the
test fail.
- An env-gated live check (`PAPERCLIP_CIRCLEBACK_LIVE=1`) starts the
real Circleback sign-in for both connectors and compares the requests.
It never runs in CI. I did not run it for this PR.
- Regression: `ai-connection-router`, `connector-tool-policy`,
`mcp-apps`, `plugin-lifecycle-restart`, `tool-access-service`,
`braintrust-connection`, `remote-mcp-connectors`,
`connection-intents-service`, `tool-gateway` (796 passed).
- `ui`: `app-connect-policy.test.ts`, `AppsConnect.test.tsx` (new plugin
cases) pass.
- `tsc --noEmit` for shared, server, ui (only pre-existing ungenerated
Storybook catalog errors), plugin SDK and CLI.
- `pnpm verify --explain` selects a broad plan because a doc under
`doc/plugins` changed; CI supplies that coverage.

## Risks

- No migration. Built-in connectors only gain one string check per
discovery; the plugins table is read only when a plugin connection is
present.
- Circleback's reviewed risk is now data. The behavior is the same, and
the existing policy tests and new tests cover it.
- Connection intents (agent connection requests in chat) still search
only built-in apps. Plugin connectors can be set up from the Connectors
page. This is a follow-up.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file editing, test execution).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 15:51:39 +00:00
DottaandClaude Opus 5.5 1d3b4201e3 feat(google-workspace): remaining Gmail read tools in the REST tool set (#22014)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents use Gmail through the built-in Google Workspace connector
> - Google's hosted Gmail MCP server is a Developer Preview, so
Paperclip launches on its own REST-backed tool set in
`server/src/google-workspace-rest-tools/` (#22000)
> - That folder has 3 of the 9 Gmail tools on the reviewed allowlist
(`toolScopes.gmail`)
> - Agents cannot read a whole thread, list drafts, list labels or
search single messages
> - This pull request adds the 6 missing read tools, with the same
names, inputs and results as Google's MCP tools where Google documents
them
> - The benefit is that Gmail is complete on the REST transport, and a
message looks the same from every tool

## Linked Issues or Issue Description

Refs #22000

**Problem**
The REST-backed Gmail tool set implements only `get_message`,
`search_threads` and `create_draft`. The allowlist also names
`get_thread`, `get_draft`, `list_drafts`, `list_labels`, `list_threads`
and `search_messages`.

**Proposed solution**
Implement the 6 read tools in the folder. Follow the existing patterns:
one file per tool, pure request and result functions, synthetic unit
tests.

**Why it matters**
Google's reviewers compare an email in Paperclip with the same email in
Gmail. Full threads and drafts must show complete bodies.

## What Changed

- Add `get_thread`, `list_drafts` and `list_labels`. Their schemas come
from Google's Gmail MCP reference.
- Add `get_draft`, `list_threads` and `search_messages`. Google's
reference does not list them, so their schemas copy the sibling tools.
The README lists them under "Not verified against Google's servers".
- Move separately stored body reads from `get_message` into
`readMessageBodies` in `gmail/message.ts`. All tools that return bodies
use it. A body that cannot be read fails the call with "Could not read
the full body of this message".
- Add `gmail/parallel.ts`: a 10-at-a-time read pool that stops after the
first failure, and a helper that skips items deleted between list and
get.
- Cap page sizes (50 for threads, messages and drafts; 500 for labels,
which Gmail returns in one request). The README has a table of Google
requests per call at the maximum page size.
- Update descriptions: `search_threads` now points to `get_thread` and
`list_labels`. The `label:` search operator takes label names (spaces as
hyphens), not IDs. This corrects the wording in the original tools.
Google's MCP reference says IDs; the README records the difference, and
a test prevents drift.
- Cap separately stored body reads at 20 per call. The README lists
requests and quota units per call at the maximum page size.
- `allowlist.test.ts` now asserts that Gmail implements all 9
allowlisted tools and that only `create_draft` writes.

## Verification

- `cd server && npx vitest run src/google-workspace-rest-tools`: 15
files, 156 tests pass. Tests use synthetic fixtures and a fake `fetch`
only.
- `cd server && npx tsc --noEmit -p .`: passes.
- `pnpm verify`: exit 2 (the README is a broad input). Required CI
supplies the broad coverage.
- Tests prove: each tool maps messages exactly as `get_message` does;
separately stored bodies are fetched; a failed body read is an explicit
error; request counts at the maximum page size (51); invalid IDs send no
request; caller cancellation stops the call.

## Risks

- Low risk. Nothing outside the folder imports it yet
(`boundary.test.ts`).
- `get_draft`, `list_threads` and `search_messages` have no Google
schema. A switch back to Google's MCP servers would drop them unless
Google adds them.
- `list_labels` leaves out `threadsTotal` and `threadsUnread`: REST
returns them only with one request per label.
- One call reads at most 20 separately stored bodies. Beyond that it
returns an explicit "Could not read the full body of every message"
error and fetches none. The worst single call is 71 requests and 3,405
Gmail quota units (per-user limit: 6,000 per minute). A second maximal
call in the same minute can get a `rate_limited` tool error.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (file edits, shell, web fetch of Google's public reference).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 09:51:17 -06:00
DottaandClaude Opus 5.5 c652c813bc feat(google-workspace): per-service transport switch for REST-backed tools (off by default) (#22016)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents use Google Workspace through one built-in `google-workspace`
connection and the tool gateway
> - Today every Google tool call goes to Google's hosted Workspace MCP
servers, which are a Developer Preview and cannot be offered to the
public
> - #22000 added Paperclip's own Gmail tools over Google's REST APIs in
`server/src/google-workspace-rest-tools/`, but nothing calls them yet
> - This pull request adds one per-service transport switch and connects
the gateway to the REST tools behind it. Every service stays on Google's
MCP servers, so production behavior does not change
> - The benefit is that a service moves to the REST tools with a
one-line, reviewed change once all its tools exist, and moves back the
same way

## Linked Issues or Issue Description

Refs #22000

**Subsystem affected**
Tool gateway and catalog discovery for the built-in Google Workspace
connection.

**Problem or motivation**
Google's Workspace MCP servers are a Developer Preview. Paperclip
launches on its own tools over Google's REST APIs. The REST tools exist
(#22000), but the gateway cannot call them, and there is no way to
choose them per service.

**Proposed solution**
Add a per-service transport setting (`google_mcp` or `paperclip_rest`)
in the shared Google Workspace module. Add one server adapter that is
the only importer of the REST folder. When a service is set to
`paperclip_rest`, the gateway call path and discovery use the adapter in
process. All gateway controls stay the same.

**Alternatives considered**
A loopback or hosted MCP endpoint for the REST tools. It adds a
connection, JSON-RPC framing and the protocol probe to every call, and
authenticated public deployments refuse loopback addresses. An
environment variable or instance setting. A plain constant keeps every
flip a reviewed code change.

## What Changed

- `packages/shared/src/google-workspace-connectors.ts`: adds
`GOOGLE_WORKSPACE_TOOL_TRANSPORT` (every service is `google_mcp`),
`transport` on the result of `googleWorkspaceToolTarget`, and
`googleWorkspaceServiceToolNames`.
- New `server/src/services/google-workspace-rest-dispatch.ts`: the only
importer of the REST folder. It keeps only the bearer token from the
gateway's credential headers, passes the gateway's egress-guarded fetch
and cancellation signal, and returns either the MCP result or a token
problem. `googleWorkspaceRestTransportGaps()` lists allowlisted tools
that a `paperclip_rest` service does not implement. Today Calendar is
eligible (9 of 9) and Gmail is not (3 of 9); nothing is flipped.
- `server/src/services/tool-gateway.ts` (`executeRemoteHttpTool`): for a
`paperclip_rest` tool, the gateway rechecks the current grant, calls the
adapter, and returns through the same result handling as the MCP path.
Google's insufficient-scope answer maps to `oauth_insufficient_scope`. A
rejected token forces a refresh and returns
`oauth_refreshed_retry_required` without a replay. A passed deadline
returns `tool_timeout`. The audit record names the protocol `Google
Workspace REST`. The grant recheck is now one shared helper for both
paths. A timed-out Google Workspace write, on either transport, now
returns `tool_timeout` with a warning that the change may or may not
have been made; a timed-out read keeps the plain timeout message.
- `server/src/services/tool-access/catalog/discovery.ts`: for a
`paperclip_rest` service, discovery and health run one REST probe with
the caller's token, then list the REST tools under the same names and
scope filter. A 401 refreshes and probes again. A second 401 on a
managed grant marks it `needs_reauthorization`. Any other probe failure
makes only that service unavailable.
- `server/src/google-workspace-rest-tools/`: adds `probe.ts` and its
export. Each service has an ordered list of cheap reads with the scopes
that allow each (Gmail: `users.getProfile`; Calendar: one calendar list
entry, one primary-calendar event, or one minute of free/busy). The
probe uses the first read the grant allows. A grant that allows none is
listed without a probe, and an insufficient-scope answer never drops a
service. It also and lets `boundary.test.ts` allow exactly the adapter
as the importer. The README now describes the implemented seam, how to
flip a service, and the removal procedure.
- `doc/connections/GOOGLE-WORKSPACE.md`: describes the transport
setting.
- `tests/runner-e2e/source-sdk-public-inputs.json`: refreshed pins for
the changed shared files.

## Verification

- `cd server && npx vitest run
src/__tests__/google-workspace-rest-transport.test.ts`: 15 tests. They
were red before the gateway and discovery changes (10 failed), and green
after. With Gmail set to `paperclip_rest` through a test-only change of
the map, the tests use the real managed setup, discovery, gateway,
policy and screening with a fake Google fetch. They prove: discovery
lists the same tool names filtered by scopes after one probe; a read
runs with the caller's token and is audited; a draft waits for approval
and then runs; a tool outside the grant and a narrowed grant are refused
before any Google request; Google's insufficient scope maps to
`oauth_insufficient_scope`; a 401 refreshes the token, is not replayed,
and the retry uses the new token; an injected email is blocked; a passed
deadline aborts the Google request and returns `tool_timeout`; the
experimental-feature gate stops calls and discovery; a failed Gmail
probe leaves Docs usable and Gmail comes back; a lasting 401 asks for
reconnection. A timed-out draft carries the uncertain-write warning in
the error, the invocation record and the audit entry on both transports,
and an approved REST draft whose request was lost returns the warning.
With the default map, a call goes to `gmailmcp.googleapis.com` and the
adapter is never called.
- `cd server && npx vitest run
src/services/google-workspace-rest-dispatch.test.ts
src/google-workspace-rest-tools`: eligibility (no partial flip),
bearer-only forwarding, signal and fetch pass-through, token outcome
mapping, probe, boundary.
- `cd packages/shared && npx vitest run
src/google-workspace-connectors.test.ts`: every service defaults to
`google_mcp`.
- Existing Google tests are unchanged and pass:
`google-workspace-injection-screening.test.ts`, the Google cases in
`tool-gateway-service.test.ts` and `tool-access-service.test.ts` (142
Workspace/Google/Gmail cases).
- `npx tsc --noEmit` for `packages/shared` and `server`: clean.
- `pnpm verify`: exit 2 (a doc under `doc/` is a global CI input), so
required CI supplies broad coverage.
- Speed, stubbed fetch, 300 calls after warm-up: the adapter adds about
15 µs per call. A full `gmail__get_message` through `executeTool` took a
median of 18.2 ms on REST and 18.3 ms on MCP (p90 21.6 ms and 22.3 ms).
The REST path adds no waits, no extra round trips and no dynamic
imports.

## Risks

- Low risk for production: every service stays on `google_mcp`, and the
default-map test proves the adapter is not called.
- The server now loads the REST folder at start. It uses only modules
the server already loads (MCP SDK, zod).
- A later flip must add a probe and the API host for each new REST
service. The `Record` type of the probe table makes typecheck fail when
a REST service has no probe.
- On `paperclip_rest`, MCP resources, MCP Tasks and elicitation do not
exist for that service.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code
with tool use (shell, file edits, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 09:49:49 -06:00
Nicky LeachandClaude Opus 5.5 79c8cf8b6b fix(heartbeat): record environment drift once for plugin-owned workspaces (#21947)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Before each run, the heartbeat compares a fingerprint of the
workspace config with the fingerprint saved on the reused execution
workspace
> - An environment change is replacement-class drift. When the task asks
to reuse its workspace, the "replace" decision does not replace the
workspace. It only keeps the old fingerprint, so the drift stays visible
> - A workspace-provider plugin owns the files of the workspaces it
prepares. The environment only decides where the runner executes, so an
environment change cannot make those files wrong
> - A managed sandbox environment can change its template id with each
release. Each existing plugin workspace then logs a "replace" freshness
notice on every later run, and nothing ever clears it
> - This pull request records environment drift as a refresh for
plugin-owned workspaces, and makes Core's fingerprint win over the
provider row's stale copy
> - The benefit is that the notice appears once after a real change and
then stops. Reuse does not change

## Linked Issues or Issue Description

Refs #19349. That PR stopped cosmetic sandbox updates from advancing the
environment revision. This PR covers real config changes on plugin-owned
workspaces.

**What happened?**

A managed plugin environment changed its `driverConfig` (a new sandbox
template id after a release). After that, every run on an existing
plugin workspace printed:

```
[paperclip] Execution workspace reuse freshness action "replace" because execution workspace configuration changed: environment.
```

The run result showed `workspaceReused: true`, with the same workspace
id before and after. The stored fingerprint stayed the same on every
run, so the notice repeated forever.

**Expected behavior**

The workspace is reused, as it is now. The new fingerprint is saved, the
drift is reported once, and later runs report no drift.

**Steps to reproduce**

1. Prepare a plugin-owned workspace with a plugin environment.
2. Change the environment's `driverConfig`.
3. Run the task twice more with workspace reuse. Both runs report
`replace` for `environment`, and the stored `configFingerprint` does not
change.

## What Changed

- `server/src/services/heartbeat/workspaces.ts`:
`resolveExecutionWorkspaceConfigFreshness` takes an optional
`providerOwnedStorage` flag. With the flag, `environment` drift does not
require replacement. Other replacement-class drift (repo, strategy,
mode, project workspace, realization) still gives `replace`.
- `server/src/services/heartbeat/workspace-preparation.ts`:
  - Pass `providerOwnedStorage: Boolean(pluginPrepared)`.
- The plugin path spread the provider row's metadata over Core's
metadata. That row still holds the fingerprint it was prepared with, so
a refresh could never save the new one. Core's `configFingerprint` now
wins. The rest of the provider metadata, including its `config`
snapshot, is unchanged.
- `doc/heartbeat-internals.md`: describe the provider-owned storage
rule.

The freshness decision never controls reuse here. With reuse requested,
restore is `requestedShouldReuseExisting` for every action, and the
plugin picks the workspace before the decision is made. This change only
alters what Core saves on the workspace row and what it reports.

## Verification

- Red first. Without the fix, both new tests failed: the decision was
`replace`, not `refresh`.
- With only the resolver change, the preparation test still failed
because the stored fingerprint did not change. This shows the metadata
merge fix is necessary.
- `npx vitest run src/__tests__/heartbeat-workspace-session.test.ts
src/services/heartbeat/workspace-preparation.test.ts` (in `server/`): 2
files, 333 tests pass.
- `npx tsc --noEmit -p tsconfig.json` (in `server/`): passes.
- New tests:
- `heartbeat-workspace-session.test.ts`: environment drift on
provider-owned storage gives `refresh` and persists the fingerprint. The
same drift on Core-owned storage still gives `replace`. Environment plus
repo drift on provider-owned storage still gives `replace`.
- `workspace-preparation.test.ts`: a new `environmentDrift` case runs
three preparations on one plugin workspace. Run 2 records one `refresh`
freshness operation for `environment` with `workspaceReused: true` and
saves a new fingerprint. Run 3 records no freshness operation.
- Not run locally: `pnpm verify` and the full suite. CI supplies that
coverage.

## Risks

- Low risk. Reuse does not change, and no schema changes.
- Behavioral shift: a plugin-owned workspace whose environment changed
now saves the new fingerprint. A later return to the old environment
config is reported as drift again, as for any other refresh.
- If a future plugin environment change does affect the files of a
provider-owned workspace, this rule would hide it. The provider owns
those files, so the provider is responsible for that check.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, test runs) and extended thinking.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
canary/v2026.1117.0-canary.819400
2026-11-17 07:18:20 -08:00
DottaandClaude Opus 5.5 c41ca4a992 feat(google-workspace): REST-backed Calendar tools in server/src/google-workspace-rest-tools (#22017)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents use Google Workspace through the built-in Google connector
and its tool gateway
> - Google's hosted Workspace MCP servers are a Developer Preview, so
Paperclip launches on its own tools over Google's REST APIs
(`server/src/google-workspace-rest-tools/`, added in #22000 with three
Gmail tools)
> - Calendar has nine allowlisted tools and none of them exist in the
REST tool set yet
> - The tools must keep Google's MCP names, inputs and results, so
Paperclip can switch back to Google's servers later
> - This pull request adds the nine Calendar tools, and the HTTP support
they need, inside the isolated folder
> - The benefit is that agents can read and change calendars with
exactly the scopes the allowlist grants

## Linked Issues or Issue Description

Refs #22000

**Feature description**
Add the nine Calendar tools that `toolScopes.calendar` allows to the
REST-backed Google Workspace tool set: `list_calendars`, `get_event`,
`list_events`, `search_events`, `suggest_time`, `create_event`,
`update_event`, `delete_event` and `respond_to_event`.

**Problem it solves**
The REST tool set has only Gmail tools. A Calendar grant has no tools
when the connector runs on REST.

**Proposed solution**
One file per tool under `calendar/`, with pure argument-to-request and
response-to-result functions. Names, fields and annotations come from
Google's Calendar MCP reference. Each tool uses only the scopes its
allowlist entry permits. The README lists the endpoint and scope for
each tool, the time rules, the notification defaults, the `suggest_time`
algorithm, and every difference from Google's tools.

## What Changed

- `calendar/`: nine tools, shared event mapping (`event.ts`), input
building (`event-input.ts`), JSON schemas, synthetic fixtures, and
`probeCalendar()` (one `calendarList.list` with `maxResults=1`) for a
health check.
- `calendar/time.ts`: time zone arithmetic with `Intl` only. It reads
offset-less times in a given IANA zone, handles DST gaps and repeated
hours (Temporal `compatible` rule), and never guesses a zone.
- `google-http.ts`: adds `www.googleapis.com` to the allowed hosts, but
only under `/calendar/v3/` (checked after URL normalization). Adds
PATCH, PUT and DELETE. All are writes: never retried, and failures after
dispatch carry the uncertain-outcome warning. A POST marked `readOnly`
(free/busy) is a read. Adds `If-Match`. Maps 412 to "nothing was
changed" and 410 to not found.
- `tools.ts`: registers the `calendar` service.
- `allowlist.test.ts`: asserts Calendar is fully implemented (9 of 9).
- `README.md`: Calendar section (endpoints and scopes, time handling,
notifications, `suggest_time`, differences) and two open questions.

## Verification

- `cd server && npx vitest run src/google-workspace-rest-tools`: 19
files, 330 tests pass. Of these, about 210 are new Calendar tests in 11
files, plus new `google-http` tests for PATCH, PUT, DELETE, read-only
POST, `If-Match`, 410, 412 and the path restriction, and the 9-of-9
allowlist assertion. Table-driven tests cover all-day versus timed
events, IANA zones with 30-minute and half-hour offsets, DST gaps and
repeated hours in New York, London and Lord Howe, recurring series and
single occurrences, pagination (including `startTimeDesc` paging),
free/busy fallback, notification mapping, and write failures (no retry,
uncertain-outcome warning). All fixtures are synthetic; no network.
- `cd server && npx tsc --noEmit -p .`: passes.
- `pnpm verify`: exit 2 (the README is a broad input), so required CI
supplies broad coverage.

## Risks

- Low risk. Nothing outside the folder imports it yet, so no runtime
path changes until the gateway wiring lands.
- Write tools follow Google's documented notification defaults:
`create_event`, `update_event` and `delete_event` email all attendees by
default; `respond_to_event` emails external attendees by default. Tool
descriptions say this, and `NONE` sends nothing.
- `freeBusy.query` does not accept `calendar.events`. A read and write
grant has no free/busy scope, so `suggest_time` falls back to reading
each attendee's calendar. That fallback cannot see coworkers who share
only free/busy. The README proposes adding `calendar.events.freebusy` to
the `calendar.write` profile.
- `delete_event` refuses a whole recurring series on purpose: it deletes
single events and single occurrences only, and sends no delete for a
series ID. This differs from REST and from Google's tool. `update_event`
and `respond_to_event` on a series ID change the whole series; their
descriptions say so, and the result shows series (`recurrence`) or
occurrence (`recurringEventId`).
- `list_events` page tokens carry the first page's time range, so later
pages stay consistent.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with
tool use (shell, file edits, web fetch of Google's public reference
pages).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 15:01:29 +00:00
DottaandClaude Opus 5.5 d1b5cfe038 feat(connections): let users delete data read through a connection (#22010)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents call connected apps, such as Google Workspace, through
Paperclip's tool gateway
> - Paperclip saves part of each tool result, approval previews, app
views and the full run transcripts that contain tool output
> - No user action deleted this data. Removing the connection left it in
place
> - Google's restricted-scope review requires that users can delete the
data an app obtained, and that the privacy policy says how
> - This pull request adds "Delete data read through this connection"
for any connection, on the connection's page and as an option when you
remove it
> - The benefit is that the picker text ("They stay there until you
delete them") is now true, and the privacy policy can link to a help
page

## Linked Issues or Issue Description

**Subsystem affected**

Connections (tool gateway audit tables, run logs, run events) and the
Connectors UI.

**Problem or motivation**

Paperclip stores content from a connection's tool calls in several
places and gives users no way to delete it:

- `tool_invocations.arguments_summary/result_summary` and
`tool_call_events.arguments_summary/request_summary/result_summary` keep
the first ~4,000 characters of each redacted value
(`summarizeToolValue`, `tool-access-policy.ts` `redactArguments`). No
retention limit.
- `tool_action_requests` keeps `canonical_arguments_summary`,
`preview_markdown` and `signed_arguments`. The linked approval card
(`issue_thread_interactions`) keeps copies in `payload.detailsMarkdown`,
`payload.toolAction.previewMarkdown`,
`payload.toolAction.argumentsSummaryJson` and
`result.toolAction.resultSummary`.
- `tool_mcp_app_views` keeps `tool_input` and `tool_result` for MCP app
views.
- Run transcripts hold full tool output: the run log (`local_file` with
an optional S3 mirror, or `s3_segments`/`s3_archive` object journals),
`heartbeat_run_events.message/payload`, and
`heartbeat_runs.stdout_excerpt/stderr_excerpt`. Neither run-log store
had a delete path.
- Connection removal (`removal.ts`) archives the connection and revokes
credentials. It deletes no content.

**Proposed solution**

A durable, resumable, company-scoped deletion request per connection.
See "Design" below.

**Alternatives considered**

- Delete synchronously in the request. Rejected: an interrupted request
leaves partial state that needs a person to retry it.
- Delete only the run events that mention the connection. Rejected: tool
output also reaches the model's later messages, so the safe unit is the
whole run transcript.
- Delete the audit rows. Rejected: counts, outcomes and hashes feed the
activity history and usage counts.

## Design

**Rule.** The job never drops a reference to content it has not
confirmed deleted. It is marked complete, and writes the
`tool_connection.content_deleted` activity entry, only after every step
succeeded. A step that cannot confirm its work throws. The request then
stays pending, the UI says "Paperclip keeps trying until it finishes",
and the worker retries it.

**Store inventory.** Every place that stores tool arguments, results or
previews. I found them from `summarizeToolValue`, `redactArguments`,
`writeAudit`, and every column or JSON field named like summary,
preview, arguments, result, toolInput or toolResult.

| Store | Content | Cleared by this PR | Proof |
| --- | --- | --- | --- |
| `tool_invocations.arguments_summary`, `result_summary`, `progress` |
Redacted request/result preview, provider progress | Yes: `summary`
becomes `""` with `contentDeletedAt`; `progress` null; size and sha256
kept | content-deletion test 1 |
| `tool_call_events.arguments_summary`, `request_summary`,
`result_summary` | Same previews | Yes, same | test 1 |
| `tool_access_audit_events.details.argumentsSummary` (`writeAudit`) |
Argument preview | Yes, same; decision, ids, size, hash kept | test 1 |
| `tool_action_requests.canonical_arguments_summary`,
`preview_markdown`, `signed_arguments` | Approval preview and arguments
| Yes, for settled requests | test 1, card retry test |
| Approval card in `issue_thread_interactions`:
`payload.detailsMarkdown`, `payload.toolAction.previewMarkdown`,
`payload.toolAction.argumentsSummaryJson`,
`result.toolAction.resultSummary` | Approval preview, arguments,
executed result | Yes. Cards are found from every settled request of a
batch, in the same transaction | test 1, card retry test |
| `heartbeat_runs.context_snapshot` and `agent_wakeup_requests.payload`
(`toolAction(s)[].resultSummary`,
`paperclipAgentMessage.untrustedToolResults[].resultSummary`) | Approved
action result delivered to the agent | Yes, for settled runs and settled
wakes | test 1 |
| `tool_mcp_app_views.tool_input`, `tool_result`, `model_context` | App
view input, result, shared context | Yes: rows deleted | test 1 |
| Run log: object journal, archive journal, original `local_file` log
and its legacy S3 mirror | Full transcript | Yes, every copy, for each
settled run that called the connection or received an approved result |
run-log-store tests, test 1 |
| `heartbeat_run_events.message`, `payload` | Transcript events | Yes,
except structural events (lifecycle, terminal, presentation, usage,
compaction, session/turn start, `native.*`) | test 1 |
| `heartbeat_runs.stdout_excerpt`, `stderr_excerpt`, and every string
`summary`, `result`, `message`, `text`, `output`, `content` or `body` at
any depth of `result_json` (including `nativeResult`, `acceptedResult`,
`semanticResult` and their `continuation`) | Output excerpts and final
answer | Yes; schema and control fields kept | test 1, canary |
| `approvals.payload` (formal board approval) | Tool name, risk and
hashes only | Not needed: no content | — |
| `tool_mcp_tasks` | Request hash and sealed provider task id; no result
| Not needed: no content | — |
| `tool_mcp_elicitations.questions` and their question cards | Questions
the provider asked a person | No. These are provider prompts, not tool
results, and the answered question stays in the task. Follow-up if
wanted | — |
| `tool_mcp_skills`, `tool_mcp_skill_files`, `tool_catalog_entries` |
Skills and tool descriptions the provider publishes | No. These are
provider definitions, not data read through tool calls | — |
| Transcripts of later runs that read an app view's `model_context` |
Shared app context | No. Nothing records which runs read it. The help
page says so. Follow-up | — |
| `issue_comments.body`, issue documents (including continuation
summaries), files, chat messages | Task content written by agents or
Paperclip | No, by design. The user removes it by editing or deleting it
or the task. The confirmation says in bold that anything agents wrote
into tasks is not removed, and so does the help page. The canary
excludes `issue_comments.body` by name for this reason | canary asserts
it is kept |
| Raw provider traces | Full provider traffic | No. Admin opt-in,
encrypted, expire within 24 hours | — |
| Feedback exports already sent | Run logs the user chose to share | No.
Already outside Paperclip | — |
| Content of runs of a deleted task or company | Same stores | No.
Follow-up | — |

**Durable request (schema change).** I added one table,
`tool_connection_content_deletions`, and migration `0490`. I followed
`doc/DATABASE.md`: edited the schema, exported it from
`packages/db/src/schema/index.ts`, ran `pnpm db:generate`, and ran the
migration drift and compatibility tests. The classifier marks it
`online_compatible` (a new table and its indexes only). No existing
table can hold a durable per-connection request with a cutoff time,
cursor, counts and retry state. Without it, an interrupted deletion
could resume only when a person pressed the button again.

**Resume after restart.** The worker uses the shared
`registerDeadlineWork` lifecycle on a new
`tool-connection-content-deletion` queue. The request insert signals the
queue in the same transaction. At server start the worker runs once,
`next()` finds any pending row, and processing continues from the saved
phase and cursor. Nobody presses anything. `next()` returns a deadline
only while a pending row exists, so an idle instance sends no recurring
query. A failure records `attempts` and `last_error` and retries after
30 seconds.

**One run selection.** `runsWithStoredContentSql` selects the runs whose
transcripts are deleted: runs that made a tool call or tool-call event
through the connection, and runs whose context lists one of the
connection's approved action requests, limited to runs that still hold a
log, an excerpt, result text or a content event. The dialog count, its
in-progress warning and the job's `runs` phase all use this one
function.

**Phases.** `invocations`, `events`, `auditEvents`, `appViews`,
`deliveredResults` (50 action requests per page; matching runs and wakes
are locked with `FOR UPDATE` before their documents are rewritten),
`runs` (the shared selection, 50 run ids per page, then one run at a
time), `done`.

**Canary test.** `tool-connection-content-deletion-canary.test.ts` puts
a unique marker in a tool call's arguments and result, an approved
action with a preview, a result delivered to a later run and its wake, a
native run that quotes the marker in its answer and events, a run log
migrated from a local file with an S3 mirror to an object journal, an
MCP App view and the audit tables. It checks that the marker is first
present in the database, the object store, the legacy mirror and the log
files. After the deletion it scans every `text`, `varchar`, `json` and
`jsonb` column of every table found through `information_schema` (no
hand-written table list), every object in both stores and every file
under the log directory. Any hit fails with the table and column or the
key. It runs three times: a complete deletion, a deletion interrupted
twice by storage failures and resumed, and two deletions for different
connections on one task at the same time. The only exclusion is
`issue_comments.body`, named with its reason.

**Batching and time budget.** Tool rows are read in pages of 200 with
keyset pagination on the existing `(company_id, created_at)` indexes,
bounded by the request time. Each page's writes and the saved progress
commit in one transaction. Counts are therefore exact, and a crash
repeats at most one page, idempotently. Runs advance one at a time from
the shared selection. Each 1,000-event page is its own step. Log removal
is its own step and runs before its progress transaction, so object
storage calls never run inside a database transaction. The 10-second
budget is checked after every step. Nothing changes on the agent boot or
tool-call path.

**In-flight work.** A run counts as settled only when its status is
terminal, `finished_at` is set, cost accounting is done, and no native
finalization is open. Calls, approvals, events, app views, delivered
results and transcripts tied to other runs or wakes are skipped and
counted. The confirmation says "N runs are still working and will be
skipped. Delete again after they finish."

**Permissions.** Board users with configure access (connection manager
or creator) can delete, like `DELETE /tool-connections/:id`. Agents,
viewers and other companies are refused. The activity entries
`tool_connection.content_deletion_requested` and
`tool_connection.content_deleted` hold counts only.

**UI reads.** The dialog loads counts once when it opens. While a
deletion runs, it polls `GET
/tool-connections/:id/stored-content/deletion`, a single indexed row
read, every 2 seconds. Polling stops when the job completes or the
dialog closes. The Delete button is not gated on counts.

## What Changed

- `packages/db`: new `tool_connection_content_deletions` table and
migration `0490` (online compatible: new table and its indexes only).
- `packages/shared`: `ToolConnectionStoredContent`,
`ToolConnectionContentDeletion` and counts types; optional
`contentDeletedAt` on `ToolRedactedValueSummary`.
- `server`: `RunLogStore.remove`, which deletes every copy of a run's
log (`composeRunLogStore`); `content-deletion.ts` service; deadline
worker wired into startup and pre-drain; `GET
/tool-connections/:id/stored-content`, `POST` and `GET
/tool-connections/:id/stored-content/deletion`, and `DELETE
/tool-connections/:id?deleteStoredContent=true`; OpenAPI entries.
- `ui`: "Delete data" row on the connection's Permissions tab with a
confirmation that shows counts, skipped runs, and "This cannot be
undone"; "Also delete data read through this connection" checkbox in the
Connectors removal dialog.
- Docs: user help
`docs/guides/board-operator/deleting-connection-data.md` (for the
privacy policy link) and an API note in
`doc/connections/CONNECTOR-PLAYBOOK.md`.
- `tests/runner-e2e/source-sdk-public-inputs.json` refreshed for the
shared type change.

`server/src/routes/tool-access.ts` (3,010 lines) and
`ui/src/pages/apps/Browse.tsx` (1,722 lines) grow by about 40 and 20
lines. The routes reuse that file's connection access checks; the dialog
reuses the existing removal confirmation. The new logic lives in its own
files.

## Verification

- `server/src/__tests__/tool-connection-content-deletion.test.ts`
(embedded PostgreSQL): content is gone from every "Yes" store in the
inventory; rows, hashes and sizes stay; another connection and another
company are untouched; a running run is skipped and is deleted after it
settles; a storage failure leaves the request pending with `retrying`,
and the next pass completes it; an approval-card update that fails once
is cleared on the retry; an unrecognised log backend keeps the pointer,
keeps the job pending and writes no completion entry; a zero time budget
yields after every step and still finishes; a repeat request returns the
pending one; agents and outsiders are refused; removal with
`deleteStoredContent=true` queues the deletion.
- `run-log-store.test.ts` and `object-run-log-store.test.ts`: `remove`
deletes local and mirrored logs and object journals, removes a migrated
run's original local file and mirror with its journal, refuses an object
log when object storage is not configured, and is idempotent.
- `ui/.../StoredContentDeletion.test.tsx`: confirmation text, counts,
skipped-run warning, and the request.
- Red: these tests fail on `master` because `remove`, the service and
the routes do not exist. Green on this branch.
- Ran: the tests above, `openapi-routes`,
`server-startup-feedback-export`, `tool-connection-removal`,
`packages/db` migration drift and compatibility tests, all
`ui/src/pages/apps` tests (950), `pnpm check:token-gates`, server and UI
`tsc --noEmit`. `pnpm verify` exits 2 (global CI input changed), so
required CI supplies broad coverage.

## Risks

- Deleting is permanent by design. The confirmation says so.
- A `local_file` log written on another replica's local disk stays on
that pod's disk until the pod is replaced; its S3 mirror is deleted. New
logs on S3-configured instances use object journals, which are deleted.
- On versioned buckets, earlier object versions remain until the
bucket's lifecycle policy removes them (the existing journal delete path
does not remove versions).
- Not in this PR (follow-ups): the "No" rows of the inventory (except
agent-written task content, which is out of scope by design), mainly the
same deletion when a task or a company is deleted, and later runs that
read an app view's shared context.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
canary/v2026.1117.0-canary.819200
2026-11-17 08:55:35 -06:00