mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-16 11:59:26 +01:00
21bf7ebcee4f9bd898cfafee3baabfff93aefa17
834
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
21bf7ebcee |
feat(connectors): connect Tailscale networks to SSH environments (#20617)
## Thinking Path > - Paperclip manages work for AI agents. > - SSH environments run tasks on computers that users own. > - A Paperclip server needs network access before it can reach a private computer. > - A reusable Tailscale connection belongs in Connectors. > - Connected machines should become choices in the existing SSH environment form. > - This change connects a tailnet and resolves selected machines through a private network client. > - Agents retain their choice of harness and AI account. ## Linked Issues or Issue Description **Subsystem affected** Connectors, environment settings, SSH execution, company secret storage, and runtime packaging. **Problem or motivation** Entering a Tailscale address in the SSH form does not connect the Paperclip server to that tailnet. Requiring a separate device key for every environment also duplicates network setup. **Proposed solution** Add a Tailscale Connector with **Connect Tailscale Network**. Use brokered OAuth consent to enroll one private network client per connection. Let SSH environments select a connected network and stable machine identity. Preserve the enrolled node state through the company vault so a restart does not reuse the one-time token. **Alternatives considered** A separate Tailscale environment type would duplicate SSH behavior and mix network access with harness selection. This change uses the existing `ssh` driver. Related #15648 adds OAuth client credentials and a health check; this change uses user consent and supplies outbound task execution. Related #11743 concerns HTTPS brokering. **Roadmap alignment** This extends the existing Connected Apps, environment, and Personal & Shared AI Accounts capabilities. This PR is stacked on #21248, which provides SSH account selection independently of Tailscale. The hosted OAuth broker is a companion control-plane change. ## What Changed - Preserve Cloud reauthentication in the current browser tab. Explain canceled or failed consent in the connector and offer a retry of the same authorized saved connection. - Record Cloud enrollment starts initiated through the Tailscale connector in the company activity log. - Check install-preparation descendant liveness through the kernel process table in the Runner support test. Keep the direct child’s strict reaping assertion and reject any non-zombie descendant. - Align the application image stage contract with the Tailscale client stage after the upstream Runner-image refactor. Keep the exact stage list and check every application stage, including `tailscale-client`, for prohibited Runner compiler commands. - Group Tailscale consent, discovery, daemon lifecycle, pairing, and deletion under `server/src/services/connectors/tailscale/`, with their service tests. Update existing route, runtime, environment, vault, and company-deletion imports. Keep the existing contracts, authorization, audit, and transaction owners. - Handle early stdin closure in the Kubernetes real-shell test helper. Stop the input stream on `EPIPE`, preserve nonzero child exits, and reject a zero exit when input was lost. Add two real-process regression cases. Production transfer code and confinement checks are unchanged. - Read configured Tailscale executable paths when a client starts, after supported `.env` loading. Keep explicit command overrides available. - Discard pending pairing results after switching to Direct SSH or closing the form. Retain the existing company, connection, machine, and port checks. - Store network credentials with an immutable server-owned purpose from their first vault row. Exclude them from generic catalogs, bindings, environment projection, and value reads. Use a dedicated audited server consumer for pinned connector reads. Block unclassified legacy serialized node state at the same value boundary. Keep SSH private-key binding separate from connector credentials. - Gate network queries and setup on verified local-implicit or instance-admin access. Hide Tailscale catalog rows for other members, show an admin requirement on direct setup navigation, and avoid device discovery before management access is known. - Keep failed-client cleanup in the network manager. A failed request cannot disconnect a newer retry for the same identity; explicit connector disconnect and enrollment retirement still stop their networks. - Check for readable Ed25519, ECDSA, or RSA host public keys before pairing edits `authorized_keys`. Print a known-hosts entry for every available key, or fail before changing authorized keys if none are available. - Return self-hosted Cloud enrollment to the Tailscale connector. Save the unprefixed setup path and let the shared callback add the verified company prefix. Keep external and unrelated destinations rejected. - Hide pairing instructions after an SSH port change. Ignore late key-selection results for the old port. Operators prepare again for matching known-hosts output. - Retire connector references before deleting a company or its vault credential. Stop company daemons after the database commit. Keep failed secret retirement retryable and prevent late node-state writes from restoring deleted connections. Discard retired state without blocking behind the vault deletion lock. - Reload a mismatched credential before replacing a ready daemon. Recheck the active entry and manager generation after the read. An older request cannot stop a newer healthy socket or cross shutdown. - Pin connector credentials to the exact vault version and SHA-256 digest recorded by trusted enrollment and node-state writes. Ignore generic secret rotations. Advance both fields with a compare-and-set guard. Reject unpinned or tampered credentials before network startup. - Save an explicit Direct SSH selection and clear old Tailscale fields after config merging. Preserve the operator's entered direct host. - Route all Tailscale SSH runtime access through the connection service after SSH-key loading. Pin access to the environment's saved connector identity and recheck after device discovery. Remove standalone key-based daemon startup and reject raw enrollment keys or credential-only network configurations. Align the shared config type and SDK pins. - Correct two E2E assumptions exposed during connector qualification. Keep the two-window chat fixture stale until its own mutation. Validate the original failed run and its permitted write-revocation timestamp in the legacy recovery browser test. - Add the Tailscale Connector page, catalog entry, OAuth return flow, connected-network management, device discovery, and disconnect. Document all six routes in OpenAPI and the API reference, including their instance-admin requirement. - Add company-scoped connection storage in `0480_lazy_lethal_legion`, trusted credential version/digest fields in `0481_round_pet_avengers`, and server-owned secret purpose/backfill in `0482_aberrant_rumiko_fujikawa`. Generate all three migrations through the maintained command and retain the five newest snapshots. - Add machine pairing that generates a fresh SSH key per request, stores its private half in the company vault, and uses the generated public half in the install command. Do not trust editable saved metadata. Preserve existing authorized keys, including a file without a final newline. Keep SSH username, workspace, and trusted host keys explicit. - Use the existing SSH driver. Resolve the machine's current address from its stable device ID. Reject disconnected, removed, offline, and wrong-company selections. - Hide Network when Direct is the only option. Select a sole connected tailnet automatically and hide its selector. Remove the requested setup helper copy. Do not restrict the harness to Claude. - Run a private userspace Tailscale daemon with a separate socket. Persist its node state before execution. Restore it after restart, bound startup, serialize reconnect cleanup, and close it during server shutdown. - Guard OAuth completion with a durable attempt marker. Disconnect and newer authorization invalidate older callbacks. Close replaced daemons and remove cancelled enrollment secrets without changing newer connection state. - Preserve SSH host verification and tailnet-specific session identity. Honor the configured Tailscale executable path, including spaces and OpenSSH percent escaping. - Package checksum-pinned Tailscale clients and their license in the standard runtime image. Exercise the client build stage in PR CI and include it in image smoke checks. ## Verification - Merge status: all CI and image checks pass, and current-head Apex is 5/5 with no unresolved comments. GitHub rejected a direct merge because the active master ruleset requires code-owner approval for `.github/workflows/pr-trusted.yml`. The [CODEOWNERS workflow pattern](https://github.com/paperclipai/paperclip/blob/master/.github/CODEOWNERS#L3) applies; eligible maintainers already have review requests. Auto-merge is enabled with the squash method for the verified head. The PR remains open pending that required approval; no branch policy was bypassed. - Final revision: `fdb29b2cc366f4945b7f4c3e2b2d25070329a21b`, including master `999709b0a6dd416af335b5debded13e1de7fcb7a`. [Final CI verification passed](https://github.com/paperclipai/paperclip/actions/runs/43518584814/job/132213508320). All required checks pass, all applicable test/build gates are green, and the branch is mergeable against master. SSH prerequisite #21248 and Cloud broker paperclipai/paperclip-cloud#1314 are already merged. - [Apex reviewed this exact revision at 5/5](https://github.com/paperclipai/paperclip/pull/20617#issuecomment-6776629437). There are no unresolved review threads. The PR is classified as Critical impact. - Final image qualification passes for the [application Runner path](https://github.com/paperclipai/paperclip/actions/runs/43518584522/job/132206784267), [amd64 Runner image](https://github.com/paperclipai/paperclip/actions/runs/43518584544/job/132207296198), and [arm64 Runner image](https://github.com/paperclipai/paperclip/actions/runs/43518584544/job/132207296156). Typechecks, Runner support, native/Rust tests, provider tests, serialized-server suites, browser shards, and the remaining applicable CI checks pass. This is source/image CI qualification, not a production rollout. - The last master conflict affected only Docker stage declarations and their boundary test. Both the checksum-pinned Tailscale stage and upstream `runner-lockfile` guard are retained, including the upstream `deps` inheritance. The Tailscale client stage is byte-identical. All 180 cases across five Docker/image contract suites pass locally, along with server and UI typechecks. The maintained 543-input SDK check has no drift. Runtime and lock pins are unchanged. Secret/PII and whitespace scans pass. - Seven OAuth regression cases failed before the fixes. The current-tab test uses the real handoff helper, checks that the session is stored before reauthentication navigation, mounts the shared return page, resumes consent, and verifies handoff cleanup. Callback tests check trusted saved-connection context, fixed safe outcomes, failure audit metadata, denied membership and expired-state rejection, JSON API errors, and no provider text in browser returns. UI retries require a connection in the current company's catalog. All 80 OAuth route/service/UI/shared-handoff cases pass; server and UI source/test typechecks pass. - [Server shard 4/12 on the prior refresh](https://github.com/paperclipai/paperclip/actions/runs/43510727031/job/132180301436) passed 2,784 cases and skipped 21, then failed a reassignment fixture's `TRUNCATE companies CASCADE` cleanup with PostgreSQL `40P01`. Its automatic retry found the surviving `REA` prefix. The unchanged test passes all three cases locally. A controlled reader/cleanup overlap reproduces the same deadlock, but the exact competing CI query was not logged. Each case now has a distinct company prefix and identifier; the owned database is removed at suite end. The three test bodies and assertions are byte-identical. Eight local reassignment/lock-release cases pass, and the [final server shard 4/12 passes](https://github.com/paperclipai/paperclip/actions/runs/43518584814/job/132207159392). Production reassignment logic, deadlines, and retries are unchanged. - The initial master integration passed 330 cases across 17 suites. Coverage includes connector state, device discovery, network lifecycle, pairing, routes, SSH binding and exports, environment persistence, and connector UI. All three unpublished migrations were regenerated after master's `0479_short_darkstar`: `0480_lazy_lethal_legion`, `0481_round_pet_avengers`, and `0482_aberrant_rumiko_fujikawa`. Each SQL body is byte-identical to its earlier unpublished migration. Generated snapshots preserve existing master tables and add only the intended Tailscale table and secret-purpose column. Exactly five snapshots remain, 0478–0482. Numbering, safety, compatibility, and snapshot-drift checks pass. The later Docker refresh did not alter this migration sequence. - Earlier attempts remain part of the verification record. On `1c020a0`, Rust cache restoration and server shard 5/12 were interrupted by shutdown signals, Runner shard 2/3 lost communication with no retrievable log, and an amd64 provider download failed with npm `ECONNRESET`. Same-head retries recovered, without establishing the cause of the communication loss. Later superseded runs also had cancellations with incomplete logs. These interrupted attempts are not counted as passes. The final revision has fresh successful CI. - Local platform limits remain recorded. The Kubernetes suite on macOS passed 191 cases, failed 12 Linux `/proc/self/fd` transfer cases, and skipped four Linux-only cases. Unchanged source reproduced the same 12 failures. An initial run also lacked provider dependencies until the maintained preparation command installed them. Those local attempts are not green qualification. Final Linux provider CI passes. Local Docker qualification was unavailable; final image checks passed in CI. - Before PR separation, a live tailnet/SSH Claude Code task completed on Ubuntu ARM64 without provider login for the remote user. The selected Paperclip subscription ran it, wrote the expected file, posted completion, and marked the task done. File contents and private runtime cleanup were checked. This is prior live evidence; the final revision was not live-retested. Other providers have automated coverage only. Hosted registration for unrelated tailnets and production concurrency remain qualification limits below. - The Superagent security and supply-chain reports remain neutral and inconclusive. Earlier attempts to restart the failed security scan returned HTTP 404. Supply-chain intelligence lacked coverage for extracted pinned dependencies and the shared workflow reference. These reports do not establish successful security qualification. The required CI and review gates pass; this external coverage gap remains disclosed. ## Risks - The Superagent security and supply-chain reports are inconclusive. The security scan re-run request returned HTTP 404, and the supply-chain scanner lacks coverage for the extracted dependencies and shared workflow reference. These neutral results are not successful security qualification. - The server-owned secret purpose is internal database metadata, not an HTTP-writable field. The generic secret service owns enforcement so direct API reads and runtime projection share the same restriction. Previously copied node state remains valid outside Paperclip until the enrolled device is revoked in Tailscale; a local disconnect cannot stop an external copy. - Tailscale network configurations now require a connected Connector and selected machine. The unreleased key-only path is removed; any saved configuration from that path needs Connector setup. - The connector remains a pilot. The local test used an OAuth app registered in the consenting user's tailnet. Qualify hosted app registration for unrelated customer tailnets before general availability. - This implementation supports one serving process per instance. Concurrent replicas need exclusive ownership of the persisted network identity before production rollout. Preserve the database and company vault master key across restarts. - Each machine-pairing preparation creates a new vault secret. Unused generated keys follow the existing secret deletion lifecycle. A saved SSH key remains available as an explicit choice. - Apply additive migrations `0480_lazy_lethal_legion`, `0481_round_pet_avengers`, and `0482_aberrant_rumiko_fujikawa` before application rollout. The new credential version and digest columns are nullable. Existing unpinned connections require fresh OAuth consent; the migration deliberately does not trust or backfill a generic secret value. Regenerate or renumber migrations if master advances the sequence before merge. - The target needs an SSH server, its chosen harness, and allowed tailnet access. A reachable network does not grant SSH or provider authorization. Pairing reads standard `/etc/ssh` host-public-key files; custom host-key locations require manual known-hosts configuration. - Roll back the application revision to remove new entry points. Existing connection secrets and registered Tailscale nodes retain their separate lifecycle. Do not drop the table or delete node state as part of an application rollback. - Production qualification still needs denied-policy, revocation, cancellation, and deployment-overlap checks. Local Docker and a production deployment were not performed. ## Model Used OpenAI Codex, gpt-6.1-sol, xhigh reasoning, with local code execution and tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used - [x] I have checked ROADMAP.md and this extends existing connector and environment features - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR using the feature request fields - [x] I have not referenced internal issues, local URLs, or private machine identifiers - [x] My branch name describes the change - [x] I have run focused tests locally and they pass - [x] I have added tests for new behavior and failure paths - [x] I have updated documentation - [x] I have documented migration and runtime risks - [x] All Paperclip CI gates are green - [ ] Required code-owner approval received for the workflow change - [x] Greptile Apex is 5/5 on the final head with no unresolved comments - [x] I will address CI and review findings before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
5df1ada9c1 |
feat(plugins): send configured environment on plugin task resume (#21352)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - #21335 lets a plugin task provider that restarts the Runner on `resume` opt in (`resumeCredentialVersion: 1`) to receive the claimant run's harness credential > - A provider restarting the Runner also needs the agent's configured environment, which Core sends on `resume` only during a session handoff > - Without it, a Runner a provider moves to a new computer starts without the agent's configured environment variables > - This pull request widens that opt-in to both launch inputs, under a name that says so, before any provider adopts the old name > - The benefit is that a provider can restart a parked Runner on resume with the same credential and environment a `submit` would carry ## Linked Issues or Issue Description Refs #21289 Follows: #21335 **What existing behavior does this improve?** The opt-in resume launch inputs added in #21335. **Subsystem affected** Plugin SDK environment task schema (`describe` capability), the plugin Runner task owner's `resume`, and the plugin environment task docs. **Current behavior** With `resumeCredentialVersion: 1`, `resume` carries `credential`. `configuredEnvironment` is sent on `resume` only for a session handoff. **Proposed behavior** The capability is renamed `resumeLaunchInputsVersion: 1`. With it, `resume` carries `credential` as before and also `configuredEnvironment` (same builder as `submit`) when the provider advertises `configuredEnvironmentVersion: 1`. Session handoff is unchanged. **Reason and benefit** A provider that restarts the Runner on resume gets both launch inputs it needs. **Breaking changes** The capability name changes from `resumeCredentialVersion` to `resumeLaunchInputsVersion`. No provider advertises the old name yet (it merged in #21335 and has no adopter), so nothing changes behavior. ## What Changed - Rename `resumeCredentialVersion` → `resumeLaunchInputsVersion` in the SDK `describe` result schema, executor plumbing, owner input, docs and tests. - With the capability, `resume` also adds `configuredEnvironment` when the provider advertises `configuredEnvironmentVersion: 1`. A normal resume does not mark the variables as delivered, since a live Runner keeps the environment it started with. - Docs: the "Release and resume" section describes both fields; the "Configured environment" section points to it. - The runner-e2e SDK public-input pin is updated. ## Verification - SDK vitest `tests/environment-task-release-resume.test.ts` and `tests/environment-tasks.test.ts`: 68 passed. - Server vitest `plugin-runner-task.test.ts` and `environment-task-runtime.test.ts`: 153 passed, including a normal resume with the capability (credential sent, variables added only with `configuredEnvironmentVersion: 1`, delivered flag unset) and redaction of both fields from metadata writes and sanitized errors. - `native-session-executor.test.ts -t "uses a task-only"`: 5 passed. - runner-e2e `source-sdk-evidence.test.ts` and `source-sdk-public-type.test.ts`: 46 passed. - `pnpm typecheck` in `server/` (builds shared and plugin-sdk) passed. - Not run locally: the full suite and `pnpm verify`; required CI covers them. ## Risks - A provider restarting the Runner gets the claimant run's current variables, which can differ from the ones the parked Runner started with. That is intended. - Rollback: revert this pull request. No migration. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, test runs), implemented by a sub-agent and reviewed before opening. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
12aa08721c |
feat(ssh): use the agent AI account in new environments (#21248)
Use the agent-selected AI account for new SSH environments. Preserve
remote-machine authentication for existing environments and expose it as
an advanced option. Keep managed credentials private while preserving
supported conversation state and credential refreshes across turns.
Validated with focused route, adapter, lifecycle, and real-process tests;
final CI passed on
|
||
|
|
0199c963e2 |
feat(apps): ship the redesigned sign-in step for a pasted MCP server (#21469)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Apps lets an operator connect a remote MCP server by pasting its
address ("Connect your own MCP server")
> - Step 2 of that flow put every sign-in choice in an "Advanced
authentication" box with wrapped pill buttons and a hint paragraph
> - The box hid the choice behind a vague label, and its shape did not
match the access line ("Connects for everyone… Change") right below it
> - The owner approved a redesign: a stated sign-in default with a
Change link, and five options with their fields directly below each
option
> - This pull request wires that redesign into the real screen, removes
the old step, and updates copy that named the removed box
> - The benefit is a screen that reads as two stated answers and one
action, with the same default path (paste the address, press Check link)
## Linked Issues or Issue Description
No public issue exists. Enhancement:
**What existing behavior does this improve?**
The sign-in section of the "Connect your own MCP server" step in Apps.
**Subsystem affected**
UI: Apps connection setup (`ui/src/features/connections`). Small copy
changes in the server OAuth client registration message and docs.
**Current behavior**
Step 2 shows an "Advanced authentication" collapsible with five wrapped
pill buttons and a hint paragraph below them. The fields for the chosen
mode appear further down. Guidance banners tell the operator to look
"under Advanced authentication". The page also shows the "Unverified
server" badge twice (header and body). The stepper reads "Pick app · Add
your key", even when the server needs no key.
**Proposed behavior**
Step 2 states the sign-in choice in one line ("Paperclip asks the server
how to sign in.") with a Change link. Change shows five radio cards: Let
Paperclip check, Key or token, Browser sign-in, Custom headers, No
sign-in needed. The fields for the selected option are directly below
it. Done collapses the list to a one-line summary (for example "Signs in
with 1 custom header."). The list stays open while the chosen option
still needs a value. Client ID and secret are behind "Use your own
client ID" unless they have values or the server asked for them. The
stepper reads "Pick app · Connect".
**Reason and benefit**
The choice is visible as a stated default in the same shape as the
access line. Each option keeps its fields next to it. No copy points at
a box that does not exist.
**Breaking changes**
None. The five auth modes and their request payloads do not change.
**Additional context**
Related open PRs change server behavior for the same flow. They are not
duplicates. Refs #15192, Refs #15450.
## What Changed
- `ui/src/features/connections/GenericMcpConnectStep.tsx`: new step
component (same props as the old `LinkConnectStep`). It uses stable
field ids `generic-mcp-key`, `generic-mcp-client-id`,
`generic-mcp-client-secret` and a `data-testid` on the sign-in
Change/Done link.
- `ui/src/features/connections/ConnectionSetupFlow.tsx`: render
`GenericMcpConnectStep`; delete `LinkConnectStep`,
`GENERIC_AUTH_MODE_OPTIONS`, `SegmentedOption` and unused imports;
change the pasted-link stepper label from "Add your key" to "Connect"
(`STEP_LABELS` is used only by the pasted-link flow; curated apps keep
their own `["Add your key"]`).
- `ui/src/pages/apps/generic-mcp-connect.ts`: `oauth_challenge` and
`oauth_manual_client_required` guidance now say "below" instead of
"under Advanced authentication".
- `server/src/services/tool-access/oauth/client-registration.ts`: the
manual-client message no longer names the removed box.
- `doc/connections/GENERIC-REMOTE-MCP.md`: describe the sign-in line,
Change, and the five options.
- Comments in `packages/shared` and `server/src/services/tool-access.ts`
reworded. `tests/runner-e2e/source-sdk-public-inputs.json` regenerated
for the two shared files (blob hashes only).
- `ui/storybook/stories/connection-setup/byo-mcp-sign-in.stories.tsx`:
14 stories for the shipped component; annotation is now `current` /
`production`.
- Tests: `AppsConnect.test.tsx` generic MCP block uses the sign-in
toggle and radio options; new tests for the collapsed summary, the
open-while-empty-key rule, and a single "Unverified server" label. New
`GenericMcpConnectStep.test.ts` unit-tests `genericMcpAuthSummary`.
Before: "Advanced authentication" box with pill buttons and a hint;
fields below the pills; two "Unverified server" badges; stepper "Pick
app · Add your key".
After: one sign-in line with Change; five radio cards with fields under
the selected card; one "Unverified server" badge in the header; stepper
"Pick app · Connect". Storybook: `byo-mcp-sign-in--choosing`,
`byo-mcp-sign-in--server-asked-for-key`.
## Verification
- `cd ui && pnpm exec vitest run src/pages/apps/AppsConnect.test.tsx
src/pages/apps/generic-mcp-connect.test.ts
src/features/connections/GenericMcpConnectStep.test.ts
src/features/connections/ConnectionSetupFlow.architecture.test.ts
src/features/connections/ConnectionSetupFlow.oauth.test.ts`: 5 files,
300 tests pass.
- `cd ui && node ../scripts/run-tsc.mjs -b`: pass.
- `pnpm -s typecheck` in `packages/shared` and `server`: pass.
- `pnpm check:token-gates`: all gates clean.
- `node scripts/storybook-catalog.mjs`: pass (generated indexes not
committed).
- `pnpm verify`: exit 2 (broader verification required, because
`doc/connections/GENERIC-REMOTE-MCP.md` is a global CI input). Required
CI supplies that coverage. The focused files above were rerun on the
final revision after the rebase.
- Headless Playwright against the Storybook dev server: the Default,
Choosing and Server-asked-for-key stories render the new step, the new
guidance copy, and the "Pick app · Connect" stepper.
## Risks
- Low risk. UI change on one screen; request payloads
(`genericConnectPayload`) do not change.
- Tests that click the first collapsed "Change" button now skip the
sign-in toggle, because the access line also has a "Change" button.
- The server message for `oauth_manual_client_required` changes text. No
code matches on the text; the client uses the error code.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, headless Playwright).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
|
||
|
|
9d9ee92d15 |
refactor(evals): remove the orphaned chat TestOps dispatch guard (#21241)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The chat catalog eval harness sends selected cases to TestOps plans through GitHub workflows. > - PR #18322 added a dispatch guard script for the `chat-feature-testops` workflow. The guard checked each dispatch event before the workflow could use credentials. > - PR #18625 retired the duplicate chat TestOps plans. That PR deleted the `chat-feature-testops` workflow, the workflow step that ran the guard, and the only import of the guard function. > - The guard file stayed in the repository after that removal, but nothing calls it now. > - This pull request deletes the unused guard file. > - The benefit is less test support code to maintain. No protection is lost, because the workflow it guarded no longer exists. ## Linked Issues or Issue Description No public issue exists for this cleanup. The problem is described here, following the enhancement template. **What existing behavior does this improve?** The chat TestOps eval support under `tests/runner-e2e`. The guard script `testops-chat-dispatch.ts` verified workflow dispatch events for a retired workflow. **Current behavior** `tests/runner-e2e/testops-chat-dispatch.ts` holds a guard for the `chat-feature-testops` workflow. That workflow was deleted in #18625. A search of all tracked file content finds no import, no run command, and no test that uses the guard file today. **Proposed behavior** The repository does not keep the unused guard file. The runner-e2e typecheck program and the test tree load one support file less. **Reason and benefit** The guard protected a workflow dispatch path that no longer exists. Keeping the file suggests a live guard where none runs. Deleting it removes 31 lines of test support code. No regression coverage is lost: no test executes the file, and the contract it guarded is retired. **Breaking changes** None. The deleted file has no callers. No workflow, script, test, or documentation references it. ## What Changed - Deleted `tests/runner-e2e/testops-chat-dispatch.ts` (31 lines of test support code). No production code changed. ## Verification - Searched all tracked file content for `testops-chat-dispatch`, `validateChatTestOpsDispatch`, and `ChatTestOpsDispatch`. Only the deleted file matched. - Checked every selection path that could run the file: it is not a `*.test.ts` file; the runner-e2e harness selects test files through the catalog manifest, which does not name it; the root vitest projects do not include `tests/`; the `node --test` script lists do not name it; no workflow runs it. - Confirmed the history: #18322 added the file; #18625 deleted the workflow step `node tests/runner-e2e/testops-chat-dispatch.ts` and removed the only import and the assertions that referenced it; no commit touched the file after that. - Local limitation: this workspace has no installed dependencies, so the layered typecheck and the vitest suites could not run locally. The CI lanes on this pull request must confirm them. ## Risks - Low risk. The file had no callers. The remaining risk is a hidden reference that a text search cannot find. The typecheck lane and the CI test lanes on this pull request would find such a reference. ## Model Used - GLM by Z.ai, model id `z-ai/glm-5.3`, run through opencode by the Paperclip Runner. Extended reasoning enabled, with shell and file tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [ ] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge |
||
|
|
e5158293c5 |
perf(ui): keep the live socket when a cached session is verified (#20591)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The board UI keeps one live-updates WebSocket per company and saves
a copy of its query cache in `localStorage` so a returning visit can
paint from it
> - On a returning hard load of the Mine inbox, about eight shell and
inbox reads ran a second time 1.5–2.5 s after load (session, health,
company list, agents, inbox dismissals, user directory, approvals,
execution workspaces)
> - The cause is the socket's sign-in key. A session restored from the
saved cache has the user but no session id. The first session read adds
the id, the key changes, the socket closes and opens again, and that
second open refreshes every active query, session and health included
> - This pull request keeps the socket when the verified session belongs
to the same user as the restored one
> - The benefit is one read of each shell request per returning load and
one socket instead of two, with no change to the catch-up read that
covers the gap before the first subscription
## Linked Issues or Issue Description
**What happened?**
A returning hard load of `/:companyPrefix/issues?view=mine` sent
`auth/get-session`, `health`, `companies?scope=accessible`, agents,
inbox-dismissals, user-directory, approvals and execution-workspaces
twice. The second copy started about 1.5–2.5 s after load on the hosted
staging instance.
**Steps to reproduce**
1. Open Mine once so the app saves its query cache.
2. Reload the page.
3. Watch the network panel. The live-updates socket opens, closes, opens
again, and the reads above repeat.
**Expected behavior**
Each read runs once. The socket opens once.
Related: #20398 (socket reconcile skips settings and preferences),
#20407 and #20419 (Mine reveal and its perf guard).
## What Changed
- `ui/src/context/LiveUpdatesProvider.tsx`: the socket auth key comes
from `nextSocketAuthScope`. When the previous scope has the same user
and no session id, the first known session id is adopted without a new
key. A different user, or a different session id once one is known,
still produces a new key and a new connection.
- `ui/src/context/LiveUpdatesProvider.recovery.test.tsx`: two tests. A
restored session (no id) verified for the same user keeps the socket and
invalidates nothing; a later replacement session still reconnects. A
verified session for another user reconnects.
- `tests/perf/collections/shell-read-waves.spec.ts` and
`returning-visit-fixture.ts`: browser checks on a production build with
every API response delayed 300 ms. With a saved cache, each of the eight
reads runs at most once and the socket opens once. Without a saved
cache, session and health run once, no read runs more than twice, and
the socket opens once.
- `tests/perf/collections/README.md`: documents the new spec and
fixture.
## Verification
- `cd ui && pnpm exec vitest run src/context/`: 16 files, 286 tests
pass. The new same-user test fails on `master` (socket count 2) and
passes with this change.
- Browser check on a disposable local instance (production UI build,
`local_trusted`, embedded PostgreSQL, Chromium headless, 1440×900, API
+300 ms):
`PAPERCLIP_COLLECTION_PERF_BASE_URL=http://127.0.0.1:<port> pnpm exec
playwright test --config tests/perf/collections/playwright.config.ts
shell-read-waves.spec.ts`
- `master` build (`99edcd28a7`): returning visit fails with
`{"auth/get-session":2,"health":1,"companies?scope=accessible":2,"agents":2,"inbox-dismissals":1,"user-directory":1,"approvals":1,"execution-workspaces":0}`.
- This branch: both tests pass; every count is 1 and one socket opens.
- Timing evidence (same instance, 5 fresh contexts per arm, API +300 ms,
host load average about 20; `mine-returning-visit.spec.ts` timing case
from the companion Mine PR). Duplicate shell reads are summed over the 5
loads:
| Build | Saved cache: time to rows, median (range) | Saved cache:
duplicate shell reads | Cleared cache: median (range) | Cleared cache:
duplicate shell reads |
| --- | --- | --- | --- | --- |
| `master` | 1028 ms (906–3011) | 5 (session ×2 each load) | 1502 ms
(1423–1592) | 5 (company list ×2, the catch-up) |
| This PR | 895 ms (880–930) | 0 | 1364 ms (1328–1408) | 5 (company list
×2, the catch-up) |
The time difference is small and inside host noise; the claim here is
the removed reads and the single socket. On the busier hosted instance
the second wave was larger (about eight reads), because more reads have
answered before the session read returns.
- Review follow-up (
|
||
|
|
64ec273845 |
perf(ci): run the typecheck lane's checks in parallel and drop its duplicate server type check (#21404)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Every pull request runs the trusted PR workflow, and the `Typecheck + Release Registry` job is one of its required checks > - That job takes about 9.2 minutes on the 4 vCPU / 16 GB fleet runner (run 43344818387), against a target of 7 minutes or less for every job > - The job runs six steps one after another, and the first step type-checks the whole server program a second time, at about 13.5 GiB, which makes the runner swap > - Typecheck results are the check that authors most want back quickly, so this job is on the critical path of most pull requests > - This pull request removes the duplicate server type check and runs the independent checks in parallel through one small script, inside the same memory budget > - The benefit is about 39% less wall time for the job's checks on the same machine, with peak memory down from about 14 GiB (plus swap) to about 8.5 GiB ## Linked Issues or Issue Description **What existing behavior does this improve?** The `Typecheck + Release Registry` job in `.github/workflows/pr-trusted.yml`. It takes about 9.2 minutes. The target for every job in the workflow is 7 minutes or less. **Subsystem affected** CI (`.github/workflows/pr-trusted.yml`) and the typecheck scripts under `scripts/` and `server/scripts/`. **Current behavior** After about 75 s of setup, the job runs six steps in sequence on a 4 vCPU runner, about 470 s in all (run 43344818387): `typecheck:build-gaps` 121 s, test programs 85 s, server layers 128 s, test projects 38 s, CLI guidance test 5 s, release registry tests 95 s. Most of these steps use one thread. The `typecheck:build-gaps` step builds the server with a full type check of the whole server program. The server layer step then checks the same files again, and the `Build` job checks them a third time. That full program peaks at about 13.5 GiB, so the job peaks at about 14 GiB of 16 GiB and uses swap. **Proposed behavior** The job runs `node scripts/ci-typecheck-lane.mjs`. The script builds the server without type checks (`PAPERCLIP_TSC_EMIT_ONLY=1`, as the Docker image does) and still checks the runtime assets. It runs the steps that write shared files first, one at a time. Then it runs the remaining checks in parallel, one per CPU, with the server declaration layers first, because the test layers and test projects read them. The checks that are not on that path run at a lower CPU priority. **Reason and benefit** On the same runner, the job's checks take 61% of the time and about 60% of the memory (see Verification). The job should take about 6 minutes on the fleet instead of 9.2. **Breaking changes** None. The same files are type-checked with the same options. Local commands do not change. `pnpm run typecheck:build-gaps` still works as before. ## What Changed - `scripts/ci-typecheck-lane.mjs` (new): runs the job's checks. It writes shared outputs first, in sequence: links, plugin SDK, server build without type checks, the runtime asset check, the sandbox provider installs, and the Runner typecheck (it rebuilds the eval kernel, which other checks read). Then it runs these checks in parallel: the server declaration layers, each server test layer, each test project, the test programs, the other build-gap workspaces, the CLI guidance test and the release registry tests. A passing check prints its output in a collapsed log group. A failing check prints its full output between `=== FAILED` and `=== end of` lines, with an error annotation. The script ends with a table of each check's start, duration and result, and exits non-zero if any check fails. - `server/scripts/typecheck-layers.mjs`: new `--layer <name>` option that checks one layer against the configs and declarations of an earlier `--declarations-only` run, and an exported `listLayers`. - `scripts/typecheck-test-programs.mjs`: new `--prepare-only` and `--skip-prepare` options, so the installs happen before the parallel checks start. - `scripts/run-typecheck-build-gaps.mjs`: new `--list-packages` option, so the script still finds the build-gap workspaces and still fails when one has no `typecheck` script. - `.github/workflows/pr-trusted.yml`: the `typecheck_release_registry` job runs the script instead of the six steps. No other job changes. - Tests: new `scripts/__tests__/ci-typecheck-lane.test.mjs` (added to `test:release-registry`), new cases in `server/scripts/typecheck-layers.test.mjs` and `scripts/typecheck-test-programs.test.mjs`, and the workflow wiring assertions in `.github/scripts/tests/ci-plan.test.mjs` and `scripts/__tests__/runner-release-crate-cache.test.mjs`. - `tests/runner-e2e/source-sdk-public-inputs.json`: repinned, because `package.json` changed. - `doc/DEVELOPING.md`: new "Typecheck lane in CI" section, and the new options. ## Verification This PR's own CI ran the old job definition, because PR runs use the workflow from master. So I measured the commands on a stand-in machine: a temporary workflow (`.github/workflows/typecheck-lane-bench-temp.yml`, now deleted) on GitHub-hosted `ubuntu-latest` (4 vCPU Intel Xeon 6975P-C, 15.7 GiB, no secrets, `contents: read`). It ran in runs [43444356017](https://github.com/paperclipai/paperclip/actions/runs/43444356017), [43445393958](https://github.com/paperclipai/paperclip/actions/runs/43445393958), [43446786105](https://github.com/paperclipai/paperclip/actions/runs/43446786105), [43448707837](https://github.com/paperclipai/paperclip/actions/runs/43448707837) and [43450786493](https://github.com/paperclipai/paperclip/actions/runs/43450786493). Each step ran under `/usr/bin/time -v`, and a sampler read `/proc/meminfo` every 0.25 s. **Old steps, one at a time (3 runs, run 43445393958):** | Step | Wall s | CPU s | Peak RSS of one process | System memory peak | | --- | --- | --- | --- | --- | | `typecheck:build-gaps` | 128–155 | 376–400 | 13.2–13.6 GiB | 14.2–14.7 GiB | | ...of which the full server `tsc` | 40–51 | 146–164 | 12.8–13.8 GiB | 13.7–14.7 GiB, swap up to 738 MiB | | ...replaced by the emit-only server build | 6–10 | 20–22 | 2.9 GiB | 3.9 GiB | | Test programs | 67–87 | 147–170 | 1.9 GiB | 2.9 GiB | | Server layers | 102–127 | 150–172 | 4.0–4.2 GiB | 5.4–5.6 GiB | | Test projects | 31–37 | 49–59 | 2.1–2.2 GiB | 3.1–3.2 GiB | | CLI guidance test | 2–4 | 2–3 | 0.2 GiB | 1.6 GiB | | Release registry tests | 82–95 | 159–183 | 1.7 GiB | 3.0 GiB | CPU seconds are about twice the wall seconds for the `tsc` steps even on one checker thread. So each step uses about two of the four vCPUs. **Old sequence and new lane on the same runner (4 runs, run 43450786493).** Each job copied one installed checkout, then ran both, in alternating order: | Run | Old steps, wall s | New lane, wall s | Ratio | Old memory peak | New memory peak | | --- | --- | --- | --- | --- | --- | | 1 | 503 | 314 | 0.62 | 13.98 GiB | 8.33 GiB | | 2 | 647 | 388 | 0.60 | 14.04 GiB, swap 637 MiB | 8.13 GiB | | 3 | 627 | 385 | 0.61 | 13.98 GiB | 8.63 GiB | | 4 | 382 | 237 | 0.62 | 14.10 GiB, swap 1.1 GiB | 7.99 GiB | Hosted runners differ in speed by up to 1.7 times, so only the same-runner ratio is meaningful. Both arms include a cold compile of the Runner release crate (45–80 s) that the fleet restores from cache. The fleet ran the old steps in 470 s; at the measured ratio, the new lane should take about 290 s, and the job about 6 minutes with setup. The new lane peaks at about 55% of memory, with no swap. Local checks (on a loaded Mac, for correctness only): - `node --test scripts/__tests__/ci-typecheck-lane.test.mjs` — 8 pass. - `node --test .github/scripts/tests/*.test.mjs` — 807 pass. - `node --test scripts/__tests__/e2e-shard.test.mjs scripts/__tests__/release-verify-workflow.test.mjs scripts/__tests__/runner-release-crate-cache.test.mjs scripts/__tests__/run-typecheck-build-gaps.test.mjs` — pass. - `pnpm exec vitest run scripts/typecheck-layers.test.mjs` in `server/` — the new cases pass. ## Risks - This changes a required job that every PR depends on, and its own CI cannot prove it. Please watch the first `Typecheck + Release Registry` runs on master after merge: the wall time (expect about 6 minutes), the summary table at the end of the step, and any `terminated by` message. - A type error now shows in that check's `=== FAILED` block, not in a step of its own. Passing checks are folded, so the failure is the only expanded block. - When the server declaration layers fail, the test layers and test projects are reported as skipped. Before, the test layers still ran after a source error. Fix the source errors first; the next run checks the tests. - The Honeycomb CI time board and any step-name queries see one step, `Typecheck, CLI guidance safety and release registry tests`, instead of six. The script prints the duration of each check. - The release registry tests run beside the type checks at normal priority. Their slowest test took 46–78 s here, against a 180 s timeout, and 114 s on the slowest stand-in runner. - Low risk for memory: the new lane peaked at 8.0–8.6 GiB of 15.7 GiB, far below the old 14 GiB. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), through Claude Code, with tool use (shell, GitHub CLI, file editing). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
0ece025749 |
perf(ci): cut repeated CPU work in the Runner support suite (#21390)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Every pull request runs the "Runner support suite" CI job (`pnpm test:e2e:runner:unit`) > - That job took 8.1–10.6 minutes, depending on the spot instance type, and was one of the last PR jobs to finish > - The target for every PR job is 7 minutes or less > - Measurement showed the suite is CPU-bound: each of the 693 Vitest files loads `@playwright/test`, the runner catalog and shared modules again in its own worker, and a few modules did quadratic or repeated work when they loaded > - This pull request removes that repeated work and reuses compiled code across workers > - The benefit is a job 75–86 s (13–16%) shorter on each instance type, with the same files, tests and assertions. It does not reach 7 minutes; the floor is explained below ## Linked Issues or Issue Description **What existing behavior does this improve?** The duration of the "Runner support suite" PR job. **Subsystem affected** CI, the Runner E2E support suite under `tests/runner-e2e/`, and one shared module (`packages/shared/src/connection-routing.ts`). **Current behavior** In 26 recent green PR runs the job took a median of 526 s on `m7i-flex.xlarge` (17 runs), 549 s on `m7i.xlarge` (3 runs) and 604 s on `m6a.xlarge` (6 runs). Vitest ran 693 files on 3 workers. The CPU was about 82% busy. 52% of the tracked time was module evaluation. **Proposed behavior** The job is shorter on every instance type. The same files and tests pass, and no assertion changes. **Reason and benefit** The PR workflow finishes sooner, so every pull request gets its result sooner. **Breaking changes** None. ## What Changed Measurements below are from the job's own runs on this PR. The spot fleet picks `m7i-flex`, `m7i`, `m6a` or `m5a` (all 4 vCPU); speed differs by up to 2x between them, so each comparison uses the same instance type. This suite has no Vitest retries, so a flaky test fails the job. ### Baseline (recent green PR runs on master, before this PR) | Run (job id) | Job | Test step | Vitest wall | Vitest split | | --- | --- | --- | --- | --- | | 131933126599 | 523 s | 474 s | 469.8 s | import 52%, tests 43%, transform 5% | | 131932775003 | 510 s | 461 s | 457.1 s | import 51%, tests 44%, transform 5% | | 131926764360 | 527 s | 479 s | 475.2 s | import 51%, tests 43%, transform 5% | | 131924082813 | 541 s | 496 s | – | – | | 131912527065 | 517 s | 468 s | – | – | - One Vitest invocation (693 files, 3 workers, the Vitest default of CPUs − 1), then 386 `node:test` tests in about 3 s. About 45–50 s of the job is checkout and dependency restore before the test step. - Summed tracked time was about 1,297 s for 470 s of wall time: effective concurrency 2.76 of 3 workers. Module evaluation was 674 s: 4,164 modules evaluated 77,699 times, because each file runs in a fresh isolated worker. - A CPU sampler (temporary, removed) showed 82% average CPU use with 3 workers and 95–99% with 4 workers. With 4 workers and no other change the wall time did not improve (455 s). The suite was CPU-bound, not idle. - Top modules by summed evaluation time (diagnostic run): `@playwright/test` 158 s over 390 files, `tests/runner-e2e/catalog.ts` 56 s over 141 files, `packages/shared/src/connection-routing.ts` 40 s over 317 files, `acpx/dist/runtime.js` 39 s. - Slowest files: `chat-rendered-links` 43 s, `results-root` 43 s, `chat-feature-flow` 43 s, `report` 40 s, `manual-private-transfer` 26 s, `private-framework-report` 24 s, `chat-catalog-fingerprint` 24 s, `hire-delegate-flow` 23 s, `source-sdk-evidence` 19 s, `unit-launcher` 16 s. No single file is near the 7-minute target, so there is no longest-file floor. - Slowest tests: the 256 MiB archive-cap test 23 s, the reload-once reply wait 20 s (plus two at 10 s), the documented pnpm validation 16 s, the retained name-only identity replay 14 s, the selected-case fingerprint replay 13 s. ### Changes, one at a time, with what each one bought 1. **Compile cache.** `run-unit.mjs` sets `NODE_COMPILE_CACHE` to `<tmpdir>/paperclip-runner-e2e-compile-cache` unless it is already set. Vitest workers and the Node processes that tests start reuse V8 compiled code. Node validates each entry against the source. `@playwright/test` evaluation went from 158 s to 107–124 s, `acpx` from 39 s to 26–27 s, and `zod` from 17 s to 9–10 s. Vitest went from 478 s to 441 s on a comparable run (this run also included change 2). 2. **Reuse import scans in the chat catalog fingerprint.** `fingerprintChatDependencyClosure` ran one esbuild parse for each of about 2,300 files on every call. The parse result is now cached by file path and content SHA-256 for the process. The fingerprint inputs and digest are unchanged. `chat-feature-flow` went from 43 s to 12–14 s, `chat-catalog-fingerprint` from 24 s to 11–13 s, and `chat-campaign-identity` from 13 s to 8 s. 3. **Hash each suite definition once.** `buildRunnerMatrix` computed `suiteDefinitionHash(suite)` for every execution, thousands of times, every time a test file loaded the catalog. It now computes it once per suite. The values are the same. `catalog.ts` evaluation went from 51 s to 9.6 s. Vitest went from 441 s to 389 s. 4. **Fake timers for the reload-once reply waits.** These three tests in `chat-rendered-links.test.ts` waited out the real 10 s render floor twice. They now advance fake time with `vi.runAllTimersAsync()`. The assertions on results, reload counts and attempt phases are unchanged. The file went from 43 s to 2.8 s. This freed a worker slot; on its own it did not shorten the CPU-bound run. 5. **Derive each aggregator identity once.** `connection-routing.ts` called `aggregatorAppIdentity` (two regex replaces) for every index entry for each of the 133 Arcade apps while it loaded, in 317 test files and in the server. The identity of each entry is now computed once. I compared the resulting `AGGREGATOR_SUPPORT_INDEX` (1,579 entries) and `searchAggregatorServices` results with master: they are identical. The SDK public input pins are refreshed for this change. 6. **Not shipped: four workers.** With the CPU work above removed, four workers cut Vitest from 379 s to 341 s (job 418 s). In the next run, `chat-installed-lane.test.ts` failed: the descendant of a timed-out preparation child was still visible to `process.kill(pid, 0)` right after cleanup. This is the process-tree-under-load risk, so the suite keeps the Vitest default of 3 workers. Docs: `tests/runner-e2e/README.md` describes the compile cache, how to turn it off, why the worker count stays at the default, and the cost of work at module load. ### Result | Run | Job | Test step | Vitest wall | Effective concurrency | | --- | --- | --- | --- | --- | | Baseline median, `m7i-flex` (17 runs) | 526 s | 465 s | 462 s | 2.76 | | Baseline median, `m7i` (3 runs) | 549 s | 476 s | 473 s | – | | Baseline median, `m6a` (6 runs) | 604 s | 535 s | 531 s | – | | This PR, `m7i-flex` (job 132007703795) | 440 s | 383 s | 379 s | 2.79 | | This PR, `m7i` (job 132027384596) | 474 s | 424 s | 420 s | 2.78 | | This PR, `m6a` (job 132033745288) | 526 s | 471 s | 466 s | 2.79 | | This PR, `m5a` (job 132040621237; dependency cache miss) | 880 s | 768 s | 761 s | – | The last three rows are the final configuration. The `m7i-flex` row is the same code before a README-only change. Module evaluation fell from 77,699 to 63,135 evaluations; part of that drop came from master changes merged into this branch, so I attribute time only to the per-change measurements above. ## Verification - CI on this PR is the experiment. Every push ran the real job on the real runner. The table above lists each run. - Local correctness checks, one process at a time: `chat-rendered-links.test.ts` (58 passed), the six chat fingerprint test files (37 passed), `catalog.test.ts` (92 passed), `packages/shared` `connection-routing.test.ts` and `connection-search.test.ts` (21 passed), and `node tests/runner-e2e/run-unit.mjs connection-auth.test.ts` (launcher path, 386 node:test passed). - Equivalence check for the connection routing change: the index and search results are byte-identical to master. - Flakiness: 10 runs of this branch with 3 workers (including the three final runs on `m7i`, `m6a` and the 2x slower `m5a`) passed with 0 failed tests and 0 timeouts. No port conflicts, OOM kills or `ps` timeouts appear in the logs. The suite has no retries. The 4-worker setting failed once in 3 runs, so it is not used. ## Risks - Low risk. No test is skipped or removed. No assertion is weakened. - The worker count is unchanged, so the load on process, port and temp-directory tests is lower than before (less CPU work per file). - The compile cache directory lives in the OS temp directory. Node rejects a stale or partial entry and compiles again. `NODE_DISABLE_COMPILE_CACHE=1` turns it off. - The connection routing change runs in the server too. It produces the same index, and it makes loading the module cheaper. ### What sets the remaining floor The job is about 45–55 s of checkout and dependency restore plus the Vitest run. The Vitest run is CPU-bound on 4 vCPU (2 physical cores). A fourth worker gives about 40 s more but fails a process-tree cleanup test under load (`chat-installed-lane.test.ts`, which checks the descendant immediately after cleanup). The largest remaining cost is per-file isolation: every file loads `@playwright/test` (about 0.3 s CPU each, 390 files), `zod` schemas from `packages/shared`, and `acpx` again. Vitest estimates that `isolate: false` would save about 140 s now. I did not use it: 116 files use `vi.mock`, and module state shared between files would make failures depend on which files share a worker. If more headroom is needed, the next step is a split across two machines (out of scope here because it changes `.github/`). Two duration-balanced halves (`vitest --shard=1/2` and `--shard=2/2`, about 190–235 s of Vitest each on `m7i-flex`/`m6a`) would put each job at about 4–5 minutes on every instance type except `m5a`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Claude Code agent with tool use (shell, GitHub CLI) and extended reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable (one test moved to fake timers; behavior changes are equivalence-checked) - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
a762b6bffd |
test(evals): keep the spend fixture's pointer to the tasks that record the charges (#21429)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - #21408 rewrote the spend-breakdown eval setup text in user words > - Its first wording ("Launch has recorded API charges on COM-3 (Morgan) and COM-4 (Riley)") dropped the old pointer that these tasks hold the charges > - In two paid runs the agent then gave correct totals but did not link the tasks that record them, which the case contract needs ("linked records support the totals") > - This pull request restores that pointer in plain words > - The benefit is that the setup gives the same information as before, without jargon ## Linked Issues or Issue Description Refs #21408. This commit was pushed to #21408 after it merged, so it is a follow-up. **What happened?** Paid runs 43451957487 and 43455387724: `budgets-costs.spend-breakdown` failed `actual-agent-breakdown-and-incomplete-total` because the reply did not link COM-3 and COM-4. **Expected behavior** The setup says where the charges are recorded, as the original text did. **Steps to reproduce** 1. Run `budgets-costs.spend-breakdown` with the #21408 wording. ## What Changed - The setup description now says "Launch's API charges this week are recorded on tasks COM-3 (Morgan) and COM-4 (Riley)." ## Verification - Paid run 43460401711 with this wording: `budgets-costs.spend-breakdown` passed. The reply links COM-3, COM-4, the project and the unpriced run, with no internal terms. ## Risks - Low. Eval setup text only. ## Model Used - Claude, claude-opus-5-5, with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
edb6945ff3 |
chore(lockfile): refresh pnpm lockfiles (#21439)
## Thinking Path > - Paperclip uses a dependency lock for reproducible builds. > - Dependency changes on master need matching lock entries. > - Paid eval reports also pin a reviewed offline correction. > - Reports must install before paid model work starts. > - This automated PR keeps the lock and report proof together. ## Linked Issues or Issue Description **What happened?** Dependency manifests on master changed after the last committed lock refresh. **Expected behavior** Frozen installs must match those manifests. Eval reports must retain matching offline lock proof. **Steps to reproduce** Run a frozen install before and after this lock refresh. ## What Changed - Refresh the dependency lock from the manifests on master. - Refresh the sandbox provider lockfiles from their manifests on master. - Include the matching offline report correction and hashes. - Resync the SDK public source proof's lock pins from the refreshed lock. ## Verification - Pinned pnpm resolution completed with lifecycle scripts disabled. - The report proof refresh accepted the reviewed committed baseline. - The workflow rejected changes outside the lock and report proof files. ## Risks The lockfile bot merges this PR without CI or review when every added package version is at least 3 days old on npm. Otherwise the PR waits for a maintainer, and the daily refresh tries again. The bot merges only the exact commit this workflow generated from manifests already reviewed on master. Unknown report lock baselines remain blocked. This update changes no permissions or paid enable gates. ## Model Used None — generated by pnpm and the deterministic report-lock updater. ## Checklist - [x] Thinking path and problem description included - [x] Generated lock and matching report proof included - [x] Release-age gate decides between bot merge and maintainer review Co-authored-by: lockfile-bot <lockfile-bot@users.noreply.github.com> |
||
|
|
e7e28ba241 |
feat(queue): show tasks waiting for an agent's slot as waiting, with their place in line (#21405)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Each agent runs at most `runtimeConfig.heartbeat.maxConcurrentRuns` tasks at once. Other runs wait in the agent's queue with status `queued` > - On hosted staging, one agent had all 20 slots full and 295 queued runs. Some tasks waited 3 to 7 hours and never started > - The UI showed these tasks as live or working: a pulsing "Live" badge in the task header and in task rows, a spinning "Working … for 4 hours" in the task chat, and "N live" counts that included every queued run > - People could not tell that work was waiting, why it waited, or what to change > - This pull request makes a deep queue an honest, visible state: "Waiting for Codie · 20 slots max · 41 of 295 in line · Configure" > - The benefit is that operators see the real state and the one setting that changes it ## Linked Issues or Issue Description Refs #21391, #21349, #21242 **What happened?** A queued run that never started looked the same as a running run. The task header, the task rows in lists and the inbox, the task chat and the sidebar counts all showed queued runs as live or working. **Expected behavior** A run that waits for one of its agent's concurrency slots shows that it waits. It names the agent, the agent's limit and the run's place in line, and it links to where the limit is set. The place in line is the order in which the server starts the agent's queued runs. **Why does it matter?** With a deep queue, the UI said "working" for hours while nothing ran. Operators could not see the cause or the fix. ## What Changed - Server: a new module `server/src/services/heartbeat/queued-run-line.ts` now holds the facts that the queued-run dispatcher orders by (`queuedRunDispatchFacts`). The dispatch path in `queue.ts` and the new "place in line" read use this one builder, so the line the UI shows is the dispatch order. The logic moved without change, including the recovery-share fix from #21391. - Server: `executionProjectionsForRuns` adds `agentQueue` (agent, limit, running count, position, queued count) and the label "Waiting for <agent>" with cause `agent_at_capacity` to a queued, never-started run when its agent has no free slot. It reads one batch of agents and running counts. Then it builds one line per full agent (at most 8 agents per read), using only the three context keys that ordering needs. No new polling. - Server: the place in line uses `queuedRunDispatchLine`, an O(n log n) replay of the dispatcher. The plain replay rescans the queue after each step, which is quadratic (measured: 3.6 ms at 300 runs, 76 ms at 3,000, 600 ms at 10,000, all on the event loop). Dispatch itself is unchanged: it stops after the free slots. Each agent's line is memoized for 5 s and shared by concurrent reads. In a preview worktree, the line applies the same creation-time cutoff as dispatch. - Server: the task list summary adds `runningDescendantCount` (descendants whose run started) next to `liveDescendantCount`. `liveDescendantCount` still decides whether a blocked parent is covered. The "n live below" chip uses the new count. - Shared: the new `ExecutionAgentQueue` type and the optional field `ExecutionProjection.agentQueue`. - Server: `/companies/:companyId/live-runs` lists running runs before queued runs, so a deep queue cannot push running work out of the 50-row limit. - UI: a new `AgentQueueWait` component shows "Waiting for Codie · 20 slots max · 41 of 295 in line · Configure", with no spinner and no elapsed timer. Configure opens the agent's Advanced view, where the run policy (`maxConcurrentRuns`) is edited. The task chat (both renderers), the run ledger and the Storybook story use it. - UI: the task header and the task rows (task list, inbox, legacy views) show a quiet "Waiting" instead of the pulsing "Live" when every live run of the task has not started. The full sentence shows on hover. "n live below" counts only started sub-tasks. A waiting sub-task still covers its blocked parent. - UI: the sidebar "N live" counts (Dashboard, agents) count only started runs. - UI: the task chat pill now names other server holds on a queued run (for example "Waiting for maintenance") instead of "Working". - Docs: DESIGN.md records the waiting-for-slot hold. - Storybook: `runs-agent-queue-wait` stories (full line, task chat, task row, one slot, mobile), with play checks. ## Verification - Red/green server test: `server/src/services/execution-projection-agent-queue.test.ts` (embedded Postgres). Before this change, the projection has no `agentQueue`. Now it checks the agent name, limit, running count, positions 1 to 4 in dispatch order (a person's retry first, then priority, not creation order), no wait while a slot is free, and isolation from other companies' runs. It also checks that the positions equal the line that the dispatch builder produces. - `pnpm exec vitest run` on the new test, `queued-run-dispatch-order`, `recovery-wake-herd`, `heartbeat-queued-run-claim-isolation` and `agent-live-run-routes` in `server/`. - Runtime cost (rule `runtime-dependency-boundaries`): the assumption is that the line stays cheap for a deep queue. `queued-run-line.test.ts` checks that `queuedRunDispatchLine` equals the dispatcher's claim-every-run order over 400 random queues (ages around the aging boundary, recovery and person runs, blocked runs, deferred share runs, all share counters). It also orders 10,000 runs in under 250 ms (the quadratic replay takes about 600 ms). The DB test projects a 300-run queue end to end with correct positions 1 to 300. Remaining limit: the line is up to 5 s stale. - Red/green: `issue-list-assignee-filter-routes.test.ts` now asserts `runningDescendantCount: 0` for a parent whose only live descendant is queued and never started. - UI tests: `liveIssueIds.test.ts` (waiting tasks, started runs keep "Live", no "running below" count for waiting sub-tasks) and `TaskChatLiveRunPill.test.tsx` (waiting line with no spinner and the Configure link, other holds named). The suites for the run ledger, sidebar, sidebar agents, task detail, task columns and task chat also pass. - The Storybook stories render in headless Chromium and their play checks pass. - After deploy: open a queued task on staging and confirm the waiting state. ## Risks - Cost on read: a projection read that includes a queued run of a full agent now reads that agent's queued runs (only small columns) and the dispatch facts. This is at most one line per full agent every 5 s, for at most 8 agents per read. Reads with no queued, never-started runs do no more work. - The place in line is the server's prediction of dispatch order. A claim that is refused at start time (a gate, a blocker) can let the next run start first. The position updates on the next refresh. - The process-local "declined aged claim" hint that the dispatcher uses is not visible to other servers, so the projection does not use it. This can change only the aged-oldest share step. - Configure opens the agent's Advanced view (the YAML run policy). The agent's Runtime tab hides the run-policy fields today. - Low risk otherwise: no schema change and no migration. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, headless browser) and subagents for the UI survey. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
e62fe18c72 |
fix(evals): wait for the primary save, select the offered Launch choice, and pace the keyboard reorder (#21396)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The chat eval catalog checks real chat journeys against saved product state > - Full cohort 12 (OpenCode with `openrouter/deepseek/deepseek-v4.1-flash-1017`) failed three cases where the agent did the correct thing > - In each case the harness, not the product, broke the journey: it left a page too early, could not answer a choice card, or pressed keys faster than the drag library listens > - A harness that breaks the journey cannot measure the product > - This pull request makes the harness act like a person in these three journeys, counts a verified card choice as a human action, and keeps each check able to fail a broken product > - The benefit is that these three cases measure the product again ## Linked Issues or Issue Description **What happened?** In full cohort 12, three cases failed for harness reasons: - `agents.choose-primary`: the CEO answered correctly. The harness clicked the crown and confirmed. Then it waited for "My primary agent" with `.first()`. The sidebar already showed the CEO's crown, so this wait passed at once. The harness then navigated away before the browser sent `PUT /primary-agent/me`. The server log has no PUT. The read-back still showed the CEO. - `tasks.ambiguous-task`: the CEO changed nothing and asked a correct question card. The card linked both tasks and offered "COM-1 (Launch project)", "COM-2 (Experiments project)" and "Both tasks". The harness could only answer with a text box or an "Other" option. It threw "The linked question has no supported explicit clarification input". - `steering.edit-queued-message`: the person's edit saved (PATCH 200). The keyboard reorder (Space, ArrowUp, Space) sent no `PUT /queued-comments/order` in 2 gestures. Cohort 9 had the same failure. **Expected behavior** Each journey completes as a person does it, and the oracle grades the saved result. **Steps to reproduce** 1. Use master `f7ddab8e94`. 2. Run the Runner Full-Stack E2E workflow with `all=false` and the three case ids `chat-features.runner-opencode.local.<case>`. 3. Read each case's `playwright.log` and `server.log`. **Additional context** - Primary agent: the crown fills at once (optimistic update) and the save starts on confirm. The mutation lives in the app-level `PrimaryAgentProvider`, so in-app navigation does not cancel it. In the failed run the confirm and the full-page `goto` happened within the same second, and the PUT never reached the server. Only a full page load in the first milliseconds can lose the save. A person cannot do that in practice, so this is not a product bug. No UI change. - Keyboard reorder: dnd-kit's `KeyboardSensor` starts the drag on Space, adds its keydown listener in a `setTimeout`, and `sortableKeyboardCoordinates` ignores arrows until the rows are measured on a later render. Keys pressed back to back can arrive before this. In the production Storybook story (headless Chromium), back-to-back `locator.press` moved the item in 6 of 8 tries. Keys about 150 ms apart moved it 5 of 5 times, also after an Edit and Save. With the drag-start wait and a paint between keys it moved 16 of 16 times, also at 6x CPU throttle. The keyboard reorder works for people, so this is not a product bug. No UI change. ## What Changed - `runtime-primary.ts`: wait for the browser's own `PUT /primary-agent/me` that names Morgan, require 2xx, and poll the read-back until Morgan is primary before leaving the page. Look for the "My primary agent" crown only in the header that holds Morgan's avatar. Add `isPrimarySaveRequest`. Oracle `runtime-primary.v6`. - `chat-linked-question.ts` / `internal-tasks.ts`: add `chosenTaskOption`. It finds the one option of the bound saved question that names the Launch task's identifier (a setup literal) and no other candidate's. It uses the option id for the rendered DOM id and the payload's submit label. It does not read option wording. The harness selects that option, checks `aria-checked`, submits, and waits for the `respond` POST. The old text and "Other" paths stay for other cards. The checks that nothing changes before the answer and that only COM-1 changes after it are unchanged. - `chat-catalog-contract.ts` / `internal-tasks.ts`: a chosen option saves no comment, so the flow counted only one human action and failed `catalog-original-human-query`. `tasks.ambiguous-task` now joins the cases that may count a bound UI answer. The case declares `expectedHumanQuestionAnswer` only after the harness answered through the choice, and names the chosen question and option. The flow then verifies the answered card independently: answered by this person, exactly the Launch option and no typed text, one audit event, and one continuation run after it. Oracle `internal.ambiguous-task.v8`. - `chat-catalog-contract.ts`: `chatCatalogHumanQuestionPolicies` takes the answering person and mirrors the server's user rule in `evaluateIssueThreadInteractionResolverAudience`. `anyone` and `human_only` admit the person. `not_creator` admits a person who did not create the card. With no resolver (a pending card) `not_creator` stays rejected, so other callers keep their behavior. For the ambiguous-task choice, an unaddressed card is accepted, as the server allows. - `runtime-queue.ts`: `keyboardMoveQueuedMessageUp` waits for the handle's `aria-pressed="true"` after Space. It lets the page paint before ArrowUp and before the closing Space. It records whether each gesture started a drag. Oracle `edit-queued-message.structural.v7`. - No scenario YAML changed, so the TestOps manifest needs no entry. ## Verification - `npx vitest run --config tests/runner-e2e/vitest.config.ts` for `chat-linked-question.test.ts`, `chat-catalog-consistency.test.ts`, `runtime-primary.test.ts`, `runtime-queue.test.ts`, `internal.test.ts`, `external-company-secret.test.ts`, `external-personal-secret.test.ts`, `turn-response-card.test.ts`, `runtime-task-coordination.test.ts`: pass. - Red/green: the two new keyboard tests fail on the old `keyboardMoveQueuedMessageUp` and pass on the new one. The resolver-policy test fails on the old `chatCatalogHumanQuestionPolicies` (`not_creator` answered by a person who is not the creator) and passes on the new one; `not_creator` answered by its creator and a pending card stay rejected. The choice-accounting test fails on the old allow list. - `chosenTaskOption` tests use the cohort 12 card shape, a reworded card, and cards with no single choice (both, duplicate, neither, `COM-10`, a different card). - `npx tsc -p tests/runner-e2e/tsconfig.json --noEmit`: pass. `node --test tests/runner-e2e/chat-full-catalog-testops-manifest.test.mjs`: 10 pass. - Paid narrow run 1 (commit `33399f8370`): https://github.com/paperclipai/paperclip/actions/runs/43446445525. `agents.choose-primary` passed (browser `PUT /primary-agent/me` 200). `steering.edit-queued-message` passed (browser `PUT /queued-comments/order` 200). `tasks.ambiguous-task` selected COM-1 (`respond` 200, only COM-1 became high) but failed `catalog-original-human-query`, because the flow did not count the choice. Later commits fix that. - Paid narrow run 2 (`c90c2aa26a`) was cancelled by run 3 before its case finished. - Paid narrow run 3 (`tasks.ambiguous-task` only, final verifier `fee3bbd453`): https://github.com/paperclipai/paperclip/actions/runs/43452010284 ## Risks - Low risk. Test harness only. - The ambiguous-task harness now answers through a structured choice when one exists. If the agent asks a card with no option that names only the Launch task, the old text and "Other" paths still apply. - `chatCatalogHumanQuestionPolicies` accepts `not_creator` only when a resolver is passed. Callers that check pending cards pass none, so their behavior is unchanged. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
b7774a01cf |
fix: name activity actors and explain stopped tasks in plain words (#21406)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - People must never see internal jargon or raw ids in chat replies or
task comments
> - Chat eval cohort 12 found three sources: activity records show the
board user only as the raw id `local-board`, agents turn internal field
names such as `checkoutRunId` into words like "no checkout", and a
recovery notice says "the original assignee is not invokable"
> - The recovery notice appeared on 170 fixture tasks. It is not a loop:
each task got it once, because recovery blocked every in-progress task
of an agent that a person had paused
> - This pull request names activity actors, lists these internal terms
in the chat guidance, rewords the notice, and leaves in-progress work of
a manually paused agent alone
> - The benefit is plain replies and no false "blocked" tasks after a
person pauses an agent
## Linked Issues or Issue Description
**What happened?**
- `activity.audit-change`: the agent read `GET
/api/issues/{id}/activity` and wrote that the change was made by "the
board user identity (`local-board`)". Activity rows carry only
`actorType` and `actorId`.
- `activity.company-overview`: the agent wrote "no checkout" to explain
why no one is working on a task.
- `tasks.organize-open-work`: 170 of 501 fixture tasks got one system
comment each: "Paperclip cannot safely continue automatic recovery
because the original assignee is not invokable...". The owners were
paused by a person, and the stranded-work sweep blocked each in-progress
task. The existing rule already keeps `todo` work of a manually paused
agent, because blocking it fights the pause and strands the task after
resume.
**Expected behavior**
Replies name actors by name ("Board", a member's name, an agent's name)
and describe states in plain words. A person's pause does not block the
agent's in-progress work. A stopped-task notice says what happened and
what to do, in plain words.
**Steps to reproduce**
1. Change a task's priority as the local board user, then ask the chat
agent who changed it.
2. Pause an agent that owns in-progress tasks with no active run, and
let the stranded-work sweep run.
## What Changed
- `server/src/services/activity.ts`, `server/src/routes/activity.ts`,
`packages/shared/src/types/activity.ts`: company and task activity reads
add `actorName` (agent name, member name, "Board" for `local-board`,
"Paperclip" for the system). Raw ids stay in their existing fields.
- `server/src/services/agent-conversations.ts`: the existing "no raw IDs
or internal codes" rule now names checkout, native run and local-board
as internal terms, and tells the agent to name an actor by `actorName`.
- `server/src/services/recovery/service.ts`: the notice is now "<agent>
can't run right now, so Paperclip stopped this task instead of retrying
it. It is still assigned to them. To continue, resume or fix the agent,
or assign the task to someone else." The manual-pause skip now covers
`in_progress` as well as `todo` work with no run.
## Verification
- `cd server && npx vitest run
src/__tests__/heartbeat-paused-assignee-new-task.test.ts`: 4 pass. With
the old recovery service the two new tests fail (`expected 'blocked' to
be 'in_progress'`, and the old notice text).
- `cd server && npx vitest run src/__tests__/activity-service.test.ts -t
"names each activity actor"`: passes (the method did not exist before).
- `cd server && npx vitest run
src/__tests__/agent-conversations.test.ts`: passes.
- `src/__tests__/activity-routes.test.ts` has three tests that time out
locally on unchanged master code too; CI runs them.
- Paid eval run of `activity.audit-change` and
`budgets-costs.spend-breakdown` on a branch with this change.
## Risks
- The chat prompt grows by about 40 words. This is a small part of the
existing directive, so time to first token does not change in a
measurable way.
- A budget- or error-paused agent still gets the notice, now in plain
words. Only a manual pause keeps in-progress work as is; the sweep picks
it up after resume.
## Model Used
- Claude, claude-opus-5-5, with tool use and code execution in Claude
Code.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
04f12fd2be |
test(evals): use plain words in the spend fixture and a relative link in the handoff message (#21408)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The chat catalog evals check that a chat agent does real product work for a person > - Two eval setups put text in front of the agent that a real person would not write > - `budgets-costs.spend-breakdown` wrote "imported synthetic metered-API receipt tasks", "native run" and "native subscription usage" into the conversation's task description, and the agent repeated these terms to the person > - `agent-chat.multiple-handoffs` pasted an absolute `http://127.0.0.1:<port>/api/attachments/...` link into the person's message, and the agent echoed it > - This pull request writes both in user-facing terms, so the eval does not teach the agent the jargon that it then is marked down for > - The benefit is that a jargon finding in these cases points at the product, not at the fixture ## Linked Issues or Issue Description **What happened?** In chat eval cohort 12, the spend-breakdown reply said "completed native run", "Native subscription usage" and "imported synthetic metered-API receipt fixtures". All three phrases come from the description that the eval setup saves on the conversation task. The multiple-handoffs reply had an absolute `127.0.0.1` attachment link, copied from the setup's own chat message. **Expected behavior** Eval setup text uses words and links that a person would use, so the reply is graded on product behavior. **Steps to reproduce** 1. Read `tests/runner-e2e/chat-catalog-domains/identity-accounting.ts` (spend-breakdown setup description). 2. Read `tests/runner-e2e/chat-catalog-domains/runtime-multiple-handoffs.ts` (setup chat message). ## What Changed - spend-breakdown setup description: "Launch has recorded API charges on <task> (Morgan) and <task> (Riley). Morgan also finished one work session on Launch (run <id>) whose usage is saved but not yet priced. Leave out charges from other projects and earlier weeks. Plan subscription fees are separate from API charges." The same facts, scope and exclusions stay. The cost events and the graders do not change. - multiple-handoffs setup message: the logo link is the relative route `/api/attachments/<id>/content?download=1`. The grader binds the message by the attachment id, which stays. ## Verification - `npx vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/chat-catalog-fingerprint.test.ts tests/runner-e2e/chat-catalog-domains/identity-accounting.test.ts tests/runner-e2e/chat-catalog-domains/runtime-multiple-handoffs.test.ts tests/runner-e2e/chat-boundary-fingerprint.test.ts`: 85 pass. - Narrow paid run of `budgets-costs.spend-breakdown` on a branch with this change. ## Risks - The setup description no longer says that the charges are imported test data. The charges are real cost records in the disposable company, so the agent reports them as recorded charges, which is correct for the person. ## Model Used - Claude, claude-opus-5-5, with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
944b1c64d7 |
feat(mcp): keep long provider calls durable with the MCP Tasks extension (#21263)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents use connected apps through Paperclip's MCP client in the tool gateway > - Some provider actions run for minutes: CI runs, deploys, batch jobs > - Today such a call is lost if it outlives the tool timeout or if Paperclip restarts or deploys mid-call, and a retry can start the work a second time > - Recovery must just work, and the target rate of runs that need a manual retry is 0% > - This pull request implements the MCP Tasks extension (`io.modelcontextprotocol/tasks`, 2026-07-28) in the MCP client > - The benefit is that a long provider call becomes a durable task: it survives timeouts, restarts, and deploys, and it is never started twice ## Linked Issues or Issue Description **Problem** A long provider tool call holds an HTTP request open. When it passes the tool timeout, or when Paperclip restarts or deploys during the call, the result is lost. The agent then often repeats the call, and the provider does the work again. **Proposed solution** Implement the client side of the MCP Tasks extension. Declare it on stateless tool calls. Store each returned task durably, wait on it during the call, keep polling it in the background after the call ends, and let the same call attach to the stored task instead of sending `tools/call` again. **Why it matters** CI, deploy, and batch actions are where agents lose the most work today. A durable task handle makes these actions safe across timeouts, restarts, and deploys, without a person. Refs #21119, Refs #21140, Refs #21219 (the MCP Apps, stateless MCP client, and MCP Skills work this builds on). ## What Changed - **Off by default:** MCP Tasks are behind the new instance experimental setting `enableMcpTasks` ("Durable app actions (MCP Tasks)"), default off. When it is off, Paperclip never declares the extension, nothing attaches to or polls tasks, pending rows do not block idle sleep, and behavior matches master. A test covers the off state. - `mcp-http.ts`: the tasks extension is declared per request, only on HTTP `tools/call` and `tasks/*`. Discovery, listings, and stdio servers do not declare it. `tasks/*` requests put the task ID in `Mcp-Name`. - New `mcp-tasks.ts`: parses task results, bounds the server's timing hints, sends `tasks/get`, `tasks/update`, and `tasks/cancel`, and seals task IDs at rest (AES-256-GCM, bound to company and connection; a keyed hash gives uniqueness). - New company-scoped table `tool_mcp_tasks` (migration `0475`, online-compatible). It links the task to the call, run, task, connection, and grant. It stores no provider result. - `tool-gateway.ts`: - A returned task is recorded, then polled up to the normal tool timeout. Its result goes through the same validation, redaction, and recording as a direct result. - Past the timeout, or when the caller disconnects, the agent gets `provider_task_pending` ("call again with the same arguments"). The task stays durable. - A call with the same endpoint, tool, full arguments, grant, agent, task, and actor attaches to the undelivered task. A retry with the same idempotency key also attaches, and it can never start different work under that key. - Input: URL requests show the provider link (Paperclip never answers them). Form requests become a question card; only the person's answer is sent with `tasks/update`. Other requests stop with a clear "needs input" error. - Stopping the run sends `tasks/cancel` and marks the task cancelled. A caller that only disconnects keeps the task. - `failed` becomes a tool error with the provider's message. An unknown task or a passed TTL becomes `provider_task_expired`. Every task stops being tracked at its TTL, or after 7 days at most. - Audit entries: `tool_gateway.task_started`, `task_input_sent`, `task_cancelled`. - New `mcp-task-work.ts`, registered at startup with the shared deadline scheduler: it resumes polling pending tasks after a restart, cancels tasks whose task closed or run was stopped, and wakes the assignee once when a task settles after its run ended. - Docs: `doc/connections/MCP-TASKS.md`, linked from `doc/connections/README.md`. Background work (per `rules/sleep-aware-background-work.mdc`): trigger is a task row write or release (`mcp-tasks` queue) and one startup pass. The deadline is the earliest `next_poll_at`. Cadence is the server's `pollIntervalMs` within 5 s – 5 min, with backoff; 5 min while waiting for a person. It is quiet when no row has a next poll. This is the narrow external-source exception: providers give no wake Paperclip can receive, and polling stops with the task. ## Verification - `server/src/__tests__/mcp-tasks.test.ts`: 20 tests with a fake 2026-07-28 server that returns tasks only to a client that declares the extension and checks `Mcp-Name`. They cover: - polling to completion, with the result identical to a synchronous call; - timeout, then an attaching call with no second `tools/call`; - a keyed retry, and a reused key with different arguments (refused); - restart mid-task: a fresh gateway with no in-memory state polls the row, wakes the agent once, and the recovered run gets the result once with no second `tools/call`; - no wake while the run is still going; - URL input (link, never answered), and form input (question card, then `tasks/update` with the answer); - cancellation by the run, by closing the task, and no cancellation when only the caller leaves; - failure, an unknown task, and TTL expiry (ends in under 3 s, no hang); - another agent never attaches; - synchronous calls unchanged and older servers never see the extension. - Red/green: with attach disabled, the timeout, keyed-retry, and restart tests fail (3). With `tasks/cancel` disabled, both cancellation tests fail (2). Before `tools/call` declared the extension, 13 end-to-end tests failed. All pass with the change. - Existing suites pass: `mcp-protocol-versions`, `mcp-http`, `mcp-apps`, `mcp-skills`, `server-startup-feedback-export`, `tool-gateway`, `tool-gateway-discovery-http`, `remote-mcp-connectors` (342 tests). - `tsc --noEmit -p server` is clean. `pnpm --filter @paperclipai/db check:migrations` passes (the migration is classified online-compatible). Snapshots pruned to five. - Agent boot and time-to-first-token: no change in the boot or first-turn path. The only new per-call cost is one indexed lookup on stateless tool calls. - Not run: a live provider that returns tasks. No public MCP server that returns tasks was available without new credentials. ## Risks - Shipped off by default. Review findings that only affect the setting-on path after this merge are tracked as follow-ups. - Migration: a new table with indexes and foreign keys to `companies`, `tool_invocations`, and `tool_connections`. Online-compatible. - Behaviour change only for servers that return tasks, which they may do only to clients that declare the extension. Older servers and stdio servers are unchanged. - A task that settles while nobody is waiting stays attachable for 1 hour, so a deliberate repeat of the same call within that hour collects the earlier result instead of starting new work. - The wake after a background task settles uses the generic task wake; the agent must call the tool again to collect the result. - Follow-ups: `notifications/tasks` via `subscriptions/listen`; sampling and roots input requests; showing a long action's progress in the task chat. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with tool use (shell, file edits, local test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
11840f9904 |
fix(evals): state the product's handoff authority in the handoff-same-task workflow fixture (#21383)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Cases move one primary task between agents through handoffs
> - The chat eval `cases.handoff-same-task` writes a workflow document
on the case's primary task during setup
> - That document said: "Case handoff authority belongs to the actual
primary task execution." The product has no such rule.
> - #21372 changed the grader to accept the product's real rule. The
fixture text still told the agent the wrong rule.
> - This pull request rewords the sentence to match
`server/src/services/case-handoffs.ts`.
> - The benefit is that the setup does not steer the agent to a
procedure that the product does not require.
## Linked Issues or Issue Description
Refs #21372
**What happened?**
The setup workflow document in
`tests/runner-e2e/chat-catalog-domains/integrator-case-handoff.ts`
stated a handoff authority rule that the product does not have.
**Expected behavior**
The fixture states the product's rule. The assignee may hand off its own
task unless another run owns that task ("Agents may hand off only their
own task" and "Another run owns this task" in `case-handoffs.ts`).
**Steps to reproduce**
Read the `workflow_document_text` of the `cases.handoff-same-task`
setup. Compare it with the agent handoff checks in `case-handoffs.ts`.
**Paperclip version or commit**
master
|
||
|
|
8af889f612 |
feat(mcp-apps): view messages with consent, app context, full screen and provider tool icons (#21266)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - MCP Apps (#21119) show interactive widgets from MCP servers, such as Circleback meeting cards, with the agent's reply in the task chat. > - The first version deferred three parts of the MCP Apps spec. A widget could not propose a chat message (`ui/message`). It could not give the agent context (`ui/update-model-context`). It could not go full screen. > - Paperclip also ignored the `icons` that MCP servers declare on tools (SEP-973). > - The security review of #21119 showed what these features need from day one: explicit consent, untrusted-data labels and strict icon sanitizing. > - This pull request adds the three host features and provider tool icons, with those protections built in. > - The benefit is that widgets can now work with the person and the agent, not only display data. ## Linked Issues or Issue Description Refs #21119 **Subsystem affected** MCP Apps host (task chat widgets), tool catalog, run wake prompt. **Problem or motivation** The MCP Apps host refused `ui/message` (-32601), dropped `ui/update-model-context` and kept every view inline. MCP tool `icons` were not parsed or shown. So widgets could not ask the agent a follow-up, the agent could not see what the person selected in a widget, large widgets were cramped, and tool rows used only generic icons. **Proposed solution** Add consent-gated view messages, per-view app context delivered to the task's next run as untrusted data, a full-screen dialog, and sanitized provider icons. Details are under "What Changed". **Alternatives considered** - Send view messages with a new server route that creates the comment itself. Rejected: comment creation has many rules (agent chat ownership, paused conversations, queued delivery, wakeups, receipts). The widget's Send button uses the same send path as the composer. A small route then records which view the comment came from. - Keep the iframe in place for full screen with CSS. Rejected: chat containers can create containing blocks that break `position: fixed`. A moved iframe always reloads, so the dialog hosts a new frame instead. **Roadmap alignment** Continues the MCP feature work after #21119, #21140 and #21219. It does not touch the MCP Tasks work (gateway `tools/call` polling and elicitation). ## What Changed - **`ui/message`.** The host shows "<App> wants to send: …" with **Send** and **Dismiss** under the widget (and inside the full-screen dialog). Nothing is sent before the click. Send posts the text as the viewer's own comment through the chat's normal send path (`handleThreadAdd`). `POST /api/issues/:issueId/mcp-app-views/:viewId/messages` then logs `issue.mcp_app_message_sent`. It accepts only the caller's own comment on the same task, matched by `clientRequestId`, from the last 10 minutes. One proposal per view can wait at a time. The view gets `{}` after the send, or -32000 "Message sending denied" after Dismiss. Only `user` text up to 8,000 characters is accepted. - **`ui/update-model-context`.** `PUT /api/issues/:issueId/mcp-app-views/:viewId/model-context` stores the latest context per view in `tool_mcp_app_views.model_context` (text blocks and `structuredContent`, 16 KB at most). Updates are applied in order and deduped in the browser and on the server. A change logs `issue.mcp_app_context_updated`, and an empty update clears the context. The task's next run gets the three most recently updated contexts as an `## App context (untrusted)` wake-prompt section. The section quotes the data as JSON and says it is third-party data, not instructions. The read runs in parallel with the other run preparation and never fails the run. CLI adapters and the native runner render it through the same `renderPaperclipWakePrompt`. - **Who may do this.** Messages and context need a signed-in person with `issue:comment` on the task. Agents and API keys get 403. Other tasks and companies get 404. `message` and `updateModelContext` are advertised in `hostCapabilities` only where the viewer can send messages. - **Full screen.** The host advertises `availableDisplayModes: ["inline","fullscreen"]` and handles `ui/request-display-mode`. It honors the view's declared modes and answers with the resulting mode. It sends `host-context-changed` with `displayMode` and the container (fixed `height`/`width` in full screen), and again when that container resizes. A new **Open full screen** control sits next to "Widget from …". Full screen is a Paperclip dialog that closes with Exit, Escape or the view's own request. - **Provider tool icons.** `icons` on MCP tool descriptors are sanitized into the new `tool_catalog_entries.icons` column, outside the version hash, so a new icon never quarantines a reviewed tool. Only `https:` URLs and `data:image/*` URIs are kept, and SVG only as `data:`. A `data:` icon is at most 32 KB, a URL at most 2,048 characters, with at most 4 icons per tool. `GET /api/companies/:companyId/tool-icons` lists them keyed by gateway tool name, in one cached request per chat. Task chat tool rows and the app's actions list show them through `ProviderToolIcon`. It renders `<img referrerPolicy="no-referrer">` at a fixed size and never inlines markup. It checks the source again before rendering and falls back to the taxonomy icon. - **Schema.** Migration `0477` adds `tool_catalog_entries.icons`, `tool_mcp_app_views.model_context` and `model_context_updated_at`, all nullable. - **Docs and stories.** `doc/connections/MCP-APPS.md` documents the new behavior, and the finished items are removed from "Limits and follow-ups". `Tasks/MCP App views` has new stories: "View asks to send a message", "Full screen" and "Tool rows with provider icons". The fixture view now uses `ui/message`, `ui/request-display-mode` and `ui/update-model-context`. ## Verification - `pnpm exec vitest run server/src/__tests__/mcp-apps.test.ts server/src/__tests__/tool-catalog-publication.test.ts server/src/__tests__/openapi-routes.test.ts packages/shared/src/mcp-icons.test.ts packages/shared/src/mcp-apps.test.ts`: pass (embedded Postgres). These tests cover: - Context: stored, deduped, logged once, 413 when too large, cleared, scoped to the task and company, refused for agents (403), and rendered into the wake prompt as untrusted data. - Messages: attributed only to the viewer's own recent comment on the same task. A teammate's comment, an unknown request id or an old comment gets 404. An agent gets 403. - Icons: stored sanitized, and the version hash and status do not change. Another company sees nothing. - `cd ui && pnpm exec vitest run src/components/mcp-app/ src/components/ProviderToolIcon.test.tsx src/pages/apps/app-detail/`: 69 pass. These tests cover: - The host protocol for each new method: capabilities, consent wait, one proposal at a time, invalid input, ordered and deduped context, the size limit, and the display-mode rules. - The consent card (nothing sent without Send). - The full-screen dialog (one frame at a time). - Icon rendering (no-referrer `img`, no inline SVG, fallback). - `cd ui && pnpm typecheck` (including Storybook) and `tsc --noEmit` for the server: clean. - Red/green: the new route and host tests fail on `master`, where the routes do not exist and the host answers -32601 or drops updates. The icon descriptor test failed when the publication write was removed. - Not run locally: the full suite, and `packages/adapter-utils/src/server-utils.test.ts › runChildProcess › does not crash when the child closes stdin before a delayed write`. That test times out on this Mac with or without this change, so it is a local platform issue. CI covers both. ## Risks - Migration risk is low: three nullable columns, no backfill. - Every run on a task now does one extra indexed read, in parallel with the other preparation. Errors are logged and the run continues without app context. - If the attribution request fails after the comment is sent, the comment stays without the `mcp_app_message_sent` activity row. This is documented. - Switching between inline and full screen restarts the view, so state inside the widget resets. This is documented as a limit. - Icon names shortened with a hash by the gateway (very long names) do not match, and those rows keep the taxonomy icon. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and code execution. One subagent (the same model) built the provider-icon part. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
3b89461210 |
fix(evals): accept the case owner's direct handoff from chat in handoff-same-task (#21372)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Cases let one primary task move between agents through handoffs > - The chat eval `cases.handoff-same-task` asks the case owner, Morgan, in chat: "Hand this case to Riley for review with the findings you saved." > - The eval failed in every cohort. Morgan's chat run handed the case to Riley directly. The grader required a separate Morgan run on the primary task to do the handoff. > - The product allows the direct route. The assignee may hand off its own task from any of its runs while no other run owns the task. The case's expected outcome does not ask for a separate run. > - This pull request grades the run that recorded the handoff, by provenance, and accepts both the direct and the routed shape. > - The benefit is that the eval measures the feature (same task, new owner, saved findings, real review) and not one tool sequence. ## Linked Issues or Issue Description **What happened?** Paid run 43413553210 on master |
||
|
|
d9b42b50fd |
feat(presentation): let a person move a waiting continuation forward without starting a second run (#21373)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - When a run is interrupted, Paperclip records a fresh continuation and continues the task by itself. Since #21353 the task reads as continuing (`working`, no Retry) for as long as that continuation waits > - That is truthful, but a person has no control while it waits. On hosted staging one agent had 21 running and 209 queued runs, and continuations waited 30 to 110 minutes. Pacing (#21234, #21354) admits a waiting continuation into the agent's queue within minutes, so most of that wait is in the queue, behind other work > - #21353 removed Retry from this state because a Retry started its own run, which raced the continuation and could give the task two runs > - A Retry that observes the queued continuation run cannot start a run: admission sees a queued run and joins the Retry to it (`coalesced`, `runId` = that run). Under #21234's ranking, a queued run that a person joined starts ahead of system recovery and ordinary work > - This pull request keeps the continuing state and, once the queued continuation has waited 15 minutes, also offers Retry labelled "Move it forward now" > - The benefit is that a person can move a long-waiting task to the front of its agent's queue, and the click can never create a second run ## Linked Issues or Issue Description Refs #21353, #21348, #21234, #21354, #21264, #21320, #21200 **What happened?** On hosted staging, tasks with a recorded continuation waited 30 to 110 minutes behind a deep agent queue. The task showed "continuing" with no action. The person could only wait. **Expected behavior** The task still never shows "failed". After a long wait the person can move the waiting continuation forward. The click must not start a second run. ## Which states a Retry joins (established before building) I read #21348's fold (`foldWaitingRetriesIntoContinuation`) and its tests, and drove the real click path (`requestTaskRetry` followed by `resumeTaskRetry`, as the route does) in a probe against each record state: | Continuation state | What a Retry does today | Offer Retry? | |---|---|---| | Recorded, waiting for delivery (`continuationDelivery: "pending"`, no run yet) | The Retry is admitted at once as its own `task_retry` run, for every isolation (`fresh_isolated_workspace`, `writer_ended`, native). The delivery pass then marks the record delivered (superseded). #21348's fold never runs: it folds only a Retry that is still `deferred_issue_execution` when the continuation run is created, and only for `fresh_isolated_workspace`. Nothing keeps the Retry deferred, so the click races the delivery pass. | **No** (unchanged) | | Delivered, continuation run `queued` | `requestTaskRetry` sees the latest run is busy and records the Retry as `coalesced` into that run in the same transaction. No run is created; `resumeTaskRetry` acts only on deferred Retries. The run is now ranked as a person's (`joinedByUser` in `queue.ts`). | **Yes, after 15 minutes** | | Continuation run `running` | Same join, but the run already started, so there is nothing to move forward. | No | | Continuation ended | A Retry gets its own run (the existing failed-run path). | Existing behaviour | So the Retry offered here is joined by the existing busy-run admission in `requestTaskRetry`, not by #21348's fold. Its source run is the queued continuation itself (the run the presentation names as `observedRunId`), not the run the continuation was created for. ## What Changed - `server/src/services/task-interruption-presentation.ts`: for a task whose latest run is `queued`, created 15 minutes ago or more, and named as `continuationRunId` by a delivered fresh-continuation record, the projection returns `state: "working"`, `canRetry: true`, `retryAction: "move_forward"`, responsibility `system`, `observedRunId` = that queued run. It does not offer this when a person's wake already joined the run, when the run's agent is not the assignee, or when a person-owned wait applies (approval, reconnect, budget, pause and the others are checked first). Two small reads run only for queued runs older than 15 minutes. The existing restore-hold check that withdraws `canRetry` also withdraws `retryAction`. - `packages/shared/src/types/execution-projection.ts`: new optional `retryAction?: "move_forward"` on `TaskInterruptionPresentation`. It only changes the label; `canRetry` still decides whether Retry is offered. - `ui/src/components/ExecutionBlockerNotice.tsx`: a `working` presentation with `retryAction: "move_forward"` shows "Waiting for the agent to finish other work. Paperclip continues this task by itself." and a "Move it forward now" button that calls the same task Retry. Token-only classes; `pnpm check:token-gates` is clean. - One join-only action, `move_forward`, carried from the offer to admission (contracts kept in sync): - `packages/shared/src/validators/issue.ts`: `taskRetrySchema` accepts `retryAction: "move_forward"`. The route already passes the body to `requestTaskRetry`. - `server/src/services/task-retry.ts` (`admitTaskRetry`): right after the stale check, still under the task row lock and before any other Retry path, a `move_forward` Retry whose observed run is no longer `queued` or `running` returns `{ accepted: false, alreadyMovedOn: true, requestId: null }` and writes nothing. This block was cleared with the owner of Retry admission. The owner will later extend it so that "continuation recorded, not yet delivered" also counts as actionable, after their pending-delivery change lands (about two lines on their side). Until then this presentation never offers the button in that state, so the early return cannot misfire from our UI. - `ui/src/api/issues.ts` and `ui/src/hooks/useTaskRetry.ts`: the click sends `retryAction: "move_forward"` when the offer has it. `alreadyMovedOn` shows "The task already moved on, so nothing new was started. It now shows its current state." - What a click does at each timing: continuation `queued` → joined to that run (moves it forward); `running` → joined (nothing more to do); ended → "already moved on", nothing started. - A joined click cannot become a run later. The busy path writes it as `coalesced` with `runId` set to the continuation and `finishedAt` set (`task-retry.ts:456-461`). Promotion after a run ends (`heartbeat/run-control.ts:944` `releaseIssueExecutionAndPromote`; `modules/wake-queue/adapters/postgres.ts:89` `DEFERRED_WAKE_STATUS` and the selects near `:1358`), `resumeTaskRetry` (`run-control.ts:833-839`) and `resumeExecutionWaitComments` (`:876`) pick up only `deferred_issue_execution` wakes. - `ui/src/pages/Inbox.tsx`: the failed-run row does not show its Retry for this state. That row sends the failed run as the observed run, which is no longer the latest run, so its Retry would be refused as stale. - `doc/execution-semantics.md`: one paragraph on the offer, when it ends, and why there is none before delivery. ## Verification - Red (before the product change, new tests only), in `server/`: `pnpm exec vitest run src/__tests__/task-interruption-automatic-path.test.ts src/__tests__/recovery-wake-herd.test.ts`: 2 failed, 58 passed. Both failures are the missing offer (`canRetry: false`, no `retryAction`) for a continuation queued 15+ minutes. - Green after merging master |
||
|
|
8e41247b65 |
fix(evals): grade full cohort 11 cases on saved state and make the interaction DOM id unique (#21366)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The chat eval catalog checks real chat journeys against saved
product state, not reply wording
> - Full cohort 11 (OpenCode with DeepSeek v4.1 Flash, master
|
||
|
|
0bb16ace54 |
fix(evals): grade the company-secret binding confirmation from saved state (#21368)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The configure-company-secret eval checks that a person saves a
deployment key securely and that Morgan receives only a reference to it
> - After the person approves the binding, a separate agent run must
confirm the outcome in chat
> - The check required that confirmation to contain one of a few words
("bound", "available", "configured", "ready", "added")
> - A correct confirmation said the binding was "applied" and "exposes"
the variable, so a correct outcome failed
> - This pull request grades the confirmation from saved state: who
posted it, after what, and which binding exists
> - The benefit is that the eval checks the real outcome, and the
reply's wording is left to human review
## Linked Issues or Issue Description
Refs #21350
**What happened?**
`chat-features.runner-opencode.local.secrets.configure-company-secret`
failed only `human-attributed-and-confirmed-after-authorization`. The
run saved the key, the binding approval was accepted and executed, and
the binding to Morgan exists as `env.DEPLOYMENT_API_KEY`. I evaluated
each part of the check against the retained observation. The secret was
created by the person. The approval path was valid. The reply came from
the bound continuation run and carried the bound interaction. The
proposal-phase check passed. `/Morgan/`, `/deploy/` and the negation
guard passed. Only `/bound|available|configured|ready|added/` failed:
the reply said "the binding you approved is applied" and "exposes it as
the environment variable `DEPLOYMENT_API_KEY`".
**Expected behavior**
The confirmation passes when the bound post-authorization run posted it
after the intended binding existed. Wording is not graded by keywords.
**Steps to reproduce**
Grade the retained confirmation with the old check. It fails although
every structural condition holds.
## What Changed
- In `external-company-secret.ts`,
`human-attributed-and-confirmed-after-authorization` no longer checks
the reply text. It requires:
- the secret was created by the current person;
- the existing authorization proof (an executed approval or native
binding authorization, or the human-attributed binding);
- a later reply whose saved comment is in this company and source
conversation, authored by the lead agent and created by that reply's
run. When a native binding is required, the run must be the bound
post-authorization run (the approval's continuation run, or the bound
continuation). When a question or confirmation card started the flow,
the run must carry that interaction;
- one binding to Morgan for the saved secret, naming the intended
variable (`env.DEPLOYMENT_API_KEY` by default, or the person's requested
name on that existing path);
- the reply comment was created at or after that binding's `createdAt`.
- The observation now always collects the source conversation's saved
comments (before, it did so only on the follow-up and card paths), so
every path can bind the reply to its run and time.
- The truthfulness pattern for the proposal phase and the other wording
patterns are dropped from this check. The credential-in-chat guard
elsewhere in the case is unchanged.
- No scenario YAML changed, so the TestOps manifest needs no entry.
## Verification
- New test built from the retained confirmation text. It passes with the
new grade and fails with the old one, on this exact check. It also fails
when the reply is saved before the binding.
- New rejections in the grade tests: no saved reply comment; a reply
saved before the binding; a reply from the initial run; a reply authored
by another agent; a reply in another conversation; a binding naming
another variable; a binding without a creation time.
- Fixtures now carry the saved reply comment and binding time. Company
and secret tests, plus catalog consistency and contract tests: 197
passed. `pnpm test:e2e:runner:typecheck` passes.
- A narrow paid run of the case on this branch is linked in a comment.
## Risks
- Low risk. The change is in eval code only.
- The confirmation's wording is no longer checked by this grade, by
design. A reply that misstates the outcome would need human review.
- One extra comment read during observation on the direct path.
## Model Used
- Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, used through
Claude Code with tool use and code execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
386dbe76a1 |
fix(evals): grade the company-secret setup reply by its rendered link (#21350)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The chat catalog evals check that a chat request reaches the correct saved outcome > - The configure-company-secret case checks that chat sends the person to the organization secret entry and never asks for the key in chat > - The case found the settings route by splitting the reply's markdown into tokens, and it judged the reply's wording with regular expressions > - The same tokenizer failed the personal-secret case: a link followed by a colon lost its route (fixed in #21344) > - This pull request grades the company setup reply the same way: it finds the rendered link, clicks it, and requires the New secret entry to appear > - The benefit is that the case measures what a person can actually do, and the reply's wording is left to human review ## Linked Issues or Issue Description Refs #21344 **What happened?** `companySecretSetupURL` read routes from the reply's markdown with a regular expression. It removed only `.,;!?])}` from the end of each token, so a reply such as `Open [Secrets](/COM/company/settings/secrets): choose New secret` yielded the route `/COM/company/settings/secrets):` and failed, although the rendered link was correct. The same bug failed the personal-secret case. The reply was also gated on wording patterns. **Expected behavior** A reply passes when its rendered message has a visible link to this company's secrets settings, and a person who clicks it reaches the organization secret entry. **Steps to reproduce** Grade a company setup reply whose markdown link to the secrets settings is followed by `:`. The old `companySecretSetupURL` returns `undefined`, although `deploymentSecretGuidance` accepts the same reply. ## What Changed - Add `external-secret-setup-link.ts`: the shared rendered-link check. It finds a visible link in the reply's rendered message to this company's `/company/settings/secrets`. It clicks the link, requires the case's entry control to be visible, and then returns to the conversation. The personal-secret check (`external-personal-secret-link.ts`) now uses it, with no change in behavior. - Add `external-company-secret-link.ts`. The company entry is **New secret** on the company **Secrets** tab, which is selected if needed. The company link takes no query. - `external-company-secret.ts`: - The comment handoff no longer checks the reply against wording patterns. - The setup link is followed only when no question or confirmation card carries the handoff. - The proof is recorded as `company-secret-reply-link.json`, and the `honest-secure-human-handoff` grade uses it. The proof must belong to a reply of the initial run. - Card paths are unchanged. - No scenario YAML changed, so the TestOps manifest needs no entry. `chat-catalog-descriptors.json` lists the new source files. ## Verification - `external-company-secret-link.test.ts`: - An ungated unit test covers the link-target rules. - A Chromium test (gated by `PAPERCLIP_CHAT_EVAL_SECRET_ROUTE_CANARY=1`, like the other secret-route browser tests, and run locally) shows that these cases pass: a link followed by a colon; a company tab selected after the click; tabs that render after navigation. - These cases fail: no link (wording only); a plain-text path; wrong-company and foreign links; a link with a query; a hidden link; a page with no secret entry. - Red check in the same test: the old `companySecretSetupURL` returns `undefined` for the colon-terminated link, while its wording guard accepts the reply. - The company grade tests now require the link proof. They reject a missing proof, a proof bound to another comment, and a proof that did not reach the entry. - Personal and company secret tests, catalog consistency and contract tests: 350 passed. `pnpm test:e2e:runner:typecheck` passes. ## Risks - Low risk. The change is in eval code only. - A company setup reply without a clickable settings link (only menu directions) no longer passes the comment path. Question and confirmation card handoffs are unchanged. ## Model Used - Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, used through Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
2b69cdfcd6 |
fix(evals): grade the personal-secret setup reply by its rendered link (#21344)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The chat catalog evals check that a chat request reaches the correct saved outcome > - The personal-secret case checks that chat sends the person to their own secure value entry, and never asks for the value in chat > - The case judged the reply's wording with regular expressions, and it found the settings route by splitting the reply's markdown into tokens > - A valid reply failed because its link was followed by a colon, so the tokenizer read the route as `/COM/company/settings/secrets):` > - This pull request grades the rendered link instead: it finds the visible link, clicks it, and requires the person's own-value entry to appear > - The benefit is that the case measures what a person can actually do, and the reply's wording is left to human review ## Linked Issues or Issue Description **What happened?** `chat-features.runner-opencode.local.secrets.personal-secret` failed with "Actual reply must direct the supported own-value settings path for this definition". The retained screenshot and DOM snapshot show a correct reply. The turn rendered as two messages, and both linked **My secrets** to `/COM/company/settings/secrets` and said **Set my value**. The wording check accepted the reply. The link check returned nothing. It extracted routes from the reply's markdown with a regular expression and removed only `.,;!?])}` from the end of each token. In the final message the link was followed by a colon (`[My secrets](/COM/company/settings/secrets): choose`), so the token was `/COM/company/settings/secrets):`, which does not match the settings path. An earlier passing run had `,` after the link, which the check removed. **Expected behavior** A reply passes when its rendered message has a visible link that a person can click to reach their own value entry for the required definition. Wording is not graded by keywords. **Steps to reproduce** Grade a setup reply whose markdown link to the secrets settings is followed by `:`. The old `personalSecretSetupURL` returns `undefined` for the retained reply, although its wording check returns `true`. ## What Changed - Add `tests/runner-e2e/chat-catalog-domains/external-personal-secret-link.ts`. It finds a visible link in the reply's rendered message (`#comment-<id>`) whose target is this company's `/company/settings/secrets`. An optional `definition` query must name the required definition. Other origins, companies, paths, credentials, fragments and query keys are refused. It clicks the link and requires the person's own-value entry to be visible. A link that names the definition opens that definition's detail sheet, so the check requires **Set my value** in the sheet, which must show `PERSONAL_PUBLISHING_KEY`. Otherwise it selects **My secrets** if that tab is not active and requires a visible **Set value** button on the row bound to the required definition (`my-secret-<definitionId>`). It then returns to the conversation and records the proof as `personal-secret-reply-link.json`. - In `external-personal-secret.ts`, the setup reply is no longer checked against wording patterns. It must still be the agent's own comment from the bound run. The grade uses the recorded link proof, which must belong to a reply of the initial run (a turn can post several messages). The reply link is now followed before the human-entry phase. - The confirmation and question paths are unchanged. When a question or confirmation card already carries the handoff, the reply link is not checked, so an optional link cannot fail those paths. - No scenario YAML changed, so the TestOps manifest needs no entry. `chat-catalog-descriptors.json` lists the new source file for this case. ## Verification - `external-personal-secret-link.test.ts`: - An ungated unit test covers the link-target rules. - A Chromium test (gated like the other secret-route browser tests by `PAPERCLIP_CHAT_EVAL_SECRET_ROUTE_CANARY=1`, run locally) uses the retained two-message reply. These cases pass: the retained reply; a link carrying `?definition=` that opens the definition's detail sheet; a page where the **My secrets** tab is selected after the click. - These cases fail: no link (wording only); a plain-text path; a wrong-company link; a foreign link; a wrong-page link; a hidden link; a link that reaches another definition's row; a definition link whose sheet shows another key; a link that reaches no own-value entry. - Red check: the old `personalSecretSetupURL` returns `undefined` for the retained reply, while `personalSecretGuidance` returns `true`. - The personal-secret test files, catalog consistency and contract tests pass (210 tests). `pnpm test:e2e:runner:typecheck` passes. - A narrow paid run of the case on this branch is linked in a comment. ## Risks - Low risk. The change is in eval code only. - A reply that points to the settings page without a clickable link (for example, text menu directions only) no longer passes this path. That matches the stated contract: the person reaches their entry through a link in the reply. Question and confirmation cards are unchanged. - The company-secret case uses the same markdown tokenizer and could hit the same trailing-colon problem. It is not changed here. ## Model Used - Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, used through Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
70be453c38 |
test(e2e): accept either order when a person's message and the fallback settle a legacy hold (#21275)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - When a run is cut off, either a person's message or the fresh-continuation fallback's disposition worker can settle the task's recovery hold, and the task continues. > - The e2e spec `legacy-failure-continuation` "message" case asserted an internal field that depends on which of the two settles the hold first. > - When the fallback's worker wins, it stamps the cut-off run as superseded. The spec then fails, even though the person sees the same outcome. > - This pull request makes the spec assert what a person sees, not which path won. > - The benefit is that the spec stops flaking for every PR and in the release source check. ## Linked Issues or Issue Description Refs #21135, #21250 **What happened?** `tests/e2e/legacy-failure-continuation.spec.ts` › "legacy startup hold: message reaches a new agent response" failed on #21250 at |
||
|
|
d16becac04 |
test(evals): fix cohort-10 secrets entry, steering graders, private-task settle and queue brief (#21296)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The chat eval catalog checks real chat journeys against saved
product state, not reply wording
> - Full cohort 10 failed six cases where the product worked but the
eval was out of date or raced
> - #21225 redesigned Settings → Secrets, and #21243 changed when a
queued comment counts as delivered
> - The private secret steps, two steering graders, one final run check
and one fixture did not follow
> - This pull request updates those eval layers and keeps each one able
to fail a broken product
> - The benefit is that these six cases measure the product again
instead of the stale harness
## Linked Issues or Issue Description
Refs #21225
Refs #21243
**What happened?**
In full cohort 10 (OpenCode, local), six cases failed for eval reasons:
- `secrets.configure-company-secret` and `secrets.personal-secret`: the
private browser step waited for the old "Create secret" dialog and its
Provider field, and for the old personal form text. #21225 renamed the
dialog to "New secret", removed Provider, moved Key under "More
options", and moved the definition name into the "Set your value for …"
heading.
- `steering.steer-active` and `steering.discard-queued-message`:
Morgan's running turn read the queued comments through task history and
cited them in its accepted result. Under #21243 that is delivery, so no
continuation ran. The graders still required one more run and timed out.
- `tasks.private-task`: the final run check read the ledger while the
CEO's `chat_task_completed` follow-up was still running.
- `steering.queue-follow-up`: the brief was only a reference string, so
the continuation asked a question instead of building.
**Expected behavior**
Each case grades the product's current contract on saved state, and
still fails a broken product.
**Steps to reproduce**
Run the six cases at `origin/master` with the OpenCode local profile.
See the cohort-10 evidence for each case.
## What Changed
- `external-private-browser-worker.ts`: the company step opens "New
secret", finds the dialog by its "Who provides the value" radiogroup,
selects Organization, opens "More options" for a requested key, and
picks Paperclip-encrypted storage only when the "Stored in" picker is
shown. The saved secret must now have provider `local_encrypted`. The
personal step binds the form by the heading "Set your value for Personal
publishing credential". Credential handling is unchanged.
`manual-private-*.ts` is unchanged.
- `external-company-secret-create-dialog.test.ts`: the canary page uses
the new dialog. It adds a several-vaults variant (valid), a
wrong-storage result and the retired "Create secret" form (both
rejected). Source anchors follow `Secrets.tsx`.
- `external-personal-secret-entry-dialog.test.ts`: checks that the
worker heading matches what `SetMyUserSecretDialog.tsx` renders, and
rejects another definition and the "Replace your value" form.
- New `runtime-cited-delivery.ts`: `owedAfterTurn` (queued comments the
turn did not read at the current revision and cite) and
`continuationCarriesOwed` (one continuation carries each owed comment
once, with its text and the earlier-read warning, and none that the turn
handled).
- `runtime-active-steering.ts` (oracle `steer-active.v9`): records the
queued comments (person and any CEO relay) and the worker's receipts.
Delivery passes if every queued comment was handled in turn with no
continuation, or one continuation carries exactly the owed ones. Run
count is 2 plus one when anything is owed. The website the worker
registered last must still render blue.
- `runtime-discard.ts` (oracle `runtime-discard.v7`, `minimumRunCount:
4`): the same rule for the font note and the later contact request. The
run that handled the contact request must save its labelled section. The
discarded request must never be cited. The gate no longer says "Leave
saved follow-ups for their ordinary continuation", because that
contradicts the #21243 contract.
- `internal-private-delegation.ts` (oracle `private-task.v3`):
`verifyResult` waits for the delegated work and its completion follow-up
to settle, in the pattern of `identity-plan-review.ts`. A failed or
cancelled run still stops the wait. A worker still running at the
deadline stays bounded active work.
- `runtime-queue.ts` (oracle `runtime-queue.v4`): the gate gives a
complete website brief (product, page, hero copy, features, pricing,
company details). It does not contain the word "contact", so the brief
does not complete the queued requirement.
- Oracle bumps for the secrets cases:
`company-secret-controller-baseline.v22`, `personal-secret.v21`.
`chat-catalog-descriptors.json` follows all bumps and source files. No
scenario YAML changed, so the TestOps manifest is unchanged.
## Why each new check still catches a broken product
- Secrets: the old check matched removed UI text, so it failed a working
product. The new check uses roles and labels, and still fails a missing
form, a failed POST, wrong saved metadata and storage outside Paperclip.
- Steering: the old check required a continuation, which #21243 removes
when the turn cites the comment. The new check still fails an uncited
read with no continuation (the lost-comment bug), a citation without a
current-revision read, a cited comment that was not applied (no blue
page or no saved label), a replay of a handled comment, and any extra
run.
- Private task: the old check read the ledger at a random point after
acceptance. The case still fails if the follow-up fails or is cancelled.
- Queue brief: the grader is unchanged. The brief test fails if the
brief leaks the contact requirement.
## Verification
- `npx vitest run --config tests/runner-e2e/vitest.config.ts
tests/runner-e2e/chat-catalog tests/runner-e2e/chat-catalog-domains`:
285 files, 3948 tests pass.
- Calibration uses the retained cohort-10 receipts. The steer-active
worker read and cited both queued comments. The discard worker read and
cited the font note and the later contact request. Both now pass.
Reworded-valid variants (target handled in turn, relay or font note owed
to one continuation) pass. The broken variants listed above fail.
- `PAPERCLIP_CHAT_EVAL_SECRET_ROUTE_CANARY=1` secrets canaries: 3 of 3
pass. RED: the old worker against the new dialog page fails
(`locator.fill` timeout).
- `pnpm test:e2e:runner:typecheck`: pass. `node --test
tests/runner-e2e/chat-full-catalog-testops-manifest.test.mjs`: 10 of 10.
`node tests/runner-e2e/source-sdk-public-inputs.mjs
|
||
|
|
d1d71a7755 |
test(restart-chaos): staging twin cells, follow-up/queue/subtask grading, and full Daytona cleanup (#21325)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The restart-chaos harness (`tests/restart-chaos/`) proves that a release swap (two servers that overlap, then the old one stops) does not break running agent work > - The staging twin runs that swap with two local servers and real Daytona sandboxes, and it is the gate before any hosted proof > - A loaded twin wave (48 tasks, 20 runs plus 20 queued, 2 swaps) reported 31 per-task failures and 2 extra turns, but the product did the correct thing in almost all of them > - Three harness defects made the noise: the graders called a required follow-up run an extra turn, called a healthy queued run an orphan, and called a parent that waited for its subtasks a stall. Cleanup also left archived Daytona sandboxes in the shared org on every wave > - This pull request lands the twin harness work and fixes the three grader defects and the cleanup gap, each with a red/green harness test > - The benefit is a stop-rule grade that shows only real failures (here: 1 Retry banner) and makes the follow-up wait visible, and a wave that leaves no sandbox behind ## Linked Issues or Issue Description Refs #21175 and #21227 (the server's comment and child-result delivery rules; this harness takes their outcome from the server's wake record and does not re-check them). **What happened?** Wave `w-20261113T052703-s7301` (loaded twin L1, on Daytona) graded 31 tasks FAIL and listed 2 extra turns. The release-swap grade was 82 / 82 tasks completed without human action. - t007 and t010 (OpenCode): a person posted a follow-up comment during the first run (05:44:56 and 05:49:21). OpenCode takes no input mid-turn. The first run finished `done` at 05:53 and said "No board comments arrived". The deferred wake was promoted, and the second run answered the comment after 16 to 17 minutes in the 20-deep queue. The graders counted the second run as an extra turn and as a `completion_once` duplicate. - t021 to t048: each waited about 483 s queued for a slot in the deliberate 20-deep queue. U1 allows 5 minutes queued plus 3 minutes without an owner, so all 28 failed `no_owner_no_action`. - t001: a parent ended its turn `yielded` to wait for its two subtasks. The observer saw no run for 3 minutes, called it a stall, and rested it `in_progress`. The parent completed later through its `issue_children_completed` wake. R1, R3, R5 and U1 failed. - Cleanup: every Daytona wave left 1 to 8 archived `paperclip-create-*` setup sandboxes and reported `remainingSandboxes: []`. Daytona does not delete archived sandboxes. **Expected behavior** - A run that starts with a person's comment that no earlier run received is required. Only a run that starts with no undelivered input is an extra turn. - A run queued while every slot of its agent is taken is healthy for the declared queue time. A run queued while its agent has a free slot still fails. - A parent that waits for open subtasks is in a named wait, not a stall. - Cleanup deletes the sandboxes that this wave created and fails when one remains. **Steps to reproduce** 1. Run the L1 loaded twin (command in the README section "Staging twin cells"). 2. Read `staging-mix-summary.md` and `SUMMARY.md`. ## What Changed - Cherry-picked the 8 twin harness commits from `fastloop/staging-twin-loaded`: staging-mix under the release-swap driver on Daytona, `--server-from` on Daytona, probe agents verified while the first tasks run, image matched to the `--server-from` checkout, and the knobs `RESTART_CHAOS_K15_KILL_OLD_AFTER_SECONDS` / `_KILL_OLD_SWAPS`, `RESTART_CHAOS_K15_FREEZE_NEW_SECONDS` / `_FREEZE_SWAPS`, `RESTART_CHAOS_MIX_BANNER_REST_SECONDS` and `RESTART_CHAOS_MIX_FIRST_SLOT_SECONDS`. A misplaced doc block in `supervisor.ts` is moved back to its function. No code comments are added. - Cleanup (`daytona-sweep.ts`): after the environment delete, the harness deletes every sandbox that carries both of this wave's UUID labels (company and environment), archived ones included. `assertHarnessMayMutate` still refuses any sandbox without both labels and any `fleet-sbx-*` snapshot. The `paperclip-create-*` name refusal is removed, because every plugin sandbox (the fleet's included) has that name, so the labels are the real ownership proof. A sandbox left after deletion fails the wave (`undeletedSandboxes`, warm-shutdown verdict, `ttft-baseline`). - Follow-ups (`follow-up-delivery.ts`, `wake-evidence.ts`, `staging-mix-retry.ts`, `staging-mix.ts`): delivery comes from the server's own record. A comment that the server judged delivered never becomes a run. A run that the server started from a wake carrying a person's comment posted after the first run started (a retry included) is required, not an extra turn. `completion_once` keeps the first completion as the baseline and does not count a later completion by such a run (truth-table test). The grader does not re-check the server's delivery rule; the server's tests cover it. The summary adds `extra turns: N` and `follow-up answered in a later run: n, queued wait p50/max, posted to answered p50/max`. - Declared queue (`universal.ts`, `agent-slots.ts`, `wave-grade-inputs.ts`): with `--mix-queue` above 0, U1 allows ceil(queue / runs) × (longest turn + 5 min) queued (27 minutes for L1). Only queued time while the run's agent has a free slot counts against the 5-minute bound. Slot use comes from a new `wake-evidence/agent-slots.json` (run start/finish and each agent's `maxConcurrentRuns`). - Subtask wait (`child-wait.ts`, `observer.ts`, `grader.ts`): an idle parent after a `yielded` or `blocked` turn, until its `issue_children_completed` wake, is a named wait (`waitingOnChildren`) for R5 and U1. The observer no longer gives up on a parent while a subtask is open. An older record that rested during that wait takes the completion from the wake evidence. - Review follow-ups: the first staging-mix slot moves earlier without adding a swap; each sandbox delete pass has a deadline; a frozen incoming server is always resumed (`server-freeze.ts`); the first completion is always the baseline for `completion_once` (truth-table test); a follow-up run that failed before its turn keeps the answer owed for its retry; `agent-slots.json` saves the server's effective run cap (`maxConcurrentRunsSql`); the wave-end cleanup flow moved to `daytona-cleanup.ts` (`main.ts` stays under 1,000 lines). - README: one-command entries for the twin cell (G, G2, G3) and L1, the knobs, the stop-rule grading, the new summary lines, the U1 queue rule, the subtask wait and the cleanup rule. ## Verification - Red/green tests (red = new tests against the previous source): - Cleanup, `daytona-sweep.test.ts`: red 4 failed (for example "expected [] to deeply equal [ 'archived', 'archiving', 'started' ]": nothing was deleted). Green: pass. - Follow-ups, declared queue, subtask wait (`staging-mix-retry.test.ts`, `universal.test.ts`, `child-wait.test.ts`, `wake-evidence.test.ts`): red 6 failed (for example the t010 run listed as an extra turn, U1 fail for the t030 queue shape, t001 "rested_not_done"). Green: pass. The tests use the real timings of t001, t010 and t030. - Full harness suite: `npx vitest run --config tests/restart-chaos/vitest.config.ts`: 71 files, 811 tests pass. `npx tsc -p tests/restart-chaos/tsconfig.json --noEmit`: clean. - `pnpm verify` exits 2 (the harness README is a global CI input). Required CI supplies the broad coverage. - One-off offline analysis (a scratch script, not part of this PR): the saved wave `w-20261113T052703-s7301` regraded with these graders, from a copy of its results: | | Before | After | | --- | --- | --- | | Per-task FAIL | 31 (t001 R1, t007 and t010 `completion_once`, t021 to t048 U1) | 0 | | Extra turns | 2 (t007, t010) | 0 | | Retry banner shown | 1 (t020, 3 s, completed without a click) | 1 (t020) | | Follow-up answered in a later run | not shown | 4: t001, t007, t010, t012. Queued wait p50 983 s, max 1739 s. Posted to answered p50 1420 s, max 2255 s | t001 and t012 started their follow-up run before completion, so they were never extra turns. They are listed because the wait is real. This wave predates `agent-slots.json`, so the regrade used only the declared queue bound for U1. - No Daytona wave ran for this PR. There is no live proof here that the cleanup deletes archived sandboxes in the real org or that the new evidence file is written on a live wave. The delete path uses the same `DELETE /sandbox/<id>` call as the Daytona SDK's `deleteSandbox`. ## Risks - The harness now deletes Daytona sandboxes in an org that the Cloud fleet shares. The guard is the wave's two freshly minted UUID labels, filtered on the server and checked again on every row, plus the `fleet-sbx-*` refusal. A sandbox without both labels is never deleted. - When the server wrongly re-delivers a comment that a finished run already received, the harness counts that run as a required follow-up, not as an extra turn. The server's own delivery tests catch that class, not this grader. `completion_once` still fails when that run also reports completion and is not the first. - The declared queue bound is generous (27 minutes for L1). A stuck queue with full slots for longer than that still fails. A queue with a free slot fails after 5 minutes when slot evidence exists. Without slot evidence only the declared bound applies. - Test harness only. No product code changes. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
22c9300dac |
fix(recovery): a run that fails before the agent is asked anything continues by itself on a fresh computer (#21297)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - A native run on a remote sandbox opens a provider session and then
waits at a launch gate before the first prompt goes to the provider
> - On remote OpenCode sandboxes the gate saves the provider launch
files; when that save fails, no prompt was ever sent
> - Today that failure ends as `native_session_interrupted` with
ambiguous recovery evidence, so the task goes blocked and the user must
press Retry
> - Nothing was sent to the provider, so nothing can be duplicated: the
system can safely continue by itself
> - This pull request types that failure, proves it from the Runner turn
ledger at the failure point, and continues the task on a fresh sandbox
in the existing interrupted-preparation lane
> - The benefit is that a setup failure before the first prompt no
longer needs a human, and the user sees plain words instead of internal
codes
## Linked Issues or Issue Description
Refs #21287
Refs #21304
**What happened?**
On a remote OpenCode sandbox, the provider launch-files gate failed
before the first prompt (for example
`native_provider_artifacts_transfer_timeout`). The run ended blocked
with a Retry banner. The run error showed internal codes and raw byte
counts to the user.
**Expected behavior**
A failure that happens before any prompt reaches the provider continues
automatically on a fresh sandbox. The task shows as working while the
continuation is pending. The visible text is plain and has no internal
codes or raw numbers.
**Steps to reproduce**
1. Use an OpenCode agent on a Daytona sandbox environment with
controller persistence.
2. Arm the crash point `native.provider_artifacts.before_capture` in
fail mode (or let the launch-file upload time out).
3. Start a task. The first run fails at the gate before any prompt.
4. On master the task goes blocked with Retry. With this change a
successor starts on a new sandbox and the task completes.
**Agent adapter(s) involved**
OpenCode (native runtime, remote sandbox).
## What Changed
- New typed cause `native_failed_before_provider_turn`
(`server/src/services/native-runtime/failed-before-provider-turn.ts`).
The executor raises it only when the launch gate fails, the run was not
cancelled, and the Runner turn ledger proves that no turn command was
issued. The proof is taken at the failure point, before the session
closes. In all other cases the original error is rethrown and today's
path stays.
- The executor quarantines the failed run's session root, so the
successor starts in a new root.
- `heartbeat/predecessor-retirement-fallback.ts` accepts the new cause:
it records the proof on `runnerProfileJson.failedBeforeProviderTurn`,
marks the run's reusable sandbox leases for replacement, and schedules
the successor in the existing `interrupted_preparation` lane (same cap
of 12 and the same backoff).
- `remote-workspace-unchanged-proof.ts` accepts the runner process,
runner command and pre-turn events only when that proof record is
present. It still refuses any turn evidence, an unsettled save by
another run on the same workspace, and setup writes. When the proof
refuses, the run's epoch is abandoned unapplied: the files are kept and
recorded, and the next run restores the last committed save into a fresh
root.
- When the proof refuses and the epoch is abandoned, the replacement
mark stays. The next acquisition replaces the sandbox. #21304's
retention hold makes `retireReplacedReusableSandboxLease` park the old
lease as `retained` instead of `pending_cleanup`, so the task continues
on a fresh sandbox and the sandbox that holds the abandoned files is
kept. When the proof holds (only the Runner's own session folder was
written), the old sandbox is replaced and cleaned up as usual.
- `task-interruption-presentation.ts` shows a pending
`interrupted_preparation` retry as `working` with no Retry until it is
due (plus the dispatch grace).
- The visible run error is now plain: "Setting up the computer took too
long, so the task is continuing on a fresh one." (for a setup timeout)
or "The computer could not be set up before the agent started, so the
task is continuing on a fresh one." The technical cause code stays in
`runnerProfileJson.failedBeforeProviderTurn.technicalCause`, the server
log and the run log.
- New crash point `native.provider_artifacts.before_capture` (fail mode
throws the typed transfer timeout at the gate).
`tests/runner-e2e/source-sdk-public-inputs.json` is refreshed for the
changed registry file.
- `tests/restart-chaos/main.ts`: `RESTART_CHAOS_EXTERNAL_CRASH_CONTROL`
lets an outside driver arm fail-mode crash points without a restart
fault.
## Verification
Local (targeted):
- `server/src/__tests__/heartbeat-native-predispatch-retry.test.ts`: 8/8
pass. New cases: fresh remote sandbox and fresh plugin-task sandbox
continuation (now also asserts the plain visible text with no
identifier-looking tokens, and the technical cause in the proof record),
the ambiguous hold when the ledger cannot prove no turn, and backoff
plus the lane cap.
- `server/src/services/remote-workspace-unchanged-release.test.ts`:
27/27 pass. New cases: hand-on when the ledger proves no turn; refusal
on a recorded turn, an unsettled predecessor save, and setup writes. Two
more cases follow the marked lease through the next acquisition's
replacement decision (`readReplaceBeforeReuseMarker` +
`reusableSandboxReplacementPlan` + `retireReplacedReusableSandboxLease`)
and then through `sweepPendingCleanupLeases`. "replaces the marked
sandbox when the failed run provably left its files unchanged": the old
lease goes to `pending_cleanup` and the sweep destroys it. "continues on
a fresh sandbox and parks the one that holds abandoned files, which
cleanup leaves alone": the old lease is `retained` with the
abandoned-files reason and is held by the receipt, the sweep does not
destroy it, and the file is still there.
- Red without the product change (product files reset to the base, tests
kept): 8 failed, 25 passed of 33. All 8 failures are the new cases. The
visible-text assertion alone was red on the previous commit: the error
read "The computer could not be prepared before the agent started
(native_provider_artifacts_transfer_timeout: the upload of 536870912
bytes did not finish within 120 s). The task continues on a fresh
computer."
- `task-interruption-presentation.test.ts`,
`native-provider-artifact-gate.test.ts`,
`crash-point-call-sites.test.ts`: 79/79 pass.
`packages/shared/src/crash-points.test.ts`: 16/16.
`tests/restart-chaos/crash-points.test.ts`: 12/12.
- `tsc --noEmit` clean for `server`, `packages/shared` and
`tests/restart-chaos`.
Daytona (local servers, Daytona sandboxes, image `sha256:787c3e9c…`,
these commits rebased onto
|
||
|
|
41998d551c |
feat(plugins): send the claimant's credential on plugin task resume (#21335)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Plugin environment drivers run agent tasks on remote computers, and a warm follow-up run resumes a parked Runner through the `resume` task operation > - `resume` carries only the durable identity and a bootstrap ticket, which is enough while the parked Runner process is still alive > - A provider sometimes has to start the Runner again to resume the session: after it moved the parked session to a new sandbox, or after the computer rebooted > - A Claude Runner cannot start without its credential, and today only the portable handoff path (`handoffCredential`) delivers one > - This pull request lets a provider opt in to receiving the claimant run's harness credential on `resume` > - The benefit is that a provider can continue a parked session on a new computer without Core orchestrating a handoff ## Linked Issues or Issue Description Refs #21289 Related: #21305 **What existing behavior does this improve?** Warm resume for plugin task providers that restart the Runner to resume a session. **Subsystem affected** Plugin SDK environment task schema, the server's plugin Runner task owner, and the plugin environment task docs. **Current behavior** `submit` can carry the harness credential (`credential`), but `resume` cannot. A provider that must start the Runner again on resume has no credential for a Claude task. **Proposed behavior** A provider that returns `resumeCredentialVersion: 1` from `describe` receives `credential` on `resume` whenever Core would send it on `submit`, resolved for the claimant run. Other providers see no change. **Reason and benefit** Providers can restart a Claude Runner on resume without the Core-run handoff protocol. **Breaking changes** None. The field is opt-in: the SDK worker parses operations strictly, so Core sends it only to providers that advertise the capability. ## What Changed - `resume` gains an optional `credential`, with the same schema and validation as `submit`'s. - `describe` gains an optional `resumeCredentialVersion: 1` capability. - The plugin Runner task owner adds the credential on `resume` when the provider advertised the capability and Core resolved a credential, using the same resolution as `submit` (`preparePluginRunnerCredentials`). After a warm rebind, the claimant run's credential is sent. - `doc/plugins/PLUGIN_ENVIRONMENT_TASKS.md` documents the field and its handling rules. - The runner-e2e SDK public-input pin is updated. `handoffCredential` on the session handoff path is unchanged. ## Verification - SDK vitest `tests/environment-task-release-resume.test.ts` and `tests/environment-tasks.test.ts` (68 passed): `resume` accepts and rejects the same 11 credential cases as `submit`; the capability is optional and rejects other versions; a resume credential reaches the plugin through the serialized worker RPC. - Server vitest `plugin-runner-task.test.ts` and `environment-task-runtime.test.ts` (151 passed): credential sent with the capability and a resolved credential; omitted without a credential or without the capability; the rebound claimant's credential is sent; the credential appears in no metadata write and not in the sanitized error, even when the worker error echoes it. - `native-session-executor.test.ts -t "uses a task-only"` (5 passed): the executor forwards the capability to the owner. - runner-e2e `source-sdk-evidence.test.ts` and `source-sdk-public-type.test.ts` (46 passed). - `tsc --noEmit` for `packages/plugins/sdk` and `server`. - Not run locally: the full suite and `pnpm verify`; required CI covers them. ## Risks - No server-side provider check on resume, unlike `submit`: the lease has no reliable harness at resume time. Protection relies on the owner, whose rebind refuses a changed harness, account or credential identity. - The credential is the claimant run's current one, which can be newer than the one that started the parked Runner. That is intended. - A provider must advertise the capability only after its SDK pin includes this schema; roll Core out first. - Rollback: revert this pull request. No migration. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, test runs), implemented by a sub-agent and reviewed before opening. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
2483cb46c2 |
feat(plugins): deliver configured environment to plugin Runner tasks (#21280)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - An agent gets environment variables from its execution environment, its own configuration, its project, and its routine. Secrets are resolved for each run. > - A plugin environment can run the agent as a native Runner task on its own host, through the environment task contract. > - That contract has no field for the agent's variables. Runs on a plugin environment get only the variables that the plugin sets itself. > - This pull request adds a versioned contract field for the values, sends them on a session handoff, lets Codex commands inherit the variables in an external sandbox, and lets a plugin protect the variables that it sets itself. > - The benefit is that plain and secret variables work on plugin environments in the same way as on local environments. ## Linked Issues or Issue Description Refs #21233. That pull request by @lucktastic finds the same gap and proposes the same capability and field names (`configuredEnvironmentVersion: 1`, `configuredEnvironment`). This pull request uses the same names, so providers implement one contract. It differs in these places: - Codex commands get the variables through an external-sandbox denylist, not through key names sent at `prepare`. A retained Runner does not need a new preparation when a key is added or removed. - A session handoff `resume` gets the values, because it starts a new Runner on another computer. Value changes on the same computer need no new field: Core's process reuse key includes the values, so Core starts a new task. - A driver can declare `protectedEnvironmentKeys`. Paperclip rejects or warns when one is saved. - The run log names skipped variables only for an older provider. Thank you to @lucktastic for the diagnosis and the contract shape. **What happened?** An agent has environment variables, plain and secret. Its runs on a plugin task environment do not get them. `PAPERCLIP_CONFIGURED_ENV_KEYS` in the run lists only the variables that the plugin sets. **Expected behavior** Each run gets the agent's configured variables. **Steps to reproduce** 1. Set a plain and a secret environment variable on an agent. 2. Assign the agent to a plugin environment that provides Runner tasks. 3. Start a run. Read the names in `PAPERCLIP_CONFIGURED_ENV_KEYS` in a command that the agent runs. 4. The agent's names are not in the list. ## What Changed - **SDK contract** (`packages/plugins/sdk/src/environment-tasks.ts`): - A provider advertises `configuredEnvironmentVersion: 1` on `describe`. - `submit` gets `configuredEnvironment`. `resume` gets it only for a session handoff. - Names, count, and sizes have the same limits as the Runner projection. - **Server**: - `createRunnerdBackendWithinSessionClaim` gives the run's configured projection to the plugin task owner. The projection has no provider credentials and no generated runtime keys. - `createPluginRunnerTaskOwner` sends the fields only to a provider with the capability. An older provider gets no new field, and the run log names the variables that were not set. The log never prints values. - **Codex** (`codex-security-config.ts`): - An externally sandboxed launch uses `inherit="all"`, `ignore_default_excludes=true`, and an `exclude` list of Codex's own credentials, instead of an `include_only` list fixed at preparation time. - A test reads the Runner's Codex environment keys, and fails if a new key is not classified. - Host launches keep the bounded allowlist. - **runnerd** (`codex_provider.rs`): `runtime_notification_codex_args` still grants the notification files to read-only profiles when no allowlist is present. Plan mode in an external sandbox needs these grants. - **Protected keys**: - An environment driver can declare `protectedEnvironmentKeys` in its manifest. - Paperclip rejects a protected key on an agent or environment that uses the driver. The 422 has code `protected_environment_variable`. - On a project, a routine, or another agent, Paperclip saves the key. The response has a `configurationWarnings` notice that names the environments. The project and routine pages show the notice as a toast. The agent pages already show configuration warnings. - **Docs**: `doc/plugins/PLUGIN_ENVIRONMENT_TASKS.md` describes the provider obligations. - `tests/runner-e2e/source-sdk-public-inputs.json`: refresh the reviewed SDK source pin. ## Verification - `packages/plugins/sdk`: `vitest run tests/environment-tasks.test.ts` (38 pass). Accepts bounded values. Rejects bad names, NUL bytes, too many variables, and oversized values. - `server`: `vitest run src/services/native-runtime/plugin-runner-task.test.ts` (95 pass). Values go only to a provider with the capability, never in `prepare`. `resume` sends values only for a handoff. - `server`: `vitest run src/services/protected-environment-keys.test.ts src/__tests__/project-routes-env.test.ts` (10 pass). Rejection, notices, and the project route response. - `server`: all route suites that import the project, routine, environment, or agent routes (45 files, 1689 pass). - `packages/shared`: `vitest run src/validators/plugin.test.ts` (64 pass). - `packages/paperclip-runner`: `vitest run src/drivers/codex src/configured-environment.test.ts src/live/runner-provider-launch.test.ts` (444 pass). - `packages/paperclip-runner/runner`: `cargo test -p paperclip-runner-core --lib runtime_notification` (9 pass, 2 ignored as before). - `pnpm typecheck` in `packages/shared`, `packages/plugins/sdk`, `packages/paperclip-runner`, `server`, and `ui`. - Not run locally: `pnpm verify --full` and browser suites. Required CI covers the final revision. ## Risks - **Existing providers:** low risk. Core sends no new field to a provider without the capability. Their runs behave as before, with one added log line when variables were skipped. - **Codex in an external sandbox:** commands now see every variable in the Runner environment except Codex's own credentials. The execution environment builds that environment from an allowlist. If the Runner gives Codex a new credential, the drift test fails until the key is classified. - **Saving env:** new rejections happen only when an installed driver declares `protectedEnvironmentKeys`. No driver declares them today. - **Providers:** they must keep the values transient and never save or log them. The doc lists these obligations. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
4f4d823cfc |
feat(environments): forward the warm idle window to plugin lease acquisition (#21305)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Plugin environment drivers run agent tasks on remote computers, and warm leases keep a parked Runner between runs for an idle window that Core records > - Some providers stop an idle computer on their own schedule (for example a sandbox auto-stop interval) > - If that provider timer is longer than the Runner idle window, the provider keeps paying for a computer whose Runner is already gone; if it is shorter, a warm resume can find the computer stopped > - The plugin cannot align the two because it never sees the idle window at acquire > - This pull request forwards the warm idle window to `environmentAcquireLease` as `runnerLifecyclePolicy` > - The benefit is that a provider can stop its computer just after the Runner idle window, without Core knowing anything about the provider's placement ## Linked Issues or Issue Description Refs #21289 Refs #21066 **What existing behavior does this improve?** Warm lease setup for plugin environment drivers. **Subsystem affected** Server environment runtime (`createPluginEnvironmentDriver`) and the plugin SDK acquire parameters. **Current behavior** Core records the warm lifecycle policy (`runnerTaskLifecyclePolicy`, including `idleTimeoutMs`) on the lease, but `environmentAcquireLease` does not receive it, so a provider with its own idle shutdown has to guess. **Proposed behavior** For a warm lease, `environmentAcquireLease` receives `runnerLifecyclePolicy: { mode: "warm", idleTimeoutMs }`, the same value Core records. Other lifecycle modes do not receive the field. **Reason and benefit** A provider can align its idle shutdown with the Runner idle window. **Breaking changes** None. The field is optional and additive. ## What Changed - `PluginEnvironmentAcquireLeaseParams` has a new optional `runnerLifecyclePolicy`. - Core sends it only for warm leases. - `doc/plugins/PLUGIN_RUNNER_EXECUTION.md` documents the field. - The runner-e2e SDK public-input pin is updated for the SDK change. An earlier revision of this PR also made Core try `environmentResumeLease` before a session handoff. That part is removed: placement is the plugin's decision, so Core should not decide to retry the same computer before moving the session. The plugin will own that inside its own resume instead. ## Verification - `pnpm typecheck` in `server/` (builds shared and plugin-sdk) passed. - `vitest run src/__tests__/environment-runtime.test.ts -t "portable task placement|delegates plugin environment leases with acquisition budget"` passed (5 cases). The portable test asserts the first warm acquire receives `runnerLifecyclePolicy` and the lease records `runnerTaskLifecyclePolicy`; the non-warm delegation test asserts the field is absent. Making the forwarding unconditional fails both delegation cases. - The full suite was not run locally; required CI covers it. ## Risks - A plugin that does not know the field ignores it. - Rollback: revert this pull request. No migration. Note: the branch name still reflects the earlier scope (resume before handoff). ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
74096c19f4 |
fix(evals): admit the in-flight chat completion follow-up on a settled source (#21313)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The chat catalog evals check that a chat request reaches the correct saved outcome > - The `source-settled-with-active-work` run policy checks that the CEO chat is settled while delegated work can continue > - After a delegated task finishes, the product starts its normal `chat_task_completed` follow-up on the source chat > - The policy required every source run to be `succeeded` and the source to be `waiting`, so a re-grade during that follow-up failed > - This pull request admits that one product follow-up while it is in flight, and keeps every other source run strict > - The benefit is that the eval grades the saved private-delegation outcome, not the timing of the reload ## Linked Issues or Issue Description **What happened?** The OpenCode cell of `chat-features.runner-opencode.local.tasks.private-task` failed with "Independent oracle survives browser reload: expected true, received false". The saved state was correct. The CEO answered, Morgan saved the private draft in the private project, and the unrelated member could not read it. The grade passed for its full 5-second stability window. Morgan's run then finished, and 73 ms later the product started the CEO `chat_task_completed` follow-up on the source chat. The harness reloaded the page and re-graded while that follow-up was running. The only failed check was `catalog-settled-source-separate-work`, because it required all source runs to be `succeeded`. **Expected behavior** The case already counts this follow-up in its run bound (`expectedRunCount: 3`, `minimumRunCount: 2`). A queued or running product follow-up on an answered source must not fail the case. **Steps to reproduce** Run the private-task case. Let the worker finish within about 5 seconds of the first passing grade. The post-reload re-grade sees the follow-up as `running` and fails. ## What Changed - Add `chatCatalogSourceCompletionFollowUp` to `tests/runner-e2e/chat-catalog-run-policy.ts`. It identifies the lead's `chat_task_completed` run on the source whose every completion update names separate work in the run ledger. - In the `source-settled-with-active-work` check, admit at most one such follow-up while it is `queued`, `running` or `succeeded`. Accept a source `conversationState` of `active` only while that follow-up is in flight. - Keep the other rules strict. The original answer must exist and must have succeeded. Any other active, failed or extra source run still fails. The declared run-count bound still applies. - Also admit the follow-up before its turn is prepared. The server claims the run first and saves the completion updates after that, so a queued run, or a running run in preparation, has only its delivery IDs. It qualifies only while queued or running, when automation woke it, and after a separate worker run in the ledger succeeded. - Add a red/green test that reproduces the observed race and covers the negative cases. ## Verification - `npx vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/chat-catalog-run-policy.test.ts`: the new test fails before the change (1 failed, 13 passed) and passes after it (14 passed). The queued-snapshot cases were added after review. - The related runner-e2e tests (`chat-catalog-run-policy`, `chat-catalog-readiness`, `chat-catalog-failure-retirement`, `runtime-dependency`, `internal`) pass: 200 tests. - `pnpm test:e2e:runner:typecheck` passes. - Paid run [43374419951](https://github.com/paperclipai/paperclip/actions/runs/43374419951) on the first commit of this branch: `chat-features.runner-opencode.local.tasks.private-task` passed. ## Risks - Low risk. The change is in eval grading only. It does not change product code. - The check accepts a follow-up that is still running at grade time. A later failure of that follow-up is not observed by this case. Completion delivery is covered by its own completion cases. ## Model Used - Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, used through Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
23c2343015 |
feat(server): read a retained provider pack from its Runner image (#21308)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Runner executes agent work in a sandbox. It needs a provider pack of agent tools, and the server uploads that pack when the sandbox does not have it > - A retained session keeps the pack it started with. When that pack is not the server's own, the server downloads the pack's transport archives from a registry (#19779) > - Today the server looks for them in the application image tagged `sha-<commit that built the pack>` > - `master` now publishes the Runner one time for each set of inputs, as `ghcr.io/paperclipai/paperclip-runner`. A pack from that image names the commit that published the image, and few commits get an application image, so that tag often does not exist > - This pull request reads such a pack from its Runner image, and keeps the old source for every other pack and for a Runner image that cannot be read > - The benefit is that a retained session can still get its pack when the pack came out of the published Runner image ## Linked Issues or Issue Description Refs #19779 (the retained pack source), #21228 (the pack field `runnerInputsDigest`), and #21204, #21229, #21230 (the Runner image). No issue exists for this change. **Problem or motivation** `retained-provider-pack-oci.ts` reads `ghcr.io/paperclipai/paperclip:sha-<runnerSourceRevision>`. It needs that image to exist, to carry the pack, and to have one layer with a known build-history line. For a pack from a published Runner image, `runnerSourceRevision` is the commit that published the image. `docker.yml` writes `sha-<commit>` only when its `master` run for that commit completes, and a newer push can replace a waiting run. The restart-chaos harness has the same assumption in another place: it requires the pack of a sandbox image to be built by the commit in the image's `org.opencontainers.image.revision` label. **Proposed solution** A pack that names its Runner image in `payload.runnerInputsDigest` is read from `ghcr.io/paperclipai/paperclip-runner:inputs-<digest>`. A sandbox image can name the commit that built its Runner in a label of its own. No application image carries a pack with that field yet, and no image carries that label, so behaviour today does not change. **Alternatives considered** - Keep the application image as the only source and require an application image for every Runner publication. Publication would then depend on `docker.yml`, which serves one ref at a time. - Take a fixed number of layers ("the image has four, take the last two"). A server in the field cannot be changed when the image gets another layer. The server now uses the rule that holds for any number of layers: the full image is the slim image plus transport layers. - Fall back to the application image for every failure of the Runner image. A Runner image that exists and does not match the pack is a fault, and a second source would hide it. **Roadmap alignment** This is build and runtime infrastructure. It adds no product feature and does not overlap an item in `ROADMAP.md`. This is part 3 of 4 of a stack ([stack #21309](https://github.com/paperclipai/paperclip/stack/21309)). Merge the parts in order: 1. #21306: the resolver and the no-rebuild mode 2. #21307: the report-only job in `docker.yml` and the pull-request check of the application image's Runner path 3. #21308: reading a retained provider pack from its Runner image (this pull request) 4. #21310: a registry build cache for the Runner image build This pull request uses no code from parts 1 and 2. It is stacked on them because all four parts edit `doc/runner-image-contract.md`. ## What Changed - `server/src/services/native-runtime/retained-provider-pack-oci.ts`, for a pack whose `payload.runnerInputsDigest` is 64 hexadecimal characters: - The server reads the full index `inputs-<digest>` of the Runner image. The index must carry `io.paperclip.runner.inputs-digest` equal to the pack's value, `org.opencontainers.image.revision` equal to the pack's `runnerSourceRevision`, and a valid digest in `io.paperclip.runner.slim-digest`. - It reads the slim index by that digest, never by the slim tag, and checks the bytes it gets against the digest. - For the pack's platform, the layer digests of the slim image must be the first layer digests of the full image, in the same order. The layers after them are the transport layers. There must be at least one and at most eight, and their total size has the existing limit. - The server downloads only those layers. It checks each against its descriptor and refuses every entry that is not a transport archive, the receipt or the `provider-pack.archives` directory. A file may not be in two layers. - When the Runner image cannot be read, the server reads the application image as before. That covers a registry error status for the token, for either index or for either manifest: for example 404 for a tag or a slim index that does not exist. Nothing is written before that point. - A Runner image that can be read and does not match fails with `runner_remote_provider_source_invalid`. The server does not try the application image then. - The same file, for every other pack: the old path, unchanged in what it checks. Its code moved into functions that both paths use (`request`, `pullToken`, `platformImage`, `receiveTransportLayer`). - The downloaded layer file is removed as soon as it is unpacked. Before, the importer removed it after its checks, so `retained-provider-pack-source.ts` loses that one line. - The cache record gets an optional `runnerImage` field with the inputs digest and the digests of the transport layers. `transportLayerDigest` is the last of them and `transportLayerBytes` their total. An older server ignores the new field. - `tests/restart-chaos/daytona-image.ts`: when the image has a non-empty label `io.paperclip.runner.source-revision`, the pack must be built by that commit. Otherwise the pack must be built by the commit in `org.opencontainers.image.revision`, as before. The receipt field `providerPackRunnerSourceRevision` records the pack's commit. - `doc/runner-image-contract.md` has a new section, "Reading the transport archives alone". `tests/restart-chaos/README.md` describes the label. One choice needs a reviewer's eye. When the full index can be read and the slim index cannot (a registry error status), the server falls back to the application image; it does not fail. Without the slim manifest the server cannot tell which layers are transport layers, so it has not accepted any byte of the Runner image, and the old source has its own complete checks. A slim index that is read and has other bytes than its digest still fails. ## Verification Linux (`node:24.22.3-bookworm`, GNU tar 1.34, zstd 1.5.4, the worktree mounted read-only): - In `server/`: `retained-provider-pack-source.test.ts` (34 tests, 15 of them new), `provider-pack-archive.test.ts`, `remote-provider-pack-verification.test.ts`, `remote-provider-pack.test.ts`: 127 tests pass in 4 files. - `node node_modules/vitest/vitest.mjs run --config tests/restart-chaos/vitest.config.ts tests/restart-chaos/daytona-image.test.ts`: 25 tests pass, 5 of them new. macOS (Node 26.12.0): - `daytona-image.test.ts`: 25 tests pass. - `retained-provider-pack-source.test.ts`: the 15 new tests pass. Six older tests fail, with and without this change. They run the real uploader, which needs GNU tar, and this host has bsdtar. - `tsc` on the two changed server files and the test, and on the two restart-chaos files, with the settings of their test projects: no error. The new cases: - Read from the Runner image with 6 registry requests (two indexes, two manifests, two layers), none to the application image and none for the slim tag. - Take every layer after the slim image's: one transport layer, and three. - Fall back when the tag does not exist, when the token is refused, and when the slim index cannot be reached. - Refuse, with no request to the application image: an index for other inputs, an index for another revision, no slim annotation, a slim annotation that is a tag, a slim index with other bytes than its digest, a slim image that is not a prefix of the full image, a slim image equal to the full image, nine transport layers, and a transport layer that holds a pack file. - Four deliberate faults in the reader each make a test fail: the slim index bytes are not checked, no prefix check, the slim annotation is not validated, and no bounds on the number of transport layers. A real run, in the same Linux container (`linux/arm64`), with no credential: - The script took `provider-pack/provider-pack.json` of `linux/arm64` from the published image `ghcr.io/paperclipai/paperclip-runner:inputs-0e6662ad9c8c6451556c76bf326723645d4afaab2decc73cd5e0025f7dd8d1ca`. Pack digest `sha256:604c29cb32a181843cc4d425871f88c0e65c8185b0a158c801fda4eb1bfce52f`, `runnerSourceRevision` `25e9a9c2e9277f3be8649be8c809cc81bde6f0c5`. - `resolveProviderPackUploadSource`, with another pack as the server's own, returned `{ archiveOnly: true }` after 23 seconds. That function calls `receiveRetainedProviderPack` and then the importer's checks: the manifest inside the overlay archive is the selected manifest, every archive matches the receipt, and the content of every file archive matches the digest of its file. - The server code made 9 requests: one anonymous token request, the full index, the `arm64` manifest, the slim index `sha256:f63ea436…4f3c` by digest, its `arm64` manifest, and two layers, each with one redirect. No request went to the application image. - The cache record: ```json { "schema": "paperclip.retained-provider-pack-source.v1", "sourceRevision": "25e9a9c2e9277f3be8649be8c809cc81bde6f0c5", "providerPackDigest": "sha256:604c29cb32a181843cc4d425871f88c0e65c8185b0a158c801fda4eb1bfce52f", "imageIndexDigest": "sha256:84f2c051aa4ed16c5cc138c08e3533fa417e92177f7d15af5f2aa3e0ba467162", "imageManifestDigest": "sha256:73dbe864fbab6208616f0aa6f1610a7ece4ef22f68a73d30901cbaea60139589", "imageConfigDigest": "sha256:740e48463d5caf0bfead4b923cdbd8b220acb93d058419c3db9d00c7f1831d65", "transportLayerDigest": "sha256:35433b9f788641666b470f41c37fef9fc9b872262b7edd5bd79f0e288f75bcdf", "transportLayerBytes": 426602696, "runnerImage": { "inputsDigest": "0e6662ad9c8c6451556c76bf326723645d4afaab2decc73cd5e0025f7dd8d1ca", "transportLayerDigests": [ "sha256:6a340088ca2cfa85f10f0385d9c8f5cdccb22c3a2bbb2062620b1d90616ba5b5", "sha256:35433b9f788641666b470f41c37fef9fc9b872262b7edd5bd79f0e288f75bcdf" ] }, "receiptSha256": "2afdd26645845369d9e851b1e6e2884eb6c3884ea7c4866a1eeb2efb9fb230d7", "verifiedTarPaths": true } ``` - `uploadVerifiedProviderPack` then installed the pack from that source through a local stand-in for the sandbox, with no further registry request, in 4 seconds. The installed tree is 1,249,060,841 bytes. Its `provider-pack.json` is the selected manifest, all 7 artifacts of the manifest have the digest the manifest names, and the pack's own Node prints `v24.22.3`. Never run: - The same real run for `linux/amd64`, whose transport layers are about 1 GB. - A session in a real sandbox. The real run used a local stand-in for the sandbox. - The fallback against the real registry. Fixtures cover it. - The full test suite and the full typecheck. CI must supply them. ## Risks - The old path was moved into shared functions. Its checks are the same, and its 19 existing tests pass on Linux, but a review must read that code as changed. - The layer file of the old path is now removed before the importer's checks, not after them. The importer does not read it. - The reader depends on two rules of the image contract: the full index names its slim index, and the full image starts with the slim image's layers. A Runner image that breaks either rule is refused, with no fallback. - The fallback covers a registry error status only. A network error ends the import, as it does today. - The label `io.paperclip.runner.source-revision` is new and no image carries it yet. The change to the sandbox image that sets it must use this name. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Fable 5.2 (`claude-fable-5-2`), Anthropic, used through Claude Code with tool use (shell, file edits, local Docker builds). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com> |
||
|
|
bc71c23e0a |
fix(test): remove the Pi fault calibration file race and halve its runtime (#21286)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The Runner support suite checks the Pi provider fault helper. The
helper kills the exact Pi child process on Linux with a pidfd.
> - `tests/runner-e2e/pi-provider-fault.test.ts` failed in 3 unrelated
PRs on 2026-11-12. One failure passed on a rerun of the same commit.
> - The failures had two causes. A fixture file race gave a JSON parse
error. A CPU-heavy calibration ran past the 25 s outer timeout on the
slower shared fleet.
> - This pull request removes the file race and removes about half of
the calibration work. It also states the outer timeout margin.
> - The benefit is a deterministic test that runs in about 3.4 s instead
of about 5.9 s on Linux.
## Linked Issues or Issue Description
**What happened?**
The test "calibrates the exact Pi fault helper with metadata negatives
and native Linux pidfd" failed intermittently in the "Runner support
suite" lane. There were two errors:
- Job 131195539083: `test_real_descriptor_launch_then_exact_child_pidfd
(descriptor=7)` failed at `json.loads(ready.read_text())` with
`JSONDecodeError: Expecting value: line 1 column 1 (char 0)`.
- Jobs 131420813114 and 131420614853: `spawnSync /usr/bin/python3
ETIMEDOUT` after 25 s. Both jobs ran on the `paperclip-public-pr-x64`
fleet (m5n.xlarge, 4 cores, load average about 4). In those jobs, every
file in the lane ran about 1.8x slower than on the hosted `ubuntu-24.04`
image.
**Expected behavior**
The test passes on every run of the same commit.
**Steps to reproduce**
1. Error A: add a delay between file creation and write in the fixture
child (for example `openSync`, then `Atomics.wait` for 500 ms, then
`writeSync`). Run `/usr/bin/python3 -B
tests/runner-e2e/pi-provider-fault.test.py` on Linux. All four real
calibrations fail with the same `JSONDecodeError`.
2. Error B: run the suite on a loaded 4-core runner. The Python suite
needs more than 25 s.
**Paperclip version or commit**
`master` at
|
||
|
|
db3d7223e0 |
fix(verify): report stale SDK public input pins locally, before CI (#21283)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The Runner support suite proves the plugin SDK public types against
pinned source. The pins are in
`tests/runner-e2e/source-sdk-public-inputs.json`.
> - The pins cover every file in `packages/shared/src` and
`packages/plugins/sdk/src`, and the SDK build inputs (for example
`pnpm-lock.yaml`).
> - A PR that edits a pinned file and does not refresh the registry
fails three files in the "Runner support suite" lane.
> - Between 2026-11-12 10:27Z and 2026-11-13 01:14Z, this failure
occurred in 14 CI runs on 12 PRs.
> - The local check, `pnpm verify`, returned exit 2 ("let CI cover it")
for these changes. It did not mention the pins. Authors pushed, and CI
failed.
> - This pull request makes `pnpm verify` find stale pins before the
push. It also makes the opaque fixture error name the file and the fix.
> - The benefit is that authors find and fix the problem in seconds,
locally, instead of after a 6-minute CI lane.
## Linked Issues or Issue Description
**What happened?**
Three files in the "Runner support suite" lane failed together on many
unrelated PRs: `source-sdk-evidence.test.ts`,
`source-sdk-public-inputs.test.mjs` and
`source-sdk-public-type.test.ts`. Two of the three failed with the
opaque error `Reviewed SDK reference source changed`.
**Root cause**
The pins on master are not stale. The tests pass on master (58/58). Each
failing PR edited a pinned input and did not refresh the registry. I
compared the PR heads with their merge base:
- All 14 failing heads (12 PRs) changed a pinned input. None of them
changed `source-sdk-public-inputs.json`.
- All 19 heads in the same window that changed a pinned input and passed
this lane also refreshed the registry. This includes later heads of
three PRs that failed first (#21119, #21121, #21156).
- Two of the 12 PRs are Dependabot PRs that change only `pnpm-lock.yaml`
(#19593, #19594). The lockfile refresh bot already resyncs the pins for
lock changes on master.
**Steps to reproduce**
1. Start from `origin/master`.
2. Add one line to `packages/shared/src/types/heartbeat.ts` and commit
it.
3. Run `node node_modules/vitest/vitest.mjs run --config
tests/runner-e2e/vitest.config.ts source-sdk-evidence.test.ts
source-sdk-public-inputs.test.mjs source-sdk-public-type.test.ts`.
4. The result is the same as CI: 3 files fail, with the same 4 failures.
5. Run `pnpm verify --explain`. It returns exit 2 and does not mention
the pins.
**Expected behavior**
`pnpm verify` reports the stale pins and the refresh command before the
author pushes.
## What Changed
- `tests/runner-e2e/source-sdk-public-inputs.mjs`: add
`staleSdkPublicInputsInWorkingTree(root)`. It applies the existing drift
guard to the working tree (tracked and untracked files that git does not
ignore). Add `isSdkPublicInput(file)` and the shared failure message
`staleSdkPublicInputsMessage(stale)`. The committed-tree guard keeps its
behavior and now uses a shared comparison.
- `scripts/local-verify.mjs`: when the diff touches a pinned input, the
registry or `report-lock.json`, `pnpm verify` checks the pins first. If
they are stale, it exits 1 and prints the files and the refresh command.
The check reads files only. It does not write the registry.
- `tests/runner-e2e/source-sdk-reviewed-fixture.ts`: the error `Reviewed
SDK reference source changed` now names the file and gives the refresh
command.
- `tests/runner-e2e/source-sdk-public-inputs.test.mjs`: new test for the
working-tree guard (uncommitted edit, untracked file, ignored file). The
CI drift test uses the shared message.
- `doc/DEVELOPING.md`: document the new check under "Incremental local
verification".
No assertion is weakened. No pin is changed. No file under `.github/` is
changed.
## Verification
- Master, without this change: the three files pass (58/58).
- Red: on master plus a one-line edit to
`packages/shared/src/types/heartbeat.ts`, the three files fail with the
CI signature. `pnpm verify --explain` returns exit 2 and does not
mention the pins.
- With this change on the same red commit: `pnpm verify` exits 1 in
about 15 s and prints `tests/runner-e2e/source-sdk-public-inputs.json is
stale for 1 pinned input(s): packages/shared/src/types/heartbeat.ts
(changed). Commit your change, run ... --write, and commit the updated
registry.`
- Green: I followed the printed command (`node
tests/runner-e2e/source-sdk-public-inputs.mjs <HEAD> --write`, then
commit). The three files then pass (59/59, with the new test).
- `node --test scripts/__tests__/local-verify.test.mjs`: 14/14 pass.
- `pnpm verify` on this branch returns exit 2 because
`scripts/local-verify.mjs` is a global CI input. Required CI supplies
the broad coverage.
## Risks
- Low risk. The new check only reads files. It runs only when the diff
touches a pinned input.
- `pnpm verify` now fails (exit 1) for a change that also fails in CI.
Before, it returned exit 2.
- The check needs a working `git ls-files` in the repository. `pnpm
verify` already needs git.
- Dependabot lock PRs still fail this lane until the lockfile bot
refreshes the pins. This PR does not change that path.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- Claude (Anthropic), Claude Code agent with tool use (shell, git,
GitHub CLI). Commits are attributed to Claude Fable 5.2.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
|
||
|
|
0c04ff76eb |
fix(native-runtime): a follow-up that arrives as a warm session closes continues the task (#21279)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A native agent session on a remote sandbox stays warm after a run, and an idle timeout closes it after about 5 minutes > - When a person's follow-up comment arrived just as that idle close began, the new run failed with `native_session_supervisor_busy` > - The idle close of the finished owner could also fail its final checkpoint with `native_runner_checkpoint_owner_lost`, because the task already pointed at the queued follow-up run > - Then the retry was held for workspace repair and the task stayed blocked; the environment could not be deleted (409) because its warm runner could not be stopped > - This pull request makes the follow-up wait for the idle close that is in flight, and lets the finished owner save its final checkpoint while its successor still waits in the queue > - The benefit is that a follow-up that lands at the warm session's idle close continues the task with no person involved ## Linked Issues or Issue Description Refs #18011 (the checkpoint ownership fence this change extends). **What happened?** A follow-up comment arrived 295–340 s after the previous run ended, while the warm session's idle close was in progress. The new run failed with `native_session_supervisor_busy`. In another case, the idle close failed its final checkpoint with `native_runner_checkpoint_owner_lost`, the next run failed `native_instruction_materialization_handoff_unavailable`, and the retry was cancelled with `execution_reconciliation_required`. The task stayed blocked. A hosted instance showed the same class as "previous run's session on this sandbox could not be retired". **Expected behavior** The follow-up run waits for the idle close (inside the existing bounded release wait) and then starts. The finished owner saves its final checkpoint and closes. The task completes hands-off. **Steps to reproduce** 1. Run an opencode agent on a Daytona sandbox with a warm session (idle timeout 300 s). 2. Let a run finish. 3. Post a comment on the task 298–306 s after the run ended. **Paperclip version or commit** master `444dc18d98`. **Deployment mode** Local server with Daytona remote sandboxes (restart-chaos harness, light lane). The same class was seen on a hosted staging instance. ## What Changed - `claimWarmNativeInstructionCopy` now also waits when an idle close is in flight for the session scope (`closingWarmNativeSessions`). The wait uses the same deadline as the existing release wait (`warmSessionReleaseWait`), so it adds no new wait and nothing before `turn.start` on the normal path. - `validateNativeRunnerCheckpointOwner` accepts the finished owner's final checkpoint (accepted result, no new work) when `issues.executionRunId` points at a successor run that is `queued` or `scheduled_retry` for the same company and task and has no `native_run_finalizations` row. - Tests: a server test for a follow-up that arrives while the idle timeout closes the warm session; four new scenarios (`queued_successor`, `scheduled_retry_successor` × `runner.suspend`/`turn.stop`) and four new negatives (`queued_wrong_issue`, `queued_new_work` × both commands) in the early-retirement checkpoint matrix. - Restart-chaos harness: `RESTART_CHAOS_MIX_FOLLOW_UP_AFTER_END_MS=<min>-<max>` and `RESTART_CHAOS_MIX_FOLLOW_UP_COUNT` post the staging-mix follow-up a set delay after the task's last run ended, so a cell can aim the follow-up at the warm idle close. The restart-chaos README describes both variables. A failed post is retried with the same `clientRequestId`, the grade check `after_end_follow_ups_served` needs an agent reply after each post and the ledger marker, and the task budget includes the planned waits. - Harness: the wave database cleanup (`tests/runner-e2e/controller-database.ts`) no longer waits forever when a group signal (for example `timeout` or Ctrl-C) already stopped the embedded Postgres. Before, `embedded-postgres` `stop()` waited for an exit event that had already fired, and an interrupted wave hung after the Daytona R6 sweep. ## Verification - Fix 1, red without the product hunk: `native-session-executor.test.ts -t "idle timeout is closing"` fails with `promise rejected "Error: native_session_supervisor_busy"`. Green with the fix: 1 passed. - Fix 2, red without the product hunk: `native-runner-restart-recovery.integration.test.ts` fails 4 of 208 (`queued_successor` and `scheduled_retry_successor` × `runner.suspend`/`turn.stop`) with `native_runner_checkpoint_owner_lost`. Green with the fix: 208 passed, 14 skipped. The two new negative cases pass in both states. - Harness hang fix, red without the fix: the new `controller-database.test.ts` case ("finishes cleanup when the database already stopped because a group signal reached it") times out on cleanup (`expected 'hung' to be 'cleaned'`). Green with the fix: 5 passed, 1 skipped. - After the rebase onto master `7154030ef6`, both server test files together: 1639 passed, 21 skipped. - Restart-chaos harness tests (`staging-mix`, `staging-mix-retry`, `observer`): 75 passed, including a test for the after-end retry, the `after_end_follow_ups_served` grade check and the added budget. `tsc --noEmit` passes for `server`, `tests/runner-e2e` and `tests/restart-chaos`. - `pnpm verify` exits 2 because `tests/restart-chaos/observer.ts` is a global CI input. Required CI supplies the broad coverage. - Daytona cell on this branch (4 opencode tasks, warm sandboxes, 1 vCPU, follow-up comment posted 300.8–308.2 s after each task's last run ended, image built for master `444dc18d98`): 4/4 tasks done hands-off, 8/8 runs succeeded, 0 Retry banners, 0 `owner_lost` / `supervisor_busy` / "could not be retired" / `execution_reconciliation_required` lines in the server log, environment delete clean, warm-shutdown grade pass, 0 sandboxes left. The harness exited by itself after 1076 s. - Same cell before the fix (master, 8 tasks): 1/8 tasks blocked (`native_session_supervisor_busy` → `native_runner_checkpoint_owner_lost` → `execution_reconciliation_required`). With fix 1 only: 1/8 still blocked (`owner_lost` on the idle close). With both fixes: 8/8 done and 0 failed runs (cut by the 20-minute cap before cleanup), then 4/4 above. - The cell grade `T:completion_once` fails on all 4 tasks by design: the timed follow-up comment arrives after the task is done, so the agent posts a second completion comment. ## Risks - Fix 1 can delay a follow-up by up to the existing warm release wait (30 s) while an idle close finishes. Before this change, that run failed. If the close fails or the wait expires, the claim returns null and the run uses a fresh instruction copy, as before. - Fix 2 widens the checkpoint ownership fence from #18011. It does not reopen what #18011 closed: - The exemption applies only to the finished owner's final save: the checkpoint must hold an accepted result and no new work, so no provider turn can be granted through it. - The successor must still be unclaimed (`queued` or `scheduled_retry`) and must have no native coordinator (`native_run_finalizations`) row. No controller has started it, so nothing can have consumed or depended on the predecessor's checkpoint. Once the successor is claimed, the existing preparing/claimed rules from #18011 apply unchanged. - The successor must belong to the same company and the same task. - Cases that must still reject, with tests in the same matrix: - `queued_wrong_issue`: a queued successor for a different task is rejected with `native_runner_checkpoint_owner_lost` and the stored checkpoint does not change. - `queued_new_work`: a `turn.start` (new work) while the successor is queued is rejected with `native_runner_checkpoint_owner_lost` and the stored checkpoint does not change. - All earlier rejections stay: unclaimed native coordinator, dispatching, expired or foreign preparation, foreign native preparation, wrong task, new work, unaccepted result, replaced controller. - Not changed: an owner whose close fails still blocks environment delete (409). That is a separate product decision. ## Model Used - Claude (Anthropic), model ID `claude-opus-5-5`, used through Claude Code with tool use (shell, file edit, test runs). Earlier commits on this branch were written by the same model in an earlier session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
b9308f9828 |
fix(ui): keep long agent names inside task columns (#21155)
## Thinking Path > - Paperclip helps people manage AI agents and their work. > - Task views show the assignee and creator in fixed-width columns. > - Long names can extend past those columns and cover adjacent values. > - The identity labels already support text truncation but need a width limit. > - This pull request limits each identity to its task column. > - Names stay on one line, avatars keep their size, and adjacent values stay readable. ## Linked Issues or Issue Description **What happened?** Long agent and human names extend outside the Responsible and Kicked off by columns in task views. **Expected behavior** Names fit inside each column. Long names show an ellipsis. Avatars remain visible. The title attribute retains the full name. **Steps to reproduce** 1. Open a task view with Responsible and Kicked off by columns visible. 2. Use a long agent name for the assignee and creator. 3. Observe names covering adjacent columns. ## What Changed - Limit agent and human identity width to the containing task column. - Add a production-component Storybook example for long names, names without spaces, and short names. - Add browser layout checks for both identity columns, avatar size, full text, adjacent values, and row height in both themes at 640px and 1200px. ## Verification - The browser regression failed before the fix: an identity reached x=424 while its column ended at x=144. - The focused Chromium suite passes all four cases at 640px and 1200px in light and dark modes. - `pnpm --dir ui exec vitest run src/components/IssueColumns.test.tsx src/components/IssuesList.test.tsx`: 83 tests pass. - `pnpm --filter @paperclipai/ui typecheck`: passes. - `node scripts/check-token-gates.mjs` and `git diff --check`: pass. - `pnpm verify` returned exit 2 because the browser test requires broader CI coverage. The complete PR CI run supplies that coverage. - All tests, typecheck, build, and e2e gates pass on final commit `f73568822add60b4659f2cbdad0196fdfe010d3b`: https://github.com/paperclipai/paperclip/actions/runs/43271777908. - The separate commitperclip review passed: https://github.com/paperclipai/paperclip/actions/runs/43271777344. - Greptile gives this commit 5/5. There are no open findings or review threads. - All 61 checks are complete: 57 success, 1 neutral, and 3 skipped. The Storybook workflow was skipped; the focused browser suite passed locally. ## Risks Low risk. The width limit applies only to identity labels in shared task columns. The full name remains in the DOM and the existing title attribute. No API or stored-data changes. ## Model Used OpenAI Codex, based on GPT-6, with code tools and Chromium browser checks. The runtime does not expose a more specific model ID. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
3b4c3265ed |
fix(recovery): offer Try again on a model-unavailable failure and recheck the model first (#21244)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - When a native run fails, the task thread shows a failure marker, and some markers offer Try again > - A `native_provider_model_rejected` failure blocks the task for the person and shows "Model unavailable", but it has no Try again control > - Recovery must just work: after the person fixes the model, they need one obvious control that continues the task, and that control must not start a run that is sure to fail > - This pull request shows Try again on the "Model unavailable" marker and makes the retry recheck the selected model before it starts a run > - The benefit is that the person fixes the model, presses Try again, and the task completes, and a too-early press explains which setting to change instead of failing again ## Linked Issue or Issue Description Refs #21210 **What happened?** Paid run 43321773136, `recovery.retry-exhausted` (OpenCode, `openrouter/deepseek/deepseek-v4.1-flash-1017`). The run failed with `native_provider_model_rejected`. The task stayed blocked and the thread showed "Model unavailable", but no Try again control appeared (`task-chat-run-failed-try-again` not found in 30 s). Only the "Run failed" and "Usage limit reached" markers offer Try again. **Expected behavior** The "Model unavailable" marker offers Try again. Pressing it checks the selected model first. If the provider-rejected model is still selected, no run starts, and the person sees which setting to change with a link to it. After the person changes the model, Try again starts one retry of the original run. **Steps to reproduce** 1. Set an agent's model to an id the provider rejects. 2. Assign it a task. The run fails with `native_provider_model_rejected`. 3. Open the task. Before this change, the marker has no Try again. ## What Changed - `server/src/services/native-runtime/model-rejected-retry.ts`: `rejectedModelRetryRefusal` reads the model the failed run sent (`runnerProfileJson.nativeExecutionInput.provider.model`) and the model a new run would use (the task's model override for its assignee, else the agent's model). If they are the same, it returns a 409 refusal with code `native_provider_model_still_unavailable`, the setting to change (`agent_model` or `task_model_override`) and the rejected model. - `server/src/routes/agents.ts`: the exact failed-run retry (`POST /agents/:id/wakeup`, `reason: retry_failed_run`) applies this check before it queues anything. - `ui/src/components/task-chat/adjacent-errors.ts`, `TaskChatThreadView.tsx`: one `isRetryableFailureMarker` rule, now including "Model unavailable". - `ui/src/lib/model-retry-refusal.ts`, `ui/src/pages/IssueDetail.tsx`: a refused retry shows "Model still unavailable" with the server's message and an "Open <agent> model settings" link to `/agents/<agent>/runtime`. A task override refusal names the override in the message. - Eval `recovery.retry-exhausted` (oracle `runtime.retry-exhausted.v7`): the rejected model stays selected after the failure. The person presses Try again (409, toast names the model, its link opens the agent's model settings, no run starts in 20 s, task stays blocked), restores the working model, presses Try again (202), and the one retry of the original run uses the working model, saves the brief and completes the task. 3 runs: failed 1, succeeded 2. The budget-hold and two follow-up chat phases are removed; the case now tests the model recheck. `recovery.yaml` context and expected outcome updated, recorded in `chat-full-catalog-testops-manifest.json` `changedFields`, pinned in its test, descriptors and inventory digest updated. ## Verification - RED on master: `TaskChatThread.test.tsx` "offers Try again on a model-unavailable failure and retries that run" fails (no control). `agent-live-run-routes.test.ts` "refuses a retry while the rejected model is still selected and starts no run" fails (the route queues the run). - GREEN: both pass. `agent-live-run-routes.test.ts` 101/101. - `server/src/services/native-runtime/model-rejected-retry.test.ts` (embedded PostgreSQL), 6/6: refuses while the agent still selects the rejected model (names the agent setting); names the task override; allows the retry when the agent or the override now selects another model, or the run failed for another reason; ignores another company's run. - `ui/src/lib/model-retry-refusal.test.ts`: names the setting and links to `/agents/morgan/runtime`; no link for a task override; other errors fall through to the generic toast. - Server `tsc --noEmit` clean. UI `tsc` clean apart from the pre-existing generated storybook catalog. `node scripts/check-token-gates.mjs`: clean. - Eval: `npx vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/chat-catalog tests/runner-e2e/chat-catalog-domains tests/runner-e2e/chat-*.test.ts`: 339 files pass. New calibration rejects an accepted early retry, a run started while the model is rejected, a refusal without the model, setting link or blocked task, a retry that used the old model, and duplicate or missing retries. `pnpm test:e2e:runner:typecheck`: pass. Manifest test 10/10. `source-sdk-public-inputs --check`: `changed: false`. - Paid run: Runner Full-Stack E2E 43338695850 on throwaway branch `chat-eval/qa-mr-paid`. Result to follow in a comment. ## Risks - The recheck compares the selected model with the model the provider rejected. It does not call the provider. A model the provider rejected stays refused until the person selects another one; a different but also unsupported model starts a run that fails with the same marker again. - The refusal message is a toast (15 s). The marker detail still says to choose a supported model. - No persisted shape changes; runs saved by older builds carry the same `runnerProfileJson` field. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
db802d1230 |
fix(heartbeat): deliver a queued comment again unless the reading run cites it as handled (#21243)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - A person can add a comment to a task while the agent's run on that
task is still working
> - #21175 made a `get_task_history` read of a queued comment's current
revision count as delivery
> - A read is not handling. In `agent-chat.multiple-handoffs`, Morgan's
running turn read the "Launch sooner" comment, set it aside, and saved
"Launch later". No continuation ran, so the instruction was lost.
> - This pull request counts a read as delivery only when the run's
accepted result cites the comment as handled. Otherwise one continuation
still delivers it, and that continuation is told the comment may already
be applied.
> - The benefit is that a person's comment is never silently lost, and
the #21175 protection against repeated completed work stays in place
## Linked Issue or Issue Description
Refs #21175
Refs #21203
Refs #21210
**What happened?**
Paid run 43321773136 (OpenCode,
`openrouter/deepseek/deepseek-v4.1-flash-1017`), case
`agent-chat.multiple-handoffs`. Morgan's running turn called
`get_task_history` and received the CEO's queued "Launch sooner" comment
at its current revision (`task_history.read` at seq 75). The model
judged the comment as an unauthenticated agent instruction and saved
"Launch later" from the brief. Its accepted result did not mention the
comment. `deliveredQueuedCommentIds` counted the read as delivery, the
release drain dropped the held comment, and no continuation ran.
**Expected behavior**
A read counts as delivery only when the run records the comment as
handled. A comment that was read but not recorded as handled is
delivered once in a new turn. The new turn's prompt says that an earlier
turn already read the comment and may have applied it.
**Steps to reproduce**
1. Start a native run on a task.
2. Queue a comment behind the run.
3. Let the run read `get_task_history` and finish without applying the
comment.
4. Before this change, no continuation runs and the comment is lost.
## What Changed
- `server/src/services/queued-comment-delivery.ts`: the history-read
delivery rule now also requires the reading run's `run.result.accepted`
event to cite `comment:<id>` (top-level `evidence[].ref` or a
criterion's `evidenceRefs`, prefix match so an annotated ref counts).
The read revision, succeeded-run and finished-after-revision rules are
unchanged. All consumers (release drain, stale-continuation save,
ownership reconciler, first-turn queued input, remote-Stop resume) use
this one rule.
- New `readByEarlierTurnCommentIds`, `commentsArrivedDuringRun` and
`taskHistoryHandlingGuidance`.
- `get_task_history` (both tool authorities) returns
`arrivedDuringThisTurn` (the `comment:<id>` refs of comments other
authors posted or edited after the run started) and a `handling`
instruction to cite each applied comment in `paperclip_finish` /
`paperclip_block` evidence. The comment list itself is unchanged. A
failure to compute the list returns no guidance, so the comment stays
owed.
- `buildPaperclipWakePayload` marks a wake comment `readByEarlierTurn`
when another finished run of the agent read its current revision.
`renderPaperclipWakePrompt` adds one line after that comment: check the
current task state first, and do not repeat work that is already done.
- `doc/run-log-events.md`: the delivery rule and the new fields.
- Evals: `agent-chat.multiple-handoffs` (oracle
`runtime-multiple-handoffs.v6`) and `steering.edit-queued-message`
(oracle `edit-queued-message.structural.v6`) accept in-turn consumption
only with a current-revision read and a citation. A read without a
citation must get exactly one continuation whose saved wake marks
`readByEarlierTurn`, and the final saved revision must hold the change
("Launch sooner"; blue). `runtime-queue.ts` reads citations from the
run's `run.result.accepted` event. No scenario YAML changed.
## Verification
- RED on master `b1ef1f3022`,
`server/src/__tests__/heartbeat-held-comment-wake.test.ts`: 10 of 41
fail. The key one: "settles a held message as soon as its run closes the
task (read mid-turn: true, cited: false)" delivered 0, expected 1.
- GREEN: 41/41. The three required shapes:
- read, not cited: one continuation, and its wake marks
`readByEarlierTurn` (`...read mid-turn: true, cited: false` and the
stale-queued-run variant).
- read and cited: no continuation (`...cited: true`).
- never read: one continuation, as before.
- Edited after the read, or a failed reader: still owed.
- `server/src/services/queued-comment-history-read.test.ts` (remote-Stop
resume): succeeded+cited not owed; succeeded+uncited owed; failed+cited
owed. 4/4.
- `packages/adapter-utils/src/server-utils.test.ts`: the prompt line
renders only for `readByEarlierTurn`.
- Neighbouring suites: `muse-control`, `dot-runner`,
`paperclip-runner-tool-authority`, `paperclip-control-plane-port`,
`muse-native-follow-up`, `runner-prp-coordinator`,
`native-session-handoff`, `execution-continuation-history-read`,
`heartbeat-comment-wake-batching`, `ownership-reconciliation-report`,
`run-preparation`: all pass. Server and adapter-utils `tsc --noEmit`:
clean.
- Eval graders: `npx vitest run --config
tests/runner-e2e/vitest.config.ts tests/runner-e2e/chat-catalog
tests/runner-e2e/chat-catalog-domains`: 283 files pass. New calibration:
the observed lost-instruction case (read, not cited, no continuation)
fails; read-not-cited with one warned continuation passes; an unwarned
continuation, a missing final change, and a cited read plus a
continuation fail.
- `pnpm test:e2e:runner:typecheck`: pass. `node --test
tests/runner-e2e/chat-full-catalog-testops-manifest.test.mjs`: 10/10.
`node tests/runner-e2e/source-sdk-public-inputs.mjs <sha> --check`:
`changed: false`.
- Paid run: Runner Full-Stack E2E 43338695850 on throwaway branch
`chat-eval/qa-mr-paid` (this change plus the model-retry fix on master
|
||
|
|
bc2cdeaa7c |
fix(evals): grade full cohort 9 chat cases on saved state and served links (#21249)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The chat eval catalog checks that agents in chat do the right thing. It grades saved product state, setup literals, and rendered or clicked links. > - Full cohort 9 ran OpenCode with `openrouter/deepseek/deepseek-v4.1-flash-1017` on master |
||
|
|
c1cf43e666 |
test(restart-chaos): count staging-mix completions only from DONE, and attribute runs after DONE (#21226)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The restart chaos harness grades staging-mix tasks across server
swaps, and reports duplicated side effects separately from completion
> - `completion_once` counted every comment that mentions `DONE-<nonce>`
as a completion
> - In wave `w-20261112T223803-s7101`, task t001's blocked run fcc55827
wrote that it "will … post `DONE-<nonce>`" later, and the harness
counted that as a second completion next to the real one from run
37724b59
> - Runs that start after a task is done are also hard to attribute
without a manual trace through the wake evidence
> - This pull request counts only real completions, and lists each run
started after DONE with its wake reason and actor
## Linked Issues or Issue Description
Refs #21168, #21173.
**What existing behavior does this improve?**
The staging-mix `completion_once` check and the wave summary.
**Subsystem affected**
Test harness only: `tests/restart-chaos/`. No product code changes.
**Current behavior**
Any agent comment that mentions the DONE marker counts as a completion.
A blocked or progress comment that promises to post the marker later
becomes a false "duplicate completion". The summary does not say why a
run started after the task was done.
**Proposed behavior**
A run completes the task only when it posts the bare `DONE-<nonce>`
comment (the rule `isDoneComment` already uses), or posts a comment that
mentions the marker at or after the first DONE comment. The summary gets
a "Runs started after the task's DONE" section. It lists each such run
with its wake reason, source, trigger and requesting actor, taken from
the saved wake evidence.
**Reason and benefit**
The duplicate count stays honest, and extra turns can be attributed
straight from the wave results.
## What Changed
- `staging-mix.ts`:
- Comments keep their creation time.
- The new `completionRunIds` counts a run as completing the task when it
posts the bare DONE comment, or posts a later comment that mentions the
marker.
- `done_comment_posted` and `done_comment_once` are unchanged.
- `staging-mix-retry.ts`:
- `firstDoneAt` anchors DONE on the first bare DONE comment, else on the
issue's completion time.
- `runsAfterDone` joins each later run to its wake row in
`wake-evidence/<task>.json`.
- The results JSON gains `extraTurns`, and the summary prints one line
per run, for example `run 91f0c3d2 started … after DONE; wake
issue_recovery_action_restored (automation/system) by system sweep`.
- Runs after DONE are filtered by when they started, not when they were
queued. A run queued after DONE that never started is listed separately
as "queued after DONE, never started" (`queuedAfterDone`).
- `wake-evidence.ts` keeps only issue status transitions: `issue.*`
entries whose status is an issue status. In t001 of
`w-20261112T223803-s7101`, a `runner.api_called` entry with HTTP status
201 was being listed as a status change.
- README: the `completion_once` row, the results paragraph and the
wake-evidence status note.
## Verification
- `pnpm exec vitest run --config tests/restart-chaos/vitest.config.ts`:
68 files, 774 tests pass.
- `node scripts/typecheck-test-projects.mjs tests/restart-chaos`:
passes.
- A regression test copies t001's shape from `w-20261112T223803-s7101`:
the blocked comment from run fcc55827 at 22:56 mentions the marker as a
future step, and the bare DONE comment comes from run 37724b59 at 23:12.
The test checks `completion_once` passes with `runs=1`. It fails on
master, which reports two runs.
- The earlier duplicate case still fails `completion_once`: a different
run's completion summary posted after the DONE comment.
- Tests cover the run-after-DONE join, the DONE anchor, and the summary
line.
## Risks
- Low risk. Test harness only.
- A comment from a run that was cut off before it could post the bare
marker, but that mentioned the marker before the first DONE comment, is
no longer counted as a completion. That is the intended change.
- Known master CI failures, not touched here: the storage S3 multipart
throttle test and the Runner native-distribution-integrity test. Recent
cancelled-runner CI failures ("The runner has received a shutdown
signal") are infrastructure, not tests.
- `ci / General tests (server (11/12))` failed on head
|
||
|
|
2d1c2db64a |
feat(mcp): host the MCP Skills extension for connected apps (#21219)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents use connected apps through MCP. Paperclip already hosts MCP Apps views and speaks stateless MCP 2026-07-28. > - The MCP Skills extension (SEP-2640, Final) lets an app publish skills (instructions and files) next to its tools. Hugging Face already serves 25. > - Paperclip did not read them, so agents missed each app's own workflow guidance. > - Skills are a prompt-injection surface. The spec makes manifest binding, verification, origin tagging and approval revocation host requirements. > - This pull request makes Paperclip a Skills host: discover, review, verify, and deliver app skills to the agents that use the app. > - The benefit is that connected apps can teach agents their workflows, under board control. ## Linked Issues or Issue Description **Problem** Connected MCP servers can publish skills with the `io.modelcontextprotocol/skills` extension. Paperclip ignored them. A board could not see, review or approve them, and agents never received them. **Proposed solution** Implement the host side of SEP-2640. List skills during catalog refresh, record them per connection and company, bind board approval to the full file manifest, and load approved skills into agent runs with verified, lazily fetched content. **Why it matters** Apps ship workflow guidance with their tools. Without the extension, each team must copy that guidance into company skills by hand, and it goes stale. Refs #21119, Refs #21140 (the MCP Apps and stateless MCP client work this builds on). ## What Changed - **Client capability.** Every stateless request declares `io.modelcontextprotocol/skills`. Legacy handshake sessions do not. The capabilities that `server/discover` advertises are remembered beside the protocol era. - **Discovery.** Catalog refresh calls `skills/list` (paged) only when the server declared the extension. A listing inside its `ttlMs` is reused. A skills failure never fails the tool refresh. Nothing runs while a run starts. - **Schema.** New `tool_mcp_skills` (company, connection, skill URI; frontmatter, sorted manifest, manifest digest, review state) and `tool_mcp_skill_files` (verified bytes keyed by connection, URI and digest). Both cascade with the connection. - **Governance.** New skills wait for review. Approval binds to the order-free digest of every `{uri, digest, size}`; a request for any other digest gets 409. A changed manifest or frontmatter revokes approval and logs `tool_mcp_skill.approval_revoked`. Dynamic, oversized and invalid skills are declined with a reason. Approve and decline use the company skill policy (`skills.install`, `external_package`, `mcp-connection:<id>`). - **Run time.** Approved skills of connections the agent can use load in parallel with other skill preparation, inside a 3 s budget. `SKILL.md` is read with the agent's grant and checked for size, SHA-256 and frontmatter. A failure calls `skills/get` and takes the skill out of use. Each skill becomes a per-run bundle on the connector skill path, named in its app's namespace, tagged with its origin. Paperclip writes the frontmatter: `allowed-tools` only if the board approved it. - **Supporting files.** `GET /api/agents/me/mcp-skills/:skillId/files?path=` serves manifest-listed files to an agent in an active run, through the skill's own connection. The sandbox bridge allows the route. - **Board API and UI.** `GET/POST /api/companies/:companyId/mcp-skills…` (list, read instructions, approve, decline). The Skills page shows "Skills from connected apps" with app, state, size, requested tools and plain-language notes. It renders nothing when no app publishes skills. - **Docs.** `doc/connections/MCP-SKILLS.md`, linked from `doc/connections/README.md`. - **Shared frontmatter parser.** It now decodes YAML single-quoted scalars. Before, 5 of Hugging Face's 25 skills kept their quotes and would fail the frontmatter check; now all 25 match. The generated runner copy is regenerated. File size notes: `server/src/services/tool-gateway.ts` grows by about 200 lines. The run-time reads must use its grant resolution and `callAssignedConnectionProtocol`, like MCP Apps view reads. Splitting the gateway is a separate refactor. `CompanySkills.tsx` and `CompanySkills.production.tsx` gain 2–4 lines each; the section lives in its own component. ## Verification - `server/src/__tests__/mcp-skills.test.ts` (15 tests) uses a fake stateless skills server and embedded Postgres. It covers: discovery only after the server declares the extension; the capability in `_meta`; `ttlMs` reuse; lazy fetch (no reads at listing or approval; `SKILL.md` at load; supporting files on read); a stale-digest approval refused; a reorder that keeps approval; a changed or added file that revokes it; dynamic skills declined; a same-size tampered `SKILL.md` rejected, refreshed with `skills/get`, and taken out of use; mismatched frontmatter rejected; reads outside the manifest refused; the same URI and name from two servers kept apart; company scoping; a slow app skipped within the budget and cached later; HTTP authorization (board only, agent needs an active run, run header must match) and activity logging. - Red/green: with the digest check disabled, the tamper test fails; with the manifest comparison disabled, the revocation test fails. Both pass with the code. - `server/src/__tests__/mcp-protocol-versions.test.ts`, `mcp-apps.test.ts`, `connection-instructions.test.ts`, `tool-gateway-discovery-http.test.ts`: 47 pass (two capability assertions updated for the new extension). - `packages/shared/src/frontmatter.test.ts` adds a single-quoted scalar case; the runner contract test passes (23 tests). - `ui/src/components/skills/ConnectedAppSkills.test.tsx` (3) and `ui/src/pages/CompanySkills.test.tsx` (44) pass. `packages/adapter-utils/src/sandbox-callback-bridge.test.ts` (89) passes. - `pnpm --filter @paperclipai/server typecheck` passes. UI `tsc` passes except the known missing generated Storybook catalog files. - `pnpm verify` returned exit 2 (doc change is a global CI input); required CI supplies broad coverage. - Live check against `https://huggingface.co/mcp` without credentials, through Paperclip's own services and the real egress guard: 25 skills discovered as pending; three approved; cold load 135 ms, cached load 13 ms; a supporting file read through the manifest; the bundle tagged with its origin. - Storybook: `Agents and reusable capabilities/Skills and Skill Studio/Connected app skills` (needs review, all states, tool request, name collision, reading instructions). ## Risks - **Migration:** two new tables only. No changes to existing tables. Safe with the previous release running. - **Run start:** companies without an approved app skill pay one indexed read. With approved skills the wait is capped at 3 s; late or failing skills are left out with a `skills.mcp_unavailable` run warning and never fail the run. - **Agent harness enforcement:** Paperclip cannot gate each code-execution tool call an agent makes while following an app's skill. Approval is explicit and per skill, the approval note says agents may run commands the skill describes, and `allowed-tools` is never granted without a separate tick. - **Shared-home adapters** receive app skills like company skills. A revoked skill leaves on the next skill sync. - Not in this PR: per-connection opt-in for dynamic skills; serving Paperclip's own skills from its public MCP endpoint; `resources/directory/read`; app skills on an agent's skills tab. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits, local test runs, live HTTP checks). Extended reasoning enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
826d41e646 |
feat(ui): redesign Settings → Secrets around what people manage (#21225)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents need API keys and tokens. People manage them in Settings →
Secrets.
> - The page showed every credential in one table. Tokens that Paperclip
saved for connections, chat channels and model sign-ins sat beside the
few secrets a person added, many with record IDs as names.
> - Each row repeated an always-true status, a version and a key. New
secret opened on two stacked mode switches and two storage pickers.
> - A person could not see which agents use a secret without opening it,
and had to open it again after creation to give an agent access.
> - This pull request redesigns the page, the New secret dialog, the
details panel, the Vaults tab, the import dialog, the Proposals tab and
My secrets. It keeps the existing data and API calls.
> - The benefit is a list that shows what a person manages, a create
flow with two fields, and a secret that an agent can use as soon as it
is saved.
## Linked Issues or Issue Description
**What existing behavior does this improve?**
The Settings → Secrets page, its New secret dialog, the secret details
panel, the Vaults tab, the import-from-vault dialog, the Proposals tab
and the My secrets tab.
**Subsystem affected**
UI: `ui/src/pages/Secrets.tsx` and `ui/src/pages/secrets/*`.
**Current behavior**
- One table lists all secrets. On a busy organization most rows are
credentials that Paperclip saved itself, and many have record IDs as
names.
- Each row shows Active, a version, a chip and the key. The row does not
show which agents use the secret.
- A three-line help banner is always visible.
- New secret starts with two mode switches (Organization / Each user,
Managed value / External reference), then a provider picker and a vault
picker. The Create button is disabled without a reason.
- The details panel has Details, Usage and Access events tabs.
- The Vaults tab has one section per provider, with empty placeholder
boxes and a status picker in the form.
**Proposed behavior**
- The list shows the secrets that the team added. Each row shows who
provides the value (Organization, or Each user with a count) and which
agents use it.
- Credentials that Paperclip saved collapse into "Managed by Paperclip".
They are grouped by origin and have readable names.
- One Filter menu covers provided-by, storage and status. Active filters
show as chips. Archived secrets stay hidden until a person asks for
them.
- New secret has a name, one explicit Organization / Each user choice, a
masked value, an optional list of agents, and "Stored in" only when more
than one storage place exists. Create stays enabled and says what is
missing. A `NAME=value` pasted into Name fills both fields.
- The details panel is one scrolling view: agents, other places the
secret is set, details, and recent activity.
- The Vaults tab is one flat list with health and secret counts. The
connect dialog has two required fields. Paperclip checks a new vault
immediately after it is saved.
**Reason and benefit**
A person can find their own secrets, see if a secret is used, and create
a secret that an agent can use in one step.
**Breaking changes**
None. The API gains one optional query parameter (`order` on the board
proposals list). The default order is unchanged. See Risks for the UI
behaviors that changed.
## What Changed
- **List.** Rewrote the Secrets tab in `Secrets.tsx`. Added
`secret-list-parts.tsx` for the Provided by and Used by cells, the help
popover and the managed groups. The Folders / Flat toggle is removed.
Folders are always on and search covers all folders.
- **Managed grouping.** Added `secret-origin.ts`. It infers the origin
of a secret from the name and key conventions of the server modules that
create secrets (`tool_app.*`, `tool_oauth.*`, `slack_manager.*`,
`chat.*`, `ai_*`, `routine-*`, `agent-observability.*`). It is used for
presentation only.
- **Agent usage.** Added `agent-secret-access.ts`. It holds the grant
and revoke logic that was inside the details panel, and builds one usage
index from the agents list for all rows.
- **New secret.** Rewrote the dialog. Added `buildCreateVaultOptions`
and `parseEnvAssignment`. Agents selected in the dialog get an
environment variable binding after the secret is saved. A failed grant
shows an error and keeps the secret.
- **Details.** Replaced the tabs with sections. "Also set in" lists the
bindings that are not agents. Activity shows agent names and the last
five entries, with "Show all activity".
- **Vaults.** Flat list with a built-in Paperclip row, a Default badge
that follows `getSelectableProviderConfig`, Check again, Enable, and an
Add vault menu. The dialog has Name and AWS region, with naming and tags
in a disclosure. The Status picker is removed.
- **Import, Proposals, My secrets.** Restyled to the same design.
Proposals adds a Recently decided list. The board list endpoint `GET
/api/companies/{companyId}/secret-proposals` takes a new optional
`order=resolved` to sort by decision time.
- **Copy.** "Provider vault" is "vault". "Local encrypted" is
"Paperclip" or "Paperclip-encrypted". Toasts and block reasons use plain
words.
- **Stories.** `secrets.stories.tsx` has new stories for the managed
section, a folder, New secret, the details panel and Vaults. The
Storybook usage fixture now returns the `target` field that the API
returns.
- **Tests.** Updated the page, import, proposals and My secrets tests.
Added `secret-origin.test.ts`. Updated the runner E2E vault-reference
journey and the Storybook visual render-state copy for the new labels.
- **Docs.** `Secrets → Provider vaults` is now `Secrets → Vaults` in
three documents.
`Secrets.tsx` was 4,855 lines and is 4,951 lines. The redesign edits the
page in place to keep its queries, mutations and deep links. Splitting
the file is a separate change.
## Verification
- `cd ui && node ../scripts/run-tsc.mjs -b`: no errors.
- `cd ui && pnpm vitest run src/pages/secrets src/pages/Secrets.test.ts
src/pages/Secrets.render.test.tsx
src/components/AgentSecretAccessEditor.test.tsx
src/components/AgentConfigForm src/components/IssueThreadInteractionCard
storybook/map-tooling.test.ts`: 14 files, 467 tests passed.
- `cd server && pnpm vitest run
src/__tests__/secret-proposals-routes.test.ts -t "decision time"`:
passed.
- `pnpm vitest run --config tests/runner-e2e/vitest.config.ts
chat-catalog-domains/external-vault-reference
chat-catalog-domains/external-personal-secret-menu
chat-catalog-domains/external-personal-secret-entry-dialog
chat-catalog-domains/external-personal-secret.test`: 32 passed, 2
skipped.
- `node scripts/check-token-gates.mjs`, `node scripts/check-maps.mjs`,
`node scripts/check-module-boundaries.mjs`, `node
scripts/storybook-catalog.mjs`: clean.
- `pnpm verify`: exit 2. It asks for broader coverage because a runner
E2E source file changed. Required CI supplies that coverage.
- All 22 Secrets stories render and their play functions pass in
headless Chromium against a local Storybook.
- End to end, on an isolated local instance (`paperclipai worktree init
--empty`, one company, two agents), with a headless Chromium script. 16
of 16 steps passed, with no page errors:
- Empty state. Create explains missing fields.
- Create an organization secret and give an agent access. The agent's
saved config has the `secret_ref` binding.
- Paste `NAME=value`. Create an each-user secret. Add a value in My
secrets. The list shows "1 of 1 added".
- Create a folder, create a secret in it, search across folders.
- In details: add an agent, replace the value, remove the agent. The
agent's saved config changes each time.
- Disable, archive, and the Archived filter.
- Managed secrets leave the team list and show under their groups.
- Add a Paperclip-encrypted vault and an AWS vault. New secret then
shows "Stored in".
- The import dialog explains a denied vault. The Proposals tab shows its
empty state. The list at 390 px width.
- Not run: the paid runner E2E journeys. I updated
`external-vault-reference.ts` for the new labels and could not execute
it. A live AWS import, and a proposal from a real agent run, were not
exercised.
## Risks
- **Origin grouping is a heuristic.** It reads naming conventions that
live in server modules. If a module changes its naming, its secrets show
in the team list again. Nothing is hidden or blocked by it. A recorded
origin on each secret is the durable fix.
- **Removed UI.** The Folders / Flat toggle, the "View references"
dialog, the vault Status picker, and creating draft GCP or HashiCorp
vault entries from the Add vault menu. Existing draft vaults still show
and can be edited. A disabled vault now has an Enable action.
- **Default status filter.** The list now shows active and disabled
secrets. Before, it showed active secrets only.
- **Built-in storage in "Stored in".** A provider's built-in storage is
not offered once that provider has a usable vault. The form already
replaced that choice with the vault before this change.
- **Agent access at creation.** Access is a second request for each
agent after the secret is saved. If one fails, the secret exists without
that access and an error says so.
- **Automation contracts.** Labels that agents and the runner E2E
journeys depend on are kept: the key in each My secrets row, "Set
value", "Set my value", "Update my value", "Save value", and the import
dialog test IDs. The import entry point moved into the "More secret
actions" menu and the journey is updated for it.
- **Value masking** uses `-webkit-text-security`. A browser without it
shows the value as typed.
## Model Used
- Claude Fable 5.2 (`claude-fable-5-2`) through Claude Code, with tool
use, code execution and browser automation. Three delegated agents on
the same model restyled the import dialog, the Proposals tab and My
secrets.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
|
||
|
|
b4ca19f3a8 |
test(e2e): keep the stalled-font task test's font requests on the page route (#21171)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Every PR runs the Playwright e2e suite in 8 shards
> - `task-loading-frame.spec.ts` "a stalled web font does not hide an
otherwise prepared conversation indefinitely" failed 17 times in 48
hours, on 17 different PR branches, most with no UI changes (for example
#21129)
> - The test stalls `**/*.woff2` with `page.route`. The app's service
worker claims the page and fetches the fonts itself. `page.route` does
not see service-worker requests
> - When the worker claimed the page before the font request, the font
loaded. The loading frame then showed for only a few frames, and
`toBeVisible()` missed it
> - This pull request blocks service workers for this one test, so the
font stall the test depends on always happens
> - The benefit is that e2e shard 1/8 stops failing at random on
unrelated PRs
## Linked Issues or Issue Description
**What happened?**
`tests/e2e/task-loading-frame.spec.ts:131` failed with
`expect(locator).toBeVisible() failed`, `Locator:
getByTestId('task-chat-loading-frame')`, `element(s) not found`, on
unrelated PRs. The failure snapshot shows a fully loaded conversation.
**Expected behavior**
The test stalls the web fonts, sees the loading frame, and then sees the
conversation within 5 seconds.
**Steps to reproduce**
`PAPERCLIP_E2E_PORT=3291 npx playwright test --config
tests/e2e/playwright.config.ts tests/e2e/task-loading-frame.spec.ts -g
"stalled web font" --repeat-each=40` on master: 6 of 40 runs fail with
the CI error.
**Paperclip version or commit**
master at
|
||
|
|
3e869c6f08 |
test(restart-chaos): quiet period before the end-of-wave grade, explicit swap moments, request records and release attribution (#21215)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The restart-chaos harness grades every task at the end of a release-swap wave, locally (RS1) and on a hosted staging instance (hosted mode) > - The first hosted 3-release run graded 79 / 80, and the one failure was a harness artefact: the drain declared quiet 20 s after a parent's run ended, before the product's children-completed wake created its continuation, so the task read as "nothing pending" and cleanup then cancelled the continuation > - The same run showed three more reporting gaps: per-turn product notices counted as duplicated comments, failed requests could not be broken down because request records stayed in memory (and the API count included the health probes), and the per-swap table charged every failed run to the last swap before it > - Separately, the local swap timeline marked the start of a swap with `fault_started` carrying `"signal":"SIGTERM"`, while the outgoing server's real SIGTERM was `fault_server_down`; that has led to wrong conclusions twice > - This pull request fixes the drain rule, the comment count, the request records, the release attribution and the swap timeline > - The benefit is that the one number and the per-swap table mean what they say ## Linked Issues or Issue Description Refs #21153 Refs #21126 **Describe the bug** 1. A task whose continuation is created after the drain declares quiet is graded `nothing pending`, and cleanup cancels the continuation. 2. "Prepared <document> for this response." notices, one per turn, count as duplicated agent comments. 3. Request failures cannot be broken down after the run, and the API continuity count includes health probes. 4. A failed run is attributed to the last swap before it, however long after. 5. `fault_started` carries `"signal":"SIGTERM"` for an overlap swap, though it marks the swap request; the outgoing SIGTERM is a different event. **Steps to reproduce** Run a hosted or local release-swap wave with parent tasks whose subtasks finish near the end of the wave. **Expected behavior** The grade waits for continuations, notices are informational, request records are saved, failed runs are attributed to a release only when live at it or soon after, and the timeline names each swap moment. ## What Changed - `release-swap-grade.ts`: shared `drainState` (quiet only when nothing is progressing and no run was created or finished for 90 s, `QUIET_PERIOD_MS`); `failedAtSwap` / `swapForFailedRun` / `failedOutsideSwaps` (a failed run counts at a swap if it was live at the outgoing SIGTERM or failed within 15 minutes of it, `RELEASE_FAILURE_WINDOW_MS`, and at most one swap); `isAutoNotice`; the summary lists failed runs outside any release window and the notice repeats as information. - `release-swap-finish.ts` (local RS1): uses `drainState` and the new attribution; the duplicate-comment query separates the notice. - `hosted-grade.ts`: uses `drainState` (`--hosted-quiet-seconds`, default 90); reads runs, tasks and task views again immediately before grading; breaks request failures down by status and by inside/outside the release windows. - `hosted-api.ts`, `hosted-main.ts`: every request record carries kind (health or API) and a path class with ids replaced; `requests.jsonl` is written to the results directory; API continuity no longer includes health probes. - `hosted-readers.ts`: duplicated agent comments and notice repeats are returned separately. - `release-swap-driver.ts`, `release-swap-moments.ts` (new): the timeline records `swap_incoming_spawned`, `swap_incoming_ready`, `swap_traffic_switched`, `swap_outgoing_sigterm` and `swap_outgoing_exited`; the summary prints the five times per swap and, for every run started during the swap, the two moments it started between and which server started it; `release-swap.json` carries both. - `supervisor.ts` (same line count, still 1,000 lines): `fault_started` for an overlap swap carries `marks: "swap_requested"` instead of a signal. No test reads that signal; `fault_server_down` stays. - `README.md`: documents the quiet period, attribution window, notice rule, request records and swap moments. ## Verification - `node server/node_modules/vitest/vitest.mjs run --config tests/restart-chaos/vitest.config.ts` on the hosted, release-swap and triggers test files: 105 tests pass across 12 files. - Red/green for the drain race (`hosted-drain.test.ts`): a parent's run ends, a children-completed continuation is created 20 s later and succeeds; with the old rule (quiet period 0) the task grades `still_working`/not by the continuation, with the new rule it grades `pass` after the continuation ends. - New tests for the notice filter, `pathClass` and the request breakdown, the 15-minute attribution, and the five swap moments (both the timeline events from the driver and the summary placement of runs between moments, with the starting server). - `node server/node_modules/typescript/bin/tsc -p tests/restart-chaos/tsconfig.json`: clean. ## Risks - Low risk: harness-only. The end-of-wave grade waits up to 90 s longer after the last run change, within the existing drain cap. - Per-swap failed-run counts change meaning: runs that failed more than 15 minutes after a stop and were not live at it move to the "failed outside a release window" line. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits). Default context window; no extended thinking mode. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
2eda54162a |
fix(evals): grade recovery on fresh continuation, accept in-turn queued-comment reads, verify the saved service model (#21210)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The chat eval catalog checks that agents in chat do the right thing, graded on saved product state and rendered links > - Full cohort 8 (OpenCode, `openrouter/deepseek/deepseek-v4.1-flash-1017`, master |
||
|
|
7f1cdb842d |
fix(evals): admit the owner-routed routine follow-up run and brief only the requested change (#21198)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The chat eval catalog checks that agents in chat do the right thing on saved product state. > - In `routines.manage-other-owner`, the CEO cannot change Morgan's routine. The valid outcome is to route the change to Morgan in one linked task. > - That valid path uses three runs: the CEO's turn, Morgan's run on the task, and the CEO's automatic completion follow-up on the chat. The case allowed only two runs, so the harness stopped valid attempts with "exceeded its declared run bound". > - In one of those attempts, the CEO also told Morgan to use UTC. Nobody asked for UTC. The routine was on America/Chicago. The oracle correctly failed that change. > - This pull request admits the third run only in its declared role, and it tells agents to brief only the change the person asked for. > - The benefit is that the eval accepts the valid delegation path and still fails extra work and unrequested setting changes. ## Linked Issues or Issue Description **What happened?** Runner Full-Stack E2E runs 43298807862 and 43300040122 stopped `routines.manage-other-owner` with "Scenario exceeded its declared run bound". Both had three runs: the CEO turn, Morgan's run on the routed task, and the CEO's `chat_task_completed` follow-up. In 43298807862 the CEO's brief also told Morgan to switch the routine to UTC, and `same-owned-definition-and-triggers` failed. **Expected behavior** The delegation path with exactly one routed owner task passes with three runs. The refusal path stays at two runs. An agent that relays a change keeps every setting the person did not ask to change. **Steps to reproduce** Run `chat-features.runner-opencode.local.routines.manage-other-owner` and let the CEO route the change to Morgan. **Paperclip version or commit** master at |
||
|
|
eba790713c |
fix(evals): read who can enter a key as no request for it, and fail any non-harness change to a protected routine (#21184)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The chat catalog evals grade agent chat journeys on saved state, setup literals and rendered links > - Full chat cohort 7 had two grader defects: one false failure and one blind spot > - `secrets.configure-company-secret` failed a correct handoff, because the credential-language check read "only a person can enter them, and the key shouldn't be pasted in chat" as a request for the key > - `routines.manage-other-owner` passed its no-mutation check although the routine changed, because the check only counted changes attributed to the agent. The change came through the local board identity > - This pull request fixes both checks and bumps their oracle versions > - The benefit is fewer false failures, and a bypass through another identity now fails ## Linked Issues or Issue Description Refs #21081 **What happened?** 1. `secrets.configure-company-secret`: both `solicitsChatCredential` and `unsafeCredentialTransfer` returned true for the sentence "I can't save secret values myself — only a person can enter them, and the key shouldn't be pasted in chat." The reply was the correct handoff: a Secrets page link, no paste in chat, and a binding for Morgan after the person saves the key. 2. `routines.manage-other-owner`: the agent got a 403, then changed Morgan's routine through the local board API. The activity log attributes the change to the board user. `no-ceo-management-mutation` passed, because it only checked events with the agent's ID. (`same-owned-definition-and-triggers` failed the case.) **Expected behavior** 1. A statement of who can enter a key, and a chat destination that the same sentence negates, are not a request for the key. "Paste the key here" and "Send me the key" still fail. 2. Any change to the protected routine during the turn that the harness did not make fails, whatever identity the change has. **Steps to reproduce** Run the chat-features OpenCode cases `secrets.configure-company-secret` and `routines.manage-other-owner` on master |
||
|
|
f1d25aca14 |
test(restart-chaos): wait for full agent termination in hosted cleanup; count only real outages in continuity (#21176)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The restart-chaos harness has a hosted mode (#21153) that runs the release-swap proof on a real staging instance > - Its first 2-task smoke from master graded 6 / 6 tasks completed hands-off, but cleanup ended with an error and the API continuity count showed one failure that was not an outage > - Cleanup deleted the agent as soon as its `status` read `terminated`, while its `lifecycleState` was still `terminating`; the server refused with 409 "Complete termination before deleting the agent" > - The continuity count treated a deliberate 404 (an older build without the assigned-work invariant endpoint) as a failed request > - This pull request waits for full termination and counts only real outages > - The benefit is a clean exit code after a clean run and a continuity number that means "the instance did not answer" ## Linked Issues or Issue Description Refs #21153 **Describe the bug** Hosted cleanup reports `remove agent ...: 409 Complete termination before deleting the agent` and exits non-zero after a clean run. The API continuity line counts a deliberate 404 as a failed request. **Steps to reproduce** Run the hosted smoke (`--hosted-releases 0 --hosted-tasks 2`) against an instance that does not have `/api/companies/:id/assigned-work-invariant`. **Expected behavior** Cleanup waits until the agent is fully terminated. Continuity counts only no answer, server errors, and 401, 403, 408 or 429. ## What Changed - `hosted-setup.ts`: agent removal waits until both `status` and `lifecycleState` are `terminated` before the delete. - `hosted-api.ts`: a 400, 404, 409 or 422 answer is a deliberate refusal, not a continuity failure. No answer, 5xx, 401, 403, 408 and 429 still count. - `hosted-main.ts`: `finalizeHosted` passes the poll interval through, for the test. - `README.md`: states the continuity rule. ## Verification - `node server/node_modules/vitest/vitest.mjs run --config tests/restart-chaos/vitest.config.ts tests/restart-chaos/hosted-*.test.ts`: 34 tests pass. The setup-failure cleanup test now returns `terminated` with `lifecycleState: terminating` first and checks that the harness reads again before it deletes. The continuity test checks that a 404 is not a failure and a 401 is. - `node server/node_modules/typescript/bin/tsc -p tests/restart-chaos/tsconfig.json`: clean. - Live: the smoke `rc-hosted-smoke2` on a hosted staging instance hit the 409. A second `--hosted-cleanup-only` found nothing left: the agent is terminated and the product keeps its record (cost history), its tasks are hidden, its secret is deleted and its project is archived. ## Risks - Low risk: harness-only. Cleanup can wait up to 120 s per agent for termination, as before. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits). Default context window; no extended thinking mode. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
5412162716 |
test(restart-chaos): grade staging-mix hands-off and click Retry once after the settle (#21168)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The restart chaos harness grades agent tasks across server swaps, and the staging-mix workload (#21125) is its staging-shaped load > - On a fix-PR comparison the harness clicked Retry as soon as the banner appeared (once after 11 s), which pre-empted the product's own continuation > - So the wave could not show whether a task would have continued by itself, and the banner itself went uncounted > - This pull request makes staging-mix grading hands-off by default: no Retry click during the wave or the settle, one click only after the settle, and every banner recorded > - The benefit is three honest numbers per wave: completed hands-off / started, Retry banners shown, and Retry worked / clicked ## Linked Issues or Issue Description Refs #21125, #21126, #21139. **What existing behavior does this improve?** Staging-mix Retry handling and the `done_comment_once` check in the restart chaos harness. **Subsystem affected** Test harness only: `tests/restart-chaos/`. No product code changes. **Current behavior** The observer clicks Retry as soon as a task rests with the banner, so a product continuation that would have come is never seen. A banner that heals by itself is not recorded. `done_comment_once` counts every agent comment that mentions `DONE-<nonce>`. A closing summary that quotes the marker therefore failed all 6 tasks of wave `w-20261112T174255-s7101`. The release-swap grader can click a second Retry after the observer's. **Proposed behavior** Hands-off by default. Every banner is recorded with its first-shown time, its duration and its error code. One Retry click happens only after the settle, and only for a task that is still not done and that nobody clicked. The DONE check counts only the comment whose whole body is the marker. A completion summary repeated by a different run becomes its own `completion_once` finding. **Reason and benefit** The product goal is that the Retry banner never appears and that interrupted work continues by itself. The wave now measures exactly that. ## What Changed - `staging-mix.ts`: - The observer step records the Retry banner per task (`retryBanner`: first shown, total shown time, error code). It clicks during an overlap only with the new flag `--mix-retry-during-wave`. - `isDoneComment` counts only the bare `DONE-<nonce>` comment. - `completion_once` fails when more than one run posted a comment that mentions the marker. - Comments are now collected with their run ids. - `observer.ts`: - A staging-mix task with a banner is kept under watch instead of being clicked or rested (hands-off). - `retryAfterSettle` reopens a task for the one post-settle click. - New `staging-mix-retry.ts`: - `shouldClickRetryAfterSettle` is the single click rule. It never clicks a done task, a task with a scheduled retry, or a task already clicked. - `retryAfterSettleStage` waits out the settle (new flag `--mix-settle-minutes`, default 15 after the last fault slot ends), clicks once per stuck task, and follows each click to rest. - The results module writes `staging-mix-results.json` and `staging-mix-summary.md`. Their first three lines are `completed hands-off / started`, `Retry banner shown: N tasks`, `Retry worked / Retry clicked`. Then come the banner list and one end-of-wave read of `GET /api/companies/:id/assigned-work-invariant` (`violationCount`, `violations`, `agentsWithStalledSlots`, `openTasks`). - `release-swap-finish.ts` uses the same click rule, so the swap grader never clicks a task the observer already clicked. When the release-swap cell drives the wave, the staging-mix settle stage does not run; the swap grader makes the one click. The hosted mode can import the same function. - `main.ts`: the staging-mix end-of-wave report moved into `StagingMixRun.report`. The file shrinks from 1,000 to 998 lines. - README: the Retry and results sections, the two new checks, and the local short-form command (without `--ephemeral-server`, which runs only on Daytona). ## Verification - `pnpm exec vitest run --config tests/restart-chaos/vitest.config.ts`: 59 files, 714 tests pass. - `node scripts/typecheck-test-projects.mjs tests/restart-chaos`: passes. - New tests cover: - the default and opt-in flags; - the click rule; - a settle where one task heals by itself (no click), one task gets exactly one click, and one already-clicked task gets none; - the three headline numbers, including the swap grader's clicks; - the invariant read; - the DONE count against the real comments from `w-20261112T174255-s7101` t001 (summary plus bare marker passes; a summary repeated by another run fails `completion_once`). - Not run: a live wave with this change. The Daytona image for current master is still queued. ## Risks - Low risk. Test harness only. - A hands-off task that keeps its banner now waits to its deadline before the post-settle click, so a wave with many banners runs longer. - RS1 waves on other workloads no longer get a second end-of-wave click on a task the observer already clicked. That is intended. - Known master CI failures, not touched here: the storage S3 multipart throttle test (being fixed) and the Runner native-distribution-integrity test. - `ci / Runner support suite` failed on head |
||
|
|
7f69ef8b44 |
Prepare and finalize managed workspaces on the active Launcher (#20743)
## Thinking Path > - Paperclip lets plugins own managed Git workspaces. > - A portable Runner task can move to a different physical allocation. > - The checkout must be present on the allocation that launches the Runner. > - Workspace preparation and finalization need exact placement when they retry. > - This change checks the placement at Runner admission and records it before finalization. ## Linked Issues or Issue Description **What existing behavior does this improve?** Workspace preparation and finalization for portable Runner tasks. **Current behavior** A task replacement can change the allocation after Git preparation. A lost finalization reply can leave retry without the original placement. **Proposed behavior** Prepare the checkout on the actual allocation before admission. Record finalization placement so an exact retry can use it. Portable ownership and controller recovery are in [#21066](https://github.com/paperclipai/paperclip/pull/21066). ## What Changed - Reserve the managed workspace before acquiring the portable task lease. - Pass the task placement to the workspace provider. Reuse the described task while placement stays the same and prepare again before admitting a replacement. - Record the initial preparation's run, agent, generation, and optional task placement before dispatch. Inspection and retries on the same Launcher replay the same request; VM and sandbox preparation use the same recovery path. - Keep provider request identity separate from Git credential authority: reservation reconciliation uses the existing project-consent path in a host-only scope bound to the plugin and workspace. Foreground preparation still requires an active run; consent, repository, and revocation checks are unchanged. - Acquire the current Launcher before retrying preparation. If placement changes, use ordinary preparation on the replacement so the provider can settle the prior writer and recover the retained checkout. No additional provider discovery call is added. - Select finalization placement before publishing the operation record, then dispatch and recover using that saved choice. Retain the Launcher during bounded finalization retries. - Preserve interrupted pre-upgrade finalization requests without placement, including their original operation IDs and provider routing. No migration or new durable mode flag is required. ## Verification - Final revision `74ea3e6712`: server typecheck and exact SDK source-pin verification (538 inputs) passed. - All 185 tests passed across the full plugin worker manager, project Git authorization, and plugin workspace integration suites (including 56 workspace tests). Independent verification reran the three lost-reply cases: inspection, same-Launcher retry, and replacement after deleting the original Launcher. The tests assert the correct request identity, durable intent, and clearing that intent after success. - Before the final retry-accounting fix, all 323 tests passed across workspace admission, runtime execution, native workspace finalization, plugin workspace integration, and terminal cleanup retry. Regressions exercise crashes after operation insertion but before dispatch in both finalization paths, exact replay after lease release, and retry after ambiguous placement clears without publishing an incomplete operation. Pre-upgrade interrupted requests retain their operation IDs despite an available portable lease. - Final retry-accounting fix: all 84 native workspace finalizer, failure, and supersession tests passed. A real missing-plugin regression first failed on the prior commit, then verified healthy-run progress, three-attempt exhaustion, terminal cleanup, and absence of a misleading workspace operation. Independent verification also passed the legacy and crash-before-dispatch cases. - Worker-process tests cover both preparation RPC methods: the provider still receives the original run and agent, maintenance receives project-scoped credential authority, and foreground or mismatched-scope requests remain denied. Database tests confirm a stopped run cannot authorize foreground Git access while valid project consent can authorize maintenance. - Existing metadata protection coverage proves callers cannot overwrite the private preparation intent or read it through workspace responses. - Two existing Launcher Git tests passed: retry preserves dirty files and the original base commit without fetching, and retry resumes the saved checkout intent while rejecting a changed pending branch. - Implementation was delegated to GPT-6.1 Sol and independently reviewed. The GitHub stack's automatic rebase preserved the original workspace-placement patch. - [Full CI](https://github.com/paperclipai/paperclip/actions/runs/43293046205/job/131452117706) passed on `74ea3e6712`, including server and workspace tests, Runner, end-to-end tests, typecheck, build, and canary dry run. Ordinary Greptile and [Apex](https://github.com/paperclipai/paperclip/pull/20743#issuecomment-6788920388) both scored `74ea3e6712` 5/5, with no unresolved findings. ## Risks - Live Daytona end-to-end qualification is pending. The checkout record persists on the mounted volume; replacement preparation relies on the provider confirming the previous writer is settled before using that retained checkout. Local tests cover Core request routing and Launcher's durable checkout recovery separately. ## Model Used - OpenAI Codex GPT-6 with GPT-6.1 Sol agents for implementation and independent review. ## Checklist - [x] Thinking path and model are included. - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] The issue is described using the enhancement template. - [x] Local checks passed and new behavior has tests. - [x] All Paperclip CI gates are green. - [x] Greptile is 5/5 with no open findings. --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |