mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-17 13:01:05 +01:00
784bc06276d93c608ea7ee082e2f5cb5e8e8fe26
874
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
d258f478fc |
test(ci): stop the live-run census hanging server shards, and name any stalled test file (#21941)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - PR CI splits the server tests across 12 shards; `ci / verify` fails when any shard fails > - Since 2026-11-16 the server shards were cancelled at their 20-minute job limit with no output (shard 2/12 five times, plus shards 1, 3 and 9), which blocked unrelated pull requests until reruns > - Turbo prints a task's output only when it ends, so a cancelled shard never names the stuck file > - Recomputing each cancelled shard's file list at its own merge commit put `issue-recovery-live-run-agent-census.test.ts` in 12 of 14 of them; it is also the slowest file in passing shards (235-300 s) > - That census started 9-14 real agent runs per file while `setTimeout` was faked, waited on them with no bound, and had 30-minute test timeouts, so a stuck wait could never fail inside the job > - This pull request stops the census from starting real runs, bounds every wait, and adds a CI reporter that fails a run naming any file that stops making progress > - The benefit is that this file cannot hang a shard, and the next stuck file fails in minutes with its name and step in the log ## Linked Issues or Issue Description Refs #21867 (the same fix for the sibling agent-row census), #21831 (the behavior this census guards) **What happened?** `ci / General tests (server (2/12))` was cancelled at 20m16s in actions runs 43865258195, 43926799428, 43932238377, 43934278368, 43944290677 and 43935722497 (both attempts), and other server shards hit the same limit (for example 43901003914 shard 1, 43945768035 shard 3, 43845343831 shard 9). The hung jobs show CPU near 0 for the last 9 minutes and printed nothing after `Running //#test:run:general`. Shard membership changes with every merge commit. To compare the shards, I recomputed each job's file list at its exact merge commit with `GITHUB_WORKFLOW=PR` (the PR workflow removes the native-runner suites from these shards, so a local dry run without it gives a different list). `server/src/__tests__/issue-recovery-live-run-agent-census.test.ts` was in 12 of the 14 hung shards. Why that file: - During its 10-minute fake-timer warm-up, the "live" task was `in_progress` with no run. That is real assigned work, so recovery woke its agent and a real `process`-adapter run started while `setTimeout` was faked. The run lost its lease ("Legacy controller lease lost"), recovery retried, and a local run of the file showed 13 real run executions. - No wait in the file had a bound, and each test had a 30-minute timeout, so a delivery pass waiting on a faked timer or a run holding a lock never failed inside the 20-minute job. - Under local CPU contention, 3 of 4 copies of the old file failed the census assertion (39 settled checks, caused by the stray runs), and each copy took about 270 s with 9-14 real runs. Fixing the runs exposed a second problem in the fixture: the seeded tasks were stamped "now". The first 5-minute backstop after the live run's first hint re-checks rows changed within 5 minutes before the last full pass, so it re-checked all 860 seeded tasks. The old file passed only because the stray runs happened to trigger that backstop during the warm-up. **Expected behavior** The census never starts a real run, every wait has a bound that names what it waited for, and a CI test file that stops making progress fails the job quickly with its name. ## What Changed - `issue-recovery-live-run-agent-census.test.ts` - The live task is a settled review until its run starts; `startLiveRun` now writes the checkout (`in_progress` + `executionRunId`) the way execution start does. - Population agents and the held agent take no on-demand wakes (`wakeOnDemand: false`), so a recovery check cannot dispatch a run. The census counts queue checks, so this does not change what it measures. - The seeded population is backdated 2 hours ("settled long before the window", as the file's comment already said). - Each step (seed, worker start, run start, run-event tick, fake-timer advance, pass settlement, direct writes) has a real 20-second bound captured before the timers are faked, with a message naming the step. Test timeout is 4 minutes instead of 30. - New `scripts/ci/stalled-file-reporter.mjs`, enabled in GitHub Actions from the root `vitest.config.ts`: when a running file starts no test or hook and reports no result for 8 minutes (`PAPERCLIP_VITEST_FILE_STALL_MS` overrides), it prints a `::error file=...` annotation naming the file and the step it was in, cancels the run and exits 1. The slowest healthy server file runs about 5 minutes in total, so 8 minutes without any progress means a stuck file. Tests: `scripts/ci/stalled-file-reporter.test.mjs` (added to `test:release-registry`). ## Verification - Census, red: the old file runs 13 real agent runs locally (`heartbeat execution setup failed`, `LegacyControllerLeaseLostError`); with 4 copies under CPU contention, 3 failed with 39 settled checks at about 270 s each. - Census, green: 4/4 tests pass, 0 real runs, about 80 s locally. - Census still guards #21831: with the `ag.status <> 'running'` line removed from `changed-tasks.ts`, the first test fails with 534 settled checks and 540 backstop checks, the numbers #21831 recorded. - Reporter, red/green: the real-Vitest case (a file awaiting a promise that never settles, with a 10-minute test timeout) is still running when the 20-second test bound ends without the reporter; with the reporter it exits 1 in about 8 s and prints the annotation naming the file and the test. - Reporter on a real server file: `PAPERCLIP_VITEST_FILE_STALL_MS=3000` on `issue-recovery-recorded-decision-idle.test.ts` prints the annotation naming the file and step and exits 1. - `node --test scripts/ci/stalled-file-reporter.test.mjs`: 5/5 pass. ## Risks - Test and CI-tooling change only. No product code changed. - The census's agents no longer take on-demand wakes. The census measures recovery queue checks, not dispatch, and still fails without the #21831 fix. - The reporter stops the whole CI run when one file stalls for 8 minutes. A legitimate single test or hook that runs more than 8 minutes with no other progress in its file would now fail; none exists today (the slowest whole file is about 5 minutes). The limit is configurable. - Other fake-timer recovery files still have long test timeouts and unbounded pass waits (for example `issue-recovery-activity-hints`, 30 minutes). The reporter now caps any of them at 8 minutes without progress and names the file; bounding them is follow-up work. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
f33c4cf6f7 |
feat(links): cache rich link status in the database and load it on idle (#21746)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task text contains links to pull requests and other work. > - A link has useful status that should be visible when a task opens. > - The current cache lives in the server process and the browser. > - A new browser or server restart has no stored status to show. > - This pull request stores link status in the database and reads it in batches. > - Idle page work updates stale status without a hover or a delay to task data. ## Linked Issues or Issue Description **Problem or motivation** PR status is unknown in a new browser until the user points to each link. The process cache cannot supply saved status after a server restart. Related work: #20593 and #20654. A search found no other PR for database link caching. **Proposed solution** Store status by company, viewer and canonical resource URL. Read saved status in one indexed batch query. After document load and core request settlement, use browser idle time to update missing or stale status. Use the same query for hover and idle work. **Alternatives considered** Adding provider calls to task reads would delay task loading. Company-wide reuse of personal GitHub results would cross viewer access boundaries. A periodic server worker would create work when no page needs the result. **Roadmap alignment** This extends the shipped rich-link feature. It does not duplicate a roadmap project or add a new provider integration. ## What Changed - Add an additive, repeat-safe `link_previews` migration. Store channel, resource kind, free-form status, JSON metadata and freshness. The table has no task relationship or GitHub status enum. - Bound each company/viewer cache to 1,000 rows across concurrent server instances. Remove rows expired for over 30 days during writes. Limit provider lookups to 600 requests per minute per scope/process. Keep cached batch reads available during backoff. - Add a database store owned by the link service. Reuse canonical results across tasks. Keep company and viewer isolation. Save hover and idle results. Preserve last known status during an outage and cache unavailable results. - Add a board-only batch GET endpoint, including viewer-role access. It reads at most 50 URLs with one indexed select and makes no provider calls. Both endpoints use private, no-store HTTP responses. - Add a page coordinator for mounted rich links. Reuse browser data, share pending reads, and defer provider work until document load, core request settlement and browser idle time. Limit automatic work to 200 unique links and three requests in flight. Remove queued work when links leave the page. Keep encoded batch URLs below 6 KB. - Keep exact navigation URLs, status styling, anchored cards and keyboard access. Add a page-load Storybook case and browser regression. The link service owns cache policy and provider calls. The browser coordinator owns page timing. The shared contract adds only the batch limit and response shape. The database adds storage because process and browser caches cannot serve a new browser session. Plugin resolver registration is outside this change; storage accepts future provider channels and status values. ## Verification - **All 93 affected tests pass:** 40 link service, real PostgreSQL and HTTP cases, plus 53 OpenAPI cases. Two SQL regressions and one HTTP case failed before the storage repair. Concurrent writers now retain at most 1,000 rows per company/viewer; cleanup removes long-expired rows and keeps recent stale status. Request backoff preserves cached reads and isolates companies/viewers. The earlier eight concurrent hydration regressions also pass. Focused store/service/unit types pass. Commands: `pnpm exec vitest run server/src/__tests__/link-previews.test.ts server/src/__tests__/link-preview-store.integration.test.ts server/src/__tests__/link-preview-routes.test.ts --maxWorkers=1` and `pnpm exec vitest run server/src/__tests__/openapi-routes.test.ts --maxWorkers=1`. - **All 50 affected server tests passed before the later server-only repairs** on the master-integrated source: 20 recovery/deadline cases, one authenticated review-counts HTTP case, 10 link-service cases, 13 real PostgreSQL cache cases and six link-route cases. Command: `pnpm exec vitest run server/src/__tests__/issue-recovery-deadlines.test.ts server/src/__tests__/tool-connection-review-counts-http.test.ts server/src/__tests__/link-previews.test.ts server/src/__tests__/link-preview-store.integration.test.ts server/src/__tests__/link-preview-routes.test.ts --maxWorkers=1`. - Reproduced the CI watchdog assertion locally twice before the test repair. Ordinary assignment recovery can win `wakeReason` while the watchdog wake coalesces into the same successful run with its full mandate. The test now verifies the persisted watchdog identity, watched issue and stopped fingerprint, plus exactly one review run. All 21 recovery/HTTP cases passed before integrating master; all 50 affected cases pass after integration. No runtime behavior was changed for this repair. - **Fresh static Storybook build and all 22 browser cases pass**, with two workers and retries disabled. They cover states, anchored hover, keyboard access, exact navigation, duplicate requests, reload persistence, expiry and saved-status hydration followed by one idle refresh. The first browser attempt could not launch because this session lacked Chromium; installing the expected browser resolved setup. That attempt had no behavioral verdict. - The cache repair remains covered by four PostgreSQL red/green regressions for expired memory and newer durable status during provider failure/timeout, plus check-time precedence and failed-read recovery. Earlier UI/database/focused server type and migration safety/compatibility/replay checks remain unchanged-input evidence. - Master `77ddf260bf9a3de3dcb8456a60f4f809d67401fd` is integrated cleanly. Link migration 0489 and its snapshot chain are unchanged. SDK source validation checks 545 inputs with no stale source identities or changed runtime/lock pins. Whitespace validation passes. - `pnpm verify --explain` returns exit 2 and requires broad CI. No full local suite or deployed acceptance is claimed. Earlier local full-server type checks exceeded memory; current-head CI supplies broad type/build/test coverage. - [Previous CI 43900336895](https://github.com/paperclipai/paperclip/actions/runs/43900336895) passed full types/build but failed on embedded PostgreSQL startup (`ECONNREFUSED`), the watchdog assertion above, and unrelated plan-task browser cleanup (activity-log foreign key on company deletion). One failed-jobs retry was requested. Its browser shard 7 now passes without code changes; the exact deletion-race cause remains unproven. The watchdog assertion is repaired in this revision. The previous failures remain recorded, not counted as an overall pass. - [Prior CI 43905482848](https://github.com/paperclipai/paperclip/actions/runs/43905482848) passed full types, build and all eight browser shards. Several test jobs were cancelled; the run has no overall pass. Its concurrent update finding is repaired. [Next CI 43909197530](https://github.com/paperclipai/paperclip/actions/runs/43909197530) also passed types, build and all browser shards, but several server jobs were cancelled. Greptile and Paperclip Review gave that revision 5/5. Its security scan found unbounded retained cache rows. The quota, retention and request-limit repair above addresses that finding. All ten review threads are resolved. - [Current CI 43912947278](https://github.com/paperclipai/paperclip/actions/runs/43912947278), fresh code reviews and a fresh security scan are pending on `335d89ef1668dcdeb66bb2059512a7c2da29e273`. Auto-merge is disabled until the current revision clears its gates. Published head: `335d89ef1668dcdeb66bb2059512a7c2da29e273`. Local head: `b37b87c704a2ba8a65233c689a6a743f4011ce14`. Both have tree `643ebd19b9729b1e6a6bb695442269c3db32be3a`; GitHub tree creation verified equality. ## Risks - Each company/viewer retains at most 1,000 rows. Writes remove excess rows and entries whose freshness expired over 30 days ago. Inactive scopes remain bounded and are cleaned on the next write. Company deletion removes all rows. No periodic worker runs. Scope writes use a transaction lock; the existing deadline and concurrency budget cover that work. - Last known status can be stale until idle or hover work completes. Provider failures keep that status through the negative-cache window. - Automatic enrichment adds bounded provider traffic. It runs only for mounted supported links, once per mount, with the existing timeout and server concurrency limit. - Cache storage is viewer-scoped because provider results can be private. New viewers load their own results. This prevents reuse across users with different repository access. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository tools, code execution and browser automation. The runtime does not expose the exact serving model ID or context-window size. No subagents were used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Paperclip <Paperclip-Paperclip@users.noreply.github.com> |
||
|
|
b816c95f96 |
chore(lockfile): refresh pnpm lockfiles (#21936)
## Thinking Path > - Paperclip uses a dependency lock for reproducible builds. > - Dependency changes on master need matching lock entries. > - Paid eval reports also pin a reviewed offline correction. > - Reports must install before paid model work starts. > - This automated PR keeps the lock and report proof together. ## Linked Issues or Issue Description **What happened?** Dependency manifests on master changed after the last committed lock refresh. **Expected behavior** Frozen installs must match those manifests. Eval reports must retain matching offline lock proof. **Steps to reproduce** Run a frozen install before and after this lock refresh. ## What Changed - Refresh the dependency lock from the manifests on master. - Refresh the sandbox provider lockfiles from their manifests on master. - Include the matching offline report correction and hashes. - Resync the SDK public source proof's lock pins from the refreshed lock. ## Verification - Pinned pnpm resolution completed with lifecycle scripts disabled. - The report proof refresh accepted the reviewed committed baseline. - The workflow rejected changes outside the lock and report proof files. ## Risks The lockfile bot merges this PR without CI or review when every added package version is at least 3 days old on npm. Otherwise the PR waits for a maintainer, and the daily refresh tries again. The bot merges only the exact commit this workflow generated from manifests already reviewed on master. Unknown report lock baselines remain blocked. This update changes no permissions or paid enable gates. ## Model Used None — generated by pnpm and the deterministic report-lock updater. ## Checklist - [x] Thinking path and problem description included - [x] Generated lock and matching report proof included - [x] Release-age gate decides between bot merge and maintainer review Co-authored-by: lockfile-bot <lockfile-bot@users.noreply.github.com> |
||
|
|
77ddf260bf |
fix(runner-e2e): keep the supervisor off the heartbeat graph and ship the plugin SDK build (#21860)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The paid Runner Full-Stack E2E workflow proves chat features against a real source server > - Since master |
||
|
|
698c371b1b |
fix(ui): keep new Recents tasks ready to open from cache (#21904)
Track warmed tasks per query client, company, and account so tasks added to the top Recents rows receive the existing idle preload. Preserve the batch limit, cache ownership, and retry-on-revisit behavior. Add unit and browser regressions for later additions, held API responses, editor retention, and message position. Extend controlled performance coverage and document timing results. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
991f773d7f |
fix(ui): align recent task status icons and move fold controls right (#21859)
## Thinking Path > - Paperclip helps people manage AI agents for work. > - The sidebar shows recent tasks and their subtasks. > - A left fold control moves parent tasks away from the status column. > - Status icons must stay in one column so people can scan tasks. > - This pull request moves the fold control to the right. > - The child guide now starts below the parent status icon. ## Linked Issues or Issue Description Follow-up to #21649. Related sidebar alignment work: #20926. Related experimental Recents work: #12314. This change corrects the status column in the shipped subtask group. **What happened?** Parent tasks with subtasks moved 20 pixels to the right. Their status icons did not align with tasks that had no subtasks. **Expected behavior** All top-level task status icons align. The child guide is centered below the parent status icon. The fold control is on the right. **Steps to reproduce** 1. Open a sidebar with a recent parent task and a task that has no subtasks. 2. Compare their status icons. 3. Expand the parent and compare the child guide with its status icon. ## What Changed - Remove the left offset from parent task rows. - Put the fold control in a separate right slot. - Center the child guide below the parent status slot. - Add browser checks for alignment, narrow rows, unread parents, both themes, and separate Classic menu and fold controls. ## Verification - The browser regression failed before the fix: the status icons differed by 20 pixels. - Final browser run: 44 checks passed. It covers 20 states in both themes, folding, paging, unread state, light and dark alignment, and the Classic action menu. - The focused component and request-performance suites passed: 46 tests. - Final source is based on master `29903bed59b1`. UI typecheck and the focused browser-test typecheck passed. The published head is `b8287aec1f55aa182b44622d72b20fe7ed7ce0e6`. Its tree matches the tested local tree `958e8d901ed1fec9ae17c164271eda4d25edca45`. - `pnpm verify --base origin/master` returned exit 2. It requires broad CI because the browser suite changed. Focused checks pass. [CI 43803650911](https://github.com/paperclipai/paperclip/actions/runs/43803650911) passes typecheck, build, all eight E2E shards, policy, Runner checks, all nine serialized server suites, all twelve general server shards, and [final verification](https://github.com/paperclipai/paperclip/actions/runs/43803650911/job/133095032900). All 62 check runs are clean: 57 success, 3 skipped, and 2 neutral. No checks are pending or failed. The legacy Snyk status also passes. - [Greptile review](https://github.com/paperclipai/paperclip/pull/21859#issuecomment-6907272673) is 5/5 on head `b8287aec1f55aa182b44622d72b20fe7ed7ce0e6`. The check passed. There are no review threads or actionable findings. The separate source review also reports 5/5. - Screenshots use production components with HTTP and session fixtures in Chromium on Linux. No deployed-instance or image-baseline comparison is claimed. - No database or API behavior changes. ## Risks Low risk. The change adjusts layout in the shared task group. The fold button remains separate from the task link and the Classic action menu. Exact unread counts and request limits keep their current behavior. ## Model Used OpenAI Codex, GPT-6. Exact deployment model ID was not supplied by the runtime. ## Checklist - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work (this is a sidebar layout bug fix) - [x] I have linked public issues or described the issue in this PR - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and has no internal task ID - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (no command or subsystem behavior change) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
f7a3497e9d |
fix(evals): compare browser zones by clock, wait for the reloaded profile name, bind accounting reads by returned run (#21760)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The chat-feature eval catalog runs real agent conversations and grades the saved outcome > - Chat-eval cohort 16 (OpenCode with a DeepSeek model) had three failures that were not product failures of the chat agent > - One setup check compared time zone names, one observer read a form before it filled, and one grader accepted only exact query digests > - An eval that fails valid behavior hides real regressions behind noise, so each check must grade the outcome, not an incidental detail > - This pull request fixes the three checks, and it fixes a first-frame empty display name on the Profile page > - The benefit is that these three cases now fail only for the behavior they test ## Linked Issues or Issue Description **What happened?** Three chat-eval cases failed in cohort 16 (OpenCode, `openrouter/deepseek/deepseek-v4.1-flash-1017`) on correct behavior. (1) `activity.routine-audit`: setup threw "Browser timezone override did not apply (America/Buenos_Aires)". At 11:00-11:59 UTC the picker selects `America/Argentina/Buenos_Aires`, and Chrome reports that zone by its alias `America/Buenos_Aires`. The check compared names. (2) `navigation-preferences.edit-profile`: the name saved correctly, but the observer read the display-name field once, right after reload, and got an empty value. (3) `budgets-costs.spend-breakdown`: the answer was correct, but the grader accepted company cost reports only for `period=all` or `period=month`. The agent read `costs/by-agent` and `costs/summary` for the requested week, and that range covers the unpriced run. **Expected behavior** Each case passes when the product and the agent do the requested work. Each case fails when they do not. **Steps to reproduce** 1. Run `chat-features.runner-opencode.local.activity.routine-audit` between 11:00 and 11:59 UTC. 2. Run `chat-features.runner-opencode.local.navigation-preferences.edit-profile` and look at `profileAfterReload`. 3. Grade the retained cohort 16 observation of `budgets-costs.spend-breakdown` with the old grader. **Additional context** The run tool inputs are saved only as SHA-256 digests, so a grader binds a read by recomputing the digest of inputs it can check. For the cohort 16 spend-breakdown reads, the inputs were recovered this way: `from=2026-11-09T00:00:00.000Z`, `to=2026-11-16T00:00:00.000Z`. ## What Changed - `internal-routine-audit.ts`: the browser override check compares the UTC offset of the requested and reported zones, now and 182 days later, instead of the zone names. An unknown zone or a different clock still fails setup. - `identity-profile.ts`: after reload, `settledFieldValue` waits up to 15 s for the display name to reach the saved value, then records what the person sees. The grade still fails a name that does not arrive. - `ProfileSettings.tsx`: unedited fields show the session's saved name and image from the first frame. Before, a session restored from the saved read cache rendered an empty name until an effect filled it. - `identity-accounting.ts`: `readSavedAccounting` also accepts a company cost report whose checked `from`/`to` covers the unpriced run's cost event. The run log keeps only the input digest, so the grader recomputes digests for covering ranges: UTC day boundaries (with or without milliseconds or seconds, date-only, end of day), local midnights at each whole-hour offset, and the run's own `startedAt` and cost-event instants. A pending run (no cost event) counts only through `costs/summary`, `costs/data-quality` or its own run record, because the other reports sum cost events. A link to the run does not count, because the run id is in the task description. The run-log redaction is not changed. - Oracle versions bumped: `routine-audit.v7`, `spend-breakdown.structural.v6`, `identity-profile.v4`. ## Verification - Red/green: each new test fails on the old code and passes on the new code. - `internal-remaining.test.ts`: every candidate zone, reported by its own name, its ICU name and known browser aliases, passes at every hour in January and July. A different clock, a zone with daylight saving time and an unknown name fail. - `identity-personal.test.ts`: the settled read returns the saved name after empty reads, and returns a wrong name when the saved name never arrives. - `ProfileSettings.test.tsx`: the first rendered frame with a cached session shows "Alex Chen", not `""`. - `identity-accounting.test.ts`: the cohort 16 week-range reads pass, as do other covering ranges. Negative controls fail: no reads, run-list reads with unchecked inputs, ranges that exclude the run, another company, a denied read, a read by another run, and a read of a different run record with a link to the target run. - The retained cohort 16 spend-breakdown observation passes the new grader, and fails with its reads removed. - `pnpm test:e2e:runner:unit`: 686 files passed. `pnpm test:e2e:runner:typecheck` passed. `ui` ProfileSettings tests passed. `scripts/check-token-gates.mjs` is clean. - One narrow paid run of the three cases on this branch (result in a comment below). ## Risks - Low risk. The changes are in eval code, plus a small Profile page state change. - A covering range in a form outside the recomputed set (for example 08:00 at -03:00) is not recognized. That is a false negative, not a false pass. - The Profile page now keeps an unedited field in step with the session. A refetch still resets drafts, as before. ## Model Used - Claude, model ID `claude-opus-5-5`, used through Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
c5218f5ec4 |
test(restart-chaos): count distinct warm close failures and let R5 wait out environment acquire (#21852)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The restart-chaos harness (`tests/restart-chaos`) runs zero-downtime
gate waves and grades each wave and each task
> - Gate wave A1 of rerun 6 (w-20261116T001158-s7106) failed. Its "warm
sessions that did not close" count was 28, but only 2 sessions failed to
close. Task t010 also failed R5, but its run was healthy.
> - The harness's environment-delete retry loop re-ran one failing close
every 3 s, so one failed session was logged 27 times. The harness
counted log lines, not failures. Also, no sandbox capture existed from
the time of the failed idle close, so the cause was not recorded.
> - R5 saw 253 s of silence on t010's run aedc1f6e. The run spent 251 s
in `environment.acquire` (a Daytona restore of an archived sandbox, with
one 90 s plugin call timeout). The "run started" event is written only
after acquire, so the run had no event during that time. The run then
succeeded.
> - This pull request makes the warm-shutdown grade count distinct (run,
reason) failures, captures the sandbox at the first idle-close failure,
and lets R5 wait up to 10 minutes while a running run has no run event
yet.
> - The benefit is a wave grade that shows the real number of failed
sessions, records why a close failed, and does not report a slow
environment acquire as a dead run.
## Linked Issues or Issue Description
**Bug description**
Three findings from gate wave A1 of rerun 6 (w-20261116T001158-s7106):
1. `warm-shutdown.ts` reports one failure per log line. The wave showed
29 shutdown close failures (28 "did not close" lines and 1 "cleanup
failed before controller shutdown" line) for 2 sessions: run 300f0d61 (2
lines) and run 70d909cc (27 lines from the environment-delete retries).
No sandbox capture was taken when the idle close of 70d909cc failed, so
why the Runner did not suspend is not known.
2. R5 reported `silent_dead_run aedc1f6e… 181s` for task t010. The run
was in environment acquire for 251 s. It then succeeded.
3. Task t024 failed its content check `subtask_1_done`. The child saved
its ledger with Read-tool line-number prefixes (`1\tROUND 1 ...`). The
grader requires lines that start with `ROUND`. This is model behaviour:
a ledger that the agent corrupts is a real content failure. This pull
request does NOT change that check.
**Steps to reproduce**
1. Grade the A1 server log lines (fixture
`fixtures/a1-warm-close-failures.log`) with `gradeWarmShutdown`. The
result has 29 `shutdownCloseFailures` entries.
2. Grade the t010 samples (fixture
`fixtures/t010-slow-environment-acquire.json`) with `gradeR5`. The
result is `fail: silent_dead_run aedc1f6e…`.
**Expected behavior**
1. The grade shows 2 distinct shutdown close failures, each with its
line count, and the verdict stays "fail".
2. R5 passes a run that has no run event yet for up to 10 minutes. R5
still fails a run with no event past 10 minutes, and a started run that
is silent past 3 minutes.
## What Changed
- `warm-shutdown.ts`: `shutdownCloseFailures` and `idleCloseFailures`
are now lists of distinct `{ runId, reason, lines, firstLine }` entries,
keyed by run and failure reason. The reason drops the JSON detail after
the error text (for example timings), so retries of the same failure
merge. A line with no parsable run keeps its own message as the reason.
Each distinct shutdown failure still adds a problem, so the verdict is
unchanged in meaning: any distinct failure fails the wave.
- `warm-shutdown.ts`: new `watchFirstIdleCloseFailure` splits server
output into lines per stream and reports the first idle-close failure
once. New optional `sandboxCaptures` input and output, listed in the
markdown section.
- `supervisor.ts`: new `onServerOutput(chunk, source)` hook, called for
the primary and the second server's stdout and stderr. A hook error is
recorded in the timeline and never breaks logging.
- `main.ts`: on a Daytona warm wave, the first idle-close failure
triggers one `captureRunnerLogs("idle-close-failure-<run>")` (the class
R probe: processes, sockets, Runner runtime files including
`runner-state.json`, and logs). The harness waits for that capture
before the environment cleanup and before the warm-shutdown grade.
- `observer.ts` / `types.ts`: a running run whose events read back empty
(the read succeeded and there is no event yet) gets `preparing: true` in
its sample. A failed events read does not set it.
- `grader.ts`: new `PREPARATION_LIMIT_MS` (10 minutes). `deadRunningRun`
uses it when the run sample has `preparing`.
- Why 10 minutes: the observed acquire took 251 s, including one 90 s
plugin call timeout. The Daytona provider bounds each start or restore
at 5 minutes by default (`DEFAULT_DAYTONA_OPERATION_TIMEOUT_MS`), so 10
minutes covers one timed-out restore and its retry. This is the same
bound #21730 gave an active provider turn.
- New fixtures: `fixtures/a1-warm-close-failures.log` (the 34 real
close-failure lines of the A1 server log) and
`fixtures/t010-slow-environment-acquire.json` (the 87 real t010 samples
of run aedc1f6e, with the time of its first run event).
- `README.md`: document the distinct-failure count, the idle-close
capture and the preparation bound for R5.
- t024: no change. Decision: the ledger check stays strict, because a
ledger that the agent saved with line-number prefixes is a real content
failure.
## Verification
- Red on `origin/master` sources, green with this change (9 tests):
- `warm-shutdown.test.ts` › "reports each distinct (run, reason) once,
not each log line (w-20261116T001158-s7106 A1: 2 sessions, 29 lines)": 2
failures (300f0d61 with 2 lines, 70d909cc with 27 lines), 2 problems,
verdict fail.
- `warm-shutdown.test.ts` › "still fails on one distinct failure, and
keeps different reasons for the same run apart" (guards: the 27 lines of
one session still fail; two reasons for one run give two problems; an
unparsable line still fails).
- `warm-shutdown.test.ts` › "names the first idle-close failure once,
across chunk boundaries and interleaved streams" (a forced-stop line
does not trigger it).
- `warm-shutdown.test.ts` › "lists the sandbox captures taken at the
first idle-close failure", plus the two updated shape tests.
- `triggers.test.ts` › R5 live evidence › "does not flag a run that is
still acquiring its environment before its first run event
(w-20261116T001158-s7106/t010)": the real samples still fail without
`preparing`, and pass with `preparing` on the samples before the first
run event.
- `triggers.test.ts` › R5 live evidence › "still fails a run silent past
the preparation bound, and a started run silent past the stall bound"
(guards: 9.5 min preparing passes; 10.5 min preparing fails; 4 min quiet
without `preparing` fails).
- `triggers.test.ts` › R5 live evidence › "marks a running run as
preparing only while its events read back empty" (the flag clears after
the "run started" event; a failed events read never sets it).
- `npx vitest run --config tests/restart-chaos/vitest.config.ts`: 71
files, 833 tests pass.
- `npx tsc -p tests/restart-chaos/tsconfig.json --noEmit`: clean.
- No wave was run. The live idle-close capture path in `main.ts` is not
exercised by a unit test.
## Risks
- Low risk. The change touches only the test harness.
- `warm-shutdown.json` changes shape: `shutdownCloseFailures` and
`idleCloseFailures` are now objects, not strings. Scripts that read
these fields as strings must read `firstLine`. Summary counts now show
sessions, not lines, so they are lower than before for the same log.
- A run that dies before its first run event now fails R5 after 10
minutes instead of 3. The fault is still caught, but later.
- The idle-close capture can add up to 45 s (the capture deadline)
before the environment cleanup on a wave that had an idle-close failure.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), used in Claude Code
with tool use and code execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
b5447432bd |
fix(ui): keep the active company on Cases and Status navigation links (#21794)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The board UI builds company-prefixed links, such as `/PAP/cases`,
from short paths in the sidebar
> - The Cases and Status links use `/cases` and `/status`, but these
roots were not in the board route list
> - The router read `cases` as a company prefix and left the link
unprefixed, so the link only worked through the unprefixed redirect
> - When a person clicked Cases before the company list loaded, the
redirect picked an arbitrary company and showed its empty case list
> - The same race made `tests/e2e/cases-read-only.spec.ts` fail in "ci /
e2e shard (2/8)" on unrelated PRs (#21683, #21715, #21765)
> - This pull request adds both roots to the board route list and makes
the e2e test force the race
> - The benefit is that Cases and Status always open in the company the
person is in, and the flaky e2e test becomes deterministic
## Linked Issues or Issue Description
Refs #21683
Refs #21715
Refs #21765
**What happened?**
The read-only Cases e2e test failed about once per day with `Expected:
3, Received: 0` links in the case list (job 132875164249 on #21765, job
132680942843 on #21715). The failure screenshot shows the sidebar of a
different company ("Passive artifacts …", created by the previous spec)
and "No cases yet". The server log shows that the page asked for the
cases of the other company and never for the test company.
**Expected behavior**
A click on the Cases or Status link opens that page for the company in
the URL, also when the click happens before the company list has loaded.
**Steps to reproduce**
1. Have two or more companies.
2. Open `/<PREFIX>/issues` on a slow connection (the e2e test holds `GET
/api/companies`).
3. Before the company list loads, open More and click Cases.
4. The URL becomes `/cases`. When the list loads, the app redirects to
`/<OTHER>/cases` and shows another company's empty case list.
**Paperclip version or commit**
`master` at
|
||
|
|
171e70c11d |
refactor: drop span and run-log event catalogs from the observability docs (#21779)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - `doc/observability.md` and `doc/run-log-events.md` described every OpenTelemetry span and every run-log event > - `rules/data-paths.mdc` told authors to update `doc/run-log-events.md` for each event, so both files only grew: 2,313 and 1,263 lines > - Each span and event is already defined next to the code that writes it, and its name is a literal string that a search finds in one step > - This pull request removes the per-span and per-event catalogs and keeps only what code does not say > - The benefit is two short docs that do not drift from the code, and no rule that makes them grow again ## Linked Issues or Issue Description Refs #21670, #21700, and #21743 (the same cleanup for other docs). **Issue type** Outdated or confusing documentation. **Where is the issue?** `doc/observability.md`, `doc/run-log-events.md`, and `rules/data-paths.mdc` **What's wrong?** The two docs repeated span names, span attributes, and event payloads that the code defines. Operator setup started at line 138 of `doc/observability.md`. The rule required a doc edit for every new run-log event. **Suggested fix** Keep operator setup and the Sentry privacy contract. Point to the code for span and event definitions. Remove the rule line that requires a doc entry per event. ## What Changed - `doc/observability.md` is now 619 lines. It keeps: how to enable tracing, `service.version` resolution, scope, how to enable Sentry, DSN delivery, environment attribution, privacy settings, server request data, the core of "Run failure context", browser data, fail-open behavior, the default capture set, and operator responsibilities. Kept text is unchanged except the intro and two sentences noted below. - Deleted from `doc/observability.md`: budget, Slack receipt, task detail, and browser tracing notes; controller preparation phases; native Runner spans; sandbox startup spans; sandbox duplex transport; shared skill preparation; invocation attribution; and about 500 lines of per-failure detail in "Run failure context". - Replaced that per-failure detail with one paragraph. It names the failures that `unexpected` mode keeps out of Sentry, and says the remaining diagnostic contexts are defined in `server/src/sentry.ts`. "Default capture set" now says the same. - `doc/run-log-events.md` is now 23 lines. It says what the run log is, where events are defined and appended, and keeps the payload storage note. - `rules/data-paths.mdc`: the run-log line now says to define an event where it is written, not in a doc. The span-allowlist line now says the allowlist is in the code that opens the span. The review levels and globs do not change. - Repointed links in two feature maps, two agent-access map files, `tests/runner-e2e/SUITES.md`, `doc/runner-api-tools.md`, and the Telemetry README. Updated the `honeycomb-tracing` skill and one code comment in `packages/adapter-utils/src/http2-bridge-server.ts`. ## Verification - A script built the new `doc/observability.md` from line ranges of the old file and asserts the first line of each range. - A link check on all changed Markdown files found 0 missing files and 0 missing anchors. The remaining inbound anchors (`#enabling-tracing`, `#sentry-error-monitoring`) still exist. - No test reads either doc. - `node scripts/check-maps.mjs` passes. - `node --test .github/scripts/tests/*.test.mjs`: 808 pass, 0 fail. - `pnpm verify` was not run. This worktree has no installed dependencies. The only code change is a comment. ## Risks - The privacy contract no longer lists each run-failure diagnostic context field. `server/src/sentry.ts` is now the list. A reviewer of a Sentry change must read that file. - No browsable list of span names or run-log event names remains. A search for the quoted name finds the producer. - `.greptile/files.json` still scopes `rules/data-paths.mdc` to both docs. Both files still exist, so it does not change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Claude Code desktop, with tool use. Earlier analysis in this session used Claude Fable 5.2. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [ ] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
3706f520fd |
test(scaling): keep ramping through recovered runs, read spend from cost records, and wait for termination before deleting agents (#21776)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The scaling effort uses a real-agent ramp harness (`scripts/scale-agents-ramp.ts`, #21695 and #21757) to load a hosted test instance through the product API > - Its first run with agents that really ran (10 agents, 105 tasks, 106 runs) stopped on one failed run, recorded $0 model spend, and left cleanup errors > - Read-only inspection of the instance showed three harness faults. The failed run had a same-run bootstrap retry scheduled 33 s later, but the harness stopped and cleaned up first. Run rows carry no cost, but cost records do (63.7 cents for the run's agents). Terminated agents leave the company agent list, so the harness treated them as done and sent deletes that Paperclip refused with 409 > - These faults make the ramp stop too early, make its spend cap inert, and make cleanup report the wrong cause > - This pull request keeps the ramp going through runs that Paperclip recovers by itself, reads spend from cost records, records board latency per route, and makes cleanup wait for termination and name what it left behind > - The benefit is that the next ramp grades the product, not the harness, and its spend cap works ## Linked Issues or Issue Description Refs #21695 Refs #21757 **What happened?** The ramp stopped at step 1 with `failed_runs_zero failed (observed 1)`. The run failed with `runner_ingress_bind_conflict` before its provider session opened. Its run events show `same-run resume persisted` with `recoverable: true` and `nextAttemptAt` 33 s later. The harness stopped within 9 s and paused the agents, so the retry was blocked (`native_retry_admission_ineligible`). The result showed `Recorded model spend: $0`. The harness read `costUsd` from run rows, which is always null; `GET /api/companies/<id>/costs/by-agent` holds 63.71 cents over 104 cost events for the 10 agents. Cleanup reported two `DELETE /api/agents/:id returned 409: Complete termination before deleting the agent`. Those agents were still `lifecycleState: terminating`; the harness waited on the company agent list, which hides terminated agents, so it treated them as done. A later `--cleanup-only` changed nothing because it also found agents only through that list. The board API p90 failed (765 ms), but the result had no per-route timings to say which request was slow. **Expected behavior** A failed run whose task finishes by itself is counted, not a stop. A failed run whose task does not finish within a grace stops the ramp. The share of failed runs is graded at the step end. Spend comes from Paperclip's cost records. Cleanup deletes an agent only after termination completes, and names the agents and sandbox leases still pending. **Steps to reproduce** 1. Run `node --test scripts/scale-ramp/scale-ramp.test.ts` with the new fake and tests and the `origin/master` `ramp.ts`, `cleanup.ts`, `criteria.ts` and `report.ts`. 2. 16 tests fail, among them every new test (recovered run, unrecovered run, spend from cost records, spend cap, board per route, termination wait, still-terminating agents and `--cleanup-only`, leases released during the wait). 3. With this change, all 60 pass. **Paperclip version or commit** Instance on `2026.1106.0+2564.git.3244780bd`; harness from #21757. ## What Changed - `criteria.ts`: replace `failed_runs_zero` with `failed_runs_unrecovered_zero` (immediate) and `failed_run_rate` (at the step end). Step metrics add `runsEnded`, `runsFailedRecovered`, `runsFailedUnrecovered`, `runsFailedPending` and `board.byRoute`. - `ramp.ts`: a run counts as failed once it was seen `failed` or `timed_out`. It is recovered when its task reaches `done`. It is unrecovered when the task rests not done, needs Retry, or has no live run for `--recovery-grace-minutes` after the failure. A task whose interruption has an automatic retry scheduled (`state: retry_scheduled`, `nextRetryAt` or `retryWait`) does not count as needing Retry. Spend comes from `costs/by-agent` for the run's agents on each poll, and each step reports the spend in its window. Board probes keep a distribution per route. - `config.ts`: add `--max-failed-run-percent` (default 2) and `--recovery-grace-minutes` (default 15). - `cleanup.ts`: read agents by id as well as from the list, so terminated agents in the state file are found. Wait for `lifecycleState` `terminated` or `rejected` by id before a delete. Report `agentsStillTerminating`, `activeSandboxLeaseIds` and `sandboxLeasesReleasedDuringWait`, and name each lease still active with its run and acquire time. - `report.ts`: the summary shows recovered, unrecovered and pending failed runs, board latency per route, and each cleanup error. - `fake-paperclip.ts`: run rows carry no cost; `costs/by-agent` returns cents per agent. Optional failure of the n-th run, a scheduled recovery run, a termination delay (terminated agents leave the list and a delete answers 409 until termination completes), and leases that stay active after their run. - `doc/plans/2026-11-13-thousand-concurrent-agents.md`: the runbook describes the failed-run criteria, the spend source and the cleanup behavior. ## Verification - `node --test scripts/scale-ramp/scale-ramp.test.ts`: 60 of 60 pass. - Red check: with the new fake and tests but the `origin/master` `ramp.ts`, `cleanup.ts`, `criteria.ts` and `report.ts`, 16 tests fail, including all 8 new ones. - `tsc -p scripts/scale-ramp/tsconfig.json`: no errors. - Evidence from read-only GETs on the test instance: run `f1892d43` events (bind conflict, scheduled same-run retry, blocked after the pause), run rows (`costUsd` null on all 106), `costs/by-agent` (63.71 cents for the 10 agents), agent rows by id (`terminating` while absent from the list), lease rows. - Not run: a new real ramp (needs the owner's go-ahead). ## Risks - Low risk. The change touches only the scale-ramp harness, its fake, its tests and one runbook section. No product code changes. - A failed run no longer stops the ramp at once. An unrecovered one stops it after the grace (15 min by default), so a broken step can run up to that long. The step-end failed-run rate still fails a step with more than 2% failed runs. - Spend now needs cost read access for the board token. If `costs/by-agent` cannot be read, the result reports spend as not measured and the spend cap does not trigger. - Cleanup can now take up to 5 more minutes while it waits for termination by id. ## Model Used Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, read-only HTTP GETs to the test instance). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
62033b9994 |
fix(qa): schedule recovered cleanup promptly and wait for task panels (#21660)
## Thinking Path > - Paperclip lets people manage AI agents and their saved work. > - QA journeys check browser controls and the cleanup of owned agent resources. > - A recovered workspace queues sandbox destruction after its files are saved. > - The cleanup queue treated that first attempt as a retry and waited five minutes. > - Agent termination correctly waited for destruction, so QA cleanup failed first. > - This PR runs that first cleanup attempt promptly and keeps the existing safety gates. > - It also waits for task-panel transitions before the browser acts again. ## Linked Issues or Issue Description **What happened?** The full scheduled QA smoke completed with 23 passed and 1 failed. Stop/redirect passed all eleven behavior checks but failed cleanup. The cancelled run had a saved workspace and a bound stop receipt before agent termination. Its sandbox was destroyed about five minutes later. The shared cleanup schedule applied the retry delay before the first destruction attempt. The browser regressions also found that the current panel helper can consume all four actions while an opener remains visible during panel mounting. **Expected behavior** A recovered workspace's first cleanup attempt runs on the next eligible worker pass. A failed attempt retains normal backoff. Agent termination still waits for completed resource cleanup. Task-panel navigation waits for the next controls before it acts again. **Steps to reproduce** 1. Queue recovered-workspace destruction after the cleanup worker has become idle. 2. Read the next cleanup deadline and advance one normal worker pass. Before this fix, the deadline is five minutes late and no teardown runs. 3. Open task document controls with a delayed side-panel transition. Before the panel fix, both collapsed-panel controls fail. Refs #19224 and #19292 for retaining stop proof during workspace recovery. This change keeps the full cleanup requirement for agent termination. Current master supplies the shared task-panel helper; this branch uses it and retains additional browser regressions. ## What Changed - Use one shared first-attempt predicate for the cleanup deadline, sweep selection and ownership report. It now includes recovered workspace cleanup bound to the same run. - Preserve worker spacing, in-flight claims, ownership and saved-file checks, retry delays and attempt limits. - Wait for panel controls after opening the panel. Wait for the selected tab or menu action to finish. - Add real PostgreSQL cleanup regressions, keep the Chromium controls, and document the cleanup timing. ## Verification Tested on Linux x86_64, Node v25.9.0. Local commit `7c8e0f12aa4b19aaf53669141ab3299ce76db963` and published head `892c5dd4b4638796a138400d8bbe30aee19c71dd` have the same tree, `d135a98bdff58e3320cdc83976ffe1cf47f48b66`. - Red: both new real-PostgreSQL cleanup tests failed on the old code. The deadline was five minutes late and the first normal sweep made zero teardown calls. - Green: 87 cleanup, ownership and lifecycle tests pass. The restored-work case stays unconfirmed until the real cleanup worker destroys the lease; failure still backs off and the worker becomes idle again. - Red after master integration: both delayed document-panel browser cases failed. Green: all 18 Chromium support checks pass. - 77 journey contract tests pass. - Product E2E typecheck passes with the documented low-memory path. The initial ordinary attempt was killed by the local memory limit and is not counted as a pass. - The only follow-up after the passing checks removes new code comments to follow the contribution guide. Runtime and test behavior are unchanged. Final-head [CI run 43714742904](https://github.com/paperclipai/paperclip/actions/runs/43714742904) completed successfully at this exact head. All 63 PR checks are terminal: 58 success, 2 neutral and 3 conditional skips; no failures or pending checks. Typecheck, build, tests and the final verify gate pass. Greptile and Paperclip Review both score this head 5/5 with no actionable findings; both historical inline threads are resolved. - Hosted QA recovery is still unverified. No extra model run, merge, deployment or QA setting change was made. ## Risks Recovered workspace destruction starts sooner. Existing producer, file-preservation, ownership, claim and provider checks still decide whether teardown is allowed. Failed attempts retain the existing cooldown. Agent termination and purge gates are unchanged. The panel helper waits for visible controls and still fails when controls or saved values are absent. The PostgreSQL integration uses a controlled provider boundary. It does not prove a new live Daytona or deployed QA pass. That requires the fixed revision on QA and a later normal scheduled smoke. ## Model Used OpenAI GPT-6-based Codex, with tool use and code execution. The host does not expose a more specific model ID or context-window size for this run. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
5b750fdf25 |
feat(ui): add foldable sidebar subtasks with bounded refreshes (#21649)
## Thinking Path > - Paperclip helps people manage AI agents for work. > - The sidebar helps people return to recent tasks. > - Recent parents need foldable child tasks. > - Busy companies must not refresh every child group for unrelated work. > - Exact unread counts must preserve personal reads and task access. > - This pull request adds child groups and bounds their refresh work. > - People can scan subtasks without repeated background requests. ## Linked Issues or Issue Description **Subsystem affected** Task navigation in the standard and Classic sidebars. **Problem or motivation** Recent parent tasks did not show their subtasks. The first implementation refreshed every previously viewed group. One unrelated update caused 52 child-page requests and one summary request after viewing 50 groups. Exact child counts also repeated privacy and unread checks per child. **Proposed solution** Show direct subtasks with independent folding and ten-row pages. Refresh affected groups and summary batches. Stop offscreen reads. Bound child-request concurrency. Aggregate viewer facts once and preserve the existing inbox unread rule. **Roadmap alignment** This extends existing task navigation. The existing feature PR is reused for its performance repair. No new PR duplicates it. ## What Changed - Add shared parent groups to both sidebar layouts. Preserve Classic row actions and active-child visibility. - Keep unread state personal to each task. Folding and paging do not mark tasks read. Counts include readable children beyond ten. - Target live refreshes with authorized current and previous parent IDs. Reuse parent metadata from the narrow run-binding read. Resolve fallback lookups once per delivery batch. Keep the previous summary fallback for older servers. - Limit child requests to three at a time. Cancel active and queued reads when groups leave the viewport. - Discard extra cached pages on Show less and collapse. - Compute exact navigation counts in one SQL statement. Materialize readable candidate IDs once. Use indexed participation sources and conservative unread candidates before the shared inbox rule. - Skip recursive privacy walks for standalone open tasks without a project. Keep the recursive rule for all restricted or inherited shapes and all delegated principals. - Add request-count, cancellation, reparenting, read-receipt, inbox-parity and privacy regressions. Include a disposable PostgreSQL benchmark. - Update product and API documentation. Retain the production Storybook states. ## Verification - Published head: `4148861437c1b6b635479ad92509dbc71155a232`. Its tree matches local `04be9d66d481d9bcd1c4d6158acf063a1e65ac7e`: `a23533e93d3d155918f0254d296a7fd2f0c7c12f`. - Rebased on master `ca23330c69b1f2b25bdadfcbda68317db97deaef`. Remote publication preserves prior feature commits. - The request regression failed before the fix and passes after it. An unrelated event now sends zero child requests. Current servers also avoid the unrelated summary request. Old-server events keep one summary fallback. - A relevant update after Show less reads one page. Reparenting refreshes two affected groups. Offscreen reads stay at zero and resume once on return. Active and queued cancellation tests pass. - The same synthetic authenticated HTTP workload with 50,000 read children improved from 767 ms to 92.84 ms median. Five warm samples per case. The all-unread case improved from 463.85 to 351.65 ms. Untouched children improved from 208.01 to 131.09 ms. The old-receipt case improved from 799.70 to 245.80 ms. These local measurements are not a production capacity guarantee. - Focused PostgreSQL routes, inbox-parity, privacy and WebSocket tests passed: 38 tests. - All 325 affected UI tests passed. All 38 focused PostgreSQL, privacy and WebSocket tests passed. UI typecheck and low-memory server declaration checks passed. - All 41 Chromium Storybook checks passed. They cover 20 states in both themes and keyboard, folding, paging and unread behavior. They use production components with HTTP fixtures. - Both new review findings failed as regressions before their fixes and pass afterward. Explicit child reads include waiting external conversations. Deleting a child publishes a separate parent-authorized update without deleted-child content. Both review threads are resolved. - `pnpm verify --explain` returned exit 2 because product documentation requires broader CI. [The prior CI run](https://github.com/paperclipai/paperclip/actions/runs/43711489306) found type errors in the new test fixtures. The same errors failed locally. The fixture fix passes `node server/scripts/typecheck-layers.mjs --layer test-tree-0` and all 21 issue summary route tests. Production code is unchanged. That CI run then found a live-delivery read regression. The existing test failed locally with eight statements against its six-statement budget. A new PostgreSQL regression also found three parent lookups for two same-batch calls. Run delivery now reuses parent metadata from its narrow run-binding query. Fallback lookups resolve to one promise. Both regressions pass without increasing the budgets. All 74 targeted route, WebSocket and privacy tests pass. The affected below, above and test-tree-0 typecheck layers pass. [CI attempt 2](https://github.com/paperclipai/paperclip/actions/runs/43715541480/attempts/2) passes on the published head. All 62 check runs are terminal and clean: 57 succeeded, two were neutral and three were skipped. Typecheck, build, policy, all general and serialized server suites, E2E and the [final verify gate](https://github.com/paperclipai/paperclip/actions/runs/43715541480/job/132829783231) pass. The separate security status passes. Attempt 1's server shard 7/12 reached its 20-minute job limit without a test verdict. One retry of that job passed without a source change. [Greptile completed successfully on this head](https://github.com/paperclipai/paperclip/runs/132816163906) with a [5/5 score](https://github.com/paperclipai/paperclip/pull/21649#issuecomment-6883916606). All five review threads are resolved. The final CI gate is complete. - A full local Storybook build exceeded available memory while server typechecking ran. Browser verification uses the current-source development server. Rebuilding stale Runner and SDK declarations resolved the initial server check errors. The first browser attempt lacked Chromium; all checks passed after installation. - Reproduce the database measurements with `pnpm --filter @paperclipai/server exec tsx scripts/benchmark-sidebar-subtasks.ts`. - Extra local shard diagnosis ran 164 suites. Under this shell's `0077` creation mask, 157 suites passed and three had fixture permission or timing failures. Rerunning those three with `0022` resolved the permission failures. The remaining ten cases passed with one worker: two workspace cases and eight archive cases. The eight archive cases also passed on the pre-performance-fix revision, whose archive source is identical. The broad local shard is not claimed as one passing execution. The same-head CI retry supplies complete final CI coverage. - Nothing is merged or deployed. ## Risks - Exact counts still grow with the readable hierarchy and comment history. The change reduces repeated work; it does not cap or approximate counts. - A relevant summary batch can contain several recent parents. Older servers retain a broader summary fallback until upgraded. - Live delivery adds bounded, permission-checked parent lookups. Short delivery-window reuse limits repeated reads. - The authorization fast path applies only to open tasks with no privacy parent and no project. Restricted cases retain the shared predicate. - No migration is required. ## Model Used OpenAI Codex, based on GPT-6. Used code execution, native GitHub tools, PostgreSQL and Chromium verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked a public issue or described the issue in this PR - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal issue id - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open findings - [x] I will address all review comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
fb3b09aaef |
refactor: split the runner-e2e and restart-chaos READMEs into a runbook and a catalog (#21743)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - `tests/runner-e2e` is the paid Product E2E harness and `tests/restart-chaos` is the restart chaos harness > - A README is the first file a person or an agent opens in a directory > - These two READMEs grew to 3,805 and 2,277 lines. Most of each is a catalog with one section per suite or per fault scenario, and almost none of it is ever trimmed > - This pull request splits each README into a runbook and a catalog file > - The benefit is a README that can be read in full, and a clear place for the next suite or scenario note ## Linked Issues or Issue Description Refs #21670 and #21700 (the same cleanup for `doc/DEVELOPING.md` and `doc/DATABASE.md`). **Issue type** Outdated or confusing documentation. **Where is the issue?** `tests/runner-e2e/README.md` and `tests/restart-chaos/README.md` **What's wrong?** In the last 60 days, `tests/runner-e2e/README.md` gained 3,476 lines and lost 182. `tests/restart-chaos/README.md` gained 2,411 and lost 134. The setup, evidence, and CI sections sit between suite notes. The first 225 lines of the runner-e2e README are suite notes with no heading. **Suggested fix** Keep how to run the harness in the README. Move the per-suite and per-scenario sections to a catalog file in the same directory. ## What Changed - `tests/runner-e2e/README.md` is now 767 lines: credentials, local commands, the Daytona image, evidence and cleanup, billing, the dashboard, GitHub Actions, and the report preview. - New `tests/runner-e2e/SUITES.md` (3,055 lines) holds every per-suite section. The two blocks of suite notes that had no heading now have one. - `tests/restart-chaos/README.md` is now 592 lines: run, fault timing, Daytona mode, hosted mode, universal checks, grading notes, pieces, output, and the crash-point suite. - New `tests/restart-chaos/SCENARIOS.md` (1,694 lines) holds every workload and fault scenario section. - All text is moved without wording changes. Nothing is deleted. The only edits are three "(below)" references in the restart-chaos README, which are now links. - Repointed 22 anchor links and 3 plain links in `doc/evals.md` and `maps/feature-map/cases.md`. - Updated seven code comments in `tests/restart-chaos/` that name a moved section, and two skills that list the runner-e2e docs to read. ## Verification - The split script asserts that each source line is kept or moved exactly one time, and checks the first line of each range. - A link check on the four files and on every file that links into them found 0 missing files and 0 missing anchors. - `tests/runner-e2e/daytona-image.test.ts` reads the README for a Docker build recipe. The recipe is still in the README. I did not run that test locally. - `node scripts/check-maps.mjs` passes. - `node --test .github/scripts/tests/*.test.mjs`: 808 pass, 0 fail. - `pnpm verify` was not run. This worktree has no installed dependencies. ## Risks - Low risk. The change is documentation and comments only. - A link to a moved section from outside this repository lands on the top of the README. - The catalog files are still long. This change moves them off the first-read path. It does not prune them. ## Model Used - Claude Fable 5.2 (`claude-fable-5-2`), Claude Code desktop, with tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [ ] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com> |
||
|
|
819f61f1fd |
fix(evals): repair four cohort-15 chat cases and prefix board links in completion feedback (#21682)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The chat-feature eval catalog runs real chat journeys. It grades saved state, rendered links and run ledgers. > - Chat-eval cohort 15 (OpenCode with `openrouter/deepseek/deepseek-v4.1-flash-1017`, master `36101ebcb9`) failed four cases where the agent did the right thing or where our own server text misled it. > - One failure was a harness race introduced by #21667. Two failures came from graders that read free-form text with keyword regexes. One failure came from an unprefixed approval link in the server's completion feedback, plus a grader that required a literal prefix. > - This pull request fixes the harness step, replaces the free-form graders with saved-state and clicked-link checks, and makes the server emit company-prefixed board links. > - The benefit is that these cases measure the feature again. The retained correct outcomes pass, and the relevant broken outcomes still fail. ## Linked Issues or Issue Description Refs #21667 **What happened?** In chat-eval cohort 15, four cases failed: - `projects.share-project`: the harness followed the agent's ID-based project link. The app then rewrote the URL to the project slug and redrew the page. The owner step timed out because Riley's option in the people picker was "detached from the DOM". #21667 introduced this path. - `skills.author-skill`: the saved skill was a correct, richer checklist. Per-sentence keyword regexes rejected it. For example, "ignore the mouse" tripped the forbidden word "ignore". - `inbox-search.show-decisions`: the reply linked the right approval and tasks and said the question was "addressed to a different teammate". A regex required the word "Riley". - `agents.hire-needs-approval`: both replies linked `/approvals/<id>` without the company prefix. The agent copied that link from `server/src/services/native-runtime/native-completion-feedback.ts`. The grader required a literal `/<prefix>/approvals/<id>`, although the board redirects the unprefixed route. **Expected behavior** Correct agent behaviour passes. Grading uses saved state, setup literals and rendered or clicked links, not free-form wording. The server gives agents canonical company-prefixed board links. **Steps to reproduce** 1. Use the retained cohort-15 evidence for the four cases. 2. Run the old graders on the retained skill markdown and decisions reply. Both fail. 3. For share-project, open a project configuration page by ID while the app rewrites the route after a delay, then use the access picker before the rewrite completes. 4. For hire-needs-approval, finish a task with a pending linked approval. The feedback links `/approvals/<id>`. **Paperclip version or commit** master `36101ebcb9` (cohort 15); fixed on top of current master. ## What Changed - `projects.share-project` (`internal-privacy.ts`, `runtime-support.ts`): after the harness follows the agent's link, it waits until the URL settles on the project's canonical route key. This reuses the #21667 approach for agent profiles: `settleAgentProfileRoute` now calls a shared `settleRouteKey`, and `projectRouteKey` follows the app's `projectRouteRef`. The owner's picker selection (`addProjectAccessMember`) now retries as one unit. If the dialog or picker closes, or an option detaches, it reopens them, searches again and selects again. It continues only when the picker shows the person and Add is enabled. - `skills.author-skill` (`integrator-skills.ts`): removed the keyword-regex checklist grader, the waiver regexes, and the reply regexes for "global activation" and "attached to me" claims. The saved-state checks stay: one agent-created writable skill, its current revision authored by the responding agent, the same bytes on the skill and the revision, the Skills/Studio inspection with reload, and assignment scope from saved assignments and audit rows. The checklist check now requires the setup literals `375`, `WCAG` and `AA` as whole tokens in the saved SKILL.md and in the rendered editor. Whether the skill still asks for keyboard-only navigation and visible labels, and whether it waives a check, is left to human review. Oracle version `skill-authoring.v10`. - `inbox-search.show-decisions` (`internal-decision-search.ts`): removed all reply regexes, including the `Riley` requirement and the wording-based separation checks. The grade now uses saved state (pending decisions unchanged, answered history unchanged, each person's own attention queue and resolver) and rendered links to the exact approval, the question's task and the history task. Oracle version `decision-search.v3`. - Product (`native-completion-feedback.ts`): the completion feedback now links `/<prefix>/approvals/<id>` and `/<prefix>/issues/<identifier>`. This fixes the approval link and the four sibling task links in the same file. - `agents.hire-needs-approval` (`runtime-hiring.ts`): the harness now clicks each reply's rendered link to its approval and records where the UI lands. The grade accepts a prefixed link, or an unprefixed link that the board redirect lands on `/<prefix>/approvals/<id>` with the proposed hire visible. Without such a link, the harness still opens the approval directly so later phases keep their evidence, and the grade fails. Oracle version `runtime-hiring.v4`. - No eval YAML changed, so the TestOps manifest does not change. `chat-catalog-descriptors.json` records the three oracle version bumps. ## Verification - Red/green tests (each new test fails on the old code and passes on the new code): - `internal-privacy-share-route.test.ts` (real Chromium): the project page rewrites its route after 0, 400 and 1500 ms, and one case closes the picker while its first results render. With the old flow, the 400 ms rewrite and the closed picker both time out on the option click (the cohort-15 failure). With the fix, Riley is added on the settled route in all cases. - `integrator-skills.test.ts`: the retained cohort-15 SKILL.md passes. The old grader rejects it. A stale skill body, another author, and a lost 375 width still fail. - `internal-decision-search.test.ts`: the retained cohort-15 reply and a reworded reply pass. The old grader rejects the retained reply. A missing or wrong approval, question-task or history-task link fails. - `native-completion-feedback.test.ts`: a pending linked approval produces `[the pending approval](/<prefix>/approvals/<id>)`, and a pending review stage produces a prefixed task link. The old code produces `/approvals/<id>`. - `runtime-hiring.test.ts`: unprefixed links that the redirect lands on the right approval pass. The old grader rejects them. A link that did not land, landed on another approval, was not clicked, or is another href still fails. - `npx vitest run --config tests/runner-e2e/vitest.config.ts chat-catalog`: all passed (one negative control in `internal.test.ts` changed from a reply wording to "approved while listing"). - `pnpm test:e2e:runner:typecheck` and `server` `tsc --noEmit`: passed. - `server` vitest for `native-completion-feedback`, `native-current-wake-completion-feedback` and `heartbeat/run-completion`: 201 tests passed. - Narrow paid run of the four full case ids on this branch: run 43657390762 (results are added in a comment). ## Risks - Low risk. The product change only adds the company prefix to links in agent feedback text. It adds one indexed company read, and only on the branches that return a link. - Less is graded automatically, by design. `skills.author-skill` no longer checks that the skill keeps the keyboard-only and visible-label checks or that it does not waive a check. `inbox-search.show-decisions` no longer checks how the reply names resolvers or separates the person's decision from others' and from history. `skills.author-skill` no longer checks whether the reply describes the assignment scope truthfully. A person reviews this meaning. Saved state and links still catch the plausible broken outcomes listed above. - `agents.hire-needs-approval` now needs a reply link that really opens the approval. A reply without a link to its approval fails as before. ## Model Used - Claude, model ID `claude-opus-5-5` (Claude Code agent with tool use and code execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
7c0c116edf |
test(e2e): accept every dependency wake producer for the one automatic resume (#21732)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - When a blocker task is completed, Paperclip wakes the blocked dependent task automatically > - Two producers emit that `issue_blockers_resolved` wake: the issue update route and the issue-graph liveness backstop (plus workspace finalize); they share one idempotency key and coalesce into one wake run > - The QA task-dependency-resume journey requires the run's `contextSnapshot.source` to be exactly `issue.blockers_resolved` > - On QA both producers emitted 43 ms apart; the run was created once, but the later backstop wake recorded its source on the run, so the journey failed a resume that happened exactly once > - This pull request accepts any of the three producers and keeps every other resume condition > - The benefit is that the journey tests the user-visible promise (one automatic resume caused by this blocker) instead of which internal producer won a race ## Linked Issues or Issue Description **What happened?** QA run (launch 3937) failed task-dependency-resume on `dependency_one_automatic_resume`. The dependent's activity log shows `issue.blockers_resolved_wake_emitted` from `issue_update` at 06:59:39.973 and from `issue_graph_liveness_backstop` at 06:59:40.016 with the same idempotency key and the same wake run. Exactly one run executed, started after the blocker completed, and succeeded. **Expected behavior** The journey accepts one automatic, blocker-caused resume regardless of which producer's label the coalesced run kept. **Steps to reproduce** Grade the retained evidence with `contextSnapshot.source = "issue_graph_liveness.backstop"`: `dependency-one-automatic-resume` fails although every other condition holds. ## What Changed - `task-dependency-resume-evidence.ts`: the wake source must be one of `issue.blockers_resolved`, `issue_graph_liveness.backstop`, `workspace.finalize` (the producers of this wake reason in `server/src`). - Still required: one dependent run, `invocationSource` automation / `triggerDetail` system, wake reason `issue_blockers_resolved`, the resolved blocker id, start after blocker completion, committed native run, no leftover checkout. ## Verification - Red/green: new cases "accepts the same automatic resume when the issue_graph_liveness.backstop / workspace.finalize dependency wake recorded its source" fail before and pass after. - Existing mutants still fail: "mutation name used as wake source" (`issue.blocker_done`) and "missing saved wake source". - `tests/runner-e2e` task-dependency tests: 3 files, 108 tests pass. ## Risks - Low risk: journey grading only. Separate note: the coalesced run keeps the last producer's source label; whether it should keep the first is a provenance question, not addressed here. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
9ebc276b42 |
test(restart-chaos): let R5 wait out a slow model request and stop calling held wakes undispatched (#21730)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The restart-chaos harness (`tests/restart-chaos`) grades each task of a chaos wave with rules R1–R6 > - R5 fails a run that stays `running` with no new evidence for more than 3 minutes (`silent_dead_run`) > - In wave w-20261115T044649-s7106, task t028 failed R5. Its run was quiet for 303.6 s (05:22:13.005 to 05:27:16.647) because one DeepSeek request through OpenRouter was slow. Every sample showed an active provider turn and no tool running. The run then finished normally. > - A slow model request is not a dead run. False failures hide real failures and make the wave number wrong. > - This pull request lets R5 wait up to 10 minutes while a provider turn is active and no tool runs. It also stops the release-swap grade from calling queued wakes "never dispatched" while a live run holds them. > - The benefit is a wave grade that counts only real dead runs and real lost wakes. ## Linked Issues or Issue Description **Bug description** R5 reports `silent_dead_run` for a run that waits on one slow model request. The samples show `providerTurnActive=true` and no tool in flight for the whole gap. The run then finishes normally. Separately, the release-swap grade detail says "2 queued wake(s) never dispatched" for a task whose run was still running. Those wakes wait behind the active run by design. **Steps to reproduce** Grade the t028 samples of wave w-20261115T044649-s7106 with `gradeR5`. The result is `fail: silent_dead_run 7b51b352… 182s`. Grade a release-swap task that has a stale running run and queued wakes. The detail contains "never dispatched". **Expected behavior** R5 passes a quiet run while a provider turn is active and no tool runs, up to 10 minutes. R5 still fails a quiet run with no active provider turn after 3 minutes, and a provider turn quiet for more than 10 minutes. The grade does not call wakes undispatched while a queued or running run holds them. ## What Changed - `grader.ts`: add `PROVIDER_TURN_LIMIT_MS` (10 minutes). `deadRunningRun` uses this limit instead of the 3-minute `STALL_LIMIT_MS` when the run sample has `providerTurnActive`. A run with a tool in flight or a live process is still skipped as before. - Why 10 minutes: the observed slow request took 303.6 s. Common provider HTTP clients (the OpenAI and Anthropic SDKs) time out a request at 10 minutes. After 10 minutes the provider client has given up, so continued quiet means the run is dead, not slow. A turn whose process died also keeps `providerTurnActive` set, so the bound still catches it. - `release-swap-grade.ts`: omit the "N queued wake(s) never dispatched" note while the task has a queued, running or scheduled-retry run. The stale-run note stays. - New fixture `fixtures/t028-slow-provider-turn.json`: the real t028 samples from 05:21:30 to the run's end (116 samples). - `README.md`: document the provider-turn bound for R5 and the queued-wake rule. ## Verification - Red on `origin/master`, green with this change: - `triggers.test.ts` › R5 live evidence › "does not flag a slow model request while the provider turn is active with no tool running (w-20261115T044649-s7106/t028)" - `triggers.test.ts` › R5 live evidence › "still fails a silent run with no active provider turn, and a provider turn silent past the provider-latency bound" (guards: 4 min quiet with no provider turn fails; 9.5 min quiet with a provider turn passes; 10.5 min quiet with a provider turn fails) - `release-swap-grade.test.ts` › "does not report queued wakes as never dispatched while a run holds them (w-20261115T044649-s7106/t019)" (also checks that wakes with no live run still get the note) - `npx vitest run --config tests/restart-chaos/vitest.config.ts`: 71 files, 826 tests pass. - `npx tsc -p tests/restart-chaos/tsconfig.json --noEmit`: clean. - No wave was run. ## Risks - Low risk. The change touches only the test harness grader. - A run whose provider turn is active but whose process is truly dead now fails R5 after 10 minutes instead of 3. The fault is still caught, but later. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), used in Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
6400698a8e |
test(e2e): find the native Plan chip in the Coordinator chat thread (#21727)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The QA Cloud plan-revise-accept journey checks that a person can
revise and approve an agent's Plan
> - It first checks that the saved Plan chip is visible, scoped to a
settled `task-chat-turn` so a live "Writing plan" chip cannot count
> - The Coordinator workspace renders the Plan chip directly in the chat
thread; there is no `task-chat-turn` element and no `data-settled`
attribute
> - So the journey never finds the chip and fails before the person
reviews the plan
> - This pull request also accepts the chip in the chat thread
> - The benefit is that the plan journey reaches the revision and
approval steps it exists to test
## Linked Issues or Issue Description
**What happened?**
On QA Cloud (build
|
||
|
|
a78be8b601 |
feat(native): send launch preparation and run digests to plugin task prepare (#21717)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Environment plugins can run a native Runner as a plugin task, and a warm session keeps that Runner between turns > - On each warm turn Core sends the plugin a `prepare` operation with the full launch requirements for the new run > - The requirements' `requirementsDigest` covers the run identity, and the run identity changes on every turn > - So a plugin cannot tell that only the run moved, and it must redo its whole preparation every warm turn > - This pull request gives the plugin two digests: one that stays the same while its preparation stays valid, and one that names the run > - The benefit is that a plugin can transfer only the run identity when nothing it reconciles changed ## Linked Issues or Issue Description **What existing behavior does this improve?** The `prepare` operation that Core sends to an environment plugin for a plugin Runner task, on cold starts and on warm attaches. **Subsystem affected** Native runtime, plugin Runner tasks, the Runner launch contract, and the plugin SDK environment task contract. **Current behavior** A warm attach sends the full requirements. The only digest available, `requirementsDigest`, includes `runId`, `turnId` and `itemId`, so it changes on every turn even when the launch is otherwise the same. **Proposed behavior** Core computes two digests for every `prepare`: - `preparationDigest`: the launch requirements an environment reconciles when it prepares a Runner: - the runner, lease and session identity; - provider, model and settings; - the prompt and instruction text; - the runtime context's instruction and skill bundles, working copy and connection instructions. It leaves out the `runId`, `turnId` and `itemId`, and the runtime context's MCP assignment and binding (with `aggregateDigest`, which hashes the assignment). MCP reaches the Runner through Core's tool authority on every run attachment, never through files the environment publishes. Core already continues a warm session across an MCP-only change (`nativeRuntimeContextsMatch` with tool refresh). The binding is also named per run (`native-mcp:<runId>`). So the digest stays the same across warm turns of one Runner, lease and session, including when the agent's MCP access changes. - `runDigest`: `preparationDigest` plus everything it leaves out: `runId`, `turnId`, `itemId`, and the MCP assignment and binding. It is unique to one run's launch. A plugin that advertises `launchDigestVersion: 1` in its task description gets both fields on `prepare`. **Reason and benefit** A plugin can compare `preparationDigest` with its saved preparation and transfer only the run identity when they match, instead of a full preparation. **Breaking changes** None. Plugins without `launchDigestVersion` get no new fields. ## What Changed - `packages/paperclip-runner/src/live/runner-provider-launch.ts`: add `runnerLaunchPreparationDigest` and `runnerLaunchRunDigest`. The preparation digest drops the MCP assignment and binding; the run digest takes the full requirements and includes them. Both use the same canonical JSON as `runnerLaunchRequirementsDigest`, with their own schema tags so the three digests never collide. Export them from the package and the server's vendored runner shim. - `packages/plugins/sdk/src/environment-tasks.ts`: add optional `preparationDigest` and `runDigest` to the `prepare` operation, and optional `launchDigestVersion: 1` to the task description. The capability gates both digests. - `server/src/services/native-runtime/plugin-runner-task.ts` and `native-session-executor.ts`: send both digests on `prepare` when the description advertises `launchDigestVersion: 1`. - `server/src/services/environment-task-runtime.ts`: recompute both digests from the requirements, and refuse a `prepare` whose digests are missing in part or do not match. - `tests/runner-e2e/source-sdk-public-inputs.json`: refresh the SDK public source pin. - Tests: - the preparation digest is stable across runs and changes when the runner, lease, session, instructions, model, approval policy or settings change; - with a runtime context, an MCP-only change (the run's binding, no binding, a changed access set) keeps the preparation digest and changes the run digest; - a change to the instruction bundle, instruction entry, a skill bundle, the skill set, the working copy or the connection instructions changes the preparation digest; - the run digest changes per run; - Core sends the digests only when advertised; - the runtime refuses forged or partial digests before it calls the plugin. ## Verification - `npx vitest run src/live/runner-launch-preparation.test.ts` in `packages/paperclip-runner`: 26 passed. - `npx vitest run src/services/native-runtime/plugin-runner-task.test.ts src/__tests__/environment-task-runtime.test.ts` in `server`: 160 passed. - `npx vitest run source-sdk-public-inputs.test.mjs source-sdk-evidence.test.ts source-sdk-public-type.test.ts` in `tests/runner-e2e`: 59 passed. - `pnpm typecheck` in `server` and in `packages/paperclip-runner`: clean. - Not run locally: the full suite. CI covers it. ## Risks Low risk. The digests are new optional fields behind a plugin capability version. Core computes them and checks them again before it calls the plugin, so a plugin can rely on them. They prove only that the launch requirements are unchanged. A plugin that skips work on a match must still check its own state, such as files it published into the sandbox. If the runtime context carries values that change per run, the preparation digest changes too, so a plugin does its full preparation. That is safe, but the plugin saves nothing. The MCP binding is the one such value Core sets per run today, and MCP is excluded entirely. A plugin that does publish MCP configuration into its sandbox must not rely on this digest for it. Values outside the requirements are not covered at all: the account identity and credentials, the GitHub broker, project IDs, and the plugin's own runtime revision. A plugin must keep checking those itself. A match lets a plugin skip publishing its launch configuration, not computing the descriptor: `prepare` must still return a launch whose `requirementsDigest` matches the current requirements, which carry the current MCP assignment and binding. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, code search). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
8cc26221af |
perf(native): accept a plugin's in-preparation project-auth check on warm attach (#21710)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Environment plugins can run a native Runner as a plugin task, and a warm session keeps that Runner between turns > - On a warm turn Core attaches the kept Runner to the new run: the plugin prepares the new run, then Core checks managed project authentication > - A warm attach has no submit step, so Core sends a separate `check_project_auth` operation after `prepare` > - That separate operation is one more plugin dispatch and Launcher round trip on every warm turn, about 0.12–0.18 s, and it is serial > - This pull request lets the plugin do the check inside `prepare` and say so, so Core does not send the second operation > - The benefit is a faster warm turn with the same proof and the same failure behavior ## Linked Issues or Issue Description **What existing behavior does this improve?** The warm attach of a plugin Runner task (`plugin-runner-task.ts` `prepareLaunch`). When `launched` is true, Core always calls `checkProjectAuth` after `prepare`. **Subsystem affected** Native runtime, plugin Runner tasks, and the plugin SDK environment task contract. **Current behavior** Cold starts fold the check into submit (`projectAuthCheckAtSubmit`). Warm attaches send `prepare`, then a separate `check_project_auth`. Measured on a plugin sandbox, the separate operation costs 0.12–0.18 s on each warm turn. **Proposed behavior** The `prepared` result has a new optional flag, `projectAuthChecked`. A plugin sets it only when it ran the managed project-authentication check inside this preparation, for this launch's configuration, provider and projects, and the check allowed them. Core then records the same lease proof that an allowed `check_project_auth` records, and skips the separate check. **Reason and benefit** One fewer serial round trip on every warm turn. The plugin already runs the check for this task; this removes a dispatch, not a check. **Breaking changes** None. The flag is optional. A plugin that does not set it, or that saw a refusal, gets the current behavior. Plugins detect the flag with the SDK schema before they send it. ## What Changed - `packages/plugins/sdk/src/environment-tasks.ts`: add optional `projectAuthChecked: true` to the `prepared` result, with a doc comment that defines it. - `server/src/services/environment-task-runtime.ts`: when a `prepared` result has `projectAuthChecked`, record `runnerTaskProjectAuthValidation` for the launch's configuration, provider and projects, without `atSubmit`. This is the proof that `submit` and `read_file` require. - `server/src/services/native-runtime/plugin-runner-task.ts`: do not send `check_project_auth` when the prepared result has `projectAuthChecked`. - Tests: a warm attach with the flag makes only `prepare`; without it, `prepare` then `check_project_auth`. The runtime test covers all four combinations of `projectAuthCheckAtSubmit` and `projectAuthChecked`. ## Verification - `npx vitest run src/services/native-runtime/plugin-runner-task.test.ts src/__tests__/environment-task-runtime.test.ts` in `server/`: 157 passed. - Not run locally: full typecheck and full suite. CI covers them. ## Risks Low risk. Core trusts the flag the same way it trusts a plugin's `check_project_auth` answer and `projectAuthCheckAtSubmit` today. A refusal still goes through Core's own check, so the conflict error is unchanged. The plugin side ships separately and sends the flag only when its SDK accepts it. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, code search, tracing queries). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
554e6d415c |
fix(restart-chaos): credit a follow-up only to a run that started a provider turn, and post a missed follow-up after the run ends (#21692)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The restart-chaos harness runs staging-mix waves across server swaps and grades each task for duplicate side effects, extra turns and follow-up delivery > - In one wave, the grader reported two false failures > - Task t005: the grader credited a follow-up to a run that failed before its provider turn. It then counted the run that really served the follow-up as a duplicate completion and an extra turn > - Task t009: the agent finished before the follow-up window opened. The harness never posted the follow-up, but the grader still required the post > - False failures hide real regressions, so the grader must match what the product did > - This pull request credits a follow-up only to a run that started a provider turn, and posts a missed follow-up after the run ends > - The benefit is that both false failures go away, and the t009 case still tests the product: the product must answer the late follow-up ## Linked Issues or Issue Description Refs #21598 **What happened?** In wave `w-20261114T231451-s7110`, two tasks failed on harness defects, not product defects. - t005 (`T:completion_once`, a duplicate side effect and an extra turn): run 76f30f70 was woken for follow-up comment 3f6a9eb9. It failed before any provider turn, so the agent never received the comment. Recovery run fdf1f793 served the comment. It added the FOLLOWUP line and did not post DONE again (DONE comments = 1). The grader credited 76f30f70 with the answer and flagged fdf1f793 as a second completion and an extra turn. - t009 (`T:follow_up_posted`): the follow-up was due at 23:25:19. The harness posts only during a swap overlap (opened 23:28:02.7) or at the 85% point (about 23:29:15). The agent finished at 23:27:56.0, so the harness never posted. The grader still required the post. **Expected behavior** - t005 grades one completer (60bdd4dc) and no extra turn. fdf1f793 is the run that answered the follow-up. - t009 gets its follow-up after the run ends. The grade then requires the product to answer it. **Steps to reproduce** 1. Build wake evidence with three runs: a done run, a run woken by a follow-up comment that has no provider turn, and a recovery run that re-completes. 2. Grade with `gradeStagingMix`, `completedAtAnchor` and `runsAfterDone`. 3. Drive `stagingMixStep` with the t009 timings: the run ends before the overlap and before the 85% point. ## What Changed - `tests/restart-chaos/wake-evidence.ts`: record `providerTurnStarted` for each run. It is true when the run has a `turn.submitted`, `turn.accepted` or `turn.started` run event. - `tests/restart-chaos/follow-up-delivery.ts`: a run with `providerTurnStarted: false` does not receive its comments. The comments carry to the next run that starts a provider turn. That run is the answering run. Add `undeliveredFollowUpRuns`. Evidence without the field keeps the old behavior. - `tests/restart-chaos/staging-mix-retry.ts`: `runsAfterDone` does not list a run that failed before it could deliver a follow-up as an extra turn. - `tests/restart-chaos/staging-mix.ts`: when no run is active after the hold or the 85% point, `stagingMixStep` posts the follow-up after the run ended. It records the post as an after-end follow-up, so `after_end_follow_ups_served` requires an agent reply and the ledger line. The `follow_up_posted` detail says "after the run ended". - `tests/restart-chaos/observer.ts`: a staging-mix task with an unposted follow-up does not finish yet. - Tests and `tests/restart-chaos/README.md`. ## Verification - Red on master, green with the fix: - `a follow-up whose run failed before any provider turn (w-20261114T231451-s7110 t005) > credits the follow-up to the run that served it, with one completer and no extra turn` - `... > lets only the serving run re-complete, and still fails a later recovery run that answers nothing` - `staging-mix board step > posts a follow-up the run finished before as an after-end follow-up the product must answer (w-20261114T231451-s7110 t009)` - `staging-mix board step > does not post an after-end follow-up between runs before the during-run window closes, or while a run is queued` - `TaskObserver > does not finish a staging-mix task whose run ended before its planned follow-up was posted` - Guards that still grade as failures on master and with the fix: - `... t005 > still fails a recovery re-completion when the run that received the follow-up started its provider turn` - `a done the server rejected as incomplete > keeps a run woken by recovery after a real completion as an extra turn and a duplicate completion (G5c t004)` (#21598) - `completion_once truth table: a later completion that answers nothing` - `wake evidence against real PostgreSQL` now checks `providerTurnStarted` false, then true after a `turn.started` event. - `npx vitest run --config tests/restart-chaos/vitest.config.ts --maxWorkers=3`: 71 files, 823 tests pass. - `tsc -p tests/restart-chaos/tsconfig.json --noEmit`: clean. - Not run: a new wave. The wave's database is gone, so the real t005 evidence cannot get `providerTurnStarted` from the new query. The tests use the real t005 and t009 values. ## Risks - Low risk. The change affects only the restart-chaos harness. - A run that failed after `turn.submitted` but before the provider read the input counts as a delivery. This keeps the grader strict: it can report a false duplicate, but it cannot hide a real one. - A staging-mix task with a planned follow-up that cannot be posted now waits until its deadline instead of resting. The grade still reports the post errors. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), via Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
92308f839d |
test(e2e): use the Coordinator task panel, inbox and plan controls in QA journeys (#21701)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The QA Cloud product journeys check real user paths on a hosted test stack > - #19360 (2026-11-05) made the Coordinator workspace standard; its task side panel starts with no tabs, and the header control opens a tab chooser > - The journeys were written for the classic panel, which opened straight on Properties, and for older inbox and plan controls > - On QA the smoke suite passed 8 of 24 journeys; 13 failures trace to these out-of-date journey steps > - This pull request updates the journeys to use the shipped Coordinator controls the way a person does > - The benefit is that the QA journeys test the product again instead of failing on removed controls ## Linked Issues or Issue Description **What happened?** The QA staging smoke suite passed 8 of 24 journeys. 13 failures come from journey steps that target the classic UI: - Task side panel: the journeys click "Show properties" and expect property rows. In Coordinator the click opens a tab chooser ("Open a side panel tab"), and the user picks Properties or Artifacts. The product's own e2e tests (`tests/e2e/artifact-tab-arrival.spec.ts`, `artifact-uploads.spec.ts`, `disconnect-resilience.spec.ts`) and the chat-catalog runtime cases already use the chooser. - Inbox: desktop Coordinator inbox rows show unread state as a bold title and have no per-row "Mark as read" button (`doc/plans/2026-11-06-chat-contacts-pins.md`). The `r` shortcut marks the hovered row read on the Mine view. - Plan: the Coordinator Plan chip is named "Open Plan revision N in task sidebar", not "Open Plan revision N". **Steps to reproduce** 1. Run `pnpm test:e2e:runner -- --target-config <managed local config>` with task-properties, task-edit-lifecycle, task-document-persistence and project-task-context. 2. See 0/4 pass on master; each stops at the tab chooser. **Expected behavior** The journeys reach the same saved-state checks as before through the shipped controls. ## What Changed - New `product-journeys/task-panel.ts` helper: opens the side panel and selects a tab (Properties or Artifacts) from the chooser, an open tab, or the "Open a new tab" button. It waits for the page instead of a single non-waiting visibility check, and does not close an already open panel. - task-properties, task-edit-lifecycle, task-dependency-resume, project-task-context and task-document-persistence (and routine/agent journeys that reuse `openDocumentAuthoring`) use the helper. task-properties reopens the panel after reload before it checks "You". - inbox-triage: when the row has no visible "Mark as read" button, open the Mine view, blur search, hover the row and press `r` (as `tests/e2e/coordinator-inbox-views.spec.ts` does). - Plan journey: `planPreviewLabel()` accepts the optional " in task sidebar" suffix and still requires the exact revision. - No product code changes. An earlier revision of this PR changed "Show properties" to select Properties; it is reverted because the product e2e suite asserts the chooser is the intended design. All saved-state assertions are unchanged: assignment, labels, status, documents, read state and plan revision are still read back through the API, so a broken feature still fails. ## Verification - Real browser journeys on a managed local instance against unmodified master UI: 4/4 pass (task-edit-lifecycle, task-properties, task-document-persistence, project-task-context); 0/4 before. - Inbox fallback checked by hand on a local instance: an unread task on the Mine view became `isUnreadForMe=false` after blur + hover + `r`. - `tests/runner-e2e` unit tests for product-journeys and plan-revision: 47 files, 1240 tests pass. The task-properties contract test now models the chooser and asserts the opener and the Properties option are used before assignment and after reload. New `planPreviewLabel` test covers revision 12 vs 1 and a missing revision. - QA Cloud rerun follows once the QA harness pin moves to a master commit that contains this change. ## Risks - Low risk: test code only. If the chooser changes again, the helper fails visibly at the panel step. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
328a954cc1 |
test(scaling): add a real-agent ramp harness for hosted instances (#21695)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The control plane must run 1,000 agents at once on one workspace (`doc/plans/2026-11-13-thousand-concurrent-agents.md`) > - Above about 50 agents, the evidence is a load model only. Nobody has run 200 or 1,000 real agents > - Phase 3 of the plan proves the finish line with real runs on hosted instances, in a fixed order > - This pull request adds the harness for those runs. It does not run it against any real instance > - The benefit is that each proof step is one command with a written cost estimate, explicit pass/fail criteria, and a guaranteed cleanup ## Linked Issues or Issue Description **What problem does this solve?** The 1,000-agent plan needs real runs at 50, 200, 500 and 1,000 agents. No tool exists that creates real agents and tasks on a hosted instance, ramps concurrency, measures the control plane, and judges the plan's finish line. **Proposed solution** A script under `scripts/` that ramps real agents through the product API, stops at the first failed criterion, writes a machine-readable result, and always cleans up what it created. **Alternatives considered** Extend the restart-chaos hosted mode. That mode grades release swaps at up to 30 concurrent tasks. The ramp reuses its target guard and its cleanup order, but it has a different workload, different measures and a different stop rule. ## What Changed - `scripts/scale-agents-ramp.ts` and `scripts/scale-ramp/`: the harness. It reads the base URL from a flag or the environment and the credentials from the environment only. It creates or reuses a company, then creates agents, tasks, a project and an optional provider secret. It ramps concurrency in steps (default 10, 50, 100, 200, 500, 1,000) and keeps each agent on one small deterministic task at a time. - It measures per step: tasks started, finished, failed and needing Retry; failed runs; peak running agents; queue wait; time to first token and first activity (from sampled run events); run duration; board GET latency. An optional metrics endpoint supplies event-loop, statement and pool figures. A missing source is reported as "not measured", never as zero. - It evaluates the criteria from the plan's finish line plus harness criteria, writes them into `result.json` and `SUMMARY.md`, and stops at the first failure. - Cleanup runs in a `finally`: pause agents, cancel live runs, archive tasks, terminate and delete agents, delete the stored secret, archive the project and the created company, and wait for sandbox leases. `--cleanup-only <run id>` finishes a cleanup after a crash. `--dry-run` prints the plan and the estimate and makes no request. - Target guard: `--target-kind` is required and must match the host. Shared staging and production hosts need a second explicit flag. The script asks for the host name before it starts unless `--yes` is given. A run above the spend threshold needs `--confirm-spend <max USD>`. - `scripts/scale-ramp/scale-ramp.test.ts`: 28 tests against an in-process fake API with a virtual clock. They cover ramp logic, stop on failure, criteria evaluation, cleanup after a mid-run crash and an interrupt, `--cleanup-only` with and without a state file, dry run, the production-host guard, the spend gate, and that credentials never appear in output or files. - `scripts/scale-ramp/tsconfig.json` is added to the test programs that CI typechecks. The test file joins the `test:release-registry` node test list, which CI runs. - `doc/plans/2026-11-13-thousand-concurrent-agents.md`: a Phase 3 runbook with commands, cost, criteria, proof order, target prerequisites and open risks. ## Verification - `node scripts/ci-profile/node-test.mjs scripts/scale-ramp/scale-ramp.test.ts scripts/typecheck-test-programs.test.mjs`: 47 tests pass. - `node node_modules/typescript/bin/tsc -p scripts/scale-ramp/tsconfig.json`: no errors. - `node cli/node_modules/tsx/dist/cli.mjs scripts/scale-agents-ramp.ts --base-url https://scale1.test.paperclip.app --target-kind test-stack --steps 50 --hold-minutes 30 --dry-run` prints the plan, the estimate and the criteria, and makes no request. - The harness was not run against any real instance. That needs the owner's go-ahead. ## Risks - Low risk for the product: the change adds a script, tests and docs only. No server code changes. - A real run creates real agents, tasks and sandboxes and spends money. The guard, the typed confirmation, the spend gate, the task cap and the recorded-spend stop limit this. The runbook lists the open risks. - The cost defaults are assumptions until a first real run records actual spend. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
ac114d9479 |
refactor: cut DEVELOPING.md to a setup and workflow guide (#21670)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - `AGENTS.md` tells every contributor and agent to read `doc/DEVELOPING.md` for setup, commands, testing, and worktrees > - That file grew to 5,495 lines. Most of it was notes on how one subsystem behaves, added one pull request at a time > - Readers could not find the commands. Agents also read behavior notes as rules > - This pull request cuts `doc/DEVELOPING.md` to a 660-line setup and workflow guide. It moves the subsystem notes to focused docs > - The benefit is a guide that a person or an agent can read in full, and a clear place for each kind of note ## Linked Issues or Issue Description **Issue type** Outdated or confusing documentation. **Where is the issue?** `doc/DEVELOPING.md` **What's wrong?** The file was 5,495 lines. It opened with 112 lines about case handoffs before it reached Prerequisites. Unrelated sections sat below the "OpenClaw Docker UI One-Command Script" heading. Statements about current behavior read like policy. **Suggested fix** Keep setup, commands, and workflow in `doc/DEVELOPING.md`. Move each subsystem note to a doc for that subsystem. Add an index. ## What Changed - `doc/DEVELOPING.md` is now 660 lines: prerequisites, start dev, health checks, local run, test drive, Docker, instance layout, database, storage, secrets, worktrees, test commands, Storybook, CLI, and an index of the other docs. - 15 new docs hold the moved text without wording changes: `ci-workflows`, `test-infrastructure`, `worktrees`, `hot-restart-deploys`, `heartbeat-internals`, `runner-internals`, `workspace-repositories`, `workspace-storage`, `skills-and-teams-catalog`, `ui-internals`, `case-handoffs`, `chat-connector-development`, `instance-configuration`, `integration-api-notes`, `database-backups`. - "Publish a branch Storybook" moved to `doc/STORYBOOK-DEPLOYMENT.md`. The privacy rollout notes moved to `doc/ISSUE-PRIVACY.md`. - Deleted about 280 lines: the story-by-story Storybook notes, "Agent Artifact Uploads" (a copy of `doc/AGENT-ARTIFACTS.md`), two pointer-only sections, "Workspace feature checks (2026-10-09)", and "Chat and task navigation caching". The test-drive build-lock paragraph is now four lines. - Repointed every inbound link and every script or workflow comment that named a moved section. - `AGENTS.md` now says where a behavior note goes, so the guide does not grow again. ## Verification - A script split the file by line range. It asserts that each of the 5,495 source lines is kept, moved, or dropped exactly one time. - A link check on every changed Markdown file found 0 missing files and 0 missing anchors. - `node scripts/check-maps.mjs` passes. - `node --test .github/scripts/tests/*.test.mjs`: 808 pass, 0 fail. - `node --check` passes on the three scripts with changed comments. - `pnpm verify` was not run. This worktree has no installed dependencies, and the change has no runtime code. ## Risks - Low risk. The change is documentation and comments only. - A link to a moved section from outside this repository will land on the top of `doc/DEVELOPING.md`. - The new docs are not in the audited prose list in `.github/scripts/ci-inputs.mjs`. An edit to one of them runs full CI. An edit to `doc/DEVELOPING.md` does not. This PR does not change that list. ## Model Used - Claude Fable 5.2 (`claude-fable-5-2`), Claude Code desktop, with tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [ ] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com> |
||
|
|
fc9e4cbbb3 |
perf(ui): show a returning task's saved chat before any server response (#21671)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The board UI opens a task as a chat. A returning browser restores a saved read cache from local storage before the first render. > - #21650 made a returning hard load share the one chat bootstrap. But the saved cache held no chat, so the first chat frame still waited for that response. > - The owner approved saving recent comments in the browser on 2026-11-14, so that cached tasks show their chat at once. > - This pull request saves the first chat frame of the 20 most recently opened tasks, with size, age, revocation and isolation guardrails. The restored chat shows at once and revalidates through the bootstrap. > - It also adopts the hard-load bootstrap at app start and sends the other live-run readers through it, so readers that start before the task page mounts do not send their own requests. > - The benefit is that a hard load of a task the user opened before shows its conversation before any server response. ## Linked Issues or Issue Description **Describe the enhancement** A hard load of a task that the user opened before still waits for one server round trip before its chat appears. The saved read cache did not include comments. **Current behavior** With every API response delayed by 800 ms, a returning load shows the chat at 892 ms (median). The frame waits for the chat bootstrap response. **Proposed behavior** The saved read cache also keeps the first chat frame of recently opened tasks. A returning load shows that frame before any response, then revalidates through the bootstrap. **Why** The owner approved saving recent comments in the browser for this on 2026-11-14. Cold and cached loads must both be as fast as possible. Related: #21650 (returning load shares the chat bootstrap), #20594 and #20591 (returning inbox from the saved cache), #20592 and #21557 (cold task load from one bootstrap). ## What Changed - `ui/src/lib/persisted-query-cache.ts`: saves the reads the first chat frame waits for (`taskChatFirstFrameRef`): recent comments, their input facts, recent activity, the recent run window, interactions, attachment metadata, live and active runs, work products and the plan document. - Only for the 20 saved recent tasks, in the record of the task's own user and company. - At most 50 comments per task (the newest; `hasMore` is set), 192 KiB per task, 1 MiB in total, newest tasks first. - If storage refuses a record, the record is saved again without chat, so chat never displaces the rest of the cache. - All other field exclusions (tokens, credentials, environment values, embedded documents) still apply. Only `comments` is allowed, and only for these keys. - Expiry is the same as every other entry: 24 hours after the read. - A 401, 403 or 404 on the task or on any of its chat reads removes the task and its saved chat from memory and storage at once. - `ui/src/lib/issue-chat-bootstrap.ts` and `ui/src/main.tsx`: `adoptHardLoadChatBootstrap` adopts the bootstrap that the HTML started, at app start, under the task's restored id and identifier. Readers that start before the task page mounts share it. - `IssueDetail.tsx`, `LiveRunWidget.tsx`, `IssueRunLedger.tsx`, `useTaskPluginPanels.ts`: their `live-runs` reads share the loading bootstrap. Before, whichever observer mounted first could send its own request. - `doc/DEVELOPING.md`: the read-cache rules now state what is saved, the bounds, why, and the owner's approval on 2026-11-14. - Tests: `ui/src/lib/saved-task-chat.test.ts` (guardrails), a new case in `restored-task-chat-bootstrap.test.ts`, and an always-on perf test: "a returning task load shows its saved chat before any API response, then revalidates". ## Verification Local production build, local_trusted instance, CDP delay on every API request, fresh browser context per run, JavaScript in the HTTP cache. Time to the revealed history with the latest message and an editable composer. Saved read cache present against cleared (other storage kept), 5 alternating pairs each. | API delay | Build | Cleared | Present | Requests before frame (present) | | ---: | --- | ---: | ---: | ---: | | 800 ms | master `36101ebcb9` | 1,911 ms (1,895–1,920) | 892 ms (887–898), released by the bootstrap | 33 | | 800 ms | this PR | 1,922 ms (1,912–1,936) | 571 ms (556–588), before any response | 36 | | 300 ms | master | 916 ms (907–920) | 570 ms (564–576) | 37 | | 300 ms | this PR | 928 ms (910–998) | 586 ms (568–622) | 37 | - At 300 ms the bootstrap answers before the page mounts on both builds, so they are the same. At 800 ms the saved chat removes the wait for the bootstrap: −321 ms against master. - The new perf test holds every response for 2.5 s. It fails on master (frame at 2,595 ms, first response at 2,524 ms) and passes with this PR. - Guardrail tests, red/green: I disabled each guardrail in turn in `persisted-query-cache.ts` and ran `saved-task-chat.test.ts`. Each mutant made its test fail, and all 19 tests pass with the code. Mutants: no chat saved, no 50-comment cap, no 20-task cap, no per-task budget, no total budget, no retry without chat when storage refuses, no 24-hour expiry, no revocation on a chat read, chat stored in the wrong company record, and no field exclusions. - Sign-out and other-user isolation use the existing per-user records and their clearing (no new code). The tests cover them for saved chat. - "Never overwrites newer data" is the query cache's hydrate rule; the test covers it. "The bootstrap replaces saved data" uses the restored-entry rule from #21650; the test covers it. - App-start adoption test: it fails when the adoption call is removed, and passes with it. - Focused Vitest: 25 files that use the changed modules, including `IssueDetail.test.tsx` and `TaskChatThread.test.tsx`: 1,156 passed. `pnpm --filter @paperclipai/ui typecheck` passes. - Browser journeys against this build: `layout-stability.spec.ts`, `plan-startup.spec.ts`, `cold-bootstrap.perf.spec.ts`: 20 passed (2 opt-in benchmarks skipped). Leftover early reads on staging (one run after #21650 started `issues/:ref?view=core` and `issues/:ref/live-runs` before the frame): I could not reproduce them locally. That included sidebar Recents and a bootstrap delayed to 1.2 s with CDP initiator stacks. Both reads would go to the server only if no bootstrap was being tracked for the task, or if an observer used a direct read. This PR removes both openings: the bootstrap is adopted at app start, and every `live-runs` observer shares it. I will check the request timeline on staging after deploy. ## Risks - Privacy: recent comments of up to 20 recent tasks now stay in local storage for up to 24 hours, or until sign-out. The owner approved this on 2026-11-14. The bounds, the revocation and the field exclusions are tested. Saved data is per user and company, and sign-out clears it in every tab. - Storage: saved chat is limited to 1 MiB in total. A record that storage refuses is saved again without chat. - A saved chat can be up to 24 hours old when it shows. It is marked for revalidation, and the bootstrap answer replaces it within one round trip. A refused revalidation removes it. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, browser automation through Playwright and Chrome). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
36101ebcb9 |
fix(evals): grade five cohort-14 chat cases by saved state and clicked links (#21667)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The chat-feature eval catalog runs real chat journeys. It grades saved state, rendered links and run ledgers. > - Chat-eval cohort 14 (OpenCode with `openrouter/deepseek/deepseek-v4.1-flash-1017`, master `d4498ae7ac`) failed five cases where the agent behaviour was correct or acceptable. > - Four failures came from graders that read free-form reply text or required a setup label to be absent. One failure came from a harness race with a client-side route rewrite. > - This pull request fixes those graders and that harness step. It does not change the product. > - The benefit is that these cases measure the feature again: the retained correct replies pass, and the relevant broken behaviour still fails. ## Linked Issues or Issue Description **What happened?** In chat-eval cohort 14 (run 43637708886), five cases failed although the agent behaved correctly: - `projects.share-project`: a regex over the free-form reply rejected a correct board-only handoff that linked the project access settings. - `teams.install-team`: the grader required the literal skill slug, name or key. The reply named the blocked skill "document-maintenance"; the slug is `doc-maintenance`. - `steering.discard-queued-message`: the final plan removed the contact section but cited the setup label in a "Latest direction" note. The grader required the label to be absent everywhere. - `workspaces.inspect-git`: the grader required the reply to name `existing-notes.txt`. The person asked about "website files", and the agent correctly left the unrelated file out. - `runs-adapters.reset-session`: the harness opened the agent actions menu on `/agents/<id>/runtime`. The app then rewrote the route to `/agents/<urlKey>/runtime` and redrew the page, so the "Reset sessions" click timed out. **Expected behavior** Correct agent behaviour passes. Grading uses saved state, setup literals and rendered or clicked links, not reply wording. Broken behaviour (an agent-made grant, a kept contact section, a changed unrelated file, a missing first reply) still fails. **Steps to reproduce** 1. Use the retained cohort-14 evidence for the five cases. 2. Run the old graders against the retained replies and revisions. Each case fails. 3. For reset-session, open an agent profile by id while the app rewrites the route after a delay, then open the actions menu before the rewrite completes. **Paperclip version or commit** master `d4498ae7ac` (cohort 14); fixed on top of current master. ## What Changed - `projects.share-project` (`internal-privacy.ts`): removed the reply regex. The grade now requires (a) a rendered link to the project's configuration page (where Manage access lives) that the harness clicks and that opens the Manage access control, and (b) saved state with no agent-made access grant: Riley is absent from the membership before the owner acts, and no agent `project.access_member_added` or visibility `project.updated` activity exists. The owner step still completes when the link is missing, so later phases keep their evidence. - `teams.install-team` (`external-team-conditional.ts`): removed `namesTeamSkill`. The case contract is "reports conflicts or partial failures truthfully"; truthfulness is meaning, so a person reviews it. The partial outcome is already graded from saved state: the catalog skill is still missing beside the local skill that holds its slug, and the installer recorded a warning. A skill link was not chosen because the blocked catalog skill has no installed page to link, and the contract only asks for links to the installed agents and project. The check is now "a first reply was delivered". - `steering.discard-queued-message` (`runtime-discard.ts`): the later contact request now names a contact person and an attempt-specific phone (a setup change in the harness act step, not the YAML). The grade collects the contact section's distinctive whole lines from the revision that saved it: lines in the section whose heading carries the label, contact name or phone, plus lines that carry one of those markers, excluding lines that also appear elsewhere in that revision. None of those whole lines may remain in the final revision, and the contact name and phone may not appear anywhere in it (substring match on those distinctive literals). A note that quotes the label inside a longer line, and shared lines such as `---`, do not count. - `workspaces.inspect-git` (`runtime-git.ts`): stopped requiring `existing-notes.txt` in the reply. The question asks about website files, and the unrelated file's bytes are still checked unchanged. No YAML change, so no TestOps manifest change. - `runs-adapters.reset-session` (`runtime-support.ts`): `openRuntimeAgentProfile` now waits until the URL settles on the agent's canonical route key (`urlKey`, or the same `deriveAgentUrlKey` the app uses) before callers use profile controls. This also covers the other callers of this helper. ## Verification - Red/green tests: - `internal-privacy-share.test.ts`: the retained cohort-14 reply and a reworded reply pass. An agent grant, an agent visibility change, a pre-owner membership, a root-only link and an unopened link fail. The old regex returns false on the retained reply. - `external-team-conditional.test.ts`: the retained cohort-14 reply passes. The same reply fails when saved state shows no blocked skill or no installer warning. A missing first reply fails. - `runtime-discard.test.ts`: the retained cohort-14 plan passes with the label still cited in a note. Shared lines remaining passes. Removing only the contact name and phone fails. Keeping the heading line, a contact detail line, or reformatted details with the contact name fails. I also ran the new check on the full retained revisions: retained passes, kept section fails. - `runtime-git.test.ts`: the retained cohort-14 reply passes. A changed unrelated file still fails. - `runtime-support-profile-route.test.ts` (real Chromium): the profile rewrites the route after 0, 400 and 1500 ms. With the fix, "Reset sessions" is clicked on the settled page. Without the fix, the click times out (the cohort-14 failure). - `npx vitest run --config tests/runner-e2e/vitest.config.ts chat-catalog`: 291 files passed. - `pnpm test:e2e:runner:unit`: all passed (one unrelated hook timeout under load passed on rerun). - `pnpm test:e2e:runner:typecheck`: passed. - Narrow paid run of the five full case ids on this branch: run 43643474282 (results are added in a comment). ## Risks - Low risk. Only eval graders, fixtures and one harness wait change. - `teams.install-team` no longer checks how the reply names the blocked skill. A person reviews that meaning; saved state still proves the partial outcome. - `projects.share-project` now needs a link to the project configuration page. A reply that links only the project overview fails `owner-access-page-linked`. - `steering.discard-queued-message` does not detect a contact section that the final revision rewrites line by line with new wording. A person reviews that. The later request text changed (it now lists a contact name and phone), so earlier retained runs used a different request. ## Model Used - Claude, model ID `claude-opus-5-5` (Claude Code agent with tool use and code execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
ecdeed5e7e |
perf(ui): open a task restored from the saved read cache through the chat bootstrap (#21650)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The board UI opens a task page as a chat. A hard load of a task URL starts one chat bootstrap request before the app's JavaScript runs. > - A returning browser also restores a saved read cache from local storage. This cache holds the core fields of recently opened tasks, but no chat history. > - The task page started the bootstrap only when the core task was not cached. A restored core task skipped it. The early bootstrap request ran, but nothing used its answer. > - So every first-frame read went to its own endpoint, and the comment input facts waited for the recent comments. A returning load was slower than a cold load: on staging, 3.6–3.8 s against 1.9–2.4 s. > - This pull request treats a task that only the saved cache holds as cold for the chat. The page adopts the bootstrap, the restored task's own revalidation shares it, and the answer replaces the restored data. > - The benefit is that a returning load is now faster than a cold load, not slower. ## Linked Issues or Issue Description **Describe the bug** A hard load of a task page is slower when the browser has a saved read cache than when it has none. Measured on staging on 2026-11-14 (hard load, JavaScript in the HTTP cache, 3 alternating pairs on one task): saved cache cleared 1.9–2.4 s, saved cache present 3.6–3.8 s, until the chat composer exists. **Steps to reproduce** 1. Open a task page so that the saved read cache (`paperclip:query-cache:v1:*`) holds it. 2. Close the tab and open the same task URL again. 3. Compare the time to the first chat frame with a load where the saved read cache is cleared. **Expected behavior** A load with a saved read cache is at least as fast as a load without one. The chat's first-frame reads share the one bootstrap request. **Actual behavior** With a saved read cache, the bootstrap request runs but its answer is not used. About 16 more per-endpoint reads run before the frame, and the frame waits for a second, serial wave of reads. Related: #20355 (chat bootstrap endpoints), #20592 (cold task open from one bootstrap), #20591 and #20594 (returning Mine inbox from the saved read cache), #21557 (cold-load perf spec). ## What Changed - `ui/src/lib/persisted-query-cache.ts`: records which queries a page restored from the saved read cache. `isRestoredSavedRead(client, key)` is true until a read or a local write in this page replaces that data. - `ui/src/pages/IssueDetail.tsx`: starts (or adopts) the chat bootstrap when the core task is missing or only restored. It passes the restored task's id and identifier as aliases, so reads keyed by the id share the request while it loads. - `ui/src/lib/issue-chat-bootstrap.ts`: `readFromIssueChatBootstrap` lets a key that holds only restored data share the bootstrap, like an empty key. `startIssueChatBootstrap` accepts aliases. - `ui/src/lib/issueChatBootstrapCache.ts` and `ui/src/lib/issueDetailCache.ts`: the bootstrap answer replaces restored data (core task under every ref). The restored core task's revalidation shares the bootstrap. A core task that was read in this page keeps its own small read. - `ui/src/hooks/useIssuePlanDocument.ts`: the first plan read shares the bootstrap's plan projection instead of reading `documents/plan` before the frame. - `tests/perf/issue-detail/cold-bootstrap.perf.spec.ts`: an always-on request-shape test for a returning load with a saved read cache, and an opt-in benchmark of alternating pairs (saved cache present and cleared). The harness now clears storage after the warm-up page closes, because closing the app saves its read cache again. - `ui/src/lib/restored-task-chat-bootstrap.test.ts`: unit tests for the restore-then-bootstrap path. - `tests/perf/issue-detail/README.md`: documents the returning-load test and benchmark. ## Verification Local production build (`PAPERCLIP_UI_DEV_MIDDLEWARE=false`), local_trusted instance, CDP delay of 300 ms on every API request, fresh browser context per run, JavaScript in the HTTP cache. Time from navigation start to the revealed history with the latest message and an editable composer. | Build | Saved read cache | Runs | Median | Spread | API requests before frame | Frame released by | | --- | --- | ---: | ---: | ---: | ---: | --- | | master `95a6ab31e6` | cleared | 5 | 913 ms | 908–957 | 33 | company list (shell stage 2) | | master `95a6ab31e6` | present | 5 | 1,204 ms | 1,178–1,215 | 49 | `activity?view=comment-inputs` (serial second wave) | | this PR | cleared | 7 | 942 ms | 925–997 | 35 | company list (shell stage 2) | | this PR | present | 7 | 573 ms | 566–658 | 37 | app boot and route mount (all task data is the bootstrap) | - Paired delta with this PR (present − cleared): median −376 ms over 7 pairs, all negative (−267 to −400). On master it was +292 ms over 5 pairs, all positive. - With an 800 ms API delay (this PR, 5 pairs): present 890 ms, released by the bootstrap response; cleared 1,910 ms. - Cold load with no app storage (existing benchmark, 7 runs): master 910 ms, this PR 939 ms. The host load average was about 20 during the PR run (other work on the machine). Requests before the frame are 35 in both. I make no claim of a cold change. - Request-shape tests (`cold-bootstrap.perf.spec.ts`): the new returning-load test fails on master (the timeline shows `?view=core`, `recent-comments`, `activity`, `runs`, `interactions`, `attachments`, `live-runs` before the frame) and passes with this PR. The existing cold test passes. - Unit tests: `restored-task-chat-bootstrap.test.ts` has 10 cases. With the behavior changes reverted (the restore marker kept so the file imports), 3 fail: the bootstrap refreshes the restored task and fills the chat, id-keyed reads share the loading bootstrap, and a partial answer. All 10 pass with this PR. The other 7 cases are guards that hold on both: a stale refresh reads the server, a failed bootstrap falls back, a 403 removes the restored task from memory and storage, another company's answer is not seeded, an answer is not used after another user signs in, and another user's saved task is not restored. - Focused Vitest: 27 files that use the changed modules, including `IssueDetail.test.tsx` and `TaskChatThread.test.tsx`: 1,208 passed. - Browser layout journeys against the PR build: `layout-stability.spec.ts` and `plan-startup.spec.ts`, 17 of 17 passed (reveal order, reading anchor, older history, Back, comment links, drafts during background refresh, failed and stalled reads with Retry, plan projection with stalled plan reads). - `pnpm verify` exit 2 (the README is a global CI input), so required CI supplies the broad coverage. Commands: ```sh cd ui && npx vitest run src/lib/restored-task-chat-bootstrap.test.ts PAPERCLIP_ISSUE_PERF_BASE_URL=http://127.0.0.1:<port> pnpm exec playwright test --config tests/perf/issue-detail/playwright.config.ts cold-bootstrap PAPERCLIP_ISSUE_PERF_BASE_URL=http://127.0.0.1:<port> PAPERCLIP_ISSUE_PERF_COLD_RUNS=7 pnpm exec playwright test --config tests/perf/issue-detail/playwright.config.ts cold-bootstrap -g returning ``` ## Risks - Low risk. The change only affects data that the saved read cache restored and that no read in this page has confirmed. Data read in this page keeps the old rule: a refresh asks the server. - The saved read cache content is unchanged. It still excludes comments and bootstrap payloads, so the first chat frame still needs one response (the bootstrap). It cannot appear before any task response without saving chat history, which the documented allowlist excludes. - If the bootstrap fails, stalls or omits the task, each read falls back to its own request, as on a cold load. A 403 or 404 on the restored task's fallback read removes it from memory and storage, as before. - Account changes: a bootstrap answer is bound to the session's user, as before. The saved cache restores data only for the same user. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, browser automation through Playwright). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
9b581d689b |
fix(chat): report an outcome once and create no run when an earlier reply gave it (#21489)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - In agent chat, a finished delegated task or a declined tool action wakes the chat agent so it tells the person > - #21401 stopped a second completion report when an earlier reply already gave it > - But the suppressed completion still created a zero-second heartbeat run, and declined tool actions still got a second report > - Run accounting must count only real turns, and the person must get each outcome once > - This pull request settles an already-reported outcome before it becomes a run, and applies the same rule to declined tool actions > - The benefit is one report per outcome, no phantom run, and no extra model call ## Linked Issues or Issue Description Refs #21401 **What happened?** Chat eval cohort 13 (OpenCode, `openrouter/deepseek/deepseek-v4.1-flash-1017`, master `df1b12ad8e`) showed three problems after #21401. 1. `questions-and-approvals.duplicate-answer`: the completion wake was suppressed, but it still created a 0-second run (`runtimeMode: legacy`, no usage). The bounded-native-runs check rejected that run. 2. `skills.studio-test`: the CEO's single reply reported the Studio result after it read the test task done (0 passed, 1 failed, links COM-2). The completion was then suppressed, but the eval waited for a separate completion run and timed out after 240 s. 3. `questions-and-approvals.decline-tool`: the person declined the tool review one second after the turn for their message started. That turn read the declined action and said so. The decline continuation, which waited behind that turn, then posted the same declined outcome again. **Expected behavior** Each finished task or declined tool action is reported once in the conversation. An outcome that an earlier reply already reported creates no heartbeat run. **Steps to reproduce** 1. In chat, delegate a short task, or start a governed tool read that needs approval. 2. While it waits, send a message that wakes the chat agent. 3. Let the task finish, or decline the tool review, while that turn runs. The turn reads the outcome and reports it. 4. See the completion wake become a 0-second run, or see a second decline report from the decline continuation. **Paperclip version or commit** master `df1b12ad8e` ## What Changed - `server/src/services/chat-completion-delivery.ts`: `settleReportedChatCompletions` marks each undispatched completion that an earlier reply reported as delivered, with that reply as its response. It returns true only when nothing else is owed. The outbox `deliver` also checks this before it requests a wake, so no wake and no run exist for an already-reported completion. - `server/src/modules/wake-queue`: a new transaction port, `skipAlreadyReportedChatWake`, runs in the release drain before a deferred wake is promoted. When the earlier reply already reported every outcome of the wake, the wake ends `skipped` ("Already reported by an earlier reply in this conversation") and no run is created. Wakes that carry queued comments or a failed-chat retry are never skipped. - `server/src/services/chat-interaction-outcome-report.ts` (new): the same structural rule for declined tool actions. A conversation agent run that lists its interactions records `chat_interaction.declined_read` for each declined tool action. The decline continuation is settled when the same agent replied in the conversation after the decline, from a run with that receipt, posted after the read. The rule covers only declined tool-action confirmations. Answers, plan reviews and accepted actions keep their own turn. - `server/src/routes/issues.ts`: `GET /issues/:id/interactions` calls the recorder for agent runs. - The run-start check from #21401 stays as a fallback for a completion turn that was admitted before the earlier reply landed. - `tests/runner-e2e/chat-catalog-domains/integrator-skill-studio.ts`: the Studio oracle (v5) accepts either the completion turn's reply or an earlier CEO reply as the delivery. The earlier reply counts only from saved state: no completion run exists, the replying run has a `chat_handoff.completed_read` receipt for the test task, the reply was posted after that read, it is the only such reply, and it links the saved result. No wording check is added. - `doc/run-log-events.md`: documents both read receipts and the skipped wake outcome. ## Verification - `cd server && npx vitest run src/__tests__/chat-completion-delivery.test.ts src/modules/wake-queue/adapters/chat-declined-outcome-reported.test.ts src/modules/wake-queue/application/use-cases.test.ts`: all pass. - Red: with the drain check and the outbox check disabled, 5 tests fail: "starts no completion wake or run ...", "settles a deferred completion wake without a run ... (true)", "settles the decline continuation with no run ...", and both "settles an already-reported chat wake without a run" cases. - Negative controls stay green: a linking reply without a done read, a decline the earlier turn never read, a non-tool-action card, and an accepted card all still run their continuation. - `npx vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/chat-catalog-domains/integrator-skill-studio.test.ts`: 34 pass. Red on the old oracle: "accepts the earlier reply the server took as the completion report" fails. New negative cases reject no done read, a read of another task, a reply before the read, a reply from another run, two replies, an extra completion run, and a missing task link. - A narrow paid eval run of `duplicate-answer`, `studio-test` and `decline-tool` on this branch follows in a comment. ## Risks - A completion turn admitted before the earlier reply landed (a narrow race) still ends as a short succeeded run with no model call, as in #21401. - If an earlier reply read the outcome but did not mention it, the person does not get a second message. The read receipt, the reply-after-read order and, for completions, the task link make this unlikely. - The decline rule covers only declined tool-action confirmations. Other interaction outcomes are unchanged. ## Model Used - Claude, claude-opus-5-5, with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
64f3abbdef |
test(evals): write the hard-stop setup note in plain words (#21487)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The chat eval catalog checks that the chat agent explains a budget hard stop to a person > - The `budgets-costs.explain-hard-stop` setup puts a note on the conversation. The note said the receipt was "an imported synthetic accounting fixture, not live model-provider billing" > - In chat-eval cohort 13, the agent repeated that eval jargon to the person > - This pull request rewrites the setup note in plain words, the same way #21408 and #21429 did for spend-breakdown > - The benefit is that the eval tests the explanation a real person would get, without setup jargon to copy ## Linked Issues or Issue Description Refs #21408 Refs #21429 **What happened?** Chat-eval cohort 13 ran OpenCode with `openrouter/deepseek/deepseek-v4.1-flash-1017` on master `df1b12ad8e`. The `budgets-costs.explain-hard-stop` reply gave a correct diagnosis, but it said "The charge is an imported accounting fixture, not live model-provider billing". That wording came from the eval's own setup note. **Expected behavior** The setup note uses plain words. It keeps the references the scenario needs, and it does not give the agent test jargon to repeat. **Steps to reproduce** 1. Run `chat-features.runner-opencode.local.budgets-costs.explain-hard-stop`. 2. Read the conversation note and the reply. ## What Changed - `budgetIncidentSourceDescription` in `tests/runner-e2e/chat-catalog-domains/identity-budget-incidents.ts` builds the note. The note names the task, the recorded hard-stop incident and the separately paused agent. It no longer says that the receipt is an imported synthetic fixture. - The grader is unchanged. It already proves the receipt's origin from saved state (`receiptOrigin`), not from the reply. The note's task identifier, incident reference and paused-agent name are unchanged. The grader matches no literal from the note. - TestOps manifest: no change is necessary. The case's oracle version, source file list, query, context and expected outcome are unchanged. ## Verification - `npx vitest run -c tests/runner-e2e/vitest.config.ts chat-catalog-domains/identity-budget-incidents.test.ts` (16 tests pass). - Red/green: the new test checks that the note keeps the task, incident and paused-agent references and contains none of the setup jargon. It fails on master (no plain-words note) and passes with this change. - A narrow paid run of `budgets-costs.explain-hard-stop` (with the two sibling cohort-13 PRs) ran on a combined branch. The results are in a PR comment. ## Risks - Low risk. Only eval setup text changes. The scenario difficulty is the same: the note states the same facts about the stop and the manual pause, without the test-origin remark. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), used through Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
0ab63cf4d9 |
fix(restart-chaos): ignore a done the server rejected as incomplete (#21598)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The restart-chaos harness runs staging-mix waves and grades each task for duplicated side effects and extra turns > - The grader counts a run as a completer when its last accepted result reports `done` > - The server can reject that `done` as incomplete. The status arbiter then queues a same-agent continuation wake with key `native-completion-incomplete` > - The grader ignores the rejection. It reports a false duplicate completion and anchors completion on the rejected run, so the continuation run that finished the work shows as an extra turn > - This pull request marks a `done` as rejected when a `native-completion-incomplete` wake for the issue follows it, and ignores that `done` unless the same run posted the DONE comment > - The benefit is that wave results stop reporting false duplicates and false extra turns, so real regressions stay visible ## Linked Issues or Issue Description **What happened?** In wave `w-20261114T081656-s7105`, task t005, run d80d3ab3 reported `done` with no DONE comment. The server rejected it as incomplete and queued a `native-completion-incomplete` continuation wake. Run a777038c then posted the only DONE comment and completed the issue. The grader reported "duplicate completion: runs d80d3ab3, a777038c" and listed a777038c as an extra turn. **Expected behavior** The grader reports one completer (a777038c), anchors completion on a777038c, and lists no extra turn. **Steps to reproduce** 1. Build wake evidence where run A has an accepted `done` result and no DONE comment. 2. Add a continuation wake with payload `continuationIdempotencyKey: "native-completion-incomplete"` created after A's result. 3. Add run B from that wake. B posts the DONE comment and reports `done`. 4. Grade the task with `gradeStagingMix`, `completedAtAnchor` and `runsAfterDone`. **Additional context** This is a test-harness change only. It changes no product code. ## What Changed - `tests/restart-chaos/follow-up-delivery.ts`: add `rejectedDoneRuns`. It attributes each `native-completion-incomplete` wake to the latest run whose accepted `done` came at or before the wake. - `tests/restart-chaos/wake-evidence.ts` (`completionEvidence`) and `tests/restart-chaos/staging-mix-retry.ts` (`evidenceCompletion`): set a new optional `doneRejected` flag on each completion run. - `tests/restart-chaos/staging-mix.ts`: `completedAtAnchor` and `completionRunIds` ignore a rejected `done`, unless that run posted the DONE comment. - `tests/restart-chaos/staging-mix-retry.test.ts`: add the t005 red/green test, a test for a rejected run that posted DONE itself, and a G5c t004 guard. - `tests/restart-chaos/README.md`: document the rule. ## Verification - Red on master, green with the fix: `a done the server rejected as incomplete > anchors on the run that finished the work, with one completer and no extra turn (w-20261114T081656-s7105 t005)`. On master the anchor is the rejected run's result time (1794646168026) instead of the finisher's (1794646218506). - Guard, green on master and with the fix: `a done the server rejected as incomplete > keeps a run woken by recovery after a real completion as an extra turn and a duplicate completion (G5c t004)`. - `npx vitest run --config tests/restart-chaos/vitest.config.ts --maxWorkers=3`: 71 files, 816 tests pass. - `tsc -p tests/restart-chaos/tsconfig.json --noEmit`: clean. - The real t005 evidence from the wave now grades as one completer (a777038c) with no extra turns. - `pnpm verify` exits 2 because `follow-up-delivery.ts` is a global CI input. Required CI supplies the broad coverage. ## Risks - Low risk. The change affects only the restart-chaos grader. - The grader trusts the wake key. If the server queued `native-completion-incomplete` for another reason, the grader would skip the latest `done` before it. The server queues this key only from the incomplete branch of `arbitrateNativeStatus`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), via Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
0b2b7aa737 |
chore(lockfile): refresh pnpm lockfiles (#21604)
## Thinking Path > - Paperclip uses a dependency lock for reproducible builds. > - Dependency changes on master need matching lock entries. > - Paid eval reports also pin a reviewed offline correction. > - Reports must install before paid model work starts. > - This automated PR keeps the lock and report proof together. ## Linked Issues or Issue Description **What happened?** Dependency manifests on master changed after the last committed lock refresh. **Expected behavior** Frozen installs must match those manifests. Eval reports must retain matching offline lock proof. **Steps to reproduce** Run a frozen install before and after this lock refresh. ## What Changed - Refresh the dependency lock from the manifests on master. - Refresh the sandbox provider lockfiles from their manifests on master. - Include the matching offline report correction and hashes. - Resync the SDK public source proof's lock pins from the refreshed lock. ## Verification - Pinned pnpm resolution completed with lifecycle scripts disabled. - The report proof refresh accepted the reviewed committed baseline. - The workflow rejected changes outside the lock and report proof files. ## Risks The lockfile bot merges this PR without CI or review when every added package version is at least 3 days old on npm. Otherwise the PR waits for a maintainer, and the daily refresh tries again. The bot merges only the exact commit this workflow generated from manifests already reviewed on master. Unknown report lock baselines remain blocked. This update changes no permissions or paid enable gates. ## Model Used None — generated by pnpm and the deterministic report-lock updater. ## Checklist - [x] Thinking path and problem description included - [x] Generated lock and matching report proof included - [x] Release-age gate decides between bot merge and maintainer review Co-authored-by: lockfile-bot <lockfile-bot@users.noreply.github.com> |
||
|
|
8c0c7c6abb |
perf(ui): load the @task mention pool after the task page settles; add a cold-load perf spec (#21557)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - People often open a task by its URL, for example from a link in a message or after a reload > - #20592 made a hard load of a task URL read the chat's first frame from one `chat-bootstrap` request > - The task page still sends a large read as it mounts: 100 full recently updated tasks for the `@task` mention picker. On staging this read returned 395 KB and took 3.9 s of server time > - The hosted database is CPU-bound, so reads that do not affect the first frame must not compete with the reads that do > - Also, no test measured a cold direct load under realistic latency or guarded the one-bootstrap request shape > - This pull request starts the mention pool after the task page settles. It also adds a cold-load perf spec with a delay on each API request and an always-on request-shape test > - The benefit is less concurrent server work during a cold task open, and a regression guard for the one-round-trip first frame ## Linked Issues or Issue Description Refs #20592 (hard-load chat bootstrap that this pull request measures and guards). Refs #20355 (the `chat-bootstrap` endpoints). **What existing behavior does this improve?** The task page loads the `@task` mention pool when it mounts. **Subsystem affected** Board UI task page (`ui/src/pages/IssueDetail.tsx`) and the issue-detail perf harness (`tests/perf/issue-detail`). **Current behavior** `GET /api/companies/:id/issues?limit=100&sortField=updated&sortDir=desc` starts with the chat's first-frame reads. It starts before the first chat frame in 14 of 14 local cold loads. The picker needs it only after the person types `@`. **Proposed behavior** The read starts after the page has painted and its own reads have finished. On a page that keeps reading, it starts after at most 5 s. **Reason and benefit** The read never affects the first frame. Deferring it removes a large concurrent read from the busiest part of a cold open. **Breaking changes** None. If the person types `@task` in the first moments after a cold load, the task suggestions can appear about half a second later. ## What Changed - `ui/src/hooks/usePageSettled.ts`: new hook. It returns `true` once `whenPageSettles` runs (the page has painted, the tab is visible, no query is fetching, and the browser is idle), or after `PAGE_SETTLED_MAX_WAIT_MS` (5 s). The cap keeps a long-running read from keeping the picker empty. - `ui/src/pages/IssueDetail.tsx`: the mention-pool query is enabled only when `usePageSettled()` is true. - `ui/src/hooks/usePageSettled.test.tsx`: tests that the hook waits for in-flight reads and that the cap applies. - `ui/src/pages/IssueDetail.test.tsx`: mocks the hook to `true`, so existing tests keep their behavior. - `tests/perf/issue-detail/cold-bootstrap.perf.spec.ts`: new spec. It delays every API response (300 ms by default) and opens the task URL in a new browser context, with the JavaScript in the HTTP cache and no app storage. The finish event is the first readable frame: the history is revealed, the latest saved message is visible and the composer is editable. - The request-shape test always runs. It requires one `chat-bootstrap` request and no per-endpoint first-frame read before the frame. Reads that start after the live socket is created are excluded, because they are its catch-up refresh. - The benchmark runs only when `PAPERCLIP_ISSUE_PERF_COLD_RUNS` is set. It writes the median, the spread, the request count before the frame, the last requests before the frame, the JavaScript bytes transferred and a request timeline. - The delay uses CDP `Fetch` interception instead of Playwright routing, because routing disables the HTTP cache. - `tests/perf/issue-detail/README.md`: documents the spec. - My earlier draft also added a deadline for a stalled bootstrap. Master already has one (`CHAT_BOOTSTRAP_SHARE_DEADLINE_MS`, 5 s), so this pull request does not include it. ## Verification Measurement setup: - Production UI builds of master `4a78124a3e` and of this branch, served by one local server (`PAPERCLIP_UI_DEV_MIDDLEWARE=false`, embedded PostgreSQL). - A delay of 300 ms on each API request. - Cold direct task URL with 24 saved comments and a plan. - Master and the branch alternated: 2 rounds of 7 runs each. - The host was shared with other agent sessions. | Build | Runs | First readable frame, median | p25–p75 | Min–max | API requests before frame (median) | Mention pool started before frame | |---|---:|---:|---:|---:|---:|---:| | master | 14 | 1,422 ms | 1,336–1,525 | 1,284–7,398 | 50 | 14 / 14 | | this branch | 14 | 1,390 ms | 1,338–1,457 | 1,272–2,207 | 49 | 0 / 14 | - The spec adds the API delay through CDP request interception, so the HTTP cache stays on. The measured loads downloaded 0 KiB of JavaScript in every run. The spec records this as `jsTransferBytes`. Each build has one outlier (7,398 ms on master, 2,207 ms on this branch), from host contention. - With this branch, the mention pool starts after the frame in every run (350–400 ms after it in an earlier set). - The first-frame timing is neutral locally. The local database answers the pool read in milliseconds, so it does not compete with other reads. The intended benefit is lower concurrent server load on a busy database. This pull request does not show it as a local latency gain. - On master and on this branch, the hard load adopts the bootstrap and the frame is no longer held by task reads. The frame now comes right after the shell reads return: `auth/get-session` and `instance/settings/experimental`, then `companies` and `workspace-preferences/me`. Profiling marks confirm this: the chat thread mounts after `workspace-preferences/me` returns, and all readiness gates are ready within 30 ms. - The request-shape test passed 5 of 5 on master and 5 of 5 on this branch. - `layout-stability.spec.ts`, `plan-startup.spec.ts` and `cold-bootstrap.perf.spec.ts` against this branch's build: 18 passed. - `ui` Vitest for `usePageSettled.test.tsx` and `IssueDetail.test.tsx`: 278 passed. The `ui` typecheck passed. `pnpm check:token-gates` passed. - `pnpm verify` exits 2 because `tests/perf/issue-detail/README.md` is a global CI input. Required CI supplies the broad coverage. To reproduce the measurement: ```sh PAPERCLIP_ISSUE_PERF_BASE_URL=http://127.0.0.1:3287 PAPERCLIP_ISSUE_PERF_COLD_RUNS=7 PAPERCLIP_ISSUE_PERF_LABEL=branch \ pnpm exec playwright test --config tests/perf/issue-detail/playwright.config.ts cold-bootstrap -g benchmark ``` ## Risks - Low risk. The only product change is when one read starts. - If the page keeps a query in flight, the pool waits up to 5 s. During that time, `@task` suggestions list only agents, projects and members. - The request-shape test treats reads that start after the live socket is created as catch-up refreshes. If the socket connects before the task page mounts, such a test would not see a first-frame read that started later. The test still requires exactly one bootstrap request before the frame. - The perf spec is not in the default CI browser suites. It runs when the issue-detail harness runs. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used in Claude Code with tool use (shell, file editing, Playwright browser runs). It wrote the change, the tests and the measurements. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
87d363a5c0 |
perf(ui): reveal a returning Mine visit from the saved cache (#20594)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The Mine inbox is the board's main list of work for one person; the
app saves its query cache in `localStorage` so a returning visit can
paint from it
> - On a hard load the saved cache gave no benefit: Mine restored its
rows but still held them until the first approvals, join-request and
dismissal reads answered, because those items sort in among task rows by
time and a late one would push rows down (#20407)
> - Those three reads were not saved, so every returning visit waited on
the network for items it showed a minute before
> - This pull request saves what Mine showed from those reads, as one
record per user and company, and lets it answer the wait on a returning
visit while every read refreshes in the background
> - The benefit is that a returning visit shows its rows with no API
response at all; the first visit keeps its bounded wait unchanged
## Linked Issues or Issue Description
**What happened?**
A returning hard load of `/:companyPrefix/issues?view=mine` with the
saved cache present (`paperclip:query-cache:v1:<user>:<company>`) showed
rows no sooner than with the cache cleared. On the hosted staging
instance, rows appeared about 40 ms after `GET
/companies/:id/join-requests?status=pending_approval` returned in 7 of
10 runs.
**Steps to reproduce**
1. Open Mine once so the app saves its query cache.
2. Delay API responses (for example 300 ms) and reload.
3. Rows appear only after the approvals and join-request reads answer.
**Expected behavior**
Rows show from the saved cache at once. The reads refresh in the
background.
Related: #20407 (kept approvals and join requests in the reveal gate),
#20408 and #20419 (Mine snapshot and its perf guard), #20591 (removes
the second wave of reads; merged), #20592 (task chat preload; merged).
## What Changed
- `ui/src/lib/inbox-mine-reveal-sources.ts` (new): the record `["inbox",
companyId, "mine-reveal-sources", userId]` holds the approvals visible
in Mine, the pending join requests, and the dismissals of those items
and of Mine's failed runs. A record over 256 KB is replaced by an empty
`incomplete` record that never answers the wait. `readMineRevealSources`
accepts a record only for the same user and company.
- `ui/src/lib/persisted-query-cache.ts`: the saved cache keeps that
record per user and company, only when its data names the same user and
company as its key.
- `ui/src/pages/Inbox.tsx`:
- Mine writes the record once approvals, join requests and dismissals
all have server answers.
- On a returning visit with a restored Mine snapshot, the saved
approvals, join requests and dismissals stand in while their reads are
pending, and after a transient failure. A refusal (401/403/404) or a
server answer replaces them. The first-attempt wait is skipped only when
saved values stand in; the first visit is unchanged.
- Selection fix: the selected row's key is recorded only when the
selection moves. Before, every list change recorded the row now at the
old index before the re-anchor ran, so a row inserted above the
selection (a late approval, or any live insert) moved the selection to
another row.
- `ui/src/hooks/useInboxBadge.ts`: `useInboxDismissals` takes an
optional saved `fallback` and returns `hasServerData`. Other callers are
unchanged.
- Tests:
- `Inbox.test.tsx`: a returning visit reveals rows and the saved
approval while the reads are pending, then keeps selection on the same
row when a newer approval arrives above it; a transient failure keeps
the saved approval and a 403 drops it; a record stamped for another user
is ignored and the page waits as a first visit does.
- `inbox-mine-reveal-sources.test.ts`: projection, size limit,
user/company checks, persistence scope.
- `persisted-query-cache.test.ts`: the record round-trips for the same
user, restores as invalidated, and another user's record is not saved.
- `useInboxDismissals.test.tsx`: fallback while pending, after a
transient failure, dropped after a refusal, and no fallback.
- `tests/perf/collections/mine-returning-visit.spec.ts` +
`returning-visit-fixture.ts`: production-build browser checks and the
timing case; README updated.
## Verification
- `cd ui && pnpm exec vitest run src/pages src/lib src/hooks
src/components/coordinator`: 453 files, 5574 tests pass.
- Browser checks on a disposable local instance (production UI build,
`local_trusted`, embedded PostgreSQL, Chromium headless, 1440×900):
`PAPERCLIP_COLLECTION_PERF_BASE_URL=http://127.0.0.1:<port> pnpm exec
playwright test --config tests/perf/collections/playwright.config.ts
mine-returning-visit.spec.ts`
- With every read of the company held, task rows and saved approvals
show and no company read has answered. Fails on `master`, passes here.
- An approval dismissed after the cache was saved stands in, then leaves
when the reads answer. Fails on `master` (no saved approvals), passes
here.
- Another company's Mine never shows the saved company's rows. Passes on
both.
- Timing (5 fresh contexts per arm, alternating saved/cleared, every API
response +300 ms, same instance and fixture, host load average about
20). "Responses before rows" counts API responses that finished before
the first row:
| Build | Saved cache: median (range) | Responses before rows | Cleared
cache: median (range) |
| --- | --- | --- | --- |
| `master` (`99edcd28a7`) | 1028 ms (906–3011) | 16–19 | 1502 ms
(1423–1592) |
| This PR | 625 ms (589–672) | 0 | 1632 ms (1359–1692) |
| This PR + #20591 | 625 ms (604–671) | 0 | 1495 ms (1398–1585) |
Saved cache: difference of medians 403 ms. The remaining 625 ms is
JavaScript download and route-chunk loading in a fresh browser context;
no API response precedes the rows. Cleared cache: no change is claimed;
the spread overlaps.
- Existing collection specs on this build: `parent-context.spec.ts`
passes; `startup-regressions.spec.ts` has 4 failures (partial
failure/context error and two 429 cases) that fail the same way on the
`master` build. `mine-workspaces.spec.ts` was not run (needs a database
URL for its fixture).
- Review follow-up (
|
||
|
|
f6c5986b61 |
feat(connections): machine sign-in (MCP OAuth client credentials) for organization connections (#21473)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents use apps through connections to remote MCP servers
> - Today an organization connection must sign in as a person (browser
sign-in) or use a static key
> - Service accounts and CI-style integrations have no person who can
sign in
> - The MCP OAuth Client Credentials extension (2026-07-28) defines
machine-to-machine sign-in for this case
> - This pull request adds "Machine sign-in" for organization
connections, with a Paperclip-held signing key (recommended) or a client
secret
> - The benefit is that organizations can connect automation-owned MCP
servers safely, with credentials only in the vault and bound to one
sign-in service
## Linked Issues or Issue Description
**Subsystem affected**
Connections: remote MCP OAuth (server `tool-access/oauth`, tool gateway
token refresh) and the custom MCP server setup flow in the UI.
**Problem or motivation**
Organization-owned automation (service accounts, CI integrations) cannot
connect an MCP server that requires OAuth, because no person can
complete a browser sign-in. The MCP OAuth Client Credentials extension
(`io.modelcontextprotocol/oauth-client-credentials`) solves this with
`grant_type=client_credentials` and RFC 7523 signed client assertions.
**Proposed solution**
Add machine sign-in for organization connections only. Paperclip creates
a keypair per connection, keeps the private key in the company vault,
and shows the public key (PEM and JWK) for the admin to register. A
client secret is also supported. The sign-in service (issuer and token
endpoint) is pinned at setup. Tokens use the existing organization grant
refresh path, so caching, concurrent dedupe and 401 re-request reuse the
existing lease and single flight.
**Alternatives considered**
Reuse the deployment-wide `PAPERCLIP_TOOL_OAUTH_*` client env path that
already sent `client_credentials`. Rejected: it is not per connection,
it keeps secrets in env, and it has no issuer pin. Route machine sign-in
through `connectGalleryApp`. Rejected: its rollback deletes the draft on
a failed test, which would lose a key the admin already registered.
**Roadmap alignment**
Phase 1 of the enterprise MCP plan (no enterprise single sign-on
needed). OIDC login and enterprise-managed authorization (ID-JAG) are
later phases.
## What Changed
- `oauth/client-credentials.ts`: keypair generation (RS256, or ES256
when the service requires it), RFC 7638 key IDs, and RFC 7523 assertions
(`iss` = `sub` = client ID, `aud` = token endpoint, 60 s `exp`, unique
`jti`) with Node `crypto` only.
- `oauth/client.ts`: the token exchange accepts a signed assertion and
then sends only `client_assertion_type` and `client_assertion` (no
`client_id` or secret), as the extension specifies.
- `oauth/client-credentials-service.ts`: discovery of the sign-in
service (protected-resource metadata, then RFC 8414/OIDC metadata with
issuer check, or a token URL entered directly), issuer pinning,
vault-only key and secret handling, draft creation, test-and-activate,
and no-downtime rotation (test the new secret first; stage a new key and
switch only after the provider accepts it).
- `oauth/grant-refresh.ts` and `oauth/connection-refresh.ts`: machine
sign-in requests a new access token with the connection's own
credentials instead of needing a refresh token. It never falls back to
deployment-wide OAuth clients.
- `catalog/discovery.ts`: a rejected token on a machine sign-in
connection never re-runs discovery or rewrites the pinned endpoints.
- `mcp-http.ts` and the tool gateway: declare
`io.modelcontextprotocol/oauth-client-credentials` in client
capabilities only for connections that use it.
- Routes (with OpenAPI and activity logging): `POST
/companies/{companyId}/tools/machine-sign-in`, `GET|POST
/tool-connections/{id}/machine-sign-in[/test|/rotate|/rotate/complete]`.
Removal activity now records `machineSignIn: true`. The runner API
treats these routes as sensitive.
- Shared validators and types for machine sign-in.
- UI: "Machine sign-in" under Advanced authentication in the custom MCP
server flow (organization connections only), `MachineSignInSetup`
(choose Paperclip key or client secret, register the public key, test),
and Storybook stories for each setup state.
- Docs: `doc/connections/MCP-CLIENT-CREDENTIALS.md`, linked from
`doc/connections/README.md` and `GENERIC-REMOTE-MCP.md`.
## Verification
- `server/src/__tests__/tool-access-client-credentials.test.ts`: 16/16
pass. It uses a fake sign-in service that verifies real signatures
against registered keys, and a fake MCP server. It covers RS256 and
ES256 assertion claims and signatures, assertion-only client
authentication, the secret flow (`client_secret_basic`), discovery and
direct token URL, issuer pinning (refuses a different sign-in service
and sends it nothing), the pinned endpoint surviving a hostile
challenge, the token cache (0 token requests while live), concurrent
dedupe (5 callers on 2 service instances produce 1 token request), 401
re-request, key and secret rotation, a refused personal connection,
secret redaction in responses, rows and activity, and vault cleanup on
removal.
- Red/green: with the grant-refresh machine sign-in branch and the
discovery pin guard disabled, 5 runtime tests fail (pin, cache/dedupe,
401 re-request, both rotations). Restored: 16/16 pass. The other tests
cover new routes and services that do not exist on `master`.
- `ui/src/features/connections/MachineSignInSetup.test.tsx`: 3/3.
`ConnectionSetupFlow.architecture` and `.oauth` tests: 19/19.
- `openapi-routes.test.ts`: 51/51. `runner-api.test.ts`: 2153/2153.
`mcp-http.test.ts`: 9/9. Existing `tool-access-service.test.ts` refresh
and legacy client-credentials tests: 54/54 (filtered).
- Server package `tsc --noEmit`: clean. UI `tsc`: clean except the
pre-existing generated Storybook catalog imports.
- Not run locally (machine under heavy load): full suites, repo-wide
typecheck, browser check of the live app. CI covers breadth.
## Risks
- No migration. New config lives in `config.oauth` (`grantType:
"client_credentials"` plus `clientAuth`). Older env-configured
client-credentials connections have no `clientAuth` and keep their
original path.
- The capability declaration adds one extension key only for machine
sign-in connections. Other requests are unchanged.
- Rotation has no settings-page control yet; it is available through the
API. This is a follow-up.
- Low risk to agent boot and time to first token: a live token is read
exactly like a browser sign-in token.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edits, test runs).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
a726dce4f0 |
feat(mcp): show live progress for long connected-app tool calls in the task chat (#21449)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents call tools from connected apps through the Paperclip MCP gateway > - Some tool calls take a long time: exports, builds, large searches > - The MCP progress utility lets an app report how far a call is, but Paperclip ignored it > - So the task chat showed only a spinner until the call finished, with no sign of life > - This pull request reads the app's progress and shows it on the tool's row while the call runs > - The benefit is that people can see that a long call is working, how far it is, and how long it has run ## Linked Issues or Issue Description **Subsystem affected** Connected apps (MCP gateway) and the task chat. **Problem or motivation** A long tool call from a connected MCP app shows only a spinner in the task chat. Paperclip does not send a `progressToken`, and it drops `notifications/progress` from the app. **Proposed solution** Send a `progressToken` on each `tools/call` that an agent run makes. Read the app's progress notifications from the call's response stream. Show a progress bar (or an indeterminate sweep), the latest message, and elapsed time on the running tool row. Keep the latest state so that a reload mid-call shows it. **Alternatives considered** A new transport for progress. The existing company live-events socket already carries per-run events with run read checks, so this change uses it. **Additional context** Spec: https://modelcontextprotocol.io/specification/2026-07-28/basic/utilities/progress ## What Changed - `mcp-http.ts`: `mcpHttpRequestInit` can add `_meta.progressToken`. `readMcpHttpResponse` gives each notification on the response stream to an observer before the result. The observer cannot fail or delay the call. New `mcpProgressNotificationOf` matches a notification to its token. - `tool-gateway.ts`: remote HTTP and local stdio `tools/call` from an agent run send a per-call token and forward progress to a reporter. Calls without a run send no token. - New `server/src/services/tool-call-progress.ts`: checks each update (finite, increasing, plain single-line text of at most 200 characters). It publishes a `heartbeat.run.tool_progress` live event at most every 250 ms (latest wins) and saves the latest update on the invocation at most every 1 s. Nothing is awaited by the call. A final `done` event goes out when the call ends. - DB: nullable `tool_invocations.progress` jsonb column (migration `0479`, `ADD COLUMN` only). A save never writes onto a finished call. - Live delivery: the new event follows its run like transcript output. A client gets it only for runs it follows, after the run read check, in the run's company only. - New route `GET /api/issues/:issueId/tool-call-progress` (task read check) returns the latest progress of the task's running calls, for a reload mid-call. Documented in `docs/api/issues.md`. - UI: `ToolCallProgressProvider` (one per chat) keeps the latest update per call. A running tool row subscribes and shows a bar or sweep, the message, the percent and elapsed time (design tokens, reduced motion honored). Finished rows drop it and never fetch or re-render for progress. `LiveUpdatesProvider` ignores the event for query invalidation. - `packages/kv-demo-mcp-server`: new `slow_with_progress` demo tool (`steps`, `stepMs`, `withTotal`) in its own file. - Storybook: `Tasks/Tool progress` stories (running with total, running without total, done). - Docs: new `doc/connections/MCP-PROGRESS.md`; `MCP-PROTOCOL-VERSIONS.md` no longer lists progress as unsupported. ## Verification - Red/green: on the original source, 5 of the new server tests fail (token in `_meta`, SSE with interleaved progress and result, notification matching, run-interest delivery, gateway end-to-end). With this change they pass. - `server/src/__tests__/mcp-http.test.ts` (13), `tool-call-progress.test.ts` (9: throttling, latest wins, bounded text, out-of-order drop, finish, failures never throw, company/run/tool scoping), `tool-call-progress-store.test.ts` (2, embedded Postgres: company and task scoping, no write onto a finished call), `realtime/live-event-delivery.test.ts` (run interest), `tool-gateway.test.ts` end-to-end against the real SDK kv demo server over Streamable HTTP: 6 provider updates over about 1.2 s give 3 to 7 throttled events, all for the right company, run, task and tool, the last one `done`; no event reaches another company; the invocation keeps the progress; the result is unchanged. - UI: `tool-call-progress.test.tsx` (10: name matching, row states, plain-text rendering, drop on complete, restore after reload, no fetch for finished rows). All 67 task-chat test files pass. - `packages/kv-demo-mcp-server`: 13 tests, including progress over HTTP with and without a total. - Storybook: checked the three stories in a browser. The bar fills to 40%, the sweep animates, the elapsed time ticks, and the done row shows no progress. - `pnpm verify`: see the CI checks for the final revision. ## Risks - Low. The migration adds one nullable column. - An SSE response is now bounded per message, not per stream, so handled notifications (such as progress) do not use up the result's size limit. Each message is still bounded. - Progress code runs only while a run's tool call is in flight. It adds a few bytes to the request and throttled, fire-and-forget work. Agent boot and time to first token do not change. - A row finds its call by run and tool name. If one run calls the same tool twice at the same time, both rows show the newer call's progress. - Progress from a call that became an MCP Task is not shown. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Claude Code agent with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
01f764fe61 |
test(evals): pin the case clock in the original-deadline rejection tests (#21501)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The Runner support suite tests the chat catalog observers that grade
agent work
> - One observer test fails at random on unrelated pull requests
> - The test simulates a late poll result with a fixed clock jump from
the fixture time. The observer computes its deadline from a later clock
read. If 2 ms or more pass between the two reads, the jump does not pass
the deadline.
> - This pull request pins the clock to the fixture time for the
deadline rejection cases, as the sibling deadline tests do
> - The benefit is a deterministic test that still fails when the
late-deadline guard is removed
## Linked Issues or Issue Description
**What happened?**
`tests/runner-e2e/chat-catalog-domains/identity-plan-decision-observer.test.ts`
> "rejects late await original deadline without resetting it" failed in
the "ci / Runner support suite" job of PR #21418 (run 43503383519,
2026-11-13 20:38Z). The error was: `expected [Function] to throw error
including 'CONTROL_SOURCE_NOT_SETTLED' but got
'CONTROL_AFTER_SOURCE_SETTLED'`. PR #21418 changes only a plan document
and a benchmark script.
**Expected behavior**
The test passes every time. The result must not depend on how fast the
test worker runs.
**Steps to reproduce**
1. Add `await new Promise(r => setTimeout(r, 2));` before the `act` call
in the "late await" case.
2. Run `node node_modules/vitest/vitest.mjs run --config
tests/runner-e2e/vitest.config.ts
chat-catalog-domains/identity-plan-decision-observer.test.ts`.
3. The test fails with the CI error. With a gap of 0 or 1 ms it passes.
**Paperclip version or commit**
`master` at
|
||
|
|
0a96f972e1 |
Connect Tailscale networks through device sign-in (#21526)
## Thinking Path > - Paperclip manages AI agents and the environments where they run. > - SSH environments can reach machines through a company Tailscale connection. > - The initial connector requires a managed OAuth app that Tailscale limits to its own tailnet. > - Customers need to connect their own tailnets without registering an app or supplying a key. > - This pull request uses Tailscale's normal device sign-in for new connections. > - Customers approve the Paperclip device and then select their machines in SSH environments. ## Linked Issues or Issue Description **Problem or motivation** The current Tailscale enrollment path cannot serve independent customer tailnets through the documented self-service OAuth app model. It also requires Cloud broker configuration and pilot admission before a customer can start. **Proposed solution** Start a private Tailscale device in the tenant and open its normal sign-in link. Save its encrypted device state before reporting a successful connection. Retain the existing SSH machine selection and agent account behavior. **Alternatives considered** Auth-key entry would require customers to copy a credential. A public OAuth integration still needs a confirmed registration path. Device sign-in uses a documented Tailscale flow without either requirement. See [device authentication](https://tailscale.com/docs/features/tsnet#device-creation-and-authentication) and [OAuth app limitations](https://tailscale.com/docs/features/oauth-apps). **Related PR search** Searched GitHub PRs for `tailscale` and `device sign-in`, and issues for `tailscale`. This follows the merged network connector in [#20617](https://github.com/paperclipai/paperclip/pull/20617) and preserves the SSH agent-account support in [#21248](https://github.com/paperclipai/paperclip/pull/21248). The open [#15648](https://github.com/paperclipai/paperclip/pull/15648) proposes storing an OAuth client and checking its health; this change instead enrolls a device through interactive sign-in so customers do not need an OAuth app. ## What Changed - New connections use device sign-in. Existing connected devices and pending legacy OAuth callbacks remain supported. - Pending sign-ins are bound to their company and initiating user. They have a fixed ten-minute deadline. Device state uses server-owned vault secrets with trusted version and digest checks. - The active Tailscale CLI waits for consent and device approval. Cancellation, expiry, secret deletion, and stale completion cannot restore a cancelled connection. - The connector stays open during sign-in. It provides a link when popups are blocked and shows device approval separately. Progress checks stop on completion, failure, expiry, or unmount. - Shared response types, OpenAPI contracts, SDK source pins, and the API/setup documentation describe the new flow. No database migration or Cloud provider configuration is required. ## Verification - 109 focused tests cover network processes, the real database and encrypted vault, routes, UI, and legacy connection behavior. - Server and UI typechecks pass. - Follow-up `29cf296a597`: reproduced the missing OpenAPI route failure locally. All 70 OpenAPI and Tailscale route tests and server typecheck pass after synchronizing the API contract. - Follow-up `79de05f38b9`: the transient-recovery regression failed before the fix; all 20 connector UI tests and UI typecheck pass after it. The query-error policy check reports no findings. - The public SDK input registry check passes. - [CI attempt 2](https://github.com/paperclipai/paperclip/actions/runs/43535594257/attempts/2) passed on the final head, `29cf296a597e470d432b84767050bc539caa280e`: 57 successful checks, no failures or pending checks. The interrupted canary, E2E shard, and serialized-suite jobs passed on rerun. Greptile is 5/5 on this head with no unresolved review threads. - `pnpm verify --explain` exits 2 because the documentation change selects broad CI coverage. Focused local checks passed; broad CI passed on `29cf296a597e470d432b84767050bc539caa280e`. - Live local qualification on `29cf296a597e470d432b84767050bc539caa280e`: native device sign-in completed with interactive user approval. An isolated Ubuntu 24.04 ARM64 EC2 machine appeared in discovery and passed SSH pairing and workspace probes. The assigned Claude Code agent used a Paperclip-managed subscription, created `hello_world.txt` containing exactly `Hello, world!` plus a newline, posted a completion comment, and marked the task done. An independent read through the private connector verified the bytes, remote user, and hostname. The remote machine had no stored Claude login and no leftover temporary credential directories. - Restarted the local Paperclip server and verified the same Tailscale device identity, online peer discovery, successful SSH probe, and unchanged file without new consent. Temporary public SSH ingress was removed. - Local qualification covers one user tailnet and the final Core revision. Browser UI acceptance, two unrelated tailnets, a hosted restart, and an agent task on the final staging revision are not yet verified. ## Risks - The enrollment method changes. It must pass the live qualification above before broad rollout claims. - Existing encrypted device state and the vault master key must survive a restart or redeploy. Device expiry or removal still requires reconnecting. - The existing one-serving-process-per-instance constraint remains. Concurrent replicas need durable network ownership. - Pending setup uses bounded progress requests and an event-driven CLI with a deadline. It adds no idle database scan. ## Model Used OpenAI GPT-6 Astra (`gpt-6-astra`) through Codex in T3 Code, with xhigh reasoning. Context window size was not exposed by the runtime. No subagents were used. ## Checklist - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] My branch name describes the change and contains no internal ticket or instance details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented the risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
47a2c82d35 |
fix(heartbeat): reach the previous server's live runs first after a release swap (#21450)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - During a release swap, the outgoing server stops and the incoming server takes over the live runs on remote Runners > - The incoming server reached those runs late: it waited up to 60 s for the outgoing server's controller lease to lapse, re-verified each run's whole log from object storage, and deferred a reattach when the agent directory was busy. Queued starts competed with the reattaches for the same resources. > - On hosted staging, the later a live run was reached, the more likely it was to fail. Runs reached in under 150 s never failed; runs reached after 420 s almost always failed. > - This pull request hands live runs over at the start of shutdown, makes a reattach skip work that the same run already did, and holds queued starts while handed-over runs are being reached, with a 180 s cap. > - The benefit is that live runs are reached within seconds of a swap instead of minutes, so they continue instead of failing. ## Linked Issues or Issue Description **What happened?** Hosted staging, release swaps on 2026-11-13 (06:05Z to 09:45Z). For each run that crossed a swap we measured the gap from the last Runner output to the moment the new server recorded the Runner's process identity: - gap under 150 s: 0 of 9 runs failed - gap over 420 s: 14 of 15 runs failed - median gap: 180 s for runs that resumed, 470 s for runs that failed In one traced run the incoming server waited on `remote_controller_lease_live` three times (09:14:21, 09:15:18, 09:16:17), claimed at 09:16:59, and spent 85 s in `task.prepare`. Of that, `heartbeat.prepare.run_output` took 58.7 s, because it re-read the run's whole log from object storage. A control run spent 23 s in `task.prepare`. Queued starts used the same preparation path, the same 8 object-I/O slots and the same database pool, and nothing gave a reattach priority. The first-attempt failures were `PRP command session.snapshot timed out` and `codex_history_read_failed`. The retry then failed with `runner_harness_state_mismatch: resumed_not_suspended`, `runner_remote_process_identity_unavailable` or `provider_turn_lost_on_restore`. **Expected behavior** Every live run is reached within about 30 s of the swap. Queued work starts after that, or after a bounded wait. **Steps to reproduce** 1. Run a two-server release swap with live remote runs and queued runs. 2. Observe `native.recovery.transition` "waiting" events with `remote_controller_lease_live`, then the claim, then a long `heartbeat.prepare.run_output` span. 3. The new tests in `reattach-phase.test.ts` and `heartbeat-native-early-hand-off.test.ts` model this. **Paperclip version or commit** `master` at |
||
|
|
1140984b82 |
feat(plugins): tell an opted-in workspace provider the run's environment (#21512)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - A plugin can be a project workspace provider and also an environment
driver for the agent runs that use those workspaces
> - Core asks the workspace provider to prepare the project workspace
before the run acquires its environment lease
> - The provider knows the run, agent, project, and execution workspace,
but not the environment that the run selected
> - A provider whose compute exists per run (for example, a sandbox that
it creates from the environment's driver config) cannot start or find
the run's compute for the workspace work
> - This pull request lets a provider opt in to receive the run's
selected environment, when that environment is one of the provider's own
drivers
> - The benefit is that such a provider can prepare the workspace on the
run's own compute, with no change to the order of operations in Core
## Linked Issues or Issue Description
**What existing behavior does this improve?**
Plugin workspace preparation for runs whose environment is a driver of
the same plugin.
**Subsystem affected**
Plugin SDK workspace provider params, the plugin manifest workspace
provider declaration, and `callWorkspaceProvider` /
`preparePluginWorkspace` in the server.
**Current behavior**
`prepare` params carry `runId`, `agentId`, `projectId`, and
`executionWorkspaceId`. They carry `taskLease` only when Core acquired
the lease first. They never name the run's environment.
**Proposed behavior**
A provider that declares `workspaceProvider.workspaceEnvironmentVersion:
1` receives `environment: { id, driverKey, config }` when the run's
selected environment is a plugin environment of that same provider.
`config` is the environment's `driverConfig`, the value that the
driver's acquire receives.
## What Changed
- SDK: optional strict `environment` field on
`pluginWorkspaceParamsSchema`.
- Manifest: optional `workspaceProvider.workspaceEnvironmentVersion: 1`,
next to the sibling version fields. Core sends the new field only to
providers that declare it, because providers parse the params with the
strict schema.
- Server: `callWorkspaceProvider` adds `environment` only when the
provider opted in, the environment driver is `plugin`, its `pluginKey`
is this provider, and it names a `driverKey`. The field is not part of
the operation ID.
- Server: `preparePluginWorkspace` passes the run's environment to its
prepare call. This covers normal, verify-existing, fresh, and
replacement prepares.
- Calls that do not have the run's environment do not send it: finalize,
inspect, remove, reservation reconciliation, and the retry-candidate
verification in `fresh-workspace-continuation.ts`.
- Lease acquisition, the `sessionHandoffVersion` gate, and the order of
operations do not change.
- Docs: `doc/plugins/PLUGIN_RUNNER_EXECUTION.md` describes the
capability and its exceptions.
## Verification
- `packages/plugins/sdk/tests/workspace-provider.test.ts`: 11 passed.
The schema accepts the field and rejects malformed values.
- `packages/shared/src/validators/plugin.test.ts`: 65 passed. The
manifest accepts only version 1.
- `server`: `plugin-workspaces.integration`,
`plugin-fresh-workspaces.integration`, `project-environments`, and
`heartbeat/workspace-preparation`: 226 passed, against embedded
Postgres. The new integration test checks the exact shape, an unchanged
operation ID, verification prepares, and omission for an undeclared
provider, another plugin's environment, non-plugin drivers, a missing
`driverKey`, and no environment. It fails when the server stops passing
the environment.
- `pnpm typecheck` passes in `packages/shared`, `packages/plugins/sdk`,
and `server`.
- The full suite was not run locally. CI runs it.
## Risks
- Low risk. Providers that do not declare the capability get the same
params as before.
- `config` is the stored `driverConfig`, in the same form that the
driver's acquire gets. Secret references in it reach the provider in
that same form.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, code search, test execution).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [ ] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
0749d74f8b |
chore(lockfile): refresh pnpm lockfiles (#21516)
## Thinking Path > - Paperclip uses a dependency lock for reproducible builds. > - Dependency changes on master need matching lock entries. > - Paid eval reports also pin a reviewed offline correction. > - Reports must install before paid model work starts. > - This automated PR keeps the lock and report proof together. ## Linked Issues or Issue Description **What happened?** Dependency manifests on master changed after the last committed lock refresh. **Expected behavior** Frozen installs must match those manifests. Eval reports must retain matching offline lock proof. **Steps to reproduce** Run a frozen install before and after this lock refresh. ## What Changed - Refresh the dependency lock from the manifests on master. - Refresh the sandbox provider lockfiles from their manifests on master. - Include the matching offline report correction and hashes. - Resync the SDK public source proof's lock pins from the refreshed lock. ## Verification - Pinned pnpm resolution completed with lifecycle scripts disabled. - The report proof refresh accepted the reviewed committed baseline. - The workflow rejected changes outside the lock and report proof files. ## Risks The lockfile bot merges this PR without CI or review when every added package version is at least 3 days old on npm. Otherwise the PR waits for a maintainer, and the daily refresh tries again. The bot merges only the exact commit this workflow generated from manifests already reviewed on master. Unknown report lock baselines remain blocked. This update changes no permissions or paid enable gates. ## Model Used None — generated by pnpm and the deterministic report-lock updater. ## Checklist - [x] Thinking path and problem description included - [x] Generated lock and matching report proof included - [x] Release-age gate decides between bot merge and maintainer review Co-authored-by: lockfile-bot <lockfile-bot@users.noreply.github.com> |
||
|
|
21bf7ebcee |
feat(connectors): connect Tailscale networks to SSH environments (#20617)
## Thinking Path > - Paperclip manages work for AI agents. > - SSH environments run tasks on computers that users own. > - A Paperclip server needs network access before it can reach a private computer. > - A reusable Tailscale connection belongs in Connectors. > - Connected machines should become choices in the existing SSH environment form. > - This change connects a tailnet and resolves selected machines through a private network client. > - Agents retain their choice of harness and AI account. ## Linked Issues or Issue Description **Subsystem affected** Connectors, environment settings, SSH execution, company secret storage, and runtime packaging. **Problem or motivation** Entering a Tailscale address in the SSH form does not connect the Paperclip server to that tailnet. Requiring a separate device key for every environment also duplicates network setup. **Proposed solution** Add a Tailscale Connector with **Connect Tailscale Network**. Use brokered OAuth consent to enroll one private network client per connection. Let SSH environments select a connected network and stable machine identity. Preserve the enrolled node state through the company vault so a restart does not reuse the one-time token. **Alternatives considered** A separate Tailscale environment type would duplicate SSH behavior and mix network access with harness selection. This change uses the existing `ssh` driver. Related #15648 adds OAuth client credentials and a health check; this change uses user consent and supplies outbound task execution. Related #11743 concerns HTTPS brokering. **Roadmap alignment** This extends the existing Connected Apps, environment, and Personal & Shared AI Accounts capabilities. This PR is stacked on #21248, which provides SSH account selection independently of Tailscale. The hosted OAuth broker is a companion control-plane change. ## What Changed - Preserve Cloud reauthentication in the current browser tab. Explain canceled or failed consent in the connector and offer a retry of the same authorized saved connection. - Record Cloud enrollment starts initiated through the Tailscale connector in the company activity log. - Check install-preparation descendant liveness through the kernel process table in the Runner support test. Keep the direct child’s strict reaping assertion and reject any non-zombie descendant. - Align the application image stage contract with the Tailscale client stage after the upstream Runner-image refactor. Keep the exact stage list and check every application stage, including `tailscale-client`, for prohibited Runner compiler commands. - Group Tailscale consent, discovery, daemon lifecycle, pairing, and deletion under `server/src/services/connectors/tailscale/`, with their service tests. Update existing route, runtime, environment, vault, and company-deletion imports. Keep the existing contracts, authorization, audit, and transaction owners. - Handle early stdin closure in the Kubernetes real-shell test helper. Stop the input stream on `EPIPE`, preserve nonzero child exits, and reject a zero exit when input was lost. Add two real-process regression cases. Production transfer code and confinement checks are unchanged. - Read configured Tailscale executable paths when a client starts, after supported `.env` loading. Keep explicit command overrides available. - Discard pending pairing results after switching to Direct SSH or closing the form. Retain the existing company, connection, machine, and port checks. - Store network credentials with an immutable server-owned purpose from their first vault row. Exclude them from generic catalogs, bindings, environment projection, and value reads. Use a dedicated audited server consumer for pinned connector reads. Block unclassified legacy serialized node state at the same value boundary. Keep SSH private-key binding separate from connector credentials. - Gate network queries and setup on verified local-implicit or instance-admin access. Hide Tailscale catalog rows for other members, show an admin requirement on direct setup navigation, and avoid device discovery before management access is known. - Keep failed-client cleanup in the network manager. A failed request cannot disconnect a newer retry for the same identity; explicit connector disconnect and enrollment retirement still stop their networks. - Check for readable Ed25519, ECDSA, or RSA host public keys before pairing edits `authorized_keys`. Print a known-hosts entry for every available key, or fail before changing authorized keys if none are available. - Return self-hosted Cloud enrollment to the Tailscale connector. Save the unprefixed setup path and let the shared callback add the verified company prefix. Keep external and unrelated destinations rejected. - Hide pairing instructions after an SSH port change. Ignore late key-selection results for the old port. Operators prepare again for matching known-hosts output. - Retire connector references before deleting a company or its vault credential. Stop company daemons after the database commit. Keep failed secret retirement retryable and prevent late node-state writes from restoring deleted connections. Discard retired state without blocking behind the vault deletion lock. - Reload a mismatched credential before replacing a ready daemon. Recheck the active entry and manager generation after the read. An older request cannot stop a newer healthy socket or cross shutdown. - Pin connector credentials to the exact vault version and SHA-256 digest recorded by trusted enrollment and node-state writes. Ignore generic secret rotations. Advance both fields with a compare-and-set guard. Reject unpinned or tampered credentials before network startup. - Save an explicit Direct SSH selection and clear old Tailscale fields after config merging. Preserve the operator's entered direct host. - Route all Tailscale SSH runtime access through the connection service after SSH-key loading. Pin access to the environment's saved connector identity and recheck after device discovery. Remove standalone key-based daemon startup and reject raw enrollment keys or credential-only network configurations. Align the shared config type and SDK pins. - Correct two E2E assumptions exposed during connector qualification. Keep the two-window chat fixture stale until its own mutation. Validate the original failed run and its permitted write-revocation timestamp in the legacy recovery browser test. - Add the Tailscale Connector page, catalog entry, OAuth return flow, connected-network management, device discovery, and disconnect. Document all six routes in OpenAPI and the API reference, including their instance-admin requirement. - Add company-scoped connection storage in `0480_lazy_lethal_legion`, trusted credential version/digest fields in `0481_round_pet_avengers`, and server-owned secret purpose/backfill in `0482_aberrant_rumiko_fujikawa`. Generate all three migrations through the maintained command and retain the five newest snapshots. - Add machine pairing that generates a fresh SSH key per request, stores its private half in the company vault, and uses the generated public half in the install command. Do not trust editable saved metadata. Preserve existing authorized keys, including a file without a final newline. Keep SSH username, workspace, and trusted host keys explicit. - Use the existing SSH driver. Resolve the machine's current address from its stable device ID. Reject disconnected, removed, offline, and wrong-company selections. - Hide Network when Direct is the only option. Select a sole connected tailnet automatically and hide its selector. Remove the requested setup helper copy. Do not restrict the harness to Claude. - Run a private userspace Tailscale daemon with a separate socket. Persist its node state before execution. Restore it after restart, bound startup, serialize reconnect cleanup, and close it during server shutdown. - Guard OAuth completion with a durable attempt marker. Disconnect and newer authorization invalidate older callbacks. Close replaced daemons and remove cancelled enrollment secrets without changing newer connection state. - Preserve SSH host verification and tailnet-specific session identity. Honor the configured Tailscale executable path, including spaces and OpenSSH percent escaping. - Package checksum-pinned Tailscale clients and their license in the standard runtime image. Exercise the client build stage in PR CI and include it in image smoke checks. ## Verification - Merge status: all CI and image checks pass, and current-head Apex is 5/5 with no unresolved comments. GitHub rejected a direct merge because the active master ruleset requires code-owner approval for `.github/workflows/pr-trusted.yml`. The [CODEOWNERS workflow pattern](https://github.com/paperclipai/paperclip/blob/master/.github/CODEOWNERS#L3) applies; eligible maintainers already have review requests. Auto-merge is enabled with the squash method for the verified head. The PR remains open pending that required approval; no branch policy was bypassed. - Final revision: `fdb29b2cc366f4945b7f4c3e2b2d25070329a21b`, including master `999709b0a6dd416af335b5debded13e1de7fcb7a`. [Final CI verification passed](https://github.com/paperclipai/paperclip/actions/runs/43518584814/job/132213508320). All required checks pass, all applicable test/build gates are green, and the branch is mergeable against master. SSH prerequisite #21248 and Cloud broker paperclipai/paperclip-cloud#1314 are already merged. - [Apex reviewed this exact revision at 5/5](https://github.com/paperclipai/paperclip/pull/20617#issuecomment-6776629437). There are no unresolved review threads. The PR is classified as Critical impact. - Final image qualification passes for the [application Runner path](https://github.com/paperclipai/paperclip/actions/runs/43518584522/job/132206784267), [amd64 Runner image](https://github.com/paperclipai/paperclip/actions/runs/43518584544/job/132207296198), and [arm64 Runner image](https://github.com/paperclipai/paperclip/actions/runs/43518584544/job/132207296156). Typechecks, Runner support, native/Rust tests, provider tests, serialized-server suites, browser shards, and the remaining applicable CI checks pass. This is source/image CI qualification, not a production rollout. - The last master conflict affected only Docker stage declarations and their boundary test. Both the checksum-pinned Tailscale stage and upstream `runner-lockfile` guard are retained, including the upstream `deps` inheritance. The Tailscale client stage is byte-identical. All 180 cases across five Docker/image contract suites pass locally, along with server and UI typechecks. The maintained 543-input SDK check has no drift. Runtime and lock pins are unchanged. Secret/PII and whitespace scans pass. - Seven OAuth regression cases failed before the fixes. The current-tab test uses the real handoff helper, checks that the session is stored before reauthentication navigation, mounts the shared return page, resumes consent, and verifies handoff cleanup. Callback tests check trusted saved-connection context, fixed safe outcomes, failure audit metadata, denied membership and expired-state rejection, JSON API errors, and no provider text in browser returns. UI retries require a connection in the current company's catalog. All 80 OAuth route/service/UI/shared-handoff cases pass; server and UI source/test typechecks pass. - [Server shard 4/12 on the prior refresh](https://github.com/paperclipai/paperclip/actions/runs/43510727031/job/132180301436) passed 2,784 cases and skipped 21, then failed a reassignment fixture's `TRUNCATE companies CASCADE` cleanup with PostgreSQL `40P01`. Its automatic retry found the surviving `REA` prefix. The unchanged test passes all three cases locally. A controlled reader/cleanup overlap reproduces the same deadlock, but the exact competing CI query was not logged. Each case now has a distinct company prefix and identifier; the owned database is removed at suite end. The three test bodies and assertions are byte-identical. Eight local reassignment/lock-release cases pass, and the [final server shard 4/12 passes](https://github.com/paperclipai/paperclip/actions/runs/43518584814/job/132207159392). Production reassignment logic, deadlines, and retries are unchanged. - The initial master integration passed 330 cases across 17 suites. Coverage includes connector state, device discovery, network lifecycle, pairing, routes, SSH binding and exports, environment persistence, and connector UI. All three unpublished migrations were regenerated after master's `0479_short_darkstar`: `0480_lazy_lethal_legion`, `0481_round_pet_avengers`, and `0482_aberrant_rumiko_fujikawa`. Each SQL body is byte-identical to its earlier unpublished migration. Generated snapshots preserve existing master tables and add only the intended Tailscale table and secret-purpose column. Exactly five snapshots remain, 0478–0482. Numbering, safety, compatibility, and snapshot-drift checks pass. The later Docker refresh did not alter this migration sequence. - Earlier attempts remain part of the verification record. On `1c020a0`, Rust cache restoration and server shard 5/12 were interrupted by shutdown signals, Runner shard 2/3 lost communication with no retrievable log, and an amd64 provider download failed with npm `ECONNRESET`. Same-head retries recovered, without establishing the cause of the communication loss. Later superseded runs also had cancellations with incomplete logs. These interrupted attempts are not counted as passes. The final revision has fresh successful CI. - Local platform limits remain recorded. The Kubernetes suite on macOS passed 191 cases, failed 12 Linux `/proc/self/fd` transfer cases, and skipped four Linux-only cases. Unchanged source reproduced the same 12 failures. An initial run also lacked provider dependencies until the maintained preparation command installed them. Those local attempts are not green qualification. Final Linux provider CI passes. Local Docker qualification was unavailable; final image checks passed in CI. - Before PR separation, a live tailnet/SSH Claude Code task completed on Ubuntu ARM64 without provider login for the remote user. The selected Paperclip subscription ran it, wrote the expected file, posted completion, and marked the task done. File contents and private runtime cleanup were checked. This is prior live evidence; the final revision was not live-retested. Other providers have automated coverage only. Hosted registration for unrelated tailnets and production concurrency remain qualification limits below. - The Superagent security and supply-chain reports remain neutral and inconclusive. Earlier attempts to restart the failed security scan returned HTTP 404. Supply-chain intelligence lacked coverage for extracted pinned dependencies and the shared workflow reference. These reports do not establish successful security qualification. The required CI and review gates pass; this external coverage gap remains disclosed. ## Risks - The Superagent security and supply-chain reports are inconclusive. The security scan re-run request returned HTTP 404, and the supply-chain scanner lacks coverage for the extracted dependencies and shared workflow reference. These neutral results are not successful security qualification. - The server-owned secret purpose is internal database metadata, not an HTTP-writable field. The generic secret service owns enforcement so direct API reads and runtime projection share the same restriction. Previously copied node state remains valid outside Paperclip until the enrolled device is revoked in Tailscale; a local disconnect cannot stop an external copy. - Tailscale network configurations now require a connected Connector and selected machine. The unreleased key-only path is removed; any saved configuration from that path needs Connector setup. - The connector remains a pilot. The local test used an OAuth app registered in the consenting user's tailnet. Qualify hosted app registration for unrelated customer tailnets before general availability. - This implementation supports one serving process per instance. Concurrent replicas need exclusive ownership of the persisted network identity before production rollout. Preserve the database and company vault master key across restarts. - Each machine-pairing preparation creates a new vault secret. Unused generated keys follow the existing secret deletion lifecycle. A saved SSH key remains available as an explicit choice. - Apply additive migrations `0480_lazy_lethal_legion`, `0481_round_pet_avengers`, and `0482_aberrant_rumiko_fujikawa` before application rollout. The new credential version and digest columns are nullable. Existing unpinned connections require fresh OAuth consent; the migration deliberately does not trust or backfill a generic secret value. Regenerate or renumber migrations if master advances the sequence before merge. - The target needs an SSH server, its chosen harness, and allowed tailnet access. A reachable network does not grant SSH or provider authorization. Pairing reads standard `/etc/ssh` host-public-key files; custom host-key locations require manual known-hosts configuration. - Roll back the application revision to remove new entry points. Existing connection secrets and registered Tailscale nodes retain their separate lifecycle. Do not drop the table or delete node state as part of an application rollback. - Production qualification still needs denied-policy, revocation, cancellation, and deployment-overlap checks. Local Docker and a production deployment were not performed. ## Model Used OpenAI Codex, gpt-6.1-sol, xhigh reasoning, with local code execution and tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used - [x] I have checked ROADMAP.md and this extends existing connector and environment features - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR using the feature request fields - [x] I have not referenced internal issues, local URLs, or private machine identifiers - [x] My branch name describes the change - [x] I have run focused tests locally and they pass - [x] I have added tests for new behavior and failure paths - [x] I have updated documentation - [x] I have documented migration and runtime risks - [x] All Paperclip CI gates are green - [ ] Required code-owner approval received for the workflow change - [x] Greptile Apex is 5/5 on the final head with no unresolved comments - [x] I will address CI and review findings before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
5df1ada9c1 |
feat(plugins): send configured environment on plugin task resume (#21352)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - #21335 lets a plugin task provider that restarts the Runner on `resume` opt in (`resumeCredentialVersion: 1`) to receive the claimant run's harness credential > - A provider restarting the Runner also needs the agent's configured environment, which Core sends on `resume` only during a session handoff > - Without it, a Runner a provider moves to a new computer starts without the agent's configured environment variables > - This pull request widens that opt-in to both launch inputs, under a name that says so, before any provider adopts the old name > - The benefit is that a provider can restart a parked Runner on resume with the same credential and environment a `submit` would carry ## Linked Issues or Issue Description Refs #21289 Follows: #21335 **What existing behavior does this improve?** The opt-in resume launch inputs added in #21335. **Subsystem affected** Plugin SDK environment task schema (`describe` capability), the plugin Runner task owner's `resume`, and the plugin environment task docs. **Current behavior** With `resumeCredentialVersion: 1`, `resume` carries `credential`. `configuredEnvironment` is sent on `resume` only for a session handoff. **Proposed behavior** The capability is renamed `resumeLaunchInputsVersion: 1`. With it, `resume` carries `credential` as before and also `configuredEnvironment` (same builder as `submit`) when the provider advertises `configuredEnvironmentVersion: 1`. Session handoff is unchanged. **Reason and benefit** A provider that restarts the Runner on resume gets both launch inputs it needs. **Breaking changes** The capability name changes from `resumeCredentialVersion` to `resumeLaunchInputsVersion`. No provider advertises the old name yet (it merged in #21335 and has no adopter), so nothing changes behavior. ## What Changed - Rename `resumeCredentialVersion` → `resumeLaunchInputsVersion` in the SDK `describe` result schema, executor plumbing, owner input, docs and tests. - With the capability, `resume` also adds `configuredEnvironment` when the provider advertises `configuredEnvironmentVersion: 1`. A normal resume does not mark the variables as delivered, since a live Runner keeps the environment it started with. - Docs: the "Release and resume" section describes both fields; the "Configured environment" section points to it. - The runner-e2e SDK public-input pin is updated. ## Verification - SDK vitest `tests/environment-task-release-resume.test.ts` and `tests/environment-tasks.test.ts`: 68 passed. - Server vitest `plugin-runner-task.test.ts` and `environment-task-runtime.test.ts`: 153 passed, including a normal resume with the capability (credential sent, variables added only with `configuredEnvironmentVersion: 1`, delivered flag unset) and redaction of both fields from metadata writes and sanitized errors. - `native-session-executor.test.ts -t "uses a task-only"`: 5 passed. - runner-e2e `source-sdk-evidence.test.ts` and `source-sdk-public-type.test.ts`: 46 passed. - `pnpm typecheck` in `server/` (builds shared and plugin-sdk) passed. - Not run locally: the full suite and `pnpm verify`; required CI covers them. ## Risks - A provider restarting the Runner gets the claimant run's current variables, which can differ from the ones the parked Runner started with. That is intended. - Rollback: revert this pull request. No migration. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, test runs), implemented by a sub-agent and reviewed before opening. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
12aa08721c |
feat(ssh): use the agent AI account in new environments (#21248)
Use the agent-selected AI account for new SSH environments. Preserve
remote-machine authentication for existing environments and expose it as
an advanced option. Keep managed credentials private while preserving
supported conversation state and credential refreshes across turns.
Validated with focused route, adapter, lifecycle, and real-process tests;
final CI passed on
|
||
|
|
0199c963e2 |
feat(apps): ship the redesigned sign-in step for a pasted MCP server (#21469)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Apps lets an operator connect a remote MCP server by pasting its
address ("Connect your own MCP server")
> - Step 2 of that flow put every sign-in choice in an "Advanced
authentication" box with wrapped pill buttons and a hint paragraph
> - The box hid the choice behind a vague label, and its shape did not
match the access line ("Connects for everyone… Change") right below it
> - The owner approved a redesign: a stated sign-in default with a
Change link, and five options with their fields directly below each
option
> - This pull request wires that redesign into the real screen, removes
the old step, and updates copy that named the removed box
> - The benefit is a screen that reads as two stated answers and one
action, with the same default path (paste the address, press Check link)
## Linked Issues or Issue Description
No public issue exists. Enhancement:
**What existing behavior does this improve?**
The sign-in section of the "Connect your own MCP server" step in Apps.
**Subsystem affected**
UI: Apps connection setup (`ui/src/features/connections`). Small copy
changes in the server OAuth client registration message and docs.
**Current behavior**
Step 2 shows an "Advanced authentication" collapsible with five wrapped
pill buttons and a hint paragraph below them. The fields for the chosen
mode appear further down. Guidance banners tell the operator to look
"under Advanced authentication". The page also shows the "Unverified
server" badge twice (header and body). The stepper reads "Pick app · Add
your key", even when the server needs no key.
**Proposed behavior**
Step 2 states the sign-in choice in one line ("Paperclip asks the server
how to sign in.") with a Change link. Change shows five radio cards: Let
Paperclip check, Key or token, Browser sign-in, Custom headers, No
sign-in needed. The fields for the selected option are directly below
it. Done collapses the list to a one-line summary (for example "Signs in
with 1 custom header."). The list stays open while the chosen option
still needs a value. Client ID and secret are behind "Use your own
client ID" unless they have values or the server asked for them. The
stepper reads "Pick app · Connect".
**Reason and benefit**
The choice is visible as a stated default in the same shape as the
access line. Each option keeps its fields next to it. No copy points at
a box that does not exist.
**Breaking changes**
None. The five auth modes and their request payloads do not change.
**Additional context**
Related open PRs change server behavior for the same flow. They are not
duplicates. Refs #15192, Refs #15450.
## What Changed
- `ui/src/features/connections/GenericMcpConnectStep.tsx`: new step
component (same props as the old `LinkConnectStep`). It uses stable
field ids `generic-mcp-key`, `generic-mcp-client-id`,
`generic-mcp-client-secret` and a `data-testid` on the sign-in
Change/Done link.
- `ui/src/features/connections/ConnectionSetupFlow.tsx`: render
`GenericMcpConnectStep`; delete `LinkConnectStep`,
`GENERIC_AUTH_MODE_OPTIONS`, `SegmentedOption` and unused imports;
change the pasted-link stepper label from "Add your key" to "Connect"
(`STEP_LABELS` is used only by the pasted-link flow; curated apps keep
their own `["Add your key"]`).
- `ui/src/pages/apps/generic-mcp-connect.ts`: `oauth_challenge` and
`oauth_manual_client_required` guidance now say "below" instead of
"under Advanced authentication".
- `server/src/services/tool-access/oauth/client-registration.ts`: the
manual-client message no longer names the removed box.
- `doc/connections/GENERIC-REMOTE-MCP.md`: describe the sign-in line,
Change, and the five options.
- Comments in `packages/shared` and `server/src/services/tool-access.ts`
reworded. `tests/runner-e2e/source-sdk-public-inputs.json` regenerated
for the two shared files (blob hashes only).
- `ui/storybook/stories/connection-setup/byo-mcp-sign-in.stories.tsx`:
14 stories for the shipped component; annotation is now `current` /
`production`.
- Tests: `AppsConnect.test.tsx` generic MCP block uses the sign-in
toggle and radio options; new tests for the collapsed summary, the
open-while-empty-key rule, and a single "Unverified server" label. New
`GenericMcpConnectStep.test.ts` unit-tests `genericMcpAuthSummary`.
Before: "Advanced authentication" box with pill buttons and a hint;
fields below the pills; two "Unverified server" badges; stepper "Pick
app · Add your key".
After: one sign-in line with Change; five radio cards with fields under
the selected card; one "Unverified server" badge in the header; stepper
"Pick app · Connect". Storybook: `byo-mcp-sign-in--choosing`,
`byo-mcp-sign-in--server-asked-for-key`.
## Verification
- `cd ui && pnpm exec vitest run src/pages/apps/AppsConnect.test.tsx
src/pages/apps/generic-mcp-connect.test.ts
src/features/connections/GenericMcpConnectStep.test.ts
src/features/connections/ConnectionSetupFlow.architecture.test.ts
src/features/connections/ConnectionSetupFlow.oauth.test.ts`: 5 files,
300 tests pass.
- `cd ui && node ../scripts/run-tsc.mjs -b`: pass.
- `pnpm -s typecheck` in `packages/shared` and `server`: pass.
- `pnpm check:token-gates`: all gates clean.
- `node scripts/storybook-catalog.mjs`: pass (generated indexes not
committed).
- `pnpm verify`: exit 2 (broader verification required, because
`doc/connections/GENERIC-REMOTE-MCP.md` is a global CI input). Required
CI supplies that coverage. The focused files above were rerun on the
final revision after the rebase.
- Headless Playwright against the Storybook dev server: the Default,
Choosing and Server-asked-for-key stories render the new step, the new
guidance copy, and the "Pick app · Connect" stepper.
## Risks
- Low risk. UI change on one screen; request payloads
(`genericConnectPayload`) do not change.
- Tests that click the first collapsed "Change" button now skip the
sign-in toggle, because the access line also has a "Change" button.
- The server message for `oauth_manual_client_required` changes text. No
code matches on the text; the client uses the error code.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, headless Playwright).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
|
||
|
|
9d9ee92d15 |
refactor(evals): remove the orphaned chat TestOps dispatch guard (#21241)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The chat catalog eval harness sends selected cases to TestOps plans through GitHub workflows. > - PR #18322 added a dispatch guard script for the `chat-feature-testops` workflow. The guard checked each dispatch event before the workflow could use credentials. > - PR #18625 retired the duplicate chat TestOps plans. That PR deleted the `chat-feature-testops` workflow, the workflow step that ran the guard, and the only import of the guard function. > - The guard file stayed in the repository after that removal, but nothing calls it now. > - This pull request deletes the unused guard file. > - The benefit is less test support code to maintain. No protection is lost, because the workflow it guarded no longer exists. ## Linked Issues or Issue Description No public issue exists for this cleanup. The problem is described here, following the enhancement template. **What existing behavior does this improve?** The chat TestOps eval support under `tests/runner-e2e`. The guard script `testops-chat-dispatch.ts` verified workflow dispatch events for a retired workflow. **Current behavior** `tests/runner-e2e/testops-chat-dispatch.ts` holds a guard for the `chat-feature-testops` workflow. That workflow was deleted in #18625. A search of all tracked file content finds no import, no run command, and no test that uses the guard file today. **Proposed behavior** The repository does not keep the unused guard file. The runner-e2e typecheck program and the test tree load one support file less. **Reason and benefit** The guard protected a workflow dispatch path that no longer exists. Keeping the file suggests a live guard where none runs. Deleting it removes 31 lines of test support code. No regression coverage is lost: no test executes the file, and the contract it guarded is retired. **Breaking changes** None. The deleted file has no callers. No workflow, script, test, or documentation references it. ## What Changed - Deleted `tests/runner-e2e/testops-chat-dispatch.ts` (31 lines of test support code). No production code changed. ## Verification - Searched all tracked file content for `testops-chat-dispatch`, `validateChatTestOpsDispatch`, and `ChatTestOpsDispatch`. Only the deleted file matched. - Checked every selection path that could run the file: it is not a `*.test.ts` file; the runner-e2e harness selects test files through the catalog manifest, which does not name it; the root vitest projects do not include `tests/`; the `node --test` script lists do not name it; no workflow runs it. - Confirmed the history: #18322 added the file; #18625 deleted the workflow step `node tests/runner-e2e/testops-chat-dispatch.ts` and removed the only import and the assertions that referenced it; no commit touched the file after that. - Local limitation: this workspace has no installed dependencies, so the layered typecheck and the vitest suites could not run locally. The CI lanes on this pull request must confirm them. ## Risks - Low risk. The file had no callers. The remaining risk is a hidden reference that a text search cannot find. The typecheck lane and the CI test lanes on this pull request would find such a reference. ## Model Used - GLM by Z.ai, model id `z-ai/glm-5.3`, run through opencode by the Paperclip Runner. Extended reasoning enabled, with shell and file tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [ ] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge |
||
|
|
e5158293c5 |
perf(ui): keep the live socket when a cached session is verified (#20591)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The board UI keeps one live-updates WebSocket per company and saves
a copy of its query cache in `localStorage` so a returning visit can
paint from it
> - On a returning hard load of the Mine inbox, about eight shell and
inbox reads ran a second time 1.5–2.5 s after load (session, health,
company list, agents, inbox dismissals, user directory, approvals,
execution workspaces)
> - The cause is the socket's sign-in key. A session restored from the
saved cache has the user but no session id. The first session read adds
the id, the key changes, the socket closes and opens again, and that
second open refreshes every active query, session and health included
> - This pull request keeps the socket when the verified session belongs
to the same user as the restored one
> - The benefit is one read of each shell request per returning load and
one socket instead of two, with no change to the catch-up read that
covers the gap before the first subscription
## Linked Issues or Issue Description
**What happened?**
A returning hard load of `/:companyPrefix/issues?view=mine` sent
`auth/get-session`, `health`, `companies?scope=accessible`, agents,
inbox-dismissals, user-directory, approvals and execution-workspaces
twice. The second copy started about 1.5–2.5 s after load on the hosted
staging instance.
**Steps to reproduce**
1. Open Mine once so the app saves its query cache.
2. Reload the page.
3. Watch the network panel. The live-updates socket opens, closes, opens
again, and the reads above repeat.
**Expected behavior**
Each read runs once. The socket opens once.
Related: #20398 (socket reconcile skips settings and preferences),
#20407 and #20419 (Mine reveal and its perf guard).
## What Changed
- `ui/src/context/LiveUpdatesProvider.tsx`: the socket auth key comes
from `nextSocketAuthScope`. When the previous scope has the same user
and no session id, the first known session id is adopted without a new
key. A different user, or a different session id once one is known,
still produces a new key and a new connection.
- `ui/src/context/LiveUpdatesProvider.recovery.test.tsx`: two tests. A
restored session (no id) verified for the same user keeps the socket and
invalidates nothing; a later replacement session still reconnects. A
verified session for another user reconnects.
- `tests/perf/collections/shell-read-waves.spec.ts` and
`returning-visit-fixture.ts`: browser checks on a production build with
every API response delayed 300 ms. With a saved cache, each of the eight
reads runs at most once and the socket opens once. Without a saved
cache, session and health run once, no read runs more than twice, and
the socket opens once.
- `tests/perf/collections/README.md`: documents the new spec and
fixture.
## Verification
- `cd ui && pnpm exec vitest run src/context/`: 16 files, 286 tests
pass. The new same-user test fails on `master` (socket count 2) and
passes with this change.
- Browser check on a disposable local instance (production UI build,
`local_trusted`, embedded PostgreSQL, Chromium headless, 1440×900, API
+300 ms):
`PAPERCLIP_COLLECTION_PERF_BASE_URL=http://127.0.0.1:<port> pnpm exec
playwright test --config tests/perf/collections/playwright.config.ts
shell-read-waves.spec.ts`
- `master` build (`99edcd28a7`): returning visit fails with
`{"auth/get-session":2,"health":1,"companies?scope=accessible":2,"agents":2,"inbox-dismissals":1,"user-directory":1,"approvals":1,"execution-workspaces":0}`.
- This branch: both tests pass; every count is 1 and one socket opens.
- Timing evidence (same instance, 5 fresh contexts per arm, API +300 ms,
host load average about 20; `mine-returning-visit.spec.ts` timing case
from the companion Mine PR). Duplicate shell reads are summed over the 5
loads:
| Build | Saved cache: time to rows, median (range) | Saved cache:
duplicate shell reads | Cleared cache: median (range) | Cleared cache:
duplicate shell reads |
| --- | --- | --- | --- | --- |
| `master` | 1028 ms (906–3011) | 5 (session ×2 each load) | 1502 ms
(1423–1592) | 5 (company list ×2, the catch-up) |
| This PR | 895 ms (880–930) | 0 | 1364 ms (1328–1408) | 5 (company list
×2, the catch-up) |
The time difference is small and inside host noise; the claim here is
the removed reads and the single socket. On the busier hosted instance
the second wave was larger (about eight reads), because more reads have
answered before the session read returns.
- Review follow-up (
|
||
|
|
64ec273845 |
perf(ci): run the typecheck lane's checks in parallel and drop its duplicate server type check (#21404)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Every pull request runs the trusted PR workflow, and the `Typecheck + Release Registry` job is one of its required checks > - That job takes about 9.2 minutes on the 4 vCPU / 16 GB fleet runner (run 43344818387), against a target of 7 minutes or less for every job > - The job runs six steps one after another, and the first step type-checks the whole server program a second time, at about 13.5 GiB, which makes the runner swap > - Typecheck results are the check that authors most want back quickly, so this job is on the critical path of most pull requests > - This pull request removes the duplicate server type check and runs the independent checks in parallel through one small script, inside the same memory budget > - The benefit is about 39% less wall time for the job's checks on the same machine, with peak memory down from about 14 GiB (plus swap) to about 8.5 GiB ## Linked Issues or Issue Description **What existing behavior does this improve?** The `Typecheck + Release Registry` job in `.github/workflows/pr-trusted.yml`. It takes about 9.2 minutes. The target for every job in the workflow is 7 minutes or less. **Subsystem affected** CI (`.github/workflows/pr-trusted.yml`) and the typecheck scripts under `scripts/` and `server/scripts/`. **Current behavior** After about 75 s of setup, the job runs six steps in sequence on a 4 vCPU runner, about 470 s in all (run 43344818387): `typecheck:build-gaps` 121 s, test programs 85 s, server layers 128 s, test projects 38 s, CLI guidance test 5 s, release registry tests 95 s. Most of these steps use one thread. The `typecheck:build-gaps` step builds the server with a full type check of the whole server program. The server layer step then checks the same files again, and the `Build` job checks them a third time. That full program peaks at about 13.5 GiB, so the job peaks at about 14 GiB of 16 GiB and uses swap. **Proposed behavior** The job runs `node scripts/ci-typecheck-lane.mjs`. The script builds the server without type checks (`PAPERCLIP_TSC_EMIT_ONLY=1`, as the Docker image does) and still checks the runtime assets. It runs the steps that write shared files first, one at a time. Then it runs the remaining checks in parallel, one per CPU, with the server declaration layers first, because the test layers and test projects read them. The checks that are not on that path run at a lower CPU priority. **Reason and benefit** On the same runner, the job's checks take 61% of the time and about 60% of the memory (see Verification). The job should take about 6 minutes on the fleet instead of 9.2. **Breaking changes** None. The same files are type-checked with the same options. Local commands do not change. `pnpm run typecheck:build-gaps` still works as before. ## What Changed - `scripts/ci-typecheck-lane.mjs` (new): runs the job's checks. It writes shared outputs first, in sequence: links, plugin SDK, server build without type checks, the runtime asset check, the sandbox provider installs, and the Runner typecheck (it rebuilds the eval kernel, which other checks read). Then it runs these checks in parallel: the server declaration layers, each server test layer, each test project, the test programs, the other build-gap workspaces, the CLI guidance test and the release registry tests. A passing check prints its output in a collapsed log group. A failing check prints its full output between `=== FAILED` and `=== end of` lines, with an error annotation. The script ends with a table of each check's start, duration and result, and exits non-zero if any check fails. - `server/scripts/typecheck-layers.mjs`: new `--layer <name>` option that checks one layer against the configs and declarations of an earlier `--declarations-only` run, and an exported `listLayers`. - `scripts/typecheck-test-programs.mjs`: new `--prepare-only` and `--skip-prepare` options, so the installs happen before the parallel checks start. - `scripts/run-typecheck-build-gaps.mjs`: new `--list-packages` option, so the script still finds the build-gap workspaces and still fails when one has no `typecheck` script. - `.github/workflows/pr-trusted.yml`: the `typecheck_release_registry` job runs the script instead of the six steps. No other job changes. - Tests: new `scripts/__tests__/ci-typecheck-lane.test.mjs` (added to `test:release-registry`), new cases in `server/scripts/typecheck-layers.test.mjs` and `scripts/typecheck-test-programs.test.mjs`, and the workflow wiring assertions in `.github/scripts/tests/ci-plan.test.mjs` and `scripts/__tests__/runner-release-crate-cache.test.mjs`. - `tests/runner-e2e/source-sdk-public-inputs.json`: repinned, because `package.json` changed. - `doc/DEVELOPING.md`: new "Typecheck lane in CI" section, and the new options. ## Verification This PR's own CI ran the old job definition, because PR runs use the workflow from master. So I measured the commands on a stand-in machine: a temporary workflow (`.github/workflows/typecheck-lane-bench-temp.yml`, now deleted) on GitHub-hosted `ubuntu-latest` (4 vCPU Intel Xeon 6975P-C, 15.7 GiB, no secrets, `contents: read`). It ran in runs [43444356017](https://github.com/paperclipai/paperclip/actions/runs/43444356017), [43445393958](https://github.com/paperclipai/paperclip/actions/runs/43445393958), [43446786105](https://github.com/paperclipai/paperclip/actions/runs/43446786105), [43448707837](https://github.com/paperclipai/paperclip/actions/runs/43448707837) and [43450786493](https://github.com/paperclipai/paperclip/actions/runs/43450786493). Each step ran under `/usr/bin/time -v`, and a sampler read `/proc/meminfo` every 0.25 s. **Old steps, one at a time (3 runs, run 43445393958):** | Step | Wall s | CPU s | Peak RSS of one process | System memory peak | | --- | --- | --- | --- | --- | | `typecheck:build-gaps` | 128–155 | 376–400 | 13.2–13.6 GiB | 14.2–14.7 GiB | | ...of which the full server `tsc` | 40–51 | 146–164 | 12.8–13.8 GiB | 13.7–14.7 GiB, swap up to 738 MiB | | ...replaced by the emit-only server build | 6–10 | 20–22 | 2.9 GiB | 3.9 GiB | | Test programs | 67–87 | 147–170 | 1.9 GiB | 2.9 GiB | | Server layers | 102–127 | 150–172 | 4.0–4.2 GiB | 5.4–5.6 GiB | | Test projects | 31–37 | 49–59 | 2.1–2.2 GiB | 3.1–3.2 GiB | | CLI guidance test | 2–4 | 2–3 | 0.2 GiB | 1.6 GiB | | Release registry tests | 82–95 | 159–183 | 1.7 GiB | 3.0 GiB | CPU seconds are about twice the wall seconds for the `tsc` steps even on one checker thread. So each step uses about two of the four vCPUs. **Old sequence and new lane on the same runner (4 runs, run 43450786493).** Each job copied one installed checkout, then ran both, in alternating order: | Run | Old steps, wall s | New lane, wall s | Ratio | Old memory peak | New memory peak | | --- | --- | --- | --- | --- | --- | | 1 | 503 | 314 | 0.62 | 13.98 GiB | 8.33 GiB | | 2 | 647 | 388 | 0.60 | 14.04 GiB, swap 637 MiB | 8.13 GiB | | 3 | 627 | 385 | 0.61 | 13.98 GiB | 8.63 GiB | | 4 | 382 | 237 | 0.62 | 14.10 GiB, swap 1.1 GiB | 7.99 GiB | Hosted runners differ in speed by up to 1.7 times, so only the same-runner ratio is meaningful. Both arms include a cold compile of the Runner release crate (45–80 s) that the fleet restores from cache. The fleet ran the old steps in 470 s; at the measured ratio, the new lane should take about 290 s, and the job about 6 minutes with setup. The new lane peaks at about 55% of memory, with no swap. Local checks (on a loaded Mac, for correctness only): - `node --test scripts/__tests__/ci-typecheck-lane.test.mjs` — 8 pass. - `node --test .github/scripts/tests/*.test.mjs` — 807 pass. - `node --test scripts/__tests__/e2e-shard.test.mjs scripts/__tests__/release-verify-workflow.test.mjs scripts/__tests__/runner-release-crate-cache.test.mjs scripts/__tests__/run-typecheck-build-gaps.test.mjs` — pass. - `pnpm exec vitest run scripts/typecheck-layers.test.mjs` in `server/` — the new cases pass. ## Risks - This changes a required job that every PR depends on, and its own CI cannot prove it. Please watch the first `Typecheck + Release Registry` runs on master after merge: the wall time (expect about 6 minutes), the summary table at the end of the step, and any `terminated by` message. - A type error now shows in that check's `=== FAILED` block, not in a step of its own. Passing checks are folded, so the failure is the only expanded block. - When the server declaration layers fail, the test layers and test projects are reported as skipped. Before, the test layers still ran after a source error. Fix the source errors first; the next run checks the tests. - The Honeycomb CI time board and any step-name queries see one step, `Typecheck, CLI guidance safety and release registry tests`, instead of six. The script prints the duration of each check. - The release registry tests run beside the type checks at normal priority. Their slowest test took 46–78 s here, against a 180 s timeout, and 114 s on the slowest stand-in runner. - Low risk for memory: the new lane peaked at 8.0–8.6 GiB of 15.7 GiB, far below the old 14 GiB. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), through Claude Code, with tool use (shell, GitHub CLI, file editing). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
0ece025749 |
perf(ci): cut repeated CPU work in the Runner support suite (#21390)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Every pull request runs the "Runner support suite" CI job (`pnpm test:e2e:runner:unit`) > - That job took 8.1–10.6 minutes, depending on the spot instance type, and was one of the last PR jobs to finish > - The target for every PR job is 7 minutes or less > - Measurement showed the suite is CPU-bound: each of the 693 Vitest files loads `@playwright/test`, the runner catalog and shared modules again in its own worker, and a few modules did quadratic or repeated work when they loaded > - This pull request removes that repeated work and reuses compiled code across workers > - The benefit is a job 75–86 s (13–16%) shorter on each instance type, with the same files, tests and assertions. It does not reach 7 minutes; the floor is explained below ## Linked Issues or Issue Description **What existing behavior does this improve?** The duration of the "Runner support suite" PR job. **Subsystem affected** CI, the Runner E2E support suite under `tests/runner-e2e/`, and one shared module (`packages/shared/src/connection-routing.ts`). **Current behavior** In 26 recent green PR runs the job took a median of 526 s on `m7i-flex.xlarge` (17 runs), 549 s on `m7i.xlarge` (3 runs) and 604 s on `m6a.xlarge` (6 runs). Vitest ran 693 files on 3 workers. The CPU was about 82% busy. 52% of the tracked time was module evaluation. **Proposed behavior** The job is shorter on every instance type. The same files and tests pass, and no assertion changes. **Reason and benefit** The PR workflow finishes sooner, so every pull request gets its result sooner. **Breaking changes** None. ## What Changed Measurements below are from the job's own runs on this PR. The spot fleet picks `m7i-flex`, `m7i`, `m6a` or `m5a` (all 4 vCPU); speed differs by up to 2x between them, so each comparison uses the same instance type. This suite has no Vitest retries, so a flaky test fails the job. ### Baseline (recent green PR runs on master, before this PR) | Run (job id) | Job | Test step | Vitest wall | Vitest split | | --- | --- | --- | --- | --- | | 131933126599 | 523 s | 474 s | 469.8 s | import 52%, tests 43%, transform 5% | | 131932775003 | 510 s | 461 s | 457.1 s | import 51%, tests 44%, transform 5% | | 131926764360 | 527 s | 479 s | 475.2 s | import 51%, tests 43%, transform 5% | | 131924082813 | 541 s | 496 s | – | – | | 131912527065 | 517 s | 468 s | – | – | - One Vitest invocation (693 files, 3 workers, the Vitest default of CPUs − 1), then 386 `node:test` tests in about 3 s. About 45–50 s of the job is checkout and dependency restore before the test step. - Summed tracked time was about 1,297 s for 470 s of wall time: effective concurrency 2.76 of 3 workers. Module evaluation was 674 s: 4,164 modules evaluated 77,699 times, because each file runs in a fresh isolated worker. - A CPU sampler (temporary, removed) showed 82% average CPU use with 3 workers and 95–99% with 4 workers. With 4 workers and no other change the wall time did not improve (455 s). The suite was CPU-bound, not idle. - Top modules by summed evaluation time (diagnostic run): `@playwright/test` 158 s over 390 files, `tests/runner-e2e/catalog.ts` 56 s over 141 files, `packages/shared/src/connection-routing.ts` 40 s over 317 files, `acpx/dist/runtime.js` 39 s. - Slowest files: `chat-rendered-links` 43 s, `results-root` 43 s, `chat-feature-flow` 43 s, `report` 40 s, `manual-private-transfer` 26 s, `private-framework-report` 24 s, `chat-catalog-fingerprint` 24 s, `hire-delegate-flow` 23 s, `source-sdk-evidence` 19 s, `unit-launcher` 16 s. No single file is near the 7-minute target, so there is no longest-file floor. - Slowest tests: the 256 MiB archive-cap test 23 s, the reload-once reply wait 20 s (plus two at 10 s), the documented pnpm validation 16 s, the retained name-only identity replay 14 s, the selected-case fingerprint replay 13 s. ### Changes, one at a time, with what each one bought 1. **Compile cache.** `run-unit.mjs` sets `NODE_COMPILE_CACHE` to `<tmpdir>/paperclip-runner-e2e-compile-cache` unless it is already set. Vitest workers and the Node processes that tests start reuse V8 compiled code. Node validates each entry against the source. `@playwright/test` evaluation went from 158 s to 107–124 s, `acpx` from 39 s to 26–27 s, and `zod` from 17 s to 9–10 s. Vitest went from 478 s to 441 s on a comparable run (this run also included change 2). 2. **Reuse import scans in the chat catalog fingerprint.** `fingerprintChatDependencyClosure` ran one esbuild parse for each of about 2,300 files on every call. The parse result is now cached by file path and content SHA-256 for the process. The fingerprint inputs and digest are unchanged. `chat-feature-flow` went from 43 s to 12–14 s, `chat-catalog-fingerprint` from 24 s to 11–13 s, and `chat-campaign-identity` from 13 s to 8 s. 3. **Hash each suite definition once.** `buildRunnerMatrix` computed `suiteDefinitionHash(suite)` for every execution, thousands of times, every time a test file loaded the catalog. It now computes it once per suite. The values are the same. `catalog.ts` evaluation went from 51 s to 9.6 s. Vitest went from 441 s to 389 s. 4. **Fake timers for the reload-once reply waits.** These three tests in `chat-rendered-links.test.ts` waited out the real 10 s render floor twice. They now advance fake time with `vi.runAllTimersAsync()`. The assertions on results, reload counts and attempt phases are unchanged. The file went from 43 s to 2.8 s. This freed a worker slot; on its own it did not shorten the CPU-bound run. 5. **Derive each aggregator identity once.** `connection-routing.ts` called `aggregatorAppIdentity` (two regex replaces) for every index entry for each of the 133 Arcade apps while it loaded, in 317 test files and in the server. The identity of each entry is now computed once. I compared the resulting `AGGREGATOR_SUPPORT_INDEX` (1,579 entries) and `searchAggregatorServices` results with master: they are identical. The SDK public input pins are refreshed for this change. 6. **Not shipped: four workers.** With the CPU work above removed, four workers cut Vitest from 379 s to 341 s (job 418 s). In the next run, `chat-installed-lane.test.ts` failed: the descendant of a timed-out preparation child was still visible to `process.kill(pid, 0)` right after cleanup. This is the process-tree-under-load risk, so the suite keeps the Vitest default of 3 workers. Docs: `tests/runner-e2e/README.md` describes the compile cache, how to turn it off, why the worker count stays at the default, and the cost of work at module load. ### Result | Run | Job | Test step | Vitest wall | Effective concurrency | | --- | --- | --- | --- | --- | | Baseline median, `m7i-flex` (17 runs) | 526 s | 465 s | 462 s | 2.76 | | Baseline median, `m7i` (3 runs) | 549 s | 476 s | 473 s | – | | Baseline median, `m6a` (6 runs) | 604 s | 535 s | 531 s | – | | This PR, `m7i-flex` (job 132007703795) | 440 s | 383 s | 379 s | 2.79 | | This PR, `m7i` (job 132027384596) | 474 s | 424 s | 420 s | 2.78 | | This PR, `m6a` (job 132033745288) | 526 s | 471 s | 466 s | 2.79 | | This PR, `m5a` (job 132040621237; dependency cache miss) | 880 s | 768 s | 761 s | – | The last three rows are the final configuration. The `m7i-flex` row is the same code before a README-only change. Module evaluation fell from 77,699 to 63,135 evaluations; part of that drop came from master changes merged into this branch, so I attribute time only to the per-change measurements above. ## Verification - CI on this PR is the experiment. Every push ran the real job on the real runner. The table above lists each run. - Local correctness checks, one process at a time: `chat-rendered-links.test.ts` (58 passed), the six chat fingerprint test files (37 passed), `catalog.test.ts` (92 passed), `packages/shared` `connection-routing.test.ts` and `connection-search.test.ts` (21 passed), and `node tests/runner-e2e/run-unit.mjs connection-auth.test.ts` (launcher path, 386 node:test passed). - Equivalence check for the connection routing change: the index and search results are byte-identical to master. - Flakiness: 10 runs of this branch with 3 workers (including the three final runs on `m7i`, `m6a` and the 2x slower `m5a`) passed with 0 failed tests and 0 timeouts. No port conflicts, OOM kills or `ps` timeouts appear in the logs. The suite has no retries. The 4-worker setting failed once in 3 runs, so it is not used. ## Risks - Low risk. No test is skipped or removed. No assertion is weakened. - The worker count is unchanged, so the load on process, port and temp-directory tests is lower than before (less CPU work per file). - The compile cache directory lives in the OS temp directory. Node rejects a stale or partial entry and compiles again. `NODE_DISABLE_COMPILE_CACHE=1` turns it off. - The connection routing change runs in the server too. It produces the same index, and it makes loading the module cheaper. ### What sets the remaining floor The job is about 45–55 s of checkout and dependency restore plus the Vitest run. The Vitest run is CPU-bound on 4 vCPU (2 physical cores). A fourth worker gives about 40 s more but fails a process-tree cleanup test under load (`chat-installed-lane.test.ts`, which checks the descendant immediately after cleanup). The largest remaining cost is per-file isolation: every file loads `@playwright/test` (about 0.3 s CPU each, 390 files), `zod` schemas from `packages/shared`, and `acpx` again. Vitest estimates that `isolate: false` would save about 140 s now. I did not use it: 116 files use `vi.mock`, and module state shared between files would make failures depend on which files share a worker. If more headroom is needed, the next step is a split across two machines (out of scope here because it changes `.github/`). Two duration-balanced halves (`vitest --shard=1/2` and `--shard=2/2`, about 190–235 s of Vitest each on `m7i-flex`/`m6a`) would put each job at about 4–5 minutes on every instance type except `m5a`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Claude Code agent with tool use (shell, GitHub CLI) and extended reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable (one test moved to fake timers; behavior changes are equivalence-checked) - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
a762b6bffd |
test(evals): keep the spend fixture's pointer to the tasks that record the charges (#21429)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - #21408 rewrote the spend-breakdown eval setup text in user words > - Its first wording ("Launch has recorded API charges on COM-3 (Morgan) and COM-4 (Riley)") dropped the old pointer that these tasks hold the charges > - In two paid runs the agent then gave correct totals but did not link the tasks that record them, which the case contract needs ("linked records support the totals") > - This pull request restores that pointer in plain words > - The benefit is that the setup gives the same information as before, without jargon ## Linked Issues or Issue Description Refs #21408. This commit was pushed to #21408 after it merged, so it is a follow-up. **What happened?** Paid runs 43451957487 and 43455387724: `budgets-costs.spend-breakdown` failed `actual-agent-breakdown-and-incomplete-total` because the reply did not link COM-3 and COM-4. **Expected behavior** The setup says where the charges are recorded, as the original text did. **Steps to reproduce** 1. Run `budgets-costs.spend-breakdown` with the #21408 wording. ## What Changed - The setup description now says "Launch's API charges this week are recorded on tasks COM-3 (Morgan) and COM-4 (Riley)." ## Verification - Paid run 43460401711 with this wording: `budgets-costs.spend-breakdown` passed. The reply links COM-3, COM-4, the project and the unpriced run, with no internal terms. ## Risks - Low. Eval setup text only. ## Model Used - Claude, claude-opus-5-5, with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
edb6945ff3 |
chore(lockfile): refresh pnpm lockfiles (#21439)
## Thinking Path > - Paperclip uses a dependency lock for reproducible builds. > - Dependency changes on master need matching lock entries. > - Paid eval reports also pin a reviewed offline correction. > - Reports must install before paid model work starts. > - This automated PR keeps the lock and report proof together. ## Linked Issues or Issue Description **What happened?** Dependency manifests on master changed after the last committed lock refresh. **Expected behavior** Frozen installs must match those manifests. Eval reports must retain matching offline lock proof. **Steps to reproduce** Run a frozen install before and after this lock refresh. ## What Changed - Refresh the dependency lock from the manifests on master. - Refresh the sandbox provider lockfiles from their manifests on master. - Include the matching offline report correction and hashes. - Resync the SDK public source proof's lock pins from the refreshed lock. ## Verification - Pinned pnpm resolution completed with lifecycle scripts disabled. - The report proof refresh accepted the reviewed committed baseline. - The workflow rejected changes outside the lock and report proof files. ## Risks The lockfile bot merges this PR without CI or review when every added package version is at least 3 days old on npm. Otherwise the PR waits for a maintainer, and the daily refresh tries again. The bot merges only the exact commit this workflow generated from manifests already reviewed on master. Unknown report lock baselines remain blocked. This update changes no permissions or paid enable gates. ## Model Used None — generated by pnpm and the deterministic report-lock updater. ## Checklist - [x] Thinking path and problem description included - [x] Generated lock and matching report proof included - [x] Release-age gate decides between bot merge and maintainer review Co-authored-by: lockfile-bot <lockfile-bot@users.noreply.github.com> |