mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-03 17:51:58 +01:00
33fc7fcddc80d37f356aa4b46936f3f6413aebb1
6141
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
33fc7fcddc |
fix(db): clean up owned PostgreSQL test servers (#18627)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Database, server, and CLI tests use real PostgreSQL fixtures. > - Each fixture must own its server until PostgreSQL has stopped. > - The old teardown returned after five seconds even when its server remained alive. > - A startup failure could then mark the host unsupported and skip database coverage. > - This pull request waits for owned shutdown and makes supported-host startup failures fail collection. > - A shared-server command reduces process creation while keeping each fixture's database isolated. ## Linked Issues or Issue Description Refs: #7756 for related development-process ownership. That development-server issue remains outside this test-fixture fix. Related PR: #16799 collects macOS startup diagnostics. This PR changes test shutdown and failure behavior. **What happened?** Repeated test runs can exhaust macOS PostgreSQL semaphore resources. The fixture abandoned slow shutdown and swallowed stop errors. The support probe converted ordinary startup failures into skipped integration coverage. **Expected behavior** Tests must stop their PostgreSQL servers, retain files if stop fails, and report resource exhaustion as a failure. Fixtures on a shared server must not close another database's clients. **Steps to reproduce** Start a fixture whose PostgreSQL stop exceeds five seconds, or make a supported-host startup fail with semaphore exhaustion. The former helper returned from teardown or reported unsupported coverage. The new regression tests exercise both paths without exhausting host resources. **Paperclip version or commit** Source checkout based on current master. **Deployment mode** Local source development and CI tests. ## What Changed - Add explicit ownership for test PostgreSQL children. Wait for fast shutdown, then request immediate PostgreSQL shutdown if needed. Keep cleanup idempotent and fail when exit cannot be confirmed. - Publish reusable templates only after clean shutdown. Do not publish a crashed or immediately stopped cluster. - Stop owned children on POSIX worker signals and process exit. Preserve test directories when shutdown fails. Remove conflicting dependency exit hooks. - Retry port collisions only. Fail collection for resource exhaustion, unavailable configured servers, and other startup failures on supported platforms. Preserve skips for explicitly unsupported platforms. - Move administrative-server support into the common fixture used by database, server, and CLI tests. Close clients at database scope and migrate cloned templates before use. - Add a directly invoked shared-server runner to own one test server and run suites against separate disposable databases. Preserve child exit status and support an existing disposable server. - Unify cancellation: stop and wait for the test process group before PostgreSQL teardown. Add nested-process coverage, real worker-exit coverage, shutdown regressions, shared-database isolation checks, and setup documentation. Replace ignored heartbeat completion polls with the existing execution drain before asserting persisted outcomes. - Finish owned cluster-directory cleanup after signal-driven cancellation. Preserve prepared templates across shared-fixture tests. Remove stopped directories asynchronously, using native removal on POSIX with a Node fallback. - Align database and CLI fixture hook budgets with the existing 30-second server budget. This lets the ten-second shutdown protocol finish and report errors. The idle-sleep suite uses the existing 90-second fixture budget for its unusually large final directory cleanup. ## Verification - The semaphore-exhaustion regression failed against the old helper because it resolved to `supported: false` instead of rejecting. - All 37 focused tests pass on the final runtime code (`e4de00ab40`). They include real worker exit, SIGTERM, nested-process SIGINT/SIGTERM cancellation with PostgreSQL available during child cleanup, failed template shutdown, unsupported platforms, stale SQL templates, and directory removal. A child exiting with status 7 makes the shared-server runner return status 7 after cleanup. - Full local `pnpm -r typecheck` and `pnpm build` pass on the final runtime code (`e4de00ab40`). Subsequent changes only adjust server-test lifecycle waits, the idle-sleep cleanup budget, and documentation; the server typecheck also passes with those final tests. - A fresh shared-server run on `90b65fd8ad` passed all 68 database files and 297 tests. Earlier preparation runs had two ten-second teardown timeouts and a stale-template failure; the final helper migrates cloned templates and permits the full shutdown budget. These earlier failures remain accounted for here. - The local full repository attempt stopped on a Discord receiver-lease test after 338 passing chat tests. That exact case passed in isolation. This full-suite attempt is not a pass. - On `90b65fd8ad`, [server shard 6](https://github.com/paperclipai/paperclip/actions/runs/41496495253/job/125452949369) and its one unchanged retry reported two 15-second cached-input heartbeat timeouts and a 30-second idle-sleep teardown timeout. All ten cached-input cases then passed locally with their original deadlines. A local parallel run of the full files reported eight behavioral/hook failures and two suite teardown timeouts (281 tests passed). - Serial idle-sleep runs passed all 225 test assertions but overran the 30-second final hook. Tracing measured client closure in 4 ms, PostgreSQL exit after 5 seconds, and synchronous directory removal taking another 35 seconds. Asynchronous Node removal and native removal also exceeded that hook under the full workload. Cleanup now remains asynchronous and awaited, and only this heavy suite's final hook uses the existing 90-second fixture budget. No PostgreSQL shutdown deadline or test assertion was weakened. The serial run then passed the complete idle-sleep file, including all 225 tests and teardown. Its heartbeat file passed 63/64 cases, exposing an ignored three-second completion poll. After replacing that poll with an execution drain, another timer case exposed the same pattern (63/64 passed). All remaining ignored completion polls now await execution drain without changing test deadlines. The complete heartbeat file then passed all 64 tests. The intentional asserted lock-contention poll is unchanged. The final server typecheck passes, and a process audit found no PostgreSQL server left by the completed local runs. - On final head `2cf316d3f571e4516890cba743ecffeb9703cce3`, the previously failing server shard 6, database lane, full typecheck, and build passed. The [first CI attempt](https://github.com/paperclipai/paperclip/actions/runs/41506358136) then lost self-hosted runners in the canary and serialized-server lanes, with additional jobs cancelled. The server-2 lane had one 30-second timeout in the unchanged workspace-recovery HTTP Retry test (2,282 other tests passed). That exact test passed locally with its original deadline (28.78 seconds including imports). One unchanged retry of only failed and cancelled jobs completed successfully on the exact final commit (workflow attempt 2). The workspace-recovery timeout case and every interrupted lane passed; successful first-attempt jobs were retained. The additional Paperclip Review check remains queued, waiting for its assigned agent; it has no unresolved GitHub review threads. Greptile completed review on this exact final head at 5/5, with zero unresolved review threads. All four original review findings and their threads are resolved; additional Windows shutdown, cancellation-directory, and template-reuse findings are fixed. - `pnpm check:module-boundaries` and `git diff --check` passed. Post-run process checks found no PostgreSQL servers left by the completed verification runs. ## Risks - Immediate PostgreSQL shutdown can require crash recovery. This fallback applies to disposable test clusters and runs only after fast shutdown times out. - No process can handle `SIGKILL` or machine power loss. Host resource cleanup remains a separate backstop. - Directory cleanup on cancellation applies to the shared-server runner's own cluster. Directly terminated test workers stop their PostgreSQL process but retain disposable files for inspection. The process-exit guard cannot await file removal. - The two-second `pg_ctl -W` deadline bounds signal dispatch. It is separate from the five-second and thirty-second server-exit waits. A control-command failure fails cleanup and retains its directory; it does not report successful cleanup or force-delete live files. - Shared fixtures require a disposable PostgreSQL server and a role that can create databases. The directly invoked shared command is opt-in; the existing `pnpm test` command remains available. - These changes do not stop unrelated development servers or delete host-wide IPC resources. Clean shutdown uses the owned `pg_ctl` binary on every supported platform. Its real control path was tested on macOS. A Windows host was not available for live verification; POSIX signal tests skip on Windows. ## Model Used - OpenAI Codex, based on GPT-6. The runtime does not expose the exact serving variant or context-window size. The agent used reasoning, shell execution, code editing, and GitHub tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
53f480d1b3 |
fix(native): preserve warm continuity and deferred review replies (#18631)
## Thinking Path > - Paperclip manages AI agents and their saved work. > - Product E2E tests must check saved personal files across separate turns. > - The server records file collection in the durable `instruction_save` run-log event. > - The warm test read optional terminal result metadata instead and rejected a recorded successful save. > - This pull request reads the complete event stream for the current run and uses its latest save receipt. > - The prompt now specifies the exact nine bytes required by the temporary-file transfer count. > - Human review actions select the actual interaction card, which can share its interaction ID with a navigation button. > - A later real three-turn run exposed a missing reply after workspace finalization moved to its durable recovery owner. > - Reply recovery handled completed tasks but omitted ordinary tasks that awaited review. > - This pull request also publishes their recorded final reply once while preserving the pending review. ## Linked Issues or Issue Description **What happened?** The [retained live continuity attempt](https://github.com/paperclipai/paperclip/actions/runs/41477302876/job/125420001074) saved the correct memory file and unchanged 8 MiB binary. Its durable run log contained a successful `instruction_save` receipt. The oracle read `resultJson.instructionSave`, which was absent, and failed before turn two. The temporary file also had a trailing LF. Its prompt allowed that result, while the byte-count check required nine bytes. **Expected behavior** The oracle must use the documented authoritative save event. A missing receipt, a later failed save, incorrect transfer counts, or another run's events must fail. The prompt must specify the bytes that its independent check requires. **Steps to reproduce** 1. Run `daytona-warm-continuity.runner-codex.daytona.warm-three-turn` with the configured protected credentials and immutable image. 2. Complete a native turn whose terminal result has no `instructionSave` field. 3. Compare the oracle output with the same run's successful `instruction_save` event. A subsequent [three-turn attempt](https://github.com/paperclipai/paperclip/actions/runs/41495337334/job/125451304553) passed every personal-file assertion and reused the same warm process. Its first final reply was recorded as a completed agent-message event but was absent from comments. The log shows `native_workspace_finalization_busy`; the durable recovery path excluded `needs_review` replies. The final message matcher correctly failed. This failure remains recorded. The observed application revision was `aab87ce70744660a4a21c77278b0ede267c04720`. Related work: #14695 and #18494. The retained failed result keeps its original grade and definition. ## What Changed - Read all run-event pages through the existing bounded collector. - Require events to belong to the requested run and select the latest `instruction_save` event. - Retain strict byte counts and reject missing, failed, pending or superseded receipts. - Specify nine ASCII bytes and no trailing newline for the temporary file. - Scope human review actions to the interaction card and its exact interaction ID. - Advance the warm suite definition to version 5 and document its evidence source. - Recover recorded final replies for ordinary native tasks with an accepted `needs_review` result and terminal state. - Preserve pending reviews, existing reply selection, company/turn/digest checks and external governed-response authorization. - Update the task feature and agent access maps. ## Verification - Real PostgreSQL regression: deferred review reply discovery failed before the production repair (expected one candidate; found zero). The final-message event and pending review were present. - Recovery cases cover one publication, no repeated provider run, unchanged review, and rejection of absent/wrong-turn final messages, bad digest and suppressed comments. - Red: seven new boundary regressions failed against the old receipt lookup. - Green: 169 focused oracle, event pagination and catalog checks passed. Product E2E typecheck passed after rebuilding local package declarations. - Retained-evidence replay confirms that the new lookup finds the real successful save event. The original attempt remains failed because its temporary file included an LF. - Recursive typecheck passed before the final recovery change; server typecheck passed afterward. The recovery and control-plane suites passed all 112 tests. Full build and map checks passed. - The broad local test run reached one unrelated 30-second timeout in `attention-retention-work.test.ts` (maintained-review/confirmation batching). The same unchanged case then passed on one targeted retry (39.9 seconds including module load). The original failure remains retained. The broad run also reported one unchanged workspace-conflict case, which passed on its targeted retry (14.85 seconds including module load). Both original failures remain retained. The redundant broad local run was stopped after current-head CI completed all required gates: 55 successful checks and 3 selection skips. No all-green local broad-run claim is made. - The separate artifact-transfer cell already passed on the recorded application revision: independent source and delivered bytes matched at 407,485 bytes, with a completed task, succeeded native run and cleared execution locks. This does not establish a three-turn continuity pass. - [Real native three-turn acceptance](https://github.com/paperclipai/paperclip/actions/runs/41499102994/job/125465795210) **passed** at `3b207a42943e9eeb371d4f2d5623de72093eb6d5`, suite definition 5. Turn one exercised `committed_task_response_recovered`. All three completed final replies were persisted once and in order. All three strict personal-file receipts passed. The unchanged 8 MiB binary survived; later transfers copied only 36 and 54 bytes. The lease was created, resumed, and resumed with the same runner PID/start time, runner instance, native session and provider session. Final task was `done`, all three runs succeeded, and checkout/execution locks were null. - The aggregate startup summary has no separate provider-process PID proof. The raw run records independently prove the stable runner process and provider session stated above. This single native cell does not qualify the legacy cell or a full provider matrix. - The passing campaign reused the signed immutable runner image. It required no disk expansion. Earlier campaign image builds used the recorded temporary 120 GiB mitigation; default 60 GiB build capacity remains unqualified. All scoped completed probes were removed and independently returned 404; org CPU/disk usage returned to zero. This does not resolve unknown historical resources under a different credential identity. - Current-head Greptile is 5/5; review-thread pagination is complete with zero threads. All 55 current-head checks are successful; 3 selection checks are skipped. The paid campaign, result enforcement and report publication all completed successfully. ## Risks - Publishing a reply must not approve the work. Recovery only copies the accepted turn’s completed final message. It preserves the pending review and does not start an executor. - Conversation and external governed-response paths retain their existing separate authorization. The latest save event can record failure after an earlier success. The oracle rejects that case. Event pagination, missing receipts and cross-run evidence also fail closed. The exact temporary-file byte count remains required. The review selector still requires the intended interaction ID and enabled action on its actual card. The production change is limited to ordinary internal-task reply recovery. It still requires an accepted result, a matching successful terminal disposition, the completed final message, and existing publication authority. ## Model Used OpenAI Codex, based on GPT-6, with shell and code tools. The exact deployment variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1102.0-canary.5 |
||
|
DottaandPaperclip
|
0fd82e386e |
feat(evals): stream completed chat catalog results to TestOps (#18647)
## Thinking Path > - Paperclip lets people manage AI agents and inspect their work. > - The chat catalog uses real Product E2E cases and reports their original grades to TestOps. > - The current report matrix waits for every selected worker. > - A large selection can hide completed results for hours. > - Each case can use the existing private reader and safe exporter after its own worker ends. > - This PR gives each case a separate upload and readback job, then verifies final coverage without another upload. > - People can inspect completed cases while the rest of the selection runs. ## Linked Issues or Issue Description Refs #18542, #18625. **What existing behavior does this improve?** The existing chat catalog reports results to TestOps only after all selected workers finish. **Current behavior** One slow case holds back the report and upload jobs for all other cases. **Proposed behavior** Each finished case gets its own protected reader, safe projection, upload and backend verification. The final job checks the whole selection through read-only requests. It does not upload results again. **Reason and benefit** A person can inspect a completed grade and its safe evidence receipts while other cases run. This changes future runs. It does not change an in-flight run. ## What Changed - Add one same-commit reusable workflow with separate worker, private-reader and TestOps-upload jobs. Each job uses only its own protected environment. Callers pass empty named secret placeholders and do not inherit secrets. - Reuse the original exporter and grade checks. Freeze each case's three Allure JSON files once. Allow known sibling results during per-case readback. Keep final exact selection coverage. - Submit each bundle at most once. Reconcile uncertain responses by readback. Retain only closed publication, failure and upload-intent receipts. A rerun attempt still fails the existing attempt-one admission guard. - Preserve failed original grades, failed cleanup and missing evidence. No missing original can become a passing result. - Keep external-input cases in the serial lane. Small native-only selections allow two cases at once. Small mixed selections allow one native and one external case. Larger selections retain eight native cases and one external case. - Add source, privacy and failure controls to the existing unit roster. Document the reporting flow and runner-group admission requirement. - Fix an existing corruption-test fixture that could write the same byte back to a random IV. Read and flip one bit instead. Keep the storage rejection and unpublished-plaintext assertions unchanged. ## Verification - 152 focused Node controls pass across the shared client, API, selection, CLI and streaming tests. The retained parent controls show the missing multipart and partial-readback behavior, plus the small external-selection error before its fix. - The focused workflow security control passes. Actionlint passes for both workflows. Changed JavaScript files pass syntax checks. Git diff checks pass. - Local controls use synthetic data and mocked HTTP. They make no provider call and do not start an app, database or native build. - Previous head `b51406df` passed PR CI (48 jobs pass and one is skipped). The review-driven test-only successor `0c65cc21` then failed one unchanged snapshot-storage corruption test, plus the final CI summary. The latest head includes its narrow fixture repair. Its first CI attempt separately hit runner shutdown/lost-communication failures, two setup/timeouts and a heartbeat-cancel cleanup HTTP 500/lock timeout; those observations are retained without claiming their underlying cause. One same-head failed-and-cancelled-only recovery now passes at `bf9c3e8f`: 48 jobs pass and one is skipped, including required `ci / verify` and `ci / e2e`. All 31 already-passing jobs retained their original execution records; all 15 recovered leaves and both dependent summaries passed. The first attempt remains recorded; the recovery does not establish the underlying causes of those failures. A small AWS run after delivery must prove that one case and both attachments are visible before another original case ends. That live proof is pending; local controls do not prove vendor ingestion or environment-secret resolution. - Retained earlier fixture-calibration failures remain recorded. The final integrated Node control run is green. A deterministic Node crypto/filesystem control also reproduces the no-op corruption mutation with a valid IV byte equal to `X`, then passes after flipping that byte. This control uses no app or database. The failing CI byte was not captured, so the exact historical byte remains unknown. ## Risks - The called workflow needs its exact master ref in the existing selected-workflow runner group. No IAM role change is needed. - Environment-only secrets must resolve in the called jobs. The caller passes no actual model or reporting key. The small live proof must confirm this behavior. - TestOps does not document an idempotency key for this upload route. An uncertain write is never resent automatically. If readback cannot verify it, the workflow fails with retained closed receipts. - The existing job run can already be marked finished by the selection watch. The live proof must record its actual stage and prove subsequent uploads are accepted. This PR does not reopen, stop or close that job run. - Backend ingestion does not prove that an already-open TestOps page refreshes automatically. - The corruption-test repair changes no production storage or cryptography. New-head AWS CI must qualify the existing database-backed owner test. - Each case adds short protected reader and upload jobs. The app source, original case clocks, graders, model, budgets, raw-evidence privacy boundary and zero eval retries stay unchanged. The versioned stream concurrency group serializes new streaming runs; an old immutable batch run can continue in its original group. ## Model Used OpenAI `gpt-6.1-sol`, with tool use and code execution. The session does not expose a context-window size. Independent source peers used the same model family. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
946735d03a |
Persist task sorting in URLs and show newest routine tasks first
Save task sort choices in URLs across task views. Open routine tasks by creation time, newest first, with stable sorting before pagination. Co-Authored-By: Paperclip <Paperclip-Paperclip@users.noreply.github.com> |
||
|
DottaandPaperclip
|
1f8f2e195f |
perf(ui): reuse parent metadata when loading the inbox (#18645)
## Thinking Path > - Paperclip helps people manage AI agents and their work. > - The Mine inbox combines task rows with approval and failure items. > - It waits for parent captions before it reveals the initial list. > - It fetched rich diagnostics for every parent, including parents already in the task index. > - Parent captions only use fields that the index already provides. > - This change reuses loaded parents and fetches missing parent metadata through the index view. > - Failure items keep their rich interruption data and the initial reveal boundary stays in place. ## Linked Issues or Issue Description **What existing behavior does this improve?** The initial load of Mine and other inbox views that need parent task captions. **Subsystem affected** `ui/` collection reads. The API contract and database schema stay the same. **Current behavior** The inbox fetches all parent and failed-run task IDs with `view=rows` after the primary task request. This adds active-run, blocker, review, interruption and recovery work to the reveal path. The observed hosted case fetched four parents. One was already in the 31-row task index. The other three were completed parents. The serial context response took 1.0–2.9 seconds during the diagnostic reloads. **Proposed behavior** Reuse parents from the selected source index. Fetch missing parents with `view=index`. Keep `view=rows` for every failed-run task, including overlaps with parents or source rows. Keep both role-specific ID lists in the query key. Drop old placeholder rows when the new index supplies a parent. **Reason and benefit** This removes unnecessary diagnostic work from a required serial request. Five alternating same-four-parent requests on the hosted app had index times of 189, 162, 156, 128 and 169 ms, versus rich-row times of 282, 211, 329, 219 and 284 ms. All requests missed the response cache. All consumed parent fields matched. This isolates the projection cost; it is not a deployed UI speedup claim. **Breaking changes** None. Membership, ordering, nesting, archive state, authorization and failure presentation retain their owners and contracts. Refs: #18323 and #18492 for the prior collection and startup work. This change addresses the remaining parent-context read. ## What Changed - Partition missing parent metadata from failed-run context. Load the two batches concurrently, with at most two batch requests in flight. - Keep richer failure rows last in the context result. Filter placeholders by company and current requested IDs. Track completed failure reads, including missing tasks, so a parent placeholder cannot reveal an unverified failure card. Keep existing verified cards mounted through refresh. - Cover delayed captions, reuse of indexed parents, refreshed parent captions, failure overlaps, resolved failures and parent-to-failure cache transitions. - Add a real browser check for delayed parent metadata and stable geometry through deferred diagnostics. Give the test its own disposable company. - Add a reusable local timing harness and update the collection verification recipe. ## Verification - Baseline source: `0710c5684243a663c09ff907011ab88838173a02`. Final measured candidate: `c8c69b5af11966b4144707a3da21e840c73c6cca`. Production UI assets were built from the same checkout and dependencies, with only the Inbox module changed between variants. - Targeted regression failed on the old implementation and passed after the change. All 58 Inbox component tests pass, including existing archive/undo and order checks. - Full workspace typecheck passes. The final UI typecheck passes. Full build passes. Token gates, map checks and diff whitespace checks pass. - All 12 collection browser checks pass, including pagination and reading-position restoration. - The full CI workflow passes on `c8c69b5af11966b4144707a3da21e840c73c6cca`, including all server, chat, workspace, serialized, runner and E2E shards. [CI run](https://github.com/paperclipai/paperclip/actions/runs/41502803275). Greptile reports 5/5 on this commit with no unresolved threads. The separate Paperclip Review check is still queued. Its first attempt saved a text result but did not complete the GitHub assessment. Normal retry was rejected because that attempt already recorded a result. The current-head review request is queued. The review service is now unavailable, so this external gate remains open. - The duplicate local `pnpm test:run` was interrupted after the equivalent CI run passed. Its six chat shards passed 1,828 tests. The general-server phase was stopped. This local full-suite run is incomplete, not a passing result. - Local synthetic fixture: four parents and 30 children, with three completed parents archived. The fixture also has 30 archived child records. No agent or provider calls ran. Mac arm64, Node 24.22.2, Chromium, production assets, 1440×900, native CPU, loopback network, fresh HTTP/query caches for each cold sample. - Five alternating local pairs produced identical 31-row membership and order and no browser errors: | Journey | Baseline median (range), ms | Candidate median (range), ms | Median paired saving, ms | | --- | --- | --- | --- | | Cold first usable Mine rows | 310 (252–435) | 258 (250–261) | 50 | | Warm Recent → Mine | 60 (56–64) | 67 (61–71) | -7 | | Task → Back to Mine | 42 (40–47) | 43 (40–45) | 0 | The final cold result has a material control limitation. The median primary-request start was 259 ms for baseline and 210 ms for candidate. Much of the cold difference occurs before the changed request. The parent response medians were 19.3 and 13.5 ms, with fewer parent IDs requested. Do not attribute the full 52 ms difference between cold medians to this fix. Four cold pairs were faster and one was 6 ms slower. Warm Recent → Mine was about 7 ms slower at the median; Back was unchanged. These runs establish no warm-navigation gain. The earlier five-pair experiment showed the same startup confound and mixed warm/Back results. All samples remain in the local evidence. A hosted final-shape comparison timed out and was cancelled by closing its owned browser tab; it supplies no result. The candidate has not been deployed. Reproduce the contract and geometry check against an isolated production UI with `pnpm exec playwright test --config tests/perf/collections/playwright.config.ts parent-context.spec.ts`. Run `tests/perf/collections/measure-mine-context.ts` against baseline and candidate loopback production origins that share one disposable API, as documented in `tests/perf/collections/README.md`. Preserve upstream startup timings as controls. Required parent responses remain inside the reveal boundary. ## Risks - Explicit-ID `index` and `rows` requests use the same company, visibility and actor filters. They do not use the inbox archive filter. An archived parent is still returned by either projection. A deleted or unreadable parent is absent in both. A second index request for a failure-parent overlap would not restore missing caption data. - A failed-run task must never reuse parent-only metadata. The query key includes each projection role. Completed reads carry explicit failure-ID coverage. The tests cover overlaps, pending role changes, late heartbeat data, known absence, and transitions to empty views. - Old parent context must not override a fresh indexed parent. The current-ID placeholder filter and caption-refresh regression cover this case. - The hosted app still has unrelated server contention. This PR reduces parent request work. It does not claim to fix all hosted loading delay. Deployment acceptance should confirm the serving revision, compare matched Mine reloads and inspect errors and row stability. Revert this change if failure items or captions regress. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, tool use, browser diagnostics and code execution. The exact runtime model identifier and context-window size are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
5a07c30c1a |
fix(workspaces): reduce temporary disk use during remote capture (#18644)
## Thinking Path > - Paperclip manages AI agents and preserves their completed work. > - Native remote finalization must save workspace files before it releases the writer. > - Portable capture copied the working tree and Git history, repacked the history, and then wrote an archive. > - These temporary copies can exhaust a small sandbox after normal development work. > - A completed provider turn then fails finalization and file preservation repeats the same failure. > - This change streams checked working files and keeps only an independent Git copy in scratch. > - The save still rejects changed sources and incomplete output. ## Linked Issues or Issue Description Related: #18403 added portable remote workspace storage. #18172 handles disk pressure in Daytona directory transfer. This failure occurs earlier, while the portable archive helper creates its source archive. That earlier repair does not cover this path. **What happened?** A completed native run failed workspace finalization three times. Its file-preservation retries also failed. In a retained 10 GB Linux sandbox with about 1.5 GB free, the deployed helper failed during `git repack -a -d` with `Out of diskspace`. The working files used about 603 MB, and Git used about 444 MiB. **Expected behavior** Capture a coherent workspace when free space can hold one full archive plus an independent Git copy. Preserve working files, staged changes, unpushed commits, and existing recovery gates. **Steps to reproduce** 1. Use a disposable Linux sandbox with a Git repository and generated working files. 2. Leave enough free space for an archive and a Git copy, but not a second working tree and another repack. 3. Run the portable capture helper. The old helper fails while creating temporary data. 4. Run the fixed helper against the same source and free-space limit. Verify each archived file hash. The small regression uses real file serialization and a controlled filesystem capacity failure at archive creation. It failed with `ENOSPC` before the production edit and passes afterward. **Paperclip version or commit** The observed staging server served `42c64d4ec4b9b0898556fcdf6ba304c4285a7819`. The repair is based on `a92f616cd0`. **Deployment mode** Hosted staging. Linux x64, Node 25.9.0, Daytona sandbox. Local tests use a macOS source checkout. ## What Changed - Stream working files from checked file handles. Keep only normalized Git metadata in scratch. - Remove the second repack after a mirror clone that already uses `--dissociate` and `--no-hardlinks`. - Check each streamed file's hash, size, mode, and filesystem identity. Recheck the source inventory and Git state before returning a receipt. - Remove incomplete archives after failure or cancellation. Keep pre-existing archive files intact. - Add space-limit, concurrent-change, cancellation, and existing-file safety cases. Update execution semantics and workspace verification recipes. ## Verification - Red before the production change: the capacity regression failed with `ENOSPC`. Green afterward: 54 archive and transport tests pass. - The existing tests restore ordinary, linked, and borrowed Git repositories after their sources disappear. They retain staged, unstaged, and unpushed work. Credential and path guards also pass. - Live comparison on the same retained sandbox: the deployed helper failed after 19.4 seconds with Git disk exhaustion. The fixed helper completed in 35.3 seconds. It produced a 1,081,872,803-byte archive with 24,111 entries. An independent reader checked every archived file hash and the manifest digest. The source commit stayed unchanged. Diagnostic scratch was removed. - This live probe exercised the changed capture boundary. It did not deploy a server, upload a production snapshot, finalize the failed run, or replay its provider turn. Full task recovery still needs rollout and the normal export recovery path. - `node scripts/check-maps.mjs` and `git diff --check` pass. - Current-head CI passes: [PR workflow](https://github.com/paperclipai/paperclip/actions/runs/41499985180), commit `d42700f7f4b0bef14a5969a44c2da0b2513e0ed2`. Greptile rates the same head 5/5 with no actionable findings or unresolved threads. - The independent Ponytail advisory also rates this exact head 5/5 with no findings. Its structured check remains queued because the automatic request was bound to the earlier head. A [fresh current-head review request](https://github.com/paperclipai/paperclip/pull/18644#issuecomment-6585296011) was accepted and queued. Its predecessor has a successful workspace export but remains active in finalization, so the new review cannot start yet. This review-service publication gate must complete before claiming every check is green. - `pnpm -r typecheck` and `pnpm build` pass. The final server-only typecheck also passes after the test additions. - Supported-runtime verification on Node 24.11.0: all 54 archive/transport tests and all 46 existing workspace lifecycle tests pass. The lifecycle suite uses a disposable real Postgres database and local filesystem transport with encrypted snapshot storage. It checks save/restore after losing the original allocation, ownership, retention, and file continuity. This is separate from the live Daytona capture probe. - The first extra lifecycle attempt could not start its local Postgres binary because the installed Darwin package lacked library aliases. Local dependency setup was repaired before the passing run. No production or repository code changed for that setup. - The full local `pnpm test:run` on the host default Node 26.4.0 stops in chat-test collection with `No test found ... in lines ...`; it has no suite verdict. A clean retry repeats the failure. The unchanged `a92f616cd0` test wrapper reproduces it with the same installed dependencies, after correcting an initial baseline setup that lacked built package entries. This is a local test-runner limitation, not a full local pass. Current-head CI remains the full-suite gate. - One focused run timed out while deeply comparing 4 MB buffers under concurrent build load. Complete SHA-256 comparisons retain the content check without that assertion cost. The capacity case then passes in under one second, and the final 54-test run passes. The disk allowance and production code did not change. ## Risks - Streaming reads the live source instead of a copied working tree. File identity, content hashes, mode checks, the final inventory read, and Git signatures must reject concurrent changes. Regression tests exercise these checks. - Capture still needs space for the full archive and normalized Git metadata together. It cannot export from a completely full disk. - Archive framing and restore validation stay compatible. The change has no database migration or new execution authority. - The incident remains unrecovered until the fixed server performs normal workspace finalization. No deployment or merge is included in this PR. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository inspection, code editing, shell tools, browser diagnostics, and automated tests. The exact serving variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1102.0-canary.4 |
||
|
DottaandPaperclip
|
026fd9e345 |
fix(server): keep controller identity independent of Git probes (#18649)
Read the existing boot timestamp without synchronous Git diagnostics. Keep the voice-retention precision test clock fixed. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Chris KaubischandClaude Opus 5.5
|
b0f5451fdf |
fix(claude-local): skip the 2.1.291 copy hint in the setup-token prompt parser (#15550)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - AI connections let a user connect a Claude subscription for all
agents of an organization
> - In a Docker deployment the server runs `claude setup-token` in a PTY
and parses its output for the sign-in URL
(`server/src/services/local-ai-browser-login.ts`,
`packages/adapters/claude-local/src/server/setup-token-parse.ts`)
> - The Docker image installs `@anthropic-ai/claude-code@latest`, and
`claude` 2.1.291 adds a copy hint line between the URL and the code
prompt
> - The parser requires the prompt on the first non-blank line after the
URL, so it returns null and the UI waits without an error
> - This pull request lets the parser skip that exact hint line
> - The benefit is that Claude subscription sign-in works again with the
current CLI
## Linked Issues or Issue Description
No issue exists. I searched open and closed issues and pull requests for
"Preparing browser sign-in" and the setup-token parser.
**What happened?**
In the "Claude - Subscription" connection dialog, the UI shows
"Preparing browser sign-in…" and "Start sign-in again". It shows no
sign-in link and no error. "Start sign-in again" starts the same wait
again. After 300 s the run fails with "Sign-in ended before the account
was connected".
**Expected behavior**
The dialog shows the sign-in link and the code field a few seconds after
the start.
**Steps to reproduce**
1. Run the Docker image with `claude` 2.1.291 (`@latest` at the time of
writing).
2. Open the AI connection dialog and select "Claude - Subscription".
3. The dialog stays on "Preparing browser sign-in…".
**Paperclip version or commit**
`master` at
|
||
|
DottaandPaperclip
|
a92f616cd0 |
fix(db): attribute bounded captures to execution owners (#18634)
## Thinking Path > - Paperclip manages agent work through shared server and database services. > - High query volume can spend main-thread time on SQL construction and result decoding. > - The existing bounded response capture counts rows and bytes, but it cannot name the operation that executed a query. > - Bundled callers can be missing, and measuring every parsed JSON value adds work to the diagnostic. > - This pull request carries a semantic execution owner into each measured query and bounds response sizing. > - Operators can rank query volume by owner before choosing a performance fix. ## Linked Issues or Issue Description Refs #17504, #17522, #17537, #17564. **What existing behavior does this improve?** The opt-in `PAPERCLIP_DB_RESPONSE_CAPTURE_WINDOW` diagnostic. **Subsystem affected** Database diagnostics and existing server execution boundaries. **Current behavior** The capture groups by table, operation, and builder caller. It cannot distinguish execution owners that reuse a prepared query. Bundled deployments need source maps for caller locations. The capture visits every returned value and serializes parsed JSON again to estimate its size. **Proposed behavior** Version 2 captures use fixed semantic owners for HTTP requests, delivery queues, execution phases, and budget work. They count completed calls, returned rows, errors, and completion wall time. They sample successful results 1, 17, 33, and so on, with fixed cell and string/byte limits. Complex values remain explicitly unmeasured. Semantic ownership works without source maps. **Reason and benefit** A short census can identify which operations issue the most queries or return the most rows. It preserves each query's execution owner through pooled connections, concurrent operations, and prepared-query reuse. It reduces diagnostic work without claiming per-query CPU attribution. **Breaking changes** Opt-in capture output moves from `entry.v1`/`summary.v1` to `entry.v2`/`summary.v2`. Sampled primitive byte lower bounds replace all-result byte estimates. Consumers must use the new fields and must not extrapolate them into a total. There is no database migration or application API change. ## What Changed - Added `withDatabaseOperation(owner, action)` to the DB package. The server owns semantic labels. Queries retain execution ownership in their completion closure. - Added fixed owners at existing HTTP, delivery, budget, and execution timing boundaries. Delivery work replaces the context of the producer that woke it. - Added complete call/row/error counters and completion wall durations within the existing bounded window. Durations include waits and are not CPU measurements. - Sampled and bounded primitive-value sizing. Complex and oversized values remain unmeasured. Known owners skip repeated stack extraction. - Preserved the deadline, group limit, paced output, query outcomes, and existing coverage exclusions. Updated the operator documentation. - Added real PostgreSQL owner-isolation and prepared-query tests, server scope tests, and plain bundled-runtime proof. - Moved projection-test and benchmark full-response sizing into an explicit test helper. Existing payload-size thresholds remain unchanged; these synthetic checks do not use the production sampler. ## Verification - Final-commit DB capture suites: 43 tests passed in 13.86 seconds against the actual PostgreSQL driver, including concurrent owners, prepared reuse, transaction failure, a dedicated client, sampling, and a query crossing the deadline. DB typecheck passed. - Server attribution and existing delivery/execution/budget suites: 53 tests passed. Final concurrent HTTP GET/JSON POST scope refinement: 5 tests passed. - Runtime bundle suite: 8 existing tests passed. The new no-source-map owner test passed after its fixture was corrected to use a driver thenable. An ordinary native Promise can bypass an overridden `.then`; the real PostgreSQL tests separately cover driver integration. - Observer-only experiment: three alternating pairs per case on Node 24.22.2, macOS arm64, 3,000 calls per run after warm-up. For an already-materialized synthetic JSON result, median capture CPU fell from 82.37 to 1.87 microseconds/call. For a 100-row primitive result, it fell from 8.10 to 4.40 microseconds/call. Every result retained identity and all calls were counted. The new diagnostic intentionally measures less byte data; these are observer costs, not database latency or a hosted application speedup. Disabled-case timings were below one microsecond/call and too small for a performance conclusion. - Final lifecycle regression: 28 unit tests and DB typecheck passed. Closing the last capture disables and replaces owner storage; overlapping captures retain it. - Final-commit full recursive typecheck passed in 38.38 seconds and full build passed in 40.81 seconds. The build stamp matches `83ed4ba9cebd68d48929c46f15ab32d5c0b9687b`. Focused server attribution tests also passed 5/5 before the test-helper-only migration. Full test acceptance will use the repository's parallel CI gates; the unsharded local command takes hours. - Verification failures retained: initial sandbox runs could not open local IPC/PostgreSQL listeners; permitted reruns exercised the real services. One old test expected JSON serialization and was updated to assert explicit unmeasured complex values. The first bundle fixture used the wrong promise boundary, as described above. A retained-callback regression also demonstrated that disabling and reusing the same AsyncLocalStorage instance could resurrect an old owner; replacing the instance after the last capture closes fixes that case. - The first CI run found older projection tests consuming the previous full-response fields. Those consumers now use the full-response test helper, preserving their byte thresholds. The two complete affected route suites plus helper tests passed 115/115 in 73.08 seconds. A temporary full-row query negative control then failed the unchanged 2,048-byte limit at 5,000,812 bytes and was restored; five other service suites passed 34/34 and the affected chat cases passed 3/3. All three previously failing CI shards passed on the corrected head. A local skill-list benchmark smoke check completed 255 captured queries with no in-flight queries or overflow; this was integration verification under concurrent test load, not a timing benchmark. - Two final-head runner jobs lost communication with GitHub before reporting a test result. One targeted retry of those jobs and their dependent aggregate passed. All 53 final-head checks passed, with 3 intentional skips; previously passing suites were preserved. - Hosted capture remains pending until the reviewed revision is deployed and a bounded window is explicitly configured. Reproduce the focused checks with Node 24 and the repository pnpm version: ```sh pnpm exec vitest run packages/db/src/response-capture.test.ts packages/db/src/response-capture-postgres.test.ts pnpm --dir server exec vitest run --config vitest.config.ts src/__tests__/database-operation-attribution.test.ts scripts/bundle-runtime.test.mjs pnpm -r typecheck pnpm build ``` ## Risks - Owner context is active while a valid future capture window is armed, up to 30 minutes before accounting starts. No-window and expired calls take the direct path. Diagnostics still add bounded work while enabled. - A nearest nested owner is reported. Broad `http.request` and unattributed groups remain visible; this does not identify every service callsite or SQL shape. - The existing census excludes tagged/raw explicit-execute/streaming paths and internal protocol commands. Its counts are not PostgreSQL transaction totals. - Completion wall time includes queue/database/network/scheduling waits. Concurrent durations overlap. Sampled bytes exclude complex values, large strings, and values beyond the sizing limits. - Activation is startup-only. Operators must choose the window after artifact preparation, allow rollout/warm-up time, and remove the expired setting afterward. ## Model Used OpenAI Codex, GPT-6 family, with code execution, repository inspection, and parallel agents. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
6aabcff7d1 |
fix(ui): add space above failed attempt notices (#18636)
Add token-based top padding outside failed-attempt notices and assert the full 16px gap in the browser check. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
5f0bb518b0 |
fix(runner): handle blank assigned-tool schema references (#18637)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Native agents discover assigned app tools through a governed
gateway.
> - OpenCode receives search and call tools instead of the full app
catalog.
> - A blank optional schema reference selected schema lookup instead of
search.
> - Agents received an unknown-tool error and could abandon valid app
access.
> - This pull request treats blank references as search and explains
invalid schema requests.
> - Agents can find their assigned tools while the gateway keeps its
access checks.
## Linked Issues or Issue Description
**What happened?**
An agent called `paperclip_search_assigned_tools` with
`{"query":"github","schemaTool":""}`. The server tried to find a tool
with an empty name. It returned `assigned_mcp_tool_unknown` instead of
search results. The agent then reported that its GitHub tools were
missing.
**Expected behavior**
A blank optional schema reference must permit normal tool discovery. A
bad schema reference must explain how to search again.
**Steps to reproduce**
1. Assign an app tool to a native agent that uses on-demand discovery.
2. Search with `{"query":"github","schemaTool":""}`.
3. Observe the unknown-tool error instead of the assigned tool.
**Paperclip version or commit**
Reproduced from source before this fix, on master at
|
||
|
DottaandPaperclip
|
60a99307b3 |
fix(server): settle cleanup after reusable sandbox destruction (#18633)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A web-server replacement must preserve work and let completed cleanup become idle. > - Several task turns can use one remote sandbox through recorded lease handoffs. > - The last owner can destroy that sandbox while an earlier owner still waits for its own termination receipt. > - A completed agent-file cleanup can also retain a retry deadline from a competing callback. > - This pull request resolves confirmed destruction through the existing handoff contract and settles obsolete cleanup retries. > - The benefit is correct cleanup recovery without deleting history or weakening sleep checks. ## Linked Issues or Issue Description Refs: #18554. Related work: #14695 handles warm agent-file ownership; #13291 handles interrupted conversation continuation. This PR fixes cleanup recovery after confirmed final destruction. It does not change provider process reuse or start a new run. **What happened?** After ten qualified staging web-server replacements, a bounded database-idle check failed. Completed native runs still owned pending agent-file cleanup. Normal workspace archive confirmed final sandbox destruction. An earlier lease in the same recorded handoff chain remained unresolved. Two completed cleanup receipts also kept stale retry deadlines. **Expected behavior** Resolve an earlier owner's cleanup only when the complete recorded handoff chain leads to an exact provider-confirmed destruction receipt. Clear a retry when its cleanup is already complete. Keep real or uncertain work pending. **Steps to reproduce** 1. Complete a remote run, reuse its sandbox for a second run, then archive the completed workspace. 2. Recover agent-file collection for the first run after the final lease confirms destruction. 3. Separately, reconstruct a completed cleanup receipt that still has a retry deadline. 4. Observe that both records remain unfinished without this fix. **Paperclip version or commit** Observed on `1a6017d6460267ad89d7e1784e2b521c28d8e9ac`. The changed source was unchanged in master `97c4dc969e13364464765ffbde74955005bf44d3`, which is this branch's base. **Deployment mode** A managed ephemeral web server with PostgreSQL, object storage and a remote sandbox. Regression tests use local PostgreSQL and no live provider or model calls. ## What Changed - Resolve remote termination at read time through the existing strict reusable-lease handoff validator. Require one final owner and a valid destruction receipt. Preserve historical lease rows. - Use the same proof in agent-directory cleanup and remote-stop inspection. Keep stopped, ambiguous, broken and cross-company chains blocked. - Reconcile completed cleanup receipts with stale retry deadlines through the existing deadline worker. Compare the receipt before clearing a deadline. - Add recovery, refusal, restart quietness and concurrent-obligation regressions. - Document the cleanup contract and preserve deployment evidence, failed checks and remaining acceptance work in the existing plan. ## Verification - Both owning service suites passed 169 tests before the final receipt comparison was added. - The seven-case selection passed again after the release-race review fix: `pnpm exec vitest run server/src/__tests__/agent-directory-working-copies.test.ts -t 'stale retry|destroyed successor chain|stale-deadline settlement'`. - Repository-wide `pnpm -r typecheck` and `pnpm build` passed on final review fix `7935d99bed`. The full `pnpm test:run` is still running. Latest-head CI has 55 successful checks, three skips, and one neutral review result. Greptile is 5/5. The Paperclip reviewer’s race finding is fixed; its optional query-deduplication suggestion is documented and the fresh lock-boundary checks are retained. - The reviewer identified an early return after a lost receipt comparison. The strengthened race regression failed before the fix and passed afterward: release now acts on the returned authoritative row, completes the new cleanup obligation, and preserves canonical files. - The first focused attempt failed the new stale-deadline test because the public release guard skipped settlement. That guard was fixed. The original failure is retained. A sandboxed typecheck could not fetch pnpm signature metadata; the normal network-enabled command passed without changing verification policy. - Prior deployed evidence: ten Railway replacements, 12,521 reads, 1,566 verified saves, 34 matching file hashes, no HTTP failures, actual browser reconnects of 1.16–1.29 seconds, and one remote execution retained across replacement. Two harness false alarms were separately adjudicated without redeploying those steps. - That deployment evidence predates this PR. Current-image natural idle, automatic sleep/wake browser return and the isolated production canary remain unverified. Production has not changed. ## Risks - Cleanup authority must not cross a company or allocation boundary. The existing validator checks the entire recorded chain; stopped successors and unexpected owners remain unresolved. - Confirmed destruction permits no-save cleanup. It does not claim that uncollected agent files were saved. - Read-time resolution adds a scoped lease-history query only for expired reusable leases without direct termination proof. No new recurring worker or polling cadence is added. - The deadline update must not erase new work. It requires the same cleanup receipt and preserves concurrent changes. - No schema or public API changes are included. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository inspection, code execution and test tools. The precise backend revision and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0710c56842 |
fix: retry saved chat input after a successful run (#18621)
## Thinking Path > - Paperclip manages AI agents and the work they receive. > - Connected conversations save incoming requests before they can run. > - A request can remain queued when the previous turn finishes successfully. > - Task Retry sent that case through the failed-run retry guard, which rejected the successful run. > - This change resumes the saved request through its existing admission owner. > - The operator can continue saved work while preserving the prior result and current permissions. ## Linked Issues or Issue Description **What happened?** Task Retry returned a conflict for a restricted connected conversation when its latest run succeeded and newer input remained saved. A GitHub review can reach this state when a new PR head arrives during the previous review. The prior review cannot publish against the new head, but its completed turn is not a failed run that can be replayed. **Expected behavior** Retry should resume the saved request with its original source identity and current authorization. It must preserve the previous successful result and must not replay that turn. **Steps to reproduce** 1. Start a review through a restricted GitHub endpoint. 2. Send a PR synchronize event while that review runs and retain its deferred input receipt. 3. Let the original run finish successfully with the task still open. 4. Click Retry. Before this change, the failed-run guard rejects the request. With this change, the saved input creates one successor through normal admission. Related work: #17725, #16965, #15033. ## What Changed - Select the newest saved input by its canonical chat action and wake receipt. - Store its receipt ID in the task retry intent and resume it through the existing chat owner. - Preserve caller permissions, source authorization, task state, agent state, cleanup waits, and duplicate-click handling. - Exclude saved-input intents from generic board-message replay. Keep board messages in separate queued receipts and allow independently saved input after successful board turns. - Add signed webhook, HTTP endpoint, accepted-result preservation, and refusal tests. Update execution semantics and capability maps. Stabilize the existing Teams lifecycle test with explicit database result ordering. ## Verification - Current-head [CI passes](https://github.com/paperclipai/paperclip/actions/runs/41482727909): 54 successful checks, one neutral review, three expected skips, and a successful Snyk status. Greptile gives commit `53f02e9105` 5/5 with no unresolved review threads. - Reproduced the original HTTP 409 before the fix. Reproduced both review findings with three failing cases before fixing them. - Passed 25 focused and related GitHub retry cases, 52 ordinary queue tests, and three native startup cases against disposable PostgreSQL. Earlier related verification also passed 83 GitHub retry cases and 109 recovery/receipt cases. - Passed `pnpm -r typecheck`, `pnpm build`, `node scripts/check-maps.mjs`, and the module boundary check. The final server typecheck passed after the review fixes. - The full local `pnpm test:run` attempt stopped on an existing Slack queue deadline assertion with a 1 ms database/host clock difference. That case passed in isolation. A second attempt was stopped when review changes superseded its source. Full local completion is not claimed; all chat, server, browser, build, and runner CI jobs passed on the final commit. - The runner CI job needed one rerun after a two-second startup timing assertion failed. That case and its two related cases passed locally; the rerun passed without code changes. - Live GitHub review publication with this patch is not verified. The signed provider fixture uses mocked GitHub responses. This branch is not deployed. ## Risks - This changes how Retry dispatches deferred external input after a successful run. It relies on the existing inbound owner to recheck source authority before execution. - Tests cover paused connections and agents, changed source bindings, completed tasks, pending cleanup, duplicate clicks, and preservation of the accepted result. - No schema migration or public API shape change is required. ## Model Used OpenAI GPT-6 through Codex, with reasoning, code execution, and browser tools. The exact deployed model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d26628e35a |
test(server): stop voice report waits from crossing refresh deadlines (#18628)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Voice calls retain reports that background work refreshes for a finite period. > - Tests must check backlog processing without starting later refreshes. > - The backlog test advances fake time while real database work is still running. > - On a slow runner, this starts refreshes and breaks the expected request count. > - This pull request advances time only to scheduled passes before the first refresh. > - The test can check each report once and still prove that the worker becomes quiet. ## Linked Issues or Issue Description Refs #18607. This is a separate test failure from that PR's task-list avatar changes. The test was added in #17586 and is unchanged by #18607. **What happened?** [Server shard 5/12](https://github.com/paperclipai/paperclip/actions/runs/41472540522/job/125369899753) failed in `voice-call-reports.test.ts`, in "bounds report reconstruction and resets a partial scan when the same owner restarts". The test expected zero active work but found two active holds. Its logs contain 170 requests for 105 sessions, including 65 second attempts. The final `ci / verify` failure follows from this failed shard. **Expected behavior** Process all 105 reports once. Wait for each pass to settle before advancing to another scheduled pass. Keep the clock before the first 60-second refresh deadline. **Steps to reproduce** 1. Run the named test against a disposable PostgreSQL database. 2. Temporarily delay the first provider response by 3,250 ms with `node:timers/promises`. 3. Use the same 15-second wait budget for the old and new versions. 4. The old polling makes 315 requests across three attempts per session and fails. The new polling makes 105 requests, one per session, and passes. **Paperclip version or commit** Observed at PR head `7f1d137a53489e2a45ba05e506b5ba17b51d5882`. **Deployment mode** GitHub Actions. Local verification uses disposable PostgreSQL 17. ## What Changed - Keep fake time fixed while a report pass waits on database work. - Advance to the coordinator's next scheduled pass only before the first report refresh. - Disable implicit fake-time advancement in the assertion polling. - Allow 15 seconds for the 105-report wait and 30 seconds for the full fixture test. - Retain all assertions for bounded reconstruction, restart, exact session IDs, one attempt per report, and no later database or provider activity. ## Verification - Controlled comparison with the same injected delay and 15-second wait: old polling fails with 315 requests; fixed polling passes with 105 requests. The temporary delay is removed from this change. - All 52 tests pass across `voice-call-reports.test.ts`, `scoped-deadline-work.test.ts`, and `delivery-work-coordinator.test.ts`. - `git diff --check` passes. - The initial embedded PostgreSQL run could not start because its macOS package lacks a `libzstd.1.dylib` alias. The passing tests use a separate disposable PostgreSQL 17 cluster. - The full 15-test voice-report suite passes again after rebasing onto current master. - `pnpm -r typecheck` passes. - `pnpm test:run` was started, then stopped during chat-suite startup after no tests completed in that invocation. It has no complete result. Its cause is not established. - `pnpm build` passes. - Greptile gives 5/5 with no inline findings on `caa6db568deb9df98437d1a532cc345e9f5b16dc`. Its check completed successfully for that exact commit. - The first CI attempt passed all 12 server shards, including the voice-report regression. Three other jobs were interrupted: Runner Vitest 2/2 received a runner shutdown signal; Typecheck + Release Registry and E2E shard 5/8 have GitHub annotations reporting lost runner communication. These are incomplete checks, not passing retries. [First attempt](https://github.com/paperclipai/paperclip/actions/runs/41488319181/attempts/1). - The single retry passed for all three interrupted jobs. The final `ci / e2e` and `ci / verify` gates passed. [Successful second attempt](https://github.com/paperclipai/paperclip/actions/runs/41488319181/attempts/2). - Final commit `caa6db568deb9df98437d1a532cc345e9f5b16dc`: all 56 reported checks are terminal and clean. The two optional Storybook checks are skipped. GitHub reports the PR as clean and mergeable. ## Risks - Low risk. The change affects one integration test. Production code and report refresh behavior are unchanged. - The test still has a finite wall-clock timeout. A database that cannot process the fixture within that budget will fail. ## Model Used OpenAI GPT-6 through Codex, with repository tools and code execution. The runtime does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
40b8075916 |
feat(ui): show scheduled task waits as Monitoring (#18624)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Task indicators tell an operator whether work is active, waiting, or
complete.
> - An agent can yield until a saved monitor checks CI or another
external result.
> - The task currently keeps the In Progress icon between those runs.
> - Changing the stored status would affect monitor admission and
recovery.
> - This pull request adds a read-only Monitoring display state with a
blue clock.
> - Operators can see and filter scheduled waits while execution keeps
its existing rules.
## Linked Issues or Issue Description
**What existing behavior does this improve?**
Task status indicators, task collection filters, board columns, and
search facets.
**Current behavior**
A task that waits for a scheduled monitor shows the same In Progress
icon as active work.
**Proposed behavior**
Show Monitoring with a static blue clock when an agent-owned task has a
saved ordinary monitor and no active run, execution lock, review
participant, or active recovery action. Return to the normal task
indicator when the check starts. Preserve review, blocked, terminal, and
quota-recovery states.
**Reason and benefit**
Operators can distinguish scheduled waits from current agent work
without reading the task history.
**Breaking changes**
None to stored task statuses or execution rules. Task reads gain
optional displayStatus. Collections and search gain a separate read
filter. Existing status filters keep their lifecycle meaning.
Related historical work: #4396 covered a monitoring label exemption.
This change uses the saved monitor and a read projection.
## What Changed
- Add one SQL projection for task detail, compact lists, summaries,
relations, search results, and display-status counts.
- Show the blue clock in shared task icons and chips, task detail and
breadcrumbs, properties, references, list rows, recent navigation, and
agent task cards.
- Add Monitoring to task filtering, grouping, sorting, board columns,
and search facets. Prevent a drag from creating this derived state.
- Keep stored lifecycle statuses, monitor scheduling, checkout,
recovery, permissions, and agent status filters unchanged.
- Add database and UI regression coverage, full-page Storybook examples,
and product and access-map documentation.
## Verification
- Passed `pnpm -r typecheck`.
- Passed `pnpm build`.
- Passed `pnpm check:token-gates` and the map/catalog structural checks.
- Ran the new database suite on a disposable PostgreSQL 17 database. It
covers read surfaces, filter pagination/counts, queued/running checks,
rescheduling, completion, retained monitors, review, and recovery holds.
- Ran existing search, list, board, sidebar, task status, and filter
tests. Corrected board queries use displayStatus and eight columns.
- The full local `pnpm test:run` stopped in a chat integration shard: 12
failures (four timeouts and eight FIFO publication assertions). That run
spanned a rebase. A focused rerun on the current source passed all 21
selected cases without assertion or timeout changes. The original cause
is not established; this does not claim the full local suite passed.
Final-head required CI passed (ci / verify and ci / e2e, run
41488267209).
- Passed 82 tests for task API summaries, recovery precedence, search,
and optimistic updates; 54 tests for status cards, database reads, and
board moves; and 27 tests for blocker relations and recent-task cache
updates. One intermediate database setup hit its 20-second hook timeout;
the serial rerun passed.
- Greptile is 5/5 on
|
||
|
DottaandPaperclip
|
b9e94b85a6 |
feat(observability): trace Daytona lease acquisition and resume (#18626)
## Thinking Path > - Paperclip manages work performed by AI agents. > - The server prepares a remote execution environment before the agent can respond. > - OpenTelemetry traces show the time spent in each preparation phase. > - A retained Daytona sandbox can take minutes to resume from an archive. > - The provider worker previously exposed no child timings for this operation. > - This pull request adds bounded lease phase spans under the existing worker trace. > - Operators can separate provider startup from lookup, teardown waits, and workspace verification. ## Linked Issues or Issue Description **What existing behavior does this improve?** Daytona environment acquisition and resume tracing. A long worker invocation currently hides the operation that consumed its time. Related tracing foundation: #10999. No duplicate open PR was found. **Subsystem affected** The Daytona sandbox provider and the server's existing plugin span validation boundary. **Proposed behavior** Show lookup, start or recovery, teardown waits, workspace verification, and fresh acquisition setup as timed child spans. Group verification steps beneath one parent. Record the bounded provider state before start, including archived. **Reason and benefit** An operator can identify which step delayed the first agent response. SDK startup includes its internal readiness wait; this change does not claim to measure provider queue time separately. **Breaking changes** The opt-in trace schema gains fixed span names and one closed state attribute. Execution, authorization, resource ownership, deadlines, and retry policies keep their existing contracts. ## What Changed - Add lease phase spans through the existing invocation tracer. - Add the fixed span names and state values to host validation. Define the state key in the shared startup attribute contract. - Preserve the exact work result or error if tracing fails. Never repeat work through the tracing wrapper. - Test worker receipts for archived and warm resume, recovery, concurrency, failure, disabled tracing, and cleanup. - Document the waterfall and its attribution limits. ## Verification - Daytona plugin suite: 525 passed, 7 skipped. The expanded tracing suite separately passes all 12 cases. - Host span validation and instrumentation tests: 46 passed, 3 skipped. - Standalone Daytona TypeScript check passes. - Module boundary and whitespace checks pass. - Full recursive typecheck and full build pass. - The full local Vitest attempt atcanary/v2026.1102.0-canary.3 |
||
|
DottaandPaperclip
|
97c4dc969e |
test(evals): retire duplicate chat TestOps plans (#18625)
## Thinking Path > - Paperclip manages AI agents and preserves their work history. > - Product E2E cases use TestOps plans to select real AWS evaluations. > - The full chat catalog now includes the former smoke and completion-pilot cases. > - The two old bootstrap workflows can recreate their smaller plans after deletion. > - This pull request removes those workflows and stops their plan-creation functions before API access. > - The full-catalog workflow keeps its shared clients, evidence rules, case identities, and measured results. ## Linked Issues or Issue Description Refs #18542, #18516. The user requested removal of the duplicate smoke and completion-pilot plans and investigation of their recreation paths. Both old workflows accepted manual dispatch only. Their bootstrap operations created plans by name when the plans were absent. They had no schedules in their workflow definitions. The old workflow records were disabled before plan removal. An existing pilot execution was allowed to finish successfully. This change does not cancel executions or delete cases, results, launches, or private evidence. ## What Changed - Remove `chat-feature-testops.yml` and `chat-completion-testops.yml`. - Make the former singleton plan creator reject every call before API reads or mutation markers. - Reject pilot bootstrap before API access. Keep the full-catalog branch and its integration validation. - Remove tests that require the retired workflow files. Keep shared helper and full-catalog controls. Run the trusted-checkout boundary test against the active catalog workflow. - Replace old setup instructions with retirement guidance and the active full-catalog route. The master sync preserves the incoming typed launch-search fix from #18620, including both regression tests. The only conflict was the retired pilot documentation. Its search-format note remains as a retained-helper contract. ## Verification - Two new retirement controls fail against the parent code because it reaches an API or mutation marker. Both pass after the change. - Six owning Node suites after the master sync: 146 passed (144 before the sync). Shared contract: 22 passed. Catalog and retained export controls: 12 passed. Focused workflow and release controls: 7 passed. - Two modified API modules pass Node syntax checks. `git diff --check` passes. - The pure workflow-security file has the same four failures and 13 passes on both the untouched parent and this branch. Two assertions expect eight trusted bootstrap steps where master now has nine. Two lists omit the already-shipped catalog workflow. This change does not repair or relax those unrelated assertions. - A broader release-test invocation had 73 passes and 20 sandbox failures from blocked mock HTTP listeners or npm permissions. Its focused affected workflow control passes. A standard Vitest setup attempt stopped on a missing `acpx/runtime` bundle dependency before tests. No local app, database, provider, or native runtime was qualified. - Current-head PR CI passes: 48 successful jobs and one skipped job, including both required checks. Greptile is 5/5. Independent source review passes. No new model run is needed for this retirement. ## Risks - Existing integrations that point at either retired workflow will stop dispatching. This is the requested behavior; use the full catalog instead. - Historical evidence readers still use shared helpers. Those helpers remain, including the transport and official-context parser used by the active catalog. - Workflow removal is a repository change. TestOps plan deletion and disabling obsolete executors are separate service actions. Neither action should delete shared cases or measurements. ## Model Used OpenAI `gpt-6.1-sol` through Codex, with source editing, shell tools, and independent source review. The context-window size is not asserted. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
3f7811eef0 |
fix(runner): keep requests responsive during retired backup cleanup (#18619)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Remote native runs keep durable harness backups for continuation and
recovery.
> - Publishing a new backup retires earlier backup directories in the
server process.
> - The synchronous permission walk and recursive removal block
unrelated requests.
> - This pull request makes that cleanup asynchronous and keeps
publication waiting for completion.
> - The benefit is continued request progress while large retired
backups are removed.
## Linked Issues or Issue Description
**What happened?**
Publishing a backup with many small files blocks the Node event loop
during directory permission repair and removal. A local comparison with
20,000 retired files reproduced 1.37–1.95 second event-loop stalls. This
is a pause candidate. A short hosted profile has not established it as
the dominant source of sustained CPU use.
**Expected behavior**
Unrelated requests can progress during cleanup. Publication still waits
for complete removal, preserves rollback on a failed lease stamp, and
leaves live files and symlink targets unchanged.
**Steps to reproduce**
Run the benchmark in `doc/performance/native-backup-publication.md`. It
compiles the actual publication service from a pinned baseline and the
working tree, prepares immutable retired backups, and sends foreground
HTTP probes from a separate process.
**Paperclip version or commit**
Measured baseline: `774ddc72713d6aa7cb516691c71f1fc23bcfa5ff`. The
measured treatment source is committed in this branch. Current head:
`0a69cf2d43c2f9fcf032d96c1dceb56163bf21ba`. The runtime patch is
unchanged from the measured source; later commits only strengthen the
test and benchmark evidence paths.
**Deployment mode**
Local compiled service on macOS arm64 with Node v24.22.2. No hosted
rollout has occurred.
Related work: #18380 changed retention scheduling. #13676 included the
existing immutable-backup cleanup behavior. Neither change makes
publication cleanup asynchronous. The GitHub duplicate search also found
#11929, which concerns database dumps rather than native harness
backups. No matching open event-loop cleanup issue was found.
## What Changed
- Use asynchronous filesystem operations for retired-backup cleanup.
Visit directories serially and skip unnecessary metadata calls for file
entries.
- Await cleanup before publication proceeds, during rollback, after
successful rotation, and in executor finalization.
- Exercise immutable current and previous backups in the publication
test. Preserve the existing symlink-isolation and rollback tests. Add a
real separate-process HTTP probe that must complete while publication is
still active.
- Add a repeatable service benchmark and document its fixture, metrics,
and acceptance limits.
## Verification
Draft status: focused correctness tests, the local comparison, and
repository typecheck pass. The complete local test gate is incomplete.
The build passed. Complete CI, hosted acceptance, and final-head review
remain pending.
- Final focused Vitest run: 2 files, 4 tests passed in 649 ms with the
normal server configuration. The regression warms a separate Node
client, waits for its HTTP request during the stamp callback, queues the
response for the next event-loop turn, and then allows final
retired-tree removal to start. The async treatment serves that response
before publication completes. The original synchronous cleanup fails the
same final regression in a separate source fixture. That expected
control failure is retained.
- Syntax and `git diff --check` passed.
- `pnpm -r typecheck` passed on head `332c57b88c` in 152.18 seconds,
including the native Runner build. Validation uses Node v24.22.2 and the
repository-pinned pnpm v9.15.4 with isolated dependencies.
- `pnpm test:run` was intentionally stopped after 1,390.24 seconds (exit
143). All six chat shards passed, covering all 1,817 parameterized cases
across their exact partitions. The general server phase was advancing
but incomplete. The repository timing manifest estimates another roughly
two hours for the serial remainder, so validation moved to final-head
parallel CI. This is not a full local test pass. The command began on
`332c57b88c`; later changes only refine the regression and benchmark,
and the final regression has its own focused pass.
- Benchmark failure/metadata checks pass in a tiny isolated Git fixture:
an injected worker exit 17 is reported in 0.44 seconds without hanging,
and the result includes a staged source change when the unstaged diff is
empty.
- `pnpm build` passed in 41.80 seconds on `9f7e4e669b`. The final commit
only releases a pending test probe on failure; runtime and build sources
are unchanged. An injected client exit verifies that final test failure
path exits promptly in 0.97 seconds instead of hanging.
- Three alternating baseline/treatment pairs used fresh fixtures with
100 directories × 100 files in each of two retired backups. Each
publication removed 20,000 files. A separate process sent HTTP probes at
fixed 5 ms intervals, including overdue arrivals.
- Every run verified the replacement manifest, complete cleanup, and
unchanged symlink-target contents and mode. All 2,395 HTTP requests
completed without errors. The largest measured generator delay was 8.8
ms.
| Metric | Baseline median (range), ms | Treatment median (range), ms |
| --- | ---: | ---: |
| Foreground request p95 | 1,357.7 (1,287.4–2,014.0) | 1.8 (1.7–25.9) |
| Maximum event-loop delay | 1,439.7 (1,371.5–1,954.5) | 19.9
(17.4–40.7) |
| Publication completion | 1,429.1 (1,360.2–1,952.9) | 1,439.5
(897.9–1,649.6) |
| Process CPU during publication | 1,194.1 (1,041.8–1,225.5) | 2,423.1
(1,995.7–2,514.7) |
The treatment served 180–329 requests while publication was active. The
baseline served zero during that interval. Process CPU includes
foreground work that the baseline cannot serve until cleanup completes.
The result supports responsiveness, with a CPU tradeoff. It does not
establish lower total CPU or a hosted latency gain.
Reproduce with Node v24.22.2:
```sh
pnpm --filter @paperclipai/server exec vitest run src/services/native-runtime/native-harness-backup-cleanup.test.ts src/services/native-runtime/native-harness-backup-publication.test.ts
node server/scripts/benchmark-native-backup-publication.mjs --baseline-ref
|
||
|
AlexFrontegg
|
8424a51018 |
fix(redaction): scan long letter/digit runs in linear time (#16999)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Paperclip removes secrets from run logs, tool results and diagnostics with `redactSensitiveText` before it stores or shows them. > - The JSON field-name patterns in that function start with an unanchored `[A-Za-z0-9_-]*`. Each start position inside a long run of letters and digits scans to the end of the run, so the cost grows with the square of the run length. > - Base64 of text is one long run of letters and digits. A 40 KB base64 blob of source code in a shell command takes about 5 s to redact, and a large Gmail RAW tool result does not finish. > - The redaction runs on the main event loop, so one such chunk blocks every request and every run on the instance. > - This pull request makes these scans start only at the start of a run. The scans become linear, and the redacted output does not change. > - The benefit is that large base64 payloads no longer freeze the server, and every secret that was redacted before is still redacted. ## Linked Issues or Issue Description Fixes #15393. The tool content guard runs `redactSensitiveText` on tool results. A Gmail RAW body is base64url of a MIME message, which is one long letter/digit run. `validateToolContent` on a Gmail-shaped result with a 68 KB `raw` field takes 59 s on master and 19 ms with this change; a 5.4 MB `raw` field takes 0.65 s. Thanks @ts-beat for the clear report in #15393. Refs #12309 (false positives in the same patterns; this pull request does not change which text matches). I searched issues and pull requests for "redaction", "redactSensitiveText", "quadratic", "backtracking", "hangs the server" and "100% CPU". #15393 reports the hang. No open pull request fixes it. **Subsystem affected:** secret redaction (`server/src/redaction.ts`, `packages/adapter-utils/src/command-redaction.ts`), used by run logs, tool content guards and run diagnostics. **Problem:** `compactRunLogChunk` redacts each run log chunk before it truncates the chunk. A shell command such as `printf '%s' '<base64>' | base64 -d > file.ts` with base64 of source text makes one long letter/digit run. Base64 of ASCII text contains `+` or `/` only where every third byte is `>`, `?`, `~` or DEL, so random base64 does not show the problem. One `redactSensitiveText` call on such a tool-call line takes (local measurement): | Blob | Before | After | | --- | --- | --- | | 4 KB | 66 ms | 1.1 ms | | 8 KB | 272 ms | 0.5 ms | | 16 KB | 1.1 s | 1.0 ms | | 40 KB | 5.4 s | 2.5 ms | | 1 MB | not measured | 36 ms | The `\b`-anchored command patterns (`KEY=value`, JWT, `Authorization`) and the CLI option pattern have the same quadratic cost on dash-separated runs, for example base64url data. **Proposed solution:** start the unanchored field-name scans only where no name character precedes them (`(?<![A-Za-z0-9_-])`). A prefixed name such as `myApiKey`, `x_auth_token` or `GITHUB_TOKEN` still matches, because the scan from the start of the run absorbs the prefix. **Alternatives considered:** - Truncate or cap the input before redaction. Rejected: a cut can split a secret and leave part of it unredacted, and the scan stays quadratic below the cap. - Run the scan in a worker thread with a deadline, as #15393 suggests. That limits the damage, but the CPU cost stays, and a timeout must then drop or block the content. It is still useful as an additional defense. ## What Changed - `server/src/redaction.ts`: `JSON_SECRET_FIELD_TEXT_RE` and `ESCAPED_JSON_SECRET_FIELD_TEXT_RE` start a field name only where no name character precedes it. The lookbehind comes after the optional opening quote, so quoted keys behave as before. The `Authorization` word scan uses the same start instead of `\b`. - `packages/adapter-utils/src/command-redaction.ts`: the `KEY=value` pattern uses the same start. The JWT pattern starts only at the first word character of a run. A CLI option name can no longer span `--`, so each `--` starts its own short scan. - Redaction still runs before truncation. `compactRunLogChunk` does not change. Why the output does not change: when an old pattern could match from the middle of a run, it could also match from the start of that run, because the `[A-Za-z0-9_-]*` prefix absorbs the extra characters. The engine tries the start of the run first. So the old patterns never matched from the middle of a run, and the new start condition only skips positions that cannot produce a match. The old JWT pattern also always matched from the first word character of a run. A CLI option name always ends at the end of its run, so the match from the last `--` in the run ends at the same place. ## Screenshots No UI change. ## Verification - New tests: - prefixed secret field names in plain, single-quoted and escaped JSON are still redacted, also when the name is glued to the end of a long run; - a 48 KB base64-of-source tool call through `compactRunLogChunk` stays intact, the secret in the same command is redacted, and the call finishes within 1 s; - `redactSensitiveText` and `redactCommandText` handle 48-120 KB letter/digit, dash-separated, `--`-separated and base64url runs within 1 s, and redact the secret that follows each run; - `x--token`, `--x--token`, `a-b--api-key=`, `prefix-MY_API_KEY=` and `--<jwt>` are still redacted, and `my-token is here` is not changed. - Without the change, the timing tests fail: 9.3 s, 9.8 s, 32 s and 60 s against the 1 s budget. The correctness tests pass with and without the change. - Mutation check: I reverted each of the six pattern changes one at a time. Each revert makes at least one test fail. - Differential fuzzing: 1.1 million random inputs built from secret names, quotes, escaped quotes, dashes, dots, colons and JWT parts gave the same output from the old and new `redactSensitiveText`, `redactCommandText` and `redactDiagnosticText`. - `cd server && npx vitest run <files>`: all 33 server test files that use the redaction helpers, with embedded Postgres, 1730 passed. - `npx vitest run --project @paperclipai/adapter-utils packages/adapter-utils/src/command-redaction.test.ts`: 25 passed. - `npx vitest run --project @paperclipai/adapter-utils`, complete suite: 80 of 82 files passed in the parallel run. The 2 other files (`github-launcher-environment`, `sandbox-managed-runtime`) had 5 s timeouts in git-based tests and passed on rerun. ## Risks - Lookbehind assertions need ES2018. `redactCommandText` also runs in the UI bundle. Node and current browsers support lookbehind, and the server (`server/src/log-redaction.ts`) and the UI (`ui/src/components/task-chat/task-chat-attachments.ts`) already use it. - A change to a redaction pattern could leak a secret. The reasoning above, the unchanged existing tests and the differential fuzzing show the same output as before. - Not changed here, found while checking: `redactDiagnosticText` backtracks exponentially on an escaped JSON secret value made of many escaped backslashes with no closing quote, and `redactSensitiveText` redacts a value only up to its first quote of either kind (`{"apiKey":"abc\"tail"}` keeps `tail"`). Both behave the same before and after this pull request. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) by Anthropic, in Claude Code, with tool use (shell, code search, and test runs) and extended thinking. A human directed the design choices. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
0748b90062 |
fix(server): stop goal-less native runs before board task closure (#18616)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A task owns its active run and its tool access. > - A board user can close a task while that run is still working. > - The task update route stopped native runs only when they had an active session goal. > - Providers without goals kept running after the route cleared task ownership. > - This pull request stops those native runs before a board closure releases ownership. > - The benefit is an orderly stop instead of tool denials and failed finalization. ## Linked Issues or Issue Description **What happened?** A board user marked a task Done while its native OpenCode run was active. The route cleared its execution lock without stopping the provider. Later tool calls failed with `paperclip_runner_tool_binding_not_authorized`. The final wake fence then failed with `native_current_wake_comments_binding_changed`, leaving workspace finalization in retry. Cancelled also released ownership before its later Stop call. **Expected behavior** A board closure must confirm native Stop before it releases task ownership. A failed Stop must leave the task open. The provider does not need to support session goals. **Steps to reproduce** 1. Start a native task with a provider that has no active session goal. 2. Mark the task Done while its run is active. 3. Observe that the old route updates the task without cancelling the run. ## What Changed - Reuse the existing audited cancellation path before a board user marks an active native task Done or Cancelled. - Keep the existing 409 response when Stop is unconfirmed. - Record the board user in the Stop metadata. The existing acknowledged-Stop policy then leaves queued input and recovery quiet until a later explicit wake. - Inspect additional runs only when a task has an execution owner; tasks without a run lock keep their existing completion path. - Keep native semantic completion, legacy completion, and goal control behavior. - Add route regressions for stop ordering, failed Stop, and legacy completion. - Update the execution contract and task maps. ## Verification - Red before the fix: three regressions failed. Done had no Stop; Cancelled stopped after the update; an unconfirmed Stop allowed Done. - A second red proved that a Stop receipt without the route's operator metadata was not recognized by the real recovery policy. Both cases pass after the metadata repair. - Final local result on macOS arm64 / Node.js 26.4.0: **348 tests pass across eight suites**. These cover the route, native self-completion authority, cancellation, acknowledged Stop, recovery queues, dependency wakeups, and completion telemetry. - CI found seven sibling route failures from an unnecessary active-run lookup on tasks without execution ownership. Those cases failed locally before the correction and pass afterward. - Earlier concurrent local runs hit a module-load hook timeout and one watchdog socket hang-up. The isolated watchdog case and the complete final eight-suite run pass with a 90-second startup budget. No assertion was removed. - `node scripts/check-maps.mjs` and `git diff --check`: pass. - `pnpm build` and `pnpm -r typecheck`: pass. Server build and typecheck also pass after the Stop metadata change; the final server build includes the lookup correction. - `pnpm test:run`: attempted; the unchanged chat shard selector failed before executing its selected tests (`No test found ... in lines ...`). This is separate from the PostgreSQL startup failure. CI supplies full-suite verification. - PostgreSQL integration and live-provider acceptance did not run locally. Embedded PostgreSQL failed before app startup with `could not create semaphores: No space left on device`. No usable isolated local database runtime was available. CI supplies the PostgreSQL regression environment. The full live OpenRouter workflow remains unverified. - The first final-head CI pass hit a 15-second timeout in the unchanged `attention-retention-work.test.ts` blocker-history fixture. Cleanup then deadlocked and later inserts reused IDs. This service test does not call the changed route. The original log is [retained here](https://github.com/paperclipai/paperclip/actions/runs/41476191754/job/125384289220). The one allowed rerun also failed on the same commit: the `running` case exceeded 15 seconds again ([rerun log](https://github.com/paperclipai/paperclip/actions/runs/41476191754/job/125393915025)). The rerun otherwise passed 2,346 tests. No retention source or assertion was changed. This repeatable failure is the remaining CI gate; the PR is **not ready to merge**. The next step is to isolate that retention fixture/query in a usable PostgreSQL environment and repair its cause before rerunning CI. - Greptile reviewed final commit `b3cb41af2d911e9ced219e846cada3ecec835dbf` at **5/5**, with no unresolved review threads. Greptile is green. CI is blocked by the repeated retention test timeout described above. - CI native cancellation evidence: all 16 tests in `server/src/__tests__/native-cancellation-request.integration.test.ts` pass ([job log](https://github.com/paperclipai/paperclip/actions/runs/41476191754/job/125384289222)). - CI PostgreSQL wake-queue evidence: all 96 tests in `server/src/modules/wake-queue/adapters/postgres.test.ts` pass ([job log](https://github.com/paperclipai/paperclip/actions/runs/41476191754/job/125384289470)). ## Risks Board closure now waits for native Stop even when the provider has no goal. A failed or uncertain Stop keeps the task open. The change uses the existing cancellation and workspace-preservation path. It adds no retry loop, credential access, schema change, or publication authority. Direct service updates and agent semantic completion retain their existing paths. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, shell execution, and browser inspection. The exact serving snapshot ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
912cff9392 |
fix(ui): link chat message identities to agent overviews (#18615)
## Thinking Path > - Paperclip lets people manage and converse with agents. > - Chat replies identify their author with an avatar and name. > - Those identities currently offer no path to the responding agent's profile. > - The existing overview route already provides that context within the current company. > - This PR links the shared chat identity to that overview while preserving its layout and typography. > - People can inspect the responding agent by clicking either its name or avatar. ## Linked Issues or Issue Description Related: #17169 links the conversation's page header. This change covers the author identity above messages and active turns. **What existing behavior does this improve?** Navigation from chat message authors to their agent overview. **Subsystem affected** ui/ — shared task and standing-chat identity component. **Current behavior** An agent's name and avatar above its message are plain text and an image. **Proposed behavior** The name and avatar form one company-scoped link to the actual author's overview. Both retain their appearance, including no underline on hover. An identity without an agent ID remains plain text. **Reason and benefit** Readers can inspect the responding agent directly from the conversation. **Breaking changes** None; this adds navigation to an existing read surface. ## What Changed - Link the shared agent identity to its existing overview route when an ID is available. - Preserve the existing avatar, text, spacing, and non-underlined appearance. - Document this entry point and the desktop/mobile verification recipe. ## Verification - `pnpm -r typecheck` — passed. - `pnpm build` — passed. - Existing bubble, runner-turn, turn, task-thread, and AgentChat tests — 461 passed. - `pnpm check:token-gates`, `node scripts/check-maps.mjs`, and `git diff --check` — passed. - Chromium production `TaskChatThread` in `Layout` (mixed-author Storybook fixture): dark and light desktop identity screenshots are pixel-identical before/after and on hover; no underline, and the href points to the responding author in the same company. The preview is UI evidence, not backend/provider acceptance. The separate standing-chat preview had stale history/email fixtures, and the optional mobile preview placed the identity behind its composer; neither is claimed as acceptance evidence. - Added router integration tests click both name and avatar from a different agent's chat and assert the author's company-scoped overview; unknown IDs remain unlinked. All 30 bubble/identity tests pass. - Full UI project: 10,284 tests passed, one 5-second timeout in the unchanged design-sandbox snapshot test. Its isolated rerun passed all 12 tests; the initial timeout remains recorded, and its cause was not established. - Final UI typecheck passed after adding the tests. - `pnpm test:run` stopped before running tests: its unchanged server chat shard filters selected no tests. The full UI result and isolated rerun are recorded above. This is not a claim that the full local repository suite passed. - [Final-head CI run](https://github.com/paperclipai/paperclip/actions/runs/41475021544) passed, including build, typecheck, unit/integration suites, all browser shards, and both aggregate gates. Greptile is 5/5 with its review thread resolved. The separate Paperclip Review check is still queued, waiting for its assigned agent; no assessment has been published yet. ## Risks Low risk: navigation uses the existing company-aware router and explicit author ID. Unknown identities stay unlinked. The shared identity is used by standing chats, task replies, and runner headers. ## Model Used OpenAI GPT-6, running as Codex with reasoning, code editing, and tool execution. The exact served model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
a6afe3e230 |
Fix TestOps launch preflight search encoding (#18620)
## Thinking Path > - Paperclip manages AI agents that do real work. > - Product evals verify the saved outcome of that work. > - TestOps plans select and dispatch these evals. > - Operator launchers check a unique launch name before they request work. > - TestOps expects an encoded filter, but the launchers send plain name text. > - This change uses the vendor filter format in the shared TestOps API owner. > - The existing checks can then prevent duplicate launch requests. ## Linked Issues or Issue Description Refs #18516, #18608, and #18542. The completion pilot registered successfully after #18608. Its optional API launcher then stopped with HTTP 400 in [run 41477551241](https://github.com/paperclipai/paperclip/actions/runs/41477551241). It stopped before the launch mutation marker. No evaluation was dispatched by that request. The launch search API uses Base64 of UTF-8 JSON filters. The launchers sent plain name text. I confirmed the format in the deployed public [vendor filter codec](https://paperclip.testops.cloud/es-testops-CICoDnkU.js) and its launch UI. The encoded name filter displayed the existing pilot launch. The TestOps UI separately dispatched [the genuine three-case pilot](https://github.com/paperclipai/paperclip/actions/runs/41478906153); this PR does not create another run or claim that pilot passed. ## What Changed - Add the typed name-filter encoder to the existing TestOps API module. - Use it in the backlog and completion launch preflight requests. The full-catalog route uses the same completion adapter. - Preserve the exact launch name, nonce, selection, authorization, once-only markers, and no-resend behavior. - Add protocol controls through both real launch functions. Document the wire format. ## Verification - Red: both new consumer controls reproduce the HTTP 400 boundary with plain name text. The other 109 API controls pass. - Green: all 111 API controls pass after the fix. All 166 related API, pilot contract, and full-catalog controls pass with Node 24.22.2. - The first broad local invocation lacked the `yaml` dependency in the new worktree. Its failure log is retained. Reusing the existing installed dependencies allowed all 166 controls to run and pass without further source changes. - The vendor-format query displayed the existing uniquely named pilot launch in the authenticated TestOps UI. No launch POST or model call was used for this read-only verification. - Diff check passes. All required CI checks pass on exact head `b55bdc6753fa042e4c2465e7b9b366b8126b4730` ([run 41480643021](https://github.com/paperclipai/paperclip/actions/runs/41480643021)). Both current-head reviews are 5/5, with zero unresolved findings. No TypeScript, compiled product, workflow, grader, model, clock, retry, or evidence-boundary changes. ## Risks The vendor can change its search format. The format is kept in the existing provider-specific API module, and both launch consumers have protocol controls. Existing duplicate detection and uncertain-response refusal remain in place. The already-running UI pilot is independent of this fix. ## Model Used OpenAI GPT-6 through Codex assisted with reasoning, repository inspection, code execution, and native accessibility tool use. The host does not expose the exact model revision or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Nicky LeachandPaperclip
|
f8a888a072 |
feat(runner): define bounded startup lifetime contracts (#18490)
## Thinking Path > - Paperclip manages AI agents and their work. > - Native sessions and plugin workers have separate startup lifetimes. > - A caller timeout does not prove that a worker or an accepted host write has stopped. > - Shared startup code needs an exact deadline and an actual settlement contract. > - This PR adds those contracts and tests them before any production caller uses them. > - The dependent integration in #18405 will use the contracts with verified cleanup. > - This split lets reviewers assess the lifetime rules separately from runtime activation. ## Linked Issues or Issue Description Refs #18405. This PR extracts the foundation from that change. The integration will depend on this PR. **Problem or motivation** A bounded startup must distinguish the caller's wait from actual worker completion. A late worker reply or an accepted host request can outlive the original RPC promise. A shorter timeout must not discard that responsibility or authorize a replacement run. **Proposed solution** Add one Core execution context, optional Runner startup deadlines, and worker settlement and retirement APIs. Test the contracts with real child-process IPC. Keep production context creation, transport projection, capability negotiation, durable cleanup, and Stop/recovery integration in #18405. **Alternatives considered** One combined PR mixed the lifetime contract with its activation and cleanup policy. Activating a shorter deadline before the complete transport and cleanup path is unsafe. Splitting only the SDK declarations would not provide a useful behavioral review boundary. **Roadmap alignment** This supports the existing plugin, sandbox execution, and bounded recovery capabilities. It adds no new product workflow, endpoint, migration, provider or runtime dependency. ## What Changed - Add `NativeExecutionContext`. It retains the selected input, binds one transport target, rejects work after cancellation or expiry, and tracks actual worker settlement. Expiry alone does not complete pending work. - Add worker startup groups. A caller timeout retains the group's responsibility. Retirement revokes obsolete invocation scopes, waits for accepted host handlers, and retires only the verified group's bookkeeping. Each cleanup wait is bounded; an unfinished handler remains pending for a later retry. - Add an optional Runner startup signal and absolute deadline for session open and retained attachment. Clean up abort listeners after synchronous failures. Existing callers keep their default timeout path. - Expose the remaining confirmed controller-lease budget without a new query or renewal timer. - Add optional SDK context and preparation-retirement fields. No production bridge sends or advertises them in this slice. - Keep the required lease mock update with the new `remainingMs()` contract so this foundation typechecks independently. Production activation files, including the native executor and environment task bridge, match the current master base. No production caller constructs `NativeExecutionContext` or passes the new Runner startup option. This PR adds no durable cleanup producer and claims **no standalone TTFT improvement**. The dependent integration must enable the context, transport and verified cleanup together. ## Verification Initial split verification on `8ebc8a2f9c6c02e90f920356138b3742a4067b28`, based on `9a3f395737e7a92faf99971459979061d423531a`: - From `server`: `../node_modules/.bin/vitest run src/services/native-runtime/native-execution-context.test.ts src/services/native-runtime/native-execution-context-worker.test.ts src/services/native-runtime/native-execution-lease.test.ts` — **17 passed**. Includes real child IPC, caller timeout with late reply, worker death, accepted host-handler draining, obsolete cleanup invocation rejection, unrelated work preservation, context scope, expiry, and confirmed lease budget. - From `packages/paperclip-runner`: `../../node_modules/.bin/vitest run src/native-session-runtime.test.ts` — **188 passed**. Includes existing callers without the new option and new synchronous-failure/expired-deadline cases for fresh and retained sessions. - Direct TypeScript checks for `packages/plugins/sdk/tsconfig.json` and `packages/paperclip-runner/tsconfig.json` — passed. Built only their shared/eval TypeScript prerequisites. - From `server`: `../node_modules/.bin/tsc --noEmit` — passed. - Checked the partition against the reviewed combined candidate. The twelve selected files match exactly. The thirteenth file carries only the required test-mock update. The production executor, bridge and target wiring remain equal to the base. - `git diff --check`, added-line secret/private-data scan, and added-code-comment scan — passed. No generated files or run artifacts are committed. - The first old-base split passed its focused tests but failed server typechecking because the unchanged executor test mock lacked `remainingMs()`. The original failure is retained. Moving that one-line mock update into this foundation resolved the split dependency; the final current-master checks above pass. - Full local CI suites, Rust builds, deployments and provider calls were not run, as requested. GitHub CI will provide full regression coverage. Current-head GitHub CI, standard review and Apex are complete; exact acceptance is recorded below. Review follow-up `d7635f4d280dab757a3666703737bcda955c9361` fixes three reproduced lifetime edge cases. A retiring startup group rejects new calls and shares one retirement promise; failed bounded retirement remains retryable. Transport identity uses a fixed field tuple rather than object-property order. Finishing a context rejects pending lookup waits without aborting an already-started backend or pretending the worker completed. All three new regressions failed before the fix; the updated foundation selection passes **20 tests**, including actual IPC and failed-retirement retry. Server typechecking and added-line checks pass. These replace the earlier 17-case selection, rather than adding 20 unique cases to it. The original failures and passing reruns are retained. Apex follow-up `cb8860bab5ab9d6357a352e11f276be0d0cdb26d` keeps removed worker handles reachable while they own startup calls, accepted host handlers or retirement work. Drain/retirement includes active and retained handles; completion callbacks remove settled handles. Stop and replacement remove only the exact handle they observed. This adds no database/network operation, polling loop or generic shutdown wait. The SDK, Runner, context and transport bridge are unchanged. Seven additional IPC regressions failed on the prior manager and pass after the fix, covering removal/replacement, Stop-all, host-handler settlement, bounded retirement failure/retry, pruning, and replacement ownership. The updated foundation selection passes **27 tests**, with server typechecking passing; these supersede the earlier 20-case selection. A separate real-PostgreSQL/child-IPC characterization uses the integration's actual SDK host, storage registry and run-control path with the exact corrected manager bytes. Before the fix, cleanup released run ownership while an accepted host write was blocked, then the late write restored obsolete metadata. After the fix, ownership and cleanup remain pending until the write completes; cleanup then removes the stale metadata before releasing ownership. The original failure, corrected result and source snapshots are preserved. Final head `cb8860bab5ab9d6357a352e11f276be0d0cdb26d` passes GitHub CI (53 successful checks, 2 expected skips, Snyk success), standard Greptile **5/5** and fresh Apex **5/5**. All review threads are resolved and the PR is conflict-free. ## Risks The shared Runner helper now observes synchronous operation failures through its asynchronous cleanup path. The affected runtime file's 188 tests pass, but full regression coverage still depends on CI. The new lifetime APIs require honest settlement signals. A future caller must not treat a deadline, worker crash or Stop intent as verified retirement. Production activation remains dependent on the complete integration and its cleanup tests in #18405. This foundation introduces no shorter production startup budget or new cleanup policy by itself. The context's transport projection is process-local. It must not be persisted or used as a general authorization cache. No production caller supplies it in this PR. There is no database migration, new dependency, or deployment requirement for merging this foundation. ## Model Used OpenAI Codex, based on GPT-6, assisted with repository inspection, the split, tests and command execution. The exact serving model ID and context-window size were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal issue identifier - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes — this description documents the inactive contracts; operational activation documentation remains with #18405 - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green on `cb8860bab5` - [x] Current-head standard Greptile and Apex are 5/5 with no unresolved findings - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
412834a860 |
test: overlap long recovery and maintenance waits (#18544)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - CI verifies recovery, execution authority, and background work before changes merge. > - Diff selection saves work, but a selected file still runs its cases in sequence. > - The three longest files in a recent full master run each took over eight minutes. > - Many of these cases must observe real lease expiry or idle periods. > - This pull request gives independent cases separate fixtures so those periods can overlap. > - The benefit is a shorter test path with the same assertions and observation windows. ## Linked Issues or Issue Description Refs #18464. The latest measured successful master [Cloud readiness run](https://github.com/paperclipai/paperclip/actions/runs/41300933157) spent 581 seconds in issue recovery, 569 seconds in native restart recovery, and 512 seconds in maintenance server routes. File-level selection cannot divide those long serial files. Shortening their lease or idle windows would weaken the regressions they protect. ## What Changed - Split issue recovery into four files with separate database and worker fixtures. Keep all 18 cases. - Split native restart coverage into recovery, lease renewal, preparation, checkpoint cost, and disk-loss files. Keep all original 79 cases, including the existing container-qualified cases. The master update also retains its newly added recovery cases. - Close each native case's database clients after pending failure reports drain. The shortened file must not rely on idle timeouts to stay below PostgreSQL's connection cap. - Run the native Runner lane with two isolated fork workers. Register every new file in the lane and verify that the partition still includes every suite. - Give each maintenance case an owned database. Run at most two cases together with test-local assertion contexts. Keep both real 90-second quiet windows and all 17 cases. - Preserve serial execution when any explicit shared `PAPERCLIP_*_TEST_DATABASE_URL` is present. An admin URL that creates a separate database per fixture can run concurrently. - Refresh shard weights from the protected master run. Mark the initial split-file weights as conservative estimates until master measures them. - Anchor the attention-retention fake clock to PostgreSQL time. Its fixed clock started trailing real database timestamps after 12:00 UTC on November 1, causing an archive-deadline test to time out. Keep its deadline assertions and timeout unchanged. - Document the fixture and concurrency rules. Leave runner authorization and workflows unchanged. ## Verification - Final head `c9a4ff11b3ba81277341294c7721b34fb8705ede`: [full PR CI passed](https://github.com/paperclipai/paperclip/actions/runs/41477848686), including typecheck, build, all test shards, E2E, and the final verification gate. The rollup has 55 successful checks, one neutral review check, four expected skips, and no pending or failed checks. Greptile reviewed this exact head at 5/5 with no actionable findings. - November 2 master update: merge commit `c9a4ff11b3ba81277341294c7721b34fb8705ede` resolves the native recovery import conflict while retaining the new busy-row and remote continuity cases from master. All 38 test-runner checks and the three new busy-row cases passed locally. - Retention verification after the master update: the first local full run passed 52 cases and timed out at 15 seconds in the paged-backlog case. That case passed in isolation. The unchanged master suite passed all 53 cases (82.95 seconds), and a full PR rerun also passed all 53 (76.65 seconds). The initial timeout cause remains unproven; no timeout or assertion was weakened. - Previously verified head `fd288a8e8c88df54dcdca7b7e42366b019182782`: [full PR CI passed](https://github.com/paperclipai/paperclip/actions/runs/41312625439). There are 55 successful checks, one neutral review check, four expected skips, and no pending or failed checks. Greptile reviewed this head at 5/5 with no actionable findings. - Issue recovery: 18/18 passed before and after the split on the same local machine and PostgreSQL fixture path. Wall time changed from **630.6 seconds to 319.7 seconds** (49% less). All case titles match. - Maintenance also passed all 17 cases on AWS: structured file cost dropped from **511.8 to 282.6 seconds** across the linked master run and the first PR run. These are different CI runs, so the matched local comparison below is the stronger isolated measurement. - Maintenance: 17/17 passed before and after with the restored pinned dependencies. Wall time changed from **488.1 seconds to 317.2 seconds** (35% less). All 120 assertion lines remain. Both real 90-second windows also passed a final targeted run. - Native restart: 71 passed and eight existing container-only skips before and after the split. Wall time changed from **632.1 seconds to 346.8 seconds** (45% less). All 79 cases and 254 assertion lines remain. - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - Extra TypeScript checking of the moved recovery and maintenance tests passed. The standard server config excludes `src/__tests__`. A separate check of attention-retention fixtures reports existing type errors; the unchanged baseline has identical diagnostics, with none introduced by the clock fix. - `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs scripts/ci/test-selection*.test.mjs`: 61 passed. - `pnpm test:run`: stopped in the second chat partition: 335 passed and four failed (two GitHub retry-state assertions and two Teams timeouts), after 339 passed in the first partition. These chat tests are unchanged. All four pass in a focused run (19.1 seconds). The full-suite ordering/load cause remains unproven. All three CI chat jobs passed. The previously failing cases passed there in 1.3, 1.3, 2.4, and 2.2 seconds; see [the chat job log](https://github.com/paperclipai/paperclip/actions/runs/41309691650/job/124857580714). The attention-retention timeout repeated in CI and in an isolated local run. Its fake clock was fixed at November 1 noon, which had fallen behind real PostgreSQL timestamps. Anchoring it to the database clock passes all 53 cases; the failed case now takes 425 ms. - The native CI lane passed all 162 runnable cases with eight existing skips after client cleanup: **650.7 seconds on the earlier master run to 391.0 seconds on this PR**. - CI found native connection exhaustion after the long waits moved out of the main file. The same failure reproduced locally at a 100-connection cap. Per-case client cleanup fixes the shortened file at that cap, and all five native files now pass together at that cap (71 passed, eight existing skips). - Early attempts exposed a missed fixture reference and a missed reporting setup hook during extraction; both were restored. The first CI policy run also required moving the new native fixture under test helpers; that boundary check now passes locally. A separate auxiliary command picked up the host pnpm version and removed dependencies. The original maintenance baseline reproduced the same missing-package failures, so that failed attempt is excluded from timing comparisons. A later unchanged-baseline run passed all 17 cases. The pinned install was restored before the passing maintenance run. The audit also covered heartbeat process recovery, deferred receipts, chat, workspace recovery, attention, and connection management. Their shared fake clocks/provider mocks or existing partitioning need different changes. The 16,400-row attention case and the large checkpoint stream remain intact because their size protects real limits. ## Risks - Two isolated processes use more memory and database connections than one. Concurrency is capped at two. Shared database overrides force serial execution. - Moving setup into helpers can change test lifecycle order. The split suites keep the same case assertions and use independent process state; focused runs and the full CI suite check the move. - Split-file weights are estimates based on case timings from the linked run plus repeated setup cost. The next protected-master refresh replaces them. Weights never permit tests to be skipped. - Local timing varies with host load. CI measurements after merge are needed to establish the overall PR wall-time improvement. ## Model Used OpenAI Codex, GPT-6, with reasoning, repository inspection, tool use, and code execution. The exact backend revision and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (changed suites and focused reruns; the full local-run caveat is recorded above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
baa4a2e64e |
fix(runner): restore Codex auth before source recovery (#18612)
Restore disposable Codex authentication and safe provider configuration before cold source recovery. Preserve source receipts and conversation state until reconciliation, and rebuild assigned tools from the admitted launch binding. Validate remote launch destinations, clear stale authentication when no selected credential exists, and atomically publish SSH file transfers without following retained links. Red/green recovery regressions, a real two-turn Codex/MCP probe, and all current PR checks pass. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
42c64d4ec4 |
fix(runner): preserve deliberate OpenCode approval waits (#18610)
## Thinking Path > - Paperclip coordinates agent work and human approval. > - A saved Plan must wait for the user before work continues. > - The native runner interrupts the provider after it saves that wait. > - OpenCode can report the interruption with an idle event and no final answer. > - Empty-answer recovery treated that intentional pause as an incomplete answer. > - This change stops answer recovery for an interrupted turn. > - The runner still requires a real provider terminal before it settles the wait. ## Linked Issues or Issue Description **What happened?** OpenCode could send a corrective model request after Paperclip interrupted a turn for a saved approval request. The native runner then failed with `governed_wait_provider_terminal_unproven` because it did not receive the required terminal event within the existing drain window. **Expected behavior** An interrupted turn must stop requesting answers. A provider idle or cancelled event must remain available to the native wait settlement. An HTTP abort acknowledgement alone must not count as a terminal. **Steps to reproduce** 1. Start an OpenCode turn that has no final answer. 2. Interrupt that turn after saving a durable approval request. 3. Have the provider return `session.idle` or an idle `session.status`. 4. Observe that the old driver sends a second prompt instead of emitting the terminal event. The controlled regression fails before the fix for both idle event forms: it observes two prompts instead of one. The affected driver blob is identical in the deployed source and the current base. Related: #18364 added the empty-answer recovery path. ## What Changed - Record interruption intent for the active OpenCode turn before asynchronous settlement. - Cancel its answer-recovery timer and prevent corrective requests for that turn. - Preserve provider terminal checks and avoid a delayed abort of a later turn. - Cover both idle event forms, an existing correction, durable question handoff, in-flight tools, missing terminal evidence, and a later authorized turn. - Document the interruption contract. ## Verification - The new regression failed before the fix for the expected extra prompt. A second red test confirmed the sibling durable-question handoff also sent that unwanted prompt. - Interrupted idle events now produce explicit cancellation terminals. Tests reject a failed terminal and require normal completion on the later authorized turn. - All 305 targeted tests passed across the OpenCode driver, normal empty-answer recovery, interruption, backend publication, and native session runtime suites. - Runner TypeScript check passed: `tsc -p packages/paperclip-runner/tsconfig.json --noEmit`. - `git diff --check` passed. - Greptile reviewed the current commit at 5/5 with no findings. Both earlier Paperclip review findings are fixed and their threads are resolved. - The additional Paperclip advisory reviewer could not assess the final commit because its earlier workspace save did not finish. Its check is neutral; this is not a passing assessment. No reviewer configuration or workspace safety gate was changed. - Repository CI passed on run 41470761437, attempt 2. Attempt 1 retained a 15-second timeout in an unchanged voice-report test; only the failed jobs were retried once, on the same commit. The identical test also passed in an isolated baseline check against disposable PostgreSQL with its original timeout. - Live QA acceptance is pending. No eval prompt, model, allowance, timeout, or assertion changed. ## Risks The change affects OpenCode interruption and answer recovery. A silent provider must remain unproven; the negative control checks that case. Ordinary answer recovery remains enabled on authorized turns. No schema, permission, or provider configuration changes are required. ## Model Used OpenAI GPT-6 through Codex, with reasoning, code execution, and browser tools. The runtime does not expose a more specific model identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1102.0-canary.2 |
||
|
Nicky LeachandPaperclip
|
aab87ce707 |
Prove workspace delivery before inspecting cleanup candidates (#18493)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Completed tasks leave workspaces that need safe cleanup. > - Providers without ancestry inspection currently need a recorded merged PR before cleanup can start. > - A squash merge changes commit identity, so direct ancestry cannot prove that delivery. > - This change proves delivery from the finalized HEAD and captured destination before storage inspection. > - Fresh storage checks and a second delivery check still protect removal. > - This reuses the existing worker and receipts. It adds no table or timer. ## Linked Issues or Issue Description Refs #18410. Related: #13400 addresses workspace identity during inspection; this change adds destination-aware delivery discovery. ## What Changed - Resolve direct ancestry or discover an associated merged GitHub PR without a recorded work product. Require exact HEAD, destination repository and branch, and a reachable integration commit. - Use the latest settled finalization receipt as a candidate. Reject missing, failed, pending, conflicting, or changed evidence. - Inspect storage only after positive remote proof for providers that disable ancestry inspection. Recheck HEAD, branch, source identity and delivery before removal intent. - Add an optional provider capability for a specific PR recovery hint and an optional prepared destination receipt. Older providers keep their existing protocol. - Wake cleanup when finalization writes settle or gain a workspace binding. Keep remote-only retries for concrete unresolved delivery. Park missing or unsupported candidate identity. - Reuse the source project's explicit GitHub consent for delivery checks through the existing authorization service. Revalidate the bound repository and consent before removal; missing or revoked consent retains storage without switching identities. Public sources remain anonymous. - Update the contracts, documentation, access map, and targeted regressions. ## Verification - Initial-head GitHub CI passed at `5acd4c818be6f1764f8319f42891f9956c357c26`, including full build and typecheck. Standard Greptile is 5/5 with no open threads. The first Apex review scored 3/5 and found startup latency from remote proof lookups and removal retry identity changes across provider capability changes. Both corrections are pushed in `9d1a38df3d`, with the qualified destination-ref length fix. Red regressions reproduced each issue. Latest targeted checks passed: cleanup/worker 26, provider integration 41, SDK contract 8. Head `9d1a38df3d` passed standard and Apex reviews at 5/5 with no open threads, plus all CI including full build/typecheck. Canary Dry Run initially failed on an npm 404 fetching `rollup-4.67.3.tgz`; the exact tarball subsequently returned HTTP 200, and one targeted retry passed without source changes. Aggregate verification also passed. After #18403 merged, master introduced conflicts in shared workspace cleanup and metadata handling. Resolved them in `7700832536`, preserving remote snapshot cleanup and Git delivery-proof cleanup. Targeted merge verification: 93 tests passed in the first run (59 opt-in skips), then all 16 remote lifecycle tests passed with explicit database opt-in; map checks passed. The isolated database is stopped. The merge head received standard review 5/5. Its CI found two failures in unchanged test cleanup paths: native runtime temporary-directory ENOTEMPTY and agent-chat cancellation lock timeout. Those failures remain recorded and require successful final-head verification. The subsequent project-consent change has targeted coverage: authorization36, resolver33 and cleanup19 tests passed, including revoked consent, anonymous public access and source-binding races. The positive consent test failed before the fix. Current head `c772679e05` has green GitHub CI, including full build/typecheck and both previously failing test jobs, plus standard Greptile 5/5 with no open threads. Neither cleanup failure required changes to its test path. Apex also completed at 5/5 on this exact head (check `124651548532`, summary updated 2026-10-31 20:45 UTC), with no open review threads. The PR remains draft. Earlier failures remain recorded above. - Final focused run on the rebased branch: **182 tests passed across six files**, covering the resolver, delivery cleanup, provider bridge, finalization notifications, existing workspace service and SDK contracts. The disposable test database was stopped afterward. - Targeted resolver tests: 33 passed. These cover ancestry, discovered squash PRs, wrong HEAD/base/repository, removed integration commits, bounded responses, cancellation and credential isolation. - Targeted cleanup integration tests: 17 passed. Provider bridge integration: 36 passed. SDK contract tests: 7 passed. Finalization notification tests: 9 passed. - A negative control that removed the final delivery check failed the race regression as expected. The check was restored. - Read-only live GitHub probe: the new resolver discovered PR #18410 from HEAD `3f5ade4cd42b707040e9a82fdaed343e3d736a09`, without PR hints. It proved integration commit `6484f96c28bfb11a793ce4be6cc5ac15643a5a02` reachable from observed master `5bc029bf6c1db6647e96acea323f3723fd1d3eba`. GitHub also advertised the exact original HEAD at `refs/pull/18410/head`. - `node scripts/check-maps.mjs` passed. Structural checks do not prove deployment behavior. - Local tests are targeted by user request. Full build, typecheck and test suite run in GitHub CI. A standalone server typecheck could not complete because local Runner build declarations were absent; no changed-file diagnostics were reported. - Initial local execution hit a stalled pnpm wrapper, sandbox socket denial and stale generated SDK declarations. Direct installed binaries, the authorized isolated test database and rebuilt SDK resolved those setup failures. No deployment was performed. ## Risks - Destructive cleanup still requires fresh storage safety and lifecycle checks. Remote proof alone never authorizes removal. A persisted removal intent preserves existing recovery semantics. - Legacy workspaces without an unambiguous destination or completion HEAD remain retained. This does not guess the original default branch. - Private repositories use the explicit standing GitHub consent on the bound source project, with current membership, grant, repository access and trust checks. No separate company token is required when that consent is valid. Missing or revoked consent and unavailable or rate-limited access retain storage; the worker does not borrow a task or agent identity. - A known undelivered commit keeps the existing five-minute retry because external GitHub changes have no guaranteed ingress to this lane. This can keep cleanup scheduled. Missing evidence parks instead. - Providers need the new recovery capability to clean a squash merge whose original HEAD is reachable only through its PR ref. Providers without it remain conservative. - The resolver currently supports GitHub.com. Existing local and other-provider ancestry paths stay in place. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools and parallel code agents. The exact runtime model ID and context-window size were not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
9203ab52f0 |
fix(evals): bind complete unassigned TestOps plan buckets (#18608)
## Thinking Path > - Paperclip manages agents that do real work. > - Product evals check the saved outcome of that work. > - TestOps plans select the exact automated cases to run. > - A new plan can report its unassigned cases as a job-statistics bucket. > - The pilot registration guard rejects that bucket as a foreign job. > - This change binds only a complete, known unassigned bucket and records assignment diagnostics before writes. ## Linked Issues or Issue Description Refs #18516 and #18542. The metadata-only bootstrap in [41464514278](https://github.com/paperclipai/paperclip/actions/runs/41464514278) created the dedicated plan and verified its GitHub job definition, then stopped at `plan_job_mismatch` before job or case-assignment mutations. No model work ran. The retained receipt did not include the assignment rows. The live failure's exact shape remains unconfirmed. The API's job-statistics carrier permits an absent job. The old guard rejects a complete unassigned bucket. This PR covers that case and records the actual rows in the next closed inspection. ## What Changed - Accept one closed unassigned bucket with the exact selected-case count. Support an explicit null job or an omitted optional job field. - Assign the exact selected cases once. Keep the existing owned assignment without assigning it twice. - Reject foreign jobs, mixed buckets, wrong counts and unknown unassigned fields before mutations. - Retain only bounded assignment kinds, numeric job IDs and counts in the existing observation receipt. - Merge current master, including the full-catalog adapter from #18542. Retain its context-aware plan and job checks alongside the new pre-write assignment inspection. The helper consumes that context’s expected case count. - Document the inspection and binding rules. The helper takes the expected count from its owner. This does not import or run the full-catalog route from #18542. ## Verification - Red: both complete unassigned carriers fail on the old guard; the other 70 dispatch/API controls pass. - Green: all 72 dispatch/API controls passed before the master sync. After the sync, all 104 related pilot and full-catalog Node controls pass. They include ordinary registration, no duplicate assignment, foreign-job rejection, incomplete and unknown bucket rejection, launch fences, uncertain-response protection and result readback. - Diff check passes. No TypeScript or compiled product code changes. Full repository checks were not run locally. All required CI checks pass on current head `a8bec6418065a7faae28c8daab3efdfc9b184797` ([run 41473510972](https://github.com/paperclipai/paperclip/actions/runs/41473510972)). Both current-head reviews are 5/5 with no unresolved threads. Before the master sync, one unchanged server timing test failed on attempt one and passed on the only targeted rerun; that original failure remains in CI attempt one. No rerun was needed on the current head. - Live acceptance still requires another metadata-only inspection, followed by a genuine three-case TestOps launch. This PR does not claim a new passing measurement. ## Risks An overly broad unassigned rule could replace an existing job. The new rule permits only one known bucket with the exact selected-case count and no foreign job. All final binding checks still apply. Unknown assignment shapes stop before writes. Original case queries, model, clocks, graders, evidence rules and retries are unchanged. ## Model Used OpenAI GPT-6 through Codex assisted with repository inspection, code execution and tool use. The host does not expose the exact model revision or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id - [x] I have run the relevant local controls and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
71a20453e1 |
fix(workspaces): honor inherited remote workspace bindings (#18611)
## Thinking Path > - Paperclip manages AI agents and their work. > - Task creation can bind a child to its parent's execution workspace. > - Native sandbox runs reserve that workspace before the provider starts. > - Remote reservation rejected inherited isolated workspaces and operator branches because the original source task differed. > - This check stopped valid child tasks before their existing access and writer checks could run. > - This pull request uses the persisted binding and existing source-access checks to admit that continuation. > - The benefit is that authorized child tasks can continue saved work while the single-writer lock stays effective. ## Linked Issues or Issue Description **What happened?** A child inherited an active isolated remote workspace through normal task creation. Its next run failed during setup with `Workspace ownership changed.` No provider work started. The reservation guard treated the original `sourceIssueId` as exclusive ownership, although task creation had persisted a valid child binding. **Expected behavior** A bound child or follow-up can reserve the workspace after the prior writer saves. Current task/run access, source visibility, trust, company scope, closed-workspace state, and writer checks must still apply. The original source task remains provenance. **Steps to reproduce** 1. Create a task with an active saved remote isolated workspace or operator branch. 2. Create a child without an explicit workspace override. 3. Confirm that the child inherited the execution workspace ID. 4. Start its native sandbox run. The old reservation guard rejects it before provider execution. **Paperclip version or commit** Reproduced on source based on `06bc8db75c71142ed108e0f331394c5187015d6a`. The same guard is present at `1a6017d6460267ad89d7e1784e2b521c28d8e9ac`. Related: #17156 proposes separate workspaces for child tasks. This repair preserves the current inheritance contract and enforces the remote writer lock. #16911 addresses child bindings across repositories. ## What Changed - Remove the original-task equality restriction for active remote workspaces. Keep the existing persisted-binding and retained-source authorization path. Explicit task-root selection follows the existing V1 filesystem contract; task-assignment grants do not define filesystem permissions. - Keep closed workspace rejection and unfinished-writer checks. - Add database-backed regressions for inheritance, replay, competing children, an unbound task, private source data, company isolation, and closed records. - Extend real save/restore tests for inherited isolated workspaces and operator branches. Keep shared-workspace discovery coverage. - Update the development guide and workspace maps. ## Verification - Red: before the repair, the inherited isolated/operator cases fail with `Workspace ownership changed.` The shared case passes. The unfixed suite has five failures and four passes. - Green: 11 reservation cases pass with real PostgreSQL 18 in an isolated Docker container. - An earlier run of the four affected workspace/preparation/trust suites passed 191 tests. The review follow-up passes all 15 selected reservation/save/restore cases. Inherited cases remove the original files, use a distinct allocation and target, restore the saved file, append child work, and independently read that work from the child's new snapshot. Shared-source discovery stays covered. The filter excludes 42 other integration cases. - `pnpm -r typecheck`, `pnpm build`, `node scripts/check-maps.mjs`, and `git diff --check` pass. A direct server typecheck also passes after the final test edit. - `pnpm test:run` stopped in its first chat shard: 338 passed, one Discord receiver-lease setup failure, 1,478 outside the shard. The failing test passes in isolation on both the unchanged baseline and the fix. Its suite-order failure is not explained by that retry and remains recorded. The unchanged baseline also fails three other Discord publication cases in that chat shard, while its next chat shard passes. This does not reproduce or explain the original lease failure; both results remain recorded. CI is complete on `e697f6e1b8`: 54 successful check runs, one successful legacy status, three skipped checks, and one neutral check. No check is pending or failed. Greptile gives this commit 5/5 with no remaining findings. The security reviewer accepted the V1-contract explanation; both review threads are resolved. - Live sandbox acceptance remains unverified. The available test credentials could not resolve the required sandbox snapshot. The provider rejected creation before workspace execution, and a scoped follow-up found no allocation. No live model turn, deployment, or original-task recovery is claimed. - Local environment: macOS arm64, Node 26.4.0, pnpm 9.15.4. Native PostgreSQL startup hit the host semaphore limit; tests use disposable Docker databases. No shared database was reset. ## Risks - A bound child can now write after the prior writer releases the workspace. Incorrect binding or access checks could expose source data. Regressions cover unbound, private-source, and foreign-company rejection. - Concurrent children must not become two writers. A real transaction race grants exactly one reservation; saved-file tests keep the second writer blocked until save completes. - No schema or migration changes. This PR does not change task-creation inheritance, requested models, credentials, trust, approval gates, or recovery budgets. ## Model Used - OpenAI Codex, based on GPT-6. The exact serving model ID and context window are not exposed in this session. Used reasoning, tool use, code execution, and browser diagnostics. No sub-agents were used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass — the targeted workspace suites pass; the full local chat-suite failure and baseline comparison are documented above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
27db8354e4 |
fix(ui): make failed attempt notices stand out (#18174)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/257/changes/paperclip/257 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task conversations show when an agent could not finish a saved request. > - The exhausted-attempt notice currently looks like ordinary muted status text. > - People can overlook both the failure and its available Retry action. > - This pull request gives failed attempts a padded, tinted surface and a recognizable error icon using existing tokens. > - The benefit is a clearer failure state while automatic recovery keeps its quiet presentation. ## Linked Issues or Issue Description Related: #17386 and #14156 cover recovery behavior. This PR only improves the exhausted-attempt notice's presentation. Searches found no duplicate styling PR. **What happened?** “Couldn’t finish this attempt. Your message is saved.” appears as muted text without a background or padding above the task conversation, making the failure easy to miss. **Expected behavior** An exhausted attempt should have a distinct, readable surface with internal spacing and an obvious Retry action when permitted. **Steps to reproduce** 1. Open a task whose system-owned interruption is `failed`. 2. Find the notice above the conversation. 3. Compare it with an automatic retry countdown. Previously, both had the same muted inline treatment. ## What Changed - Style failed attempts with a rounded destructive-tinted background, token-based padding, foreground text, and a decorative error icon. - Center the icon with the message and Retry button, including when the message wraps. - Keep the Retry button from shrinking on narrow screens. - Check the center positions in the existing browser layout test. - Add a production-component Storybook example beside a saved message. - Document the exhausted-attempt treatment in the design guidance. ## Verification - `pnpm check:token-gates` and `node scripts/check-maps.mjs`: pass. - `pnpm exec vitest run ui/src/components/ExecutionBlockerNotice.test.tsx`: 8 tests pass. These cover Retry, countdowns, read-only access, active and resolved states, admission waits, and admission errors. - Chromium layout: 4 cases pass at 320px and 1280px in light and dark themes. The icon, text, and Retry button have the same vertical center. The previous code failed this check by 8–10px. - Updated desktop and mobile screenshots are saved with the task. - `pnpm --filter @paperclipai/ui typecheck` and `pnpm --filter @paperclipai/ui build`: pass after the workspace SDK is built. - Full local typecheck and build need Cargo. Cargo is not installed in this sandbox. The full local Vitest run did not complete in the prior session. CI must verify these checks. - The updated head `4ba8de9da3e5c0c084cb5f752f371eccff31376a` passes all test jobs, typecheck, build, and all eight E2E shards. The final CI summary is queued. - Greptile completed its review of this head with a score of 5/5. It found no actionable issues. There are no review threads. - The separate Paperclip Review check is queued. Its review task stopped before it could publish the result. This check must complete before merge. ## Risks Low risk: this is a presentation-only change. The failed state takes more vertical space. Recovery, authorization, retry timing, and API contracts retain their existing behavior. Colors, spacing, radius, and icon sizing use existing design tokens. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository tools, code execution, and browser inspection. The exact runtime model ID and context-window size were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
774ddc7271 |
test(evals): add the full chat catalog to TestOps (#18542)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat feature evaluations check real user requests through the product. > - The YAML catalog has 244 scenarios, but the shipped TestOps completion route selects only three. > - The preserved catalog already supplies the execution mappings and independent oracles. > - A larger TestOps plan needs exact identities, bounded selection and source-bound private evidence. > - This pull request adds that plan route without merging the catalog app into master or replacing its grades. > - Users can run the full catalog or a selected subset and see the actual outcome and prerequisites. ## Linked Issues or Issue Description Refs: #18516 and #18515. This extends their completion pilot and reuses its trusted reporting path. **What existing behavior does this improve?** The existing three-case TestOps completion workflow. **Current behavior** The remaining catalog scenarios cannot be selected through that TestOps job. The one-case backlog plan has a separate strict singleton contract. **Proposed behavior** A separate candidate-scoped plan maps all 244 scenarios. Existing case aliases, jobs, plans and result history stay intact. Registration is distinct from a measured pass. ## What Changed - Add a source-derived 244-row manifest. It binds the 242 domain descriptors and two legacy tasks to the unchanged YAML queries, contexts, outcomes and original bounds. The JSON snapshot avoids importing the unmerged domain tree into master. - Reconcile exact case identities and a dedicated full-catalog plan. Reuse legacy aliases, including the existing backlog case, without renaming them. Preserve history and stop on conflicting aliases or uncertain writes. - Extend the shared completion adapter with a separate trusted full-catalog workflow context. Return a compact observed binding that fits one 48 KB protected variable. Keep the three-case pilot contract intact. - Add a fixed-source manual workflow. Small selections run serially. Larger selections allow eight independent workers and one serial external-input worker. Each worker and report reader keeps a singleton approval and exact-key AWS session. The eight-cell private authority limit is unchanged. - Validate and merge metadata-only singleton projections before one protected Allure upload and complete paginated readback. Preserve failed originals and cleanup failures beside passed results. Keep screenshots and raw evidence private. - Discard a changing full-catalog page-count snapshot and retry within the same observer bounds. Malformed page fields still fail at once. A pass still needs one consistent exact selection and all final evidence checks. - Keep workflow source and app source separate. The full lane builds candidate `3b66c3c144360ddf32714a0742fd774b8ceafb16`. Its only changes from the existing `de0941835cc8f0d01c982574ee258e110babdbeb` pin are finite TestOps host admission and its offline test. This is not current-master app qualification. - Give full selections above three cases a bounded metadata-read allowance above the existing 120-second deadline ceiling. Readback has two upper bounds: at most twelve polls with five-second pauses, and a size-derived deadline of at most 374 seconds, whichever exhausts first. There is no guaranteed minimum wait or processing SLA. Small selections and pilot deadlines stay unchanged. Final identity, grade and evidence predicates stay intact. - Document the real readiness limits and setup path. Five coverage gaps, six evidence exceptions, eighteen additional-credential declarations and thirty-seven external-input declarations overlap. They are not new pass claims. ## Verification - Related Node controls passed: pilot API and dispatch, 244-row manifest, idempotent reconciliation, full selection, three-page result readback, source guards and partition/output-size bounds. The latest ten adapter controls include the actual registration receipt's variable-size limit and the real bootstrap facade's closed failure codes and privacy checks. - A deterministic 244-result replay with one second per HTTP response failed at the old deadline and now verifies after 127 virtual seconds. A three-second response replay still rejects at 375 virtual seconds. The full-lane allowance accounts for paginated reads and four-way result batches; it is not a guaranteed network SLA. All 55 related API/adapter controls pass. They also reproduce the changing-page-count failure before the fix, then verify the consistent snapshot. Permanently changing counts fail after twelve polls. A malformed numeric field still fails before a retry. - Owning export controls: 26 passed. They include all 244 synthetic carriers, mixed passed/failed originals, missing and foreign projections, privacy filtering and exact source binding. - Candidate host owner: 31 offline controls passed. The new finite-catalog case failed before admission changed. - Preserve two actual local regressions: canonical scenario provenance failed before its one-line fix; the full registration binding exceeded the variable limit before compact output. Both now pass. - Initial local harness setup attempts failed before test execution. The final narrow owner configuration avoids provider-bundle setup. - Full repository tests, build and typecheck were not run locally. Normal required PR CI provides the remote checks. - Owning TypeScript no-emit, actionlint, paid-workflow policy with unsafe controls, and diff checks passed. - Local tests used Node 26.4.0. These are synthetic source controls. No local app, database, native build or provider qualification ran. - CI [41307571776](https://github.com/paperclipai/paperclip/actions/runs/41307571776) on `23e518be` recorded one 15-second timeout in `attention-retention-work.test.ts:153`. All other ordinary leaves passed. The test used a fixed JavaScript date but its retention deadline comes from PostgreSQL `updatedAt`. The exact failing await and original SQL timestamp were not captured. The corrected pre-sync head’s server shard (2/12) passed on AWS in CI 41309030928. Current master independently fixed the same clock assumption in #18543; conflict resolution adopts that whole upstream file. No local PostgreSQL run or timeout increase was used. - Current-head normal CI [41463567187](https://github.com/paperclipai/paperclip/actions/runs/41463567187) passes all required checks on `fc6c2309cfc7d1e5a83f0a597c7fa1d2beb987e1`, with Greptile 5/5 and all review threads resolved. The final diff has 23 catalog files. Current master’s whole retention test is adopted; the redundant earlier test change is absent. The valid changing-page snapshot regression passes with all 55 related API/adapter controls. No new TestOps import or live evaluation has run for this change. After delivery, use metadata-only bootstrap and a three-case normal TestOps selection to verify the new route. Do not claim a new 244-case measurement. ## Risks - The app target remains the preserved catalog candidate. Each result records its target and workflow revisions. No old campaign ordinal, acceptance gate or core fix is changed. - Some mapped cases need private account configuration or have known source/evidence gaps. Their original checks and evidence rules can still fail. A registration does not establish readiness or success. - Whole-plan execution can take hours and spend provider and AWS money. Original per-case limits and zero retries remain. Unknown setup usage is not free. This PR does not run all 244 cases. - The raw private `SOURCE_REF` field retains the prior integration-lineage label. It is not a verified checked-out branch. The actual candidate checkout and all authority checks use immutable `3b66c3c144360ddf32714a0742fd774b8ceafb16`; public projections set `source.ref` to null. - TestOps has no atomic bulk upsert. Reconciliation retains partial progress and performs no automatic resend after an uncertain response. - The trusted inventory keeps a 12 MiB limit per public carrier file. The larger parser ceiling does not promise admission above that limit. The existing 244-case fixture produces a 1,359,465-byte normalized carrier; this is not a guarantee for future payload sizes. - Raw evidence remains under the existing private retention owner. Public GitHub artifacts and TestOps attachments contain only allowlisted metadata and availability receipts. - The new workflow needs the exact master entry in the existing runner group's selected-workflow list before a live run. No repository or runner-group visibility is broadened by this source change. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) assisted with implementation and independent source review, using code and tools. The host does not report the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id - [x] I have run the relevant local controls and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1102.0-canary.1 |
||
|
DottaandPaperclip
|
39e5ccd1a8 |
fix(ui): remove New Task heading from composer (#18604)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People use the task composer to describe work for an agent. > - The Coordinator workspace added a visible New Task heading above the editor. > - This heading adds text and space that the task form does not need. > - This pull request removes the heading and its empty header space. > - The composer keeps its task fields and creation controls. ## Linked Issues or Issue Description **What happened?** The task composer shows a New Task heading when Coordinator is on. Commit |
||
|
|
06bc8db75c |
fix(runner): install verified API artifacts in execution workspaces (#18494)
## Thinking Path > - Paperclip manages AI agents and their work. > - Native agents use authorized API tools to read saved results. > - Copying a large response as model-generated text can lose bytes. > - The execution workspace needs an exact copy of the saved artifact. > - This change copies the authorized bytes without shell API credentials. > - The runtime verifies size and SHA-256 before it reports success. > - A live evaluation checks capture origin, installation and delivered bytes. ## Linked Issues or Issue Description **What happened?** A native agent lost one byte while copying a 24576-byte API text page. Existing capture and paging controls passed. The runtime had no verified saved-artifact-to-workspace copy tool. A separate historical screening refusal does not establish capture corruption. **Expected behavior** An authorized native run can install the complete saved artifact. The stored and installed bytes must match in size and SHA-256. A conflicting file must remain unchanged. A lost publication receipt must not cause an unsafe repeated write. **Steps to reproduce** Run `server/src/services/native-runtime/artifact-transfer.integration.test.ts` on baseline `c46b8f35add9255fc71be294d27f5a7865bcb47f`. Its capability assertion fails because `download_artifact` is absent. Apply this PR and run the same regression with the filesystem and API controls. It verifies 388131 synthetic bytes through the authority and installation boundary. Refs #18429. Related #17393, #17086 and #17285 cover paging, transfer foundations and remote output reads. This PR adds the inbound installation contract. ## What Changed - Add `download_artifact` for supported standard-mode native runs, with a 10 MiB limit. - Reuse current company, task, run, asset-content and responsible-user access checks. Check authority again before publication. - Bind installation to the current execution lease. Stage through confined Linux descriptors and publish with an atomic create-only link. Keep API credentials out of execution commands. - Verify bytes at storage and execution boundaries. Reject unsafe paths, symlinks, different existing files, truncation and corruption. Reconcile an unknown publication through verification only; an identical explicit retry remains safe. - Add API documentation, access-map entries and content-free audit records. - Preserve only the pinned Daytona SDK numeric HTTP status when failed creation cleanup is unconfirmed. Keep provider exception messages and headers private. - Keep only the finished runner binary and portable provider pack in their builder layers. Remove Rust targets/downloads and the Node install workspace/store before export. Report tree sizes. - Add one explicit Codex/Daytona Product E2E cell. Its independent oracle checks the server capture audit, redacted tool receipts, stored source, delivered attachment and browser link after reload. Uploaded substitutes and one-byte loss fail. ## Verification - Merge update at `6766b46553cab5b9787ade87a002eccaf50e68fc`: merge current master and preserve both explicit suites in the catalog. Focused catalog, image and oracle checks: 122 pass. Filesystem and authority integration: 20 pass. Maps, module boundaries and whitespace checks pass. New-head CI and Greptile are running. Full local build and typecheck remain unavailable because cargo is absent. - Behavioral red/green: the absent-tool regression failed before implementation and passed afterward. The initial missing-package attempt is recorded as setup failure. - Linux filesystem and real PostgreSQL/authority integration: 19 filesystem tests and 1 authority test pass. Existing API authority, capture and paging controls: 70 pass, 2 skipped. Provider transport in these integration controls is simulated. - Qualification calibration: 118 focused tests pass. The real persisted-event redaction case and a byte-identical uploaded substitute each failed before their oracle repair. The unchanged controls then passed. Harness typecheck passes. - Full recursive typecheck and build pass locally on the integrated production source at `086f7a2901`. Later commits change only qualification tests and documentation. - Structural map/module checks and whitespace checks pass. - Full local `pnpm test:run` was attempted by the implementation worker and stopped under its 8 GiB memory limit, with no full-suite verdict. Current-head CI must supply the full test result. - The complete local Product E2E support suite has 13 failures. The unchanged pre-qualification baseline reproduces all 13: host PostgreSQL semaphore capacity, stale timeout/profile/workflow inventory expectations, and expired fixed-date allowance fixtures. This is not a full local suite pass. - Earlier CI at `086f7a2901` failed a browser test during cleanup cancellation with a database lock timeout. It remains recorded. All browser shards passed on both later fully checked heads. - Two qualification campaigns were canceled during setup before provider execution while correcting the oracle. They provide no live product verdict. - Previous-head CI at `28fa001457f59df1c300c016dc569049e72d0061`: 53 checks pass, 3 intentional selection skips; no failed or pending checks. - Real image/export regression: campaign 41272351780 compiled the Linux runner and passed root/unprivileged CLI and transport checks, then failed BuildKit export with `no space left on device` (exit 102, filesystem 99.99%). No provider cell ran. Removing the Rust build trees alone was insufficient: campaign 41273604815 also failed during provider-pack export with exit 102. Commit `5bbb549c54` also consolidates dependency installation and pack construction so the disposable workspace/store can be removed in the same layer. All 122 image/oracle/catalog checks pass. Image and remote-pack publication then passed in campaign 41275196741 after cleanup changes plus a scoped 60-to-120 GiB expansion of only its delete-on-termination builder disk. The VM terminated and the expanded volume was deleted. This does not establish that cleanup alone makes the default 60 GiB builder sufficient. - The Daytona admission probe then failed before agent creation, with unconfirmed provider cleanup. The queued continuity case was canceled because it shares this prerequisite. No artifact-transfer or continuity success is claimed. - Status-diagnostic regression: the original generic wrapper failed the HTTP-status assertion. After the repair, the full plugin suite passes 501 tests, with 7 skips; its standalone TypeScript check passes. Controls reject invalid status values, avoid accessors, preserve the original create status and exclude credential canaries. - Previous-head Greptile: 5/5; its summary names `28fa001457f59df1c300c016dc569049e72d0061`. All review threads are resolved. - Live artifact transfer and three-turn warm continuity: NOT VERIFIED. The diagnostic case in https://github.com/paperclipai/paperclip/actions/runs/41277598289 confirmed Daytona HTTP 401 before agent creation. The configured `DAYTONA_API_KEY` in the existing `runner-e2e-paid` environment needs verification or refresh. The queued continuity control was not run after its shared admission prerequisite failed. No further provider attempt is planned until the credential changes. Unconfirmed provider cleanup remains recorded. ## Risks - Source-provenance changes can repeat the provider dependency install. The combined build layer trades install-cache granularity for lower retained disk use. - Linux descriptor semantics and Node are required on the execution host. Unsupported providers and restricted runtime profiles fail closed. - Storage reads buffer at most 10 MiB. A provider outage can leave a reserved staging file. Partial bytes never become the destination. - A lost publication receipt can leave a complete file with an unknown result. Recovery verifies the existing file without installation. Conflicts are never overwritten. - Artifact contents remain untrusted. The copy does not bypass screening or authorize downstream execution. - Live qualification uses the governed Codex API profile. It does not establish every model, subscription profile, platform or historical failure cause. ## Model Used OpenAI GPT-6 in Codex assisted with diagnosis, code, tests and qualification. The desktop runtime does not expose a more specific model deployment ID or context-window size. The model used reasoning, source inspection, tools and code execution. The native implementation agent is configured with OpenAI `gpt-6.1-sol`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
67948173f4 |
fix(sentry): report saved-message task attempt failures (#18541)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Tasks retain user messages when an execution attempt cannot finish. > - The task interruption projection drives the saved-message failure notice. > - That state can occur without a new terminal run transition, which the existing Sentry reporter requires. > - This pull request reports that failed task state through the existing Sentry capture path. > - Operators can find the failure through the same tag and saved run links. ## Linked Issues or Issue Description Refs #13446 and #16300. The related PR search found no open implementation of this change. **What happened?** A task can show “Couldn’t finish this attempt. Your message is saved.” without a matching run-failure Sentry event. A stale saved-message wait can produce this state after the previous run succeeded. **Expected behavior** When run-failure reporting is enabled, this failed task state also reports with `event_kind=run_failure`. **Steps to reproduce** 1. Enable the backend Sentry DSN and `PAPERCLIP_SENTRY_RUN_FAILURES=unexpected` or `all`. 2. Leave a saved-message wait on an unfinished task after its last run has ended. 3. Read the task after the wait grace period expires. 4. The task shows the failure notice. Before this change, that read does not report it to Sentry. **Paperclip version or commit** Base commit: `e122478368`. ## What Changed - Report the system-owned failed presentation from the existing batch task reader. The pure projection remains unchanged. - Reuse `captureRunFailure()` with `event_kind=run_failure`, `failure_source=task_interruption`, and `error_code=task_attempt_failed`. - Preserve the actual saved run status and company, agent, task and run references. Load the run agent's adapter through a company-scoped join in the existing run query, including after reassignment. Add no database query or background timer. - Send fixed error text. Do not read chat bodies, prompts or raw run diagnostics for this event. - Suppress repeated reports for the same company/task/run/intent with a bounded process-local cache. Failed or disabled captures do not consume the report. - Document the enabled modes, event fields, duplicate behavior and cache limits. ## Verification - Regression before implementation: the two new Sentry tests failed because the capture entry point was absent; the existing 38 Sentry tests passed. - Focused verification: 332 tests passed across `run-failure-sentry.test.ts`, `task-interruption-query.test.ts`, `task-interruption-presentation.test.ts`, and `run-failure-report.test.ts`. The reassignment regression first reproduced the missing adapter, then passed after the query correction. - Command: `PAPERCLIP_TEST_POSTGRES_ADMIN_URL=<local-test-admin-url> pnpm exec vitest run server/src/__tests__/run-failure-sentry.test.ts server/src/__tests__/task-interruption-query.test.ts server/src/services/task-interruption-presentation.test.ts server/src/services/__tests__/run-failure-report.test.ts`. - PostgreSQL tests used isolated databases on a local PostgreSQL 17 server, with automatic cleanup. Initial embedded starts failed because installation scripts had not restored library links; after repair, the host semaphore limit prevented new clusters. The existing external-database helper supplied the working test path. - `pnpm -r typecheck` passed. - `pnpm build` passed. - `git diff --check` passed. - `pnpm test:run` was attempted. It stopped in the unchanged chat integration shard collection step with “No test found” for line filters. The complete local suite has not passed. - Sentry capture assertions use the SDK test double. Live Sentry ingestion and a deployed browser journey were not tested. ## Risks - An enabled instance can send more Sentry events when task reads discover retained failures. It can emit a task-state event alongside the separate terminal run event. The source tag distinguishes them. - Deduplication is process-local and retains 1,000 reports. Restarts, multiple replicas or cache eviction can report a retained failure again. - No schema, permission, execution or UI changes. The existing opt-in reporting gate stays in place. ## Model Used OpenAI GPT-6 through Codex, with tool use, code editing and test execution. The exact runtime model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f6fc336a35 |
fix(ui): reveal task conversations without loading layout shifts (#18545)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Tasks combine saved messages, documents, run output, monitors, and
decisions.
> - Compact history makes long tasks load quickly.
> - The loading view still showed the real task header before the thread
scrolled to recent messages.
> - Different loading frames also changed the viewport, composer bounds,
and header typography.
> - This pull request uses one loading frame and reveals the prepared
conversation once.
> - The benefit is a stable first view without a material loading-speed
penalty.
## Linked Issues or Issue Description
**What happened?**
A cold task open could show a generic skeleton, then the real title and
monitor above placeholder messages, then jump to recent content. The
loading view changed width on desktop and height on mobile. Its title
line height differed from the real title.
**Expected behavior**
Keep the loading viewport and composer in place. Prepare the initial
reading position out of view. Reveal the recent conversation once its
data, fonts, scroll position, and composer bounds settle. Cap the font
wait at three seconds if the font host does not answer.
**Steps to reproduce**
1. Open a task with a long title, long saved replies, document cards, a
monitor, and a pending decision.
2. Reload it with CPU and network throttling.
3. Watch the header appear above placeholders, then move offscreen when
the recent conversation appears.
**Paperclip version or commit**
|
||
|
TonioandClaude Opus 5.5
|
8ea240218d |
feat(ui): bring the Refine-v2 layout to the agent page (#18378)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The agent page is where an operator reads and changes one agent. > - The page grew one tab at a time. Ten tabs sit in one flat list, the header holds many buttons, and the audit links leave the agent. > - A design pass, called Refine-v2, made a calmer layout for this page. It ran in a prototype on fixture data only. > - This pull request moves the Refine-v2 layout and behavior into the live agent page. Where the prototype and the live page differ, the live behavior stays. For example, the floating save bar stays on screen. > - The benefit is a shorter navigation, edits in place, and audit data that stays inside the agent. ## Linked Issues or Issue Description Refs #13716 Refs #9166 Both pull requests mount plugin tabs or slots on the agent page. They touch the same files, so one of the three needs a rebase after the others merge. **Subsystem affected** The agent detail page in `ui/` (`ui/src/pages/AgentDetail.tsx`, its sidebar, header actions, and the shared agent config form). **Problem or motivation** The agent page is hard to scan. The navigation shows ten items in one list. The header shows Run now, Pause, Clear error, and Assign task as separate buttons. The name, title, and manager are form fields deep in the Harness tab. Activity, Runs, Costs, and Budgets open company-wide pages, so the operator loses the agent context. Each tab uses a different card and table style. **Proposed solution** Use one layout system for every agent tab. Group the navigation into Agent and Audit. Edit the name and title in the header. Move the work actions into the actions menu. Show audit views inside the agent. Keep the persistent save bar and add a status dot to it. **Alternatives considered** We considered a flag that lets operators opt in to the new layout. We did not use a flag, because two layouts on one page double the test surface. We also considered moving the save button to the end of the scrolling content, as the prototype did. We kept the floating save bar, because it is always visible. **Roadmap alignment** `ROADMAP.md` has no item for the agent page. This change is a UI refinement of an existing page. It adds one optional field to an existing API request and no database schema change. ## What Changed - **Navigation.** The agent sidebar shows two groups. Agent has Overview, Identity, Harness / Runtime, Secrets & variables, Permissions / Trust, and Advanced. Audit has Activity, Runs, Costs, and Budgets. Skills, Tools, Channels, and API Keys leave the navigation. Their URLs still open. The Instructions tab is now Identity, at `/identity`. The old `/instructions` URL redirects. - **Header.** The agent character shows at a larger size. The name and title edit in place. Enter or blur saves. Escape cancels. An error restores the old value. A rename updates the URL. While a tab has unsaved changes, the header refuses the edit and asks the operator to save or discard first. The third line shows the title and a link to the manager. A crown button to the right of the name replaces the "Set as my primary" link and the separate primary indicator. It is outlined and dimmed until the agent is your primary, then fills with a short pop. Replacing an existing primary asks for confirmation first, and the pop plays after the dialog closes. Reduced motion skips the pop. Run now, Pause or Resume, Clear error, and Agent history move into the actions menu. Assign task is the primary button. For an agent that still needs setup, Finish setup is the primary button and Assign task moves into the menu. - **Agent history.** A new dialog in the actions menu replaces the Revisions entry. Each row shows a plain-language summary, the person or agent who made the change, the date, and Restore. Restore asks for confirmation. A failed restore shows its reason inside the confirmation. - **Overview.** Four metric tiles show activity this week, runs this week, costs this month, and budget used. Each tile links to its audit view. When the list behind a count is capped, the tile shows a lower bound, for example 200+. The activity tile follows live updates. Profile, Skills, and Recent tasks panels follow. The Profile panel lists direct reports under their label in at most two rows, then a "+N more" button that expands the list. The Latest run card is removed. - **Identity.** The file preview shows the raw text. A pencil button switches to a plain text editor. Copy and Delete are icon buttons. A chip next to the file name says "Managed by you" or "Managed externally". Per-file history and unsaved edits stay. - **External instructions.** "Advanced settings" opens Root path and Entry file fields and a Load profile button. Load profile asks the server to switch only if the root directory and the entry file already exist. If either is missing, the server refuses and the panel shows the reason. The entry file can have any name inside the root. - **Instructions bundle API.** `PATCH /agents/:id/instructions-bundle` accepts an optional `requireExistingEntry` flag (`packages/shared` validator, `server` service, `ui` client). With the flag, the server checks the root directory and entry file before it creates or copies anything. Without the flag, the behavior does not change. - **Harness / Runtime.** The config form gets a `panels` layout: Harness and Execution side by side, then the runtime test, then environment variables. The identity and run-policy sections are hidden from this tab. - **Secrets & variables and Advanced.** These tabs use the same filled panels and table headers. - **Permissions / Trust.** The trust presets show as selectable cards. An Allowed connectors panel shows up to three connector logos and links to Apps, filtered to installed connectors. Capabilities sit in their own panel. Changes on this tab save at once, with no save bar. - **Audit inside the agent.** Activity, Runs, Costs, and Budgets render inside the agent page. Runs is locked to the current agent. A failed budget incident action shows its reason under the incident. - **Save bar.** The floating save bar stays. It shows an amber dot for unsaved changes and a green dot with "Changes saved" after a save. Discard stays. - **Leave guard.** If the form has unsaved changes and the operator clicks a link, a dialog asks to keep editing or discard the changes. Browser back and the header actions keep the native confirm. - **Apps.** `/apps?source=installed` opens the Apps page on the Installed filter. - **Tokens.** New `--agent-detail-*` tokens in `ui/src/index.css` hold every spacing and fill value. Each fill is a mix of foreground into background, so one value works in light and dark themes. - **Tests and maps.** Tests cover the navigation model, sidebar, header menu, Overview layout, Identity editor, and helpers. The agent-configuration recipe in `maps/feature-map/agents.md` describes the new page. ## Verification - `cd ui && npx vitest run`: 849 test files and 10,081 tests pass. - `cd server && npx vitest run src/__tests__/agent-instructions-routes.test.ts src/__tests__/agent-instructions-service.test.ts`: 40 tests pass. - `npx playwright test --config tests/e2e/playwright.config.ts tests/e2e/onboarding.spec.ts tests/e2e/chat-adapters-ui-providers.spec.ts`: 26 tests pass. - `cd ui && pnpm typecheck` passes. This includes the Storybook catalog check. The `server` and `packages/shared` typechecks pass. - `cd ui && pnpm build` passes. - `pnpm run check:token-gates` reports all gates clean. - `node scripts/check-maps.mjs` and `node scripts/storybook-catalog.mjs --check` pass. - Manual check on a local instance, in light and dark themes: 1. Open an agent. Confirm the sidebar shows the Agent and Audit groups. 2. Click the agent name in the header. Type a new name and press Enter. Confirm the URL changes. Rename it back. 3. Open the actions menu. Open Agent history. Confirm the rows and the Restore confirmation. 4. Open Identity. Click the pencil. Change a line. Confirm the save bar shows the amber dot. Click a sidebar link. Confirm the discard dialog. Click Keep editing, then save. Confirm the green dot. 5. Open Harness / Runtime, Secrets & variables, and Permissions / Trust. Confirm the panels and the trust cards. 6. Open Runs under Audit. Confirm the list shows only this agent and the page stays inside the agent. ## Risks - This changes a high-traffic page for every operator at once. There is no flag to switch back. - Run policy and Reports to are no longer editable on the agent page. The header shows the manager as a link only. - Skills, Tools, Channels, and API Keys are reachable by URL only. Operators who use them from the sidebar must use the URL. - The pull request conflicts with #13716 and #9166 in `AgentDetail.tsx` and the agent sidebar. - Storybook stories for the agent page and `DESIGN.md` are not updated in this pull request. - The new `requireExistingEntry` flag is opt-in. Existing API callers keep the old behavior, which creates a missing external directory and entry file. - I ran two e2e spec files locally, not the full `pnpm test:e2e` or `pnpm test:release-smoke`. CI runs the full e2e suite. I did not run the repo-wide `pnpm -r typecheck`, because the local toolchain has no Rust. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), in Claude Code, with reasoning and tool use (shell, file edits, and a browser for visual checks). A person directed every design decision. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
1a6017d646 |
fix(server): resume suspended sessions after controller replacement (#18554)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A web server replacement must preserve a completed conversation as well as active execution. > - A warm remote Runner can suspend before the replacement admits another turn. > - The replacement restores its controller checkpoint from PostgreSQL, but lacks the Runner suspension receipt. > - It therefore tries to adopt a live process and rejects the safely suspended source. > - This change restores the verified receipt and preserves retained launch files while the old authority is reconciled. > - Later turns can keep the same provider conversation after the old web server filesystem is deleted. ## Linked Issues or Issue Description Refs #18552, #18543, #18366. **What happened?** The real rolling-deployment journey passed both HTTP handoffs, with 765 reads and 78 verified saves. The first three agent turns kept their native and provider sessions. After the second replacement, the fourth continuation failed with `remote_warm_session_not_idle`. The remote Runner was suspended with an empty outbox and no pending terminal. A focused regression reproduced that exact failure. After restoring its suspension receipt, the same test exposed a stale launch-file read during source recovery. **Expected behavior** A replacement can resume the same provider conversation from a settled suspended Runner. It must validate the physical sandbox and all persisted identity fields. A live source must still pass the existing adoption checks. **Steps to reproduce** Enable the Linux Runner fixture for `native-runner-restart-recovery.integration.test.ts`. Run the `diskless-warm-suspended` cases. The fixture completes and gracefully suspends each source, deletes the controller projections, and admits a later turn from another process. It does not write a synthetic suspension receipt. ## What Changed - Reuse the existing per-turn suspended-receipt reader before warm admission chooses live adoption or suspended recovery. - Keep the receipt read retryable when a failed remote read leaves only the controller projection. - Leave every non-suspended warm state to the existing live-adoption checks. A preliminary backpressure observation cannot reject admission before that owner inspects the current Runner state. - Preserve current sandbox binding, settled-outbox, exact-identity, local-evidence and admission checks. - Pass the structured launch specification to remote asset preparation. Source reconciliation keeps the old authority's retained launch files; a current-authority launch still stages its current files. - Add two Linux Runner/PostgreSQL regressions for two controller-cache deletions and three turns on one provider thread, including an interrupted receipt read. - Update the deployment qualification record, including the failed live evidence and the successful staging database-idle observation. ## Verification - Before the fix, the focused regression fails with the exact live `remote_warm_session_not_idle` error. - Both new regressions pass with real PostgreSQL and Linux Runner. The provider is deterministic; this is not a live model qualification. - The first receipt-only fix failed on the deleted controller skills cache. The completed fix passes that same scenario without recreating fixture launch assets. - All 1,067 surrounding launch supervision, warm recovery, launch-asset and controller recovery tests pass. The exact `cc23019691` server build passes. All ten Linux continuity cases pass, including live adoption, graceful suspension, per-turn resume, changed configuration and failed-read retry. Each case checks provider-thread identity and exact turn counts. The live Codex/Daytona journey passes on the same clean source and exact build: 518 reads, 53 saves and 53 verified saves with zero probe failures; both predecessor homes deleted; the active execution completes once; all four turns preserve the provider conversation and native receipts; saved files and the final command output are independently checked. No automatic retries. Child credential and object-storage bucket cleanup pass. Browser reconnection remains a separate hosted acceptance check. - The review refinement has a failing-before regression: only the preliminary read reports transient backpressure; the subsequent adoption reads the real Runner state. It and both suspension cases pass on the final source (three cases), with server typecheck. The existing idle check remains unchanged. An initial retry hit a fixture port left by the intentional pre-fix failure; a fresh container resolved that infrastructure error. - Independent cleanup verification found two archived setup sandboxes left by the live harness. Both were deleted and confirmed absent. The failed automatic-cleanup check remains recorded. - No claim of the ten-deploy Railway campaign or production canary yet. ## Risks - A cold warm-session admission can now make one bounded remote state read when local lifecycle evidence is absent. Its failure stops admission and remains retryable. - This path cannot replace contradictory local evidence, transfer authority, or treat a suspended process as a live one. - Source recovery must keep the previous authority's launch files until reconciliation completes. The launch identity selects that behavior; ordinary launches still prepare current assets. - No schema migration or public API change. ## Model Used OpenAI GPT-6 Codex, with reasoning, repository editing, shell execution and integration-test tools. The exact runtime model variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge CI qualification note: first final-head run `41322376054` passed the native/runtime and remaining suites but failed the unrelated `automatic_message_on_reactivation` browser fixture at its pre-reactivation deferred-queue count (0 rather than 1). The unchanged test passed locally against isolated PostgreSQL (1 test, 43.6 s); inspection found an asynchronous company-pause/fixture-seeding timing window. Original CI logs and local evidence are retained. The single failed-job retry passed; all final-head CI gates are green. Both reviews are 5/5 and no review threads remain unresolved. The first local embedded-Postgres launch failed before tests during cluster initialization; the isolated external-Postgres test passed, and its disposable database was removed after correcting the private launcher cleanup call. --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1101.0-canary.13 nightly/v2026.1102.0-nightly.0 |
||
|
DottaandPaperclip
|
c0550e7010 |
fix(server): preserve sessions and defer busy overlap recovery (#18552)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A web server replacement must preserve remote execution and later task continuation. > - The active-run handoff now survives deletion of the departing server home. > - The next admission still compared disposable instruction and skill paths as configuration. > - A native receipt could also settle without the old adapter returning its terminal metadata. > - A replacement can also collide with a run row locked by the serving controller. > - This change preserves durable session identity and terminal evidence, and defers locked recovery work so overlap can reach readiness. ## Linked Issues or Issue Description Refs #18543, #18366, #9977, #18118. **What happened?** A real Codex/Daytona journey replaced both control-plane servers and deleted their homes. Reads, saves, saved files and active execution passed. A later useful answer used a new provider session. All four run records lacked the native terminal contract. The whole qualification therefore failed. A later attempt also exposed startup failure: the replacement waited for a row lock held by the serving controller and aborted on its one-second lock timeout. **Expected behavior** An unchanged managed instruction entry and immutable skill version retain the session across server cache relocation. Actual content and configuration changes still reset it. A committed native receipt retains its terminal contract even when the originating adapter never returns. A busy recovery row defers through the existing scheduler while the replacement reaches readiness. **Steps to reproduce** Run the `rolling-web-deployment.runner-codex.daytona.recover-controller` Product E2E cell with separate controller homes and shared PostgreSQL/S3. The focused regressions reproduce the failures without a model call. ## What Changed - Read the managed instruction entry digest from the company-scoped durable file head. Do not restore or read the cache for this check. - Omit disposable instruction paths only when that durable authority exists. Fingerprint selected immutable skill versions in both runtime skill inputs. - Preserve the existing local and external instruction path checks, and unversioned skill identity. - Publish the accepted native terminal contract in the same transaction as the canonical result receipt. - Skip locked recovery rows and schedule the same run for retry. Avoid an audit write that would block on the same row. No ownership is transferred while busy. - Keep replacement readiness independent of an in-progress skill selection; migration and cache eviction still require exact pins. Missing durable skill versions still refuse startup. - Project only the instruction digest in SQL instead of loading the entire agent-file manifest per admission. - Add real PostgreSQL lock contention tests for the issue, run and coordinator, including the heartbeat startup caller and later claim. - Limit native test teardown to its owned database on a shared test host. The old behavior also fails on clean master. - Add cache-loss, scope, changed-content, changed-version and finalization regressions. Update the deployment qualification record. ## Verification - Both focused regressions failed before the source changes. - All 249 tests pass across `heartbeat-workspace-session`, `agent-file-durability`, and `paperclip-control-plane-port`. - Server typecheck and exact-revision build pass. - Assembled admission ran 59 passing cases but failed during native-suite teardown. The teardown failure reproduced on clean master. All 16 native resumption cases and teardown pass after the database-scope fix. - All three row-lock regressions fail with the pre-fix lock timeout and pass with deferred recovery. The full recovery suite passes 74 cases, including the startup caller; eight Docker-specific cases were not enabled. - The real attempt at `d2165dec5d` failed when B aborted on the run lock. Its credential and bucket were removed; all three provider sandboxes were independently confirmed absent. The next real attempt at `aae8a53d01` failed a different readiness gate while a run was preparing skill pins. That race now has a failing-before regression; all 33 readiness, durable-file and managed-skill tests pass after the fix. Its owned resources are cleaned. The `a15a12651c` live attempt passed both web replacements: 765 reads, 78 saves and 78 verified saves with zero HTTP probe failures. The first three turns preserved the provider session and native terminal contract. The fourth continuation failed because a safely suspended remote Runner was treated as live/idle; that distinct recovery gap is being handled in a follow-up. All three owned sandboxes were independently confirmed absent. No claim of full Railway qualification yet. ## Risks - The new fingerprint shape may reset a session once when upgrading from older fingerprint metadata. It is stable across subsequent cache replacements. - A versioned skill is identified by its immutable selected version; materialization and admission still enforce availability. Unversioned skills retain their existing path boundary. - A native terminal describes the accepted execution result. Workspace and issue settlement remain separate authorities. - This does not yet prove the ten-deploy staging campaign, natural database autosuspension or the production canary. ## Model Used OpenAI GPT-6 Codex, with reasoning, repository editing, shell execution and live integration-test tools. The exact runtime model variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1101.0-canary.12 |
||
|
DottaandPaperclip
|
7751fc3107 |
fix(server): complete volume-free controller admission and recovery (#18543)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Its volume-free web server must admit remote warm execution without
depending on a local disk.
> - Fresh run rows retain the initial legacy label during workspace
preparation.
> - The native runtime selector commits the execution mode later.
> - The earlier guard rejected valid remote runs before that selection.
> - This PR checks storage during preparation and the resolved lifecycle
during runtime selection.
> - Agent-file recovery also has to reconstruct disposable cache paths
from the retained remote directory.
> - The deployment test removes the complete predecessor home, including
immutable caches, to enforce that contract.
## Linked Issues or Issue Description
Refs #18518, #18529, #18533.
**What happened?**
A real Codex/Daytona deployment test failed during setup because the
volume-free workspace guard checked the initial run label. After fixing
admission, controller retirement first failed on an immutable skill
cache. Removing that cache exposed a production recovery failure: B
rejected the saved agent-directory path from A, even though the remote
execution identity was unchanged. The next live attempt recovered that
directory and transferred the same runner, but adoption then tried to
recreate provider launch assets from the deleted controller home.
**Expected behavior**
Allow remote workspace preparation before native selection. Reject
actual legacy and per-turn execution at the runtime boundary. Delete the
entire stopped predecessor home, including immutable caches, without
following links to shared dependencies. B must reconstruct its local
agent-file projection from the exact registered remote lease while
preserving the save baseline and unsaved remote edits. Reattaching an
already-running provider must not prepare a new launch or require the
old launch files. The full backend must recover the admitted
instructions from the matching durable run checkpoint.
**Steps to reproduce**
Run the explicit
`rolling-web-deployment.runner-codex.daytona.recover-controller` Product
E2E cell with a complete server/UI build, disposable PostgreSQL/S3, and
an owned Daytona fixture. The new boundary regressions also reproduce
the premature guard and missing runtime rejection without a model call.
**Paperclip version or commit**
The failures were observed on
`247552fe0719dcc92f7b42c50b658be6f624986d`. This branch is based on
`e122478368`.
## What Changed
- Validate Runner and remote workspace authority before runtime
selection without trusting the initial run label.
- Reject resolved legacy execution and local-source fallback, preserving
the native warm-policy check before admission.
- Rebind remote agent-file caches on ephemeral controllers and retrieve
their validated contents without uploading over the retained execution.
Preserve pending saves when the remote collector is unavailable.
- Restore over-quota remote directories without applying save quotas to
run admission; ordinary saves still enforce quotas.
- Skip provider launch asset materialization when adopting an existing
Runner. A real-process regression deletes the old skill/instruction
sources before reattachment.
- Recover admitted system instructions through Runner-owned checkpoint
validation for the same run, session and context digest. Fresh launches
still compose current instructions.
- Make the attention-retention test advance past its database-assigned
deadline, fixing a clean-master clock mismatch.
- Serialize remote collection and cleanup across controllers with an
operation-scoped Postgres advisory lock; fence stale cache-path updates.
- Keep synthetic deployment probe tasks in a separate disposable company
so they cannot block the business completion oracle.
- Share the launcher's directory cleanup with rolling retirement. Reject
a symlink root and skip linked descendants.
- Test admission with real PostgreSQL transactions and cleanup with real
read-only directories and external symlinks.
- Update developer documentation and the canonical qualification record.
Broader deployment acceptance remains in progress.
## Verification
- `pnpm -r typecheck` and `pnpm build` passed.
- All 156 workspace preparation, runtime selection and ephemeral
contract tests passed. Two new regressions failed before the source fix.
- Seven focused Product E2E cleanup/replacement tests and `pnpm
test:e2e:runner:typecheck` passed.
- `pnpm test:run` stopped at an unrelated 15-second chat integration
timeout after 338 passes. That exact test passed in isolation without
increasing its timeout. The later full-suite shards did not run.
- `pnpm test:e2e:runner:unit` reported 5,898 passes and 13 failures.
Clean master reproduced 11 failures in unchanged onboarding
expectations, operating receipts, Pi catalog assertions, workflow
bootstrap counts and embedded PostgreSQL startup. The other two workflow
assertions were fixed by the intervening master commit. These failures
are retained, not counted as passes.
- All 122 workspace-preparation tests pass after the reviewed
host-fallback fix. The first broader attempt exposed a test environment
leak; restoring the ephemeral flag between cases resolved it.
- Agent-directory and instruction regressions passed 179 tests. Five
focused cache-loss cases pass: different home, same path on an empty
filesystem, foreign lease, concurrent controller, and missing collector.
- The live fixture at `a18f6b85a3` gracefully stopped A and deleted its
entire home, then failed the 420-second remote reattachment deadline on
the saved local agent-directory path. That failure is retained; exact
owned sandbox cleanup was independently confirmed. The next live attempt
at `e14ca5f3e9` restored the cache and transferred the same runner, but
failed while rematerializing launch assets from A. Both failures are
retained and their owned sandboxes confirmed absent. No active-overlap,
ten-deploy, or production-canary acceptance is claimed.
- All 71 related Runner adoption/recovery cases pass, including the new
removed-context regression (failed before the source fix).
- The revised full agent-directory suite passed 110 cases and failed
three cleanup timing cases. All 20 cleanup/cache-loss cases then passed
in isolation with unchanged timeouts, including over-quota recovery.
- Server and Runner TypeScript checks pass after the latest corrections;
the live retry remains pending.
- The live test at `67714af9ff` adopted the same remote execution after
A was deleted, then failed while the full backend read the retired
instruction bundle. All owned sandboxes were independently confirmed
absent. Its regression now passes with actual backend recovery and no
new thread or turn. All 59 backend factory tests and focused checkpoint
identity tests pass. Runner and server TypeScript checks pass; the exact
committed server build passes. A fresh paid-model retry followed.
- The attention-retention CI timeout reproduced on clean master. The
deadline correction passes all 53 tests in that file with unchanged
timeouts.
- The real `cd690de4d3` run transferred the same runner/session after
deleting A, completed, and completed the next revision. During handoff,
372 reads and 38 saves had zero failures; all 38 saves were verified.
The full attempt was interrupted because the probe's open task
contaminated the business company oracle. This remains incomplete
evidence. The scope regression fails before the fixture fix, then
passes; all nine deployment support tests and E2E typecheck pass. All
three owned sandboxes were confirmed absent and temporary
credentials/storage were cleaned up. A clean corrected attempt is
running.
## Risks
- The runtime guard must still reject legacy and per-turn execution
before dispatch. Tests cover these refusals and a valid resolved warm
policy.
- Directory permission changes apply only during cleanup of stopped,
test-owned homes. Traversal does not follow symlinks. This changes no
production cache permissions.
- Cache reconstruction must not change remote execution authority or
rebase the original save baseline. Tests verify foreign-lease rejection,
preserved receipts, and absence of local-to-remote upload.
- Postgres locking is scoped to active remote file work and adds no idle
polling; local/durable-server locking remains unchanged.
- No schema or public API changes. External conversation exemptions
remain unchanged.
## Model Used
OpenAI GPT-6 Codex, with reasoning, repository editing, shell execution
and live integration-test tools. The exact runtime model variant and
context-window size are not exposed in this session.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Latest real qualification at
canary/v2026.1101.0-canary.11
|
||
|
Dotta
|
e122478368 |
test(evals): run and record the chat completion pilot through TestOps (#18516)
## Thinking Path > - Paperclip evals must verify real user outcomes. > - A passing manual pilot does not make its cases runnable from TestOps. > - The existing TestOps adapter owns one separate backlog case. > - The three-case completion pilot needs its own exact selection and reporting path. > - This workflow binds a real TestOps job run to reviewed catalog source and AWS execution. > - Results retain their original verdict, checks, source and private evidence. ## Linked Issues or Issue Description Refs #18364, #18377, #18515 and #18509. This is the main-side companion to merged #18515. The immutable evaluated source is its actual integration-branch squash merge, `de0941835cc8f0d01c982574ee258e110babdbeb`; its tree matches the reviewed catalog head. The host admission remains inert until this main-only workflow exists. The broader catalog stays off master. The existing TestOps connection can run only the separate backlog cell. The project-work, project-defaults and instance-restart cases currently require manual campaign dispatch and private retrieval. This change creates a dedicated reusable TestOps plan and records fresh execution outcomes there. Source checks and registration do not establish a new live pass. ## What Changed - Add a default-branch-only workflow with exact TestOps selection, immutable source/build outputs, serial AWS execution and zero eval retries. No source/model/budget selector is accepted. - Register the three exact reusable cases and a separate plan with the existing reviewed GitHub integration. Preserve opaque job definitions. Validate the observed single Branch/defaultValue carrier, PATCH only enable/integration, and use the job’s default branch without rewriting its definition or adding plan overrides. Fence the initial operator launch by actual GitHub run number and nonce; normal TestOps Run remains repeatable. - Reuse the existing private-evidence authority, scanners, bounds and exact-key S3 policies. Authenticated selection supplies only this original attempt's exact source and case identity. The three trusted protocol files are unchanged. - Skip report retrieval after an unsuccessful build or skipped execution; retain evidence and failed grades when evaluation actually ran. - Check out trusted main after the target in both build and execution jobs, and reject any target tree containing the reserved trusted-helper path. A real temporary-Git-tree regression exercises the actual YAML guard. - Keep reporting credentials in separate protected jobs. Export only closed verdict/check/accounting metadata and private-evidence receipts to TestOps. Verify final PNGs privately and record their checksums; keep all screenshot bytes in protected AWS storage and out of public GitHub artifacts. Preserve failed grades and failed cleanup. - Verify actual same-run case results and complete result-owned attachment metadata within a fixed processing window. Missing or inherited attachments never qualify. Keep source/result links and stable case identities. ## Verification - 68 dispatch, selection, registration and readback controls pass. A checkout-boundary regression fails on the old ordering, accepts an ordinary target and rejects a candidate-owned helper tree before trusted checkout. They cover real workflow environment wiring, actual GitHub metadata guards, exact subset selection, wrong sources/cases/jobs, replays and attachment readiness. - 38 exporter controls pass, including the existing backlog exporter and the unchanged private-evidence authority consumer. Red/green controls prove project work accepts the catalog’s 1–3 run range in both export gates, while 0/4 runs are rejected; the actual pinned-source control verifies all three cases’ run bounds. Cancelled, failed and scheduled TestOps job stages are rejected even with complete results. The three new cases retain all 35 independent checks; private diagnostics and all image bytes are excluded from the public artifact. Extra media files and directories are rejected. - The actual checked-in workflow environment is exercised against the actual pinned source matrix and host function for all three cases before provider access. Local fixture controls make zero provider calls and do not claim live AWS qualification. - All 71 workflow-policy controls pass. Actionlint and owning typecheck pass. Relevant TypeScript build dependencies pass. - Full support suite: 5,827 passed, 13 failed, four skipped. A clean unchanged-master baseline reproduces all 13 failures in the same five files. They include fixture expectation drift, operating-receipt fixtures, a profile pin, pnpm bootstrap counts and local database startup. This change updates the exact TestOps workflow registries; the two affected boundary controls now pass. Baseline failures remain retained and are not presented as a green full support suite. - The catalog dependency passed CI and current-head 5/5 review, has no unresolved threads, and is merged. All current-head CI checks pass; both Greptile and Paperclip reviews are 5/5, with no unresolved threads. No new TestOps mutation, provider call or passing launch is claimed yet. A fresh three-case TestOps launch and verified server-side results remain the delivery gate. ## Risks This is a paid execution and reporting boundary. The new workflow accepts only trusted default-branch actor identities and a verified project/plan/job/case selection. Its evaluated source is immutable and separate from main reporting code. The full catalog is not promoted. Screenshots, raw evidence and hidden provider data stay in protected storage. Public GitHub artifacts carry only the closed metadata projection. TestOps receives checks and private-evidence availability/checksum receipts. Unknown setup or AWS accounting stays unknown. One passing launch will not establish full-catalog reliability. ## Model Used OpenAI GPT-6 through Codex, with repository inspection, code execution and tool use. Exact hosted revision and context-window size are not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
247552fe07 |
test(server): avoid deep traversal in durable file assertions (#18533)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Cloud web servers restore agent files from shared storage after replacement. > - The durability test checks restoration, corruption, cancellation and retry. > - Its generic equality assertions traverse every byte in many large buffers. > - The post-merge check for #18531 exceeded the 15-second test timeout. > - This change uses native buffer equality and keeps every byte check. ## Linked Issues or Issue Description Refs #18531. The post-merge Cloud-source verification run timed out in the new cold-folder test. Other shards passed. The unchanged test passed locally. This change removes avoidable assertion cost; it does not increase test or production timeouts. ## What Changed - Use `Buffer.equals` for the cold-folder content assertions. - Keep the concurrency, cancellation, cleanup, old-projection and retry assertions. - Record packaged-image proof, the hosted deployment state and the original CI failure in the existing plan. ## Verification - Eight durability tests passed against a real local PostgreSQL fixture. Object storage uses controlled asynchronous streams. - The same test file took 10.03 seconds before and 5.90 seconds after this change. The changed cold-folder case took 561 ms after the change. - The first local attempt could not start embedded PostgreSQL and ran no tests. The existing external test database enabled the comparison. - Original failed CI: https://github.com/paperclipai/paperclip/actions/runs/41296253723 . That failure remains in the qualification record. - Server typecheck and build passed. All latest-head PR checks passed, including the full test, typecheck, build and browser gates. One cancelled runner job was retried after its log showed an infrastructure shutdown; the retry passed. This PR changes no production code or database schema. ## Risks - A failed content check now reports a boolean mismatch instead of a large byte diff. Equality still covers buffer length and every byte. - Local timing is evidence for the fixture, not a general CI performance guarantee. - Hosted active-agent and rollout acceptance remain incomplete as recorded in the plan. ## Model Used OpenAI Codex, based on GPT-6. The exact runtime snapshot and context-window size are not exposed. Used code editing, shell execution and test analysis. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1101.0-canary.10 |
||
|
DottaandPaperclip
|
1d348bbef0 |
fix(server): restore durable agent folders with bounded concurrency (#18531)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A replacement web server restores agent files from shared storage. > - The old restore reads each object in sequence. > - Modest network latency makes a 1,024-file folder exceed its deadline. > - This pull request uses at most eight concurrent restore streams. > - Every file must pass its size and hash checks before the folder is published. ## Linked Issues or Issue Description Refs #18518 and #18526. Local deployment qualification found that a 64 MiB folder could not be restored after the old server filesystem was removed. ## What Changed - Restore at most eight files concurrently within the existing file authority service. - Cancel sibling transfers on failure. Await all workers before removing the temporary folder. - Keep the existing deadline, file-transfer database pool, authorization and publication rules. - Add regression coverage for bounded concurrency, corruption, settled cancellation, retained previous bytes and retry. - Document the behavior and update the deployment plan with local qualification evidence. ## Verification - Real PostgreSQL and S3-compatible storage: 1,024 files of 64 KiB each, plus instructions, with 25 ms added to each object GET. Delete the entire predecessor home and disable local adoption before reading through the successor. - Before: the read failed after 30.065 seconds. After: the same case passed in 4.180 seconds and fetched 1,025 objects. All owned test resources were cleaned. - The durable-file suite passed eight tests against PostgreSQL. Server TypeScript compilation passed. All 56 latest-head CI checks are complete and green or intentionally skipped, including full typecheck, build, unit and browser suites. Greptile is 5/5 with no review findings or open threads. - The plan separately records a released-container overlap proof: 130 reads, 26 saves, zero failures, two deleted predecessor containers and a retained authenticated session. That image predates this restore change. ## Risks - A cold folder can use eight simultaneous streams instead of one. Two file-transfer database connections bound the simultaneous restore callers. Streams verify bytes without loading entire objects into memory. - The 30-second deadline is unchanged. The measured size and latency do not qualify every folder at the maximum storage limits. - No schema, API or UI changes. Paid-model and hosted deployment acceptance remain separate work. ## Model Used OpenAI Codex, GPT-6. Tool use, code execution and repository review. The runtime does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1101.0-canary.9 |
||
|
DottaandPaperclip
|
d941ed3879 |
feat(server): verify existing-volume ephemeral enrollment (#18529)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Cloud web servers can now keep durable state in PostgreSQL and S3.
> - Existing servers still have files and a decision signing key on
their volume.
> - The volume must stay attached until every data owner verifies its
migration.
> - This pull request adds a dedicated maintenance process for that
transition.
> - Cloud can preserve the original key and remove the volume only after
a matching receipt.
## Linked Issues or Issue Description
Refs #18518 and #18526. Existing-volume enrollment is a required step in
the zero-downtime web deployment work. This is the Core maintenance
side; the Cloud transition is a separate private change.
## What Changed
- Package a standalone maintenance entry that starts no normal app
routes or execution workers.
- Coordinate the existing agent-file, run-evidence and skill migration
owners. Follow every page and verify shared bytes before final
eligibility.
- Require stopped writers. Refuse active execution, missing local bytes,
local workspaces and external instructions.
- Read the existing decision key without generating a replacement.
Preserve normal key creation for ordinary server startup.
- Require an attempt token for status and export. Export the original
key only after successful migration. Keep status and errors free of
secret material.
- Document the Cloud ownership boundary, deadline, retry and retention
requirements.
## Verification
- Full workspace typecheck and build passed before the review fixes. The
final server TypeScript compilation passed after correcting Drizzle's
optional predicate type.
- Production runtime bundling passed: 2,702 inputs, 122 chunks,
including the maintenance executable.
- The focused suite passed 21 cases using PostgreSQL and real
S3-compatible storage. It covers multi-page migration of 26 agents,
exact reads after deleting the predecessor home, corrupt-object
rejection, active/missing-data refusal, and the bundled plain-Node
process with authenticated export and clean SIGTERM exit.
- Regression cases refuse unsaved terminal working copies, require
explicit stopped/cleaned receipts for unavailable copies, clean 101
expired traces through the owning service, and enforce the deadline
after migration succeeds.
- Initial integration attempts exposed fixture cleanup ordering
problems; corrected runs pass. CI also caught a Drizzle predicate type
error, now fixed and compiled locally.
- All repository CI gates passed on
|
||
|
DottaandPaperclip
|
73cd8eb9a5 |
fix(server): deliver answers promptly during web handoff (#18530)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Its web servers can overlap while Railway replaces one deployment.
> - An answer can reach the replacement while the original server owns
the native execution.
> - A competing delivery claim can make the owner miss the first
notification.
> - The normal retry interval then delays the answer by 30 seconds.
> - This pull request gives pending answers a one-second retry in
ephemeral mode and verifies delivery between separate processes.
## Linked Issues or Issue Description
Refs #18518 and #18526. This closes a measured native-question delivery
delay during control-plane web deployment overlap.
## What Changed
- Register question delivery through its owning service. Use a
one-second retry for ephemeral servers and retain the configured cadence
for ordinary servers.
- Keep the existing durable claims, response identity, authorization and
continuation behavior.
- Add a two-process PostgreSQL regression for a claim held by the
non-owning server. Verify one command, settlement, idle behavior and
reactivation by another committed answer.
- Keep the assembled startup fixture on the real registration function.
Update the deployment plan with current evidence and remaining
acceptance work.
## Verification
- The same local two-process race took 30.157 seconds before the fix and
1.197 seconds after it. The command sink is simulated; PostgreSQL,
notification transport, interaction response, delivery claim and worker
are real.
- The cross-process regression and assembled startup suite passed 122
tests. No further sweeps or pending inspections occur after settlement.
A second answer wakes the owner through PostgreSQL without a test-owned
sweep.
- Server TypeScript compilation passed. All PR CI gates passed on
canary/v2026.1101.0-canary.8
|
||
|
DottaandPaperclip
|
16787e4167 |
Qualify rolling web deployments and preserve Stop through handoff (#18526)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The web server owns request handling and remote execution coordination. > - Deployment must start a ready replacement while the old server still serves. > - An outage test does not prove that two live servers can overlap safely. > - This pull request adds a separate Product E2E cell for that boundary. > - The existing run, file, recovery, and retained-session checks still apply. ## Linked Issues or Issue Description Refs #18518. The runtime prerequisite is merged; this PR qualifies its ephemeral server and deployment contracts. ## What Changed - Add the explicit `rolling-web-deployment` Codex Daytona workflow. - Start independent A/B/C homes and compiled builds behind a test ingress. - Wait for readiness before routing, then drain and delete the old home. - Observe active authority before handoff and use the existing read-only recovery oracle. - Require ordinary reads and saves to succeed throughout replacement. Read back every accepted save exactly once. - Preserve the five-minute outage suites and their existing validators. - Add a model-free supervisor smoke and an unready replacement control. - Require graceful exit before deleting an old home; forced cleanup cannot satisfy the oracle. - Preserve a durable Stop through lease release/expiry until the recovered execution owner acknowledges it. Permit only fenced reattachment of the original execution for Stop cleanup; submit no new turn. - Exclude events saved by A during shutdown from evidence of B making progress. ## Verification - E2E TypeScript check passed. - Affected harness checks: 143 passed. Catalog, fixture and report checks: 120 passed. - Actual supervisor smoke passed two replacements: 436 reads, 45 saves, zero request failures, and confirmed owned resource cleanup. - B became ready in 18.7 seconds; C became ready in 109.2 seconds. A prior C timeout remains failed evidence. - Fresh browser proof: 11 checks passed, 246 reads, 50 acknowledged saves, zero failures, 1,058ms reconnect, both old homes deleted and cleanup confirmed. - Stop lease-gap regression: three PostgreSQL delivery tests passed with a simulated provider edge. Current server TypeScript build/vendoring passed. - Full harness: 5,814 passed, 17 failed, four skipped. Two shutdown races passed the focused file; two report timeouts passed an isolated 42-test rerun. Onboarding/workflow-security/Pi/date-expired fixtures are unchanged on inspected master; the host embedded PostgreSQL fixture remains a setup limitation. This is not a green full local suite. - CI at `172509b8be` passed all gates. Greptile is 5/5 with every review thread resolved. The earlier access-service race failure passed its isolated regression and the subsequent full CI. - Final review corrections: six focused executor cases, three PostgreSQL command-delivery cases, 19 harness cases and server/E2E typechecks passed. - Three live attempts failed before agent creation: two harness setup gaps were fixed, then provider authentication failed. The saved OpenAI key returns 401; a refresh is pending. The Daytona probe reported cleanup uncertain, and no sandbox with that attempt label was found through the authenticated staging manager. No live model execution is claimed. - Live Runner overlap, released-container qualification and hosted deployment proof remain pending. Merging this opt-in qualification lane does not claim those results. ## Risks - The source lane shares workspace dependencies and does not establish container packaging correctness. - The live workflow needs an explicitly owned S3 bucket, PostgreSQL, pinned remote image and provider credentials. - Qualification cleanup stops all owned processes after failure. A failed replacement must not be reported as a pass. ## Model Used OpenAI Codex, GPT-6. Tool use, code execution and repository review. The runtime does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1101.0-canary.7 |
||
|
DottaandPaperclip
|
2b2b5a43b7 |
fix(daytona): restore the recorded directory for file recovery (#18527)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A stopped sandbox can hold workspace files that still need a durable copy. > - File recovery must reopen the exact allocation and directory recorded by the run. > - Workspace preparation can place that directory below the allocation root. > - Daytona resume returned the allocation root even when recovery needed the recorded child directory. > - This pull request lets the file-recovery owner request that exact directory. > - Ordinary resume keeps its allocation-root behavior. The recovery source checks stay strict. ## Linked Issues or Issue Description **What happened?** A stopped native run had a prepared remote workspace. The recovery service stopped and retained its allocation. The next file-recovery step failed with `source_binding_invalid` and `reopened_cwd_mismatch`. Daytona resumed the correct allocation and matched its sentinel, but returned the allocation root instead of the workspace directory recorded on the lease. **Expected behavior** The file-recovery owner must reopen its exact recorded directory without creating an allocation, an empty directory, or an agent turn. Ordinary execution resume must keep returning the allocation root for normal workspace preparation. **Steps to reproduce** 1. Prepare a task workspace below a Daytona allocation root. 2. Stop and retain the allocation with unsaved workspace files. 3. Request source-bound file recovery. 4. Before this change, the provider returns the allocation root and the strict cwd check rejects it. **Paperclip version or commit** Reproduced from `050365e09d88411960fc3bacc1c08c46c0d9b6fa`. **Deployment mode** Docker with the bundled Daytona provider. Local tests use synthetic provider boundaries and disposable PostgreSQL. Refs: #18517, #15749. Related retained-export work: #17916. That work does not change the Daytona resume handler. ## What Changed - Add an optional driver-only `requestedCwd` resume field to the SDK contract and existing host forwarders. - Set the field only after the owned file-recovery path verifies the recorded source. - Require Daytona to match the requested directory to the recorded lease directory. Require a normalized absolute path without NUL bytes. Keep the allocation and sentinel checks. - Keep ordinary resume unchanged when the field is absent. - Add provider, host, and real workspace-preparation regressions. Document the separate directory contracts. ## Verification - The original provider regression failed for reusable and ephemeral leases. The scoped regression also failed to read a real retained file from the wrong root. Ordinary resume controls passed before the fix. - Current provider handler: **345/345 passed**. This includes exact-directory, malformed, missing, foreign, sentinel, reusable, ephemeral, ordinary-root, and real-file cases. - Removing only the owned producer's cwd request makes its real PostgreSQL regression fail with the original `reopened_cwd_mismatch`; the source file was restored exactly afterward. - Current host selection: **10/10 passed** against disposable PostgreSQL 17. It covers the inspected owner, sandbox forwarding, generic plugin forwarding, normal resume without the field, same and later task workspace preparation, and ownership rejection. The same-workspace control retains saved bytes and does not append another workspace path. - Current SDK and Daytona builds, server build, module boundaries, maps, and diff checks passed. The server used its normal unchanged-Runner cache. Its canonical input digest matched the previously verified build, and the canonical assessor accepted the copied original stamp and outputs. The changed SDK and server compiled from current source. - Local package-wide validation exposed two unchanged 30-second timeouts in the 145,000-member tar-listing cases. A single-file control reproduced both timeouts. That test mocks this provider module. The failures remain recorded; no timeout changed. The earlier package run had 497 passes, two failures, and seven skips. It is historical evidence from the abandoned broader resume experiment, not final-source package acceptance. - Initial host attempts timed out during database setup before any assertion. The final host selection used the existing supported pre-migrated disposable template. No test timeout changed. - Greptile reviewed exact head `8d0e577d1234ad394887255f13707c010dee6d24`: **5/5, no actionable issues, zero review threads**. [Current review](https://github.com/paperclipai/paperclip/pull/18527#issuecomment-6555253315). - The official [Ponytail review](https://github.com/paperclipai/paperclip/pull/18527#issuecomment-6555237850) also assessed exact head `8d0e577d1234ad394887255f13707c010dee6d24`: **5/5, no findings**. Its check is terminal neutral; its completed source assessment is not a test-execution claim. - [Primary CI](https://github.com/paperclipai/paperclip/actions/runs/41286802674) initially lost its spot runner in the UI/CLI workspace lane. The log records an explicit shutdown signal and no test assertion result. UI, CLI, lane scripts and dependencies are unchanged; the adjacent identical-source baseline lane passed. The single failed-lane retry passed at the same source. The initial failure is retained. All exact-head CI gates are now green. - No live repair or image deployment is claimed by this PR. Commands: ```sh node packages/plugins/sandbox-providers/daytona/node_modules/vitest/vitest.mjs run --config packages/plugins/sandbox-providers/daytona/vitest.config.ts packages/plugins/sandbox-providers/daytona/src/plugin.test.ts node node_modules/vitest/vitest.mjs run --project @paperclipai/server server/src/services/workspace-recovery.test.ts server/src/__tests__/environment-runtime.test.ts -t 'recovers a protected retained source|owned export directory|prepares the (same|later) task workspace|does not reopen a retained source|resumes only the exact stopped sandbox lifecycle|explicit export directory' pnpm -C packages/plugins/sandbox-providers/daytona build pnpm --filter @paperclipai/server build node scripts/check-module-boundaries.mjs node scripts/check-maps.mjs git diff --check ``` ## Risks - A provider that does not support the optional field can still return a different directory. Recovery keeps failing closed on that mismatch. - Ordinary resumes omit the field. The tests cover the same workspace and a later task so that normal preparation does not gain an extra path prefix. - The field grants no source ownership, stop proof, file-persistence proof, or execution authority. Existing allocation, company, task, run, claim, and cwd checks remain in force. - No schema migration, retry-policy change, model replay, or historical-row repair is included. Existing failed attempts remain historical facts. Deployed file recovery still needs its normal admission and independent persistence and physical-stop receipts. ## Model Used - OpenAI GPT-6 through Codex. Used reasoning, source inspection, tool use, and code execution. The exact serving model identifier and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7e595fd055 |
test(e2e): verify native hiring, delegation and teammate reuse (#18351)
## Thinking Path > - Paperclip manages real work by AI agents. > - Daily QA must verify that work on the staging deployment. > - Classic Agent Chat coverage already exercises hiring and teammate reuse. > - The deployment journey should share that behavior and its ownership oracle. > - Saved documents, exact callback replies and physical cleanup establish success. > - The corrected journey and complete daily suite now pass on QA. ## Linked Issues or Issue Description **Subsystem affected** Product E2E deployment journeys and their shared browser controls. **Problem or motivation** QA must prove that one newly hired teammate completes two different tasks and returns both saved results to the lead conversation. Names, chat claims and successful-run counts alone cannot establish authorship or completion. **Proposed solution** Reuse the classic hiring oracle and production composer. Verify immutable creation audits, initial assignment, current project and ownership, saved document revisions, exact native authors, durable callbacks and their own task links after reload. Bound execution to five lead turns and two child turns. **Alternatives considered** A separate QA implementation would duplicate the classic journey. Name matching or later reassignment would accept broken initial ownership. Links elsewhere in the conversation cannot substitute for either callback's own result link. **Roadmap alignment** This verifies existing features. Related merged repairs: #18389, #18465, #18459, #18463, #18502, #18503, #18504, #18506, #18507 and #18514. Separate dependency #18517 repairs file preservation after lifecycle cancellation and is merged as Core `050365e`, which also contains #18514. This primary PR now differs from that protected mainline in only 16 unchanged H harness/documentation paths; it adds no application production delta. The measured QA deployment serves Core `050365e`, with its matching bundled provider pack and unchanged approved model/ROM. ## What Changed - Add the shared `hire-delegate-reuse` journey, catalog, types, dispatch, launch fingerprints, journey-feature entries and documentation. Hire one teammate, delegate a checklist, then reuse the same teammate for a saved review. Require seven canonical native runs: five lead and two child turns. - Require exact company, lead, chat and native creation receipts. Both tasks must be created for the intended teammate using immutable creation-audit assignee fields. Independently verify current project and ownership. Creation-time project is not recorded in that audit and is not claimed by this assertion. - Verify both current document revisions belong to the exact child runs and preserve the original checklist. - Use ordinary production rich-editor agent/project autocomplete controls. Verify exact reference chips, then send once. Preserve the shared default sender. - Require each durable, run-owned callback reply to contain its own exact completed-task link. Collect links through the actual production Markdown renderer and issue-reference decoration, then apply the existing company-scoped route authority. An unrelated table or code fence cannot hide a rendered link; foreign, wrong-task and code-only links remain failures. - Preserve approved connection, account, grant and responsible-user attribution. Verify the child's independent installation and retain the immutable binding snapshot. Do not copy credentials or broaden grants. - Record ownership before later observations can fail. Retry only typed transient observation GET failures. Preserve authority and cleanup failures. - Retain all 14 daily journeys, the model matrix, deadlines, seven-run cap and 500-cent aggregate allowance split 300/200. ## Verification - **Current composed head:** `afa3f5552ab0c1a9f0ee7327dfa5665d57cc7bcb`. It normally merges mainline `1e8e322` and exact #18517 source `adb1d4e`, retaining the intermediate `b2d9eb4` and `6fd1042` ancestry. The only merge conflict was the Playwright support registration; it is resolved to the exact previously qualified union. All 16 H harness/documentation paths are byte-identical to `5c0830f`; all four #18517 paths are byte-identical to `adb1d4e`. Every other tracked path matches actual protected mainline `050365e`. A clean merge preview of `050365e` and this head equals the current head tree, so no additional source commit is needed. GitHub's three-dot display still lists the four already merged helper files because #18517 used a squash merge; those blobs equal protected mainline and add no net application change. No journey, oracle, cap or model change accompanies this composition. A final clean merge preview on intervening mainline `006d5691a6245d5a710e25ba3df6d609754e163a` preserves all unrelated changes and adds exactly the same 16 measured H blobs, with no net App production delta. That later mainline is not represented as the AppCore measured by the QA run. - **Local qualification:** 149/149 focused tests across six suites, including the report-catalog case, and 11/11 real Chromium checks passed on `b2d9eb4`. All H modules, browser inputs and parser/package dependencies are byte-identical at `afa3f55`; those unchanged suites were not repeated. Current-source transitive types, maps, module boundaries and diff checks pass. Official Node24 QA runtime imports pass again using exact public `afa3f55` and private `8671fa9`, with a clean credential-free environment and network disabled. The initial local report-catalog failure is retained: the child `tsx` process was denied its local IPC socket before importing the report or writing an artifact. The same source and selected case pass after correcting only sandbox IPC permission; no source workaround or deselection was used. - **Current canonical CI and reviews pass.** [CI 41275109691](https://github.com/paperclipai/paperclip/actions/runs/41275109691) completed successfully at exact `afa3f55`: all 59 contexts are terminal and clean (55 success, three skipped, one neutral). Fresh exact-head Greptile and official Paperclip reviews both scored 5/5; all 27 review threads are resolved with pagination complete. The official review is static and its check is neutral; it did not execute tests or live QA. A separate exact-head independent composition assessment found no actionable issues at 4/5 confidence and performed no live actions. **Fresh QA qualification passed: all 14 daily journeys, including the corrected H7, passed against deployed Core `050365e`.** Prior check conclusions and review scores retain their actual heads. The previous `5c0830f` completed all 59 contexts cleanly, actual exact-head Greptile and official Paperclip reviews both scored 5/5, and all 27 review threads were resolved. Its separate independent journey-source review at `2b803f08` remained 4/5 with a unit-helper authorship caveat. Intermediate `b2d9eb4` received an actual exact-head Greptile 5/5 with zero open findings, then was updated to compose the valid dependency review fix before live QA. That score and its pending CI are not attributed to `afa3f55`. - Existing causal controls remain unchanged: invalid immutable assignee, either missing callback link, foreign/code/wrong-task links and an earlier handoff fail against the relevant prior source. Renderer controls prove that tables and fenced code elsewhere in a valid callback do not hide its actual task link. Both composer and document-navigation browser registrations remain present. - #18503 merged after its actual QA document case passed four canonical checks and five phases, positively established zero native work, and completed cleanup. Its measured files are unchanged in this composition. #18506 and #18507 merged after their own checks; their causal worker-settlement controls remain intact. - #18514 is merged as `1e8e322`. It reproduces the selected-task metadata-only-project startup failure against deployed `756599e` and passes 19 real-PostgreSQL selected controls after repair. Both direct and assembled baseline controls fail before it. Real source, explicit selection, policy and workspace-isolation guards remain enforced. It is included in measured Core `050365e`; fresh H7 reached and completed both delegated native tasks. - #18517 initial source `6fd1042` passed 320/320 focused and sibling Stop/recovery tests, including 20 preservation controls. Its actual official review was 4/5 and found a valid follow-up: the new live-controller guard should wait for actual lease expiry rather than repeat locked checks every 30 seconds. Both an already-live lease and a real under-lock renewal reproduce that issue against `6fd1042`. Corrected source `adb1d4e` passes all 22 preservation controls and 124 directly affected tests across three suites, exact-head server build/stamp, typecheck and structural checks. It defers before locking and rechecks renewed expiry afterward, preserving the transactional owner guard. The old score/failure and successful owner tests remain attributed to their actual sources. Independent exact-head review of `adb1d4e` found no actionable issues at 4/5 confidence. This is local/source evidence, not physical QA recovery. Its current [CI 41274963476](https://github.com/paperclipai/paperclip/actions/runs/41274963476) completed successfully with all 59 contexts clean; both actual current reviews are 5/5 with zero unresolved threads. It merged normally as `050365e` after those gates. QA now runs that image. The historical allocation is positively stopped and retained; its files-only export remains a separately tracked repair, not a passing-test assertion. - The historical [first H7 pass](https://pages.paperclip.ing/evals-qa-smoke-41226065043-1/) remains attributed to public `5d3f143`, private `dabad62` and App `b7f1c37`: all checks and five phases passed, seven native runs succeeded, both saved documents and exact callbacks were verified, and physical cleanup completed. An optional runtime-metadata receipt was inconclusive and was not grade authority. Material failed attempts remain failed: | Attempt | Actual result and correction | | --- | --- | | [41231853638](https://pages.paperclip.ing/evals-qa-smoke-41231853638-1/) | 13 passed; hiring omitted initial teammate/project assignment and started extra work. The production composer repair retains the ownership negative. Later recovery stopped physical work without changing the failed grade. | | [41238711372](https://pages.paperclip.ing/evals-qa-smoke-41238711372-1/) | All seven native runs succeeded, but the final callback had no durable chat reply. Cleanup passed. #18502 repairs premature completion-outbox retargeting; the fresh measured H7 now verifies both durable, run-owned callback replies and their own task links. | | [41253428446](https://pages.paperclip.ing/evals-qa-smoke-41253428446-1/) | 12 passed; hiring rejected a mentioned metadata-only project before delegation, and document reopening hit an immediate-control-probe race. #18504 and #18503 repair those separate boundaries. Historical missing-task absence remains quarantined; physical termination does not manufacture absence evidence. | | [41266385118](https://pages.paperclip.ing/evals-qa-smoke-41266385118-1/) | On `5c0830f` and App `756599e`, 13 passed, including the exact composed document helper. Hiring created the correctly assigned task, but its child admission failed before runtime selection because the organizational project had no repository source. #18514 repairs that selected-task boundary. The cancelled native lead retained an active prepared-workspace lease; cleanup 41271035231 failed with `owned_work_stop_unconfirmed`. #18517 addresses its file-preservation admission. Launch 1110, original evidence and automatic Slack FAIL remain failed. | | [CI 41257779002](https://github.com/paperclipai/paperclip/actions/runs/41257779002) / [CI 41263476916](https://github.com/paperclipai/paperclip/actions/runs/41263476916) | Existing enrollment and snapshot-retention fixtures observed intermediate I/O before worker settlement. #18506 and #18507 have controlled causal failures, complete-settlement observers and preserved deadlines/negative controls. Runtime scheduling behavior is unchanged. | **Fresh measured acceptance:** [QA run 41285131220](https://github.com/paperclipai/paperclip-cloud/actions/runs/41285131220) and its [permanent report](https://pages.paperclip.ing/evals-qa-smoke-41285131220-1/) passed **14/14**, zero failed or incomplete, with all five phases and complete cleanup for every case. The strict verifier independently matched original grades, current provenance and completed manifests: public harness `afa3f5552ab0c1a9f0ee7327dfa5665d57cc7bcb`, private workflow `8671fa93e27501fabbfffa377bec6dea453b5db7`, AppCore `050365e09d88411960fc3bacc1c08c46c0d9b6fa` and matching Cloud image/provider pack. All model, ROM, deadline and run-budget controls remained unchanged. All three workflow jobs completed successfully; the automatic Slack notification reported PASS with links to this run. The passing TestOps launch was closed normally after evidence verification. The hiring case passed all 11 check entries across nine IDs and five phases, with exactly five lead and two child native runs. Both saved documents match their live current revision/body hashes and exact hired-agent run attribution. The immutable original checklist remains intact; the same teammate authored the review. Both own callback replies contain their exact delegated task link. Independent canonical readback confirmed all seven native runs succeeded and committed their accepted result, assessment and decision, with no live owner and zero pending or failed effects; each of the seven exact owned leases has its own bound DESTROYED receipt and successful cleanup. All seven final snapshot operations match their source run, workspace, task, epoch, SHA, generation and export location. The four earlier lead snapshots are retained SUPERSEDED ancestors of the current committed snapshot; the latest lead and both child snapshots are COMMITTED. The initial diagnostic required current COMMITTED status for every historical snapshot and correctly remained false until the separate lineage diagnostic established this supported supersession. The product oracle and original grade were not changed. Canonical Suite metadata is `qa-basics`; the daily plan was saved with its standard suite-only query and synchronized, retaining all 14 ACTIVE cases. The older failed allocation remains separately stopped, retained, isolated from reuse and owned by terminated actors. That permits fresh campaigns without replaying old work. Its same-allocation files export is still unresolved and has separate bounded files-only recovery work; it is not claimed by this successful campaign. Original failed grades, reports and automatic FAIL notifications remain unchanged. No failed result was closed, relabelled, hidden or retried unchanged to obtain green. ## Risks The journey requires Agent Chat, normal hiring permission, a scoped lead and a provider that finalizes owned resources. Missing or extra work, wrong ownership, absent saved documents, missing callbacks and unconfirmed cleanup must fail. The deployed app and bundled runner pack must match the verified Core. The old run's cancellation ACK and terminal database row do not prove physical termination or saved files. Its coordinator has no accepted result or decision; pending-effect absence cannot be inferred from that missing decision. File recovery must preserve the exact allocation and original destroy disposition without a provider turn. Current source checks, reviews, fresh H7 and full-14 aggregate all pass. Canonical Suite metadata and actual 14-case daily-plan synchronization were verified after closing the genuine passing launch. The original failed allocation still has a separate files-export debt; its STOPPED receipt is not a SAVED or DESTROYED claim. ## Model Used OpenAI Codex, GPT-6 family. The session does not expose a more specific backend model ID or fixed context window. Used reasoning, repository tools, TypeScript and browser test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge - [x] Historical passes and material failed attempts retain their actual source and outcome - [x] The composed document helper passed normal QA with complete cleanup on its actual source - [x] The corrected hiring journey passes fresh normal H7 with complete cleanup - [x] The active hiring case has canonical Suite and daily-plan membership after the passing launch is closed --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
006d5691a6 |
feat(server): support volume-free web server handoff (#18518)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its web server also owns controller admission and user commands. > - A rolling deployment starts a replacement before the old server exits. > - Shared database state alone does not preserve local agent files, browser chunks, or in-process commands. > - This pull request adds an opt-in ephemeral server contract and a local overlap test. > - The benefit is a web server that can relinquish traffic and local files without losing acknowledged work. ## Linked Issues or Issue Description Refs #18403, #18327, #18335, #18341. **Current behavior** The control-plane web server still has local agent-file authority and local command targets. A new container can miss saved bytes or send a command to the wrong process. An existing browser tab also needs chunks from its original build. **Proposed behavior** Eligible servers share only PostgreSQL, object storage and stable secrets. A replacement waits for readiness. The old server drains admitted requests, keeps owner commands available through handoff, and releases controller leases. Explicit maintenance adopts existing files before the volume is removed. **Reason and benefit** This is the application prerequisite for rolling app-only deployments. Schema migrations and permanent multi-replica operation are outside this change. ## What Changed - Store agent-file manifests in PostgreSQL and immutable bytes in S3. Treat local roots as disposable projections. Preserve existing permission checks and edit conflicts. - Add an explicit migration, eligibility and object-retention command to the production runtime bundle. - Add `/api/ready`, admitted-request drain, browser restart signaling and retained immutable UI chunks. - Carry transaction-aware work hints across controllers during handoff and active execution. Keep listeners closed when idle. - Deliver durable Stop and runtime responses to the execution owner. Recheck request lifecycle and access before response dispatch. - Reject local-only configuration in ephemeral mode. Keep the feature disabled for existing installations by default. - Add an isolated PostgreSQL/S3/proxy/browser deployment harness with independent server homes and retained failure evidence. ## Verification - All CI checks pass on `ce17b95019d3cf1cabd357d19b6d37f96782cc5c`: workspace typecheck, build, Runner checks, server/general tests and browser shards. Greptile reviewed the runtime changes at 5/5 with all findings resolved; the final commit updates maps and the plan. - Local targeted checks passed: coordinator 27; Stop/recovery 33; PostgreSQL command delivery 6; route/workspace regressions 8; bootstrap Stop 1; restart recovery 75. PostgreSQL command tests simulate the provider edge. - Local browser proof passed 91 reads and 19 saves; reconnect took 1,013ms. The tab opened Audit from S3 after deletion of A's whole home/build. This uses compiled workspace code and local-trusted auth, not released containers or live agents. - Local workspace typecheck/build passed; current server and runtime bundle passed. The earlier broad local suite was stopped after three hours and is not a passing receipt. Its retained failures and isolated reruns are documented in the plan. - Stacked #18526 adds stricter graceful-exit proof, an unready replacement control, live Runner qualification and a Stop handoff correction. Hosted enablement remains gated on those results and Cloud #1096. No hosted stack has been enabled. ## Risks - Apply migration 0422 before enabling the feature. Existing data must pass each owner's migration and eligibility checks. Do not discard a volume based on a single migration page. - Durable manifests remain authoritative after the opt-in is disabled. Rolling back to a binary that ignores them is unsupported. - Cold directory restoration and complete shutdown timing still need qualification before enabling a hosted stack. - Retained UI assets must cover the supported browser and rollback window. Cleanup is explicit; no new idle maintenance polling is introduced. - This change does not claim production zero downtime or authorize a fleet-wide enablement. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model ID and context-window limit are not exposed in this session. Used reasoning, code editing, shell execution and browser automation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>nightly/v2026.1101.0-nightly.0 canary/v2026.1101.0-canary.6 |
||
|
DottaandPaperclip
|
050365e09d |
Preserve native workspace files after lifecycle Stop (#18517)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent termination must stop execution and preserve unfinished workspace files. > - A native run can acknowledge a lifecycle Stop, then finish without an accepted result. > - The file preservation worker required a board actor even when the exact native Stop receipt was valid. > - This left a prepared remote workspace allocated without a file recovery action. > - This pull request admits the exact audited receipt for file preservation only. > - The existing recovery service can then save the files and stop the same allocation without another agent turn. ## Linked Issues or Issue Description **What happened?** A cancelled native run can have a terminal, unowned coordinator and a prepared remote workspace. Agent lifecycle cancellation records a valid run-only Stop receipt without a board user. The preservation worker rejects that receipt because it uses the board Stop predicate. The source lease remains active. A resolved execution watchdog does not supply the missing file recovery action. **Steps to reproduce** Seed a cancelled native run with a terminal-failure coordinator, no result or owner, an acknowledged and audited run-only Stop, and a prepared remote workspace. Omit board attribution. Start the registered cancellation preservation worker. Before this change, it records no workspace recovery marker. **Expected behavior** Record file recovery for the exact bound allocation. Keep the cancelled result, accounting, task state and watchdog history. Use the normal recovery service to stop, save and clean up that allocation without another provider turn. **Paperclip version or commit** Reproduced on source `1e8e32290c657227857aeda2620a73db52bab4f2` with real local PostgreSQL 17. The observed deployment uses a sandbox provider and remote workspace storage. Related current workspace work: #18514. Public PR and issue searches found no duplicate repair. ## What Changed - Expose the existing exact run-only Stop receipt check. Reject empty audit identifiers. - Use that receipt only in cancelled workspace preservation. Keep board Stop and continuation checks unchanged. - Defer preservation while the run still holds a live controller lease. - Schedule that wait at the actual controller lease expiry. A lease renewal observed under the preservation lock also sets the next deadline. - Add a PostgreSQL regression for the remote-workspace-only lifecycle record, resolved watchdog history, normal recovery admission, ownership controls, rollback and idle behavior. - Document the separate file recovery obligation and its physical cleanup limits. ## Verification - Before the fix: the registered-worker PostgreSQL regression fails because the file recovery marker is absent. - After the original repair at `6fd1042efd`: 20/20 focused tests and all five sibling suites pass, 320/320 tests. The exact-head server build and typecheck pass. - The repository reviewer then found that a live controller caused a fixed 30-second retry. Both the existing-lease and real row-lock renewal cases fail against that source. The new `adb1d4eccdd48be4c86906d5bd111a6e39f20191` schedules the actual recorded deadline before locking and after the locked reread. The lock still enforces the ownership guard. - At `adb1d4eccd`: all 22 focused PostgreSQL controls pass. The directly affected retention and board receipt sibling suites also pass, 124/124 tests across three files. They prove no intermediate reads or transactions before expiry, preservation after expiry, and no extra execution. Server typecheck and the exact-head server build pass. - Map checks, module boundary checks and `git diff --check` pass. - Run `pnpm exec vitest run server/src/services/native-runtime/native-cancelled-workspace-preservation.test.ts server/src/services/native-runtime/native-failed-workspace-recovery.test.ts server/src/services/acknowledged-native-stop.test.ts` with the supported external test-database variable set to a disposable local PostgreSQL database. The earlier broader run also covered run control and workspace recovery. - Exact-head [canonical CI](https://github.com/paperclipai/paperclip/actions/runs/41274963476) completed successfully. All 59 current-head checks are terminal with zero failures; required `ci / verify` and `ci / e2e` pass. Both [Greptile](https://github.com/paperclipai/paperclip/pull/18517#issuecomment-6553528545) and the [repository assessment](https://github.com/paperclipai/paperclip/pull/18517#issuecomment-6553542139) reviewed exact `adb1d4eccd` at 5/5, with zero findings or unresolved threads. The repository assessment's check concludes `NEUTRAL`; the validated assessment itself is 5/5. Earlier reviews, including the valid 4/5 deadline finding at `6fd1042efd`, remain in review history. The [earlier CI run](https://github.com/paperclipai/paperclip/actions/runs/41273965963) is also retained. No old result substitutes for a current-head gate. - Real sandbox recovery and physical cleanup remain unverified. Earlier failed QA outcomes remain failed. This PR changes no test grade or historical row. - Repo-wide local tests and typecheck were not run. The focused server checks cover this change; canonical CI supplies the remaining repository gates. ## Risks - The newly admitted receipt permits only file preservation. It cannot authorize a continuation, accept a native result or certify provider termination. - Recovery still requires exact company, task, run, lease, allocation and workspace bindings. Accepted results and live owners remain with their existing owners. - Provider or storage failure can leave recovery pending. The source must remain retained until saved-file and physical termination receipts exist. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI GPT-6 through Codex. Used reasoning, source inspection, tool use and code execution. The exact serving model identifier and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1101.0-canary.5 |
||
|
DottaandPaperclip
|
1e8e32290c |
Give metadata project tasks private remote workspaces (#18514)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A task can belong to a project without using that project's files. > - The specification gives a task without a selected or configured source its own private folder. > - The remote source planner instead rejects such a task when its project has no configured workspace. > - This stops an ordinary delegated task before native execution starts. > - This pull request selects an empty, task-owned remote source for that narrow default case. > - Explicit source choices, existing workspace authority, and repository requirements retain their current checks. ## Linked Issues or Issue Description Refs #18504 and #18351. The earlier reference-project fix covered additional project mentions. This fix covers the task's own organizational project. **What happened?** A task in a project with no configured workspace fails with `remote_workspace_source_unavailable`. Its project has no workspace policy, its four workspace selection and binding fields are null, and it requests no repository, branch, or local folder. The failure occurs before native runtime selection. A derived project folder exists in the project facade even when no source is configured; that derived path does not grant file access. **Expected behavior** Use the task's private remote workspace. Keep the project on the task for organization. Do not adopt the project folder, agent home, or another repository. Preserve explicit workspace and source requirements. **Steps to reproduce** 1. Create a project without configured workspaces or a workspace policy. 2. Assign a normal task in that project to a native sandbox agent. Leave all workspace selections and bindings unset. 3. Start the task with remote workspace storage enabled. 4. On the prior source, observe the source-unavailable error before allocation. On this source, observe a company-owned remote workspace whose `sourceIssueId` is the task and whose filesystem project and source IDs are null. **Paperclip version or commit** Reproduced against the deployed source planner at `756599e4ea069c9338c320fe8846523d3dfb92a0`. This branch starts at `987e0932bf66e703750cd04698b3c7e8993580a7`; the relevant planner and preparation inputs at that parent match the deployed source. **Deployment mode** Native sandbox execution with remote workspace storage. The regression tests use real PostgreSQL with the current migrations. ## What Changed - Allow an implicit empty source only after a company-scoped query confirms the project has no workspace rows and no explicit workspace policy. - Keep existing bindings, pending selections, explicit source choices, Git requirements, and local folder choices outside this default. - Add real PostgreSQL source controls and an assembled heartbeat preparation control through workspace reservation and the environment acquisition boundary. - Clarify the existing task-owned folder contract in the implementation specification. ## Verification - Causal RED: the new source control and the assembled allocation control both fail against the unchanged deployed planner with the observed source-unavailable error. - GREEN: 19 focused PostgreSQL controls pass across the source planner and assembled heartbeat preparation suites. The allocation control checks the persisted company/task owner, null filesystem project IDs, private remote root, original organizational project, and unchanged project and host sentinel files. - Negative controls retain configured repository sources and reject a configured source without a URL, an explicit missing source, an unknown selection, a required project policy, a required branch, an explicit local folder, a foreign project, and unsupported read-only project sources. - Current-source transitive TypeScript checks and `node scripts/check-maps.mjs` pass. The map check is structural evidence only. - Fresh exact-head Greptile is 5/5 with zero unresolved threads at `680461e180232d0f4a057528617e39b64b4f0e3a`. [Review](https://github.com/paperclipai/paperclip/pull/18514#issuecomment-6552946468). The official [Paperclip assessment](https://github.com/paperclipai/paperclip/pull/18514#issuecomment-6552931726) is 4/5 and remains recorded unchanged. A separate non-author source review found no actionable issue after the full producer/consumer trace, with manual confidence 4/5. Neither static review ran the tests or proved deployed native execution. - Full repository tests, build, and type checks passed in [canonical CI](https://github.com/paperclipai/paperclip/actions/runs/41269733360). All 59 checks are terminal: 55 success, three skipped, and the official Paperclip check neutral; no failed or pending checks. The successful build checked out merge preview `03dce9f9bc708e778dd214ca85dfa1ba58e0a806`, with parents current main `9dbbfa4fb298448db83ace946b338bff321d8b2a` and this exact head. Its four helper blobs match this head; the four added mainline document-harness paths remain unchanged. Local tests use source aliases with the current DB, shared, and Runner source, rather than borrowed generated declarations. - Actual QA staging qualification is pending deployment. The previous failed journey and cleanup evidence remain failed; these local checks do not relabel them. ### Review follow-up - Fresh workspace retry: the normal producer sets `taskDirectory:true` for this empty-source case. [Heartbeat](https://github.com/paperclipai/paperclip/blob/680461e180232d0f4a057528617e39b64b4f0e3a/server/src/services/heartbeat.ts#L2288-L2296) separates the saved filesystem project from the organizational project. [Environment selection](https://github.com/paperclipai/paperclip/blob/680461e180232d0f4a057528617e39b64b4f0e3a/server/src/services/heartbeat/environment-selection.ts#L506) selects the fresh task directory. Preparation passes that flag, and the planner's existing task-directory return runs before the new binding guard. The proposed fresh failure omits that producer flag. The existing fresh-intent and binding checks stay strict. This is source evidence, not an extra runtime test claim. - Empty policy object: the [public project policy schema](https://github.com/paperclipai/paperclip/blob/680461e180232d0f4a057528617e39b64b4f0e3a/packages/shared/src/validators/project.ts#L24-L43) requires `enabled:boolean`. The project PATCH schema makes outer fields optional and does not make nested policy fields optional. A clear uses `null`; `{}` is rejected. A defensive parser's treatment of unknown stored data does not establish a valid `{}` producer. Keep the narrow fallback closed for non-null policies. - Boolean grouping: JavaScript evaluates the existing predicate as `sourceId || (projectId && !implicitTaskDirectory)`. The explicit-source and metadata-default controls cover those branches. The optional parentheses suggestion does not identify a behavior change; this reviewed source remains frozen. - The official 4/5 assessment and its static verification limits remain visible. The independent manual review withdrew its initial fresh-retry concern after the full producer trace. Actual staging qualification still requires deployment and a new real journey result. ## Risks - The new query runs only for a source-less default admission. It reads one authorized project and uses `NOT EXISTS` for that project's source rows. It adds no recurring poll, write, provider call, or local folder access. - A project with an explicit workspace policy remains subject to the prior source requirement. This conservative rule does not reinterpret policy fields. - The allocation control uses the real planner and PostgreSQL reservation. It stops at environment acquisition, so it proves workspace ownership rather than provider filesystem execution. Live staging qualification is still required. - No API, database schema, permission, model, budget, run limit, or QA journey oracle changes are included. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use, and code execution. This session does not expose a more precise backend model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1101.0-canary.4 |