mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-31 06:04:11 +01:00
09c05c55050fede448196637cbaa543a154da76e
361
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Devin FoleyandPaperclip
|
5d79d88d68 |
fix(server): let settled native finalizations permit idle sleep (#18184)
## Thinking Path > - Paperclip manages AI agents and their work. > - Native runs retain finalization records for recovery and audit history. > - Idle sleep checks durable work before it stops an instance. > - The current check treats every retained finalization as pending work. > - A committed record can remain after its run and status effects have finished. > - This pull request recognizes complete records without deleting them. > - Instances can sleep while unfinished execution and recovery work remain protected. ## Linked Issues or Issue Description **What happened?** A retained `native_run_finalizations` row prevents idle sleep even after the run has finished and its status decision has been applied. The current gate checks whether any row exists. **Expected behavior** Complete finalization history should remain available without counting as pending work. Active leases, retries, accounting, recovery, and incomplete status effects must still prevent sleep. **Steps to reproduce** 1. Finish a native run and retain its committed finalization. 2. Confirm that its accepted result, assessment, applied decision, and delivered effects agree. 3. Request an idle report under a valid quiet admission hold. 4. Observe that the retained row reports background work before this change. **Paperclip version or commit** Reproduced on master at `cd0c86e5e1d427c8fa8aa8c3a9499391ee9b425b` with a local PostgreSQL fixture. **Deployment mode** Server with PostgreSQL. No provider calls are needed to reproduce the gate failure. Related PR and issue searches found no duplicate for this check. ## What Changed - Replace the blanket finalization-row check with a completion proof bound to the same company, issue, run, accepted result, assessment, and applied decision. - Require the owning run's successful workspace-finalization receipt; retain unresolved leases, retries, accounting, recovery, and status effects. - Share final-reply candidate discovery with the presentation recovery worker, so a completed run cannot hide a required reply. The pure query stays with native presentation; idle inspection uses it in the existing read-only transaction and five-second statement limit. - Register open warm sessions and in-flight close/checkpoint promises with local idle inspection. Historical controller/provider process metadata remains preserved. - Add PostgreSQL and local-warm regressions and document the retained-history rule. ## Verification - PostgreSQL sleep/finalization and chat-monitor continuation suites: 300 tests passed, including 56 finalization cases. - Local-warm regression: passed; an open session and its pending close/checkpoint both block sleep until retirement completes. - Server typecheck and server build: passed after review fixes. - Full workspace typecheck and build: passed before the review additions; hosted checks cover the updated revision. - An earlier local full invocation encountered two chat recovery failures that passed in isolation, and two existing Git workspace fixture failures caused by Xcode system Git initializing `main`. Those two pass with `init.defaultBranch=master` scoped to the test subprocess. That superseded full invocation was stopped for review revisions; no full local pass is claimed. No machine configuration or unrelated source was changed. - Fresh Apex review: 5/5 on `6846203fe2383bae1ede888fbaaedf83e9946e73`, with all findings addressed. - Hosted CI: fully green on `6846203fe2383bae1ede888fbaaedf83e9946e73`, including the final verify gate. Twelve jobs were interrupted by a simultaneous runner shutdown; rerunning the failed jobs completed successfully. - No hosted deployment of this change is claimed. ## Risks This changes an automatic-stop gate. An incomplete proof could stop an instance with pending work. The SQL fails closed and retains the existing admission, local work, plugin, schedule, backup, and provider-allocation gates. PostgreSQL cases cover unresolved leases, retry/recovery intent, accounting, projections, rejected results, unapplied decisions, and pending effects. No migration or history deletion is required. Reverting the predicate restores the conservative gate. ## Model Used OpenAI GPT-6 through Codex, with repository tools and local PostgreSQL verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky Leach
|
e2cfdc7e0c | feat(plugins): expose GitHub credential broker to task providers (#18178) | ||
|
DottaandPaperclip
|
9f3fc2094f |
Fix shared workspace seed capacity and trace deferred copyback (#18181)
Apply the existing company quota to company-shared workspace seed buckets. Preserve individual seed limits and recheck size after verification. Trace each deferred native workspace finalization check and its resumed copyback under the same hashed run ID. Keep existing admission, ownership and retry behavior. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f1911a4c53 |
perf(skills): move repository packages out of Postgres (#18082)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/205/changes/paperclip/205 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents receive selected company skills before each turn. > - Imported skills can need files from the whole source repository. > - The current path stores those files in Postgres and reads large JSON payloads during preparation. > - This pull request stores verified packages in the configured file or object store and keeps preparation queries small. > - It also converts old rows automatically and restores missing files from the exact saved commit. > - The result reduces database traffic while preserving skill versions and repository dependencies. ## Linked Issues or Issue Description **What existing behavior does this improve?** Company skill import, version storage, and preparation before agent turns. **Current behavior** Repository snapshot bodies live in JSONB. Skill preparation can read those bodies again even when the files already exist on disk. Historical packages and native assets also need a bounded cleanup path. **Proposed behavior** Store package archives and manifests through the existing storage service. Keep version identities, receipts, hashes, and recovery metadata in Postgres. Convert legacy rows after the stored package passes read-back verification. A missing package can use legacy bytes or its exact Git commit under current repository authorization. No separate data-migration command is required. **Reason and benefit** The warm path sends no repository bodies through SQL and reads no object bytes. Cleanup runs through the existing deadline coordinator. Current versions, active runs, explicit pins, and authored content retain their protection. **Breaking changes** The normal application schema upgrade is required. Single-version and skill-file responses keep their existing shape. Version lists, current-version summaries, and test-run version summaries now contain file names and kinds without file bodies. Diff and restore fetch selected versions through the existing single-version endpoint. Older server builds must not write after conversion unless they support the new package format. The storage override and rollback limits are documented in `doc/skill-package-storage.md`. **Additional context** Related work: #15731 preserves repository dependencies; #17923 changes version-cache scheduling; #18080 changes delivery to remote Codex homes. This change owns package persistence, acquisition, and expiry. It does not replace those delivery or scheduler changes. ## What Changed - Add company-scoped archive receipts, exact-commit recovery recipes, migration deadlines, and a durable object-deletion queue. - Add bounded archive upload, read-back verification, extraction, concurrent request coalescing, and verified local caches. - Convert old snapshots and versions automatically. Keep legacy bytes when verification fails. - Recover missing files through current source authorization and the original commit. Validate repository identity, file hashes, modes, and the aggregate hash. - Expire unused recoverable Git packages and native assets after 30 days. Preserve active pins and the durable copy of authored or unproven content. - Keep warm preparation queries metadata-only. Bound file digest caches and avoid repeated body hashing when file identities are unchanged. - Extend the existing storage contract with cancellation. Register package work with the existing coordinator. Optional conversion retries retain the original bytes and may wait for ingress without blocking sleep. - Add regression tests, operating guidance, a deep trace, benchmark receipts, and an optional maintenance command. ## Verification ### Final master integration and review repair — October 29, 2026 Current head: `a6065424249e763dfeac37161f1b52a9f25bd9b7`. Commit `f8bbdd3dbd` resolves the conflicts with master `76a796e964`; the final two commits repair the Apex deletion findings below. - Keep both package-maintenance and version-retention notifications in their write transactions. - Keep the shared run-pin predicate and master's controller-lease deadline metadata. Package expiry reads the updated protected-version result. - Preserve every master migration entry. Generate migration **0415 (`0415_zippy_zemo.sql`)** after master's 0414. Its skill SQL is byte-for-byte unchanged from the previously tested idempotent migration. Retain only the five newest schema snapshots. Migration numbering and safety checks pass. - Repair failed file/folder deletion: preserve a complete temporary subtree until transaction commit, restore on upload or SQL failure, and restore callback failures before releasing the mutation lock. Both PostgreSQL regressions reproduced missing files before the fix and now pass. They cover binary data, executable permissions, empty folders, unlisted files, unchanged database/version rows, successful retry after upload-claim expiry, and historical reads. - Reconcile lost commit replies under the same advisory and row locks. Check the saved version and source identity before restoring. Keep committed deletions intact and preserve backups when the database outcome is unknown or a newer edit exists. Four real-PostgreSQL outcome tests pass; the prior code failed three of them. - Full workspace typecheck and production build pass on the final commit. All **204** focused PostgreSQL package/cache/retention/authoring and filesystem-boundary tests pass, with none skipped. - Apex review **18 passes 5/5**, with zero new comments on the exact final head. All **34** review conversations are resolved. - [Final-head CI](https://github.com/paperclipai/paperclip/actions/runs/40914116508) completes successfully on attempt 2. The head has **55 successful checks**, two neutral reports, two skipped checks, and a successful Snyk status. No checks are pending or failing. All eight browser shards pass. - Runner Vitest 2/2 timed out once in the unchanged ACPX controller-replacement fixture. The same job passed on the preceding commit. One unchanged CI retry passes all five ACPX recovery cases, **2,253** Runner package cases and **126** native server integration cases; their standard one and five skips remain. The initial timeout's root cause is not established. Local diagnosis also hit the fixture's eight-second session.open timeout with an authenticated Runner; no local Runner pass is claimed. - Both full local `pnpm test:run` attempts were stopped when Apex identified deletion defects and source repair became necessary; neither is claimed as a passing full local run. The final **204** focused tests, full workspace typecheck, and production build pass. Full-suite acceptance uses the final-head CI jobs. The superseded merge-head CI included runners terminated by shutdown signals; each source repair started a new full run. - Auto-merge is enabled for squash merge. The final head is conflict-free and all checks pass. GitHub's active master ruleset still requires a code-owner approval for the dependency files. The existing review requests are with `devinfoley`, `nickyleach`, and `forgottendev`; no policy bypass was used. - The earlier hosted stress and full native receipts below remain tied to `a2e597d800`; they are not relabeled as a deployment of this final integration. The earlier local PostgreSQL startup failure remains recorded below; PostgreSQL ran successfully for this integration verification. ### Full native skill execution — October 29, 2026 The full native-agent acceptance gap is closed on candidate `a2e597d800eb989d565780e7aa947d5368676444`. Two qualifying native turns used OpenAI API / `gpt-5.4-mini` in real Daytona sandboxes, on the same disposable hosted TEST tenant as the stress test. A follow-up on the same task consumed a newly saved skill version. The task text asked for the assigned skill workflow. It did not contain the proof nonce or expected receipt. The assigned authored skill contained a Python verifier and a random package-local nonce. The agent read the actual installed skill files and executed that verifier against the imported, automatically converted 1,360-file repository package. It checked the repository file count and reference/binary bytes, and directly executed the repository's executable probe. Independent verification matched the emitted JSON to the nonce and expected byte hashes. The agent persisted the same receipt as its task response; native finalization committed, workspace finalization succeeded, and task state became Done. | Qualifying turn | End-to-end run time | Recorded skills.prepare | Repository files | Binary hex | Shell receipt SHA-256 | |---|---:|---:|---:|---|---| | Initial task | 47.881 s | 1822 ms | 1360 | `00ff07800a` | `a1deb30dc095f2306c7aa9cc5ec396da12c68961cf5b809f4d7c20f7fc68493b` | | Corrected follow-up, new nonce/version | 37.974 s | 377 ms | 1360 | `00ff07800a` | `dbcdc33fa1376b4edc136feae1cdf770560a73292467e93619efdda20e4282a7` | Both successful shell receipts reported `probe: STRESS18082` and `assetSha256: e8ea7a2f15f7bb3b332a46adbef848465884c8996595aa234748befd544d1f3a`. Their proof nonces, selected version IDs and receipt hashes differed, and both ran on the same task. These are observed timings, not latency SLOs; they include real provider work. The service stress-test timings remain a separate measurement boundary. Earlier failures remain in the evidence: - The original five-minute Daytona allocation timeout never reached model execution. After normal owned-stack restore, its durable cleanup record was `expired` / `success`. The cause of that original allocation timeout remains unknown; a later success establishes recovery only. - Initial continuation setup used unsupported generic agent-update fields and returned HTTP 409 before dispatch. Setup was corrected to use the supported skill-assignment and versioned instruction commands. - The intervening completed native turn read repository bytes and ran the probe, but omitted the verifier JSON under the older test agent's overly-specific instruction file. Its strict receipt validator failed; it is not counted as a qualifying turn. Correcting the test agent's instruction file and sending an explicit task-scoped wake produced the qualifying follow-up above. - The pinned application's immutable rate card omits `gpt-5.4-mini`, so completed usage triggered the normal block-on-unpriced budget policy. Supported append-only accounting corrections estimated the three completed runs using the [official OpenAI model rate card](https://developers.openai.com/api/docs/models/gpt-5.4-mini): $0.75 input, $0.075 cached input, $4.50 output per million tokens, assuming standard processing. Estimates totaled 12.482655 cents. Original usage and missing-price records are preserved; the $1 hard stop and unpriced-usage guard stayed active. - Model shell attempts included a missing `xxd` command and incorrect guessed file paths. Those failed calls and subsequent successful actual verifier calls remain visible in complete event streams. No zero-tool-error claim is made. No product source changed during this continuation. The previously measured contention, cold-start and memory limits remain. The later merge resolution is covered by the separate integration checks above; these hosted receipts qualify their recorded candidate. ### Hosted stress test — October 29, 2026 **Hosted acceptance:** full native-agent execution and a changed-skill follow-up pass on the recorded hosted candidate. The final integration section above records the subsequent conflict resolution and its separate checks. Tested baseline `929e840595c8933ca485ac7a2e99413e1e11ca64` and this exact candidate sequentially on one disposable TEST tenant, with external Neon PostgreSQL and actual S3. The fixture contained **1,360 files / 22.4 MB / ten skills sharing one repository**. Measurements cover deployed skill-preparation and native-context services in an adjacent diagnostic process, not a complete agent turn. The container had **24 vCPU / 24 GB**. | Warm workload | Median before → after | p95 before → after | DB returned value bytes/preparation before → after | |---|---:|---:|---:| | One skill, serial | 1,060 → 125 ms | 1,303 → 155 ms | 22,610,389 → 1,771 | | One skill, four concurrent | 4,415 → 297 ms | 5,143 → 328 ms | 22,610,389 → 1,771 | | Ten skills, serial | 5,723 → 535 ms | 6,057 → 600 ms | 224,739,030 → 10,671 | | Ten skills, four concurrent | 24,410 → 1,986 ms | 24,833 → 2,221 ms | 224,739,030 → 10,671 | - Candidate warm preparation performed **zero S3 GETs**. Ordinary-pool queries fell from 16 to 9 for one skill and 61 to 9 for ten, excluding dedicated lock connections. DB numbers measure returned values before Drizzle decoding, not wire bytes. Baseline/candidate sample counts were 30/100, 32/100, 5/50 and 8/48 respectively; the small baseline ten-skill samples are comparative evidence, not SLO estimates. - **Application-UID soak:** 800 ten-skill preparations over 10.3 minutes, four workers paced by 2.5 seconds; **zero failures**, median **559 ms**, p95 **775 ms**, p99 **910 ms**, zero warm S3 reads. All 726 accompanying health/company-list requests returned HTTP 200. Their end-to-end p95 was 569/502 ms, including network latency. The probe overlaps most of the soak. Soak DB-response capture was limited to its first 60 seconds. - **Important remaining costs:** an unpaced 16-way ten-skill burst had p95 **8.32 seconds** because company-level preparation serializes. Cold preparation took approximately **1.1–2.5 seconds** for one skill and **7.4–10.0 seconds** for ten. Ten-skill diagnostic RSS remained about **1.17 GiB** (baseline 1.18 GiB), and the soak peaked at **1.35 GiB**. A concurrent whole-container sample was 3.45 GB. This does not establish suitability for a small-memory host or eliminate per-entrypoint native asset duplication. - **Upgrade without a manual data migration passed:** legacy snapshot JSON shrank from 22,465,691 bytes to `[]`, with a stored package and zero pending conversions. - **Recovery passed:** sixteen concurrent cold requests coalesced into two S3 GETs; full bytes/binary/executable modes matched; same-size corruption repaired; an injected storage failure preserved warm access and failed cold access safely. A disabled public-Git source denied recovery, and after the actual five-minute backoff its exact SHA reconstructed all 229 files. Cross-company descriptor access was denied. Private-Git credential revocation was not tested. - **Retention passed:** reclaimed 27,754,496 allocated cache bytes, deleted an unreferenced package row and both objects, preserved aged authored history, and expired/reacquired reconstructible Git content. A version pinned by the failed native run was correctly retained. Only owned fixture timestamps were aged. - **Browser UI passed:** authored skill creation, saved edit surviving reload, version history, and restoration of earlier content. - **Initial native attempt (superseded by the qualifying turns above):** one real run failed at Daytona allocation after five minutes (`setup_failed`), before any model execution. Provider cleanup also reported failure. That attempt does not count as a passing native turn. Its receipt and all evidence remain preserved; its cleanup later recorded success after normal restoration. - Initial A/B commands ran as SSH root. Diagnostic-owned lock/cache permissions caused one setup HTTP 500 and the first UID-1000 soak to fail; exact fixture paths were repaired. The final soak, retention and recovery ran as application UID 1000. A fresh UID-1000 one-skill batch also passed (30 samples, median 116 ms, p95 321 ms, 1,771 DB value bytes/preparation). Earlier harness-export, SSH, npm provenance, backoff and UI-wait failures remain recorded alongside their successful retries. - Private evidence includes raw timings, assertions, HTTP probes, screenshots and failure/cleanup receipts. Initial failed-campaign cleanup suspended and retained the owned TEST tenant and released the shared QA account lease at **15:22:41 UTC**. The tenant was subsequently resumed under its existing ownership for the native execution qualification above. Final cleanup completed at **15:53:09 UTC**: the account lease is released and the tenant is suspended/retained under the original October 30, 12:24:52 UTC retention deadline. Its separate stack lease remains held; no active or pending-cleanup environment lease remained. The original retention window is deliberately preserved even after successful continuation. - Earlier hosted-tested head: `a2e597d800eb989d565780e7aa947d5368676444`, integrated with master `929e840595`. The earlier merge conflicts were resolved at this head. [Current-head CI](https://github.com/paperclipai/paperclip/actions/runs/40884087629) passed; all checks are complete with 55 successes, two neutral reports, two skips, and no failures. [Apex review 15](https://github.com/paperclipai/paperclip/pull/18082#issuecomment-6480257258) passed 5/5 with zero findings on this exact head. All review conversations are resolved. - Ten real-filesystem boundary cases and server typecheck pass locally for the security fix. Coverage includes cache roots, descendants, individual files, symlink aliases, recursive ancestors, a future concurrent cache fill, and allowed sibling directories. [Current-head private-project integration tests](https://github.com/paperclipai/paperclip/actions/runs/40884087629/job/123483306405) passed all 29 cases, including authorized and unauthorized imports of a cached private package. Full typecheck, build, server suites, all eight browser shards, and Runner checks passed in current-head CI. Local database tests could not run because PostgreSQL could not initialize; this limitation is recorded below. - The earlier generated migration preserved the newest five snapshots. The final integration renumbers its unchanged SQL to 0415. Migration safety/numbering checks pass. Its SQL is byte-for-byte identical to the previously verified idempotent skill migration; repeated application on this latest schema was blocked by the same local database startup failure. - The broader verification before the final migration-only rebase passed 1,025 package, lifecycle, remote-launch, native-session, and asset-retention tests, 52 focused UI tests, full workspace typecheck/build, token gates, and map checks. The linked complete CI run passed before the latest focused fixes. - A disposable MinIO instance passed multipart upload, verified read-back, and cold restoration of an 18 MB archive. - Twenty warm samples with a synthetic 1,349-file, 22.8 MB repository retained five metadata queries and zero object reads. Median preparation was 256 ms for one selected skill and 691 ms for ten, with 5 ms emulated network delay each way. These are local measurements, not production latency guarantees. - No production instance was changed. Earlier verification failures and their disposition are retained below. <details> <summary>Earlier failures, review findings, fixes, and rerun evidence</summary> - The first CI pass found a stale inventory-span expectation and a missing repository-read span. Both are fixed; all three observability tests pass. One browser test copied page text into its YAML parser after reload; its first unchanged-head retry passed, but the race recurred after rebasing. The test now waits for the renamed URL and completed reload before copying the editor, retaining its original assertions. The earlier full local suite did not complete cleanly: it reported setup timeouts and assertions, while isolated relevant reruns passed. The complete current-head CI run closes that broad verification gap. No production instance was changed. - The first Apex review scored 2/5 and identified six issues. The fixes protect authored history even when it shares an in-flight Git upload, recheck upload expiry during orphan deletion, bound failed-upload cleanup, retain per-version kind/encoding metadata, add sanitized failure diagnostics, and correct the migration guide. The final focused run passes all 140 tests; the preceding eight-file run passed all 149 tests, including local storage and S3 diagnostics. Server typecheck and migration safety/numbering checks pass after the fixes. The second Apex review scored 1/5 and found unbounded maintenance storage calls and mismatched package/Git batch limits. Those are fixed with a 30-second cancellable cleanup deadline, shared 100 MiB/10,000-file scanner and storage limits, and bounded Git read batches. The proposed manual lockfile edit was rejected with evidence of the enforced CI-owned lockfile policy. All 19 storage/recovery unit tests and server typechecking pass. The new database-backed cleanup/conversion tests were locally skipped because the host exhausted its PostgreSQL semaphore capacity (confirmed directly by initdb). That gap is closed by current-head CI: [package-store tests](https://github.com/paperclipai/paperclip/actions/runs/40771320775/job/123109300889) pass all 18 cases and [package-worker tests](https://github.com/paperclipai/paperclip/actions/runs/40771320775/job/123109301157) pass all 7, with no skips in either suite. The corrected YAML browser shard passes. A separate planning-fixture test failed its cleanup DELETE with an activity-log foreign key violation after its behavior assertions passed; its one unchanged-head retry passed. The third Apex review scored 1/5 and identified a one-slot pool deadlock, projected path limits, cleanup holding the preparation lock during archive reads, and a publication/expiry race. These are fixed with transaction-local reads, prefix-aware bounds, isolated verification, and shared package row locks with a publication grace. The separate Paperclip review also found stale historical inspection, eager history downloads, and conversion retries blocking sleep. Those are fixed. Local PostgreSQL capacity recovered: the new database regressions ran, including the one-slot fork, paused storage during a warm turn, and a locked publication during expiry. The expanded run passed 144/145 tests; its remaining assertion still expected file bodies from the version list and was updated to fetch the selected version. The corrected company-skill suite passes all 81 tests, and the final package/projection run passes all 27. All 52 focused UI tests pass after updating their query-provider fixtures; the first UI run exposed a fixture remount, and the follow-up exposed an editing syntax error, both corrected. Full workspace typecheck and build, token gates, and map checks pass. The sandbox initially blocked the build tool’s local IPC socket; the authorized build rerun passed. Master then claimed migration 0401. The branch is rebased on |
||
|
Nicky LeachandPaperclip
|
247de89eae |
fix(runner): preserve warm tasks during recovery (#18169)
## Thinking Path > - Paperclip manages agent work and the lifetime of its execution resources. > - A warm native task can keep one Runner alive between successful turns. > - Foreground completion selects whether that resource stays running or stops. > - Background recovery must use the same selected policy. > - Recovery currently reads that policy only from workspace copy-back records. > - A task without such a record can be stopped immediately after foreground retention. > - This pull request saves the resource policy with the native runtime and permits the retained task to refresh its control connection. ## Linked Issues or Issue Description **What happened?** Background native finalization can stop a successful warm Runner when its run has no Core workspace-sync record. Foreground completion retains that same task. The stopped Runner cannot receive the later drain and suspend commands. Cold resume then correctly rejects its unproven checkpoint. Hosted diagnostics confirmed the recovery reconciler attempted environment release immediately after foreground retention. The provider connection returned HTTP 404 within the same second. A database-backed regression reproduced the stop without a workspace-sync record. A separate authorization gap also prevents the current retained task from refreshing its connection endpoint after the run succeeds. **Expected behavior** Foreground completion and recovery must use the selected resource policy. A successful warm task stays running until its idle shutdown can suspend it. Failed or cancelled work still stops. The current retained task can refresh its existing control route, but cannot prepare or submit new execution. **Steps to reproduce** 1. Select a warm native runtime on a reusable remote execution environment. 2. Complete a turn without a Core workspace-sync record. 3. Let native finalization recovery settle the still-active run lease before foreground retention finishes. 4. Before this change, recovery defaults to `stop_and_retain` and invokes the provider stop path. 5. Separately, request the retained task's connection endpoint after successful completion. Before this change, Core rejects that request. **Paperclip version/commit** The hosted failure used Core |
||
|
DottaandPaperclip
|
6f278dd6d7 |
fix(runtime): settle recovery lease and connection lifecycles (#18167)
## Thinking Path > - Paperclip manages AI agents and their work. > - A remote runner can survive a controller restart. > - Recovery renews its execution lease during slow preparation. > - The native executor then shares that renewal timer. > - Preparation kept its reference until all finalization finished. A tick after lease release treated completion as ownership loss and detached the retained runner. > - A Linux continuity test also found that buffered wire events could arrive before their connection was registered. Cleanup then kept a completed promise under an undefined key and never finished. > - This pull request transfers renewal ownership after execution acquires its reference. It also registers each connection before wire callbacks can run. > - Recovered turns can complete, retain the warm runner, and settle connection cleanup. ## Linked Issues or Issue Description **What happened?** A recovered remote run could complete, release its coordinator lease, then detach during instruction collection. The next turn could find the old runner still using the ingress port. Recovery then stopped because it could not prove process ownership. **Expected behavior** Preparation keeps the lease alive until execution owns renewal. Execution stops renewal before releasing the lease. Actual ownership loss still detaches the runner. Buffered wire callbacks must use the registered connection, and cleanup must finish. **Steps to reproduce** 1. Recover an in-flight remote runner after a controller restart. 2. Let the recovered turn complete. 3. Hold instruction collection open through the next lease renewal tick. 4. Send a follow-up that needs the retained runner. Related: #17439 introduced remote controller recovery. #18011 preserves checkpoint authority through warm preparation. This change fixes the lifetime of preparation's renewal reference. ## What Changed - Pass an internal lease-acquired callback from recovery through heartbeat dispatch. - Release preparation's renewal reference after the executor registers its own. Keep final cleanup for exits before dispatch. - Add a PostgreSQL regression for completion while finalization is still pending. Cover callback forwarding, acquire-before-release ordering, and failure before dispatch. - Register the authority connection before subscribing to wire events. Handle an immediate close before accepting buffered data. - Add regressions for a message or close delivered during subscription. - Extend Linux continuity coverage with preparation renewal and delayed instruction collection after recovery. - Update the continuity guide and session verification recipes. ## Verification - The new recovery regression failed before the fix because renewal detached the runner after lease release. The 76 recovery, renewal, and dispatch tests then passed. The final ordering and early-exit checks passed again after replacing fixed sleeps with controlled interval ticks. - Both buffered-wire regressions failed before the connection fix. All 170 control-plane tests passed afterward. Existing tests cover stale authority, ownership loss, pending operations, and closed ingress. - Linux ARM64 acceptance passed at `a7daefacceb8bd994d0b79d284cdf54f3e3829d1`, with Node 24.19.0, PostgreSQL, the real Rust runner, TLS ingress, and a deterministic Codex protocol fixture. The test interrupted the first turn, recovered it, held finalization open for a renewal interval, and completed two follow-ups. It retained one provider process, three turns in one history, and the same runner identity. Final suspension and connection cleanup completed. The ordinary warm-reuse case also passed three turns with one provider process. - Earlier Linux attempts completed the turns but hung during cleanup. A debugger found a fulfilled promise keyed by an undefined connection. The regression reproduces that defect. One later setup attempt timed out while staging an unstripped 373 MB debug binary. The passing run used the same binary with debug symbols removed. - Reproduce Linux acceptance with the `same-server` case in `doc/testing-native-remote-continuity.md`. - `pnpm -r typecheck`, `pnpm build`, map validation, and `git diff --check` passed at the final revision. The full local `pnpm test:run` command was attempted, but the serial run had not finished after 13 minutes. It was stopped after the equivalent CI lanes passed. It is not counted as a full local pass. - Greptile and the Paperclip review both scored the final revision 5/5 with no findings. All Runner verification lanes passed. The first CI attempt hit timing failures in voice recovery, agent YAML configuration, and the task-title outage browser test. The full YAML and voice-recovery files passed locally (88 and 19 tests). These paths were unchanged and passed on the prior revision. All three passed on the single retry. Final current-head status: 55 successful checks, one neutral check, and two skipped Storybook checks. No pending or failed checks remain. - Live Daytona/model acceptance and recovery of the hosted task remain unverified. This PR does not deploy the change. ## Risks - Releasing preparation before execution starts renewal could leave a gap in authority. The callback runs after the executor registers its reference. - Preparation errors must not leak renewal. The recovery promise retains idempotent final cleanup. - Wire callbacks can run during subscription. Message and close regressions check that registration is complete before either callback runs. - No schema, API, model, permission, lease duration, or retry policy changes. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, code execution, and browser diagnostics. The exact serving model identifier and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
07c6df260b |
fix(server): schedule archived runner evidence from saved deadlines (#17925)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/85/changes/paperclip/85 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Retired native sessions leave continuity and quarantine evidence on disk. > - The old hourly loop mixes evidence retention with backup expiry, which has different database eligibility rules. > - Evidence already has saved creation timestamps and local publication signals. > - This pull request gives evidence its own registered worker and actual next deadline. > - The worker stops filesystem and SQL activity when its evidence obligations settle. ## Linked Issues or Issue Description Stack slice 65. Parent: #17923 (`codex/sleep-64-skill-version-retention`). Related foundation: #15625. **What happened?** At parent `7b94aa185c29673372646bbf2e07a2fbd16bc394`, an hourly backup/evidence tick still inspects native state roots. Evidence's private due time is checked by that recurring tick instead of the shared scheduler. Empty evidence and future retention windows have no reason to be rediscovered every hour. **Expected behavior** Reconstruct evidence once. Wake from the existing publication signal. Register the earliest saved retention deadline. Keep busy, failed, or incomplete evidence recoverable. Preserve backup eligibility as a separate duty. **Steps to reproduce** Start retention with an empty evidence root or archives inside the seven-day window. The old timer still visits the combined owner. The new evidence worker disarms when empty, or registers the saved expiry with no evidence filesystem or SQL calls between relevant changes and that deadline. ## What Changed - Add an evidence worker through the delivery coordinator and shared scheduler. Track startup reconstruction without delaying HTTP readiness. Unsubscribe local publication before delivery shutdown. - Select `backups` for the existing backup timer and `evidence` for the new worker. Existing callers that omit the selection retain combined behavior. Backup expiration remains the next separate migration. - Use hourly retries for concrete busy or failed evidence, with earlier saved expiries and one-second continuations independent. Restore candidate/removal/byte summaries. Schedule publication at the first removable instant; preserve legacy manual-tick semantics. - Expose the existing earliest evidence deadline. Preserve one-second bounded continuation, concrete failure/busy retries, cleanup-source archive exclusions, symlink protection, late activity checks, atomic rename, and interrupted removal recovery. - Preserve non-ENOENT inspection errors as recovery obligations. Retry a failed session root in a new evidence pass instead of continuing past it and forgetting it. - Register the first removable instant after the inclusive retention boundary. Keep the existing manual dry-run publication semantics. - Add real filesystem/isolated PostgreSQL lifecycle tests and disabled/held-startup wiring cases. Add no table or migration. The archive and quarantine writers already call `noteNativeRunnerEvidence`; this PR connects that existing signal to the scheduler. Shared code owns scheduling, while evidence rules stay with the retention service. See [`doc/plans/2026-10-27-runner-evidence-retention-sleep-audit.md`](doc/plans/2026-10-27-runner-evidence-retention-sleep-audit.md). ## Verification - Adopted actual prerequisite squash `b06ea111a254c62e0a55199b010d82e0adb33bf0` from #17923 through normal merges. Qualified runtime/package/UI/map trees remain unchanged. The final nine-path diff includes one browser-spec readiness repair. - Runtime qualification remains **652 passing TypeScript cases across 22 complete files**, plus **29 passing Rust cases** in native-backend, runtime-notification and configured-environment targets. One actual closed Pi snapshot launch is intentionally skipped because its external distribution root is not provided. Accepted source is `b80eee32f344ae0218d8b791c14f2ea174da09cc`; all runtime and earlier sleep-test bytes are preserved. - The complete outage-recovery browser file passes **all five journeys** on a fresh isolated database after four additive rendered-history readiness assertions. Every original assertion, outage control and timeout remains byte-identical when those new lines are removed. The intentionally unresolved first-session scenario keeps its pre-auth interaction and delayed-owner coverage. The prior CI/local timeout evidence and one rejected broader precondition remain in the audit. - Adapter/Runner/database builds, server/Runner typechecking, Rust formatting/workspace checks, server build, module boundaries, migration order and generated TypeScript/Rust profiles pass. PostgreSQL 17 fixtures apply all 413 journal entries through index 414; source hashes and temporary fixture imports restore exactly. - **Inherited validation exception:** Copilot profile 81's separate source-identity check fails on the merged Rust configured-environment hash. Its inputs and attestations equal upstream and stay unchanged here. Grok's identity check passes. No release attestation is regenerated or new live-provider qualification claimed. Required exact-head CI/security and Greptile 5/5 still must pass. - [The evidence audit](doc/plans/2026-10-27-runner-evidence-retention-sleep-audit.md) retains historical failures, counts and limitations. No full local repository-suite, live provider, external-alarm, measured suspension or fleet-savings claim. Fresh final-head CI/security, exact-head Greptile 5/5 and actionable Ponytail findings remain separate merge gates. ## Risks - Live publication is the existing server-owned executor in one process. Independent filesystem writers need an explicit signal contract. Startup reconstructs restart state. - Busy sessions and concrete inspection/removal failures retain retries. Unknown cleanup-source archives remain protected. - The existing scan enumerates directory names before applying its entry/removal budget. This PR preserves that behavior and does not claim a bounded total directory-listing allocation. - An in-process deadline cannot wake stopped compute. Cloud sleep-safety and external-alarm coverage stay unchanged. Physical evidence reclamation can wait for a supported host wake. - Backup expiration still has an hourly database eligibility scan. This slice removes only the archived-evidence duty from that loop and does not claim full instance suspension. ## Model Used OpenAI Codex (GPT-6-based coding agent), using repository inspection, normal Git merges, composition proofs and complete PostgreSQL/filesystem/startup suites. No subagents were used in this reconciliation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Codex <noreply@openai.com> |
||
|
DottaandPaperclip
|
c32e1ba6da |
fix(onboarding): keep approved child task handoffs runnable (#18160)
## Thinking Path > - Paperclip manages work through tasks, agent runs, dependencies, and human decisions. > - The first-task workflow asks for approval before it creates an execution task. > - File-producing work must run on a child task before onboarding can finish. > - A child blocked by its onboarding source cannot start while the source waits for that child. > - This change rejects that dependency during task creation and later edits, and gives the agent a correction. > - The shipped recipe shows the dependency direction and requires the actual approved inputs in the child instructions. ## Linked Issues or Issue Description **What happened?** After human approval, an onboarding run could create a child with the onboarding source in `blockedByTaskIds`. Creation saved a blocked child with no wake. The parent could then wait for that child. A handoff could also refer to input from the parent conversation without including the input in the child description. **Expected behavior** The approved child can execute before its waiting parent. Invalid first-task dependency input fails before task creation and leaves the same command key available for a corrected call. The child instructions include the required approved input and constraints. **Steps to reproduce** 1. Accept a proposal on a server-created first task. 2. Call native `create_task` with `status: "todo"` and the onboarding source in `blockedByTaskIds`. 3. Observe a blocked child with no scheduled wake. Block the parent on that child to expose the stalled handoff. **Paperclip version or commit** The regression reproduces on `f8484a9f8d1f68e9534fee4f29e00e716dc680f4`. The repair is based on current master. **Deployment mode** Core native and REST task creation. The same server boundary serves self-hosted and hosted instances. Related: #18080 fixed remote delivery of the selected first-task skill. This change handles the later child handoff boundary. ## What Changed - Reject a task blocked by its authenticated first-task source during creation and later dependency edits. Later agent edits also check the locked target's server-owned onboarding parent, protecting a child running under its own execution. Compare UUID identity in PostgreSQL so alternate supported UUID spellings cannot bypass the check. The error explains the correction. - Add a native child/dependency recipe with self-contained approved input guidance. Clarify the general `create_task` input descriptions and update their generated contracts. - Cover native and REST rejection, corrected retry, preserved valid dependencies and replay, and an executable recipe with exact input bytes read from the child's own context. Thread the authenticated run through native dependency edits. Keep blocker corrections independent of a new proposal decision. - Update execution semantics and the existing onboarding map entries. ## Verification - Red on `f8484a9f`: the new native regression received a created blocked child instead of the expected rejection. - Review regressions were red on the original PR head: uppercase, unhyphenated and braced source UUIDs created blocked children; a later child PATCH returned 200 and saved the reversed blocker. - Review follow-up reproduced both native and REST child-own-run bypasses on `75fa75a6`: the native dependency call applied the reverse edge; the child's own PATCH returned 200. The regression uses the actual authenticated child execution and an already checked-out task. - All 95 tests across first-task admission, assets, and the full native tool-authority suite pass. The shipped recipe is validated against the native schemas and exercised through real task/dependency services with disposable PostgreSQL. Rejected PATCH tests preserve existing authorization and attempt-audit behavior and require no requested task-field, relation or wake change. Compatibility controls retain ordinary manual parent blockers, explicit board edits, other blockers and empty-list corrections. - Runner TypeScript build, generated contracts, capability inventory, protocol manifest, and map checks passed. - Server typecheck and all 95 focused tests pass with the child-run follow-up. All latest checks pass on `f24509ae`: 55 successful checks, one neutral review check, four skipped checks, and a successful legacy security status. Both automated reviews are 5/5 on this exact commit. - Prior-head CI failures are preserved: the Runner close fixture had unconfirmed provider drain while suspension succeeded; its exact fake-provider subprocess row passed locally with unchanged sources. The browser outage test lost its title textarea before any outage write. A bounded local control with a fully loaded composer passed every original outage assertion. The prior failure is an editor-readiness race; its exact asynchronous ordering remains unproved. No UI, browser-test, or timeout change is included. Neither failure is counted as a pass or silently discarded. - The first `f24509ae` CI attempt had two interrupted jobs: Typecheck and General server 9/12. Their logs record runner shutdown signals. The server job also records lost runner communication. The same-head failed-job rerun passed both jobs and the required aggregate. It reused the other successful results. No source, test, or timeout change was made for the rerun. The failed attempt remains preserved. Full repository checks pass in CI; the local full repository typecheck, test, and build commands were not repeated. - No external model or provider calls are used by these checks. Real-provider acceptance at this revision remains separate. ## Risks - The new rejection uses the authenticated run and the server-owned onboarding origin. Ordinary tasks, other blockers, empty-list corrections, and already committed creation replay retain their existing behavior. Blocker edits do not reapply the creation approval gate. - Nonempty blocker edits perform two indexed identity reads; the task lookup covers at most the run source and locked structural parent. Other patches and empty-list corrections add no source lookup. The update guard adds no parent row lock behind a child. Existing REST authorization checkout and attempt auditing remain in place before the domain mutation guard. - The explicit input recipe guides the model. Schema and persistence tests do not prove that a future model follows every instruction. - The tool guidance changes the generated semantic catalog identity. The maintained generated files are synchronized in this change. ## Model Used OpenAI `gpt-6.1-sol` through Codex, with `ultra` reasoning effort, repository tools, code generation, and test execution. The context window is not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used and stated the available runtime detail - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the bug issue template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket ID - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
27fbdb2a74 |
perf(workspaces): reuse verified seeds across task branches (#18159)
## Thinking Path > - Paperclip manages AI agents and their workspaces. > - Remote runs stage a recoverable copy of the selected host workspace. > - New tasks use separate workspace IDs and Git branches. > - The verified seed cache currently treats those identities as different content. > - This causes repeated clones and seed verification when task files and Git commits are identical. > - This pull request shares immutable seed content within a company and applies each task's branch to a private archive. > - It also removes credential broker calls for two local Git snapshot commands. ## Linked Issues or Issue Description **What happened?** New remote tasks rebuild the same Git seed under different workspace IDs and branches. In a measured staging run, preparation took 29.7 seconds. Git clone took 6.1 seconds and optional seed publication took 7.8 seconds. Publication overlapped the 4.4-second transfer, so those values cannot be added. Local `diff-files` and `write-tree` commands also request GitHub credentials. **What did you expect?** Reuse a verified seed when filesystem contents, Git HEAD, origin, and nested repository layout match. Preserve each task's branch, recovery archive, ignored files, and copyback behavior. Keep local snapshot plumbing independent of GitHub credential capture. **Steps to reproduce** 1. Configure a remote native environment and a Git project workspace. 2. Create two tasks from the same source commit with different task branches and workspace IDs. 3. Inspect the workspace preparation spans. Both tasks currently clone and publish the same seed content. **Paperclip version and deployment mode** Reproduced on Core `25f1070fbdabd96b26cdf199e624a77905f3ca37`, with a managed remote sandbox and native Runner. The local comparison used a public repository at a fixed commit with roughly 13,000 entries. Refs #17886, #16398, #16399, #17357. The Git launcher extension builds on Jamie Buchanan's work in #17886. The other PRs establish the seed verification, recovery, and eviction contracts preserved here. A search found no open PR for sharing these verified seeds across task branches. ## What Changed - Use a company-scoped cache bucket for native host-current preparation. Keep the existing bucket and company storage limits and eviction rules. - Remove branch names from the shared generation identity. Keep file contents, Git HEAD, origin, and nested layout in that identity. Keep the default workspace-scoped identity unchanged for other callers. - Store a detached Git archive in the shared cache. Copy it and append the selected branch metadata before upload and private per-run persistence. Existing durable-seed replay can read the private archive without the new cache option. - Forward the scope through the execution target and both command-runtime paths. - Treat `diff-files` and `write-tree` as local Git commands. Network commands, unknown commands, and managed commit identity keep their existing credential path. - Test new workspace IDs, named and detached branches, nested repositories, dirty and ignored files, symlinks, recovery, corrupt caches, refused publication, and copyback. ## Verification - Current head `6087bac274d50804d1b332646e43f3b8d41d798f`: 53 checks passed, two optional checks skipped, Greptile 5/5, no unresolved review threads. Linux CI passed. - Adapter-utils TypeScript 7.0.2 typecheck passed. - Server package typecheck passed, including its Runner and plugin build prerequisites. - 140 seed-cache, seed-reuse, and sandbox-runtime tests passed after rebasing. - 44 Git launcher and command-runtime tests passed after rebasing. - 30 native workspace preparation, warm history, and retention-configuration tests passed. All 56 retention tests also passed (5.72 seconds) after repairing missing library aliases inside the installed macOS Postgres package. That local dependency repair changed no repository source. The earlier startup timeouts occurred before test assertions. - The new seed reuse regression failed before the change because the second branch cloned again. It passes through the execution target and both command-runtime paths after the change. - Clean local preparation comparison: two alternating measured pairs, with an existing verified company seed and new task branches. Baseline: 19.63 and 9.85 seconds. Shared cache: 5.18 and 4.80 seconds. Median: 14.74 to 4.99 seconds. Each baseline cloned and published; each shared-cache run skipped both. Branch and file checks passed. The complete comparison loop took 58 seconds. - The comparison used real Git, archives, snapshot workers, and local file transport. It used no model calls. It does not establish a complete staging response time. The first generation still pays the cold preparation cost. ## Risks - A shared cache hit requires an identical generation that is still present. The company bucket retains the existing 16-generation/1 GiB bucket limit; busy companies may miss more often. - The branch binding appends entries to a private tar archive. Tests cover the host platform; Linux CI must also pass before merge. - A cache miss, corruption, or capacity refusal must retain the private recovery path. Existing tests exercise those cases. - Git plumbing can still encounter repository-specific filters or missing objects. This extends the existing local-command policy by two commands; it does not grant network credentials to them. - The latest measured real staging reply took 77.8 seconds. This change targets a bounded part of that path and does not claim the under-15-second response goal is met. ## Model Used OpenAI Codex, GPT-6 family, with reasoning and repository tools. The session does not expose the exact model revision or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
ba07f85cad |
perf: schedule agent timers from durable deadlines (#17876)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/58/changes/paperclip/58 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent timers must start due work and preserve queue admission rules. > - The server woke their scan on every heartbeat interval, even with no eligible timer. > - Idle scans keep the database and instance active. > - This pull request derives the next deadline from saved policy and work records. > - Supported producer transactions notify the shared coordinator when eligibility changes. > - An idle timer worker stops scanning and resumes when real work or a deadline requires it. ## Linked Issues or Issue Description Refs #17863. This is slice 63 of the recurring-work migration and follows the merged issue-monitor deadline PR. Related: #16632 changes timer issue scope; #16881 changes live-run eligibility. This PR preserves the current admission policy. Those policy changes are separate work. A future time-sensitive predicate must expose its next transition deadline. **What existing behavior does this improve?** Agent timer heartbeat scheduling in the server. **Current behavior** A recurring heartbeat wakes the timer scan even when companies are paused, timers are disabled, or no actionable task exists. **Proposed behavior** Register the earliest eligible interval or positive daily-cap reset deadline. Re-read on supported producer commits and startup. Leave no recurring database scan when no timer obligation remains. ## What Changed - Keep policy and deadline calculation in the heartbeat queue. Scheduling retains company, reporting-chain, worktree, and CAS gates. - Use the existing deadline coordinator. Remove only the timer wake from the server heartbeat loop. - Notify before existing transaction writes that change eligibility or deadlines. Cover configuration, lifecycle, company budgets, task ownership, recovery, conversations, accounting corrections, and completion. - Use the lifecycle status-change hook for saved hire rejection. Cover checkout's second adoption transaction after an initial failed CAS. - Preserve pending work through idle and maintenance drains. Check cancellation between due agents. Reconstruct on restart. - Document the supported writer model and producer inventory. Add no schema, table, or provider branch. ## Verification - **1,710 passing cases across 49 complete files**, no skips, on accepted source `32db86c1974dcbf794f01a8c5e42024e2ad071fc`. Coverage includes real timer policy/CAS/producers, monitor and shared deadline consumers, native reconstruction, Slack lifecycle, skills/settings/configuration, queue isolation and agent lifecycle. - Database build, production server typechecking/build, module boundaries, target migration order and Runner profile identity pass. Twenty-nine fresh PostgreSQL 17 schemas plus per-file fixtures applied all 412 journal entries through index 413; all selected source hashes and temporary fixture imports were restored exactly. - Four new deadline composition cases verify future/overdue timing, minimum scan spacing and disarming. A controlled mutation dropping only the inherited continuation option fails; the composed source passes with identical assertions and timers. - Normal parent reconciliation preserves task-drain expiry and both deadline contracts. The final monitor-parent publication changes only its audit, preserving every qualified runtime and selected test source. - Historical failures and review corrections remain in [the timer audit](doc/idle-agent-timers.md). No full local repository-suite, live provider, measured suspend/wake, external-alarm or fleet-savings claim. Final-head CI/security, exact-head Greptile 5/5 and actionable Ponytail findings remain separate merge gates. ## Risks - Process-local notifications cover supported service/API writers in one server process. Direct SQL and independent replicas need an explicit ingress contract. Startup reconstruction is not live cross-process notification. - Notifications can batch for up to the configured heartbeat interval. Known deadlines retain their scheduler path; failed reconciliation retains bounded retry. - Enabled timers without an actionable-work requirement remain real recurring obligations. Zero caps require policy changes; positive reached caps use UTC midnight. - The timer fixture substitutes adapter enqueue. Existing admission tests also pass, but this is not deployed provider or sleep qualification. - Cloud does not gain legacy agent timer alarms in this PR. Existing sleep-safety blockers remain. No database/host suspension or fleet savings claim is made. - Parent-stack CI and review must pass before merge. This PR does not authorize merging its dependencies. ## Model Used OpenAI Codex (GPT-6-based coding agent), using repository inspection, normal Git merges, composition proofs, a controlled red/green regression and complete PostgreSQL integration suites. No subagents were used in this reconciliation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Codex <noreply@openai.com> |
||
|
|
0f6f0a3fee |
fix(server): schedule issue monitors without heartbeat polling (#17863)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/56/changes/paperclip/56 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Issue monitors save a future check on an existing issue. > - The heartbeat timer still scans monitors on every tick, even when none are due. > - The saved deadline and claim already provide durable scheduling state. > - This pull request gives monitors their own deadline worker and connects state changes to it. > - Idle monitor work stops querying. Existing admission and recovery rules remain in force. ## Linked Issues or Issue Description Refs #17840, #16325. Related: #17171 addresses rearming a monitor after its admitted run fails. This PR changes when monitor inspection runs and follows the merged routine prerequisite. **What existing behavior does this improve?** Issue-monitor scheduling and idle database use. **Current behavior** Heartbeat ticks call the combined agent and monitor sweep. The existing next-deadline query only retains future monitors. Polling covers overdue leftovers, stale claims, and changes that make a saved monitor eligible. **Proposed behavior** Use the shared deadline worker. Retain overdue work with bounded retries. Wake after committed changes to monitor state, issue eligibility, company state, or native ownership. Keep legacy agent ticks separate for the next slice. ## What Changed - Add monitor-specific sweep and deadline methods to the existing scheduling owner. Keep the manual combined method. - Include overdue work and the strict five-minute claim fence. Exclude active native owners until their state changes. - Register monitors with the shared deadline worker. Heartbeat ticks wake only the legacy agent timer worker. - Connect direct issue-state writers and existing execution-control notifications to monitor work. Checkout uses a short transaction around its existing CAS. Provider work stays outside it. - Add real database lifecycle tests and retain routine, startup, admission, execution-control, and native-projection coverage. - Record the obligation, producer inventory, recovery behavior, and limits in `doc/plans/2026-10-27-issue-monitor-sleep-audit.md`. ## Verification - **1,233 passing cases in 35 complete files**, no skips, on accepted source `c24aadfc91e9394d73c799f9e05fc67202160557`. Coverage includes real monitor scheduling/admission, native settlement, committed producers, retained routines and all shared deadline consumer integration suites. - Database build, production server typechecking/build, module boundaries, target migration order and Runner profile identity pass. Twenty-four fresh PostgreSQL 17 schemas plus per-file fixtures applied all 412 journal entries through index 413; every selected source hash and temporary fixture import was restored exactly. - Normal parent reconciliation preserves master's admitted-wake result and scoped issue deletion while keeping committed monitor notifications. All other paths match the automatic merge; packages remain byte-exact to the qualified parent. The final routine-parent publication changes only its audit. - Historical failures and corrections remain in [the monitor audit](doc/plans/2026-10-27-issue-monitor-sleep-audit.md). Old delegated-Slack rate and Runner drain CI failures are retained; the current inherited clock fixture and package source are byte-identical to merged #17835. Fresh final-head CI remains required. - No full local repository-suite, live provider, deployed suspend/wake, external-alarm, production-load or fleet-savings claim. Final-head CI/security, exact-head Greptile 5/5 and actionable Ponytail findings remain separate merge gates. ## Risks - A missing producer notification can strand a monitor after polling is removed. The audit covers ordinary writes, direct status/assignment writers, recovery, and native ownership changes. - The writer model is process-local. Independent SQL writers and other replicas need a supported notification or reconciliation contract. - Overdue work uses the configured heartbeat interval as a retry floor for the monitor worker. New notifications during that backoff wait for its next pass. Normal state-change bursts coalesce for one second. - Notification fanout can inspect monitors during other active work. An unchanged idle subsystem has no recurring monitor scan. - The legacy agent timer still polls. Cloud sleep proofs and external alarm coverage are unchanged. ## Model Used OpenAI Codex (GPT-6-based coding agent), using repository inspection, ordinary Git merges, composition proofs, PostgreSQL fixtures and complete local integration suites. No subagents were used in this reconciliation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Codex <noreply@openai.com> |
||
|
|
c8b7afee06 |
fix(server): schedule native environment lease settlement (#17801)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/24/changes/paperclip/24 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip manages AI agents and the resources used by their work. > - Native run completion has several independent settlement duties. > - The recurring run reaper repeated environment-release discovery for completed native history. > - Existing lease rows already record releases that still need work. > - This pull request wakes the existing release owners from committed changes and schedules only unfinished release debt. > - Completed history becomes quiet while stop evidence, workspace preservation, and retention policy keep their existing owners. ## Linked Issues or Issue Description Refs: #17793, #17717. **What problem does this solve?** Periodic native finalization replay calls environment release even when no lease needs work. This adds recurring database traffic to an idle instance. **What is the proposed solution?** Register one scoped worker on the existing delivery coordinator. Discover work from native finalizations and existing environment leases. Wake it from native, lease, and settings transactions. Keep the release owner's saved pending-cleanup deadlines. Keep retries only while a concrete lease still needs settlement. **What alternatives have you considered?** A slower unconditional scan still queries idle history. A new job table duplicates existing lease truth. Replacing the compound callback would also change runtime-service and agent-status duties; those remain separate slices. This is slice 58. Parent #17793 has merged. This publication targets master and preserves that landed prerequisite. Fresh current-head checks and review remain required. ## What Changed - Keep ordinary company-only finalization hints out of environment settlement. Prove zero candidate SQL after quiet and preserve real lease delivery. - Reuse the existing durable destroy claim for run-owned cleanup, excluding concurrent attempts and fencing late receipts while preserving cleanup recovery. - Scope lease cleanup notifications to their company/run so settlement does not rediscover the entire backlog after each release. - Read only `nativeWorkspaceSync` and `workspaceExportRetry` JSON fields when discovering environment cleanup. Preserve saved run data and existing release decisions. - Reuse the environment runtime's release predicate for discovery. Add no table or independent timer. - Wake from existing finalization, projection, lifecycle, recovery, lease, and settings transaction boundaries. Simple lease acquisition now uses a transaction too. - Preserve recorded retention policy, pending-cleanup deadlines, original lease/provider ownership, and current release guards. - Re-read durable lease state after a release. A logged or swallowed per-lease failure does not discharge the obligation. - Delegate only periodic environment release. Preserve default live/manual behavior, runtime-service release, and agent-status callbacks. - Add an owner/producer/lifecycle audit and real database tests. ## Verification Accepted production source: `9596fa022b57feea026c4728c80bdc36037b804f`. Publication head: `fb252c8b0e52ce355e7d98db2e392e4831f99c61`. The actual parent squash is adopted by a normal merge with the complete accepted child tree unchanged. Only audit text follows. - **4,172 distinct cases in 64 complete server files pass**, plus **20 selected chat cases** (4,192 selected total). The main batch reports 3,440 passes and 49 skips. Both complete Stop/read files then pass 51 + 6 cases on separate fresh databases through their supported override. Their eight pure cases are counted once; skipped cases are never counted as passes. Separate process recovery passes 410; review response passes 273. The other 1,794 chat cases were not run locally. - **Quiet producer proof:** the controlled unscoped baseline fails with three candidate SELECTs after three routine company-only hints. The scoped case performs zero, releases a real eligible committed-run lease, and returns to quiet. The same test hash spans red and green. - **Static gates pass:** database build; server typecheck/build; module boundaries; migration order; current Copilot profile identity. Packages equal the accepted parent byte for byte. - **Fresh fixtures:** 39 prepared PostgreSQL 17 databases plus fresh helper-owned databases. The schema has all 412 journal entries through index/tag 0413_tired_kronos. All 65 original test hashes restore exactly; the indirect Stop fixture is also hashed. No assertion or timeout changes. - **Preserved records:** original 49 skips and their completed coverage remain documented. Parent handoff timeout and prior chat-idle failure records remain in their owning audits. This does not claim full repository, full chat, live provider, or deployed sleep acceptance. - [Current integration and native lease audit](https://github.com/paperclipai/paperclip/blob/fb252c8b0e52ce355e7d98db2e392e4831f99c61/doc/plans/2026-10-26-native-environment-settlement-sleep.md#current-master-integration-and-native-lease-quiet-proof-2026-10-29). Fresh CI, security, and exact-head Greptile 5/5 must pass before merge. <details><summary>Earlier revision verification and failure history</summary> Receiver failure fairness: a controlled 1,100 ms batch reproduced a 30-second delay for healthy due receivers. The worker now supplies its existing per-item deadline through `nextRunAtOnError`. All 20 receiver lifecycle tests pass on the owner and combined tip `31c12f1d68ff4a51e06fc2be2abf2c2aee9f9d3a`; server production typechecking passes on the owner. The failed receiver retains its own backoff, and settled receivers return to zero recurring SQL. The five-second assertion budget is unchanged. See [the owning audit](https://github.com/paperclipai/paperclip/blob/6f3a0b8de149fb5620f15909473eb7c4455b9824/doc/plans/2026-10-25-chat-receiver-discovery-sleep-audit.md#healthy-receiver-progress-after-a-slow-failed-batch). Current-head CI and review remain required. Approved delegated Test-tab calls now evaluate caller and target policies in the existing claim transaction. Real HTTP regressions first showed a missing caller charge and shared-limit self-rejection. All 78 connection-management cases and 92 gateway cases now pass on the owning branch, along with server production typechecking. All 78 connection-management cases also pass at combined tip `d25f830db89c1623bff3547bfc68015c1c66e0cf`. The three new cases cover shared charges, separate charges, rollback on denial, duplicate approval, and rejection of a later authorized call. Intermediate counter/identity fixture failures and a reused-database Slack collision are recorded in [the owning audit](https://github.com/paperclipai/paperclip/blob/ae9bff1c06e34702f64a399e58b45a657c133bd1/doc/idle-approved-tool-recovery.md#delegated-test-tab-admission-after-policy-changes). The earlier combined checks below remain evidence for their stated revisions. Current-head CI and review remain required. Inherited voice-cleanup verification: the controlled slow-provider regression and all 13 lifecycle tests passed on its owner and combined tip `ab22d8ed0c09028315607414b83b375e5cc0a60a`. That repair changed only the fixture clock advancement and its audit note. Current head: `6f3a0b8de149fb5620f15909473eb7c4455b9824`. This remains a single-duty PR stacked on #17793. - Integrated pinned master `8548173fc9efdc0fa07644fb92dcc0b6851a95e5` with normal merges. Every branch passed generated-schema checks and its target-branch migration-order gate. Existing SQL is unchanged; the 25 stack migrations follow master’s new migration. - Duty-specific producer coverage, lifecycle tests, and earlier failure records: [2026-10-24-recurring-server-sleep-stack.md](https://github.com/paperclipai/paperclip/blob/7e619453567e078a7d745f28bfe8e5ede571f282/doc/plans/2026-10-24-recurring-server-sleep-stack.md), [2026-10-26-native-environment-settlement-sleep.md](https://github.com/paperclipai/paperclip/blob/7e619453567e078a7d745f28bfe8e5ede571f282/doc/plans/2026-10-26-native-environment-settlement-sleep.md). - Combined tip `51e6cbe67979b55b340ea9f0f434c5b5c682ccd6`: **355 startup/cancellation/recovery/dispatch/preservation tests**, **75 connection-management tests**, and production server typechecking pass. These combined checks do not mean every intermediate PR independently ran the full repository suite. - Real master-to-stack database upgrade passes: **379 unchanged prior migration receipts plus 25 new migrations**, with the gateway responsibility fixture preserved. - Owning-branch checks: 84 runner installation/settlement cases plus runner typechecking and generated TypeScript/Rust profile agreement; 253 GitHub review workflow cases. Stop-attribution and held-worker readiness regressions failed before their fixes. - Failure accounting: the initial combined run passed 349 cases and failed six idle assertions in a reused preservation database containing old cancelled runs. Repeating the combined command with a fresh preservation database passed. Earlier invalid project selection, merge import/type errors, and the interrupted full local suite are not counted as passing checks. - **Latest-head CI and review remain required.** Prior CI findings include GitHub idle observation, route teardown, attention cleanup, touch-picker E2E, workspace health, and voice-cleanup timing; local passes do not erase those failures. No full local repository-suite, deployed sleep, fleet cost, or live Copilot qualification claim is made. </details> ## Risks - Existing lease owners retain destruction claims, provider identity, stop evidence, retention policy, and pending-cleanup deadlines. Incorrect changes here can release a live resource or lose a cleanup obligation. - Logged or swallowed release failures remain concrete debt. Real retries must retain their saved floor across unrelated hints. - In-process signals do not cover independent SQL or other replicas without ingress. - This delegates only periodic environment settlement. Runtime-service and agent-status duties remain separate. Existing Cloud blockers remain; no deployed sleep, database suspension, external alarm coverage, or cost savings is claimed. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, code execution, real PostgreSQL tests, and GitHub tools. The exact runtime model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes:` / `Closes:` / `Refs:` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Codex <noreply@openai.com> |
||
|
|
829f9e7eaf |
fix(server): settle Muse terminal assignments from committed changes (#17793)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/12/changes/paperclip/12 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its external-agent module owns Muse assignments, operation receipts, and worker holds. > - Running assignments and retained terminal history must not cause an empty maintenance scan every five seconds. > - Terminal run changes and operation results already have durable database transactions. > - This pull request wakes the existing maintenance worker from those commits and reconstructs work at startup. > - The worker retries only concrete settlement debt. It retains qualification deadlines and authority checks. ## Linked Issues or Issue Description Refs: #17772. This is slice 57. Parent #17772 has merged. Related: #17000, #17525, #17752, #17758. The publication targets master and preserves the merged parent. **What existing behavior does this improve?** Muse terminal assignment settlement and the existing Muse maintenance worker. **Current behavior** A running assignment can keep maintenance scanning every five seconds. Terminal reconciliation rewrites unchanged history. Native finalization also calls that reconciliation during periodic replay. Discovery excludes a revoked original binding even when its assignment still needs settlement. **Proposed behavior** Committed run changes and operation receipts wake the existing worker with the original company, run or binding identity. Startup reconstructs actionable work from existing rows. Maintenance processes bounded batches and disarms after settlement. Default live and manual callbacks remain. The periodic callback delegates this one duty. **Reason and benefit** Configured but idle instances stop querying for this obligation. Late receipts, restarts, drain and historical bindings still have recovery paths. No table or independent timer is added. **Breaking changes** No schema, endpoint or provider protocol changes. Explicit manual reconciliation retains its unbounded default. This does not alter Cloud sleep admission or claim that all instance work can sleep. ## What Changed - Add an owner predicate shared by discovery and locked settlement. Fenced private-worker uncertainty remains held without an automatic retry loop. - Preserve original binding generation and company/agent/run identity, including revoked bindings. Fence only the terminal assignment during reconciliation. - Extend existing maintenance with scoped hints, bounded batches, concrete-debt retry, startup reconstruction and quiet drain behavior. - Register notifications inside existing terminal, projection, cancellation, operation-result and send-failure transactions. Keep qualification and OAuth signals explicitly global when coalesced with scoped changes. - Serialize only four domain IDs. Real native-finalizer tests caught and prevent a circular-object serialization failure at this boundary. - Use existing restore/worktree suppression and settings reactivation. Delegate both periodic finalizer branches while retaining resource callbacks and default live/manual settlement. - Add the producer audit and regression evidence in `doc/plans/2026-10-26-muse-terminal-settlement-sleep.md`. ## Verification Accepted production source: `584015018ed4a9d57d2b0f192b75847503b54e53`. Publication head: `305014b030d6c5eb891c70bc57036f4192d4fdad`. Normal adoption of the actual parent squash preserves the complete accepted child tree. Only audit text, a migration-count clarification, and the proved Photon test extension follow. - **3,715 distinct cases in 52 complete server files pass**, plus **20 selected chat cases** (3,735 selected total). The main 49-file batch passes 3,008; process recovery passes 410; review response passes 273; the full Photon file passes 24. The other 1,794 chat cases were not run locally. - **Quiet producer proof:** moving only the Muse fanout outside its parent guard produces six candidate SELECTs and fails the regression. Keeping the guard yields zero and still settles a real terminal run. The same test hash spans the controlled red/green pair. - **Photon review evidence:** repeated active callbacks produce zero settlement hints/candidate SQL. Three concurrent attention recoveries produce exactly one of each. The existing locked attention guard was correct; production needed no fix. The parent security check accepted the resolved finding before merge. This PR keeps the additional settlement queue in the existing regression. - **Static checks pass:** database build; server typecheck/build; module boundaries; migration order; current Copilot identity. Packages equal the accepted parent byte for byte. No new live provider qualification is claimed. - **Fixtures and failures:** 34 fresh PostgreSQL 17 fixtures have 412 applied migration receipts through tag 0413_tired_kronos, plus fresh per-call helper fixtures. All 52 original test hashes restore exactly. The additional Photon test hash matches its passing proof. Earlier remote chat passive-idle failure remains recorded; both passive/missing cases pass in the new selection. Parent handoff timeout/fresh-file-pass accounting remains in its audit. No assertion or timeout was weakened. - [Integration and quiet producer audit](https://github.com/paperclipai/paperclip/blob/305014b030d6c5eb891c70bc57036f4192d4fdad/doc/plans/2026-10-26-muse-terminal-settlement-sleep.md#current-master-integration-and-quiet-producers-2026-10-29). Fresh CI, security, and exact-head Greptile 5/5 must pass before merge. <details><summary>Earlier revision verification and failure history</summary> Receiver failure fairness: a controlled 1,100 ms batch reproduced a 30-second delay for healthy due receivers. The worker now supplies its existing per-item deadline through `nextRunAtOnError`. All 20 receiver lifecycle tests pass on the owner and combined tip `31c12f1d68ff4a51e06fc2be2abf2c2aee9f9d3a`; server production typechecking passes on the owner. The failed receiver retains its own backoff, and settled receivers return to zero recurring SQL. The five-second assertion budget is unchanged. See [the owning audit](https://github.com/paperclipai/paperclip/blob/f7477933469fdc960f555425dce66eff25a94e54/doc/plans/2026-10-25-chat-receiver-discovery-sleep-audit.md#healthy-receiver-progress-after-a-slow-failed-batch). Current-head CI and review remain required. Approved delegated Test-tab calls now evaluate caller and target policies in the existing claim transaction. Real HTTP regressions first showed a missing caller charge and shared-limit self-rejection. All 78 connection-management cases and 92 gateway cases now pass on the owning branch, along with server production typechecking. All 78 connection-management cases also pass at combined tip `d25f830db89c1623bff3547bfc68015c1c66e0cf`. The three new cases cover shared charges, separate charges, rollback on denial, duplicate approval, and rejection of a later authorized call. Intermediate counter/identity fixture failures and a reused-database Slack collision are recorded in [the owning audit](https://github.com/paperclipai/paperclip/blob/bbd31dea5ef3a4875950a87b4636af8b6bde858e/doc/idle-approved-tool-recovery.md#delegated-test-tab-admission-after-policy-changes). The earlier combined checks below remain evidence for their stated revisions. Current-head CI and review remain required. Inherited voice-cleanup verification: the controlled slow-provider regression and all 13 lifecycle tests passed on its owner and combined tip `ab22d8ed0c09028315607414b83b375e5cc0a60a`. That repair changed only the fixture clock advancement and its audit note. Current head: `f7477933469fdc960f555425dce66eff25a94e54`. This remains a single-duty PR stacked on #17772. - Integrated pinned master `8548173fc9efdc0fa07644fb92dcc0b6851a95e5` with normal merges. Every branch passed generated-schema checks and its target-branch migration-order gate. Existing SQL is unchanged; the 25 stack migrations follow master’s new migration. - Duty-specific producer coverage, lifecycle tests, and earlier failure records: [2026-10-24-recurring-server-sleep-stack.md](https://github.com/paperclipai/paperclip/blob/ff196791db6dada2dc04ea9db0d58c5c3beb4a41/doc/plans/2026-10-24-recurring-server-sleep-stack.md), [2026-10-26-muse-terminal-settlement-sleep.md](https://github.com/paperclipai/paperclip/blob/ff196791db6dada2dc04ea9db0d58c5c3beb4a41/doc/plans/2026-10-26-muse-terminal-settlement-sleep.md). - Combined tip `51e6cbe67979b55b340ea9f0f434c5b5c682ccd6`: **355 startup/cancellation/recovery/dispatch/preservation tests**, **75 connection-management tests**, and production server typechecking pass. These combined checks do not mean every intermediate PR independently ran the full repository suite. - Real master-to-stack database upgrade passes: **379 unchanged prior migration receipts plus 25 new migrations**, with the gateway responsibility fixture preserved. - Owning-branch checks: 84 runner installation/settlement cases plus runner typechecking and generated TypeScript/Rust profile agreement; 253 GitHub review workflow cases. Stop-attribution and held-worker readiness regressions failed before their fixes. - Failure accounting: the initial combined run passed 349 cases and failed six idle assertions in a reused preservation database containing old cancelled runs. Repeating the combined command with a fresh preservation database passed. Earlier invalid project selection, merge import/type errors, and the interrupted full local suite are not counted as passing checks. - **Latest-head CI and review remain required.** Prior CI findings include GitHub idle observation, route teardown, attention cleanup, touch-picker E2E, workspace health, and voice-cleanup timing; local passes do not erase those failures. No full local repository-suite, deployed sleep, fleet cost, or live Copilot qualification claim is made. </details> ## Risks - In-process signals do not capture independent SQL or other replicas. Supported external events still need ingress. - Pending assignments and operation receipts retain their original binding identity. A new agent configuration cannot take their ownership. - Genuine qualification, OAuth, and recovery deadlines remain obligations. They must not be discarded merely to make the worker quiet. - The parent coarse-hint guard prevents recurring Muse discovery; real company/binding authority changes retain their signals. - No full local repository suite, full chat suite, live Muse or Copilot qualification, deployed sleep, database suspension, external alarm coverage, or cost savings is claimed. Existing Cloud blockers remain. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, code execution, real PostgreSQL tests, and GitHub tools. The exact runtime model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Codex <noreply@openai.com> |
||
|
|
4d0847c4ca |
fix(server): settle external conversations from committed work (#17772)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/6/changes/paperclip/6 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip manages AI agents and the tasks that they execute. > - External conversations must become idle after an authorized response is published and all related work is complete. > - Native replay and stranded-task recovery still discover this settlement work through recurring scans. > - A new deadline worker must preserve response ordering, monitor handoffs, and unfinished work. > - This pull request gives settlement one shared readiness predicate and activates its worker from committed changes. > - Existing rows provide recovery truth. No new table or per-feature timer is added. > - The benefit is a settlement worker that becomes quiet and resumes when relevant work changes. ## Linked Issues or Issue Description Refs #17758. This is slice 56. Slice 55 has merged. Related: #17606 (chat milestones), #17663 (bounded run reads), #17665 (monitor proof), and #13809 (answered Slack conversation lifecycle). The provider applicability remains Slack and Speko-owned conversations. This does not extend Telegram settlement from #16559. **What happened?** Conversation settlement depended on periodic native replay and generic task recovery. An exhausted monitor also cleared its schedule before recovery admission. During that gap, an answered conversation could look finished. Failed or deferred admission lost the monitor schedule. **Expected behavior** Relevant committed changes activate settlement. The worker uses the owning service's evidence and becomes quiet when no settlement is owed. Monitor recovery stays pending until admission, and restart does not admit an already accepted recovery twice. **Steps to reproduce** 1. Create an active external conversation with a successful run and an authorized published response. 2. Keep execution, approval, a goal, or a monitor pending. Let the settlement worker become quiet. 3. Resolve that prerequisite through its owning service. Verify that the conversation settles without another periodic sweep. 4. Delay or fail exhausted-monitor recovery admission. Verify that the monitor remains pending and the conversation cannot settle during the handoff. **Paperclip version or commit** Accepted production source: `50f4432ed88537b8d522cf1dce24c33588db0bac`. Parent #17758 merged as `8ac5a128e66fb672e88106412f76cf278599bfc4`. The publication adopts that actual squash through a normal merge with an unchanged complete child tree. ## What Changed - Preserve quiet after coarse company-only maintenance hints. The shared inbound-notice and response-availability helpers keep settlement enabled by default. Only the milestone dispatcher suppresses their duplicate settlement fanout. Its scoped issue-review signal and the explicit company-authority signal still wake real work. Four database cases prove both quiet and real company-wide producer delivery. - Preserve source-generation checks for chat handoff, task-before-comment locking, issue-scoped decision invalidation, and current conversation event columns from master. - Track initial external conversation settlement through shutdown without blocking HTTP readiness on endpoint or task locks. - Reuse the real queue's coalesced recovery receipt after restart, without admitting the same monitor recovery again. - Share the existing conversation readiness predicate between bounded discovery and locked settlement. Preserve current provider policy, run/publication proof, company boundaries, and endpoint-to-issue lock order. - Register a scoped deadline worker through the delivery coordinator. Preserve startup, restore/worktree suppression, drain/resume, failure recovery, and joined shutdown. - Extend existing review, milestone, wake-notice, and finalization signals. Add notifications at independent issue, comment, goal, session, and wake-receipt unblocks. - Keep monitor owner recovery pending until guarded admission. Reuse an accepted receipt for the same monitor schedule after restart, even when its limit reason changes; retain compatibility with legacy receipt keys. Commit board escalation and its monitor clear together. - Delegate periodic native settlement while preserving resource callbacks. Keep direct/manual defaults and preserve generic recovery ordering with a read-only readiness guard. - Add the producer and lifecycle audit in `doc/plans/2026-10-26-external-conversation-settlement-sleep.md` and update the stack ledger. ## Verification Accepted production source: `50f4432ed88537b8d522cf1dce24c33588db0bac`. Publication head: `b256bb52647affd9f5279ea8c40f48bbe40f031f`. The complete accepted child tree stays unchanged when adopting merged parent #17758. This publication adds only the audit document. - **3,129 distinct cases in 37 complete server files pass**, plus **18 selected chat-response cases** (3,147 selected cases). The other 1,796 chat cases were not run locally. Full settlement, queue, scheduling, run-state, goals, interactions, task Retry, decision targets, status reassessment, workspace, process recovery, and review consumers are covered. - **Preserved failure:** the first 35-file command passed 2,445 cases and timed out one restored-work handoff at its existing 15-second limit. That aggregate command failed. The unchanged full 89-case actions file then passed on a fresh fixture; the affected case took 22 ms. Root cause was not established. No timeout or assertion changed. The 34 other files passed originally. Distinct counts exclude duplicate actions passes. Fresh CI remains a separate gate. - **Indirect wake regression:** the original unconditional baseline failed two quiet checks. Guards alone left the chat case failing with three candidate queries. The final controlled indirect baseline fails one of four cases; the repair passes all four with the same test hash. Positive controls preserve real company-only producers. - **Source and fixture controls:** all 38 selected test files restore to their exact committed hashes after fixture imports. Fresh PostgreSQL 17 fixtures use 413 migrations. All five manual merge resolutions preserve current authority, locking, and event-column contracts. - **Static checks pass:** database build; server typecheck and build; module boundaries; migration order; Copilot profile identity. Runner packages equal the accepted parent byte for byte. No new Runner/package qualification is claimed. - [Current integration and wake audit](https://github.com/paperclipai/paperclip/blob/b256bb52647affd9f5279ea8c40f48bbe40f031f/doc/plans/2026-10-26-external-conversation-settlement-sleep.md#current-master-integration-and-indirect-wake-audit-2026-10-29). Latest-head CI and exact-head Greptile 5/5 must pass before merge. <details><summary>Earlier revision verification and preserved failure history</summary> Receiver failure fairness: a controlled 1,100 ms batch reproduced a 30-second delay for healthy due receivers. The worker now supplies its existing per-item deadline through `nextRunAtOnError`. All 20 receiver lifecycle tests pass on the owner and combined tip `31c12f1d68ff4a51e06fc2be2abf2c2aee9f9d3a`; server production typechecking passes on the owner. The failed receiver retains its own backoff, and settled receivers return to zero recurring SQL. The five-second assertion budget is unchanged. See [the owning audit](https://github.com/paperclipai/paperclip/blob/83204a67880d980cfbea3771cee19a3cda0022b7/doc/plans/2026-10-25-chat-receiver-discovery-sleep-audit.md#healthy-receiver-progress-after-a-slow-failed-batch). Current-head CI and review remain required. Approved delegated Test-tab calls now evaluate caller and target policies in the existing claim transaction. Real HTTP regressions first showed a missing caller charge and shared-limit self-rejection. All 78 connection-management cases and 92 gateway cases now pass on the owning branch, along with server production typechecking. All 78 connection-management cases also pass at combined tip `d25f830db89c1623bff3547bfc68015c1c66e0cf`. The three new cases cover shared charges, separate charges, rollback on denial, duplicate approval, and rejection of a later authorized call. Intermediate counter/identity fixture failures and a reused-database Slack collision are recorded in [the owning audit](https://github.com/paperclipai/paperclip/blob/ef9ae5767430d0093efc736ae0b303ba5afaae65/doc/idle-approved-tool-recovery.md#delegated-test-tab-admission-after-policy-changes). The earlier combined checks below remain evidence for their stated revisions. Current-head CI and review remain required. Inherited voice-cleanup verification: the controlled slow-provider regression and all 13 lifecycle tests passed on its owner and combined tip `ab22d8ed0c09028315607414b83b375e5cc0a60a`. That repair changed only the fixture clock advancement and its audit note. Current head: `83204a67880d980cfbea3771cee19a3cda0022b7`. This remains a single-duty PR stacked on #17758. - Integrated pinned master `8548173fc9efdc0fa07644fb92dcc0b6851a95e5` with normal merges. Every branch passed generated-schema checks and its target-branch migration-order gate. Existing SQL is unchanged; the 25 stack migrations follow master’s new migration. - Duty-specific producer coverage, lifecycle tests, and earlier failure records: [2026-10-24-recurring-server-sleep-stack.md](https://github.com/paperclipai/paperclip/blob/d6d8e4ffa6b330ebf4647aec997ee2dde6abc321/doc/plans/2026-10-24-recurring-server-sleep-stack.md), [2026-10-26-external-conversation-settlement-sleep.md](https://github.com/paperclipai/paperclip/blob/d6d8e4ffa6b330ebf4647aec997ee2dde6abc321/doc/plans/2026-10-26-external-conversation-settlement-sleep.md). - Combined tip `51e6cbe67979b55b340ea9f0f434c5b5c682ccd6`: **355 startup/cancellation/recovery/dispatch/preservation tests**, **75 connection-management tests**, and production server typechecking pass. These combined checks do not mean every intermediate PR independently ran the full repository suite. - Real master-to-stack database upgrade passes: **379 unchanged prior migration receipts plus 25 new migrations**, with the gateway responsibility fixture preserved. - Owning-branch checks: 84 runner installation/settlement cases plus runner typechecking and generated TypeScript/Rust profile agreement; 253 GitHub review workflow cases. Stop-attribution and held-worker readiness regressions failed before their fixes. - Failure accounting: the initial combined run passed 349 cases and failed six idle assertions in a reused preservation database containing old cancelled runs. Repeating the combined command with a fresh preservation database passed. Earlier invalid project selection, merge import/type errors, and the interrupted full local suite are not counted as passing checks. - **Latest-head CI and review remain required.** Prior CI findings include GitHub idle observation, route teardown, attention cleanup, touch-picker E2E, workspace health, and voice-cleanup timing; local passes do not erase those failures. No full local repository-suite, deployed sleep, fleet cost, or live Copilot qualification claim is made. </details> ## Risks - Signals cover supported in-process writers. They do not supply change capture for independent SQL or other replicas. - Real company-wide changes can inspect active candidates. Coarse maintenance hints do not activate settlement. Configured idle and retained history must remain quiet. - Monitor receipt binding and admission order must preserve company, agent, issue, and exact schedule identity. Current parent handoff locks and source-generation checks remain intact. - The local restored-work timeout is retained above. A fresh full-file pass does not establish its cause; latest-head CI must also pass. - Generic recovery and Muse/resource duties remain separate slices. This does not make the whole server idle. Existing Cloud sleep blockers remain. No deployed sleep, database suspension, external alarm coverage, fleet cost, or live provider qualification claim is made. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, code execution, PostgreSQL tests, and GitHub tools. The exact runtime model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Codex <noreply@openai.com> |
||
|
|
8ac5a128e6 |
fix(server): resolve native finalization notices from committed wakes (#17758)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native finalization can commit before its recovery notice is resolved. > - The periodic orphan pass revisits committed runs to resolve those notices. > - Retained completed history should not require repeated discovery. > - This PR moves that one duty to transaction wakes and concrete failure retries. > - Existing notice rows reconstruct work after restart. The action service still owns resolution. > - Other completion and resource duties keep their current recovery paths. ## Linked Issues or Issue Description Refs #17752 (stack parent: committed run projection), #17748 (accepted-result commit), and #17734 (workspace finalization). These are distinct recovery obligations. This slice incorporates parent #17752 at `a49a2dac13b0e1c19da5d3d6b435c781703b1c0b`, now merged as `50fe32684d2d4e1db129e3c1bb455571344a0c09`. Related #17724 retires obsolete exact-session notices for terminal runs with no result and preserves their replay hold. This slice resolves notices after a result committed; it does not change that no-replay policy. **Current behavior** Committed native finalization recovery repeatedly inspects active watchdog notices. A notice can also be updated in place to describe a different run, so resolving a previously selected ID alone is unsafe. **Proposed behavior** Wake a scoped worker from relevant committed writes. Reconstruct active finalization notices at startup. Resolve only the current notice after the native owner rechecks its binding under lifecycle locks. Stop scheduled work when no matching obligation remains. **Reason and benefit** Notice recovery becomes quiet after settlement and reacts to real changes without a polling delay. Reused notices remain attached to their current incident. This does not claim that every remaining server duty or the host can sleep yet. ## What Changed - Track the initial recovery-notice sweep through shutdown without blocking HTTP readiness on task locks. - Require issue/run scope for ordinary finalization fanout. Routine company-only hints leave dormant notices quiet; dedicated owner and explicit settings signals retain their scopes. - Extract the native finalization notice predicate and resolution owner. Keep the existing recovery-action service responsible for writing the resolution. - Bind candidates by company, issue and run. Require a committed result and an active or escalated finalization notice. Exclude held ownership, pending status effects and unfinished run projection. - Recheck coordinator, issue, run and matching actions under locks. A newer incident cannot be resolved from a stale candidate snapshot. - Use the shared scoped worker, transaction signals, retry backoff and lifecycle handling. Add no table, per-feature timer or provider branch. - Signal action insertion, update, supersession and resolution; finalization and effect changes; run projection and ownership release; issue changes; settings activation; and the recovery route that reopens an action. - Preserve no-op resolution quiet. Serialize only the three supported notification IDs, even when a caller supplies a larger internal object. - Delegate only notice resolution from the periodic path. Keep manual/default finalization and other completion/resource work. - Add real PostgreSQL producer, concurrency and lifecycle coverage and a detailed audit. ## Verification - Current local source `730e00a7b007a36d188ca0a328ef050d3a25d1ee` incorporates the parent with a normal merge. All 1,209 parent-only, eight owned-only and 12 automatic shared paths are preserved. The two manual paths retain both chat import sets and scope the new notice fanout. The entire package tree matches parent profile 80 byte-for-byte. - Controlled real-database regression: original notice-only unscoped fanout produces three candidate SELECTs for routine company hints. The same zero-query assertion passes after restoring scoped fanout. The real notice writer then creates and resolves an action for a committed run and returns to quiet. Both phases restore production bytes exactly; the 22 cases outside this focused pattern are not counted as passing there. - The complete main command passes **1,873 cases in 21 files**, without skips, in 309.69 seconds. It includes all 23 notice-worker cases, full presentation/ownership read coverage, 88 startup cases, and the projection/result/physical-work consumers. Process recovery passes 410 in 163.54 seconds and external-chat response passes 273 in 53.48 seconds. Total complete-file acceptance: **2,556 cases in 23 complete files**, with no skips. The additional focused chat selection passes 18 cases in 13.44 seconds; 1,796 unselected chat cases do not count. Total selected passes: **2,574**. Normal merge `a0f2b3f4348b078219a53d8cc135c84202ba505e` adopts actual parent ancestry with this complete tested child tree unchanged. - The chat selection covers the real reply/notice row-lock regression, Slack and Telegram response recovery, current-authority denial cases, exact canonical response binding and internal GitHub attachment navigation. Other cases in the giant chat file remain unselected; this is not full chat-suite acceptance. - Production server typecheck, build, module boundaries, migration order and source identity checks pass. Twenty dedicated PostgreSQL 17 databases use the unchanged migration 413 schema. Embedded-only invocations use the existing fresh-per-call recovery helper. All temporary imports restore exactly and all 24 selected test hashes match after completion. No assertion or timeout is relaxed. - Parent package acceptance remains applicable: 130 passing installation/compatibility/transport cases, 22 installed ACPX/Grok contract cases and 26 native-provider Rust cases. One existing opt-in closed Pi snapshot case is skipped because its distribution root is unset. This does not qualify a live provider. - Failure history and scope are retained in `doc/plans/2026-10-26-native-recovery-notice-sleep-audit.md`. Current-head acceptance: all 57 checks are clean, including protected CI and security. Greptile gives 5/5 on `2a8777bbed8b1e10810e3d6d99aae3f25b4854ef`, with no actionable findings. Prior scores do not substitute. No Ponytail verdict is present under the authorized missing-review policy. No full local repository suite, deployed sleep, expanded Cloud wake coverage or measured savings is claimed. ## Risks - Resolution takes coordinator, issue, run and action locks in the native recovery order. Contention must preserve the current incident; the real concurrent-upsert and competing-resolver cases exercise this boundary. - Process-local signals cover instrumented application writers. They do not add support for independent SQL writers or replicas without supported ingress. Startup reconstructs the existing durable rows. - Completed finalization is not proof that external conversation settlement, Muse reconciliation or physical cleanup finished. Those obligations retain their owners and recovery paths. - Discovery uses narrow keyset pages. No fleet benchmark, host suspension or cost reduction is claimed. Cloud wake coverage remains unchanged. - Parent #17752 has merged. This PR targets master at `50fe32684d2d4e1db129e3c1bb455571344a0c09`. Fresh current-head CI and review must pass before the authorized protected merge. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The exact serving model ID and context-window size are not exposed in this session. No subagent was used for this slice. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Codex <noreply@openai.com> |
||
|
|
50fe32684d |
fix(server): wake committed native run projection from durable changes (#17752)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A native run has separate result acceptance, task decision, run status, and resource settlement steps. > - A task decision can commit before its run status is saved. > - Recovery currently revisits committed decisions through the periodic orphan pass. > - This PR gives the existing run projection repair its own transaction-driven worker. > - Existing records reconstruct unfinished work after restart. Completed history becomes quiet. > - Other completion and resource obligations keep their existing recovery paths. ## Linked Issues or Issue Description Refs #17748 (stack parent: accepted-result commit), #17734 (workspace finalization), and #17717 (retained session cleanup). Related #17614 preserves terminal notifications. This slice is prepared on current parent #17748 at `b2efdb2dbbbef5b2227ffa2bd2a9a6cfe3867fe2`; the parent has merged as `28e38da7e6bdd6af4092dda56bc2be05bed47af5`. **Current behavior** The native recovery pass revisits committed decisions to repair the heartbeat-run status and metadata. A crash between decision and run writes creates real work, but periodic scanning also revisits retained history. **Proposed behavior** Wake projection repair after committed producer changes. Reconstruct mismatches on startup and settings activation. Retry concrete failures through the shared scheduler. Keep the run projection before resource settlement. **Reason and benefit** The repair worker performs no recurring SQL after its work settles. A real decision, effect settlement, or ownership release reactivates it. This is one worker migration; it does not establish host or database suspension. ## What Changed - Extract the existing projection predicate and write into their native-runtime owner. The finalizer keeps initial metadata enrichment and task decision authority. - Discover accepted, committed, mismatched run projections with company, issue and run bindings. Exclude pending status effects and held runner ownership. Read narrow fields instead of whole result payloads. - Use existing transaction hints, scoped delivery scheduling, failure backoff and lifecycle handling. Add no tables or per-feature timers. - Require issue/run scope for ordinary finalization fanout. Routine company-only hints leave dormant projection quiet; explicit settings and dedicated producer signals keep their existing contract. - Connect finalization, pending effects, lifecycle status writes, issue changes and worktree execution activation. Preserve outer transaction commit and rollback behavior. - Lock the current run before deciding which terminal event and completion time to save. Concurrent live and recovery projection must not announce the same transition twice. - Delegate only already-committed projection from the periodic pass. Its candidate scan uses the same pending predicate to defer remaining consequences and cleanup until projection finishes. - Add real PostgreSQL producer, race and lifecycle tests plus an owner audit. ## Verification - Current local source `a83b9449233a275176e637c765059f414305d7c1` incorporates the parent with a normal merge. All 1,208 parent-only, seven owned-only and ten automatic shared paths are preserved. The only manual conflict keeps new projection fanout inside the ordinary issue/run guard. The complete package tree matches parent profile 80 byte-for-byte. - Controlled real-database regression: the original projection-only unscoped fanout produces three candidate SELECTs for routine company hints. The same zero-query assertion passes with scoped fanout. A real committed result then repairs the run and emits one terminal event before returning to quiet. Production bytes restore exactly in both phases. Twenty cases outside that focused pattern are not counted as passing there. - The complete main command passes **1,830 cases in 19 files**, without skips, in 301.14 seconds. It includes all 21 projection cases, 85 startup cases, required quality review and heartbeat run lifecycle, plus physical-work and result-commit consumers. Separate process recovery passes 410 in 163.58 seconds and external-chat response passes 273 in 53.26 seconds. Total: **2,513 distinct passes in 21 complete files**, with no skips. Normal merge `d4b46fd29cb23126d556cc3aeb883e6b41f1a8a5` adopts the actual parent squash with this entire tested child tree unchanged. - Production server typecheck, build, module boundaries, migration order and source identity checks pass. Seventeen dedicated PostgreSQL 17 databases use the unchanged migration 413 schema. Embedded-only calls use the existing fresh-per-call recovery helper. All temporary imports restore byte-exact and all 21 selected test hashes match after completion. No assertion or timeout is relaxed. - Parent package acceptance remains applicable: 130 passing installation/compatibility/transport cases, 22 installed ACPX/Grok contract cases and 26 native-provider Rust cases. One pre-existing opt-in closed Pi snapshot case is skipped because its distribution root is not configured. No live provider qualification is claimed. - The owner audit at `doc/plans/2026-10-26-native-run-projection-sleep-audit.md` retains prior failures and their disposition. Current-head acceptance: all 57 checks are clean, including protected CI and security. Greptile gives 5/5 on `a49a2dac13b0e1c19da5d3d6b435c781703b1c0b`, with no actionable findings. Old scores do not substitute. No Ponytail verdict is present under the authorized missing-review policy. - These checks do not claim a full local repository suite, deployed suspension, expanded Cloud wake coverage or measured cost savings. ## Risks - Process-local signals cover instrumented application writers. Independent SQL writers and replicas without supported ingress are not newly covered. Durable startup reconstruction remains enabled. - Projection must precede remaining committed resource settlement. The real reconciler test proves the periodic path defers cleanup while projection is pending and resumes it after repair. - Status decisions, external conversation settlement, watchdog resolution, Muse reconciliation and resource release keep their own authority and recovery. A committed result alone is not proof of completed cleanup. - Candidate discovery uses narrow bounded pages. There is no large-fleet performance or deployed sleep claim. Existing Cloud wake coverage and sleep blockers remain unchanged. - Parent #17748 has merged. This PR targets master at `28e38da7e6bdd6af4092dda56bc2be05bed47af5`. Current-head CI and review must pass before the authorized protected merge. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The exact serving model ID and context-window size are not exposed in this session. No subagent was used for this slice. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Codex <noreply@openai.com> |
||
|
|
28e38da7e6 |
fix(server): wake native result commits from durable work (#17748)
## Thinking Path > - Paperclip manages AI agents and preserves their results through failures. > - Native completion must save workspace output before it commits the accepted result. > - The orphan recovery pass repeatedly revisits this work even when the subsystem is idle. > - Accepted results, workspace receipts, retry dates and leases already provide durable truth. > - This PR wakes a scoped worker from those writes and schedules only real unfinished work. > - It preserves existing finalization authority and keeps later projection and resource recovery separate. ## Linked Issues or Issue Description Refs #17734 and #17717. This is slice 53 of the existing sleep stack, targeting master after parent #17734 merged at `a019d00a393b57f531afe8dd8ab0b8a1820d1c4c`. A current search found no duplicate accepted-result commit PR. **What problem does this solve?** Native accepted-result recovery currently depends on periodic orphan scanning. Quiet instances keep doing work to discover that no result needs a commit. **What improvement do you propose?** Use the shared delivery coordinator with transaction-bound producer signals. Reconstruct unfinished results at startup. Preserve saved retry and lease deadlines. Remove only uncommitted-result execution from the periodic pass. **Why is this useful?** The accepted-result worker becomes quiet with no work or retained completed history. Real results and completed workspace operations reactivate it without a polling delay. This does not claim that the whole server or database can sleep yet. ## What Changed - Track the initial result-commit sweep through shutdown without blocking HTTP readiness on recovery locks. - Keep ordinary company-only finalization hints from reconstructing dormant result work. Existing company settings and dedicated owner signals remain supported. - Add a native result-commit worker. Bind candidates by company, issue, run, accepted result and a successful workspace receipt. Exclude committed, terminal, held and pending-effect work. - Signal both accepted-result writers and workspace receipt writes before their transaction changes. Extend the existing finalization fanout for retry, lease, effect and completion transitions. - Add an optional atomic due-date check to the existing finalizer claim. A lease collision or later retry does not consume a failed attempt. - Signal native retry cancellation so it removes the saved wake after commit. - Register through the existing heartbeat suppression and shared delivery lifecycle. Preserve outer commit/rollback, drain/resume, settings activation and shutdown behavior. - Delegate only uncommitted-result execution from the periodic orphan pass. Keep committed projection and resource settlement recovery enabled. - Add real producer/lifecycle tests and a detailed audit. No new tables, provider branches or per-feature timers. ## Verification - Current local source `2af88fa55f961a74340880240019f6cd31e21bf1` incorporates the parent with a normal merge. All 1,203 parent-only, nine owned-only and 12 automatic shared paths are preserved. The only manual conflict keeps the new result fanout inside the existing ordinary issue/run scope guard. The complete package tree matches parent profile 80 byte-for-byte. - Controlled real-database regression: the original result-only unscoped fanout produces three candidate SELECTs for routine company hints. The same zero-query assertion passes after restoring the scoped fanout. A real workspace receipt and accepted result then commit exactly one decision and return to quiet. Red and green restore production bytes exactly. Twenty cases outside the focused pattern are not counted as passing there. - The complete main selection passes **1,746 cases in 16 files**, without skips, in 277.97 seconds. This includes all 21 result-worker lifecycle cases and 82 startup cases, including held result recovery, HTTP readiness and shutdown joining. Separate process recovery passes 410 cases in 162.42 seconds and external-chat response passes 273 in 53.41 seconds. Total: **2,429 distinct passes in 18 complete files**, with no skips. Normal merge `998239b019defe52e7b5d64f884de4e871ccd736` adopts the actual parent squash and preserves this complete tested child tree. - Production server typecheck, build, module boundaries, migration order and source identity checks pass. Fifteen dedicated PostgreSQL 17 databases use the unchanged migration 413 schema. Embedded-only calls use the existing fresh-per-call recovery helper. All temporary imports restore byte-exact, and every selected test hash matches after completion. No assertion or timeout is relaxed. - Runner validation belongs to the unchanged parent package tree: 130 passing installation/compatibility/transport cases, 22 installed ACPX/Grok contract cases and 26 native-provider Rust cases. One existing opt-in closed Pi snapshot case is skipped because its distribution root is not configured. This does not claim live provider qualification. - Earlier Greptile 4/5 on `656f6722be1aedf3d52b7f853956e15838204278` correctly identified startup blocking on result recovery. The prior branch `108cf0dc8296bf451b721ddac55ecbcd035d00f0` already tracks its ready promise through the existing lifecycle owner. Current full startup acceptance verifies the fix in the merged context. The later Greptile service error is not a pass; current-head acceptance now passes: all 57 checks are clean, including protected CI and security. Greptile gives 5/5 on `b2efdb2dbbbef5b2227ffa2bd2a9a6cfe3867fe2`, with no actionable findings. No Ponytail verdict is present under the authorized missing-review policy. - The owner audit at `doc/plans/2026-10-26-native-result-commit-sleep-audit.md` retains earlier failures and their disposition. These checks do not claim a full local repository suite, deployed app/database suspension, expanded Cloud wake coverage or measured cost savings. ## Risks - Process-local signals cover instrumented application writes in the owning server process. They do not cover independent SQL writers or another replica without supported ingress. Durable startup reconstruction remains enabled. - The initial native decision and its run projection are separate transactions. The periodic reconciler still repairs already-committed results and releases resources. This PR must not be used to infer that committed means fully settled. - Candidate discovery adds bounded, narrow reads at startup and after signals. No large-fleet query benchmark is claimed. - Existing Cloud alarm coverage, host sleep safety and resource blockers are unchanged. No deployed sleep or savings claim is made. - Parent #17734 has merged. This PR targets master at `a019d00a393b57f531afe8dd8ab0b8a1820d1c4c`. Fresh current-head CI and review remain required before merge. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The exact serving model ID and context-window size are not exposed in this session. No subagent was used for this slice. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Codex <noreply@openai.com> |
||
|
|
a019d00a39 |
fix(server): wake native workspace finalization from committed work (#17734)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native runs can accept a result before workspace copy-back finishes. > - The periodic orphan pass repeatedly tries this recovery, including cases that require operator action. > - Copy-back needs exact ownership evidence. A generic timer cannot grant that authority. > - This pull request wakes workspace recovery from committed changes and local owner completion. > - Saved retries use the existing scheduler. Empty work, completed barriers, and operator holds become quiet. ## Linked Issues or Issue Description Refs #17717. This is slice 52 of the recurring-server sleep stack, targeting master after parent #17717 merged at `4c56702e06c698cc10d4a86514669aba0f4d9a9b`. Related work: #17709 (status reassessment), #17687 (unsafe export recovery), and #16478 (workspace repair holds). This PR covers native workspace copy-back and validation. Result commit/projection and resource settlement remain separate recovery duties. It does not claim full-instance sleep. ## What Changed - Register a company/run/task-scoped workspace worker. Reconstruct accepted, unfinished work from existing records. Use saved retry and lease deadlines. - Notify from both accepted-result writers (control-plane and PRP coordinator), workspace receipts, coordinator changes, operator confirmation, retained workspace repair, and relevant task updates. Notifications follow the outer transaction. - Distinguish active local ownership, busy external ownership, pending Cloud evidence, operator-required evidence, and changed bindings. Only concrete external ownership/evidence waits receive a scoped retry. - Preserve advisory locks, signed stop proof, exact binding checks, physical callback joins, and existing failure policy. Join physical operations during shutdown. - Respect the existing heartbeat execution-suppression policy before discovery and execution; wake suppressed work on settings activation. - Skip physical workspace recovery in the enabled periodic orphan pass. Keep direct/startup reconciliation and the remaining result/cleanup recovery paths. - Keep routine company-only finalization hints quiet. Ordinary work requires issue/run scope; explicit settings and owner-release events keep their company-wide contract. - Preserve the parent's entire Runner/package tree, profile 80 and all historical identities. Add the dated obligation, producer, lifecycle and limitation audit. ## Verification - Current local source: `4059b24e3ea5c623be54ade18ec9b6ab78eb0289`, prepared by a normal merge of the current parent. The sole manual conflict keeps ordinary workspace fanout inside the existing issue/run guard. All 1,198 parent-only, 15 owned-only and 10 automatic shared paths are preserved. The complete package tree matches the parent byte-for-byte. - Controlled regression: the original workspace-only unscoped fanout performs three candidate SELECTs for routine company hints. The same zero-query assertion passes with scoped fanout. A real accepted result then produces one successful workspace settlement and returns to quiet. The failed baseline remains in the audit. - The first complete local group passes **1,693 cases in 14 files**, including all 19 workspace-worker cases. It covers both accepted-result producers, physical ownership, export retry/failure, finalization recovery, arbiter corpus, issue recovery actions, control-plane port, workspace recovery, heartbeat recovery, startup, retained cleanup and native execution. - Separate process-recovery and external-chat response files qualify the changed recovery/settlement paths without sharing their general test database with other files. Both pass in full: 410 process-recovery cases in 162.79 seconds and 273 external-chat cases in 53.79 seconds. The total is **2,376 distinct passes in 16 complete files**, with no skips. Normal merge `5c46c52873158529736b8349e4d44658865108ee` adopts the parent squash with this entire tested child tree unchanged. - Production server typechecking, build, module boundaries, migration order and source-profile checks pass. Thirteen dedicated PostgreSQL 17 fixtures use the unchanged migration 413 schema. Embedded-only paths use the existing recovery helper with fresh per-call fixtures. All temporary imports are restored byte-exact. No test assertion or timeout is relaxed. - Runner validation belongs to the unchanged parent package tree: 130 passing installation/compatibility/transport cases, 22 installed ACPX/Grok contract cases and 26 native-provider Rust cases. One existing opt-in closed Pi snapshot case is skipped because its distribution root is not configured. These are source-scoped checks, with no live provider qualification claim. - The owner audit at `doc/plans/2026-10-26-native-workspace-finalization-sleep-audit.md` preserves earlier failures and their disposition. Current-head acceptance: all 57 checks are clean, including protected CI and security. Greptile gives 5/5 on `7df5c9c44ea9e171be3c61b4430d3727b10f3337` with no actionable findings. Earlier scores do not substitute for this result. No Ponytail verdict is present under the authorized missing-review policy. - No claim of a full local repository suite, deployed app/database sleep, expanded Cloud wake coverage or measured cost savings is made. ## Risks - Supported wake producers are in-process application services. This adds no cross-replica or independent SQL-writer transport. - A real unresolved export still needs recovery. Busy owners and pending Cloud stop evidence retain a 30-second scoped check. No timeout authorizes copy-back. - Result projection and resource release still use their existing recovery. Some existing cleanup failures remain best effort; this PR does not infer cleanup success from a workspace barrier. - Cloud alarm coverage and host-sleep safety remain unchanged. No deployed suspension or cost savings have been measured. - Parent #17717 is merged. Current base is master at `4c56702e06c698cc10d4a86514669aba0f4d9a9b`. Fresh current-head CI and review must pass before this slice merges. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The exact serving model ID and context-window size are not exposed in this session. No subagent was used for this slice. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Codex <noreply@openai.com> |
||
|
|
4c56702e06 |
fix(server): schedule retained native-session cleanup (#17717)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native runs can retain a local session after an interrupted durable suspend. > - The orphan reaper repeatedly scans completed history for that cleanup duty. > - Idle scans keep the database active. Removing them requires complete wake and ownership paths. > - This pull request moves retained-session cleanup to committed signals, local release events and saved deadlines. > - Existing physical proof checks still decide whether cleanup is safe. The shared scheduler does not grant execution authority. > - The benefit is quiet retained history with recoverable cleanup and owned shutdown. ## Linked Issues or Issue Description Refs #17709 and #18135. This is slice 51 of the existing recurring-server sleep stack. Both referenced dependencies have merged. The target is `master`. **What happened?** The periodic orphan pass scans retained native sessions even after their cleanup is complete or no automatic action is safe. Some eligible sessions also depend on a local session, environment lease or original maintenance callback releasing ownership. An earlier decision notification can be consumed before the run's later terminal projection commits. **Expected behavior** Reconstruct existing work at startup. Wake from committed writes and actual owner release. Register the saved claim expiry or a probe for a specific live process. Stop recurring discovery when no obligation remains. Preserve original callback ownership and all existing cleanup proof checks. **Steps to reproduce** 1. Retain completed native runs with a durable-suspend close failure. 2. Let the periodic orphan pass run with no new work. 3. Observe repeated cleanup discovery. Separately, delay terminal projection after its decision notification is consumed and observe the missing cleanup wake. **Paperclip version or commit** Parent #17709 at reviewed head `3b75f4bc1da1370304f069fc287af3d693575716`, landed as `df6156c2e37f542a69f393dde31e357b82b7cac1`. Current integrated master: `31b927e3013a8a4b3266854b740cf40a24753788`. ## What Changed - Move scoped, paged discovery to the retained cleanup owner. Preserve explicit manual recovery. - Register the duty with the shared scheduler and remove it from the periodic orphan pass. Keep concrete lease expiry and live-process probe obligations. - Use typed outcomes for settled/refused work and process, session, environment, maintenance or lease waits. Keep physical proof and quarantine decisions in the executor. - Wake from committed finalization and terminal projection writes, durable process/lease release, and local session or callback settlement. Routine company-only finalization hints leave dormant cleanup quiet. - Join duplicate cleanup and serialize inspections on one database lane. Keep original Runner callbacks owned until settlement and shutdown. - Preserve all master profile fixtures and the branch's historical identities. Save the colliding sleep profile 79 separately. Generate combined Copilot profile 80 from its source and retain master's other provider profiles. - Add the owner audit and regression coverage. No new table, migration or independent maintenance timer is added. ## Verification - Server acceptance: **2,342 distinct passing cases in 18 complete files**. The 14-file unaffected group contributes 1,532 passes. Full startup contributes 76, process recovery 410, environment service 51, and external-chat response 273. The 29-case cleanup worker suite and 977-case executor suite run in full. No case in these files is skipped. - Controlled scope regression: removing only the ordinary finalization issue/run guard wakes dormant retained cleanup; the same assertion passes after restoring it. A real local owner release still resumes the durable candidate. - Preserve the first broad failure: 1,607 passed and one startup shutdown case failed because the coordinator mock lacked `stop()`. Add only that mock method. All 76 unchanged startup cases then pass. Do not treat the first broad command as green. - Dedicated PostgreSQL 17 databases use the existing migration 413 schema. Embedded-only fixtures use the existing recovery helper with fresh per-call databases. All temporary imports are restored byte-exact. No assertion or timeout is relaxed. - Current-master integration preserves both complete source histories. The server, UI and database trees are identical to the server-tested source. All master historical profile fixtures remain byte-exact. The prepared sleep profile 79 remains independently inspectable. - Current runtime validation at `d08f764e3efe75e96929173afe7ee8739121ad9f` passes all 12 commands (22 installed ACPX/Grok contract cases and 26 native-provider Rust cases): `pnpm install --frozen-lockfile --ignore-scripts`; Copilot/Grok identity and generated profile checks; Runner and server typechecking; server build; Copilot/Pi installation, profile compatibility and transport settlement files (130 passed; one opt-in closed Pi snapshot case skipped because its distribution root is not configured); installed ACPX model/reconnect package contract and Grok identity tests; Rust native-provider backend tests; module boundaries and migration order. - Earlier 94-case installation/settlement acceptance applies to the prepared profile 79 source. It is retained as history and does not substitute for the current profile 80 validation above. - Failure history and duty-specific evidence remain in `doc/plans/2026-10-26-retained-native-session-cleanup-sleep-audit.md`. Exact-current-head acceptance: all 60 checks are clean, including protected CI and security; Greptile gives 5/5 for `281d579de0d08e907194b8b790a0597b07d16b7b` with no actionable findings. No Ponytail review is present under the authorized missing-review policy. - These checks do not claim a full local repository suite, live Copilot or Daytona qualification, deployed host/database suspension, or measured cost savings. ## Risks - Missing a supported producer could strand cleanup. Tests exercise real finalizer, heartbeat, environment and process writers. Process-local signals cover in-process writers; no multi-replica or external SQL notification transport is added. - A timeout can leave an original callback active. Cleanup and shutdown must keep waiting for it. The existing global maintenance drain is conservative across roots. - A retained live process still has a scoped 30-second absence probe. A saved lease keeps its actual deadline. Operational errors retain retry work. Immutable proof refusals and operator-required histories do not gain automatic retry authority. - Cleanup keeps all existing artifact, identity, claim, no-launch and quarantine checks. Terminal status alone does not authorize physical cleanup. - This PR does not expand Cloud alarm coverage or relax host sleep eligibility. It proves worker behavior, not deployed app/database suspension. - The Copilot source identity has changed; these checks do not claim new live provider qualification. - The parent has merged. This PR is merged only after its current-head protected CI and review gates pass under the existing campaign authorization. ## Model Used OpenAI Codex, GPT-6 family, with code execution and repository tools. The exact serving variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Codex <noreply@openai.com> |
||
|
|
df6156c2e3 |
fix(server): wake native status reassessment from committed changes (#17709)
## Thinking Path > - Paperclip manages agents, tasks and their durable native results. > - Changed evidence, task authority or review retirement can require a new status assessment. > - Periodic orphan recovery repeatedly reads history to find these changes. > - This change gives reassessment an owner driven by committed changes and saved deadlines. > - Existing result, assessment, decision and task rows remain durable truth. > - The status committer still verifies current sources and authority before writing. > - Settled or insufficient evidence becomes quiet until a supported producer changes it. ## Linked Issues or Issue Description Refs #17702 (merged), #17679 and #17680. This is slice 50, now based on master. It owns native status reassessment; workspace settlement, retained sessions and other orphan duties remain separate. **What happened?** Periodic recovery repeatedly inspects completed native results. Removing that scan without complete writer coverage or current authority checks could strand new evidence or commit an old result over newer work. **Expected behavior** Reassess from existing durable facts after relevant outer transactions commit. Retain real retry/lease deadlines and restart/drain recovery. Preserve current authority, quality-review and supersession checks. Make no recurring SQL when work is settled or dormant. **Steps to reproduce** Complete a native result awaiting evidence or review, leave it idle, then approve a real work product or retire the selected automatic review. Its saved result should advance once when authorized, or record the existing nonterminal business gate, without a periodic scan. ## What Changed - Extract reassessment and its pure status policy from finalization recovery into their owning modules. Add a scoped worker using the existing delivery coordinator and scheduler; no queue table or migration is added. - Retain the three durable discovery triggers: changed work-product history, authoritative task status/version, or retirement of the selected automatic completion review. Preserve selected-source, ownership, effects, retry and lease fences under the existing committer. - Notify from actual task/work-product, claim release, approval, interaction, OAuth, event evidence and workspace-settlement owners before their transaction writes. Keep provider calls outside short transactions and nested writes behind outer commit. - Scope ordinary milestone/finalization/coalesced-input wakes to an issue/run. Keep genuine company authority restoration explicit. Real Photon and coalescer regressions cover all three response/status queues. - Move the current parent's required quality policy into the owner and apply it to every reassessment that could complete a task. An unmet review records the existing in-progress decision and next action, avoiding an operational retry loop. Preserve conversation/session reset locks and the workspace omission-barrier signal. - Skip reassessment in periodic recovery while preserving explicit/startup recovery and the historical supersession guard before independent workspace work. Document the source contract, producer inventory, failures and qualifications in the linked audit. ## Verification - Broad source `9fddb3f0b1157df32b64fb1f22e9d6e9624470e8` passes **2731 distinct cases in 28 complete files**, with no skips: 2458 producer/owner/authority/workspace/quality cases in 394.43 seconds and all 273 governed-response cases in 53.74 seconds. Nineteen dedicated fresh PostgreSQL 17 databases migrate through 0413; eighteen embedded-only invocations preserve fresh-per-call isolation through the existing recovery helper. All 22 temporary imports restore exactly and all 28 hashes match. - Final runtime source `5ea3b0fd016bf7e63ba82a7044825a46a0cd2082` passes **1211 cases in five complete affected files** in 108.23 seconds after the narrowly reproduced evidence-trigger fix: all 43 worker, 145 corpus, 32 finalization-recovery, 968 native-session and 23 quality-review cases. Four fresh databases plus the quality helper's own fresh fixture use PostgreSQL 17. Its one adapted import restores exactly; all five hashes match. The other 27 broad-run test files are unchanged. These qualifications are source-scoped and not cumulatively double-counted. - Required/optional real regressions reproduce retry loops on retired-review and evidence-approval paths before repair; both become SQL quiet afterward with unchanged assertions/time limits. Controlled removal of only ordinary scope guards reproduces two coarse-hint SQL failures; exact guard restoration passes. Real producer, outer transaction, claim, authority, 103-item backlog, retry and drain cases pass. Photon transport is mocked; callbacks, locks and observed SQL are real. - Normal merges preserve 12661 parent/unrelated entries, 35 owned entries and 21 automatic shared results, with five original conflicts reviewed explicitly. The parent's actual squash reconciliation preserves the entire held child tree. The final quality fix changes only its owner and tests. Fourteen added comments are removed with identical nontrivia tokens; no directives are removed and the final comment check is empty. - Database build, server types/build, boundaries and exact-head migration order pass. The database tree is identical to merged #17702: no new SQL, queue table or snapshot is generated. Earlier benchmarks, host limitations, failed assertions and their repairs remain in `doc/plans/2026-10-26-native-status-reassessment-sleep-audit.md` and `doc/idle-native-status-reassessment.md`. - This qualifies targeted external PostgreSQL behavior, not embedded bootstrap on the semaphore-exhausted host, a full local repository suite, deployed sleep or measured fleet savings. Fresh exact-head CI and review remain required. ## Risks - Notifications retain the process-local supported-writer contract. Independent SQL writers and uncoordinated replicas are not covered by a new transport. - Reassessment competes with current authority changes. Existing row/source locks and retries remain required; arbitrary missing identities have no row to lock, so this is not a general predicate-lock guarantee. - Discovery pages 100 rows but retains outstanding candidates in memory. It remains proportional to backlog size, and cancellation occurs between queries. - Other orphan/workspace/session duties and Cloud sleep eligibility remain unchanged. Missing supported producer coverage could strand dormant work, so the producer audit and real callback/transaction tests remain part of acceptance. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, shell execution and PostgreSQL tests. The runtime does not expose a more specific serving snapshot or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (targeted scope is stated above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Codex <noreply@openai.com> |
||
|
|
79ce86630a |
fix(server): schedule governed review response recovery (#17702)
## Thinking Path > - Paperclip manages agents, tasks and their external chat responses. > - A completed chat turn can owe a response while an earlier review gate remains pending. > - Periodic recovery revisits that response history even when no work can advance. > - This change uses the existing delivery coordinator and saved retry deadlines. > - Approval, interaction and authority changes wake the affected issue or run after commit. > - The existing presentation owner verifies the durable result and current authority. > - Settled and dormant history becomes quiet without losing future eligible responses. ## Linked Issues or Issue Description Refs #17696 (merged) and #15625. This is slice 49, now based on master. It owns governed-review response presentation. Accepted responses and pending decision effects keep their existing owners. **What happened?** Native finalization recovery repeatedly inspects committed review responses. Removing the scan without covering approval, interaction, task-policy and authority changes would strand responses that become eligible later. **Expected behavior** Recover an owed response once from existing durable facts. Enforce authority at publication, wake after relevant outer transactions commit, retain real retry deadlines and restart/drain recovery, and stop recurring SQL when no response can advance. **Steps to reproduce** Complete an external chat turn while an earlier completion review remains pending. Block its response behind an approval or interaction, then resolve that blocker. The response must appear once without a periodic scan or resolving the original review gate. ## What Changed - Add a scoped governed-review worker using the existing coordinator, finalizations, decisions and comment receipts. No queue table or transaction wrapper is added. - Keep recovery and publication policy in the native finalizer. Preserve binding, source/session authority, ownership, deadline and deleted-comment fences; match coordinator-before-issue lock ordering. - Notify through actual approval, interaction, OAuth, task-policy and lifecycle producers before their owned writes. Serialize approval resolution with task linking so a concurrent link retains its wake. Nested transactions still settle at outer commit. - Scope ordinary milestones, finalization and coalesced input to their issue/run. Use the explicit company authority boundary for genuine global restoration, including membership changes. Both response workers share this boundary. - Register under maintenance and drain gates, disable the two periodic repair entrances, and retain fresh finalization and explicit recovery. Add the concurrent 0413 comment-receipt index and migration registration; retain five snapshots. - Preserve the producer inventory, source qualifications and earlier failures in the owner audit. Retain a real Photon regression proving routine ownership assertions stay quiet and concurrent authority restoration wakes each worker once. ## Verification - Scoped repair source `2341714720b5c273f0ce011b48345af0ef39227f` passes **2053 distinct cases**: 1703 in 15 complete producer/authority/recovery files, including all 273 governed-response cases and 95 wake-queue adapter cases, plus 350 selected chat cases. Another 1463 chat cases are outside that selection. Thirteen fresh PostgreSQL 17 fixtures migrate through unchanged 0413; the one temporary queue-fixture import restores exactly. All sixteen captured hashes match. - Two actual repeated coarse-hint SQL regressions fail before repair and pass afterward. Actual membership restoration passes before and after. The same assertions and time limits prove dormant work stays quiet and real authority restoration still publishes once. Real coalesced input covers both queues after commit. - Combined source `586f5674cd9798b33cd22437255e0480b5726d20`, including independent master #18129, passes **1297 distinct cases**: 1093 in five complete server files, 185 in five runner files, 14 Node protocol cases and five UI registry cases. This includes the complete 22-case Photon file for both response workers. The external Photon transport is mocked; Paperclip callbacks, transaction fences and observed SQL are real. The earlier and combined case counts are separate, with no cumulative double-counting. - A normal merge retains both complete owned and independent master binary patches. All twelve new-context hashes and sixteen earlier repair hashes match. Three dedicated fresh 0413 fixtures plus the live-event helper's own fresh database use PostgreSQL 17 without source adaptations. Database build, server types/build, boundaries, full-SHA migration order and generated TS/Rust profiles pass. The new inherited Runner vendor inputs rebuild successfully. - Database bytes remain identical to the previously qualified 0413 source. The recorded populated 0412-to0413 upgrade retains all 411 prior receipts and 38 companies, appends one receipt and none on replay; the index is valid and ready. The earlier 2828-case acceptance, CLI journey, setup failures and their repairs retain their recorded source scope in `doc/plans/2026-10-26-governed-review-response-sleep-audit.md`. - Parent #17696 merged with 58 clean checks, Greptile 5/5 and Ponytail 5/5. Its Photon warning was disproved by real callback/SQL evidence and the security check became successful. This PR still requires its own fresh exact-head CI and review gates. No deployed sleep, external alarm or fleet savings claim is made. ## Risks - Notifications use the existing process-local writer contract. Independent SQL writers and uncoordinated replicas are outside its coverage. - Genuine startup and company authority reconstruction remain proportional to the owed backlog. Queries page 100 rows, but discovery accumulates the candidates; this is not a constant-cost global operation. - Current publication authorization, durable binding, ownership and lock ordering remain mandatory. The worker does not resolve the original review gate. - The concurrent index uses existing migration replay handling. Other recovery duties and Cloud sleep eligibility remain separate. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, shell execution and PostgreSQL tests. The runtime does not expose a more specific serving snapshot or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Codex <noreply@openai.com> |
||
|
Nicky LeachandPaperclip
|
80a2b9284b |
Define durable runtime activity events (#18129)
## Thinking Path > - Paperclip manages agents and tasks. > - Runner reports durable run events through PRP. > - Runtime clients need a typed record of reported activity. > - The event must keep the normal run authority and read boundaries. > - This pull request adds the generic runtime activity event contract. > - Later PRs can publish and consume this contract without private plugin coupling. ## Linked Issues or Issue Description First feature prerequisite extracted from #18050. Runner publishing is #18050; the dependent consumer is #18131. Related independent test stability work: #18128. The parent search and review history is preserved in [the verification archive](https://github.com/paperclipai/paperclip/pull/18050#issuecomment-6489369676). **Subsystem affected** Runner event protocol, server run-event persistence and redaction, and UI event taxonomy. **Problem or motivation** Core needs a durable, typed activity record tied to the authenticated run. A text marker is not an event contract. **Proposed solution** Add `runtime.activity.recorded` as a P1 event with a closed `{ namespace, activityKey }` payload. Runner knows no plugin or panel identity. Run/session identity, sequence and emission time stay in the outer envelope; Core resolves company, task and agent through the authorized run binding. Use the existing authenticated persistence, acknowledgement, replay and redaction paths. ## What Changed - Define and validate the event in PRP v1, v2 and v3. Update the generated protocol artifacts and event catalog. - Preserve saved and live event redaction. Test persistence, replay deduplication and the existing run-read boundary. - Register the event in the UI taxonomy without changing panel display behavior. - Advance the source-bound Copilot identity to 78. Retain all published historical identity fixtures without changing their bytes. ## Verification - Root `pnpm typecheck` and `pnpm build`: passed for the revised activity contract. - Node protocol checks: 14 passed. Provider identity and registry tests: 185 passed. Server tests: 103 passed. UI event registry: five passed. - Maintained Grok and Copilot identity generator checks and `git diff --check`: passed. - The first activity-contract review scored 4/5 because the dependent producer and readers still used the earlier contract. Both dependent PRs are now updated and restacked; the finding is resolved and review is running again. - Initial extraction retained a consumer-only import in one server suite. That suite failed to load. The consumer case and import were moved out; the final three server files passed all 103 cases. - Earlier panel-specific head `6137c50bc` had green CI and standard/Apex 5/5. Those scores do not qualify this revised activity contract. Fresh CI and reviews are pending. Earlier combined results in the archive apply only to their recorded source revisions. ## Risks This change is additive. It adds no route, publisher, filesystem transport, provider sandbox grant, or panel display behavior. Activation hints grant no permission to read a plugin view. Provider identities bind the changed protocol source; historical identities remain decode-only. No schema migration or dependency change is required. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository tools and code execution. The exact serving snapshot is not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
ec3b558277 |
fix(server): schedule accepted native response recovery (#17696)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native runs store accepted results and their finalization decisions. > - A completed run can still owe its public answer after an interruption. > - Periodic recovery scans for that answer even when no new work exists. > - This pull request wakes recovery from committed changes and saved retry deadlines. > - The existing owner still verifies the result, source, and current publication authority. > - Settled or dormant response work no longer needs its own periodic scan. ## Linked Issues or Issue Description Refs #17690 (merged). This is slice 48 of the sleep migration stack, now based on master. Related: #17661 handles GitHub terminal publication; this change handles native accepted-response presentation. Governed-review presentation remains a separate duty. **What happened?** Native finalization recovery repeatedly checks already accepted results for a missing answer. Removing that check without complete notifications would lose responses when evidence arrives late, an agent resumes, or admission settles after execution. **Expected behavior** Recover the answer from its saved result when a supported change makes it eligible. Retain real retry deadlines. Stop querying when the answer is selected or cannot currently be published. Reconstruct work after restart. **Steps to reproduce** 1. Commit a native accepted result with no selected response comment. 2. Run finalization recovery while its source is unavailable or locked. 3. Restore that source or release the lock. The answer must appear once without repeating provider execution. ## What Changed - Honor recorded non-create decisions and skip-comment policy in both discovery and recovery. Apply durable conversation/result/continuation guards before recovery locks. Preserve genuine unanswered and legacy recovery. - Scope routine response wakes to the affected issue/run. Coalesced input names its active run; pending-effect settlement names its issue. Sixteen existing global authority/configuration hooks use one explicit company invalidation boundary. No new transaction wrappers or tables are added. - Add an accepted-response worker using the existing delivery coordinator and scoped deadline helper. Preserve saved retry deadlines and pending-effect fences under the owner's lock. - Share the finalization notification boundary. Wake on finalization settlement, exact accepted-result/final-message evidence, agent availability, retry admission, and existing chat authority changes. - Keep lifecycle policy in services. A narrow pre-write status hook lets lifecycle retain ownership of its transaction and status transitions. - Register the worker during enabled maintenance startup. Remove its authority subscription on shutdown. Periodic replay skips this duty on both committed and fresh-commit paths; ordinary finalization and explicit recovery retain their behavior. - Add a partial index and expression statistics for unpresented native runs. Migration 0412 supports concurrent replay and adds no queue table. - Add real PostgreSQL and lifecycle coverage. Preserve the implementation and producer audit in the plan document. ## Verification - Additional real Photon callback acceptance on the published production code disproves the security warning: ten active assertions cause zero response-recovery hints/SELECTs, while three concurrent assertions during an actual attention-to-active recovery cause exactly one wake/SELECT. The existing locked status guard already enforces this boundary. No production change was needed; the resolved finding now has a successful current-head security check. The regression will be retained with the next slice. - Eligibility repair source `892f36e5f66ba1af123034936a7160c64c3ab883` passes **1489 distinct server cases**: 1139 in six complete recovery/finalizer/monitor/summary/workspace suites (81.02 seconds), plus 350 selected chat cases (297.68s). Another 1463 chat cases are outside the pattern; the complete 1813-case file is not claimed. All eleven new regressions are included in that count. Five dedicated fresh PostgreSQL 17 databases migrate through unchanged 0412, with no fixture-source adaptations. Database build, server types/build, boundaries and full-SHA migration ordering pass. All seven test source hashes match. - The exact published baseline first fails eight behavioral assertions while the three genuine-answer cases pass. The same corrected test source passes all eleven after repair: recorded none/reuse remain suppressed; five ineligible history families take zero recovery row locks across repeated issue hints; missing/legacy/create-null answers still publish once; fresh periodic finalization defers presentation to its worker. Earlier fixture setup and variable/compiler failures remain explained in the audit, rather than erased by later success. - Repair source `2fc8b0614aaf9ac7337045de4ced7325f95edc45` passes **1821 server cases**: 1496 in 13 complete producer/lifecycle/authority suites (110.63 seconds), plus 325 selected chat cases (278.99 seconds). Coverage includes all native-session cases, queue admission, status decisions, membership/board access, the complete GitHub workflow, accepted-response lifecycle and real authority restoration. The other 1477 chat cases are outside this selection; the complete 1802-case file is not claimed for this repair run. - Actual PostgreSQL regressions first reproduce a coalesced-input hint missing its run scope and two routine hint paths producing 12 response-history SELECTs each after three notifications. The same assertions pass with run-scoped coalescing and zero response-history SELECTs for unscoped routine hints. Actual membership restoration still publishes the waiting response once. Assertion time limits are unchanged; this measures real driver SQL and persisted producer work. - Eleven dedicated fresh PostgreSQL 17 databases migrate through 0412. One embedded-only wake-queue fixture import temporarily uses its own external database and restores byte for byte. All 14 selected source files match pre-run hashes. This qualifies external-database behavior rather than reproducing embedded CI locally. - Master advanced independently with #18126's Pi archive retry fix. Normal merge `2a535bc61259ef99387369e8b32498f774e5ba2c` preserves both complete binary patches. All accepted server/database/UI/test/CLI trees stay identical; the three independent runner installer/manifest files are retained. All 13 installer tests pass, including stalled header/body cancellation. Combined server types/build, boundaries and full-SHA migration order pass. A subsequent normal merge of #18128 at `e8b78cce5bb557b086bee9ca930eeaab1d09b244` retains its three test-only file changes with both complete patches preserved. The changed HTTPS backfill case passes on a fresh PostgreSQL fixture (1 selected case; 178 outside the pattern), and combined types/build, boundaries and migration order pass again. All 14 repair source hashes remain unchanged. The independent browser fixture changes are preserved; no local browser run is claimed. - The earlier source `9feec170ef8174011fc3c5d3d6d76f0a0f39fea0` passed the 4171-case full baseline before this scope repair. That scope is preserved in the audit rather than represented as a current full rerun. All 0412 database bytes remain unchanged: original concurrent index/statistics/ANALYZE, prior executable history, journal entries and retained snapshots. Five snapshots 0408 through 0412 remain. The populated 0411-to0412 upgrade preserved 410 prior receipts and 38 companies, appended one receipt and none on replay; index/statistic checks passed. - The earlier P2 security failure, Greptile's former conditional dependency on #17702, and the subsequent actual Ponytail eligibility findings are recorded in `doc/plans/2026-10-25-native-response-presentation-sleep-audit.md`. This PR now includes the issue-scoped pending-effect notification fix itself, so it is independently correct before #17702. Fresh published-head CI, Greptile 5/5 and a clean security scan remain required. No deployed-sleep, external-wake or fleet-savings claim is made. ## Risks - The writer contract is process-local. Independent SQL writers and other replicas are not covered by these notifications. - Unpresented backlog reconstruction still has memory and time cost proportional to candidate count. Pages contain at most 100 rows; execution uses the shared fair batch. Selected history is excluded. Health-only signals do not trigger reconstruction. - Publication policy remains with its current owner. Digest, company/run binding, source authorization, selected-answer suppression, and ownership fences remain required. - The rest of native recovery and governed-review presentation still run separately. This PR does not relax Cloud sleep eligibility or add external wake coverage. - The stack parent has merged and its actual ancestry is reconciled. This publication still needs fresh current-head CI and review. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, shell execution, and PostgreSQL tests. The runtime does not expose a more specific model identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Codex <noreply@openai.com> |
||
|
|
a8a038e659 |
fix(server): schedule native pending effect recovery (#17690)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native status decisions store effect targets and a delivery ledger. > - Historical pending or failed effects still need recovery. > - The periodic finalizer queries that ledger even when work is complete. > - This pull request gives pending effects their own commit-driven worker and saved deadlines. > - Completed history becomes quiet while incomplete effects remain recoverable. ## Linked Issues or Issue Description Refs #17687 (merged). This is slice 47 of the sleep migration stack, now based on master. **What problem does this solve?** Periodic native finalization repeatedly checks effect delivery. A tenant with completed history should not perform recurring SQL for this duty. Historical unresolved effects must still recover without duplicating their targets. **Proposed solution** Reconstruct current pending effects at startup. Wake from committed coordinator changes. Use the saved retry and ownership deadlines. Keep the existing dispatcher as the only recovery mutation owner. **Alternatives considered** A slower unconditional scan still creates idle traffic. A new queue table duplicates existing durable state. Replaying materialization can duplicate effects. This change uses the existing ledger, transaction signals, and shared scheduler. ## What Changed - Add a scoped worker with 100-row discovery pages and fair 25-entry dispatch batches. Delivered, superseded, and terminal-failure history has no recurring scan. - Preserve exact company/issue/run/decision binding and validate materialized targets before settling effects. Recheck phase, saved result, retry, and lease under the coordinator lock. Missing targets retain recovery. - Notify from decision replay, accepted-result recovery, coordinator claim/retry/release, attention replay, and execution-control recovery. Only two DB-only producer writes need a new transaction wrapper. Nested transactions use the existing outer-commit boundary. - Exclude pending-effect candidates from periodic finalization so it cannot skip ahead of their owner. Keep explicit and startup recovery behavior compatible. Respect maintenance disablement, drain, standby, and shutdown. - Add concurrent, idempotent migration 0411 with partial indexes for pending effects and their current coordinator binding. Retain the five newest schema snapshots. - Document producers, query evidence, failure history, and scope in `doc/plans/2026-10-25-native-pending-effects-sleep-audit.md`. ## Verification - Source `ba38ee465df02b6026286b5d5e15ae49fe07a367` passes all 2284 distinct cases across 25 complete files, without skips. The main selection passes 2225 in 203.94 seconds; safe-replacement passes 59 in 26.03 seconds. Coverage includes all 968 native-session cases, the 94-case status-authority/pending-effect corpus, actual decision/lease producers, Stop, queue, interaction, reconstruction and Slack consumers. The audit gives per-file counts. - The complete 94-case corpus passes again on the same database in 16.03 seconds. Retained PSC/OUT rows and an unrelated sentinel remain unchanged. The repeat is not counted again as distinct coverage. - Twenty-one separate fresh PostgreSQL 17 databases migrate through 0411 before setup. Four embedded-only fixture imports temporarily use their own external database on this IPC-exhausted host and are restored byte for byte. Assertions and timing limits stay unchanged. This qualifies external-database behavior; it does not claim an unmodified embedded CI run. - Database build, server typecheck/build, module boundaries and exact-head migration order pass. Original runtime/schema edits and unrelated upstream blobs are preserved. Both original concurrent index statements, all prior executable history, journal entries and retained snapshot bytes are unchanged; five snapshots 0407 through 0411 remain. - Populated 0410-to0411 upgrade preserves all 409 prior migration receipts and 38 companies, appends one receipt and adds none on replay. Both indexes are valid and ready on fresh and upgraded databases. - Adopting the latest prepared prerequisite changes ancestry and parent audits only. All accepted core trees remain identical. The audit retains earlier failures and qualifications. Actual landed-parent ancestry is now reconciled; fresh published-head CI and exact-head Greptile 5/5 remain required. No deployed sleep, Cloud-wake or fleet-cost claim is made. - Current parent context: integration `5f7cd7bc589ad0169dfb5f33df611ead56510ca3` changes only the parent's GitHub observation bound and independent cleanup assertion. All production code, packages and 25 accepted test files remain unchanged. Both focused GitHub idle cases pass in 15.26 seconds and the complete six-case cleanup suite passes in 8.47 seconds on two fresh 0411 PostgreSQL fixtures. This does not claim the full parent workflow was rerun here. ## Risks - A missing producer can delay work until restart. The audit covers current supported writers. Independent SQL and other processes remain outside the local notification contract. - Recovery validates existing targets; it cannot recreate a missing target or silently mark it delivered. Such work retains an error and retry. - The indexes add write/storage cost and use the existing restart-safe concurrent migration runner. - Other finalization duties still poll until their separate migrations. This PR does not prove host/database suspension, extend Cloud alarms, or claim fleet savings. - The actual prerequisite squash is adopted through a normal merge. Its complete tree equals the reviewed parent, and the complete accepted child tree is preserved. Fresh published-head CI and review remain required. ## Model Used OpenAI Codex, based on GPT-6, with reasoning and code-execution tools. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Codex <noreply@openai.com> |
||
|
DottaandPaperclip
|
4e715eb7ea |
fix(server): schedule historical unsafe export recovery (#17687)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native runs can retain accepted results after an unsafe workspace export fails. > - Historical recovery can retire an old notice or resume the accepted result without another provider turn. > - The periodic reaper repeatedly scans failures that cannot currently recover. > - Existing rows and committed changes can identify the actual recovery obligation. > - This pull request wakes recovery from those changes and retries only owed work. > - Dormant history becomes quiet while ownership and explicit operator decisions remain protected. ## Linked Issues or Issue Description **What happened?** The native finalization reaper repeatedly visits historical unsafe workspace-export failures. Cancelled tasks, newer runs, changed contracts, and explicit operator decisions can make those rows permanently ineligible. They still produce periodic database reads. **What did you expect?** Retire stale unsafe-export notices and recover eligible accepted results. Stop reading dormant history until a relevant committed change occurs. Keep a retry while a real obligation is blocked by ownership or a transient failure. **Steps to reproduce** Retain an unsafe-export terminal failure whose task is cancelled or superseded. Leave the instance idle after startup. The periodic reaper continues reading that history. Prerequisite #17680 is merged into master. Related: #14402 established safe export finalization; #14329 was an earlier historical recovery proposal. #17679 owns separate policy-review cleanup. This PR changes only historical unsafe-export scheduling and its producers. ## What Changed - Register a scoped worker through the existing coordinator and scheduler. Remove this duty from periodic reaping. Preserve the manual/startup recovery entry point. - Distinguish stale notices, eligible accepted results, dormant history, and concrete ownership contention. Reuse the existing mutation owner and lock order. Preserve explicit settlement vetoes and exact result/contract identity. - Signal relevant task, run, action, and deletion writes before they commit. Existing transaction boundaries handle nested commits and uncertain outcomes. Notifications contain only scope IDs. - Retain a retry only for owed work. Reconstruct work at startup, preserve drain/resume, and stop through the shared lifecycle. - Add concurrent, idempotent partial index migration 0410 for unsafe terminal failures. Keep the five newest generated schema snapshots. - Document producer coverage, query plans, supported writer boundaries, and test failures in the unsafe-export audit. ## Verification - Source `2a13dd08a6cc52f0adab09e3df9ce31e97b746b9` passes all 2188 cases across 24 complete files, without skips. The main selection passes 2129 cases in 192.58 seconds; safe-replacement passes 59 in 26.04 seconds. Coverage includes all 968 native-session cases, real export/action producers, explicit continuation, workspace reconstruction, interaction routing, Stop, queue and Slack consumers. The audit gives per-file counts. - Twenty separate fresh PostgreSQL 17 databases migrate through 0410 before setup. Four embedded-only fixture imports temporarily use their own external database on this IPC-exhausted host, then are restored byte for byte. Assertions and timing limits stay unchanged. This is qualified external-database acceptance, not an unmodified embedded CI run. - Database build, server typecheck/build, module boundaries and exact-head migration order pass. Normal integration preserves the complete original runtime/schema patch and unrelated upstream blobs. Original concurrent index SQL, all prior executable history, journal entries and retained snapshot bytes are unchanged; five snapshots 0406 through 0410 remain. Added comment removal preserves the full non-comment TypeScript token streams. - A populated 0409-to0410 upgrade preserves all 408 prior migration receipts and 38 companies. It appends exactly one receipt; replay adds none. The index is valid and ready on fresh and upgraded databases. - The owning audit retains earlier fixture failures, skipped checks and notice-retirement regression history. Older combined stack checks retain their original scope in their owning audits. Actual parent `73d24aa5acfcf17b3f676b4da7cd05eb70b60b99` is reconciled through a normal merge. Its tree equals the reviewed ancestor and the complete accepted child tree is unchanged. Fresh published-head CI and Greptile 5/5 remain required. No deployed sleep, fleet-cost or live provider qualification claim is made. - Inherited prefix correction: current integration `8f43c9f81fac23b50997e7f0e1d0d9cf383a8109` changes only two values in a previously unselected corpus fixture. All production code, the complete package tree and all 24 accepted test files remain byte-identical to the 2188-case source above. The additional complete 72-case corpus passes in 14.00 seconds and again on the same database in 13.08 seconds. Retained PSC/OUT and sentinel rows remain unchanged. The repeat is not double-counted. The parent audit retains the original collision and rejected cleanup attempt. - Latest parent context: integration `4d51231abe8b68c21ce896d0d2ac16865978c8ab` carries only the parent's one-argument GitHub observation correction and an independent master cleanup assertion. All production code, packages, 24 accepted suite files and the additional corpus are unchanged. The two focused native-presentation cases pass in 15.14 seconds and the complete cleanup-removal suite passes all six cases in 8.31 seconds on two fresh 0410 PostgreSQL fixtures. This does not claim the full parent workflow was rerun here. Current head `b28399babac72ba9db1c4a4ef882c5e87aded38c` has passing CI and Greptile 5/5 with no actionable findings. The first CI attempt lost runners: fourteen jobs explicitly received simultaneous shutdown signals, two reported lost runner communication, and two aggregate gates failed downstream. One failed-job retry on unchanged source passes. The underlying host shutdown cause remains unconfirmed; no code fix or extra retry is claimed. [CI run and attempts](https://github.com/paperclipai/paperclip/actions/runs/40819408528). No Ponytail verdict is present. ## Risks - A missing future producer can delay dormant-history discovery until restart. The audit names current producers. Independent SQL writers and other server processes remain outside the process-local signal contract. - Physical ownership and uncertain operations retain retries. Time alone does not authorize taking workspace ownership. No provider turn or unsafe archive retry is added. - The recovery owner remains authoritative if state changes after discovery. Terminal UI status does not authorize resource destruction or accepted-result replay. - The partial index adds matching-row write/storage cost. It builds concurrently through the existing restart-safe migration runner. - This proves the changed worker becomes quiet. It does not prove host suspension, external wake coverage, or fleet cost savings. Other sleep migrations remain. This slice still requires fresh current-head CI and reviews. ## Model Used OpenAI Codex, based on GPT-6, with reasoning and code-execution tools. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
73d24aa5ac |
fix(server): schedule automatic completion review retirement (#17680)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native finalization preserves accepted work and resolves task status. > - A historical cleanup removes automatic completion-review cards that current policy no longer creates. > - The periodic finalization pass repeats that lookup even after all matching cards are gone. > - This PR gives this cleanup its own committed hints and pending-card retries. > - The existing owner retains withdrawal receipts and protects genuine reviews. > - Idle history stops causing recurring cleanup reads. ## Linked Issues or Issue Description Refs #17679. This is a separate stacked slice for automatic completion-review retirement. Obsolete version-change reviews belong to the parent PR. **What needs to be improved?** `reconcileNativeFinalizations` repeatedly discovers old automatic completion reviews. The discovery helper also converts SQL errors to an empty result, which would let a new event-driven worker forget failed reconstruction. **Why is this improvement important?** Idle tenants should not query for absent cleanup work. Failed discovery must preserve the retry, and removing a historical card must preserve the durable trigger for later evidence reassessment. **Proposed solution** Use the existing card, decision, effect, assessment and contract rows. Reconstruct candidates at startup, signal the existing decision commit, and retry exact pending cards. Keep the direct cleanup call inside active finalization. ROADMAP.md and the related native review work were inspected; this change owns the scheduling migration only. ## What Changed - Add automatic completion-review registration to the shared scoped deadline worker. Reconstruct 100-row keyset pages and dispatch fair batches of twenty. - Signal company/issue IDs before the native decision transaction writes. Keep outer-transaction settlement with the existing work-signal boundary. - Split raw discovery from the public helper's compatibility error handling. Worker lookup failures propagate so the shared scheduler retains dirty scope and retry. - Reuse the complete historical predicate and existing withdrawal owner. Recheck the selected card identity, payload and system author fields in the conditional update. - Preserve the withdrawal reason, chat publication, activity record, direct finalizer call, startup/manual recovery, and subsequent evidence reassessment. - Skip only this duty in periodic reaping. Retain admission, drain/standby, retry and joined shutdown. - Add producer/lifecycle documentation. Add no table, schema migration, provider branch or independent timer. ## Verification - Current source `3f4d35ac0154aedc76cc39486d3758c73b1e6ff4`: all 462 cases across seven complete files pass with no skips in 49.31 seconds. This covers both review workers through the real status-authority corpus, startup/timing, recovery, finalization, external chat and governed milestones. Nested corpus fixtures are not double-counted. - Five separate fresh PostgreSQL fixtures migrated through parent 0409 before setup. Database build, server typecheck/build, module boundaries and target migration order pass. The whole database source tree matches the parent; no migration or table is added. - Normal integration preserves substantive owned edits and unrelated upstream blobs. Actual landed-parent ancestry and fresh current-head Greptile 5/5 and clean CI remain required. This is worker and integration acceptance, not deployed sleep qualification. - The inherited P2 fix changes only two fixture prefix values. All production code, other fixtures, assertions and timing limits are unchanged. The full 72-case corpus also passes again on the same database in 12.95 seconds. Retained PSC/OUT rows and an unrelated sentinel remain unchanged after both runs. The repeat is not double-counted among the 462 distinct cases. The parent audit retains the original collision and rejected cleanup attempt. - CI correction: the 40f head fails the passive native-presentation initial idle observation in chat shard 1. An unchanged local pair passes. A controlled committed hint plus a 600 ms held run attempt makes the original one-second observation fail; the identical probe passes with the established five-second observation bound. This demonstrates the coalescing boundary, not the unrecorded exact CI token timeline. No unchanged-head CI retry was used. - The probe is removed. The only permanent code change is that initial poll bound. It still requires zero active work and no deadline, followed by two minutes with no SQL, transaction or provider writes. Production code, worker/business deadlines, rollback and committed-settlement assertions are unchanged. All 274 cases in the named GitHub agent-review workflow pass in 232.06 seconds. Other cases in the large file are outside that named selection. The earlier 462-case selection retains identical source. Both owning audits preserve the failure and scope limits. Fresh updated-head CI and reviews remain required. - Master integration: independent master `49fe8b87a49aa5a8985a08dd3d5ce57bdd24756c` adds one company-skill cleanup assertion. A normal merge preserves both complete binary patches. All production code and the accepted GitHub integration file are unchanged. The full six-case cleanup-removal suite passes in 8.16 seconds on fresh PostgreSQL. Database history is unchanged. ## Risks - A normal native decision can cause one scoped lookup even when it creates no historical card. It retains no periodic scan when settled. - Startup reconstruction cost scales with historical data and joins. No constant-cost or fleet-savings claim is made. - Process-local signals cover instrumented writers. Independent SQL writers and concurrent older controllers need explicit integration. - Retirement and status reassessment are distinct obligations. The existing durable withdrawal receipt and remaining finalization owner are preserved. - Failed withdrawals retain retries while exact pending cards remain. A database lookup failure must not be interpreted as completion. - Other finalization and recovery duties remain. Cloud wake coverage and host-sleep eligibility are unchanged. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, shell tools, and PostgreSQL tests. The runtime does not expose a more precise model ID or context-window size for this turn. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Remote acceptance: head `be668fb975f119efeb4830405d4bb1ab65791f87` has 53 successful gates, verified with `gh pr checks` (no non-passing checks). Greptile reviewed that head at 5/5. There are no unresolved review threads. The PR remains draft and unmerged. Current published head `181bf800a79449961e09c995f2785dd7f8060bb3` has passing CI, fresh Greptile 5/5 and Ponytail 5/5 with no actionable findings. Earlier failures remain recorded in the audits. --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
dfd5b1ac62 |
fix(server): wake obsolete policy review cleanup from commits (#17679)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native finalization repairs saved results and related task state. > - Its periodic pass also searches for obsolete policy-review cards. > - Those cards have a precise historical signature and existing durable rows. > - This PR gives that cleanup its own committed signals and pending-work retries. > - The existing owner still protects real reviews and later task changes. > - Settled review history no longer causes repeated cleanup queries. ## Linked Issues or Issue Description Refs #17677. This PR is stacked on cancelled workspace preservation. It owns only obsolete policy-review cleanup within the native finalization reconciler. **What needs to be improved?** Every periodic native finalization pass queries for old version-change review cards, even after all such cards are gone. The parent reaper combines this cleanup with unrelated finalization and recovery duties. **Why is this improvement important?** Empty queries prevent idle tenants from becoming quiet. Removing the query must preserve historical repair, real review requests, task status authority, and cleanup after a failed write. **Proposed solution** Reconstruct matching pending cards at startup. Signal the existing native decision transaction before its writes. Schedule bounded cleanup only while an exact card remains pending. Skip this duty in periodic reaping while keeping startup and manual entry points. ROADMAP.md and related PR search found no duplicate of this migration. ## What Changed - Register obsolete-review cleanup with the existing scoped deadline worker. Reconstruct in 100-row keyset pages and process fair batches of twenty. - Use company/issue hints from the existing status-decision commit boundary. Serialize only the two IDs; preserve outer commit and rollback. - Reuse the exact historical signature and existing cleanup owner. Keep status-version, decision-pointer, separate-review, activity and chat-publication guards. - Recheck selected payload and system-card identity in the conditional withdrawal so a concurrent edit cannot be cancelled from a stale snapshot. - Retain a 30-second retry only while a matching pending card survives. Preserve drain, standby, startup and joined shutdown. - Skip only this duty in the periodic finalization pass. Automatic completion reviews, unsafe exports, status effects, evidence reassessment and workspace finalization remain separate migrations. - Add the owner audit and validated external-test database support for the existing status-authority corpus. Add no schema, table, independent timer or provider branch. ## Verification - Current source `4cc0d03ae96bd92a4fdad44168c2a8647d42466c`: all 443 cases across seven complete files pass with no skips in 46.46 seconds. This covers the full status-authority corpus and cleanup worker, startup/timing, heartbeat recovery, native finalization, external chat and governed milestones. The audit gives per-file counts without double-counting fixtures inside the corpus matrix. - Five distinct fresh PostgreSQL fixtures migrated through parent 0409 before setup. Database build, server typecheck/build, module boundaries and target migration order pass. The whole database source tree matches the parent; no migration or table is added. - Normal integration preserves substantive owned edits and unrelated upstream blobs. The earlier circular notification-payload failure remains in the audit. Actual parent `51c3f8924cb0bd5fe7cbda15ad9b8f06a6a11055` is reconciled through a normal merge, with its full tree equal to the reviewed ancestor and the entire accepted child tree unchanged. Fresh current-head Greptile 5/5 and clean CI are still required. No deployed sleep qualification is claimed. - Review correction: the old head reproduces a fixed-prefix collision on a same-database corpus rerun. The initial deletion attempt passed assertions but failed teardown on non-cascading audit foreign keys; it was removed. The final two-value change uses each fixture company UUID in its prefix and changes no production behavior, assertions, timing limits or teardown. - The complete 56-case corpus passes a second time on the same database in 10.15 seconds. Retained PSC/OUT rows and an unrelated sentinel company remain unchanged after both runs. The repeat is not double-counted among the 443 distinct cases. The audit retains both failures and their causes. Fresh CI and Greptile are required for the updated head. ## Risks - Startup reconstruction still scans historical candidates and joins; its cost scales with history. Scoped commits use existing indexes. No recurring query remains after this worker settles. - The signal covers instrumented single-process writes. Independent SQL writers and concurrent older controllers need explicit integration. - A normal native decision can cause one scoped lookup, even when it creates no obsolete review. It does not retain a periodic scan. - Failed withdrawals retain retries while matching cards remain. Actual cleanup still uses the existing locking, authority and activity owner. - Other finalization duties still run. This PR does not establish whole-instance quiet, external alarm coverage, provider suspension or fleet savings. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, shell tools, and PostgreSQL tests. The runtime does not expose a more precise model ID or context-window size for this turn. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Current published head `9aa67199da64d700fd6230829d38f2ef522a07fe` has fresh Greptile 5/5, Ponytail 5/5 with an unconditional ship verdict, and all current CI checks passing or skipped. The reproduced rerun P2 is fixed and its thread is resolved. --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
51c3f8924c |
fix(server): schedule cancelled workspace preservation (#17677)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A cancelled native run may still owe preservation of prepared workspace files. > - The existing recovery receipt and source lease record that obligation. > - The orphan reaper still searches for these cancellations on every pass. > - This PR registers committed cancellations with the shared deadline worker. > - The existing retention owner keeps file pinning and ownership checks. > - Idle and completed history no longer causes this preservation scan. ## Linked Issues or Issue Description Refs #17674 and #17145. Stopped instruction collection is merged into master. It owns file-preservation discovery only. The existing environment cleanup worker continues to own stop-only teardown. **What needs to be improved?** `preserveCancelledNativeWorkspaces` runs from every orphan-reaper pass. It queries even when no cancelled native workspace needs a file-recovery hold. Current foreground Stop can also defer retention after a lock or database failure, so startup-only recovery would be insufficient. **Why is this improvement important?** Empty queries keep idle tenants active. Removing them must not lose user files, weaken Stop ownership, replay cancelled work, or turn a retained allocation into destructive cleanup. **Proposed solution** Register the existing durable obligation from committed run-status, cancellation-acknowledgement and workspace-reference writes. Reconstruct it at startup. Keep retries while actual preservation remains unresolved. Disarm after the existing owner records the hold. Related-PR search and ROADMAP.md review found no duplicate of this scheduling migration. ## What Changed - Add a company/run-scoped preservation worker with 100-row keyset reconstruction and fair batches of five. - Separate pending file debt from immediate coordinator/lease readiness. Retain a 30-second retry for known unfinished preservation and shared failure backoff for errors. - Signal the existing status, acknowledgement, workspace-reference, deletion and settings transactions. Add no table or independent timer. - Keep exact source checks, no-workspace contracts, task/coordinator/run lock order, retention receipt and no-replay recovery card with their owners. - Recheck cancellation acknowledgement, run status and coordinator phase under the retention locks. Preserve the existing foreground preterminal API. - Keep startup/manual recovery; skip only this duty in periodic reaping. Preserve drain/standby and shutdown behavior. - Add concurrent, replay-safe partial index migration 0409 and retain the five newest snapshots. - Add the producer and lifecycle audit in `doc/plans/2026-10-25-cancelled-workspace-preservation-sleep-audit.md`. ## Verification - Current source `21b46a8e4293ca670235117458b984493a181759`: all 1472 cases in 16 complete files pass without skips. The owner passes 84 cases in 14.93 seconds. Fifteen consumer files pass 1388 cases in 133.46 seconds, including all 968 native-session cases and Stop, dispatch, restart, workspace-sync and upgrade-readiness coverage. - Twelve separate fresh PostgreSQL 17 databases migrated through 0409 before setup. The embedded-only lifecycle fixture temporarily used its own external database on this IPC-exhausted host and was restored byte for byte. Assertions and timing limits stayed unchanged. This qualifies external-database behavior; it does not claim an unmodified embedded CI run. - Database build, server typecheck/build, module boundaries and exact-head migration order pass. The complete original runtime/schema patch and unrelated current-parent blobs are preserved. Original concurrent index SQL, all earlier executable history and retained snapshot bytes are unchanged; five snapshots 0405 through 0409 remain. - A populated 0408-to0409 upgrade preserves all 407 prior receipts and 38 companies. Exactly one receipt is appended; replay adds none. The preservation index is valid and ready on fresh and upgraded databases. - Actual parent `82529741418e138d9d623bf13d2fa22c1ff79d36` is merged through a normal ancestry reconciliation. Its complete tree equals the reviewed parent. The entire accepted child tree and all source hashes are preserved. The audit retains earlier failures, qualifications and per-file counts. Published head `b34f0589847f33754ecde9413fd36bf007353d2c` has Greptile 5/5 and all 57 checks passing or skipped. No Ponytail review is present. These checks do not establish deployed host/database suspension or Cloud alarm delivery. ## Risks - Signals cover instrumented single-process writers. Independent SQL writers and additional replicas need integration. - Status signals may inspect a run without relevant file debt once. They do not create recurring work. Discovery and memory still scale with actual candidates. - Missing ownership or stop evidence remains pending. Active retries are intentional while a concrete file obligation is unresolved. - Retry timing is process-local; startup reconstructs durable debt. The worker does not overwrite another native-finalizer deadline. - Admitted database operations may finish after drain begins. Shutdown joins them. Provider teardown remains with the existing cleanup owner. - The partial index adds storage and write cost. Its concurrent migration uses the existing interrupted-index repair path. - Cloud alarm coverage, sleep eligibility and host suspension are unchanged. This proves worker behavior, not fleet savings. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, shell tools, and PostgreSQL tests. The runtime does not expose a more precise model ID or context-window size for this turn. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8252974141 |
fix(server): schedule stopped instruction collection (#17674)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Runs can stop while their private instruction edits still need collection. > - Existing copy rows and stop evidence describe that unfinished work. > - The orphan reaper still queries for it on every maintenance pass. > - This PR activates collection from committed copy and run transitions. > - It uses saved retry deadlines when stop proof or collection remains pending. > - Active copies and completed history no longer cause collection polling. ## Linked Issues or Issue Description Refs #17671 and #17670. Both prerequisites are merged into master. Captured saves and post-save release remain separate reasons with their own workers. **What needs to be improved?** `recoverStopped` queries instruction copies on every orphan-reaper pass, even when no terminal run owes collection. A run can become terminal without changing its copy row, so copy-only notifications would miss real work. **Why is this improvement important?** Repeated empty queries prevent idle tenant instances from becoming quiet. Removing them must preserve edits, stop proof, current ownership, and recovery after restart. **Proposed solution** Use the existing transaction signals, scoped deadline worker, and scheduler. Reconstruct eligible copies at startup, signal supported copy and terminal-run writers, and retain saved retry deadlines for unresolved work. Preserve the existing collection and directory owners. Related PR search found the warm agent-file ownership work (#14695), verified stop boundary (#14311), and task-workspace cleanup (#16464). This PR does not replace those contracts. ROADMAP.md has no duplicate migration. ## What Changed - Add scoped stopped-instruction collection with keyset reconstruction and fair batches of 20. - Signal before copy preparation, mutation and adoption writes, and before terminal run writes in their existing transactions. - Cover heartbeat, native finalization/restart/session recovery, execution-control, queue and dispatch producers. - Signal settings changes and company/agent deletion. Preserve drain, standby, restart and shutdown ownership. - Recheck terminal-run eligibility in the due-time claim. Keep terminal status separate from process-stop proof. - Retain existing stop checks, attempt limits, saved candidate bytes and remote-resource fencing. - Skip only this lane in periodic orphan recovery. Preserve manual/startup recovery. - Add concurrent, replay-safe partial index 0408 and retain the five newest snapshots. Add no table. - Record the owner and producer audit in `doc/plans/2026-10-25-stopped-instruction-collection-sleep-audit.md`. ## Verification - Source `cb2818a8f46ef961f744977a258f0e9042e31a42`: all 1469 cases across 14 complete files pass without skips in 141.69 seconds. This includes all 968 native-session cases, filesystem saves, startup, terminal-run producers, dispatch, queue, execution-control, finalization and lifecycle. - Eleven separate fresh PostgreSQL 17 databases migrated through 0408. The embedded-only lifecycle fixture temporarily used its own external database on this IPC-exhausted host and was restored byte for byte. Assertions and timing limits stayed unchanged. This qualifies external-database behavior; it does not claim an unmodified embedded CI run. - Database build, server typecheck/build, module boundaries and exact-current migration order pass. The original concurrent index SQL is unchanged. All prior executable migration history, journal entries and retained snapshot bytes are preserved; the five newest snapshots are 0404 through 0408. - A populated 0407-to0408 upgrade preserves all 406 prior migration receipts and 38 companies. It appends exactly one receipt. Replay adds none. The collection index is valid and ready on fresh and upgraded databases. - The normal merge of actual parent `267842fc3c6e6542deed8a1bcc2c78bec8df7862` preserves both complete patches, all independent sidebar changes from #18117, and the full accepted server/package/test trees. The complete sidebar test file passes all 36 cases on the combined tree. React act warnings remain. - Prior failures and fixture qualifications remain in the audit. Published head `d176d731f083c7edd2ede688cb901d1e66d8f1d8` has Greptile 5/5, all 57 checks passing or skipped, and no unresolved review threads. No Ponytail review is present. These local tests do not establish deployed host/database suspension or Cloud alarm delivery. ## Risks - Process-local signals cover instrumented writers. Independent SQL writers and additional replicas still need integration. - Terminal-run signals can cause one scoped inspection even if no instruction copy exists. They do not create a recurring scan. - Reconstruction and memory scale with actual pending candidates; active candidates are filtered by the run join. - Admitted file/provider operations can outlive a later drain request. Shutdown joins them. Uncertain ownership remains pending rather than being discarded. - The partial index adds storage and write cost. Its concurrent migration uses the existing interrupted-index repair path. - Cloud alarms, sleep eligibility and provider suspension are unchanged. Worker quiet is not a claim of deployed host/database sleep or measured savings. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, shell tools, and PostgreSQL/filesystem tests. The runtime does not expose a more precise model ID or context-window size for this turn. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
27270e9bf8 |
fix(server): wake interrupted retry cancellation from committed intent (#17667)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A parked scheduled retry has no executor to finish an interrupted Stop request. > - Its saved cancellation fence already records the work that remains. > - The orphan reaper currently scans for that work on every heartbeat, including idle periods. > - This PR registers the existing records with the shared delivery scheduler and wakes recovery after committed writes. > - Recovery keeps the existing cancellation guards and settlement behavior, then becomes quiet. ## Linked Issues or Issue Description Refs #17666, the merged preceding PR in the recurring-server-work migration stack. This PR targets master. Related cancellation work found in the dedup search: #12301 and #7340. This PR changes the trigger for interrupted parked-retry cancellation; it does not replace their cancellation or retry policies. **What happened?** The periodic orphan reaper queries scheduled retries with cancellation fences even when no cancellation needs recovery. Its first-20-row scan can also stop behind malformed evidence. **Expected behavior** Committed cancellation intent wakes bounded recovery. Failed work retains a retry. Startup reconstructs durable work. Empty or settled work performs no recurring SQL for this duty. **Steps to reproduce** 1. Start a server with heartbeat scheduling enabled and no interrupted retry cancellations. 2. Observe the orphan reaper repeatedly query for cancellation fences. 3. Cancel a scheduled retry and fail the final status write after its fence commits. Recovery must complete the existing Stop request without running a provider or dropping its metadata. **Paperclip version or commit** Current parent squash `7afee8f0622e273d10506b0febfb68f143d168b1`. Source acceptance `66107ae0a7ad85fd0476f91c4f8f98ae74a7a048`. **Deployment mode** Self-hosted server with the existing single-process transaction signal model. Local evidence uses PostgreSQL 17. ## What Changed - Add one domain-owned recovery worker over existing heartbeat run rows. No new table. It delegates to run control and preserves original cancellation, budget, and operator metadata. - Signal cancellation fences, lifecycle transitions, dispatch deferral/promotion/suppression, relevant settings changes, and deletion within their owning transactions. Wrap only the ordinary database fence in a short transaction; provider calls stay outside it. - Keep startup/manual recovery compatible and remove this duty from the periodic orphan sweep. Other reaper duties remain for separate PRs. - Extend the shared scoped worker to retain pending work with a memory-only admission retry while its owner is paused. Without pending work, it schedules nothing. Existing GitHub consumers retain their behavior. - Add a concurrent idempotent partial index and lifecycle/producer tests. Enable guarded disposable PostgreSQL URLs in affected test harnesses. - Reconstruct one hundred candidates per scope per scheduler pass. Resume from the saved scope cursor and retain fair twenty-item execution batches. Manual recovery starts cancellation before reading the full backlog. - Require the recovered browser reply to be saved before testing reload. Keep all original assertions and time limits. - Record the full producer audit, lifecycle limits, and failed attempts in `doc/plans/2026-10-25-scheduled-retry-cancellation-sleep-audit.md`. ## Verification Published head: `405e533394b26215e22d0452e9b0207d3f784a79`. Source acceptance: `a35fa43b98e2bd378d4a920d2775ce243654838b`. The final commit changes the audit only. - All 363 backend cases in ten complete owner and consumer files pass without skips. This includes actual Stop/retry producers, transaction settlement, interrupted-write recovery, pause/resume, shutdown, fair backlog processing and quiet after settlement. Seven separate fresh PostgreSQL 17 databases prevent fixture interference. - The two embedded-only backend fixtures temporarily use separate external databases on this IPC-exhausted host. Their fixture imports and native client teardown are restored byte for byte afterward. Assertions and timing limits are unchanged. CI must still pass with the original embedded fixtures. - Two paging regressions fail on the prior worker and pass on the fix. They cover cancellation before full reconstruction, a malformed first page, fair batches, a newly committed intent behind the cursor, and no later idle SQL. Per-candidate admission reads remain deliberate safety checks. - The repeated CI reload failure is reproduced with a temporary twelve-second fixture settlement delay. The trace retains the same five-second missing-reply assertion, even though a cleanup race masked it in the top-level error. Requiring exactly one saved reply from the resumed run passes under the identical delay. The fault is removed. All six browser journeys pass on the final source in 40.5 seconds, with zero retries and the original 120-second case and five-second reload limits. - Browser acceptance uses the canonical CLI/server/static-UI path and a fresh external database. Only fixture database-URI setup is temporarily adapted and then restored. This is not an unmodified embedded CI-environment reproduction. The six cases include the controlled case; repeated runs are not added to the distinct count. - Actual database build, server typecheck/build, module boundaries and exact-current migration order pass. The database source tree is unchanged from the verified 0405-to0406 upgrade: all 404 prior receipts and 300 companies stay intact, exactly one receipt is added, replay adds none, and the index is valid on upgraded and fresh databases. The five snapshots remain 0402 through 0406. - All earlier failures remain recorded in the audit. Prior head `48b3fcb83f305616df02a4337a42ec0867bb4b80` had exact-head Greptile 5/5 but failed the repeated browser assertion. It was not retried blindly. Its completed Ponytail review's backlog concern and old migration paragraph are addressed here. Old scores do not satisfy new-head gates. - Current-head CI passes. Exact-head Greptile and Ponytail both give 5/5, with no remaining actionable findings. These checks do not prove deployed instance or database suspension. ## Risks - Only supported in-process writers produce local transaction signals. Independent SQL writers or other replicas need a separate ingress contract. Startup reconstructs saved obligations. - A real failure retains bounded retries. Unknown commit outcomes retain the shared intent hold. Neither is discarded to report quiet. - Per-owner drain uses a memory-only admission recheck. The shared helper change affects existing consumers, which have focused regression coverage. - The partial index is concurrent and idempotent; it adds index storage. Each scoped load reads at most one hundred rows. The shared helper retains discovered pending IDs between passes, so a large unresolved backlog still uses memory. - Cloud alarms, idle-safety exemptions, and other reaper duties remain separate. Merge requires passing current-head CI and review gates. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, shell tools, and local PostgreSQL tests. The runtime does not expose a more precise model identifier or context-window size for this turn. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
scotttongandPaperclip
|
71f0c0c99e |
feat: add opt-in Coordinator workspace for chat, tasks, and decisions (#17983)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/147/changes/paperclip/147 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People coordinate work through conversations, tasks, and requests for input. > - These surfaces must share task state and preserve the context of each decision. > - The existing workspace makes people switch surfaces to follow that context. > - This pull request adds an optional Coordinator workspace with linked conversations, tasks, documents, and decisions. > - Each person can enable it in Settings > Experimental and return to the classic workspace. > - The benefit is a coherent path from a request to an assigned task and reviewed work. ## Linked Issues or Issue Description **Subsystem affected** UI, server, shared contracts, and database. **Problem or motivation** People need a consistent way to start tasks from agent conversations, answer questions, review work, and return to the source context. Decisions and task state must agree across these surfaces. **Proposed solution** Add a personal, default-off workspace experiment. Keep one conversation per agent. Add task proposals, compact task and decision cards, contextual document panels, and a Needs You queue with search, filters, sorting, grouping, and history. **Alternatives considered** Replacing the classic workspace for everyone would remove the opt-out path. Keeping separate decision state in each surface would create conflicting answers. This change uses the existing canonical task and interaction records instead. **Roadmap alignment** This supports the existing agent chat, task management, and human review capabilities. It is an experimental presentation choice. The user requested this isolated implementation after design review. No Discord discussion is claimed. ## What Changed - Add per-person workspace preferences, a default-off setting, and an additive migration. - Add Coordinator navigation, pinned destinations, task lists, agent switching, and contextual panels. - Link task suggestions and creation to the source conversation. Default new task ownership to the current agent unless the user specifies another owner. - Share pending questions, decision receipts, exact review targets, and history across tasks, chat, and Needs You. - Preserve task access, assignment authorization, revision binding, and required output review gates. - Add compact document links, relative timestamps, composer controls, and reduced-motion support for task and decision transitions. - Add component stories, regression coverage, and capability-map documentation. ## Verification - Final head: `4b839aae1e314bd7e8635a766f1d205ff1c03ba6`, based on master `8bc051245f77a47bee0f8ba45896b8bfa5922e0a`. Verified October 29, 2026 at 00:10 UTC: **MERGEABLE, 56 checks passed, four conditional skips, no pending or failed checks**. [Current CI run](https://github.com/paperclipai/paperclip/actions/runs/40790639688) passed without a rerun on this head. - Fresh Greptile **5/5**, zero new comments and zero open review threads; Superagent passed with no suspicious changes. [Checks](https://github.com/paperclipai/paperclip/pull/17983/checks). - Local workspace typecheck/build passed after the latest rebase. The final account-read patch passed UI typecheck/build and all 12 related preference/settings tests, including a delayed real-client request across account changes. Earlier scoped checks cover review deduplication/replay, decision deep links, cross-surface state, access boundaries, and migration replay. - Local database/browser rechecks after the latest rebase were limited by exhausted macOS shared-memory IDs: `initdb` failed before database creation. The local run passed 31 UI/snapshot tests and skipped 24 database tests; those skips and the blocked local browser start are not claimed as passes. The independent Linux CI database and browser checks passed. No running dev database or host setting was changed. - Upstream migration 0403 is preserved. Preferences and durable review retry aliases are generated as replay-safe migrations `0404_tense_silver_sable.sql` and `0405_harsh_avengers.sql`; five newest snapshots are retained. Every detected conflict was rebased and pushed before waiting for CI/review, as requested. - The PR is unmerged. The active master ruleset still requires code-owner approval for the changed Paperclip skill; no approval or rule change was performed. <details> <summary>Earlier verification failures, integration history, and evidence</summary> These entries preserve prior checkpoints and failures; the current acceptance result is above. - Focused UI and server checks from the isolated build are recorded in the implementation plan. - Browser checks covered the opt-in shell, navigation, task/document links, and decision card states. No new live provider trial is claimed for this handoff. - Current acceptance head: `4b839aae1e314bd7e8635a766f1d205ff1c03ba6`, rebased onto master `8bc051245` immediately after its GitHub review-repair migration took 0403. Preferences are generated as idempotent `0404_tense_silver_sable.sql`, preserving upstream SQL/journal/snapshot history and five newest snapshots. GitHub reports MERGEABLE after the immediate lease-protected push. Scott's updated instruction is to rebase and push every conflict resolution first, without waiting for CI or Greptile; verification follows the push. - The final reviewed feature tree was consolidated for repeatable single-commit rebases. The complete previous commit history remains reachable through local recovery ref `codex/coordinator-recovery-before-rebase-20261028` (resolved snapshot `0e1bbdf4e`). The actual rebase completed, and its tree hash exactly matches that resolved snapshot. The force-with-lease required the known remote head `0e6ef7c44`, protecting against concurrent branch changes. No PR merge or permission/runner setting change occurred. Fresh CI/review are in progress for the current head. - `2001132fa` received Greptile 5/5 with one nonblocking account-switch read finding. Current head detaches path-coalesced preference GETs and disables HTTP caching, retaining React Query's account-scoped sharing. A delayed real-client GET regression proves account B can load its saved choice before account A responds, and A's late response only updates A's cache. Preference/settings tests 12/12 and UI typecheck passed; its review thread is resolved. Fresh current-head CI/reviews are pending. The prior CI chat shard 3 ended with “The operation was canceled” and no assertion failure ([job 123171595631](https://github.com/paperclipai/paperclip/actions/runs/40789688333/job/123171595631), `/tmp/pr17983-2001132-chat3.log`); no code/timeout change is attributed to that cancellation. - `843e823b4` reached fresh Greptile 5/5 with zero comments/open threads and Superagent pass. All browser tests including the repaired YAML shard passed, and 54 checks passed. General server shard 9 ended with “The operation was canceled” and no failed assertion; its test step was marked cancelled (job [123164350893](https://github.com/paperclipai/paperclip/actions/runs/40787501761/job/123164350893), log `/tmp/pr17983-843e-server9.log`). A single unchanged-head rerun was started after the workflow completed, but master moved while it ran. A local nested-clone check timed out in its 30-second database-startup hook and then failed cleanup before the 20 database cases ran; this is retained as a local verification limit (`/tmp/pr17983-843e-nested-clone-check.log`), not claimed as a pass or proof of the CI cancellation cause. - Following Scott's instruction, the new conflict was rebased and pushed immediately without waiting for that rerun. Upstream's `0403_natural_cerise.sql` is preserved; regenerated preferences and retry-alias migrations are `0404_tense_silver_sable.sql` and `0405_harsh_avengers.sql`, replay-safe with five newest snapshots retained. The YAML helper conflict combines upstream's click and header behavior with the editor-selection guard and injected regression. Recovery ref `codex/coordinator-recovery-843e823b4` preserves pre-rebase work. Current head is MERGEABLE. Full workspace typecheck/build passed; UI and snapshot checks passed 31 tests, but 24 database tests were skipped by the local support probe, and the local browser server could not initialize PostgreSQL. A direct disposable `initdb` diagnostic established the host limitation: `could not create shared memory segment: No space left on device` (`shmget`, shared-memory IDs exhausted, not disk space). No dev database or host setting was changed. Diagnostics: `/tmp/pr17983-2001132-initdb-probe.log`, `/tmp/pr17983-2001132-db-probe.log`; checks: `/tmp/pr17983-2001132-tests.log`, `/tmp/pr17983-2001132-browser.log`, `/tmp/pr17983-2001132-types.log`, `/tmp/pr17983-2001132-build.log`. These local skipped/blocked checks are not claimed as passes; fresh CI on independent runners is in progress. - `d18f683cd` reached fresh Greptile 5/5 with zero comments/open threads and Superagent pass, but [E2E shard 7](https://github.com/paperclipai/paperclip/actions/runs/40785966243/job/123159719925) failed parsing an Advanced YAML clipboard read beginning with “Skip to Main Content.” The log is retained at `/tmp/pr17983-d18f-e2e7.log`. Current head hardens the test helper to verify editor focus and selection, clear stale clipboard content, and require the YAML header at the start of a fresh copy before parsing. Saved-state assertions and timeouts are unchanged. All six Advanced YAML browser journeys passed on a disposable local instance (41.0s); the exact lost-response journey also passed with a one-time injected surrounding-page copy, verifying rejection/reacquisition (17.6s). The precise browser focus/clipboard timing in CI remains unproven; the observed invalid-read path is now guarded and deterministically exercised. Logs: `/tmp/pr17983-yaml-clipboard-scope-browser.log`, `/tmp/pr17983-yaml-clipboard-injected-browser.log`. Fresh current-head CI and reviews are pending. - `19e6dcb0b` passed Superagent and Greptile 5/5, with one nonblocking deep-link finding. Current head preserves attention links using `?decisionId=`: the requested pending decision is rendered even beyond the first 40 cards, focused and scrolled into view, then only that parameter is consumed so later refreshes do not refocus. Other URL parameters remain. Needs You tests 10/10 and UI typecheck passed; its review thread is resolved. The preceding post-rebase regression run passed 45/45 tests. Fresh current-head CI/reviews are pending. - `6d772b7cc` reached Greptile 5/5 with zero comments/open threads and Superagent pass. While 32 checks had passed, master moved and caused a documentation conflict in `maps/feature-map/onboarding.md`. Rebased immediately, preserving both onboarding paragraphs. The complete four-commit branch was pushed as `19e6dcb0b` before waiting on CI/review, per Scott's instruction. A transient Git index lock interrupted the first continue; the command sequence briefly pushed intermediate `34e298c07`, then the completed rebase was immediately pushed. All feature and security/retry fixes are present; local recovery ref `codex/coordinator-recovery-6d772b7cc` preserves the pre-rebase history. The only tree differences from that ref are upstream's two smoke-test files and the combined documentation paragraph. GitHub reports MERGEABLE. Fresh current-head verification is in progress; prior-head review results are not current acceptance. - `21efa3120` passed Superagent but Greptile scored 4/5 for a valid retry-key persistence bug: an acknowledged duplicate key could reopen a resolved review. Current head adds an additive, replay-safe `0404_light_swarm.sql` migration for indexed, task-scoped aliases from acknowledged retry keys to the canonical interaction. Alias registration and retry lookup occur under the existing task lock; payload/actor mismatch still conflicts, and each canonical review permits at most 100 aliases. Original and acknowledged keys replay the terminal result rather than creating a new gate. Five newest migration snapshots are retained. Service/routes tests 165/165 passed; final quality review plus migration replay/snapshot checks 25/25 passed; server typecheck passed. The P1 thread is resolved in code; fresh current-head reviews and CI are pending. - `69c2c7543` passed 55 checks and Greptile 5/5 with no open Greptile findings, but Superagent check [123140675812](https://github.com/paperclipai/paperclip/pull/17983/checks?check_run_id=123140675812) correctly flagged unbounded pending quality-card creation. Current head fixes that under the existing task-row lock: equivalent requests with fresh idempotency keys reuse the pending card; conflicting requests for an unresolved deliverable return the existing interaction ID; pending cards are capped at ten per task across all reviewers. Distinct outputs remain independent, and already-answered items in a partially pending batch can receive a fresh review. Added real-database concurrency, cross-card duplicate, partial-answer, and hard-cap regressions. Quality review 22/22 tests and final server typecheck passed; earlier service/routes 163/163 tests passed before the final partial-answer refinement. One added assertion initially expected a service HTTP error, but duplicate keys are rejected earlier by the existing shared schema; the assertion now verifies that validation. Fresh current-head CI/reviews are pending; the action-required security result is not treated as passed. - `c9c575e5f` completed all 56 checks successfully with four conditional skips, Greptile 5/5, and Superagent pass. Its two P2 review findings are fixed in the current head: independently bound quality approvals now each remain visible in attention; changing/clearing URL-backed assignee/status filters updates the URL and survives task navigation. Both review threads are resolved. Regression evidence: attention service 34/34 tests passed, CoordinatorTasks 20/20 passed, UI and server typechecks and the full workspace build passed on `69c2c7543`. The first new UI test run caught an assertion racing React Router's transition; waiting for the navigation before asserting passes without weakening the navigation/remount checks. These older-head green checks are not claimed for the new fix commit. - Preceding `0e6ef7c44` completed all 56 checks successfully with four conditional skips, fresh Greptile 5/5 and no open findings, and passed Superagent. A fifth migration-metadata collision arrived before the final Runner job finished. Those green results are preserved as prior-head evidence, not current acceptance. - Previous fully verified migration head: `0e6ef7c44696a377f35e69521b93587576374bd0`, integrating master `f8226b051` after the fourth migration-number collision. Master added terminal GitHub-response migration 0401 while the final native check ran; upstream 0401 SQL/journal/snapshot is preserved unchanged, and preferences are generated as idempotent `0402_yielding_blink.sql` on top. Exactly the five newest snapshots remain. GitHub reports MERGEABLE after the push. All 1,155 relevant migration/review/interaction/native-execution/remote-assets tests, workspace typecheck/build, migration guards and whitespace checks pass. Fresh exact-head CI/Greptile/Superagent are pending. - Immediately preceding head `36c67cf03` completed all 56 checks successfully with four conditional skips, fresh Greptile 5/5 and no open findings, plus passed Superagent security. No retry was needed for that head's CI. Master advanced and introduced only a migration-metadata conflict while its final native check ran, so that green result does not establish acceptance of the current integration. Four consecutive collisions (0398, 0399, 0400, 0401) during verification are retained as concrete integration churn; a brief pause of competing migration merges was requested, without assuming it happened or changing any repo setting. The PR has not been merged. - Prior fully verified head: `36c67cf03`, on the clean `30059c0ed` integration of master `efd65c112`. The last Greptile P2 is fixed: Needs attention retains the participant-agent scope through authoritative server membership queries for each pending task, including historical participation when another agent currently owns the task. It does not approximate participation with current assignment or enumerate every company task. The actual view-switch regression and all 31 focused task/routing/collection tests, UI typecheck/build, token gates and whitespace checks pass. Greptile completed a fresh 5/5 review on this exact head (review 19) with zero new comments and no open review threads. CI and Superagent remain in progress; no current-head CI failure has been reported. - Head `30059c0ed` received Greptile 5/5 with that one nonblocking participant finding, now fixed. Superagent passed that head. Current-base integration restored the ephemeral RunsOn fleet after a superseded workflow retained queued aggregate jobs: normal cancellation was insufficient; the single force-cancel request was interrupted but readback confirmed that obsolete run cancelled and current CI began on the trusted fleet. No cancellation retry or runner/security/workflow changes were made. Its partial CI was superseded by the necessary participant fix and is not current acceptance. - Previous compatibility/base-sync head: `30059c0ededc0eab4e04e8299a884f36db8b292f`, integrating master `efd65c112` and the Inbox/workspace compatibility fixes. GitHub reports MERGEABLE after the push. Coordinator keeps Inbox standalone under More without rewriting the instance's Combined Inbox preference; opting out restores that preference. Saved merged-Inbox URLs resolve to their original owner, task status presets remain usable, and canonical repeated `workspace` parameters retain scoped, deduplicated pagination (including Needs attention). All 192 broader routing/Inbox/sidebar/privacy/review tests and the final 16 task-page tests pass, as do workspace typecheck/build, token gates and whitespace checks. The two review threads are resolved. Fresh exact-head CI, Greptile and Superagent results are pending. - The preceding `d5b9be5bd` remained mergeable but its CI routing gate intentionally fell back to GitHub-hosted runners: “stacked base merge commit does not contain the current base branch.” All 45 downstream jobs stayed queued. Current master was merged cleanly to refresh the pinned base; no runner/security settings or workflow logic were changed. Greptile scored that preceding head 4/5 with the two compatibility findings fixed here. Its automatic Superagent scan was inconclusive at checkout; the documented retry was still pending when superseded. Those results do not establish acceptance of this current head. - Previous approval/migration head: `d5b9be5bd3b8d6b0437e42bf48138377f34fa67d`, integrating master `ba5bba86d` and the stale-output approval correction. GitHub reports MERGEABLE after the push. The fresh P1 finding is fixed: every submitted item now validates and locks its bound output inside the resolution transaction, whether or not it carries a required-quality-review binding. Real database regressions verify that an advanced document revision or deleted attachment refuses the entire batch without recording approvals, and current ordinary output reviews still resolve. All 272 combined migration/review/interaction tests pass; workspace typecheck/build passed. Fresh exact-head Greptile and CI are pending. The automatic Superagent scan ended inconclusive at checkout; one documented on-demand retry is pending. These are explicit verification limits, not passed security results. The P1 thread is resolved. - Master consumed the next migration slot three times during this verification pass: confirmation recovery took 0398, continuation recovery took 0399, and GitHub acknowledgement recovery took 0400. Each upstream SQL/journal/snapshot is preserved unchanged. Preferences are now generated as idempotent `0401_white_lenny_balinger.sql` from upstream 0400; exactly five snapshots remain, and replay coverage locates this migration by table content. This is recorded concurrent integration churn, not a claim that older green results apply to the latest base. - Head `ed3b68da9` fixed the two history/URL findings with 26 focused regression tests, server/UI typechecks, workspace build and token gates passing. Greptile then scored that head 4/5 with the stale-output P1 corrected in the current head. Its partial CI is superseded, not counted as current acceptance. - Previous filter-fix head: `ed3b68da9`, based on the `8458d39d9` merge of master `063cfd438`. The two fresh Greptile findings are fixed: history outcome filters now match recorded per-item verdicts in SQL before pagination (mixed reviews remain in both applicable outcomes), and same-page URL assignee changes/back/forward navigation update that filter without clearing independent filters. All 26 focused database/navigation/collection tests, server/UI typechecks, workspace build, token gates and whitespace checks pass. The two review threads are resolved. Fresh exact-head CI, Greptile and Superagent results are pending. - The `8458d39d9` migration integration also passed all 328 targeted migration/interaction/confirmation tests, workspace typecheck/build and Superagent security scan. Greptile scored that head 4/5 with the two functional gaps fixed in the current head; those prior-head checks are not current acceptance. - Migration integration head: `8458d39d9721f4cb312c2390070a0609f8cddd1a`, integrating master `063cfd43813cabf18417605fb9eefb9aacf95c8d`. GitHub reports MERGEABLE after the push. Master advanced twice during this conflict-resolution pass: first its confirmation-recovery migration took 0398, then its continuation-recovery migration took 0399. Both upstream migrations and journal entries remain unchanged; Coordinator preferences are now generated as idempotent `0400_sad_guardian.sql` from upstream's latest snapshot. Exactly the five newest snapshots remain. The replay regression now locates the table migration by content instead of hardcoding a renumbered filename. - All 328 targeted question/review/confirmation/migration tests passed again after the second merge. Workspace typecheck/build and fresh exact-head CI/reviews are in progress. The preceding `0c46f46d6` passed workspace typecheck/build and Superagent security scan, but was superseded because master consumed migration 0399 while those checks ran. That prior-head result is not acceptance of the current head. No production settings or runner/security permissions were changed, and the PR has not been merged. - Previous integration head: `0c46f46d6f2b23417ab872ce3cb6ff2673647287`, integrating master `287b04d72f1b9fca1412e05435c75b8302ba0710`. GitHub reports MERGEABLE after the push. The renewed conflicts were `0398_snapshot.json`, `_journal.json`, and the top imports in `issue-thread-interactions.ts`. Upstream's `0398` confirmation-recovery index and journal history are preserved unchanged; Coordinator preferences were regenerated as idempotent `0399_bouncy_miek.sql`, with replay coverage updated and exactly the five newest snapshots retained. Both terminal confirmation notifications and Coordinator task assignment/required review behavior are preserved. - All 328 targeted interaction/review/confirmation/migration tests pass, as do workspace typecheck, map validation and whitespace checks. The workspace build and fresh exact-head CI/Greptile/Superagent checks are pending. The first migration-generation attempt hit the sandbox's local IPC restriction before generating files; the unchanged authorized local generation completed successfully. Prior-head green checks below do not establish acceptance of this merge. - Previous acceptance head: `8b7dcf739ed15afc51351fdfa0e0e5fbc75ab788`, with master `1aa3a21ac` integrated and GitHub reporting MERGEABLE. Greptile completed a fresh 5/5 review on this exact head with zero new actionable findings and no open review threads. The [fresh CI workflow](https://github.com/paperclipai/paperclip/actions/runs/40752864319) completed SUCCESS on attempt 2 at 19:40:08 UTC, including all eight E2E shards. Final completed check set: 55 succeeded, four conditional checks skipped, zero pending/queued/running checks, and one neutral/inconclusive Superagent scan. The auxiliary commitperclip review and isolated native Runner compilation/cache-reuse check both passed. This is not a claim that Superagent produced a passed security verdict. GitHub still reports MERGEABLE against current master `60b8089a44b4eaa37e627adbfdf2f130bda13335`. - Current-head E2E: all eight shards and the aggregate passed, including the previously failing YAML browser journey. Serialized server shard 9/9 failed: the protected YAML projection test exceeded its original 15-second timeout; its late update request then reached the next test's reset mock (the captured payload is the prior test's model/label update). All 88 tests in that exact unchanged file passed locally in 38.03 seconds, with no timeout or assertion changes. The original slow-request cause remains unproven. CI failure log: `/tmp/pr17983-8b7-serialized9.log`; local recovery: `/tmp/pr17983-adapter-yaml-repro.log`. The single bounded unchanged shard rerun passed, and the workflow completed SUCCESS. Rerun log: `/tmp/pr17983-8b7-serialized9-rerun.log`. The initial failure is retained; a passing retry does not establish its original slow-request cause. - Greptile scored preceding head `4873704be` 5/5 with one valid P2: opened task decision records missed company-scoped live invalidation. The current head uses the existing company decision prefix for resolution and effect-completion events, and resets private record payloads on access changes. Its thread is resolved. All 107 focused record/live-update tests, UI typecheck/build, token checks and whitespace checks pass. - CI on `4873704be` failed E2E shard 7/8 in the Advanced YAML lost-creation-response journey: its clipboard helper selected page text starting with “Skip to Main Content” instead of YAML. The exact unchanged journey passed in a disposable local browser instance (1 test, 17.1 seconds). No assertion, timeout or production behavior was weakened; the original failure cause remains unproven. Failure log: `/tmp/pr17983-487-e2e7.log`; local check: `/tmp/pr17983-agent-yaml-repro.log`. The full unchanged Advanced YAML browser suite also passed locally (6 tests, 37.8 seconds). Fresh current-head CI subsequently passed this journey and all other E2E shards. - Both the automatic and one supported on-demand Superagent scan on `4873704be` ended neutral/inconclusive: “Pinned PR scan merge-base checkout failed or timed out.” These do not count as a passed security verdict. The current head's automatic scan and one documented on-demand retry also returned the identical neutral checkout failure, with zero annotations or further diagnostics. This is an external scanner blocker, not a passed security verdict. No security settings or permissions were changed. - Recorded decision links now render and focus the full source-task record in both Classic and Coordinator, including recorded choice, submitted inputs, and the existing effect-result resolver. Fresh terminal records take precedence over older open-decision cache entries. Context and account checks keep mismatched or inaccessible records out of the DOM; no decision-history tab was added to tasks or chat. The shared company live-run feed now drives live-only filtering, including ended runs and terminal tasks with stale run entries. All 213 focused tests, UI typecheck/build, token and whitespace checks pass; the final focus follow-up passed all 188 task-detail tests and UI typecheck. The first new navigation test failed because its test host omitted company context; the host was corrected before the passing run. - Previous integration head: `375c5e89516705050d94bab99336212841b2e6e1` merges master `1aa3a21ac` after four additional upstream commits. The sole conflict was competing imports in `server/src/services/issues.ts`; both quality-review policy and inbound wake-notice notifications are preserved. The PR-specific review import now sits beside execution-policy imports instead of the frequently changing first line. GitHub mergeability is rechecked after the push. - On the combined master tree, workspace typecheck/build, all 153 affected UI tests, map validation, and whitespace checks pass. Four server suites passed 77 tests; the fifth suite's module-import hook timed out while broad verification was running. Its unchanged isolated run passed all 41 tests in 7.2 seconds with the original timeout. This records recovery without claiming the original import-timeout cause is proven. CI on `375c5e895` passed every substantive job, including all eight E2E shards and Superagent; Greptile found the two functional gaps fixed in the current head. A short-lived `669d90876` commit added the main fixes before the focused navigation follow-up. - Immediately preceding head `9d9737b49` completed all 56 checks successfully with four conditional skips, Greptile 5/5 and no open findings, plus a successful Superagent scan. The signoff E2E shard passed on its single accepted rerun; its initial cancellation/timing failure is retained below. Master moved again while those checks ran, requiring this final integration rather than treating the prior head's green result as current acceptance. - Previous head `9d9737b496911a6054d0ea22d1b42b28b115e9ab` includes master `d30c8f42b842327ee1c18c41a2efd6ba623da5cd`; that head was subsequently conflicted by moving master. The master merge preserved durable account-bound writes and Coordinator UX. Workspace typecheck/build, 416 focused UI tests, and 142 affected comment-route tests passed. A stable full UI run on preceding head `938ad1665` passed all 9,833 tests across 836 files; its final full workspace build also passed. - Greptile found nested proposal project inheritance and Classic bound-revision links on `91be74952`. Both were fixed with 142 interaction-route tests, 113 focused UI tests, and UI/server typechecks. Its next review identified a valid draft-preservation bug in the new Classic loading UI. Head `9d9737b49` keeps loading/error notices outside editors and annotations while preserving dirty drafts; 115 focused review UI tests pass, including pending/failed revision loads and continued draft editing. All related threads are resolved. Greptile completed a fresh 5/5 review of `9d9737b496911a6054d0ea22d1b42b28b115e9ab` with no open findings. - CI on `938ad1665` exposed an integration regression with the newly merged durable startup saves: the personal preference gate blocked the task page before its first session lookup. The guarded title editor now mounts while that lookup is pending, with presentation-specific tools still gated. All 216 affected component tests pass. The unchanged five-test `disconnect-write-recovery.spec.ts` browser suite passes against a newly launched disposable local-trusted PostgreSQL instance on port 33219, including the exact failed startup-save journey. No live provider, hosted authentication or production proof is claimed. The helper-owned test server exited and the port has no listener. UI typecheck, built-browser assets, token and whitespace checks pass. - Initial durable-write merge tests found duplicate imports and unscoped draft fixtures, fixed before 416 focused tests passed. A sandbox typecheck could not create its IPC socket; the supported rerun passed. The first revision-link test used a wrong Return to latest button assertion, corrected before the 113-test pass. The startup E2E reproduction initially still failed because an outer shell gate also blocked the page; the subsequent fix passed all five cases without changing the test or its timeouts. A focused loading-header assertion initially used an incorrect fixture title; correcting it preceded the 216-test pass. These failures remain recorded. - Earlier routing gates fell back to GitHub-hosted runners while master was moving or conflicted. Superseded queued runs were cancelled, then the `938ad1665` routing gate explicitly selected the ephemeral RunsOn fleet. Its CI actively executed and completed most tests; only the diagnosed startup E2E shard and aggregate failed, alongside the valid Greptile finding. Auxiliary GitHub-hosted workflows also completed successfully on earlier heads, so queueing alone is not an administrative blocker. Fresh [CI on `9d9737b49`](https://github.com/paperclipai/paperclip/actions/runs/40744012678) is the acceptance target; the previously failing startup E2E shard now passes. A different signoff E2E shard failed during executor resubmission when its fixture run was cancelled as `issue_continuation_waiting_on_review`. The exact unchanged journey passed locally (1 test, 46 seconds), and the preceding head with identical server/signoff code passed that CI shard. This supports a bounded rerun but does not establish the timing failure's root cause. After the workflow finished, GitHub accepted one rerun of that failed shard and its dependent aggregate; both passed, completing all checks on that head. The failure log is retained at `/tmp/pr17983-9d9737-e2e7.log` and the local reproduction at `/tmp/pr17983-signoff-repro.log`. No assertion or timeout was weakened. - On the first merge, full workspace typecheck/build, all 9,622 UI tests, 267 tests across seven affected server suites, two migration checks, map validation, token gates, and the PR diff whitespace check passed. Greptile then found duplicate local task-search filtering and lost completed item-review detail. Both are fixed, with 42 passing focused UI tests and passing UI typecheck/build. - A subsequent Greptile review found attachment confirmations without working links and mobile history without a visible host. Confirmation layouts now derive the authenticated immutable-content endpoint from the attachment ID, and Needs You uses the existing full-screen mobile sheet. All 119 focused tests pass, including missing/stale display URLs, both review layouts, and both mobile history entry buttons. UI typecheck/build and token checks pass. Full workspace typecheck/build also pass after the second master merge. - Greptile gave head |
||
|
Nicky LeachandPaperclip
|
c57af3c564 |
Reuse prepared Runner storage and installed skill selections (#18089)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/207/changes/paperclip/207 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip manages AI agents and their work. > - The native Runner prepares selected skills, instructions, and provider storage. > - Repeated preparation imports the company skill inventory and rebuilds Claude storage. > - Execution environments can already maintain prepared resources. > - This pull request lets them reuse account storage while Core keeps execution policy. > - New retry runs resolve current skills and keep the failed attempt's history. ## Linked Issues or Issue Description **What existing behavior does this improve?** Native Runner preparation and retry skill selection. **Current behavior** Explicit installed selections still trigger company inventory reconciliation. Plugin preparation receives account identity only on warm transitions. Claude's isolated mode requires a session-specific configuration directory. **Proposed behavior** Skip inventory imports when explicit selections are installed, while retaining current authorization and version resolution. Send account identity on initial preparation. Allow an execution environment to provide persistent Claude storage without enabling preconfigured permission behavior. A new retry resolves current skill assignments. **Reason and benefit** Remove repeated preparation work without adding environment-specific policy to Core. Related: #13984 describes a similar storage-lifetime problem in the separate claude_local adapter; this PR does not change that adapter. ## What Changed - Derive Claude’s configuration path from the single authoritative prepared-storage input; separately supplied CLAUDE_CONFIG_DIR values are overridden. - Add an explicit prepared Claude storage binding, forward it to the sidecar, and retain recovery and provider lifetime fencing. - Refresh Core-owned Claude settings after acquiring the lifetime lease. Reuse environment-prepared skills. - Reuse installed explicit skill selections and preserve missing-selection initialization and current access checks. - Reserve the prepared-home marker for environment provisioning; task overrides are rejected and configured bindings are stripped before native launch. - Clear inherited skill selection snapshots on new retries. Keep admitted execution recovery and failed attempt metadata. ## Verification - Current single-input follow-up `8dbc2f6ec`: 154 focused Runner checks pass with one existing skip; four Grok identity checks and four Pi packaged-probe checks pass. Shared immutable source identities advance to Grok10/Pi32/Copilot74. All56 current-head CI checks pass (four intentionally skipped); standard and enhanced reviews are both5/5 with no unresolved threads. - Focused Runner sandbox tests: 29 passed. Focused environment tests: 16 passed. - company-skills-empty-runtime: 10 passed. - Plugin Runner and runtime selection database tests were attempted. Local execution is blocked because embedded PostgreSQL cannot load libzstd.1.dylib; these checks remain for CI. - New database regressions cover installed-selection reuse, initial account preparation, and retry snapshot removal. - Prepared-home trust-boundary regressions: 4 targeted checks passed (task schema, all env binding scopes, native executor). - Focused Pi public-install probe checks: 4 passed. The build smoke probe now asserts the current Pi 32 profile and exact digest. - CI found and fixed retry-context TypeScript inference, stale Pi identity fixtures, Docker source identity inputs, and the skill-observability test’s old assumption that every selection imports inventory. Cold and reused selection coverage remain. - CI chat shard 2 initially failed three GitHub/Discord idle-database assertions. An unchanged-head rerun passed; the original cause remains unexplained. - Runtime head `f5b53679a` passes all 56 current CI checks (four jobs intentionally skipped), including the full build/public-install probe, TypeScript, Rust, Runner and server matrices. Greptile is 5/5 with no unresolved review threads. - The requested enhanced review scored 5/5 on `f5b53679a`, with one nonblocking documentation request. Documentation follow-up `1ac5d26a8` adds only the documented existing ownership/0700 requirement; the thread is resolved and all 56 current-head checks pass (four intentionally skipped). The subsequent single-input runtime follow-up8dbc2f6ec repeated the complete CI → standard5/5 → enhanced5/5 cycle successfully. - Broader verification is delegated to GitHub CI as requested. The initial policy check rejected a generated lockfile repair; that change was removed because Core CI already resolves stale importer entries. ## Risks Prepared storage must be provisioned by the execution environment before launch. It is not credential isolation. Core replaces managed Claude settings under the existing provider lease. No new per-run harness homes are created by this option. New retries may select a newer skill version; recovery of an already admitted provider execution keeps its saved input. ## Model Used OpenAI Codex, GPT-6 family, with reasoning and tool use. The exact model revision is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked related context above - [x] I have described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change - [x] I have run focused local tests; database and full verification remain for CI as explained above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f8484a9f8d |
fix(native): deliver selected skills to remote Codex homes (#18080)
## Thinking Path > - Paperclip coordinates company agents and preserves their assigned instructions and approval rules. > - Native runs pin an immutable skill inventory for each execution. > - Built-in remote Codex launches prepare that inventory on the controller, but transfer only authentication, configuration, and the separate context assets. > - Codex needs the selected skills under its isolated home. An explicit skill path does not load a skill outside its registered discovery roots. > - This pull request transfers the sealed selected inventory to the path already used by explicit skill inputs. > - Each launch replaces stale assignments inside the authorized workspace while preserving provider history. ## Linked Issues or Issue Description Refs #18025. The corrected first-task instructions exist in the selected bundle, but remote Codex must receive those bytes before it can follow them. The legacy skill resolver work in #16646 and #9323 uses different paths and does not repair native remote launch staging. **What happened?** A fresh remote native Codex run could not read its assigned first-task skill. It produced an approval question instead of the required confirmation. The Product E2E journey correctly rejected that question before child creation. **Expected behavior** Codex receives the selected immutable skill at the same provider-local discovery path used by explicit skill input. Removed or unassigned skills do not remain available after a new launch. **Steps to reproduce** 1. Prepare an isolated Codex home with one assigned skill. 2. Run the built-in remote launch staging path against a fresh remote filesystem. 3. Resolve an explicit invocation of that skill. Before the repair, its provider-local file is missing. ## What Changed - Transfer only the prepared selected skill inventory into the isolated remote Codex home before other launch writes. - Reject writable files, symlinks, missing entries, and extra unassigned packages in the controller projection. - Validate each provider-home path component below the authorized workspace. Reject redirected retained paths before deleting stale skills. - Create the private discovery root before native sandbox transfers, which omit the source-root metadata and can ignore the requested directory mode. - Replace stale or empty inventories without changing provider conversation history, preconfigured homes, other providers, or plugin-provided launch preparation. - Exercise the real home materializer and turn resolver through the existing test-only Runner surface. Document the remote discovery contract. - Synchronize an existing Telegram attachment-failure test with the durable provider-effect commit, preserving its required final states. This CI repair is a separate test-only commit. ## Verification - The new production-staging regression failed before the repair because the resolved skill file was absent. - All 23 focused staging checks pass. They cover delivery, file modes, empty/revoked inventories, history retention, extra packages, writable files, source symlinks, target symlinks, redirected ancestors on sandbox/SSH, and an authorized workspace alias. The filesystem fixture packs and extracts actual tar bytes without applying the requested root mode; it preserves exact projected-file digests and executable bits. - The unchanged materializer and prepared-launch contracts passed 26 tests. Eight selected native caller checks passed; other cases in that filtered run were not selected. - Independent probes used official Codex 0.160.0, the real materializer, production staging, the real resolver, and a local model endpoint. The original staging supplied no skill message. The final staging supplied the complete selected skill body with its exact SHA-256. A final probe also ran production Daytona `performSyncIn`, backing only its SDK exec/upload/chmod edge with the local filesystem under umask `022`: the pre-review source reproduced root `0755`, and the final source retained root `0700`, package `0555`, file `0444`, and the exact selected digest. All model requests returned local 403; there were no upstream requests or credentials. - Full local `pnpm -r typecheck` and `pnpm build` passed on verification commit `e7d4e7a9`, based on `60b8089a`. The full local `pnpm test:run` on that frozen tree was not completed: later embedded PostgreSQL startup failed because the host could not allocate semaphore sets (`semget`, SEMMNI/SEMMNS), and the owned attempt was stopped with its log preserved. Its earlier four caller-fixture failures and one workspace-conflict failure are recorded separately below; the later resource error is not an explanation for them. Delivery commit `567d838d` replayed the initial six-file patch onto `ee67d2ec`; production was last changed in `23456f51`, followed by test-only commits `002fa4a7` and `6e8fa595`. All 24 staging checks, all 26 materializer/prepared-launch checks, and the complete native executor file (968 tests, zero skipped) pass locally. Server typecheck also passed after these test edits. The complete CI matrix passed on `002fa4a7`; current-head CI is tracked separately. - Initial-head CI exposed four caller fixtures that manually wrote auth/config without preparing the skills inventory. The same four failures reproduced locally. Those fixtures now use the real home preparer for their existing empty inventory and assert its transfer; all 34 retained/replacement scenarios and the complete 966-test caller file pass. A sandboxed full-file attempt could not bind local PostgreSQL (`EPERM`); it was stopped with its log preserved, then the identical command passed with local fixture access. Initial-head CI also recorded timeouts in workspace recovery, approval-queue backoff, and heartbeat attribution. Their three complete files pass unchanged locally (102 tests); the next CI head also passed those cases. The original slowdown remains unexplained. The consequent approval cleanup foreign-key failure is a timeout teardown race already recorded in the owning worker documentation. - The frozen full local run also observed an existing workspace-conflict test fail. The test and owning workspace code are unchanged between the verification and delivery trees, and the isolated delivery case passes. The original failure remains unresolved: stopping the later resource-blocked run prevented the final assertion detail from being emitted. That isolated pass does not replace the earlier result. - CI on `23456f51` caught a Telegram test reading `provider_effect=processing` after observing the provider mock call. Production commits `processed` later, so the test now waits for the unchanged complete durable-state assertion. All 18 selected Telegram attachment/media cases pass (the other 1,670 cases were not selected). The same CI attempt also recorded an E2E runner shutdown followed by cancellation, without a failed Playwright assertion; the necessary new test-only head reruns the matrix. Earlier failures remain preserved and are not reported as passes. - Review requested an executor-level preconfigured-home negative test. Retained and replacement workspace cases now drive the actual launcher, assert successful context staging, and exclude the isolated home's skills/auth/config transfers. A counterfactual that removes only the executor exclusion makes both tests fail; the exact production bytes were restored before the complete 968-test pass. Flat and nested selected package delivery is also covered. Optional arbitrary remote-stderr expansion is deferred; the repair keeps the existing closed error code and report surface. - Final head `6e8fa595` passes the complete CI matrix, including typecheck, build, Runner, browser, and aggregate gates. Two first-attempt jobs were interrupted by runner shutdown/lost communication; one unchanged-head rerun of those jobs and their aggregates passed. The underlying host interruption cause remains unknown. Both current-head automated reviews are 5/5 with the implemented finding resolved. - The original checked-in lockfile fails a fresh frozen install on the baseline. The repository's maintained resolution-then-frozen sequence passes. No lockfile change is included. - Independent source review found and then verified the retained-ancestor guard. This PR does not claim that the full live Cloud onboarding campaign has passed. It changes no journey assertion or approval gate. ## Risks Launch preparation adds an asynchronous inventory walk, one bounded remote cleanup command, and one transfer of the selected skill tree. Those costs grow with selected files and bytes and occur only at provider launch. A failed validation, cleanup, or transfer prevents launch. Remote Node is already required by the shared SSH/sandbox launcher. The selected packages remain read-only; the private discovery root retains Codex's normal writable mode. No database, API, credential-routing, or deployment change is included. ## Model Used OpenAI Codex based on GPT-6, with reasoning, source inspection, tool use, and code execution. The host does not expose the exact serving model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
c74171d620 |
fix(recovery): retire superseded native workspace retries (#18077)
## Thinking Path > - Paperclip coordinates agents while retaining their work and execution authority. > - A failed native run can schedule an exact-session retry and retain a sandbox. > - An explicit task Retry can already have completed successfully on a successor. > - Recovery correctly refuses the old run after its task execution lock changes. > - It nevertheless leaves that old retry scheduled and its workspace without a preservation owner. > - This change retires that proven superseded retry and registers exact-source preservation together. > - Existing stop receipts and saved-file verification still control workspace switching. ## Linked Issues or Issue Description **What happened?** A native sandbox run failed before starting a provider turn. An explicit Retry completed on a linked successor. The original coordinator remained in `retryable_failure` with an active sandbox lease. Restart recovery returned `issue_execution_lock_changed`, while the ordinary sweep kept delaying it. A pending workspace selection then made every new task run fail during setup. **Expected behavior** Stop replaying a failed retry after its linked explicit successor has completed. Preserve the original files through the existing system worker. Keep the current task, ownership, pending choice, and unresolved action outcomes intact. **Steps to reproduce** 1. Persist a failed, unowned native run with a prepared remote workspace and `retryable_failure` coordinator. 2. Complete a same-company/task/agent explicit Retry linked to that run. 3. Release the task's old execution lock and schedule a workspace selection. 4. Run startup or ordinary retry recovery. Observe the old retry remain scheduled and the selection fail with an unfinished-recovery conflict. **Paperclip version or commit** Regression reproduced against `644937c931`. The current repair is based on refreshed master `60b8089a44`. **Deployment mode** Native Runner with remote sandbox workspaces. The reported runtime used ACPX Claude with per-turn lifecycle. The original provider session-resume rejection is not independently reproduced or claimed fixed here; this PR addresses the subsequent stranded recovery. Related: #17839 records preservation when restart classification becomes terminal. This change handles the earlier lock-mismatch return and ordinary retry admission after a completed explicit successor. It uses the same preservation service and does not require that PR. ## What Changed - Share a narrow superseded-retry settlement between startup recovery and the ordinary retry sweep. - Require a failed source with no coordinator owner or process, its exact prepared source lease, and a completed linked successor under the same company, task and agent. - Atomically retire replay and register preservation. Do not alter task state or create another provider run. - Add PostgreSQL lifecycle/safety regressions, a real-file preservation and workspace-selection regression, and execution/map documentation. ## Verification - Red: both startup and ordinary admission regressions left the old coordinator in `retryable_failure` with no preservation obligation. - Green: the same two cases retire the retry and retain the exact lease without another run or task mutation. - The assembled PostgreSQL/filesystem regression passes: workspace selection remains blocked before preservation, the existing service saves `saved.txt` and preserves newer local work, then the normal selection guard applies the pending shared workspace. - All 90 native retention/resumption cases pass. Three neighboring startup-recovery cases pass, including the blocked-owner inventory. Full recursive typecheck and full build pass at runtime head `23ddb213b4`; final `19d7738ae3` only moves the function comment into execution documentation. Map, module-boundary and diff checks pass. The duplicate full local Vitest run was stopped after 20 minutes in its first long general shard once the complete current-head hosted test gates passed; no complete local-suite pass is claimed. - Current-head CI has 53 passing checks and two intended Storybook skips, including full typecheck, all test shards, build, Runner verification, browser shards and both aggregate gates. The separate `commitperclip PR Review` job is still queued on `ubuntu-latest` with no assigned runner; its outcome remains pending. Greptile reviewed exact head `19d7738ae3` at 5/5, with zero unresolved threads. - Live acceptance passes on exact final source `19d7738ae3`: a disposable Daytona Linux sandbox, the current immutable staging fleet snapshot, SDK 0.203.0, a Darwin arm64 source host and Node 26.4.0. The real native workspace preparation/export and provider stop/start paths preserve an unpredictable report and newer host files. The ordinary workspace-selection guard stays blocked until preservation commits, then applies the pending shared workspace. - The old workload's start count and side-effect marker each remain one. Recovery creates no new run or native result. The sandbox finishes stopped and is then deleted by the probe; a final provider read confirms it is absent. - A second live acceptance also passes on the original runtime's older immutable snapshot with the source explicitly archived before recovery. It verifies cold source reopening, the same report/newer-file checks, one workload start and side effect, pending workspace application, and confirmed deletion. Its initial probe attempt used immediate archive/deletion reads and failed during asynchronous cleanup observation; bounded provider-state polling fixed the probe without a product change. The abandoned probe allocation is independently confirmed absent. - Scope: database fixtures represent the already-failed run and completed explicit successor. The old workload is a persistent shell process with observable file/side-effect markers, not a paid Claude turn. This qualifies the repaired recovery handoff and real provider preservation; it does not reproduce the initial ACPX session-resume rejection. - The original hosted task has only been inspected read-only. No rollout or original-task recovery is claimed. ## Risks - Incorrect successor classification could retire legitimate work. Matching requires the server-persisted explicit Retry link, same company/task/agent, later successful committed execution, no current source execution lock, no recorded process, no coordinator owner, and no workspace finalization owner. - Retirement is not a stop receipt. The exact source remains retained, unknown effects remain unknown, and workspace switching still waits for the existing worker's stop and file-persistence proof. - A failed preservation transaction rolls back retirement. No new scan, retry budget, schema or public API is introduced. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use and code execution. The host does not expose a more specific model ID or the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass — focused suites and live acceptance; full-suite status above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green — separate template-review job still queued as noted above - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups — exact head `19d7738ae3` - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
c4b6168e08 |
fix(server): disclose assigned tool search page bounds (#18056)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/187/changes/paperclip/187 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the control plane for AI agents at work. > - Native agents find connected app tools through the assigned-tool search tool. > - Large catalogs require small pages and an offset for the next page. > - The input schema enforces a maximum of 20 results, but its descriptions omit that number. > - Providers can render the integer field as a plain number and hide its numeric bounds. > - This pull request puts the existing range and default in both descriptions. > - Agents can choose a valid page size before they make a search call. ## Linked Issues or Issue Description **What happened?** An agent can request `limit: 50` for `paperclip_search_assigned_tools`. The tool rejects the call with `invalid_tool_call` at `/properties/limit/maximum`. Its model-visible description does not state the accepted range. **Expected behavior** The tool description and the `limit` description must state the range of 1 to 20 and the default of 5. Agents can use `nextOffset` to read more results. **Steps to reproduce** 1. Inspect the assigned-tool search description in a provider tool list. 2. Render the integer schema as a TypeScript number field. 3. Request `limit: 50` without a visible page-size bound. 4. Observe the validation error. Retry with a valid size and offset. ### Hypothesis and thesis for a fix The numeric schema is correct, but the descriptions omit the page limit. Put the existing range and default in text that all providers can see. Keep the strict validation in the assigned-tool discovery owner. This should reduce avoidable retries. It does not guarantee that every model call will be valid. ## What Changed - State the accepted range and default in the search description and the `limit` property. Add the existing default as schema metadata. - Test the tool listing and calls through the real HTTP MCP bridge for both compact delivery paths. - Document the page size and offset procedure in the Runner README and verification maps. ## Verification - Red: the two new bridge tests fail against the unchanged implementation because the listing omits the numeric guidance. - Green: all 37 tests in `server/src/services/native-runtime/assigned-mcp-tools.test.ts` pass after the change. The tests cover default pages, a 20-result page, offset continuation without duplicates, and rejection of 21 and 50. Existing tests cover grants, revocation, modes, approvals, aliases, and schema chunks. - Current head `7d658d0c4e4d9db3feb53a5c508214440ff058c4`: 37/37 focused tests pass locally. Greptile is 5/5 with no unresolved threads. Current-head CI is green, including typecheck, build, unit tests, Runner checks, and all browser shards ([CI run, attempt 2](https://github.com/paperclipai/paperclip/actions/runs/40733284523/attempts/2)). Browser shard 7 initially failed in a newly merged agent YAML editor test when its clipboard read included the whole page. The single retry passed without a code change; the exact cause is unconfirmed. The original failure remains in [the CI log](https://github.com/paperclipai/paperclip/actions/runs/40733284523/job/122975224906). - Run the focused suite from `server/`: `pnpm exec vitest run src/services/native-runtime/assigned-mcp-tools.test.ts`. - `node scripts/check-maps.mjs` and `git diff --check` pass. - The bridge uses real local HTTP. The app gateway uses synthetic descriptors. No deployed build or live model improvement is claimed. - Full local verification is incomplete. Recursive typecheck and server typecheck require `cargo`, which this workspace lacks. A direct server TypeScript check was killed by the workspace. The full test suite had no final verdict before the resource stop. CI must supply the remaining verification. ## Risks Low risk. This change describes existing limits and adds default metadata. It does not change search execution, page caps, gateway access, revocation, approvals, or audit. Models can still send invalid calls. This change has not measured their future retry rate. ## Model Used OpenAI Codex, GPT-6 family, with tool use and code execution. The exact provider model ID and context window are not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used, its capabilities, and unavailable model metadata - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs - [x] I have described the issue in-PR following the bug report fields - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] The public branch name describes the change and has no internal task ID - [x] I have run the focused tests locally and they pass - [x] I have added tests for the reported failure - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open recommendations or follow-ups - [x] I will address reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
a2fbce9305 |
Fix plugin session and directory reuse on follow-up turns (#18045)
## Thinking Path > - Paperclip preserves task state when agents receive follow-up work. > - Plugin computers keep their process state outside the controller. > - A local receipt proves that a previous plugin run suspended. > - A cold resume left that receipt in place after the session became live. > - The next warm follow-up rejected the stale receipt as an identity mismatch. > - A separate follow-up path also treated a fresh task directory as a Git worktree. > - This PR consumes the obsolete receipt after admission and preserves the bound directory on follow-up. ## Linked Issues or Issue Description Related: #17820. This change retains the controller restoration added by #18018. **What happened?** A plugin session resumed from a suspended checkpoint and completed a turn. Its next warm follow-up failed with `runner_state_identity_mismatch: prior_owner_terminal_state_indeterminate`. The journal had the completed run's identity. The local suspension receipt still had the preceding run's identity. Two separate task histories showed this sequence. After a fresh retry completed in a repository-free plugin directory, an ordinary follow-up could instead fail with `The selected plugin environment does not support Core Git worktree preparation.` The saved task settings contained only `mode: isolated_workspace`, which introduced the default Git strategy on the next admission. **Expected behavior** Keep the same admitted provider session during warm follow-ups. Reuse the saved plugin task directory without Core Git preparation. Retain ownership checks and explicit strategy settings. **Steps to reproduce** 1. Complete a warm plugin task and let its session suspend. 2. Resume the checkpoint and complete another turn. 3. Send a follow-up while that session remains warm. 4. Separately, complete a fresh retry in a repository-free plugin directory and send an ordinary follow-up. **Paperclip version or commit** Both regressions reproduce against base `820c45846` with the added tests. **Deployment mode** Hosted plugin execution. Local reproduction uses the real filesystem and a disposable PostgreSQL 18 database, with synthetic provider execution. ## What Changed - Remove the local plugin suspension receipt after provider recovery and durable run admission publish a live session. Admission still checks the receipt first. Normal suspension writes a new receipt. - Recognize a saved fresh plugin directory through its company, task, workspace, owner, attempt and path. Use the existing directory-selection path. Explicit strategy settings still apply. - Add cold-to-warm, database-backed workspace-selection and ownership regressions. Update execution semantics and session verification recipes. ## Verification - Red: the cold-to-warm regression failed on unchanged executor code with `prior_owner_terminal_state_indeterminate` on the follow-up. - Red: the PostgreSQL regression selected a Git strategy on unchanged environment-selection code. The explicit-strategy denial case already passed. - Green: all 95 tests in `workspace-preparation.test.ts` and `provider-fresh-workspace.test.ts` pass against a disposable local PostgreSQL 18 database, selected with the existing `PAPERCLIP_WORKSPACE_PREPARATION_TEST_DATABASE_URL` test seam. - Green: 92 focused executor cases pass, including plugin checkpoint recovery, warm rotation and denial cases. The full `plugin-runner-task.test.ts` suite also passed (38 tests). - Green: direct server check, `node node_modules/typescript/bin/tsc -p server/tsconfig.json --noEmit`. - Green: `node scripts/check-maps.mjs`, `git diff --check`, and a local diff scan for secrets, personal data and live identifiers. - Full local native suites: 988 of 1,004 cases passed. Sixteen integration cases could not start their bundled PostgreSQL cluster. The failure persisted outside the sandbox. These cases are not claimed as passing. An earlier sandboxed attempt also denied process and loopback operations; those denials disappeared when run outside the sandbox. - The first broad local commands could not locate Cargo on `PATH`. Rust is installed under `~/.rustup`; its Cargo executable was verified. Per the requested scope, subsequent local verification stays targeted to changed TypeScript paths; no Rust source changed. CI Typecheck + Release Registry and Rust verification passed on the initial commit. The complete repository test suite was not run locally. - Review regression: an explicit agent-level Git strategy reproduced the incorrect directory fallback before correction. After including the raw agent strategy in the existing check, all three targeted cases (implicit, task-level explicit, agent-level explicit) pass. The server TypeScript check passes after this correction. - CI on `a81668c4b`: the first workspace-a 2/2 attempt failed in unchanged UI `AgentActionButtons.test.tsx` (`calls clearError and refreshes agent-related queries`, zero mock calls). [Original failed job](https://github.com/paperclipai/paperclip/actions/runs/40717870938/job/122921463517). That shard passed on `294c5dfc3`. The failure is retained here. The UI shard passed on final commit `687000287`; no unrelated UI edits or local UI suite runs were added. - CI also exposed three existing local-directory recovery failures on `a81668c4b`: the review correction applied the raw agent strategy guard to local directories. [Original failed server job](https://github.com/paperclipai/paperclip/actions/runs/40717870938/job/122921463229). Narrowed that guard to adapter-managed directories, preserving local recovery behavior. All 20 targeted cases (three plugin selection cases and 17 local recovery scenarios) pass afterward, and server TypeScript verification passes. - Final commit `687000287d3339b7e15e35d95515b3c75c273d7a`: [hosted CI is green](https://github.com/paperclipai/paperclip/actions/runs/40720169342), including the corrected server shard, all other server/workspace shards, Runner checks, typechecking and build. All 55 non-skipped PR checks pass. Greptile scores this exact commit 5/5, with no unresolved review threads. GitHub reports no merge conflicts. - Pre-merge hosted acceptance was requested. The maintained deployment CLI stopped during artifact discovery: no published private Cloud composition exists for `687000287`. The current private build requires canonical Core master readiness and standard-image provenance. The Core-only branch preview omits the private computer plugin needed for the journey. No campaign or tenant update was started; private-preview deployment support is a separate prerequisite. No live deployment or provider acceptance run is claimed. ## Risks The receipt is removed only after the existing runtime publishes an admitted session. Earlier admission failures keep their proof. A later normal suspension creates a new receipt. The change preserves existing identity, lifecycle and controller checks. Directory selection remains limited to the exact bound, active, repository-free plugin recovery directory. Foreign owners, changed paths, wrong attempts and explicit Git strategies do not gain this fallback. No schema, API, credential, permission or fleet-default change is required. Existing failed runs and recovery holds remain history. This PR does not edit quarantined state or replay an interrupted task. Rollback restores the previous code and can reintroduce the two admission failures. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use and code execution. A more specific model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
567c043b17 |
feat(server): expose bounded hot-upgrade readiness (#18037)
## Thinking Path > - Paperclip coordinates agent work and keeps its authority in the server. > - Remote native runners can preserve execution through a server replacement. > - Deployment operators need evidence that current runs support that recovery. > - Process-local drain counters do not establish this fact. > - This PR adds an optional, operator-only readiness observation from existing records. > - A deployment consumer can use that report while the existing runtime retains recovery authority. ## Linked Issues or Issue Description Refs #17439. **What happened?** A deployment cannot distinguish recoverable remote native executions from runs that require a task drain. The existing status endpoint reports only process-local counts. **Expected behavior** An instance operator can request a bounded readiness report without stopping admission or exposing checkpoint material. Missing ownership or persistence evidence must prevent readiness. Ordinary status reads must keep their current cost. **Steps to reproduce** Run a remote native task. Read the task-drain status. It reports active work but supplies no recovery eligibility. Deployment code therefore has no affirmative signal to select an active-run handover. ## What Changed - Add `GET /api/instance/task-drain?upgradeReadiness=1` for instance administrators and existing Cloud control callers. - Return aggregate eligibility from current run identities, controller leases and bound opaque checkpoint headers. Unknown work, pending admission and incomplete reads prevent a positive report. - Bound the explicit read to 1,001 rows, encrypted headers of at most 256 KiB each and 8 MiB total, and a two-second SQL statement timeout in a read-only transaction. Keep ordinary status reads in memory. - Add PostgreSQL, count-transition and route authorization tests. Update the API contract, operator docs and access maps. ## Verification - All seven readiness cases pass against an isolated PostgreSQL 18 fixture. They cover missing checkpoints, legacy adapters, missing process/session identities, foreign owners, expired leases, missing lease tokens, failure state and in-flight work. The positive SQL fixture uses the real Runner checkpoint writer and encryption. Regular sandbox workspace identity and plugin environment-lease identity are tested with independent IDs. Stopped/released/undispatched plugin tasks and every mismatched identity field are rejected. Local/legacy/malformed/oversized or mismatched headers are rejected. A held PostgreSQL table lock proves the two-second cancellation budget, rollback/connection release and unrelated query responsiveness. - All 123 focused readiness, instance-settings and Cloud control assertion tests pass, including operator-only access and no readiness read on ordinary status requests. Runner’s 168 checkpoint tests pass. Runner and server typechecks pass after review fixes. - Workspace typecheck, full build, module-boundary and map checks passed locally before the master rebase. Current-head GitHub CI is now green, including typecheck, build, all grouped/serialized server and Runner tests, browser shards, policy and review-dispatch checks. Greptile completed cleanly at 5/5 for `f15fb0c7e9dfb975090cd5d791a1c8e6701275f8`; all actionable threads are resolved. - The broad local test run was stopped after the macOS embedded PostgreSQL fixture failed to bootstrap. The SQL test was then run on the isolated PostgreSQL fixture. No assertion or timeout was weakened. One unrelated existing route test exceeded its 15-second limit under concurrent local builds; it passed on one isolated retry and the subsequent whole route suite passed. - Hosted active-run upgrade acceptance has not run. This PR supplies an observation; it does not change rollout policy or establish live continuity by itself. ## Risks This is an instance-wide observation, so company members and agent keys must not receive it. A readiness snapshot can age as work changes; shutdown and startup retain their existing execution authority and fencing. The report does not validate a replacement artifact or authorize a schema migration. It decrypts only bounded headers through the checkpoint owner and delegates format/identity inspection to Runner; full journal restoration and physical sandbox authentication remain with recovery. No migration, recurring query, new checkpoint write or wire protocol change is included. The additive Runner header inspection contract reuses its existing parser. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, source inspection, tool use and code execution. The host does not expose the exact serving model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass for the affected contracts; the broad-run limitation is recorded above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
1ed0af4186 |
fix(server): wake native chat progress from committed changes (#17639)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat users receive coarse progress updates while native runs execute. > - A recurring chat pass scans saved run events even when no progress changes. > - This traffic prevents an idle database from staying idle. > - Events, lineage changes, and publication state already identify when projection can change. > - This PR gives native progress its own commit-driven worker and schedules only unfinished work. > - Idle progress stops scanning while restart and recovery preserve delivery. ## Linked Issues or Issue Description Refs #17606. This is slice 29 of the recurring-server-work stack, based on `codex/sleep-28-chat-run-milestones`. It owns native chat progress only. Related historical optimization: #16845. The parent and this PR must land in stack order. **What happened?** Native chat progress is still called by the one-second chat reconciliation loop. It repeatedly selects run events with no new work. Its cursor also truncates PostgreSQL timestamp precision, which can repeat a rejected full page forever. **Expected behavior** Committed events and eligibility changes should wake the projection owner. Existing rows should reconstruct work after restart. A settled worker should have no recurring database activity. Full batches, lock contention, and errors must retain recovery. **Steps to reproduce** Start an otherwise idle instance with chat history. Observe the native progress lane in the recurring chat coordinator. For the cursor defect, create a full page of suppressed candidates sharing a microsecond timestamp and one eligible later candidate. The old cursor fails to advance. ## What Changed - Extract native progress from lifecycle milestones into its owning service. Keep the public combined API and budget compatible. - Register a scoped worker with the existing delivery coordinator and scheduler. Remove native progress from the recurring chat coordinator. - Notify before allowlisted event writes in both persistence owners. Preserve outer commit, rollback, duplicate detection, and the ordinary non-progress log fast path. - Extend the existing lineage/authority notification boundary once. Failed-publication replay and operator retry carry their issue scope. First working publications signal progress so commit ordering cannot strand an already-inspected event. - Limit notification payloads to company/task IDs so wider caller inputs never serialize database handles or private fields. Load task-binding fields only for progress event kinds; other native/recovery events retain their narrow lock query. - Preserve current safe phrases, cadence, phase deduplication, provider policy, authorization, and final-output precedence. - Retry concrete issue/run lock contention. Disarm deleted or terminal candidates. Preserve scoped dirty work across batches, failures, drain and shutdown. - Preserve PostgreSQL timestamp precision in pagination. Add a bounded regression for the former infinite rejected-page scan. - Document the producer audit, lifecycle contract, failures, and remaining limits in `doc/plans/2026-10-25-native-chat-progress-sleep-audit.md`. ## Verification Published head: `b7ee820e49fd06c780b36b79d9b1c0883557fb73`, targeting master after #17606 merged at `5f0da562dbfdf3229f3cd4bb3dc742ab541986b4`. A normal ancestry merge preserves the complete 18-path native-progress patch byte-for-byte relative to its verified parent, with all six unrelated current master paths intact. No new production change was introduced during final reconciliation. - **163 distinct selected cases pass on the verified unchanged patch:** 137 across eight complete suites (native progress, atomic event append, all five milestone worker suites, and app admission wiring), plus 26 real chat integration cases selected by `native progress`. The other 1,595 chat integration cases were skipped, not passed. - The full worker suites cover restart, committed events, trace reprojection, duplicate/no-progress events, first-working publication ordering, concrete lock contention, issue scope, backlog, drain/resume, failures and shutdown. Settled history leaves no recurring SQL. The real chat cases verify operator retry and replay, microsecond pagination, authority, cadence, phase deduplication, final/question precedence and redaction. - The sole earlier parent conflict preserves both this slice's issue-scoped retry wake and the exact voice conversation scope. The current parent's milestone producers, onboarding completion guard, identity query/read-scope coverage, migration 0397 and voice cleanup/retention repairs remain present. - Server production typechecking, build and module-boundary checks pass. The selected server tests live under the excluded `src/__tests__` path and were executed separately. Each database owner uses a new isolated app-migrated PostgreSQL fixture. - [Current-parent acceptance and failure ledger](https://github.com/paperclipai/paperclip/blob/b7ee820e49fd06c780b36b79d9b1c0883557fb73/doc/plans/2026-10-25-native-chat-progress-sleep-audit.md#current-milestone-parent-acceptance-october-28) records current counts and scope. Older combined checks and failures retain their original revision scope and are not added to 163. No new table, dependency, workflow, migration, or scheduler contract is introduced. - **Exact-head Greptile is 5/5 and all 55 current checks are clean.** No Ponytail review is present, which is an explicitly authorized nonblocker; no failing verdict is waived. Older head/parent reviews are not current passes. Actual failing Ponytail findings must be fixed. Missing/stopped/paused turns without a verdict are authorized nonblockers; active unfinished reviews retain the original per-head timer with a 25-minute monitor cutoff respecting the 30-minute maximum. - No complete local repository matrix, live provider tenant, external wake, physical host/database suspension or savings claim is made. Following stack duties remain separate. ## Risks - Signals are process-local under the existing single active writer contract. Independent SQL writers or another active process need an explicit wake integration. - Allowlisted direct log writes now use a transaction. Native bound runs register transaction intent; unrelated direct event types retain their atomic-statement path. - The shared eligibility boundary wakes both lifecycle and native projection. It does not decide provider policy or copy event payloads. - The existing 24-hour selection window, event-time cadence and authority checks remain. A suppressed event is not retried just because wall-clock time passes. - This establishes worker quiet, not whole-instance suspension. Cloud alarm coverage and conservative sleep gates do not change. Other chat maintenance remains on its existing cadence. - This actual-master reconciliation still requires fresh exact-head review and CI before protected merge. Subsequent slices retain their own gates. ## Model Used OpenAI Codex, GPT-6 family, with repository inspection, code execution and browser-test tools. The exact serving variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0791da0cc5 |
fix(server): synchronize routine history during native status commit (#18033)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/173/changes/paperclip/173 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip manages work assigned to AI agents. > - A routine stores each invocation and its execution task. > - Native completion commits task status without using the public issue route. > - That route synchronizes routine history, but the native status transaction did not. > - A successful task could remain listed as an unfinished routine invocation on a local instance. > - This pull request uses the existing database projection in the native status transaction. > - Task status and routine history now commit or roll back together. ## Linked Issues or Issue Description **What happened?** A real routine execution completed successfully. Its task had status `done` and retained its routine origin and invocation identity. Public routine history still returned `issue_created`, with `completedAt: null` and a linked task whose status was `done`. The existing five-minute reconciliation runs only on cloud-managed instances. It does not close this path for a local instance. **Expected behavior** An authoritative native task status change must update the matching routine invocation through the existing routine status rules. It must not update another task, routine or company. **Steps to reproduce** 1. Create a routine on a local instance and run it through the public routine API. 2. Let its native execution finish and commit the task as done. 3. Read the routine run history. Before this change, the invocation can remain `issue_created` while its linked task is done. **Paperclip version or commit** The retained failure used merged production `39e811c9c5`. The missing hook is also present on this branch's base, `8049fbf51d`. Related: #16633 contains the existing cloud-only reconciliation. A bounded search found no duplicate native synchronization fix. ## What Changed - Synchronize routine history inside the native status transaction after the task status projection. - Require matching company, invocation and linked task before synchronization. Preserve supported custom display origins. - Add database-backed tests for terminal and active states, replay, invalid custody, rollback and a real plugin-managed routine dispatch with a custom origin. - Keep the routine status projection in a database-only helper. Both the routine service and native status committer use it without loading heartbeat execution from the committer. ## Verification - Own-source server TypeScript check passed after the helper extraction. It includes the new test source and production dependencies. Local package exports point at another checkout, so the check used explicit paths to this checkout's source and disabled declaration emit. - Existing pure `status-arbiter.test.ts`: 20 tests passed. `git diff --check` passed. - Initial required CI run 40711603938 built and typechecked successfully. It executed all ten new database tests: nine passed; the public-dispatch fixture failed because it assumed status version zero. The fixture now reads the saved version and decision binding. - That CI run also exposed a heartbeat mock dependency cycle. The database-only helper removes the committer-to-routine-service-to-heartbeat import. Required CI on the new head must confirm both repairs. - Initial browser shard 3/8 failed during cleanup: legacy run cancellation waited for a heartbeat row lock. Its cause is under investigation; the failure remains recorded. - A local runtime-execution check failed during module import from stale package exports, before any tests or database setup ran. It is not passing evidence. No local database, provider or live eval was run. - Latest-head required CI run 40717315212 executed all ten new database tests successfully in server shard 5/12 (2,163 passed; three unrelated skips). The heartbeat runtime suite executed 23 tests successfully in server shard 9/12 (1,704 passed). - Latest-head browser shard 3/8 passed all 33 tests. This does not establish the cause of the preserved earlier cleanup lock timeout. - Required CI run 40717315212 completed successfully on head `2d77f4302b36303741abad07266bbe9e376ba6a5`. All 48 workflow jobs are green after one failed-jobs-only infrastructure retry. The first attempt had seven explicit runner shutdowns and three lost-runner communication failures. The exact chat shard host was independently confirmed as an AWS Spot termination. Original logs and annotations remain retained. - Greptile completed at 5/5 on this exact head, with no outstanding findings. All review threads are resolved. The independent follow-up review found no issues. - No live qualification or corrected eval pass is claimed. The PR remains open and unmerged, with auto-merge off. ## Risks - Matching routine executions perform the existing bounded status synchronization queries inside the status transaction. A database failure rolls back both projections. - Inconsistent invocation or linked-task metadata now prevents synchronization. No unrelated invocation is inferred or repaired. - The existing scheduled repair remains unchanged. This PR does not change accounting, provider behavior or eval budgets. ## Model Used - OpenAI `gpt-6.1-sol`, with reasoning and tool-assisted source inspection and coding. The session did not supply a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes:` / `Closes:` / `Refs:` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal ticket id or instance-derived details - [x] I have run relevant credential-free tests locally and they pass; database coverage is pending CI - [x] I have added or updated tests where applicable - [x] I have checked documentation; the existing routine status contract is unchanged - [x] I have considered and documented risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
5f0da562db |
fix(server): wake chat lifecycle milestones from committed changes (#17606)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Connected chats show safe notices when a run starts, waits, finishes, or fails. > - A recurring scan projected these notices even when no eligible work existed. > - Eligibility depends on committed run state, conversation lineage, and current authority. > - Existing database rows can reconstruct that work after restart. > - This PR wakes a scoped worker from those writes and removes lifecycle polling. > - Native progress events keep their separate existing owner until the next migration. ## Linked Issues or Issue Description **What happened?** The chat maintenance timer repeatedly scanned run lifecycle state. Empty and settled instances still queried the database. A run-status event alone cannot replace the scan: prompt receipts, continuation receipts, identity changes, and restored authority can make a run eligible later. **What did you expect?** Committed changes wake projection. Startup reconstructs eligible work. A completed pass stops until another relevant change arrives. Failed work retains recovery without scanning settled history forever. **Steps to reproduce** Configure an automatic chat endpoint. Finish its work, or leave it unused. Observe recurring lifecycle projection queries from the chat maintenance callback. Stacked on #17592. Related foundation: #15625. This PR owns lifecycle milestones only. ## What Changed - Preserve upstream coalesced Cloud membership writes together with locked membership restoration and its committed chat notification. - Signal milestone eligibility from the real Slack account-link transaction. - Register the lifecycle worker with the existing delivery coordinator. Remove lifecycle projection from the periodic chat callback. Keep native progress in a separately named lane and preserve the explicit combined API. - Reconstruct company/issue scopes from the existing selector. Drain bounded batches with company fairness. Retry failures, retain writes through drain/resume, and disarm settled work. - Signal from supported run, presentation, interaction, receipt, lineage, conversation, recovery, and authority writers. Preserve their transaction ownership, including nested savepoints and uncertain settlement. No new table is added. - Preserve provider authorization, authored-final suppression, responsible-user continuity, deduplication, the 24-hour selection window, and raw-output restrictions. Keep provider interpretation in its owner. - Preserve PostgreSQL timestamp precision in lifecycle pagination. Rejected submillisecond candidates must not repeat the same page and starve later notices. - Document producer coverage and alternate-writer dispositions in `doc/plans/2026-10-25-chat-run-milestone-sleep-audit.md`. ## Verification Published head: `1d056f68c5b1005933b9e12de8745bf1c4cbeeca`, targeting master after #17592 merged at `7d75ae0569e41786640cf09647ae8849411d2f22`. A normal merge integrates the actual landed parent. The entire 77-path milestone patch is binary-identical to the locally verified preparation across that ancestry reconciliation; both unrelated master paths are preserved. The four earlier producer/test integration conflicts retain scoped milestone and voice notices, current onboarding completion admission, and expanded identity query/read-scope coverage. - **330 distinct selected cases pass on the verified unchanged patch:** 305 across 14 complete suites (all five milestone worker suites, run identity, board claim, Cloud provisioning, auth session routes, Cloud actor authentication, stale-lock recovery, GitHub credentials, project Git authorization, and app admission wiring), plus 25 selected real chat producer cases. The remaining 1,590 chat integration cases were skipped, not passed. - The real producer cases cover saved milestone discovery, GitHub and Slack identity changes, account-link rollback/reconfirmation, ordinary ingress reactivation, post-close action/delivery receipt settlement, all ten authority-restoration paths including Cloud synchronization, and changed native review metadata. - Worker coverage verifies outer commit, rollback, uncertain-write recovery, restart, bounded batching/fairness, timestamp precision, retries, drain/resume, and settled SQL quiet. Authorization and existing publication deduplication remain with their owning services. - Server production typechecking, build, and module-boundary checks pass. The tests above were executed separately. Production typechecking excludes `src/__tests__` and does not replace those runs. Each database owner uses its own newly created and app-migrated PostgreSQL fixture. - [Current-parent acceptance and preserved failure ledger](https://github.com/paperclipai/paperclip/blob/1d056f68c5b1005933b9e12de8745bf1c4cbeeca/doc/plans/2026-10-25-chat-run-milestone-sleep-audit.md#current-parent-integration-and-acceptance-october-28) records commands, counts, producer scope, conflict dispositions, and limits. Earlier combined-stack counts and failures keep their original revision scope; local acceptance does not erase old CI failures. - No new migration, queue table, dependency, workflow, or shared scheduler contract is introduced. Parent migration 0397, bounded consent recovery, voice cleanup/retention repairs, and runner checkpoint restoration remain present. - **Exact-head Greptile is 5/5 and all 55 current checks are clean.** No Ponytail review is present, which is an explicitly authorized nonblocker; no failing review is waived. Older reviews and parent checks are not current passes. Actual failing Ponytail findings must be fixed; an absent/stopped/paused turn without a verdict is an authorized nonblocker. Active unfinished Ponytail uses its original request/start timer, with a 25-minute monitor cutoff respecting the user's 30-minute maximum. - No complete local repository matrix, live provider tenant, external alarm, physical host/database suspension, or fleet savings claim is made. Native progress remains a separate following slice. ## Risks - A future eligibility writer must notify the existing projection boundary. Independent SQL writers or another active server process are outside this process-local signal contract. - Authority and lineage remain sensitive. The worker uses the existing selectors and publication policy; it does not grant execution or disclosure authority. - Native progress and other publication maintenance still have recurring work. This PR removes one reason, not the entire chat maintenance timer. - Worker quiet does not prove database suspension, host sleep, or external alarm coverage. Cloud sleep blockers remain unchanged. - Merge eligibility requires passing current-head CI and Greptile 5/5 after this actual-master reconciliation. Subsequent stack slices retain their own gates. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model ID and context-window size are not exposed. Used repository inspection, reasoning, shell execution, PostgreSQL tests, browser E2Es and GitHub tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details available in this session) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run the documented focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
820c458465 |
fix(runner): restore per-turn controller checkpoints before resume (#18018)
## Thinking Path > - Paperclip coordinates agents and preserves their work between turns. > - Remote native sessions save controller checkpoints in the database. > - External persistence leaves a small identity marker on local disk. > - A later `per_turn` continuation must restore that checkpoint before it reads the controller identity. > - The restore path handled warm sessions only, so valid per-turn continuations failed before provider execution. > - This PR restores the referenced checkpoint and retains the existing ownership and lifecycle checks. ## Linked Issues or Issue Description Related: #17439 added external controller persistence and the warm restore path. #18011 repairs warm preparation and fresh bootstrap checks; it does not cover ordinary per-turn continuation. **What happened?** A remote OpenCode run completed with `per_turn` lifecycle and saved its controller checkpoint in the database. Its next turn reused the normalized session. Admission read the remaining external-persistence marker as a full controller file and failed with `runner_state_identity_mismatch: durable_identity_unreadable`. The prior run's complete checkpoint and harness backup still existed. The failure occurred before provider work started. **Expected behavior** Restore the exact prior controller checkpoint before identity inspection. Then apply the existing company, task, session, prior-owner and suspended-state checks before a provider can start. **Steps to reproduce** 1. Complete a remote native OpenCode task with `per_turn` lifecycle and external controller persistence. 2. Send a follow-up that resumes the same saved session without a session reset. 3. Observe admission reject the valid external controller marker before provider execution. **Paperclip version or commit** Observed at `40c1ab36cdac70c85dc8fe5e912431834fdb691b`. The same warm-only restore remains on current master. **Deployment mode** Hosted Linux with Daytona and native OpenCode. The original model was `openrouter/deepseek/deepseek-v4.1-flash-1017`. ## What Changed - Restore external controller state for ordinary remote per-turn continuations through the existing checkpoint loader. - Bind that read to the current company and task, the marker's run, and the expected normalized session. - Keep local, warm and restart-recovery paths, prior-owner checks and harness backup checks intact. - Refresh a prior-run projection from the latest checkpoint after a rejected active-owner attempt. Keep current-run full state and the existing legacy file-backed fallback intact. - Add restoration and denial regressions, execution documentation and session verification recipes. ## Verification - Red: the real controller generated an external marker; a valid per-turn OpenCode continuation failed with the exact `durable_identity_unreadable` error before tool-authority construction. The repair passed the same case and its ownership guards. - Review regression red/green: a real controller advanced its checkpoint after an active-owner rejection. Before the follow-up fix, a later valid continuation retained a null attach seed. After the fix, it restored the newer attach template and prepare command. Legacy full-file roots with no database checkpoint retain their existing behavior. - At current head `1f6589233e6121d082a0c0c4444ea895b6e8cd7c`, 86 focused executor cases and all 14 execution-driver cases pass. Coverage includes checkpoint restoration, direct/backup suspension, Codex, legacy files, warm admission and canonical bootstrap, missing or malformed evidence, wrong bindings, active/foreign owners and unsuspended runners. Tests stop before provider launch. - The real PostgreSQL encrypted-checkpoint integration passes, including normalized-session refusal before write and stale-writer fencing. Independent source review found no actionable safety defect. - `node scripts/check-maps.mjs`, `git diff --check`, full `pnpm -r typecheck` and `pnpm build` pass on current head. - The local full suite did not complete: native PostgreSQL startup failed under the host sandbox, an initial Docker rerun lacked migrations, and the migrated retry crossed the review-fix revision change. Logs are retained; that mixed-revision run was stopped and its owned database removed. Full test evidence comes from current-head CI. - Current-head CI passes: 53 successful checks, two intended Storybook skips, none pending or failing. Greptile is 5/5 on that exact head with no open review threads. The only manual merge conflict with #18011 was in the verification recipes; both recipes and the unchanged per-turn restore helper are preserved. **Live continuation acceptance remains unverified, so this PR stays draft.** Two disposable OpenCode/Daytona attempts never reached turns two and three: - At `72b49208b1`, the first turn executed tools and produced the exact 33-byte workspace note, but exceeded its four-minute settlement deadline. The first completion lacked a registered attachment; the second decorated a typed attachment reference with filename/size/hash text. The strict UUID parser rejected that reference before checking a valid criterion reference. The scoped attachment, asset and publication receipt exist. Replaying the actual guard rejects the original input before a database lookup; correcting only that reference passes against the recovered row and receipt. This did not show a publication-persistence or controller defect. The task remained unsettled. - At `1f6589233e`, a separately approved replacement used a temporary prompt-only fixture change requiring exact `deliverable:<attachmentId>` references, with metadata in `verification.detail`. Production code, grading and deadlines were unchanged; the fixture patch was removed afterward. This attempt had no recorded provider authentication, harness-ready, model, tool or result event. Ingress acquisition took 84.072 seconds; transport connection took 205.835 seconds within the 239.832-second failed native execution. A local settlement GET also timed out after 30 seconds. The underlying startup stall is unresolved; these observations do not establish a remote provider outage. Final recovery was blocked with `ambiguous_remote_recovery_evidence`; no successful settlement or continuation is claimed. Both attempts used a Darwin arm64 source host with Node 26.4.0, isolated PostgreSQL 18, a Linux x64 runner and OpenCode 1.18.34. The representative qualified model was `openrouter/deepseek/deepseek-v4-flash-0731`; the original incident used `openrouter/deepseek/deepseek-v4.1-flash-1017`. The controller defect occurs before provider work. Runtime artifacts came from immutable build `e7ea2e2961`, whose Runner package source is identical to the candidate. The current-source manifest, artifact, distribution, asset and all 110 archive-hash checks passed. Each attempt had $10 company/agent model budgets and zero automatic workflow/task retries. Accounting was incomplete; zero recorded spend is not proof of zero cost. Earlier setup failures are also retained: host PostgreSQL bootstrap denial, cached-pack compatibility admission before provider work, and an [official artifact build](https://github.com/paperclipai/paperclip/actions/runs/40696838122) that stopped during an unrelated Pi npm install at its 300-second bound. No manifest or integrity guard was weakened. Owned local processes and databases are stopped, and the temporary fixture patch is removed. Standard fixture cleanup had left one replacement sandbox started; it was stopped and deleted through the SDK. Every owned sandbox was then verified absent with a final 404 readback. Resource cleanup does not establish successful task finalization. An earlier head retained one [voice-recovery timing failure](https://github.com/paperclipai/paperclip/actions/runs/40695892339/job/122843005098): a deadline test observed two failed-provider inspections instead of one. A single isolated shard rerun passed all 1,579 tests without a source change. Its timing explanation remains an inference. The current reconciled head passed without a retry. ## Risks Restoration writes a local projection before the existing admission checks. The encrypted checkpoint loader validates company/task/run and normalized session before writing. A later ordinary continuation refreshes a prior-run projection so rejected admission cannot freeze an active owner's older checkpoint. Active, foreign or unsuspended prior owners still cannot start a successor. Missing or malformed evidence remains blocked. The change adds no retry, credential, provider binary or lifecycle-policy change; it preserves the existing legacy fallback. This PR does not deploy the repair or recover the original hosted tasks. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use, code execution and collaborating agents. The session exposes GPT-6 as its model version; a more specific model ID and context-window size are unavailable. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
25f1070fbd |
fix(runner): preserve checkpoint proof across warm preparation (#18011)
## Thinking Path > - Paperclip coordinates agent work and preserves its execution state. > - Native Runner checkpoints preserve authority across controller changes. > - Warm sessions also need checkpoint writes while their successor prepares. > - External persistence replaces the local state file with a marker. > - Two existing checks treated those valid states as lost ownership or prior activity. > - This PR uses the existing preparation lease and canonical controller state to keep those checks accurate. > - The result is safe warm-session retirement and fresh-session startup on reused sandboxes. ## Linked Issues or Issue Description Related: #17439 and #17320. The open #17401 changes the recovery welcome window and does not cover these paths. **What happened?** A configuration change could fail before the next provider turn with `runner did not durably suspend before checkpoint`. The task already pointed to its preparing successor, but that successor did not yet have a native coordinator. The old completed runner's maintenance checkpoint was rejected. A separate fresh session on a reused sandbox could fail with `resumed_unavailable`. Checkpoint registration had replaced the full local state file with an external-persistence marker. The bootstrap check still required the old full-file format. Process supervision then reported unconfirmed termination before launch. **Expected behavior** Retire the completed warm runner under the same process's live preparation lease. Admit a fresh remote session only when the registered controller proves exact identity, no prior activity and pending bootstrap commands, and the remote session directory can be claimed atomically. **Steps to reproduce** 1. Complete a native warm turn with external checkpoint persistence. 2. Change configuration so instruction preparation must retire that session, then submit a follow-up. 3. Separately, admit a fresh native session on a reused sandbox lease after external checkpoint registration. **Paperclip version or commit** Reproduced in the checkpoint implementation introduced by #17439. Fixed against current master. **Deployment mode** Source checkout tests and a hosted Docker/Daytona incident. No deployment or original-task retry is part of this PR. ## What Changed - Accept completed-run maintenance under the exact successor's live, same-process preparation lease only before a native coordinator exists. - Preserve original controller generation, accepted-result, no-new-work, company and task checks. - Pass the registered controller through the existing connection callback and read its current canonical state for untouched-session proof. - Keep marker-only, identity, activity, backup and atomic remote-directory guards. - Add regressions and update execution semantics and session verification recipes. ## Verification - Warm retirement red: the real Postgres writer rejected the preparing successor's `runner.suspend` checkpoint with `native_runner_checkpoint_owner_lost`. - Warm retirement green: 14 Postgres ownership cases pass, including stale/foreign/expired owners, conflicting native authority and new provider work. - Bootstrap red: real controller registration and external persistence caused the resumed and replacement positives to fail. Six external negative cases passed. - Bootstrap green: all 34 bootstrap cases, the bootstrap-state predicate and 13 connection-driver cases pass (48 total). - `node scripts/check-maps.mjs` and `git diff --check` pass. - Full `pnpm -r typecheck` and `pnpm build` pass. - Full local `pnpm test:run` remains partial/interrupted. The default attempt could not initialize embedded Postgres on this Mac. A dedicated Postgres 18 attempt ran for 17 minutes without a completed file result before interruption; the generic helper separately confirmed unsupported embedded Postgres (`initdb` failed after five attempts) and ignores the external database URL. The generic probe skipped one test. Setup and interrupted-run logs are retained; neither is claimed as a behavioral red or a full local pass. - Real source-hosted Codex/Daytona acceptance was attempted at `6ce7fcacc4` with `gpt-6.1-sol`, `xhigh`. The actual task stopped before provider work at `runner_ingress_auth_failed`, with zero authenticated connections and zero source events. A separate environment-qualification handshake reached challenge/response; that does not prove task authentication. The three-turn warm reuse, configuration retirement, useful output and reload workflow remains unverified, so this PR stays draft. - The live attempt used the cached Linux amd64 Runner binary `7437a4e32a099180889be534fcfb2a086046514490c1448aba46c55848983ed8` from the immutable Daytona image. The current staging binary could not be copied because SSH timed out, and the configured current EXE image returned registry 403. No compatible current local or CI-downloadable artifact was available. Earlier setup failures and cleanup receipts are retained; no provider success is claimed. Disposable test resources were cleaned up. - Exact-head Greptile completed at 5/5 with no open feedback: https://github.com/paperclipai/paperclip/pull/18011#issuecomment-6469515952. All current-head CI checks pass: 53 successful checks and two intended Storybook skips. ## Risks The preparation fallback affects maintenance checkpoints only after an accepted result. An existing native coordinator prevents that fallback. Stale controller generations, expired or foreign preparation leases, wrong tasks and new provider work remain denied. The bootstrap check retains the live controller object and reads current state each time. It does not trust a marker or restore local state beneath a live controller. Pending turns, prior events/connections, backups and existing remote directories retain recovery gates. No schema, provider binary, credential, permission or lifecycle-policy change is required. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use, code execution and collaborating agents. This session does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
356c7de9b7 |
fix(runner): bind Grok packs to current admission source (#17972)
## Thinking Path > - Paperclip manages agents and their work. > - Native agents use verified Runner and provider-pack artifacts. > - A protected installed pack can be reused when its selected provider profile matches. > - Grok kept profile 1 when its sidecar gained effort, search, and startup diagnostics. > - An older Grok sidecar can therefore pass compatibility and reject the new session fields before provider startup. > - This PR binds the current Grok profile to its admission source so a fresh run selects the current verified pack. ## Linked Issues or Issue Description Refs: #17922, #17942, #17962. Related installation contract: #14024. No duplicate repair was found. **What happened?** The selected Grok profile digest stayed unchanged across native session contract changes. The installed-pack verifier correctly compared that digest, but the unchanged value let a pre-effort sidecar satisfy a current controller. That old parser rejects `grokEffort` and `grokDisableWebSearch` before the new startup diagnostics run. **Expected behavior** A fresh Grok run must use a pack that supports its current session contract. Source-only app changes must still permit warm reuse when the selected Grok profile matches. Reattachment must keep its exact admitted artifact authority. **Steps to reproduce** 1. Retain a protected pack with published Grok profile 1 and ACPX interface 1. 2. Upgrade the controller to the native effort and search contract. 3. Start Grok 4.7 with High effort and disabled web search. 4. The old profile passes installed-pack compatibility, but its sidecar rejects the new fields. **Paperclip version or commit** Reproduced from master at `09ad9adce2cf4742944353a3707a2f908e9bb35e`. ## What Changed - Advance Grok to profile 7. Keep historical versions 1–6 decodable and keep the Grok 1.0.13 runtime pin. - Stamp the Grok admission source identity into a generated launcher comment. Keep the existing physical command-byte digest check unchanged. Bind effort/search/credential handling, sidecar admission, and Rust backend/session/diagnostic source. - Check the source attestation in the existing TypeScript/Rust profile build gate. Keep its required fixture in the Docker context. Refuse to overwrite a versioned identity after source changes. - Add regression coverage for rejecting the old pack, current-profile warm reuse, exact historical recovery authority, and actual sidecar dispatch of Grok 4.7 High with both search policies. - Re-attest Copilot as profile 71 because its bound shared recovery decoder now retains Grok 6. Preserve Copilot 70 and all earlier identities. Keep the compatibility algorithm, other provider identities, dependencies, lockfile, and workflows unchanged. ## Verification - The production installed-pack regression failed before the profile update: the old Grok pack was accepted. It now passes. - 105 server manifest, artifact, archive, and authority tests passed. - 246 Runner profile, physical launcher integrity, Grok effort/search/credential, and actual sidecar process tests passed; six existing platform cases were skipped. - 31 Node packaging and identity tests passed, including an actual generator CLI regression that preserves all authority files when a versioned identity would change. - 55 Rust ACPX unit tests and 17 real sidecar pipe tests passed. After the final profile declaration, all 55 unit checks passed again. - Final `pnpm -r typecheck` and `pnpm build` passed at the final source. - CI on the preceding head caught Copilot’s stale attestation for the shared recovery decoder. The same current-source test failed locally, then passed after profile 71 was generated with version 70 retained. All 106 affected Copilot/profile checks pass. Final head `78e6d76fc52f492371e15574482b3641735a6d89` passed the full CI workflow; the failed prior job log is retained. - Review follow-up: mutating the Rust backend’s actual Grok descriptor field left the prior identity unchanged. This regression failed before binding `acpx_provider_backend.rs` and passes afterward. Profile 7 retains the prior versioned fixture and recovery decoder. All 179 affected Runner checks and 75 focused server pack/manifest checks passed after this change. - `pnpm test:run` hit the recorded embedded PostgreSQL bootstrap failure after five attempts. The full run also exposed a current-release fixture with the historical Grok digest; it now reads the current profile. Its 11 focused manifest tests now pass. The failed local run log is retained; final exact-head CI passed. - The same filtered-context Docker build that failed CI now passes locally after adding its required source fixture. - A separate probe used the built sidecar in a private HOME with no credentials. It admitted the current profile and the new Grok fields, then returned the closed working-directory rejection before provider launch or inference. - A historical parser probe reproduced the pre-effort sidecar rejection. These probes establish the contract boundary; they do not establish native subscription task acceptance. - Initial restricted local ownership tests failed because loopback listeners were denied. The same tests passed with loopback access. One new dispatcher fixture first lacked required instructions; the completed request passed. These setup failures are retained in local evidence. - Live native task acceptance remains pending deployment. Earlier deployed attempts failed before tool output. No successful native shell proof is claimed. - Final exact-head merge gate: all 62 contexts are clean (57 success, four intentional skips, one informational neutral formal review). Greptile and Paperclip reviews are both 5/5, with no unresolved actionable threads. The formal review uses the existing report-only neutral policy. ## Risks - The first fresh Grok run after upgrade may need to stage the current provider pack. Matching current profiles keep their existing reuse path. - Fresh admission rejects older Grok profile identities. Historical decoding and exact reattachment authority remain separate; this change does not reinterpret an old identity or replace an active session's pack. - Future changes to bound source require a new Grok profile attestation. The build gate fails if the checked-in identity is stale. - Model, effort, search, credential, approval, and retry policy are unchanged. ## Model Used - OpenAI Codex, based on GPT-6, with reasoning, tool use, and code execution. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused checks; full-suite database failure is retained above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
6cc765bd81 |
feat(live): deliver recoverable transcripts to interested views (#17456)
## Thinking Path > - Paperclip should show live agent output without repeating presentation work for every view. > - Execution, acknowledgements and durable checkpoints must remain independent of browser interest. > - The server already owns authorization and redaction at delivery. > - A shared browser connection can declare which visible runs need full transcripts. > - Catch-up must handle subscription races, disconnects, terminal output and slow consumers. > - This PR delivers the backend and UI contract together without a database migration. ## Linked Issues or Issue Description **What existing behavior does this improve?** Visible transcript consumers receive recoverable output through one shared browser connection. **Current behavior** Independent views can create duplicate connections, presentation work and recurring history reads. A subscription race or filtered native history can lose the saved tail. **Proposed behavior** Negotiate run interests, subscribe before catch-up and retain independent canonical recovery cursors. Keep lightweight lifecycle and attention events available without full transcript delivery. PR 4 of #17097 → #17439 → #17455 → #17456. The runtime feature is independent of hot-upgrade recovery; the stack order is for review convenience. ## What Changed - Add additive WebSocket capability negotiation, replacement interests, revision acknowledgement and resynchronization notices. - Reference-count visible consumer interests on one company-scoped browser connection. Preserve older-client/server behavior. - Reuse canonical structural/current-user/registered-secret redaction for plugin run notifications through the same bounded delivery queue. Keep producer and uninterested streams free of recipient lookups. - Refresh application queries after the matching recovery acknowledgement, including one follow-up after an older in-flight read. Ordinary interest acknowledgements add no refresh. - Keep authorization and redaction at delivery. Browser interest cannot change harness execution, PRP ACKs, checkpoints, accounting or canonical logs. - Deduplicate by sequence and log identity. Catch up terminal output and saved tails without recurring reads during healthy negotiated streaming. - Bound delivery to 1,000 items and 1 MiB. Overflow requires resynchronization; persistent congestion closes the connection. - Preserve compact-history expansion from current master across paginated catch-up. Snapshot the original read window before React can defer its update. ## Verification On current source `f6710f0b3c`, Linux [job 122604880503](https://github.com/paperclipai/paperclip/actions/runs/40624101891/job/122604880503) executes all five run-read permission cases successfully (2.574 seconds), including grant and revocation on the same connection. Its shard passes all 2,430 cases. Current [server-shard job 122604880416](https://github.com/paperclipai/paperclip/actions/runs/40624101891/job/122604880416) has 1,513 passes, one failure and three skips: an unchanged nested-clone expiration/restore test exceeds its 15-second deadline. The preceding `5efc6c48bf` source has identical non-document bytes; [its corresponding job 122601359272](https://github.com/paperclipai/paperclip/actions/runs/40623017917/job/122601359272) passes all 21 nested-clone cases, including that case in 9.505 seconds. The exact timing cause is unproven. The single diagnostic [retry job 122609332597](https://github.com/paperclipai/paperclip/actions/runs/40624101891/job/122609332597) passes all 21 nested-clone cases (61.316 seconds), including the formerly timed-out case in 7.003 seconds, and all 1,514 shard cases (three skips). The failure remains recorded; no timeout or assertion was changed. After the server squash merge and qualification retarget, all 50 focused server cases and all 91 focused UI cases pass again, with both typechecks and token gates. Runner vendor preparation rebuilds current master inputs before those tests. This source needs fresh CI and reviews; prior CI and browser receipts do not qualify it by inference. Current candidate `f6710f0b3c` now targets master `6987c2f20b`, after merged #17097, #17439 and #17455. Its diff contains only transcript delivery and its documentation. It adds no database migration. Retargeting changes only the evidence document; all production and test code is byte-identical to `5efc6c48bf`. On the eighth integration, all 50 focused server delivery cases and all 91 focused UI cases pass, with both typechecks and token gates. The following 35-case receipt belongs to the earlier delivery correction. Four low-queue authorization/decoration stall variants and the late-rejection case fail before correction. They now verify the five-second read deadline, old and negotiated clients, revisions that cannot reset the deadline, unaffected sibling connections, rejected late results, and timer cleanup. The first combined toast run has four loopback EPERM failures and 31 passes; its permitted repeat passes all 35. Earlier 28-case evidence excluded the toast file. The permission correction requires the current HTTP company-scope transcript permission, plus task access, for both native events and legacy logs. Prior Linux source `49a8783e9b`, job 122549913643, executes all five run-read cases, including grant/revocation on the same connection, successfully. That source placed this file in server shard 11; later shard assignments can differ and must be checked against their actual inventories. The older `405bedd189` receipt retains a separate scheduled-retry teardown race and inherited fixture collision; neither is waived. The new head must pass again in Linux CI. The seven extracted stream-recovery tests preserve their assertions. All 39 related legacy/native cases, UI typecheck and token gates pass after restoring the initially missed fixture helper. Earlier full workspace build/typecheck and Chrome journeys at `445a76563c` cover two views, company switches, reconnect, terminal catch-up, reload and zero recurring healthy transcript reads. Those browser receipts are historical source evidence and do not qualify this new head by inference. The [qualification record](https://github.com/paperclipai/paperclip/blob/f6710f0b3c84c6fdb2021df67206261b446dfc97/doc/plans/2026-10-22-hot-upgrade-qualification.md) retains exact earlier UI/history, source-boundary and cleanup failures with their fixes and limits. The workload-cost test preserves bounds and authorization callback counts for zero interests, one viewer, multiple browsers and a slow viewer. Its socket congestion is simulated. It measures neither SQL nor network costs. Current permission checks and the final hosted active-agent/viewer matrix remain unmeasured; no production savings are claimed. Fresh exact-head CI, legacy status, completed Greptile 5/5, resolved feedback and ordinary merge eligibility remain required. The earlier recipient corrections pass all 47 registry/plugin/delivery/WebSocket/toast cases and all 174 UI provider/recovery/interest cases, both typechecks and token gates on source `b202e59a25`. A real bus/host-service test registers a synthetic run secret through the production registry and reproduces its raw notification before the fix; native/legacy/final notifications now redact it, disposal prevents late worker notification. A later P1 review reproduces lost final notifications on settings-read and secret-resolution failure. Both cases fail before correction. Any delivery failure or subscription timeout now sends one content-free stream error and removes the plugin subscription. It never changes the agent execution or run status. Later events start no new redaction reads. The final integrated six-file server lane passes all 50 cases, with server typecheck and token gates. Existing browser delivery behavior is unchanged. Storage and secret-provider seams are scripted. Two real React Query/provider regressions first miss a run during resync; both now refresh after the valid replacement ACK, including pending-read and stale-scope cases. The prerequisite qualification batch passes all 118 cases and E2E typecheck after master integration. Fresh CI, security scan and reviews remain pending on this head. The contribution-rule review correction removes 104 explanatory comments introduced by this slice and keeps the ordering, deadline and plugin-delivery explanations in the delivery document. Existing comments and functional test/lint directives remain. All edited source files have identical parsed non-comment tokens before and after the change. All 50 server delivery cases and all 91 cases in six affected UI suites pass, along with token gates. The new head requires fresh CI, security and review. ## Risks A slow or disconnected viewer must resynchronize from authorized canonical history. It may see a display gap until catch-up completes. Company changes and revoked access must remove delivery interests. This slice adds no database migration and cannot change execution or recovery authority. Full transcript delivery evaluates current company-scope permission per delivery. Active plugin recipients also perform canonical redaction reads; viewer-load database costs on this final source remain unmeasured. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools and code execution. The runtime does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
09ad9adce2 |
feat(server): adopt running remote agents through durable controller checkpoints (#17439)
## Thinking Path > - Paperclip must keep admitted remote work alive while its main server is replaced. > - Runner owns recovery mechanics and exposes opaque checkpoints through #17097. > - The server owns company scope, encryption, Stop, tool authority and execution ownership. > - A successor must prove current ownership before it adopts the original execution. > - Checkpoint bursts must also leave capacity for normal requests and lease renewal. > - This PR persists and adopts that authority through the existing runtime boundaries. ## Linked Issues or Issue Description **What existing behavior does this improve?** Remote agents retain the original execution during a bounded main-server outage. **Current behavior** A server restart can lose the controller's local state and connection. An unsafe replacement could repeat a turn or an uncertain mutation. Checkpoint writes in the request pool can also delay ordinary requests during an aligned burst. **Proposed behavior** Restore encrypted control records from Postgres and adopt the original runner under a fenced lease. Recover known tool receipts through current tool authority. Give control checkpoints a shared pool capped at two connections per database owner. Depends on #17097. Successors are #17455 and #17456. Qualification and presentation remain separate review slices. ## What Changed - Add company/run-bound encrypted checkpoint records and migration 0394. Keep the SQL idempotent after renumbering against current master. - Recover startup and expired ownership through native runtime and heartbeat. Preserve the sandbox lease, admitted runner artifact, harness configuration, native session and active turn. - Keep Stop, finalization, preparation ownership and stale-writer fencing transactional. Preserve graceful detach and natural crash takeover. - Recover committed tool receipts through the existing authority. Reauthorize reads and hold uncertain mutations. - Bound startup retention with the existing coordinator and preserve zero idle scans. - Share at most two checkpoint connections per database client. Keep application and liveness capacity separate. Close the checkpoint pool during shutdown; late writes cannot reopen it. - Count SQL across both application and checkpoint clients. Prove request/liveness progress, command recovery and failure after pool shutdown with real Postgres. ## Verification Project creation recovery now uses the existing project activity receipt owner. It retains authenticated input normalization, fingerprint checks and current run authority after the receipt lock. An absent receipt returns no result before mutation. The new lost-HTTP-response and no-effect regression uses real API routes and Postgres. The seven transport cases pass locally. This Mac skips all 20 database cases. On current source `5de803c2cd`, Linux [job 122593415890](https://github.com/paperclipai/paperclip/actions/runs/40620576702/job/122593415890) executes all 20 project-tool cases successfully (9.480 seconds), including the committed-project lost-response regression. Its shard passes 1,627 cases with three unrelated skips. Server typecheck passes. The transport regression initially has two failures; a subsequent incorrect expectation about schema defaults was corrected before the final pass. Ordinary execution adds no checkpoint writes or recurring work. Current candidate `5de803c2cd` includes the project-creation receipt recovery fix and master `2cd747d1b6`. Migration 0394 retains the prior idempotent checkpoint DDL. Every master SQL file and journal entry is preserved. Snapshots 0390–0394, migration checks and server typecheck pass. The ordinary warm-turn regression fails on its second turn before `a219ab51ad`. After correction, three completed warm turns select the correct prior checkpoint and preserve the original session. All 288 provider-wiring cases pass. The control plane, external persistence callback, identity rotation and filesystem are real; the backend and database-load seam are scripted. The 13 warm admission/configuration variants also pass after final master integration. This does not qualify an underlying harness. At current source `5de803c2cd`, Linux [job 122593415907](https://github.com/paperclipai/paperclip/actions/runs/40620576702/job/122593415907) passes all 940 native-session-executor cases (61.431 seconds), including the warm external-checkpoint behavior; the shard passes 2,358 cases with one skip. Current [Runner job 122593415475](https://github.com/paperclipai/paperclip/actions/runs/40620576702/job/122593415475) passes the native recovery lane: 115 passes and five container-only skips, including the encrypted restore guard, fresh preparation (63.189 seconds), blocked claim lock (1.226 seconds) and recovery beyond 101 blocked owners (3.897 seconds). The first lane passes 2,215 cases with one skip. The SQL cost test reports zero idle and streaming SQL for both output volumes; this is component evidence, not a production capacity claim. At prior candidate `1ff7bb44f4`, Linux job 122549307215 passes all 938 native-executor cases, including the three-turn external-checkpoint regression. Job 122549306469 passes the native recovery lane with 115 passes and five container-only skips, including the encrypted-restore session guard, fresh preparation (63.664 seconds), claim-lock contention (3.961 seconds), and 101-owner recovery (3.824 seconds). This Mac skips the real-Postgres suite. The earlier full local executor attempt has 921 passes and 16 failures during embedded-Postgres initialization before assertions. The original fixture collisions and their corrections remain recorded. At earlier candidate `8fbe62b41d`, server shard job 122535673841 has 3,876 passes, one failure and one skip in an unchanged expiry-worker retry/drain timing test. The same server/database tree on qualification source `b64cac6951` passes all 20 expiry cases and all 3,877 shard cases. The single diagnostic retry, job 122541891915, passes all 20 expiry cases and all 3,877 shard cases (one skip). No assertion was weakened. The original failure remains recorded and its exact timing race is not proven. All Actions checks and legacy status passed on `8fbe62b41d`, with completed Greptile 5/5 and no open feedback. The next candidate `1ff7bb44f4` also passed all Actions checks and Greptile 5/5. Its Paperclip Review remained queued before the new master conflict required integration. The new candidate needs fresh exact-head CI and reviews; old checks do not qualify it. Migration generation is checkpoint-only, all master SQL and journal entries remain unchanged, and migration checks, server typecheck and all 13 warm-owner variants pass after integration. The first migration check was blocked by TSX sandbox IPC; its permitted repeat passes. The first warm-test command used an incorrect path and ran no tests; the corrected command supplies the 13-case receipt. All 16 database pool-lifetime/options/teardown cases pass after granting loopback permission; the first batch had two EPERM failures and 14 passes. Earlier semantic-tool/HTTP recovery, adoption, encryption-batch, zero-streaming-write and idle-SQL receipts remain source-attributed in the [qualification record](https://github.com/paperclipai/paperclip/blob/2ff66f57da0da35bac1c19a2366ea4b84d87a986/doc/plans/2026-10-22-hot-upgrade-qualification.md). That record retains earlier assertion, setup and cleanup failures and their corrections. No failed gate is waived. The historical 300-session component campaign passes four after phases: steady read p99 is 107–108 ms at 500 reads/s and 100 writes/s; aligned-burst read p99 is 103–105 ms after the bounded pool, versus 584–689 ms before. This comparison isolates the persistence corrections. It does not measure the final stack's hosted active-agent capacity. The hosted zero-agent API pair leaves SQL and CPU essentially unchanged, but both arms fail absolute latency gates and candidate peak cgroup memory fails its gate. The active-agent/viewer matrix, different-artifact upgrade and rollback remain open. Run `PAPERCLIP_TEST_DATABASE_URL=<owned-disposable-postgres> pnpm --filter @paperclipai/server exec vitest run src/services/native-runtime/native-runner-restart-recovery.integration.test.ts`. Previous head `f2db8a6568` had one failed workspace setup gate: [job 122555672441](https://github.com/paperclipai/paperclip/actions/runs/40608984238/job/122555672441) passes all 4,169 UI tests and 285 CLI cases (one skip), but the company import/export suite reaches its 30-second health deadline during doctor startup. Its output does not reach the server-start step. The CLI, server, database, shared, adapter and plugin trees are identical on qualification source `a866619d7b`, whose [job 122555886663](https://github.com/paperclipai/paperclip/actions/runs/40609057937/job/122555886663) passes that fixture and all 286 CLI cases. The one diagnostic retry, [job 122561057225](https://github.com/paperclipai/paperclip/actions/runs/40608984238/job/122561057225), passes the formerly failing fixture and all 286 CLI cases. All Actions checks and legacy status on `f2db8a6568` pass. Its exact-head Greptile completed at 5/5 with no open feedback. The current integrated head requires fresh checks and a completed Paperclip Review before merge. The original failure remains recorded; its exact timing cause is unproven. A repeated failure requires investigation and repair rather than another retry. All current-head CI, legacy status, exact-head completed Greptile 5/5, resolved feedback and ordinary GitHub merge eligibility remain required before merge. The latest integration preserves master’s published Copilot v70 profiles and every published migration. Checkpoint migration `0394_conscious_whizzer.sql` was generated with `pnpm db:generate` and retains the prior idempotent DDL. The Runner vendor rebuild and replay parity checks pass. Server typecheck and all 13 warm-owner variants pass after that rebuild. The imported report-lock patch is byte-identical to master; its intentional patch-context whitespace is retained, and the feature delta passes whitespace checks. No hosted experiment was run. ## Risks The pool adds up to two active connections per instance, shared across agents. Checkpoint delivery can wait behind that bound; it still requires its commit. Steady component-test Postgres CPU rises by about 1.7 percentage points of a two-CPU allocation. These corrections do not remove the stack's control and renewal writes. The migration and key must survive replacement. Fresh-database tests and local load do not establish different-artifact hosted upgrade or rollback continuity. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools and code execution. The runtime does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
39e811c9c5 |
fix(runner): prepare diagnostic proxy and reduce bundled skill queries (#17941)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native turns reconcile company skills and select verified remote runtime packs. > - Catalog imports repeat folder transactions for skills in the same bundled category. > - Diagnostic requests can also reuse an older image proxy that lacks the phase collector. > - This pull request reuses categories within one import and selects the current proxy for fresh diagnostics. > - Each import has its own category map, and recovery keeps its admitted runtime identity. > - The result is less preparation work and a runtime that can provide the requested measurements. ## Linked Issues or Issue Description Refs #17918. **What happened?** A hosted run retained `PAPERCLIP_OPENCODE_PERFORMANCE_DIAGNOSTICS=1` but emitted no provider phase batch. Its saved pack authority selected an older image proxy. The deployed server proxy contained the collector. The image proxy did not. Bundled inventory reconciliation also repeats folder transactions for skills in the same category. The paired local benchmark exposes that preparation cost. **Expected behavior** A fresh diagnostic request must use a verified proxy that supports the current collector. Bundled skill reconciliation should resolve each category once per import. **Steps to reproduce** 1. Use a protected image pack with the same OpenCode interface revision and an older proxy hash. 2. Enable `PAPERCLIP_OPENCODE_PERFORMANCE_DIAGNOSTICS=1` for a fresh remote run. 3. Observe that the current backend accepts the older proxy and cannot emit the requested phase batch. ## What Changed - Reuse each bundled category during one import. Every skill still runs its existing validation, access lookup, value comparison, and write path. - Compare the installed proxy hash with the current build when a fresh run requests diagnostics. - Keep compatible-pack reuse for ordinary runs and exact admitted identity for recovery. - Cover selection through the real backend, including recovery after diagnostics are enabled. - Document runtime preparation costs and admission limits for diagnostic requests. The skill importer owns category reuse. The provider verifier owns the artifact comparison. The native backend passes the resolved diagnostic requirement at the existing admission seam. The changes stay in those existing owners and share one latency investigation and rollout. ## Verification - Before the fix, the new verifier and backend cases failed because both accepted a stale proxy. - After the fix, 63 focused cases passed in 8.10 seconds. They cover normal reuse, current diagnostic proxy selection, stale proxy rejection, selected ACPX profiles, and admitted recovery identity. - 110 skill and folder service cases passed against isolated real PostgreSQL databases. Only cluster bootstrap used the existing local PostgreSQL service. Queries, migrations, authorization, file handling, and transactions ran on their real paths. - A paired benchmark used the same immutable skill snapshot, authenticated actor, 170-skill inventory, and 3.5 ms delay in each network direction. It checked every delivered file and executable bit. Warm median fell from 3.319 s to 2.482 s (25.2%). SQL calls fell by 81 in each matched sample, from 241–243 to 160–162. The bundled phase median fell from 2.596 s to 1.721 s. The final measurement uses commit `37eac8d8de`; only the skill importer differs from baseline `7951455073`. These local results are not hosted acceptance. - `pnpm -r --workspace-concurrency=1 typecheck` passed with the pinned pnpm 9.15.4 and a 6 GiB Node heap. Earlier attempts exposed a subprocess toolchain mismatch, a local IPC restriction, and a fixture type error. Each was corrected before the passing run. - Module boundary and whitespace checks passed. - `pnpm build` passed for all workspaces. - `pnpm test:run` did not complete. Embedded PostgreSQL initialization failed after repeated startup attempts. The owned test subtree was stopped after preserving the failure log. This is not a full local test pass. On final commit `37eac8d8de`, all 53 successful check contexts passed; two optional Storybook checks were skipped. Greptile is 5/5 with no unresolved findings. The draft awaits merge approval. - The catalog change has a measured local gain. The proxy change fixes missing phase capture. Real hosted response latency remains above the target and needs a new matched measurement after delivery. ## Risks The category map belongs to one import and one company. The importer still resolves current skill data on every invocation. Fresh diagnostic runs can prepare a current verified pack, which can add startup time. Admission can fail when matching runtime assets are unavailable. Ordinary runs keep the existing fast compatibility path. Recovery keeps the already admitted identity, even if diagnostics are enabled later. The operator endpoint gate, collector limits, and closed span fields are unchanged. ## Model Used OpenAI GPT-6 through Codex, with repository inspection, shell execution, browser tools, and Honeycomb queries. The session does not expose a verifiable backend deployment ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d40f9ac377 |
fix: enforce first-task completion decisions and retain E2E evidence (#17940)
## Thinking Path > - Paperclip lets people delegate work to agents and approve proposed work. > - The onboarding first task asks for scope, proposes a task, and waits for a human decision. > - An answer to its opening question supplies scope. It does not approve a proposal. > - Task creation already checks the saved proposal decision. > - Native completion used the general completion policy and could mark the onboarding parent Done without that decision. > - This PR checks the trusted decision before completion feedback and again when the native status is committed. > - The first task stays in review until a saved human decision permits closure. Ordinary completion and acknowledged refusal keep their current behavior. > - The Product E2E guard also retains closed scalar evidence so future failures show which boundary condition was observed. ## Linked Issues or Issue Description Refs: #17914. Profile compatibility follow-up to #17922. **What happened?** A native onboarding run can submit a valid low-risk Done result after only the opening answer. The native finalizer then marks the first task Done even though no human has accepted or rejected a proposal. **Expected behavior** The opening answer must not permit completion. A trusted saved proposal decision must permit it. A saved rejection permits acknowledgment and closure, but does not authorize new work. A later pending proposal must keep the task in review. **Steps to reproduce** Create a server-owned onboarding first task with an answered opening question, no proposal decision, and a valid low-risk completion contract. Pass a Done result through native completion feedback, the control-plane port, and the finalizer. Before this fix, the persisted status is Done. The regression expects In review. Ordinary completion and acknowledged rejection pass as controls. ## What Changed - Share the trusted latest-proposal and human-response checks with completion admission. Keep task creation acceptance-only. - Refuse premature native completion feedback. Keep an already saved premature result in review during finalization. - Check the decision again under the existing task and run locks before committing Done. - Cover accepted decisions, saved refusals, forged context, pending or changed proposals, and a real database race. - Retain a closed scalar receipt when the existing E2E pre-acceptance guard fails. Validate and display it without task content, IDs, filenames, or raw rows. Preserve the original failure, blocked dependents, and historical reports. - Include the evidence module in the journey definition hash. Bind the receipt to its exact failure code and check its closed keys against the full type. - Repair base-branch profile drift found by the full harness check: include the active Copilot v66 fixture in image hashing and update stale Pi version/hash expectations from #17922. Keep all older image inputs and strict hash checks. - Update the onboarding feature and agent access maps and E2E evidence documentation. ## Verification - Semantic regression before implementation: one failure and two passing controls on PostgreSQL 18. The failed assertion observed Done instead of In review. - The local database uses an owned disposable container with an immutable image digest. Actual migrations, feedback, control-plane port, finalizer, and status commit run against PostgreSQL. A temporary test adapter replaces only the embedded database transport because this host has exhausted its PostgreSQL semaphores. - Native control-plane suite: 55 tests passed, including 17 new controls. Existing task-creation admission: 25 passed. Existing completion feedback: 19 passed. - Credential-free E2E support: 206 Vitest suites passed, 4,407 tests passed, and two skipped. All 137 Node support checks passed. Harness typecheck passed. - The failure-receipt regression had 11 failed assertions before capture was implemented. Two failure-code binding assertions fail without their gate. Eight placement/binding cases first validate the receipt-free control, then reject the misplaced receipt. - The untouched base reproduced the three profile compatibility failures. The narrow repair passed all 83 affected tests before the full harness run. - Private workflow preparation and credential-free onboarding preview: nine compatibility checks passed. Ownership, recovery, evidence, shared-contract and workflow compatibility: 77 passed. The actual provider preview also confirms immediate success deletion, earlier declared failure eligibility, limits, and zero retries without credential access. Map checks and git diff checks passed. - Required repository typecheck passed for the product fix. The combined branch build passed. The local full repository suite was stopped after repeated PostgreSQL startup failures; a separate initdb probe confirmed host SysV semaphore exhaustion. No shared IPC resources were changed. [Hosted CI](https://github.com/paperclipai/paperclip/actions/runs/40591020873) passed the full database suites, typecheck, build, runner checks, serialized suites, and browser shards on final head `1ac4c23605a03bfd1f1f8a017b14ce29fda85f6b` (55 success, two expected skips, one advisory neutral). - Independent source review approved the product guard, atomic race regression, evidence contract, and maps with no actionable findings. Both automated reviews are 5/5 on final head `1ac4c23605a03bfd1f1f8a017b14ce29fda85f6b`; prior findings are fixed and resolved. Final CI is green and the PR is conflict-free. - No Cloud workspace, provider call, release publication, or deployment is part of this verification. These tests do not identify a historical Cloud failure or qualify live onboarding. - Local dependency installation uses the existing CI-generated lockfile from #17811. No lockfile change is included. ## Risks - Existing onboarding runs without a trusted saved proposal decision remain in review even when the agent reports success. This is the intended approval boundary. - Decision changes can race completion. The status commit checks the decision under its existing locks and uses the existing status-race recovery path. - A saved refusal permits closure only. Creation admission still rejects it. - Evidence readers reject malformed receipts and unrelated failure codes. The version-1 key registry is checked against every receipt field at compile time. Existing attempts do not gain evidence after the fact. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`), ultra reasoning, through Codex. Used repository inspection, code execution, and collaborating coding agents. The runtime did not report a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes:` / `Closes:` / `Refs:` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal ticket or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
dc8cbdb819 |
perf(server): narrow native session resume database reads (#17934)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/94/changes/paperclip/94 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip manages AI agents and their work. > - Native agents resume prior provider sessions from saved run checkpoints. > - Resume selection must respect every newer progress barrier. > - The prior query returned large context and profile JSON for up to 32 candidates. > - Most candidates need only process, status, binding, and checkpoint-presence checks. > - This PR selects the same source in one SQL snapshot and returns only its required payloads. > - This reduces database response bytes while preserving session authority and saved task context. ## Linked Issues or Issue Description **What existing behavior does this improve?** Native task/session resume selection in `findNativeSessionResumeRun`. **Current behavior** At baseline `bf3ef12f3a78d0b15a32ad57f7e16520fbf6cd5e`, a resume lookup reads complete context and profile JSON for up to 32 runs, then reads provider-event evidence. A synthetic 15-run history returns about 4.44 MB even though one checkpoint is selected. **Proposed behavior** A bounded CTE finds the first checkpoint or progress barrier. The same SQL statement returns the selected source's native input, checkpoint, tool fingerprint, wake envelope, canonical task brief, and process identity. All fallback barriers and the 32-run limit stay in place. **Reason and benefit** Rejected candidates no longer transfer saved prompts and execution inputs to Node. A single snapshot also prevents a gap between candidate selection and payload hydration. **Breaking changes** None to public APIs or stored history. This private service returns the fields consumed by runtime selection and checkpoint rebinding. It keeps complete required nested values and distinguishes missing brief fields from explicit JSON nulls. **Additional context** Checked `ROADMAP.md`. This is a targeted improvement to existing resume behavior. Public searches for native session resume projections found no duplicate. Related work includes #6573 (durable agent sessions) and #17011 (managed Claude resume opt-in); neither changes this database read. Provider-refusal recovery in #17676 is a separate concern and is unchanged. ## What Changed - Select the newest progress barrier within the existing exact company/agent/task/session and 32-run window in one SQL snapshot. - Return three required profile fields, the full wake envelope, and the saved task's ID, title, and description only for the selected source. - Preserve invalid non-null checkpoint barriers, provider evidence, process authority, terminal-status rules, timestamp precision, and exact string binding checks. - Add real PostgreSQL regression coverage and a reproducible baseline/treatment benchmark. Document the read and benchmark command. ## Verification - `pnpm -r typecheck` passed. `pnpm build` passed. The final server `tsc --noEmit` also passed. The later benchmark-only relocation passed `pnpm check:module-boundaries` and the real PostgreSQL proof; runtime and regression-test source are unchanged. - Related suites: `native-session-resume-projection.test.ts`, `native-session-resume.test.ts`, `heartbeat/retries.test.ts`, and `heartbeat-retry-scheduling.test.ts`: 78 passed and 54 skipped across 3 passed files and 1 skipped file. Embedded PostgreSQL/platform gates caused the skips. Resume orchestration and the 10 new SQL tests ran against isolated PostgreSQL 17.11. The final 10-test projection rerun passed. - New SQL cases cover all seven provider evidence kinds, each process authority field, all terminal statuses, wrong bindings and runtime, invalid checkpoints, null/scalar/array/object JSON, missing versus null brief fields, the 32-row bound, ordering ties, future barriers, submillisecond timestamps, and a change committed before the one selection statement. - `pnpm test:run` was attempted. The host's embedded PostgreSQL startup failed after five attempts. The broad run recorded failures/skips in embedded-database suites and was stopped after that concrete host limitation (exit 143). It is not a local full-suite pass. The full final-head CI matrix passes on `dcde4e3dac2173321d7be2406e7775af8411826a`: 55 successful checks, one neutral check, and two optional Storybook skips. Greptile completed at 5/5 on that exact head, with zero unresolved findings. The initial benchmark-location policy failure was fixed before this final run. - Reproduce the benchmark with `pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-native-session-resume-reads.ts --baseline-ref=bf3ef12f3a78d0b15a32ad57f7e16520fbf6cd5e`. The optional `PAPERCLIP_SESSION_RESUME_BENCHMARK_DATABASE_URL` must name an isolated database ending in `_test`. - The harness imports the pinned baseline module, invokes both actual services, captures complete generated SQL and parameters, compares every retained consumer field, and captures `EXPLAIN (ANALYZE, BUFFERS, FORMAT JSON)`. Its incompressible wide fixture keeps a complete 96 KiB native input in the selected result. Raw PostgreSQL text DataRow frames are measured exactly; RowDescription, protocol command metadata, TLS, and TCP are excluded. | Synthetic local case | Baseline DataRow bytes | Treatment DataRow bytes | Reduction | | --- | ---: | ---: | ---: | | 15 runs, one selected checkpoint | 4,441,143 | 104,599 | 97.64% | | 15 unused bootstraps | 4,435,590 | 0 | 100% | | One small selected source | 6,395 | 6,319 | 1.19% | The committed-head benchmark uses two warmups and seven alternating samples per variant. Local round-trip medians and ranges were: | Synthetic local case | Baseline median (range), ms | Treatment median (range), ms | | --- | ---: | ---: | | Wide history | 20.32 (18.63–24.37) | 1.63 (1.40–2.21) | | Unused history | 18.52 (17.63–20.35) | 1.20 (1.13–1.56) | | Small selected source | 1.05 (0.95–1.70) | 0.78 (0.73–1.30) | The small case has overlapping ranges and a 1.19% byte change, so it is neutral. `EXPLAIN` server execution time was higher for the treatment (wide: 0.402 ms versus 0.028 ms across the two baseline statements). `EXPLAIN` does not serialize result rows; the new JSON presence checks do extra server work while avoiding large result transfer. These are local synthetic round-trip measurements, not production latency claims. The final proof ran on `dcde4e3dac2173321d7be2406e7775af8411826a` with no runtime source diff. ## Risks - Selection policy now has a SQL form. Regression tests preserve the pure selector's barrier semantics, including invalid non-null checkpoints and JavaScript Date precision. - Checkpoint-presence checks can still detoast candidate profiles inside PostgreSQL. The change reduces returned bytes; it does not eliminate all database JSON work. - The selected checkpoint, native input, and wake envelope can still be large because consumers require them in full. No new payload cap is introduced. - There is no migration, cache, background polling, ownership lock change, deployment, or merge in this PR. ## Model Used OpenAI GPT-6 through Codex. The host did not expose an exact runtime model ID or context-window size. The agent used reasoning, code execution, real PostgreSQL, and GitHub tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
c3a9a8e92a |
perf(server): scope run identity reads to operation needs (#17931)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Tool and connection operations capture the current accepted requester. > - Capture locks the task and run, then reconciles pending steering. > - Most callers use identity or task fields from the returned run. > - The default query also transfers saved prompts, runner configuration, results, and logs. > - This pull request gives each production caller an explicit read scope. > - Narrow scopes keep their authority checks and return only selected columns. > - The Muse project path still reads its complete saved execution input. ## Linked Issues or Issue Description **What existing behavior does this improve?** Operation-time run identity capture for native tools, gateway sessions, project and workspace access, Slack, voice, connection management, and connection token admission. **Current behavior** These callers use the default full run read. Large saved execution documents cross the database connection even when the caller only needs the current accepted requester or task binding. **Proposed behavior** Use typed identity, task, broker, and project-execution scopes. Keep task-before-run locks, company and agent predicates, active status validation, and pending-steering reconciliation. Return the complete saved native input only where project admission uses it. **Reason and benefit** A synthetic PostgreSQL 17.11 fixture matches a wide run with a 142 KB context and a 77 KB runner profile. Its locked run DataRow falls from 225,101 bytes to 190–605 bytes for authority reads. The project scope returns 66,004 bytes because it must retain a complete 64 KB execution input. These are local DataRow measurements, not a forecast of production billing or latency. **Breaking changes** None to public APIs, saved data, or authority policy. Internal narrow return types omit columns that were not selected. The default full compatibility read and the existing GitHub-credential and authentication scopes keep their current contracts. **Additional context** The public issue and PR duplicate searches found no matching open change. This is one independent query optimization. It does not require other performance PRs. ## What Changed - Add typed scopes for accepted identity metadata, task authorization, token attribution, and saved project execution input. - Assign explicit scopes at all production `captureRunIdentity` call sites. - Keep the full native execution input for Muse project admission while omitting unrelated profile, context, result, and log data. - Narrow the broker's initial read and the existing Slack and connection-management historical reads to the same required fields. - Extend steering and acknowledgement regressions to all new scopes. Add real PostgreSQL projection, malformed JSON, tenant and agent, session-generation, and saved-input coverage. - Add a service-generated SQL benchmark with exact raw DataRow accounting, retained-field equality, plans, and alternating transaction timings. - Document the scope contracts and reproducible benchmark. ## Verification - Baseline: `623f1dc289a14a699ebbe3f09b94179d38fc2ae2`. - `vitest run server/src/__tests__/run-identity-projections.test.ts server/src/__tests__/run-identity.test.ts`: 54 tests passed on an owned disposable PostgreSQL 17.11 database. - The tests retain task-before-run and audit foreign-key lock behavior. Every scope rejects uncertain steering, accepts its durable acknowledgement, and preserves requester replay ordering. - A full-row mutation makes the narrow identity regression fail. The restored treatment passes. - `pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-run-identity-projections.ts --baseline-ref=623f1dc289a14a699ebbe3f09b94179d38fc2ae2` captures the real locked SQL for both versions. - Every retained column and selected JSON field matches the baseline. Identity contexts match. The complete stored run remains unchanged. Capture keeps the same five queries. | Scope | Before DataRow bytes | After DataRow bytes | Reduction | | --- | ---: | ---: | ---: | | Identity | 225,101 | 190 | 99.92% | | Task authorization | 225,101 | 410 | 99.82% | | Token broker | 225,101 | 605 | 99.73% | | Project execution input | 225,101 | 66,004 | 70.68% | - DataRow bytes include row framing and raw text-format column values. They exclude response metadata, TLS, and TCP. - Two warmups and seven alternating samples measure the complete capture transaction. Several timing ranges overlap. The primary claim is byte reduction; production latency is not verified. - Initial fixture runs exposed an empty external schema and an incomplete synthetic conversation binding. The test setup now applies migrations and supplies the required user/state fields. The final 54 tests passed. - `pnpm -r typecheck` and `pnpm build` passed locally. - Four existing skill-setup tests passed. Four assembled connection-management/native-tool consumer cases passed with real JWT/API-key authentication. These include per-request identity freshness and task-scoped activity authority. - A CI-only stale mock assertion expected two capture arguments; it now verifies the explicit identity scope. A repository-wide capture mock audit found no other stale argument assertions. - Review follow-up: the projection suite uses the existing embedded-support guard while always enabling an explicitly supplied external database. The audit-lock regression merges URL query parameters and retains existing PostgreSQL options. - `vitest run server/src/__tests__/run-identity-projections.test.ts server/src/__tests__/run-identity.test.ts server/src/__tests__/company-skills-routes.test.ts server/src/__tests__/project-skill-setup-identity.test.ts`: 123 tests passed. The external PostgreSQL URL included both `application_name` and `options`; lock tests verified `lock_timeout=1s` and retained `statement_timeout=5s`. - Full local tests are not run because the host's embedded PostgreSQL setup is unavailable. Greptile reviewed `06e1dd53a3fb24df74d5fbfb2ad397f291f1e30f` at 5/5 with zero unresolved threads. The full final-head CI matrix passes. All 58 checks are terminal: 55 success, one neutral, and two optional Storybook skips. - No migration, lockfile, workflow, new polling, or cache change. ## Risks A narrow read must retain each caller's actual fields. The caller audit includes Slack's wake-request fallback, conversation generation, legacy broker attribution, and the Muse parser's complete native input. JSON selection retains value types, including malformed legacy values. The saved-input scope preserves missing-key behavior. The default full read still returns the complete stored row for compatibility. Locks and live authority checks are unchanged. ## Model Used OpenAI GPT-6, accessed through Codex. The exact runtime model ID and context-window size were not exposed in this session. Used repository inspection, code execution, and PostgreSQL validation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f60ac29cf1 |
fix(runner): migrate Grok Build with native reasoning effort (#17922)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Harness selection maps supported agents to Paperclip Runner. > - The native Runner already supports Grok Build, but its migration mapping excludes Grok. > - Grok agents also have a thinking-effort setting that the native execution contract does not carry. > - This pull request adds the migration path and preserves explicit Grok effort through startup and resume. > - Operators can upgrade an existing Grok agent without silently losing its model, account, or effort. ## Linked Issues or Issue Description **What happened?** Existing Grok Build agents have no Runner selector or upgrade action. Adding only the mapping would route conversion through the Codex fallback and lose Grok execution semantics. Native execution also ignores the legacy reasoning-effort field. **Expected behavior** A qualified Grok agent can explicitly upgrade. The saved model, account, instructions, history, and supported effort remain intact. Unsupported legacy settings fail before saving. The provider must acknowledge the selected effort before inference. **Steps to reproduce** 1. Open the runtime page of an existing Grok Build agent on a qualified Linux x64 or macOS ARM64 computer. 2. Observe that the Runner selector and upgrade action are absent. 3. Compare the native execution input with the saved legacy reasoning-effort setting. Related work: #13882 introduced native Grok Build. #15422 covers the earlier default-runner work. This change extends the current shared selection contract. ## What Changed - Add the Grok harness mapping, qualified platform limits, upgrade action, and company-scoped computer qualification. - Translate Grok settings to ACPX without changing the selected account or explicit model. Resolve the legacy default model to the native default. Preserve the effective legacy search policy. Reject incompatible command, argument, permission, and turn-limit settings. - Add native execution input v10 for explicit Grok effort and search policy. Preserve historical schemas. Enforce disabled search through Grok requirements and bind it across startup and recovery. - Apply and verify ACP `reasoning_effort` before prompting. Bind the value to provider descriptors, checkpoints, runtime identity, and resume checks. - Use the Grok catalog and effort choices in the task composer. Apply task model and effort overrides while retaining the agent account and search policy. Update runtime controls, portability/default handling, documentation, and capability maps. - Advance Pi and Copilot source attestations for changed shared transport and recovery code. Keep historical identities intact and update all Rust integration fixtures. ## Verification - Migration and review regressions were reproduced before the fixes. Coverage includes the Grok mapping, default model, explicit effort, implicit search policy, task overrides, and incompatible legacy settings. The broad migration, route, immutable-input, and composer check passed 478 tests. The final catalog and task-input check passed 135 tests. - Runner startup, recovery, search policy, and profile checks passed 361 tests, with one platform skip. The final source-identity, search-admission, and transport event-pump check passed 89 tests. - All 757 Rust workspace tests passed for the broad implementation. After the final source attestation update, all 55 ACPX profile/session tests passed. - `pnpm -r typecheck` and `pnpm build` passed for the implementation. Server and Runner typecheck passed after the final production change. Server `tsc --noEmit` passed after the final test-fixture change. Docker context integrity, token gates, maps, generated protocol data, and replay goldens passed. - The v10 startup fixture initially used an ambient local Grok login and failed on CI. An explicitly empty credential home reproduced all four failures. The fixture now supplies a synthetic key and a private empty home. A separate case checks that missing credentials fail closed. All seven selected Grok v10 and Claude v9 startup cases pass without host credentials. - The full local suite could not complete because embedded PostgreSQL failed before database-backed fixture setup. The broad executor-suite attempt hit the same limit and was stopped. Neither attempt is counted as passing. Current-head CI must verify those paths before merge. - Earlier CI found missing Rust input fields, stale source attestations and exact fixture pins, a missing Docker context entry, and an outdated import expectation. Those failures were fixed and checked locally. The CI logs retain the original failures. - Different earlier CI runs also failed unchanged cleanup pagination, device-login timeout, and chat lost-response recovery cases. Later attempts passed the cleanup and login cases without changes to those implementations. Their causes remain unproven. The original evidence remains in [the first run](https://github.com/paperclipai/paperclip/actions/runs/40563402840), [the login run](https://github.com/paperclipai/paperclip/actions/runs/40571128328), and [the chat run](https://github.com/paperclipai/paperclip/actions/runs/40573072163). A passing retry does not establish a root-cause fix. - Both reviews rate current head `16fba9b4cdf12bc7945e6628912e79da42959f35` 5/5. All current-head CI gates pass in [run 40576580699](https://github.com/paperclipai/paperclip/actions/runs/40576580699). All 60 check runs completed cleanly, including the four intentional skips and the neutral 5/5 Paperclip review. The legacy security status is successful. No review threads remain unresolved. - Staging migration and a real shell task are pending deployment. This PR does not claim full Grok provider qualification. Reviewers can verify the operator path by upgrading a qualified Grok agent, checking its saved model and effort, and running a shell task through native execution. ## Risks - Explicit effort and search policy require the matching Runner and sidecar build. Older immutable execution inputs keep their existing schemas. - Providers that omit or reject the selected effort fail before a prompt. Unsupported effort does not fall back silently. - Existing agents retain their saved runner until an explicit change. New Grok agents use native execution only on qualified platforms. - Grok token and cost reporting remain unverified, as before. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model ID and context-window capacity are not exposed in this session. Used reasoning, source editing, shell execution, and browser inspection. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and the focused suites pass; the full local suite has the environment limitation recorded above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |