mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-01 23:43:16 +01:00
9a79769c726da3ad2b1c784d1c6940c05dc46efb
429
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
9a79769c72 |
Merge landed cards and current native admission into retention
Preserve the original retention owner and producers alongside current native provider-pack admission, route execution-lock protection, canonical work-signal imports, and master CI/UI changes. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b6b57079b9 |
fix: evaluate status cards on committed changes and deadlines (#18401)
Reviewed and verified at
|
||
|
DottaandPaperclip
|
b7f1c3769c |
feat(native): record verified provider-pack session admission (#18469)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Remote native sessions can use an installed pack or a retained earlier pack. > - The current app revision does not identify the pack that a session admitted. > - Core already verifies the selected pack and binds its launch artifacts. > - Normal run records lack a closed receipt for that admitted selection. > - This pull request records that receipt after session admission and attachment. > - Operators can compare the actual Core admission identity with their expected pack. ## Linked Issues or Issue Description Refs #17749 for protected provider-pack selection and #18343 for retained controller recovery. The related OpenCode producer repair is #18456. Those changes do not publish the closed run receipt added here. A search found no existing `nativeProviderPackAdmission` PR. **What happened?** Core can admit a compatible installed provider pack with a source revision that differs from the current app. A retained session can also keep its original pack. The normal run record does not show the admitted pack digest, proxy digest, and source revision. An expected launch path or current app revision cannot fill that evidence gap. **Expected behavior** The run has a closed, server-owned receipt after actual session admission and attachment. The first receipt remains immutable history. A separate current-proof state verifies only the actual current admission epoch. Retained recovery needs positive original provenance, and missing legacy evidence stays unknown. **Steps to reproduce** 1. Select and verify an installed pack whose digest differs from the current app pack. 2. Open the native session successfully. 3. Inspect the normal run profile for a bound admission receipt. **Paperclip version or commit** The original selected base was `8bcc5a01edd5d87540401521e6f30d3df43e1848`. The branch now includes exact actual prerequisite parent `3ee4c5b6138da82d0b1f51aa010540561bf43c22`; the PR diff against it is the five receipt, executor, test and documentation files. **Deployment mode** Core remote native execution with provider-pack selection and retained session attachment. ## What Changed - Retain the verified selected pack identity only after a backend returns the matching admitted session. - Publish `runnerProfileJson.nativeProviderPackAdmission` from the session callback after attachment succeeds. - Mark the receipt with `stage: session_admitted` and `authority: core_verified_selection`. - Store bound company, agent, run, session, runner, logical `runnerEnvironmentLeaseId`, pack digest, OpenCode proxy digest, and source revision fields. Retained attachments identify `inheritedFromRunId`. The logical Runner scope is distinct from the physical environment lease. - Lock the current coordinator claim before the profile write. Check owner, attempt, generation, controller boot, lease expiry, phase, and absence of an accepted result. Merge JSON fields without replacing other profile data. - Restore retained metadata only from the exact prior closed Core receipt, positively verified prior state and matching retained binding. Capture same-run prior provenance from the locked claim row before invalidation; cross-run recovery reads only the exact source. Legacy absence remains unknown while recovery continues. - Preserve the first receipt as history. Atomically mark a new opaque admission epoch pending in the existing mandatory claim transaction. Publish current verification only for the matching claim and epoch after actual attachment; changed or unsupported admission becomes unknown without blocking authorized execution. - Preserve synchronous session publication and quarantine ordering before the best-effort receipt write. Fresh replacement selects its newly verified pack; a captured retained-process flag cannot relabel the replacement with an older pack. Preserve the existing private pack authority even when a local dirty revision is ineligible for the closed receipt. - Add focused admission, recovery, identity, PostgreSQL ownership, lock, and JSON-preservation tests. Document the receipt contract. ## Verification - RED: the actual managed-backend path verified and admitted an installed pack but had no closed metadata. Failed open published no evidence. Additional controls fail when retained recovery invents missing history or dirty-revision eligibility gates the existing private authority. - RED calibration: removing mandatory pending invalidation fails the actual pre-admission control; reusing the retained flag for fresh replacement falsely restores the old pack. Removing the epoch fence fails the real PostgreSQL stale-callback case; reintroducing a receipt-specific execution error fails both authorized Runner/logical-lease replacement controls. - GREEN: 49 focused Core tests, all 33 admission identity/transaction tests (20 pure and 13 PostgreSQL), and all 23 existing private-authority tests passed on exact integrated head `165a9096cf3e0d34d2db1e6b52960ad4ac0ad132`. No all-skipped run is counted. - Final test-only head `11ac34baf00552a5d322b5f5f1a8e491859cacec` adds the positive executor-callback SQL-writer wiring control, exact session/identity/claim/epoch assertions, profile preservation, and null-session zero-write check. Removing only the production writer call makes it fail on a missing receipt. All70 focused identity/Core tests pass with this closure; production code, the13 PostgreSQL tests and the23 private-authority tests are byte-identical to the qualified165 source (106 distinct passing controls across the combined proof). - The PostgreSQL tests used an owned loopback-only cached Postgres 17 container with the supported `PAPERCLIP_TEST_POSTGRES_ADMIN_URL` helper and aliases to current DB/shared source. The container was removed afterward. They cover immutable history, JSON preservation, stale claims and epochs, expired leases, terminal runs, changed/unsupported replacements, and a failed write retaining pending proof. - Historical pushed `f7f12fe` CI executed all 24 earlier admission tests, including nine PostgreSQL cases, in server5. Its warm-quarantine failure exposed the new hook-ordering issue fixed here. The same run also encountered a six-character UUID fixture-prefix collision and the independently repaired cached issue-link browser selector; those historical failures are not claimed as green. - Node 24.22.2 and Vitest 5.0.3. The credential-free normal QA runtime import preflight passes with networking disabled. Map contracts, token gates and `git diff --check` pass. - Local TypeScript validation still sees stale borrowed Runner/shared declaration mismatches outside the changed ranges. No diagnostics remain in the new helper or changed test/hook ranges; clean current-head CI is required. - Repository-wide typecheck, full tests and build were not completed locally. A full local executor attempt reached this host's unsupported native PostgreSQL listener and was interrupted; its result is not counted. Canonical run `41214512239`, attempt 2, is successful on exact final head `11ac34baf00552a5d322b5f5f1a8e491859cacec`. All 59 checks are terminal: 55 success, 3 skipped and 1 neutral; none failed or pending. The actual Linux suites execute all 33 admission tests (including 13 PostgreSQL cases) and all 1,022 executor tests successfully. The [original five failures and bounded unchanged-source controls](https://github.com/paperclipai/paperclip/pull/18469#issuecomment-6545593767) remain preserved. One failed-only request was made; GitHub expanded the reusable workflow call and dependents. No second request was made. Fresh exact-head Greptile review is 5/5, all review threads are resolved, and the PR is mergeable. - These local results claim no live provider admission, rollout or recovery. ## Risks - The authority is Core's verified selection plus successful session admission. It is not an independent provider statement or continuous runtime measurement. - Receipt fields cannot grant execution or replace the existing provider, process, lease, workspace, or recovery checks. - Legacy missing receipts and unavailable database reads remain unknown. The current release cannot label an older admitted pack. - The closed receipt requires a clean, 40-character hexadecimal source revision. Verified local `-dirty` packs still execute and retain their existing private recovery authority; their closed admission proof stays `unknown`, so it cannot certify an immutable source revision. - The first receipt is historical. Consumers must require current state `verified`; pending or unknown cannot certify the current pack. Changed Runner scope or pack evidence does not add execution policy. - A receipt save failure leaves the atomically invalidated state pending. The existing artifact authority, cancellation, workspace and process checks remain responsible for execution safety. - The receipt adds no manifest, environment, credential, or filesystem path to the run record. ## Model Used - OpenAI GPT-6 through Codex, with reasoning, code editing, and tool use. The session does not expose an exact model build ID or context-window capacity. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
3ee4c5b613 |
Make saved execution receipt recovery react to durable work (#18409)
Reviewed and verified at
|
||
|
DottaandPaperclip
|
42fd568cf8 |
Drive issue recovery from committed changes and exact deadlines (#18402)
Reviewed and verified at
|
||
|
DottaandPaperclip
|
8e414f54f4 |
Merge final cards revision ownership and parent acceptance
Co-Authored-By: Paperclip <noreply@paperclip.ing> * fix/sleep-merge-18401: Scope status-card refresh tasks to their claim revision test(chat): retain completion identity before link metadata loads test(chat): locate completion through its rendered task link fix(chat): recover committed native Agent Chat replies (#18465) perf(heartbeat): drive retry promotion from durable deadlines (#18387) fix(ui): make the Coordinator composer footer opaque (#18455) fix(testops): verify unexpanded result job runs (#18462) fix(runner): preserve OpenCode written-answer defaults (#18456) fix(native): fence retries when replaying an audited Stop (#18457) test: await async outcomes and drive saved deadlines deterministically (#18416) |
||
|
DottaandPaperclip
|
37bbb20cda |
Merge final deferred execution acceptance and locator proof
Co-Authored-By: Paperclip <noreply@paperclip.ing> * commit 'd6d0b67e8a7574ee63271bbc49e6b207a3065e80': test(chat): retain completion identity before link metadata loads test(chat): locate completion through its rendered task link fix(chat): recover committed native Agent Chat replies (#18465) perf(heartbeat): drive retry promotion from durable deadlines (#18387) fix(ui): make the Coordinator composer footer opaque (#18455) fix(testops): verify unexpanded result job runs (#18462) fix(runner): preserve OpenCode written-answer defaults (#18456) fix(native): fence retries when replaying an audited Stop (#18457) test: await async outcomes and drive saved deadlines deterministically (#18416) |
||
|
DottaandPaperclip
|
d6d0b67e8a |
Merge current master native chat recovery into deferred receipts
Co-Authored-By: Paperclip <noreply@paperclip.ing> * commit '11b49125db5ea1b274e69faf9d1abe057cdbb551': fix(chat): recover committed native Agent Chat replies (#18465) perf(heartbeat): drive retry promotion from durable deadlines (#18387) fix(ui): make the Coordinator composer footer opaque (#18455) fix(testops): verify unexpanded result job runs (#18462) fix(runner): preserve OpenCode written-answer defaults (#18456) fix(native): fence retries when replaying an audited Stop (#18457) test: await async outcomes and drive saved deadlines deterministically (#18416) |
||
|
DottaandPaperclip
|
11b49125db |
Merge current master into scoped issue recovery work
Retain scoped recovery registration and producer gates while preserving landed audited-Stop retry fencing and committed native Agent Chat response recovery. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
12b0cc6304 |
fix(chat): recover committed native Agent Chat replies (#18465)
## Thinking Path > - Paperclip manages agent work and keeps its results available to users. > - Agent Chat needs a visible reply before the conversation can wait for more input. > - Native result commitment and reply presentation are separate obligations. > - A completed conversation can retain a valid final message without a saved reply. > - Existing reply recovery excludes these completed conversation turns. > - This pull request repairs that persistence gap through the existing recovery worker. ## Linked Issues or Issue Description **What happened?** A native Agent Chat turn completed successfully and saved its workspace. Its accepted result and final assistant message were durable, but no reply appeared in the conversation. The chat stayed active. The deployment hiring journey in #18351 then reached its deadline while waiting for the first reply. **Expected behavior** Recover the exact accepted turn's final reply once, then use normal conversation settlement. Keep later user input active. Do not run the agent again or replace a selected or deleted reply. **Steps to reproduce** 1. Create an active conversation with one admitted native turn. 2. Commit an accepted DONE result and a final-channel assistant message for that exact run, session and turn. 3. Finish workspace finalization before live heartbeat reply presentation. Leave no saved reply. 4. Run normal response recovery. Before this fix it finds no conversation candidate and leaves the chat active. **Paperclip version or commit** Reproduced with actual services at frozen master `8bcc5a01edd5d87540401521e6f30d3df43e1848`. The production predicate also exists in the deployed revision that exposed the failure. Related work: #18351. The broader receipt and recovery work in #18118 and #18409 does not touch these source files. ## What Changed - Discover unanswered committed Agent Chat turns through the existing native response presentation worker. - Recheck company, agent, responsible user, current execution and conversation generation under the normal transaction locks. - Require the accepted result digest and an exact final message from the same native run, session and accepted turn. Keep external publication authority separate. - Commit the reply and presentation receipt before normal conversation settlement. Later input stays active. Recovery does not repeat agent work. - Preserve skip decisions and selected or deleted replies. Add database controls for foreign identity, stale sessions, partial commitment, duplicates and ordinary internal task behavior. - Document the recovery boundary in the developer guide. ## Verification - Real-service RED/GREEN uses all 418 canonical migrations in isolated PGlite and the exact new regression fixture. Before the fix: zero candidates, no reply, recovery returns false and the chat stays active. After the fix: one candidate, one exact reply, recovery returns true and the chat waits. A repeat returns false and creates no second reply. - All 20 canonical regression cases pass locally through a fixture-only PGlite bridge. Only local database provisioning and `createDb` selection differ from the normal embedded PostgreSQL fixture. Production services, SQL queries, transactions, protocol validation and assertions remain real. Linux embedded PostgreSQL CI is required before merge. - Focused current-source no-emit typing passes for both production modules and the new regression file, with normal Express augmentation and source aliases. The broader local presentation/ownership and conversation-settlement PGlite bridge run was interrupted before it reported results; it is not counted as a pass. No repository-wide local build is claimed from borrowed workspace dependencies. Clean-build CI and canonical PostgreSQL consumer tests remain required. - `git diff --check` and the scoped source/test secret scan pass. No captured run data, credentials or private target identities are committed. - No live journey retry, deployment, configuration change or test-plan admission was performed. The original failed journey remains failed; this production fix still needs deployed qualification through the unchanged journey. - Current-head [canonical Linux CI](https://github.com/paperclipai/paperclip/actions/runs/41207018345) passed all 57 terminal checks with no failure or pending check. The new 20-case database regression actually passed in [server shard 8](https://github.com/paperclipai/paperclip/actions/runs/41207018345/job/124555006090), at 13:59:25 UTC (2,643 ms). Typecheck, build and the existing consumer lanes passed. Fresh [Greptile review](https://github.com/paperclipai/paperclip/pull/18465#issuecomment-6544208002) on exact `f32a432e10b377ae6277d6f728995d5fedcf9d7b` is 5/5; the review-thread inventory is empty with no pagination. No live deployment or journey pass is claimed by these checks. ## Risks Eligible active conversations can gain a previously missing reply. Recovery accepts only the exact saved final message under its accepted result and current conversation identity. It does not run a model, repeat hiring or change a prior test grade. Reply persistence precedes waiting-state settlement; existing conversation recovery retains ownership of a later interrupted settlement. No schema or public API changes are included. ## Model Used OpenAI Codex, GPT-6 family. The session does not expose a more specific backend model ID or fixed context window. Used reasoning, repository tools, TypeScript and real database test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
baf651c8d0 |
perf(heartbeat): drive retry promotion from durable deadlines (#18387)
Reviewed and verified at
|
||
|
DottaandPaperclip
|
aab54d5bec |
Merge current master and generated card schema into attention retention
Preserve master attention, issue, native cleanup and tool authority changes alongside the exact reviewed retention patch. Keep bounded blocker facts, finalization precedence, source locks, archive eligibility and producer intent unchanged. Publication remains dependent on the parent qualification. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
56f8ff7bd5 |
Merge current recovery and regenerate status-card migration after master 0419
Preserve the published concurrent comment uniqueness and read-state backfill migration. Regenerate the combined schema as migration0420 with the repository workflow and retain five snapshots. Add both upgrade-path controls without changing status-card execution policy. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
3549fad1dc |
Merge commit '0b4e48be9b881484dc5c48225a3b501595329ce2' into codex/sleep-merge-18409
* commit '0b4e48be9b881484dc5c48225a3b501595329ce2': fix(ui): retain startup autosave through session recovery (#18412) perf(server): recover independent native replies concurrently (#18421) perf(chat): scope task questions and batch inbox queue checks (#18422) fix(native): recover controllers across webserver replacement (#18343) fix(ui): restore task document authoring and add persistence journey (#18356) fix(tools): stabilize tied profile entry order (#18427) fix(adapter-utils): tolerate entries removed during workspace snapshots (#18426) test(runner): qualify Kimi Code lifecycle and recovery (#18029) feat(runner): deliver native Kimi Code (#18391) Preserve selected Google services when setup resumes (#18048) feat(runner): add pinned Kimi Code native runtime (#18028) fix(ui): show agent-created subtasks inline in the task feed (#18320) fix(evals): preserve opaque TestOps parameters (#18394) feat(chat): deliver proactive task messages to agent chats (#18374) fix(ci): track repository inputs across test projects (#18398) chore(lockfile): refresh pnpm-lock.yaml (#18414) fix(runner): build on Windows and add a Windows CI lane (#18250) |
||
|
DottaandPaperclip
|
12d8c3043e |
fix(native): fence retries when replaying an audited Stop (#18457)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native runs keep Stop intent and retry state in durable records. > - A provider error can leave a failed retry with an earlier acknowledged Stop. > - The replay path returned the existing receipt before it closed that retry. > - This pull request applies the existing retry fence to the validated replay. > - The same Stop can settle without another provider cancellation or a new receipt. ## Linked Issues or Issue Description Refs #14733 for the related caller-bound Stop contract. This change covers a later retry that already has a Stop acknowledgement. The older CI-only #14743 is not a production delivery PR. **What happened?** A failed native run can have a retryable coordinator and an earlier audited Stop. A repeat Stop returns the old acknowledgement without moving the coordinator to `terminal_failure`. The final failed-to-cancelled transition requires that fence, so the Stop cannot settle through that path. **Expected behavior** A valid replay closes the same retry and keeps the original intent and acknowledgement. It does not send a second provider cancellation. Physical stop and workspace cleanup still require their own evidence. **Steps to reproduce** 1. Store a failed native run with a `retryable_failure` coordinator. 2. Store a matching run-scoped Stop with a dispatched acknowledgement and audit IDs. 3. Call `cancelNativeSession` for that same run and inspect the coordinator. **Paperclip version or commit** Reproduced on public master `f2db909393de124caf6a5380a52d4e0e1d621661`. **Deployment mode** Core native execution. The regression uses the real cancellation executor with controlled persistence and provider boundaries. ## What Changed - Reuse the existing guarded coordinator fence for a validated Stop replay. - Test repeated replay, unchanged receipts, one fence, and no second provider cancellation. - Reject foreign company, run, issue, scope, effect, and missing-audit receipts before any write. - Check that an acknowledged Stop with an unconfirmed remote lease still fails the physical-stop guard. - Document the separate retry fence and physical-cleanup requirements. ## Verification - RED on the selected public base: the new executor case returned success but left the coordinator in `retryable_failure` with a future retry. One case executed and failed for that assertion. - GREEN: 62 focused tests passed across `native-session-executor.test.ts` and `native-cancellation-request.test.ts`. These include ownership, caller conflict, changed-phase, and receipt controls. - Command from `server/`: `node ../node_modules/vitest/vitest.mjs run src/services/native-runtime/native-session-executor.test.ts src/services/native-runtime/native-cancellation-request.test.ts -t 'native session cancellation|caller-bound native cancellation|failed native retry caller admission'`. - Canonical Linux CI passed all 1,018 executor cases and all 32 caller-cancellation cases. The exact file completion lines are in [server shard 2](https://github.com/paperclipai/paperclip/actions/runs/41203989386/job/124545366271) and [server shard 10](https://github.com/paperclipai/paperclip/actions/runs/41203989386/job/124545366219). - Node 24.22.2 and Vitest 5.0.3. Map contract checks and `git diff --check` pass. - Local dependencies are linked from an existing checkout. Full local TypeScript validation reports older Runner/shared declarations that lack current Kimi and continuity APIs. The changed ranges have no diagnostics. The canonical clean CI Typecheck + Release Registry gate passed. - A broader two-file run was stopped when an unchanged integration case tried to start embedded Postgres and hit the host listen restriction. It is not a passing full-suite result. No host settings or other services were changed. - Repository-wide `pnpm -r typecheck`, `pnpm test:run`, and `pnpm build` were not completed locally. Current-head CI passed against clean source and dependencies. - Fresh [Greptile review](https://github.com/paperclipai/paperclip/pull/18457#issuecomment-6543772487) on `dbc1c1a5b71450194d568725592969660864cb5a`: 5/5 with no actionable findings. The review-thread inventory has no threads and no pagination. Full [current-head CI](https://github.com/paperclipai/paperclip/actions/runs/41203989386) passed: 59 terminal checks, 55 successful, 3 skipped, and 1 neutral. No failure or pending check remains. Merge status is CLEAN. No live recovery, rollout, or physical termination is claimed. ## Risks - A wrong or stale Stop must not close another retry. Existing run/coordinator locks, caller checks, receipt binding, and the final compare-and-set remain in place. - This fix changes logical retry settlement only. Provider termination, workspace preservation, lease cleanup, and continuation gates remain separate. - The regression controls persistence and provider boundaries. It does not establish the physical state of a remote provider. ## Model Used - OpenAI GPT-6 through Codex, with reasoning, code editing, and tool use. The session does not expose an exact model build ID or context-window capacity. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0b4e48be9b |
Merge current controller recovery with scoped issue deadlines
Preserve keyed concurrent checks and their default serial behavior alongside issue-recovery reconstruction spacing and before-pass preparation. Qualify keyed ordering, joined preparation, exact deadline preemption and retained late-write reconstruction debt across drain. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d1f9534c54 |
Merge current master qualification into event-driven run dispatch
Preserve retained-controller recovery deadlines and rearming together with company-scoped dispatch forwarding. Keep both source owners and all incoming master changes; no execution or retry policy changes. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
c2971badde |
perf(server): recover independent native replies concurrently (#18421)
## Thinking Path > - Paperclip manages work for AI agents. > - A native run can commit its result before its reply is saved. > - The response recovery worker saves replies from those committed results. > - It checks each candidate in series, so one task can delay unrelated tasks. > - This pull request checks up to four independent tasks at once. > - Each task keeps serial checks, and the existing coordinator owns retries and shutdown. ## Linked Issues or Issue Description **What existing behavior does this improve?** Recovery of committed native task replies. **Current behavior** The worker checks one candidate at a time in a batch of up to 25. A database lock or a slow read for one task delays replies for other tasks. **Proposed behavior** Check up to four company-and-task groups at once. Check candidates within each group in their existing order. Keep the current batch limit and company selection. **Reason and benefit** A ready task reply can be saved while another task waits for a database lock. The change reduces waiting under a reply backlog. **Breaking changes** Replies for different tasks can be saved in a different order. There are no API or schema changes. Other deadline workers keep serial dispatch. Related work: #17696 adds the recovery worker. #17501 reduces its read payloads. #18367 adds its timing spans. These changes are already merged. The PR search found no open change that adds this concurrency. ## What Changed - Add optional keyed concurrency to the shared scoped deadline worker. - Enable a limit of four for native response recovery, keyed by company and task. - Await all active checks before the pass ends. Retain per-candidate failure backoff, drain behavior and pending work. - Test concurrency limits, task order, company separation, drain/resume, shutdown, retries and idle wake behavior. - Add a PostgreSQL regression that holds one task's finalization row lock while a second task saves its exact accepted reply. - Document the worker's concurrency and sleep contract. ## Verification - The PostgreSQL lock regression fails with serial dispatch. It passes with this change. Both accepted replies are saved once after the lock is released. - `pnpm exec vitest run server/src/__tests__/scoped-deadline-work.test.ts server/src/__tests__/delivery-work-coordinator.test.ts server/src/__tests__/company-work-batch.test.ts server/src/__tests__/deadline-work.test.ts server/src/services/execution-preparation-timing.test.ts` — 55 tests pass. - `pnpm exec vitest run server/src/services/presentation-ownership-read.test.ts` with its documented dedicated PostgreSQL URL — 21 tests pass. The disposable database was dropped. - `pnpm -r typecheck` — passes. - `pnpm build` — passes. - One matched local replay with 25 older pending replies and one fresh reply used the real worker, a 10-connection PostgreSQL pool and 8 ms delay in each network direction. The saved fresh reply was first observed after 29.193 s with serial dispatch and 9.798 s with this change. All 26 replies were saved exactly once in both runs, with no worker errors. Both disposable databases were dropped. This is one synthetic workload pair, not a staging measurement. - `pnpm test:run` was attempted. Embedded PostgreSQL could not initialize its template after five attempts, so the broad run was stopped. The changed PostgreSQL tests pass on the separate local server. Remote CI is green. This PR does not claim a measured staging speedup. - The [first CI browser shard](https://github.com/paperclipai/paperclip/actions/runs/41162257463/job/124422204506) failed during company cleanup after its task assertions and empty-run check passed. Company deletion hit the `activity_log_company_id_companies_id_fk` constraint. The fixture uses paused process agents and reported no runs. The one failed-jobs rerun passed all 45 browser tests. All CI checks are now green. ## Risks - Four checks can use more database connections than serial recovery. The limit applies to this queue only; the batch limit stays at 25. - The shared helper has more dispatch logic. Its default remains serial. Tests cover task order, retry deadlines, drain and waiting for active checks during shutdown. - This improves reply recovery under backlog or lock contention. It does not establish a target for total agent-turn latency. ## Model Used OpenAI Codex, based on GPT-6, with code editing, tool use and local test execution. This session does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
492d537340 |
fix(native): recover controllers across webserver replacement (#18343)
## Thinking Path > - Paperclip manages AI agents and their work. > - Native execution runs in a remote Runner while the webserver owns its controller. > - Encrypted PRP checkpoints and session snapshots already survive a webserver restart in Postgres. > - Ordinary warm and per-turn admission still needs local markers and supervisor metadata to find that state. > - Replacing the webserver with an empty native controller directory can therefore lose a usable retained session. > - This change adds bounded recovery metadata to the existing snapshot transaction and reconstructs local projections from the selected durable source. > - The benefit is native execution continuity across webserver replacement, with the existing ownership and agent lifecycle policies. ## Linked Issues or Issue Description **What existing behavior does this improve?** Native remote controller recovery and retained-session continuation during a webserver upgrade. **Subsystem affected** server/ — native runtime, checkpoint persistence, recovery scheduling and internal upgrade reporting. Product E2E qualification also changes. **Current behavior** Without both the local PRP marker and supervisor file, normal continuation can return before discovering the saved Postgres checkpoint. The snapshot already exists in Postgres, but its local reuse compatibility envelope cannot be reconstructed. **Proposed behavior** Select the existing durable source, validate its identity and ownership, restore its PRP state and supervisor projection, then use normal authenticated admission. Missing or inconsistent evidence holds recovery. It does not authorize fresh provider work. **Reason and benefit** A replacement webserver must recover active execution and later continue its retained session without the old server's native files. This is one component of deployment continuity. HTTP traffic switching, workspace storage migration and Railway rollout remain separate. **Breaking changes** No new public endpoint, schema migration or database pool. The private run-profile envelope is additive. Disk-independent rollback also requires a reader for the staged controller format and compact checkpoint part; the first production release containing these readers is the floor. Older file-backed sessions need an owner checkpoint or the bounded internal importer. **Additional context** Related foundations: #17401, #17916, #17254, #14621, #14622. The implementation contract and open acceptance gates are in [the dated plan](doc/plans/2026-10-30-native-controller-checkpoints.md). ## What Changed - Persist a versioned recovery envelope, at most 4 KiB, beside the redacted session snapshot in its existing transaction. Keep it out of public run projections. - Compare frozen admission separately from prepared attachment instructions during staged recovery, preserving exact template/identity/authority checks. - Bound the existing lazy checkpoint pool at three connections after measured two-slot burst backlog; keep request/renewal capacities and SQL operation counts unchanged. The tuning is a separate commit. - Discover and restore the exact selected retained source without local native markers. Preserve the 32-candidate window, progress barriers and current authority checks. - Fence snapshot and result writes. Acquire coordinator, issue and run locks in order and check expiry after lock waits. Combine snapshot ownership/read work into one statement. - Reuse existing recovery deadlines and drain/resume scheduling. Retained-owner waits preserve the original unstarted run and actual lease deadline. Stop, ownership and provider-progress fences precede deferral. Add bounded native evidence reporting, retained-source import and retention protection. - Admit a settled warm source with an authenticated idle goal-read suffix while retaining active-work and mixed-authority holds. - Permit lifecycle `turn.stop` checkpoints after accepted results under existing ownership fences. Claim the existing successor coordinator before retiring the old warm session, and stage shared attachments only after verified retirement. Preserve omitted isolated-home defaults and existing fresh skill-selection order during recovery without hiding real configuration changes. - Carry the exact durable six-field PRP binding during active recovery after a healthy warm handoff. Keep the Runner mismatch guard and preserve opaque attachment IDs. - Preserve an already verified active or idle warm process during remote preparation; revalidate its proof/liveness and retain later authenticated PRP attachment. Ordinary resumed leases retain suspension checks. - Drain retained source events under their original authority, reconcile immutable source accounting and block unknown source tool dispatch before target work. - Persist compact staged handoff intent through authenticated adoption and recover direct/listener command-loss windows without another Runner or duplicate attachment. - Rebuild suspended lifecycle receipts for resumed prepared tasks, and retry missing lifecycle reads after partial PRP materialization without overwriting contradictory evidence. - Add normal warm/per-turn diskless continuation coverage, real Postgres races and SQL-cost checks. Add a reproducible paired load workload. - Give only the explicit outage fixture a 15-minute provider idle window, with a red/green catalog regression preserving all production timing and lifecycle policies. - Publish the goal-less task-session pointer in the existing post-acceptance terminal checkpoint transaction, with current ownership, accepted-result, reset and newer-progress fences. Keep a private bound receipt so late completion cannot duplicate or resurrect publication. - Use a retained warm sandbox in the dedicated continuity fixture, and accept an exact output token followed by an explanation. Keep the strict identity oracle. - Extend the explicit Daytona cell to a five-minute active outage and another replacement after completion. Require retained provider identity, a new tool receipt and a useful follow-up result. Add an owned loopback Postgres E2E fixture for hosts that cannot start embedded Postgres. ## Verification **Merge recommendation — `fc68584ce6231a02e47ecd743f55b036ccea034f`: ready for the qualified native-controller component.** Full CI, fresh review, local application paths, controlled performance qualification and the strict live Linux/Daytona case pass. Independent resource cleanup is complete. Broader rollout, browser reconnection and the unexecuted transport/platform combinations remain integration gates. Integrated 484 passed full local recursive typecheck/build and ordinary active→completion→cache-loss→C component qualification. Fresh review then found a valid attached-wake failure: preparation appends attachment instructions before the verifier compares execution to the frozen admission. 9f4 fixes that by separately checking the pre-preparation admission and exact prepared execution. Both hashes are captured privately per admission; the full template, identity, configuration and authority checks remain. Its regression fails before the fix; 36 focused cases, 42 staged source/target cases and server typecheck pass afterward. The review thread is resolved. Exact-head full CI passes ([run 41168228822](https://github.com/paperclipai/paperclip/actions/runs/41168228822): 48 successful jobs, one intended skip); the complete `pnpm test:run` partitions are forced, with the five native suites in the Runner lane. This is not a claim that an unsharded local command passed. The fresh exact-head Greptile check succeeds with 5/5 and all six threads resolved. The original two-connection database comparison closes **steady PASS and burst FAIL**. Every relative CPU/WAL/request-p99 gate passes and all operations/ownership remain correct, but candidate arm 3 drains in 1,015.99 ms and baseline arm 4 in 1,059.67 ms against the unchanged 1,000 ms absolute gate. Those results remain failed. A separate 2→3→4→2 diagnostic reproduces drain times 1,051.57 / 956.79 / 882.02 / 1,065.72 ms: three is the smallest tested passing cap, and the repeated two-cap failure survives run-order/cache effects. Separately reviewable `fc68584ce6` changes the existing lazy checkpoint pool default to three connections. Request capacity stays ten and liveness two; idle release, shutdown fencing, transaction counts and pool count are unchanged. Thirteen DB lifecycle/cap tests and DB typecheck pass. Full recursive typecheck/build pass at exact |
||
|
DottaandPaperclip
|
59da15066a |
feat(runner): deliver native Kimi Code (#18391)
Deliver native Kimi Code setup, exact models, scoped credentials, verified packages and runtime packs. Preserve current upstream release attestations and restore verified same-run remote retry dispatch without weakening process ownership. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
98a68b492f |
Merge verified sleep prerequisites into 01
Co-Authored-By: Paperclip <noreply@paperclip.ing> * commit 'd287074fba0ab0365d61416dadc392cc3078eee5': Remove unused aggregate watchdog diagnostics test: settle delegated chat completion before plan assertions test: settle saved retry cleanup before idle assertion fix: wake retry scheduling only when retry state changes # Conflicts: # server/src/services/heartbeat/retries.ts # server/src/services/native-runtime/native-provider-capacity-retry.ts |
||
|
Dotta
|
d287074fba |
Merge verified sleep prerequisites into 07
Co-Authored-By: Paperclip <noreply@paperclip.ing> * commit '78bd3560a886c7fd3eea03baccb6cf49f6e8499a': Remove unused aggregate watchdog diagnostics test: settle delegated chat completion before plan assertions test: settle saved retry cleanup before idle assertion fix: wake retry scheduling only when retry state changes # Conflicts: # server/src/services/heartbeat/retries.ts |
||
|
DottaandPaperclip
|
822b173281 |
test: preserve retry producer boundaries in receipt recovery
Co-Authored-By: Paperclip <noreply@paperclip.ing> * fix/sleep-forward-02b-security: fix: wake retry scheduling only when retry state changes |
||
|
Dotta
|
63c922a193 |
Merge commit 'c2ff3eb92113fb16713b270e34932636956ff16b' into fix/sleep-forward-02b-security
* commit 'c2ff3eb92113fb16713b270e34932636956ff16b': fix: wake retry scheduling only when retry state changes |
||
|
DottaandPaperclip
|
c2ff3eb921 |
fix: wake retry scheduling only when retry state changes
Separate retry-schedule notifications from ordinary queued-run eligibility. Preserve exact deadlines and prove native producer commit/rollback behavior without global retry scans on ordinary agent or issue edits. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8ff812cdde |
feat(chat): deliver proactive task messages to agent chats (#18374)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agent Chat gives each person a persistent conversation with each
agent.
> - Tasks and routines produce results outside that conversation.
> - Those runs need a safe way to bring a requested result back to the
person.
> - The server must select the agent's own chat and check the source
task's access rules.
> - This pull request adds proactive chat messages and unread
indicators.
> - People can see the result, open its source task, and reply in the
same chat.
## Linked Issues or Issue Description
**Subsystem affected**
Server, native runner tools, shared contracts, database, and UI.
**Problem or motivation**
An agent can report progress on a task or comment on an authorized task.
It cannot send a routine result to its own standing chat with the
responsible person. A morning review routine therefore cannot leave a
visible digest in that person's chat.
**Proposed solution**
Add `send_chat_message({body, idempotencyKey})` and `POST
/api/agents/me/chat/messages`. The server selects the company, agent,
and responsible person from the active run. It saves an attributed
comment and audit record in one transaction. The chat shows a
source-task link and per-person unread state.
**Alternatives considered**
A destination parameter on a generic comment tool would expose
unnecessary routing authority. A new delivery worker would add another
durable delivery path. This change uses a dedicated tool and the
existing comment and live-event paths.
**Roadmap alignment**
This extends the experimental Agent Chat capability in `ROADMAP.md`. The
existing `enableAgentChat` setting controls it. Related public work:
Refs #13282, Refs #13283, and Refs #14408. Searches for proactive chat
and unread chat found no duplicate PR.
## What Changed
- Add a focused messaging service shared by the native tool and HTTP
route. It checks current run ownership, execution mode, recipient
membership, task access, and task privacy.
- Reuse the canonical company/agent/person conversation. Save the
comment and audit record together. Keep task completion and chat
execution separate from delivery.
- Add source-task provenance and source-scoped retry keys. Identical
retries return the saved receipt. Changed content returns a conflict.
- Enforce the conversation's fixed agent at the shared comment-write
boundary. Keep normal reply cancellation and session checks.
- Add chat unread state, accessible agent dots, aggregate Chat
indicators, and visible-message read cutoffs. Refresh through existing
live events.
- Add the request-key constraint, native tool contracts, documentation,
capability maps, stories, and regression coverage.
## Verification
- Local delivery/security proof: all 38 proactive-message cases pass.
Eight credential-scope regressions and four quota regressions fail
against the prior source. The repaired delivery and canonical
cross-issue cap suites pass all 49 cases; all 31 OpenAPI tests, server
typecheck, and capability-map checks pass. The shared per-run cap covers
native and HTTP sends, mixed ordinary mutations, concurrent last-slot
contention, and retry/recovery without double charging.
- Credential tests preserve the authenticated person and authorizer;
reject bridge/test credentials and persisted skill-test runs after
task-mode changes; and recheck API-key revocation, scope, and
responsible person after destination locking. The earlier
delivery/native-authority suites passed all 74 cases before the four
quota additions.
- Four PostgreSQL contention cases cover native and HTTP sends plus
retries during a native chat mutation. All four deadlock against the
earlier source and pass with destination-before-source/run locking.
Recovery and human-reply delivery enqueue only the human comment; a
proactive digest never wakes the chat agent.
- Passed 108 companion conversation, native-authority, and OpenAPI
tests; 112 runner catalog/workflow-report tests; 47 completion-delivery
tests; and 25 project-handoff tests. Focused runner tests cover native
tool availability without generic API tools.
- All 327 affected UI tests, UI typecheck, token gates, and catalog
checks pass. The read hook retries at most three times, respects server
retry delay, coalesces only seen cutoffs, recovers on reconnect, and
cancels on scope changes.
- Real browser journeys with a deterministic process agent verify
routine-to-chat delivery, first-chat creation, unread state across
reload, the source link, secondary chat navigation, and a follow-up
reply referring to the digest. This does not qualify live GitHub
retrieval or model output quality.
- The Storybook showcase uses production components with deterministic
network fixtures. Rendered interactions pass for the landing, ordinary
read success, transient read recovery, and persistent failure after
bounded retries. Both Chat navigation variants update from two unread
chats to one while only Kodi's dot clears. Mobile and light variants
were reviewed. The existing Coordinator Recents label contrast finding
is unchanged; no Linux visual baseline archive exists.
- Current master's independent sandbox migration retains 0418.
Regenerated the feature migration as 0419 from the combined schema and
retained exactly the newest five snapshots; numbering/safety, replay,
and schema-drift checks pass.
- Workspace typecheck and build passed before PR preparation; canonical
server/UI typechecks passed again on the combined tree. Migration
`0419_slippery_impossible_man.sql` passes safety and schema drift checks
on disposable PostgreSQL, including interrupted index repair, concurrent
migrators, one-time owner read-state backfill, preserved receipts, and
two SQL replays. A default five-second local snapshot comparison timed
out; a diagnostic 30-second run passed without drift.
- Earlier failures are retained in the record: full local `pnpm
test:run` and broad shards encountered embedded-PostgreSQL startup,
timing, and provider-recovery failures. Thirteen selected
external-channel cases pass in isolation on the original and changed
sources; an unordered comment-row assertion now explicitly orders both
reads. The earlier route registry, native tool count, runner catalog,
and accessible Chat badge-name failures were fixed and passed full
remote CI on `920b3bb7db`.
- Repeated managed-skill server8 failures came from CI's newer synthetic
merge: its Cloud guard correctly rejected the old local-disk fixture.
Master independently repaired the fixture in #18385; this branch keeps
that repair unchanged. All 57 managed-skill and Cloud prerequisite cases
pass on the combined tree, and that fixture has no remaining PR diff.
- Current reviewed head `dc16f314739a6d977bed21cb1446d0e80a541578`:
Greptile 5/5 with no findings, Superagent security assessment passed,
advisory Paperclip Review 5/5 with no findings, and zero open review
threads. All 55 current-head check runs pass, with five expected skips
and the advisory review neutral at 5/5; Snyk also passes. No checks are
pending or failing. Protected squash auto-merge is enabled and is
waiting for required code-owner approval. A normal exact-head merge was
refused by the base-branch policy; no admin bypass was used. The
previous head's Superagent quota finding is fixed and covered above.
## Risks
- Chat remains company-visible. Delivery rejects a source task with
private-task, project, or ancestor restrictions.
- Read state uses a comment timestamp cutoff. The server retains
database precision and never moves a read receipt backward.
- The new unique constraint changes comment query planning. An existing
regression now uses an explicit row order. The migration builds the
unique index concurrently, then takes a brief table lock to attach it.
It uses the existing top-level migration and interrupted-index recovery
path.
- The upgrade backfills missing chat-owner read-state rows at the latest
existing agent comment. Existing read receipts remain intact. New chats
can still be unread before the human participates.
- Quota observation commits atomically with delivery. A rejected send
rolls back its attempted denial audit row and any first-chat creation;
the canonical warning remains in server logs, and previously spent quota
remains enforced. Identical/conflicting retries and receipt recovery
spend no extra slot.
- This version delivers in-app only. It adds no external-channel, push,
or email delivery.
- Local full-suite acceptance remains limited by the failures recorded
above. The feature stays behind `enableAgentChat`.
## Model Used
OpenAI Codex. PR preparation and CI work used `gpt-6.1-sol` with tool
use and code execution. The earlier implementation also used Codex, but
its exact model ID was not recorded in the visible session. The
context-window size was not provided.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run focused tests locally and they pass; the broad local
failures are recorded above
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
|
||
|
DottaandPaperclip
|
688f35763d |
merge: preserve attention, status-card and recovery source ownership
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
9b4beff72b |
merge: preserve status-card and recovery producers in the review stack
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
290d12cf4b |
merge: rehearse issue-recovery predecessor for saved receipts
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
dee5296b8a |
merge: rehearse current dispatch predecessor for issue recovery
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
fcc107739c |
merge: rehearse current master and UI prerequisite for dispatch
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ac6c5191ee |
Drive saved execution receipts from committed work and deadlines
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
4ef018cfe6 | test: model retention reads in native cleanup transaction fixture | ||
|
Dotta
|
0fe6e038c2 | Restore independent recovery duties and bound committed-source work | ||
|
Nicky Leach
|
9e904ee519 | perf(server): overlap independent heartbeat preparation reads (#18372) | ||
|
Dotta
|
ab5108611d | Run recovery producer acceptance on isolated PostgreSQL fixtures | ||
|
Dotta
|
dc4456c76b | Drive issue recovery from scoped source writes and saved deadlines | ||
|
DottaandPaperclip
|
0fac69e167 |
fix(server): scope run retention hints to affected companies
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
d334f59c98 |
Merge remote-tracking branch 'origin/master' into codex/sleep-status-card-evaluation
* origin/master: fix(evals): support official TestOps job metadata (#18385) fix(config): require shared storage for managed cloud (#18350) feat(skills): make managed content independent of web-server disk (#18334) Add a portable agent question and answer resumption journey (#18346) fix(ci): isolate test selection and require current CI CODEOWNERS (#18376) fix(evals): capture TestOps registration after watch (#18375) fix(runner): allow session reset after confirmed task retirement (#18373) fix(evals): batch task search result readback across renders (#18370) fix(ui): keep task plugin views available for manual access (#18363) test: wait for the evidence expiry worker to settle (#18241) # Conflicts: # server/src/services/company-skills.ts # server/src/services/native-runtime/paperclip-runner-tool-authority.ts |
||
|
DottaandPaperclip
|
4af412eaea |
refactor: drive status-card evaluation from committed work
Move generation sequencing into a feature-owned deadline worker and retain persisted evaluation, eligibility and relative-search obligations. Add transactional producer hints and initial PostgreSQL acceptance coverage; follow-up completes the producer audit and remaining lifecycle gates. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
2955e0e22e |
fix(server): scope run retention hints to affected companies
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
15cc12dba6 |
fix(heartbeat): scope and batch dispatch wakeups by owner
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b1104c6dd7 |
fix(server): scope run retention hints to affected companies
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0f3017c96d |
perf(server): archive attention from source deadlines
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
b57f5e5869 |
Merge remote-tracking branch 'origin/master' into codex/sleep-run-task-recovery
* origin/master: fix(config): require shared storage for managed cloud (#18350) feat(skills): make managed content independent of web-server disk (#18334) Add a portable agent question and answer resumption journey (#18346) fix(ci): isolate test selection and require current CI CODEOWNERS (#18376) fix(evals): capture TestOps registration after watch (#18375) fix(runner): allow session reset after confirmed task retirement (#18373) fix(evals): batch task search result readback across renders (#18370) fix(ui): keep task plugin views available for manual access (#18363) test: wait for the evidence expiry worker to settle (#18241) |
||
|
DottaandPaperclip
|
0e15c92073 |
feat(skills): make managed content independent of web-server disk (#18334)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents use installed skills and exact version pins during execution. > - Managed skills still depended on mutable directories on the web server. > - A replacement server could lose acknowledged edits or remove an unassigned skill whose directory was absent. > - This pull request makes immutable versions the authority for managed content and installed repository reads. > - The server can rebuild delivery files from durable content after replacement. > - This establishes skills readiness for ephemeral web servers. General server draining and workspace recovery remain separate work. ## Linked Issues or Issue Description **Subsystem affected** Skills services, shared contracts, database receipts, Skill Studio, and runtime delivery. **Problem or motivation** A fresh web server must discover, read, edit, and deliver protected installed skills using the same database, object storage, secrets, and supported application artifacts. It must not need its predecessor's files or process memory. **Proposed solution** Use `managed` ownership with a null source locator. Read installed content by immutable version. Prepare packages before a short publication transaction. Persist execution selections before materialization. Recover exact pinned runtime assets and reject deletion while active or recovering pins exist. **Alternatives considered** Reconstructing a mutable installed directory preserves the old source-of-truth ambiguity. Permanent dual writes create two competing authorities. This change uses one managed model after a bounded legacy conversion. **Roadmap alignment** This improves the existing Skills Manager, Studio, version history, and Cloud execution capabilities. The maintainer supplied the implementation contract. This does not add a separate skills product or deploy fleet changes. Related work: #18082 supplies immutable package storage. #17273 addresses edits outside the API for linked local skills; this PR keeps that source category separate. #2648 is a closed earlier ephemeral-filesystem proposal. ## What Changed - Add managed ownership and focused content publication and execution materialization modules. Keep the archive format and package storage tables. - Publish managed create, save, delete, fork, rename, restore, and catalog content with version checks. Add durable create receipts. Preserve file-save receipt responses. - Return file version identities. Keep Studio drafts after a conflict and provide an explicit reload action. A delayed reload cannot overwrite newer typing or navigation. - Persist exact runtime selections, including empty selections. Rebuild pinned assets and verify their admitted digests on a replacement server. - Preserve source access on forks and disconnected copies. Keep the existing workspace policy category for managed skills. - Add bounded conversion, eligibility, and independent verification phases. Load persisted Cloud configuration in the standalone CLI without seeding or changing it. Reject partial current versions against installed inventories; recover only complete saved content or the exact source. Document storage migration blockers and the compatible rollback floor. - Add separate-process PostgreSQL/S3 qualification and update domain, retention, policy, and UI regressions. - Pass the plugin worker manager into Studio execution, admit restricted `skill_test` mode through native Runner contracts, and preserve admitted scope across native and REST tools. Native tests cannot create ordinary tasks, browse company data, or use external tools. - Select the exact durable Studio revision even when unassigned or Beta skills is off. Persist Studio output and audit atomically with native task completion, preserving the result on replay. ## Verification - Final revision `13e8af480b88ba6fea5737bac4bd81104d604792` passed full CI: all 49 workflow jobs completed successfully ([run 41127396098](https://github.com/paperclipai/paperclip/actions/runs/41127396098)). This includes typecheck, test suites, Runner checks, browser checks and build. Greptile review 15 is 5/5 on that same revision, with zero unresolved threads. The PR is ready for review. - Local workspace typecheck and build passed after integrating master. Focused checks passed: 33 execution/completion cases, 52 Runner HTTP cases, 16 managed-content cases, 17 Studio cases, the new real-process CLI regression, map checks and UI token gates. The CLI regression proves inspection does not seed missing Cloud configuration. Server typecheck passed after that fix. - The independent-process PostgreSQL/S3 matrix passed through normal HTTP skill routes. It covers A → B → C replacement, lost commit responses, upload crashes, concurrent saves, binary contents and executable delivery. Metadata listing and warm selected delivery retain zero object downloads. Local timing measurements are recorded; they do not establish hosted latency. - Hosted A → B → C passed on the owned disposable test tenant with external Postgres, S3 and a real remote Paperclip Computer. Predecessor temporary homes were absent. C read and executed B's acknowledged managed save. New runs used repository v1 while v2 was installed. Hashes matched an independently checked exact upstream commit. - Compatible image rollback to `6518b323` passed. Studio retained the unassigned fork's C save. A fresh remote run executed the newer managed content and historical repository pin. The actual shell event confirmed expected hashes, stdout, exit 0 and mode 0555. - Runtime candidate `9200e3fcb4` deployed successfully ([Cloud run 41124147876](https://github.com/paperclipai/paperclip-cloud/actions/runs/41124147876)). The final Studio test delivered only the unassigned fork's exact v3, executed its script and persisted a successful output snapshot. Independent command-event inspection confirmed all three hashes, stdout, exit 0 and mode 0555. All 14 installations were version-backed. Independent verification checked all 25 versions: 23 remote objects, two bundled database snapshots, zero blockers. - The deployed image is `9200e3fcb4`. Later master integration, the client-only delayed-reload draft guard and the CLI bootstrap have separate automated proof. The CLI bootstrap was also exercised against the tenant's stored configuration and storage through the same read-only services. This PR does not claim that the deployed image is the final PR revision. - Live Studio checks exposed worker wiring, admission, access scope, exact selection and completion-publication gaps. They are fixed. Failed attempts remain in the evidence. The full local suite exceeded 80 minutes under host contention and was stopped. Later combined/isolated local attempts also timed out; they are not counted as passes. Full CI covers the integrated suites. Earlier CI found an obsolete restart-scope fixture, which was corrected and passed all 52 HTTP cases. - Reproduction and migration: [`doc/skill-ephemeral-web-servers.md`](doc/skill-ephemeral-web-servers.md). Source revisions, deployments, versions, run IDs, hashes and failure history: [`doc/plans/2026-10-30-ephemeral-web-skills.md`](doc/plans/2026-10-30-ephemeral-web-skills.md). ## Risks - Legacy conversion requires stopped admission and retired filesystem-authoritative writers. Pre-conversion binaries are not supported rollback targets afterward. - Cloud readiness requires remote object storage. Local provider receipts and provider mismatches remain blockers until objects are copied and verified. - Linked local sources retain filesystem semantics and are outside the cloud guarantee. - Cold reads still fetch the existing archive format. Warm runtime preparation retains the verified-cache path. - Bundled reconciliation preserves an installed head. Existing release snapshots remain selectable. An older server must not replace a newer head with its shipped files. - No production volumes, fleet defaults, or general deployment coordination change in this PR. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, code execution, and browser use. The exact backend model ID and context-window size are not exposed in this session. No subagents were used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
6174b4d7c8 |
fix(runner): allow session reset after confirmed task retirement (#18373)
## Thinking Path > - Paperclip manages AI agents and their work. > - Native warm sessions retain a Runner owner between turns. > - Closing that owner must stop execution and preserve valid recovery state. > - A failed suspension can leave a rejected close promise after the task has stopped. > - A session reset then retries that same promise and blocks every fresh session. > - This PR separates task retirement proof from checkpoint reuse for a new session. > - A reset can proceed after exact stop proof, completed handoff, and file collection. ## Linked Issues or Issue Description Refs #18011. That change preserves checkpoint authority during warm preparation. This change handles a stopped owner whose checkpoint cannot be reused. Related: #14695 covers managed files retained with warm processes. A search found no duplicate stopped-owner reset fix. **What happened?** A warm Runner can stop and complete its task handoff while close reports `runner did not durably suspend before checkpoint`. The transport caches that rejected close promise. Core retains the warm owner. Each session reset retries the same promise before it can launch a provider. A controller restart clears the in-memory owner and permits recovery. **Expected behavior** A reset to a new normalized session should release the old owner when its exact task has proved that execution stopped and its handoff completed. Core must collect stopped instruction files before it admits the new session. An unconfirmed owner must continue to block replacement. **Steps to reproduce** 1. Complete a warm native run through a plugin task provider. 2. Change configuration so Core retires the warm owner. 3. Let the task stop and complete, but fail durable suspension proof. Close keeps its rejected promise. 4. Reset the session and start another run on the same task environment. 5. Observe the old close error before any new provider work starts. ## What Changed - Keep a lease-bound retirement receipt in the plugin task owner. It requires `executionStopped: true` and successful task completion after credential handoff. - Accept an unrecoverable suspension error only during retirement for a different normalized session, with that exact owner's receipt and settled Core tool results. - Collect stopped instruction files before releasing the warm owner. Preserve old checkpoint evidence. - Keep same-session, active-owner, incomplete-handoff, unrelated-owner, unresolved Core tool results, and other close-error fences. - Report Core tool settlement on the close error after ingress retirement and drain. Default missing proof to false. Expose the error through the server's Runner shim. Add regression tests and document the recovery boundary. ## Verification - The reset regression fails with the new handling disabled and passes with it enabled. - Both affected suites passed: 1,032 tests. The final 15 focused owner/reset cases pass after adding the tool-settlement guard. - The assembled regression uses the real task-owner and backend binding. Its controlled task provider proves stop and completion. Close returns the same rejected promise on every call. - Negative cases cover incomplete handoff, other close errors, same-session continuation, provider change with the same session ID, unrelated receipts, unresolved Core tool results, and failed file collection. Collection can later succeed without replaying stop or reading the old checkpoint. - Five real-process transport cases pass. They cover an insufficient suspension deadline, the cached rejected promise, late tool completion within and beyond the deadline, interruption, and result-persistence failure. Unsettled callbacks produce `semanticToolsSettled: false`. - `pnpm -r typecheck` passed with the repository's pinned Rust toolchain on PATH. After the final contract change, Runner TypeScript build and server `tsc --noEmit` passed. - The local full-suite run was stopped at the requester's direction. Full regression and build coverage are delegated to GitHub CI. - Full GitHub CI passed on `ef0040d80af679ce6d4a130468e15f75678c1f4c`. Standard Greptile reviewed the same commit: 5/5, zero comments. Apex then reviewed that same commit: 5/5, zero comments. Final checks confirm no unresolved review threads and no merge conflicts. - Initial full CI on `00c002fd5a` failed one scheduled-retry route test at 18/20 cancellations. Controlled cancellation latency reproduced the short poll-budget failure. Before the next cycle, master advanced with #18322, which coordinates that test at the first completed batch. The separate local timeout commit was dropped during rebase. The upstream test passes locally. The rebased run passed 46 targeted owner/reset and scheduled-retry route cases. This PR has no scheduled-retry source or test diff. - No deployed provider run or tenant rollout is claimed by these source tests. ## Risks The exception applies only to a new normalized session and an exact completed plugin task. The old checkpoint remains evidence and cannot authorize reuse through this path. The original suspension failure can remain visible on its run. This fix prevents repeated startup failures after a reset; it does not hide unconfirmed cleanup or make every suspension succeed. Missing task receipts and failed file collection continue to block replacement. No schema migration, plugin protocol change, Rust Runner binary change, or new polling is required. Tests use a controlled task boundary; they do not certify every external provider implementation. ## Model Used OpenAI Codex, GPT-6 family, with tool use and local code execution. The exact runtime model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal issue or instance details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] Greptile Apex is 5/5 with no unresolved findings - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
294d8ca1c2 |
perf(heartbeat): schedule retry promotion from durable deadlines
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f53db0c70f |
perf(server): overlap OpenCode setup and trace reply recovery (#18367)
Overlap eligible OpenCode GitHub and assigned MCP preparation. Trace deferred reply discovery and recovery, and forward the native GitHub transport recorder. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
9f524ec1a7 |
fix: persist native Runner usage before transport acknowledgement (#18354)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent execution must preserve usage so Paperclip can price work and enforce budgets. > - Remote Runner execution can outlive its web receiver. > - The receiver previously acknowledged Runner events before its later usage recorder saved the accounting receipt. > - This pull request saves required native usage receipts before that transport acknowledgement. > - A replacement receiver can accept the same retained report and create one ledger charge. ## Linked Issues or Issue Description **What happened?** Native remote Runner usage could leave its durable outbox before the web receiver saved its accounting receipt. The later recorder required a local receipt directory. A replacement web receiver could lose the only remaining report when the database write failed. **Expected behavior** The Runner retains unaccepted structured usage. Postgres owns accepted receipts. Repeated delivery preserves one receipt for each source report and one charge for the run. Missing usage keeps accounting pending. **Steps to reproduce** 1. Start a supported built-in native remote Runner. 2. Make the receipt database insert fail after the provider reports usage. 3. Replace the web receiver and its local directory. 4. Reconnect the same Runner and inspect the saved receipt and ledger. **Paperclip version or commit** The initial investigation used `ebd05e9f4c88c66af7473db86bbaacfd1544aa0a`. **Deployment mode** A web server with a remote native Runner and shared Postgres. Related work: #18327 owns output logs and provider traces. This change has no dependency on its S3 export path. #18031 covers interrupted Codex receipt recovery. #18005 covers legacy spool reads. #17516 covers Codex stop-boundary retention. ## What Changed - Add native producer receipt delivery through the existing Runner callback before transport acknowledgement. - Reuse `run_usage_receipts`, stable source sequences, and deterministic report identities. Reject conflicting replay and retry a lost commit response with the same receipt. - Restore per-turn usage from structured receipt facts. Preserve provider turns that share one PRP envelope turn. - Bind retained warm transport callbacks to the current run authority epoch, allowing old attachment control acknowledgements while preserving accounting ownership. - Persist Codex session baselines and cumulative totals in the existing provider checkpoint. Mark qualified whole-run deltas, preserve them across resume/restart/attachment, and leave old unmarked reports incomplete and unpriced. - Normalize the bounded final usage tail before the terminal event. Retain later increases for explicit reconciliation, block further attachment, and fail an affected successor without changing its baseline. - Complete runs from the accepted immutable receipt and its exact saved rate-card amount; preserve existing live-budget enforcement. - Skip the local receipt recorder for supported built-in native remote Runner execution. Keep local, legacy, and external Dot/Muse behavior unchanged. - Keep pricing, reservations, ledger settlement, source fencing, and budget holds with their existing owners. - Put the actual Runner integration test in the existing Rust-cached native test lane. Build before opening Postgres and give cold compilation its own bounded budget. ## Verification - Fresh Greptile review is 5/5 on `e7376a50c8162770da1d581ecf25c8cffac64520`, with no unresolved findings. Parent and independent peer review approved the final changes. - Real PostgreSQL 18.6 and a real Rust Runner exercise failed receipt insertion, receiver-directory loss, replay from the same producer, and exactly one 25-cent ledger charge. The actual accounting spool path stays absent. Optional logs/traces are disabled. The provider is controlled; no paid request is made and the fixture never pre-saves the expected receipt. - Producer checks passed 32 usage/provider tests,37 terminal controls, corrected completion, all 3 previously failing exit cases, and both full transport goal-continuation cases. Rejected active-goal completion retains its 120-token tail before failure. Baseline 100→successor150→33 duplicate old snapshots→old120 retains explicit incomplete reconciliation evidence and positive budget exposure. - Heartbeat executor checks passed 982/982. PostgreSQL tests preserve exact 1.002 USD pricing, the 100.2-cent ledger amount, budget holds, source takeover, partial/final receipts, cost-only corrections, equal rate-card replay, and stale-completion interleaving. Newer incomplete usage remains pending rather than settling an older snapshot. - Runtime replay and strict usage-schema checks passed 2/2; generated contracts and Rust formatting pass. Test-partition contracts passed 32/32. - [Final combined-head evidence](https://github.com/paperclipai/paperclip/pull/18354#issuecomment-6531526158) records full workspace typecheck/build and 41/41 accounting tests against all five cloud PR heads, including this final accounting commit. It includes commands and evidence limits. - Exact-head CI is green: 53successful checks, 6intentional skips, and the legacy Snyk status passed. Typecheck, build, Runner, server, browser, and aggregate gates pass. Runner Vitest1/2 initially failed an unchanged mocked Antigravity login fixture; its focused case passed locally, then one unchanged job retry passed 131 files/1995 tests, including all 11 Antigravity tests. The initial failed attempt remains recorded in CI. Initial local attempts exposed the host SysV semaphore limit and corrected harness path, IPC readiness, and queue-counter defects. A temporary test loader selected an existing POSIX-semaphore PostgreSQL 18.6 build without changing repository dependencies. An optional expanded debug provider run was interrupted at its unrelated 8192-turn rollover stress case; no full-file local pass is claimed. Required failure cases were run directly, and the full Runner CI remains required. ## Risks - The guarantee covers built-in native remote Runner execution. Local native execution, legacy adapters, external Dot/Muse execution, and server-originated model calls retain their existing contracts. - This does not provide traffic switching or general web-server failover. The existing controller identity, checkpoint, and generation gates still apply. - The receipt JSON gains optional structured producer facts. No schema migration or accounting S3 spool is added. Text output and raw trace payloads are excluded. - Codex accounting authority requires a Runner artifact built from this provider baseline implementation and qualified by digest. Update receivers and strict usage-schema readers before releasing that Runner; PRP does not negotiate this payload field. Existing unmarked Codex reports remain pending/unpriced. No deployed Runner artifact is claimed fixed. - Usage arriving beyond the bounded terminal tail may require manual reconciliation or a fresh session. Its evidence remains in the existing producer checkpoint/outbox; no adjustment ledger or recurring scan is added. ## Model Used OpenAI `gpt-6.1-sol` assisted with the implementation, tests, and documentation through code execution and repository tools. The context window size is unavailable. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |