mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-01 15:35:12 +01:00
110e82d2a3d22ee6af67f525ddad5eb6c2dccd63
3050
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
110e82d2a3 |
Carry landed master through 18399
Preserve original owner patch sequences and all new master semantics. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
9a79769c72 |
Merge landed cards and current native admission into retention
Preserve the original retention owner and producers alongside current native provider-pack admission, route execution-lock protection, canonical work-signal imports, and master CI/UI changes. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b6b57079b9 |
fix: evaluate status cards on committed changes and deadlines (#18401)
Reviewed and verified at
|
||
|
DottaandPaperclip
|
c0db98d007 |
Carry qualified fixture wake reason through 18399
Normal parent merge; only the qualified queued-run fixture changes. Preserve every other owned and upstream blob. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8fc9220ff9 |
Carry qualified fixture wake reason through 18388
Normal parent merge; only the qualified queued-run fixture changes. Preserve every other owned and upstream blob. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
21795e222a |
test: specify assigned wake reason in held-read fixture
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
be92e67ef5 |
Carry qualified queued-run fixture repair through 18399
Normal parent merge; only the qualified queued-run fixture changes. Preserve every other owned and upstream blob. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
2c486ca6e5 |
Carry qualified queued-run fixture repair through 18388
Normal parent merge; only the qualified queued-run fixture changes. Preserve every other owned and upstream blob. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f9bf5ab36d |
test: await admitted legacy heartbeat executions
Use tracked execution settlement and explicit adapter-entry barriers instead of independent short polls. Add a committed-read control proving the pre-registration drain gap while preserving durable state and concurrency assertions. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e40de0c9e8 |
perf(ci): refresh test weights and narrow graph dependencies (#18464)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Contributors need fast, reliable CI feedback. > - Diff-aware CI selects test files and partitions them into runner jobs. > - Stale cost estimates put several slow suites on the same runner. Broad imports also couple small tests to large modules. > - This PR refreshes test costs, runs independent graph discovery in parallel, and narrows three dependency boundaries. > - Selection safeguards and CODEOWNER-only AWS routing stay in force. ## Linked Issues or Issue Description Refs #18398. This follows the audit of [PR #18427's verification run](https://github.com/paperclipai/paperclip/actions/runs/41169978784). The audit found 403 selected general-server files without timing weights. Three suites measured at 232s, 122s and 116s shared one shard because each received a roughly 2s fallback weight. Local graph discovery across four projects took 43.7s serially and 21.7s with two workers. A reverse-order repeat took 37.7s serially and 22.6s with two workers. Both comparisons produced identical selection results. This is a local measurement, not a claim about production CI speed. ## What Changed - Refresh general-server costs for 1,285 of 1,309 current files and serialized costs for all 163 files from two successful protected-master Cloud readiness runs. Retain prior costs for unmeasured files, including dedicated chat and native-runner suites. - Record passing server file costs, including collection, setup and environment preparation. A weekly hosted workflow validates trusted source runs and opens a timing-only PR. It tests shard coverage, pauses while a refresh PR is open, and never merges automatically. - Discover at most two project graphs concurrently in isolated, heap-limited Node processes. Cache repeated inventory reads within one selection. Worker errors and timeouts retain the full task plan. - Fetch only the merge commit and parents for the selector. A real shallow-clone regression proves the complete diff still includes more than 300 files, renames and deletions. - Share the identical log viewer and run details between both AgentDetail pages. The viewer test imports the shared implementation and runs each of its 13 distinct cases once. - Extract the pure API error class while preserving the API client's public exports. Import delivery work signals through their existing narrow DB export. ## Verification - PASS: `pnpm -r typecheck` and `pnpm build`. - PASS: 284 CI selection, timing, shallow-diff, sharding and routing tests. - PASS: 74 further cache, routing and graph-proof tests. - PASS: 94 targeted UI assertions across the viewer, run invocation, API errors/client and progress helpers. All 76 assertions in the 12 server suites made independently selectable also pass. - PASS: `pnpm check:token-gates`, module-boundary checks, YAML parsing and `git diff --check`. - PASS: [Full PR CI and required `ci / verify` aggregate](https://github.com/paperclipai/paperclip/actions/runs/41206683345), including all 12 general-server shards, all nine serialized shards, workspace groups, chat suites, Runner lanes and build checks. - PASS: [GitHub hosted real-workspace selector and cache/routing qualification](https://github.com/paperclipai/paperclip/actions/runs/41206683205). - PASS: Both Greptile and Paperclip reviews are 5/5 with no actionable findings or inline comments on `7dc65c51b8`. The PR has no merge conflicts. - PASS: The production refresh parser accepted all 107 structured file-cost records from a real successful server CI job, including setup/loading cost. - Local `pnpm test:run` completed all six chat shards successfully. The duplicate monolithic local run was stopped after the complete remote suite passed; it was not completed locally. The focused log-viewer command improved from 18.84s (26 duplicate variant cases) to 8.49s (all 13 distinct behavior cases). The dependency audit confirms 12 server files and the API-error test no longer need conservative always-run retention. Repartitioning the audited PR with the new costs changes the slowest shard from 810s to 591s of summed historical test execution (1,124 of 1,142 files measured). This is a workload simulation, not measured new CI wall time. ## Risks - Two graph processes use more peak memory. Each has a 2 GiB heap limit and a three-minute timeout. Any failure preserves the original task plan. Real workspace qualification runs in GitHub CI. - Historical logs contain test/hook costs only. New reporter records add loading/setup costs after merge. The manifests identify measurement coverage and source runs. These weights change placement, never permission to skip tests. - Runner load adds timing noise. The refresh uses medians and requires at least 90% measurement coverage in each lane. - GitHub may require approval to run checks on a PR created with `GITHUB_TOKEN`. Refreshes remain reviewable and do not merge themselves. - The shared viewer is a mechanical extraction. Both pages retain their existing exports, and the behavior assertions remain covered. AWS runner authorization is unchanged. ## Model Used OpenAI Codex, GPT-6, with repository inspection, reasoning, code execution and GitHub tools. The exact deployment ID and context-window size were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused checks and six chat shards; complete suite passed in CI as detailed above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b7f1c3769c |
feat(native): record verified provider-pack session admission (#18469)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Remote native sessions can use an installed pack or a retained earlier pack. > - The current app revision does not identify the pack that a session admitted. > - Core already verifies the selected pack and binds its launch artifacts. > - Normal run records lack a closed receipt for that admitted selection. > - This pull request records that receipt after session admission and attachment. > - Operators can compare the actual Core admission identity with their expected pack. ## Linked Issues or Issue Description Refs #17749 for protected provider-pack selection and #18343 for retained controller recovery. The related OpenCode producer repair is #18456. Those changes do not publish the closed run receipt added here. A search found no existing `nativeProviderPackAdmission` PR. **What happened?** Core can admit a compatible installed provider pack with a source revision that differs from the current app. A retained session can also keep its original pack. The normal run record does not show the admitted pack digest, proxy digest, and source revision. An expected launch path or current app revision cannot fill that evidence gap. **Expected behavior** The run has a closed, server-owned receipt after actual session admission and attachment. The first receipt remains immutable history. A separate current-proof state verifies only the actual current admission epoch. Retained recovery needs positive original provenance, and missing legacy evidence stays unknown. **Steps to reproduce** 1. Select and verify an installed pack whose digest differs from the current app pack. 2. Open the native session successfully. 3. Inspect the normal run profile for a bound admission receipt. **Paperclip version or commit** The original selected base was `8bcc5a01edd5d87540401521e6f30d3df43e1848`. The branch now includes exact actual prerequisite parent `3ee4c5b6138da82d0b1f51aa010540561bf43c22`; the PR diff against it is the five receipt, executor, test and documentation files. **Deployment mode** Core remote native execution with provider-pack selection and retained session attachment. ## What Changed - Retain the verified selected pack identity only after a backend returns the matching admitted session. - Publish `runnerProfileJson.nativeProviderPackAdmission` from the session callback after attachment succeeds. - Mark the receipt with `stage: session_admitted` and `authority: core_verified_selection`. - Store bound company, agent, run, session, runner, logical `runnerEnvironmentLeaseId`, pack digest, OpenCode proxy digest, and source revision fields. Retained attachments identify `inheritedFromRunId`. The logical Runner scope is distinct from the physical environment lease. - Lock the current coordinator claim before the profile write. Check owner, attempt, generation, controller boot, lease expiry, phase, and absence of an accepted result. Merge JSON fields without replacing other profile data. - Restore retained metadata only from the exact prior closed Core receipt, positively verified prior state and matching retained binding. Capture same-run prior provenance from the locked claim row before invalidation; cross-run recovery reads only the exact source. Legacy absence remains unknown while recovery continues. - Preserve the first receipt as history. Atomically mark a new opaque admission epoch pending in the existing mandatory claim transaction. Publish current verification only for the matching claim and epoch after actual attachment; changed or unsupported admission becomes unknown without blocking authorized execution. - Preserve synchronous session publication and quarantine ordering before the best-effort receipt write. Fresh replacement selects its newly verified pack; a captured retained-process flag cannot relabel the replacement with an older pack. Preserve the existing private pack authority even when a local dirty revision is ineligible for the closed receipt. - Add focused admission, recovery, identity, PostgreSQL ownership, lock, and JSON-preservation tests. Document the receipt contract. ## Verification - RED: the actual managed-backend path verified and admitted an installed pack but had no closed metadata. Failed open published no evidence. Additional controls fail when retained recovery invents missing history or dirty-revision eligibility gates the existing private authority. - RED calibration: removing mandatory pending invalidation fails the actual pre-admission control; reusing the retained flag for fresh replacement falsely restores the old pack. Removing the epoch fence fails the real PostgreSQL stale-callback case; reintroducing a receipt-specific execution error fails both authorized Runner/logical-lease replacement controls. - GREEN: 49 focused Core tests, all 33 admission identity/transaction tests (20 pure and 13 PostgreSQL), and all 23 existing private-authority tests passed on exact integrated head `165a9096cf3e0d34d2db1e6b52960ad4ac0ad132`. No all-skipped run is counted. - Final test-only head `11ac34baf00552a5d322b5f5f1a8e491859cacec` adds the positive executor-callback SQL-writer wiring control, exact session/identity/claim/epoch assertions, profile preservation, and null-session zero-write check. Removing only the production writer call makes it fail on a missing receipt. All70 focused identity/Core tests pass with this closure; production code, the13 PostgreSQL tests and the23 private-authority tests are byte-identical to the qualified165 source (106 distinct passing controls across the combined proof). - The PostgreSQL tests used an owned loopback-only cached Postgres 17 container with the supported `PAPERCLIP_TEST_POSTGRES_ADMIN_URL` helper and aliases to current DB/shared source. The container was removed afterward. They cover immutable history, JSON preservation, stale claims and epochs, expired leases, terminal runs, changed/unsupported replacements, and a failed write retaining pending proof. - Historical pushed `f7f12fe` CI executed all 24 earlier admission tests, including nine PostgreSQL cases, in server5. Its warm-quarantine failure exposed the new hook-ordering issue fixed here. The same run also encountered a six-character UUID fixture-prefix collision and the independently repaired cached issue-link browser selector; those historical failures are not claimed as green. - Node 24.22.2 and Vitest 5.0.3. The credential-free normal QA runtime import preflight passes with networking disabled. Map contracts, token gates and `git diff --check` pass. - Local TypeScript validation still sees stale borrowed Runner/shared declaration mismatches outside the changed ranges. No diagnostics remain in the new helper or changed test/hook ranges; clean current-head CI is required. - Repository-wide typecheck, full tests and build were not completed locally. A full local executor attempt reached this host's unsupported native PostgreSQL listener and was interrupted; its result is not counted. Canonical run `41214512239`, attempt 2, is successful on exact final head `11ac34baf00552a5d322b5f5f1a8e491859cacec`. All 59 checks are terminal: 55 success, 3 skipped and 1 neutral; none failed or pending. The actual Linux suites execute all 33 admission tests (including 13 PostgreSQL cases) and all 1,022 executor tests successfully. The [original five failures and bounded unchanged-source controls](https://github.com/paperclipai/paperclip/pull/18469#issuecomment-6545593767) remain preserved. One failed-only request was made; GitHub expanded the reusable workflow call and dependents. No second request was made. Fresh exact-head Greptile review is 5/5, all review threads are resolved, and the PR is mergeable. - These local results claim no live provider admission, rollout or recovery. ## Risks - The authority is Core's verified selection plus successful session admission. It is not an independent provider statement or continuous runtime measurement. - Receipt fields cannot grant execution or replace the existing provider, process, lease, workspace, or recovery checks. - Legacy missing receipts and unavailable database reads remain unknown. The current release cannot label an older admitted pack. - The closed receipt requires a clean, 40-character hexadecimal source revision. Verified local `-dirty` packs still execute and retain their existing private recovery authority; their closed admission proof stays `unknown`, so it cannot certify an immutable source revision. - The first receipt is historical. Consumers must require current state `verified`; pending or unknown cannot certify the current pack. Changed Runner scope or pack evidence does not add execution policy. - A receipt save failure leaves the atomically invalidated state pending. The existing artifact authority, cancellation, workspace and process checks remain responsible for execution safety. - The receipt adds no manifest, environment, credential, or filesystem path to the run record. ## Model Used - OpenAI GPT-6 through Codex, with reasoning, code editing, and tool use. The session does not expose an exact model build ID or context-window capacity. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
51ea624344 |
fix(server): reject a native run's own task status change (#18411)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Native runs (the Paperclip Runner) report their outcome with
`paperclip_finish` or `paperclip_block`, and the status arbiter then
sets the task status
> - Every runner tool, `paperclip_finish` included, requires that the
task's `executionRunId` is the current run
> - When the run sends its own status change through `PATCH
/api/issues/{id}` (for example with `call_api`), the issue update clears
`executionRunId` and `checkoutRunId`, so the run removes its own
execution lock
> - After that, every tool call fails, the run cannot report, same-run
recovery spends its retries, and the task goes to `blocked`
> - This pull request rejects that status change with a 409 before it
commits, and the message tells the run to report with `paperclip_finish`
or `paperclip_block`
> - The benefit is that the run corrects itself in the same turn, and
the task does not get stuck
## Linked Issues or Issue Description
No public issue exists. This is the bug description.
**What happened?**
An agent on the Paperclip Runner followed task instructions that said
"set the task to `in_review` and keep it assigned to yourself" while it
waited on a scheduled monitor. It sent `call_api` with `PATCH
/api/issues/{id}` and `{ "status": "in_review" }`. The server applied
the change and cleared the execution lock. Then:
1. `get_task`, `set_task_monitor`, and `paperclip_finish` all failed the
run binding check. The provider saw only "Paperclip semantic operation
failed", or the monitor-wait message from `paperclip_finish`.
2. The `call_api` receipt for the PATCH showed `status: null, outcome:
unknown`, because the authority check after the response failed when the
lock was gone.
3. The session ended with no semantic result. Exact-checkpoint resume
ran out of retries (`native_session_retry_exhausted`), and recovery
would not replay the run because the mutation outcome was unknown. The
task went to `blocked`.
This occurred on each check where the agent tried the status change.
Runs that did not try it completed normally.
**Expected behavior**
A native run that holds the task's execution lock cannot remove its own
lock. It gets a clear, actionable error and reports its outcome through
`paperclip_finish` or `paperclip_block`.
**Steps to reproduce**
1. Assign a task to an agent that uses the Paperclip Runner (native
runtime).
2. While the run executes, have it send `call_api` with `PATCH
/api/issues/{id}` and `{ "status": "in_review" }` for its own task.
3. Look at the next tool calls of the run. They fail, `paperclip_finish`
cannot complete, and recovery blocks the task.
**Paperclip version or commit**
`master` at
|
||
|
DottaandPaperclip
|
4a685a8ca4 |
Carry qualified card interval label repair through 18399
Normal parent merge; only the two qualified card interval paths change. Preserve every other owned and upstream blob. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
674036e758 |
Carry qualified card interval label repair through 18388
Normal parent merge; only the two qualified card interval paths change. Preserve every other owned and upstream blob. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8c99d86e16 |
Label interval summary writeback with its refresh deadline
Keep membership provenance from describing an interval-owned evaluation deadline; retain existing cadence and source handling. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
bd78176b40 |
Carry qualified card search repair through 18399
Normal parent merge; only the five qualified card/search paths change. Preserve every other owned and upstream blob. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d2ef6c81d0 |
Carry qualified card search repair through 18388
Normal parent merge; only the five qualified card/search paths change. Preserve every other owned and upstream blob. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e0986ac344 |
Paginate watched status-card issues before unrelated search results
Add an internal result-kind constraint owned by company search, preserving all/comments/documents task matching and default mixed search. Apply it to card snapshots and preview without unrelated entity producer fanout. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
3dc77ca942 |
Carry final qualified cards repair through 18399
Normal parent merge; only the three qualified status-card paths change. Preserve every other owned and upstream blob. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
42ece47d98 |
Carry final qualified cards repair through 18388
Normal parent merge; only the three qualified status-card paths change. Preserve every other owned and upstream blob. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
9a9668425b |
Preserve completed shared status-card refresh generations
Guard stale cleanup with the durable generation ledger and a transaction-bound canonical cancellation. Coalesce the original completed manual request, retain future manual work, and preserve one-connection and outer-commit effects. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
3ee4c5b613 |
Make saved execution receipt recovery react to durable work (#18409)
Reviewed and verified at
|
||
|
DottaandPaperclip
|
42fd568cf8 |
Drive issue recovery from committed changes and exact deadlines (#18402)
Reviewed and verified at
|
||
|
DottaandPaperclip
|
508115ef70 |
Carry qualified cards repair through routine status work
Preserve the complete routine status slice and additive retention ownership. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8e414f54f4 |
Merge final cards revision ownership and parent acceptance
Co-Authored-By: Paperclip <noreply@paperclip.ing> * fix/sleep-merge-18401: Scope status-card refresh tasks to their claim revision test(chat): retain completion identity before link metadata loads test(chat): locate completion through its rendered task link fix(chat): recover committed native Agent Chat replies (#18465) perf(heartbeat): drive retry promotion from durable deadlines (#18387) fix(ui): make the Coordinator composer footer opaque (#18455) fix(testops): verify unexpanded result job runs (#18462) fix(runner): preserve OpenCode written-answer defaults (#18456) fix(native): fence retries when replaying an audited Stop (#18457) test: await async outcomes and drive saved deadlines deterministically (#18416) |
||
|
DottaandPaperclip
|
37bbb20cda |
Merge final deferred execution acceptance and locator proof
Co-Authored-By: Paperclip <noreply@paperclip.ing> * commit 'd6d0b67e8a7574ee63271bbc49e6b207a3065e80': test(chat): retain completion identity before link metadata loads test(chat): locate completion through its rendered task link fix(chat): recover committed native Agent Chat replies (#18465) perf(heartbeat): drive retry promotion from durable deadlines (#18387) fix(ui): make the Coordinator composer footer opaque (#18455) fix(testops): verify unexpanded result job runs (#18462) fix(runner): preserve OpenCode written-answer defaults (#18456) fix(native): fence retries when replaying an audited Stop (#18457) test: await async outcomes and drive saved deadlines deterministically (#18416) |
||
|
DottaandPaperclip
|
1d800abd49 |
Scope status-card refresh tasks to their claim revision
Keep concurrent refreshes of one revision deduplicated while preventing a superseded refresh from cancelling a newer revision task before its claim. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d6d0b67e8a |
Merge current master native chat recovery into deferred receipts
Co-Authored-By: Paperclip <noreply@paperclip.ing> * commit '11b49125db5ea1b274e69faf9d1abe057cdbb551': fix(chat): recover committed native Agent Chat replies (#18465) perf(heartbeat): drive retry promotion from durable deadlines (#18387) fix(ui): make the Coordinator composer footer opaque (#18455) fix(testops): verify unexpanded result job runs (#18462) fix(runner): preserve OpenCode written-answer defaults (#18456) fix(native): fence retries when replaying an audited Stop (#18457) test: await async outcomes and drive saved deadlines deterministically (#18416) |
||
|
DottaandPaperclip
|
11b49125db |
Merge current master into scoped issue recovery work
Retain scoped recovery registration and producer gates while preserving landed audited-Stop retry fencing and committed native Agent Chat response recovery. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
12b0cc6304 |
fix(chat): recover committed native Agent Chat replies (#18465)
## Thinking Path > - Paperclip manages agent work and keeps its results available to users. > - Agent Chat needs a visible reply before the conversation can wait for more input. > - Native result commitment and reply presentation are separate obligations. > - A completed conversation can retain a valid final message without a saved reply. > - Existing reply recovery excludes these completed conversation turns. > - This pull request repairs that persistence gap through the existing recovery worker. ## Linked Issues or Issue Description **What happened?** A native Agent Chat turn completed successfully and saved its workspace. Its accepted result and final assistant message were durable, but no reply appeared in the conversation. The chat stayed active. The deployment hiring journey in #18351 then reached its deadline while waiting for the first reply. **Expected behavior** Recover the exact accepted turn's final reply once, then use normal conversation settlement. Keep later user input active. Do not run the agent again or replace a selected or deleted reply. **Steps to reproduce** 1. Create an active conversation with one admitted native turn. 2. Commit an accepted DONE result and a final-channel assistant message for that exact run, session and turn. 3. Finish workspace finalization before live heartbeat reply presentation. Leave no saved reply. 4. Run normal response recovery. Before this fix it finds no conversation candidate and leaves the chat active. **Paperclip version or commit** Reproduced with actual services at frozen master `8bcc5a01edd5d87540401521e6f30d3df43e1848`. The production predicate also exists in the deployed revision that exposed the failure. Related work: #18351. The broader receipt and recovery work in #18118 and #18409 does not touch these source files. ## What Changed - Discover unanswered committed Agent Chat turns through the existing native response presentation worker. - Recheck company, agent, responsible user, current execution and conversation generation under the normal transaction locks. - Require the accepted result digest and an exact final message from the same native run, session and accepted turn. Keep external publication authority separate. - Commit the reply and presentation receipt before normal conversation settlement. Later input stays active. Recovery does not repeat agent work. - Preserve skip decisions and selected or deleted replies. Add database controls for foreign identity, stale sessions, partial commitment, duplicates and ordinary internal task behavior. - Document the recovery boundary in the developer guide. ## Verification - Real-service RED/GREEN uses all 418 canonical migrations in isolated PGlite and the exact new regression fixture. Before the fix: zero candidates, no reply, recovery returns false and the chat stays active. After the fix: one candidate, one exact reply, recovery returns true and the chat waits. A repeat returns false and creates no second reply. - All 20 canonical regression cases pass locally through a fixture-only PGlite bridge. Only local database provisioning and `createDb` selection differ from the normal embedded PostgreSQL fixture. Production services, SQL queries, transactions, protocol validation and assertions remain real. Linux embedded PostgreSQL CI is required before merge. - Focused current-source no-emit typing passes for both production modules and the new regression file, with normal Express augmentation and source aliases. The broader local presentation/ownership and conversation-settlement PGlite bridge run was interrupted before it reported results; it is not counted as a pass. No repository-wide local build is claimed from borrowed workspace dependencies. Clean-build CI and canonical PostgreSQL consumer tests remain required. - `git diff --check` and the scoped source/test secret scan pass. No captured run data, credentials or private target identities are committed. - No live journey retry, deployment, configuration change or test-plan admission was performed. The original failed journey remains failed; this production fix still needs deployed qualification through the unchanged journey. - Current-head [canonical Linux CI](https://github.com/paperclipai/paperclip/actions/runs/41207018345) passed all 57 terminal checks with no failure or pending check. The new 20-case database regression actually passed in [server shard 8](https://github.com/paperclipai/paperclip/actions/runs/41207018345/job/124555006090), at 13:59:25 UTC (2,643 ms). Typecheck, build and the existing consumer lanes passed. Fresh [Greptile review](https://github.com/paperclipai/paperclip/pull/18465#issuecomment-6544208002) on exact `f32a432e10b377ae6277d6f728995d5fedcf9d7b` is 5/5; the review-thread inventory is empty with no pagination. No live deployment or journey pass is claimed by these checks. ## Risks Eligible active conversations can gain a previously missing reply. Recovery accepts only the exact saved final message under its accepted result and current conversation identity. It does not run a model, repeat hiring or change a prior test grade. Reply persistence precedes waiting-state settlement; existing conversation recovery retains ownership of a later interrupted settlement. No schema or public API changes are included. ## Model Used OpenAI Codex, GPT-6 family. The session does not expose a more specific backend model ID or fixed context window. Used reasoning, repository tools, TypeScript and real database test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
63c77102a8 |
Merge qualified retention parent into routine status work
Preserve routine status producers alongside current native, security, retention, and deferred execution owners. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
baf651c8d0 |
perf(heartbeat): drive retry promotion from durable deadlines (#18387)
Reviewed and verified at
|
||
|
DottaandPaperclip
|
18c2fc767d |
Merge final saved-receipt test qualification through cards
Carry only the qualified C1 fixture update; preserve retention and migration runtime inputs. Co-Authored-By: Paperclip <noreply@paperclip.ing> * commit '8179a607ea2ec4858c29c8ae1f29ea196dbb48b8': test(server): observe restored Retry before draining unrelated work |
||
|
DottaandPaperclip
|
8179a607ea |
Merge qualified saved-receipt readiness controls through the recovery parent
Carry only the final C1 test fixture and unrelated-process regression. Preserve all status-card, migration and runtime inputs. Co-Authored-By: Paperclip <noreply@paperclip.ing> * commit '939b90abe3f455762185110e42918ae379af6e64': test(server): observe restored Retry before draining unrelated work |
||
|
DottaandPaperclip
|
939b90abe3 |
test(server): observe restored Retry before draining unrelated work
Prove a completed saved Retry remains observable while another real process holds the global drain open. Preserve exact target completion, stdout, execution count, deadline and final quiet assertions, and always release the held provider before teardown. Align two inherited fixture literals with current types. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
aab54d5bec |
Merge current master and generated card schema into attention retention
Preserve master attention, issue, native cleanup and tool authority changes alongside the exact reviewed retention patch. Keep bounded blocker facts, finalization precedence, source locks, archive eligibility and producer intent unchanged. Publication remains dependent on the parent qualification. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
56f8ff7bd5 |
Merge current recovery and regenerate status-card migration after master 0419
Preserve the published concurrent comment uniqueness and read-state backfill migration. Regenerate the combined schema as migration0420 with the repository workflow and retain five snapshots. Add both upgrade-path controls without changing status-card execution policy. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
3549fad1dc |
Merge commit '0b4e48be9b881484dc5c48225a3b501595329ce2' into codex/sleep-merge-18409
* commit '0b4e48be9b881484dc5c48225a3b501595329ce2': fix(ui): retain startup autosave through session recovery (#18412) perf(server): recover independent native replies concurrently (#18421) perf(chat): scope task questions and batch inbox queue checks (#18422) fix(native): recover controllers across webserver replacement (#18343) fix(ui): restore task document authoring and add persistence journey (#18356) fix(tools): stabilize tied profile entry order (#18427) fix(adapter-utils): tolerate entries removed during workspace snapshots (#18426) test(runner): qualify Kimi Code lifecycle and recovery (#18029) feat(runner): deliver native Kimi Code (#18391) Preserve selected Google services when setup resumes (#18048) feat(runner): add pinned Kimi Code native runtime (#18028) fix(ui): show agent-created subtasks inline in the task feed (#18320) fix(evals): preserve opaque TestOps parameters (#18394) feat(chat): deliver proactive task messages to agent chats (#18374) fix(ci): track repository inputs across test projects (#18398) chore(lockfile): refresh pnpm-lock.yaml (#18414) fix(runner): build on Windows and add a Windows CI lane (#18250) |
||
|
DottaandPaperclip
|
12d8c3043e |
fix(native): fence retries when replaying an audited Stop (#18457)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native runs keep Stop intent and retry state in durable records. > - A provider error can leave a failed retry with an earlier acknowledged Stop. > - The replay path returned the existing receipt before it closed that retry. > - This pull request applies the existing retry fence to the validated replay. > - The same Stop can settle without another provider cancellation or a new receipt. ## Linked Issues or Issue Description Refs #14733 for the related caller-bound Stop contract. This change covers a later retry that already has a Stop acknowledgement. The older CI-only #14743 is not a production delivery PR. **What happened?** A failed native run can have a retryable coordinator and an earlier audited Stop. A repeat Stop returns the old acknowledgement without moving the coordinator to `terminal_failure`. The final failed-to-cancelled transition requires that fence, so the Stop cannot settle through that path. **Expected behavior** A valid replay closes the same retry and keeps the original intent and acknowledgement. It does not send a second provider cancellation. Physical stop and workspace cleanup still require their own evidence. **Steps to reproduce** 1. Store a failed native run with a `retryable_failure` coordinator. 2. Store a matching run-scoped Stop with a dispatched acknowledgement and audit IDs. 3. Call `cancelNativeSession` for that same run and inspect the coordinator. **Paperclip version or commit** Reproduced on public master `f2db909393de124caf6a5380a52d4e0e1d621661`. **Deployment mode** Core native execution. The regression uses the real cancellation executor with controlled persistence and provider boundaries. ## What Changed - Reuse the existing guarded coordinator fence for a validated Stop replay. - Test repeated replay, unchanged receipts, one fence, and no second provider cancellation. - Reject foreign company, run, issue, scope, effect, and missing-audit receipts before any write. - Check that an acknowledged Stop with an unconfirmed remote lease still fails the physical-stop guard. - Document the separate retry fence and physical-cleanup requirements. ## Verification - RED on the selected public base: the new executor case returned success but left the coordinator in `retryable_failure` with a future retry. One case executed and failed for that assertion. - GREEN: 62 focused tests passed across `native-session-executor.test.ts` and `native-cancellation-request.test.ts`. These include ownership, caller conflict, changed-phase, and receipt controls. - Command from `server/`: `node ../node_modules/vitest/vitest.mjs run src/services/native-runtime/native-session-executor.test.ts src/services/native-runtime/native-cancellation-request.test.ts -t 'native session cancellation|caller-bound native cancellation|failed native retry caller admission'`. - Canonical Linux CI passed all 1,018 executor cases and all 32 caller-cancellation cases. The exact file completion lines are in [server shard 2](https://github.com/paperclipai/paperclip/actions/runs/41203989386/job/124545366271) and [server shard 10](https://github.com/paperclipai/paperclip/actions/runs/41203989386/job/124545366219). - Node 24.22.2 and Vitest 5.0.3. Map contract checks and `git diff --check` pass. - Local dependencies are linked from an existing checkout. Full local TypeScript validation reports older Runner/shared declarations that lack current Kimi and continuity APIs. The changed ranges have no diagnostics. The canonical clean CI Typecheck + Release Registry gate passed. - A broader two-file run was stopped when an unchanged integration case tried to start embedded Postgres and hit the host listen restriction. It is not a passing full-suite result. No host settings or other services were changed. - Repository-wide `pnpm -r typecheck`, `pnpm test:run`, and `pnpm build` were not completed locally. Current-head CI passed against clean source and dependencies. - Fresh [Greptile review](https://github.com/paperclipai/paperclip/pull/18457#issuecomment-6543772487) on `dbc1c1a5b71450194d568725592969660864cb5a`: 5/5 with no actionable findings. The review-thread inventory has no threads and no pagination. Full [current-head CI](https://github.com/paperclipai/paperclip/actions/runs/41203989386) passed: 59 terminal checks, 55 successful, 3 skipped, and 1 neutral. No failure or pending check remains. Merge status is CLEAN. No live recovery, rollout, or physical termination is claimed. ## Risks - A wrong or stale Stop must not close another retry. Existing run/coordinator locks, caller checks, receipt binding, and the final compare-and-set remain in place. - This fix changes logical retry settlement only. Provider termination, workspace preservation, lease cleanup, and continuation gates remain separate. - The regression controls persistence and provider boundaries. It does not establish the physical state of a remote provider. ## Model Used - OpenAI GPT-6 through Codex, with reasoning, code editing, and tool use. The session does not expose an exact model build ID or context-window capacity. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0b4e48be9b |
Merge current controller recovery with scoped issue deadlines
Preserve keyed concurrent checks and their default serial behavior alongside issue-recovery reconstruction spacing and before-pass preparation. Qualify keyed ordering, joined preparation, exact deadline preemption and retained late-write reconstruction debt across drain. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ab51cabbbd |
test: await async outcomes and drive saved deadlines deterministically (#18416)
Reviewed and verified at
|
||
|
DottaandPaperclip
|
d1f9534c54 |
Merge current master qualification into event-driven run dispatch
Preserve retained-controller recovery deadlines and rearming together with company-scoped dispatch forwarding. Keep both source owners and all incoming master changes; no execution or retry policy changes. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ab0b15d234 |
Merge master into qualified test lifecycle fixes
Preserve master retention I/O settlement and held-I/O coverage alongside the controlled fairness clock and exact run completion barriers. Retain the delegated-completion fixture without changing production code. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
c2971badde |
perf(server): recover independent native replies concurrently (#18421)
## Thinking Path > - Paperclip manages work for AI agents. > - A native run can commit its result before its reply is saved. > - The response recovery worker saves replies from those committed results. > - It checks each candidate in series, so one task can delay unrelated tasks. > - This pull request checks up to four independent tasks at once. > - Each task keeps serial checks, and the existing coordinator owns retries and shutdown. ## Linked Issues or Issue Description **What existing behavior does this improve?** Recovery of committed native task replies. **Current behavior** The worker checks one candidate at a time in a batch of up to 25. A database lock or a slow read for one task delays replies for other tasks. **Proposed behavior** Check up to four company-and-task groups at once. Check candidates within each group in their existing order. Keep the current batch limit and company selection. **Reason and benefit** A ready task reply can be saved while another task waits for a database lock. The change reduces waiting under a reply backlog. **Breaking changes** Replies for different tasks can be saved in a different order. There are no API or schema changes. Other deadline workers keep serial dispatch. Related work: #17696 adds the recovery worker. #17501 reduces its read payloads. #18367 adds its timing spans. These changes are already merged. The PR search found no open change that adds this concurrency. ## What Changed - Add optional keyed concurrency to the shared scoped deadline worker. - Enable a limit of four for native response recovery, keyed by company and task. - Await all active checks before the pass ends. Retain per-candidate failure backoff, drain behavior and pending work. - Test concurrency limits, task order, company separation, drain/resume, shutdown, retries and idle wake behavior. - Add a PostgreSQL regression that holds one task's finalization row lock while a second task saves its exact accepted reply. - Document the worker's concurrency and sleep contract. ## Verification - The PostgreSQL lock regression fails with serial dispatch. It passes with this change. Both accepted replies are saved once after the lock is released. - `pnpm exec vitest run server/src/__tests__/scoped-deadline-work.test.ts server/src/__tests__/delivery-work-coordinator.test.ts server/src/__tests__/company-work-batch.test.ts server/src/__tests__/deadline-work.test.ts server/src/services/execution-preparation-timing.test.ts` — 55 tests pass. - `pnpm exec vitest run server/src/services/presentation-ownership-read.test.ts` with its documented dedicated PostgreSQL URL — 21 tests pass. The disposable database was dropped. - `pnpm -r typecheck` — passes. - `pnpm build` — passes. - One matched local replay with 25 older pending replies and one fresh reply used the real worker, a 10-connection PostgreSQL pool and 8 ms delay in each network direction. The saved fresh reply was first observed after 29.193 s with serial dispatch and 9.798 s with this change. All 26 replies were saved exactly once in both runs, with no worker errors. Both disposable databases were dropped. This is one synthetic workload pair, not a staging measurement. - `pnpm test:run` was attempted. Embedded PostgreSQL could not initialize its template after five attempts, so the broad run was stopped. The changed PostgreSQL tests pass on the separate local server. Remote CI is green. This PR does not claim a measured staging speedup. - The [first CI browser shard](https://github.com/paperclipai/paperclip/actions/runs/41162257463/job/124422204506) failed during company cleanup after its task assertions and empty-run check passed. Company deletion hit the `activity_log_company_id_companies_id_fk` constraint. The fixture uses paused process agents and reported no runs. The one failed-jobs rerun passed all 45 browser tests. All CI checks are now green. ## Risks - Four checks can use more database connections than serial recovery. The limit applies to this queue only; the batch limit stays at 25. - The shared helper has more dispatch logic. Its default remains serial. Tests cover task order, retry deadlines, drain and waiting for active checks during shutdown. - This improves reply recovery under backlog or lock contention. It does not establish a target for total agent-turn latency. ## Model Used OpenAI Codex, based on GPT-6, with code editing, tool use and local test execution. This session does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d5766e4074 |
perf(chat): scope task questions and batch inbox queue checks (#18422)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Standing chats show recent messages and current task questions. > - Recent-message pagination bounds history work, but it does not bound question work. > - The question read built the full company attention feed before it selected the chat's tasks. > - Each feed read also checked seeded queue membership with one insert per item. > - This pull request scopes question collection in the attention owner and batches queue membership checks. > - Chats keep their current question cards and initial layout while doing less work. ## Linked Issues or Issue Description Follow-up to #18358 and #18323. Related work: #18312. **What happened?** A hosted standing chat remained slow after recent-message pagination shipped. On commit `4eaa435bf4`, an embedded-browser reload made the conversation usable at about 8.6 seconds. Recent comments finished at about 1.9 seconds. The pending-question request held the initial reveal until about 8.4 seconds. That request called the complete company attention feed and then filtered its results. The separate badge summary also did broad work. **Expected behavior** Chat question cost should depend on the relevant tasks. Reading an existing queue must not require a separate insert for every member. Current access and decision policy must remain in force. **Steps to reproduce** 1. Create a standing chat with questions on tasks assigned to its agent. 2. Add many unrelated tasks and pending questions to the company. 3. Open the chat from a fresh application cache. 4. Compare recent-comment arrival with the pending-question response and content reveal. **Paperclip version or commit** Baseline: `909e29dd071b1aeeb24c2933af049da90a6066d3`. Its affected services match the observed hosted revision `4eaa435bf4`. ## What Changed - Add an internal task-question projection to the attention service. Filter pending interactions by company, task IDs and question kind before card enrichment. - Keep the same access, audience, dismissal, queue, snooze, retention, archive and training-example logic. Ranks are local to the returned question list. - Use the projection from the conversation receipt service. Keep current chat audience and session checks there. - Batch seeded membership inserts in groups of 500. Keep the queue lock, conflict protection, audit events and activity logging for each new membership. Existing members produce no new audit records. - Add real PostgreSQL regression cases and a reproducible paired benchmark. No schema migration or UI reveal change is needed. ## Verification - `pnpm -r typecheck` passed. The final server typecheck also passed after the queue change. - `pnpm build` passed. - Real PostgreSQL service coverage: 57 attention and chat-handoff tests passed. Seven queue-route tests passed, including concurrent reads, mixed existing/new items, duplicates, disabled seeds and audit preservation. - Red/green proof: the unrelated-attention regression fails with the baseline conversation implementation and passes with the scoped read. - `node scripts/check-module-boundaries.mjs` and `git diff --check` passed. - Full `pnpm test:run` was attempted and stopped with exit 130 after embedded PostgreSQL startup failures and 17 failures in the untouched native-session executor suite. It is not a local pass. The focused Docker/PostgreSQL results above completed before Docker Desktop became unavailable. A direct native PostgreSQL probe identified the host semaphore limit (`could not create semaphores: No space left on device`). No shared kernel settings or other test clusters were changed. [Full CI](https://github.com/paperclipai/paperclip/actions/runs/41162314995) passed on `2809f096b3fb2c01b44a96a1b82a27809c9359df`, including all eight browser shards. Final status: 55 successful checks/statuses, three skipped and one neutral. Greptile is 5/5 with no findings or unresolved threads. The fresh local browser retest is blocked because Docker Desktop cannot start and native PostgreSQL cannot allocate semaphores. The original hosted reproduction above is complete; no candidate browser pass is claimed. Local service benchmark: PostgreSQL 17.11, two relevant tasks, five alternating pairs per scenario, after warmup. These are local service timings, not a claim about deployed latency. | Fixture | Read | Baseline median (range) | Candidate median (range) | SQL calls | | --- | --- | --- | --- | --- | | 1,000 unrelated tasks/questions | Chat questions | 670 ms (632–1,905) | 12 ms (11–15) | 1,076 → 22 | | 1,000 unrelated tasks/questions | Company badge summary | 641 ms (622–717) | 167 ms (161–171) | 1,068 → 69 | | Small company | Chat questions | 38 ms (37–52) | 13 ms (12–16) | 52 → 22 | | Small company | Company badge summary | 34 ms (33–39) | 34 ms (32–40) | 44 → 43 | All question fields and ordering matched, except rank is relative to the scoped list. Summary fields matched except the generation timestamp. The interaction plan changed from 1,002 selected rows to two rows through the existing company/issue/status index. The benchmark loads baseline attention, conversation and queue modules together so the queue change cannot contaminate the baseline. ```sh pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-chat-questions.ts \ --baseline-ref=909e29dd07 --unrelated=1000 ``` ## Risks - Scoped and complete feeds must share policy. This change reuses the same attention collection and finalization path and tests dismissals, named audiences, snoozing, archives and private access. - Batched inserts must audit only new members. The concurrent PostgreSQL test checks unique membership and one event/activity record per new item across batch boundaries. - Full inbox summaries still collect the company feed. The small-company summary timing is unchanged. Other chat requests and transcript hydration can still contribute to hosted latency. - The first reveal still waits for question cards to preserve layout. This follow-up has not been deployed, so its hosted effect remains unverified. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository tools, code execution and browser profiling. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
492d537340 |
fix(native): recover controllers across webserver replacement (#18343)
## Thinking Path > - Paperclip manages AI agents and their work. > - Native execution runs in a remote Runner while the webserver owns its controller. > - Encrypted PRP checkpoints and session snapshots already survive a webserver restart in Postgres. > - Ordinary warm and per-turn admission still needs local markers and supervisor metadata to find that state. > - Replacing the webserver with an empty native controller directory can therefore lose a usable retained session. > - This change adds bounded recovery metadata to the existing snapshot transaction and reconstructs local projections from the selected durable source. > - The benefit is native execution continuity across webserver replacement, with the existing ownership and agent lifecycle policies. ## Linked Issues or Issue Description **What existing behavior does this improve?** Native remote controller recovery and retained-session continuation during a webserver upgrade. **Subsystem affected** server/ — native runtime, checkpoint persistence, recovery scheduling and internal upgrade reporting. Product E2E qualification also changes. **Current behavior** Without both the local PRP marker and supervisor file, normal continuation can return before discovering the saved Postgres checkpoint. The snapshot already exists in Postgres, but its local reuse compatibility envelope cannot be reconstructed. **Proposed behavior** Select the existing durable source, validate its identity and ownership, restore its PRP state and supervisor projection, then use normal authenticated admission. Missing or inconsistent evidence holds recovery. It does not authorize fresh provider work. **Reason and benefit** A replacement webserver must recover active execution and later continue its retained session without the old server's native files. This is one component of deployment continuity. HTTP traffic switching, workspace storage migration and Railway rollout remain separate. **Breaking changes** No new public endpoint, schema migration or database pool. The private run-profile envelope is additive. Disk-independent rollback also requires a reader for the staged controller format and compact checkpoint part; the first production release containing these readers is the floor. Older file-backed sessions need an owner checkpoint or the bounded internal importer. **Additional context** Related foundations: #17401, #17916, #17254, #14621, #14622. The implementation contract and open acceptance gates are in [the dated plan](doc/plans/2026-10-30-native-controller-checkpoints.md). ## What Changed - Persist a versioned recovery envelope, at most 4 KiB, beside the redacted session snapshot in its existing transaction. Keep it out of public run projections. - Compare frozen admission separately from prepared attachment instructions during staged recovery, preserving exact template/identity/authority checks. - Bound the existing lazy checkpoint pool at three connections after measured two-slot burst backlog; keep request/renewal capacities and SQL operation counts unchanged. The tuning is a separate commit. - Discover and restore the exact selected retained source without local native markers. Preserve the 32-candidate window, progress barriers and current authority checks. - Fence snapshot and result writes. Acquire coordinator, issue and run locks in order and check expiry after lock waits. Combine snapshot ownership/read work into one statement. - Reuse existing recovery deadlines and drain/resume scheduling. Retained-owner waits preserve the original unstarted run and actual lease deadline. Stop, ownership and provider-progress fences precede deferral. Add bounded native evidence reporting, retained-source import and retention protection. - Admit a settled warm source with an authenticated idle goal-read suffix while retaining active-work and mixed-authority holds. - Permit lifecycle `turn.stop` checkpoints after accepted results under existing ownership fences. Claim the existing successor coordinator before retiring the old warm session, and stage shared attachments only after verified retirement. Preserve omitted isolated-home defaults and existing fresh skill-selection order during recovery without hiding real configuration changes. - Carry the exact durable six-field PRP binding during active recovery after a healthy warm handoff. Keep the Runner mismatch guard and preserve opaque attachment IDs. - Preserve an already verified active or idle warm process during remote preparation; revalidate its proof/liveness and retain later authenticated PRP attachment. Ordinary resumed leases retain suspension checks. - Drain retained source events under their original authority, reconcile immutable source accounting and block unknown source tool dispatch before target work. - Persist compact staged handoff intent through authenticated adoption and recover direct/listener command-loss windows without another Runner or duplicate attachment. - Rebuild suspended lifecycle receipts for resumed prepared tasks, and retry missing lifecycle reads after partial PRP materialization without overwriting contradictory evidence. - Add normal warm/per-turn diskless continuation coverage, real Postgres races and SQL-cost checks. Add a reproducible paired load workload. - Give only the explicit outage fixture a 15-minute provider idle window, with a red/green catalog regression preserving all production timing and lifecycle policies. - Publish the goal-less task-session pointer in the existing post-acceptance terminal checkpoint transaction, with current ownership, accepted-result, reset and newer-progress fences. Keep a private bound receipt so late completion cannot duplicate or resurrect publication. - Use a retained warm sandbox in the dedicated continuity fixture, and accept an exact output token followed by an explanation. Keep the strict identity oracle. - Extend the explicit Daytona cell to a five-minute active outage and another replacement after completion. Require retained provider identity, a new tool receipt and a useful follow-up result. Add an owned loopback Postgres E2E fixture for hosts that cannot start embedded Postgres. ## Verification **Merge recommendation — `fc68584ce6231a02e47ecd743f55b036ccea034f`: ready for the qualified native-controller component.** Full CI, fresh review, local application paths, controlled performance qualification and the strict live Linux/Daytona case pass. Independent resource cleanup is complete. Broader rollout, browser reconnection and the unexecuted transport/platform combinations remain integration gates. Integrated 484 passed full local recursive typecheck/build and ordinary active→completion→cache-loss→C component qualification. Fresh review then found a valid attached-wake failure: preparation appends attachment instructions before the verifier compares execution to the frozen admission. 9f4 fixes that by separately checking the pre-preparation admission and exact prepared execution. Both hashes are captured privately per admission; the full template, identity, configuration and authority checks remain. Its regression fails before the fix; 36 focused cases, 42 staged source/target cases and server typecheck pass afterward. The review thread is resolved. Exact-head full CI passes ([run 41168228822](https://github.com/paperclipai/paperclip/actions/runs/41168228822): 48 successful jobs, one intended skip); the complete `pnpm test:run` partitions are forced, with the five native suites in the Runner lane. This is not a claim that an unsharded local command passed. The fresh exact-head Greptile check succeeds with 5/5 and all six threads resolved. The original two-connection database comparison closes **steady PASS and burst FAIL**. Every relative CPU/WAL/request-p99 gate passes and all operations/ownership remain correct, but candidate arm 3 drains in 1,015.99 ms and baseline arm 4 in 1,059.67 ms against the unchanged 1,000 ms absolute gate. Those results remain failed. A separate 2→3→4→2 diagnostic reproduces drain times 1,051.57 / 956.79 / 882.02 / 1,065.72 ms: three is the smallest tested passing cap, and the repeated two-cap failure survives run-order/cache effects. Separately reviewable `fc68584ce6` changes the existing lazy checkpoint pool default to three connections. Request capacity stays ten and liveness two; idle release, shutdown fencing, transaction counts and pool count are unchanged. Thirteen DB lifecycle/cap tests and DB typecheck pass. Full recursive typecheck/build pass at exact |
||
|
DottaandPaperclip
|
74970cc21e |
Merge qualified sleep review fixes and prerequisites
Co-Authored-By: Paperclip <noreply@paperclip.ing> * commit '5726ecf7a39339c2d68b094d9bbe1b1d0c436845': fix(retention): wake blocker retention after no-run workspace export Bound blocker workspace history reads by candidate workspace Validate eligibility before first attention retention materialization fix(retention): exclude finalized blockers before graph bounds test: hold lifecycle verification before retry identity assertions Exercise saved legacy input through its eligibility producer |
||
|
DottaandPaperclip
|
5726ecf7a3 |
Merge qualified sleep review fixes and prerequisites
Co-Authored-By: Paperclip <noreply@paperclip.ing> * commit 'f07e5cf343785503266d5a3a06a7c3ad31d163bb': test: hold lifecycle verification before retry identity assertions Exercise saved legacy input through its eligibility producer |
||
|
DottaandPaperclip
|
e7ff68d5d8 |
Merge qualified retention producer repair
Co-Authored-By: Paperclip <noreply@paperclip.ing> * commit 'a0e2a3515d420270b30cd5e89fbb966ef641b8a9': fix(retention): wake blocker retention after no-run workspace export |
||
|
DottaandPaperclip
|
0cd37fbb41 |
Merge qualified retention performance repair
Co-Authored-By: Paperclip <noreply@paperclip.ing> * commit '57f914747c0d2aefaca9ceccdeeddb55489dad67': Bound blocker workspace history reads by candidate workspace |