mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-24 18:19:05 +02:00
70d64663dfec5c9e84ab18ccd729eac25a89bd28
5511
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Dotta
|
70d64663df |
Merge branch 'codex/plain-task-repository-admission' into codex/dependable-repository-conversion
* codex/plain-task-repository-admission: style: format runner continuation regression sources |
||
|
DottaandPaperclip
|
219513a59d |
style: format runner continuation regression sources
Co-Authored-By: Paperclip <paperclip@paperclip.ing> |
||
|
Dotta
|
a7593dcc67 |
Merge branch 'codex/plain-task-repository-admission' into codex/dependable-repository-conversion
* codex/plain-task-repository-admission: fix: carry server-owned workspace continuations through runner bridges |
||
|
DottaandPaperclip
|
b852cdec59 |
fix: carry server-owned workspace continuations through runner bridges
Co-Authored-By: Paperclip <paperclip@paperclip.ing> |
||
|
DottaandPaperclip
|
9456bbdf51 |
Merge the complete setup repair into repository qualification
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
941467168c |
fix: accept server-resolved workspace continuations in Codex
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
9b924462b9 |
test: retain lifecycle state for repository journeys
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
9dcccc9e46 |
fix: resume pending repository work with an empty work list
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
6df51567e7 |
Merge current master into the complete repository setup repair
Co-Authored-By: Paperclip <noreply@paperclip.ing> * origin/master: fix(evals): classify latency from observation bounds (#16829) docs: define module ownership and shared change review (#17087) fix: stop idle MCP event polling (#17036) fix(recovery): let preservation owners reopen retained sources (#17053) perf(budgets): scope accounting and deliver enforcement asynchronously (#17046) # Conflicts: # tests/runner-e2e/README.md # tests/runner-e2e/installed-cli.test.ts |
||
|
DottaandPaperclip
|
d942a5c874 |
test: create conversion-compatible Claude baseline settings
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8748c2b5df |
Merge retained task-source reuse repair
Co-Authored-By: Paperclip <noreply@paperclip.ing> * codex/plain-task-repository-admission: fix: preserve plain task sources through shared workspace reuse |
||
|
DottaandPaperclip
|
3568193a71 |
fix: preserve plain task sources through shared workspace reuse
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
53732c811b |
fix(evals): classify latency from observation bounds (#16829)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E probes measure when acknowledgements and replies become visible. > - A poll gives a time interval for an event. > - The first positive poll can occur after a target that the event could still meet. > - This pull request compares the full interval with a strict or inclusive target. > - Reports retain the raw observations and the original verdict when the timing verdict changes. ## Linked Issues or Issue Description **What happened?** A historical acknowledgement was absent at 1901 ms and first present at 2427 ms. A report marked the 2000 ms target as failed. The interval crosses that target, so the available evidence cannot give a pass or fail verdict. **Expected behavior** Return pass when the full interval meets the target. Return fail when the full interval rules out the target. Return inconclusive for overlap, invalid evidence, or insufficient evidence. Respect strict and inclusive targets and open and closed endpoints. **Steps to reproduce** Run the deterministic timing tests. Use the acknowledgement interval `(1901, 2427] ms` with `<= 2000 ms`. Compare the late reply interval `(52965, 53334] ms` with `<= 30000 ms`. The first is inconclusive. The second fails. **Paperclip version or commit** The source includes master at `14afc12124017d4ec30d0f4bfd6892f8ce475d21`. Related PR: [#13710](https://github.com/paperclipai/paperclip/pull/13710) changed chat latency and session reuse. This change corrects probe evidence and report verdicts. ## What Changed - Add a shared interval classifier with explicit endpoint and target operators. - Start observation before dispatch. Use a monotonic clock. Bound asynchronous reads by request start and completion. - Measure the first visible response separately from message creation and final settlement. Share browser reads with an optional acknowledgement probe. - Keep raw bounds, clock resolution, message metadata, failed reads, and original verdicts in retained report evidence. - Add deterministic coverage for threshold equality, missing data, edited messages, asynchronous reads, teardown, report correction, and a finite-domain oracle. - Bound observer shutdown to one second across pending and final reads. Retain timeout counts. Ignore late completions and start no more reads after shutdown. - Document the timing contract in the existing Product E2E guide. - Keep current startup reuse evidence after the master merge. Update four stale test expectations for qualified Copilot admission. Keep the explicit suite and provider restrictions. ## Verification - Current head: `c42e87e6d3b4b347e7cb176f56819c1d0abe106d`. - Passed: `pnpm test:e2e:runner:unit` (3,287 Vitest tests and 128 native completion tests; two explicit skips). - Passed: `pnpm test:e2e:runner:typecheck`, `pnpm -r typecheck`, `pnpm build`, `pnpm check:token-gates`, and `git diff origin/master --check`. - Passed: [exact-head CI](https://github.com/paperclipai/paperclip/actions/runs/39607140418), including both required checks (`ci / verify` and `ci / e2e`), all general and serialized test jobs, all eight browser shards, build, canary dry run, and the clean public npm-install smoke. - Greptile reviewed the complete current diff and returned [5/5 with no actionable issues](https://github.com/paperclipai/paperclip/pull/16829#issuecomment-6266704814). All review threads are resolved. - The redundant local `pnpm test:run` was stopped after the exact-head CI test jobs passed. It did not produce a complete local verdict; the full repository test evidence is the CI run above. - The [earlier broad run](https://github.com/paperclipai/paperclip/actions/runs/39249282597) failed on the old head. Its full test command timed out after 15 minutes. Five E2E support failures came from ambient `NODE_PATH` and a missing historical Git object. Current master supplies the closed-environment launcher and retained source fixture. The full support suite passes with these fixes. - After the current master merge, four stale Copilot assertions and the nested unit launcher failed. Copilot is now qualified. Those assertions now require no qualification override and retain the rejected suite/provider cases. The full documented unit command passes. - The first local typecheck after the merge used stale generated package declarations and failed. The typechecks pass after `pnpm build`. - These checks use synthetic timing evidence. They make no live provider calls and claim no new live latency measurement. ## Risks - The browser diagnostic depends on the existing message and receipt DOM markers. Controlled tests cover visibility, receipt separation, concurrent reads, and teardown. - Corrected timing verdicts can become inconclusive. The report keeps the original verdict and does not change the overall workflow grade. - This change adds no database migration or product surface. ## Model Used - OpenAI GPT-6 Codex. Model family: GPT-6. This session does not expose a more specific model ID or context window size. The agent used reasoning, shell tools, code inspection, and deterministic tests. The earlier implementation session did not retain its exact model ID. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
65fd231271 |
Merge the measured workspace and completion repairs
Co-Authored-By: Paperclip <noreply@paperclip.ing> * codex/plain-task-repository-admission: fix: validate actual task roots and unfinished response waits # Conflicts: # doc/plans/2026-10-20-dependable-everyday-work.md |
||
|
DottaandPaperclip
|
91c026f169 |
fix: validate actual task roots and unfinished response waits
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
14afc12124 |
docs: define module ownership and shared change review (#17087)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Coding agents extend connectors and shared control-plane services. > - Provider details can leak into shared services during these changes. > - Moving code into a new file does not explain why shared behavior must change. > - This pull request adds module ownership and shared-change review rules. > - The rules help agents justify the change before implementation and review. ## Linked Issues or Issue Description **Issue type** Missing documentation. **Where is the issue?** Repository architecture rules and the connector authoring playbook. **What's wrong?** Subsystem rules describe some ownership boundaries. The repository lacks a general rule for deciding which behavior belongs in a feature module, what shared contracts must change, and how to review the resulting scope. An integration can therefore add provider-specific logic to shared code or bundle large platform changes into one connector PR. **Suggested fix** Add a repository-wide rule and load it in Greptile. Apply the rule during connector planning and PR preparation. Keep provider protocol behavior with its owner. Require a reason for shared changes and checks through affected consumers. Related public context: [Notebook connector PR #16984](https://github.com/paperclipai/paperclip/pull/16984). This PR changes instructions only. ## What Changed - Add a concise module-boundary rule for ownership, narrow contracts, shared controls, delivery scope, and verification. - Register the rule as repository-wide Greptile context. - Add planning and PR review steps to the connector playbook. Keep tests and docs with their behavior. ## Verification - Passed rule frontmatter, Greptile registration, referenced-file, and playbook-link checks. - Passed `git diff --check origin/master...HEAD`. - Reviewed the rule against the relevant connector failure and allowed cases: small dispatch branches, one-consumer shared capabilities, and unrelated fixes. - Fresh Greptile and Paperclip reviews both scored commit `759572fe503e1b6dfa9527f53de6c326a2c3bff6` 5/5. No actionable findings or unresolved review threads remain. - Final current-commit GitHub checks: 55 successful, 1 neutral, and 2 skipped; no pending or failed checks. Both required `ci / verify` and `ci / e2e` gates passed. - Initial CI attempt: the build and runtime boot smoke test passed. The spot runner received a shutdown signal during the clean public npm install check, canceling that step and failing the build job. [Interrupted job log](https://github.com/paperclipai/paperclip/actions/runs/39601811838/job/119228042468). Retried failed jobs once. Attempt 2 passed the complete build, runtime boot, and clean-install verification. The initial runner interruption remains recorded here. - Full local application checks are unavailable: this checkout has no workspace dependencies, and `pnpm -r typecheck` stopped during pnpm bootstrap because registry fetches failed and the requested pnpm version could not be verified. Local application tests and build were not run. GitHub checks must pass before merge. ## Risks - The rule could cause unnecessary abstractions or extra PRs if applied mechanically. It permits small dispatch branches and useful capabilities with one consumer. It rejects arbitrary file counts and unrelated restructuring. - Prose and metadata checks do not prove that future agents will follow the rule. Hosted review receives it when this branch is reviewed. ## Model Used - OpenAI Codex: `gpt-6-astra` authored the guidance; `gpt-6.1-sol` prepared delivery. Both used reasoning, filesystem tools, and GitHub tooling. Context window size was not recorded in the available model settings. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused documentation and metadata validation; application checks unavailable as stated above) - [x] I have added or updated tests where applicable (no runtime change; targeted structural checks cover the changed instructions) - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
16922a1832 |
test: qualify conversion prerequisites before runner selection
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0cd0746d19 |
test: replay repository setup with recorded staging models
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
c6ee96a3df |
fix: stop idle MCP event polling (#17036)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - MCP task subscriptions and Dot mailboxes notify external clients when work changes. > - Their delivery workers queried SQL every two seconds, including when no work existed. > - Those queries prevent idle databases from becoming quiet across many hosted instances. > - Existing activity, mailbox, subscription, and delivery records already preserve the work. > - This pull request uses committed-write signals and saved retry deadlines to drive delivery. > - Idle subscriptions become quiet while startup, retry, drain, and shutdown recovery remain supported. ## Linked Issues or Issue Description Related sleep migrations: #15625, #17000, #17008. Searched open PRs and issues for MCP event sleep/polling work; no duplicate found. **What happened?** Each public MCP and Dot event service started a two-second interval. It checked settings, scanned subscriptions, created delivery receipts, cleaned expired records, and attempted delivery even without changes. **Expected behavior** An empty or subscribed-but-idle service produces no recurring SQL or network traffic. Committed events trigger prompt delivery. Only unfinished work retains a retry deadline. **Steps to reproduce** Configure a public MCP origin and start the server. Observe recurring settings queries with MCP disabled, or subscription/queue queries with it enabled. No incoming task changes are required. **Paperclip version or commit** Initially inspected master at `8b4ab65b1c3d863d282752e83307943720a5c85c`; rebased onto `c2daf906d9` after master advanced. The only conflict was overlapping imports in the heartbeat queue; both were preserved. ## What Changed - Register separate MCP and Dot workers with the existing delivery coordinator. Remove both unconditional intervals. - Notify from committed activity, comment, mailbox, subscription, and experimental-settings writes. Centralize comment insertion in one small helper so normal callers do not manage recovery state. - Preserve outer transaction ownership, uncertainty reconciliation, and requests deferred during drain or standby. - Restore delivery retry/abandoned-claim deadlines on startup. Scan subscriptions in pages of 20; drain full source and delivery batches without another external event. Scope subscription refreshes to their subscription or shared Dot binding, issue activity to its task, comment discovery to its company, and mailbox/assignment changes to their binding. Retry deadlines do not rescan unchanged sources. - Run expired-record cleanup opportunistically. Preserve on-use expiry checks and the existing one-time automatic Dot challenge behavior. - Add real PostgreSQL lifecycle coverage and guarded external test database options for existing MCP/Dot integrations. Document producers and the supported writer model. ## Verification - Final head `e54fb34485921016ef988f7df9cca173a9ebdfc8`: all CI checks passed (55 successful checks/statuses, one neutral review, two intentionally skipped Storybook checks). Mergeable with no unresolved review threads. Greptile: 5/5 on this exact commit; security scan passed. - All 284 affected tests passed in one local run after rebasing: 16 real-PostgreSQL lifecycle tests, 24 coordinator/scheduler/standby tests, 84 public MCP tests, 54 Dot runner tests, and 106 interaction-service tests. - Server TypeScript passed after refreshing generated Runner/plugin SDK dependencies from master. Full workspace typecheck and build passed before the rebase; final CI repeated typecheck and build successfully on the rebased head. - Coverage includes SQL/network quiet with configuration and history, committed writes, outer commit/rollback, drain/standby, retry restoration, reenable/unsubscribe, concurrent writes, shutdown, source/delivery/subscription batch boundaries, Dot challenges and reconnects, native broker fences, comment discovery, and company-isolated scans. - CI exposed an existing lock-order test whose insert proxy missed nested transactions and waited indefinitely. The fixture now observes nested inserts, preserves the lock assertion, and fails explicitly if interception is missed. The complete 106-test interaction suite and the formerly hanging CI shard pass. The warm-standby/settings test doubles were also updated for the new coordinator/transaction boundaries. - Earlier fixture failures (required timestamps, cleanup, external database selection, async waits) were corrected. One existing OAuth first-poll timing assertion and one E2E company-cleanup race passed on single reruns; the final full CI run passed without a rerun. No OAuth or company-deletion behavior was changed. - The local full `pnpm test:run` attempt could not initialize embedded PostgreSQL after five attempts. It was stopped; affected integrations were verified with separate disposable PostgreSQL 17 databases, and final CI supplied the full-suite result. Embedded-only local skips are not counted as passes. ## Risks - Notifications use one live server/database scope. Independent SQL writers and separate live replicas do not trigger these process-local workers. Startup recovers durable state; continuous multi-writer discovery needs a notification/ingress contract. - Idle physical cleanup waits for startup or another enabled sweep. Subscription and authorization expiry remain enforced on use. - Existing delivery claims, retry bounds, signatures, and receipt deduplication remain in place. No new tables or migrations. - Host sleep-safety checks remain conservative. MCP/Dot retry deadlines are not yet covered by Paperclip Cloud's scheduled-wake contract. This proves worker quiet, not deployed host suspension or fleet savings. ## Model Used OpenAI Codex (GPT-6), with reasoning, local code execution, and tool use. The runtime does not expose a more specific model version or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d20304da70 |
test: verify configured repositories after real runner conversion
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ed5452863c |
style: keep combined setup rationale in review documentation
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a7511226e8 |
fix: prepare nested repositories in declared plain task roots
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ab067f59b0 |
style: keep repair rationale in the PR and plan
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
eaacadeba3 |
fix(recovery): let preservation owners reopen retained sources (#17053)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Recovery must save retained workspace files before ordinary execution can resume. > - The environment runtime blocks access to retained sources to prevent unsafe reuse or deletion. > - The recovery worker also used that ordinary resume path. > - The guard returned no lease, so recovery reported an invalid source binding before it could save files. > - This pull request gives the physical preservation owner a separate, source-bound resume path. > - Ordinary execution and destructive cleanup keep their existing guards. ## Linked Issues or Issue Description Refs #16965. **What happened?** A retained workspace with a workspaceRestoreRecovery marker could not recover through the real environment runtime. The runtime returned null before calling the provider. Recovery classified this as source_binding_invalid. Earlier recovery fixtures replaced the runtime facade, so they did not exercise this guard. **Expected behavior** The verified preservation owner can reopen its exact stopped allocation to save files. Other callers remain blocked. Recovery preserves newer local files and does not start another agent or repeat a completed external action. **Steps to reproduce** 1. Seed a terminal source run with a retained sandbox, a recovery action, and the workspaceRestoreRecovery lease marker. 2. Give the source a verified stop receipt. 3. Call recoverRetainedWorkspace through environmentRuntimeService. 4. Before this change, the recovery action returns to blocked with source_binding_invalid. No files are copied. **Paperclip version or commit** Reproduced on master at |
||
|
DottaandPaperclip
|
30b753f614 |
perf(budgets): scope accounting and deliver enforcement asynchronously (#17046)
## Thinking Path > - Paperclip manages AI agents and their paid work. > - Budgets must prevent new work and stop reported overruns. > - Cost settlement also ran serial lifecycle reconciliation for every eligible company agent. > - That made request latency grow with unrelated idle agents and delayed results on stop delivery. > - This change separates atomic admission and accounting from durable asynchronous enforcement. > - Scoped live usage detects overruns without writing provisional ledger charges. > - The result is constant query scope for ordinary settlement and recoverable stop delivery. ## Linked Issues or Issue Description **What happened?** Settlement and budget checks entered company-wide lifecycle reconciliation. A company with 1,000 unrelated idle agents performed 1,001 accounting transactions for one ordinary cost evaluation. User requests could also wait for provider termination. Running usage checkpoints did not enforce reported monetary exposure. **Expected behavior** Keep the final reservation gate authoritative. Commit charges, scope blocks, and enforcement intent before returning. Deliver stop requests after commit. Read only applicable policies and execution scopes for ordinary charges. Stop long-running work when supported reported or priced usage establishes an overrun. **Steps to reproduce** 1. Create a disposable company with one company budget and idle agents. 2. Evaluate a charge below the budget with 0, 17, and 1,000 unrelated idle agents. 3. Compare query counts, company accounting transactions, and duration. 4. Use the controlled partial-usage and delayed-stop fixtures to check enforcement and request responsiveness. **Paperclip version or commit** The measured baseline is `6c9b87d0922f1ebfc43e8b0c36889ac5ba7b9837` from `master`. Related public work: Refs #16976, #2006, #3539, #13763, #14296, #16853, #13075. These cover the affected service charge, cancellation, and accounting paths. This change removes their shared reconciliation overhead. ## What Changed - Batch applicable policy exposure under the existing company accounting lock. Preserve exact fractional money and zero-reservation dispatch fences. - Commit a durable final budget dispatch claim before provider I/O. Retain a separate final issue/run ownership transaction through synchronous handoff, without holding the company accounting lock or awaiting provider work. Preserve native recovery lease checks and duplicate handoff fencing. Browser handoff rechecks current policy; it reuses a consumed rate slot only for the identical policy generation. - Separate ledger spend, live observed spend, and effective held capacity. Use the maximum of reservation and live observation for each unfinished operation. - Deliver budget cancellation through the existing coordinator. Revalidate generations and persist queued/running stop fences before external termination. Bound stop concurrency to four. - Apply agent holds only to the affected agent. Project budgets leave other projects running. Project real company transitions in bounded lifecycle batches. - Replace periodic broad reconciliation with durable pending work, startup repair, receipt replay, and concrete retry/window deadlines. Retain drain and idle-safety checks. - Enforce accepted cumulative usage checkpoints without provisional charges. Final accounting atomically replaces live exposure and settles reservations. - Add operator-gated budget spans, regression coverage, map updates, and a real-PostgreSQL comparison harness. No schema or first-party telemetry change. ## Verification - Full workspace `pnpm -r typecheck` passes on the final ownership fix, rebased onto `bbcf5dfe98`. Production `pnpm build` passes on final head `4c5f4de8b1`. Map, module-boundary, token, and Storybook catalog checks pass. - Focused real-PostgreSQL coverage includes the 292-case rebased accounting/dispatch run, 63 browser cases, 52 run-control/cancellation cases, 42 policy-generation cases, 77 lifecycle/commentary cases, and the exact Muse confirmation regression. Native executor unit coverage passes all 768 cases. Final ownership coverage passes: 74 adapter/use-case cases, nine public regression cases, and the additional deleted-project race. These exercise competing ownership, budget changes and charges between phases, zero/nonzero duplicate claims, rollback, recovery contention, read-only capacity rechecks, and controlled HTTP startup after commit. Another 74 pure/startup/tracing cases pass. The complete CI matrix and both 5/5 reviews passed on `1ac9b58`; its external security scan identified a stale-owner handoff race. The final fix restores synchronous issue/run ownership fencing after the accounting transaction commits. Final head `8e84f55ce8` has fresh Greptile 5/5 with zero new comments, Paperclip review 5/5, and a passing security scan. The complete latest-head [CI matrix](https://github.com/paperclipai/paperclip/actions/runs/39535189808) passes, including budget, browser, Runner, build, and canary jobs. The unchanged Runner close fixture passed on its sole same-head job retry. - Broad local `pnpm test:run` invocations were stopped before required source updates (rebase and review fixes). They are not claimed as full passes. Final full-suite coverage comes from the complete latest-head CI matrix; local focused coverage and the incomplete broad invocations are reported separately. - Reproduce performance with `pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-budget-enforcement.ts --baseline-ref=6c9b87d0922f1ebfc43e8b0c36889ac5ba7b9837`; repeat with `--reverse`. The harness uses disposable real PostgreSQL, 20 warm samples per variant/count, and records raw timings, SQL/lock counts, agent reads, unchanged agent state, and query plans. Final production-source measurement revision: `eff713a4f72c7f4b7a4002a701cb481d4bfccd95`; the extended harness is committed in `4c5f4de8b1`. | Unrelated idle agents | Baseline median / p95 | New median / p95 | Baseline SQL / locks | New SQL / locks | New agent reads | | --- | ---: | ---: | ---: | ---: | ---: | | 0 | 1.80 / 2.33 ms | 2.99 / 3.88 ms | 5 / 1 | 7 / 1 | 0 | | 17 | 15.53 / 19.55 ms | 2.24 / 2.60 ms | 90 / 18 | 7 / 1 | 0 | | 1,000 | 978.35 / 1,790.43 ms | 2.16 / 2.52 ms | 5,005 / 1,001 | 7 / 1 | 0 | - These measurements use local PostgreSQL 18.1 and Node 26.4.0 arm64. They isolate the cost-evaluation core and complete admission path, without hosted-region or provider latency. Prior-revision reversed-order controls also preserved the constant query counts; the final measurements above use the final production source. - Complete reserve → final ownership gate → immediate offline provider result is measured separately, with one responsible agent and one active operation at each unrelated-agent count. Admission p95 baseline/new: 0 agents **11.54 / 29.77 ms**; 17 agents **9.31 / 27.62 ms**; 1,000 agents **24.51 / 74.05 ms**. Baseline admission was already scoped; the stronger final fence adds bounded overhead. New counts stay constant at 53 Drizzle statements, three transactions, two company-accounting locks, and five agent-reading statements bound to the responsible agent. Zero unrelated-agent reads, exactly one provider callback per sample, and no leftover runs or holds. All admission scenarios stay below 250 ms. No external provider calls are included. - The partial-usage/delayed-termination fixture passed 20/20: receipt persistence p95 13 ms, receipt return to observed durable Stop fence p95 28 ms, complete receipt-start-to-fence p95 50 ms, unrelated database request p95 1 ms while termination was blocked. After releasing the controlled termination fixture, acknowledgement p95 was 14 ms. These are separately measured phases; fixture acknowledgement is not live-provider termination latency. - Models and provider deadlines are unchanged. No deployment, live inference, or staging acceptance is claimed. A quiet 30-minute follow-up waits for the merged revision on staging, then repeats classifier and controlled budget-stop acceptance in staging, recording hosted bookkeeping, queue delay, and actual provider acknowledgement separately. - Development failures retained: finalizer notification regressions (fixed; 27/27 affected cases passed), invalid browser admission fixture assumptions (fixed; browser coverage now passes 63/63), and first-head lifecycle/dispatch CI regressions (reproduced and repaired). A prior-head Cases E2E test hit its unchanged announcement-dismiss handler race while opening a filter; the same journey passed on `1ac9b58`. The external security review then found a stale-owner race between accounting commit and provider entry; final verification covers a competing run in that gap while retaining provider I/O outside the accounting lock. Early PostgreSQL startup attempts hit local semaphore pressure; serial reruns succeeded. The final-head Runner lane had one failure among 1,823 tests: the unchanged `timeout-attachment` fixture failed at its first process close after 6.28 seconds. The sole same-head job retry passed all 1,819 active tests (four skipped), including that fixture in 1.85 seconds. Three isolated local probes could not run correctly because the Rust fake-provider fixture binary was absent; cleanup masked the original startup failure. CI builds that fixture through its normal lane. These probes are inconclusive and are not counted as passes. ## Risks - This changes accounting, lifecycle, and cancellation ordering. Durable generations, source/sequence checks, ownership leases, and company locking remain required for correctness. - Provider stop acknowledgement can be slow or fail. Stop intent remains pending until existing termination proof succeeds; cancelled row status alone does not acknowledge delivery. - Mid-run limits depend on provider usage reports or supported price calculations. Providers that report only at completion cannot provide exact mid-call detection. - Signals are process-local wake hints for the supported single-active-writer model. Durable state and startup recovery remain authoritative. - There are no migrations, new queue tables, or cached balances. Repeat the classifier and controlled budget-stop acceptance after normal rollout. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, tool use, code execution, and parallel coding agents. The host does not expose the exact backend model revision or context-window size, so those values are not asserted. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
84a613f66a |
fix: correct routine repository admission before accepting a blocker
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
f220914293 | fix: retain local-board responsibility during repository preparation | ||
|
Dotta
|
12b2a8eff9 | test: verify first-admission context and prepare source dependencies | ||
|
DottaandPaperclip
|
de14edc606 |
test: exercise automatic repository continuation through a real task
Use production hire instructions and the existing paid browser harness. Compare delivered bytes with the actual checkout, require retained workspace and context, and reject timer or user-driven continuation. Keep current qualified provider admission free of ambient overrides and retain explicit pending-profile gate controls. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
dff63551fe |
fix: resume unfinished work after repository preparation (#17047)
## Thinking Path
> - Paperclip manages agents that perform work for a company.
> - Task repositories are prepared at normal run admission.
> - A running native task can stage a repository while its work remains
unfinished.
> - The controller did not offer that saved setup as a continuation.
> - The agent could add a timer or wait for a user restart.
> - This change uses the existing same-agent continuation and status
ledger.
> - The accepted explanation and proven conversation remain available
after setup.
## Linked Issues or Issue Description
Refs #17045. That prerequisite has merged. This PR targets master.
**What happened?**
A projectless native task can request a repository for the next
admission. An unfinished task has no server-selected repository
continuation. An explicit same-agent continuation also loses chat
provenance.
**Expected behavior**
An unfinished standard task can yield and resume through the existing
admission queue. Its explanation and final result retain the authorized
conversation. A task that only stages a repository can complete without
another run.
**Steps to reproduce**
1. Start a native standard task in a task directory.
2. Request an authorized repository through prepare_repository.
3. Yield with unfinished work and no explicit continuation.
4. Observe that the former driver rejects the yield, although the
pending repository is saved.
## What Changed
- Offer a saved same-agent continuation for the current task's pending
repository.
- Accept that current server continuation in the shared provider driver.
- Resolve chat provenance through applied status decisions, delivered
effects, and admitted wakes.
- Preserve accepted setup explanations without repeating an owned
question card.
- Document optional reproduction details and configured repository
discovery.
- Extend database regressions and workspace access maps.
- Preserve Copilot profile v52 and bind the shared continuation source
in v53. Include the active identity in the QA image fingerprint.
## Verification
- Five focused server suites: 100 checks pass.
- Shared continuation driver: 12 checks pass.
- Server typecheck passes.
- Map contract and whitespace checks pass.
- Existing API and driver assertions fail on the former implementation.
- Real dispatcher and queue cohort: 43 checks pass. The original
reply-loss regression fails before the dispatch receipt fix.
- Continuation driver and installation identity cohort: 62 checks pass.
- Regenerated capability inventory and contract checks pass.
- Final response precedence cohort: 109 checks pass. The stale-comment
case fails on the former implementation.
- All required CI checks, build, and both source reviews pass on
canary/v2026.1021.0-canary.6
|
||
|
|
8b873e0112 |
fix(adapter-utils): skip the materialize lock for an already-current skill (#17061)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A remote Claude ACP run stages the selected runtime skills into the sandbox. The host archives the materialized skills directory with tar and uploads it. > - Concurrent runs of one agent share that skills directory. The directory key is the selected skill set and its content. > - Each run calls `materializePaperclipSkillCopy` for each skill. The function always created and removed a sibling `<skill>.lock` directory, even when the copy was already current. > - The archive step lists the directory entries first and then gives the names to tar. When a concurrent run removed its lock between these two steps, tar failed with `Cannot stat` and the whole run failed as `adapter_failed`. > - This pull request checks the materialized fingerprint before it takes the lock. An up-to-date copy now leaves no transient entries in the shared directory. > - The benefit is that concurrent runs of one agent no longer fail each other's skill staging. ## Linked Issues or Issue Description **What happened?** A remote `claude_local` run failed before the agent started. The skill staging step ran `tar -c ... -T entries.nul` on the shared runtime-skills directory. Tar stopped with `tar: <skill>.lock: Cannot stat: No such file or directory` and exit status 2. Another run of the same agent was materializing the same skills at the same time. **Expected behavior** A run that finds its skills already materialized does not change the shared skills directory. Concurrent runs can archive that directory safely. **Steps to reproduce** 1. Materialize a skill into a skills directory. 2. Call `materializePaperclipSkillCopy` again in a loop for the same skill. 3. At the same time, call `createTarballFromDirectory` on the skills directory in a loop. 4. Before this change, most archives fail with `Cannot stat` on `<skill>.lock`. **Agent adapter(s) involved** `claude_local` (ACP, remote sandbox). The function is shared, so `grok_local` and the Codex and Gemini ACP skill paths also benefit. ## What Changed - `packages/adapter-utils/src/server-utils.ts`: `materializePaperclipSkillCopy` hashes the source and checks the materialized sentinel before it takes the lock. When the copy is current, it returns without creating the lock directory or a temporary copy. The existing check inside the lock stays, so a copy that is missing or out of date still uses the lock as before. The source is hashed once per call, the same as before. - `packages/adapter-utils/src/server-utils.test.ts`: a new test holds a live lock from another owner, then materializes an already-current skill. The call must return at once and must not change the lock or the directory. ## Verification - `cd packages/adapter-utils && npx vitest run src/server-utils.test.ts`: 132 passed. - Red/green: without the fix, the new test times out because the call waits on the live lock. With the fix, it passes. - `npx vitest run src/acpx-engine/execute.test.ts -t skill`: 15 passed. - `npx tsc --noEmit -p tsconfig.json` in `packages/adapter-utils`: passes. - Real-dependency race check (a temporary script, not committed). The script used real files and the system tar. It ran `materializePaperclipSkillCopy` for 5 skills in a loop while `createTarballFromDirectory` archived the same directory for 8 seconds: - Before the fix: 1994 of 2307 archives failed with `tar: skill-1.lock: Cannot stat: No such file or directory`. - After the fix: 0 of 2021 archives failed. - Assumption under test: the sentinel check outside the lock is safe. The sentinel is written into the temporary copy before the atomic rename, so a matching sentinel means a complete copy. The lock never protected readers: the caller already reads `SKILL.md` after the lock is released. - Remaining limit: the first materialization of a new skill set still uses the lock. A concurrent archive of the same directory is unlikely then, because a run archives the directory only after it has materialized every skill in it. ## Risks - Low risk. The change only skips the lock when the result is already correct. A missing, damaged, or out-of-date copy uses the same locked path as before. - No change to the materialized layout, the sentinel format, or the lock format. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, test runs). The exact context-window size is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>nightly/v2026.1021.0-nightly.0 canary/v2026.1021.0-canary.5 |
||
|
|
f3af9887ca |
fix(chat): keep each chat conversation in its own wake batch (#17054)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat channels let a linked user talk to an agent from Slack, GitHub and other platforms. Each external thread is a chat conversation that is bound to one issue. > - One issue can have many conversations. A GitHub pull request review makes one conversation for each inline finding thread. > - When replies arrive behind an active run, the wake queue merges them into one deferred wake. The merge did not look at the conversation. > - A reviewed chat run can attest only one conversation. A merged batch from two threads always failed attestation with `setup_failed` / `reviewed_chat_execution_binding_not_authorized`, so no reply got an answer. > - This pull request keeps each chat conversation in its own wake batch. The single-conversation attestation check stays the same. > - The benefit is that replies in different review threads each get their own attested run, one after the other. ## Linked Issues or Issue Description Refs #16793 (related: chat origin during coalescing). No public issue exists, so the bug report follows. **What happened?** A linked user replied in three different inline review-finding threads of one pull request while the agent's review run was still active. The three wakes merged into one deferred wake. The promoted run failed in about 2 seconds with `setup_failed` / `reviewed_chat_execution_binding_not_authorized` and `providerWorkStarted=false`. None of the three replies got an answer. The failed-chat retry cannot recover this batch, because a retry source holds one conversation. **Steps to reproduce** 1. Bind two chat conversations (two GitHub finding threads) to one `in_review` issue. 2. Start a run for the assigned agent on that issue. 3. As the linked user, reply once in each thread. 4. Finish the active run. 5. See one promoted run that fails with `reviewed_chat_execution_binding_not_authorized`. **Expected behavior** Each thread gets its own run. Each run attests its one conversation and answers its reply. **Cause** `findExistingDeferredWake` matched on company, agent, issue and actor only. The active-run coalesce path had the same gap. `authorizeChatConversationForBoundRun` correctly denies a batch with more than one conversation. ## What Changed - `run-dispatch/domain/wake-context.ts`: add the `chatConversationId` wake-context key, a reader, and `wakeChatConversationsConflict`. A conflict means that both wakes have a conversation and the two conversations are different. - `issue-assignment-wakeup.ts` and `chat-channels/inbound/wakeups.ts`: stamp the conversation ID on each inbound chat wake. - `chat-channels/failed-run-retry.ts`: stamp the conversation ID on failed-chat retry wakes, so later replies do not merge into a retry from another conversation. - `wake-queue` admission: do not coalesce into an active run of another conversation. The deferred-wake lookup now matches only wakes of the same conversation or wakes without a conversation. - `heartbeat/queue.ts`: the same-scope queued or running coalesce target is skipped when the conversations conflict. - `doc/execution-semantics.md`: describe per-conversation chat wake batching. - Tests: an integration regression in the GitHub review block of `chat-channels.integration.test.ts`, and unit cases for the active-run branch in `wake-queue/application/use-cases.test.ts`. ## Verification - Red before the fix: `pnpm --dir server exec vitest run src/__tests__/chat-channels.integration.test.ts -t "replies in review finding threads"`. The main case fails with one run: `{ status: "failed", errorCode: "setup_failed", error: "reviewed_chat_execution_binding_not_authorized" }`. - Green after the fix, same command, 3 of 3 pass: - Replies in two finding threads give two successive runs. Each run succeeds, has `paperclipExternalChatExecutionBound: true`, and holds only its own comment and conversation. - Two replies in the same thread still give one run with both comments. - After the linked identity is revoked, each separated run is still denied (`setup_failed`) and the adapter does not execute. - `vitest run src/modules/wake-queue src/modules/run-dispatch`: all pass, with the new cases (different conversation, same conversation, run without a conversation, non-chat wake). - Related suites pass: `heartbeat-comment-wake-batching`, `durable-chat-wakeup`, `chat-failed-run-retry-boundary`, `issue-queued-comments-routes`, `heartbeat-stale-queue-invalidation`. - Full `chat-channels.integration.test.ts`: 823 passed, 1 timed out. The timed-out test (`GitHub terminal receipt composes pinned HTTP and durable source authority (held_token)`) passes alone with and without this change. The sandbox has a 4 GB memory cap. - Not run locally: `pnpm -r typecheck` and `pnpm build`. The server `tsc` process was killed at the 4 GB sandbox memory cap. CI runs both. - Runtime dependency note: the deferred-wake lookup adds one JSONB predicate to the existing deferred-wake query. It reads the same rows. The query count does not change. The integration regression runs it against embedded Postgres. ## Risks - Low risk. No schema change and no migration. - Behavior change: chat replies from different conversations on one issue now run one after the other, not in one batch. Before this change, such a batch always failed, so no working flow becomes slower. - Wakes without a conversation (non-chat wakes, and wakes queued before this deploy) keep the old coalescing. A deferred wake from before the deploy has no conversation key, so one later reply can still merge into it. That window closes when the queue drains. - The check keeps the single-conversation attestation in `authorizeChatConversationForBoundRun`. Revoked identities are still denied. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), in Claude Code through the Paperclip agent harness. It used tool use (shell, file editing, test runs) and extended reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>canary/v2026.1021.0-canary.4 |
||
|
Nicky Leach
|
65f59e339a | Extend native instance secrets to project lifecycle plugins (#17034) canary/v2026.1021.0-canary.3 | ||
|
DottaandPaperclip
|
c2daf906d9 |
fix: preserve task continuation and question delivery (#17045)
## Thinking Path
> - Paperclip manages agent work and delivers its results.
> - One task can need several runs and a saved workspace.
> - A run can succeed while the task waits for its next run.
> - Chat fallback treated that success as completion and dropped useful
status.
> - Monitor runs also lost their conversation origin, and text-question
cards lost their prompt.
> - This change preserves the task outcome and exact delivery authority
across those boundaries.
## Linked Issues or Issue Description
**What happened?**
A Slack task requested a repository, yielded to a scheduled
continuation, and showed only a generic completion message. The next run
could not publish through the original conversation. A later text
question showed its title and Respond button without the question
context. Workspace recovery could commit the saved monitor result
without restoring its explanation.
**Expected behavior**
The original conversation shows the accepted continuation explanation,
receives the resumed result, and displays the full question when input
is needed. A successful provider run does not claim that unfinished work
is complete. Recovery delivers saved output once without another
provider result.
**Steps to reproduce**
Start an authorized chat task, prepare a repository for a later
admission, save a monitor wait, and let that monitor resume the task.
Repeat with workspace finalization interrupted and with a free-text
question. Inspect the originating conversation and the persisted output.
Related merged work: #16806 and #16975. The open monitor and recovery
changes found during duplicate search address different status and
liveness policies.
## What Changed
- Keep committed monitor summaries visible through the shared board and
chat response resolver. Preserve interaction ownership and comment
reuse.
- Recover a saved monitor explanation after finalization, with no
repeated provider result.
- Prove monitor publication origin from the admitted wake and matching
scheduling receipt. Match company, task, agent, requester, and schedule;
reject ambiguous sources.
- Include monitor continuations in the existing publication scan and use
the canonical task binding. Keep current transport authorization.
- Suppress completion fallback for yielded results. Explain missing
final output without claiming completion.
- Include question summaries, prompts, and help text in modal-backed
cards. Keep answered text private.
- Update execution semantics, capability maps, and the bounded
reliability plan.
## Verification
- Original source fails four composed PostgreSQL regressions: monitor
origin, yielded completion, question body, and saved-summary recovery.
The shared response resolver also fails before the repair.
- Repaired focused cohort: 148 tests pass across publication, question
privacy, scheduler, and native monitor suites. These use real disposable
PostgreSQL; they do not qualify live Slack.
- Full workspace typecheck and production build pass locally. Map checks
and whitespace checks pass.
- All CI gates pass at
canary/v2026.1021.0-canary.2
|
||
|
|
bbcf5dfe98 |
fix(heartbeat): renew legacy controller leases on a private pool (#17032)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A legacy (non-native) agent run keeps a controller lease on its `heartbeat_runs` row. The server renews the lease every 10 seconds. If no renewal succeeds in 60 seconds, the watchdog stops the run with `renewal_deadline`. > - #16359 added timings to lease-loss reports. The first staging event with these timings shows `deadlineLateMs=0`, `renewalPendingMs=50000`, and 108 renewals in about 22.5 minutes, where about 135 are expected. Thus the event loop was not blocked. One renewal UPDATE was in flight for 50 seconds, and renewals were slow for a long time before the loss. > - The agent had exited with code 0. The run was in post-execution `sync_back`. Other runs on the same host completed. No code path held a lock on the run row across slow work. A later `instruction_save` UPDATE on the same row completed while the earlier renewal was still pending, so a row-lock wait does not explain the stall. > - postgres.js 3.4.9 sends a new query to a busy connection when no connection is idle (`handler` → `go(busy.shift(), query)` in `src/index.js`). The query waits behind the statement that is in flight on that connection. The renewal had no timeout, and the `pending` flag stopped all later renewals. Thus one slow unrelated statement on the shared pool can stop a healthy run. > - This pull request moves lease renewal to a small private pool, bounds each attempt with a statement timeout, and retries a timed-out attempt until the existing lease deadline. > - The benefit is that slow application queries on the shared pool can no longer cancel a healthy legacy run. ## Linked Issues or Issue Description Refs #16359 (lease-loss timings that identified this cause) Refs #16358 (pre-dispatch lease loss becomes retryable; different path) Refs #16917 (open: keeps a clean adapter result after a lease lapsed; it does not prevent the loss) Refs #13389 (open: host sleep / late timers; different cause) **What happened?** A staging legacy `claude_local` run was cancelled with `Legacy controller lease lost (renewal_deadline)` after the agent exited with code 0. The timings show that the deadline timer ran on time, and one renewal UPDATE was pending for 50 seconds. **Expected behavior** A live controller keeps its lease. Slow unrelated database work does not cancel the run. **Steps to reproduce** 1. Create a database client with one pooled connection. 2. Start `select pg_sleep(5)` on that client. 3. Call `watchLegacyControllerLease(...).assertOwned()` with the same client. 4. Before this change, the renewal waits about 5 seconds behind `pg_sleep`. With a long enough slow statement, the lease deadline stops the run. ## What Changed - `packages/db`: Add `livenessDb(db)`. It gives each `createDb` client a private pool of at most 2 connections (`application_name=paperclip-controller-lease`). The pool is created on first use. Test doubles and transactions get the original client back. - `packages/db`: Add `closeLivenessDb(db, timeoutSeconds)`. The server calls it in `closeDatabaseClients`, so shutdown and failed startup end the private pool too. - `legacy-controller-lease.ts`: The watchdog renews in a transaction on the private pool. Each attempt sets a transaction-local `statement_timeout` of 10 seconds. A timed-out attempt (`57014`) is retried. After a successful retry, the local deadline is measured from that attempt's start, because that attempt's UPDATE wrote the fresh 60 second lease. The existing 60 second deadline, the abort race, and the `renewal_rejected` / `renewal_error` attribution do not change. `renewLegacyControllerLease` itself does not change, so the runtime-selection caller inside its own transaction is not affected. - Tests: a saturated shared pool no longer delays renewal; a renewal that times out on a locked run row retries and keeps the lease; the private pool is cached per client, has its own `application_name`, and closes; after a 45 second timed-out attempt and a successful retry, the next deadline is a full 60 seconds after the retry. The existing test doubles now model the transaction. - `docs/deploy/database.md`: Count the up to 2 extra connections per server process when you size `max_connections`. ## Verification - `cd server && npx vitest run src/services/legacy-controller-lease.test.ts`: 20 passed, on embedded Postgres. - Red/green: with the old `legacy-controller-lease.ts` and the new tests, the saturated-pool test fails with `expected 5018 to be less than 2000`. With the fix, it passes. - Red/green for the retry deadline: with the deadline measured from the first attempt, the retry-deadline test fails (`expected true to be false`) because the run is aborted 15 seconds after the retry. With the fix, it passes. - Mutation check: when the timeout retry is removed, the locked-row test fails with `canceling statement due to statement timeout`. - `npx vitest run src/__tests__/heartbeat-process-recovery.test.ts -t "lease|dispatch|controller|interaction"`: 43 passed. - `npx vitest run src/services/heartbeat/run-completion.test.ts src/__tests__/heartbeat-orphaned-active-lease-sweep.test.ts`: 53 passed. - `npx vitest run src/__tests__/server-startup-feedback-export.test.ts`: passed. - `cd packages/db && npx vitest run src/work-signals.test.ts src/client-options.test.ts src/client-teardown-registry.test.ts`: 24 passed. `npx tsc --noEmit` for `packages/db` passed. - Not run locally: the full server `tsc` and the full server suite. The sandbox has a 4 GB memory limit, and `tsc` was killed (exit 137). CI is the gate for the server typecheck, the full tests, and the build. ## Risks - Each server process can open up to 2 more Postgres connections. Deployments that are at their `max_connections` limit must allow for them. The docs now say this. - A renewal attempt is now 4 statements (BEGIN, `set_config`, UPDATE, COMMIT) instead of 1. Each statement is a short primary-key operation on a dedicated connection. - If many runs wait on locked run rows at the same time, renewals for other runs can queue on the 2 private connections. The 10 second statement timeout limits each wait, which is less than the 60 second lease. - Behavior for a real lease loss does not change. A rejected renewal and a non-timeout error still stop the run at once, and the 60 second deadline still applies. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code in agent mode with tool use (shell, file edit, and test execution). A read-only subagent helped with code search. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Nicky LeachandPaperclip
|
e5351b240a |
Verify native Runner setup through plugin tasks (#17029)
## Thinking Path > - Paperclip manages work for AI agents. > - An environment plugin can own a Runner task. > - The merged launch-preparation contract lets the execution owner build the provider configuration. > - Core still needs a task process owner and a setup test that uses that configuration. > - This PR connects the task API to the Runner process-owner contract. > - Operators can test a plugin environment before Core sends it task work. ## Linked Issues or Issue Description **Subsystem affected** Runner, plugin SDK, and agent environment tests. **Problem or motivation** A task-only environment cannot use the shell-based setup path. Core must verify the Runner hello exchange and process cleanup through dedicated task operations. **Proposed solution** Add a lease-bound task driver and use it for a bounded setup probe. Keep heartbeat admission and task execution in #16978. This PR follows the merged launch-preparation contract in #16997. **Alternatives considered** A provider declaration alone cannot prove that Runner and the provider connect. A shell bridge would give the plugin an unnecessary command interface. ## What Changed - Pass structured Runner launch data to process owners. Support an external connection and explicit detach. - Add typed launch authentication checks and checkpoint operations. Check lease, run, project, and retained-task authority. - Connect launch preparation, PRP, termination proof, and credential handoff to the task driver. Reject local environment overrides during installed-account setup. Pass saved-account identity and credential home directly from Core. - Run a short-lived setup probe through the existing agent environment test API. Require a verified provider hello and cleanup. Keep saved Copilot account checks metadata-only through a dedicated task operation. - Update the Copilot source identity because the shared transport source changed. Keep provider behavior and the earlier identity fixture. ## Verification - CI passes on `f987adec7`: 55 successful checks and 5 expected skips. Standard Greptile and Apex are both 5/5 with no open findings. The branch merges cleanly with current master. - Repository typecheck and build pass after the rebase onto the workspace-provider contract. Server typecheck and build also pass after the final setup fix. - The 110 focused lease, SDK, setup, Copilot, and route cases pass after the rebase. All 14 setup cases pass after the final change. These include a real local Runner with a test provider. The saved-agent model-discovery case also passes. - Apex found that Core's relative workspace marker failed the driver path check, and non-Codex sessions omitted the observed model. Both failures were reproduced and fixed. The probe leaves physical workspace selection to plugin preparation. The transport reads each provider's observed model and exposes it to the driver. All 14 setup cases, 6 transport peer cases, and both existing real Runner/OpenCode cases pass. Runner build and server typecheck pass. - The coverage audit removed two redundant cases, repeated fixture code, and two extra real Runner launches. It kept authorization, credential selection, cancellation, lost replies, termination proof, checkpoint identity, and provider setup checks. - The Copilot source-identity check passes. The added-line secret and PII scan passes. The unmerged v52 attestation was regenerated after the shared transport fix. - CI on `13faeff02` caught one stale Copilot digest assertion after attestation regeneration. The literal now matches the candidate attestation, and all 49 installation cases pass. That CI lane otherwise passed 1,818 cases. - An earlier CI run hit an unchanged UI toast timing test. The separate fix is tracked in #16952. Current CI passes that shard. CI also caught a stale model-discovery assertion in this change; the assertion is fixed. - The local full-suite run was stopped when master changed. The final CI run supplies exact-head full-suite validation. No live Cloud or provider test was run. ## Risks - Native hello tests require a saved Core account or the installed account. Local environment overrides are unsupported for installed-account probes and fail before the provider runs. - A native task provider must declare launch preparation and authentication-check version 1. It must keep the checked configuration fixed through submission. Core rejects providers that cannot make this guarantee. - A lost task reply does not prove process termination. Core retains grants until the provider proves termination and Core saves required state. - This PR does not enable native heartbeat dispatch through plugin environments. Cloud deployment and live provider validation remain separate. ## Model Used OpenAI Codex, GPT-6. The exact model ID and context window are not exposed in this session. Tool use includes code editing, local tests, and the GitHub CLI. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal issue details - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
79b2bf78dd |
Create agent records when onboarding names are submitted (#17005)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Core onboarding selects an agent and connects its provider account. > - The current flow creates the agent record after authentication. > - Setup needs a stable agent identity as soon as the user submits its name. > - This PR saves a paused setup record on name submission and finishes the hire on that same record. > - The existing sandbox authentication and approval paths continue to apply. ## Linked Issues or Issue Description **Subsystem affected** Core agent lifecycle, shared validators, and onboarding UI. **Current behavior** Onboarding creates the database record at the final hire step. Earlier authentication and saved setup have no agent record. **Proposed behavior** Create a paused agent with a setup hold when the user submits its name. Do this before adapter selection or authentication. Update that same record after adapter selection. Reuse its ID on retries and saved setup. Finish that record through the existing hire path. **Reason and benefit** Give setup a stable identity without coupling this refactor to computer provisioning or a new execution backend. **Breaking changes** Incomplete setup now leaves an inactive agent record. Existing clients can still create and hire agents without the new optional fields. The sandbox authentication flow and API key support remain in place. **Additional context** Requested Core-only refactor. Related work: #15422 changes runner defaults. This PR preserves those choices. No duplicate early-record refactor was found in the issue and PR search. ## What Changed - Save the agent when the user submits its name in both onboarding entry points. Use a provisional adapter until the user chooses one. Reopen name-only drafts at adapter selection. - Add retry-safe setup creation and same-record hire completion. Keep approval checks at final hire. - Save and restore setup configuration in both onboarding entry points. Show Finish setup for paused setup agents. - Restrict setup changes to authorized board users. Block generic resume until setup finishes. - Add lifecycle, authorization, connection, and UI coverage. Update the specification and feature maps. ## Verification - Latest focused server run: 126 lifecycle and AI-connection tests passed after the Apex fixes. An earlier broader run passed 301 lifecycle, instruction, adapter-validation, and AI-connection tests. Earlier authentication, retry, cross-company, and direct-create regressions also passed. - Name-save verification: 336 tests across the wizard, naming dialog, New agent, action-button, and wizard-step suites passed. Coverage includes retry identity, repeated submission, closing during a save, saved-model restoration, and a disabled process adapter. - Shared, server, and UI TypeScript checks passed. The latest UI package check and direct server TypeScript check passed. The server package preparation command could not finish locally because Cargo is unavailable; full build preparation runs in CI. Map checks, token gates, and `git diff --check` passed; each pushed diff was scanned for credentials and PII. - All six onboarding browser tests passed together with installed Chrome against a fresh throwaway instance after the Apex fixes. They include numbered duplicate names, first-time Copilot without a connected account, and Grok’s saved model. They verify early persistence in both entry points, closing and reopening a name-only draft, same-record adapter selection, sandbox sign-in, login reload, and failed-connection gating. Provider responses are simulated; Core persistence is real. - Regression coverage includes incomplete Copilot creation versus final validation, Kimi settings through redacted responses, stale local drafts, company switches during creation, saved environments, and provider changes with personal/shared old bindings. - Apex found five name-only setup gaps: numbered names, early Copilot account validation, the Grok model default, same-name draft ownership, and provisional runner restoration. The fixes preserve server names, defer implicit account binding until hire, set Grok’s model, avoid name-based reuse of unfinished records, and restore Auto for pending adapter choices. UI, database, and browser regressions cover these cases. - Standard Greptile found a Grok provider mismatch in the new adapter save. The save now uses ACPX with the Grok profile. Both the 129-test dialog/setup run and a real-API browser regression passed after the fix. The first browser attempt stopped on an ambiguous empty-roster button selector; it now scopes the click to the toolbar. - Moving the save point exposed a saved-model reset and timing assumptions in the same-name and portal fixtures. The source choice now preserves the restored model, and tests cover same-name recovery without a second hire. - Initial CI exposed direct-create compatibility, concurrent setup effects, fixture assumptions, and a missing-fields assumption in the provider-validation guard. These were fixed and verified locally. Review findings were addressed with authorization, recovery, credential-persistence, and restored-configuration regressions. - The final desktop capture passed after host resource cleanup. A repeat first failed before Core started because macOS had exhausted shared-memory IDs. Standalone PostgreSQL initialization reproduced the same error. Releasing only unattached control segments from this task’s stopped test databases allowed the unchanged revision to pass. No Core or system setting changed. - Local database tests required port access outside the sandbox. Missing declared development dependencies were supplied outside the branch. No live provider sign-in was tested locally. - [Full monorepo CI](https://github.com/paperclipai/paperclip/actions/runs/39522434236) is green on `82b82fec189a172a885d4dd565ae2f4a18c73efc`. Standard Greptile scored 5/5 with zero new comments (check `118969424903`). The subsequent Apex review also scored 5/5 with zero new comments (check `118972866331`) on that same commit. All review threads are resolved. - The separate Superagent Security scan returned neutral/inconclusive because its pinned merge-base checkout failed or timed out. GitHub returned 404 for check and suite rerun requests. This is not a successful security scan; the CI workflow and other security checks passed. ## Risks - Users can leave inactive setup records behind, including after name submission alone. These records cannot run until hire completes. - Setup completion touches hiring and approval boundaries. Tests cover retries, company isolation, creator permissions, and approval snapshots. - An uncertain credential-save result retains its new secret reference for retry. A confirmed client rejection removes only credentials staged by that attempt. - No schema migration or VM provisioning change is required. Agent-computer integration is out of scope. ## Model Used OpenAI Codex, based on GPT-6, with repository search, code editing, and tool execution. The exact deployment ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] Apex Greptile is also 5/5 on the same commit as standard review and green CI - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d33b40a5c3 |
fix: recover saved work and provide reliable task Retry (#16965)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The control plane must retain work when an execution or workspace export fails. > - Retained-workspace holds can leave saved input waiting with no usable Retry action. > - Users cannot verify process ownership or repair provider allocations. > - This pull request gives system recovery a durable budget and gives each deliberate Retry a durable intent. > - Safe recovery can preserve files, settle an accepted result, or prepare an independent workspace. > - One task state keeps infrastructure diagnostics out of the ordinary conversation while preserving real permission and approval gates. ## Linked Issues or Issue Description **What happened?** A task with a retained workspace can show recovery, failed-run and saved-message warnings at the same time. Historical runs without native workspace descriptors cannot use the native restore path. Old recovery metadata can prevent a new attempt even when an independent workspace would be safe. **Expected behavior** Paperclip owns bounded infrastructure retries and file preservation. A user can request another attempt with ordinary Retry. Thinking appears immediately. The service preserves completed effects and current user controls. It does not claim that starting fresh recovered the old files. **Steps to reproduce** 1. Retain a failed run and its workspace after export or process-stop failure. 2. Save a follow-up while the recovery hold is active. 3. Open the task and try to continue without infrastructure credentials or operator certification. Refs #16906, #16835, #16883. This extends their recovery and receipt contracts. Related work: #16962 and #16963. ## What Changed - Add task-level Retry with company-scoped authorization, all-status idempotency and durable admission before physical work. Keep exact-run Retry compatible. - Add typed recovery phases, finite persisted backoff, historical adoption and durable local failure diagnostics. - Preserve native workspaces or verified historical snapshots. Admit an independent fresh workspace only after execution containment. Keep old-source preservation and cleanup obligations separate. - Carry completed publication and API receipts forward. Fence uncertain mutation retries across replacement runs. Preserve connected-chat identity and authorization. - Re-enter normal dispatch for saved input. Reuse accepted results through the existing finalizer. - Use one task interruption projection across the thread, composer and task list. Show optimistic Thinking, ordinary Retry and genuine user gates. - Add PostgreSQL, filesystem, API, browser-fixture and controlled provider coverage. Document the behavior and update the feature/access maps. ## Verification - All GitHub Actions CI jobs pass on `6d8c6d27d50586d8548b31d03b47f6d73a0ee11d`: typecheck, all server/chat/workspace and serialized suites, Runner checks, eight browser shards, canary and build. The required merge gate passes. - Greptile reviewed the final commit at 5/5, with no actionable findings or unresolved review threads. Its two earlier P1 findings are fixed. - Full recursive typecheck and build pass locally on the final commit. All 16 task Retry cases pass against isolated real PostgreSQL. The unrelated-success case and both stale Retry cases fail on the previous code and pass after the fix. - The corrected gateway fixture passes its uncertain-mutation and independent-read case. Four real HTTP API cases pass for definite 403/422 rejection, completed-effect reuse, and uncertain-effect fences across deliberate and automatic retries. These focused gateway/API checks used a temporary database lifecycle harness on an existing healthy local PostgreSQL process. The standard repository helper is unchanged. - Earlier qualification covers 291 task UI cases, saved-file copyback and ancestry, current Agent Chat ownership races, nine chat Retry API browser cases, and six assembled browser recovery fixtures. A real API/PostgreSQL/native-runner journey showed Thinking in 25 ms despite a delayed response, admitted one successor in a separate workspace, retained original files, and survived reload. - The controlled Daytona test interrupts export, joins the controller, restarts it in a separate process, and verifies one agent start, one external sentinel operation, retained file contents and no premature cleanup. It does not establish abrupt-kill or live LLM/GitHub acceptance. - The full local `pnpm test:run` was attempted and was not green: an OAuth row-lock timing assertion failed, and a later suite exhausted macOS PostgreSQL semaphores. The ten OAuth timing cases passed on rerun against a separate database. Later embedded database initialization failures remain recorded; the final CI matrix passes. - Security scanning passed on production-code commit `63953edf069b111c6becc3144dffa6deb52f68b5`. Its only successor changes one test-fixture identity line. The final scan is neutral/inconclusive because its pinned merge-base checkout failed or timed out. Both rerun API requests returned 404, so no fresh successful scan is claimed. Socket and Snyk checks pass on the final head. - The optional Paperclip Review remains queued behind an earlier run in workspace finalization. Its prior completed assessment was incomplete because a comparison tool was blocked by the platform filter. It is not counted as a completed code review. GitHub does not require this check for merge. - Hosted acceptance remains pending deployment. Local and CI results do not establish recovery of a hosted task or its retained files. ## Risks - Recovery changes span admission, physical ownership and dispatch. Incorrect evidence could permit overlapping work. Existing physical locks and authoritative stop requirements remain in place. - The migration adds concurrent indexes. It requires the existing top-level migration runner and retries incomplete index creation safely. - A fresh attempt can complete while old-source preservation remains unresolved. The service retains that obligation and does not report those files as recovered. - Unknown tool mutations remain blocked until their authoritative receipt owner resolves them. Generic upstream tools have no universal result-query protocol. - Recovery and diagnostics stay inside the stack. No Cloud incident endpoint, notification worker or new outbound reporting channel is included. ## Model Used OpenAI Codex, GPT-6 family. The runtime does not expose a more specific model version or context-window size. The model used reasoning, shell execution, source editing and browser tools. No model subagents were used for this implementation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites; the full local run limitation is documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
6c9b87d092 |
trace(server): expose repository admission waits (#17040)
## Thinking Path > - Paperclip manages work performed by AI agents. > - The controller prepares a task workspace before it starts a provider turn. > - Repository admission also runs when a checkout already exists. > - One observed reuse took 16.877 seconds. Its child spans explained only 3.556 seconds. > - The missing detail covered repeated authority checks, catalog work, and publication waits. > - This change exposes those operations under the existing execution attempt. > - Operators can expand the slow phase before choosing a performance fix. ## Linked Issues or Issue Description Refs #16963, #16330, and #16515. The provider-routing work in #17019 has a different scope. **What existing behavior does this improve?** Repository-admission traces for new and reused checkouts. **Subsystem affected** server/ orchestration and operator-configured OpenTelemetry tracing. **Current behavior** The admission span can contain a long interval without child spans. Existing project checks, HTTP requests, and checkout lookup do not explain current-task checks, database awaits, credential refresh, or publication locks. **Proposed behavior** Expose nested spans for current binding and task access, individual identity reads, catalog construction, credentials, grant transactions, filesystem validation, Git inspection, and directory-lock acquisition, work, and release. **Reason and benefit** The trace can distinguish repeated checks, remote calls, database awaits, and directory contention. A database-await span includes driver, network, pool, and server time. It is not isolated SQL execution time. **Breaking changes** None. No API, schema, permission, retry, lock, or repository behavior changes. ## What Changed - Add fixed admission phases beneath the existing execution attempt. Keep the existing project, checkout, and HTTP detail. - Trace current-task authorization stages only under workspace access. Unrelated decisions cannot consume that detail budget before admission. - Trace GitHub catalog reads, credentials, provider metadata, grant row locking and writes, transaction settlement, and webhook work. - Separate directory-lock acquisition, the protected operation, and release without changing the adapter lock contract. - Document the hierarchy and timing limits. Keep endpoint gating, span caps, and hashed run correlation. Add no span attributes or exception text. ## Verification - Nine new tests passed with real PostgreSQL, real Git, and SDK-exported spans. They cover directory contention, grant-row blocking, all three binding checks, reuse without cloning, dirty files, private-task denial, receipt repair, conflict retry, endpoint-off behavior, and parent IDs. - Twenty earlier unrelated authorization decisions do not exhaust the admission detail budget. - Existing authorization and trust suites: 105 tests passed. Existing timing/context suites: 26 tests passed. Existing repository-access tests: 6 tests passed. - `pnpm -r typecheck` and `pnpm build` passed on commit `5a986d8c86644dfb7ee39b3b832e2d7c86532598`. - `pnpm test:run` could not complete locally. The temporary five-connection cluster returned `sorry, too many clients already` in chat fixtures, and the run was interrupted. It is not a pass. The complete standard CI workflow passed on the same commit, including all 12 server shards and all three chat shards. - Final current-head checks: 54 success, two neutral, two skipped, zero pending or failing. The [CI aggregate](https://github.com/paperclipai/paperclip/actions/runs/39522187940/job/118971261973) passed. [Greptile](https://github.com/paperclipai/paperclip/pull/17040#issuecomment-6305066520) and [Paperclip Review](https://github.com/paperclipai/paperclip/pull/17040#issuecomment-6305017182) both assessed this exact commit at 5/5 with no actionable findings or unresolved threads. - `node scripts/check-module-boundaries.mjs` and `git diff --check` passed. - This macOS host exhausted its SysV semaphore quota. Database tests used a temporary command-scoped initdb preload with five connections, fewer workers, and a larger lock table. No repository test infrastructure or shared host settings changed. Initial skipped tests, focused-fixture failures, and clock-order assertion failures were corrected and rerun; they are not counted as passes. The complete local run remained connection-limited and was interrupted. - Local SDK export proves parentage and privacy. Hosted Honeycomb ingestion remains unverified until deployment. No deployment occurred. ## Risks Span recording requires an operator-configured OTLP endpoint. Each fixed detail name remains capped at 16 occurrences per attempt. Later operations still execute under their enclosing phase. Timing wrappers preserve the existing operations and their errors. No migration is required. The change does not itself reduce admission latency. ## Model Used OpenAI Codex, based on GPT-6, with tool use, code execution, and delegated agents. The exact hosted model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1021.0-canary.1 |
||
|
Nicky LeachandPaperclip
|
8b4ab65b1c |
Add plugin-owned project storage and task workspaces (#17001)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Tasks can use isolated repository workspaces. > - Core currently prepares those repositories on its own filesystem. > - A storage provider may only expose repositories inside remote agent computers. > - Project storage and task workspace preparation must have one provider owner. > - This pull request adds that contract and keeps execution as a separate plugin interface. > - Core keeps authorization, durable task bindings, and lifecycle fences. ## Linked Issues or Issue Description **Subsystem affected** Plugin SDK, project workspace policy, heartbeat preparation, execution leases, and workspace lifecycle. **Problem or motivation** Core cannot prepare a task workspace when its repository storage is unavailable on the Core host. An execution lease must not be required just to prepare persistent storage. **Proposed solution** Let one plugin provide project resource lifecycle and workspace prepare, inspect, restore, and remove behavior. Core reserves a task workspace before it calls the plugin. The plugin returns an opaque durable reference. The matching execution plugin receives that reference on its task RPC. **Alternatives considered** Preparing repositories on Core would require a provider mount on Core. Preparing them only during task launch would tie persistent workspace lifecycle to execution leases and agent availability. **Roadmap alignment** This extends the remote provider model in ROADMAP.md. Related execution work is in #16978. This PR does not replace that task bridge. Native execution of managed storage requires the bridge; this change fails closed when it is absent. ## What Changed - Add one storage provider declaration with supported modes/strategies, a capability, a validated RPC contract, and a test harness. - Persist provider ownership, repository revision, and execution-workspace identity before external effects. Reconcile retries with a stable operation ID. - Prepare provider storage before environment acquisition. Keep Core filesystem preparation out of this path. - Pin the durable reference to the matching execution plugin and validate the lease binding before task dispatch. - Route inspection, reopen, and removal through the existing lifecycle process. Keep local data when remote recovery is uncertain. - Keep the application database pool free during lifecycle locks. Preserve pending workspace selections, validate source edits before retirement, and retry unfinished closed storage cleanup. - Protect provider metadata and live repository sources from caller edits. Storage edits retire only closed records with confirmed removal; retired records cannot reopen. ## Verification - Current head `8214b3ca5`: all 25 provider lifecycle tests pass with `maxConnections=1`. They cover preparation, inspection, reopening, removal, a paused provider with responsive ordinary SQL and a competing source edit, pending selection preservation, rejected source edits, cleanup retry scheduling, and providers that cache operation results, and committed cleanup notifications for storage intent and recovered state. - Final frozen-tree lifecycle run on `8214b3ca5`: all 318 tests passed in 10 files, covering workspace services, reopen, compaction, task binding, project sources, heartbeat preparation, display caching, and source locks. - Direct server `tsc --noEmit` passes. The SDK build and all 128 SDK tests passed during initial verification; later SDK changes are documentation only. - Full CI on the previous head `ec5c7209a` passed all 55 applicable checks; 2 conditional checks skipped. Standard Greptile reviewed current head `342e6d9c3` at 5/5 with no unresolved threads. All 55 applicable CI checks passed on `342e6d9c3`; 2 conditional checks skipped. The Slack browser and CLI startup jobs passed on their single unchanged-head rerun. Its Apex review returned 4/5 with one pending-restoration issue. Commit `1b58943b1` reconciles that durable intent before cleanup, advances the generation without reopening the Core row, and blocks direct removal or retirement while intent remains unresolved. Standard Greptile reviewed `1b58943b1` at 5/5 with no new findings. All 54 applicable CI checks passed on `1b58943b1`; 3 conditional checks skipped. The chat job passed on its single unchanged-head rerun. Apex then returned 4/5 for a missing cleanup wake after recovery. Commit `8214b3ca5` atomically registers cleanup intent before storage intent and recovery-result writes. Its provider test observes real committed notification hints and the visible advanced generation. Standard Greptile reviewed `8214b3ca5` at 5/5 with no new findings. All 55 applicable current-head CI checks passed on the first run; 2 conditional checks skipped. Final Apex verification completed at 5/5 on this exact head: 38 files reviewed, no new comments, and no unresolved threads. The PR is ready for review and mergeable. - The first Apex review found five issues after a standard 5/5: application-pool starvation, stale pending selections, retirement during rejected source edits, lost cleanup retries, and reused inspection identities. Commits `6d72e7fd4` and `342e6d9c3` fix each with regression coverage. Archive lock waits use dedicated connections, and source edits fail promptly if the lifecycle is busy. Retirement compares normalized values. The normalized-source regression initially failed with the raw-value comparison; after moving the check after normalization, a clean provider-suite rerun passed all 24 tests at that stage. Those threads are resolved; the earlier Apex 0/5 is not a passing acceptance result. - Earlier local integration attempts could not bind localhost inside the sandbox. The same suites passed with permission to start local PostgreSQL. - The earlier broad local `pnpm test:run` overlapped edits and reported stale-revision and checkout-path/Git-fixture failures. It was not a final-tree result. Full CI subsequently passed; the original local failure remains part of verification history. - Local repository-wide typecheck/build cannot complete because `cargo` is unavailable. Direct server TypeScript checks and affected tests pass; full Rust and build validation comes from CI. - An earlier browser run on `ff918c5a8` failed in the unchanged case-filter journey: expected “Done,” received “All statuses,” with an announcement handler recorded during the assertion. The cause is not confirmed. [The failed job](https://github.com/paperclipai/paperclip/actions/runs/39500611857/job/118896954508) remains verification history. The unchanged shard passed in the next full CI run on `ec5c7209a`; no UI or fixture change was made. - Current-head browser shard 2 failed waiting five seconds for the mocked Slack setup Success heading. This PR changes neither that UI nor its fixture. [The failed job](https://github.com/paperclipai/paperclip/actions/runs/39506070513/job/118916021232) remains verification evidence. The cause is not confirmed; the single unchanged-shard rerun passed; no UI or fixture change was made, and the original cause remains unconfirmed. - The workspace test shard passed its 3,978 general tests, then its CLI company-import/export fixture timed out after 30 seconds waiting for startup health before creating any company or workspace. [The failed job](https://github.com/paperclipai/paperclip/actions/runs/39506070513/job/118916021855) remains verification history. The child reached external-PostgreSQL startup without exiting or reporting an exception. Its cause is not confirmed; the single unchanged-job rerun passed, including the CLI fixture. No startup or fixture change was made, and the original cause remains unconfirmed. - The new cached-removal regression initially failed because its fake worker advertised availability for unrelated fixture plugins, so another workspace changed its fake checkout state. Worker availability now matches the fixture plugin; the complete 25-test provider suite passes. - On `1b58943b1`, the chat shard timed out polling for one conversation after an authorized GitHub description edit. [The failed job](https://github.com/paperclipai/paperclip/actions/runs/39510521657/job/118930193418) remains evidence. The exact case passes locally against PostgreSQL (1 passed, 1,380 unrelated cases skipped), and this PR changes neither that test nor its handler. The original cause is unconfirmed; the single unchanged-job CI rerun passed without a code or fixture change. The original cause remains unconfirmed. - The diff was checked for credentials, private references, and generated artifacts. ## Risks This is an additive provider contract with no database migration. Existing projects keep their local behavior unless their policy selects a storage plugin. A selected provider must be ready and enabled. Unknown or unsafe cleanup fails closed. Managed native execution depends on #16978. Core refuses a host Runner fallback. Providers must fence lifecycle generations, preserve ambiguous retries, and report remote recovery honestly. Provider-specific live storage and guest mount qualification remain separate from these Core contract tests. ## Model Used OpenAI Codex (GPT-6), with repository inspection, code execution, and tests. The exact model variant was not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run tests locally and they pass (affected suites; broad-run limits are documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1021.0-canary.0 |
||
|
Devin FoleyandPaperclip
|
572bef4ba7 |
fix: omit re-downloadable Codex remote plugins from failover backups (#17026)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Native Codex sessions in remote sandboxes copy the sandbox's Codex
home into a host-side failover backup after every turn
> - Ephemeral sandboxes restore that backup at the start of every turn,
so its size affects both disk use and turn latency
> - Most of each backup is remote plugin data that Codex itself treats
as a cache and re-downloads on startup
> - This pull request leaves that data out of Codex checkpoints, keeping
everything a resumed session needs
> - The benefit is backups roughly 80–90% smaller, less data moved per
turn, and much slower disk growth
## Linked Issues or Issue Description
Bug report described in-PR (bug report template). Related: #16866
(expires failover backups after their issues close).
**What happened?**
Failover backups grow quickly because each Codex checkpoint includes
`$CODEX_HOME` remote plugin data. On one instance, 204 backups held 22.1
GB:
- `codex-home/plugins/cache/openai-curated-remote`: 8.3 GB (about 29 MB
per backup);
- `codex-home/cache/remote_plugin_catalog`: 6.1 GB (one 38 MB catalog
file per backup);
- `codex-home/sessions` (the conversation history): 0.31 GB.
About 80 new backups (9.8 GB) were written per day. Ephemeral sandboxes
also copy all of this into a fresh sandbox before each turn.
**Expected behavior**
Checkpoints keep what a resumed session needs and omit data that Codex
re-downloads itself.
**Steps to reproduce**
1. Run a Codex agent with ChatGPT auth in a remote sandbox environment.
2. Inspect
`<instance>/runtime/paperclip-runner/durable-sessions/*/failover-backups/current/codex-home`.
3. Most bytes are under `plugins/cache/openai-curated-remote` and
`cache/remote_plugin_catalog`.
**Paperclip version or commit:** `master` at
canary/v2026.1020.0-canary.15
|
||
|
DottaandPaperclip
|
f89c37791c |
Add campaign sidebar and case navigation to runner E2E reports (#17010)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E campaigns test runner behavior across agent profiles and environments. > - The retained reports contain many suites, cases, and execution previews. > - A long report makes it hard to find a case or return to the campaign context. > - This pull request adds campaign navigation, case breadcrumbs, and separate global history pages. > - Users can scan all execution previews or inspect one case without losing their place. ## Linked Issues or Issue Description **What existing behavior does this improve?** Navigation in the retained Runner E2E campaign reports and history homepage. **Subsystem affected** Test report infrastructure in `tests/runner-e2e/`. **Current behavior** Reports show a long configuration matrix. The suite index and campaign history share the same page. Case links do not provide a focused inspection view. **Proposed behavior** Global overview, history, and attention pages show global navigation. A specific campaign adds a folding suite tree. All results keeps the full scrollable report. A case opens with breadcrumbs and an execution picker. **Reason and benefit** Users can find a case, compare its executions, and return to their reading position. Global pages do not show suites from an unrelated campaign. **Breaking changes** The report HTML layout changes. Retained result schemas, execution identities, and suite and execution anchors remain unchanged. **Additional context** Refs: #16996. This extends the merged report navigation and local preview work. The duplicate PR and issue search found no open PR for this change. ## What Changed - Add shared report navigation with the Paperclip brand mark and wordmark. - Scope the suite tree to the current campaign. Keep global overview, history, and attention pages separate. - Group all execution previews by suite and case. Reuse the existing evidence cards and image gallery. - Add focused case inspection, configuration selection, breadcrumbs, filter handling, and scroll restoration. - Move historical trends to the global history page. Keep billing details, missing results, and failure evidence available. - Update report documentation and browser regressions for filters, status dots, configuration reloads, breadcrumbs, and scroll restoration. ## Verification - [Final CI](https://github.com/paperclipai/paperclip/actions/runs/39500401411) passes on `89a8b5c290de3cfb9bdc061be3c88a29fd160a62`. Typecheck, build, runner checks, server suites, and all browser shards pass. - All four report browser regressions pass. They cover filters, gallery navigation, repeated suite jumps, the folding sidebar, breadcrumbs, scroll restoration, configuration selection and reload, mobile navigation, and aggregate case status. - The 63 report, history, preview-boundary, and navigation tests pass. Runner E2E typecheck passes with `node node_modules/typescript/bin/tsc -p tests/runner-e2e/tsconfig.json`. - Manual browser checks covered desktop and a 390-pixel viewport. Retained campaigns covered 209 executions in eight suites, 179 executions in seven suites, and 15 executions in one suite. Browser fixture checks also confirmed filtered inspection and aggregate status. No paid evaluations ran. - The initial report browser tests still targeted the old layout and routes. They were updated for the new navigation. Review also exposed filtered-inspection and aggregate-status defects; both were fixed before the final passing browser run. - The unchanged Cases UI grouping test [failed once on the final head](https://github.com/paperclipai/paperclip/actions/runs/39500401411/job/118898559197). Its full file also failed locally, while the isolated test passed. The same UI code passed in earlier CI. A [single CI retry](https://github.com/paperclipai/paperclip/actions/runs/39500401411/job/118903333430) passed all 3,978 UI tests and 286 CLI tests without code changes. The cause remains unresolved. - Two jobs on an intermediate head were interrupted by runner shutdown signals. Those remain interrupted results. The complete final-head CI run supersedes them for merge verification. - Workspace `pnpm -r typecheck` and `pnpm build` also passed locally before the rebase. The full local `pnpm test:run` attempt encountered repeated embedded PostgreSQL startup failures and was stopped. It is not a passing local full-suite result; final CI provides the full verification. ## Risks - Navigation and hash handling can select the wrong case or hide results. Focused tests and browser checks cover these paths. - Historical campaign links must resolve from both the preview and published report locations. - The report layout changes. Graders, prompts, retained outcomes, and billing calculations do not change. ## Model Used - OpenAI Codex with `gpt-6-astra` and `gpt-6.1-sol`. Used reasoning, repository tools, code execution, and browser verification. The runtime did not expose a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1020.0-canary.14 |
||
|
Nicky LeachandPaperclip
|
2a5a323e57 |
Add execution-owned Runner launch preparation (#16997)
## Thinking Path > - Paperclip manages work for AI agents. > - Runner can execute in a different environment from Core. > - Core owns provider selection, policy, and resource assignments. > - The execution environment must prepare its own files and paths. > - A shared contract must keep these responsibilities separate. > - This PR adds that contract to the Runner execution driver. ## Linked Issues or Issue Description **Problem or motivation** Runner transport prepares provider files on the controller before launch. An execution owner cannot prepare those files in its own environment through the existing driver. **Proposed solution** Add `prepareLaunch` to the execution driver. Supply versioned Core requirements. Accept a configuration ID, requirements digest, and ordinary provider descriptor. Validate Core policy and assignments before launch. Keep existing provider session checks. **Alternatives considered** Provider-specific ownership flags would spread the boundary across backends. Shell preparation would expose process details in the plugin API. Refs #16993, #16977. Both prerequisites are merged. This PR now targets master. Native plugin integration and execution-host preparation follow separately. PR #16978 contains an earlier, unmerged prototype. Its `persistentAgentHome` mode and skill path-map call will be replaced by this preparation contract before activation. This PR preserves current master callers of the skill helper. ## What Changed - Add the optional preparation callback and an explicit Runner artifact identity. - Keep the built-in preparation path as the default. - Validate provider policy and resource assignment identity. Preserve instruction text when resource locations change. - Persist the attachment seed after an authority epoch rotation, so a later controller restart can recover the same run. - Preserve explicit context removal on a new run. Only an omitted context can reuse the saved context. Remove its registered instruction framing when context is removed. - Save exact requirements and configurations in a private file outside Runner commands. Retain the active record across failed attachments. Use current grants for new runs. Keep incompatible session changes rejected. - Join delayed preparation during startup and warm attachment cleanup. Retain an unsettled checkpoint after a timeout. Prevent work after cancellation. - Use the supplied Runner artifact for interrupted handoff recovery. Restore the exact target configuration at each handoff crash boundary. Check the Runner state when the final ACK may be lost. - Bind the changed shared sources to Copilot profile v51. Preserve the v50 receipt and current integrity checks. ## Verification - Workspace typecheck passed. - All 487 affected Runner transport, control-plane, preparation store, builder, policy, and profile tests passed on commit `eaf8ae8d1`. - Workspace build and typecheck passed again on commit `eaf8ae8d12c7ef2d22d226e4b3f05fdbf79ef4cb`. - Both warm attachment cleanup regressions fail before the fix. They pass after the fix. The tests cover delayed preparation, timeout retention, and concurrent attachment rejection. - The routed handoff regression fails with a missing controller binary before the fix. It passes after the fix. The execution owner launches the real fixture Runner. - The 200,000-byte instruction regression fails before the fix and passes after it. Four prepared handoff crash cases verify exact request and configuration recovery. The store tests verify private file mode, conflicting records, corrupt data, and active-run retention. - A real Runner with a controlled fake provider verifies execution-host paths, exact recovery, warm reuse, incompatible profile rejection, and cancellation. It does not contact an external provider. - The regression found undefined settings in strict PRP command hashing. Requirements now contain only JSON values. The warm transition test passes without changing PRP validation. - The broader root suite on the prerequisite finished with 22,420 passing tests, seven failed tests in unchanged Git and diagnostic code, and two suite setup errors because Cargo was absent from its PATH. This PR does not claim a passing full root suite. - The CI Build job passed compilation and production runtime boot, then its runner received a shutdown signal during the clean public install check. Other jobs reported lost communication with their self-hosted runners. The failed jobs passed on retry on the same commit; successful jobs kept their results. - The current Superagent security scan is inconclusive because its pinned merge-base checkout failed or timed out. GitHub rejected the check rerun request through the available API. This is not a passing security scan. - Standard Greptile and Greptile Apex each returned 5/5 on `eaf8ae8d1`. All review threads are resolved. CI passed on attempt 2. Current checks have 55 successes, four optional skips, and the inconclusive security scan described above. The branch is ready, mergeable, and current with master. - No live provider qualification or deployment was run. ## Risks - A preparation owner must store configurations and keep retries bound to the same task and requirements. - Copilot source binding requires a new profile identity. This PR does not claim fresh live qualification. - The callback is an SDK extension. This PR does not connect it to a plugin task process owner or add a Launcher endpoint. ## Model Used - OpenAI Codex, GPT-6. Reasoning and repository tools. The exact serving model identifier and context size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
15176963b1 |
fix: stop idle skill-source polling (#17008)
## Thinking Path > - Paperclip manages agent work and the skills that agents use. > - Project repositories create durable import jobs, and installed sources can refresh automatically. > - The server queried these records every five seconds, even when no work existed. > - Idle tenant databases need to stop receiving those discovery queries. > - This change uses the existing transaction notifications and delivery coordinator. > - Saved refresh, retry, and lease deadlines determine when the worker runs next. > - Cloud does not yet wake these deadlines, so outstanding skill work retains the instance. ## Linked Issues or Issue Description **What existing behavior does this improve?** Skill-source background synchronization. This continues the idle-work migrations in #15625 and #17000. A search found no overlapping open polling-removal PR. **Current behavior** On master `b11010b0d4`, startup installs a five-second skill-source interval and also calls it from the external-object scheduler. Each sweep queries active companies, import capacity, pending imports, and automatic refreshes. **Proposed behavior** Run at startup, after a relevant transaction settles, or at a saved deadline. Stay SQL-quiet when no eligible work remains. Preserve import admission, authorization, atomic claims, retry backoff, and the existing five-minute upstream check. **Reason and benefit** Remove recurring database activity from inactive tenant instances without requiring new tables or a new scheduler. ## What Changed - Replace both recurring skill-sync calls with one delivery-coordinator registration. - Notify from attachment/detachment, source creation/settings/manual refresh/disconnection, project/company deletion, and company status changes, including budget resume. - Restore the earliest eligible `nextSyncAt` or lease-expiry deadline. Due backlogs and conflicts retry after five seconds; no idle recovery cadence remains. - Admit deferred imports again after a batch frees capacity. Preserve requests through drain, standby, restart, and commits during a sweep. - Add sleep-safety predicates for imports and eligible automatic sources until Cloud supports these deadlines. - Document the lifecycle and process-local writer contract. No schema or API changes. - Guard external test database URLs before migrations or cleanup: require a `_test` database name and reject database/options overrides. ## Verification - 147 targeted tests pass: 39 server-startup tests, 14 new real-PostgreSQL lifecycle tests, 20 shared coordinator/scheduler tests, 63 existing import/refresh/retry regression tests, and 11 test-database guard cases. - CI exposed a stale startup mock without the new `start` method. The fixture is updated and all 39 startup tests pass locally. - Tests exercise real enqueue/settings/company-reactivation producers, nested commit, rollback, saved retry/lease restart, manual refresh failure, capacity admission, concurrent commit, shutdown cancellation, and SQL quiet after startup or pause. - Sleep-safety tests reject sleep for future imports and automatic refreshes, and accept inactive retained state. - `pnpm -r typecheck` and `pnpm build` pass. - `pnpm test:run` was attempted and stopped after repeated embedded-PostgreSQL startup failures in unrelated suites. The two existing import/refresh suites initially skipped for the same host limitation; after adding explicit disposable-database options, all 63 regression tests passed against PostgreSQL 17. Full CI passed on `ddb3ee51369baa8c7153bae9fd88efb8b903f2bc`: 55 successful checks, one neutral review check, and two intentional skips. ## Risks - Notifications are process-local. Independent SQL writers and separate live replicas do not wake this worker; startup reconstructs persisted work. - Enabled automatic sources still contact GitHub every five minutes. Their upstream changes have no event ingress here. - Cloud scheduled wakes currently cover routines and issue monitors, not skill sync. This change retains instances with outstanding skill obligations. It does not claim measured host/database suspension or fleet savings. - No change to source authorization, installed content on failure, or existing claim semantics. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, and code execution. The host does not expose an exact model revision or context-window size to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1020.0-canary.13 |
||
|
DottaandPaperclip
|
b11010b0d4 |
Simplify failure banners and show only the latest error (#16982)
Remove dashed failure dividers and show only the latest adjacent run or recovery error. Preserve permitted retries, pending state and unrelated notice boundaries. Add production-component Storybook examples and browser regression coverage. Co-Authored-By: Paperclip <noreply@paperclip.ing>canary/v2026.1020.0-canary.12 |
||
|
DottaandPaperclip
|
cb2d6e80d3 |
perf: stop polling idle Muse maintenance (#17000)
## Thinking Path > - Paperclip manages AI agents and their work. > - Muse connects an external worker to a Paperclip agent. > - Its maintenance loop scans the database every five seconds, even without bindings. > - These empty scans prevent an otherwise idle database from sleeping. > - The shared delivery coordinator can react to committed writes and saved deadlines. > - This change uses that coordinator and retains recovery for unfinished assignments. > - Idle Muse maintenance can now stop querying the database. ## Linked Issues or Issue Description **What existing behavior does this improve?** Muse qualification, OAuth authority, and terminal assignment maintenance. **Current behavior** A claimed instance runs two empty scans every five seconds even when Muse has no bindings. This produces about 34,560 unnecessary queries per day. Warm standby suppression from #16974 does not cover claimed instances. **Proposed behavior** Recover once at startup. Wake after relevant transactions settle. Schedule qualification expiry at its saved deadline. Retain five-second recovery only for failures and unfinished assignments. **Reason and benefit** Remove recurring database traffic from idle Muse maintenance while preserving authority checks and recovery. Reuse the database work signal foundation from #15625 without adding tables. ## What Changed - Register Muse maintenance with the shared delivery coordinator and its idle admission rules. - Signal pairing, OAuth attachment and revocation, qualification changes, and new assignments before transaction writes. The existing transaction layer waits for outer commit. - Restore qualification deadlines from persisted bindings. Retry failed sweeps and unfinished assignments. - Add PostgreSQL tests for idle quiet, nested commit, rollback, deadlines, standby, drain, and failure recovery. Verify signals from actual Muse producers. Preserve startup, newly committed work, and failed sweep retries across idle drain without SQL polling. - Document the remaining recovery cadence and sleep limitations. ## Verification - `pnpm -r typecheck`: passed before rebase. Server typechecking also passed after rebase and the drain fix. - `pnpm build`: passed before rebase. - Final change: 33 maintenance, delivery coordinator, and scheduler tests passed. Maintenance used a fresh disposable PostgreSQL 17 database. - Muse control-plane tests: 254 passed against the same disposable database. - `pnpm test:run`: attempted, then stopped after repeated embedded PostgreSQL startup failures from this host's semaphore limit. Native-session, Muse, and AI-connection suites could not create local clusters. The complete suite runs in CI. - `git diff --check`: passed. - Final commit `e106cbf20d`: all CI gates passed, including complete test suites, E2E, build, production bundle boot, public-install smoke test, and canary dry run. Greptile and Paperclip source reviews both rated 5/5, with no unresolved findings. The focused suites accept `PAPERCLIP_MUSE_MAINTENANCE_TEST_DATABASE_URL` and `PAPERCLIP_MUSE_CONTROL_TEST_DATABASE_URL`. Use a disposable test database; fixtures remove test data. ## Risks - Unfinished assignments retain five-second recovery. This change removes idle polling, not all Muse polling. - Signals are process-local. Independent database writers need explicit integration. Startup recovers persisted state. - Qualification deadlines depend on the running process. Existing durable Muse sleep blockers remain unchanged. - The receiver keeps its batching timer. An empty buffer already makes no database calls. - No schema migration or API change. ## Model Used OpenAI Codex, GPT-6 family, with code execution and repository tools. The exact model version is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites; full local run has the environment limitation above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
2b16262280 |
Improve runner E2E report navigation and local previews (#16996)
## Thinking Path > - Paperclip is the open source app that people use to manage AI agents for work. > - Product E2E campaigns check user journeys across agent profiles and environments. > - Readers use the campaign history and reports to inspect results and retained evidence. > - The reports showed image summaries, many unselected suites, and a long sticky index. > - Individual case cards also showed routine diagnostics before the visual evidence. > - This pull request adds text summaries, feature filters, folding case indexes, and simpler cards. > - A local preview uses retained public data so designers can change reports without another paid campaign. ## Linked Issues or Issue Description **Current behavior** The report is hard to scan as the journey catalog grows. Suite links can point into hidden results. Case indexes do not show whether a case passed across its configurations. Design changes also need a realistic local preview. **Proposed behavior** Show the selected campaign as text. Link journeys to current feature-map capabilities. Add folding case indexes with status indicators, reliable section links, and a back-to-top control. Keep failed outcomes visible and put routine diagnostics under Test details. Render retained public campaign data with local templates. **Reason and benefit** Readers can find a feature or failed case faster. Designers can iterate against real retained data without starting an application instance or using provider credentials. **Breaking changes** No result schema or grading change. The report stops embedding the generated summary PNG. The publisher still produces that artifact for existing consumers. Related work: #12889 added the existing matrix browser. #12846 concerns the public image publication boundary. This PR changes report presentation and local preview behavior. ## What Changed - Replace image summaries with shared HTML suite summaries on the history and campaign pages. - Show selected executions and preserve missing results for selected cells. - Add explicit journey-to-capability associations, feature filters, and a coverage view with unmapped journeys. - Add folding case indexes, section case lists, and red, green, or amber status indicators across matching configurations. - Fix anchor navigation through filters and add a floating back-to-top control. - Simplify case cards. Keep failure reasons visible. Fold routine checks, billing, and context into Test details. - Add an original-image link to the gallery. - Add a local preview utility with public data caching and offline mode. Document the direct Node command. - Test manifest-listed evidence, rejection of unlisted files and encoded traversal, cache reuse, and offline misses with a real loopback server and a stub downloader. ## Verification - Focused report, history, and preview tests: 75 passed in 6 files. - Report Playwright spec: 4 passed in Chrome with a standalone config that does not start the application server. - `node scripts/check-maps.mjs`: passed. - `git diff --check`: passed. - Manually checked retained campaign reports, feature filters, folding indexes, failure links, card details, gallery links, mobile layout, and keyboard navigation. - `pnpm build`: passed. - `pnpm -r typecheck`: passed. - `pnpm test:e2e:runner:typecheck`: passed. - `pnpm test:run`: stopped after 15 native session database cases failed. The local Postgres package first lacked dynamic-library links. After link hydration, the same cases still failed. A direct `initdb` check then reported `could not create semaphores: No space left on device` and identified the macOS semaphore limit. The complete local suite has not passed. - `pnpm test:e2e:runner:unit`: 3,250 passed, 6 failed, 2 skipped. All six failures reproduced against unchanged `origin/master` report sources with the same installed dependencies. They concern Copilot admission and its Daytona image input, plus the launcher test that runs those assertions. No report or history test failed. - No live provider evaluation was run for this presentation change. - CI on `b1a9c06823` first failed browser shard 3/8. The agent-chat split-handoff test reached cleanup, where cancelling a run returned HTTP 500 from a Postgres deadlock. Cleanup did not restore the experiment setting, and a later test failed its initial setting check. This shard does not contain the report spec. One targeted rerun passed, and that full workflow then passed. This shows recovery; no production fix was made for the deadlock. The later preview-test commit requires its own fresh checks. To review the design, run `node --watch --import ./cli/node_modules/tsx/dist/loader.mjs tests/runner-e2e/preview.ts`. Open the history address printed by the command, then open a campaign. The README covers offline mode and cache refresh. ## Risks - Feature associations use the current map. They describe linked scenarios, not proof that a whole capability is covered. Historical grades remain unchanged. - New or unfamiliar cases remain unmapped until their assertions are inspected. - The preview needs the public history host on first access. Offline mode requires cached files. It binds to the loopback interface and serves manifest-listed public evidence. - Hiding routine diagnostics adds one disclosure step. Failed or incomplete outcomes remain visible. This is report polish. It does not duplicate planned core work in `ROADMAP.md`. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use, code execution, and browser automation. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
4145b5048a |
docs: deepen task, approval, secret, and recovery maps (#16994)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its feature map describes user journeys. Its access map describes the matching agent paths. > - The recent index and capability additions improved discovery and breadth. > - Broad baseline records still combined operations with different actors, permissions, and retry behavior. > - This pull request reviews the existing task, approval, secret, and recovery capabilities at operation level. > - Each operation now has its own access boundary, source evidence, and verification recipe. Source review remains separate from live proof. ## Linked Issues or Issue Description Refs: #16987, #16990, #16025. These related pull requests are merged. The search found no open duplicate or matching issue. **Issue type** Missing documentation. **Where is the issue?** The paired task, questions-and-approvals, secrets, and recovery files under `maps/feature-map/` and `maps/agent-access-map/`. **What's wrong?** Baseline records do not show enough detail to verify who can perform each action, which state must persist, or what happens after a denial, race, or retry. A declared scenario tool can also be mistaken for an admitted production tool. **Suggested fix** Split the four areas into source-reviewed operation records. Add compact checkpoints to their user recipes. Preserve existing capability IDs and the map schema. ## What Changed - Promote 27 existing capability records to `operation-reviewed`, with 125 operation records: tasks 32, questions and approvals 39, secrets 28, recovery 26. - Give each operation its own API, agent surface, policy, source evidence, verification recipe, and remaining gap. - Separate human, legacy agent, run JWT, native binding, mode, and resource authority. Split operations where those rules differ. - Correct production completion paths, plan acceptance and handoff, question dismissal and withdrawal, secret proposal authority, runtime recovery grants, and explicit duplicate-risk decisions. - Preserve access gaps found in source, including the binding-proposal dependency cascade and narrower native/CLI fields. - Add one checkpoint table per product recipe for saved results, denial, races, and retries. - Keep all 191 existing capability IDs. Change only eight Markdown files. Keep every operation's evidence `source-only` and verification `not-run`; the subset below does not prove an entire operation across its actors and surfaces. ## Verification Source review uses revision `3bbc2104b65caf9f7763938a01260a5673e12f7a`. Read-only area reviews and a second source review covered the changed pairs. Passed: - `node scripts/check-maps.mjs`: 35 paired areas, 191 capability IDs, valid references and fields. This is structural verification. - `node --test scripts/check-maps.test.mjs scripts/storybook-feature-index.test.mjs`: 17 passed. - `node scripts/storybook-catalog.mjs`: 308 files, 2,899 stories. Generated outputs remain ignored. - Capability comparison with the base: all 191 IDs unchanged; the four reviewed areas contain 27 capabilities and 125 source-only operation records. - `pnpm -r typecheck` and `pnpm build`. - `git diff --check`. - Focused existing interaction and proposal route tests: 80 passed. The two selected DB-backed suites skipped 119 tests because local embedded PostgreSQL could not start. These skipped tests are not passes. Live subset on a fresh local instance with a disposable PostgreSQL container: - Created the company and an unassigned task through the browser. Added a comment and completed the task. The saved status and comment survived reload. - Created a managed secret with inert dummy data through the browser. Its metadata and version survived reload. The metadata response did not contain the dummy value. - Created a question fixture through board REST. Invalid input returned 422 and left it pending. Browser dismissal kept it pending and preserved the draft. Reopening restored the choice. Submission survived reload. A duplicate response returned 409 and left the saved answer unchanged. - Used a real issued legacy agent API key. Task read and safe company-secret catalog returned 200. Full secret listing and run-only secret list/value routes returned 403. Revoked the key after the checks. - Read an empty recovery projection. Resolving a nonexistent action returned 404 and left the task unchanged. This did not execute recovery. - Closed the owned browser tab and stopped only the verified test runtime and labelled database container. Evidence survived cleanup. No provider or model ran. Limits and retained failures: - Native embedded PostgreSQL launch failed because the host could not allocate semaphores. The container-backed app was the successful alternate environment. - Full `pnpm test:run` was attempted. Fifteen DB-dependent native-session cases failed at database startup, and several DB suites skipped. The run was stopped after repeated failures of that prerequisite. The full local suite did not pass. - The sandboxed build attempt failed at the local tsx IPC socket. The required build passed outside the sandbox. - The first CI build job passed compilation and production boot, then its npm-install check was interrupted when the runner received a shutdown signal. The corrected head passed the full CI run, including the build and public-install checks. - Initial Greptile review found the recovery note field and native search/history limits were inaccurate. Corrected `resolutionNote` and the production cap of 100, then reran the map checks. Greptile reviewed the corrected head at 5/5; both earlier threads are resolved. - An initial harness used a nonexistent generic secret-value URL and received 404. The actual mapped agent routes were then checked separately. An initial recovery assertion expected 409; the actual missing-action contract returned 404. These were harness corrections, not product fixes. - Native run bindings, two-person ownership, secret delivery/rotation, external vaults, provider-stop proof, workspace repair, and actual recovery execution remain unexecuted. The map does not claim those journeys passed. The separate external **Paperclip Review** check is still in progress on `904509df80cc6de16d36906a410a66786f619b20`. The reviewer reports that its full changed-file request is pending. CI is green and Greptile is 5/5, but this external review has not published a final assessment. Its review service must finish that request and assessment. ## Risks Documentation can become stale as policy and runtime bindings change. Each operation cites source and retains its verification gaps. The records describe observed enforcement; they do not authorize broader access or certify live behavior. This pull request changes no product code, schema, or permission enforcement. ## Model Used OpenAI Codex, `gpt-6.1-sol`; 475,000-token context window; xhigh reasoning; tool use and code execution. Read-only source agents assisted the area reviews. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused map and route checks; full-suite host blocker recorded above) - [x] I have added or updated tests where applicable (existing map regression tests cover this documentation-only change) - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1020.0-canary.11 |
||
|
Devin FoleyandPaperclip
|
a2d26a83fe |
fix(inbox): keep automatic work out of Mine by default
Keep channel and routine attribution out of Mine unless the person participates directly, is assigned the task, or receives a current addressed request. Apply the same source rules to unread state, archives, and task notifications. Preserve direct comments and current human action requests. Restore the two expressions damaged by redaction during the earlier master merge. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
dc6dd23ea4 |
Add plugin Runner preparation task contract (#16993)
## Thinking Path > - Paperclip manages work for AI agents. > - Environment plugins can own Runner tasks. > - Core selects provider policy and runtime assignments. > - The execution environment must resolve its own paths and configuration. > - The task API needs a preparation exchange before execution. > - This PR adds that exchange under the existing lease authority. ## Linked Issues or Issue Description **Problem or motivation** The environment task API can start a Runner. It cannot return a prepared provider configuration from Core requirements. **Proposed solution** Add a versioned `prepare` request and `prepared` result. Keep the provider configuration bound to its task and requirements. Check the same company, run, project, and retained-session authority as submission. **Alternatives considered** A shell preparation command would expose execution details in the API. Provider-specific ownership flags would spread this contract across backends. **Roadmap alignment** This extends the existing plugin and remote execution APIs. It does not add a separate execution system. Refs #16977, #15406. The prerequisite #16977 is merged. This PR now targets master. Runner transport and provider implementations follow separately. ## What Changed - Add strict launch requirements and prepared result schemas. - Add the preparation capability version and saved configuration ID. - Apply existing lease and resource checks to preparation. - Document idempotency, cancellation, recovery, and cleanup requirements. ## Verification - `pnpm -r typecheck` passed. - Plugin SDK build passed. - All 55 Dot Runner and native Codex server integration tests passed with Cargo available on PATH. - All 18 environment task runtime tests passed. These include preparation dispatch, inactive leases, wrong run identity, unauthorized projects, result identity, and unknown command rejection. - The broader root suite on the parent refactor finished with 22,420 passing tests, seven failed tests in unchanged Git and diagnostic code, and two suite setup errors because Cargo was absent from its PATH. This PR does not claim a passing full root suite. - No live provider calls or deployment were run. - Standard Greptile and Apex each scored commit `d3ac15906240b7066f5ff505ed2ffd9ef5dd43c1` 5/5. There are no unresolved review threads. - All 55 CI checks passed. Two optional checks were skipped. The branch is ready and mergeable against master. - A fresh master fetch and merge-base check passed immediately before the Apex request. ## Risks - This defines a new optional worker operation. Existing providers keep their current operations. - A provider must validate policy and resource bindings. Schema validation alone does not validate a concrete provider descriptor. - This PR defines the exchange. It does not connect native execution to a preparation provider. ## Model Used - OpenAI Codex, GPT-6. Reasoning and repository tools. The exact serving model identifier and context size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |