mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-01 15:35:12 +01:00
6dc58115cdebb988a63bb45dce4521f5bb5013f6
83
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
6dc58115cd |
fix(server): schedule stopped instruction collection
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
854244092a |
fix(server): schedule owed agent directory cleanup
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
03e1b3cd72 |
fix(server): recover captured instruction saves from committed deadlines
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ecef7776a4 |
test(server): cover retry cancellation startup registration
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
2ca12f8063 |
fix(server): schedule tool health from connection changes and deadlines
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
17f3eaf806 |
Merge repaired plugin scheduling into tool health migration
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0d3100665d |
fix(server): back off failed approval queue projections
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
cde69869bf |
test(server): reproduce idle tool health maintenance scans
Record the producer audit and failing baseline before migrating the worker. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e04a0375dc |
fix(server): wake approval queue seeding from committed changes
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
9c5c4fbf65 |
fix(server): schedule join request retention from durable deadlines
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ea3dd40cdd |
fix(server): schedule approval retention from durable source deadlines
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
5062efb368 |
fix(server): invalidate decision targets from committed changes
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a5b5994c1d |
fix(server): recover decision effects from saved claim deadlines
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
18c11e3110 |
fix(server): schedule decision TTL expiry from saved deadlines
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
66bbef521e |
fix(server): wake decision continuations from committed changes
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
692e3ee02c |
fix(server): wake archive notification delivery from committed changes
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0956883a84 |
fix(server): wake tool-review card repair from committed changes
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
fd2d848a7b |
fix(server): schedule interrupted tool claims by their deadlines
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
5dcd193d35 |
fix(server): recover approved tool actions from committed changes
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
5b9ec3b11f |
refactor(server): isolate pending tool review expiry
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
75332b9641 |
fix(server): schedule gateway credential expiry from committed changes
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d41590ac0e |
test: verify secret expiry settlement and concurrent producer writes
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
36844a458b |
test: cover concurrent setup expiry changes and startup ownership
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
898144f4b3 |
fix: stop idle execution-control recovery polling (#17094)
Replace idle execution-control polling with workers driven by committed writes and persisted deadlines. Preserve startup recovery, bounded retries, and shutdown safety. Stabilize the Slack setup browser regression by waiting for the restored setup screen before simulating its webhook. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
30b753f614 |
perf(budgets): scope accounting and deliver enforcement asynchronously (#17046)
## Thinking Path > - Paperclip manages AI agents and their paid work. > - Budgets must prevent new work and stop reported overruns. > - Cost settlement also ran serial lifecycle reconciliation for every eligible company agent. > - That made request latency grow with unrelated idle agents and delayed results on stop delivery. > - This change separates atomic admission and accounting from durable asynchronous enforcement. > - Scoped live usage detects overruns without writing provisional ledger charges. > - The result is constant query scope for ordinary settlement and recoverable stop delivery. ## Linked Issues or Issue Description **What happened?** Settlement and budget checks entered company-wide lifecycle reconciliation. A company with 1,000 unrelated idle agents performed 1,001 accounting transactions for one ordinary cost evaluation. User requests could also wait for provider termination. Running usage checkpoints did not enforce reported monetary exposure. **Expected behavior** Keep the final reservation gate authoritative. Commit charges, scope blocks, and enforcement intent before returning. Deliver stop requests after commit. Read only applicable policies and execution scopes for ordinary charges. Stop long-running work when supported reported or priced usage establishes an overrun. **Steps to reproduce** 1. Create a disposable company with one company budget and idle agents. 2. Evaluate a charge below the budget with 0, 17, and 1,000 unrelated idle agents. 3. Compare query counts, company accounting transactions, and duration. 4. Use the controlled partial-usage and delayed-stop fixtures to check enforcement and request responsiveness. **Paperclip version or commit** The measured baseline is `6c9b87d0922f1ebfc43e8b0c36889ac5ba7b9837` from `master`. Related public work: Refs #16976, #2006, #3539, #13763, #14296, #16853, #13075. These cover the affected service charge, cancellation, and accounting paths. This change removes their shared reconciliation overhead. ## What Changed - Batch applicable policy exposure under the existing company accounting lock. Preserve exact fractional money and zero-reservation dispatch fences. - Commit a durable final budget dispatch claim before provider I/O. Retain a separate final issue/run ownership transaction through synchronous handoff, without holding the company accounting lock or awaiting provider work. Preserve native recovery lease checks and duplicate handoff fencing. Browser handoff rechecks current policy; it reuses a consumed rate slot only for the identical policy generation. - Separate ledger spend, live observed spend, and effective held capacity. Use the maximum of reservation and live observation for each unfinished operation. - Deliver budget cancellation through the existing coordinator. Revalidate generations and persist queued/running stop fences before external termination. Bound stop concurrency to four. - Apply agent holds only to the affected agent. Project budgets leave other projects running. Project real company transitions in bounded lifecycle batches. - Replace periodic broad reconciliation with durable pending work, startup repair, receipt replay, and concrete retry/window deadlines. Retain drain and idle-safety checks. - Enforce accepted cumulative usage checkpoints without provisional charges. Final accounting atomically replaces live exposure and settles reservations. - Add operator-gated budget spans, regression coverage, map updates, and a real-PostgreSQL comparison harness. No schema or first-party telemetry change. ## Verification - Full workspace `pnpm -r typecheck` passes on the final ownership fix, rebased onto `bbcf5dfe98`. Production `pnpm build` passes on final head `4c5f4de8b1`. Map, module-boundary, token, and Storybook catalog checks pass. - Focused real-PostgreSQL coverage includes the 292-case rebased accounting/dispatch run, 63 browser cases, 52 run-control/cancellation cases, 42 policy-generation cases, 77 lifecycle/commentary cases, and the exact Muse confirmation regression. Native executor unit coverage passes all 768 cases. Final ownership coverage passes: 74 adapter/use-case cases, nine public regression cases, and the additional deleted-project race. These exercise competing ownership, budget changes and charges between phases, zero/nonzero duplicate claims, rollback, recovery contention, read-only capacity rechecks, and controlled HTTP startup after commit. Another 74 pure/startup/tracing cases pass. The complete CI matrix and both 5/5 reviews passed on `1ac9b58`; its external security scan identified a stale-owner handoff race. The final fix restores synchronous issue/run ownership fencing after the accounting transaction commits. Final head `8e84f55ce8` has fresh Greptile 5/5 with zero new comments, Paperclip review 5/5, and a passing security scan. The complete latest-head [CI matrix](https://github.com/paperclipai/paperclip/actions/runs/39535189808) passes, including budget, browser, Runner, build, and canary jobs. The unchanged Runner close fixture passed on its sole same-head job retry. - Broad local `pnpm test:run` invocations were stopped before required source updates (rebase and review fixes). They are not claimed as full passes. Final full-suite coverage comes from the complete latest-head CI matrix; local focused coverage and the incomplete broad invocations are reported separately. - Reproduce performance with `pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-budget-enforcement.ts --baseline-ref=6c9b87d0922f1ebfc43e8b0c36889ac5ba7b9837`; repeat with `--reverse`. The harness uses disposable real PostgreSQL, 20 warm samples per variant/count, and records raw timings, SQL/lock counts, agent reads, unchanged agent state, and query plans. Final production-source measurement revision: `eff713a4f72c7f4b7a4002a701cb481d4bfccd95`; the extended harness is committed in `4c5f4de8b1`. | Unrelated idle agents | Baseline median / p95 | New median / p95 | Baseline SQL / locks | New SQL / locks | New agent reads | | --- | ---: | ---: | ---: | ---: | ---: | | 0 | 1.80 / 2.33 ms | 2.99 / 3.88 ms | 5 / 1 | 7 / 1 | 0 | | 17 | 15.53 / 19.55 ms | 2.24 / 2.60 ms | 90 / 18 | 7 / 1 | 0 | | 1,000 | 978.35 / 1,790.43 ms | 2.16 / 2.52 ms | 5,005 / 1,001 | 7 / 1 | 0 | - These measurements use local PostgreSQL 18.1 and Node 26.4.0 arm64. They isolate the cost-evaluation core and complete admission path, without hosted-region or provider latency. Prior-revision reversed-order controls also preserved the constant query counts; the final measurements above use the final production source. - Complete reserve → final ownership gate → immediate offline provider result is measured separately, with one responsible agent and one active operation at each unrelated-agent count. Admission p95 baseline/new: 0 agents **11.54 / 29.77 ms**; 17 agents **9.31 / 27.62 ms**; 1,000 agents **24.51 / 74.05 ms**. Baseline admission was already scoped; the stronger final fence adds bounded overhead. New counts stay constant at 53 Drizzle statements, three transactions, two company-accounting locks, and five agent-reading statements bound to the responsible agent. Zero unrelated-agent reads, exactly one provider callback per sample, and no leftover runs or holds. All admission scenarios stay below 250 ms. No external provider calls are included. - The partial-usage/delayed-termination fixture passed 20/20: receipt persistence p95 13 ms, receipt return to observed durable Stop fence p95 28 ms, complete receipt-start-to-fence p95 50 ms, unrelated database request p95 1 ms while termination was blocked. After releasing the controlled termination fixture, acknowledgement p95 was 14 ms. These are separately measured phases; fixture acknowledgement is not live-provider termination latency. - Models and provider deadlines are unchanged. No deployment, live inference, or staging acceptance is claimed. A quiet 30-minute follow-up waits for the merged revision on staging, then repeats classifier and controlled budget-stop acceptance in staging, recording hosted bookkeeping, queue delay, and actual provider acknowledgement separately. - Development failures retained: finalizer notification regressions (fixed; 27/27 affected cases passed), invalid browser admission fixture assumptions (fixed; browser coverage now passes 63/63), and first-head lifecycle/dispatch CI regressions (reproduced and repaired). A prior-head Cases E2E test hit its unchanged announcement-dismiss handler race while opening a filter; the same journey passed on `1ac9b58`. The external security review then found a stale-owner race between accounting commit and provider entry; final verification covers a competing run in that gap while retaining provider I/O outside the accounting lock. Early PostgreSQL startup attempts hit local semaphore pressure; serial reruns succeeded. The final-head Runner lane had one failure among 1,823 tests: the unchanged `timeout-attachment` fixture failed at its first process close after 6.28 seconds. The sole same-head job retry passed all 1,819 active tests (four skipped), including that fixture in 1.85 seconds. Three isolated local probes could not run correctly because the Rust fake-provider fixture binary was absent; cleanup masked the original startup failure. CI builds that fixture through its normal lane. These probes are inconclusive and are not counted as passes. ## Risks - This changes accounting, lifecycle, and cancellation ordering. Durable generations, source/sequence checks, ownership leases, and company locking remain required for correctness. - Provider stop acknowledgement can be slow or fail. Stop intent remains pending until existing termination proof succeeds; cancelled row status alone does not acknowledge delivery. - Mid-run limits depend on provider usage reports or supported price calculations. Providers that report only at completion cannot provide exact mid-call detection. - Signals are process-local wake hints for the supported single-active-writer model. Durable state and startup recovery remain authoritative. - There are no migrations, new queue tables, or cached balances. Repeat the classifier and controlled budget-stop acceptance after normal rollout. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, tool use, code execution, and parallel coding agents. The host does not expose the exact backend model revision or context-window size, so those values are not asserted. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
bbcf5dfe98 |
fix(heartbeat): renew legacy controller leases on a private pool (#17032)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A legacy (non-native) agent run keeps a controller lease on its `heartbeat_runs` row. The server renews the lease every 10 seconds. If no renewal succeeds in 60 seconds, the watchdog stops the run with `renewal_deadline`. > - #16359 added timings to lease-loss reports. The first staging event with these timings shows `deadlineLateMs=0`, `renewalPendingMs=50000`, and 108 renewals in about 22.5 minutes, where about 135 are expected. Thus the event loop was not blocked. One renewal UPDATE was in flight for 50 seconds, and renewals were slow for a long time before the loss. > - The agent had exited with code 0. The run was in post-execution `sync_back`. Other runs on the same host completed. No code path held a lock on the run row across slow work. A later `instruction_save` UPDATE on the same row completed while the earlier renewal was still pending, so a row-lock wait does not explain the stall. > - postgres.js 3.4.9 sends a new query to a busy connection when no connection is idle (`handler` → `go(busy.shift(), query)` in `src/index.js`). The query waits behind the statement that is in flight on that connection. The renewal had no timeout, and the `pending` flag stopped all later renewals. Thus one slow unrelated statement on the shared pool can stop a healthy run. > - This pull request moves lease renewal to a small private pool, bounds each attempt with a statement timeout, and retries a timed-out attempt until the existing lease deadline. > - The benefit is that slow application queries on the shared pool can no longer cancel a healthy legacy run. ## Linked Issues or Issue Description Refs #16359 (lease-loss timings that identified this cause) Refs #16358 (pre-dispatch lease loss becomes retryable; different path) Refs #16917 (open: keeps a clean adapter result after a lease lapsed; it does not prevent the loss) Refs #13389 (open: host sleep / late timers; different cause) **What happened?** A staging legacy `claude_local` run was cancelled with `Legacy controller lease lost (renewal_deadline)` after the agent exited with code 0. The timings show that the deadline timer ran on time, and one renewal UPDATE was pending for 50 seconds. **Expected behavior** A live controller keeps its lease. Slow unrelated database work does not cancel the run. **Steps to reproduce** 1. Create a database client with one pooled connection. 2. Start `select pg_sleep(5)` on that client. 3. Call `watchLegacyControllerLease(...).assertOwned()` with the same client. 4. Before this change, the renewal waits about 5 seconds behind `pg_sleep`. With a long enough slow statement, the lease deadline stops the run. ## What Changed - `packages/db`: Add `livenessDb(db)`. It gives each `createDb` client a private pool of at most 2 connections (`application_name=paperclip-controller-lease`). The pool is created on first use. Test doubles and transactions get the original client back. - `packages/db`: Add `closeLivenessDb(db, timeoutSeconds)`. The server calls it in `closeDatabaseClients`, so shutdown and failed startup end the private pool too. - `legacy-controller-lease.ts`: The watchdog renews in a transaction on the private pool. Each attempt sets a transaction-local `statement_timeout` of 10 seconds. A timed-out attempt (`57014`) is retried. After a successful retry, the local deadline is measured from that attempt's start, because that attempt's UPDATE wrote the fresh 60 second lease. The existing 60 second deadline, the abort race, and the `renewal_rejected` / `renewal_error` attribution do not change. `renewLegacyControllerLease` itself does not change, so the runtime-selection caller inside its own transaction is not affected. - Tests: a saturated shared pool no longer delays renewal; a renewal that times out on a locked run row retries and keeps the lease; the private pool is cached per client, has its own `application_name`, and closes; after a 45 second timed-out attempt and a successful retry, the next deadline is a full 60 seconds after the retry. The existing test doubles now model the transaction. - `docs/deploy/database.md`: Count the up to 2 extra connections per server process when you size `max_connections`. ## Verification - `cd server && npx vitest run src/services/legacy-controller-lease.test.ts`: 20 passed, on embedded Postgres. - Red/green: with the old `legacy-controller-lease.ts` and the new tests, the saturated-pool test fails with `expected 5018 to be less than 2000`. With the fix, it passes. - Red/green for the retry deadline: with the deadline measured from the first attempt, the retry-deadline test fails (`expected true to be false`) because the run is aborted 15 seconds after the retry. With the fix, it passes. - Mutation check: when the timeout retry is removed, the locked-row test fails with `canceling statement due to statement timeout`. - `npx vitest run src/__tests__/heartbeat-process-recovery.test.ts -t "lease|dispatch|controller|interaction"`: 43 passed. - `npx vitest run src/services/heartbeat/run-completion.test.ts src/__tests__/heartbeat-orphaned-active-lease-sweep.test.ts`: 53 passed. - `npx vitest run src/__tests__/server-startup-feedback-export.test.ts`: passed. - `cd packages/db && npx vitest run src/work-signals.test.ts src/client-options.test.ts src/client-teardown-registry.test.ts`: 24 passed. `npx tsc --noEmit` for `packages/db` passed. - Not run locally: the full server `tsc` and the full server suite. The sandbox has a 4 GB memory limit, and `tsc` was killed (exit 137). CI is the gate for the server typecheck, the full tests, and the build. ## Risks - Each server process can open up to 2 more Postgres connections. Deployments that are at their `max_connections` limit must allow for them. The docs now say this. - A renewal attempt is now 4 statements (BEGIN, `set_config`, UPDATE, COMMIT) instead of 1. Each statement is a short primary-key operation on a dedicated connection. - If many runs wait on locked run rows at the same time, renewals for other runs can queue on the 2 private connections. The 10 second statement timeout limits each wait, which is less than the 60 second lease. - Behavior for a real lease loss does not change. A rejected renewal and a non-timeout error still stop the run at once, and the 60 second deadline still applies. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code in agent mode with tool use (shell, file edit, and test execution). A read-only subagent helped with code search. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
15176963b1 |
fix: stop idle skill-source polling (#17008)
## Thinking Path > - Paperclip manages agent work and the skills that agents use. > - Project repositories create durable import jobs, and installed sources can refresh automatically. > - The server queried these records every five seconds, even when no work existed. > - Idle tenant databases need to stop receiving those discovery queries. > - This change uses the existing transaction notifications and delivery coordinator. > - Saved refresh, retry, and lease deadlines determine when the worker runs next. > - Cloud does not yet wake these deadlines, so outstanding skill work retains the instance. ## Linked Issues or Issue Description **What existing behavior does this improve?** Skill-source background synchronization. This continues the idle-work migrations in #15625 and #17000. A search found no overlapping open polling-removal PR. **Current behavior** On master `b11010b0d4`, startup installs a five-second skill-source interval and also calls it from the external-object scheduler. Each sweep queries active companies, import capacity, pending imports, and automatic refreshes. **Proposed behavior** Run at startup, after a relevant transaction settles, or at a saved deadline. Stay SQL-quiet when no eligible work remains. Preserve import admission, authorization, atomic claims, retry backoff, and the existing five-minute upstream check. **Reason and benefit** Remove recurring database activity from inactive tenant instances without requiring new tables or a new scheduler. ## What Changed - Replace both recurring skill-sync calls with one delivery-coordinator registration. - Notify from attachment/detachment, source creation/settings/manual refresh/disconnection, project/company deletion, and company status changes, including budget resume. - Restore the earliest eligible `nextSyncAt` or lease-expiry deadline. Due backlogs and conflicts retry after five seconds; no idle recovery cadence remains. - Admit deferred imports again after a batch frees capacity. Preserve requests through drain, standby, restart, and commits during a sweep. - Add sleep-safety predicates for imports and eligible automatic sources until Cloud supports these deadlines. - Document the lifecycle and process-local writer contract. No schema or API changes. - Guard external test database URLs before migrations or cleanup: require a `_test` database name and reject database/options overrides. ## Verification - 147 targeted tests pass: 39 server-startup tests, 14 new real-PostgreSQL lifecycle tests, 20 shared coordinator/scheduler tests, 63 existing import/refresh/retry regression tests, and 11 test-database guard cases. - CI exposed a stale startup mock without the new `start` method. The fixture is updated and all 39 startup tests pass locally. - Tests exercise real enqueue/settings/company-reactivation producers, nested commit, rollback, saved retry/lease restart, manual refresh failure, capacity admission, concurrent commit, shutdown cancellation, and SQL quiet after startup or pause. - Sleep-safety tests reject sleep for future imports and automatic refreshes, and accept inactive retained state. - `pnpm -r typecheck` and `pnpm build` pass. - `pnpm test:run` was attempted and stopped after repeated embedded-PostgreSQL startup failures in unrelated suites. The two existing import/refresh suites initially skipped for the same host limitation; after adding explicit disposable-database options, all 63 regression tests passed against PostgreSQL 17. Full CI passed on `ddb3ee51369baa8c7153bae9fd88efb8b903f2bc`: 55 successful checks, one neutral review check, and two intentional skips. ## Risks - Notifications are process-local. Independent SQL writers and separate live replicas do not wake this worker; startup reconstructs persisted work. - Enabled automatic sources still contact GitHub every five minutes. Their upstream changes have no event ingress here. - Cloud scheduled wakes currently cover routines and issue monitors, not skill sync. This change retains instances with outstanding skill obligations. It does not claim measured host/database suspension or fleet savings. - No change to source authorization, installed content on failure, or existing claim semantics. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, and code execution. The host does not expose an exact model revision or context-window size to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
505e1a5582 |
feat(cases): add persistent workflows with same-task agent handoffs (#16959)
## Thinking Path > - Paperclip manages AI agents and their work. > - An agent workflow needs persistent facts and a safe way to transfer work. > - Creating a task for each phase splits the objective, discussion, workspace, and review policy. > - A case can retain the facts while one primary task moves between agents. > - Transfers must survive process failure and preserve private access, budgets, and completion gates. > - This pull request adds durable case handoffs, native case tools, and read-only case pages. > - Prompts describe the workflow. Existing task controls decide when work can run and finish. ## Linked Issues or Issue Description **Problem or motivation** Prompt-driven agent teams need a persistent record that can move through triage, implementation, verification, and correction. Ordinary reassignment does not provide a durable instruction, safe release barrier, or recoverable delivery receipt for an agent handing off its own running task. **Proposed solution** Extend standalone cases with one primary task, a descriptive phase, retained access sources, and a handoff ledger. Apply assignments after source execution and workspace settlement. Deliver each transfer through the existing durable wake queue. Provide `create_case`, `read_case`, `update_case`, and `handoff_case` in the native runner. Creation requires kickoff and explicit workflow text. An explicit handoff to `done` closes the case when normal task completion is accepted. **Alternatives considered** Do not add a workflow engine, filesystem queue, configured state graph, or one task per phase. The workflow document combines transition prose with an optional Mermaid `stateDiagram-v2` map. The server does not execute the diagram. The Pipelines experiment removed on master stays removed. **Roadmap alignment** This extends Agent Reviews and Approvals, Enforced Outcomes, Bounded Run Recovery, and Activity Log attribution. Existing approval, review, budget, privacy, workspace, and responsible-user rules remain authoritative. Related work: [ordinary handoff wake recovery](https://github.com/paperclipai/paperclip/pull/15019). This feature adds a separate persisted case-handoff receipt and release barrier. ## What Changed - Add case routing fields, retained access sources, durable handoff history, delivery leases, and replay-safe migrations. - Keep the same primary task across agent phases. Preserve each returning agent's own session and the task's workspace. - Fence outgoing mutations, wait for provider and workspace release, and recover delivery without duplicate task comments or runs. Recheck current assignment permissions as the original caller before applying a delayed transfer. Scope a run Stop to that run’s own handoff intents, so repeated old Stop requests preserve newer work. Cancelling terminal intent without a source run also requires fenced current task ownership, including native retries and preparation with an old checkout. For Cloud callers, revalidate current stack membership with an uncached Cloud RPC and derive owner elevation from the current managed setting. - Keep case reads, documents, assets, Audit, and live events inside retained task/project permissions. - Add native case creation/read/update/handoff tools. Store required workflow text and initial documents on the first task. Page oversized case facts and referenced document bodies through the dedicated tool with bounded UTF-8 JSON, current routing versions, snapshot conflicts, and current source authorization on every page. - Add read-only case lists and detail pages, task-style list controls, compact dotted field tables, linked handoff tasks, and case filters in Audit. - Add native/legacy integration tests, migration/restart tests, browser coverage, and bounded live receipt fixtures. Extend the trusted sandbox condition to cover the new subscription fixture profile. - Hide obsolete paused-agent warnings on completed or cancelled task threads. - Synchronize the company-switch settings test with React effects after a CI failure. Preserve its security assertions and production remount behavior. ## Verification The branch directly includes master `9ef434fe79`. The required `ci / verify` and `ci / e2e` gates pass on head `765fccd67c`. [Current-head CI](https://github.com/paperclipai/paperclip/actions/runs/39464348614) passes typecheck, build, tests, and all eight browser shards. Security checks pass. [Greptile gives this exact head 5/5](https://github.com/paperclipai/paperclip/pull/16959#issuecomment-6291156549), with no unresolved review threads. The optional Paperclip Review check remains queued because its assigned agent is paused. It is not a required branch check. No check result was overridden. Local workspace typecheck and build pass after the master merge, along with database migration checks, token gates, map checks, Storybook catalog checks, and the Copilot profile identity check. Standard builds refresh Runner vendor output. Focused checks pass 2,008 API/paging/context tests and 7 strict case protocol tests. The final commit changes one test fixture only; its ten tests, UI typecheck, and token gates pass. No production source changed in that commit. On current head `765fccd67c`, all **194 critical workflow tests** executed and passed with no skipped tests in those files: - 38 Stop/ownership tests, including board completion intent, native retries, stale Stop, and preparation with an old checkout. - 2 actual legacy process/HTTP handoff tests, including scratch failure, a closed-stdio child that continues writing, and recovery through a fresh worker after the process group stops. - 5 migration tests, including SQL column metadata, preserved null authority context, and all six prior-numbering receipts. - 3 process-settlement and 5 scratch tests. - 88 handoff-service, 18 heartbeat handoff, and 21 native case-tool integration tests. - 7 bounded case/document-read projection tests and 7 strict protocol-contract tests. The current CI checkout was `fed65d49`, merging `765fccd67c` into master `9ef434fe79`. Its new document regressions reconstruct exact Unicode/escaped bodies and revisions through real native authority and authenticated HTTP with generic API tools disabled. They cover case-owned, primary-task, and linked-task documents; routing on every page; stale document snapshots; access loss and unlinking; unrelated/foreign/private sources; system documents; and quarantined content. Active-task document tools retain their existing behavior. These are current-head test results from the first CI attempt; the browser-only retry did not rerun these suites. The full `64fdd3403f` CI failed one separate settings test: after a company switch, it observed an unsettled API-key draft. The production component and test were unchanged from prior green source. Production switches remount both keyed settings layers; the test directly reused an unkeyed view and used a timer tick instead of waiting for React effects. The final commit replaces that fixture synchronization with React `act`. The exact secret-clearing assertions and all timeouts remain unchanged. The original assertion failure remains recorded; no unchanged retry was used. The narrow unchanged baseline and fixed fixture each pass 10 tests. The upstream result-paging merge preserves case capture namespaces and current asset privacy. Its API integration suite executed 39 of 41 tests successfully, with two skipped; the optional assigned-result PRP integration test was skipped. Neither skipped suite is counted in the 194 critical tests or as live provider proof. All 361 upstream migration entries and SQL bodies remain unchanged. Six case migrations follow master as 0363–0368 with identical SQL hashes. Official Drizzle generators produced the combined metadata and five retained snapshots. Generation reports no schema changes, and the snapshot drift test passes. The earlier `ce361ae5d0` CI passed on attempt 1, including its then-current 179 critical tests. Fresh review later identified unusable document references with generic API tools disabled; the bounded document reads and ten additional authenticated HTTP tests address that finding. The full local `pnpm test:run` attempt was cancelled after this Mac exhausted PostgreSQL semaphore capacity; it is **not** counted as a pass. Later database attempts also stopped before assertions or skipped, including one sandbox socket denial. Two original stale-Stop regressions did reproduce the bug against the old source. A REST serialization assertion passed after two preserved database-startup failures. No test timeout was increased. All 194 assertions listed above executed and passed in current-head CI. The complete 385-file, 3,963-test UI shard also passed after the fixture repair. A configured-provider receipt workflow passed 13 backend checks across triage → implementation → verification → correction → done, using one task and four work runs. The saved receipt is revision 2 with total 6 USD; case and task both closed. The original canonical attempt remains failed because its browser assertion confused the terminal destination with completed-task tooltips. The corrected full browser verifier passed on the same retained database and original runs, with no additional provider calls. The last clean-build browser readback is on `6a86c54a53`; later changes affect recovery, authorization, bounded reads, and settlement. Accounted work cost is a $1.010574 rate-card estimate; provider billing and setup-probe usage are unavailable. Copilot profile 49 passed the unchanged seven-fixture private protocol pack (46/46 checks, no retries) and the product file-edit canary after master's completion-replay change. All 60 bound implementation hashes remain unchanged on the current source. Historical profile-47/48 identities and results remain intact. See the [profile-49 compatibility record](https://github.com/paperclipai/paperclip/blob/codex/persistent-case-handoffs/doc/plans/2026-10-20-copilot-completion-recovery-compatibility-v49.json). This is bounded compatibility evidence, not a fresh full matrix, installed-package product test, or Daytona rollout. Preserved CI failures and retries: - On `20662930ac`, Slack managed setup failed a five-second success assertion after reload. Its test/application path was unchanged from a prior green head. One unchanged-source retry passed all 25 tests in that shard; no source or timeout changed. - On `9402b4f17a`, two CI workers received shutdown signals at 15:24:19 UTC. Neither log had an assertion failure before interruption. One unchanged-source retry of each passed all Runner static checks and all 32 tests in browser shard 6; both aggregates passed. The next head, `7b4374edaf`, passed all CI without retries. - On `295f289b5a`, serialized shard 9 timed out during its first device-login test, then the next test found an active login lease. Direct login/test paths were unchanged from a prior green head. Cold import plus a timed-out async body using the next test's mutable actor/store is the supported fixture-race explanation; no request trace proves that sequence. The original failure remains recorded. One unchanged-source retry passed all 294 tests in 19 files, including all 41 device-login tests. No login source or timeout changed. The subsequent `ce361ae5d0` CI passed without retries. - On `64fdd3403f`, one company-switch settings assertion observed an unsettled draft. The final test-only commit uses React `act` without changing assertions, timeouts, or production code. On `765fccd67c`, all ten settings tests and the complete 385-file, 3,963-test UI shard passed. - On `765fccd67c`, browser shard 3 completed the split-handoff assertions, then cleanup received HTTP 500 from run cancellation after a one-second PostgreSQL row-lock timeout. The extra chat run was reporting completed work while teardown began. Cleanup stopped before restoring a global experiment flag, causing a later assertion to fail. Relevant production and test paths are unchanged from an earlier green commit. Stop's lock order is consistent, and no nested self-lock was found; the exact PostgreSQL blocker was not recorded. One unchanged-shard retry passed. The original failure, trace, and downstream flag failure remain recorded. No Stop implementation, cleanup assertion, or timeout changed. The retry supports intermittent cleanup contention; it does not identify the original blocking transaction. ## Risks - Local process transfers require host-owned PID/group absence. Missing or uncertain termination evidence keeps delivery pending, independently of scratch setup. - Handoff admission and task cancellation share execution ownership. New regression tests cover temporary holds, late source writes, and uncertain provider termination. - Retained private sources use an intersection of existing permissions. This intentionally fails closed when a source is missing or an existing task cannot preserve that boundary. - Migrations add tables and indexes. They preserve master history and can replay over prior feature installs. Migration 0368 adds nullable internal authorization provenance with no default or authority backfill. Historical transfers still awaiting release without this context fail closed; an operator must cancel them and request a new handoff. - Delayed Cloud handoffs depend on the current Cloud membership RPC. An unavailable, ambiguous, expired, or mismatched response holds the handoff for retry; it does not reuse cached authority or grant access. - Bounded reads reload and reauthorize the selected projection per page. Underlying work grows with the data size and number of pages. Changed data or routing requires restarting from offset zero without the old snapshot. - Phase names and workflow diagrams remain advisory. They do not satisfy review or approval gates. - This is a large, cross-cutting PR. The maintainer requested one PR for the whole feature; the commits separate data, runtime, UI, and qualification work. ## Model Used OpenAI Codex, GPT-6 family, with tool use, code execution, and delegated code review. The exact deployment model ID and context-window size were not exposed in the session. Live fixture agents used the explicitly configured provider model recorded in their test evidence. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
cda77edc60 |
perf(server): stop readiness waiting on startup housekeeping (#16329)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - On restart, `/api/health` reports `starting` until startup recovery finishes, and deploy tooling waits for `ok` > - After heartbeat recovery, readiness also waits for housekeeping that every scheduler tick repeats and that serving requests does not need > - #16330 added OpenTelemetry spans for the main startup phases; native recovery and orphan reaping are the slow ones, and both run the native finalization reconciliation pass, which has no inner breakdown > - This pull request stops readiness waiting on that housekeeping, and records the phases of slow reconciliation passes as spans > - The benefit is faster readiness on every restart, and traces that show where a slow reconciliation pass spends its time ## Linked Issues or Issue Description Builds on #16330, which records `controller.startup.<phase>` spans. Related to #16328. **What existing behavior does this improve?** Server readiness time after a restart, and diagnosis of slow native recovery. **Current behavior** - After heartbeat recovery, readiness waits for the custom-image setup cleanup, tool connection health (remote MCP checks), decision expiry, the adapter and setup-token login reapers, environment lease cleanup (provider calls) and the retention sweep. Each also runs on every scheduler tick. On one instance they added 11–28 s to every measured restart. A throw in any of them fails startup. - On the same instance, native recovery took from 33 s to 11.5 min and orphan reaping from 30 s to 4 min across restarts in one hour. Both run the native finalization reconciliation pass. The #16330 spans time each step as a whole, but not the phases inside that pass. **Proposed behavior** - The housekeeping steps run after readiness. The scheduler interval still starts after them. - With an OTLP endpoint, a reconciliation pass of at least one second records a span with one child span per phase. **Reason and benefit** Readiness depends only on what serving requests and run recovery need. Slow reconciliation passes become attributable to a phase in the existing traces, without adding trace volume for the periodic pass. **Breaking changes** None. A failed housekeeping step no longer fails startup; it is logged and the next step runs. ## What Changed - `server/src/index.ts`: moves the housekeeping steps into `runStartupHousekeeping`, which runs after readiness under `trackHeartbeatSchedulerWork` (idle sleep and shutdown wait for it). Each step catches and logs its own failure (`startup housekeeping step failed`). The heartbeat scheduler interval starts when housekeeping finishes, unless the scheduler was stopped, so the reapers still finish before the first timer tick. - New `server/src/services/native-runtime/native-reconciliation-timing.ts`: records the phase boundaries of one pass. When the pass took at least one second and an OTLP endpoint is configured, it emits a `controller.native_reconciliation` root span (`paperclip.native-reconciliation`) with child spans `legacy_workspace_exports`, `obsolete_policy_reviews`, `automatic_completion_reviews`, `candidate_scan` and `candidates`, using the recorded start and end times. It records `scoped` and the candidate count only. A failed pass marks the pass span and the running phase. Tracing failures are swallowed. - `native-finalization-reconciler.ts`: the exported function wraps the pass in that timing and finishes it in `finally`, so failed passes are recorded; the original error is rethrown. - `doc/execution-semantics.md`: what readiness waits for, and the housekeeping order. `doc/observability.md`: the reconciliation spans. ## Verification - `pnpm exec vitest run src/__tests__/server-startup-feedback-export.test.ts src/services/native-runtime/native-reconciliation-timing.test.ts src/services/native-runtime/native-finalization-reconciler-timing.test.ts src/startup-recovery-timing.test.ts` (in `server/`): 52 passed. New tests: - Readiness is `ready` while a housekeeping step hangs, and the heartbeat interval starts only after the step settles. - A failed housekeeping step is logged, and the scheduler still starts after the remaining steps. - Span timing: nothing for a fast pass or without an endpoint; a slow pass emits the pass span and one child per phase with the recorded times; a failed pass marks the running phase and the pass; a failing tracer does not throw. - The reconciler records its phases and a failed outcome when a pass throws, and rethrows. - Three existing startup tests settle housekeeping before they read the scheduler interval or the setup cleanup. - `pnpm exec vitest run src/services/native-runtime`: 4,399 passed. - `pnpm typecheck` in `server/`: passed. ## Risks - Housekeeping now runs concurrently with the first requests and with `waitForExternalAdapters`. None of these steps is needed to serve requests, and each one also runs on every scheduler tick. - Readiness is reported earlier; the heartbeat timer starts at the same point relative to housekeeping as before. - The new spans exist only with an OTLP endpoint and only for passes of at least one second. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file edits, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
473c80d13f |
Automatically import and update project repository skills (#16066)
Automatically import skills from new GitHub project repositories and update enabled sources in the background. Preserve project-derived access, saved selections, immutable execution versions and explicit pins; add legacy adoption, refresh and connection repair in Sources. Include durable scheduling and bounded work credits, production Storybook journeys, real GitHub-to-routine acceptance receipts and final-head CI/security/Greptile qualification. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a7dd5284b5 |
feat: add company fast-response acknowledgements (#15748)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Human messages start agent work through task, chat, and external-channel intake. > - Agent startup can delay the first visible response. > - A small direct model call can acknowledge the request while normal work starts. > - This response must remain separate from evidence that the agent answered or completed work. > - This pull request adds optional company-wide fast responses with durable receipts, access checks, and cost accounting. > - Users receive prompt acknowledgement without changing the agent execution path. ## Linked Issues or Issue Description **Subsystem affected** Cross-cutting: database, shared contracts, server orchestration, connections, and UI. **Problem or motivation** Users can wait for an agent to start before they receive a contextual response. Generic progress messages do not confirm what the user asked. An acknowledgement must not falsely complete work or satisfy an unanswered turn. **Proposed solution** Add a disabled-by-default experimental fast-response model using an existing shared API connection. Generate one short acknowledgement for each accepted human turn. Use the existing agent identity and external publication outboxes. Retain platform provenance and account for the inference independently. **Alternatives considered** Running the acknowledgement through the agent runtime adds startup latency. Restricting implementation to OpenRouter leaves other existing API connections unsupported. This change supports all existing API connections and uses OpenRouter for live verification. **Roadmap alignment** This extends existing agent communication and connection settings. ROADMAP.md and open pull requests were checked. No duplicate fast-response implementation was found. The maintainer supplied the feature plan directly. ## What Changed - Add Experimental fast responses to instance settings. Keep the feature off by default. Enforce the flag at intake, dispatch, and publication. - Move Decision model and Fast response setup to company Settings → Connections when the experiment is enabled. Place Connections below Environments. Separate the model sections with clear headings and a divider. Keep Decision model in General when the experiment is off. Route Costs setup links to the available settings page. - Make the acknowledgement prompt a multiline template string. Ask for a brief, positive response in the user’s language. - Add settings, connection/model selection, sponsorship, fixed-sample testing, and Costs history. - Add a Vercel AI SDK text service for existing native, routed, Bedrock, and local API connections. - Add durable per-turn requests, a bounded worker, a three-second admission deadline, and one inference attempt. - Integrate task creation, task and agent chat, steering, queued follow-ups, experimental board chat, shared chat intake, and email. - Preserve receipt provenance and exclude receipts from completion, recovery, interaction resolution, and agent wakeups. - Recheck authorization, source validity, real replies, budgets, and destination binding before publication. Preserve accounting for suppressed or uncertain outcomes. - Add provider contracts, database race tests, channel outbox tests, UI tests, Stories, and documentation. Keep decision-model execution unchanged. ## Verification - Local `pnpm -r typecheck`, `pnpm build`, and UI token gates pass after rebase. The UI typecheck and build also pass after the final navigation fixes. All 37 focused navigation tests pass. The first full local `pnpm test:run` invocation reported seven chat failures. All seven passed in isolated reruns after fixture updates and host cleanup. This invocation was stopped after the full final-head CI suite passed. - Initial local database tests hit macOS shared-memory exhaustion. A diagnostic confirmed `shmget: No space left on device` with the system limit of 32 IDs. Three abandoned, unattached PostgreSQL IPC headers were reclaimed. The isolated rerun passed all 31 fast-response and channel tests. The two additional Slack regression cases also passed in a separate rerun. No active database was stopped. - UI token gates pass. Provider contract tests cover all supported API connection types. Fast-response tests cover uniqueness, authorization, budgets, expiry, cancellation, restart, real-reply races, and the experimental flag. - The test drive verified that disabling the experiment hides Connections and redirects its direct URL. Enabling it restores the saved OpenRouter setup. After restart on the final commit, the OpenRouter settings sample returned “Sure, I’ll take care of fixing the border around the settings panel.” in 428 ms. It used 260 input and 31 output tokens and recorded $0.0000576 in reported cost. - Earlier live OpenRouter verification produced a receipt in 1.013 seconds for a new task. The real agent then supplied its substantive answer. A separate settings test took 1.34 seconds, used 246 input and 23 output tokens, and recorded $0.0000361. - Live Slack verification on final commit `24016fa525961ebc7a4acffdc5019ce123970549` passed for a fresh initial task and a follow-up in a disposable bot DM. The initial acknowledgement appeared 2.405 seconds after the Slack message and 1.413 seconds after durable acceptance; the follow-up appeared in 2.322 seconds and 1.160 seconds respectively. The recorded generation/settlement durations were 898 ms and 697 ms. The initial call used 291 input / 39 output tokens and cost $0.00006705; the follow-up used 281 / 47 and cost $0.00007035. - The Slack acknowledgements retained the assigned agent identity and `origin: fast_response`, source-turn references, and no fabricated run ID. A status snapshot after the initial receipt showed the task still `in_progress` with its real execution run. The agent then read the CSV fixture and correctly reported two ready items, two blockers, and their owners. Both replies stayed in the originating DM, and no duplicate generic working message followed the delivered contextual acknowledgements. The `new` command started a fresh conversation session without generating a fast response. - An earlier eligible Slack turn generated a response but suppressed publication. Its reported cost was retained, generic progress remained, and the real agent completed the requested result. This is recorded separately from the two delivered acknowledgements; the history did not expose a precise suppression reason. - Slack verification used real signed provider events, the existing intake/publication paths, and a user-approved temporary webhook-only HTTPS tunnel. Subscriptions were limited to direct-message events. The test bot was paused and the tunnel stopped afterward. Live channel/group-thread and GitHub journeys remain unverified; their mocked tests cover intake, destination binding, publication, and continued underlying work. - Greptile reviewed final commit `24016fa525961ebc7a4acffdc5019ce123970549` at 5/5. Both new review findings are fixed and all threads are resolved. [Final-head CI run](https://github.com/paperclipai/paperclip/actions/runs/38060015730) passed. This includes grouped server/chat/workspace tests, serialized suites, runner checks, browser journeys, typecheck, build, release registry, policy, and the canary dry run. - Manual check: enable Experimental fast responses. Open company Settings → Connections. Select a shared API connection and model. Save and use Test response. Submit a task message and confirm that its receipt does not complete the task. ## Risks - The migration adds receipt and configuration tables plus accounting references. It uses idempotent statements and follows the current migration sequence. - A slow provider, authorization change, or real-agent reply can suppress an acknowledgement. Existing progress feedback remains available. - Unknown provider billing retains a reservation until reconciliation. External deliveries with uncertain outcomes must not be blindly repeated. - Live Slack initial-task and follow-up DM journeys pass. Live Slack channel/group-thread and GitHub journeys remain unverified. The PR remains a draft and records these gaps. Final-head repository CI and automated review pass. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository tools, code execution, and browser interaction. The runtime does not expose an exact model snapshot ID or context-window size; these are not asserted. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
835a022936 |
perf(server): wake delivery queues from transaction-aware work signals (#15625)
## Thinking Path > - Paperclip manages AI-agent work and its delivery to users and agents. > - Five delivery queues scan the database even when empty. > - Each queue already has durable rows; empty polling wastes idle hosting capacity. > - Producers can register intent inside their existing database transaction. > - Central transaction tracking can wake consumers after the outer commit without extra caller wrappers. > - A lost response needs reconciliation against PostgreSQL before an empty queue is safe to leave idle. > - This change uses one scheduler for outstanding work and lets settled, empty queues go quiet. ## Linked Issues or Issue Description Supersedes the closed #15614. Related idle-safety work: #15522 and #15599. **What existing behavior does this improve?** Delivery scheduling for feedback exports, chat completions, connection continuations, question answers, and tool-action receipts. **Current behavior** Feedback scans every five seconds. Four delivery queues scan on the heartbeat interval, normally 30 seconds. Startup and some event paths also scan them. **Proposed behavior** Each producer awaits one intent registration before its queue write. The existing `createDb` transaction boundary tracks nested savepoints and wakes the consumer after outer settlement. Startup scans restore durable work. Pending rows, failures, and unresolved transactions retain retry deadlines. Empty, settled queues have no timer or recurring scan. **Reason and benefit** Normal delivery starts after commit. Central tracking removes caller-specific post-commit plumbing. PostgreSQL transaction status resolves lost responses without new tables or a permanent uncertainty latch. **Breaking changes** No API or schema changes. The notification scope is one DB owner and its dedicated child pools. Direct SQL, separate roots, and other processes require an explicit wake/recovery integration. This path requires PostgreSQL 14+ transaction-information functions; embedded PostgreSQL uses 18. ## What Changed - Add one named-deadline scheduler and app-owned coordinator for five existing queues. - Instrument `createDb().transaction()` centrally, including nested savepoints. Dedicated child connections share their owner's signal scope. - Register work before the five queue insert paths. The first registration obtains the outer XID; unrelated transactions issue no extra SQL. Tool receipt insertion gains a small transaction around its existing insert. - Query `pg_xact_status` only for rejected, tracked transactions. Probe before scanning the queue. Retain the idle hold while PostgreSQL still reports an in-progress transaction or the probe fails. Release it after settlement and reconciliation. - Remove unconditional delivery recovery scans and caller-specific completion post-commit actions. Retain the existing task-scoped completion activity fast path and durable claims. - Coalesce wakes without postponing an earlier deadline. Retry outstanding rows and failures; suppress dispatch during idle drain while allowing transaction and queue reconciliation; suppress both during warm standby. Cancel deadlines and await active delivery sweeps on shutdown. - Serialize feedback flushes. Vote routes return after saving. Uploads have a 30-second deadline and shutdown cancellation; unfinished exports remain recoverable. - Document the writer contract and update the dated sleep inventory. ## Verification - Final head: `a4609956841107ad60c4cb50fe2acb05636b1363`. - Passed: all final-head hosted checks, with 54 successes and two intentional skips. The full test matrix, typecheck, build, canary, and aggregate verification passed in [run 37940948440](https://github.com/paperclipai/paperclip/actions/runs/37940948440). - Passed: Greptile 5/5 on the same head, with a successful check run and zero unresolved review threads. The branch is mergeable. - Passed locally after the final rebase: 50 coordinator, scheduler, and startup tests; server typecheck; server build. - Passed locally before the final import-only rebase: repo-wide typecheck and build; 237 PostgreSQL producer/delivery and database signal tests; feedback, native-question, and idle-safety regression tests. - Real PostgreSQL tests hold transactions open after an injected client response loss, then commit or abort. They verify that an empty scan cannot clear an in-progress transaction. Another test terminates a real backend and verifies recovery without replay. - Coordinator tests cover an hour with no empty-queue timers, pending/error retries, worker replacement, concurrent writes, earliest-deadline preservation, rollback during idle drain, committed work during drain, standby, and shutdown. - Local `pnpm test:run` was started and stopped after embedded PostgreSQL startup failures appeared. It did not finish and is not reported as a pass. Later local database reruns had skipped suites; those skips are not claimed as verification. The earlier PostgreSQL tests did execute and pass, and the final hosted full matrix passed. ## Risks - One XID query is added per transaction that registers delivery work. Nested transactions share that ID. Registration must be awaited before writing; new enqueue paths must follow this contract. - Work signals are local to a DB owner and its dedicated child pools. Independent roots, direct SQL, or other processes are not observed. Startup scans recover already committed rows, but do not fence late transactions from a previous process. Cross-process ownership and database failover require separate work. - A database outage or genuinely unresolved transaction keeps an idle hold until status can be reconciled. No timer clears uncertainty by assumption. - These changes remove five empty polling loops. They do not implement whole-instance sleep, database teardown, or an external waker. The earliest deadline covers only the registered queues. - Waiting tool reviews retain their existing retry cadence until their receipts can be delivered. - Feedback vote responses no longer wait for the remote upload. Sharing consent and the saved vote response are unchanged. - Real hosting-provider sleep and cost savings have not been measured. ## Model Used OpenAI Codex, GPT-6. Used reasoning, repository inspection, code editing, and command execution. The runtime did not expose the exact model revision or context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
9b624a110e |
fix: checkpoint database backups before idle sleep (#15629)
## Thinking Path > - Paperclip coordinates agent work and preserves company state. > - Hosted instances can stop compute when admission and all work sources are quiet. > - The periodic database backup timer currently blocks every automatic sleep. > - Disabling backups would remove recovery points. > - This change offers an explicit final-backup checkpoint before sleep. > - A verified archive and restart marker preserve recovery while compute is stopped. ## Linked Issues or Issue Description **What happened?** An otherwise idle instance with automatic backups enabled always reports background work. Operators cannot reclaim idle compute without disabling those backups. The work inventory also queries the database before checking known local blockers. **Expected behavior** An operator can enable checkpoint mode. The server may authorize sleep only after all other work is quiet and a fresh backup is verified under the same owned hold. Backups resume on restart with the existing retention policy. **Steps to reproduce** Acquire a bounded idle drain on an instance with no application work and automatic backups enabled. Read its owned idle safety report. It reports present even when all other work is clear. With this change and `PAPERCLIP_DB_BACKUP_IDLE_CHECKPOINT_ENABLED=1`, a successful final backup can permit sleep; backup failures still refuse it. **Paperclip version or commit** Reproduced from master at |
||
|
Devin FoleyandPaperclip
|
3367b75ccc |
fix: fence accepted work and cleanup before idle sleep (#15522)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A host can stop an idle instance to reduce unused compute. > - Zero active runs do not prove that requests, saved work, or cleanup are complete. > - A client can disconnect while its request still writes, and failed cleanup can remain only in memory or on disk. > - This pull request holds new admission and checks accepted work, cleanup, and persisted work under an owned drain. > - The host gets an empty report only while those checks remain valid. ## Linked Issues or Issue Description Refs #13413. That companion PR uses the same owned-hold vocabulary for runtime services. This PR covers HTTP requests, scheduler work, accounting, cleanup, and the instance work inventory. It does not include the preview gateway changes. **What existing behavior does this improve?** The instance task-drain API reports process counters. It does not prove that a host can safely stop the instance for idle sleep. **Current behavior** A quiet run set can coexist with an unfinished request, cleanup after a completed run, future work, or a failed accounting write. **Proposed behavior** Provide a bounded owned idle hold. Block new ingress, track accepted handler promises, inspect durable and local work, and return `none` only when the same hold stays quiet through the checks. Keep normal deployment drains compatible. **Reason and benefit** Hosts can identify eligible idle instances without treating a disconnect or failed cleanup write as completed work. ## What Changed - Add `purpose: "idle"`, a bounded TTL, unique owners, and owner-checked release to task drain. Existing holds cannot be replaced by another API request. - Gate HTTP ingress before parsers, auth, webhooks, and MCP. Gate new WebSocket upgrades. Track async handlers in nested Express routers and error middleware until they settle, even after the response or client disconnect. - Count accepted live-event WebSocket authentication through settlement, even after disconnects. Count detached built-in agent, managed-home and runtime-service startup reconciliation after readiness. - Keep health and control mutations tracked. Count control-request authentication separately from the read-only report, including concurrent user/company/membership writes. - Pause new scheduler admissions during idle holds. Count work already in flight, including database backups, and reject scans whose work generation changes. Periodic backups block sleep without a host wake schedule. - Inspect accounting and orphan-cleanup spool directories without skipping temporary or malformed entries. Retain orphan tokens through queue splices, flush failures, and buffer overflow. Keep failed usage capture counted until its database failure fence is written. - Check persisted work across companies in a bounded read-only transaction. Include deferred agent-file cleanup and saved watchdogs whose watched issues are complete. Enabled plugins and unsupported retained work remain blockers. - Require the exact idle owner and completed startup tracking before returning an empty report. Keep reports free of tenant details. - Document the hosting protocol, retry behavior, conservative blockers, and the remaining external provider-stop race. ## Verification - Current head: `8d9a599b732c2047b8c671d4799065d7c90a3567`, rebased on master `941a3fa991aeb97eb1ac390c65b7973b5f6de1ad`. Both heartbeat helper extractions are preserved. GitHub confirms no merge conflicts. - Focused heartbeat renderer/run-log, drain, control-auth, admission, route and PostgreSQL inventory checks: 234 tests passed in 10 suites after the rebase. - The POST task-drain contract includes the expected `409` conflict response. The 84 OpenAPI and instance-settings route tests and server typecheck passed after that final documentation fix. - Final accepted-upgrade/startup regression run: 104 tests passed in five suites, including success and failure after readiness or disconnect. Final server typecheck also passed. - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - `pnpm install --frozen-lockfile` and the PR diff secret scan passed. No dependency or lockfile changes are added by this PR. - The prior verification covered HTTP disconnects and early responses, async error handlers, concurrent authentication writes, signed bootstrap, saved watchdogs, backup promises, orphan cleanup, spools and failed accounting fences. Those tests passed. - The last full local `pnpm test:run` stopped in the general-server phase with 16,658 passing tests and five skill/connector fixture failures caused by an ancestor workspace skill directory. That full local run preceded this rebase and has not been repeated for the import conflict. Full current-head CI passed: 53 successful checks and two conditional skips, with no failures. - All eight review findings are fixed and their threads resolved, including accepted upgrade authentication, detached startup writes and the POST conflict contract. Current-head Apex review is 5/5 with no new findings; all eight review threads remain resolved. - No live provider stop or production change was performed. ## Risks - The hosting controller must use the owned protocol and hold external admission through its final validation and provider stop. A legacy drain cannot authorize idle sleep. During an idle hold, new requests receive 503 with `Retry-After: 1`; the host must handle queueing or retry before enabling this path. - This is a single-process protocol. An unexpected restart after the last validation can race an external provider stop. The host must serialize deploy/wake/sleep operations and bind the validation to the instance it stops. Multiple replicas need shared fencing. - Some retained state conservatively prevents sleep, including every enabled plugin. This PR does not promise that every inactive instance becomes eligible. - The HTTP adapter uses Express 5 router layers. Real Express tests cover nested routes, errors and disconnects. New routes must register before tracking is installed. Detached work must have durable state or explicit work tracking. - Unrecoverable in-memory cleanup debt keeps the instance awake until reconciliation. This change does not make such debt survive an unplanned process crash. No schema migration or provider configuration change is included. ## Model Used OpenAI GPT-6 via Codex, with reasoning, repository inspection, tool use and code execution. The exact deployment variant/model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run focused tests locally and they pass; the full local fixture limitation and clean full CI result are documented above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
799e4d556f |
fix: make accounting durable and synchronize cost reporting (#14997)
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
cad26c6bfb |
fix(tool-gateway): bound MCP discovery memory and concurrency (#14864)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents discover governed tools through the MCP gateway. > - A listing repeated policy and full run-row reads for each catalog tool. > - Parallel listings multiplied those allocations during run startup. > - One 900-tool baseline listing used 11,489 queries and about 1.7 GiB of extra heap in a fixture. > - This pull request shares reads within a listing and bounds whole listings across the process. > - The benefit is lower discovery memory use while execution still checks current policy. ## Linked Issues or Issue Description Refs #13115. Its on-demand target change affects the same listing loop. **What happened?** MCP discovery repeated roughly 13 reads per tool. Full run snapshots and repeated connection configurations caused large allocations. Per-listing bounds alone did not limit concurrent listings across gateways. **Expected behavior** Discovery reads shared inputs once per listing. The process bounds active and queued listings. Catalog payload size and policy evaluation still grow with the catalog. Tool execution checks current access rules. **Steps to reproduce** Create a company with a remote MCP connection, 900 catalog tools, large schemas, and a large run snapshot. Send concurrent tools/list requests using a run-bound gateway token. Run the committed benchmark for a deterministic reproduction. **Paperclip version or commit** Baseline: |
||
|
DottaandPaperclip
|
83076d7e7c |
feat: return completed handoffs to Agent Chat (#14408)
Return completed Agent Chat handoffs through a durable outbox and scope each generated update to its supplied tasks. Add recovery, browser delivery, result access, and calibrated quality coverage. Validated with two consecutive ten-case Claude/Codex campaigns, all CI checks, and a 5/5 review. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
422287eecd |
fix: preserve runner recovery, warm sessions, and task outcomes (#13338)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The native runner connects task messages, provider execution, and task outcomes. > - First-time user tests exposed gaps in recovery, completion permissions, message delivery, and Stop behavior. > - These gaps left usable output hidden, completed work waiting for bookkeeping, or safe work unable to continue. > - This pull request fixes the shared lifecycle and receipt paths while preserving process ownership and action checks. > - Users can continue work with accurate task state and durable messages. ## Linked Issues or Issue Description **What happened?** A stopped local Codex execution could remain blocked even after its processes had stopped and its complete transcript proved that no external action needed replay. Claude under Conservative permissions could fail to call task completion tools. Recovery could reuse an assistant item ID and overwrite prior output. A delivered comment could remain marked uncertain after navigation. Stop could look like Pause or a new recovery incident. Workspace contention could look like cancellation. A direct reply reopening Done could enter a clarification loop. **Expected behavior** Recover automatically only with verified termination and complete action receipts. Preserve answers and messages. Keep task completion available under Conservative permissions without broad tool access. Show crashes as Blocked, actual human decisions as In Review, and ordinary workspace contention as waiting. Stop the current response and allow a new direction. **Steps to reproduce** 1. Create ordinary response tasks with local Codex and Claude Code, then send follow-up messages through the task composer. 2. Interrupt a disposable local Codex runner during text-only work. Verify automatic continuation and retained output. 3. Stop a response, send a new request, answer a clarification, and reopen completed work with another message. 4. Navigate or reload while a comment submission is pending. Confirm the exact persisted request receipt settles it without removing newer draft text. 5. Run two tasks in a shared Daytona workspace. Confirm waiting does not appear as failure. **Paperclip version or commit** Initial acceptance baseline: `c9021c6721f91e2c74bd9fee9d3fd41c999d17b7`. Current integration base: `6cef9743c`. Both operator-interruption and workspace-waiting guards are preserved; native restart and legacy permission rules remain documented. **Deployment mode** An isolated source-built test-drive instance, with real local Codex and Claude Code providers and disposable Daytona environments. Related work: #13314, #13316, #13327, #13344, #13239, #13254, #13163. This PR addresses additional failures from ordinary task journeys, including controller restart handoff and repeated warm sandbox setup. Historical task status reconciliation is excluded. ## What Changed - Persist runner ownership immediately at spawn and resume an explicitly adopted runner even when the controller crashed before the first driver checkpoint. Detach the controller safely across graceful restarts, including session startup. Prevent an old finalizer from suspending or signaling an adopted runner. Checkpoint idle warm sessions before shutdown. Preserve the same run and queued follow-up messages. - Scope saved legacy queue successor checks to the queue owner while preserving ordinary task locks, operator identity, assignment gates, and exactly-once delivery. - Preserve managed Codex credential files when an old session is detached for restart; normal owned cleanup still copies refreshed auth back and removes the scoped copy. - Reuse the bound warm shared sandbox and fully verify an existing staged provider pack before using it. This avoids repeated uploads when the pack is already valid. - Add a narrow local Codex replacement path with stopped-process proof, a closed transcript inventory, exact completion receipts, and fresh-session lineage. Preserve no-replay holds when evidence is incomplete. Recovery may clear only the same run's recorded Blocked status version; manual re-blocking and dependency changes invalidate that receipt, while queued comments do not. Later blocks stop scheduled, queued, and final dispatch; queued/final checks re-read dependencies even when the task status stays In Progress. - Permit only task delivery and human-input tools through the isolated Claude runner's exact task bridge. - Scope assistant item identity to the provider turn and ignore only authority-free Codex skill-change notifications during startup. - Reconcile composer submissions by client request ID across response loss, navigation, and reload. Retain text typed during delivery. - Keep acknowledged run-only Stop neutral and show workspace contention as waiting. Project exhausted native failures as Blocked. - Restore the guarded task-page retry action for failed legacy runs, including the server-supported explicit new-attempt path for stopped conversation adapters. Preserve native/process recovery holds and avoid promising Retry while a decision or execution gate hides it. - Refresh delivered artifacts and handle direct user replies that reopen completed work without a clarification loop. - Check the embedded PostgreSQL PID, data directory, and actual port before connecting or migrating. - Document accepted behavior and add focused regressions at lifecycle, route, transcript, and UI boundaries. ## Verification - Final head `fece606ac2` passes the complete GitHub CI matrix: **34 green checks, two expected Storybook skips, no failures or pending checks**, including `ci / verify`, `ci / e2e`, full runner verification, typecheck, build, every server/workspace shard, and all browser shards. [CI run](https://github.com/paperclipai/paperclip/actions/runs/34727183287). Greptile is **5/5 with no open findings**. The final two commits only refine test fixtures; both affected suites pass 24/24 locally and in CI, with server typecheck green. - Complete local Vitest coverage uses the canonical groups/shards: all 635 general server suites, all 145 serialized suites, and all workspace packages. The aggregate began on `0a8001c18` while the final queue fix arrived: 23,903 passed, five failed, 87 skipped. The five port/socket/timing failures passed unchanged in follow-ups (60 tests in the exposure/file suites and 412 tests covering the serialized failures and unrun tails). The final queue/operator-identity suites separately passed 52/52. This is aggregate coverage plus explicit reruns, not a pristine single-command final-head run. - After integration with current master, queue/operator-identity/continuation suites passed 162/162 and affected UI suites passed 140/140. ACP Stop/continuation and legacy task/Inbox/message browser suites passed 9/9, including both task recovery Retry and thread Try again, automatic saved-message delivery, exactly one new run, Done, and retained output after reload. The default process Stop/Pause/Resume browser case passed (the native-provider case is opt-in and skipped by default). The complete Board attachment/receipt browser suite passed 11/11 on a disposable instance, covering both composers, exact receipts after lost responses, no replay, bound attachments, and newer drafts after reload. - Blocking-intent regressions cover pre-existing Blocked, a mismatched run/cause, an explicit manual re-block, changed dependencies, a queued comment after failure, and a block arriving between scheduling and provider dispatch. The negative cases reproduced before the fix. All 478 affected executor/recovery/dispatch tests passed; both database suites ran separately after availability-probe skips in the first combined command. The final late-dependency check passed all 143 affected recovery/dispatch tests (zero skips) after two new negative cases reproduced the bug. - Focused runtime regressions cover awaited runner ownership publication, authenticated adoption before the first checkpoint, old-finalizer detachment, idle and busy warm-session shutdown, rejected checkpoint propagation, provider-pack verification, and managed-Codex credential preservation. Four managed credential detachment cases reproduced the bug before the fix; normal owned cleanup still succeeds exactly once. - Live local Claude: SIGKILL 2.6 seconds into startup recovered the same run automatically in 53 seconds, then a normal follow-up completed in 24 seconds. SIGTERM 2.5 seconds into startup preserved the same run (54 seconds) and its queued follow-up (21 seconds). Answers remained visible and the task reached Done. - Live Claude Daytona: a warm follow-up retained its sandbox and fell from 121 seconds to 44 seconds. A separate cold turn took 127 seconds; after controller shutdown and checkpointing, its follow-up completed in 33 seconds with the same sandbox, workspace, native session, and runner. Both answers remained visible and the task was Done. - Other live journeys covered task completion and follow-up with local and Daytona Codex, local Codex crash recovery, Stop then new direction, clarification response, live artifact refresh, and shared-workspace waiting. - Validation limits: the opt-in native composer Stop/Pause→subtree Resume fixture exposes terminal/result ordering and subtree-cancellation attribution bugs that can leave a child task blocked; that new finding is assigned to a separate follow-up and is not claimed fixed here. Default CI skips this optional native-provider fixture. Managed-Codex credential handoff and the queue-agent integration use automated regression evidence. Cold custom provider-pack uploads still add startup latency. ## Risks - Automatic replacement remains deliberately narrow: local Codex, verified stopped identities, unchanged retained state, and a complete text/completion-only turn. Unknown actions, partial history, or changed ownership remain blocked. - Claude completion permission handling changes an upstream package patch. The exact isolated task bridge must remain pinned; unrelated tools keep their existing permissions. - New task failure projection changes user-visible status. No historical status backfill or database migration is included. - This is a broad lifecycle fix across server and UI. Live proof covers graceful local Claude restart during startup and idle Claude Daytona session recovery across controller shutdown. Live abrupt SIGKILL during local Claude startup also recovered the same run. Unknown ownership or missing action evidence still blocks reuse. Cold custom provider-pack uploads still add startup latency; this change avoids unnecessary repeat uploads. ## Model Used OpenAI GPT-6 (Codex), with reasoning, code execution, browser automation, and tool use. The exact hosted model ID and context window are not exposed in this task. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
3bafac12f7 |
refactor: remove automatic productivity reviews (#13263)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its recovery loop keeps assigned work moving after execution failures. > - Productivity review used run counts, comment counts, and elapsed time to create management tasks. > - Infrastructure failures could satisfy those rules and create more tasks without evidence that the source work needed management review. > - This pull request removes that detector and its continuation holds. > - Bounded recovery, budgets, explicit blockers, and normal review stages remain in place. > - Existing task records stay readable and unchanged. ## Linked Issues or Issue Description Refs #5897. That request describes unwanted automatic productivity reviews and asks to preserve existing tasks. This change retires the feature instead of adding another configuration switch. Related prior approaches: Refs #9191, Refs #12489. Those changes excluded infrastructure failures or bounded review creation. This removal replaces the detector rather than tuning its thresholds. ## What Changed - Delete the scheduled detector, automatic task creation, evidence refresh, and productivity continuation holds. - Remove computed productivity fields, special attention items, badges, and Storybook fixtures. - Retain historical origin values, decision compatibility, and recovery recursion exclusions. Add no migration and change no existing task data. - Update the execution contract. Replace feature tests with regressions for legacy task reads, ordinary attention, and bounded continuation in the presence of an old review. ## Verification - Targeted attention, issue-route, startup, and UI tests: 4 files and 101 tests passed. - Updated issue-route and UI tests: 2 files and 61 tests passed. - Bounded continuation regression: 2 cases passed, including a legacy review plus pre-dispatch cancellation churn. - `pnpm check:token-gates`: all four gates passed. - `git diff --check`: passed. - `pnpm build-storybook`: passed. - Greptile: 5/5 on `a5a612eea`, with no actionable findings. - Scheduler and historical recovery regressions: 2 files and 28 tests passed. - Repository `pnpm -r typecheck` and `pnpm build`: passed. - The complete `pnpm test:run` suite passed across the CI server, serialized-server, and workspace shards on `a5a612eea`. Stopped the duplicate local monolithic run after the full CI suite passed; no completed local full-suite result is claimed. The targeted local suites above passed. - CI serialized shard 5 initially hit a 10-second timeout in the first interaction-route test. The complete file passed locally (78 tests), then the single CI rerun passed. - All CI gates are green, including the build and end-to-end suites. - A local merge check against current `master` (`ce09ea40b`) completed without conflicts. ## Risks - API responses no longer include the computed `productivityReview` field. Consumers must stop using it. - The scheduler no longer creates management work from elapsed time, run counts, or missing comments. This is the intended behavior change. - Existing review tasks and explicit dependencies remain in place. Historical origins still prevent recursive recovery treatment. No task cleanup or data migration occurs. - The native review handoff repair is separate from this removal. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use, and code execution. The exact runtime model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e200104727 |
feat: review connection actions from tasks (#13063)
Bring governed connection reviews into task history and composer approvals. Share resolution with Connections, add scoped remembered permissions, and resume agents through durable outcome receipts. Keep cards compact, collapse raw results, isolate untrusted provider output, bound continuation payloads, and reconcile missed live events. Add Storybook coverage, browser journeys, and service regression tests. Verification: all PR CI gates passed, Greptile 5/5, security scans passed, five connection-review browser journeys passed, and real native Codex approval/continuation was verified against the local MCP fixture. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7ed122911b |
Add end-to-end session goals to Paperclip Runner
Add capability-aware slash-goal controls, durable provider goal state, PRP v2 negotiation, autonomous goal execution, and safe local session recovery. Integrate with current master, preserve provider session identity, and verify the browser goal/chat/replacement/clear workflow and unsupported-agent rejection. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e095b84dab |
feat(connections): connect services from native task feeds (#13058)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents use connections to reach external services. > - A fresh native task can have no service tools installed. > - The agent needs a way to discover services and ask the responsible person for access. > - This pull request brings the existing connection-intent flow into native task execution. > - The person can connect from the task, and the agent can continue with updated tools. ## Linked Issues or Issue Description **Subsystem affected** Native runner tool authority, connection intents, task interactions, and shared connection setup. **Problem or motivation** A task that needs an unconnected service cannot finish its work. Leaving the task to configure access also loses context. A resolved request must survive a restart and resume the correct agent once. **Proposed solution** Expose connection discovery and access requests as server-owned native tools. Render a durable task card and use the shared setup dialog. Persist outcome delivery and start a fresh provider session after access is ready. **Alternatives considered** Sending the person to the Connections page adds navigation and does not solve continuation. Polling for authorization consumes runs and can create duplicate requests. **Roadmap alignment** This extends the existing connection-intent runtime and setup experience. It reuses the shared access model and the native runner. Related: #12345, #12347. The service-slug fix in #12906 is related but separate. Companion evaluation PR: https://github.com/paperclipai/paperclip-evals/pull/21. ## What Changed - Expose `connections_search` and `connection_request` with server-bound company, task, agent, and responsible user. Preserve the legacy entry points. - Discover catalog services and authorized custom connections. Check installation, identity, health, and executable permissions before reporting ready. - Keep pending cards through ordinary messages. Reuse requests and retire stale ownership. Put Connect at the right of Not now. - Reuse the shared setup flow in a task dialog. Keep access additive and default to the requesting agent. Recover from cancelled or blocked OAuth windows with a new-tab fallback. - Persist outcome delivery with an idempotent wake key. Resume in a fresh session and recheck ownership before dispatch. - Add native browser fixtures, offline Storybook states, server contracts, and evaluation fixtures. Update guidance and documentation. ## Verification - `pnpm build`: passed after replaying the change on current master. - `pnpm -r typecheck`: passed. - `pnpm check:token-gates`: passed. - `pnpm --filter @paperclipai/ui build-storybook`: passed. - New continuation-policy regression cases: 16 passed. - Docker-backed PostgreSQL regressions passed for requester-only OAuth access, assignment-only expiry, terminal expiry, and credential-free setup metadata. - Shared setup and task-card UI tests: 121 passed, including configured MCP reconnect URL recovery and preserving user edits across refetch. - Storybook browser checks: all 119 passed on the latest reconnect fix. - `pnpm test:run`: 4,734 tests passed in the first server group, but embedded PostgreSQL startup failures and resulting cleanup errors prevented a complete local pass. All Linux CI lanes passed on the latest reviewed commit. One external-object route test returned an unexplained 500 on the first run; it passed twice locally and the failed shard passed on retry without code changes. - Earlier feature-checkout evidence: three deterministic native browser journeys passed, including restart delivery and an actual fixture tool result. Legacy scripted coverage also passed. All 59 added stories were inspected in light and dark themes. - Live Notion testing recorded successful provider reads. The manual test used a local-trusted instance. It does not prove authenticated/cloud deployment or every provider journey. - Native browser rerun reached the embedded PostgreSQL startup limit before bootstrap, so the latest checkout’s full native browser journey remains unverified. Both OAuth page/task regression cases passed against isolated Docker-backed PostgreSQL 17. They verify no premature task access, requester-only completion, additive retries, and reconnect preservation. - Applied both new migrations twice to isolated PostgreSQL 17. Foreign keys remained intact, duplicate active delivery keys were rejected, and failed delivery records did not block retries. Reviewer path: start a fresh test drive, enable the native runner, use an agent that can perform work directly, and ask it to summarize a Notion page. Connect from the card, then verify the resumed provider call and source-linked answer. The default test-drive CEO is instructed to delegate, so it can introduce an unrelated hiring step. ## Risks - Two additive migrations create durable deliveries and a partial unique wake index. They are idempotent. The wake index can require a maintenance window on large tables because migrations run in a transaction. - OAuth and continuation cross asynchronous boundaries. Tests cover ownership changes, retries, additive access, and restart delivery; live provider behavior still varies. - The latest requester-scope fix has not yet been exercised through live OAuth. GitHub, API-key, authenticated-user, and all recovery journeys are not claimed as verified. ## Model Used OpenAI GPT-6-based Codex assisted with implementation, tests, and review using tools and code execution. The runtime does not expose the exact model version, context window, or reasoning setting. Live evaluation used `gpt-5.6-luna`; manual native testing used `gpt-5.6-sol`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used and disclosed unavailable runtime details - [x] I have checked ROADMAP.md and confirmed this extends existing connection work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the feature issue template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [ ] I have run all required tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0ffc091473 |
feat(connections): add durable GitHub identities and webhooks (#12843)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents need source control access for repository work > - A shared token cannot preserve the responsible person's identity or an agent's dedicated identity > - GitHub App tokens also need durable refresh, repository access checks, and webhook delivery > - Paperclip already has managed connections, encrypted grants, run secret leases, and merge-confirmation behavior > - This pull request extends those systems with GitHub identities instead of adding a parallel credential system > - The benefit is durable GitHub access with explicit identity, repository, runtime, and webhook boundaries ## Linked Issues or Issue Description No public GitHub issue describes this connection change. This description follows the feature request template. **Subsystem affected** Connected Apps, connection grants, secret resolution, native Git runtime setup, webhook processing, and the Apps UI. **Problem or motivation** Users need to connect GitHub once and let agents use the correct GitHub identity. A run should use a dedicated agent account when one exists. Otherwise, it should use the responsible person's account. The connection must survive token expiry, repository access changes, and temporary instance downtime. **Proposed solution** Add user-owned and agent-owned GitHub grants to the existing connection model. Resolve one identity for MCP, Git, `gh`, health checks, and webhook bindings. Store provider tokens in the existing encrypted secret system. Refresh expiring token pairs under the existing lease and compare-and-swap path. Register signed Cloud webhook bindings and process normalized pull request and installation events through a durable local inbox. **Alternatives considered** An organization-wide GitHub token would lose person and agent attribution. Environment variables alone would bypass the managed connection and grant model. A new GitHub-only credential store would duplicate the existing secret and access systems. GitHub App installation tokens and private-key custody remain outside this first version. **Roadmap alignment** This change implements the Connected Apps direction. It also extends the shipped MCP Tool Gateway, per-agent secret access, and action-attribution systems. It does not add a repository catalog. The open repository catalog work in [#11234](https://github.com/paperclipai/paperclip/pull/11234) is related and complementary. ## What Changed - Added agent-owned connection grants and a per-agent credential policy with company and subject constraints. - Added a managed GitHub App method while keeping the personal access token method as an advanced fallback. - Added durable access-token and refresh-token handling with proactive rotation and one automatic recovery after a provider `401`. - Added GitHub identity and installation summaries without storing repository-name lists. - Added signed Cloud webhook binding, event lease, acknowledgement, local idempotency, pull request merge processing, and installation access handling. - Added one identity resolver for MCP, native Git, `gh`, checkout, health checks, and webhook bindings. - Added a class-3 run projection for `GH_TOKEN`, `GITHUB_TOKEN`, a `github.com`-only credential helper, SSH-to-HTTPS rewrite, and GitHub noreply commit attribution. - Added personal and dedicated-agent setup choices plus identity, repository, continuity, and webhook status in the Apps UI. - Added schema migrations, tests, and connection documentation. ## Verification - The current head is fully green in GitHub CI, including build, typecheck, all serialized/general server shards, all browser shards, policy, canary dry run, review, and security checks. - Live staging proof completed with a non-expiring GitHub App user token, selected-repository installation, repository add/remove refresh, managed MCP, native `gh`, HTTPS clone/push/delete, GitHub noreply commit attribution, signed merged-PR webhook acceptance, durable Cloud-to-instance delivery, and installation-access event processing. Temporary branches and temporary repository access were removed afterward. - `pnpm check:token-gates` passed. - `pnpm -r typecheck` passed before and after the rebase onto `origin/master`. - `pnpm build` passed. - The focused connector suite passed 285 tests after the rebase. - The full stable suite passed 5,790 tests and failed 22 tests across 8 general server files. The failures reproduced as shared-runner environment issues. They included `/tmp` versus `/private/tmp`, closed database connections, and invalid high ephemeral ports. The focused connection tests pass in isolation. ## Risks - Migrations add agent grant subjects and a durable connection-event inbox. Migration numbering and safety checks pass. - A raw GitHub user token enters the agent process for Git and `gh`. Per-tool Ask-first controls cannot limit those shell operations. The UI warns users about this boundary. - GitHub App user tokens can be non-expiring. Paperclip performs a continuity check every 30 days, but provider revocation still requires a reconnect. - The webhook path accepts only signed and bounded payloads. It stores a minimal normalized record and no raw provider payload. - GitHub repository permissions remain authoritative. Removed access can make a cached repository count temporarily stale, but runtime access fails immediately. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, `gpt-5.6-sol`, extended reasoning, tool use, code execution, browser control, and multi-file repository editing. The context window size was not provided. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
7b094724e6 |
fix(runner): recover native sessions across restarts (#12845)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Paperclip Runner keeps durable run and provider state outside one server process. > - A server restart can leave that runner alive or can interrupt it after a provider checkpoint. > - The old startup path used handoff intent and PID evidence, but it did not reconstruct native ownership. > - That gap could block the issue, create a replacement run, or start duplicate provider work. > - This pull request adds durable same-run recovery for coordinated and uncoordinated restarts. > - The benefit is exact recovery of the run, runner, session, provider, steering, and finalization state. ## Linked Issues or Issue Description Refs #9628. That pull request added earlier local-adapter hot-restart work. This change adds native PRP authority reconstruction and same-run provider resume. Refs #10935. That pull request handles missing hot-restart snapshots. This change also supports hard restarts with no snapshot. Refs #11624. That pull request prevents unsafe retry after an adopted legacy process exits. This change reconciles native terminal evidence before provider recovery. Refs #12070. That pull request improves process liveness checks. This change also binds recovery to a process-start fingerprint and fails closed on ambiguity. **What happened?** The server could record hot-restart intent, but startup did not rebuild native runner ownership. A live runner could not re-register its PRP authority. A dead runner could not resume the exact native and provider session on the same heartbeat run. Generic recovery could then block the issue or create replacement work. **Expected behavior** A live native runner must reconnect with the same PID and logical identities. A dead runner must resume the same durable session and heartbeat run with only a new operating-system PID. A proposed or terminal result must finalize once before any provider turn starts. Ambiguous process or session evidence must stay blocked without a signal or duplicate spawn. **Steps to reproduce** 1. Start a Paperclip Runner heartbeat and wait for an active provider turn. 2. Restart only the Paperclip server, with or without a hot-restart marker. 3. Observe that the old startup path does not reconstruct the native control-plane authority. 4. Kill both the server and runner after a provider checkpoint. 5. Observe that the old path cannot resume the exact native session on the original heartbeat run. **Paperclip version or commit** The defect was reproduced from commit `1991f31fd53e7f7794d5c2e4b93be384ade2b41d`. This branch is rebased onto the current `master`. **Deployment mode** Local development and self-hosted server deployments that use the local Paperclip Runner. ## What Changed - Added correlated hot-restart requests and version-compatible native handoff fields. - Added controller boot identity, process-start identity, controller generation, recovery state, request id, and bounded history to the native finalization ledger. - Added transactional recovery claims for live-runner reattach, dead-runner resume, and incomplete bootstrap. - Added fail-closed ownership takeover rules and process identity validation. - Added live runner adoption to the local runner transport without a duplicate spawn. - Added same-run provider checkpoint resume and legacy retry-row compatibility. - Reconciled proposed and terminal results before runner or provider recovery. - Bound the HTTP and PRP listener before startup recovery and delayed scheduling and generic reapers until classification completes. - Added restart-aware health diagnostics, run-log recovery transitions, durable runner diagnostics, and bounded shutdown finalizer draining. - Moved restart-survivable diagnostics into runner-owned, pre-redacted bounded writes; raw stdout and stderr are never persisted. - Added process-start fencing for controller, runner, and provider PIDs; startup classifies every candidate without an implicit cap. - Added crash-recoverable, contention-safe development restart-request coordination and failed-startup listener cleanup. - Added a credential-free real-process restart suite for eight restart, scale, and identity scenarios. - Documented native restart operation, persistence, diagnostics, and verification. ## Verification - The documented native restart commands passed. They ran eight real-process/database recovery scenarios and the live runner adoption transport test. - Native executor tests passed: 111 tests. - Heartbeat recovery tests passed: 124 tests. - Hot restart, health, and shutdown tests passed: 52 tests. - The broader affected server suite passed: 350 tests. - Focused native recovery and startup tests passed: 49 tests. - Runner transport and control-plane tests passed: 63 tests. - Runner-owned diagnostic tests passed for write-time bounding, credential redaction, private file modes, and raw stream non-persistence. - Development restart coordination tests passed: 11 tests. - Database migration checks and the partial-application/replay regression test passed. - Server, database, and Paperclip Runner typechecks passed. - `git diff --check` passed. - Full Paperclip PR CI passed, including build, canary, all five general server shards, all five serialized server shards, all three browser E2E shards, workspace suites, and release-registry verification. - Greptile completed at 5/5 with no outstanding findings, recommendations, follow-ups, or open review threads. ## Risks - Moderate risk. This changes startup ordering and ownership transfer for active native runs. - The migration adds nullable columns and does not rewrite existing rows. - Recovery fails closed when process or durable session identity is incomplete or contradictory. - The first implementation supports the local Paperclip Runner. Remote targets keep their existing behavior. - The real-process suite covers cleanup and asserts that no runner or provider process survives each test. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex with GPT-5. The runtime did not expose a more specific model revision or context-window size. Repository editing, shell execution, database tests, and real-process test execution were enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
141f202e40 |
Clean up experimental settings features (#12681)
## Thinking Path > - Paperclip is the open source app that people use to manage AI agents for work. > - Instance settings control optional product features and developer tools. > - The experimental settings page mixed active experiments, internal tools, and old recovery controls. > - Some workspace links also used the selected company instead of the workspace owner. > - These problems made settings hard to scan and could send users to the wrong company route. > - This pull request removes old controls, groups developer settings, and resolves workspace links from workspace data. > - The benefit is a smaller settings surface and correct workspace navigation. ## Linked Issues or Issue Description **What existing behavior does this improve?** This improves the instance experimental settings page, task watchdog controls, dependency wake recovery, and execution workspace routes. **Current behavior** The settings page shows old recovery controls and mixes product experiments with internal developer settings. Task watchdogs require an extra feature flag. Some direct workspace links use the current company prefix instead of the company that owns the workspace. **Proposed behavior** Remove the old task recovery experiment and its unused API surface. Make task watchdog controls available without the removed flag. Put worktree execution and managed environment controls in the developer section. Resolve direct workspace links from the workspace owner and reject a company prefix that does not own the workspace. **Reason and benefit** The smaller settings page is easier to understand. The server keeps only the dependency wake backstop that it still uses. Workspace links open under the correct company route. **Breaking changes** This removes the experimental issue graph recovery preview and run endpoints. It also removes the task watchdog feature flag. Task watchdog data and dependency wake behavior remain available. ## What Changed - Removed the old task watchdog and issue graph recovery feature flags. - Removed the old issue graph recovery preview, run controls, API contracts, and unused recovery implementation. - Kept resolved dependency wakes as the scheduler backstop. - Grouped product experiments and Paperclip developer settings on the instance settings page. - Made task watchdog controls available without an extra experimental flag. - Added owner-aware redirects and company checks for execution workspace routes. - Hid the false stopped-state badge while a workspace has no active runtime state. - Updated focused server and UI tests for the new behavior. ## Verification - `pnpm check:token-gates` - `pnpm -r typecheck` - `pnpm build` - `pnpm test:run` completed with 5,620 passing tests and four failures in unchanged workspace runtime port tests. The same four failures repeat when the two files run alone. - The complete GitHub CI matrix passed, including all server, serialized server, build, canary, and end-to-end jobs. ## Risks - Clients that call the removed experimental recovery endpoints must stop calling them. - The route checks depend on workspace detail access. An unknown or cross-company workspace returns the global not-found page. - There are no database migrations, lockfile changes, workflow changes, or design image changes. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex with GPT-5. The exact deployment ID and context window are not exposed. Reasoning, tool use, and code execution were enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
bc9ba7cd26 |
feat(runner): project native runs into task threads (#12321)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The experimental Paperclip Runner can execute a guarded Codex run and persist provider-neutral events. > - The task page still reads direct-adapter transcripts and cannot present those native events. > - Structured runner questions must also use the existing task interaction experience. > - Runtime selection must use the persisted run mode, not an adapter name or a current feature flag. > - This pull request projects native events and questions into the existing task thread. > - Direct adapters keep their existing transcript, composer, interaction, and finalization paths. > - The benefit is a complete native Codex task thread without a behavior change for existing adapters. ## Linked Issues or Issue Description Refs #12202. This pull request replaces that stale implementation on current `master`. **What happened?** The server persists native runner events and structured input requests. The task page only consumes direct-adapter transcripts. A native run therefore cannot present a complete transcript, usage, or question flow through the normal task experience. **Expected behavior** Native runs project persisted provider-neutral events into the existing task thread. Native structured questions use the existing interaction card. Direct adapters retain their current behavior. **Steps to reproduce** 1. Enable the experimental runner. 2. Start a native Codex run that emits progress, usage, a structured question, and a final reply. 3. Open the task page. 4. Observe that the direct-adapter transcript path cannot project the native event records. **Paperclip version or commit** `master` at `67f9867bc`. ## What Changed - Add the canonical structured-question validator and shared contract exports. - Materialize native input requests as existing task interactions. - Validate native answers and deliver them through the durable question-response receipt. - Resume the original PRP request with an idempotent `request.resolve` command. - Project native messages, tool activity, cumulative usage, and final replies into the existing transcript model. - Propagate persisted `runtimeMode` to the task page and select native handling only for `runtimeMode: "native"`. - Expire pending interactions through the shared issue service on every terminal transition, including decisions, stalled reviews, tree control, and pipeline retry cleanup. - Queue native run cancellation while a transaction is open and execute it only after the owning transaction commits. - Keep nonterminal and non-runner issue paths on their existing service call shapes and behavior. ## Verification - `pnpm --filter @paperclipai/server typecheck` — passed, including the Rust runner release build and protocol/catalog drift gates. - Focused native-thread and lifecycle suites — 18 files and 481 tests passed during review. - `issue-execution-policy-routes.test.ts` — 19/19 passed after the final transactional-queue expectation update. - `issue-agent-mutation-ownership-routes.test.ts` — 87/87 passed in the final isolated compatibility rerun. - GitHub Actions — policy, build, canary, typecheck/release registry, 5 serialized server shards, 8 general-test shards, 3 browser shards, and both aggregate gates passed on `7793f3193`. - Security — Snyk, Socket Project Report, Socket PR Alerts, and Superagent passed. - Greptile — 5/5 on `7793f3193`; all actionable review threads resolved. - `git diff --check` — passed. - Diff against `master`: 44 files. ## Compatibility Boundary - Native transcript polling only runs when the persisted run reports `runtimeMode: "native"`. - Missing or legacy runtime modes continue through `useLiveRunTranscripts`. - Legacy questions keep the existing optional free-text choice. - Native closed select sets can suppress that legacy fallback. - Terminal cleanup uses the same issue service for native and legacy interactions; only a bound native question schedules a native run cancellation. - Native cancellation happens after transaction commit, so failed or rolled-back writes do not cancel a still-valid run. - The durable delivery service checks the original native request before it considers a continuation run. - This pull request adds no migration, dependency, workflow, manifest, or lockfile change. ## Risks The main risk is routing a direct-adapter task through native handling or changing terminal issue behavior. The implementation selects the native path only from persisted runtime facts, retains the existing nonterminal call shape, and schedules native cancellation only for a validated bound native question after commit. Focused and repository-wide tests cover both paths. Native requests remain bound to the company, issue, run, and agent; answers are validated, durable, and idempotent across reconnects. ## Model Used OpenAI Codex, GPT-5 family. The client does not expose the exact deployment ID or context window. Agentic reasoning, tool use, and code execution were enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run the affected local tests and they pass - [x] I have added or updated tests where applicable - [x] I have updated the compatibility notes for this change - [x] I have considered and documented risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I addressed all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
b51112798f |
feat(apps): improve gateway and workspace connection UX (#12340)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - App connections must work in both the operator UI and agent tool gateway. > - The first stack layer adds secure remote connections. > - Operators still need clear setup, test, and recovery states. > - This pull request adds the gateway behavior and the workspace connection experience. > - The benefit is a connection flow that is easier to understand and recover. ## Linked Issues or Issue Description Refs #11965 This is stack 2 of 11. It depends on stack 1 and replaces another reviewable part of #11965. ## What Changed - Improve remote tool gateway connection behavior. - Add clearer app setup, test, and recovery states. - Add focused server and UI tests for the new paths. - Keep the diff isolated from later identity and catalog work. - Stabilize DNS-pinned remote HTTP protocol fixtures and the managed-runtime public-origin fixture for this independently tested layer. ## Verification - `pnpm -r typecheck` - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/tool-access-service.test.ts` (150 passed) - `pnpm test:run` - `pnpm check:token-gates` - `pnpm build` ## Risks - Gateway errors now surface through new user-facing states. - A stale connection can require a new setup attempt. - The change does not add a database migration. - The injected HTTP transport and public URL are test-only fixtures; production DNS pinning and runtime behavior are unchanged. > I checked `ROADMAP.md`. This stack continues the existing app connection work from #11965 and does not duplicate another planned item. ## Model Used OpenAI Codex, GPT-5. The runtime model ID and context window were not exposed. The model used reasoning, tool use, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
67f9867bc6 |
fix(interactions): deliver question answers durably (#12307)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents can pause a task and ask the user structured questions. > - The answer is durable in the issue interaction, but delivery to the next run is not durable. > - A process restart can therefore leave an answered interaction without a continuation attempt. > - Native runners also need a provider-neutral question contract before the task page can consume native events safely. > - This pull request adds a content-free delivery outbox and an optional native steering seam. > - Direct adapters keep their existing heartbeat continuation path. > - The benefit is reliable answer delivery without changing runtime selection or task-page behavior. ## Linked Issues or Issue Description Refs #12202. This pull request replaces the question-delivery foundation from that stale task-thread pull request. The task-thread projection will follow in a smaller pull request. **What happened?** Question answers were stored in the issue interaction. The server then made one in-memory continuation wake. A server stop between those operations could leave the answer stored but not delivered. The combined native task-thread pull request also made this behavior hard to review separately from UI changes. **Expected behavior** The answer and its delivery receipt must commit in one transaction. The server must retry pending receipts after a restart. Existing direct adapters must keep the current wake path. A native runtime may use the optional steering seam, but this pull request does not enable native steering in production. **Steps to reproduce** 1. Create an `ask_user_questions` interaction. 2. Answer the interaction. 3. Stop the server before the continuation wake completes. 4. Start the server again. 5. On current master, no durable record tells the server to retry the answer delivery. **Paperclip version or commit** Current `master` at `4d82f5eae`. ## What Changed - Add the `issue_question_response_deliveries` table and migration. - Store only routing state, a correlation ID, and a payload digest in the delivery row. The answer remains in the existing interaction result. - Commit an answered interaction and its pending delivery row in one transaction. - Add bounded claims, retry recovery, cumulative terminal state, and content-free activity records. - Keep every built-in direct adapter and external adapter on the existing heartbeat wake path. - Add an optional native steering seam. No production caller supplies that seam in this pull request. - Retain the provider-neutral `paperclip.question_set.v1` presentation on recovered interactions. - Run delivery immediately after an answer and sweep pending rows at startup and on the existing server interval. - Add focused database, service, route, startup, adapter-matrix, digest, and duplicate-delivery tests. ## Compatibility Boundary - This pull request does not change adapter selection. - This pull request does not start runnerd. - This pull request does not create native run records. - Direct adapters never call the native steering seam. - The existing interaction result stays authoritative for answer content. - The migration is additive and does not rewrite existing rows. - This pull request has no UI, dependency, workflow, package-manager, or lockfile changes. - The diff has 19 files. ## Verification - `pnpm exec vitest run server/src/__tests__/question-response-delivery.test.ts server/src/services/issue-thread-interactions.test.ts server/src/__tests__/issue-thread-interaction-routes.test.ts server/src/__tests__/server-startup-feedback-export.test.ts` — 4 files and 120 tests passed. - `pnpm -r typecheck` — passed for all applicable workspaces. This includes Cargo format and check, protocol drift checks, and migration safety. - `pnpm build` — passed. This includes the Rust release binary, server build, and UI production build. - `git diff --check` — passed. - Secret patterns were not present in the changed text files. - The repository token gates currently report violations from unchanged files on `master`. This pull request does not change those files. ## Risks The main risk is routing a direct-adapter answer into a native session. The service checks the persisted runtime mode, and the adapter matrix proves that all direct adapters use only the existing wake path. The new table is additive. It has foreign keys, unique correlation constraints, bounded attempts, and status checks. Activity records omit question and answer content. ## Model Used OpenAI Codex, GPT-5 family. The client does not expose the exact deployment ID or context window. Agentic reasoning, tool use, and code execution were enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes:` / `Closes:` / `Refs:` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket ID or instance-derived details - [x] I have run the affected tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have documented the new contracts and compatibility boundary - [x] I have considered and documented compatibility and security risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Dotta
|
9964b034bb |
feat(runner): add hidden server PRP coordinator (#12176)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Paperclip Runner needs a narrow server trust boundary before an adapter can start it. > - The package has durable runner transport, but the server does not host or authorize that transport. > - Native persistence exists, but no writer connects PRP events to those records. > - A direct adapter must not enter this path by accident. > - This pull request adds a hidden, run-bound PRP server coordinator. > - The benefit is a recoverable server boundary that remains unavailable to normal execution. ## Linked Issues or Issue Description Refs #11962 Refs #12129 Refs #12169 **Subsystem affected** Cross-cutting. The change affects the runner package and server orchestration. **Problem or motivation** The server cannot authenticate runnerd, commit PRP events before ACK, authorize semantic tools, or enter native finalization from a durable runner result. The application must have this hidden boundary before a guarded adapter can use the runner. **Proposed solution** Add an authenticated PRP WebSocket authority and register it only for one exact persisted native Codex run. Bind each connection and event to the company, issue, agent, run, runner, session, turn, item, and verified runner identity. Commit each event before its cumulative ACK. Project only authorized same-task read tools. Rebuild the accepted result and finalization record from durable result and terminal events. **Alternatives considered** The server could expose a broad runner API key or route semantic calls through existing adapter endpoints. Those options grant too much authority and weaken replay recovery. The server could also add the user-facing adapter in this pull request. That option would mix rollout selection with the transport trust boundary and make legacy compatibility harder to review. **Roadmap alignment** This work supports the shipped enforced-outcomes, governed-tool, and self-healing-run milestones. It does not add a new roadmap surface. ## What Changed - Add the durable PRP server authority with one-use bootstrap tickets, reconnect leases, encrypted frames, bounded state, cumulative ACKs, and idempotent commands. - Add `/api/runner/v1/connect/:runId`. Derive its `ws://` or `wss://` URL from the configured Paperclip API URL. - Register one authority only after the coordinator verifies the complete native Codex run binding. - Commit validated PRP events to `heartbeat_run_events` before ACK. Reject source gaps and conflicting replays. - Rebuild accepted results and finalization records from durable result and terminal events. Enforce finalization owner leases and retry times. - Project five same-task read operations. Recheck run, agent, task, and company authority for each call. - Keep the route hidden. No adapter selects this coordinator, and no code starts runnerd. - Vendor the compiled runner TypeScript runtime into the server package while keeping the workspace package development-only for the server. - Document the package, database writer, run-log payload, and credential exclusions. ## Verification - Run `pnpm --filter @paperclipai/paperclip-runner check:all`. All TypeScript protocol checks and 69 Vitest tests pass, including commit-before-ACK crash recovery. All 43 Rust unit tests and 13 Rust integration tests pass. Conformance and replay parity pass. - Run the focused server WebSocket, coordinator, package-build, and startup-wiring suites. All 26 tests pass, including a clean-checkout reproduction with the runner `dist` directory absent. - Run `pnpm -r typecheck`. - Run `pnpm test:run`. - Run `pnpm build`. - Confirm that the diff contains 19 files. Confirm that it contains no workflow or `pnpm-lock.yaml` change. ## Risks - The server installs the WebSocket route at startup. An unregistered or malformed run path fails closed and creates no native record. - Bootstrap tickets are one use. The private state directory uses mode `0700`, and the state file uses mode `0600`. The file stores derived authentication verifiers and never stores raw tickets or lease tokens. - The journal has explicit frame, command, event-window, and file-size bounds. A bound violation closes the runner connection or rejects the command. - A runner event reaches the database before its ACK. A crash between event commit and ACK causes a byte-equivalent replay, not a second logical effect. - The coordinator accepts only an existing queued or running native Codex row with exact company, task, agent, runner, session, and completion-contract ownership. - Existing direct adapters do not call this service. They keep their current execution, transcript, result, and finalization paths. - The server has no production dependency on the private runner package. Its build copies the compiled runtime into `server/dist`; the workspace link is development-only. This adds no external package and does not change the lockfile. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex with GPT-5. The exact deployment ID and context-window size are not exposed. The model used agentic reasoning, repository tools, code execution, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Nicky LeachandPaperclip
|
c1c46f1e4e |
feat: Claude login on the new-agent page before agent creation (#11347)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Claude local adapter supports subscription login through a sandbox > - The new-agent page must show login before the user creates an agent > - Test results must not expose raw sandbox diagnostics or secret values > - This pull request adds the login UI to both Test lanes and closes the diagnostic boundary > - The branch also adds durable cleanup recovery for failed sandbox teardown > - Reusable sandboxes must retain both their recorded teardown configuration and a valid lifecycle path until destruction succeeds > - The benefit is a usable login flow with fixed public checks, redacted server logs, and recoverable sandbox cleanup ## Linked Issues or Issue Description Related public work: [#9488](https://github.com/paperclipai/paperclip/pull/9488) adds first-class recognition for `CLAUDE_CODE_OAUTH_TOKEN` in headless and remote runs. Related public issue: [#2681](https://github.com/paperclipai/paperclip/issues/2681) requests Claude Code subscription support. This pull request adds the login transport and new-agent UI flow that those changes do not provide. **Subsystem affected:** Claude local adapter, server login probes, sandbox provider setup, cleanup recovery, and the new-agent UI. **Problem or motivation:** The Test lanes did not show the sandbox login panel in all supported cases. Test results also exposed raw probe diagnostics, and JSON escapes could end secret redaction early. **Proposed solution:** Surface the login capability through the bundled provider manifest. Prepare the same probe runtime in the ACP lane. Send diagnostics only to redacted server logs. Keep Test checks on fixed public messages. Normalize login URL hints to allowlisted HTTPS Claude and Anthropic hosts. Consume JSON escapes during redaction. Preserve failed sandbox cleanup state across retries and restarts, and prevent deletion from severing the lifecycle context of a live reusable sandbox. **Alternatives considered:** Keep raw diagnostics in Test checks or trust login URL text from the sandbox. Both choices increase information exposure. Keep separate probe behavior in the ACP lane. That choice would leave the two Test lanes inconsistent. ## What Changed - Surface the sandbox login panel on both Test lanes. - Reconcile the bundled Daytona plugin manifest so `supportsSetupTokenLogin` reaches the UI capability gate. - Prepare the ACP Test lane with the same probe runtime as the CLI Test lane. - Add the `claude_acp_login_probe_unavailable` warning when the ACP probe cannot run. - Send raw sandbox diagnostics only to redacted server logs. - Keep Test checks on fixed public messages in the ACP, managed-config, and CLI paths. - Normalize login URL hints to allowlisted HTTPS Claude and Anthropic hosts. - Redact JSON and escaped-JSON secret values, including escaped quotes and backslashes. - Preserve orphan cleanup records across provider failures, restarts, and unavailable plugins. - Atomically block environment deletion while a live reusable sandbox lease still depends on it. - Verify pending cleanup destroys plugin sandboxes with the provider configuration recorded on the lease, even after the current environment configuration changes. ## Verification - Head under review: `506b7fa2d83c36bfa5fd722ee9d95b0c7431c241`. - Focused environment route/service/runtime coverage passes: 196 tests across 3 files. - `pnpm -r typecheck` passes. - `pnpm build` passes. - The full Vitest run completed with 4,754 passing and 28 failing tests. All 23 source-test failures reproduce unchanged on parent head `58cfe61a33191ce03d965d65085d26064b4888ba`; the other 5 are duplicate executions from stale `server/dist` output. The failures are unrelated macOS path/listener and scheduler-fixture failures, so there is no new bad commit for bisect to localize. - All required CI checks pass for the current head, including build, typecheck/release registry, all server and workspace shards, serialized server suites, canary, and e2e. - A fresh Greptile review for `506b7fa2d83c36bfa5fd722ee9d95b0c7431c241` reports 5/5, “safe to merge,” with no blocking failure remaining. ## Risks - A probe or redaction change could hide useful server diagnostics. - An allowlist change could reject a valid Claude login URL. - Cleanup recovery changes could affect provider teardown ordering. - An environment with a live reusable sandbox can no longer be deleted until the owning issue or execution workspace completes teardown. - The implementation keeps public Test messages fixed and sends detail to redacted server logs. ## Model Used OpenAI GPT-5 via Codex — exact model ID: GPT-5; tool use and code execution enabled; extended reasoning enabled. The implementation author used AI-assisted development. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and documented the result - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation or confirmed no separate documentation change is needed - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |