mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-24 18:19:05 +02:00
2503d8dda4c96fd4a092fba948be4e6cb634ef0e
2220
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
2503d8dda4 |
Retain source projects and repository receipt origins
Fence configured folder owners before capture and publication. Preserve valid task continuation after configuration removal while keeping revoked or deleted project scopes private. Record origin only at new receipt creation to distinguish unavailable independent checkouts from legacy aliases. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
93b1a95969 |
Authorize file roots and preserve explicit child sources
Resolve repository publishers by physical provenance instead of repository name. Check file APIs against retained and current folder sources, and preserve explicit child source overrides before parent binding inheritance. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
4fad5c0db1 |
Merge latest master task context and chat normalization
Preserve task workspace source selection around extracted heartbeat context preparation. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
cf5987ccb1 |
Preserve shared-root denial in task workspace API fixture
Separate unrelated task files before testing the positive source-only repository path. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
fb21a3f30a |
refactor(chat): extract provider message normalization (#15823)
## Thinking Path > - Paperclip manages AI agents and keeps their work attached to company tasks. > - Chat providers deliver messages and corrections into those tasks. > - The chat channel service has grown to 39,155 lines. > - Provider message normalization has no database or worker ownership of its own. > - This pull request moves that responsibility into one module while preserving the service body. > - Reviewers can now find provider identities, timestamps, and edit revisions together. ## Linked Issues or Issue Description **What existing behavior does this improve?** Maintenance of inbound chat message normalization in the server. **Subsystem affected** Server chat channels. **Current behavior** Provider message parsers and identity rules share a 39,155-line file with credentials, runtime ownership, database admission, and publication workers. **Proposed behavior** Move provider message normalization into `chat-channels/provider-messages.ts`. Keep service exports and runtime behavior unchanged. **Reason and benefit** This creates a clear first boundary for an incremental refactor. A dated plan ranks the next four larger extractions. No duplicate chat refactor was found in the GitHub title search. **Breaking changes** None intended. No schema, endpoint, transport, or public service API changes. Stack order: #15823 (provider messages) → #15834 (resource reconciliation) → #15836 (identity links) → #15844 (publication replacement) → #15845 (failed-run retries). Merge in this order; retarget each child after its predecessor lands. ## What Changed - Move provider message parsing into `chat-channels/telegram/messages.ts`, `slack/messages.ts`, `discord/messages.ts`, `microsoft-teams/messages.ts` and `github/messages.ts`. - Keep shared actor normalization, actor data and the inbound text limit in `chat-channels/message-lifecycle.ts`. These six modules total 566 lines / 19,534 bytes. - Import provider modules directly. Follow this provider-folder convention for future extractions. - Preserve exact actor IDs, edit hashes, timestamp provenance, and Telegram message identity rules. - Narrow two input types to the Teams parser method and the retained delivery fields actually read. - Reduce `chat-channels.ts` from 39,155 lines / 1,509,090 bytes to 38,643 lines / 1,491,703 bytes. - Record five ranked extraction candidates and verification evidence in the dated plan. ## Verification - The original extraction passed complete CI and 5/5 review at `cbba30a526`. Those results are historical; the updated head is `90ac517e6c310c8cf194dbbd668006f740e58f3a`. - Structural comparison: all 21 moved declarations match the base after normalizing exports and two type-only changes. All remaining non-import declarations, including the complete service body, match the base. - Provider-folder update: the same 48 public service integration cases passed locally before and after the move on PR #15823. They cover Slack/Discord revision identity, Teams edits, GitHub edits, Telegram rich messages and provider chronology. - All 22 message declarations retain identical bodies; the service changes imports only. The four dependent extractions rebased without conflicts. - Final-stack workspace `pnpm -r typecheck` passed. Module-boundary and whitespace checks passed. - Final-stack `pnpm build` passed. The fresh `pnpm test:run` was interrupted after PostgreSQL startup failures recurred. An isolated `native-status-arbiter-corpus.test.ts` run failed during database initialization after five attempts; all 45 tests skipped. All 32 host shared-memory slots were occupied. The full local gate remains unverified; only the task-owned run was stopped. - Greptile rated updated head `90ac517e6c310c8cf194dbbd668006f740e58f3a` 5/5 with no inline review threads. - Updated-head CI passed: 53 successful checks and two skipped checks, including all chat integration shards, typecheck, build and browser suites. - Tests use disposable databases and fixture provider transports. No live third-party execution is claimed. ## Risks - A wrong module import or changed revision identity could affect message admission. Existing service-path tests and the declaration comparison cover those risks. - Authentication, company-scoped transactions, leases, background workers, and shutdown remain in the original service. - The service closure remains large. Resource reconciliation is the recommended next extraction. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, local code execution, and repository tools. The exact runtime model ID and context-window size are not exposed in this session. No separate planning or review agent was used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ce15fe7c73 |
refactor(heartbeat): extract task context preparation (#15872)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Heartbeat execution prepares task context before it selects an environment and starts an adapter. > - This preparation still occupied 319 lines inside the executor after the earlier extractions. > - It combines continuation summaries, comments, prompt variants, communication guidance, and redaction. > - This pull request moves that responsibility into one typed module under `services/heartbeat/`. > - The executor keeps admission, resource ownership, dispatch, and cleanup ordering together. > - The result makes the preparation phase easier to find and test without changing execution behavior. ## Linked Issues or Issue Description Refs #15825, #15807. This continues the incremental heartbeat extraction series. A search found no duplicate task-context extraction. **What existing behavior does this improve?** Maintenance and review of heartbeat task context preparation. **Current behavior** The executor assembles prompts and trust-filtered context inline. This hides its execution phases among preparation details. **Proposed behavior** Call `prepareHeartbeatTaskContext` at the same point in execution. Preserve statement order, prompt contents, public exports, and the shared run context object. ## What Changed - Move task context assembly to `server/src/services/heartbeat/task-context.ts`. Inputs group task state, policy, and two existing service methods. Results preserve the continuation and wake values used by later runtime selection and dispatch. - Keep company-scoped loading, quarantined-content filtering, secret redaction, and revision hashing in their original order. Add no execution resource acquisition. - Add four public-path integration tests before the move. Cover prompt variants and redacted revisions, quarantined summaries/comments, a mixed-company wake batch, and stale caller fields. - Extend the real process worker in the opt-in browser suite to check persisted prompt variants and the description revision before it completes its task. - Update the module map and acceptance-test documentation. - Reduce `heartbeat.ts` from 5,314 lines / 220,784 bytes to 4,989 lines / 206,650 bytes. The new module has 376 lines / 15,912 bytes. ## Verification - Before the move: 11 existing prompt tests and all four new characterization tests passed against the inline implementation. - Normalized syntax comparison passed for all 319 moved lines and the remaining orchestrator, including public exports. Imports and the typed boundary were reviewed separately. - `pnpm -r typecheck` passed. - `pnpm build` passed. - The clean post-move focused run passed all 26 tests in five files, with no skips: task-context characterization, task-markdown, Slack invitation, accepted-plan workspace refresh, and run-preparation. - The local `pnpm test:run` attempt was interrupted after database failures while the Mac exhausted its shared-memory slots. All 15 affected native warm-ownership database cases passed in an isolated rerun after the browser instance stopped; the other 666 cases in that file were excluded by the test-name filter. The interrupted command is not a full-suite pass. Full CI passed on `6bf86cfdc07e7fe34015f7b719218639f4232510`: 55 successful checks and two optional Storybook checks skipped. Greptile reviewed the latest commit at 5/5; its one earlier worker-test finding is fixed and resolved. - Browser E2E passed both scenarios on the final code: persistent timeouts stop after three runs and survive reload; one timeout recovers without manual Retry, verifies task-context prompt variants/revision and preserved workspace files in a real process, then completes through the run-scoped API. The first recovery attempt exposed an incorrect new test assertion because the fixture title equals its description. The worker now checks the description section specifically; the corrected recovery scenario passed in 55.9 seconds. The original failure trace was preserved. These tests use a deterministic process adapter and make no live model calls. ## Risks This is a behavior-preserving extraction. The main risks are changed closure bindings, missing return values, or changed preparation order. Typed inputs, public-path tests, and syntax comparison cover those risks. The run context is still the same mutable object. The helper acquires no leases or processes. It introduces no schema, API, or prompt policy changes. The first post-move database run skipped cases due to host resource exhaustion. All 26 focused cases passed on the subsequent clean run. Browser E2E passed after the worker assertion fix. The full local suite remains incomplete. All 15 native warm-ownership database cases passed on their isolated rerun; The full CI test suite passed on the final commit. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository inspection, code execution, and test tools. The host does not expose the exact deployment model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f29be9b58e |
Merge origin/master into task workspace ownership
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e1863d4fe7 |
Fence shared workspace readers and preserve projectless chat admission
Retain root source and reader provenance, recheck publication access, keep plain agent transfers independent of repository discovery, and resolve organizational projects without filesystem sources. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
88b7bbb5e2 |
Stabilize task lifecycle browser fixtures (#15878)
## Thinking Path > - Paperclip manages agents and their tasks. > - Browser and runner tests check task recovery and user actions. > - Some fixtures act before a prior request or run has settled. > - Those races can report a product failure before the intended assertion runs. > - This PR binds each check to the exact request, run, or geometry snapshot it needs. > - The tests retain their existing budgets and negative assertions. ## Linked Issues or Issue Description Refs: #15813, #15804. The draft stack is master → [fixture prerequisite #15878](https://github.com/paperclipai/paperclip/pull/15878) → [runtime prerequisite #15880](https://github.com/paperclipai/paperclip/pull/15880) → [Boat core #15813](https://github.com/paperclipai/paperclip/pull/15813) → [UI #15804](https://github.com/paperclipai/paperclip/pull/15804). GitHub confirms 5, 63, 97, and 75 changed files respectively, each below the 100-file limit. Existing commits and PR URLs remain intact. [Historical evidence](https://github.com/paperclipai/paperclip/tree/5909bfac9fa41405beb3fe8a65253dc28ba52835/doc/plans/assets/2026-10-10-boat) remains pinned to its original commit; current proof files stay with the UI PR. This is the test-only prerequisite extracted from the experimental Boat work. It contains five existing test-file changes. It does not add the Boat adapter or change task admission. ## What Changed - Wait for the exact signoff fixture run and each replacement run to succeed before applying the next stage decision. - Wait for the inbox Retry response for the exact failed run before navigating away. - Compare task planning time with the first run that actually started, instead of assuming the first list item started. - Read the touch target, container, and scroll position in one browser evaluation. Keep strict containment and native gesture checks. - Bind the native recovery fixture to its disposable fake Codex executable so installed provider binaries cannot override it. ## Verification Draft branch: `codex/boat-fixture-prerequisite`, commit `e318850909c0777565b43644a4d8114c6006da60`. Base: master `e733bddc45e00efb1819fffa8677f7b0d07802c5`. Five changed files. The isolated fixture-plus-runtime stack passed 24 browser cases. Six legacy recovery cases could not reach their assertions because the fixture requires embedded PostgreSQL metadata. A normal embedded PostgreSQL attempt then failed during cluster setup on this host. These are not six passing cases. No test seam, timeout increase, or product-database fixture was used. The fixture-only commit has not yet had a separate complete local browser run. All six legacy cases subsequently passed on this exact fixture commit in PR CI browser shard 8 (20/20 total, job114470999277). The original local setup failures remain recorded. The full-stack typecheck/build results belong to the assembled fixture-plus-runtime tree and are not claimed as independent qualification of this smaller commit. Review the exact run/request filters and the unchanged rejection assertions. Run the four changed Playwright specs and the native-session-resume suite through the repository's normal fixture setup. At 13:03:24 UTC, this exact head had 53 successful checks and two skips, with no pending checks or failures. All eight browser shards passed. Greptile is 5/5 on the exact head with no unresolved threads. [Exact-head legacy browser proof](https://github.com/paperclipai/paperclip/blob/9eacca2c5321bb63f061eef4bb775b6951d5846d/doc/plans/assets/2026-10-10-boat/evidence/prerequisite-legacy-browser-ci-proof.json). Feature acceptance stays in the downstream report. Deployment29 verified `05f3725af838b5a074707df1d6ed1e193bdfe49e` at 12:23:23.944 UTC. Native Claude cold execution passed at 12:39:57.782 after 14m56s of first-use preparation. Warm continuation passed at 12:42:09.943 with the same owner, boot and session identities. Normal retirement and archive followed. The ordinary saved post-idle turn then failed: its primary attempt reported the closed `LINK_COUNT` module fence at `turn_ownership_verified` (3,711ms), and same-run recovery settled failed at 12:52:09 with identity/authority errors. Natural archive completed at 12:52:21. Numeric link counts and their direction are not yet proven. A narrower failure-only diagnostic and a bounded read-only probe are under review; neither establishes a fix for this live failure. Prior staging27/28 failures remain preserved. Local complete lifecycle passes and staging Codex/computer/HMR passes remain separate source-bound evidence. No session, quarantine, provider stop or migration history was reset. These live observations do not qualify this prerequisite by themselves. ## Risks - Waiting for the wrong run could hide a task race. The fixtures match task, agent, and replacement-run identity before proceeding. - A geometry helper could weaken touch coverage. This change preserves containment and gesture assertions and only makes their inputs one snapshot. - The six legacy browser cases passed in exact-head CI; their local PostgreSQL setup limitation remains recorded. No live Boat acceptance result follows from this test-only change. Documentation impact: no product command or behavior changes. This description records fixture scope and validation limits. ## Model Used OpenAI Codex (GPT-6 family) assisted with code, test execution, and review. The host does not expose the exact primary model ID or context window. The work used tools for source inspection and local execution. Prior architecture work used `gpt-6-astra`; an independent design review used `gpt-6.1-sol`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal issue id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
c205ab39f0 |
Merge origin/master and rebase feature migration sequence
Preserve upstream usage attribution and sandbox restore retry semantics alongside task workspace persistence. Regenerate task and physical-root migrations after upstream 0333 with replay-safe guards. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
db2f00947b |
fix(runner): include Claude ACP in the server package (#15912)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The server ships a copy of Paperclip Runner for native agent execution. > - The Claude ACP bridge is a runner dependency, but it was absent from the server package. > - The runner resolves this bridge dynamically, so the import-based packaging check missed it. > - A Claude session then failed before startup and showed a generic connection error. > - This pull request ships the qualified bridge and reports a missing runtime with repair steps. ## Linked Issues or Issue Description **What happened?** A server install with the vendored runner could not load `@agentclientprotocol/claude-agent-acp/package.json`. The runtime test suggested checking the model or provider connection. The task reported an interrupted runner session. **Expected behavior** The server package includes its qualified Claude ACP runtime. A missing bundled runtime reports an installation error and explains how to repair it. **Steps to reproduce** 1. Use a server install with the vendored runner and no Claude ACP bridge. 2. Select the Paperclip Runner ACP harness and the Claude provider. 3. Run the runtime test or open a session. 4. Observe a missing-module failure before Claude starts. **Paperclip version or commit** Reproduced on `e733bddc45`. This branch is based on current master. **Deployment mode** Self-hosted server built from source. Related: #15224 upgrades the Claude bridge. This PR fixes its omission from the server package and keeps the current qualified version. A future bridge upgrade must update the server pin too. ## What Changed - Add Claude ACP `0.73.0` to the server dependencies and bundled dependencies. - Pin the qualified Claude SDK and bridge dependencies in the npm bundle producer graph. Publish its native packages as consumer optional dependencies instead of shipping the producer executable. - Bind source-built vendored runners to the shared pnpm dependency store while retaining the server manifest as the provider authority. Keep published npm and self-rooted deployment roots unchanged. - Include dynamically resolved Claude and Codex bridges in the vendor dependency check. - Preserve the missing-package category through the sidecar, Rust transport, and native session executor. Stop automatic retries for this installation failure. - Show **Runtime missing** in the runtime test and task thread. Explain the reinstall or rebuild, server restart, and runtime test steps. - Add regression tests and document the repair procedure. ## Verification - Red/green comparison with the production sidecar: the same missing-package setup returns a raw require-stack error on the original code and an actionable, non-retryable missing-runtime error on this branch. - The rebuilt server's vendored Claude installation probe passes using the source-server authority selected by the launcher, without starting a provider or making a model request. - Focused server verification: 830 tests passed. Runner installation and sidecar error tests: 83 passed, with 6 platform-specific skips. Packaging tests: 28 passed. Rust transport tests: 7 passed. - Source/deployment authority tests: 3 passed. Red/green packaging tests include a Linux-produced Claude SDK bundle, actual npm pack, and closed executable admission on Linux x64 and macOS ARM64/x64. - Full UI suite: 8,113 tests passed. - `pnpm -r typecheck` and `pnpm build` passed on the rebased branch. - Full current-head CI passed, including all general and serialized test lanes, Runner Rust/Vitest verification, typecheck, build, and all eight e2e shards. Greptile reviewed `d6a2741834ff922956560f2f689382827bbaf9e0` at 5/5 with no unresolved threads. - Local full-suite limitation: an earlier parallel attempt hit Postgres startup failures and timeouts; the affected server tests and a database fixture passed in isolation. The subsequent serial `pnpm test:run` was stopped while still running after complete CI passed. Focused local tests, full UI tests, typecheck, and build completed as listed above. - Token gates and `git diff --check` passed. ## Risks - Bundling the Claude bridge increases the server package size. The producer graph retains the qualified SDK version and existing isolation patch. Native platform declarations move to the published server manifest, while the runtime keeps its exact version, declared-slot, no-follow, and executable digest checks. - The new failure category is permanent. Operators must repair the server installation before they retry the task. - Existing failed runs retain their recorded error. The clearer message applies to new runtime tests and runs. - No database schema or migration changes. ## Model Used OpenAI GPT-6 through Codex. The system reports the model family; the exact deployment ID and context-window size are not exposed in this session. Used reasoning, browser inspection, tool use, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
1b4c803816 |
fix(costs): preserve usage attribution and explain unavailable run costs (#15948)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The cost ledger records usage and controls agent budgets. > - Some runs lose the selected model or API billing category before their usage reaches the ledger. > - The dashboard reports missing costs but does not identify the affected runs or the available actions. > - This pull request preserves usage attribution and adds a compact view of cost gaps grouped by agent. > - Operators can inspect runs, open agent settings, and record saved costs without starting new agent work. ## Linked Issues or Issue Description Refs #15634, #11938, #15879. This follows the merged subscription reporting in #15634. The older proposal in #11938 addresses part of ACP model attribution. This change also rejects conflicting models across a turn, fixes credential-backed billing capture, and explains cost gaps in the existing dashboard. The Pi receipt recovery work in #15879 covers a separate adapter recovery path. This PR preserves unresolved receipts and does not recover provider usage or reprice past entries. **What happened?** ACP results use the requested model even when the runtime selects a model. Native and OpenCode API runs can report unknown billing despite a matching API credential. The dashboard shows notices without links to the affected runs. These gaps make API and subscription totals harder to interpret. **Expected behavior** Record the model selected by the runtime. Identify API billing only from a matching credential. Keep costs unresolved when reliable usage or pricing is absent. Show the affected agents and runs. Offer an accounting retry only when saved evidence is ready. **Steps to reproduce** 1. Run an ACP agent with its model left unset, or use a matching API credential with a native or OpenCode agent. 2. Inspect the recorded model and billing category for the run. 3. Open Costs Overview for a period with unpriced usage or terminal runs awaiting receipts. 4. Inspect the notice and try to find the affected run or the relevant agent settings. **Paperclip version or commit** Current master at `657e71584a0`. The patch is rebased on that commit. **Deployment mode** Self-hosted server and local development. The cost detail tests use a disposable PostgreSQL database. ## What Changed - Preserve the runtime-selected ACP model in checkpoints and final results. Conflicting models across one turn remain unpriced. - Share credential-backed API billing checks across native and OpenCode results. Keep unknown billing when the provider and credential do not match. - Add company-scoped cost detail and accounting retry endpoints. Board viewers can read details but cannot retry. Agents and foreign companies are denied. - Limit each detail category to 25 entries. Use one consistent database snapshot and sparse indexes matching the filters and sort order. Add the unknown-billing count to the existing summary query. - Replace the notice with an expandable count line. Group entries alphabetically by agent. Show the saved blob icon, link each name to configuration, and put run and retry links beside the date. - Preserve estimate and unavailable-price labels on Providers and Billers; keep retry confirmation after the resolved notice disappears. - Fetch details only while expanded. Retain the last loaded details after a refresh error. Retry saved accounting only; do not rerun the agent. - Add capture, database, authorization, UI, and Storybook coverage. Update the implementation contract and cost reporting guide. ## Verification - `pnpm -r typecheck` passed after rebase. - `pnpm build` and `pnpm build-storybook` passed after rebase. - `pnpm check:token-gates` and `git diff --check` passed. - 1,086 focused tests passed across 17 files. These cover ACP, native and OpenCode capture, PostgreSQL reports, route authorization, OpenAPI, Costs UI, and existing accounting regressions. - Desktop and mobile browser checks passed during UI review. They cover lazy expansion, alphabetical groups, loaded blob icons, configuration and run links, keyboard retry, and layout overflow. - `pnpm test:run` was attempted with command-scoped `init.defaultBranch=master` for Git fixtures. It was stopped after a native session recovery failure reproduced on unchanged base `657e71584a0` (`archives a damaged prior epoch before completing guarded same-task replacement with real runnerd`). An MCP upload case failed in the broad run but passed in isolation on both the base and PR branch. These local limitations do not affect the 1,086 passing focused tests. - Current-head CI is green, including all unit test lanes, serialized server suites, runner checks, typecheck, build, policy, and eight browser shards. Apex re-review is 5/5 on `56de60effdb`; all four original findings are addressed and every review thread is resolved. - PostgreSQL applies migration `0333` in the integration fixtures. EXPLAIN ANALYZE on the production detail statements uses the new sparse indexes behind 5,000 healthy records in each table. - Ten provider/biller card cases cover missing and estimated prices; the retry test covers row removal and notice unmount. A live Storybook browser check verifies the confirmation and updated count. - Run `pnpm storybook`. Open **Costs / Subscriptions / Cost Data Quality Details** to inspect the production page with representative fixtures. ## Risks - Model and billing metadata change for future runs. Estimates still require complete usage and a supported price. No past cost amount is rewritten. - A missing final receipt remains unresolved for budget checks. The display changes after fifteen minutes; that time limit never authorizes settlement. - Accounting retry is a write action. It retains the existing company checks, receipt fences, idempotency, and audit logging. - Detail reads are bounded and run only after expansion. They poll every 30 seconds while open. No provider probe or new summary query is added. - Migration `0333` adds two partial indexes; it changes no stored cost amounts or table columns. Index creation scans the existing tables and may briefly block writes during deployment. No lockfile or workflow change is included. ## Model Used OpenAI Codex, based on GPT-6, with repository inspection, code editing, browser checks, and test execution. The exact serving model ID, reasoning setting, and context window were not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
657e71584a |
fix(daytona): record provider state when lease creation hits its deadline (#15946)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents can run in Daytona sandboxes. The Daytona plugin acquires a
lease before each run.
> - Some acquisitions time out. The report then says
`environmentAcquisitionPhase=create` and that the whole 300 s budget was
used.
> - The Daytona SDK create timeout is equal to the acquisition budget.
The outer timer always fires first. Thus the report never shows the
state that Daytona left the sandbox in.
> - Without that state, we cannot choose a fix. A stuck snapshot pull, a
slow start, and a missing allocation need different fixes.
> - This pull request adds one bounded, read-only provider lookup at the
create deadline. It records a fixed-enum provider state in the existing
acquisition diagnostic.
> - The benefit is that the next timeout identifies where the provider
stopped. Retry, budget, and cleanup behavior do not change.
## Linked Issues or Issue Description
Refs #15736 (bounded Daytona acquisition failure observations). This
pull request extends those observations.
**What happened?**
A Daytona lease acquisition timed out with `Daytona lease acquisition
timed out; allocation cleanup is pending`. The run-failure report gave
`environmentAcquisitionPhase=create`,
`environmentAcquisitionElapsedMs=300004`, and
`environmentAcquisitionBudgetMs=300000`. The report did not show the
Daytona sandbox state. The SDK create timeout is equal to the outer
budget, so the outer timer always fires first. The SDK never reports the
stuck state.
**Expected behavior**
When creation is still pending at the deadline, the report should show
where the provider left the allocation. Examples are `pulling_snapshot`,
`starting`, `error`, or `absent`.
**Steps to reproduce**
1. Configure a Daytona environment.
2. Start a run when Daytona does not finish create and start within the
acquisition budget.
3. Inspect the setup-failure report. Only the phase, the elapsed time,
and the budget are present.
**Paperclip version or commit**
`master` at
|
||
|
|
ba04968b10 |
fix(server): report failing workspace jobs as 422 instead of 500 (#15941)
A non-zero workspace job exit is the operator job's own result. Map it to 422 workspace_job_failed with the exit code instead of a 500 that the error handler reports to Sentry as a server crash. A job without a command is also 422. Other workspace command failures are unchanged. Co-Authored-By: Amy (Opus) <claude@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Devin FoleyandPaperclip
|
d49cbf4c4f |
Permit idle sleep with consumed retries and passive worktrees (#15907)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Idle sleep must wait for accepted work and durable cleanup.
> - The retry dispatcher keeps its due date after it promotes a retry.
> - Finished retries and passive local directories can therefore look
like pending work.
> - This pull request requires completion proof and permits known
passive directory records.
> - It retains the records and keeps real work and unknown state awake.
## Linked Issues or Issue Description
Related: #15897 and #15851. The open runtime-service work in #13413
retains its separate service and ingress guarantees. This patch does not
change those checks.
**What happened?**
Idle sleep treats a consumed retry date as pending work. It also blocks
on every retained execution workspace, including a local directory
without runtime activity.
**Steps to reproduce**
1. Promote a scheduled retry, then finish its run without accounting
debt. Retain its scheduled retry date.
2. Retain a local primary directory or Git worktree without runtime
configuration, commands, or cleanup due.
3. Request an owned idle safety report. The report returns `present` for
this history.
**Paperclip version or commit**
Reproduced on master at
|
||
|
Devin FoleyandPaperclip
|
0ba88d2195 |
fix(opencode-local): preserve deadlines in Daytona process sessions (#15905)
## Thinking Path > - Paperclip manages work done by AI agents. > - The OpenCode adapter can run its CLI in a provider sandbox. > - A long CLI run must remain observable until it exits or the host stops it. > - The legacy main command does not request a provider process session. A launch without an active run step uses one synchronous command request. > - That request can lose its response before the resolved execution budget ends. > - This PR requests the existing asynchronous session path for the main Daytona OpenCode command. It also carries the resolved command deadline through the host and provider contracts. The host still applies the effective capability gate. > - The change removes that synchronous request from eligible sandbox runs. ## Linked Issues or Issue Description Related reporting fix: #15789. A search of open and closed GitHub issues and PRs found no duplicate transport fix. **What happened?** A hosted Linux legacy OpenCode run retained a gateway-timeout response about one hour after CLI startup. It had a four-hour resolved budget and recent agent output. The earlier reporting fix correctly kept the configured policy separate from the timeout message. The original response owner and specific timer were not retained, so this PR does not claim to prove that provider cause. **Expected behavior** A Daytona target that supports persistent process sessions should dispatch the long agent command asynchronously. Paperclip should observe its output and exit through the existing session path. A single command HTTP request should not need to stay open for the full agent run. **Steps to reproduce** 1. Use legacy OpenCode with a Daytona sandbox target and a four-hour execution budget. 2. Launch outside an active measured step. The current main CLI call does not request a persistent process session. 3. In the SDK fixture, make the synchronous command request return a provider timeout before the budget ends. Keep the asynchronous session healthy for more than one hour. 4. The regression tests require session dispatch, incremental output, one command dispatch, a real exit result, and session cleanup. 5. Set a shorter command deadline and stall dispatch, a status read or a log snapshot. The command must return a timeout at its own deadline. Healthy output must not extend that deadline. **Paperclip version or commit** Observed source: `e733bddc45e00efb1819fffa8677f7b0d07802c5`. PR base: `4b59e2a2ecc67e98b336b8eb4c09959681672d54`. **Deployment mode** Hosted Linux server, legacy OpenCode adapter, Daytona sandbox. The host gate retains the old path when a provider cannot keep process sessions. ## What Changed - Add an optional sandbox session choice to the shared process helper. Leave the choice absent for existing callers. - Request that session only at the main Daytona OpenCode CLI invocation. Keep probes, preparation and other providers on their existing paths. - Carry a separate optional command deadline through the shared runner, host service and plugin SDK. Existing session callers can omit it. Zero leaves the command deadline disabled. - Bound dispatch, live output and recovery observations with one command timer. Clear it on settlement. Fence late stream callbacks. Remove deadline listeners after each read and clear interrupted polling waits. Preserve a known exit code if only the final logs time out. A timeout result does not claim that the process stopped. - Add adapter and process-helper regression coverage. Test the existing Daytona session for a virtual run beyond one hour, without command replay, with release cleanup. - Document transport selection and the limits of the timeout evidence. ## Verification - `pnpm -r typecheck` passed on Node 25.6.1 and pnpm 9.15.4 with the command deadline changes. - `pnpm build` passed with the updated contracts. The separately installed Daytona package also passes its own typecheck and build against the pinned SDK 0.203.0 and TypeScript 5.9.3. - Five focused process, adapter, host and provider suites passed all 349 tests. They cover a run beyond one hour, five stalled observation phases, healthy-stream deadlines, zero/absent budgets, late callbacks, session release, SSH and capability fallback. - The real environment-runtime forwarding tests passed: 19 selected tests, including the separate command budget and forced session flag in the worker RPC. - An earlier full local `pnpm test:run` began before a clean build and overlapped the correction. That mixed-version run was stopped and is not passing evidence. After the build, the two previously uncollected suites pass (1,659 tests). Two unchanged workspace fixtures fail with this machine's Git default branch `main`; both pass with a test-process-only `master` override. One unchanged native recovery fixture fails on both this branch and an independently installed and built checkout of base `4b59e2a2ecc67e98b336b8eb4c09959681672d54`, with the same terminal-result mismatch. It is not introduced by this change. The full local suite is not claimed green. These limitations do not replace fresh hosted CI. - Final head `8e96638cdef8ca6e91f7a42a52c83a075180c137` passes all hosted checks: 54 passed and 2 expected Storybook skips. Greptile reviewed this exact head at 18:12 UTC and scored it 5/5. Both actionable findings are fixed and resolved. GitHub reports no conflicts and a clean merge state. Earlier-head results are not used as final-head proof. - The standalone provider test command passes 373 tests, with 14 expected live-probe skips. - SDK tests use a controlled virtual clock. No live provider experiment, task replay, manual deployment, or public PR merge was performed. ## Risks The main legacy OpenCode command uses Daytona's existing session route when the effective capability gate permits it. A quiet log stream can use the existing status and snapshot fallback. Those observations add provider reads. The new optional command budget is separate from each read's timeout and returns a timeout without the host RPC's extra 30-second allowance. Existing session callers that omit it keep their previous lifetime. Callback bridge control commands retain their explicit session bypass. The capability gate retains the previous path for unsupported targets. Other providers, local and SSH execution keep their existing path. Retry policy, Stop acknowledgement and lease ownership rules stay the same. The SDK exposes no cancellation handle for an abandoned stream; existing Stop and lease teardown still own command termination. This fix does not establish the original gateway owner or prove recovery in a live environment. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository tools and code execution. The exact backend identifier and context window size are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (changed paths; the independently reproduced local baseline failure is documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
3d3b8d4371 |
fix(external-objects): distinguish hidden GitHub repos from genuinely-missing objects (#15906)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Issues reference external objects such as GitHub pull requests. The instance resolves their status and shows it in the UI. > - GitHub returns 404 both when a pull request or issue does not exist and when the repository exists but the configured credentials cannot see it. GitHub hides private repos behind a 404 on purpose. > - The resolver treated every 404 as a terminal not-found verdict. It froze references to inaccessible repos forever, and marked them "Not found (Stale)". > - This pull request probes the repository metadata to tell the two cases apart and keep frozen objects re-checkable. > - The benefit is accurate statuses instead of permanent "Not found" labels for repos the instance cannot see. ## Linked Issues or Issue Description No public GitHub issue exists for this report. **What happened?** - The issue properties panel and markdown decoration showed many GitHub pull request and issue references as "Not found (Stale)" even though the objects exist. - GitHub returns 404 for objects the configured credentials cannot access, not only for objects that are truly gone. - The resolver treated every 404 as a terminal not-found verdict, so those references were frozen and showed a misleading "stale" liveness. - Detected-but-unresolved GitHub objects were labeled "Not yet resolved" in inline markdown, which did not match the pill label "Not yet refreshed". **Expected behavior** - A reference to a repo the credentials cannot see should surface as authorization-required, not as a permanent "Not found". - A truly missing object should still be "Not found". - A terminal object (merged, closed, or confirmed not-found) should not be labeled "Stale", because it is never refreshed again. - Inline markdown statuses should match the pill statuses. **Steps to reproduce** 1. Configure GitHub credentials with no access to a private repo. 2. Add a comment or issue description that links to a pull request in that repo. 3. Wait for the external object resolver to run. 4. Observe the reference labeled "Not found (Stale)". **Paperclip version or commit** - `master` at `e733bdd` (before this change). **Deployment mode** - Self-hosted server. ## What Changed - Added a repository visibility probe to the GitHub external object provider. - A 404 for an object now probes the repository metadata with the same credentials: - Repository visible: the object genuinely is gone. Keep the not-found snapshot. - Repository hidden: return an authorization-required result instead of a terminal not-found verdict. - Probe failed (network, rate limit, 5xx): return an unreachable result and retry later. - A rate-limited repository probe (403 with `x-ratelimit-remaining: 0`) is treated as uncertain, not as missing access, so it retries instead of showing a misleading "Requires auth". - Re-check previously frozen GitHub not-found objects on their normal schedule so they self-heal once credentials or repo visibility improve. A frozen object is only re-checked until a probe confirms the repo is visible; once confirmed deleted, it is terminal and no longer polled. - Cleared any terminal verdict when a refresh fails, so a failed refresh does not freeze an object. - Kept terminal objects from displaying the "stale" liveness label. - Matched the inline markdown status label to the pill status label, so detected-but-unresolved GitHub objects say "Not yet refreshed" instead of "Not yet resolved". - Skipped the liveness suffix in markdown links and status icons when it duplicates the status label (e.g. no more "Requires auth (Requires auth)"). - Cached repository visibility probes per company+repo + credential fingerprint, so many objects in the same repo share one probe per refresh cycle instead of each issuing its own — and a verdict obtained under one token is never reused by a later refresh running under a different credential. ## Verification - Added provider tests for visible, hidden, failed, and rate-limited repository probes, per-company/per-repo probe caching, and probe-verified not-found snapshots. - Added service tests for terminal-object liveness, for re-checking frozen GitHub not-found objects, and for ensuring probe-verified not-found objects are never re-queued. - Added UI tests asserting unresolved GitHub objects render as "Not yet refreshed" (not "Not yet resolved") and that failed-first-refresh labels do not repeat the liveness text. - Ran `npx vitest run src/__tests__/external-objects-service.test.ts` in `server/`: 34 passed (incl. provider + service suites). - Ran the external-object markdown/pill/icon suites in `ui/`: 97 passed. - `ci / Typecheck + Release Registry` and all other CI gates pass on this head. Local `tsc` could not run to completion in the execution sandbox (the sandbox memory cap killed it), so CI typecheck is the authoritative gate. ## Risks - Low risk. The change is confined to the GitHub external object provider and the external object service. - A repo that genuinely does not exist and a private repo both return 404 for the probe. Both now surface authorization-required or a retry instead of an immediate not-found freeze, which stays re-checkable and self-heals. - Re-checking frozen not-found objects adds at most one repository probe per affected object until the probe produces a definitive verdict, after which the object is terminal and no longer polled. Probe results are cached per company+repo+credential, so a credential rotation always re-probes instead of inheriting a stale verdict. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Provider: DeepSeek via OpenRouter, model ID `openrouter/~deepseek/deepseek-v4-flash-latest`, running inside the Paperclip opencode adapter (`opencode_local`). - The model produced and assisted with the reasoning and code changes; the execution workspace, test suite, and git/PR steps above were run by the same agent run. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details — the branch name is assigned by the managed execution workspace and the harness requires keeping it - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green on the current head (all `ci / *`, Greptile, Socket, and review checks pass) - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
4b59e2a2ec |
fix(server): allow idle sleep with saved AI auth and verified lease history (#15897)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Automatic idle sleep must preserve accepted work, cleanup and recovery. > - Saved AI login connections are configuration without independent workers. > - Failed ephemeral leases can remain after their resources have stopped. > - Detached subscription observations and quota refreshes can outlive a response. > - This pull request distinguishes inert configuration and verified history from unfinished work. > - The benefit is safe idle eligibility while keeping credentials and audit history. ## Linked Issues or Issue Description **What happened?** An active saved AI login blocks idle sleep after its login and agent runs finish. A failed ephemeral lease also blocks after successful cleanup and provider termination. The connection guard currently hides detached quota and subscription work that can outlive the request that started it. **Expected behavior** An idle instance can sleep with saved AI credentials and verified failed-lease history. Accepted refresh work, recovery journals, unfinished login sessions, pending cleanup and run accounting must still block. **Steps to reproduce** 1. Save an AI connection with purpose `ai`, transport `runtime_auth` and status `active`. 2. Finish login and all agent work, then acquire an owned idle drain. 3. Read the idle safety report. The saved connection alone reports background work. 4. Repeat with a failed ephemeral lease whose exact provider termination and finished run are recorded. Its history also reports background work. **Paperclip version or commit** Based on `0bbd586a2f6`, after updates from master. **Deployment mode** Single-process managed or self-hosted server using the owned idle-drain protocol. Related work: #15522, #15761 and #15851. Searched open and closed pull requests for related idle-sleep changes. The open accounting recovery work in #15879 is separate; this change does not settle or discard accounting debt. ## What Changed - Exempt active saved AI runtime-auth configuration from the connection sleep check. Other connection types and unknown states keep their existing checks. - Track detached subscription discovery and provider observations until their actual promises settle. - Track underlying quota work through an HTTP timeout, including an OAuth exchange and the save of its replacement credential. - Inspect the credential-recovery directory without reading credential bytes. Journals, probes, temporary files, unexpected entries and read failures keep sleep blocked. Empty company directories left by successful recovery permit sleep. - Exempt a failed ephemeral lease only after successful cleanup, a release timestamp, an exact company/run/lease/resource termination receipt, and a finished run without retry or accounting debt. Export and cleanup intents still block. - Use the same failed-lease proof when checking finished login-session cleanup. - Add PostgreSQL and filesystem regression tests and document the behavior. ## Verification - Four focused suites passed: 186 tests covering durable guard decisions, local recovery, detached subscription work and quota timeouts. - `pnpm build` passed. - `pnpm -r typecheck` passed. - `pnpm --filter @paperclipai/server build` and a final server TypeScript check passed after the directory-scanner cleanup. - The final directory-scan recheck passed all 18 tests. - The original-base `pnpm test:run` attempt found three local failures. It was stopped before completion to update the branch after two master changes. All three also reproduce on unchanged master `e733bddc45e`: two Git default-branch fixtures in `workspace-runtime.test.ts` and one native runner recovery fixture in `native-session-resume.test.ts`. - Both Git fixtures pass with a command-scoped `init.defaultBranch=master` setting. The native fixture fails with the same result on both branches. No global Git setting was changed. - All CI gates passed on the final head `ff63497d9bef40f5f00567bc729fb17536cfe136`: [complete CI run](https://github.com/paperclipai/paperclip/actions/runs/38156312722). The only skipped checks are the two path-gated Storybook jobs. - Greptile Apex scored the final head 5/5 with no actionable findings and no unresolved review threads. The branch has no merge conflicts with master. - After the master update, `pnpm build` and `pnpm -r typecheck` passed again. All four focused suites passed again: 186 tests. - Staging deployment and live sleep/wake verification are pending the Core merge and the verified private Cloud build. No production rollout has been started. ## Risks - The failed-lease exception is deliberately narrow. Older records without exact termination evidence or a complete run receipt continue to block. - Pending OAuth recovery blocks sleep until it is recovered. No credential, history, cleanup obligation or accounting receipt is deleted. - The existing owned drain, ingress fence, local-work generation checks, plugin drain and backup checkpoint remain required. - No schema migration or provider mutation is introduced. Roll back to the previous application release to restore the conservative guard behavior. ## Model Used OpenAI GPT-6 (Codex), with repository inspection, code editing, test execution and read-only provider analysis. No additional model version or context-window size was supplied by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id or instance-derived details - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no unresolved comments - [x] I will address all reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
0bbd586a2f |
fix(server): wait for workspace restore before shutdown lease release (#15857)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Server shutdown must stop agent execution and preserve its files. > - Legacy adapters export workspace and instruction changes during settlement. > - Shutdown released their sandbox leases before this export finished. > - The provider closed command access, so the later Git export failed. > - This change stops an owned adapter through its cancellation control and waits for settlement before release. > - A bounded timeout retains the exact sandbox source behind the existing repair gate. ## Linked Issues or Issue Description **What happened?** A server restart interrupted a remote legacy run. Shutdown began releasing its sandbox lease while the adapter was still stopping its provider and restoring files. The sandbox provider closed command admission. The adapter's later Git export failed with `workspaceRestoreFailure: restore_failed`, leaving the task held for workspace repair. **Expected behavior** Shutdown keeps the exact lease available through provider stop, instruction collection, and workspace restore. Cleanup follows adapter settlement. A restore that exceeds the bounded deadline keeps its exact source for repair before later execution. **Steps to reproduce** 1. Start an owned legacy adapter with a sandbox workspace. 2. Send SIGTERM to the server while the adapter is active. 3. Delay adapter settlement until after the shutdown terminal write. 4. On the base branch, shutdown releases the lease without cancelling or joining the adapter. Its later export can encounter closed provider admission. Related: #14311 defines the provider-stop boundary for file collection. #15042 closes run admission during shutdown. This fix orders sandbox release after the adapter's existing file settlement. ## What Changed - Cancel each owned legacy adapter and join all affected runs within one shared 30-second window. - Commit a durable fence on the exact sandbox before exposing interruption. Other cleanup workers defer while export is pending. - Permit the exact aborted owner's stop-only request so shutdown does not block provider termination. - Reload current execution evidence after settlement. Release leases even when the adapter's ordinary terminal write wins the shutdown race. - Claim the current fence before recording timeout retention. An old snapshot cannot create a repair hold after successful settlement. - Recover fenced sources after controller loss, including released stop receipts. Block reuse or destruction of the physical allocation while file recovery is required, including direct resume and destroy calls with historical lease snapshots. Convert failed ordinary cleanup into stop-only recovery after its existing attempt settles. - Allow 90 seconds for shutdown in shipped Compose services. Preserve native runner restart ownership and unregistered process behavior. - Add database regressions for restore order, timeout, concurrent cancellation, another cleanup worker, controller loss, stale snapshots, stop-only settlement, and completion races. Document the shutdown order. ## Verification - The original ordering regression failed on the base implementation. - Latest recovery and environment runtime suites: 190 tests passed. The other four focused suites passed 136 tests, covering run control, shutdown, file repair, server startup, and environment runtime. - Server typecheck passed. Full `pnpm -r typecheck` and `pnpm build` passed on the latest head. - `pnpm test:run` was run locally. Its general-server group passed 18,559 tests and failed seven, with one unhandled error. The long run started before the review fixes, then loaded newer regression cases against older transformed modules. The current focused suites pass in a fresh process. The damaged-prior-epoch native-session-resume failure also reproduces on unchanged base commit |
||
|
Devin FoleyandPaperclip
|
5847f047c7 |
fix(connections): retire requests for already available tools (#15856)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Connection cards let users provide the access an agent needs. > - Users can complete setup in Apps while an earlier card stays pending. > - A stale card can then cancel recovery despite valid existing access. > - This change verifies existing access and expires the obsolete request. > - It records a system outcome and sends one durable continuation. > - Approval, identity, permission, and workspace recovery gates remain in place. ## Linked Issues or Issue Description **What happened?** An ordinary tool connection request stayed pending after the addressed user connected the service in Apps. A later conversation retry was cancelled with `issue_waiting_for_response`. **Expected behavior** A request expires when the same user and requesting agent already have usable access. The agent gets a durable tool refresh. The system records no new human approval. **Steps to reproduce** 1. Have an agent request a tool connection on an active task. 2. Connect the service through Apps with the addressed user's existing grant, agent install, and allowed tools. 3. Let a conversation run stop and retry. The pending card still blocks the retry on the base branch. Related: #12345 introduced the connection request flow. This fix uses its existing access checks and delivery queue. ## What Changed - Verify company, task ownership, user identity, membership, health, grants, installs, and task tool policy before expiring an ordinary request. - Preserve AI account, channel, additional permission, and upstream provider requests on their existing paths. - Save the system outcome, audit event, and continuation in one transaction. Notify an idle delivery worker when a request commits. - Recheck access and other questions or approvals before delivery, retry, final dispatch, and native projection. Refresh system observations for an authorized replacement account. - Rearm delivery when access changes before provider work starts. Verify the host's admission receipt against execution evidence before allowing retry without a repair incident. - Require the current delivery claim when marking an attempt complete. An old completion cannot overwrite a rearmed continuation. - Add embedded-Postgres regressions for permissions, policy, concurrency, restart, replacement accounts, both dispatch gates, and the delivery race. Keep genuine workspace repair holds blocking. ## Verification - Latest-head connection, wake-context, dispatch, and legacy recovery suites: 124 tests passed. The connection suite includes 44 real-Postgres cases. - Server typecheck passed. Full `pnpm -r typecheck` and `pnpm build` passed on the latest head. - `pnpm test:run` was run locally. Its general-server group passed 18,583 tests and failed 20. That long run started before the review fixes, then loaded new test cases against modules transformed before those fixes. The affected latest-head suites pass in a fresh process. - One full-run failure also reproduces on unchanged base commit |
||
|
DottaandPaperclip
|
abcd017898 |
Reject concealed managed publications in anonymous roots
Check the readable repository namespace independently of historical aliases, including existing reservations. Preserve unavailable unrelated folders and allow asynchronous chat edit verification its established polling budget. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
fa31e4ade7 |
Preserve native recovery transport and legacy workspace bindings
Fence host roots only for local and sandbox execution targets. Accept verified run-ID bindings for older native tasks and avoid unrelated historical path failures without weakening pending-acquisition guards. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d66a2de539 |
Fence repository acquisition by durable physical root
Preserve empty-inventory admission and immutable recovery. Reserve managed or anonymous roots before acquisition and fence legacy recovery using its verified host binding. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0e0557f123 |
Fix repository trust and settled workspace recovery guards
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
fed4802b72 |
Merge origin/master and integrate environment selection
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e733bddc45 |
perf(chat): stop polling provider action and session sync queues (#15822)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat stores pending provider actions in durable database rows. > - Three action queues still scan every second when empty. > - These scans keep idle databases active. > - This pull request wakes those queues after a commit or at their next deadline. > - Empty queues stop querying while existing recovery rules remain in place. ## Linked Issues or Issue Description Refs #15808, #15802, #15625. Related sleep work: #15750, #15757. **What existing behavior does this improve?** Provider messages and notices, accepted GitHub webhook receipts, and Slack session status synchronization. **Current behavior** Chat maintenance scans these three queues every second. It also rechecks paused Slack sync rows every 30 seconds. **Proposed behavior** Reuse the transaction-aware action workers and existing chat_actions rows. Wake after commits. Restore unfinished work on startup. Use stored retry and stale-claim deadlines. Stop scanning when empty. Park paused Slack sync rows until an endpoint change. ## What Changed - Add three topics to the existing action worker registration. No tables, migrations, triggers, or external services. - Signal producers, claims, retry updates, terminal settlements, and operator resolution inside their owning transaction. Provider I/O remains outside transactions. - Share pending-state and deadline predicates between dispatch and scheduling. Exclude terminal history and uncertain provider sends. - Preserve GitHub receipt attempt fences and provider-send quarantine. Preserve Slack's 30-minute refresh while a session is processing. - Remove these workers from periodic app maintenance. Update the sleep inventory and add regression coverage. ## Verification - Passed: 13 focused tests across scheduler, app wiring, and real PostgreSQL integration. Covers actual commit wakes with deferred fast paths disabled, GitHub retries and startup recovery, stale-send quarantine, Slack publication-to-sync delivery, paused deadlines, and idle queues. - Passed: pnpm typecheck. - Passed: full chat integration and related coordinator suites, **1,274 tests across five files**. - Passed: pnpm build. - Greptile: **5/5**, no actionable findings, completed on `dc284cd275e5f6df624d60b27954421f2bcdbee2`. - Hosted CI: **54 checks passed**, two intentionally skipped Storybook checks, no failures or pending checks on `dc284cd275e5f6df624d60b27954421f2bcdbee2`. Provider responses are simulated. PostgreSQL, transactions, service dispatch, and scheduling are real. No live Slack or GitHub App was used. ## Risks - Writers must emit the existing process-local transaction signal. Independent database clients and direct SQL still need explicit integration or startup recovery. - Accepted GitHub receipts are covered. Finding callbacks GitHub never delivered still uses the separate provider recovery lane. - Other chat maintenance and persistent transports still run. This does not establish whole-instance sleep. - Existing claim, revision, credential, and authorization checks remain authoritative during concurrent foreground and scheduled dispatch. ## Model Used OpenAI Codex (GPT-6), with repository inspection, code execution, and automated tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
30e338fba6 |
refactor(heartbeat): extract environment selection (#15825)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Heartbeat execution selects an environment before it prepares a workspace and starts an agent. > - Workspace preparation now has its own module, but its selection policy still occupies the executor. > - Environment defaults, recovery bindings, chat isolation, and workspace contention must retain their order. > - This pull request moves that selection phase behind one typed function. > - The executor keeps dispatch gates and lease cleanup. Tests cover the public path and the new boundary. ## Linked Issues or Issue Description Refs #15807. This continues the heartbeat extraction series after workspace preparation. **Current behavior** The 5,599-line heartbeat service contains a 301-line environment selection phase inside run execution. **Proposed behavior** Move that phase to `heartbeat/environment-selection.ts`. Keep the same policy, statement order, errors, and public exports. **Reason and benefit** Readers can find selection policy in one module and follow the executor's phases without reading each policy branch. ## What Changed - Add `selectHeartbeatEnvironment` with typed run, task, policy, configuration, and service inputs. Return the values needed by configuration and workspace preparation. - Preserve native recovery bindings, environment precedence, managed sandbox enforcement, Kubernetes provisioning, native chat isolation, and shared workspace contention. - Reduce `heartbeat.ts` from 5,599 lines / 234,599 bytes to 5,314 lines / 220,784 bytes. The new module is 402 lines / 18,215 bytes. - Add six public-path characterization cases before moving production code and six boundary cases. Repair plugin-test teardown to drain active execution and call the database cleanup helper. - Assert the selected local environment in the browser fixture's real worker and document the new boundary. ## Verification - Before the move: 71 tests passed, including all six new public-path cases. Three plugin tests skipped when the host could not start their database probe. - After the move: all 112 focused tests passed across eight files, with no skips. These cover environment policy, local and plugin execution, contention, recovery bindings, workspace preparation, chat isolation, and low-trust directories. - Passed `pnpm -r typecheck` and `pnpm build`. - A normalized TypeScript AST comparison confirms the 301 moved lines and remaining executor statements are unchanged, apart from the new call and imports. - Browser acceptance: the task UI recovered automatically and the worker verified its selected local environment, preserved uncommitted files, and completed the task. The persistent-failure case passed in isolation: three total runs, then a blocked task that remained blocked after reload. - The first persistent-failure browser attempt missed the retry banner within five seconds, despite a persisted scheduled retry. Its unchanged isolated rerun passed. The trace is retained; the intermittent UI failure is not yet attributed to a root cause. No assertion or timeout was relaxed. - Greptile reviewed `20953dd164decb565bc8d85081aad31a23fd33e5`: 5/5, no actionable findings. - Hosted CI passed on `20953dd164decb565bc8d85081aad31a23fd33e5`: 52 successful checks and two intentionally skipped Storybook checks, with no pending or failing checks. This includes all general and serialized test groups, Runner verification, eight browser shards, typecheck, and build. - The duplicate local `pnpm test:run` was interrupted after hosted CI completed. It is not reported as a local full-suite pass. Local completed checks are the 112 focused tests, typecheck, build, and browser cases described above. ## Risks - This selection phase enforces execution restrictions. A wrong input or import could select the wrong environment or change retry behavior. Public-path tests cover refusal, redirect, contention, and dispatch; boundary tests cover recovery and isolation. - The helper can provision a managed environment row and add concurrency notes to the supplied task context. It acquires no execution lease. The executor retains its catch/finally boundaries and resource ownership. - Kubernetes selection is tested with real database state and bootstrap settings. These checks do not claim execution against a live Kubernetes cluster or hosted sandbox provider. - No schema, migration, API contract, or intended behavior change. ## Model Used OpenAI Codex (GPT-6), with repository inspection, code editing, compiler checks, database integration tests, and browser automation. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
de9ab8e7e0 |
perf(runner): compile the TypeScript surface once per build (#15853)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Pull requests to this repository run a large CI matrix. The slowest check sets the time a contributor waits for a green run. > - After #15849 hid the verifier's downloads behind package staging, the `Build` lane is still the slowest check in five of the eight fully green PR runs created after that merge (535s to 581s; the top server shard is 529s to 560s in the other three). `Build` is about 100s of setup, 160s to 176s of `pnpm build`, and 265s to 312s of the public install verifier. > - Inside `pnpm build`, every package after `@paperclipai/shared` waits on the Runner: its `build` script takes 76s to 80s, and for 65s of that it is the only job running. The chain runs `build:typescript` three times: once directly, then again inside `check:semantic-contracts` and `check:runner-workflow-traceability`, which rebuild first so they work standalone. The server's `prepare:runner-vendor` then runs the whole Runner build a second time, so the job pays for four redundant TypeScript passes. > - Each pass is two `tsc` projects, the eval-kernel rebuild, and the schema checks, 7s to 8s on the CI runner. The dist they rebuild is the dist the first pass just wrote; nothing between the passes changes it. > - This pull request calls the two check scripts directly after the chain's own `build:typescript`, the way `generate-replay-goldens.mjs --check` already does in the same line. The standalone `check:*` scripts keep their rebuild. > - The benefit is about 30s off the `Build` job's critical path, about 15s off the Typecheck job and every Docker or release build that runs the server build, with no change to what is checked. ## Linked Issues or Issue Description **Describe the enhancement** The `Build` PR check is the slowest check in most green PR runs. Its `pnpm build` step compiles the Runner's TypeScript surface four times: three passes inside one Runner `build`, and a second full Runner `build` from the server's `prepare:runner-vendor`. Only the first pass produces a dist that the later steps read; the other three rewrite the same output. **Why is this needed** The slowest check decides the wall-clock time of every PR run. In the eight fully green PR runs created after 2026-10-11 04:10Z, `Build` topped five of them at 535s to 581s. The Runner build sits alone on the `pnpm build` critical path for about 65s, and the server build repeats it. From the job logs of four of those runs, the second and third `build:typescript` passes cost 7s to 8s each, in both Runner builds, so about 30s per job. **Proposed solution** In the Runner's `build` script, replace `pnpm run check:semantic-contracts && pnpm run check:runner-workflow-traceability` with `node scripts/generate-semantic-contracts.mjs --check && node scripts/check-runner-workflow-traceability.mjs`. Both scripts read `dist/`, which `build:typescript` wrote earlier in the same chain, and the steps between them are `--check` runs that write nothing to `dist/`. Add a guard test so the chain compiles TypeScript exactly once and both checks still run after it. **Alternatives considered** - Skip the server's `prepare:runner-vendor` rebuild when `pnpm -r build` already built the Runner. That rebuild is what makes `pnpm --filter @paperclipai/server build` correct standalone (the Dockerfile uses it), and the server's build-script test pins the command. A freshness check would be a larger semantic change; this PR shrinks the rebuild instead of removing it, and the next cycle can measure what remains. - Make `tsc` incremental across the passes. It would still re-run the schema checks and the eval-kernel build each time, and would add build-info files to the package. - Change `check:semantic-contracts` and `check:runner-workflow-traceability` themselves to stop rebuilding. They are documented standalone commands (`pnpm --dir packages/paperclip-runner check:semantic-contracts` in the capability docs) and must work from a stale dist; unchanged here. ## What Changed - `packages/paperclip-runner/package.json`: the `build` script runs `node scripts/generate-semantic-contracts.mjs --check` and `node scripts/check-runner-workflow-traceability.mjs` directly, after its own `build:typescript`, instead of the `check:semantic-contracts` and `check:runner-workflow-traceability` wrappers that rebuild first. Every other step, its order, and the standalone `check:*` scripts are unchanged. - `server/src/__tests__/server-package-build-script.test.ts`: a new test in the suite that already pins the server's Runner vendoring reads the Runner's `build` script, resolves every `pnpm run` step, and asserts that exactly one step compiles TypeScript (directly or through a wrapper), that both check scripts are in the chain, and that both come after `build:typescript`. It fails against the previous script with `expected [ 'build:typescript', …(2) ] to deeply equal [ 'build:typescript' ]`. ## Verification - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/server-package-build-script.test.ts`: 9 pass (1 new). With the `package.json` change stashed, the new test fails as described above and the other 8 pass. - In `packages/paperclip-runner`, the new chain without the cargo step (`check:protocol-manifest`, `build:typescript`, `check:capability-contract`, `check:capability-inventory`, `check:protocol-coverage`, `generate-semantic-contracts.mjs --check`, `check-runner-workflow-traceability.mjs`, `generate-replay-goldens.mjs --check`, `generate-semantic-action-catalog.mjs --check`) exits 0: `Runner stress traceability passed: 44 findings, 12 workflows, 1 explicit exclusion.` and `Replay golden snapshots and parity summaries are current.` Cargo is not installed on the host that produced this PR, so `build:binary` was not run locally; it is unchanged and runs in this PR's `Build` job. - `pnpm run build:typescript` alone takes 5.8s to 5.9s on this host, so each removed pass is in the range the CI logs show. - This PR's own `Build` job runs `pnpm build` from the branch checkout, so its log shows the new chain in both Runner builds. Expected on that job: `pnpm build` about 130s to 146s (was 160s to 176s on the four measured runs), `Build` about 505s to 551s (was 535s to 581s). **Measured on the four post-#15849 `master` runs that motivated this PR** (`Build` job logs): | Run | `pnpm build` | Runner build in `pnpm -r build` | Extra `build:typescript` passes in it | Runner rebuild in server build | Extra passes in it | | --- | --- | --- | --- | --- | --- | | 38112764439 | 169s | 76s | 04:51:52 and 04:52:00 (about 14s) | 04:53:12 to 04:53:41 | 04:53:26 and 04:53:34 (about 15s) | | 38111191543 | 176s | 79s | 04:22:40 and 04:22:48 (about 14s) | 04:24:04 to 04:24:34 | 04:24:18 and 04:24:26 (about 15s) | | 38110914319 | 160s | 75s | 04:19:07 and 04:19:14 (about 14s) | 04:20:25 to 04:20:53 | 04:20:39 and 04:20:46 (about 14s) | | 38110897745 | 172s | 80s | 04:15:16 and 04:15:25 (about 15s) | 04:16:41 to 04:17:11 | 04:16:55 and 04:17:03 (about 15s) | ## Risks - Low risk. The same two checks run against the same dist, in the same position in the chain. The only difference is that the dist is not rewritten twice between them. - If a future step between `build:typescript` and the two checks starts writing to `dist/`, the checks would read that output instead of a fresh rebuild. The steps there today (`check:capability-contract`, `check:capability-inventory`, `check:protocol-coverage`) are `--check` runs that only compare generated spec files. - The standalone `check:semantic-contracts` and `check:runner-workflow-traceability` scripts still rebuild first, so documented usage is unchanged. - The guard test lives in the server's suite because that suite already pins the Runner vendoring contract and runs in the server shards on every PR. ## Model Used - Claude (Anthropic), model ID `claude-fable-5-1`, running as a Paperclip agent through the Claude local adapter with tool use (shell, file edits, GitHub CLI). The agent collected the CI data, made the change, ran the tests, and wrote this description. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Devin FoleyandPaperclip
|
b36960b758 |
fix(server): permit verified question-delivery history to sleep (#15851)
Treat completed question-response deliveries as history only after their exact same-company handoff finishes. Preserve missing evidence, retries, accounting debt, and idle-sleep fences. Add a concurrent all-state question history index with interrupted-build repair. Keep migrations serialized without waiting snapshots that deadlock concurrent index builds. Validated by 118 focused sleep tests, six migration/snapshot tests, full repository typecheck, all CI gates, and Apex 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
07f3536e37 |
fix(connections): discover Sentry OAuth through its MCP resource (#15837)
Correct Sentry OAuth discovery and cover new and saved client registration. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b1ebe3de3c |
perf(chat): wake action queues from commits and deadlines (#15808)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat keeps durable rows for provider actions and setup messages. > - Five action queues still scan the database every second when empty. > - Empty scans add idle database traffic and prevent cost savings from sleep. > - This pull request wakes these queues when their writes commit or their next deadline arrives. > - Existing rows retain retry and restart recovery. Empty queues stop scanning. ## Linked Issues or Issue Description Refs #15802, #15625, #15750, #15757. Related allocation work: #15153. **What existing behavior does this improve?** Idle chat reconciliation for Slack Board messages, Slack slash-command task starts, Slack setup verification messages, receipt reactions, and Slack session stops. **Current behavior** The app runs these five queue scans through chat maintenance every second, including when no work exists. **Proposed behavior** Each queue has an independent worker on the existing scheduler. Producers and foreground claim/retry/settlement writes announce work inside their database transaction. The worker wakes after the outer transaction settles. Startup restores durable work. Shared dispatch and deadline predicates select real retries and abandoned claims. Empty queues disarm. **Subsystem affected** Server chat services. No schema or migration changes. ## What Changed - Register five action workers alongside the three delivery workers from #15802. Remove their scans from periodic app maintenance. - Share action eligibility between dispatch and deadline selection. Park paused Board messages and task starts until endpoint changes. Preserve uncertain-send quarantine. - Wake setup verification after account, welcome, registration, or authenticated callback changes. Exclude prerequisites that need another event. - Persist a retry time for failed provider-confirmed task admission. Its own commit notification cannot cause an immediate retry loop. - Reuse the existing transaction signals and scheduler registration code. Provider calls remain outside database transactions. - Update the instance-sleep inventory and add scheduler and PostgreSQL regression coverage. ## Verification - Passed: 37 scheduler, coordinator, and app-wiring tests, including an idle hour with no further scans or timers. - Passed: seven focused PostgreSQL chat cases. These cover deadline eligibility, startup recovery, outer-commit wakes, actual producer signals, and signed-callback verification without maintenance polling. - Passed: `pnpm typecheck`. - Passed: full chat integration suite and related publication/coordinator tests, **1,278 tests** across six files, including **1,228 chat integration cases**. - Passed: `pnpm build`. - After the review fix, seven focused PostgreSQL integration cases and the server typecheck passed again. Deadline and dispatch queries now exclude completed action history. - Greptile: **5/5** on `35a979fce6bb9ab910fab3373ec8e537fc2d01a9`, with the prior finding resolved and no new findings. - Fixed a CI timing failure in the GitHub mention integration test: asynchronous admission now has an explicit ten-second assertion deadline instead of the default one second. The failure reproduced locally; all five mention variants and all 398 cases in the matching CI shard pass after the fix. - Hosted CI passed on `35a979fce6bb9ab910fab3373ec8e537fc2d01a9`: **54 successful checks**, two intentionally skipped Storybook checks, no failures or pending checks. Provider responses are simulated in the integration suite. PostgreSQL, transactions, service dispatch, and scheduling are real. No live Slack workspace was used. ## Risks - Every producer that changes runnable state must emit a transaction signal. This change covers the five queues and their foreground dispatch/retry paths. Direct SQL and independent DB owners still need explicit integration or startup recovery. - Missing retry or claim deadlines can strand work. Tests compare the dispatch and deadline behavior, including paused and uncertain states. - Other provider maintenance, sockets, and scheduled jobs remain. This change does not establish full instance sleep. - No new tables, migrations, or provider credentials are required. ## Model Used OpenAI Codex (GPT-6), with repository inspection, code execution, and automated tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a7dd5284b5 |
feat: add company fast-response acknowledgements (#15748)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Human messages start agent work through task, chat, and external-channel intake. > - Agent startup can delay the first visible response. > - A small direct model call can acknowledge the request while normal work starts. > - This response must remain separate from evidence that the agent answered or completed work. > - This pull request adds optional company-wide fast responses with durable receipts, access checks, and cost accounting. > - Users receive prompt acknowledgement without changing the agent execution path. ## Linked Issues or Issue Description **Subsystem affected** Cross-cutting: database, shared contracts, server orchestration, connections, and UI. **Problem or motivation** Users can wait for an agent to start before they receive a contextual response. Generic progress messages do not confirm what the user asked. An acknowledgement must not falsely complete work or satisfy an unanswered turn. **Proposed solution** Add a disabled-by-default experimental fast-response model using an existing shared API connection. Generate one short acknowledgement for each accepted human turn. Use the existing agent identity and external publication outboxes. Retain platform provenance and account for the inference independently. **Alternatives considered** Running the acknowledgement through the agent runtime adds startup latency. Restricting implementation to OpenRouter leaves other existing API connections unsupported. This change supports all existing API connections and uses OpenRouter for live verification. **Roadmap alignment** This extends existing agent communication and connection settings. ROADMAP.md and open pull requests were checked. No duplicate fast-response implementation was found. The maintainer supplied the feature plan directly. ## What Changed - Add Experimental fast responses to instance settings. Keep the feature off by default. Enforce the flag at intake, dispatch, and publication. - Move Decision model and Fast response setup to company Settings → Connections when the experiment is enabled. Place Connections below Environments. Separate the model sections with clear headings and a divider. Keep Decision model in General when the experiment is off. Route Costs setup links to the available settings page. - Make the acknowledgement prompt a multiline template string. Ask for a brief, positive response in the user’s language. - Add settings, connection/model selection, sponsorship, fixed-sample testing, and Costs history. - Add a Vercel AI SDK text service for existing native, routed, Bedrock, and local API connections. - Add durable per-turn requests, a bounded worker, a three-second admission deadline, and one inference attempt. - Integrate task creation, task and agent chat, steering, queued follow-ups, experimental board chat, shared chat intake, and email. - Preserve receipt provenance and exclude receipts from completion, recovery, interaction resolution, and agent wakeups. - Recheck authorization, source validity, real replies, budgets, and destination binding before publication. Preserve accounting for suppressed or uncertain outcomes. - Add provider contracts, database race tests, channel outbox tests, UI tests, Stories, and documentation. Keep decision-model execution unchanged. ## Verification - Local `pnpm -r typecheck`, `pnpm build`, and UI token gates pass after rebase. The UI typecheck and build also pass after the final navigation fixes. All 37 focused navigation tests pass. The first full local `pnpm test:run` invocation reported seven chat failures. All seven passed in isolated reruns after fixture updates and host cleanup. This invocation was stopped after the full final-head CI suite passed. - Initial local database tests hit macOS shared-memory exhaustion. A diagnostic confirmed `shmget: No space left on device` with the system limit of 32 IDs. Three abandoned, unattached PostgreSQL IPC headers were reclaimed. The isolated rerun passed all 31 fast-response and channel tests. The two additional Slack regression cases also passed in a separate rerun. No active database was stopped. - UI token gates pass. Provider contract tests cover all supported API connection types. Fast-response tests cover uniqueness, authorization, budgets, expiry, cancellation, restart, real-reply races, and the experimental flag. - The test drive verified that disabling the experiment hides Connections and redirects its direct URL. Enabling it restores the saved OpenRouter setup. After restart on the final commit, the OpenRouter settings sample returned “Sure, I’ll take care of fixing the border around the settings panel.” in 428 ms. It used 260 input and 31 output tokens and recorded $0.0000576 in reported cost. - Earlier live OpenRouter verification produced a receipt in 1.013 seconds for a new task. The real agent then supplied its substantive answer. A separate settings test took 1.34 seconds, used 246 input and 23 output tokens, and recorded $0.0000361. - Live Slack verification on final commit `24016fa525961ebc7a4acffdc5019ce123970549` passed for a fresh initial task and a follow-up in a disposable bot DM. The initial acknowledgement appeared 2.405 seconds after the Slack message and 1.413 seconds after durable acceptance; the follow-up appeared in 2.322 seconds and 1.160 seconds respectively. The recorded generation/settlement durations were 898 ms and 697 ms. The initial call used 291 input / 39 output tokens and cost $0.00006705; the follow-up used 281 / 47 and cost $0.00007035. - The Slack acknowledgements retained the assigned agent identity and `origin: fast_response`, source-turn references, and no fabricated run ID. A status snapshot after the initial receipt showed the task still `in_progress` with its real execution run. The agent then read the CSV fixture and correctly reported two ready items, two blockers, and their owners. Both replies stayed in the originating DM, and no duplicate generic working message followed the delivered contextual acknowledgements. The `new` command started a fresh conversation session without generating a fast response. - An earlier eligible Slack turn generated a response but suppressed publication. Its reported cost was retained, generic progress remained, and the real agent completed the requested result. This is recorded separately from the two delivered acknowledgements; the history did not expose a precise suppression reason. - Slack verification used real signed provider events, the existing intake/publication paths, and a user-approved temporary webhook-only HTTPS tunnel. Subscriptions were limited to direct-message events. The test bot was paused and the tunnel stopped afterward. Live channel/group-thread and GitHub journeys remain unverified; their mocked tests cover intake, destination binding, publication, and continued underlying work. - Greptile reviewed final commit `24016fa525961ebc7a4acffdc5019ce123970549` at 5/5. Both new review findings are fixed and all threads are resolved. [Final-head CI run](https://github.com/paperclipai/paperclip/actions/runs/38060015730) passed. This includes grouped server/chat/workspace tests, serialized suites, runner checks, browser journeys, typecheck, build, release registry, policy, and the canary dry run. - Manual check: enable Experimental fast responses. Open company Settings → Connections. Select a shared API connection and model. Save and use Test response. Submit a task message and confirm that its receipt does not complete the task. ## Risks - The migration adds receipt and configuration tables plus accounting references. It uses idempotent statements and follows the current migration sequence. - A slow provider, authorization change, or real-agent reply can suppress an acknowledgement. Existing progress feedback remains available. - Unknown provider billing retains a reservation until reconciliation. External deliveries with uncertain outcomes must not be blindly repeated. - Live Slack initial-task and follow-up DM journeys pass. Live Slack channel/group-thread and GitHub journeys remain unverified. The PR remains a draft and records these gaps. Final-head repository CI and automated review pass. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository tools, code execution, and browser interaction. The runtime does not expose an exact model snapshot ID or context-window size; these are not asserted. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
50083c4e35 |
test: honor configured instance in task cwd regression
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
5d6d6eabdf |
fix: retain task cwd overrides and protect ignored host files
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
3cfe5d58df |
Merge origin/master and preserve task workspace preparation
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
07fee85eaf |
refactor(heartbeat): extract workspace preparation (#15807)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The heartbeat service prepares the workspace and dispatches each agent run. > - After the runtime extraction, `heartbeat.ts` still had 6,417 lines. > - Workspace setup occupied one contiguous 857-line block inside `executeRun`. > - This pull request moves that block into one workspace preparation module. > - The executor keeps dispatch gates and terminal cleanup. Reviewers can inspect workspace setup as one phase. ## Linked Issues or Issue Description Refs #15803 and #15578. This continues the heartbeat refactor series. The search found no open duplicate workspace preparation extraction. **What existing behavior does this improve?** It makes heartbeat workspace preparation easier to maintain. It does not change the workspace policy or execution behavior. **Current behavior** `executeRun` contains workspace resolution, freshness decisions, provisioning, persistence, issue binding, environment acquisition, and realization inline. **Proposed behavior** `prepareHeartbeatWorkspace` owns that sequence in `server/src/services/heartbeat/workspace-preparation.ts`. `heartbeat.ts` calls it at the same point and uses its returned workspace, lease, target, and metadata. **Reason and benefit** The main file falls from 6,417 to 5,597 lines. The workspace phase has a typed boundary and direct regression tests. **Breaking changes** None. There are no schema, API, feature-flag, or execution-policy changes. ## What Changed - Extract the 857-line workspace preparation block without changing its operations or their order. - Preserve native recovery bindings, workspace persistence failure cleanup, issue rebinding after realization, and the cloned restore lease. - Keep environment selection, dispatch admission, terminalization, and final lease release in `heartbeat.ts`. - Add 12 boundary tests with disposable PostgreSQL, real Git repositories, and real local environment leases. - Make the existing Stop and plan-only recovery tests await executor completion instead of short polling deadlines. Assert exactly one continuation for the specific plan-only source run. - Keep server-only Node preload hooks out of the deterministic process adapter used by Stop browser acceptance. - Refresh workspace recovery browser acceptance for the current task composer, and give its disposable test-drive an explicit 10-second snapshot deadline. Production snapshot settings stay unchanged. - Document the workspace preparation boundary in `doc/DEVELOPING.md`. ## Verification - AST comparison: all 857 moved lines match the original block. The remaining module body differs only at the extraction call. - `pnpm build`: passed. - `pnpm -r typecheck`: passed. - Focused regression coverage: 651 distinct cases across eight suites passed across targeted runs, including all 12 new boundary tests. The complete 400-test recovery suite passed together on the final commit. - Workspace bootstrap browser acceptance: both journeys passed (4.2 minutes), including preserved source edits, ignored-file exclusion, bounded retries, and persistence across reload. - Stop/queue browser acceptance: both process and native runner journeys passed (2.2 minutes), covering queueing, Stop, subtree pause/cancel, resume, and unrelated-task isolation. - [Full repository CI](https://github.com/paperclipai/paperclip/actions/runs/38062408589): all 47 jobs passed, including the general and serialized test suites, eight browser shards, build, typecheck, native runner checks, and canary dry run. - The local `pnpm test:run` was interrupted to prioritize the recovery regression; the complete repository suite was verified in CI. - Greptile: 5/5 on `c28e698b885cf5fd12f36f7b4052e7a2e6f879d7`, with no open review threads. ## Risks - This code writes durable workspace bindings and acquires environment leases. A changed operation order could break recovery or cleanup after partial setup. The extraction preserves the original statements and exception boundaries. - The new tests cover creation, reuse, immutable recovery bindings, persistence failure, acquisition and realization failure, controller ownership loss, isolation, execution policy, and lease snapshot independence. - Browser acceptance uses real application services with deterministic local workers and a fake Codex protocol provider. It does not qualify live cloud providers. ## Model Used OpenAI GPT-6 through Codex, with code editing, shell execution, and test tools. The exact serving model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b014a5ad1e |
fix: align workspace source authorization with admission strategy
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0049e01678 |
fix: preserve task policy and authorize channel workspace admission
Keep task project environment and trust separate from workspace source configuration. Authorize channel creation against locked endpoint defaults, realize first native chat worktrees safely, and enforce source authorization on legacy PATCH. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
026010de0d |
merge: integrate runtime and chat refactors with source workspace authorization
Preserve task workspace finalization timing in the extracted runtime module. Separate source workspace write policy from task reassignment and retain board assignee context through chat defaults. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
4d7f0c8fe6 |
fix: authorize actual workspace sharing and assigned board targets
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
19e9facc5b |
perf(chat): wake delivery queues from commits and deadlines (#15802)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat connectors store incoming messages and outgoing publications in durable queues. > - The app checks these queues every second, even when they are empty. > - Empty scans keep the database active and hinder instance sleep. > - This pull request wakes three queue workers after transactions settle and at known retry or recovery deadlines. > - Empty queues stop scheduling checks. Existing rows retain their ordering, retry, and recovery rules. ## Linked Issues or Issue Description **What existing behavior does this improve?** Idle database activity from incoming chat deliveries, outgoing publications, and Slack upload receipt recovery. **Subsystem affected** `server/` — chat delivery and background work scheduling. **Current behavior** The app's one-second chat timer scans the three queues together with provider and action maintenance. Empty queues continue to query PostgreSQL. **Proposed behavior** Register three independent workers with the existing delivery coordinator. Producers announce work inside their database transactions. Commit wakes the corresponding worker. Startup restores outstanding work; persisted retries and stale claims supply the next deadline. An empty queue has no recurring scan. **Reason and benefit** Remove avoidable queue reads without requiring a new table, an external scheduler, or a persistent PostgreSQL listener. **Breaking changes** No schema or public API change. Work notifications are scoped to the app's shared database owner. Direct SQL and independent database roots do not send these notifications; startup still scans durable work. **Additional context** Refs #15625, #15750, and #15757 for the preceding sleep work. Related: #15153 reduces chat query allocation while retaining polling. This PR changes when the three queue workers run; it does not add query caching. ## What Changed - Register independent incoming delivery, publication, and Slack receipt workers on the shared scheduler. - Add transaction work signals at queue writes, endpoint state changes, and action transitions that unblock publications. Keep provider I/O outside the added transactions. - Restore deadlines from durable retry times, abandoned claims, and publication lease expiry. Share publication and receipt eligibility rules between dispatch and deadline queries. Respect incoming conversation order and lifecycle dependencies. - Exclude publications blocked by unresolved Slack tool sends. Wake them when the tool receipt changes. - Wake publication dispatch when an endpoint send finishes so free slots refill. Preserve per-endpoint concurrency, conversation order, and conservative handling of uncertain sends. - Wake receipt recovery when a stale publication becomes `delivery_unknown`, and wake queues when a provider connection recovers. - Commit interaction publications with their callback tokens so dispatch cannot see an incomplete card. - Keep provider runtimes, other action outboxes, milestone and wake-notice projection, Teams protocol maintenance, GitHub recovery, and Slack session synchronization on their existing maintenance cadence. This PR does not make the whole chat subsystem idle. - Update the instance sleep inventory with this partial migration and its remaining work. ## Verification - All 1,225 PostgreSQL chat integration cases passed across the three hosted chat shards. These cover retry and lease deadlines, blocked followers, unresolved Slack sends, provider recovery, commit visibility, and endpoint slot refill. - 32 local app-wiring, queue-worker, and delivery-coordinator tests passed after updating the wiring assertion for event-driven dispatch. Fake timers cover an hour of idle time without queue scans. - Related publication, interaction, receipt, and wake-notice tests passed locally. The 13 interaction integration cases include atomic rollback when callback-token storage fails. - Local `pnpm typecheck`, `pnpm build`, and final server typecheck passed. All hosted checks passed on the final commit, including build, typecheck, test shards, release registry checks, and the canary dry run (54 successful checks; two Storybook checks skipped for this backend change). - The last local PostgreSQL rerun was skipped because the embedded database startup probe failed after five attempts. The hosted PostgreSQL cases ran successfully; their skips only partition cases across shards. - `git diff --check` passed. Greptile is 5/5 on the final commit with no unresolved findings. ## Risks - Every producer that makes queued work eligible must send a transaction work signal. The tests exercise commit visibility, concurrent wakes, worker failures, and restart recovery. Future direct SQL writers must participate in this contract. - The existing database work-signal layer reconciles uncertain transaction outcomes. This PR adds no work-signal table or schema migration. - Provider and action maintenance still polls. Further changes are required before an idle chat-enabled instance can sleep. - Tests use real PostgreSQL with simulated provider responses; they do not send messages through live provider accounts. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, tool use, and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
1fb3f9746e |
refactor(heartbeat): extract prepared runtime execution (#15803)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Heartbeat prepares, dispatches, and completes each agent run. > - After the runtime-selection extraction, the main service still had 7,295 lines. > - Provider dispatch and workspace finalization formed one large block with shared cleanup rules. > - This pull request moves that block into a typed runtime-execution module. > - The service now has 6,417 lines. Readers can follow admission and outer cleanup without reading the provider execution details. ## Linked Issues or Issue Description Follow-up to #15746. I searched open heartbeat PRs and issues. I found no competing runtime-execution extraction. **What existing behavior does this improve?** The organization and test coverage of native and legacy heartbeat execution. **Current behavior** The heartbeat service contains a 912-line block for provider dispatch, usage capture, workspace finalization, and instruction cleanup. **Proposed behavior** Move that block to `server/src/services/heartbeat/runtime-execution.ts`. Keep the atomic dispatch gate, admission, terminal run status, and outer lease cleanup in the service. Preserve run behavior. **Reason and benefit** This gives runtime execution a named boundary and direct tests. It removes 878 lines from the main service in one bounded change. **Breaking changes** None intended. There are no schema, API, or policy changes. ## What Changed - Add `executeHeartbeatRuntime` for prepared native and legacy execution, usage capture, workspace finalization, and gateway/instruction cleanup. - Return an explicit dispatch result. Update outer ownership state through immediate callbacks, including before exceptions. Read instruction state through live getters. - Move the two internal recovery error classes with their existing behavior. The caller imports the same classes for `instanceof` checks. - Add 15 database-backed execution tests. Keep the existing heartbeat regression suites. - Repair the native Stop acceptance fixture: installed Codex now takes precedence over `PATH`. A test-only loader pins the fake provider. An isolated home and dummy key keep readiness offline. Add four loader tests. - Document the module boundary and fixture setup. ## Verification - Passed: eight focused heartbeat suites (483 distinct tests, including 15 new execution tests). The boundary suite passed again after adding the failed-restore retention case from review. - Passed: `node --test tests/e2e/fixtures/composer-stop-codex-loader.test.mjs` (four tests). - Passed: `pnpm -r typecheck` and `pnpm build`. - Passed: AST comparison of all 912 moved lines after the explicit boundary substitutions. SQL template text, recovery class bodies, public exports, and the rest of the executor match the baseline. - Passed: browser Stop journeys for legacy process and native Paperclip Runner (two tests). The native fixture log proves provider `turn/start` before Stop and `turn/interrupt` after Stop. - Passed: all three ACP continuation browser journeys (same-session queued input, no replay of an interrupted write, and a new direction after composer Stop). Five relevant browser journeys passed in total. - Passed: the full CI test groups, including all server, chat, workspace, serialized-server, runner, and browser shards. - Local full-suite limitation: `pnpm test:run` first hit the macOS shared-memory segment limit while starting embedded PostgreSQL. After releasing only this task’s unused acceptance-server segments, the retry progressed without reported failures. I stopped that duplicate run after the full CI test groups passed. - Passed: all 56 PR checks/statuses on `8694c116e8d43ed8ac11bc4c58623ea3af34a675`. Greptile is 5/5 with no unresolved review threads. ## Risks - Cleanup depends on state written before an error. The module reports dispatch, retention, and recovery ownership immediately. Tests cover early returns and exceptional cleanup. - Instruction callbacks can replace saved metadata during execution. Live getters preserve those changes. - Native acceptance uses a real server, database, browser, and Rust runner with a deterministic provider. It does not verify a live provider service. ## Model Used OpenAI Codex, GPT-6 family, with reasoning and repository/tool access. The exact serving model ID and context-window limit were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ab45588d13 |
fix: retain task context in shared source authorization
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
1b41e74faa |
fix: enforce source project authority for shared workspace writes
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
fff60782d2 |
fix: bound durable workspace seed cache publication
Enforce workspace and company generation, byte, and directory caps under one company lock. Cache saturation and free-space limits preserve per-run seeds and skip optional publication without deleting existing recovery evidence. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0f8cea002a |
merge: preserve workspace callbacks through runtime selection extraction
Adopt upstream runtime-selection boundary and forward the task workspace checkpoint/phase callbacks without moving their run-log policy out of heartbeat. Runtime selection, project repository admission, and responsible-user invariants pass43 tests; server typecheck passes. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
cb0e60c510 |
fix: preserve task workspace access and continuation across edge cases
Address Apex review: authorize board mutations with complete task state, preserve pending choices through first binding, resume retained workspaces after source removal, and keep valid output when sparse capture encounters unsafe symlinks. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
2a2071340d |
refactor(server): extract heartbeat runtime selection (#15746)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Heartbeat execution prepares a runtime, dispatches work, and records the result. > - Earlier refactors separated workspace services, scheduling, and completion. > - The executor still builds native inputs and records runtime selection inline. > - This preparation must preserve admitted inputs and serialize with Stop. > - This pull request moves that phase into a typed runtime-selection module. > - Reviewers can inspect runtime preparation separately from dispatch and cleanup. ## Linked Issues or Issue Description Refs: #15732. **What existing behavior does this improve?** Maintainability of heartbeat runtime preparation and selection. **Subsystem affected** server/ — orchestration services. **Current behavior** The main heartbeat service has 8,128 lines. `executeRun` builds or restores native inputs, selects sessions and sandbox lifecycle, and records native or legacy selection inline. **Proposed behavior** Move this phase into `server/src/services/heartbeat/runtime-selection.ts`. The executor calls `selectHeartbeatRuntime` at the existing boundary and receives an explicit selection result. The main file becomes 7,275 lines, a reduction of 853 lines. **Reason and benefit** This gives runtime selection its own typed boundary and direct tests. It preserves existing decisions, database transactions, and cleanup ordering. **Breaking changes** None intended. All public heartbeat exports remain available. ## What Changed - Move native input assembly, completion contract selection, checkpoint lookup, runtime persistence, and native workspace-sync preparation into `selectHeartbeatRuntime`. - Preserve the row lock and controller lease check that serialize native selection with cancellation. - Report lifecycle and ownership changes to the executor immediately, including when a later preparation step throws. - Move sandbox lifecycle and model fallback helpers with compatible public re-exports. - Add 12 database-backed boundary tests. Update the existing session-recovery source test and developer guide. ## Verification - Structural AST comparison against current `master` confirms the 784-line selection block, moved helpers, and remaining service bodies are unchanged after normalizing boundary callbacks. All 140 public exports are preserved. - On final head `2eddaa485f`, all 12 new database-backed tests and 83 existing lifecycle, selection, fallback-warning, and session regression tests pass (95 total). - Five executor integration cases passed before the rebase. Their final-head rerun could not start because the local checkout was removed during verification; no assertions ran in that last attempt. - `pnpm -r typecheck` and `pnpm build` both pass on final head. - Local verification limitations: the full `pnpm test:run` was attempted, then stopped after database/process startup timeouts in concurrent local runs. Two unchanged optional real-runner session cases timed out on the initial run and one isolated retry; they were excluded from the 95-test focused run. CI also skips those two binary-dependent cases. - All CI gates pass on final head `2eddaa485f`: typecheck, build, general and serialized tests, runner checks, E2E, and canary dry run. - Greptile reviewed final head at 5/5 with no actionable findings. ## Risks This boundary controls native admission and cleanup ownership. Losing an update after a later exception could change resource cleanup. Immediate callbacks preserve those updates. Tests cover Stop races, lost controller leases, immutable admitted inputs, competing runtime selection, and workspace-sync failures. The change adds no schema or API behavior. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, code editing, and command execution. The session does not expose the exact model variant or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (95 focused tests on final head; broader local limitations documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d6e6d1c78a |
Stream OpenCode immediately, scope reasoning to turns, and deduplicate instructions (#15708)
## Thinking Path
> - Paperclip helps people manage AI agents and observe their work.
> - The native runner carries provider output into the task transcript.
> - OpenCode sends incremental text in `message.part.delta` events.
> - The driver ignored those events and waited for a full part snapshot.
Accepted chats also waited after lifecycle verification was complete.
> - This pull request forwards each valid chunk as it arrives and
removes unnecessary isolated startup work.
> - Valid chunks reach Paperclip sooner without changing completion or
permission rules. The matched browser experiment below does not
establish a reliable end-to-end latency improvement.
## Linked Issues or Issue Description
**What happened?**
The native OpenCode driver withheld text until OpenCode sent a full
snapshot. A local delayed-provider probe with the pinned OpenCode
1.18.34 runtime showed a 1,496 ms gap between the first native text
delta and the first Paperclip text delta (1,510 ms from provider text).
**Expected behavior**
Forward valid assistant text and reasoning deltas immediately. Preserve
repeated identical tokens, message roles, turn ownership, and final
response selection.
**Steps to reproduce**
Send an empty text part, then text deltas, and withhold the final full
snapshot. The new fixture requires the consumer to acknowledge its first
text delta before releasing the snapshot. The previous driver cannot
pass this handshake.
**Paperclip version or commit**
Baseline: `011e5669bad61c435a4745fb10812bef70376841`. The patch is
rebased onto current master.
**Deployment mode**
Local native OpenCode 1.18.34 on macOS arm64. Live diagnostics use
`openrouter/deepseek/deepseek-v4-flash-0731`. These are narrow
diagnostics, not full Product E2E qualification.
## What Changed
- Keep the macOS verified OpenCode launch pathname until child exit,
with guarded reuse and cleanup on failed launches. This fixes a
reproduced asynchronous executable-lifetime failure.
- Dispatch accepted queued chats immediately after lifecycle readiness
commits, through normal scheduler admission. Preserve ready state if
dispatch fails.
- Validate native OpenCode model access with the real hello probe
without a redundant catalog scan. Keep legacy adapter discovery
unchanged and isolate unspecified native probe directories.
- Map incremental text and reasoning to the existing canonical event
path, using observed part and message identities.
- Keep additive chunks without IDs distinct. Preserve ID-based event
deduplication.
- Clear current streaming part metadata when a new turn starts. Reject
mismatched session/message identities and unknown parts or fields.
- Disable automatic updates, remote model refreshes, and default plugins
in each isolated OpenCode process. Keep assigned MCP tools and default
model reasoning intact.
- Retain the latest cumulative text/reasoning part while its message
role is unknown, including completion metadata after more than 100
chunks.
- Expose `startTurn({ message, reasoningMode: "disabled" })` for one
OpenCode/OpenRouter turn. Carry the selection in the durable command
through Rust and the proxy. An omitted/default option restores provider
defaults on the next turn, including warm and recovered sessions. There
is no environment-level setting or automatic selection policy.
- Group optional Rust turn inputs in `CodexTurnOptions` and dispatch
through `start_turn_with_options`; retain `start_turn` for defaults.
Future turn settings can be added without another feature-specific
method.
- Expose a native-session capability for per-turn reasoning and reject
explicit unsupported selections before dispatch, including direct ACPX
and Codex backends. Runnerd checks the durable selected model as well as
the provider.
- Remove duplicate agent instructions from the shared OpenCode proxy
task envelope for all users. Send one system copy on every turn,
including prepared and resumed turns, because OpenCode reads the latest
user message’s system field. Preserve task constraints and completion
contracts.
- Document startup and streaming behavior. Add regression coverage for
role ordering, repeated chunks, final snapshot deduplication, and stale
turns.
## Verification
### Lower-load browser retry — 2026-10-10
**All 8/8 trials succeeded. In this batch, median Send → first visible
reply fell from 30.586 s to 12.476 s (59.2% lower).** Four fresh-agent
trials per variant, phase order before/after/after/before with two
trials per phase. Every adjacent-phase pair favored the candidate,
although one difference was only 0.301 s. This is evidence for the
readiness/probe changes on this host, not a production percentile or a
guarantee across providers.
| Phase/trial | Variant | Send → reply (s) | Observed ready → run (s) |
Session open → turn start (s) | Model requests |
|---|---|---:|---:|---:|---:|
| 0/0 | before | 45.042 | 25.337 | 3.629 | 2 |
| 0/1 | before | 13.689 | 0.000 | 2.443 | 2 |
| 1/0 | after | 14.696 | 0.000 | 2.526 | 1 |
| 1/1 | after | 13.389 | 0.000 | 2.470 | 2 |
| 2/0 | after | 10.871 | 0.000 | 2.706 | 2 |
| 2/1 | after | 11.564 | 0.004 | 2.443 | 2 |
| 3/0 | before | 41.582 | 21.518 | 2.376 | 2 |
| 3/1 | before | 19.589 | 9.753 | 2.707 | 1 |
Observed ready-to-run median: **15.636 s → 0 s**; zero means execution
began by the first ready observation, not literally zero latency.
Polling interval is nominally 250 ms plus HTTP time.
Session-open-to-turn-start includes controller handling, not just child
startup. All eight ran within the existing startup timeouts; no timeout
or reasoning policy was changed. Host load at fixture creation stayed
**5.47–8.73 on 18 CPUs**, versus 68.9–100.8 in the prior campaign. No
builds/tests ran concurrently with measurement.
Same merged source **b0a0158729a7913882a0e756fe7e27228d72bf53**, release
runner, rebuilt static UI, OpenCode **1.18.34**, model
**openrouter/deepseek/deepseek-v4-flash-0731**, provider-default
reasoning, and empty isolated workspaces in both variants. Only the
readiness worker and explicit-model verification source were toggled;
the launch-lifetime fix and all proxy streaming changes were held
constant. Thus this does not separately measure the original streaming
changes. One browser per phase, two fresh companies/agents/chats per
phase. Measurement begins at capture-phase Send click and ends on
visible Hi text plus two animation frames. Status/thinking is excluded;
emoji can arrive just after the first text. Successful screenshots
inspected. Each variant had three two-request turns using
`paperclip_finish` and one one-request turn; no status lookup was
observed. Reported usage includes settlement after the first visible
reply.
Runner SHA-256
`57f658c220eecd2370ffd3333cd7e5d06d16bb2798af0cf43c84e01c8207c1c3`;
proxy
`bcc02fda2bf35b38a116cd880841aa6c534f6e0b497a96d97369a1e06dcc72ac`;
rebuilt UI tree
`eee238f2eda628c4e8c3660707986da4f8dd9100c814e26938fce3756df2eae4`
(sorted relative path, NUL, binary per-file SHA-256). Worker/probe
variant source hashes are unchanged from the preserved comparison below.
Measured turn cost **$0.00612574**, plus verification probes not
included in that figure. All eight attempts retained; all source/proxy
files restored and test supervisors exited.
Current commit validation: **53 successful CI checks, 4 skipped, zero
failing; Greptile 5/5 on
|
||
|
DottaandPaperclip
|
10a873b76c |
merge: reconcile callback bridge fixtures with current master
Co-Authored-By: Paperclip <noreply@paperclip.ing> |