1934 Commits

Include renames

Author	SHA1	Message	Date
DottaandPaperclip	b3eb03fcba	fix(sentry): preserve run failure stacks and diagnostic context (#14585) ... Builds on merged #14575 and targets `master`. ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Operators use optional Sentry reports to diagnose failed agent runs. > - The shared reporter currently converts each saved error message into a new exception. > - This loses the original stack and cause chain, and omits available adapter and provider diagnostics. > - This pull request selects, redacts, and bounds useful diagnostics before it sends them to Sentry. > - Operators can locate the failing operation and correlate upstream failures without copying arbitrary run data. ## Linked Issues or Issue Description Refs #14573. That change preserves ACP provider failure details in the run record. Refs #14575. That merged PR adds recorded exit codes and signals. This PR covers stacks, causes, and structured diagnostics without duplicating those fields. A thrown setup or adapter error currently appears in Sentry with the reporter's stack. A saved provider failure can contain useful details that never reach the Sentry event. Both cases use the same shared reporting path. ## What Changed - Pass caught setup and execution exceptions, and structured adapter error metadata, through successful terminal status transitions. - Snapshot a fixed set of execution, adapter, provider, and exception fields. Preserve up to four exceptions in the cause chain. - Remove registered secret values, declared runtime environment credentials, unknown inherited environment values and encoded credential forms, and credential patterns before truncation. Mark truncated fields and bound provider details and stacks. - Rebuild sanitized Sentry exceptions with the original stacks and causes. Use an adapter stack preview when available. Omit a fabricated reporting stack when the source has no stack. - Keep contexts local to each event. Preserve the optional DSN gate and existing error-code/adapter fingerprint. - Document the fields, limits, and omitted data. - Settle leftover chat fixture outbox rows only after assertions and worker shutdown, so subsequent tests cannot claim earlier cases’ pending actions or provider I/O. This fixes the CI shard contamination exposed during verification; production chat behavior and test timeouts are unchanged. ## Verification - `pnpm -r typecheck` passed. - `pnpm build` passed. - 86 focused diagnostic, Sentry, and startup tests passed after the rebase, including the real `@sentry/node@10.71.0` SDK with an in-memory transport. - Tests cover cause chains, HTTP status and request IDs, long provider details, secret redaction, failed secret resolution, cyclic causes, size limits, and context isolation. - Five targeted heartbeat integration cases passed, including thrown and returned errors containing an opaque environment-bound credential. The earlier full heartbeat integration run also passed its integration cases. - The focused suite also passed with an unknown inherited environment value set to `1`; reporting tests use a controlled environment and separately verify short-secret redaction. - Before the fixture cleanup, the full chat shard reproduced the CI Telegram timeout at `pending recovery before restart` (354 passed, 1 failed). After the cleanup, the same shard passed all 355 tests. No timeout or production behavior changed. - Latest-head CI (`0ae9e70df321d66dda025c3a7ba4787e169e0a4f`) passed typecheck, build, all 12 server shards, all 3 chat shards, workspace and serialized suites, browser shards, Runner checks, canary validation, the real Sentry SDK contract, and security checks. Required `ci / verify` and `ci / e2e` passed. - The branch has been rebased onto `master` after #14575 merged. All 55 applicable checks passed on this head (2 unrelated Storybook checks skipped). Greptile re-reviewed the current 11-file diff at 5/5 with no findings or unresolved review threads. - Local monolithic full-suite attempts were interrupted to apply fixes; full-suite success is not claimed from those runs. ## Risks - Error messages and stacks can contain credentials. The reporter uses existing redactors and the run's encrypted secret registry, reads only known fields, and skips capture if registered-secret resolution fails. - Unknown environment values remain private by default. Unrecognized short values can mask benign matches; known public settings are explicitly allowed. - Diagnostic text is bounded and can be truncated. Truncation is explicit. Data discarded upstream cannot be recovered. - These additional fields go to the operator's configured Sentry endpoint. Arbitrary request/response objects, headers, configuration, prompts, and stdout/stderr are not copied. ## Model Used OpenAI GPT-6 through Codex, with tool use and code execution. The exact serving model ID, context window, and configured reasoning level are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-29 12:54:12 -05:00
DottaandPaperclip	2de43fc909	fix(issues): keep agent mentions as context and defer personal app authorization (#14577) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Each task has one assignee. Explicit assignment and review requests select who should act. > - An agent mention started another agent on a task it did not own. Native attachment staging then rejected that run. > - Allowing that run through startup could also let two agents work on the same task. > - Mentions should identify relevant context. They should not start work or forward comments to other tasks. > - A personal app installed on a shared agent must also wait until tool use to resolve the current user's grant. > - This pull request removes mention dispatch and keeps missing personal app credentials from blocking startup. ## Linked Issues or Issue Description **What happened?** A native agent mentioned on another agent's task failed with `paperclip_runner_attachment_staging_not_authorized`. The source task could already be complete. A nearby optional-app warning was a separate problem: personal app tools were excluded when their shared health state required attention. **Expected behavior** An agent mention is context only. It does not wake the agent, take ownership, or copy a comment onto another task. Normal feedback still reaches the assignee. Assignment and explicit review requests still dispatch work. An unavailable personal app does not block startup or produce a startup warning. Tool use requests the current user's authorization and never uses another user's grant. **Steps to reproduce** 1. Assign a task to agent A. Post a comment that mentions agent B, including a comment that closes A's task or references B's child task. 2. Confirm the comment retains its agent link and B receives no run or deferred wake. A can still receive normal feedback. 3. Install an active personal MCP connection on B. Give only Alice a grant and leave shared health at `error`. 4. Explicitly assign work to B for another user. Confirm it can finish without using the app. 5. Ask B to use the app. Confirm its tool call shows an inline connection request for the current user. Related work: Refs #11144. This change uses the existing execution-time personal grant resolution. ## What Changed - Remove mention dispatch from standalone comments and issue updates. Remove implicit forwarding of parent comments to a mentioned worker's child task. - Ignore new requests with the legacy mention wake reason before creating a run or deferred request. Preserve already accepted queue entries, which can combine assignments and feedback with a later mention. - Remove the native mention admission, staging, and finalization exceptions from this PR. Native task ownership checks remain intact. - Keep active, installed personal app tools available despite shared health errors. Remove optional-app startup warnings. Tool execution retains the current user's grant and policy checks. - Update agent instructions and product/API docs. Refresh generated capability source anchors. ## Verification - Red: comment-route regressions reproduced extra agent wakes and child comment forwarding. A separate regression proved that cancelling by the last coalesced reason could drop an accepted assignment. - Green: the targeted route, wake queue, heartbeat, workspace, responsible-user, MCP discovery, and HTTP gateway suites passed. The final queue and heartbeat rerun passed 104 tests, the restored queue adapter passed 56, and both comment-route suites passed 135. These include accepted assignment preservation, rejection of new mention requests, and normal assignee feedback. - `pnpm -r typecheck` and `pnpm build` passed locally. The full local `pnpm test:run` attempt was interrupted for review/CI fixes, so it is not claimed as a completed local pass. It exposed a cleanup timing race in the concurrent-mention assertion, now fixed and verified across 10 repetitions. CI also exposed an obsolete test waiting for the removed mention lookup; it was reproduced and fixed, then both comment suites passed. Final full-suite verification is through CI. - Final head `bd9ea4cb05a8f081c54e017760a8999f9ea6ef44`: 54 checks passed, 2 Storybook checks intentionally skipped; no pending or failing checks. Full CI includes general and serialized suites, all 8 browser shards, runner verification, typecheck, build, and canary dry run. Greptile is 5/5 on this exact commit, with no unresolved findings. - One unchanged Cursor adapter test hit its 10-second CI timeout. All 5 tests in that file passed locally; one retry of its CI shard passed all 674 tests (3 skipped). The aggregate verification gate then passed. No code or timeout was changed for that retry. - Live browser check: inserted a structured mention with the picker on a human-owned task. The saved link remained visible. Database checks found zero new runs and zero wake requests. - Live Codex runner check: explicitly assigned that task with the unavailable personal app attached. The run succeeded and committed completion without using the app or creating a connection card. - Live browser follow-up: asked the assignee to call PostHog and mentioned another enabled agent as context. Only the assignee ran. It succeeded and displayed the existing inline connection card. Only Alice's grant existed; the run belonged to a different user. - The HTTP regression covers tool discovery with no provider calls or connection cards, first use returning the current user's authorization request, and successful retry after that user's grant exists. - App checks use an isolated local fixture and a fake MCP provider. They do not use production app credentials. ## Risks - Intentional behavior change: workflows that used mentions to wake agents must use assignment, a bounded child task, or an explicit review request. - Already accepted queue entries retain their prior rules. An old entry can combine assignment or feedback with a later mention; its last reason cannot safely identify mention-only work. New mention requests create no run or deferred wake. - Personal apps with a shared health error remain discoverable. Actual tool use still requires the responsible user's grant and existing policy gates. - No database migration or public API schema change. ## Model Used - OpenAI GPT-6 through Codex, with reasoning, repository tools, code execution, and browser testing. The exact serving model ID and context-window size are not exposed in this session. - Live native-run verification used `gpt-6-astra` through the Codex provider. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-29 12:49:01 -05:00
Devin FoleyandPaperclip	e5bf9d49a5	fix(sentry): retain recorded process exit details (#14575) ... ## Thinking Path > - Paperclip manages agents and records their task runs. > - Operators can enable Sentry reports for terminal run failures. > - A failed adapter can leave only the generic message “Adapter failed.” > - The run already records a process exit code and signal, but the report omits them. > - This pull request carries those two values through a strict capture boundary. > - Operators can distinguish a nonzero exit from signal termination when the message is generic. ## Linked Issues or Issue Description **What happened?** A failed run can store `exitCode: 1` or `signal: "SIGTERM"` while its Sentry event contains only `adapter_failed` and “Adapter failed.” The existing reporter drops both recorded fields. This occurs on the current master reporting path. **Expected behavior** The opt-in report preserves bounded process exit evidence without exporting adapter output or changing run behavior. **Steps to reproduce** Enable the backend Sentry DSN and report a failed run whose message is “Adapter failed” and whose stored signal is `SIGTERM`. Before this change, the event has no signal field. After this change, `run_failure.signal` is `SIGTERM` and the existing fingerprint stays the same. Related: #12105 and #8222 describe missing adapter/HTTP failure details. #13152 changes terminal-result cleanup classification, and #12886 adds process-failure classification and runtime URL checks. None forwards these stored fields through the Sentry reporter. This change does not resolve those broader issues. ## What Changed - Forward the stored exit code and signal from the terminal run reporter. - Accept only signed 32-bit integer exit codes; use `null` for missing or malformed values. - Accept only the reporting host's Node signal constants; use `null` for missing values and `unknown` for unrecognized values. - Keep the added fields in event-local context, outside tags and fingerprints. - Cover database-backed reporting, malformed input, privacy, and isolation through the real Sentry SDK. - Document the fields and their limits. - Give the dedicated Sentry job the normal PR dependency-resolution fallback, with lifecycle scripts disabled on every install and the required real-SDK test retained. ## Verification - Before the change: 16 report-shape/exit-field assertions failed in the focused capture suite. - After the change: 91 focused Sentry, DSN, and database-backed reporting tests passed. The real SDK uses an in-memory transport. - Final real-SDK test also passed with malformed metadata; it verifies that arbitrary signal text is absent from captured events and unrelated errors inherit no run context. - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - `pnpm test:run`: exited nonzero after 718 general-server files: 14,085 tests passed, 14 failed, 86 skipped. Thirteen skill-service/cache failures reproduce on the unchanged base commit on this macOS host. One comment-wake test timed out; the complete 29-test suite passes on the unchanged base and in final-head Linux CI. Local isolated rechecks skipped because embedded PostgreSQL could not start; these are not counted as passes. The remaining local workspace/serialized lanes did not run after the failing first lane; all CI lanes passed. - Initial Sentry CI failed before tests with `ERR_PNPM_LOCKFILE_CONFIG_MISMATCH`. Its install step lacked the normal PR fallback. The repaired real-SDK job passed. Security review then requested disabling lifecycle scripts for resolved dependencies; every install now uses `--ignore-scripts`. A fresh isolated checkout passed the exact script-disabled fallback and real-SDK contract. Final-head real-SDK CI and the security scan passed. - GitHub CI: all 54 checks passed on `37e0a836e50660f7753d367bcf5a4959eaf89b90`, including required `ci / verify` and `ci / e2e`; two unrelated checks skipped. - Greptile: 5/5 on that commit. No unresolved review comments. - Merge status: conflict-free; required CODEOWNER approval for the workflow change is still pending. ## Risks Low risk: this only adds two validated fields to existing opt-in error reports. It changes no database schema, run status, retry, fingerprint, or suppression rule. Process output and adapter result payloads remain excluded. A recorded signal does not identify its sender or prove an out-of-memory kill. Missing exit evidence stays unknown; this change does not establish the cause of a historical generic adapter failure. ## Model Used OpenAI GPT-6 (Codex), with reasoning, terminal tools, and code execution. The context window size is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes:` / `Closes` / `Refs` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal ticket id or instance-derived details - [x] I have run focused tests locally and they pass; broader validation is recorded above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-29 12:29:17 -05:00
DottaandPaperclip	83076d7e7c	feat: return completed handoffs to Agent Chat (#14408) ... Return completed Agent Chat handoffs through a durable outbox and scope each generated update to its supplied tasks. Add recovery, browser delivery, result access, and calibrated quality coverage. Validated with two consecutive ten-case Claude/Codex campaigns, all CI checks, and a 5/5 review. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-09-29 10:25:52 -05:00
DottaandPaperclip	3b4b270650	fix(adapters): preserve ACP terminal failure diagnostics (#14573) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The shared ACP adapter engine records agent failures for operators. > - ACP providers can report a failure category, title, and detailed cause. > - Our patch kept only the category in the saved error, so an operator could not diagnose a failure when tracing was off. > - This pull request preserves redacted provider diagnostics in the run error, transcript, and structured run result. > - Operators can now inspect the provider message and any supplied request ID or stack trace after the run ends. ## Linked Issues or Issue Description Refs #13889 (the diagnostic gap; this PR does not update the bundled Claude version). Refs #14484 (related model-refusal classification; this PR retains diagnostics for all terminal failure categories). **What happened?** An ACP turn failed with only `ACP agent reported a terminal service failure.` The provider's title and details were available in memory but absent from the saved error and transcript. **Expected behavior** The run retains useful provider diagnostics even when raw tracing is disabled. Credentials remain redacted. A size limit must report truncation instead of silently removing the cause. **Steps to reproduce** 1. Run an ACP agent that returns an error-severity typed session failure. 2. Include an HTTP error, request ID, and stack text in its title and details. 3. Inspect the failed run with tracing disabled. Before this change, only the category survives. ## What Changed - Both pinned ACPX patches pass complete error text to the in-memory callback, so redaction happens before truncation. - The shared engine retains the sanitized category, title, and details in `resultJson.terminalSessionFailure` and includes the text in the run error and error transcript. - Diagnostics redact configured environment values even under arbitrary names, unknown launch-environment values, connection URL passwords, run credentials, and common credential syntax. Known boolean settings remain readable, while credential values are redacted even when embedded in other text. Diagnostics remove control characters and invalid Unicode. - Title and detail limits keep escaped transcript JSON below the server's chunk limit. Truncated fields include an omission count. The safe run-result projection preserves a byte-bounded diagnostic preview when the result exceeds its byte budget, with an explicit pointer to the full adapter-bounded run error and transcript. - The existing UI and CLI display the error. Diagnostics do not become assistant output. Issue continuation summaries and session-compaction prompts receive only the generic category, preventing provider text from becoming handoff instructions. Existing quota classification, warnings, timeout precedence, and control-channel failure precedence remain in place. - Regression tests cover real ACP child processes with both pinned versions in one-shot and persistent modes, credential redaction, request IDs after the old 4 KiB cutoff, transcript parsing, storage bounds, and database retrieval of oversized multibyte diagnostics. ## Verification - Full CI on `20ad4f5f1f66c46d2c260e6ad0339cbea607b4cf`: **54 passed, 2 intentionally skipped, no pending or failing checks**. Includes typechecking, build, all Vitest shards, Runner checks, browser E2E, and the canary packaging/public-install dry run. - Greptile: **5/5** on this commit. Superagent security scan passes. All review threads are resolved. - Local verification passed: shared ACP engine suite (395 tests); real Claude ACP child-process and diagnostic regressions across both pinned runtimes and both execution modes; run retrieval and model-handoff regressions (59 tests); ACPX patch packaging (16 tests); full typecheck and build. Affected package typechecks and focused tests were rerun after review fixes. - The broad local `pnpm test:run` was stopped after review edits made its cached imports stale. Fresh targeted runs pass, including both affected server suites. Cold-build import failures were also rerun after dependency builds: chat integration (1,063 tests) and tool access (351 tests) pass. The final commit's complete CI matrix is green. ## Risks - Provider diagnostic text is untrusted. This change retains more of it in company-scoped run records. Redaction and size bounds apply before persistence. - Diagnostics are limited to fields the provider supplies. Old runs cannot recover discarded error text. - No schema migration, recovery-policy change, or new Telemetry or OpenTelemetry export. ## Model Used - OpenAI GPT-6 through Codex, with reasoning, repository inspection, code editing, and test execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-29 10:18:30 -05:00
3 people	da887ea3e9	fix(runner): honor Codex effort selected in composer (#14568) ... ## Thinking Path > - Paperclip manages AI agents that work on assigned tasks. > - The task composer lets a person choose an assignee, model, and effort for the next run. > - A Paperclip Runner agent can use Codex as its provider. > - The composer hid Codex effort for that agent because it checked only the older Codex adapter. > - The native Runner input also did not carry an effort choice to Codex. > - This pull request carries the chosen effort from the composer to each Codex turn. > - People can now select a supported effort and get the effort they selected. ## Linked Issues or Issue Description Refs #14322 **What happened?** The composer showed a model but no effort slider when the assignee used Paperclip Runner with the Codex provider. A task-level model override also did not reach the native Runner input. **Expected behavior** The composer shows effort choices for a known Codex model. The next native Codex turn uses the selected model and effort. **Steps to reproduce** 1. Open a task composer. 2. Select an agent that uses Paperclip Runner with the Codex provider. 3. Select a known Codex model such as `gpt-6-astra`. 4. Open the assignee and model picker. The effort slider is missing before this change. ## What Changed - Show known Codex effort levels for Paperclip Runner Codex assignees. - Save the task effort override in the native run input and send it to Codex on each turn. - Apply the task's merged model and effort overrides when the native run starts. - Apply a task model override for OpenCode Runner without changing the agent's provider. - Add Runner effort tests and desktop and mobile Storybook cases. ## Verification - `pnpm -r typecheck` passed. - `pnpm build` passed. - `pnpm build-storybook` passed. - `pnpm check:token-gates` passed. - Focused UI, server, Runner contract, and Codex driver tests passed. - The full CI test matrix, build, typecheck, and canary dry run passed on the latest head. ## Risks - Native Runner inputs add an optional Codex effort field to the current v5 input. Older inputs keep their previous behavior. - A known model rejects an effort that its catalog does not support. Unknown models do not show a slider. > This fixes an existing composer bug. I checked `ROADMAP.md`; it does not describe this bug as planned work. ## Model Used OpenAI Codex, GPT-6. The exact deployment ID and context window are not exposed in this session. The model used reasoning, code execution, and repository tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: OpenAI GPT-6 Astra <noreply@openai.com>	2026-09-29 09:41:05 -05:00
DottaandPaperclip	7636966452	fix(inbox): keep other users’ failed runs out of Mine (#14572) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Mine inbox shows work that needs the current user. > - Failed-run rows used the latest run for every agent in the company. > - A failure from another user therefore appeared in Mine and its badge. > - Run list responses also omitted the responsible user needed to filter these rows. > - This pull request uses run ownership for personal failure routing. > - Users see their own failures and can still inspect company failures in All. ## Linked Issues or Issue Description **What happened?** An agent run started for one user failed. Its row and failure badge appeared in another user's Mine inbox. **Expected behavior** Mine and its badge include failed runs for the current responsible user. Other users' failures remain available in All and run details. **Steps to reproduce** 1. Use a company with two human users. 2. Create a failed or timed-out run attributed to the first user. 3. Open Mine as the second user. Before this fix, the failed run appears there and increases the badge. **Paperclip version or commit** Reproduced in regression tests on master at `24beb0057`. **Deployment mode** Authenticated deployment with multiple users. Tests also cover the local single-user board. Related prior work: #933 addressed inbox dismissal and badge consistency. No duplicate ownership fix was found. ## What Changed - Return `responsibleUserId` in normal and summary run lists. - Share one ownership rule across both inbox versions and client/server badges. - Select the latest run per agent before applying the ownership filter. This prevents old failures from resurfacing on shared agents. - Keep unattributed historical failures in the local board's Mine view. Hide them from authenticated users with no matching owner. - Keep company health alerts outside the personal badge, consistent with the client. - Document the routing contract and add page, badge, and database regression coverage. ## Verification - Red: the new badge cases failed with three company failures instead of one personal failure; eight Mine page cases failed across both inbox versions. - Green: 113 focused tests pass in `ui/src/lib/inbox.test.ts`, `ui/src/pages/Inbox.test.tsx`, `server/src/__tests__/heartbeat-list.test.ts`, and `server/src/__tests__/inbox-dismissals.test.ts`. - `pnpm check:token-gates` passes. - Agent calls on behalf of a user have two additional red-to-green API regressions. - Full `pnpm -r typecheck` and `pnpm build` pass. Server typecheck also passes after the agent-call fix. - All CI test shards and browser tests pass on `243bfa681`. The duplicate local `pnpm test:run` was stopped after the CI test lanes completed; it did not finish locally. ## Risks - Authenticated users no longer receive unattributed legacy failures in Mine. Those failures remain visible in All. - The server badge no longer counts company health alerts, matching the existing client badge. - No migration, run state, retry behavior, or company access rules change. ## Model Used - OpenAI GPT-6 through Codex, with reasoning, terminal execution, and browser tools. The exact deployment variant and context window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-29 09:40:21 -05:00
DottaandPaperclip	24beb00575	feat(runner): add rich ACP transport and durable interaction foundation (#14430) ... Add shared rich ACP transport, durable questions and permissions, verified provider packaging, and bounded activity and plan presentation. Keep Cursor, Copilot, and Pi pending their separate provider qualification. Persist interaction settlement before publication, fence failed writes until fresh recovery, and preserve owned-process cleanup. Incorporate reviewed mainline integration with extended harness coverage. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-09-29 08:56:21 -05:00
DottaandPaperclip	c9b93d7e8c	fix: preserve terminal task owners during release (#14561) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Tasks record an assigned owner and separate checkout and execution locks. > - A completed task must retain its owner after execution ends. > - The release endpoint currently clears that owner when it clears the locks. > - This pull request preserves the assignee of Done and Cancelled tasks during release. > - Unfinished tasks keep the existing relinquishment behavior. > - The benefit is stable task attribution without retaining execution locks. ## Linked Issues or Issue Description **What happened?** An agent completed an assigned task, then called the release endpoint. The task stayed Done, but its assignee became null. The same defect affects Cancelled tasks. It caused the legacy Claude clarification/reuse and multiple-repository handoff E2E assertions to fail. **Expected behavior** Release must clear execution locks on terminal tasks and preserve their assignee, final status, and disposition timestamps. Release of unfinished tasks must still clear the agent assignee. Only In Progress work returns to Todo. **Steps to reproduce** 1. Create an assigned task with checkout and execution locks. 2. Complete or cancel the task. 3. Call `POST /api/issues/:id/release` as the assigned agent. 4. Read the saved task. Before this fix, its assignee is null. **Paperclip version or commit** Reproduced on master commit `d172197117a14b80a1eb2d2835a0e7cce2679656`. **Deployment mode** Local tests against real PostgreSQL through the production issue routes and services. This is a core lifecycle defect, independent of the agent adapter. Refs: #11689, #6899, #7769. These are related open release proposals. This is an independent fix limited to terminal task ownership. It does not include timer scheduling changes. ## What Changed - Preserve the current assignee when releasing Done or Cancelled tasks. - Keep all execution-lock cleanup and existing unfinished-task behavior. - Cover all seven task statuses through the release API and read back saved state. - Check disposition timestamps, activity attribution, and repeated board cleanup. - Update the API contract, agent reference, and CLI help. ## Verification - Red commit `ddaabb754`: the two terminal-owner regressions failed with `assigneeAgentId: null`; 12 other route tests passed. - Green: all 14 route tests pass, plus the existing successor-checkout race test (15 selected tests total). - Command: `pnpm --filter @paperclipai/server exec vitest run src/__tests__/issue-stale-execution-lock-routes.test.ts src/__tests__/issues-service.test.ts -t 'stale issue execution lock routes|does not let stale release clobber a successor checkout lock'`. - The local host has exhausted its SysV semaphore pool. The red/green runs used the existing test-provider hook to start disposable Docker PostgreSQL 17 instances. Routes, services, migrations, and assertions were unchanged. No database tests in the selected set were skipped. The other 134 tests were excluded by the name filter. - Capability contract and inventory drift checks pass. - `pnpm build` and `pnpm -r typecheck` pass. - The local `pnpm test:run` was interrupted after environment failures while the complete sharded CI suite ran in parallel: native PostgreSQL bootstrap fails under the host semaphore limit, and the large Git fixture hits macOS `ENAMETOOLONG`. A focused rerun confirmed these happen before the relevant assertions. The interrupted local run is not counted as a full pass. - Greptile completed on `1caeeb827e9cb658ddb71f16c2421ec20f80634e` with **5/5**, a successful check, and no review threads. - All CI gates pass on the current head: typecheck, build, general and serialized tests, Runner checks, browser E2E, release packaging, and security checks. Server shard 11 passed on one targeted retry; the first attempt had 836 passing tests but an unhandled workspace-runtime startup rejection caused by an existing timing window. All other successful jobs were reused. - No paid provider evaluations were run. ## Risks - A caller that used release to erase ownership from terminal work will now retain that owner. An explicit assignment update or the board force-release option with `clearAssignee=true` can still clear it. - No schema or migration changes. The transaction, company access, assignee/run checks, and activity log remain in place. ## Model Used - OpenAI GPT-6 through Codex, with tool use, code execution, and test debugging. The session does not expose a more specific backend model version or context-window size. - OpenAI `gpt-6-luna` assisted with read-only test discovery and review. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-29 08:47:33 -05:00
DottaandFry	3ca196b0a6	feat(agents): persist agent files across tasks without revision history (#14420) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - An agent needs personal files across tasks and sessions. > - AGENTS.md is one file in that directory. Supporting files need the same persistence. > - The Instructions Editor and agent runs must share one current directory. > - Concurrent runs should apply only the files they change. The last sync of the same file wins. > - This pull request uses existing file transport and removes temporary copies after sync. > - Old instruction-only sessions keep their restore contract. New saves do not create revision history. ## Linked Issues or Issue Description Refs #14325. This replaces its revision-oriented design with persistent agent files. Keep #14325 unmerged. Transport prerequisite #14416 merged first at `d172197117a14b80a1eb2d2835a0e7cce2679656`. This PR now targets master and remains below 100 changed files. Related work: #4513 and #8798 cover instruction tooling. This change handles run synchronization, cross-task personal files, browser editing, and old-session restoration. ## What Changed - Keep one current directory per company and agent. Point AGENT_HOME at a temporary working copy for each active run. Keep task files and provider HOME separate. - Restore text, binary files, and nested folders through workspace transport. Exclude remote agent files from task Git snapshots with a self-ignoring file inside the reserved runtime directory; never write through repository-controlled Git metadata. - Collect after the provider and child processes have stopped. Keep resumable conversation state. - Apply changed and deleted files under the agent lock. The last sync wins for the same file. Unrelated concurrent changes survive. - Remove temporary copies after successful sync, rejected sync, and staging failure. Register ownership before copying so restart recovery can remove interrupted preparation. Retry transient synchronization up to three times. Preserve the original remote lease reference until deletion succeeds; restart cleanup never acquires a replacement sandbox. Do not create captured directories or a conflict-review queue for new runs. - Keep browser editing, stale-draft protection, and streaming binary downloads. Keep the instruction entry and text editor limited to 1 MiB. - Keep historical agent-folder sync failures on their affected runs instead of repeating them above current saved instructions. Preserve legacy candidate review and current browser-save errors. Avoid duplicate quota warnings while retaining separate sync failures when they describe a different problem. - Require target-scoped caller grants for peer instruction access, while preserving self edits, responsible-user checks, and protected-change consent. - Treat full storage as a nonblocking run warning, never an agent pause or run-admission failure. Restore already-over-quota saved folders so ordinary agent cleanup can recover; warn on each run until cleanup. The run detail view shows the warning. - Allow 256 MiB per file, 2 GiB per directory, and 100,000 entries. Hash large files as streams. Check editor-save quotas with metadata instead of hashing unrelated files. - Preserve old native inputs, instruction-only copies, paths, digests, and pending legacy candidates. Adopt old revision heads once. New writes do not append history rows. - Add idempotent migration 0287 and verify upgrades from the preview tables and receipts. - Add nine interactive stories under **Agents / Persistent files**, including automatic incoming edits, stale browser drafts, and storage-limit diagnostics. ## Verification - Merge candidate: `4f5390107ec6ffd80a76d1d2e85530e66f21d079`, after merging current master and the landed transport prerequisite. Integration required no manual conflict resolution; the feature remains 99 changed files. Full workspace typecheck, production build, token gates, and 715 focused tests passed on this merge candidate. Fresh Greptile review is 5/5 with no unresolved findings. All 55 checks passed, with four conditional skips, including the build, typecheck, browser E2E, and canary dry run. A single retry recovered four jobs interrupted by runner shutdowns; no source changes were required. - Historical-warning UI fix: all 6,834 UI tests across 640 files passed, including regression coverage for three old failures, legacy preserved edits, and warnings scoped to the affected run. Full workspace typecheck, production build, Storybook build, and token gates passed. Browser-verified Storybook playtests passed for Historical Failures After Successful Save, Storage Limit, and Full Storage Run Warning. - Review follow-ups at `4e20c9fb2`: all 18 focused tests passed, including external Git directories, linked worktrees, symlinks, hardlinks, and distinct I/O failures alongside storage warnings. Server and UI typechecks, token gates, and the production build passed. - Storage warning regressions at `0724f3012`: all 33 directory tests and all five heartbeat-list tests passed, with no skips in their successful runs. They cover repeated runs while full, an already-over-quota saved folder, cleanup, warnings retained after unrelated save failures, and bounded warnings in large result JSON. Server typecheck passed after the final warning fixes. - Full workspace typecheck, production build, and token gates passed during this follow-up. Product E2E harness: 631 tests passed across 52 files; harness typecheck passed. Earlier native session/context and directory/legacy collection suites passed 537 tests; Runner unit/transport suites passed 329 tests. - **Real E2E at `0724f3012` (before this follow-up):** legacy local Codex and native Daytona Codex each passed six tasks, one server restart, seven independent assertions, and cleanup verification. Both prove browser-to-agent edits, agent-to-browser edits, nested/binary restoration, per-file last-sync-wins, a successful run after an oversized save rejection, and cleanup clearing the warning. - Native local Codex also passed the six-task quota flow before the final warning-retention fixes. That pass began at `918d1ed02` while the bounded-result warning fix was being edited, so it is not claimed as exact-final-head evidence. Its final-head rerun failed during embedded PostgreSQL bootstrap before any provider run: the macOS host had 87,365 of 87,381 SysV semaphores occupied. No unrelated services or kernel limits were changed. - The final-source report intentionally records **2/3 cells passed**, preserving the blocked native-local attempt: `tests/runner-e2e/results/agent-files-quota-final-20260928-report/`. Earlier failed attempts and provenance notes remain under `tests/runner-e2e/results/agent-files-quota-final-20260928-input/` and the original campaign directories. - Daytona used immutable image `ghcr.io/paperclipai/paperclip-daytona-runner@sha256:5643f0d801417cae3581833a1a3bc6715b325e028602738d2652c44cac5dc6bf` and its exact Linux runner binary. Controller source is `0724f3012`; image source is recorded separately. - Legacy-session compatibility and all three ACP Stop/resume browser regressions passed on the prior validated feature head `169fab46d5af21caa2269b4c1b29b69c933a6951`. They assert the same provider session is retained and interrupted writes are not replayed. Migration upgrade tests also passed earlier. - Nine interactive stories are under **Agents / Persistent files**, including **Full Storage Run Warning**. Its playtest and visual browser inspection passed; the warning states that runs continue and the editor remains available. - Prior-head checks on `4e20c9fb2`: 55 passed, two conditional jobs skipped, no failures or pending checks. All eight browser E2E shards and their aggregate passed. Fresh Greptile review is 5/5 with no findings; all review threads are resolved, the security scan passed, and GitHub reports no merge conflicts. - The broad local follow-up test run was interrupted after host semaphore exhaustion affected isolated PostgreSQL instances. It also encountered the existing macOS long-path fixture failure and two timeout failures. This is not a claim that the full local suite passed. Logs are retained; focused storage/warning tests passed. ## Risks - A later sync can overwrite an earlier edit to the same file, including a saved browser edit. There is no text merge or retained version. This is the intended last-sync-wins policy. - A save that exceeds a storage limit is rejected and its temporary copy is discarded. The run itself continues normally, and later runs restore the last saved files with a warning until cleanup. Transient sync failures get bounded retries. An I/O failure partway through a sync can leave some files updated; a failed receipt does not claim whole-folder success. - Larger folders increase copy time, network traffic, and temporary disk usage. Active runs still need working copies. Terminal runs do not accumulate archives. Operators must provision disk for agents and configured concurrency; these limits are not company-wide quotas. - A restored old native session remains instruction-only until a fresh session starts. Its original conflict fence and existing pending candidates remain compatible. - Provider processes close at the collection boundary. Conversation resume remains available, but warm process reuse is lost. - Backups must include the instance filesystem and database. External bundles keep their existing behavior until explicitly moved to managed storage. ## Model Used OpenAI Codex, GPT-6 family. The session does not expose a more specific model ID or context-window size. Reasoning, code execution, and browser tools assisted this change. Real provider E2E uses `gpt-5.6-sol`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Fry (Paperclip) <noreply@paperclip.ing>	2026-09-29 08:25:56 -05:00
Devin FoleyandPaperclip	53aad90b9e	fix: retry sandbox ACP input delivery after gateway failures (#14485) ... ## Thinking Path > - Paperclip coordinates agent work through execution adapters. > - Sandbox ACP sessions send ordered input through a remote file queue. > - A temporary provider 502 currently closes the session during an input upload. > - A lost response can occur after the sandbox has consumed the message, so a blind retry can duplicate input. > - This pull request retries gateway failures with the same sequence and drops consumed sequences at the receiver. > - The session can continue through a brief provider failure without repeating a tool call. ## Linked Issues or Issue Description **What happened?** A sandbox ACP run can fail with `ACP agent disconnected during request (connection_close, exit=null, signal=null)` when a provider input upload returns HTTP 502. The bridge destroys its local socket on the first failure and can discard the diagnostic before the proxy reads it. **Expected behavior** A temporary gateway failure should get a bounded retry. A lost response after successful delivery must not duplicate input or reorder later messages. Permanent failures must still close the session. **Steps to reproduce** 1. Run the real sandbox process bridge with an echo child and a local test runner. 2. Inject a provider 502 before preparation, after a chunk upload, or after final publication and consumption. 3. Send the next input message. Before this change, the connection closes instead of delivering it. Searched open and closed PRs for `ACP disconnect`, `bridge retry`, and `502 sandbox`. Related work: #13287 covers shutdown after bridge loss; #13793 covers large launch envelopes. This change covers ordered input delivery within a running legacy ACP session. ## What Changed - Retry input uploads up to three times for recognized Daytona and Cloudflare HTTP 502, 503, and 504 diagnostics, with 250 ms and 500 ms delays. - Give each upload separate temporary paths and discard already-consumed input sequences, including late publication from an earlier attempt. Clean failed attempts in the background without removing a published message or another attempt’s files. Cleanup cannot delay retries or shutdown. - Keep later input behind the retry. Stop queued input on permanent failure and flush a fixed diagnostic before closing the socket. Neither failure-diagnostic persistence nor shutdown-warning persistence can block teardown. - Add real-process regression tests for lost responses, late publication, retry exhaustion, immediate permanent failure, and diagnostic redaction. - Give accepted run-log file appends up to three seconds to drain before finalization computes the size, hash, and durable copy. Close the run handle to later appends. This waits only for file writes, independently of later DB progress or live-event persistence. If writes remain stalled, return null size/hash metadata and skip the final durable copy so the run can settle. Late writes cannot restart mirroring. - Preserve legacy comment attribution when final log size is unknown by reading existing entries within the unchanged 2 MB scan limit. Storage errors or a three-second read deadline return the evidence already read instead of failing the comment listing; pagination stops at the deadline. The deadline requests cancellation of the underlying local stream or S3 HEAD, GET, and response stream. A separate response timeout returns partial evidence even when filesystem I/O delays cancellation; late reads cannot append evidence or start another page. Each listing retains its existing batches of eight reads, without a shared admission cap that skips readable logs under contention. - Document the retry and log-finalization boundaries in the development guide. ## Verification - Final commit `347daa564b`: [Linux CI](https://github.com/paperclipai/paperclip/actions/runs/36506995168/attempts/2) passed. Greptile Apex review 13 scored this commit 5/5 with no new findings; all 12 review threads are resolved. - The final CI run initially hit a Cursor test timeout and four Discord credential-lock contention failures. All five cases passed in isolation. The two failed shards and their aggregate gate passed on retry without a code change. Those intermittent failures are not claimed fixed by this PR. - `pnpm --filter @paperclipai/adapter-utils typecheck` passed. - `pnpm exec vitest run packages/adapter-utils/src/execution-target-stdin-race.test.ts packages/adapter-utils/src/execution-target-sandbox.test.ts packages/adapter-utils/src/sandbox-callback-bridge.test.ts`: 262 tests passed on the final implementation, including 21 new regressions. The original three fault-injection cases failed before the fix. - The regressions cover failed and indefinitely stalled cleanup, Cloudflare gateway responses and retry exhaustion, permanent errors that must not retry, and teardown while failure logging remains indefinitely stalled. Seven Apex regression cases failed before the review fixes. Adapter-utils typecheck and build passed again after the final review change. - `pnpm exec vitest run server/src/services/run-log-store.test.ts server/src/services/run-log-store-cancellation.test.ts`: all 25 tests passed, including four new regressions that failed before the finalization fix. They cover delayed and failed appends, late-write admission, agreement between the local bytes/summary/durable copy, and a stalled append that exhausts the three-second budget. The timeout case verifies unknown metadata, no final upload, and no mirror restart after late completion. New cancellation tests use the real AWS SDK against a local HTTP server. They verify that stalled HEAD, GET, and response-body connections close on abort and that a subsequent read succeeds. Local range and already-aborted read cases also pass. - `pnpm exec vitest run server/src/__tests__/issues-service.test.ts -t 'readIssueCommentRunLogText|deriveIssueCommentRunLogAttribution'`: 14 targeted tests passed. The null-size reader case, both storage-error cases, the stalled-read case, and the cancellation/concurrent-listing cases failed before their fixes. The new regressions verify that timed-out reads are cancelled, subsequent listings recover, and two concurrent listings both retain their attribution markers. A read that ignores cancellation still returns partial evidence at three seconds and cannot resume pagination when it finishes; this regression failed before the response-timeout fix. - `pnpm --filter @paperclipai/server typecheck` and `pnpm --filter @paperclipai/server build` passed after the response-timeout change. - Full `pnpm -r typecheck` and `pnpm build` passed earlier in this PR; the affected packages were rechecked after review fixes. - Full local `pnpm test:run` failed in the general-server group: 511 files passed, 40 failed, and 158 were skipped. Failures include embedded PostgreSQL initialization, read-only cache directory renames, a macOS long-path fixture, and a workspace exposure assertion. The PostgreSQL, cache-permission, and long-path failures also reproduce with both changed implementation files restored to baseline commit `24c58e479a`. The exposure suite passes in isolation both on baseline and the fixed branch (28 passed, 3 skipped). CI runs the full suite on Linux. Later local test groups were not reached. - An earlier CI run hit the Telegram retry-timing failure fixed upstream in #14501. The branch includes that master fix. The selected recovery test passed against a fresh, migrated PostgreSQL 16 database. The embedded PostgreSQL runner is unavailable on this Mac; the isolated database was stopped and removed afterward. - No live agent turn was replayed. The tests use local child processes and injected provider failures. ## Risks Retries are restricted to recognized Daytona SDK and Cloudflare bridge gateway-error messages, which survive plugin RPC serialization. Other errors fail immediately. Temporary upload paths are now unique for all command-managed queue writes. Receiver sequence checks prevent duplicate input; retries do not restart an agent turn. Cleanup and failure logging are nonblocking and best effort; session teardown remains the final cleanup boundary. Log finalization now drains accepted local file writes for at most three seconds and ignores later appends on the closed run handle. A timeout leaves final size/hash unknown and skips the final durable upload; an existing partial mirror may remain available, but it is not claimed as a verified final snapshot. It does not wait for later DB progress or live-event persistence. Optional attribution keeps partial evidence when a read fails or times out. Cancellation closes S3 requests and response streams. Local filesystem I/O may finish after the caller deadline, but a late read cannot change the returned evidence or continue pagination. Later listings can retry after storage recovers. There is no schema, authentication, or permission change. Revert this commit to restore the previous behavior. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository inspection, code editing, and local test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally; targeted tests pass and full-suite limitations are documented above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-28 19:15:33 -07:00
Devin FoleyandPaperclip	ea371b9684	fix: retain project defaults in partial workspace overrides (#14502) ... ## Thinking Path > - Paperclip manages AI agents and their work. > - Project workspace policies define how isolated worktrees are set up. > - Tasks can override a branch without providing every setup field. > - The resolver currently replaces the entire project strategy with that partial override. > - Losing an explicit setup command can run the repository fallback script and block the task. > - This pull request keeps enabled project defaults when the task uses the same strategy type. ## Linked Issues or Issue Description **What happened?** A project uses `git_worktree` with `provisionCommand: "true"`. A task overrides only `baseRef`. The resolver drops the command. Worktree creation then invokes `scripts/provision-worktree.sh`, which can fail because its required setup is absent. **Expected behavior** A branch override keeps the project's provision, runtime provision, and teardown commands unless the task explicitly overrides them. A different strategy type must not inherit those commands. **Steps to reproduce** Configure the project with an enabled `git_worktree` strategy and `provisionCommand: "true"`. Give the task an isolated workspace with a `git_worktree` strategy and a different `baseRef`. Add a failing repository fallback provisioner. Before this change, worktree creation invokes that script. After this change, it uses the project's explicit command and succeeds. Related: #4968 concerns agent strategy and working-directory fallback. #13903 concerns gated API fields and reusable-workspace updates. #11091 concerns provision hooks on workspace reuse. None fixes partial task overrides discarding project defaults. ## What Changed - Merge a partial task strategy over the enabled project's strategy only when their types match. - Preserve explicit null values when parsing nullable strategy fields, so they can clear project values. - Keep explicit empty-string overrides and agent fallback behavior. - Exclude disabled project strategies and avoid an inherited branch template when a task pins an existing branch. - Add policy regression coverage and a real Git worktree test with a failing fallback script. - Document inheritance, explicit clearing, and no-op provisioning in the development guide. ## Verification - Policy regression: eight failures before the fix; all 41 policy tests pass after it. - Real worktree regression: passes and creates a worktree using the task's base branch without invoking the failing fallback provisioner. - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - `pnpm test:run`: general-server phase completed with 13,873 passed, 86 skipped, and 14 failures in unchanged macOS skills-cache and Git long-path tests. The same failures reproduce on unmodified base code. The command stops at that phase, so no full local pass is claimed. - CI initially failed the existing Telegram subscription recovery test on a 15-second timeout. The separate fix and investigation are in #14501. A serialized job also lost its runner; GitHub reported lost communication, and that job was rerun without source changes. All 52 final-commit checks pass, with two intentional skips. Greptile is 5/5, with no unresolved comments or merge conflicts. The chat shard passed on one unchanged rerun. The timeout cause remains unproven; #14501 adds phase diagnostics for a recurrence. ## Risks Tasks that specify a partial strategy now retain the project's omitted fields, including setup and teardown hooks. This is the intended behavior change. Inheritance requires an enabled project policy and matching strategy types. Explicit task values still win. Null and empty commands restore existing runtime defaults; they do not guarantee that no script runs. Use `"true"` for an explicit no-op provision command. No migration, live configuration change, or task replay is included. ## Model Used OpenAI GPT-6 (Codex), with reasoning, terminal tools, and code execution. The context window size is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes:` / `Closes` / `Refs` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run focused tests locally and they pass; full-suite status is recorded above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-28 18:09:20 -07:00
Devin FoleyandPaperclip	90119181e7	test: control Telegram subscription retry timing (#14501) ... ## Thinking Path > - Paperclip manages AI agents and their work. > - Telegram delivery recovers subscription changes after a restart. > - The recovery test leaves a failed action on the real one-second retry timer. > - A slow runner can cross that deadline before the test checks that no retry occurred. > - This pull request holds the fixture deadline until the explicit restart transition. > - The test still checks that recovery uses fresh provider options. ## Linked Issues or Issue Description **What happened?** The Telegram subscription recovery test expected one `setWebhook` request but saw two. A 1.5-second delay after the first failed request reproduces the failure. **Expected behavior** The test controls when the failed request becomes eligible for retry. Host speed does not change its result. **Steps to reproduce** Run the test named `retries an unknown subscription mutation after restart` with a 1.5-second delay after the first failed-action assertion. The old fixture retries too early. The updated fixture passes with the same delay. Related: #13952 fixes a separate Telegram fixture cleanup problem. This change addresses retry timing. ## What Changed - Set the stored retry deadline to 2099 before the pre-restart assertions. - Keep the existing explicit epoch deadline after restart and all provider request assertions. - Report the current test phase only when this test fails, to diagnose an observed intermittent CI timeout. - Leave production retry code unchanged. Temporary delay and per-step console tracing are not included. ## Verification - Delayed regression: failed before the change with two requests instead of one; passed after the change. - Focused Telegram durable private draft Stop group: 34 tests passed. - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - Full local chat shard: 355 tests passed twice. - `pnpm test:run`: general-server phase completed with 13,861 passed, 86 skipped, and 14 failures in unchanged macOS skills-cache and Git long-path tests. The same failures reproduce on unmodified base code. The command stops at that phase, so no full local pass is claimed. - CI exposed a separate 15-second timeout. A diagnostic run passed all 355 shard tests, with the affected test completing in under one second. Its cause remains unproven. Normal step logging is removed; a failure-only phase report remains for a recurrence. The final commit also passes the 355-test chat shard, and Greptile rates it 5/5. All 52 final-commit checks pass, with two intentional skips. There are no unresolved review comments or merge conflicts. ## Risks Low risk. This changes only the fixture deadline. It does not disable a test, extend a timeout, or change production retry behavior. Existing assertions still verify the failed action, the pending state, restart recovery, and fresh provider options. The intermittent CI timeout is not claimed fixed; phase diagnostics narrow the next occurrence without changing the timeout. No documentation change is needed for a test fixture correction. ## Model Used OpenAI GPT-6 (Codex), with reasoning, terminal tools, and code execution. The context window size is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes:` / `Closes` / `Refs` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run focused tests locally and they pass; full-suite status is recorded above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes, or explained why none is needed - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-28 18:08:57 -07:00
Devin FoleyandPaperclip	ad1f7e98ea	fix: retain diagnostic reasons for native runner failures (#14481) ... Retain bounded reasons for runner identity, harness recovery, and provider-pack read failures. Preserve existing ownership and cleanup proofs and compatibility with receipt-gated chat recovery. Verified with executor, recovery, diagnostic privacy, typecheck, build, and full CI checks. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-09-28 16:16:19 -07:00
Devin FoleyandPaperclip	4f6cf5b3ff	fix: prevent run identity locks from blocking audit checks (#14478) ... Use NO KEY UPDATE for identity locks so audit foreign-key checks can proceed while identity writers remain serialized. Preserve task-before-run ordering, company scoping, and foreign keys. Verified with PostgreSQL concurrency regressions, focused tests, typecheck, build, and full CI. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-09-28 16:16:05 -07:00
3 people	0be2afcca6	feat(ui): improve task composer controls and pending input (#14322) ... ## Thinking Path > - Paperclip lets operators assign tasks to AI agents and review their work. > - The task composer controls the next message and its assigned agent. > - Operators needed a way to choose that agent's model and effort without leaving the composer. > - The old mode selector, upload button, and input cards made the mobile composer crowded and hid normal messaging during a pending decision. > - Harnesses publish different model and effort capabilities, so the picker must follow the selected agent. > - This pull request adds one responsive composer flow, keeps pending cards visible above it, and protects Codex ACP authentication in the local test path. > - Operators can choose run settings, send a message, and answer a pending card as separate actions. ## Linked Issues or Issue Description **Subsystem affected** Task composer UI, issue thread interactions, Codex ACP credential handling, and Storybook. **Problem or motivation** The composer did not expose model or effort for the selected agent. Mobile actions wrapped poorly. Pending questions and confirmations replaced the composer. A local Codex ACP test could also reuse host authentication after the managed key was removed. **Proposed solution** Put assignee search, model search, exact model IDs, effort, and fast mode in one picker. Use a mobile dialog. Replace the direct-upload plus action and separate mode selector with an Add menu and removable Plan or Ask chips. Place pending interaction cards above the usable composer. Keep these cards pending after an ordinary message unless their creator asks for comment superseding. Replace managed ACP auth files atomically and isolate the test key from host credentials. **Roadmap alignment** ROADMAP.md does not list an overlapping composer milestone. This change improves the existing task and review flows. ## What Changed - Added the combined assignee, model, and effort picker to both task composers. Search matches agent name, role, and harness. The server uses a curated Codex list by default and honors instance-declared models. Manual IDs remain available. - Added an effort slider for known model capabilities, a conditional Codex fast control, and reset. The picker opens in a modal on mobile. - Added the Add menu for files, supported goals, Plan mode, and Ask mode. Plan and Ask are exclusive removable chips. Keyboard mode cycling remains available. - Adjusted mobile spacing, avatars, wrapping, and Send placement. Removed the composer divider. - Moved pending question, confirmation, review, and related cards above the composer. Ordinary comments now leave question and confirmation cards pending by default. The onboarding prompt retains explicit comment superseding. - Updated the Storybook composer group with responsive states and the production picker. Added UI, service, route, and browser regression coverage. - Isolated Codex ACP API-key authentication, skipped subscription auth merge and shared-home copy-back for remote API-key runs, and replaced the managed auth file atomically. ## Verification - `pnpm -r typecheck` — passed on the final local head. - `pnpm check:token-gates` — passed on the final local head. - `pnpm exec vitest run server/src/__tests__/adapter-models.test.ts ui/src/components/task-chat/ComposerRunSettingsPicker.test.tsx` — 31 tests passed, including role and harness search, declared Codex models, and filtering general OpenAI models. - `pnpm exec vitest run server/src/__tests__/issue-thread-interactions-service.test.ts` — 74 tests passed. - `pnpm exec vitest run packages/adapters/codex-local/src/server/acp.test.ts` — 42 tests passed, including remote API-key copy-back isolation. - `pnpm test:run` — attempted locally; the embedded PostgreSQL test database could not initialize on macOS. The isolated `heartbeat-run-event-sequencing` suite reproduced that environment failure. GitHub CI runs the full test matrix for this head. - `pnpm build` — passed on the final head. `pnpm build-storybook` passed after the last UI change; only server code, tests, and docs changed afterward. - Live local test drive — Codex ACP ran a task with a managed API key. The test agent was restored to its default ACP configuration afterward. - Review the interactive stories under the top-level Composer group with `pnpm storybook`. Check a narrow desktop width and mobile Plan, Ask, picker, and pending-question states. ## Risks - A pending card stays open when an ordinary comment changes the discussion. Its creator can set `supersedeOnUserComment: true` when a new comment should replace it. - Model and effort overrides persist on the task until reset or changed. An unlisted manual model ID may fail when the provider runs it. - Some harness catalogs do not report effort support. The picker hides effort for those models. - No database migration is required. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-6 via Codex. This runtime does not expose the exact model ID or context window to the task. The model used code execution and browser tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: OpenAI Codex <codex@openai.com>	2026-09-28 23:02:39 +00:00
DottaandPaperclip	18e8c121d9	fix(runner): include Grok support in public installs with sandbox prerequisites (#14024) ... ## Thinking Path > - Paperclip manages agents through a shared native runner. > - Built-in harness support should ship with Paperclip's public distribution. > - Grok already speaks ACP; it does not require a new public bridge package. > - Sandbox provisioning owns the native executable and its pinned version. > - The runner must verify that prerequisite without downloading it during npm installation. > - This change separates built-in launcher identity from external runtime identity. > - Clean npm installation and live staging checks verify the distribution boundary. ## Linked Issues or Issue Description Refs #13882, #13973, #13977, #13979. This follow-up now targets master after #13882 was squash-merged. It replaces the private `@paperclipai/grok-acp` workspace package with runner-owned assets. Current master is included so the branch also contains the merged scheduler, complete-event capture, and durable cleanup fixes. ## What Changed - Ship Grok launcher and qualification metadata inside the runner's compiled output and the public server's vendored runner tree. - Remove the separate Grok npm package and all package-manager install hooks for this runtime. - Require the checksum-verified Grok Build 1.0.13 binary at `/opt/paperclip/providers/grok/1.0.13/grok` in the selected execution environment. Provision it explicitly in the Daytona image and CI setup. - Keep native binaries outside the provider pack. Bind the built-in launcher into the pack manifest. - Preserve executable leases, descriptor-backed startup, credential fences, permissions, and exact ACP model admission. - Use `builtin:grok-acp` and `native:grok` as profile identities. Historical package-profile sessions fail closed on resume rather than being silently reinterpreted. - Resolve built-in assets from the authenticated sidecar location, including public server npm layouts. Keep the controller path out of provider environments. - Add clean npm tarball installation verification to the existing trusted canary CI job and the admitted manual EC2 verification path. It stages a unified release version and runs npm lifecycle scripts, then verifies missing-prerequisite rejection and admission after separate provisioning without credentials or inference. - Include the controller-owned provider pack in stamped Cloud images. Unstamped local images omit the pack and remain usable; remote ACPX requires full source provenance. - Correct CLI approval-page metadata for an already authenticated Cloud board user; approval authorization remains unchanged. - Honor explicit native-runner enablement in the Cloud agent picker and direct setup page, keeping the flag disabled by default. - Allow selecting the execution environment before connecting credentials. Include Grok in the existing authenticated hello-probe flow, targeting its pinned native prerequisite for runner setup. - Recover an existing subscription sign-in conflict through an explicit cancel-and-retry action, serialized after cancellation succeeds. - Preserve the selected ACPX harness before normalizing config fields, so new Grok agents use the Grok default model. - Keep the credential-free Cloud provider pack root-owned and readable after runtime UID remapping; verify manifest and referenced asset access under an unrelated unprivileged UID during image builds. - Archive prior failover backups alongside explicitly replaced harness state, preserving evidence while preventing stale backups from blocking a fresh replacement. - Update Daytona image content inputs and contract tests for the built-in assets and explicit provisioner. - Document and regression-test the shared `approve-all` default for Grok setup, saved configuration, and native execution. Explicitly saved restrictions remain unchanged. ## Verification Current merge-repair head `df09eb3e1a619430ad8419a0ee9aedd486689b05` incorporates master `f1a394bd30cb56fb9e479f98b9f50176fe921858` after the base PR was squash-merged. All 12 conflicts came from incoming files identical to the tested pre-squash base. The final tree exactly matches a three-way merge using that original base, preserving built-in Grok distribution and removal of the obsolete private package. All 252 focused runner/UI tests, six npm-isolation tests, and token gates pass. Fresh exact-head Greptile review is 5/5 with no outstanding findings; security scans and EC2 native compilation pass. All current-head CI is green: 56 successful checks/statuses and four intentional skips ([run 36468768035](https://github.com/paperclipai/paperclip/actions/runs/36468768035)). The repository owner explicitly authorized bypassing code-owner approval after all checks passed; no CI checks or repository protection settings are bypassed or changed. The only remaining PR was removed from the completed stack metadata to permit native auto-merge. Earlier integration head `78cb306ecc41b5c96577c26c1d89153b0ef865a1` includes master `3447609d2247e75e55d91493dda91a608364f672` (2026-09-28). Two master advances during verification overlapped the eval catalog; the final merge preserves Grok qualification, completion updates, and bounded API-response reading in all 348 cells. All 77 focused catalog/eval/workflow tests pass. Both native stack layers (#14397) are mergeable, and both exact-head Greptile reviews are 5/5 with successful security scans and no unresolved review threads. All current-head CI is green: 56 successful checks/statuses and four intentional skips ([CI attempts](https://github.com/paperclipai/paperclip/actions/runs/36447124691)). The initial attempt lost two EC2 runners to shutdown signals and stalled a third shard during dependency preparation; all three passed the same-commit failed-job-only retry. Trunk code-owner requirements remain enforced. The review summary’s non-blocking saved-asset offset classification note concerns code already merged in #14301; those runtime files are identical to master and outside this stack’s diff. Historical live evidence below retains its original source revisions. [Final public npm verification](https://github.com/paperclipai/paperclip/actions/runs/36445542764) passed on `76ea70cd4d13786a042af9df82f0fd7a8c85ae30`: 17 public packages, an executed offline lifecycle sentinel, unchanged consumer lock, built-in launcher, missing-prerequisite rejection, and verified separately provisioned binary/command lease. Provisioning and cleanup require no host privilege elevation; only the positive probe mounts the temporary native binary read-only. The verifier is unchanged by the final master merge. All six isolation tests and an offline npm smoke test pass. The prior head had 56 green CI checks and a 5/5 review after two unchanged tests timed out and passed a failed-job-only retry ([CI attempts](https://github.com/paperclipai/paperclip/actions/runs/36444597313)). All 56 recovery-display/lineage tests pass; re-review cleared the already-covered missed-retry concern. Earlier EC2 failures remain retained: [npm lockfile rejection](https://github.com/paperclipai/paperclip/actions/runs/36436311203), [missing compiler in the slim image](https://github.com/paperclipai/paperclip/actions/runs/36440210984), and the aggregate 15-minute test timeouts in those broad runs. Both broad attempts passed typecheck, token gates, Product E2E type/unit checks and build. The focused EC2 lane preserves the existing trusted-actor and immutable-source gates. Earlier documentation/test checkpoint `ff244c4fd78a7ede5a3e00efe09f475f133ef33e` leaves runtime behavior unchanged. 154 focused tests pass across configuration building, native provider resolution, permission policy, credentials, UI configuration, and new-agent setup (including both Grok auth modes); token gates pass. All fresh CI is green for this head: 56 successful checks/statuses and two intentional skips ([run 36367065119](https://github.com/paperclipai/paperclip/actions/runs/36367065119)). Greptile is 5/5 with no new findings. Grok already inherits the shared `approve-all` default, so unattended setup requires no manual permission change. Runtime head `bb5a9307991f1ac567b781970ef11b39d518e19b` fixes a final staging continuation failure before provider startup: explicit replacement archived the old harness but left its failover backups active, which caused `runner_harness_state_mismatch`. The regression fails before the fix and passes after it; all eight adjacent recovery-safety cases also pass. Old backups remain inspectable inside the continuity archive. All fresh CI is green at this head ([run 36360839248](https://github.com/paperclipai/paperclip/actions/runs/36360839248)), with a 5/5 review. One unrelated Cursor test timed out in the initial server shard; the same-commit failed-job rerun passed, and both attempts are retained. Staging deployment is confirmed healthy on this revision. The controller image is `ghcr.io/paperclipai/paperclip@sha256:6ad91c487910ccd2596ff7aed0a3a3ea5233d12b51b83cd6e1402237749b9673`. The final browser-created staging task passed on this exact revision with API authentication: context read → structured human question → controller restart → answer submission → same native provider session resumed → document saved → task Done. The two turns took approximately 119s and 77s. The actual write receipt was applied, and the saved document has exactly one revision containing the selected answer and requested marker. Usage and cost were not reported. [Controller image build](https://github.com/paperclipai/paperclip/actions/runs/36360889243). - Previous integration head `a44f7dbb6b6f77cd9ed893756ca453307f281e5f`: all CI green (53 successful checks/statuses, two intentional skips), including repository typecheck/build/tests, native Runner tests, browser shards, and canary installation checks. [CI run 36358672529](https://github.com/paperclipai/paperclip/actions/runs/36358672529). Greptile is 5/5 with no unresolved findings. - Focused checks cover Grok credentials, executable admission, launcher assets, provider-pack paths/permissions, workflow contracts, setup defaults, CLI authorization, and subscription conflict recovery. All 39 protocol definitions validate. Final integration checks pass 124 catalog/evidence/cache tests and nine project-form tests; token gates pass. Some local dependency checks could not load the stale installed dependency tree; the corresponding fresh EC2 checks pass. - Clean public npm installation passed on EC2 at `8b172ebcf8e02e30662d830c00f3961e3bd459ec` ([run 36164964900](https://github.com/paperclipai/paperclip/actions/runs/36164964900)): 17 unified-version packages, lifecycle scripts enabled, built-in launcher present, no separate Grok package or npm-downloaded binary, missing prerequisite rejected, separately provisioned native executable and command lease verified. No credentials or inference were used. Subsequent changes preserve this npm asset layout. - The immutable Daytona prerequisite image is `ghcr.io/paperclipai/paperclip-daytona-runner@sha256:98957d5be0ac774d086b6402b5849e8e6356fec70fb8c09fca6eb4ed6de918e0`, built from `5a2db471f3ddabe77f9f80e76ed27f996cb97fba`. The previous Cloud controller image was `ghcr.io/paperclipai/paperclip@sha256:fd914e1ab1e45f741e8e078ff452d16f082d7ac05f9b4b3506d3a3c64150d204`, built from `a44f7dbb6b6f77cd9ed893756ca453307f281e5f`; it is superseded by the latest image above. Its EC2 build verified provider-pack access under an unrelated unprivileged UID. - Browser staging at `40f898bc4cba73c1dff4e6344a3983ba0fb247ef` passed full Grok onboarding with the correct `grok-4.7` model, saved credential delivery, and pinned Daytona execution. A browser-created task read context and asked the structured human question. After a controller restart, answering the persisted question resumed the same native provider session, saved the requested document, and completed the task. Actual tool outcomes and durable state agree: one question and one document revision. The two successful turns took 42.7s and 63.1s; usage and cost were not reported. - Restricted policy returned the expected `approval_required` outcome. Functional staging tests explicitly selected `approve-all`; controller authorization and governed approvals remain enforced. Temporary board CLI access was revoked and verified rejected (HTTP 401), and the disposable onboarding agent was paused. Failures remain retained: the pre-fix continuation failure (its task remains blocked; the passing final task is fresh), the original Cloud provider-pack permission failure, the expected restricted-policy denial, the superseded npm staging failure, and an earlier monolithic CI infrastructure timeout. Browser CI exposed a project alias/form race; the final stack uses master's stronger draft-preservation fix and all browser shards pass. Historical full subscription/API protocol and Product rosters retain their original source revisions and do not qualify this packaging revision. No local Docker or Rust build was used. ## Risks The branch includes master’s draft-preservation fix for project URL aliases. It keeps the same project’s edit form mounted and clears prior data when the project or company changes. Custom sandboxes and local execution hosts must provision the pinned binary before Grok starts. Missing, changed, unsupported-platform, and symlinked executables fail admission. The new builtin profile cannot resume sessions created with the former private-package profile. Existing Claude/Codex npm bridge profiles retain their package pins. Grok restricted modes preserve the selected policy but cannot automatically admit Paperclip calls: ACP permission metadata does not independently bind tool authority, so those calls stop with `approval_required`. New Grok configurations default to `approve-all`, including API configurations that omit the mode. Existing explicitly restricted configurations remain restricted; controller authorization and governed approvals remain enforced. ## Model Used OpenAI GPT-6 through Codex, with tool use and code execution. The exact serving model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-28 14:54:44 -05:00
DottaandPaperclip	992f720262	fix: make runner task context ownership explicit (#13753) ... <!-- Write all pull request text in Simplified Technical English (ASD-STE100). --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task descriptions, comments, continuation data, skills, and execution rules enter several agent adapters. > - The same source can be rendered by more than one automatic input carrier. > - Failed resumes can also rebuild input from stale or compact context. > - This pull request gives each Paperclip-owned source one delivery owner and preserves the required transport boundaries. > - It adds deterministic adapter, interaction, runner, and browser tests for these boundaries. > - The benefit is more predictable context delivery with explicit evidence for later live qualification. ## Linked Issues or Issue Description Related: #13144 removes a duplicate environment payload and bounds wake lists. Related: #11360 addresses Hermes resume behavior. This pull request preserves compatible active-session formats while repairing context ownership and stale question creation. **What happened?** Task descriptions and comments could enter more than one automatic context block. Native transports could wrap a complete model input in a second task envelope. Some legacy and gateway adapters could omit the owned assignment on ordinary tasks or rebuild a failed resume with stale compact context. A continuation could also request a question after newer human comments had arrived. **Expected behavior** Each task or comment source has one automatic model-facing owner. Distinct comment IDs and repeated wording remain distinct. Fresh fallback attempts rebuild the required full context. A question request is rejected when newer queued human direction makes it stale. Harness access policy remains owned by execution configuration. **Steps to reproduce** 1. Build a task with a description and current comments. 2. Capture the actual adapter or runner input. 3. Compare source ownership and task-envelope nesting. 4. Queue a human comment before a continuation requests a question. 5. Trigger a failed resume and inspect the fresh retry input. 6. Run the focused adapter, interaction, runner, and browser checks. ## What Changed - Add shared prompt-section selection at the provider-attempt boundary. - Deliver owned assignment context through native, legacy CLI, ACP, gateway, cloud, Pi, Kimi, Grok, Gemini, OpenCode, Cursor, OpenClaw, and Hermes paths. - Rebuild full or compact context after resume recovery changes the attempt. Add native and Claude ACP tests of actual recovery requests. - Preserve custom templates, loaded instruction files, execution policies, and older active-session formats. - Record continuation source metadata and reject stale question creation under the issue-row lock. - Add explicit Product E2E context-integrity profiles, prerequisite gates, credential-isolation checks, and report fixtures. - Bypass service-worker forwarding for same-origin Vite development modules. A real Chromium test fails with resource exhaustion before the repair and passes after it. Production asset caching keeps its existing policy. - Add browser diagnostics and service-worker module-loading regressions. - Add an explicit zero-retry eval option. The default retry behavior remains unchanged. Each campaign records its effective policy. - Remove the model-facing working-directory sentence from four prompt builders. Existing workspace, sandbox, permission, and custom-template configuration remains unchanged. - Align the everyday workflow assertion with the current 47-entry catalog. Compared with current upstream master, the branch carries the context-ownership implementation and its tests, the explicit context-integrity catalog and evidence harness, and the focused browser regression checks. ## Verification **Merge assessment:** focused regression evidence supports merge. This is not full completion of the original broad qualification matrix. The maintainer has authorized merge after fresh verification of the master integration. - Current head: `bbd52f82114eabf09bc7b1a7e97d54a5b43bbc00`. This integrates current master `2f585ef26a1814fa209715242d1ca791b63e4c4e`. All 14 conflicts are resolved. Cancellation checks, workspace finalization, native Grok support, and both sets of tests are retained. - Current-head Greptile: **5/5**, with no blocking findings. The review names this exact commit. All **59 reported checks are terminal: 55 successful, 4 skipped, zero pending or failing**. This includes the full root general and serialized suites, separate runner checks, typecheck, build, canary, browser E2E, Docker, and security checks. The successful legacy security status is included in that total. - After integration: workspace typecheck and full build passed. Separate runner checks passed: **2,160 TypeScript tests (10 skipped), 582 Rust tests, and 39 preparation checks**. Other passing checks include 621 Product E2E harness units, 376 focused shared/adapter tests, 160 real-database/API tests, 86 Hermes tests, 18 browser-support checks, and Product E2E typechecking. The complete root suite passed in CI. The duplicate local monolithic root run was stopped after that CI result; it is not counted as a completed local pass. - New native recovery coverage retains full assignment, completion contract, and explicit skill selection after safe replacement, for old and prepared input formats. Full native session test file: **136/136 passed**. - New Claude ACP coverage captures actual fresh, resumed, and missing-session fallback requests. It verifies one assignment copy, comment order, identical text under distinct comment IDs, and full fallback context. Full file: **33/33 passed**. Both affected TypeScript checks passed. - Existing deterministic tests cover source revisions, approval and trust boundaries, completion validation, custom templates, compatible sessions, standalone driver wrapping, and maintained adapter transport requests. - Provider-free browser support: **17/17 passed** after the master merge. Service-worker unit tests: **33/33 passed**. The module-overload regression failed before the repair and passed after it in real Chromium. ### Fresh live comparisons The new batch ran exactly four Product E2E attempts. **All four passed on the first attempt; no retries.** Each has six terminal matchers plus the existing browser lifecycle and invariant checks. | Exact case ID | Control | Candidate | |---|---|---| | `core-compatibility.runner-codex.local.plan-revise-accept` | Passed | Passed | | `local-session-integrity.runner-acpx-claude.local.structured-question-restart-resume` | Passed | Passed | The plan case checks a revised canonical plan and revision-bound approval before completion. The question case restarts the server before submitting the answer, then verifies the continuation completes. Control source is `dfa4e1bda8d50a1a01746603251a9128dbe9d0d6`. Candidate source is `79fcdb5dece501d28064ea9da306603881b46f0c`. They use identical frozen definitions and provider versions: Codex `0.156.0` with `gpt-5.6-sol`; ACPX `0.13.1` / Claude ACP `0.73.0` with `claude-sonnet-5`. The September 24 head added master browser recovery and test-only changes. The September 28 head also integrates newer master changes, including cancellation, workspace finalization, and native Grok. These are frozen-source live results, not exact-head live runs. The candidate received one description copy where the control initially received three. The submitted initial plan envelopes were 7,969 versus 19,097 characters. Question envelopes were 7,592 versus 18,919. These are structural measurements, not whole-provider token or dollar savings. ### Earlier evidence and failed attempts - The preceding fresh batch has four effective passing pairs: OpenCode comment continuation and assigned skill, native Codex comment continuation, and native Claude comment continuation. It retains **11 attempts: eight passed and three failed**. - Original failures remain recorded: missing local PostgreSQL library links before task creation; host-sleep cleanup after task/page checks passed; and a Claude **control** session-open rejection before a model turn. Setup was repaired identically on both worktrees. The permitted unchanged infrastructure retries passed. The underlying Claude provider startup error was not retained and remains unknown. - Older R2 retains **17 passes and one failure** across 18 attempts, including eight both-pass native/legacy Codex/Claude pairs. Its OpenCode blank-page failure led to the service-worker repair. R2 is historical evidence: master changed the native fixed prompt and removed duplicate wake environment data afterward. - The September 24 CI run initially failed one unrelated preview readiness test (`ECONNREFUSED` on its local fixture). Its test and production code match master. Isolated local verification passed **28 tests, 3 skipped**. One unchanged CI retry passed the full shard: **831 passed, 1 skipped**, including all **31 preview-exposure tests**. The aggregate CI gate passed afterward. The precise startup cause remains unknown; a port race is a hypothesis, not a proved cause. ### Limits The original wider profile/workflow matrix, repeated trials, and remote Daytona qualification are incomplete. These results support a focused merge recommendation, not statistical equivalence or universal harness qualification. Some usage receipts are missing in both variants, so no token or dollar savings are claimed. The $500 ceiling was preserved using conservative allowances; failed attempts and unknown charges remain in the ledger. Reproduce the focused additions with `pnpm exec vitest run packages/adapters/claude-local/src/server/acp.test.ts` and `pnpm --filter @paperclipai/paperclip-runner exec vitest run src/native-session-runtime.test.ts`. Full checks use `pnpm -r typecheck`, `pnpm test:run`, `pnpm build`, and the separate runner checks. Paid evals require the frozen definitions, profiles, and credentials; do not use `--all` as a substitute for the selected cases. ## Risks - Context placement changes can affect model behavior. Deterministic checks cover the selected paths, but live qualification remains incomplete. - The stale-question guard can reject a request when queued human comments arrived during the run. This is intended. - New stored inputs and model envelopes retain compatibility readers for older active sessions. - Custom templates may intentionally repeat content. - Removing a model-facing working-directory sentence does not change filesystem, command, sandbox, or permission configuration. - The worker bypass applies only to same-origin development module paths. Cache-policy tests preserve private-response handling and production asset caching. Mounted HTTP fixture changes remain test-only. - This PR does not claim measured token savings or statistical equivalence across every harness. ## Model Used OpenAI Codex, exact model gpt-6-astra, with repository tools and code execution. Bounded supporting work used gpt-5.6-luna and gpt-6-luna. The serving context-window size is not exposed in this task. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR using the required issue fields - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run the focused local checks and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect these changes - [x] I have considered and documented risks above - [x] All current-head Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups for the current head - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-28 14:49:14 -05:00
DottaandPaperclip	f1a394bd30	feat(runner): add Grok Build through native ACP (#13882) ... ## Thinking Path > - Paperclip manages AI agents and governs their work. > - Its native runner uses structured provider protocols for sessions and tools. > - Grok Build supports ACP over stdio, but the runner did not expose it. > - Native execution requires company-scoped credentials, verified identities, and permission gates. > - This change adds Grok through ACPX for local and Daytona execution. > - Subscription login and explicit API-key execution have separate credential paths. > - Qualification grades real tool outcomes, durable state, and browser workflows. ## Linked Issues or Issue Description Refs #13845, #13847, #13850, #13878, #13901, #13973, #13977, #13979. Add **Grok Build** to `paperclip_runner` with `provider: "acpx"`, `acpxAgent: "grok"`, and model `grok-4.7`. Existing legacy Grok agents keep their adapter. Merge the three companion fixes (#13973, #13977, #13979) before treating the integrated Product qualification as deployed behavior. ## What Changed - Synchronize shared, TypeScript, Rust, server, validation, and UI provider contracts. - Run Grok native ACP stdio through ACPX and the authenticated Paperclip MCP bridge. Verify the pinned executable and exact ACP model identity. - Prefer company subscription login. Support an explicit company-secret API key without automatic paid fallback. Fence refresh and copyback to the same account and remove private runtime credentials after containment. - Preserve selected permissions, cancellation, durable session identity, resume, and restart recovery. Keep unsupported steering and goals unavailable. Preserve missing usage and cost as unknown. - Package checksum-verified Grok Build 1.0.13 for Daytona with an immutable, signed image built on EC2. - Add deterministic admission, protocol, permissions, identity, credential, failure, and cleanup checks. Add the maintained 39-case protocol roster and separate subscription/API Product profiles. - Fix live-test findings in reasoning events, reloads, idle-owner retirement, credential-home cleanup, expired-login model discovery, launcher pinning, and rerun evidence selection. - Align control-plane state readers with the transport's 64 MiB bound while retaining identity, ownership, lifecycle, and size rejection checks. - Stabilize two asynchronous CI assertions while retaining actual outcome and filesystem-evidence checks. ## Verification Current integration head `f114948376056fe0b6b34c1496ae8667b59daa63` includes master `3447609d2247e75e55d91493dda91a608364f672` (2026-09-28). Two master advances during verification overlapped the eval catalog; the final merge preserves Grok qualification, completion updates, and bounded API-response reading in all 348 cells. All 77 focused catalog/eval/workflow tests pass. Both native stack layers (#14397) are mergeable, and both exact-head Greptile reviews are 5/5 with successful security scans and no unresolved review threads. All current-head CI is green: 56 successful checks/statuses and four intentional skips ([CI run](https://github.com/paperclipai/paperclip/actions/runs/36447097232)). Trunk code-owner requirements remain enforced. The review summary’s non-blocking saved-asset offset classification note concerns code already merged in #14301; those runtime files are identical to master and outside this stack’s diff. Historical live evidence below retains its original source revisions. Earlier integration checkpoint: `24fc9b94ca0afb21ccdc8d26dbb2e4b258ad72cb`. Refreshed against master `0f14d2612`, preserving Grok qualification alongside the new accounting and lifecycle suites. All 124 focused catalog, evidence, and service-worker checks pass. The current base workflow includes the explicitly selected public-install verification lane; follow-up #14024 supplies its verifier script. CI at that earlier checkpoint was green (56 successful checks/statuses, four intentional skips), and the review is 5/5 with no unresolved findings. Prior feature CI at `fd73f0a9b1ecdf4094685054028df71739ddc3e1` passed ([run 36148259902](https://github.com/paperclipai/paperclip/actions/runs/36148259902)); that is historical evidence, not a current-head result. Paid Product measurements use frozen integrated source `2d939a92b21dcaf5c77c88b54d96784d2ddd0699`, which combines the feature with #13973, #13977, and #13979. That source passed all 52 CI checks and clean 5/5 review. Later master syncs incorporate upstream changes. Their checks remain separate from these pinned live measurements. | Check | Result and source-pinned report | | --- | --- | | Subscription protocol roster | [39/39 first attempts; 206 assertions](https://d1p6rlowie26tp.cloudfront.net/runner-protocol-evals/campaigns/gha-36046839612-1/index.html), runtime `bc6833f7`, evals `92bb4b8c` | | API protocol roster | [39/39 first attempts; 206 assertions](https://d1p6rlowie26tp.cloudfront.net/runner-protocol-evals/campaigns/gha-35926577007-1/index.html), runtime `4a1061c8`, evals `3213dbec` | | Subscription full Product matrix | [16/16 first attempts; 144 assertions; cleanup passed](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-36096908572-1/index.html), source `2d939a92` | | Subscription core repetitions | 18/18: tool use, planning approval, and Stop/resume each passed three times in local and Daytona profiles. The full matrix contains repetition one; [repeat two](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-36104551060-1/index.html) and [repeat three](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-36108100404-1/index.html) each passed 6/6. Total: 28 unique subscription attempts at `2d939a92`. | | API smoke and question continuation | [4/4 first attempts; cleanup passed](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-36147315401-1/index.html), both environments at `2d939a92` | | Historical API Product coverage | [16/16 full matrix](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-35875144860-1/index.html) and 18/18 core repetitions at `4a1061c8`; retained as measurements of that revision | | Native Daytona proof | Three subscription and three API MCP/permissions/resume runs passed at `bc6833f7`. Three expired-login admission and fenced refresh checks passed without inference. All test sandboxes were removed. | | Inspectable artifacts and UI | Current-source screenshots verify planning approval, direct Ask completion, question continuation after controller restart, and two downloadable project revisions. The project downloads pass 12 and 18 tests; all 40 independent artifact oracle checks pass. | | Provider-free checks | 116 eval-validator tests, 39 Grok definitions, and 359 enabled/external campaign cells pass. Continuation regressions above 2 MiB and 16 MiB failed before their fixes; 32 focused recovery/ownership/size checks pass. | The 32 unique current-source Product attempts have no failures, retries, or skipped cells, and all cleanup checks pass. Whole-workflow timing, model identity, image and provider-pack provenance, attempts, and accounting coverage are retained in the canonical reports. The report publisher's conservative `complete=false` flag is preserved; independent audits verify the exact selected source catalog and immutable result rows. Pins: Grok Build `1.0.13 (5e9a58528b76)`, ACPX `0.13.1`, ACP model `grok-4.7`. Linux binary SHA-256: `edf79521581bb5e6b95abef848491a6a742e860da3e237ebe86a280d30dce4c1`. Launcher SHA-256: `f0b698395a3704ed2ffaf84ea19bdb20c36c8a0a70b7c629c7b6ffe144e59e55`. Image: `ghcr.io/paperclipai/paperclip-daytona-runner@sha256:76b24edfd850219e949418b19e4ceba690e84d51d199ade426e484953329b5e9`. Image build source is `4196a4cd`, recorded separately from application source `2d939a92`; each campaign verifies the image signature and provider pack. Original failed campaigns remain available: [continuation bound](https://github.com/paperclipai/paperclip/actions/runs/36057718059), [scheduler/event capture](https://github.com/paperclipai/paperclip/actions/runs/36071063537), and [startup cleanup plus EC2 interruption](https://github.com/paperclipai/paperclip/actions/runs/36080870743). They retain their original grades. No Docker or Rust builds ran on the developer laptop for these follow-ups. ## Risks Merge packaging follow-up #14024 with this base before public release. The follow-up replaces the private Grok bridge package with a built-in launcher and makes the native binary an explicit sandbox prerequisite. Three separate, reviewed fixes are part of the tested integrated behavior: #13973 serializes task-run admission; #13977 captures complete event evidence; #13979 durably reconciles failed Daytona creation. Each has green CI and clean 5/5 review. Failed-create recovery has 277 plugin tests, 92 SDK tests, host-runtime recovery tests, and a real Daytona lost-deletion-receipt proof. The live proof uses a private file for journal persistence; database durability is covered by host tests. Worker death before delivery of a failure envelope remains outside that recovery mechanism. Subscription fixtures stage an authorized company login; interactive browser sign-in is not qualified. Local Product profiles ran on EC2 Linux. The temporary subscription credential was removed from the protected GitHub environment after all subscription audits, with absence verified. Runtime homes and refresh copyback remain ownership-fenced. Protocol results remain pinned to their original revisions; they are not relabeled as tests of the latest feature commit. New binary/model versions require qualification. Missing token usage and model cost remain unknown; runtime estimates do not establish a full bill. Automatic paid Grok scheduling remains disabled pending separate reviewed enablement. The 64 MiB bound can increase memory use for verbose sessions, and larger files still fail closed. No automatic legacy-agent migration occurs. ## Model Used OpenAI GPT-6 through Codex, with tool use and code execution. The exact serving model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-28 13:54:36 -05:00
Devin FoleyandPaperclip	e9debd3eac	fix(workspaces): allow larger status output for readiness checks (#14414) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The server checks workspace contents before it allows cleanup or branch reconciliation. > - These checks need the full Git status output to count untracked files. > - Nested task worktrees can make this output exceed the scheduler's default 1 MiB limit. > - A scan failure then blocks an otherwise inspectable workspace. > - This pull request raises the limit for these status checks to 32 MiB. > - The checks retain exact counts and still protect uncommitted work from cleanup. ## Linked Issues or Issue Description Refs #14194 and #14253. Those changes address snapshot enumeration. This PR addresses buffered execution-workspace status checks. Refs #13619 for separate work on caching these checks. The related journal identity failure is covered by #14312. **What happened?** Close-readiness checks failed when Git status output exceeded 1 MiB. A workspace with thousands of long untracked paths could not report its file count or complete readiness inspection. **Expected behavior** Allow up to 32 MiB of status output for execution-workspace readiness and branch reconciliation. Preserve exact counts. Continue to block cleanup when the workspace has uncommitted data or the scan exceeds its bound. **Steps to reproduce** 1. Create an execution workspace with a merged delivery. 2. Add 5,000 long untracked filenames under a nested task directory. The status output exceeds 1 MiB. 3. Request close readiness. Before this fix the status scan fails. After this fix it reports all 5,000 files. 4. Run the terminal-workspace sweep. Confirm that it preserves the workspace and files. **Paperclip version or commit** Reproduced on master at `14795136f5` before this fix. **Deployment mode** Server with managed Git workspaces. ## What Changed - Set a 32 MiB stdout bound for execution-workspace status scans. - Add a real Git regression with 5,000 long untracked paths and a cleanup-preservation assertion. Assert that the measured status output exceeds 1 MiB. - Document the bound and failure behavior. ## Verification - The new regression failed on master before the service change: the status result had no untracked files or count after the scan exceeded its bound. - `pnpm exec vitest run server/src/__tests__/execution-workspaces-service.test.ts server/src/services/workspace-git-operation-scheduler.test.ts` passed: 82 tests, including the new regression. The regression and server typecheck also passed after the explicit byte-count assertion. - `pnpm build` and `pnpm -r typecheck` passed. The full local `pnpm test:run` was attempted and stopped after the failures listed below. Greptile is 5/5 with zero unresolved review threads on the latest head. All checks for head `45f93ad5ad` passed (53 successful, two intentional skips). - Full local validation did not pass. The attempt reproduced 13 company/runtime skill-cache permission failures, the terminal-workspace cleanup assertion, and a heartbeat feedback timeout. It was stopped during the general-server stage after these failures. Remaining general-server tests, other workspace groups, and serialized-server stages did not complete locally. Earlier clean-master checks reproduced the cache failures and isolated cleanup retries passed. The latest-head GitHub suite passed all of these groups. - One GitHub browser shard initially failed because its GitHub-connection test checked the resume-action array before the mocked request completed. The single failed-job retry passed without a source change. All latest-head checks are green. - No browser suites ran locally. This change does not affect browser behavior. ## Risks - Each active status scan can buffer up to 32 MiB before parsing. The existing scheduler bounds scan concurrency and queue size. - Output above the bound still fails the scan and blocks destructive cleanup. The change does not truncate output or change cleanup rules. - There are no API, database, or snapshot-streaming changes. ## Model Used OpenAI Codex, GPT-6, with repository inspection, code editing, and test execution. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites; full local-run failures and incomplete stages are documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-28 09:59:56 -07:00
Devin FoleyandPaperclip	6f40e23536	fix(logging): redact cloud authentication headers (#14413) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The server records HTTP requests to help operators diagnose failures. > - Cloud requests carry tenant credentials and signed assertions in headers. > - The HTTP logger did not redact four of these headers. > - This pull request adds those headers to the existing redaction list. > - Operators retain the route, method, and response status without recording these values. ## Linked Issues or Issue Description **What happened?** HTTP request logs could contain cloud tenant credentials, session identifiers, runtime identity assertions, and cloud control assertions. **Expected behavior** The logger must redact these header values for successful requests and failed requests. **Steps to reproduce** 1. Create an Express app with the production HTTP logger and redaction configuration. 2. Send a request with the four cloud headers and distinct test values. 3. Inspect the serialized request headers for responses with status 200, 403, and 500. **Paperclip version or commit** Reproduced on `0f14d26123` before this fix. **Deployment mode** Server with cloud proxy authentication. The regression test uses an in-process Express server. ## What Changed - Redact `x-paperclip-cloud-tenant-token`, `x-paperclip-cloud-session-id`, `x-paperclip-cloud-runtime-identity`, and `x-paperclip-cloud-control` in HTTP request logs. - Test real logger output for HTTP 200, 403, and 500 with mixed-case request header names. Route 403 and 500 through the production error handler. Check response bodies, log levels, and server error context. - Check that the method, route, and status remain available. ## Verification The `server/src/__tests__/http-log-redaction.test.ts` suite passed, including all three new cloud-header cases. - Rebased onto master at `14795136f5`. - `pnpm exec vitest run server/src/__tests__/http-log-redaction.test.ts` passed: 59 tests, including all three new response-status cases. The suite and server typecheck also passed after the error-handler coverage update. - `pnpm build` and `pnpm -r typecheck` passed. The full local `pnpm test:run` was attempted and stopped after the failures listed below. Greptile is 5/5 with zero unresolved review threads on the latest head. All checks for head `373d29e2f1` passed (53 successful, two intentional skips). - Full local validation did not pass. The attempt reproduced company-skill cache permission failures, the terminal-workspace cleanup assertion, a heartbeat feedback timeout, and one process-conversation timing failure. It was stopped during the general-server stage after these failures. Remaining general-server tests, other workspace groups, and serialized-server stages did not complete locally. Earlier clean-master checks reproduced the cache failures and isolated cleanup retries passed. The latest-head GitHub suite passed all of these groups. - No browser suites ran locally. This change does not affect browser behavior. ## Risks - These four values will no longer be available in HTTP logs. Route, method, and status remain available. - This change applies to new log entries. It does not remove old entries or rotate credentials. - No schema, API, or authentication behavior changes. ## Model Used OpenAI Codex, GPT-6, with repository inspection, code editing, and test execution. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites; full local-run failures and incomplete stages are documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-28 09:59:15 -07:00
DottaandPaperclip	14795136f5	fix(runtime): finalize and recover sandbox workspace exports safely (#14402) ... Serialize native workspace finalization, validate streamed archives within bounded limits, and quietly recover unsafe exports from saved results. Preserve exact allocations for exhausted transient failures and provide export-only retry without rerunning the provider. Consolidates #14314, #14315, #14329, and #14334 while preserving the already-merged finalization label changes. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-09-28 10:57:32 -05:00
DottaandPaperclip	3447609d22	fix(runner): stream and page large API responses within capture budgets (#14301) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents use governed API tools to inspect task evidence. > - Large API results become saved assets with short previews. > - Reading an asset through the same tool used to create another asset, so the agent could not reach the rest of the evidence. > - The 10 MiB response cap also blocked useful large results. Removing all bounds allowed excessive disk use. > - This pull request streams responses up to 1 GiB and makes saved text readable in bounded pages. It adds durable run budgets and capture admission limits. > - Agents can inspect complete evidence while tool results, memory use, and capture work stay bounded. ## Linked Issues or Issue Description **What happened?** A large response became an asset. Reading that asset returned another asset and the same preview. Responses above 10 MiB failed before the agent could read any page. **Expected behavior** The agent can fetch a large response and read its saved text to EOF. Each page stays bounded. New snapshots have a generous finite limit and a durable run budget. Existing larger assets remain readable through byte ranges. **Steps to reproduce** 1. Call a GET operation that returns more than 10 MiB of text or JSON. 2. Before the fix, the tool returns `api_transport_failure`. 3. With this change, responses up to 1 GiB become streamed snapshots with artifact references. 4. Read `GET /api/assets/{assetId}/content` with `responseText: {offsetBytes: 0, limitBytes: 8192}`. Follow `nextOffsetBytes` until null. Related work: #14186 added the API fallback tools. #14218 bounded API discovery. ## What Changed - Add authenticated UTF-8 text windows to `call_api`, with byte offsets and total size. Keep each page at or below 24 KiB. - Stream new responses above 24 KiB through private temporary files into company-owned assets. Bound each capture to 1 GiB of decoded bytes. Reject oversized declared lengths before reading and count streamed bytes before writing. - Reserve capture budget in the run record before spilling. Allow 4 GiB per run. Settle successful captures to their actual size. Failed or interrupted captures retain their full 1 GiB reservation. Run restarts do not reset the budget. - Enforce a 20 GiB company snapshot quota with database reservations. Count legacy snapshots and unfinished storage work across runs and processes. Asset deletion frees quota. - Limit large captures to two per company and four per server process. Hold slots through storage upload and temporary-file cleanup. Use a 10-minute download deadline and 30-second connection/idle-read timeouts. - Return explicit size, budget, busy, and timeout errors. Preserve unknown outcomes for mutations whose response cannot be captured. - Read saved assets through authenticated storage ranges, with at most two extra bytes for UTF-8 and EOF handling. Unpaged reads return the existing asset and digest with a bounded preview. Reads create no copies and do not consume capture budget. - Keep existing assets above 1 GiB readable in pages. Use safe integer offsets and PostgreSQL `bigint` asset sizes. - Stream large S3 uploads through ordered multipart requests. Abort failed uploads and remove partial local files. - Revalidate run authority during downloads. Keep company authorization, GET-only text paging, redirect denial, and mutation replay receipts. - Document the separate 10 MiB upload limits. This PR does not raise memory-buffered attachment ingestion limits. Future large video uploads need streamed ingestion and storage quotas. ## Verification - Full workspace `pnpm -r typecheck` and `pnpm build` pass after rebasing on master. - Focused API and response tests: 1,761 pass. Cover declared and chunked oversize responses, incorrect Content-Length, exact-limit success, active-stream deadline, cancellation, cleanup, concurrency admission, and mutation outcome handling. - Real HTTP integration: 28 tests pass, including runnerd → PRP → authority → HTTP, a 12 MiB snapshot, final-page/EOF reads, cross-company denial, a persisted 3 GiB sparse asset, and large mutation receipt replay. - The HTTP suite verifies durable run-budget accounting, simultaneous runs competing for company quota, legacy snapshot accounting, deletion refunds, failed-storage reservations, cleaned-failure refunds, metadata-rollback cleanup refunds, preservation after a lost commit acknowledgement, and small/saved reads after capture-budget exhaustion. - A standalone proof streams exactly 1 GiB through the production capture helper, verifies the final bytes, and removes its temporary file. It uses repeated 256 KiB chunks and records a peak process RSS of 191 MiB. - Earlier storage verification covers exact S3 multipart boundaries, cleanup/abort failures, and a 17 MiB transfer through the real AWS SDK to a local HTTP S3 endpoint. No cloud S3 qualification was run for this follow-up. - The local full test run was interrupted for the company-quota changes. A later targeted run hit exhausted macOS shared-memory slots before tests started; two unattached PostgreSQL segments with dead owners were reclaimed before retrying. All 55 current-head checks pass at `aebb80ceeeee77d5a56b67bfffd835f2f846878c`, including the full CI test suite, typecheck, build, browser suites, security scan, and Greptile (5/5). There are no unresolved review threads. The combined rebased test catalog also passes (48 tests). - Earlier paging acceptance passed Daytona and separate staging at `7739879e9`. Those runs predate the streaming and budget changes. ## Risks - The 1 GiB response cap and 10-minute active-download deadline are intentional product limits. Larger live results must use endpoint pagination or a direct file workflow. Existing larger assets remain readable through bounded ranges. - A durable 20 GiB company snapshot quota counts stored runner-api assets and active/orphan reservations across runs and processes. The operator can set PAPERCLIP_RUNNER_API_COMPANY_CAPTURE_MAX_BYTES to a finite value of at least 1 GiB. Deleting snapshots frees capacity; possible orphan storage must be reconciled before releasing its reservation. - A failed capture uses its full reservation. A new large capture needs a full 1 GiB available, even if it later completes at a smaller size. Small reads and existing asset pages remain available. - Concurrency limits apply per server process. The run byte budget is shared through the database. - The `integer` to `bigint` migration rewrites asset metadata and takes an exclusive table lock. File bytes stay in storage. - A live endpoint is fetched once before returning its snapshot. Continue reading the saved artifact for stable pages. Mutations may commit before any size or transport error; inspect state before retrying. - Attachment uploads and native file handoffs still default to 10 MiB. Raising buffered ingestion paths to GiB sizes is separate work. ## Model Used OpenAI Codex, based on GPT-6, with code execution and repository tools. The runtime does not expose an exact serving model variant or context-window size. The earlier paging work also used browser testing and subagents. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-28 10:36:32 -05:00
DottaandPaperclip	8751e2de46	fix(ui): distinguish finalization recovery from live observation (#14326) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The task board shows which recovery actions are active. > - A native run can stop while a person must repair its workspace. > - The board previously called that state “Recovery in progress.” > - The label implied that work would continue without operator action. > - This pull request derives the label from the recovery owner and live continuation. > - Operators can distinguish scheduled recovery from a repair that needs attention. ## Linked Issues or Issue Description **What happened?** A blocked task showed “Recovery in progress” after native finalization stopped and no automatic continuation remained. **Expected behavior** Show “Recovery needed” for an idle board repair or when no live recovery path exists. Show “Recovery in progress” while the recorded continuation can run, including an explicitly admitted export whose exact callback is executing even if the old recovery action remains board-owned. **Steps to reproduce** 1. Complete a native run whose workspace export cannot be recovered automatically. 2. Inspect the task recovery action and badge. 3. Compare the board-owned action with the old “Recovery in progress” label. Related work: #14314 serializes native workspace finalization and fences stale recovery outcomes. This change reports the recovery action that currently owns the task. ## What Changed - Show “Recovery needed” for board-owned active-run recovery unless the exact native export callback is positively verified as executing. - Require the native continuation run and a live or future continuation before showing progress. - Project native activity from the exact company, source issue, and run. Include active workspace export while the original heartbeat remains failed. - Require an executing callback before using a running export row as evidence. Preserve activity for long exports and clear it when the callback joins. - Give native resume its own card explanation. Preserve ordinary watchdog observation behavior. - Remove the redundant ownership sentence from all six recovery-card explanations that used it. - Document the labels and add regression cases for stopped, scheduled, and active recovery. ## Verification - Copy-only follow-up (`c68aef04c`): all 167 focused recovery UI tests and token gates pass. No UI occurrence of the removed sentence remains. `pnpm -r typecheck`, `pnpm build`, and current-head CI pass (54 successful checks, two optional Storybook checks skipped). Greptile is 5/5 with no open review threads. The duplicate local `pnpm test:run` was stopped after the full CI suite passed; it did not complete locally. - Original regressions: seven failures before the change, then 52 focused cases pass. - Review regressions: seven UI failures and nine database failures before the follow-up. All 75 database/API recovery tests, 167 UI tests, and 18 workspace lifecycle/finalizer tests pass. A further three RED cases cover explicit board retry activity; one RED case rejects orphaned running export rows after controller loss. Wrong company, issue, run, service, phase, and completed-operation cases remain inactive. - Final recursive typecheck, production build, token gates, and complete local suite coverage pass. Embedded PostgreSQL startup/socket failures passed in isolated retries with the canonical test environment; no expected behavior was weakened. The recovery regression added during the earlier full run passed in its final complete 75-case file. - Before the copy-only follow-up, all 56 CI checks passed on `c2f84cd89c905cda85c53aaf5bb83b7250900fe6`; Greptile is 5/5 with no unresolved review threads. The final native-activity staging repeat passed on integrated source `2bedd0f23bf4698b1f8b818f6796900647030427`. - Verified on a separate staging instance: a real failed Daytona workspace export retains its board-owned repair action and displays “Recovery needed” in the task list. The repair card remains actionable without starting another provider turn. - Real staged export-only repair: the actual task list showed “Recovery in progress” while the original run had a positively identified running export operation, then Done after exact copyback of all 20,000 nonce-bound files. The accepted result and full provider session/turn/terminal envelopes remained unchanged. All 17 independent final checks passed; the browser downloaded the exact 19-byte result. The separate fixture was cleaned up with independent provider-absence verification. A control transport process restarted during repair; it did not submit another provider turn. - Final deployed-source repeat on `d884e1ab046cc76004e35e6091e9e6e2c918c9eb`: explicit per-turn ephemeral Daytona allocation, actual browser export repair, and a saved full activity projection referencing the exact executing export operation with no scheduled retry. The task list showed recovery in progress, then Done; all 21 final checks passed, including 20,000 exact host files, unchanged provider provenance, and provider deletion only after committed copyback. The downloaded 19-byte result matched independently. This integrates #14334; no source change was required here. ## Risks - The label depends on the persisted recovery action. A separate runtime defect can still stop work; this change makes that condition visible. - Future recovery kinds must supply a valid continuation path before they can display progress. ## Model Used OpenAI Codex, based on GPT-6, with code execution, browser testing, and subagent tool use. The runtime does not expose an exact serving model variant or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-28 09:33:39 -05:00
DottaandPaperclip	4b38db9622	fix(runtime): stream workspace Git snapshots through disk manifests (#14253) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Managed runs copy a selected workspace to an execution environment and restore its changes. > - Git snapshots select the files for that copy and for later recovery. > - A fixed output limit stops large generated trees before the run can start. > - Increasing the limit still keeps the complete filename lists in memory. > - This pull request stores those lists and merge baselines in disk manifests. > - Large snapshots can now complete with bounded filename buffers and explicit failure handling. ## Linked Issues or Issue Description Refs #14194. This is the streaming follow-up to the merged 32 MiB limit fix. Related: #13619 and #11621 cover workspace scan admission and demand. This change keeps the shared scheduler and changes the snapshot data path. ## What Changed - Stream changed, untracked, deleted, and ignored paths through the shared scheduler and the standalone adapter path. - Use SQLite manifests for file selection, duplicate removal, ignored-path lookup, baseline capture, and merge lookup. - Set a configurable 30-minute snapshot deadline. Keep the existing interactive scan deadlines. - Wait for each child process and pending sink write before removing temporary storage after failure or cancellation. - Use NUL archive lists and bounded deletion batches. Preserve unusual names, explicit selection, nested repositories, and source root checks. - Store manifest references in native recovery format v2. Check their location and digest before recovery reads. Keep v1 descriptors readable. - Remove temporary manifests at lifecycle completion. Use fixed-size temporary copy names for long basenames. - Admit each manifest with a SQLite page allowance based on current disk capacity. Keep a configurable free-space reserve and fail explicitly when either limit is reached. - Preserve a host file that replaces a directory deleted by the sandbox, and continue the rest of the restore. ## Verification - Current head `5ab622ec43cd16d35429d79dedee6a5d8e3d2df2` has 54 successful checks/statuses and two skipped Storybook jobs. No checks failed or remain pending. - [CI passed](https://github.com/paperclipai/paperclip/actions/runs/36318966368): typecheck, build, all test shards, E2E, Rust checks, and the aggregate verify job. - [Greptile is 5/5](https://github.com/paperclipai/paperclip/pull/14253#issuecomment-5855670338) on the current head. All four review threads are resolved. Security checks passed. - 229 focused tests passed across Git sync, runtime staging, merge, manifest integrity, native recovery, and the scheduler (214 adapter/runtime tests and 15 scheduler tests). - A real 40,000-file fixture produces 43,428,890 filename bytes. The original standalone and scheduled scans fail. The new test passes all four filename paths, complete staging, exclusion of late files, unusual names, and deletion replay. - Recovery tests reject changed bytes, symlinks, and paths outside the controller state directory. Adapter-utils typecheck passed. - A test executor returned buffered output and caused two retry integration failures. The fixture now uses the shared streaming scheduler. All 13 tests passed with `corepack pnpm exec vitest run server/src/__tests__/heartbeat-project-repositories.test.ts`. The same CI shard now passes. - Ran `pnpm -r typecheck`, `pnpm test:run`, and `pnpm build` locally. Each full local command hit SIGKILL/exit 137 in the 4 GiB container. These local commands did not pass. The current-head CI gates above provide the full verification. - A real-Git disk-capacity regression confirms a typed failure and removal of the incomplete manifest. Repeated writer attempts cannot exceed the permitted page count. - Follow-up real Daytona and separate staging qualification passed with the related archive validator (#14315) and exact-owner finalization fix (#14314). Three successive turns copied back all 60,000 files with 39,828,890 filename bytes and five unusual names. Independent host inventories verified every file and the pinned Git HEAD. Native, provider, session, and process identities stayed fixed; no retry remained. The task reached Done, and its browser-downloaded final proof matched exactly. The reusable regression is #14316, including an assertion of the effective environment idle policy. ## Risks - SQLite manifests use disk space. Each receives one quarter of the available capacity above the host reserve at creation. The reserve defaults to 256 MiB and has a 64 MiB configuration minimum. Disk capacity, filesystem quotas, per-path limits, Git resource use, and execution deadlines remain limits. - Each path and sink chunk has a 64 KiB limit. SQLite connections use a 1 MiB page cache. Invalid or incomplete records fail explicitly. - Restore transport keeps fixed and configured archive exclusions. A remotely created Git-ignored file can be transferred, but the host merge excludes it through the manifest. - Provider archive buffers, Git and tar memory, repository metadata, legacy v1 arrays, and the separate referenced-source resolver retain their own limits. Existing provider safety validators still buffer textual tar listings: Daytona allows 32 MiB and Kubernetes allows 64 MiB. These separate transport limits can stop a sufficiently large restore before merge. This change does not claim bounded total process memory or unlimited transport size. - New descriptors use v2. Existing v1 recovery remains supported; a downgrade cannot read v2 descriptors. ## Model Used OpenAI GPT-6 through Codex. The exact deployment ID and context limit are not exposed in this run. The agent used code editing, terminal execution, tests, and GitHub tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-28 08:40:36 -05:00
Devin FoleyandPaperclip	be43c23e2d	fix(recovery): preserve pending native result finalization (#14219) ... Preserve native coordinator ownership while accepted results await workspace copy-back, assessment, or arbitration. Check that ownership in the terminal update so a coordinator recorded after the liveness read is also protected. Keep terminal task authority and exhausted-retry cleanup unchanged. Validation: 28 focused recovery tests, local typecheck/build, 52 passing CI checks, and Greptile 5/5 with no open findings. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-09-27 00:22:49 -07:00
Devin FoleyandPaperclip	5ee9e751fb	fix(runner): discover assigned tools when direct catalogs exceed limits (#14218) ... Keep assigned app tools accessible when the combined Runner catalog exceeds its operation or byte limits. Reserve task and completion tools, then expose bounded discovery and call tools for large catalogs. Fetch oversized schemas in reauthorized chunks without blocking later search results. Retain task ownership, work-mode restrictions, pinned assignments, current gateway authorization, approvals, and audit. Small catalogs stay direct. Validation: 46 focused server tests, two Runner capacity tests, local typecheck/build, 52 passing CI checks, and Greptile 5/5 with no open findings. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-09-27 00:22:16 -07:00
DottaandPaperclip	640dee1802	fix(runner): acquire reusable leases when switching to warm sessions (#14187) ... ## Thinking Path > - Paperclip coordinates agent work across task turns. > - Warm runners need reusable sandbox leases. > - Lease acquisition read only the environment reuse setting. > - Startup then read the agent's new warm setting and rejected the ephemeral lease. > - This fix requests reuse before acquisition, with provider checks intact. ## Linked Issues or Issue Description **What happened?** Switching an existing native runner task from per-turn to warm fails with `runner_warm_environment_requires_reusable_lease`. Environment `reuseLease` defaults to false. Acquisition therefore creates an ephemeral lease, and capability narrowing correctly denies reuse. **Expected behavior** Changing lifecycle between turns requests a compatible lease without replacing the task workspace or changing the shared environment. **Steps to reproduce** 1. Start a native runner task with inherited lifecycle and environment reuse disabled. 2. Change the agent from per-turn to warm. 3. Continue the task on a reusable-capable sandbox provider. **Paperclip version or commit** Reproduced on `a6c4e7a`. Related: #12904 established warm workspace continuity; this fixes the earlier lease-selection mismatch. ## What Changed - Pass agent settings into acquisition and derive a run-scoped reuse request. - Respect explicit environment lifecycle overrides; leave other adapters unchanged. - Keep provider capability, ownership, cleanup and restore gates intact. - Add orchestration and database-backed transition tests; document the contract. ## Verification - Red: three new assertions failed before the fix. - Green: 156 tests pass in environment-run-orchestrator, environment-runtime, native-sandbox-lifecycle, and environment-execution-target-capabilities. - Database-backed regression proves ephemeral → reusable → resumed lease, same workspace, and unchanged stored environment. Provider RPCs are mocked, not live Daytona. - `git diff --check` passes. - Latest-head CI passes typecheck, build, tests, runner checks, E2E, and canary dry run. Greptile: 5/5; no open threads. - Recovery uses the persisted lifecycle. Direct regression: red before, six cases green after. CI runs the added Vitest cases. - Full local checks were unavailable (missing dependencies and earlier memory limits). ## Risks Warm mode now requests retained resources despite an environment's default `reuseLease:false`. Existing idle timeout and cleanup still apply. Unsupported providers remain denied. No migration, credential [REDACTED], or shared-environment mutation. ## Model Used OpenAI Codex agent; reasoning, code editing and test execution. Exact model ID and context size were not exposed to this run. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details available) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket [REDACTED] or instance-derived details - [x] I have run targeted tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-26 21:23:33 -05:00
DottaandPaperclip	f2ed0b65c4	fix(runner): enable API tools by default (#14186) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The native runner exposes tools for company tasks. > - API search and call tools cover operations without a dedicated tool. > - The current default hides these tools unless an operator sets an environment variable. > - This pull request enables the tools when that variable is absent. > - Operators can still disable the tools or restrict them to selected companies. ## Linked Issues or Issue Description Refs #13003, which added the guarded API tools. **What happened?** The native runner does not advertise `search_api` or `call_api` with the default server configuration. **Expected behavior** The tools are available without a special environment variable. Existing authorization checks still apply. **Steps to reproduce** Remove `PAPERCLIP_RUNNER_API_TOOLS_ENABLED` and `PAPERCLIP_RUNNER_API_TOOLS_COMPANY_IDS`. Create a normal runner authority. Inspect its tool definitions. **Paperclip version or commit** a6c4e7a. ## What Changed - Enable API tools when the server flag is absent. - Keep explicit disable, invalid-value rejection, company restrictions, and binding restrictions. - Test default tool definitions and run HTTP integration tests without the enabling flag. - Update operator and hiring documentation. The existing shared gate also controls `hire_agent`. - Use the same policy in the E2E evidence summary so an unset flag is not reported as disabled. - Isolate default-availability tests from operator environment variables. ## Verification - Red test: two new rollout policy assertions failed before the fix. - Focused policy, authority, and HTTP tests: 3 files passed; 41 tests passed, 2 skipped. The two runnerd transport cases require a Rust-built binary absent from this workspace. - Command: `pnpm exec vitest run server/src/services/native-runtime/runner-api-rollout.test.ts server/src/services/native-runtime/paperclip-runner-tool-authority.test.ts server/src/services/native-runtime/runner-api.integration.test.ts`. - Attempted `pnpm -r typecheck`: blocked by missing `cargo` in this workspace. - Attempted `pnpm build`: terminated at the 4 GiB memory limit. - Attempted `pnpm test:run`: stopped after memory pressure to run focused tests alone. - Repeated the 41 passing focused tests with an inherited disabled flag and a foreign-company restriction; test isolation passed. - `pnpm test:e2e:runner:unit`: 44 files and 543 tests passed. - `pnpm test:e2e:runner:typecheck` exceeded the workspace memory limit, including a retry with bounded Go memory settings. - CI results will be recorded before handoff. ## Risks - More native runs can discover API tools and the existing `hire_agent` tool by default. - The change does not remove company, run, mode, credential, lifecycle, or approval checks. - Explicit operator restrictions still take precedence. No database migration is required. ## Model Used OpenAI Codex agent. The runtime does not expose the exact model ID or context-window size. Used reasoning, repository tools, shell execution, and tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-26 21:16:33 -05:00
3 people	a6c4e7a8d1	fix(heartbeat): cancel obsolete execution continuations before dispatch (#13761) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A comment can queue execution before the task changes state. > - The task may be completed, cancelled, deleted, or reassigned before setup checks ownership. > - The ownership guard must prevent that obsolete execution from starting. > - This pull request records that guard outcome as cancellation and settles the wake request. > - Missing history and authorization errors remain failures that operators can investigate. ## Linked Issues or Issue Description **What happened?** A comment can queue a run just before a user marks its task Done. The ownership guard stops setup before adapter dispatch, but the run becomes a setup failure and can put the agent into an error state. **Expected behavior** Cancel obsolete work when the typed task ownership guard rejects it. Keep genuine setup errors visible. Do not change the task's terminal state or current owner. **Steps to reproduce** Queue a task continuation, then mark the task Done or Cancelled before continuation setup. The run should settle as cancelled without invoking the adapter. Deleting or reassigning the task must also prevent dispatch. Missing source history or explicit user authorization must still fail. **Paperclip version or commit** Refreshed against master `b2e9e82f053af14777fd68e8834fff6bb64c84c4`. Related: #13546 covers lost issue-lock claims, and #13888 covers persisted continuation decisions and retry budgets. This change handles the final continuation ownership guard during setup. Thanks to @MrBlackTongue for the original typed cancellation implementation and regression coverage. This update preserves the contributor's commits, resolves the master conflict, and narrows cancellation to task ownership invalidation. ## What Changed - Add a typed `StaleExecutionContinuationError` for `continuation_task_ownership_changed`. - Use existing cancellation settlement for that typed guard, including the run, wake request, issue execution ownership, and agent state. Suppress immediate recovery of obsolete work. - Preserve failure classification for missing source context, missing user authorization, and untyped errors, including an untyped error with identical text. - Cover Done and Cancelled tasks with real database checks. Retain company and assignment guard coverage. Verify cancellation performs no adapter dispatch or automatic replay. - Document the cancellation event and its error classification in the run-log guide. ## Verification Current head: `b875486e81`. - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - `pnpm exec vitest run server/src/__tests__/heartbeat-process-recovery.test.ts server/src/services/execution-continuation.test.ts --maxWorkers=1 --no-file-parallelism -t 'authorized continuation context|obsolete continuation setup|untyped continuation setup failures'`: 17 passed; 316 unrelated tests excluded by the name filter. - `pnpm test:run`: local full run is still finishing. It is not a clean pass: the observed failures are three company-skills cache checks (`EACCES` while renaming read-only cache directories on macOS), one tool-access assertion, and one comment-wake timeout. The three cache failures reproduce in isolation in unchanged code. The tool-access check passes in isolation, and the entire comment-wake file passes (29 tests), including explicit feedback after completion. Final full-run totals will be added when available. - [CI](https://github.com/paperclipai/paperclip/actions/runs/36282532652): all gates green on this head. An unchanged Runner workspace-diff test initially returned no diff; an isolated check passed, and the failed job plus its dependent gate passed on retry with no code change. Original failed job: 108517010989. - Greptile: 5/5 on the full current head, with no unresolved review threads. Branch is current with master and conflict-free. - Diff check and local secret/PII scan passed. No customer data or private deployment identifiers are included. ## Risks The classification change is limited to the typed ownership guard. A task with missing history remains a failure rather than being treated as an expected cancellation. Existing company, task-owner, terminal-state, and authorization checks still prevent dispatch. Cancellation retains its specific reason in the run log and does not grant a retry. No schema, dependency, or API changes; no migration is required. ## Model Used OpenAI GPT-6 via Codex assisted investigation, implementation, code review, and test execution. The exact serving model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the bug issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run the focused regression tests locally and they pass; full-suite results are tracked above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented the risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before merge Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Devin Foley <devin@paperclip.ing>	2026-09-26 17:43:33 -07:00
Devin FoleyandPaperclip	b2e9e82f05	fix: stop remote Grok runs before continuing queued messages (#14100) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The execution service owns each run and saves messages sent while it runs. > - Interrupt must stop the current executor before it delivers those messages. > - Remote Grok commands did not register the host cancellation control. > - A cancelled task run could still write Done and prevent queue recovery. > - This pull request connects remote cancellation and revokes cancelled run writes. > - Saved input can use the existing queue admission rules after verified cleanup. ## Linked Issues or Issue Description **What happened?** Interrupting a queued message marked a remote Grok run cancelled before its sandbox stopped. The old run could still post a reply and mark the task Done. Its saved follow-up remained deferred behind execution recovery. **Expected behavior** Stop revokes run write authority and waits for verified termination. Saved messages remain durable and enter one successor through normal admission after cleanup. **Steps to reproduce** 1. Run a task with `grok_local` in a remote sandbox. 2. Send a follow-up and use Interrupt while the command runs. 3. Let the old command attempt a task status update after cancellation. 4. Observe the task disposition and the saved message queue. **Paperclip version or commit** The gap is present in master at `d3e0f0a238`. **Deployment mode** Authenticated server with a Daytona sandbox. Related work: #14028 and #14046 handle bounded continuation. #13291 covers infrastructure interruption and verified remote cleanup. #13332 addresses atomic recovery holds. This change handles direct Grok operator cancellation and stale task writes. ## What Changed - Register remote Grok cancellation before preparation. Keep command ownership until the host confirms sandbox termination. - Reuse the sandbox cancellation boundary for the direct CLI invocation. Reject fresh attempts after cancellation and preserve workspace restore failure evidence. - Reject writes from cancelled task JWTs and runs with a pending stop. Preserve diagnostic reads and existing conversation error codes. - Recheck run authority under a database lock before task updates and interaction responses commit. - Preserve authorized handoffs that stop their own run. Only the server-issued stop receipt for that request permits the final task update. - Add tests for hung commands, unverified stops, early cancellation, copy-back failures, late Done, late interaction responses, authorized handoffs, exact lease receipts, and one queue successor across concurrent restart sweeps. - Document the cancellation and write-authority contract. ## Verification - Targeted adapter, cancellation-boundary, authentication, queued-message, interaction-service, and activity-route tests passed. The expanded run passed 214 tests; one new test had an incomplete fixture. After correcting the fixture, all 8 selected follow-up cases passed. - `pnpm -r typecheck`: passed on `179c86caf1bf0d89914a503d46e24af7e4b8c557`. - `pnpm build`: passed on the same commit. - `pnpm test:run`: the general-server group completed with 13,521 passed, 99 skipped, and 18 failed tests. It then stopped, so the remaining local groups did not run. Five Slack, email, and wake-batching failures passed on focused reruns after correcting the local environment. The remaining 13 failures reproduce as `EACCES` on rename in unchanged skill-cache code on macOS. Two custom-image suite setup hooks also failed to start embedded PostgreSQL after the machine exhausted shared-memory slots; all 31 tests in that file passed on rerun after the local resource issue was resolved. CI covers all test groups. - CI: 53 checks passed and 2 were skipped on the latest commit, including the aggregate verification gate. The last server shard passed on its single rerun after a preview-server startup timeout. The affected file also passed locally with 28 passed and 3 skipped. - Greptile: 5/5 on the latest commit. Both review threads are resolved. - No live deployment or staging task mutation has been performed. ## Risks - Stopping the sandbox can prevent file copy-back. The result preserves workspace restore failure evidence; termination does not imply restored files. - If provider termination fails, the adapter keeps ownership of its outstanding command and does not acknowledge Stop. - The write restriction now applies to ordinary cancelled tasks. Reads remain allowed. Task and interaction checks add a shared run-row lock to agent mutations. An exact server-issued receipt permits the task request that stopped its own run to complete its handoff. - Existing terminal tasks are not reopened automatically. An operator must correct a historical late Done before its saved queue can continue. - No schema migration or UI change. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository inspection, code execution, and test tools. The precise backend revision and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-26 17:07:07 -07:00
DottaandCodie	01d9a12185	fix: make keyboard shortcut enablement a personal preference (#14141) ... Store keyboard shortcut enablement per user and expose it in Profile settings. Co-Authored-By: Codie <Codie@users.noreply.github.com>	2026-09-26 11:50:04 -05:00
Devin FoleyandPaperclip	d3e0f0a238	fix(server): validate CLI auth challenge IDs (#14095) ... Reject malformed CLI challenge UUIDs before database access while preserving secret and authentication precedence. Document the HTTP responses and pin the supervisor test fixture to the CI-selected Node executable. Validated by 21 focused tests, root typecheck/build, and full CI. Mac broad-suite baseline limitations are documented in the PR. Greptile 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-09-26 00:48:14 -07:00
Devin FoleyandPaperclip	1e3a148f46	fix(connections): retain safe broker rejection diagnostics (#14098) ... Preserve fixed broker rejection reason codes within strict size/time limits while retaining public error codes and status. Unknown bodies remain generic; no raw response or credential material enters the error. Validated by 33 consumer tests, a synthetic producer HTTP contract fixture, root typecheck/build, and full CI. Greptile 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-09-26 00:47:47 -07:00
Devin FoleyandPaperclip	ffa32373bc	fix(runtime): honor provider acquisition timeout defaults (#14097) ... Declare provider acquisition budgets so slow Daytona creation does not hit the host’s 30-second fallback. Bound creation and setup to one deadline and preserve scoped cleanup ownership after timeout. Legacy drivers retain their original call shape. Validated by 238 provider/manifest tests, focused database and heartbeat regressions, root and standalone provider typecheck/build, and full CI. Local broad tests also expose recorded Mac baseline limitations. Greptile 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-09-26 00:47:21 -07:00
Devin FoleyandPaperclip	c3ddb288b1	fix: validate work product execution workspace references (#14063) ... Reject invalid and cross-company execution workspace references with a useful 422 before changing the work product. Hold the validated reference through the transaction so concurrent deletion cannot turn validation into a 500. Verified 13 focused tests, full typecheck/build, and green PR CI. Greptile 5/5 with no unresolved comments. The known UI copy-toast flake passed on an unchanged-commit retry and in a focused local run. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-09-25 17:06:30 -07:00
DottaandPaperclip	96bf004a79	fix: use persisted state for lifecycle continuation and retry budgets (#13888) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its control plane decides when a task can continue, wait, stop, or complete. > - Legacy continuation could change when an agent changed its wording without changing task state. > - Shared attempt counts also let repair and infrastructure retries affect each other's limits. > - This pull request uses persisted state and separate, bounded allowances for these decisions. > - If automatic repair stops, the task explains what happened and offers a guarded retry. > - Paired tests and real-provider evaluations verify that Stop, approvals, ownership, and spending limits remain authoritative. ## Linked Issues or Issue Description Related work: Refs #13761, Refs #11126, Refs #13610. These cover obsolete continuation dispatch and retry storms. Open and closed issues and PRs were searched for related lifecycle, continuation, and retry work. **What happened?** Legacy continuation depended on English wording and progress heuristics. Repair, failure retry, and productive continuation could consume shared counts. When bounded repair stopped, the task showed a technical recovery message without a clear next action. **Expected behavior** Persisted disposition and owned execution paths determine the next action. Missing disposition prompts bounded agent repair. Explicit work mode determines planning mode. Narrative changes and raw activity counts cannot replenish allowances. An exhausted repair shows a readable notice. An explicit retry checks current controls and preserves the assigned agent. **Steps to reproduce** Run `pnpm test:lifecycle-baseline`. The paired probes keep structured state constant while varying completion, planning, blocker, and progress prose. Run the explicit `lifecycle-baseline` and `continuation-accounting` Product E2E suites for real-provider coverage. In Storybook, open **Design previews / Recovery notice** to inspect the production component's normal, pending, acknowledged, unavailable, failure, and mobile states. ## What Changed - Hide the image attachment button, icon, and drop/paste hint in answer composers. Image paste and drop support remains available. - Merge current master and retain both browser regression sets. Use a production-stamped service worker in the offline recovery browser fixture. - Share one state-based legacy continuation decision across immediate, delayed, and recovered dispatch. Bind bounded repairs to their source run and episode. - Remove title and description wording from work-mode authority. Agents can still write requested plans in execution mode. - Persist separate failure-retry and productive-continuation counters. Disposition repair and resource waits cannot consume or reset those allowances. - Validate delayed repair identity, then recheck current gates before provider dispatch. Fence native startup cancellation. - Show **Agent needs attention**, a plain-language explanation, **Retry agent**, and expandable details in both task interfaces. Report request progress, acknowledgement, and errors inline. - Store typed recovery notice metadata. Recognize older active notices only through exact stored action and run IDs. Notice text never grants retry authority. - Use the existing recovery-action endpoint for retry. Recheck current action, status, owner, agent availability, dependencies, active runs, pending questions and confirmations, approvals, pause controls, and budget. Duplicate requests do not wake twice. - Add component, page, route, database, contract, and Storybook coverage. Keep the scenario inventory and executable evals here. Historical reports and snapshots live in the [commit-pinned paperclip-evals archive](https://github.com/paperclipai/paperclip-evals/blob/ce3e5afcd4a1184650f586a2b5b8be5874c66c8b/experiments/2026-09-lifecycle-authority/README.md). - Preserve unsaved project fields while the same project URL changes to its canonical alias. Do not reuse data across projects or companies. This separate fix addresses the repeated repository-editor browser failure without changing the browser test. - Keep the development service worker from intercepting Vite module reloads. Update the connection-intent browser fixture to record progress and completion through the agent API. ## Verification Merge preparation on September 25, commit `c1e8e4b7ddd9fbc4913ed55ce21b8e12906c2f97`: - Merged master `bd2030932` and resolved the browser test-list conflict by keeping both sets of regressions. - Deterministic lifecycle baseline: 1,090/1,090 assertions passed; no failures, skips, or missing selected evidence. Unit 423, runner 184, database integration 397, grading 86. - Browser support: 17/17 passed. The offline recovery test first failed with an unstamped development worker, then passed with the production stamp. Its assertions are unchanged. - Focused interaction UI and offline fallback tests: 19/19 passed. Verified the custom-answer composer in Storybook: no attachment controls or hint; entering an answer enables Next. - Recursive typecheck, production build, token gates, and diff checks passed. The worktree is clean. No new real-provider campaign was run. - Current CI and review: [Current PR CI passed](https://github.com/paperclipai/paperclip/actions/runs/36166011243): 55 successful checks and two optional Storybook skips. Greptile scored this exact commit 5/5. Hiding the question attachment controls is an intentional UI change; paste/drop remains available. Earlier recovery UI verification, commit `21be0fec0e90e86b6d662b8ee4831847cd041cdb`: - Recursive typecheck, production build, token gates, and diff checks passed. - Focused UI coverage: 338 tests passed across six suites (336 before the interaction guard, with the two affected suites rerun at 149 passed after it). Covers both task interfaces, the real page mutation, pending/error acknowledgement, stale state, and unavailable controls. - Recovery database integration: 352 tests passed before the interaction guard. The complete recovery-action and mutation-route suites passed 181 tests after it. The two new pending question/confirmation regressions failed before the fix and passed afterward, including resolved-interaction controls. Shared validator suite: 31 passed. E2E catalog suites: 34 passed. - Browser inspection passed for light/dark themes, mobile layout, expandable details, pending retry, acknowledgement, failure, and disabled retry. Storybook renders the production component; its request is simulated. - The broad local run hit two chat callback-order wait failures and was stopped after all CI unit/database/runner shards passed. Both local failures passed when rerun without the competing full-suite process. - CI exposed a repeated project-repository draft-loss race during canonical redirects. A new unit regression failed before the fix; all nine project-page tests now pass, including controls for other projects and companies. Both unchanged repository browser tests passed against a fresh local server. UI typecheck, production UI build, and token gates passed after this fix. - [Earlier PR CI passed](https://github.com/paperclipai/paperclip/actions/runs/36072486798) on `21be0fec0e90e86b6d662b8ee4831847cd041cdb`: 55 successful checks, two optional Storybook skips, and no failed or pending checks. The repository browser shard passed with the production fix. Greptile is 5/5 on this exact commit with no unresolved review threads. The PR is mergeable. Historical, source-qualified lifecycle evidence: - Lifecycle baseline: 1,074 assertions. Native session coverage: 447 tests. Product E2E support: 515 tests. Browser support: 11 tests. Full earlier verification is retained in the archive. - [Real-provider campaign: 8/8 passed, zero retries](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-35881382080-1/index.html), source `e88d210417280140b44a36449027290adcb1aeaa`. Evidence and cleanup checks passed. This includes deliberately exhausted repair cases that correctly remain blocked; it does not mean every task finished Done. This campaign predates the recovery UI change. - Archive migration verified all 16 original JSON files byte-for-byte and all 24 checksum entries. App tests do not need private archive access. [Archive PR #27](https://github.com/paperclipai/paperclip-evals/pull/27) is merged. ## Risks - Agents that omit durable disposition receive at most two repair attempts by default. Prose-only completion exposes missing state rather than silently changing scheduling. - A retry is an explicit board action. The server rechecks current controls. A successful response confirms the task returned to To do; it does not claim that the provider has already started. - Existing notice metadata remains valid. Only older active notices with matching structured evidence receive the new UI. Historical notices without that evidence keep their existing rendering. No schema migration is required. - Old run records require conservative retry accounting. Tests cover old counters, alternating retry lanes, restarts, and exhausted repairs. - Historical snapshots require private `paperclip-evals` access. The app index retains public campaign links. Live campaigns qualify specific sources and scenarios; no new real-provider campaign has run for the recovery UI commit. > This fixes existing lifecycle and recovery behavior and does not duplicate planned core work. ## Model Used OpenAI GPT-6 through Codex assisted implementation, reasoning, code execution, and review. The exact serving model ID and context window are not exposed in this task. Historical real-provider evaluations used Codex model `gpt-5.6-sol`, separately from the implementation assistant. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-25 15:28:11 -07:00
Devin FoleyandPaperclip	6bc830b62b	fix(recovery): escalate an issue whose interrupted run has spent its retry budget (#14046) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The stranded-issue sweeper is what puts an assigned issue back on a live path when its run dies. > - A failed or interrupted run gets a bounded transient retry; when that budget is spent, the retry scheduler queues nothing and reports the exhaustion. > - The sweeper treated that "nothing queued" like every other "nothing queued" and skipped the issue, on every tick, forever: `in_progress`, no run, no path, no notice. > - Three server restarts in a row (a deploy storm) are enough to spend the budget, so this is reachable in ordinary operation. > - This pull request makes the sweeper escalate a spent budget to a board-owned recovery action, the same visible `blocked` state its other dead ends use. > - The benefit is that no issue can sit assigned and silent after its retries run out. ## Linked Issues or Issue Description No existing issue found. Related work: #14028 (this branch's predecessor: queue drain-time wakes, retry interrupted corrective runs) fixed two neighbouring gaps but not this one. **What happened?** A routine-created issue's run was interrupted by a graceful server shutdown, and both bounded transient retries were interrupted by the next two shutdowns. The retry scheduler logged "Bounded retry exhausted after 2 scheduled attempts; no further automatic retry will be queued" and stopped. On every tick after that, `reconcileStrandedAssignedIssues` reached the generic continuation lane, `enqueueStrandedIssueRecovery` delegated to `scheduleRecoveryRetry`, which returned null (exhausted), and the sweeper counted the issue as `skipped`. The issue stayed `in_progress` with no run, no recovery action, and no comment for hours until a person woke the agent by hand. The same hole exists in the assigned-`todo` dispatch lane. **Expected behavior** When the transient retry budget for an interrupted or failed run is spent, the sweeper should treat it as the dead end it is: escalate the issue to `blocked` with a board-owned recovery action and a notice, exactly as it does when a continuation retry chain or an assignment retry chain is exhausted. Leaving the budget spent across restarts is correct and unchanged; leaving the issue silent is not. **Steps to reproduce** 1. Assign an agent using a conversation adapter (its interrupted runs carry `conversationContinuation`, so legacy reconciliation does not terminalize them) an issue and let a run start. 2. Interrupt the run with a graceful shutdown, let the transient retry start, interrupt it, and repeat once more so `scheduledRetryAttempt` reaches 2. 3. Run the stranded-issue sweep. Before this change: `skipped` every tick, issue `in_progress`, no run, no recovery action. After: `escalated`, issue `blocked`, one active board-owned `issue_recovery_actions` row, a "No live execution path" notice. **Paperclip version or commit** master at bd6caf51bb (2026-09-25). **Deployment mode** Managed cloud instance restarted by fleet deploys; the code path is the same for any operator whose server restarts more often than the retry budget allows. ## What Changed - `recoveryService` gains an optional `transientRetryBudgetSpent(run)` dependency; `heartbeatService` wires it as `executionFailureRetryCount(run) >= BOUNDED_TRANSIENT_HEARTBEAT_RETRY_MAX_ATTEMPTS`, the same check `scheduleBoundedRetryForRun` applies. - `enqueueStrandedIssueRecovery` takes an optional `outcome` out-parameter and sets `retryExhausted` when the failed predecessor's retry returned nothing **because** the budget is spent. A null return without it still means another authority owns the run (native runtime, legacy reconciliation) and the caller leaves it alone; the deliberate "failure recovery cannot fall through into the continuation queue" rule is unchanged. - The generic `in_progress` continuation lane and the assigned-`todo` dispatch lane escalate on `retryExhausted` via `escalateStrandedAssignedIssue` with a "No live execution path" notice (danger tone), which creates the board-owned source-scoped recovery action and moves the issue to `blocked`. Every other `enqueueStrandedIssueRecovery` caller is unchanged. - Tests (`heartbeat-process-recovery.test.ts`): an `in_progress` issue with a spent budget escalates (blocked, one active board-owned action, notice, no successor run, idempotent on the next sweep); an interrupted run with budget remaining still gets its transient retry; an assigned `todo` issue with a spent dispatch budget escalates. The existing guard "does not reset an exhausted incident budget on server restart" keeps its no-successor-run assertion and now expects the board escalation instead of nothing, with a comment on why. ## Verification ``` pnpm -r --filter './packages/**' build cd server npx vitest run src/__tests__/heartbeat-process-recovery.test.ts \ src/__tests__/issue-recovery-actions.test.ts \ src/services/recovery/successful-run-handoff.test.ts \ src/__tests__/heartbeat-task-drain-admission-release.test.ts \ src/__tests__/attention-service.test.ts \ src/__tests__/heartbeat-comment-wake-batching.test.ts npx tsc --noEmit -p tsconfig.json ``` Live reproduction: the exact stuck state (issue `in_progress`, latest run `interrupted` with `scheduledRetryAttempt` 2, the "Bounded retry exhausted" lifecycle event, sweeper `skipped` every tick) was observed on a managed instance running current master before this change was written. ## Risks - Behavior change is limited to runs whose transient budget is already spent, which previously produced no action at all. Nothing new is retried; the change only adds the escalation, so no retry loop can be introduced. - The board-owned action spawns no run. Resolving it (restore to the owner) re-dispatches through the existing recovery-action routes, the same flow as every other stranded escalation. - Native-runtime and legacy-reconciliation predecessors are untouched: they return before the exhaustion check. ## Model Used Claude (Anthropic) — `claude-fable-5-1`, extended thinking, tool use (Claude Code CLI). The incident diagnosis and the choice to escalate rather than re-dispatch were steered by the maintainer. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-25 13:17:48 -07:00
Devin FoleyandPaperclip	bd6caf51bb	fix: preserve restore failure results and stop unsafe retries (#14035) ... Preserve agent output and earlier execution errors when workspace restore fails. Report the restore phase and confirmed saved-plan links. Require verified repair before retrying unsafe archives, while preserving approval states and the retry budget. Verified with full CI, 506 focused regression tests, and Greptile 5/5 with all review threads resolved. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-09-25 11:34:31 -07:00
Devin FoleyandPaperclip	5b09d66183	fix(heartbeat): keep drain-time wakes queued and retry interrupted disposition handoffs (#14028) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The server's heartbeat scheduler queues, admits, and recovers agent runs. > - A control plane arms an instance task drain before it restarts the process, so no new run starts mid-restart. > - Two recovery paths treated that transient hold as a permanent verdict: a wake that arrived during a drain was written as `skipped` and never replayed, and a corrective disposition run that the restart interrupted counted as an exhausted attempt, so the issue went to `blocked`. > - Both outcomes leave an assigned issue with no run and no path until a person notices. > - This pull request keeps drain-time wakes in the durable queue and gives an interrupted corrective run the same bounded transient retry any interrupted run gets. > - The benefit is that a graceful restart never strands or blocks an issue by itself. ## Linked Issues or Issue Description No existing issue found. I searched open PRs that touch the task drain (#13527 adds opt-in termination of active runs; #13965 handles deferred wakes after a stale cancellation). Neither replays drain-time wakes or changes the corrective-handoff escalation. **What happened?** 1. An operator accepted a plan confirmation while the instance was in a task drain (a control plane armed it before a deploy restart). `enqueueWakeup` wrote the assignee's continuation wake with `status: "skipped"` and `heartbeatSkip.reason: "task_drain"`. The drain is process-local and cleared on the restart seconds later, but nothing replays skipped wakes. The issue stayed in `todo` with no run for 20 hours until a person retried it by hand. The stranded-issue sweeper did not help: a `todo` issue whose latest run succeeded is treated as deliberately handed back. 2. On another issue, the watchdog correctly raised `successful_run_missing_state` and queued the single corrective handoff run. A graceful server shutdown (the same deploy restart) interrupted that run with `server_shutdown_interrupted`. On the next boot, `reconcileStrandedAssignedIssues` saw a corrective run at `handoffAttempt >= maxHandoffAttempts` and escalated the issue to `blocked` with a board-owned `missing_disposition` recovery action. The agent never got to finish one corrective attempt. **Expected behavior** A task drain holds admission, not the request. A wake that arrives during a drain must run once the drain lifts or the process restarts. An interrupted corrective run is not evidence that the agent could not choose a disposition; it should be retried like any interrupted run, and escalate only when that retry budget is spent or a finished attempt still leaves no disposition. **Steps to reproduce** 1. Assign an agent an issue and `POST /api/instance/task-drain` with a Cloud control assertion (or call `startTaskDrain({})` in a test). 2. Trigger any wake for that agent (assignment, comment, or an accepted `request_confirmation`). 3. Observe the `agent_wakeup_requests` row: `status = skipped`, `reason = heartbeat.scheduling_suppressed`, `payload.heartbeatSkip.reason = task_drain`. Stop the drain or restart: no run is ever created for that wake. 4. For the second case: let a `finish_successful_run_handoff` run be interrupted by SIGTERM during a graceful shutdown, then boot. The issue moves to `blocked` on the startup sweep without any retry. **Paperclip version or commit** master at e2f1a66aa7 (2026-09-25). **Deployment mode** Managed cloud instance behind a control plane that arms task drains before deploy restarts; the code paths are the same for any operator who uses the drain endpoint. ## What Changed - `enqueueWakeup` no longer writes a wake as `skipped` when the only suppression is `task_drain`. The wake and its queued run land in the durable queue; `startNextQueuedRunForAgent` and `executeRun` already refuse to admit work while the drain is active, and the boot-time `resumeQueuedRuns` pass picks it up after the restart. `worktree_instance` and `database_restore_in_progress` suppression still write `skipped`: those holds are not transient restarts. - `reconcileStrandedAssignedIssues`: when the latest run is a corrective successful-run handoff at its attempt cap **and** that run is `interrupted`, the sweeper schedules the bounded transient retry (via `enqueueStrandedIssueRecovery` → `scheduleRecoveryRetry`) instead of escalating. The retry keeps the handoff context, so it is still the corrective run. If the retry budget is spent, or a finished attempt still leaves no disposition, escalation proceeds exactly as before. Failed corrective runs (adapter or provider failures) are unchanged: they still escalate immediately. - New sweep counter `successfulRunHandoffRetried`, included in the startup and periodic recovery log lines. - Tests: `heartbeat-task-drain-admission-release.test.ts` gains a case that a wake during a drain is queued, held while the drain is active (quiescence still reports true), and runs to completion once the drain lifts. `heartbeat-process-recovery.test.ts` gains two cases: an interrupted corrective run is retried with its handoff context and the issue stays `in_progress`; an interrupted corrective run with a spent transient budget escalates to `blocked` with the usual recovery action evidence. ## Verification ``` pnpm -r --filter './packages/**' build cd server npx vitest run src/__tests__/heartbeat-process-recovery.test.ts # 296 passed npx vitest run src/__tests__/heartbeat-task-drain-admission-release.test.ts \ src/__tests__/heartbeat-worktree-suppression.test.ts \ src/__tests__/heartbeat-scheduling-suppression.test.ts \ src/__tests__/heartbeat-task-drain.test.ts \ src/__tests__/instance-settings-routes.test.ts \ src/services/recovery/successful-run-handoff.test.ts \ src/__tests__/issue-recovery-actions.test.ts \ src/__tests__/heartbeat-comment-wake-batching.test.ts \ src/__tests__/attention-service.test.ts # 226 passed npx tsc --noEmit -p tsconfig.json # clean ``` Live reproduction of the first defect: after this change was written, a board wake issued against a draining managed instance (running current master) was again recorded as `skipped` with `heartbeatSkip.reason = task_drain`, which is the exact row the new test asserts no longer appears. ## Risks - A wake that lands during a drain now waits in the queue instead of being dropped. `getTaskDrainStatus().pendingWakes` counts only in-flight enqueue promises, so quiescence is unchanged and a control plane waiting for the drain is not held longer. If a drain is stopped without a restart, the queued run starts on the next scheduler tick. - The interrupted-handoff retry reuses the existing transient retry budget (two attempts) and the existing `hasActiveExecutionPath` skip, so a sweep cannot double-schedule. Native-runtime corrective runs still return `null` from the recovery enqueue and escalate as before. - Self-hosted instances that never call the drain endpoint see no change on the first path; the second path only changes behavior for corrective runs interrupted by a graceful shutdown. ## Model Used Claude (Anthropic) — `claude-fable-5-1`, extended thinking, tool use (Claude Code CLI). Human-directed: diagnosis of the two production incidents, the fix design, and review were steered by the maintainer. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-25 11:28:54 -07:00
Devin FoleyandPaperclip	e2f1a66aa7	feat: support default-hidden experimental settings (#13980) ... ## Thinking Path > - Paperclip is the open source control plane for AI-agent companies. > - Operators can hide settings that their users must not change. > - The server shares the effective restrictions with the UI and settings API. > - An explicit list must change whenever a new experimental flag is added. > - This pull request adds a wildcard with named exceptions to the existing setting. > - Core applies the policy to its current catalog, so new flags stay hidden automatically. ## Linked Issues or Issue Description Related: #11823 introduced settings visibility. #13907 added workspace isolation visibility. I searched related PRs and issues and found no duplicate wildcard implementation. **What existing behavior does this improve?** Operator control of experimental setting visibility through `PAPERCLIP_HIDDEN_SETTINGS`. **Subsystem affected** Shared settings policy and its existing server health and mutation consumers. **Current behavior** Operators must name every hidden experimental toggle. A new Core flag can become visible until the operator updates that list. **Proposed behavior** `instance.experimental.*` hides current and future experimental toggles. Entries such as `!instance.experimental.enableEnvironments` leave named controls available. Explicit hidden keys and the hidden parent page take precedence over exceptions. **Reason and benefit** Operators can maintain a short list of allowed controls instead of a second copy of Core's full feature catalog. **Breaking changes** Existing explicit lists and unset configuration keep their behavior. The new syntax is opt-in. Older images ignore it, so operators must retain explicit restrictions until those images are upgraded. Visibility does not change feature values. ## What Changed - Expand the wildcard into concrete catalog keys in the shared parser. - Limit exceptions to known experimental controls and preserve explicit restrictions in either input order. - Test a synthetic future catalog addition, duplicate and invalid entries, API rejection, same-value echoes, and the effective health payload. - Document the syntax and the transition for deployments with mixed image versions. ## Verification - Targeted parser, future-catalog, health, and settings-route tests pass: 95 tests across four files. - `pnpm -r typecheck` passes, including Rust checks, with the installed Cargo directory on PATH. - `pnpm build` passes. - All current-head CI gates pass, including the full test shards, Rust, build, browser E2E, and canary dry run: https://github.com/paperclipai/paperclip/actions/runs/36083292578. One unchanged runtime-exposure cold-start test passed on its first retry. - The full local `pnpm test:run` did not pass on macOS/Node 25: the first server group reported 13,356 passed, 18 failed, and 99 skipped, with six failed files (including two failed suite setups). Failures were in unchanged runtime/company skill cache, chat/email connector fixtures, embedded-Postgres setup, and workspace cleanup tests. A standalone filesystem probe reproduced the read-only-directory rename permission failure. Missing connector fixture paths, database startup failures, and two integration assertions also occurred; the remaining local groups were not reached after this group failed. The corresponding CI lanes all pass. These local failures are not claimed as fixed by this PR. - Browser suites were not run because this changes the shared policy, not UI rendering or browser workflows. Health payload and route tests cover the shared UI/API contract. ## Risks A malformed exception remains hidden and is reported as unknown. Exceptions cannot override an explicit hidden toggle or parent page. Older images ignore wildcard syntax; keep their explicit list during a mixed-version rollout. No schema or feature-value changes are included. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use, and code execution. The exact runtime variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-25 08:07:50 -07:00
DottaandPaperclip	1dbffb4c04	fix(daytona): clean up sandbox allocation after create failure (#13979) ... ## Thinking Path > - Paperclip manages AI agents and their execution environments. > - The Daytona provider creates sandboxes before it returns a lease. > - Daytona can allocate a sandbox, then time out while waiting for it to start. > - The SDK throws without returning the allocated sandbox handle. > - Paperclip previously lost the resource identity and could leave a running sandbox behind. > - This pull request keeps an exact creation identity and deletes a matching failed allocation. ## Linked Issues or Issue Description Related: #13882. Searches for Daytona create cleanup and orphan fixes found no duplicate for this path. **What happened?** In [Grok qualification campaign 36080870743](https://github.com/paperclipai/paperclip/actions/runs/36080870743), Daytona sandbox creation exceeded 300 seconds. No native runner or provider session started. The SDK threw, but an ownership-filtered provider lookup found the allocated sandbox still running. The disposable resource was then deleted separately. The original attempt and its failure remain retained. **Expected behavior** A failed create must retain enough identity to clean up an allocated sandbox. Cleanup must never delete another attempt's resource. If the provider cannot confirm cleanup during sandbox lease acquisition, the host must retain a durable cleanup record and retry the exact owned allocation after restart. **Steps to reproduce** 1. Have Daytona allocate a sandbox for an environment acquisition. 2. Make the SDK throw while waiting for startup, before it returns the sandbox handle. 3. Before this fix, acquisition fails without deleting the allocated sandbox. 4. The regression tests reproduce this failure and verify exact-owner deletion. **Paperclip version or commit** Observed at `1b0551bb7c8de3c54f4bee64dbe2c88328b3645e`. The same creation path exists on master; this fix starts from `efce9356b`. ## What Changed - Assign each cold create a unique provider name and creation-attempt label. - On create failure, look up that exact name and verify every ownership label before deletion. - Wait for provider deletion with bounded lookup and deletion calls. - Preserve the original create error after successful cleanup. Report both errors and the provider name when cleanup cannot be confirmed. - Treat a missing lookup after an uncertain create as unconfirmed cleanup: a delayed provider request might still create the sandbox. - Transfer only validated ownership fields through the acquire-lease RPC error; provider exceptions and arbitrary error data are not serialized. - Persist a pending-cleanup lease before the host retries deletion. Reuse the existing cleanup sweep and durable spool, retaining the original environment scope after its row is deleted. - Fence retries by company, environment, run, provider account, unique creation name, and every ownership label. Missing provisional-name lookups remain unresolved. - Journal an ownership-verified provider ID before host-driven deletion. Its absence then confirms cleanup after a lost deletion reply or final database update; failed journal writes block deletion. - Keep ownership labels separate from the mutable input Daytona modifies during create. - Report confirmed host-side deletion accurately while still rejecting the failed acquisition. - Add protocol, malformed-evidence, cross-scope denial, controller-restart, and deleted-environment regressions alongside the original bounded cleanup tests. ## Verification - SDK suite: 92 passed. Daytona plugin suite: 277 passed; six opt-in live tests skipped. Environment-runtime suite: 103 passed, then all four focused observation/recovery variants passed after adding a foreign-observation case. SDK and provider TypeScript checks and `git diff --check` pass. - Regressions failed before the respective fixes: missing durable ownership, misleading successful-cleanup message, and SDK mutation of the ownership labels. - Controlled real-Daytona proof at `d2ac1c95b32ca64daf039c0427137657f351c0e0`: create a real sandbox; inject an SDK failure and a failed immediate lookup; persist the validated ownership envelope; have a child journal the observed provider ID before deleting; deliberately drop its successful deletion receipt; reconcile absence from another fresh process and confirm it with a separate provider read. Passed, with no manual cleanup and zero model calls. The live proof uses an owner-only envelope file; actual host database/spool recovery is tested separately in the environment-runtime suite. - The preceding controlled attempt failed because the real SDK mutated the labels object. That failure is retained. The harness deleted its exact owned sandbox, and a fresh lookup confirmed absence. The fix copies the SDK input labels separately from the ownership snapshot. - [Repository CI](https://github.com/paperclipai/paperclip/actions/runs/36095925399) passes on this exact head, with a clean 5/5 review. The unchanged Cursor remote-command test passed its one bounded rerun after a 10-second timeout; the same test had already passed on the combined tree. The original failure is retained. Earlier `199f0852` CI and immediate-deletion proof are retained, without being promoted to the new source. - Combined-source verification uses temporary PR #13990 against the Grok feature branch. No Docker or Rust build runs on the developer laptop. ## Risks Creation failures now add up to ten seconds for lookup and fifteen seconds for deletion confirmation. The SDK still owns the initial creation timeout. If a request materializes only after the failed lookup, its durable pending record remains eligible for later cleanup; a never-observed creation name is never reported deleted from a 404. A previously observed provider ID can be reconciled as deleted. The host and SDK must be deployed together. Durable handoff applies to plugin-backed sandbox lease acquisition used by heartbeat and runner login. Probe and custom-image interactive setup retain immediate cleanup and explicit failure reporting, but do not use this lease-recovery path. A worker crash before delivering the failure envelope still cannot be recovered through this mechanism. A cleanup error never returns a lease. Successful creation behavior is unchanged except for the provider name and ownership label. ## Model Used OpenAI GPT-6 through Codex, with repository tools and code execution. The exact serving identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-25 10:06:57 -05:00
DottaandPaperclip	c96c751f39	fix(heartbeat): claim task ownership with queued runs (#13973) ... ## Thinking Path > - Paperclip manages agents and their tasks. > - The scheduler can start several runs for one agent. > - Each task still needs one execution owner. > - Task creation and recovery can queue assignments for the same task. > - The scheduler previously marked both runs running before starting either executor. > - This change claims the run and task ownership in one transaction. > - Tasks with different IDs retain concurrent execution. ## Linked Issues or Issue Description Refs #13882. Related queue work: #11830 and #13965; those address different admission and cancellation paths. **What happened?** The Grok subscription Product campaign exposed two assignment runs for one task. Task creation and periodic recovery started runs within 200 ms. Both acquired Daytona sandboxes. One failed before provider work with `paperclip_runner_attachment_staging_not_authorized` because the other owned the task. Fixture cleanup then found an active session. The original failed result is retained in [campaign 36071063537](https://github.com/paperclipai/paperclip/actions/runs/36071063537). **Expected behavior** Only the task execution owner may start provider setup. Competing queued work must wait. Different tasks may use the agent's available slots. **Steps to reproduce** 1. Set an agent's concurrent run limit to two. 2. Queue task-creation and recovery assignment runs for the same task. 3. Resume the queue while holding adapter execution open. 4. Before this fix, both runs become running. Only one has the task execution lock. **Paperclip version or commit** Reproduced on master `8781f06a8` and in the Grok campaign at `4196a4cd`. ## What Changed - Use the same company-scoped task ownership gate for assignments, direct comments, and queued comments. - Leave competing work queued while another live run owns the task. Transfer a terminal pointer only after the tracked executor and durable environment leases/finalization settle, including cleanup on another controller. - Commit the running state and task execution owner together. - Preserve the dedicated review path and the native replacement checkout guard. - Add sixteen database regressions for assignment/comment orderings, unrelated concurrent tasks, and cross-controller cleanup fences. ## Verification - Live subscription Product E2E planning passed on its first attempt in both Daytona (351,197 ms) and local execution (268,883 ms), with 6/6 matchers and cleanup passing in each, on combined source `1b0551bb7c8de3c54f4bee64dbe2c88328b3645e` ([campaign](https://github.com/paperclipai/paperclip/actions/runs/36080870743)). The local screenshots and persisted state show the same plan revised, the first approval rejected, the revised approval accepted, and one completion. The full campaign remains unqualified because of a separate startup retry and EC2 Spot interruption. - The new same-task regression failed before the fix: two running rows instead of one. - All seven focused concurrency cases passed. Three comment cases failed before the shared gate was added. Five cross-controller cleanup cases failed before the durable gate was added. A warm-retention regression also failed before its successful release receipt was admitted; missing/failed receipts and a different retention policy stay blocked. - `node ../node_modules/vitest/vitest.mjs run src/__tests__/heartbeat-stale-queue-invalidation.test.ts` from `server`: 48 passed. Native cleanup admission adds one passing test; task-drain release adds two. Total focused checks: 51 passed. - `git diff --check`: passed. - Current-head [CI](https://github.com/paperclipai/paperclip/actions/runs/36078495577) passes repository typecheck, tests, Rust checks, build, and browser suites. The unrelated repository-form browser shard passed one bounded rerun without assertion or code changes; its original navigation failure remains retained. No local Docker or Rust build was used. - The failed test's exact two sandboxes were checked: one was already absent; the other was deleted after its company, environment, and run labels matched. ## Risks The claim transaction adds a task row lock. It follows task-before-run lock order. A competing run stays queued until the current owner releases the task. The change does not alter attachment authorization, provider permissions, schemas, or public APIs. Review is 5/5 with all threads resolved on `23c0d13b6`. All CI checks pass on that exact head. Live Grok requalification will use the combined runner and scheduler source. ## Model Used OpenAI GPT-6 through Codex, with repository tools and code execution. The exact serving identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge ### Live regression verification The Daytona plan/revise/accept lifecycle in [Grok campaign 36080870743](https://github.com/paperclipai/paperclip/actions/runs/36080870743) passed on its first attempt at combined source `1b0551bb7c8de3c54f4bee64dbe2c88328b3645e`, in 351,197 ms, with all six outcome matchers and cleanup passing. The earlier competing-run failure remains retained. This is one live planning result; the complete Grok matrix and repeated qualification are separate gates, and this campaign also has separately retained infrastructure failures. --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-25 10:06:05 -05:00
DottaandPaperclip	8781f06a87	feat(connections): enable MCP aggregators by default (#13964) ... ## Thinking Path > - Paperclip helps people manage AI agents for work. > - Connections let those agents use external services with explicit access rules. > - Zapier, Arcade, Composio Connect, and Executor already have setup and runtime support. > - Their experimental switch still blocks discovery and setup by default. > - This pull request removes those gates and the Settings toggle. > - Users can connect these providers without enabling an experiment. ## Linked Issues or Issue Description Refs #13755. Refs #13941. **What existing behavior does this improve?** Apps browsing, inline setup, and agent connection search for the four MCP aggregators. **Current behavior** An instance must enable the MCP aggregators experiment before users or agents can start setup. **Proposed behavior** All four providers are available by default on local and managed instances. Old stored and managed values still parse but cannot disable them. ## What Changed - Remove the aggregator gates from Apps, inline setup, server setup, and agent search. - Remove the Settings toggle and its UI hook. - Retain the old setting key only for upgrade compatibility. Normalize it to true and ignore managed overrides, as Apps already does. - Replace opt-in fixtures with default-on coverage. Test old false values, all four setup flows, provider choice, and the removed toggle. - Update current connector guidance and remove the opt-in from the runner acceptance fixture. ## Verification - 306 focused tests passed across eight files: shared remote MCP contracts; server remote MCP lifecycle, aggregator fallback, settings normalization, and managed overlay; UI Apps browsing, setup, and experimental settings. - Server and UI TypeScript checks passed. - UI token gates and `git diff --check` passed. - The full local suite was not run, per the maintainer's instruction. All 54 CI checks passed; two checks were skipped. One unrelated workspace-preview readiness timeout passed on one failed-shard retry. - The setup fixtures use simulated MCP responses. This change does not claim new live provider acceptance. ## Risks - Existing instances now show all four providers, even if the old flag was false. This is intentional. - External provider choice, credentials, company isolation, agent grants, and tool policies still apply. Showing a connector does not authorize an external account. - No data migration is required. The compatibility key keeps old managed configuration documents valid. - Historical Zapier live acceptance remains incomplete in the existing evidence report. The maintainer explicitly requested the default-on rollout for all four existing providers; the report records that scoped exception. ## Model Used OpenAI GPT-6 (`gpt-6-astra`) through Codex, with reasoning, repository tools, and test execution. The context window size is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-24 17:31:32 -05:00
DottaandPaperclip	aa8fc86331	feat(connections): prefer native apps and ask users to choose external providers (#13941) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Connections give agents governed access to external services. > - Native connections should remain the first choice for a supported app. > - Other apps may be available through an external MCP provider. > - The user must know which external provider handles the connection and choose it before setup. > - This pull request adds ranked alternatives and server-authored instructions to connection search. > - Agents can follow the returned instructions while Paperclip validates saved choices and access. ## Linked Issues or Issue Description Related: #13879, which fixed inline MCP provider setup. This PR adds discovery and provider selection on top of that work. **Subsystem affected** Cross-cutting: shared connection contracts, server search and intent services, native runtime, CLI, inline setup UI, and evals. **Problem or motivation** An agent cannot offer a clear external-provider choice when Paperclip has no native connection for an app. Adding provider-specific branches to the core prompt would make those instructions harder to maintain. **Proposed solution** Prefer a native connection. Otherwise return verified alternatives in Composio, Arcade, Executor, Zapier order. Include an external-service disclosure, a question with None, and the next instruction in the search result. Validate the saved human choice before creating a selected fallback setup card. Reuse existing provider accounts and verify underlying app access separately. **Alternatives considered** Do not silently choose a provider. Do not claim that broad execution tools prove support for every app. Reuse existing questions and connection intents rather than add another connection model. **Roadmap alignment** Extends the existing MCP Tool Gateway & Apps and Agent evals & feedback capabilities. The MCP aggregators experiment remains the gate. No duplicate provider-routing PR was found in the public search. ## What Changed - Add a dated support index and authorized cached-tool evidence for external routes. - Return provider questions and next-step instructions from `connections_search`. - Preserve pending choices and declines across continuation. Validate company, task, agent, human, app, and current route eligibility. - Carry the selected app into new setup and account reuse, validate explicit provider requests against persisted human messages, and distinguish provider readiness from app authorization. - Sync native, MCP, REST, and CLI contracts. Keep core agent instructions provider-neutral. - Add production-component Storybooks, focused database tests, and three real-agent browser eval cases. - Record the plan, observed failures, fixes, passing evidence, and acceptance limits. ## Verification - Latest head `586f0e6cd`: 54 checks passed, 2 skipped; Greptile 5/5 and all review threads resolved. - After rebasing on master `18dac1e1e`: 64 focused shared, validator, route-contract, and database tests passed; server typecheck passed. - Embedded-browser test drive on the rebased head: native Jira card, HubSpot external-provider question, Arcade account reuse, one actual MCP read against a local synthetic fixture, reload persistence, and None preventing further calls. A real OpenAI-backed agent performed discovery and continuation. - UX observation: the agent initially combined mutually exclusive request fields; the server rejected it and the agent recovered without changing access. This extra retry remains visible in the transcript. - After rebase: 23 focused eval grader/catalog tests, affected TypeScript checks, token gates, production UI build, and Storybook build passed. Full local tests are intentionally excluded at the maintainer's request. - Before rebase: four browser/real-agent attempts passed: native Jira, None, and reuse of the second provider on two Codex profiles. - Browser evals used an isolated deterministic MCP fixture through the real Paperclip gateway. They do not prove production compatibility with all four providers. - Review `Apps / Connections / Provider choice` in Storybook. Choose Arcade, continue through Access, and verify the app name, external-service disclosure, and URL configuration. - The detailed verification report is `doc/connections/2026-09-23-aggregator-routing-verification.md`. ## Risks - The public support index is finite and can age. Account capability and app authorization still require verification after selection. - Existing installed-tool permissions remain in effect. Provider choice is not a new execution permission boundary. An early Mini attempt skipped search; clearer provider-neutral instructions made the targeted rerun pass. This is not a measured reliability rate. - Explicit requests skip provider confirmation only when a clear persisted human message or saved provider choice supports them. Other phrasing falls back to confirmation; the agent query alone is not consent. These routes do not add tool permissions. - No database migration or legacy Composio broker is added. Real-provider acceptance remains separate from fixture proof. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, code execution, and browser tools. The exact deployment variant and context-window size are not exposed in this session. Product evals separately used the repository's primary Codex and Codex Mini profiles; those agents supplied test behavior, not independent provider compatibility proof. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-24 16:36:49 -05:00
DottaandPaperclip	18dac1e1ef	feat(connections): add experimental memory providers and remote MCP access (#13942) ... ## Thinking Path > - Paperclip manages AI agents and their work. > - Connections give agents governed access to external tools. > - Agents need durable memory across tasks and execution environments. > - Mem0, Zep, Supermemory, Cognee, and Honcho provide hosted memory tools. > - This pull request adds their setup flows behind an experimental toggle. > - It also delivers assigned MCP tools through the native remote Codex runner. > - Operators can connect a provider once and use the same governed tools locally or in Daytona. ## Linked Issues or Issue Description **Problem or motivation** The Apps catalog lacks a complete set of memory providers. Remote native Codex agents also need access to assigned managed MCP tools without receiving provider credentials. **Proposed solution** Add five memory connectors behind the disabled-by-default Experimental memory connectors setting. Use the existing connection setup and permissions UI. Default their tools to Allowed. Preserve company boundaries, operator permission changes, provider scopes, and audit attribution. **Alternatives considered** Direct provider credentials in each sandbox would duplicate setup and bypass the managed gateway. The remote runner instead uses its existing protocol channel to call the gateway on the server. **Roadmap alignment** This maintainer-requested experiment supports the Memory / Knowledge and Connected Apps roadmap areas. It adds provider connections without introducing a separate memory UI. Related connector authoring documentation is tracked in #13692; no duplicate memory-provider implementation was found. ## What Changed - Add provider definitions, official branding, and the experimental setting for all five providers. - Use OAuth for Zep and Supermemory, API credentials for Mem0 and Honcho, and a bundled Cloud API bridge for Cognee with no runtime downloads or subprocesses. - Default memory tools to Allowed and classify destructive actions explicitly. - Fix personal remote credential resolution and propagate provider tool errors. - Relay assigned managed MCP tools to remote native Codex through the runner protocol. Recheck current authority for each call and rotate stale tool contracts. - Add 21 Storybook states and complete OAuth walkthrough fixtures. - Document provider research, sanitized tool inventory, and live local and Daytona proof. ## Verification - Passed workspace typecheck: `pnpm -r typecheck`. - Passed production build: `pnpm build`. - Full local `pnpm test:run`: 13,273 passed, with failures from process/readiness timeouts under parallel load. Reran all 16 affected suites with one worker: 456 passed, leaving two macOS `/var` versus `/private/var` path assertions. Both passed with `TMPDIR=/private/tmp`. No test failures remain unverified. Latest-head remote CI passes all 54 checks (two optional Storybook jobs skipped). Greptile is 5/5 with no unresolved findings. - Latest Cognee gateway regression: 71 passed, including public deployment without a runtime host and immediate recovery after a provider error. Bundled bridge tests: 25 passed. - Browser setup and real agent tasks exercised all five providers. Mem0, Cognee, Zep, and Honcho have successful store/retrieve proof. - Supermemory now has scoped read/write consent. Local storage and real Daytona write, document read, and semantic recall passed; indexing completion was verified before claiming success. - All five providers were exercised through a real Daytona sandbox and its native runner MCP relay. Provider credentials remained on the server. The final bundled Cognee bridge also passed a fresh Daytona store/recall run. All disposable sandboxes were removed and verified absent after testing. - Storybook is rebuilt and contains the experimental toggle, catalog, setup, permissions, and error states. The Zep and Supermemory access steps advance correctly. ## Risks - Provider OAuth scopes and plan limits remain independent of Paperclip tool permissions. An Allowed tool can still be rejected by the provider. - Providers can queue memory indexing; save acceptance does not prove that semantic recall is ready. - Remote tool contracts must stay synchronized with current connection authority. Regression tests cover revocation and stale contracts. - This change has no database migration. Existing connections remain usable when the experimental catalog toggle is disabled. ## Model Used OpenAI Codex, GPT-6 Astra (`gpt-6-astra`), with reasoning, repository edits, code execution, and browser automation. The exact context window size is not exposed in this session. Live acceptance agents used `gpt-5.6-sol`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-24 15:54:38 -05:00
Devin FoleyandPaperclip	2914501c08	test(chat): retire fixture leases before recovery sweeps (#13952) ... Retire verifying chat fixtures and their synthetic endpoint leases after service shutdown. Add a regression that recovers a stale receipt while preserving another company's lease. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-09-24 11:10:25 -07:00
DottaandPaperclip	b0155a681a	feat(slack): connect Paperclip conversations and scheduled messages (#13920) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Slack conversations use the same tasks and agents as the Paperclip board. > - A board reply must reach that Slack conversation and let the agent continue the work. > - An assigned agent also needs its Slack tools during normal tasks and scheduled routines. > - Both paths must keep the linked user's authority, delivery rules, and conversation history. > - This pull request adds those paths and reduces setup friction for Slack bots. ## Linked Issues or Issue Description **Subsystem affected** Server orchestration, Slack connector tools, shared contracts, and chat setup UI. **Problem or motivation** Replies entered in Paperclip did not provide a complete round trip to the linked Slack thread. Slack and board wakeups could select different model sessions for the same task. Agents also lacked their assigned Slack tools outside Slack-origin work, which prevented a routine from sending its responsible user a briefing. Inviting a bot could leave the new channel disabled. **Proposed solution** Mirror human board messages with author attribution and route the agent result to the same thread. Use the same session key across both entry points. Supply Slack tools to the connection's assigned agent in normal tasks and routines, using the current responsible user's verified link. Enable newly invited channels while preserving explicit disabled choices. Add a browser-agent setup prompt to the Slack wizard. **Alternatives considered** A separate Slack scheduler or task dispatcher would duplicate existing Paperclip workflows. Reusing the connection owner's identity would grant the wrong authority. Replaying old channel history could start unintended work. This change uses ordinary task wakeups, routine dispatch, and Slack's original invitation mention event instead. **Roadmap alignment** Extends the shipped Scheduled Routines and governed Apps capabilities. It does not add a separate task lifecycle. Related work: #13828 and #13809. Related test stabilization: #13877. The existing plugin Slack-control proposals are separate from this built-in connector change. ## What Changed - Queue human Paperclip messages for the original Slack thread with display-name attribution and stable delivery identities. Require the author’s current linked Slack identity and recheck access before delivering messages or agent replies. - Apply pause, dependency, cancellation, and closed-workspace guards before explicit Board sends request work and again when the durable outbox dispatches it. - Route agent results back to Slack and preserve model-session continuity, including replies that reopen completed tasks. - Resolve assigned Slack connections for normal agent tasks and routines. Recheck the responsible user's link, membership, and permissions at execution. - Add `slack_open_dm` for the responsible user's bot DM and request the `im:write` scope. - Enable newly discovered invited channels. Keep explicit OFF choices and normal admission and deduplication rules. - Add a copyable Slack setup prompt for a computer-use agent, with Storybook coverage. Share the prompt-button component with GitHub. - Update Slack tool documentation and runtime instructions. - Stabilize the mobile project browser test by waiting for the final canonical route before editing, preserving all persistence assertions. ## Verification - Live staging: invited the bot after the first mention. The channel became enabled and the bot answered that original mention. - Live staging: a normal Paperclip reply appeared in Slack with author attribution. The agent completed the calculation and replied once in the original thread and in Paperclip. - Live staging: a codeword entered in Slack was recalled from Paperclip. A following Slack calculation used the result from the Paperclip turn. Run metadata confirmed the same model session for both entry points. - Live staging: a scheduled routine used `slack_open_dm` and `slack_post_message` to deliver one DM. The existing app was reinstalled with `im:write`. The test routine was paused after verification. - Before the master merge: 397 focused feature tests passed. The continuity fix passed all 76 issue comment/update route tests and six focused route/integration cases. Typecheck, build, and token gates passed. - Review fixes: 47 focused integration cases passed, covering link revocation/replacement, private membership removal, guarded outbox dispatch, concurrent workers, lost scheduler responses, a real one-connection pool, exact reply provenance, and attachment retries. All 99 issue-comment route tests and the Slack catalog browser test passed. - Full local typecheck, production build, token gates, and module-boundary checks passed. The full local test command passed 25,915 tests before a 15-second timeout in `issue-thread-interaction-routes.test.ts`; that entire suite passed on isolated rerun (81 tests). Remaining serialized coverage is provided by the current-head CI shards. - An unchanged Cursor adapter test hit its 10-second limit in CI; all five tests in that file passed on a local rerun in 3.11 seconds, and the failed CI shard passed on its single retry. - The preview-server readiness test passed a local rerun (28 tests). The mobile-project readiness fix passed three repetitions of both browser tests (6/6). - Final commit `64ac0d9897f4353375996f1b1b38e5040bdeb0a0`: all CI gates passed, including all eight browser shards, all server/chat suites, typecheck, build, runner checks, and security checks. Greptile reviewed this exact commit at 5/5; all review threads are resolved. ## Risks - Human messages on a Slack-linked task now publish to its Slack thread. The task banner states this behavior. Incoming Slack messages and internal agent bookkeeping must not echo back. - Normal tasks and routines can now use the assigned bot. Authority remains bound to the current responsible user's link; it does not fall back to the connection owner. Revocation, private-context limits, and queued-write checks still apply. - Existing Slack apps need `im:write` and a reinstall to open DMs. Other existing capabilities remain available without that scope. - New invited channels default to enabled. Explicit disabled choices remain disabled. Channels created by bot tools still require a person to enable responses. - No database migration or new provider credentials are required. ## Model Used OpenAI GPT-6 through Codex, with reasoning, code execution, GitHub CLI, and browser tools. The runtime does not expose a more specific model build identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-24 10:51:45 -05:00
Devin FoleyandPaperclip	8ee8f1fd6e	ci: retire recurring public cloud image builds (#13827) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Core publishes standard images and source verification for downstream services. > - Managed services can now compose private images from the signed standard image. > - Core still builds a second public cloud image on every master push and release. > - That duplicate producer consumes build capacity and retains an obsolete readiness contract. > - This pull request retires recurring cloud publication while preserving the standard producer and rollback artifacts. ## Linked Issues or Issue Description Refs #13797 and #13789. Related: #12856 changes image dependency packaging; it does not retire this producer. **What existing behavior does this improve?** Core's recurring Docker publication and Cloud readiness workflow. **Current behavior** Master pushes call the legacy cloud publisher from Cloud readiness. Release tags and manual Docker runs call it too. Canary promotion also requires the legacy image. **Proposed behavior** Publish standard Core images and retain `Cloud source verified v1`. Let downstream services build their managed image. Keep explicit commit previews and existing images available. ## What Changed - Remove `docker-cloud.yml`, its master and release callers, and its unused cache selector. - Remove the legacy image/migrator wait and `Cloud deployable v1` job. Keep the full source verification workflow and exact source-proof name. - Make canary promotion inspect and promote the standard image only. - Preserve signed standard-image publication, direct migrator publication, and explicit `release.yml` previews. The preview path still uses the Dockerfile `cloud` target. - Update workflow, preview, build-stamp, and packaging tests. Exercise the promotion shell with mocked registry commands, including missing-image and missing-tag cases. - Document frozen legacy aliases, consumer requirements, preview compatibility, and rollback retention. ## Verification - All 377 workflow tests pass: `node --test .github/scripts/tests/*.test.mjs`. - All 129 release-registry tests pass: `pnpm test:release-registry`. - Focused source-proof, standard-image, preview, and workflow tests pass: 256 tests. - Focused image packaging/build-stamp tests pass: 16 tests. - Actionlint passes on all three changed workflow files. `git diff --check` passes. - Full local `pnpm build` and `pnpm -r typecheck` pass. - The policy follow-up updates an old assertion that required the removed readiness job. All 37 source-proof/release-workflow tests pass locally. - Full local `pnpm test:run` did not complete successfully while the Mac ran out of disk space. No full-suite pass is claimed. Removed 1.2 GiB of generated Cargo output from this isolated worktree with `cargo clean`. GitHub CI passed on the final head: 52 successful checks and 2 optional skips. - Fresh Greptile review for `4f5fe1951f0bd7f7739cf6655d395ff78f1ed944`: **5/5**, successful current-head check, zero review threads. - September 23 refresh: the unchanged PR head merges cleanly with current master `db8f8fe5b73a2697684a30261b0d306a9c631aba`. In an isolated temporary worktree, all 377 workflow tests and 29 release/preview tests pass on the combined tree. `git diff --cached --check` passes. - Refreshed Actionlint workflow validation passes with ShellCheck disabled. Full Actionlint reports the same 10 existing ShellCheck diagnostics as master, with no added diagnostics. No source changes or new PR commits were needed. - The full local build/typecheck and current-head Linux CI results above remain the verification for the unchanged PR head. They were not rerun for this metadata-only refresh. No image publication or tenant deployment was initiated for this refresh. ## Risks **Deployment prerequisite satisfied (September 23):** The combined cleanup release is deployed to staging and production, and production Support is verified. Active managed-fleet automation uses standard-image composition. Explicit immutable previews remain supported by the retained preview publisher. This PR is ready for maintainer review; keep auto-merge disabled and wait for explicit merge authorization. - A consumer still selecting `Cloud deployable v1` will stop advancing at the last legacy-ready commit. Confirm active automatic consumers use the standard-image composition contract before merge. - Legacy cloud release-channel aliases stop advancing. Standard self-hosted aliases continue. - This PR deletes no registry images, cache tags, migrators, credentials, or runner infrastructure. Existing immutable releases remain usable for rollback. - Explicit legacy previews remain for commit-specific operator deployments. Retiring that compatibility path requires a separate consumer migration. - These changes affect CI publication, not database schema or application behavior. ## Model Used OpenAI Codex, GPT-6. The runtime does not expose a more specific model identifier or context-window size. Used repository inspection, reasoning, code editing, shell tools, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-23 19:35:19 -07:00
DottaandPaperclip	a6448cd060	fix(runner): tolerate Codex account notifications during active turns (#13902) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Rust runner connects task runs to Codex app-server sessions. > - Codex sends account notifications on the connection during authentication updates. > - These notifications have no task, thread, or turn identity. > - The runner treated a valid account update as an invalid task event and stopped the run. > - This pull request classifies account updates as connection information while preserving task identity checks. > - Sandbox images can also contain an older runner with compatible metadata. The server must compare its bytes before reuse. > - Slack conversations use the deployed runner and can complete when Codex refreshes account state. ## Linked Issues or Issue Description Related: #13853. That change fixes the supported Codex version range. This fixes a separate notification failure after the version check succeeds. **What happened?** An active Codex task stopped with `thread_binding_mismatch` when the provider emitted `account/updated` without a thread ID. Slack showed that the agent stopped before completing its turn. **Expected behavior** Connection-level account updates must not stop a task or acquire task authority. Account details must not appear in task output. **Steps to reproduce** 1. Start a task with the native Codex runner. 2. Emit `account/updated` during the turn, with `authMode` and `planType` but no thread ID. 3. Observe that the old runner rejects the notification and fails the turn. **Paperclip version or commit** Reproduced after #13853. The fix is based on `b648d8cdd`. **Deployment mode** Cloud staging with a remote Codex runner and a Slack chat connection. ## What Changed - Classify `account/updated` and `account/login/completed` as connection information. - Reject account notifications that contain execution identity fields. - Reuse the existing bounded diagnostic path without publishing account payloads. - Test both notification types during two consecutive turns. Preserve existing identity rejection tests. - Compare preinstalled sandbox runner bytes with the controller artifact. Stage the deployed binary after a mismatch, failed checksum, or timeout. Keep exact retained artifact reuse. ## Verification - `cargo test --release --locked -p paperclip-runner-core`: passed, 582 test executions; two existing ignored cases. - `cargo test --release --locked -p paperclip-runner-core --test codex_provider`: 89 passed; two existing ignored cases. - `cargo fmt --all` and `git diff --check`: passed. - Repository-wide `pnpm -r typecheck`: passed. Server typecheck also passed after the artifact-selection change. - Server executor suite: 456 tests passed, including preinstalled digest match, mismatch, checksum failure, and timeout. - Repository-wide Vitest and build are running. - Deployed `5648d90d5dd762c5a8b697face269f6eea9cd59d` to one staging stack and verified its serving SHA. - Retried the failed conversation through the Slack UI. The agent replied and the task completed. - Sent a fresh Slack mention asking which bots belong to the channel. Verified successful `slack_members` and `slack_user` calls, a successful run, and the delivered Slack answer. - Sent a follow-up in the same Slack thread without another mention. The agent returned the requested names from context, with one final reply. ## Risks - Only two known connection-level notification methods change behavior. Unknown authoritative methods and malformed or conflicting identities still fail closed. - An older sandbox runner now requires one upload when its bytes differ from the controller artifact. Remote OS and architecture checks still apply. - No schema, credential, permission, Codex version-range, or UI changes. The Codex minimum remains 0.149.0. ## Model Used OpenAI GPT-6 (`gpt-6-astra`) through Codex, with reasoning, code editing, terminal tools, and browser verification. The context-window size is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-23 15:43:46 -07:00