PaperClipAI

0 Fork 0

mirror of https://github.com/paperclipai/paperclip.git synced 2026-10-23 09:07:46 +02:00

Author	SHA1	Message	Date
Nicky LeachandPaperclip	42715d459f	Support authenticated WebSocket routes and agent views in plugins (#17111 ) ## Thinking Path > - Paperclip manages agents and their work. > - Plugins provide optional views and connect to external runtimes. > - A live runtime view needs a binary browser connection. > - The current plugin API supports JSON routes but cannot proxy that connection. > - Agent detail tabs are declared in the plugin contract but are not mounted on agent pages. > - This PR completes those two host capabilities and keeps upstream credentials on the server. ## Linked Issues or Issue Description Subsystem affected Plugin scoped API routes and agent detail views. Problem or motivation Plugins cannot expose a binary or text stream through a browser session without exposing an upstream credential or adding a separate browser gateway. Agent-scoped detail contributions also have no mount on the agent page. Proposed solution Allow declared board GET routes to select a server-only WSS upstream. Run upgrades through the existing HTTP authentication pipeline and trusted Origin check. Add agent company resolution with the existing `agent:read` decision. Mount contributed detail views in agent navigation. Alternatives considered A separately hosted viewer needs its own browser credentials and cross-origin policy. JSON or SSE transport adds encoding and buffering to a binary protocol. Roadmap alignment This extends the existing plugin API and detail-tab contract. It adds no desktop provider to Core. I checked ROADMAP.md and searched related open PRs; I found no equivalent transport and agent view change. ## What Changed - Add declared WebSocket routes requiring board authentication, GET, and `http.outbound`. Reject body-based company resolution because browser upgrades cannot send a body. - Keep worker-selected upstream URLs and headers out of browser JSON responses. Strip browser cookies and Origin. Preserve binary frames and socket backpressure. - Close both sockets on disconnect or worker shutdown. Reauthorize on reconnect after a five-minute connection lifetime. Preserve HTTP idle-work accounting for the stream lifetime. - Resolve agent scope through `agent:read` and mount generic agent plugin detail views, with current agent/view breadcrumbs and browser titles. ## Verification - Final head `9d8d269040acf9dc7bb65524876406af522dacb5`: CI passes (including Runner Rust, build, and browser E2E); Greptile 5/5; no unresolved review threads; no merge conflicts. Verified 2026-10-21. - Core server TypeScript check: `node node_modules/typescript/bin/tsc -p server/tsconfig.json --noEmit`. - Focused server suites: 17 passing tests (`server/src/__tests__/plugin-scoped-api-routes.test.ts` and `server/src/http/server.test.ts`), including arbitrary build-output and metrics channels with real local text and binary WebSocket round trips, disconnect cleanup, denied actors and origins, credential exclusion, agent-read denial, and cross-company 404 equivalence. The test replaces TLS with local HTTP while retaining the real upgrade and binary stream. - UI suites: 9 passing tests for independent Insights and Notes contributions, removed contributions, built-in route preservation, sidebar links, direct plugin entry and heading updates after tab navigation. These contributions do not require the WebSocket transport. - Shared validator suite: 42 passing tests, including rejecting body-based WebSocket resolution while retaining ordinary HTTP body resolution and accepting query/entity scopes. - UI token gates pass. - UI TypeScript check: `node ../scripts/storybook-catalog.mjs` then `tsc -b`. - Shared and plugin SDK builds pass. - Full Runner build reached its unrelated Rust step and stopped because cargo is unavailable. Runner TypeScript build was run separately. No live remote VM was used. ## Risks - This adds a browser ingress path for trusted installed plugins. The plugin must select its upstream from a trusted binding. HTTP requests cannot retrieve the upstream descriptor. - Membership or binding changes are rechecked on reconnect. Connections last at most five minutes, and worker shutdown is checked every five seconds. - Initial CI caught a missing cross-company 404 guard and an old test that assumed exactly one upgrade listener. Both were corrected. Greptile's body-resolution and stale-heading findings are fixed with regression coverage; latest-head CI passes and Greptile scored 5/5 with no unresolved findings. - No schema migration. Rollback restores the previous Core image; plugins using this transport must be disabled or rolled back first. ## Model Used OpenAI Codex, based on GPT-6, with reasoning and code/tool execution. The exact runtime model ID and context-window size are not exposed in this session. ## Checklist - [x] Thinking path and issue description included - [x] Model described - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] No internal issues, private links, or runtime credentials included - [x] Descriptive branch name - [x] Focused tests and relevant typechecks run - [x] Tests and documentation updated - [x] Risks documented - [x] All CI gates green - [x] Greptile 5/5 with no unresolved findings - [x] I will address review findings before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-21 10:10:00 -07:00
Nicky LeachandPaperclip	00331e0254	Connect native Runner admission to plugin-owned workspaces (#16978 ) ## Thinking Path > - Paperclip manages work for AI agents. > - A plugin can own an execution environment and its files. > - #17029 supplies the task driver and bounded setup verification. > - Heartbeat still prepares workspace files on Core and expects a shell transport. > - This PR gives the selected plugin ownership of workspace preparation and native task execution. > - Core keeps admission, tool authority, and recovery checks. ## Linked Issues or Issue Description Subsystem affected Heartbeat workspace admission and native Runner execution. Problem or motivation A task-only plugin cannot run through Core filesystem staging or shell execution. Native admission and recovery must use the same lease and workspace ownership. Proposed solution Use the task driver from #17029. Resolve workspace and project identities without Core file operations. Connect native dispatch and restart recovery to that driver. Alternatives considered Keep workspace admission and dispatch in one change. A workspace change alone would leave dispatch on the previous transport. An execution change alone would leave Core staging files for the wrong host. ## What Changed - Enable plugin environment selection for native workspace sessions. - Resolve workspace metadata and authorized project references without Core checkout, copy, or provision operations. - Preserve agent-default and primary workspace policies. Use the pinned workspace provider for plugin-backed project storage. Reject unsupported Core service commands and cross-owner reuse. - Connect native execution to the prepared task driver. Keep assigned MCP authority on Core. - Save exact checkpoint identity, transfer selected account refreshes, and require termination proof before replacement or cleanup. - Complete managed plugin workspaces through their storage provider. Use the same check after a restart. Keep branch validation and leave task-owned files on the plugin. - Include default instruction content in runtime synchronization when no instruction directory exists. - Defer new-agent Copilot verification until the saved-agent lifecycle can acquire the agent-owned computer. ## Verification - Rebased onto `origin/master` at `898144f4b3`. Master advanced once during verification. Both manual conflicts joined import blocks: one in `native-session-executor.ts`, and one in `heartbeat/runtime-execution.ts`. The seven patches retain their logic. The first rebased head passed repository typecheck, build, all 862 focused execution/finalizer cases, and all CI gates. The latest head `93e5a72892` passes server typecheck and all 880 focused execution, finalization, and execution-control notification tests. All 54 CI/review checks pass, with two expected skips. Greptile ran automatically and scored this head 5/5 with no unresolved findings. No new review or Apex request was made for these import-only resolutions. - The successful CI workflow initially left three stale job-check reports. Targeted policy and summary reruns cleared the reports. No source change or test fix was needed. - Repository typecheck and build pass: `pnpm -r typecheck` and `pnpm build`. - Focused native execution and restart tests pass: 795 cases. The account and harness suites pass: 159 cases. Workspace admission passes: 48 cases. The SDK task lifecycle, workspace API, runtime-hook, and instruction checks also pass. - The heartbeat integration case uses the merged workspace provider contract and reaches native dispatch with no project files on Core. The new-agent case confirms that creation leaves the agent in `preparing` until lifecycle verification. - Map validation and the Copilot source-identity check pass. No Runner implementation or source-identity bump is in this PR. - The first rebased workspace run found two stale path assertions and a recovery guard that did not recognize task-owned targets. These are fixed. The storage integration fixture also needed its persisted source binding; the production authorization check stays in place. - Two prior review findings required rejection of Core service startup and cross-owner workspace reuse. The existing admission fixtures now check both failures. The installed-account authentication finding was fixed in merged #17029. - Tests reuse the current fixtures. No new provider test matrix or live Runner process is added for these fixes. - The prior head `2db273ec12` passed CI on its first attempt and reached standard Greptile 5/5. Apex scored 4/5 and found local-folder checks during plugin completion and recovery. - The completion fix reproduces both failures. It passes all 110 cases in the runtime execution, workspace finalizer, and workspace provider integration suites. Server typecheck and map checks pass. CI passed on `62d8335b4e` on its first attempt, and standard Greptile scored 5/5. The next Apex review scored 4/5. It found that a failed plugin inspection could abort restart recovery. - The restart fix records a failed workspace receipt and uses the current run retry policy. A regression first reproduced the aborted pass. It now checks retry state, retained results, and completion of another workspace in the same pass. All 68 finalizer cases and server typecheck pass. CI passed on `7d4016323b` on its first attempt. Standard Greptile and Apex both scored 5/5. Apex left one P2 about lost failure codes. - The diagnostic fix keeps the known branch and owner mismatch codes in receipts and run recovery state. It gives repair guidance for those codes and keeps unknown plugin errors generic. All 96 affected finalizer cases and server typecheck pass. Before the rebase, required CI passed on `c2fa61c6c8` on its first attempt. Standard Greptile and Apex both scored 5/5 on that head, with no unresolved findings. The new head retains these reviewed patches. - The earlier Superagent scan on `c2fa61c6c8` was inconclusive after a pinned merge-base checkout failed or timed out. The current scan on `93e5a72892` passes. - The exact rebased head passes all 48 workspace cases and server typecheck. The first local test launch could not bind PostgreSQL inside the sandbox. The authorized local-server run passed. - The prior head reached standard Greptile 5/5. Its first CI run hit an unchanged Runner cleanup timeout and an external build-runner shutdown. Both jobs passed on the single retry. The timeout cause is not established. - A local regression check found that a retained Core workspace could also enter a plugin task without file transfer. Core now rejects this move before provider work. The existing ledger test covers both directions and confirms that the source files stay intact. - The first active standard review scored 4/5. It found that a fresh plugin retry could enter Core Git preparation. This mode now uses the existing unsupported-storage error before workspace resolution or acquisition. Both unsupported-mode cases pass, and server typecheck passes. - No live Cloud or provider validation was run. ## Risks - The prerequisite #17029 is merged. This PR targets master. - Native plugin tasks support per-turn workspace sessions. Core service commands and fresh retries that import retained files remain unsupported. A workspace storage plugin can use the modes and strategies it declares. - Core-managed credentials support Codex and direct Claude accounts. Other managed credential routes fail closed. Other harnesses can use an installed account. - A provider must preserve its prepared configuration and report process termination correctly. Uncertain termination keeps the task pending cleanup. - Cloud deployment and live provider validation remain separate. ## Model Used OpenAI Codex, GPT-6. The exact model ID and context window are not exposed in this session. Tool use includes code editing, local tests, and the GitHub CLI. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal issue details - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-21 08:22:48 -07:00
Nicky Leach	65f59e339a	Extend native instance secrets to project lifecycle plugins (#17034 )	2026-10-20 20:09:16 -07:00
Nicky LeachandPaperclip	e5351b240a	Verify native Runner setup through plugin tasks (#17029 ) ## Thinking Path > - Paperclip manages work for AI agents. > - An environment plugin can own a Runner task. > - The merged launch-preparation contract lets the execution owner build the provider configuration. > - Core still needs a task process owner and a setup test that uses that configuration. > - This PR connects the task API to the Runner process-owner contract. > - Operators can test a plugin environment before Core sends it task work. ## Linked Issues or Issue Description Subsystem affected Runner, plugin SDK, and agent environment tests. Problem or motivation A task-only environment cannot use the shell-based setup path. Core must verify the Runner hello exchange and process cleanup through dedicated task operations. Proposed solution Add a lease-bound task driver and use it for a bounded setup probe. Keep heartbeat admission and task execution in #16978. This PR follows the merged launch-preparation contract in #16997. Alternatives considered A provider declaration alone cannot prove that Runner and the provider connect. A shell bridge would give the plugin an unnecessary command interface. ## What Changed - Pass structured Runner launch data to process owners. Support an external connection and explicit detach. - Add typed launch authentication checks and checkpoint operations. Check lease, run, project, and retained-task authority. - Connect launch preparation, PRP, termination proof, and credential handoff to the task driver. Reject local environment overrides during installed-account setup. Pass saved-account identity and credential home directly from Core. - Run a short-lived setup probe through the existing agent environment test API. Require a verified provider hello and cleanup. Keep saved Copilot account checks metadata-only through a dedicated task operation. - Update the Copilot source identity because the shared transport source changed. Keep provider behavior and the earlier identity fixture. ## Verification - CI passes on `f987adec7`: 55 successful checks and 5 expected skips. Standard Greptile and Apex are both 5/5 with no open findings. The branch merges cleanly with current master. - Repository typecheck and build pass after the rebase onto the workspace-provider contract. Server typecheck and build also pass after the final setup fix. - The 110 focused lease, SDK, setup, Copilot, and route cases pass after the rebase. All 14 setup cases pass after the final change. These include a real local Runner with a test provider. The saved-agent model-discovery case also passes. - Apex found that Core's relative workspace marker failed the driver path check, and non-Codex sessions omitted the observed model. Both failures were reproduced and fixed. The probe leaves physical workspace selection to plugin preparation. The transport reads each provider's observed model and exposes it to the driver. All 14 setup cases, 6 transport peer cases, and both existing real Runner/OpenCode cases pass. Runner build and server typecheck pass. - The coverage audit removed two redundant cases, repeated fixture code, and two extra real Runner launches. It kept authorization, credential selection, cancellation, lost replies, termination proof, checkpoint identity, and provider setup checks. - The Copilot source-identity check passes. The added-line secret and PII scan passes. The unmerged v52 attestation was regenerated after the shared transport fix. - CI on `13faeff02` caught one stale Copilot digest assertion after attestation regeneration. The literal now matches the candidate attestation, and all 49 installation cases pass. That CI lane otherwise passed 1,818 cases. - An earlier CI run hit an unchanged UI toast timing test. The separate fix is tracked in #16952. Current CI passes that shard. CI also caught a stale model-discovery assertion in this change; the assertion is fixed. - The local full-suite run was stopped when master changed. The final CI run supplies exact-head full-suite validation. No live Cloud or provider test was run. ## Risks - Native hello tests require a saved Core account or the installed account. Local environment overrides are unsupported for installed-account probes and fail before the provider runs. - A native task provider must declare launch preparation and authentication-check version 1. It must keep the checked configuration fixed through submission. Core rejects providers that cannot make this guarantee. - A lost task reply does not prove process termination. Core retains grants until the provider proves termination and Core saves required state. - This PR does not enable native heartbeat dispatch through plugin environments. Cloud deployment and live provider validation remain separate. ## Model Used OpenAI Codex, GPT-6. The exact model ID and context window are not exposed in this session. Tool use includes code editing, local tests, and the GitHub CLI. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal issue details - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-20 18:13:09 -07:00
Nicky LeachandPaperclip	8b4ab65b1c	Add plugin-owned project storage and task workspaces (#17001 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Tasks can use isolated repository workspaces. > - Core currently prepares those repositories on its own filesystem. > - A storage provider may only expose repositories inside remote agent computers. > - Project storage and task workspace preparation must have one provider owner. > - This pull request adds that contract and keeps execution as a separate plugin interface. > - Core keeps authorization, durable task bindings, and lifecycle fences. ## Linked Issues or Issue Description Subsystem affected Plugin SDK, project workspace policy, heartbeat preparation, execution leases, and workspace lifecycle. Problem or motivation Core cannot prepare a task workspace when its repository storage is unavailable on the Core host. An execution lease must not be required just to prepare persistent storage. Proposed solution Let one plugin provide project resource lifecycle and workspace prepare, inspect, restore, and remove behavior. Core reserves a task workspace before it calls the plugin. The plugin returns an opaque durable reference. The matching execution plugin receives that reference on its task RPC. Alternatives considered Preparing repositories on Core would require a provider mount on Core. Preparing them only during task launch would tie persistent workspace lifecycle to execution leases and agent availability. Roadmap alignment This extends the remote provider model in ROADMAP.md. Related execution work is in #16978. This PR does not replace that task bridge. Native execution of managed storage requires the bridge; this change fails closed when it is absent. ## What Changed - Add one storage provider declaration with supported modes/strategies, a capability, a validated RPC contract, and a test harness. - Persist provider ownership, repository revision, and execution-workspace identity before external effects. Reconcile retries with a stable operation ID. - Prepare provider storage before environment acquisition. Keep Core filesystem preparation out of this path. - Pin the durable reference to the matching execution plugin and validate the lease binding before task dispatch. - Route inspection, reopen, and removal through the existing lifecycle process. Keep local data when remote recovery is uncertain. - Keep the application database pool free during lifecycle locks. Preserve pending workspace selections, validate source edits before retirement, and retry unfinished closed storage cleanup. - Protect provider metadata and live repository sources from caller edits. Storage edits retire only closed records with confirmed removal; retired records cannot reopen. ## Verification - Current head `8214b3ca5`: all 25 provider lifecycle tests pass with `maxConnections=1`. They cover preparation, inspection, reopening, removal, a paused provider with responsive ordinary SQL and a competing source edit, pending selection preservation, rejected source edits, cleanup retry scheduling, and providers that cache operation results, and committed cleanup notifications for storage intent and recovered state. - Final frozen-tree lifecycle run on `8214b3ca5`: all 318 tests passed in 10 files, covering workspace services, reopen, compaction, task binding, project sources, heartbeat preparation, display caching, and source locks. - Direct server `tsc --noEmit` passes. The SDK build and all 128 SDK tests passed during initial verification; later SDK changes are documentation only. - Full CI on the previous head `ec5c7209a` passed all 55 applicable checks; 2 conditional checks skipped. Standard Greptile reviewed current head `342e6d9c3` at 5/5 with no unresolved threads. All 55 applicable CI checks passed on `342e6d9c3`; 2 conditional checks skipped. The Slack browser and CLI startup jobs passed on their single unchanged-head rerun. Its Apex review returned 4/5 with one pending-restoration issue. Commit `1b58943b1` reconciles that durable intent before cleanup, advances the generation without reopening the Core row, and blocks direct removal or retirement while intent remains unresolved. Standard Greptile reviewed `1b58943b1` at 5/5 with no new findings. All 54 applicable CI checks passed on `1b58943b1`; 3 conditional checks skipped. The chat job passed on its single unchanged-head rerun. Apex then returned 4/5 for a missing cleanup wake after recovery. Commit `8214b3ca5` atomically registers cleanup intent before storage intent and recovery-result writes. Its provider test observes real committed notification hints and the visible advanced generation. Standard Greptile reviewed `8214b3ca5` at 5/5 with no new findings. All 55 applicable current-head CI checks passed on the first run; 2 conditional checks skipped. Final Apex verification completed at 5/5 on this exact head: 38 files reviewed, no new comments, and no unresolved threads. The PR is ready for review and mergeable. - The first Apex review found five issues after a standard 5/5: application-pool starvation, stale pending selections, retirement during rejected source edits, lost cleanup retries, and reused inspection identities. Commits `6d72e7fd4` and `342e6d9c3` fix each with regression coverage. Archive lock waits use dedicated connections, and source edits fail promptly if the lifecycle is busy. Retirement compares normalized values. The normalized-source regression initially failed with the raw-value comparison; after moving the check after normalization, a clean provider-suite rerun passed all 24 tests at that stage. Those threads are resolved; the earlier Apex 0/5 is not a passing acceptance result. - Earlier local integration attempts could not bind localhost inside the sandbox. The same suites passed with permission to start local PostgreSQL. - The earlier broad local `pnpm test:run` overlapped edits and reported stale-revision and checkout-path/Git-fixture failures. It was not a final-tree result. Full CI subsequently passed; the original local failure remains part of verification history. - Local repository-wide typecheck/build cannot complete because `cargo` is unavailable. Direct server TypeScript checks and affected tests pass; full Rust and build validation comes from CI. - An earlier browser run on `ff918c5a8` failed in the unchanged case-filter journey: expected “Done,” received “All statuses,” with an announcement handler recorded during the assertion. The cause is not confirmed. [The failed job](https://github.com/paperclipai/paperclip/actions/runs/39500611857/job/118896954508) remains verification history. The unchanged shard passed in the next full CI run on `ec5c7209a`; no UI or fixture change was made. - Current-head browser shard 2 failed waiting five seconds for the mocked Slack setup Success heading. This PR changes neither that UI nor its fixture. [The failed job](https://github.com/paperclipai/paperclip/actions/runs/39506070513/job/118916021232) remains verification evidence. The cause is not confirmed; the single unchanged-shard rerun passed; no UI or fixture change was made, and the original cause remains unconfirmed. - The workspace test shard passed its 3,978 general tests, then its CLI company-import/export fixture timed out after 30 seconds waiting for startup health before creating any company or workspace. [The failed job](https://github.com/paperclipai/paperclip/actions/runs/39506070513/job/118916021855) remains verification history. The child reached external-PostgreSQL startup without exiting or reporting an exception. Its cause is not confirmed; the single unchanged-job rerun passed, including the CLI fixture. No startup or fixture change was made, and the original cause remains unconfirmed. - The new cached-removal regression initially failed because its fake worker advertised availability for unrelated fixture plugins, so another workspace changed its fake checkout state. Worker availability now matches the fixture plugin; the complete 25-test provider suite passes. - On `1b58943b1`, the chat shard timed out polling for one conversation after an authorized GitHub description edit. [The failed job](https://github.com/paperclipai/paperclip/actions/runs/39510521657/job/118930193418) remains evidence. The exact case passes locally against PostgreSQL (1 passed, 1,380 unrelated cases skipped), and this PR changes neither that test nor its handler. The original cause is unconfirmed; the single unchanged-job CI rerun passed without a code or fixture change. The original cause remains unconfirmed. - The diff was checked for credentials, private references, and generated artifacts. ## Risks This is an additive provider contract with no database migration. Existing projects keep their local behavior unless their policy selects a storage plugin. A selected provider must be ready and enabled. Unknown or unsafe cleanup fails closed. Managed native execution depends on #16978. Core refuses a host Runner fallback. Providers must fence lifecycle generations, preserve ambiguous retries, and report remote recovery honestly. Provider-specific live storage and guest mount qualification remain separate from these Core contract tests. ## Model Used OpenAI Codex (GPT-6), with repository inspection, code execution, and tests. The exact model variant was not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run tests locally and they pass (affected suites; broad-run limits are documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-20 16:35:33 -07:00
Nicky LeachandPaperclip	dc6dd23ea4	Add plugin Runner preparation task contract (#16993 ) ## Thinking Path > - Paperclip manages work for AI agents. > - Environment plugins can own Runner tasks. > - Core selects provider policy and runtime assignments. > - The execution environment must resolve its own paths and configuration. > - The task API needs a preparation exchange before execution. > - This PR adds that exchange under the existing lease authority. ## Linked Issues or Issue Description Problem or motivation The environment task API can start a Runner. It cannot return a prepared provider configuration from Core requirements. Proposed solution Add a versioned `prepare` request and `prepared` result. Keep the provider configuration bound to its task and requirements. Check the same company, run, project, and retained-session authority as submission. Alternatives considered A shell preparation command would expose execution details in the API. Provider-specific ownership flags would spread this contract across backends. Roadmap alignment This extends the existing plugin and remote execution APIs. It does not add a separate execution system. Refs #16977, #15406. The prerequisite #16977 is merged. This PR now targets master. Runner transport and provider implementations follow separately. ## What Changed - Add strict launch requirements and prepared result schemas. - Add the preparation capability version and saved configuration ID. - Apply existing lease and resource checks to preparation. - Document idempotency, cancellation, recovery, and cleanup requirements. ## Verification - `pnpm -r typecheck` passed. - Plugin SDK build passed. - All 55 Dot Runner and native Codex server integration tests passed with Cargo available on PATH. - All 18 environment task runtime tests passed. These include preparation dispatch, inactive leases, wrong run identity, unauthorized projects, result identity, and unknown command rejection. - The broader root suite on the parent refactor finished with 22,420 passing tests, seven failed tests in unchanged Git and diagnostic code, and two suite setup errors because Cargo was absent from its PATH. This PR does not claim a passing full root suite. - No live provider calls or deployment were run. - Standard Greptile and Apex each scored commit `d3ac15906240b7066f5ff505ed2ffd9ef5dd43c1` 5/5. There are no unresolved review threads. - All 55 CI checks passed. Two optional checks were skipped. The branch is ready and mergeable against master. - A fresh master fetch and merge-base check passed immediately before the Apex request. ## Risks - This defines a new optional worker operation. Existing providers keep their current operations. - A provider must validate policy and resource bindings. Schema validation alone does not validate a concrete provider descriptor. - This PR defines the exchange. It does not connect native execution to a preparation provider. ## Model Used - OpenAI Codex, GPT-6. Reasoning and repository tools. The exact serving model identifier and context size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-20 11:41:49 -07:00
Nicky LeachandPaperclip	33cc567f75	Add lease-scoped Runner task hooks to environment plugins (#15406 ) ## Thinking Path > - Paperclip coordinates AI agents and their runs. > - Environment plugins own remote capacity. > - A plugin needs a typed API to start and stop its installed Runner. > - Core must derive authority from the saved lease. > - A lost reply or controller restart must not lose cleanup authority. > - This PR adds the worker contract and the host task service. > - Native execution and Runner changes are separate PRs. ## Linked Issues or Issue Description Subsystem affected Plugin SDK and environment lease services. Current behavior Environment plugins have no typed Runner task hook. Core cannot dispatch these tasks or recover their credential handoff through a saved lease. Proposed behavior Add validated task operations, worker negotiation, company and project checks, retained-session authorization, and durable cleanup. Keep credentials out of lease metadata. Additional context This PR is the first part of a smaller series. Runner installed-home support is in #16977. Native execution and setup are in #16978. Both are stacked drafts. This PR does not enable native plugin execution, change default environments, add a route, or add a database migration. ## What Changed - Add `supportsTasks` and `onEnvironmentTask` with seven typed operations. - Validate worker requests and responses. Reject shell commands and file transfer operations. - Pin dispatch and cleanup to the saved company, lease, plugin, and driver. - Authorize projects and retained sessions. Validate the current run separately from the retained PRP identity. - Save dispatch before the provider call. Fence admission during cleanup. Require observed process termination. - Save account writeback authority on the lease. Recover Codex refreshes through the existing account and secret services. - Document the task API. Keep the Runner, heartbeat, setup, workspace, and provider changes in later PRs. ## Verification - Full `pnpm -r typecheck` and `pnpm build` passed on the separated API scope. - 201 focused tests passed: SDK worker dispatch, task validation, PostgreSQL task lineage, credential recovery, and existing environment cleanup. - The lineage test covers a third run, repeated same-run replacements, a stale submitting run, and exact termination receipts. - Initial commands hit sandbox socket/IPC restrictions. The permitted runs executed PostgreSQL and passed. A typecheck attempt lacked Cargo in PATH; the run with the installed toolchain passed. - The earlier combined implementation had a full local run with 22,286 passing tests, 120 skips, and eight failed assertions. Task fixture failures were fixed before this split. Remaining host Git and temporary-path assertions are recorded in the previous PR revision. This split does not claim a fresh full local suite pass. - The standard Greptile review scored this API head 5/5 with no unresolved threads. CI is green. The single Apex review also scored this head 5/5 with no actionable findings. - Fresh API CI lost a self-hosted runner during server shard 9. GitHub reported lost communication, with no failed test step. The affected jobs passed on retry without code changes. - CI on the prior combined head failed an unchanged workspace dev-server readiness case. No unrelated test fix is included here. Fresh CI is required for this smaller head. - This scope does not run a provider or a Cloud test. The user owns deployment and the intermediate Cloud test. ## Risks - The API is opt-in. Native plugin execution requires the later integrations. - Providers must make task submission idempotent and prove descendant termination. - A failed status read or credential export keeps the lease available for cleanup. - Codex refresh writeback requires the same active grant, secret reference, and session epoch. Revocation or account replacement discards the old refresh. ## Model Used OpenAI GPT-6 through Codex. Used source inspection, code execution, tests, and GitHub tools. The exact served model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run local checks and recorded failures and their results above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-20 09:14:23 -07:00
DottaandPaperclip	8193abb936	feat(claude): default qualified macOS and Linux targets to Paperclip Runner (#15997 ) ## Thinking Path > - Paperclip must run the selected agent on its selected host. > - Claude needs verified packages for each supported target. > - A Mac controller must select the correct Linux provider pack. > - New defaults must follow package and execution evidence. > - This PR adds Claude platform packaging and target-specific pack selection. > - Existing Codex Linux ARM64 behavior remains intact. ## Linked Issues or Issue Description Refs #15683. This PR depends on #15993 and #15994. Related scope: #15422 and #15224. ## What Changed - Reuse the merged four-target daemon producer/consumer matrix. Add target-owned Linux provider packs and carry their immutable identities through release assembly and vendoring. - Reuse master’s landed pinned Claude SDK ARM64 executable admission and retain package-integrity, architecture and no-follow admission. - Select target-specific remote packs so Mac controllers can use Linux hosts without a development pack override. - Preserve read-only runtime context on SSH startup and resume without accepting unverified old content. - Check ordinary installed Claude readiness without injected package authority. Retain Codex and Linux x64 consumer checks. - Extend Claude automatic target selection for the remaining qualified package targets. Retain unsupported-platform and external-override behavior. - Qualify managed Daytona snapshot names through provider IDs during Claude Connect. Pin native acquisition to the measured ID. Preserve a revocation marker after failed probes or target edits, and block native acquisition until requalification instead of falling back to a mutable name. Preserve immutable-image proof, company and account boundaries, and Legacy acquisition. ## Verification - Reconciled with master `9179d8dcc3e908c217fde4f63c9d08ccf8132eeb` through the prerequisite stack, retaining the merged startup fix and latest Codex Linux ARM64 behavior. - [Four-target exact-source package qualification](https://github.com/paperclipai/paperclip/actions/runs/39267321383) passed on source `04e0202ca81387b17cbe7fd094a8338648356fe9`: all four native builds, public-package assembly, Linux x64 verification and clean installed consumers on Apple Silicon, Intel Mac and Linux ARM64. The later master reconciliation adds shared OpenRouter attribution and changes runtime source. This historical result keeps its original source identity; [later qualification](https://github.com/paperclipai/paperclip/actions/runs/39281100929) passed at `dca62c7aeb4b45f0dd91a8e18d386a16a2658ef8`. It predates the subsequent TypeScript startup fix. - Fresh local product acceptance on integrated source `b2c9fe43f9cd3bc5dd9d22552da0148845ba71a4` completed a useful file and follow-up through native Claude/Opus 5.5, High effort and the API-key connection. Both actual browser downloads were independently checked; the revision preserves the original body and session continuity. Total task estimates: $0.518432800. Ordinary web and API creation saved defaults with the native experiment disabled. - Local focused verification passed 993 startup/session tests, 243 UI tests, 12 workflow security tests, direct server/UI TypeScript checks, and token/palette gates before the last review changes. Initial DB-backed snapshot pin/revocation coverage passed 18 cases; the final serial route suites pass 122 credential/permission and 232 snapshot/environment cases after correcting a test hook timeout. Current-head cloud suites pass. - Preserved the failed pre-provider local fixture and its full pending reservation. The successful fixture has no unpriced or pending usage. Historical failed Cloud journeys, accounting holds, source identities and subscription limitations remain in the dated plan. - All Docker/Rust release production was performed in cloud CI. No local Docker image was built. An inadvertently invoked local Rust prerequisite was stopped. - Current-head cloud CI is green. All three stack PRs have fresh Greptile 5/5 reviews and zero unresolved review threads. Required typecheck, test and build gates ran in cloud CI; Docker work stayed remote. Earlier failed checks remain in their original runs. - The final master merge preserves task-loading, permission, skill-retention and OpenRouter changes. Pi uses a fresh combined profile 27, preserving both historical profile-24 declarations; Copilot remains profile 22. Focused protocol checks passed 155 cases (one skipped), the updated Pi attestation check passed 18 (one skipped), and 91 evaluation/catalog checks passed. - Latest-application manual continuation passed in 37.942 seconds on `695e5c005a85bddc8711d3fcddcaf6b7e1fb64d0`, with High effort, native Opus 5.5, Anthropic API attribution, the same resumed Claude session, an independently checked 635-byte file and persisted Done status. It uses the earlier verified Mac daemon; the preceding-source package qualification passed at `dca62c7aeb4b45f0dd91a8e18d386a16a2658ef8`. That package run predates the small TypeScript startup fix and is not relabeled as qualification for it. - Upgrade limitation: changed compiled sidecar identity rejected old durable launch state and recovery hit session-open bounds before eventually resuming. That attempt was operator-cancelled and retained; the successful manual continuation does not qualify seamless upgrades. - Local latency profile, Opus 5.5/High unchanged: three alternating real-ACP loopback samples per variant reduce median provider-ready time from 5.081s to 4.034s. The matched real API task startup improves from 5.533s to 4.348s, but total reply time changes from 17.933s to 20.221s because agent work varies. This establishes a startup saving of about one second, not a consistent total-reply speedup. A real candidate follow-up passed in 20.146s and resumed the same saved session. The loopback fixture makes no external model requests; failed initial probes remain retained. - The startup change passes 107 focused runtime/effort/lease tests, 53 profile checks (one skipped), TypeScript checking and the TypeScript-only runtime build. Native hire fixture checks pass 70 tests; their first local run retained a socket-hangup failure and stale OpenCode expectation, both accounted for before the passing rerun. Broader required checks run in cloud CI. ## Risks - Experience needs improvement: successful connection setup still shows “Connection needs attention” because runtime checks are deferred. Cold provider-pack upload took 122–126 seconds per run while the task mainly showed “Thinking.” The tested Core-only preview excludes private Cloud UI plugins; this is not full Cloud distribution or subscription qualification. Desktop dark theme was tested; mobile and comprehensive accessibility were not. The portable report validates, but gallery interaction could not be inspected because host preview was queued. - The optional supply-chain scan returned neutral/inconclusive because GitHub Actions package coverage and infrastructure parsing are unavailable. It is not counted as a successful supply-chain qualification. Exact package, daemon and image integrity checks passed. - A stale, missing or wrong-architecture pack must fail admission. It must not change runner or target silently. - Provider-pack source changes invalidate affected earlier execution results. Existing unknown-cost and cleanup holds remain intact. - Live subscription execution remains unverified under the API-key qualification direction. Built-in authentication flows are retained. ARM64 cloud-image production is deferred. Merge and production deployment remain separate. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, code execution and browser testing. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] Focused local tests and final cloud verification pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-19 14:17:24 -05:00
DottaandPaperclip	98ca6ff7c5	perf: reuse verified provider files during sandbox startup (#16493 ) Build provider-pack transport layers once, reuse verified image files in private directories, and upload only missing layers. Preserve archive integrity checks and directory fallback on unsupported hosts. Verified with 44 focused tests, all 48 main CI jobs, cloud Docker Runner checks, and Greptile 5/5 on the merged head. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-19 08:45:09 -05:00
Devin FoleyandPaperclip	10d92eb468	fix(daytona): clear idle guard after verified sandbox cleanup (#16760 ) Replace the lifetime provider-use guard with scoped allocation and pending-request evidence. Fence cleanup observations against overlapping work, retain timed-out raw requests, and wait for completed remote deletion before permitting idle sleep. Verified with 436 Daytona tests, package and workspace typecheck/build, full CI, and Greptile Apex 5/5 on the reviewed head. Four deletion-boundary regressions and two concurrency regressions fail on the earlier implementation and pass with the fixes. Live-provider and local full-suite limits are documented in the PR. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-18 13:56:20 -07:00
DottaandPaperclip	bfadb66556	fix(daytona): retain already archived workspaces on stop (#16710 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Stop-only recovery must keep unfinished remote workspace files safe. > - Daytona can archive a stopped sandbox while those files remain in object storage. > - The stop handler sent another Stop request for that archived source. > - Daytona rejected that request, so recovery could not record its required receipt. > - This change accepts the freshly confirmed archived state after retention is verified. > - Recovery can proceed while the source stays cold and its files remain available. ## Linked Issues or Issue Description Refs: #16583, #16700. What happened? Stop-only cleanup rejected a retained workspace after Daytona archived it. The live provider returned `DaytonaValidationError` for Stop while its state was `archived` and auto-delete was already disabled. The host correctly kept the recovery gate closed. Expected behavior Confirm the exact source and its retention policy. Treat an archived source as having no running process. Record the stop-only receipt without starting or deleting the source. Steps to reproduce 1. Keep an unfinished native workspace with stop-and-retain intent. 2. Let its stopped Daytona sandbox become archived. 3. Request stop-only cleanup. 4. Observe the rejected Stop request and missing receipt. ## What Changed - Accept `archived` after a fresh provider read confirms that auto-delete is disabled. - Stop the source if that fresh read instead shows it has resumed. - Add regression cases for confirmed retention, unconfirmed retention, and a resumed cached handle. - Update the recovery runbook and capability map. ## Verification - The new baseline fixture failed on the old handler: 1 failed, 2 passed. - Daytona provider suites: 417 passed, 6 skipped. - Full workspace `pnpm -r typecheck`: passed. - Full workspace `pnpm build`: passed on `fc8b1c1bb8`. - Maps and Git whitespace checks: passed. - The live provider rejection was reproduced on the exact retained source. No model turn, file change or delete was performed. The fixed source handler was then run against that same archived allocation with the declared SDK version. It returned the stopped-process receipt while the provider stayed archived with auto-delete disabled. No process was started. All 58 exact-head CI checks passed; independent review is 5/5. A full local run is also in progress on a separate frozen checkout of this revision; it is not reported as complete. - [Daytona lifecycle documentation](https://www.daytona.io/docs/en/sandboxes/) confirms that archive follows Stop and preserves the filesystem in object storage. ## Risks - An archived source grants a generic stopped-process receipt. Its physical provider state stays archived. - Retention confirmation remains required. A stale archived handle that has resumed still requires Stop. - The host still checks the original allocation, ownership, stop receipt and file-recovery evidence before continuation. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use and code execution. The session does not expose an exact backend model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-18 02:50:49 -05:00
Devin FoleyandPaperclip	4415665d67	fix(server): allow proved idle waits and verify historical cleanup (#16633 ) ## Thinking Path > - Paperclip manages agents, task continuations, integrations, and their execution resources. > - Automatic idle sleep needs durable evidence that accepted work and provider cleanup are settled. > - Response-driven questions, passive board escalations, and completed connection receipts currently look like unfinished work. > - Legacy ephemeral leases and terminal image sessions need actual deletion proof before their history can become idle. > - This PR adds narrow evidence-based exceptions and read-only provider verification while preserving queues, outcomes, and ownership fences. > - Idle instances can sleep without losing responses, automatic reconciliation, or cleanup obligations. ## Linked Issues or Issue Description What happened? Instances with no executable work can remain awake because pending human responses, passive recovery records, outbound integration configuration, or completed delivery history are counted as background work. Older failed ephemeral leases and finished setup sessions also lack the evidence needed to distinguish retained resources from deleted allocations. Expected behavior Permit idle sleep only for proved response-driven waits and settled history. Continue blocking real queued work, inbound integrations without a wake contract, unfinished accounting/retries, retained resources, and uncertain provider cleanup. Steps to reproduce Finish an issue's producing run, leave a pending question or confirmation, and request an owned scheduled idle report. Separately retain a processed connection receipt or a failed ephemeral lease whose provider allocation has already been deleted. The old checks keep those instances awake even after executable work has finished. Related public groundwork: #15522, #15599, #15761, #15897, #15907, and #16325. Searched existing public idle-sleep PRs; this addresses additional blockers in the current predicates. ## What Changed - Recognize supported response-driven waits and passive board recoveries without changing task state, response audiences, or continuation policies. Bind passive issue generation into routine-only alarms too. Keep automatically polled merge-confirmation cards awake. - Treat processed events, delivered intents, disabled drafts, and outbound MCP configuration as history/configuration where proved safe. Preserve pending deliveries, downstream wake requests, and inbound GitHub bindings. - Permit only pristine Slack setup drafts with a matching disabled channel connection, no provisioning registration, and no prior provider identity/activity. A saved slash-command name remains inert configuration. - Repair missed routine execution-issue status hooks in bounded awake-only maintenance. Lock and re-read exact owners, apply existing completion/transient-failure rules, and preserve normal continuation after an issue resumes. - Allow human-only connection setup to wait only when no reusable active connection of its purpose exists; available access reblocks sleep for reconciliation. The delivery worker shares this proof, releases its local idle hold, and retains maintenance only while already awake. Passive-only ticks pause before admission during an owned drain, allowing slow backups to finish; new producer notifications still reconcile and invalidate the backup. - Add optional `environmentInspectLease` SDK discovery/dispatch and read-only Daytona/fake-provider implementations. Old workers remain conservative. Daytona status inspection uses a fresh provider GET without reconnecting or poisoning unused-worker eligibility. - Bind historical credential reads to an unrotated initial secret version and its immutable resolved-value hash; reject ambiguous histories and ambient Daytona credential/API-endpoint fallbacks before querying provider state. - Verify bounded rotating batches of historical resources on already-awake Cloud instances, outside readiness. Store host-owned deletion evidence bound to exact row identity/version and reject concurrent ownership changes. Preserve failure outcomes, cleanup/export intents, and run continuation authority. - Preserve cancellation provenance, defer contended routine rows individually, and cap actual inspection RPCs across caller timeouts and worker exits. Add database, SDK, provider, configuration-scope, and race regression coverage and update idle protocol documentation. ## Verification - `pnpm -r typecheck` and `pnpm build` passed on `24fd1c0f5ec`; server typecheck/build passed again after the final coordinator-only fix in `60f23e6e4b6`. - Final server regressions: 258 sleep-safety/connection cases and all 17 coordinator cases passed. Earlier focused checks also covered 331 combined idle/cleanup cases, 34 custom-image cases, 39 credential/cleanup cases, 39 connection-history cases, and 80 existing routine cases. Counts overlap and are not additive. - SDK: 153 passed. Daytona: 406 passed with 14 existing opt-in skips. The provider implementation also typechecks against the locked SDK used by the Cloud wrapper. - Dependency boundaries were exercised with real child workers holding provider inspection replies beyond caller timeouts, a real PostgreSQL row lock delaying one routine owner, and real connection creation → queue notification → worker hold release → resumption after access is enabled without another notification. A controlled six-second backup goes through the real coordinator, task drain, work signals, and idle-safety generation checks; maintenance permits completion, while a new committed transaction invalidates it. - Full local `pnpm test:run` completed its general-server phase with 21,122 passing, 29 failing, and 109 skipped tests; later phases did not run. That long run spanned source edits and is not a final-revision pass. The affected Slack draft assertion passes on current source. Restoring `/usr/sbin` for `lsof` produced 233 passing tests and two Git-fixture failures across the other five files (3 existing skips). Both remaining assertions pass with command-scoped `init.defaultBranch=master`; this host defaults to main and leaves both branches in their fixture. No source/global Git setting changed for that comparison. Original failure logs are retained. - A parallel typecheck/build attempt raced generated Runner outputs and failed an import. Running the build sequentially passed with no source change. - Local database checks use the maintained test-constructor seam with native PostgreSQL 16.14 because bundled PostgreSQL 18 is unreliable on this host. No database assertions were skipped in the focused runs. An interrupted combined run was superseded by the separate passing coordinator and safety/connection runs; no owned test database processes remain running. - Apex reviewed final head `60f23e6e4b615a08591e19d86443a6c35c091ce1`: 5/5, zero unresolved threads. All current-head CI gates are green, with expected policy skips. The branch is conflict-free and GitHub reports a clean merge state. - Current-head CI initially failed `plan-task-fixture.spec.ts:66` during fixture deletion: the final company DELETE hit `activity_log_company_id_companies_id_fk` after the feature assertions completed. The deletion path is unchanged here; the final diff only changes drain-gated passive maintenance. The preceding revision passed this shard. Failure evidence is retained in [the original CI job](https://github.com/paperclipai/paperclip/actions/runs/39006936642/job/117321244396); the targeted retry passed on the unchanged commit ([retry job](https://github.com/paperclipai/paperclip/actions/runs/39006936642/job/117322535013)). This shows recovery, not that the underlying company-deletion race is resolved. - These checks do not claim staging/production deployment or live provider cleanup proof. Provider tests simulate responses. Full Cloud staging verification requires a compatible Core/private-plugin composition after the Core revision is on master. ## Risks Sleep eligibility changes require the existing owned drain, final generation validation, local-work checks, and plugin drain. Unsupported states fail closed. Queued agent input and routine execution remain blocked. Outbound MCP exemptions exclude inbound GitHub grants; other unsupported inbound integrations remain awake. The inspection RPC is optional and requires a matching live worker method. Its actual outstanding calls are capped at two across workers even after the caller times out. Historical lease inspection requires its recorded plugin/config; setup inspection requires an unchanged recorded config fingerprint. Not-found under replacement account credentials is not historical deletion proof: ambiguous/rotated secret histories defer verification, and initial-version reads must match their recorded value hashes. Missing provenance (including legacy image sessions without a recorded config fingerprint), live/stopped resources, provider authorization failures, and concurrent changes defer verification. Routine repair replays existing completion/transient-failure behavior and timestamps at reconciliation time; it does not execute or discard deferred comments. No migration or destructive provider operation is added. The new marker does not grant run-resume authority. Full Cloud staging validation requires a compatible Core/private-plugin composition; no production deployment is included. ## Model Used OpenAI GPT-6 Codex, with repository/tool access and code execution. Exact model identifier and context-window size are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run relevant tests locally; passing focused checks and full-run limitations are documented above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-17 07:01:32 -07:00
Nicky LeachandPaperclip	5f5602bba7	feat(plugins): synchronize standing connection assignments (#16603 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Lifecycle plugins prepare agent runtimes outside the control plane. > - They need committed connection assignment changes without polling. > - A reader and its revision hooks must cover the same fields. > - This pull request adds a bounded standing-assignment snapshot and its matching hooks. > - Plugins can react to assignment changes while execution keeps its existing access checks. ## Linked Issues or Issue Description What happened? The agent runtime reader does not expose standing connection assignments. Changes to connection installs, active profiles, and connector feature switches do not request synchronization. Adding company-wide invalidation also exposes lock inversions in assignment writers and AI adoption. Expected behavior The owning lifecycle plugin can read a bounded assignment snapshot. Every committed change to its stored fields advances the runtime revision. Rollback removes both the change and its event. Assignment writers finish without a lock-order cycle. Steps to reproduce 1. Install a connection or bind an active profile to an agent. 2. Read its runtime inputs through the owning lifecycle plugin. 3. Rename or disable the connection, revoke the assignment, or switch connector features. 4. Expect a committed runtime event and an updated snapshot without periodic polling. Additional context Extracted from #16503. Catalog publication #16602 and the separate test maintenance PRs #16601 and #16610 are merged. This PR now targets master and is the active review. Tool-definition hooks #16604 and access/credential hooks #16605 remain drafts without review requests. ## What Changed - Add SDK types and an owning-plugin runtime snapshot for installed connections, active profile IDs, and effective connector feature switches. - Reject more than 512 connections or 128 active profiles instead of returning partial data. - Add migration 0354 for the exact stored fields used by the snapshot, with transaction-coalesced invalidation. Agent-targeted bindings and installs notify only their old/new target agents; project/routine/issue/gateway bindings are outside the snapshot and produce no notification. - Use ordered agent locks that are compatible with foreign-key references and exclude runtime readers. Prelock complete old/new target sets before replacing assignments or restoring managed profiles. - Record one company baseline at delivery startup for process-managed feature overlays. - Lock assignment inputs before install saves, task completion, and email assignment. Setup finalization reads the locked connection state and prelocks company agents before activating, re-enabling, or reviving a profile. - Make profile binding and reassignment writes transactional and lock parent profiles first. Keep target validation and profile/detail reads on the reserved transaction connection, including with a one-connection pool. - Lock company agents in order before validated AI adoption. Keep adoption and shared configuration writes at `FOR NO KEY UPDATE`, avoiding an upgrade that can deadlock foreign-key inserts during gateway creation. Configuration fields cannot change agent identity; lifecycle transitions and deletion retain their existing locks. - Document supported coverage and the inputs reserved for later PRs. ## Verification - Thirteen affected suites passed: 757 tests covering runtime hooks/assignments/skills, concurrent skill publication, tool access/catalog, task connection intents, lifecycle delivery, email, AI accounts, migration histories, snapshot drift, and the agent lifecycle module. This includes the merged catalog PR's OAuth and overlapping-refresh regressions. - The assignment suite passes all 24 cases, including bounded snapshots, company/agent isolation, rollback, health-field no-ops, process-start feature overlays, and real lock contention. - Four one-connection PostgreSQL regressions reproduce the earlier helper-read hangs before the correction; bind, unbind, delete, and reassign complete with it. The earlier adoption/gateway probe exposed both lock upgrades: changing only validation still deadlocks in shared configuration. Current scoped coverage verifies adoption with agent-targeted binding writes; gateway-only bindings no longer invalidate this snapshot. - The original ordered-adoption regression reproduces PostgreSQL `40P01` without ordered company-agent locking. Scoped-recipient tests reproduce unrelated-peer journaling before the trigger correction and verify repeated bind/unbind operations leave peers untouched, ignored binding types produce no events, both reassignment recipients are reached, and coalescing remains intact. - Six replacement contention cases cover installs, existing setup, activation, re-enabling, profile revival, and profile reassignment. Negative controls reproduce `40P01` without the corresponding target/company prelocks; all cases pass with the guards. Activation decisions use the connection row reread under lock. - An initial contention fixture failed because its settings connection lacked an MCP URL. Supplying the fixture URL and controlled lookup corrected that test setup error. - Server typecheck, database migration numbering/safety, five-snapshot retention, feature-map structural validation, and `git diff --check` passed. Migration metadata preserves historical journal entries and adds no schema change. The complete diff was reviewed for secrets and personal data before push. - Earlier hosted PR CI runs 38983440861 and 38985530069 passed; the superseded promotion run 38983440470 was cancelled. Superagent acknowledged the recipient fix and accepted the documented company-wide cost at `359b5b2f30a8a0248177367e458663d5379810d1`. Its two standard Greptile summaries were conditional on a later setup correction and were not counted as independent acceptance; the setup correction is now included here. - At `7da246aab46274db1a120a0546abbb404c5427c7`, hosted PR run 38986958988 passed: all 55 checks succeeded, with two optional Storybook checks skipped. Superagent passed, all review threads are resolved, and standard Greptile independently returned 5/5 with no blocking findings (review completed 2026-10-17 00:40 UTC). Apex independently returned 5/5 with no actionable findings (completed 2026-10-17 00:50 UTC). CI was rechecked after Apex and remains green; the branch is mergeable against master. Full local build and repository-wide tests were not run. ## Risks Migration 0354 adds database functions and triggers without changing table schema. Apply it through the normal migration flow. Agent-targeted bindings and installs only notify their affected agents. Explicit company-wide assignments, connection metadata, profile activation/removal, and connector feature switches retain synchronous, transaction-atomic company fanout. Each such transaction takes ordered row locks and inserts up to one connector runtime event per eligible company agent: O(number of eligible agents) work and journal storage, with no company-size cap or cross-transaction mutation quota. This remains an intentional scope tradeoff for this PR; deferred fanout is not implemented. Many of these conservative changes can request preparation even if an individual snapshot stays unchanged. Snapshot limits fail closed. This PR provides desired assignments, not credentials or task authorization. Later PRs must add mutation coverage before consumers can rely on immediate synchronization of tool definitions or credentials. ## Model Used OpenAI Codex, GPT-6, with reasoning, repository inspection, code execution, and testing. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-16 17:55:47 -07:00
Nicky LeachandPaperclip	5fbece9dcc	Synchronize assigned skills through agent runtime hooks (#16501 ) ## Thinking Path > - Paperclip manages shared skills and agent assignments. > - Runtime plugins need installed skills before a task starts. > - Skill content can change without changing the agent row. > - Source access and version pins also affect delivered files. > - This pull request feeds those changes into the durable runtime stream. > - The runtime reader resolves the same immutable skill sources used for execution. ## Linked Issues or Issue Description Builds on merged PR #16499. This pull request now targets `master` and contains only the five skill synchronization commits. Problem Plugins receive no durable notification when an assigned skill changes. A saved assignment can also become inaccessible after private-source access changes. Proposed solution Invalidate referencing agents at installed skill mutations. Invalidate inherited access when source bindings, project privacy, or policy change. Return authorized skill sources and version IDs through the owning runtime reader. Alternatives considered Activity-only subscriptions miss service writers. Per-task synchronization adds preparation work. Copying current files would ignore version pins. Roadmap alignment This extends existing skills, source synchronization, and plugin runtime support. ## What Changed - Defer multi-skill source runtime hooks until commit, after publication holds all skill rows. Keep file-save journaling immediate to preserve filesystem rollback. - Add transactional invalidations for skill content, companion files, versions, rename, deletion, policy, source access, and pin-setting changes. - Recognize canonical keys and supported legacy aliases. - Reuse the execution skill resolver, current agent access, and instance version-pin rules. - Resolve and materialize sources outside the agent row lock. Recheck ownership and runtime revision afterward. - Keep task-only mentions and running-task snapshots in the execution path. - Document conservative invalidations for pinned assignments and inherited access. ## Verification - Rebased onto origin/master `9f4a956b90`, which contains merged PR #16499. All five commits are unchanged according to `git range-diff`; no conflict resolutions or source changes were needed. - On rebased head `58b065941`, all 237 focused tests passed: runtime skill hooks/source publication (41), plus content service, assignment routes, policy routes, and project source access (196). - Server typecheck passed after rebuilding Runner with the installed Rust toolchain. Migration numbering/safety and map checks passed; snapshot pruning made no changes. Migration CLI checks required local IPC permission. - Real PostgreSQL coverage includes concurrent source publication/version saves, policy-only tightening/restoration, referencing-agent fanout, company isolation, no-op counters, supporting files, rename/deletion, rollback, private-source access, pinned/latest bytes, and unassignment. - Earlier stack validation passed workspace typecheck/build. Fresh hosted CI and review must verify the rebased head before handoff. - The broad local suite was previously limited by an unchanged diagnostic test that rejects "private" in an error stack under this /private/tmp worktree; that unchanged suite passed all 108 tests in the original checkout. Hosted full-suite checks provide exact-head verification. - Diff whitespace and full PR secret/PII scan passed. - All hosted CI/security checks are green on `58b065941`; fresh standard Greptile and Apex both returned 5/5 with no findings or unresolved threads. The PR targets master and is mergeable without conflicts. ## Risks - Apply migration 0348 after the runtime-stream migration in #16499. - Content invalidations include pinned assignments. The resolved version stays pinned; consumers can avoid writes when its version/hash is unchanged. - Project access and source changes invalidate only agents assigned skills from the affected sources. Company policy still invalidates the company. Source polling bookkeeping and popularity counters do not create work. - The reader rebuilds installed caches without upstream requests. Concurrent changes cause a bounded retry and then a conflict. - Private transport integration and live remote computer qualification remain separate. ## Model Used OpenAI Codex, GPT-6, with reasoning, repository inspection, code execution, and testing. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal Paperclip ticket id - [x] I have run tests locally and they pass (focused suites; broad local limitation documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-16 08:28:47 -07:00
Nicky LeachandPaperclip	9f4a956b90	Add durable agent runtime synchronization hooks (#16499 ) ## Thinking Path > - Paperclip manages agents and their runtime configuration. > - Plugins can maintain external agent computers. > - Configuration and file saves currently miss durable plugin delivery. > - A transient activity notification cannot recover after interruption. > - This pull request adds an opt-in agent runtime resource stream. > - The existing dispatcher delivers and acknowledges its committed work. > - Plugins can prepare computers when configuration changes. ## Linked Issues or Issue Description Related: #15631 and #16203 establish agent lifecycle and native durable delivery. Problem Plugins cannot reliably react to agent configuration and instruction changes without polling or waiting for execution. Proposed solution Add an opt-in `agent_runtime` stream to the existing resource journal. Record changes at the database mutation and managed file audit boundaries. Wake the dispatcher with content-free commit notifications. Provide scoped runtime inputs to the selected lifecycle plugin. Alternatives considered Route-only activity events miss service and background writers. Scheduled scans delay changes and repeatedly query unchanged state. Roadmap alignment This extends the existing plugin and agent configuration work. ## What Changed - Add durable update/delete events for agent runtime state, including an existing-agent baseline. - Capture configuration changes, instruction edits, deletion, restore, and managed run writeback. - Add commit notifications and a reconnect scan. Register one listener for configured, enabled runtime subscribers. Release it when none remain. - Keep HTTP startup independent of queued plugin callbacks. - Add `ctx.agents.getRuntime`. Require lifecycle capability, invocation company, enabled plugin, and owning lifecycle selection. - Keep legacy agent/project subscriptions isolated. Manual inbox reads retain both legacy streams. - Resolve recovered instruction paths without reading or hashing bundle contents. Use the company reporting graph for runtime health. - Repair missed file events on identical save/deletion retries after a journal failure; preserve content tokens and conditional-write checks. - Correlate file audits with runtime revisions to deduplicate recorded retries without a new table. Bound the lookup with the existing company/timestamp index. - Document the contract, migration, access boundaries, and session-connection requirement. ## Verification - Rebased the full stack onto origin/master after #16507. - Final replayed stack: `pnpm -r typecheck` and `pnpm build` passed. - All 132 affected runtime, skill, connector, publication, instruction, managed-agent, and delivery tests passed after the latest-master rebase. Workspace typecheck and build passed. - Journal failure injection verifies recovery on identical file writes, deletions, and instruction retries from an acknowledged inbox. - Deduplication follow-up: 80 runtime/instruction/delivery tests and server typecheck passed. Repeated retries create no audit/events; old hashes returning after an intervening commit still repair delivery. Built-in/native instruction writer regressions passed all 56 tests. - PR 1 follow-up tests cover valid/cyclic reporting chains, listener registration failure, multiple subscribers, global/company disable, re-enable, unload, and startup with blocked callbacks. - Database coverage includes rollback, journal failure, no-op suppression, file saves/deletion, ordering/replay, owner/company denial, independent-client commits, and real listener loss/reconnect. - SDK/shared suites passed 1,059 tests during implementation. CLI/delivery regressions also passed. - Migration safety, map checks, diff whitespace, and local secrets/PII scan passed. - Hosted CI and security checks are fully green at `b296405e2`. Standard Greptile and Apex both returned 5/5 with no unresolved findings. - The broad local suite encountered an unchanged diagnostic test that rejects the word "private" from an error stack. This worktree lives under /private/tmp. The same unchanged diagnostic suite passed all 108 tests in the original checkout. Hosted full-suite checks provide exact-head verification. ## Risks - Apply migration 0347 before enabling a runtime subscriber. Runtime notifications require session-level PostgreSQL LISTEN. Transaction-mode poolers are unsupported for this opt-in feature. - Database triggers share the mutation transaction. Journal failure prevents configuration commits. Trigger work adds a journal insert to relevant changes. - Filesystem publication and SQL commit are separate boundaries. Direct filesystem edits do not emit hooks. External clients must fence stale revisions and retain conditional-write checks. - The reader returns only that plugin's own instance secrets. Keep them out of UI data, logs, plugin state, and agent files. - No private transport integration or live remote computer qualification is included. ## Model Used OpenAI Codex, GPT-6, with reasoning, repository inspection, code execution, and testing. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal Paperclip ticket id - [x] I have run tests locally and they pass (focused suites; broad local limitation documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-16 07:53:04 -07:00
DottaandPaperclip	d75b3fc33f	perf(daytona): observe command exit before log-follow closure (#16561 ) Return completed Daytona session commands after recorded exit and a final log snapshot. Keep bounded early status reads, command deadlines, late callback fences and stream recovery. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-16 07:09:30 -05:00
DottaandPaperclip	69dded3230	fix: allow normal Daytona command status response latency (#16480 ) Allow a normal completed Daytona status response up to 500ms within the unchanged caller deadline. Keep final separated logs, stream fallback and single dispatch. Verified: 396 provider cases, full repository and standalone provider typecheck/build, matched real-Daytona component controls, and exact-head full CI/E2E, security and Apex 5/5. Whole-agent latency target remains open. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 14:42:36 -05:00
DottaandPaperclip	b1566ca494	perf(daytona): avoid log-follow delay for completed session commands (#16468 ) A completed Daytona session command now reads final separated logs without waiting for a new log-follow socket. Running commands retain streaming, recovery, and the existing total deadline. Verified with full repository typecheck/build, standalone provider typecheck/build, 395 passing provider cases, real immutable-image Daytona command controls, and fresh exact-head full CI/E2E and Apex 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 13:35:10 -05:00
DottaandPaperclip	3b6d1452f2	test(daytona): retire duplicate pack span execution (#16099 ) Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-14 23:45:50 -05:00
DottaandPaperclip	3ea39f75c1	test: consolidate duplicate owner proofs (#16075 ) Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-14 23:17:13 -05:00
Nicky LeachandPaperclip	db65a0863e	Deliver durable resource lifecycle events to plugins without polling (#16203 ) ## Thinking Path > - Paperclip manages AI agents and their work. > - Plugins keep external resources aligned with Paperclip projects and agents. > - The durable resource journal stores changes, but its plugin interface requires polling. > - Project deletion has no journal entry, so consumers must scan for missing projects. > - This PR delivers committed journal events through a native plugin callback and records deletion. > - Plugins can recover unfinished work without periodic change scans. ## Linked Issues or Issue Description Related: #15343 introduced the durable inbox. #15631 added required agent lifecycle participation. This change keeps both contracts and adds automatic delivery for the resource inbox. What happened? Plugins must schedule reads of the durable inbox. A deleted project has no durable deletion event. Consumers must scan saved bindings to detect its removal. Expected behavior Core should deliver committed resource events and replay unfinished work. Empty queues should have no polling timer. Project removal should leave a durable event. Steps to reproduce 1. Install a plugin that consumes project lifecycle events. 2. Create or update a project. The plugin must call `listLifecycle` to discover it. 3. Delete the project. The journal has no deletion entry. ## What Changed - Add `resourceLifecycle` to the manifest and `onResourceLifecycle` to the SDK. Require `events.subscribe` and a configured, enabled company scope. - Reuse transaction-aware work signals and the delivery coordinator. Wake after commit, on startup and plugin activation, and after configuration changes. - Acknowledge only a matching successful result. Retry unfinished or interrupted work. Retain deferred events without an idle retry timer. - Wake again after successful worker configuration application, and retain failed scans through idle drain (both Apex findings fixed). - Keep ordering per resource and deliver past failed or deferred resources. Keep the manual inbox methods for existing consumers. - Record project deletion in the removal transaction. Extend the journal constraint with a generated migration. ## Verification - Apex follow-up: 129 focused worker, delivery, coordinator, and application tests passed; server typecheck passed. Regression coverage verifies configuration-application races and failed scans across idle drain. - Focused lifecycle, transaction coordinator, manifest, and SDK dispatch suites: 105 tests passed, including single-connection saves, redacted failure diagnostics, wakeups retained during idle drain, and per-event retry deadlines. - Complete SDK suite plus delivery database tests: 128 tests passed. - Shared, database, and SDK typechecks passed. Direct server `tsc --noEmit` passed against rebuilt SDK declarations. - Full local `pnpm test:run` finished with 19,491 passed, 148 skipped, and 6 failed tests across 5 files (plus two suite setup errors). This run overlapped branch updates and is not final-head evidence: failures included missing Cargo, local Git default-branch assumptions, and stale in-process lifecycle modules. Final Apex-fix commit `75d028788` passed all CI gates, including the full test shards, and received Apex 5/5 with no unresolved comments. - `pnpm -r typecheck` and `pnpm build` were attempted. Both stopped because Cargo is unavailable locally. CI supplies the Rust toolchain. - Local installation used the repository's documented lockfile fallback because master has dependencies missing from its lockfile. No dependency changes are included here. ## Risks - Delivery is at least once. Plugins must re-read current state and make external effects idempotent. - Signals are process-local, like the other delivery queues. Startup recovers committed rows. Independent database writers or multiple active server processes require an explicit ownership and wake protocol. - Apply migration `0340_open_warlock.sql` before running code that records project deletion. Earlier deletions cannot be reconstructed. - Automatic delivery selects plugins with company configuration rows. `deferred` waits for a later startup, configuration change, or other work notification. - This does not replace required agent lifecycle participation or establish that external mounts are unused. ## Model Used OpenAI Codex (GPT-6), with tool-assisted code editing, code inspection, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-13 12:00:30 -07:00
DottaandPaperclip	f329b24ee1	feat: give tasks durable project-optional workspaces (#15749 ) Give tasks durable company-scoped workspace bindings independent of project assignment, channel defaults and authorized repository preparation. Reuse verified input seeds and sparse output recovery while preserving trust, privacy and controller provisioning boundaries. Full CI and E2E, Superagent, legacy checks and Greptile Apex 5/5 passed on reviewed head `753680b61c`. All review findings are resolved; the PR records focused verification and historical acceptance limits. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-12 20:45:18 -05:00
DottaandPaperclip	dabb8c5daf	test: remove repeated setup and strengthen regression proof (#16022 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its tests protect company boundaries, approvals, durable tasks, credentials and the board UI. > - Several owner suites repeat large database fixtures and render frames. > - Some tests repeat stronger proof or pass when the behavior they describe is broken. > - This pull request shares exact setup, removes proven duplicate cases and repairs weak observations. > - It preserves distinct security, lifecycle, concurrency and user-facing contracts. > - The benefit is 5,027 fewer test lines and stronger regression checks. ## Linked Issues or Issue Description Refs #15978. This continues the merged test-audit cleanup in [PR #15978](https://github.com/paperclipai/paperclip/pull/15978), which removed 11,976 test/support lines. The savings in this pull request are additional. Related open work: [PR #13460](https://github.com/paperclipai/paperclip/pull/13460) changes legacy OAuth parent/version lock ordering. This pull request retains those production paths and the existing lock-order test. Its secrets changes compact fixtures and repair the two AWS wrapper exclusion observations. Scoped title searches found no open duplicate audit or fixture-consolidation PR and no matching audit issue. The problem is described here using the enhancement template. What existing behavior does this improve? The maintenance and regression value of the existing server, UI, CLI, shared and wiki-plugin test suites. Subsystem affected Cross-cutting test code. Two UI modules also lose unused test inspection exports. Current behavior Owner tests repeat database rows, service inputs, HTTP setup and render frames. Some cases repeat stronger assertions. Two AWS lock tests record only cleanup events, so an AWS provider write can happen inside the held lock without failing those observations. Several other negative tests accept or do not observe the wrong result. Worktree fixtures also use a temporary path alias that conflicts with the existing canonical-path checks on macOS. Proposed behavior Use small local helpers that preserve exact inputs, fresh allocations, callbacks, query order and await positions. Remove only cases with named stronger keepers. Tighten negative assertions and observe actual AWS provider entry while a confirmed lock holder remains parked. Resolve temporary fixture roots to their canonical paths. Reason and benefit Remove 5,027 net test lines and 15 redundant registered executions. Remove 11 lines of unused UI inspection exports. Keep every distinct company, actor, approval, credential, persistence, stale-write and concurrency contract. No wall-time speedup is claimed for setup compaction. Breaking changes None for application behavior, APIs or stored data. The removed exports expose private UI test state and have no remaining callers. ## What Changed - Consolidate repeated runtime, workspace, portability, route, tool-access, policy, gateway and state-service fixtures. - Share repeated UI render frames and CLI/wiki-plugin setup while retaining the original actions and expected results. - Remove 15 repeated case executions only where the audit records an exact stronger keeper. - Repair weak HTTP/UI negative assertions and two AWS account-home lock observations. Keep the existing durable results and cleanup behavior. - Remove the unused `__inflightGetCount` and `__secondTickInternals` inspection exports. Keep observable request coalescing and ticker lifecycle checks. - Canonicalize temporary worktree fixture paths without weakening production path checks. - Record the owner decisions, savings, independent preservation reviews and validation in [the dated audit report](https://github.com/paperclipai/paperclip/blob/58c5c4aa241c85f4665a3159c192888bbb2a5cfa/doc/plans/2026-10-12-test-audit-rounds.md) and [the compact control evidence](https://github.com/paperclipai/paperclip/blob/58c5c4aa241c85f4665a3159c192888bbb2a5cfa/doc/plans/2026-10-12-test-audit-controls.json). ## Verification - Complete owner and retained-sibling baselines passed before each audit cutover. Independent reviewers checked the full original tests, actual helper bodies, exact inputs, receipt provenance, async order, fresh allocations and case registrations. - All 14 UI/route/wiki mutations failed at their intended assertions: ten UI/API/ticker controls, one route control and three wiki persistence/receipt controls. Each mutated production source was restored to its exact original bytes. - Both old AWS lock tests passed with their respective AWS-only lock bypass. Both repaired tests failed at the intended assertions with the same bypasses. These give 16 intended failures and two passing weak controls across 18 executions. Each source was restored exactly. - Final complete state-owner/sibling run: 329/329 passed at code revision `64370831ebe191e5ce092f85f79b6fb661df7e16`. - Final `pnpm -r typecheck`, `pnpm check:token-gates` and `pnpm build`: passed at the same code revision. - Delivery head `58c5c4aa241c85f4665a3159c192888bbb2a5cfa` includes the evidence commit and master `4b766b872993045a2e787404f0475503683e5ded`. All 27 audited test files and both application files match the locally verified code revision byte-for-byte. Current-head CI validates the integrated branch, including the incoming master changes. - All eight production-owner SHA guards for the 16 deliberate failing mutation controls still match the integrated delivery checkout. The mutation receipts therefore apply to those current owners. - Full default `pnpm test:run`: failed with exit 1 at frozen code revision `64370831ebe191e5ce092f85f79b6fb661df7e16`; HEAD and tracked source bytes remained unchanged. The first general-server group reported 878 passing files, four failing files and six skipped files; 17,846 passing cases, 24 failing cases and 105 skipped cases. Later groups did not run. Failures were `http-log-redaction` (socket hangup), `public-mcp` (initial 15-second timeout followed by hosted token “Refresh before subscribing” failures), `remove-test-home` (macOS `ENOTEMPTY` where permission errors were expected, plus cleanup failure), and `workspace-git-snapshot-streaming` (300-second timeout). All four failing test files and the relevant cleanup/MCP production owners are unchanged by this audit. Full local success is not claimed. - Focused complete-owner diagnosis at the same frozen revision: 149/152 passed, three failed, zero skipped. `public-mcp` passed 82/82 and `http-log-redaction` passed 66/66 unchanged. The two `remove-test-home` cases reproduced macOS `ENOTEMPTY` against an expected `EACCES`/`EPERM` contract; the 32 MiB workspace snapshot case reproduced its existing 300,000 ms timeout. HEAD and tracked files stayed clean. Exact blob comparison confirmed all four test files and the relevant cleanup/MCP owners match both inspected master `4b766b872993045a2e787404f0475503683e5ded` and the frozen local revision. The three remaining local failures are unresolved; no timeout, retry or assertion was weakened. - Current-head CI: [workflow 38299262149](https://github.com/paperclipai/paperclip/actions/runs/38299262149) has passed runner selection and created all downstream test/typecheck/build gates; those gates are queued, with no failures reported. Greptile completed successfully at `58c5c4aa241c85f4665a3159c192888bbb2a5cfa` with [5/5 and zero comments](https://github.com/paperclipai/paperclip/pull/16022#issuecomment-6131457592). All review threads are resolved or absent. This pull request will remain open after verification. Reviewers can reproduce the required workspace checks: ```sh pnpm -r typecheck pnpm check:token-gates pnpm test:run pnpm build ``` Browser suites were not run. Their sources and user journeys are unchanged by this audit. ## Risks - A fixture helper can hide a specialized input or change object freshness. Full-program expansion checks and independent reviews preserve the exact old inputs and execution order. - Removing duplicate proof can remove a distinct contract by mistake. The audit records the keeper for each deletion; all distinct security and lifecycle cases remain. - Concurrency checks can miss progress during a fixed delay. The AWS repairs record actual provider entry, confirm holder entry, use measured entry windows, require exact final event order and drain both operations in `finally`. - The full local default run failed. Focused diagnosis reproduced two macOS cleanup failures and one existing snapshot timeout in unchanged owners; these remain unresolved. Current-head CI is pending. No test deadline, retry policy, skip or CI shard was weakened to obtain a pass. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex with GPT-6, reasoning, repository inspection, code execution and delegated audit/review. The runtime does not expose the exact served model identifier or context-window size; those values are unavailable and are not inferred from the model catalog. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-12 22:23:11 +00:00
DottaandPaperclip	1e0cfcb4fc	test: retire redundant suites and consolidate owner coverage (#15978 ) Retire duplicate ACPX and Codex proof after migrating unique assertions to their behavioral owners. Consolidate repeated fixtures and exact case matrices while preserving lifecycle, protocol, company, credential, approval, and ordering contracts. Remove the unused runtime-controls compatibility path and document test ownership and preservation evidence. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-12 10:44:02 -05:00
Devin Foley Devin Foley Claude Opus 5.5	657e71584a	fix(daytona): record provider state when lease creation hits its deadline (#15946 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents can run in Daytona sandboxes. The Daytona plugin acquires a lease before each run. > - Some acquisitions time out. The report then says `environmentAcquisitionPhase=create` and that the whole 300 s budget was used. > - The Daytona SDK create timeout is equal to the acquisition budget. The outer timer always fires first. Thus the report never shows the state that Daytona left the sandbox in. > - Without that state, we cannot choose a fix. A stuck snapshot pull, a slow start, and a missing allocation need different fixes. > - This pull request adds one bounded, read-only provider lookup at the create deadline. It records a fixed-enum provider state in the existing acquisition diagnostic. > - The benefit is that the next timeout identifies where the provider stopped. Retry, budget, and cleanup behavior do not change. ## Linked Issues or Issue Description Refs #15736 (bounded Daytona acquisition failure observations). This pull request extends those observations. What happened? A Daytona lease acquisition timed out with `Daytona lease acquisition timed out; allocation cleanup is pending`. The run-failure report gave `environmentAcquisitionPhase=create`, `environmentAcquisitionElapsedMs=300004`, and `environmentAcquisitionBudgetMs=300000`. The report did not show the Daytona sandbox state. The SDK create timeout is equal to the outer budget, so the outer timer always fires first. The SDK never reports the stuck state. Expected behavior When creation is still pending at the deadline, the report should show where the provider left the allocation. Examples are `pulling_snapshot`, `starting`, `error`, or `absent`. Steps to reproduce 1. Configure a Daytona environment. 2. Start a run when Daytona does not finish create and start within the acquisition budget. 3. Inspect the setup-failure report. Only the phase, the elapsed time, and the budget are present. Paperclip version or commit `master` at `ba04968`. ## What Changed - `packages/plugins/sdk`: Add an optional `providerState` field to `PluginEnvironmentAcquisitionDiagnostic`. The values are a fixed enum: `absent`, `creating`, `pending_build`, `building_snapshot`, `pulling_snapshot`, `starting`, `started`, `error`, `build_failed`, `other`, `unavailable`. The reader omits an unknown value and keeps the other fields. Old envelopes stay valid. - `packages/plugins/sandbox-providers/daytona`: When the acquisition deadline fires while `client.create` is still pending, make one `client.get` of the provisional `paperclip-create-<attempt>` name. The lookup has a 5 s bound, inside the host's 30 s RPC margin. Report the state only when the name and all ownership labels match. Otherwise report `unavailable`. A not-found error reports `absent`. - The lookup does not delete. It does not set `observedProviderLeaseId`. It does not change retries, budgets, or the cleanup envelope. - After the deadline, only the timer settles the call. A late create result cannot replace the timeout error before the lookup completes. A test proves this guard. - `server`: Project the value to `run_execution.environmentAcquisitionProviderState`. It uses the same private, run-scoped receipt as the other acquisition fields. - `doc/observability.md`: Document the new field and its limits. ## Verification - `cd packages/plugins/sandbox-providers/daytona && npx vitest run --config vitest.config.ts`: 392 passed, 6 skipped. - `npx vitest run packages/plugins/sdk/tests/environment-creation-cleanup.test.ts server/src/services/__tests__/run-failure-diagnostics.test.ts server/src/__tests__/environment-acquisition-diagnostics.test.ts server/src/__tests__/run-failure-sentry-real-sdk.test.ts server/src/__tests__/environment-runtime.test.ts`: 302 passed, 11 skipped. - `tsc --noEmit` passes for `packages/plugins/sdk` and `packages/plugins/sandbox-providers/daytona`. - New tests cover state mapping, `other` for unlisted states, `absent`, `unavailable` for lookup failure and label mismatch, a hung lookup bounded to 5 s, no deletion, an unchanged cleanup envelope, and a late create that settles during the lookup. - A mutation check removed the late-settlement guard. The late-create test then failed. - The local server `tsc` was killed by the sandbox memory limit. CI must confirm the server typecheck. The server change is one line. ## Risks - Low risk. The timeout rejection can come up to 5 s later, and only for a create-phase deadline. The host RPC margin is 30 s. - The lookup is one extra read-only Daytona API call for each create-phase timeout. - This is a diagnostic. It does not fix the upstream cause. The next timeout report must show the state before a fix is chosen. - This is a Sentry run-failure report change. It is not a Telemetry change. The new value is a fixed enum. It does not contain provider IDs, causes, URLs, or paths. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), used through Claude Code with tool use (shell, file edit, test runs). Context window and reasoning configuration are set by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Devin Foley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-11 21:22:39 -07:00
Devin FoleyandPaperclip	0ba88d2195	fix(opencode-local): preserve deadlines in Daytona process sessions (#15905 ) ## Thinking Path > - Paperclip manages work done by AI agents. > - The OpenCode adapter can run its CLI in a provider sandbox. > - A long CLI run must remain observable until it exits or the host stops it. > - The legacy main command does not request a provider process session. A launch without an active run step uses one synchronous command request. > - That request can lose its response before the resolved execution budget ends. > - This PR requests the existing asynchronous session path for the main Daytona OpenCode command. It also carries the resolved command deadline through the host and provider contracts. The host still applies the effective capability gate. > - The change removes that synchronous request from eligible sandbox runs. ## Linked Issues or Issue Description Related reporting fix: #15789. A search of open and closed GitHub issues and PRs found no duplicate transport fix. What happened? A hosted Linux legacy OpenCode run retained a gateway-timeout response about one hour after CLI startup. It had a four-hour resolved budget and recent agent output. The earlier reporting fix correctly kept the configured policy separate from the timeout message. The original response owner and specific timer were not retained, so this PR does not claim to prove that provider cause. Expected behavior A Daytona target that supports persistent process sessions should dispatch the long agent command asynchronously. Paperclip should observe its output and exit through the existing session path. A single command HTTP request should not need to stay open for the full agent run. Steps to reproduce 1. Use legacy OpenCode with a Daytona sandbox target and a four-hour execution budget. 2. Launch outside an active measured step. The current main CLI call does not request a persistent process session. 3. In the SDK fixture, make the synchronous command request return a provider timeout before the budget ends. Keep the asynchronous session healthy for more than one hour. 4. The regression tests require session dispatch, incremental output, one command dispatch, a real exit result, and session cleanup. 5. Set a shorter command deadline and stall dispatch, a status read or a log snapshot. The command must return a timeout at its own deadline. Healthy output must not extend that deadline. Paperclip version or commit Observed source: `e733bddc45e00efb1819fffa8677f7b0d07802c5`. PR base: `4b59e2a2ecc67e98b336b8eb4c09959681672d54`. Deployment mode Hosted Linux server, legacy OpenCode adapter, Daytona sandbox. The host gate retains the old path when a provider cannot keep process sessions. ## What Changed - Add an optional sandbox session choice to the shared process helper. Leave the choice absent for existing callers. - Request that session only at the main Daytona OpenCode CLI invocation. Keep probes, preparation and other providers on their existing paths. - Carry a separate optional command deadline through the shared runner, host service and plugin SDK. Existing session callers can omit it. Zero leaves the command deadline disabled. - Bound dispatch, live output and recovery observations with one command timer. Clear it on settlement. Fence late stream callbacks. Remove deadline listeners after each read and clear interrupted polling waits. Preserve a known exit code if only the final logs time out. A timeout result does not claim that the process stopped. - Add adapter and process-helper regression coverage. Test the existing Daytona session for a virtual run beyond one hour, without command replay, with release cleanup. - Document transport selection and the limits of the timeout evidence. ## Verification - `pnpm -r typecheck` passed on Node 25.6.1 and pnpm 9.15.4 with the command deadline changes. - `pnpm build` passed with the updated contracts. The separately installed Daytona package also passes its own typecheck and build against the pinned SDK 0.203.0 and TypeScript 5.9.3. - Five focused process, adapter, host and provider suites passed all 349 tests. They cover a run beyond one hour, five stalled observation phases, healthy-stream deadlines, zero/absent budgets, late callbacks, session release, SSH and capability fallback. - The real environment-runtime forwarding tests passed: 19 selected tests, including the separate command budget and forced session flag in the worker RPC. - An earlier full local `pnpm test:run` began before a clean build and overlapped the correction. That mixed-version run was stopped and is not passing evidence. After the build, the two previously uncollected suites pass (1,659 tests). Two unchanged workspace fixtures fail with this machine's Git default branch `main`; both pass with a test-process-only `master` override. One unchanged native recovery fixture fails on both this branch and an independently installed and built checkout of base `4b59e2a2ecc67e98b336b8eb4c09959681672d54`, with the same terminal-result mismatch. It is not introduced by this change. The full local suite is not claimed green. These limitations do not replace fresh hosted CI. - Final head `8e96638cdef8ca6e91f7a42a52c83a075180c137` passes all hosted checks: 54 passed and 2 expected Storybook skips. Greptile reviewed this exact head at 18:12 UTC and scored it 5/5. Both actionable findings are fixed and resolved. GitHub reports no conflicts and a clean merge state. Earlier-head results are not used as final-head proof. - The standalone provider test command passes 373 tests, with 14 expected live-probe skips. - SDK tests use a controlled virtual clock. No live provider experiment, task replay, manual deployment, or public PR merge was performed. ## Risks The main legacy OpenCode command uses Daytona's existing session route when the effective capability gate permits it. A quiet log stream can use the existing status and snapshot fallback. Those observations add provider reads. The new optional command budget is separate from each read's timeout and returns a timeout without the host RPC's extra 30-second allowance. Existing session callers that omit it keep their previous lifetime. Callback bridge control commands retain their explicit session bypass. The capability gate retains the previous path for unsupported targets. Other providers, local and SSH execution keep their existing path. Retry policy, Stop acknowledgement and lease ownership rules stay the same. The SDK exposes no cancellation handle for an abandoned stream; existing Stop and lease teardown still own command termination. This fix does not establish the original gateway owner or prove recovery in a live environment. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository tools and code execution. The exact backend identifier and context window size are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (changed paths; the independently reproduced local baseline failure is documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-11 12:08:45 -07:00
Devin FoleyandPaperclip	9befc66a44	Retain bounded Daytona acquisition failure observations (#15736 ) ## Thinking Path > - Paperclip manages AI-agent work and reports failed runs. > - Some runs acquire a Daytona sandbox before model execution. > - Acquisition has one budget for creation, setup, and inline cleanup. > - A timeout currently carries cleanup ownership but loses the step that failed. > - This pull request carries a bounded observation through the existing error envelope. > - Operators can identify the first failed step without exposing provider responses or changing recovery. ## Linked Issues or Issue Description What happened? An acquisition timeout reports pending allocation cleanup without the first failed step. The provider already knows its current step, but that evidence is discarded at the worker boundary. A setup failure followed by slow cleanup is also indistinguishable from slow creation. Expected behavior The immediate setup-failure report should contain a fixed phase and bounded elapsed/budget values. Cleanup ownership, original messages, deadlines, retry policy, and task behavior should remain unchanged. Steps to reproduce Use the provider regression fixtures to stall creation, workspace setup, shell detection, expiry configuration, or sentinel upload. Also fail setup early and let cleanup consume the remaining budget. The previous source emits no phase diagnostic. Paperclip version or commit Base commit: `98e3cec78e8461774022a5ba0a777b09d4d16b96`. Deployment mode Hosted and self-hosted installations that use the Daytona sandbox provider and enable Sentry. Related public changes: #13979 established failed-allocation cleanup ownership; #14097 preserves the provider acquisition budget; #15637 carries bounded workspace-transfer evidence. Duplicate searches found no existing acquisition-phase diagnostic patch. ## What Changed - Capture the first acquisition failure before inline cleanup. Keep the observation if cleanup later reaches the deadline. - Add an optional, validated observation to the existing creation-cleanup envelope for acquire calls. Old ownership envelopes remain valid. Malformed observations do not alter cleanup. - Bind immediate reporting to an actual RPC error with matching company, environment, and run ownership. Project only three bounded fields from private host evidence. - Add provider, SDK, actual worker/client, PostgreSQL cleanup, collector, and real-Sentry regressions. Document the evidence limits. ## Verification - Passed 404 focused tests across the SDK cleanup envelope, complete Daytona provider suite, diagnostic collector, actual built worker/client subprocess, and mandatory real-Sentry SDK suite. - Passed 10 focused PostgreSQL cleanup-boundary cases. The filter excluded 152 unrelated cases. - Passed `pnpm -r typecheck` and `pnpm build` on the committed source. - Full `pnpm test:run` is running. Final outcomes will be recorded before merge. - Independent source review passed. Replacing the provider with the original source fails all 8 new phase/timing assertions. Replacing the host boundary with the original source fails all 3 owned diagnostic cases while 7 negative cases remain clean. - CI passed: 54 successful checks and 2 expected Storybook skips. This includes all server, workspace, serialized-server, Runner, browser, build, canary, and real-Sentry SDK lanes. - Greptile scored the committed source 5/5 with no review threads. - No live provider operation, task replay, runtime configuration change, or deployment was performed. ## Risks Low risk: this only adds optional diagnostic evidence. The first observed failing phase does not prove an upstream cause. Elapsed time ends at that observation, not at completion of later cleanup. Evidence is process-local and accompanies immediate failure reporting; post-restart reports cannot reconstruct it. No historical run or lease is backfilled. The wire schema remains v1 and cleanup ownership parsing is unchanged. ## Model Used OpenAI Codex, GPT-6, with repository inspection, code editing, local execution, and independent agent review. The exact model snapshot and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-09 14:25:14 -07:00
Nicky LeachandPaperclip	5badea4798	Use the agent lifecycle contract for plugin setup and status (#15391 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Some plugins must prepare resources before an agent can work. > - Those resources must stop or be removed when the agent pauses or terminates. > - The agent lifecycle module now owns these steps and their durable completion. > - Plugins use its required callback contract instead of a separate readiness query. > - This PR documents that contract and shows its progress and retry action on the agent page. ## Linked Issues or Issue Description Refs: #15631 Subsystem affected Required plugin lifecycle work and the agent detail page. Problem or motivation The lifecycle module exposes `onAgentLifecycle` and `agents.lifecycle.manage`, but the plugin authoring guide does not explain their use. The agent page displays legacy status. Setup therefore looks paused and offers Resume. Termination leaves the page before resource cleanup completes. Proposed solution Use the callback, admission guards, persisted state, and retry route supplied by the lifecycle module. Show pending phases and fixed host errors. Keep the detail page open during incomplete termination. Use lifecycle state for pause and resume controls. Alternatives considered A separate plugin readiness query would duplicate the module's state and admission rules. The final diff contains no readiness API, database commit hooks, or second admission check. ## What Changed - Document the manifest declaration, callback results, phase ordering, capability scope, retry behavior, and provider revision requirements. - Correct SDK comments for lifecycle, routing, and external object callbacks. - Show preparation, verification, pause, resume, termination, and cleanup progress from the existing agent fields. - Refresh pending steps every two seconds and other states every thirty seconds. Keep these reads separate from the agent record used by editable forms. Use the existing authorized retry route for failed steps. - Offer Pause during setup. Disable pause and resume while stop or cleanup is incomplete. Keep cleanup failures reachable on the detail page. Disable new work after termination starts. - Test required resource work across setup, pause, resume, and confirmed deletion. Test every worker callback phase, missing handlers, retry authorization, and UI behavior. ## Verification - Focused UI tests: 139 passed across lifecycle status, action buttons, API tests, and the configuration form. A regression preserves unsaved edits during an in-flight lifecycle refresh and a failed refresh. - Focused host and shared tests: 113 passed across lifecycle integration, retry authorization, plugin RPC, and manifest validation. The database suite used isolated local PostgreSQL 18 through a temporary loader outside the repository. - Module boundary tests: 11 passed. Module import/write checks and UI token gates passed. - Plugin SDK build, server direct TypeScript check, and UI TypeScript check passed. - `git diff --check` and an added-line secrets/private-reference scan passed. Fixtures use synthetic identities. - Full `pnpm -r typecheck` and `pnpm build` passed using the existing local Rust toolchain. - Visually checked all six progress phases with the application components and styles. Captured preparation and cleanup failure screenshots from the production UI build with synthetic API data; artifacts are kept outside git. - Full local `pnpm test:run` was attempted and stopped after a native-runner failure reproduced in isolation: `native-session-resume.test.ts` / `archives a damaged prior epoch before completing guarded same-task replacement with real runnerd` returns a failed Codex fixture result instead of success. The native-runtime and runner trees are unchanged from the base. No full local test pass is claimed. - All latest-head CI checks passed, including test shards, verification, E2E, and the canary packaging check. Greptile reviewed commit `96473e03a` at 5/5 with no unresolved comments. ## Risks - Selected required plugins remain a dependency for lifecycle completion. The lifecycle module retains their requirements when a plugin is disabled or removed. - Plugins must make repeated provider operations safe and fence older revisions. Host revision checks do not cancel external requests. - Polling adds one agent-detail read every two seconds during pending work and every thirty seconds otherwise. - No schema change, private VM provider implementation, resource backfill, or deployment is included. No live provider test was run. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository inspection, code editing, and command execution. The exact model revision and context window were not reported by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no unresolved comments - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-09 11:15:24 -07:00
Nicky LeachandPaperclip	2dd9811e8b	Own agent hiring and termination in a lifecycle module (#15631 ) ## Thinking Path > - Paperclip manages AI agents and their work. > - Agent creation, approval, pause, and termination change whether an agent can work. > - These changes currently use separate service and database paths. > - Some agents need setup after their record exists and before their harness can run. > - This PR adds one module that owns those state changes and their transactions. > - The module records progress and retries incomplete work after a server restart. ## Linked Issues or Issue Description Problem or motivation A hire can become active before its execution environment is ready. Direct agent state writes also bypass one common lifecycle boundary. Termination does not have a durable step for resource cleanup. Proposed solution Use agent lifecycle commands for hiring, approval, pause, resume, and termination. Keep proposed hires pending until approval. Prepare each approved hire, test its saved harness configuration, and then permit work. Wait for required plugin work before completing a step. Alternatives considered A general database commit hook would expose transaction details to callers. A timer alone would delay each new hire. This module owns each command transaction and starts its worker after commit. A periodic scan recovers work after a process stops. Roadmap alignment This change adds the lifecycle boundary for the existing agent and plugin systems. It does not add a user interface or a provider implementation. It replaces the lifecycle delivery approach in #15391. Related proposals include #11475 and #13099; this change does not add their external executor protocols. ## What Changed - Add durable approval, preparation, verification, pause, resume, stop, and cleanup states. - Keep agent creation, lifecycle state, configuration writes, and agent record deletion behind `modules/agent-lifecycle`. Keep validation, credential rules, reads, onboarding, invitations, general approvals, and company data deletion in their services. - Commit a hire before the remaining onboarding or invitation work. Serialize requests with the existing dedicated database connection helper. Reuse the hire after a failed attempt. - Keep accepted work in the run queue during setup. Require execution and cleanup evidence before completing pause or termination. - Retain the original credential owner when approving migrated hires. - Start lifecycle work after each command commits. Use database leases and revision checks for retries and late results. - Add a generic required plugin method and capability. Store the required plugin IDs for each agent. Track host and plugin completion separately. - Keep harness verification and credential rules in services. Supply their results to lifecycle through explicit ports. Reject failed logins and incomplete probes. - Remove reverse imports from lifecycle adapters into services. Share record queries and validation below both layers. - Use one hire decision path for agent IDs and approval IDs. Resolve the approval and agent change in one transaction. Expose policy reconciliation on the command object. - Save configuration and verification invalidation through one module operation. Ordinary commands own their transaction. Restrict the shared transaction integration to its listed callers. Reject identity, company, accounting, timestamp, and unknown fields. - Require completed termination before deleting agent records. The company deletion service calls the module through a separate deletion interface. - Save an approved secret before applying its agent binding. A failed binding leaves the secret approved. A retry reuses it. - Remove old permission and metadata export files. Move hire configuration rules out of lifecycle persistence. - Report failed worker steps and scans with operation context and known error codes. Omit provider messages, stacks, and configuration values. Document ordinary application use and extension rules. - Recover expired native owners only after execution stops. Retry cleanup of failed retained leases. Wait for setup in the live smoke script. - Keep manual, budget, and company pause reasons separate. Guard execution status writes with the ready state. - Add a migration, boundary checks, regression tests, and a lifecycle guide. Keep the current UI. - Limit periodic policy checks to 100 eligible agents per minute. Continue from the last agent ID. Exclude pending hires and agents in termination or a final state. Keep explicit company and budget updates immediate. ## Verification - Latest head `ccc3c86b0` aligns the shared status update schema with the PATCH route. All 15 shared tests, 104 server route/OpenAPI tests, 2 CLI tests, and shared/server/CLI TypeScript checks pass. The Greptile comment is fixed. CI is running; no new Apex review was requested for this small fix. - Rebase `75708ac94` rebases onto master at `2eb5187e4`. The backup test uses the shared 90-second timeout from master; the older timeout commit was removed. The affected backup test passes. - Agent budget edits now pass the target agent ID through enforcement. All 50 affected tests, server TypeScript, and 11 boundary tests pass. Both configuration update paths have regression coverage. CI is running on the latest head. No new Apex review was requested for these small fixes. - Rebase `9b568666a` preserves the run-control extraction and delivery-work startup changes from master. All 559 affected tests, server TypeScript, and 11 boundary tests pass. CI and Apex review are running on the final head. - The policy scan fix passes 56 lifecycle and diagnostic tests, 24 policy and caller tests, and 11 boundary tests. A regression test checks the page limit, interval, excluded states, and cursor wraparound. - Rebase `fd4698750` passes all 611 tests in eight affected lifecycle, route, and heartbeat suites. Server TypeScript and all 11 module boundary tests pass. The company deletion rollback and concurrent hire tests are included. - Follow-up `00aac7646` fixes the failed decision deletion test and the public update schema. All 126 affected server tests, 8 shared tests, 2 CLI tests, and 11 boundary tests pass. Shared, server, CLI, and UI TypeScript checks pass. - Server TypeScript passes: `pnpm --filter @paperclipai/server exec tsc --noEmit`. - All 11 boundary tests pass: `node --test scripts/check-module-boundaries.test.mjs`. The repository boundary scan also passes. - All 68 lifecycle, failure diagnostic, and harness verification tests pass. Tests reject forbidden fields through the root command, combined transition command, and restricted transaction integration. They also check invalid fields returned by configuration preparation. - Failure tests cover host work, plugin work, claims, renewals, deferrals, and scans. Log tests check known error codes and exclusion of private error contents. - All 355 tests in 10 existing caller suites pass. These tests cover secret approval, pairing, pending hires, credentials, company import, built-in agents, managed agents, instructions, and skills. - The branch is rebased onto `origin/master` at `2eb5187e4`. The queue guards and cancellation lock fix use the extracted heartbeat queue. Agent cancellation uses the extracted run-control file. The source diff scan found no secrets or private URLs. - Earlier checks cover approval retries, rollback, late verification results, policy holds, deletion rollback, concurrent company deletion, and queue admission during setup. - Before the rebase, CI passed on `00aac7646` and Greptile Apex rated it 5/5 with no open comments. CI and a new Apex review are pending for the rebased commit. The branch has no merge conflicts. - The full local build and native Dot suite were not run. The host has no Cargo. No live provider test was run. ## Risks - Background policy recovery takes more than one minute when more than 100 agents are eligible. Explicit company and budget updates still reconcile immediately. - The restricted configuration integration keeps an agent reference and its dependent records in one transaction. It cannot change a budget or invoke a lifecycle transition. A periodic scan starts pending setup after that transaction commits. - Secret approval and binding use separate transactions. A failed binding can leave an approved secret with no agent binding. - New approved hires remain unavailable until setup and verification complete. Existing clients see `paused` during these steps. This PR does not change the UI. - Migration `0320_daily_onslaught.sql` maps existing statuses and pause reasons. It does not create external resources or claim that an existing harness passed verification. - Use a coordinated server update for this migration. Older server versions can write legacy status without the new lifecycle checks. - A failed onboarding or invitation request can leave a committed hire. A retry reuses that hire and completes the remaining records. - Deletion requires completed termination. Company deletion can require another request while cleanup is pending. - Required plugins must make repeated calls safe and reject older revisions. A disabled or removed required plugin blocks completion until it is restored. - A new hire can run a second harness test after the existing UI test. Some adapters can charge for this test. - No live provider test was run. Provider implementation and resource backfill remain separate work. ## Model Used OpenAI GPT-6 through Codex, with reasoning, code tools, and local test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in this PR with the feature request field labels - [x] I have not referenced internal or instance-local issues or links - [x] My branch name describes the change - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] Targeted local tests pass - [x] All Paperclip CI gates are green on the new commit - [x] Greptile Apex is 5/5 with no open comments on the new commit - [x] I will address review comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-09 09:05:12 -07:00
Devin FoleyandPaperclip	1c4ce44e13	fix: retain bounded workspace transfer failure evidence (#15637 ) ## Thinking Path > - Paperclip manages AI agents and the work they produce. > - Sandbox work must return to the host when an agent run ends. > - A failed transfer must keep its source available for recovery. > - Some transfer errors lose their useful fields before restore diagnostics reach Sentry. > - This change records fixed transfer stages, failure kinds, and numeric RPC codes. > - The next failure can identify the operation that failed without exposing workspace contents. ## Linked Issues or Issue Description What happened? A workspace transfer can fail after model execution succeeds. Several command, archive validation, and download errors become plain errors. The existing RPC envelope then reports `unknown`, with no transfer stage or command exit code. A host RPC timeout also loses its code from restore diagnostics. Expected behavior Keep bounded evidence through the provider, worker RPC, restore result, and Sentry context. Keep the same exception, error message, retry rules, and source retention policy. A transfer timeout must remain separate from the model execution timeout flag. Steps to reproduce Return a nonzero exit code from the sandbox archive command, return a per-file download error, exceed a tar listing limit, or time out the sync-out RPC. Observe the missing fields in the saved restore diagnostic. The added tests use local fixtures for these cases. Related public work: #15479 preserves the source after restore failure. #15481 carries bounded sync-out diagnostics through RPC. This change supplies missing producer evidence and extends that same envelope. I checked the roadmap and searched open PRs for duplicate transfer diagnostic work. ## What Changed - Add optional transfer stage and failure kind fields to the existing diagnostic envelope. Capture command exits and listing deadlines at their producers. - Record only known codes from typed RPC errors at the sync-out boundary. Revalidate every field before persistence and Sentry projection. - Keep annotations private to each outbound request and restore settlement. Preserve frozen error identity and prevent evidence from leaking across concurrent or later calls. - Document the fields. Cover producer failures, quota limits, old workers, concurrent error reuse, and redaction through the real Sentry SDK. ## Verification - Focused SDK, provider, host, persistence, and real Sentry tests: 449 passed, no skips. Independent review also ran the focused contracts and all provider tests. The optional Sentry SDK is required for this check, so the contract tests cannot skip. - A compiled Daytona transfer and compiled SDK worker pass a local RPC round trip. This uses the development TypeScript loader for workspace dependency exports. The transfer and capture modules are compiled JavaScript. - `pnpm -r typecheck` and `pnpm build` passed. The excluded Daytona package also passed `tsc --noEmit`. - The initial local `pnpm test:run` stopped in the general-server group: 60 suites could not initialize embedded PostgreSQL because the isolated install skipped its Darwin library-link setup. The existing package postinstall repairs this; `initdb --version` now passes. All 60 affected suites then passed: 1,342 tests, no skips. No local full aggregate pass is claimed. - Independent source review covers privacy, concurrency, packaging, and recovery behavior. ## Risks - These diagnostics help identify future failures. They do not establish or repair the cause of a past transfer failure. - New fields are optional. Older workers remain compatible. Unknown values are omitted. - The change adds a small request-local diagnostic scope. It keeps the original thrown errors, archive confinement, cleanup order, timeout values, retry count, and source retention policy. - No schema or deployment changes are required. No live provider actions are part of validation. ## Model Used OpenAI Codex, based on GPT-6, with repository inspection, code execution, and independent agent review. The exact deployed model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-08 19:27:39 -07:00
Devin FoleyandPaperclip	3d1d5294d9	Drain idle plugin workers before automatic sleep (#15599 ) ## Thinking Path > - Paperclip runs autonomous work through agents, schedules, and plugins. > - Automatic idle sleep must preserve accepted work and unfinished cleanup. > - Every enabled plugin currently blocks sleep, including unused providers. > - An enabled flag or package version cannot prove that a worker is idle. > - This change adds a live worker drain under the existing owned hold. > - Idle workers can allow sleep while unknown work stays protected. ## Linked Issues or Issue Description Refs #15522. Related: #15391 adds plugin readiness for agent admission; this change concerns instance sleep and does not replace that contract. Subsystem affected Plugin worker lifecycle and automatic idle sleep. Problem or motivation A workspace with no pending work cannot sleep when any plugin is enabled. Removing that check alone would lose accepted RPCs, background tasks, or cleanup after a caller timeout. Proposed solution Require a live `onIdleDrain` handshake from each worker. Close admission in both processes for the exact owner and expiry. Count accepted work until completion. Continue checking durable work separately. ## What Changed - Add bounded worker holds, exact-owner release, automatic expiry, and an abort signal for plugin-owned background work. - Count host and worker requests through their real completion receipts. Keep timed-out work counted. Check active notifications and terminal routes. - Accept enabled plugins only when their current workers provide matching runtime receipts. Missing workers, old SDKs, crashes, invalid replies, and unknown cleanup still prevent sleep. - Let unused Daytona workers opt in. Once a worker contacts the provider, it remains a blocker for that process lifetime. This restriction avoids treating its existing timeout and terminal-close behavior as a cleanup receipt. - Document the plugin author contract. No schema or user-facing API is added. ## Verification - All hosted CI checks passed on `6ffb682894`, including all server, runner, workspace, build, typecheck, and end-to-end checks. One unrelated OpenCode transport test hit a five-second timeout and passed on rerun. - `pnpm -r typecheck` passed. - `pnpm build` passed. - Focused Vitest: 293 tests passed across worker admission, real child-process RPC, durable idle checks, and Daytona. - Adjacent worker manager and duplex tests: 119 passed. - Real child-process tests cover late worker completion, a host write after caller timeout, stale owners, closed admission, and release. - SDK tests cover hook failures, busy workers, expiry without another request, and late acknowledgements after expiry. - Initial long local run: 17,202 passed, 10 failed. Five failures involved parent-directory skill fixtures; four of those were already reproduced and passed in an isolated worktree. One HTTP test had a socket hang-up; all 66 tests in that file passed on rerun. The other four used new tests with runtime modules loaded before the follow-up edits; all pass in the fresh focused run above. A clean build and full-suite rerun at the final commit are running in an isolated worktree. No full-suite pass is claimed. - Follow-up: 44 focused tests passed, including repeated owned holds without an intervening normal request. Worker status changes now invalidate an in-progress scan. - Apex follow-up: 13 idle tests, 119 adjacent worker tests, 71 instance-settings route tests, and server typecheck passed. A real queued notification completes its write while preparation is in flight; synchronous and async session callback failures are logged. - Diff whitespace and added-line secret/private-reference scans passed. ## Risks - Plugins are trusted code. A hook must account for work outside SDK RPCs and keep it quiescent until its signal aborts. There is no package-name or manifest-only exemption. - This first change targets unused workers. Used Daytona workers remain awake until remote cleanup receipts are complete. Restarting does not bypass durable lease and recovery checks. - Unknown completion remains a blocker, including after a worker crash. This can retain cost but cannot authorize sleep from a timeout alone. - Enabled jobs, schedules, credentials, integrations, and other durable work remain blockers. Durable wake ownership is separate follow-up work. - Rollback restores the blanket plugin blocker. No data migration is required. ## Model Used OpenAI Codex, GPT-6. Used repository inspection, reasoning, code editing, and command execution. The runtime did not report an exact model revision or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing public issues or described the issue in this PR - [x] I have not referenced internal or instance-local issues or links - [x] My branch name describes the change and contains no internal identifiers - [x] Focused local tests pass; full CI is green (long local rerun status is recorded above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile Apex is 5/5 on `6ffb682894` with no open findings - [x] I will address all review comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-08 12:57:32 -07:00
Devin FoleyandPaperclip	465140596f	Preserve bounded workspace sync diagnostics across RPC (#15481 ) Preserve bounded error codes and HTTP/exit statuses across the environmentSyncOut worker RPC boundary, and revalidate that method-scoped envelope before attaching host restore diagnostics. Keep the original error and all recovery policy unchanged; do not transmit provider payloads or credentials. Verified real RPC roundtrip and privacy regressions, 95 focused tests, 68 independent tests, full typecheck/build and all exact-head CI. Greptile 5/5 with no unresolved comments. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-07 12:22:53 -07:00
DottaandPaperclip	b67db12d90	feat: add durable storage for private tasks (#14717 ) Add private task and project ownership, downward access grants, and immutable run/workspace provenance. Apply migration 0313 with bounded batch commits and concurrent indexes. Keep the enforcement and sharing changes in their dependent PRs. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-07 06:53:20 -05:00
Nicky LeachandPaperclip	1cebdd4c4f	Add plugin lifecycle delivery and an initial resource baseline (#15343 ) ## Thinking Path > - Paperclip manages agents and projects for work. > - Their lifecycle changes already commit to a durable journal. > - Plugins need to read these records and track completed work. > - Existing resources need a one-time baseline before delivery is enabled. > - Worker crashes must leave unfinished records available for retry. > - Each plugin needs its own progress and company access checks. > - This PR adds a pull inbox through the existing plugin SDK and job system. ## Linked Issues or Issue Description Problem or motivation Plugins cannot consume the durable resource lifecycle journal. In-process notifications can disappear during a restart and cannot record successful completion. Resource plugins need retries, company boundaries, and ordered transitions. Proposed solution Add `ctx.events.listLifecycle(companyId, limit?, afterId?)` and `ctx.events.acknowledgeLifecycle(companyId, eventId)` under `events.subscribe`. Seed a one-time baseline from current resource state. Deliver creation first, then the remaining transitions in ID order. Store acknowledgments for each plugin. Use existing plugin jobs to poll configured companies. Alternatives considered A global sequence cursor can skip lower IDs that commit later. Fire-and-forget subscriptions cannot record completion. A new dispatcher is unnecessary because plugin jobs support polling. Consumers serialize polling and use provider idempotency keys. Roadmap alignment This extends the existing plugin system and builds on #15280 and #15306. Searches found no duplicate resource inbox work. Related #13306 exposes decision events on the in-process bus. This PR includes the initial journal baseline. Provider provisioning remains separate work. ## What Changed - Add company-scoped lifecycle reads and acknowledgments to the SDK and worker RPC host. - Gate both methods by capability, invocation or proactive company scope, plugin readiness, and company enablement. - Seed hired agents and all projects once. Preserve paused/terminated agent state and archived project state. Keep pending hires behind approval. - Preserve existing history and deliver backfilled creation before partial transition histories. - Deliver project archive events from #15371 through the SDK. Seed archive intents for archived projects and update intents for active projects with a partial archive history. - Store acknowledgments per plugin and reject acknowledgments that skip earlier resource events. - Page past failed resources while retaining their pending records. Reset the page cursor each sweep to include late commits. - Add acknowledgment storage, an index for resource ordering, tests, and authoring guidance. - Preserve native identity definitions and sequence progress in JavaScript backups. Repair journal ID generators lost by older backups before seeding the baseline, without changing existing IDs. ## Verification - `pnpm -r typecheck` passed after the final origin/master rebase. - `pnpm build` passed before the final metadata rebase. The final CI build also passed. - All 46 focused lifecycle, SDK harness/RPC, migration snapshot, and legacy restore checks passed again with migration 0309. - Checks cover retries, per-plugin progress, resource order, capability/company boundaries, baseline idempotency, approval gates, archive delivery, restored projects with partial histories, and atomic migration rollback. - Full CI passed on final commit `0e674b87fc`. One unrelated Cursor fixture hit a 10-second timeout; it passed locally in under one second, and the failed server shard passed on retry. - Greptile scored the final commit 5/5 with no unresolved review threads. - The branch is rebased onto origin/master and is conflict-free. - Earlier full local runs were stopped as scope changed. CI ran the complete repository suite. - `git diff --check` and a local secret/PII scan passed. ## Risks - Apply migration `0309_loving_the_hood.sql` before starting the new server. It creates acknowledgment storage and an index, then seeds the baseline in the same transaction. Agent/project writes wait for the migration to commit. Keep these writes quiesced through the migration and activation of the new capture-capable server; do not resume an older runtime that lacks capture after the baseline. - The baseline records current desired state, not historical transitions. It includes archived projects and terminated-agent cleanup intents. It runs once with the delivery migration; no later or runtime journal backfill is planned. - Delivery is at least once. Concurrent reads can repeat an event. Consumers must serialize polling, use stable company/event idempotency keys, and acknowledge successful operations only. - Reset `afterId` at the start of every polling sweep. It is a page cursor, not a persisted high-water mark. - Deleting a plugin removes its acknowledgment records. A new installation may replay existing journal events. - Records contain identity and action. Consumers must load current authorized data before acting. Cleanup and retention policy belong to the provider plugin. - Provider calls, VM/volume provisioning, and journal retention are outside this PR. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository inspection, code execution, and tool use. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-06 12:27:09 -07:00
dependabot[bot]Priya Raman Paperclip Claude Sonnet 5	854af7df19	build(deps-dev): bump vitest from 4.1.11 to 5.0.3 (#12969 ) Bumps [vitest](https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest) from 4.1.11 to 5.0.3. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/vitest-dev/vitest/releases">vitest's releases</a>.</em></p> <blockquote> <h2>v5.0.3</h2> <h3> 🐞 Bug Fixes</h3> <ul> <li>Isolate <code>result.status</code> between <code>repeats</code> runs - by <a href="https://github.com/hi-ogawa"><code>@hi-ogawa</code></a>, <strong>Hiroshi Ogawa</strong> and <strong>Codex (GPT-6)</strong> in <a href="https://redirect.github.com/vitest-dev/vitest/issues/11218">vitest-dev/vitest#11218</a> <a href="https://github.com/vitest-dev/vitest/commit/5dbebe9e3"><!-- raw HTML omitted -->(5dbeb)<!-- raw HTML omitted --></a></li> <li>Don't print an interceptor warning in browser mode - by <a href="https://github.com/sheremet-va"><code>@sheremet-va</code></a> in <a href="https://redirect.github.com/vitest-dev/vitest/issues/11377">vitest-dev/vitest#11377</a> <a href="https://github.com/vitest-dev/vitest/commit/15cc006aa"><!-- raw HTML omitted -->(15cc0)<!-- raw HTML omitted --></a></li> <li>Don't retry when <code>test.fails</code> expectedly failed - by <a href="https://github.com/hi-ogawa"><code>@hi-ogawa</code></a>, <strong>Hiroshi Ogawa</strong> and <strong>Codex (GPT-6)</strong> in <a href="https://redirect.github.com/vitest-dev/vitest/issues/11219">vitest-dev/vitest#11219</a> <a href="https://github.com/vitest-dev/vitest/commit/b24585f08"><!-- raw HTML omitted -->(b2458)<!-- raw HTML omitted --></a></li> <li>Scope cache key generators to projects - by <a href="https://github.com/ecoyoung"><code>@ecoyoung</code></a> in <a href="https://redirect.github.com/vitest-dev/vitest/issues/11281">vitest-dev/vitest#11281</a> and <a href="https://redirect.github.com/vitest-dev/vitest/issues/11301">vitest-dev/vitest#11301</a> <a href="https://github.com/vitest-dev/vitest/commit/92ba7fc1d"><!-- raw HTML omitted -->(92ba7)<!-- raw HTML omitted --></a></li> <li><strong>browser</strong>: <ul> <li>Delay server <code>listen</code> until tests start running - by <a href="https://github.com/sheremet-va"><code>@sheremet-va</code></a> in <a href="https://redirect.github.com/vitest-dev/vitest/issues/11366">vitest-dev/vitest#11366</a> <a href="https://github.com/vitest-dev/vitest/commit/7d8ed3e9b"><!-- raw HTML omitted -->(7d8ed)<!-- raw HTML omitted --></a></li> <li>Check mock path boundaries - by <a href="https://github.com/saryn17"><code>@saryn17</code></a>, <strong>Ryosei Sato</strong> and <a href="https://github.com/sheremet-va"><code>@sheremet-va</code></a> in <a href="https://redirect.github.com/vitest-dev/vitest/issues/11361">vitest-dev/vitest#11361</a> and <a href="https://redirect.github.com/vitest-dev/vitest/issues/11362">vitest-dev/vitest#11362</a> <a href="https://github.com/vitest-dev/vitest/commit/1c3888bce"><!-- raw HTML omitted -->(1c388)<!-- raw HTML omitted --></a></li> <li>Keep config of browser-consumed environments - by <a href="https://github.com/kasperpeulen"><code>@kasperpeulen</code></a> and <strong>Claude</strong> in <a href="https://redirect.github.com/vitest-dev/vitest/issues/11378">vitest-dev/vitest#11378</a> <a href="https://github.com/vitest-dev/vitest/commit/aafc0996f"><!-- raw HTML omitted -->(aafc0)<!-- raw HTML omitted --></a></li> <li>Ignore page crash while cancelling - by <a href="https://github.com/sheremet-va"><code>@sheremet-va</code></a> in <a href="https://redirect.github.com/vitest-dev/vitest/issues/11386">vitest-dev/vitest#11386</a> <a href="https://github.com/vitest-dev/vitest/commit/7c36748fa"><!-- raw HTML omitted -->(7c367)<!-- raw HTML omitted --></a></li> <li><code>toMatchScreenshot</code> uses wrong reference on retried tests - by <a href="https://github.com/macarie"><code>@macarie</code></a> in <a href="https://redirect.github.com/vitest-dev/vitest/issues/11393">vitest-dev/vitest#11393</a> <a href="https://github.com/vitest-dev/vitest/commit/c22aba992"><!-- raw HTML omitted -->(c22ab)<!-- raw HTML omitted --></a></li> </ul> </li> <li><strong>cache</strong>: <ul> <li>Revalidate imports of cached modules - by <a href="https://github.com/sheremet-va"><code>@sheremet-va</code></a> in <a href="https://redirect.github.com/vitest-dev/vitest/issues/11381">vitest-dev/vitest#11381</a> <a href="https://github.com/vitest-dev/vitest/commit/38f98855f"><!-- raw HTML omitted -->(38f98)<!-- raw HTML omitted --></a></li> </ul> </li> <li><strong>deps</strong>: <ul> <li>Pin <code>why-is-node-running</code> to <code>3.2.1</code> to avoid users running into <code>ERR_PNPM_TRUST_DOWNGRADE</code> - by <a href="https://github.com/AriPerkkio"><code>@AriPerkkio</code></a> in <a href="https://redirect.github.com/vitest-dev/vitest/issues/11403">vitest-dev/vitest#11403</a> <a href="https://github.com/vitest-dev/vitest/commit/f6c9a4977"><!-- raw HTML omitted -->(f6c9a)<!-- raw HTML omitted --></a></li> </ul> </li> <li><strong>expect</strong>: <ul> <li>Pass current equality testers to <code>expect.extend</code> asymmetric matchers - by <a href="https://github.com/hi-ogawa"><code>@hi-ogawa</code></a>, <strong>Hiroshi Ogawa</strong> and <strong>Claude</strong> in <a href="https://redirect.github.com/vitest-dev/vitest/issues/11401">vitest-dev/vitest#11401</a> <a href="https://github.com/vitest-dev/vitest/commit/3e794a96b"><!-- raw HTML omitted -->(3e794)<!-- raw HTML omitted --></a></li> </ul> </li> <li><strong>jsdom</strong>: <ul> <li>Support Blob on jsdom 30.1 - by <a href="https://github.com/sheremet-va"><code>@sheremet-va</code></a> in <a href="https://redirect.github.com/vitest-dev/vitest/issues/11379">vitest-dev/vitest#11379</a> <a href="https://github.com/vitest-dev/vitest/commit/6c49b7197"><!-- raw HTML omitted -->(6c49b)<!-- raw HTML omitted --></a></li> </ul> </li> <li><strong>pool</strong>: <ul> <li>Preserve unique pool ids when <code>groupOrder</code> is set - by <a href="https://github.com/mtorp"><code>@mtorp</code></a> in <a href="https://redirect.github.com/vitest-dev/vitest/issues/11392">vitest-dev/vitest#11392</a> <a href="https://github.com/vitest-dev/vitest/commit/50312ebb4"><!-- raw HTML omitted -->(50312)<!-- raw HTML omitted --></a></li> </ul> </li> <li><strong>ui</strong>: <ul> <li>Split-pane handle overlapping iframe - by <a href="https://github.com/macarie"><code>@macarie</code></a> in <a href="https://redirect.github.com/vitest-dev/vitest/issues/11221">vitest-dev/vitest#11221</a> <a href="https://github.com/vitest-dev/vitest/commit/f91db0dfd"><!-- raw HTML omitted -->(f91db)<!-- raw HTML omitted --></a></li> </ul> </li> <li><strong>vitest</strong>: <ul> <li>Remove root temp dir on close - by <a href="https://github.com/abhinav-phi"><code>@abhinav-phi</code></a> in <a href="https://redirect.github.com/vitest-dev/vitest/issues/11248">vitest-dev/vitest#11248</a> <a href="https://github.com/vitest-dev/vitest/commit/7c7119cf7"><!-- raw HTML omitted -->(7c711)<!-- raw HTML omitted --></a></li> </ul> </li> <li><strong>vm</strong>: <ul> <li>Do not optimize deps from index.html - by <a href="https://github.com/ezefernandezyf"><code>@ezefernandezyf</code></a>, <strong>Hiroshi Ogawa</strong> and <strong>Codex (GPT-6)</strong> in <a href="https://redirect.github.com/vitest-dev/vitest/issues/11329">vitest-dev/vitest#11329</a> and <a href="https://redirect.github.com/vitest-dev/vitest/issues/11360">vitest-dev/vitest#11360</a> <a href="https://github.com/vitest-dev/vitest/commit/caf2887de"><!-- raw HTML omitted -->(caf28)<!-- raw HTML omitted --></a></li> <li>Don't reuse scripts across vite environments - by <a href="https://github.com/MO2k4"><code>@MO2k4</code></a>, <strong>Martin Oehlert</strong> and <strong>Claude</strong> in <a href="https://redirect.github.com/vitest-dev/vitest/issues/11395">vitest-dev/vitest#11395</a> <a href="https://github.com/vitest-dev/vitest/commit/346d3896b"><!-- raw HTML omitted -->(346d3)<!-- raw HTML omitted --></a></li> </ul> </li> </ul> <h5> <a href="https://github.com/vitest-dev/vitest/compare/v5.0.2...v5.0.3">View changes on GitHub</a></h5> <h2>v5.0.2</h2> <h3> 🐞 Bug Fixes</h3> <ul> <li>Bind <code>process</code> in case global is overwritten - by <a href="https://github.com/AriPerkkio"><code>@AriPerkkio</code></a> in <a href="https://redirect.github.com/vitest-dev/vitest/issues/11343">vitest-dev/vitest#11343</a> <a href="https://github.com/vitest-dev/vitest/commit/0b79231ad"><!-- raw HTML omitted -->(0b792)<!-- raw HTML omitted --></a></li> <li><strong>detect-async-leaks</strong>: <ul> <li>Ignore <code>process.stdio</code> handles - by <a href="https://github.com/AriPerkkio"><code>@AriPerkkio</code></a> in <a href="https://redirect.github.com/vitest-dev/vitest/issues/11333">vitest-dev/vitest#11333</a> <a href="https://github.com/vitest-dev/vitest/commit/0fd6b9790"><!-- raw HTML omitted -->(0fd6b)<!-- raw HTML omitted --></a></li> </ul> </li> <li><strong>expect</strong>: <ul> <li>Fix <code>toMatchObject</code> with asymmetric matchers - by <a href="https://github.com/ShreeBohara"><code>@ShreeBohara</code></a>, <strong>Claude Opus 5</strong>, <a href="https://github.com/hi-ogawa"><code>@hi-ogawa</code></a>, <strong>Hiroshi Ogawa</strong> and <strong>Codex (GPT-5)</strong> in <a href="https://redirect.github.com/vitest-dev/vitest/issues/11100">vitest-dev/vitest#11100</a> <a href="https://github.com/vitest-dev/vitest/commit/42523289e"><!-- raw HTML omitted -->(42523)<!-- raw HTML omitted --></a></li> </ul> </li> <li><strong>jsdom</strong>: <ul> <li>Fix <code>Request</code> with <code>Blob</code> body on jsdom 28+ - by <a href="https://github.com/harshit-d3v"><code>@harshit-d3v</code></a> in <a href="https://redirect.github.com/vitest-dev/vitest/issues/11295">vitest-dev/vitest#11295</a> <a href="https://github.com/vitest-dev/vitest/commit/d1c3ecc93"><!-- raw HTML omitted -->(d1c3e)<!-- raw HTML omitted --></a></li> </ul> </li> <li><strong>reporter</strong>: <ul> <li><code>agent</code> to respect <code>--silent</code> - by <a href="https://github.com/Raj4478"><code>@Raj4478</code></a> and <a href="https://github.com/AriPerkkio"><code>@AriPerkkio</code></a> in <a href="https://redirect.github.com/vitest-dev/vitest/issues/11271">vitest-dev/vitest#11271</a> <a href="https://github.com/vitest-dev/vitest/commit/5b95efb6d"><!-- raw HTML omitted -->(5b95e)<!-- raw HTML omitted --></a></li> </ul> </li> <li><strong>reporters</strong>: <ul> <li>Handle concurrent <code>createReport</code> calls - by <a href="https://github.com/7rulnik"><code>@7rulnik</code></a> in <a href="https://redirect.github.com/vitest-dev/vitest/issues/11278">vitest-dev/vitest#11278</a> <a href="https://github.com/vitest-dev/vitest/commit/e8e556ff7"><!-- raw HTML omitted -->(e8e55)<!-- raw HTML omitted --></a></li> <li><code>hanging-process</code> to use ESM entrypoint - by <a href="https://github.com/AriPerkkio"><code>@AriPerkkio</code></a> in <a href="https://redirect.github.com/vitest-dev/vitest/issues/11316">vitest-dev/vitest#11316</a> <a href="https://github.com/vitest-dev/vitest/commit/4e91e5668"><!-- raw HTML omitted -->(4e91e)<!-- raw HTML omitted --></a></li> </ul> </li> <li><strong>spy</strong>: <ul> <li>Fix stack overflow when spying <code>Set.prototype.add</code> - by <a href="https://github.com/fengmk2"><code>@fengmk2</code></a> in <a href="https://redirect.github.com/vitest-dev/vitest/issues/11299">vitest-dev/vitest#11299</a> <a href="https://github.com/vitest-dev/vitest/commit/a0a939653"><!-- raw HTML omitted -->(a0a93)<!-- raw HTML omitted --></a></li> </ul> </li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/vitest-dev/vitest/commit/33cadea62e8763c455c7fca38d9ab1dda87c5f75"><code>33cadea</code></a> chore: release v5.0.3 (<a href="https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest/issues/11409">#11409</a>)</li> <li><a href="https://github.com/vitest-dev/vitest/commit/346d3896b65c3c907174447a035807342799f346"><code>346d389</code></a> fix(vm): don't reuse scripts across vite environments (<a href="https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest/issues/11395">#11395</a>)</li> <li><a href="https://github.com/vitest-dev/vitest/commit/f6c9a4977ad3363f796a737834572e54c6ad5c18"><code>f6c9a49</code></a> fix(deps): pin <code>why-is-node-running</code> to <code>3.2.1</code> to avoid users running into `...</li> <li><a href="https://github.com/vitest-dev/vitest/commit/062c75d8b63519211d951d8293ea81b5a9e3c124"><code>062c75d</code></a> chore: fix standalone docs build, update exports maps (<a href="https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest/issues/11394">#11394</a>)</li> <li><a href="https://github.com/vitest-dev/vitest/commit/caf2887dee8987a60118d53933f6e9cabd6b3e2a"><code>caf2887</code></a> fix(vm): do not optimize deps from index.html (fix <a href="https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest/issues/11329">#11329</a>) (<a href="https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest/issues/11360">#11360</a>)</li> <li><a href="https://github.com/vitest-dev/vitest/commit/50312ebb4eca6a98f6d0b2b61d5d9d38cbbabcef"><code>50312eb</code></a> fix(pool): preserve unique pool ids when <code>groupOrder</code> is set (<a href="https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest/issues/11392">#11392</a>)</li> <li><a href="https://github.com/vitest-dev/vitest/commit/7c36748fad1eae9687312f2f7ceadce6ec88b5df"><code>7c36748</code></a> fix(browser): ignore page crash while cancelling (<a href="https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest/issues/11386">#11386</a>)</li> <li><a href="https://github.com/vitest-dev/vitest/commit/92ba7fc1df16a4fa5bbee3f198c582fbd56689d8"><code>92ba7fc</code></a> fix: scope cache key generators to projects (fix <a href="https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest/issues/11281">#11281</a>) (<a href="https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest/issues/11301">#11301</a>)</li> <li><a href="https://github.com/vitest-dev/vitest/commit/38f98855fa9cd7fd376afb84094eba0fda256a74"><code>38f9885</code></a> fix(cache): revalidate imports of cached modules (<a href="https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest/issues/11381">#11381</a>)</li> <li><a href="https://github.com/vitest-dev/vitest/commit/b24585f08f2ea267746a2d6ca0e43edcbb29726f"><code>b24585f</code></a> fix: don't retry when <code>test.fails</code> expectedly failed (<a href="https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest/issues/11219">#11219</a>)</li> <li>Additional commits viewable in <a href="https://github.com/vitest-dev/vitest/commits/v5.0.3/packages/vitest">compare view</a></li> </ul> </details> <br /> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Priya Raman <priya.raman@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>	2026-10-06 12:13:46 -07:00
Valentin PalkovicandClaude Opus 5.5	bb73f2fe39	feat(exe-dev): copy a source VM with exe.dev cp (#14975 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The exe.dev sandbox provider plugin gives each run its own exe.dev VM > - Today the plugin always creates that VM with `exe.dev new`, so every run starts from a bare image > - Large repositories need a long setup on each VM: toolchain, agent CLIs, package caches, and browsers for tests > - exe.dev has a `cp` command that copies an existing VM, disk and config included > - This pull request adds an optional `sourceVm` setting. When it is set, the plugin copies that VM with `exe.dev cp` instead of creating a new one > - The benefit is that operators prepare one VM once, and each run starts from it ## Linked Issues or Issue Description Refs #13575. That open PR rewrites this plugin for durable exe.dev environments. It does not add `cp`. The two changes touch the same files and can conflict. I found no issue for this. Feature description: Subsystem affected packages/plugins: the exe.dev sandbox provider plugin (`packages/plugins/sandbox-providers/exe-dev`). Problem or motivation Each run gets a fresh exe.dev VM from `exe.dev new`. We use a large monorepo (Storybook). Before the agent can work, each run must install Node, the agent CLIs, and Playwright browsers. Each run must also fill the package manager cache. This setup takes a long time on each run. Proposed solution Add a `sourceVm` setting ("Source VM" in the environment form). When it is set, lease acquisition and probes run `exe.dev cp <sourceVm> <generated-name> --json`. The command also sends the configured `cpu`, `memory`, and `disk`. The VM name, the SSH setup, the workspace, and the release and destroy steps do not change. When the setting is blank, the plugin uses `exe.dev new` as before. Alternatives considered - A custom image with `--image`: the operator must build and push a large image for each change. A private registry needs `--registry-auth`, and the plugin does not support it. - `--setup-script`: it runs on every new VM, so each run still pays the setup cost. It also has a 10 KiB limit. - `reuseLease`: it keeps one VM for one lease. It does not give each run a fresh copy of a prepared VM. Roadmap alignment The change stays inside an existing sandbox provider plugin. `ROADMAP.md` lists "Cloud / Sandbox agent support" as done and does not plan VM copies. CONTRIBUTING.md asks to discuss features in Discord `#dev` first. I open this pull request as a draft and start that discussion in `#dev`. Additional context exe.dev documents `cp` here: https://exe.dev/docs/cli-cp. exe.dev token permissions are documented here: https://exe.dev/docs/https-api. ## What Changed - `plugin.ts`: add `sourceVm` to the driver config. - `plugin.ts`: `buildCreateCommand` sends `cp` when `sourceVm` is set. - `plugin.ts`: config validation rejects `sourceVm` together with settings that `cp` cannot apply (`image`, `command`, `comment`, `env`, `integrations`, `tags`, `setupScript`, `prompt`). The error names the settings to clear. The server shows validation errors in the form, but it does not show warnings after a successful save. - `manifest.ts`: add the "Source VM" field to the "VM creation" group. Its description says that the API token must allow `cp`, because exe.dev returns 403 for a command that the token does not list. The API key description now also mentions `cp`. - `README.md`: document `sourceVm`, its limits, and the token permission. - `plugin.test.ts`: add tests for the `cp` command, the validation error, and the form field. Add `sourceVm: null` to the expected normalized config. ## Verification - `vitest run --config vitest.config.ts` in `packages/plugins/sandbox-providers/exe-dev`: 38 tests pass. The three new tests fail without the change. - `tsc --noEmit -p packages/plugins/sandbox-providers/exe-dev`: no errors. - Manual test on a self-hosted Paperclip instance (2026.1001.0). I applied the same change to the installed plugin. I prepared a source VM and set "Source VM" on an exe.dev environment. Then I ran agent tasks. Each run copied the source VM and ran in the copy. Paperclip deleted the copy at release. - With an API token that does not list `cp`, the run fails with `exe.dev API command failed (403) for: cp '<source>' '<name>' --json ...`. The new field description tells operators about this. ## Risks - Low risk. The new code runs only when `sourceVm` is set. The `new` path is unchanged. - `cp` uses the same `/exec` endpoint and its 30-second request limit. A copy of a very large disk can take longer than the limit. - Every copy inherits the source VM disk. The README tells operators not to keep secrets on the source VM. - Can conflict with #13575. ## Model Used - Provider and model: Anthropic Claude Opus 5.5. - Model ID: `claude-opus-5-5`. - Tool: Claude Code, with tool use (shell commands and file edits) and extended thinking. - Context window: the tool does not report it. - The model wrote the change and the tests. A human tested the feature on a real exe.dev setup. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) <img width="1166" height="852" alt="Bildschirmfoto 2026-10-02 um 23 04 32" src="https://github.com/user-attachments/assets/c057bee4-9f27-4a69-945e-0f71d5bcc1ba" /> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-05 22:11:59 -07:00
DottaandPaperclip	0e0b63e5a5	feat(connections): add experimental task-pinned AI routing (#14967 ) ## Thinking Path > - Paperclip manages AI agents and their work. > - AI Connections separate account access from models and harnesses. > - A pool must act as one connection while retaining each task’s account. > - Core must enforce member access and preserve session and recovery rules. > - A plugin supplies rotation policy without receiving credentials. > - This change adds durable routing and native connector setup and management. ## Linked Issues or Issue Description Subsystem affected AI Connections, Connectors, plugins, run dispatch, and session compatibility. Problem or motivation Operators need to rotate new tasks across saved accounts while each task keeps its account and session. Pool setup must fit the existing connector catalog and account workflow. Proposed solution Add an experimental router binding, a capability-gated plugin hook, and transactional task pins. Plugins declare native pooled connectors through `aiConnectionRouter`. Core hosts the existing-account picker, ordering step, and account settings. Related usage contract: #14936. Companion private plugin: https://github.com/paperclipai/paperclip-cloud/pull/643. Roadmap alignment This extends Apps and AI Connections. Core supplies generic enforcement and native connector UI; the private plugin owns rotation and quota policy. The prior duplicate search found no matching router implementation. ## What Changed - Add a router binding without changing existing concrete bindings. Keep the instance flag and new pools disabled by default. Require manual operator configuration. Show no routing toggle in Experimental settings on either open-source or Cloud installs, even after routing is enabled. - Persist company-scoped pools, one shared cursor per pool, and pins keyed by company, pool, agent, and task. Commit pins and cursor advances together with revision checks and bounded retries. Persist run-ID affinity before allocation. - Pass only authorized metadata and normalized usage to plugins. Core retains credential handling, member access checks, runtime qualification, and recovery evidence. Probe outside locks with a shared 15-second budget and freshness cache. - Resolve routing before credential preparation and backend selection. Preserve pins through turns, session resets, removed members, and quota waits. Retain admitted recovery after disable or uninstall. - Separate credential session epochs from token generations. Verified refresh preserves the epoch; reconnect and manual replacement change it. Include the credential slot ID in session and usage-cache identity, so reconnecting an indexed legacy account invalidates its old session even when both epochs are zero. - Validate pool member installations before accepting saved-agent bindings and recheck compatibility when the harness changes. Install only authorized members in the new-agent transaction and record their IDs in local activity. Pool membership cannot install a restricted shared connection. - Preserve pool bindings when agents hire teammates through either creation API or native caller runtime inheritance. Block stale manager credential references; retain explicit child authentication precedence and reject incompatible inherited pools. - Add native connector registration through plugin metadata. Reuse the Connectors catalog, setup header, account header, sidebar, dialogs, and usage display. Setup selects and orders saved connections. Advanced settings hold usage rules and member runtime defaults. New-account setup opens in another tab. - Use revision-checked pool archival from the Connectors catalog and account page. Keep task pins, cursors, recovery evidence, and underlying connections. Reject ordinary connection updates or removals that bypass pool revisions. - Add pool selectors, composer models, override notes, quota status, run details, activity records, and local run-log records. Keep session-adoption copy minimal. - Show Used by below the pool connections. List current company agents with shared avatars and profile links. Include paused agents; exclude terminated agents and agents using another pool. - Add Core stories for the generic connector workflow and runtime surfaces. Cloud stories reuse these production routes and tokens through a preview-only alias. ## Verification - Final head `73cb953bca30ed83e4505dd820edd9b5edffd28b`: full workspace `pnpm -r typecheck`, `pnpm build`, and `pnpm check:token-gates` pass locally. - All 422 focused connector/settings/shared-contract/migration tests and all 156 database-backed AI connection, hiring, reconnect, and durable-routing cases pass (69 hiring cases rerun after the final auth-precedence fix). The merged shared contract retains connection instructions and pool metadata. The pool migration is generated at sequence 0299 after the latest upstream migrations; this PR makes no lockfile changes. - All four full-app Playwright tests pass on the final head after a cold restart and migration, against the installed private plugin and isolated database, with no route or pool-API mocks. They cover hidden routing controls after manual opt-in, native pool creation, ordering, membership edits, rename, paused defaults, enabling/save/refresh persistence, stale edits, cancellation/removal, preserved underlying accounts, unavailable routers, and Used by avatars and profile links. Exact command: `PAPERCLIP_CONNECTION_POOL_E2E=1 AI_CONNECTIONS_TEST_COMPANY_ID=a37b9625-5ecf-4e29-8081-04df3d6e7d6f AI_CONNECTIONS_TEST_URL=http://127.0.0.1:3108 pnpm exec playwright test --config tests/ai-connections-app/playwright.config.ts connection-pools.spec.ts`. - [Native setup, ordering, and management screenshots](https://github.com/paperclipai/paperclip/pull/14967#issuecomment-6006976278) address the review follow-up. [Earlier selector, quota, and run-detail screenshots](https://github.com/paperclipai/paperclip/pull/14967#issuecomment-5971537316) show the runtime surfaces. Core previews: `pnpm --filter @paperclipai/ui storybook`, then Connectors / Pool host or AI Connections / Connection pools. Cloud owns its host-backed plugin stories; both repositories’ Operator Setup Required story assertions pass. - Live acceptance used OpenAI/Codex and Anthropic/Claude ACPX, resumed both exact sessions after restart, preserved pinned accounts through explicit reset and controlled quota deferral/recovery, and committed only two allocations across fourteen runs. A later UI-created task test again rotated OpenAI then Anthropic and resumed OpenAI through follow-up/restart/quota recovery. That later Anthropic execution was blocked by its saved OAuth token expiring (provider 401). No live usage probes ran. - The full local `pnpm test:run` was attempted earlier and did not complete because of macOS embedded PostgreSQL bootstrap/shared-memory failures and the 40,000-file Git fixture timeout. The focused database suites above now pass; full-suite verification is provided by the split CI lanes. The preceding CI run had one runtime readiness timeout; it passes locally both alone and inside the larger runtime suite. That larger local suite also encountered an embedded PostgreSQL setup failure and two macOS temporary-path alias assertions; those two assertions pass with canonical TMPDIR=/private/tmp. All final-head CI checks are terminal green, including full general/serialized server suites, Runner checks, browser E2E shards, canary verification, build, and typecheck. Greptile is 5/5 on that exact head with no unresolved threads. ## Risks - The migration adds routing tables and a credential epoch column. Install the private plugin only with the compatible Core contract. - Routing and each pool require opt-in. Production distribution and fleet defaults remain unchanged. - Unknown usage stays eligible. Known pinned exhaustion waits; revoked access requires operator repair. - Legacy adapters require compatible members. Runner model and effort overrides remain limited by qualified backend support. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository editing, code execution, and browser testing. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes:` / `Closes:` / `Refs:` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (targeted suites; full-suite limitations are reported above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-05 20:44:37 -05:00
DottaandPaperclip	d034ba7491	fix(interactions): derive question storage from canonical forms (#14946 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents request human input through durable issue interactions. > - A question form has a canonical presentation and a compatibility storage format. > - The creation API required agents to write both formats. > - Tool guidance told agents to split text and choice questions across those formats. > - This pull request accepts one complete canonical form and derives storage fields on the server. > - The benefit is a complete question card with stable answer and retry behavior. ## Linked Issues or Issue Description Related work: Refs #13630 and #14430. PR #13630 addresses the display of historical partial forms. This change fixes creation and keeps the check that rejects conflicting new forms. What happened? A question save supplied three compatibility questions and one canonical text question. The API correctly rejected the incomplete canonical form. The Runner's tool description encouraged this split. Sending only a complete canonical form also failed because the API required compatibility questions. Expected behavior An agent sends one complete `payload.questionSet` with every text and choice question. Paperclip derives `payload.questions` for storage and answer compatibility. Existing legacy requests remain valid. Explicitly conflicting dual forms remain invalid. Steps to reproduce 1. Call `paperclip_request_human_input` with `interactionKind: "questions"`. 2. Send `payload: { version: 1, questionSet: ... }` with a required text question and a required choice question. 3. The old API rejects the missing compatibility questions. With this change, it stores both questions and preserves the canonical form. 4. Retry with the same idempotency key. Confirm that only one interaction exists. 5. Submit both answers. Confirm that the normal resolver and continuation rules apply. Paperclip version or commit The branch is based on `cf8ad63c8`. The problem affects the native Runner and the interaction creation API. Deployment mode Server deployment with the native Paperclip Runner. Integration tests use the real interaction service and an embedded test database. ## What Changed - Add one shared canonical-to-storage projection. Reuse it for native harness question requests. - Accept canonical-only question creation at the shared validator and server boundary. - Export the input type and update the plugin SDK and its RPC contract. - Advertise a typed, complete question form in the live and scenario tool schemas. - Enforce canonical text and custom-answer constraints before ordinary or native resolution. Preserve harmless display whitespace. - Run regex matching in isolated workers with a deadline and resource limits. Both answer paths await the result before persistence. Saved native delivery uses the validated answer without taking another worker slot. - Update agent guidance and generated Runner contracts. - Test mixed forms, option-ID collisions, retries, answers, legacy requests, and conflicting forms. ## Verification - Interaction service, HTTP route, native bridge, and Runner authority suites: 221 tests passed after correcting an obsolete tool-description assertion. - Shared validator, plugin SDK, CLI, and UI compatibility suites: 67 tests passed. - Runner core tool-contract suite: 20 tests passed. AJV validates live and scenario schemas. - Final review regressions: 172 shared, service, native bridge, and authority tests passed. These cover text length, pattern, numeric limits, whitespace, custom option IDs, and historical pending cards. - Runner session suites: 67 tests passed. Published example tests: 4 tests passed. - Server typecheck and the shared/server builds passed after the compatibility fixes. - Final delivery verification: 35 response-delivery tests passed. The native delivery regression proves saved answers do not enter pattern workers; server typecheck and build passed. - Pattern security and answer-flow verification: 205 tests passed after repairing the child fixture loader. These cover pathological matching, event-loop responsiveness, worker concurrency, slot cleanup, HTTP routes, native delivery, and the full helper in a child process. - `pnpm -r typecheck` passed on the bounded-worker revision. - `pnpm build` passed on the bounded-worker revision. - All 55 GitHub checks passed on `fe457af`; four optional jobs were skipped. An unchanged Cursor adapter test timed out once in CI, passed locally, and passed on one failed-job rerun. - Reviewers can send the canonical-only mixed form above and verify that the saved interaction contains both canonical and compatibility questions. ## Risks - The creation API accepts a new input shape. Stored rows and answer contracts keep the existing shape. - The shared projection must preserve synthetic free-text option IDs. Collision and native round-trip tests cover this behavior. - Historical partial rows remain readable. New conflicting dual forms, including written-answer mismatches, remain rejected. - Existing pending cards retain the written-answer paths offered by their stored options. Canonical text constraints still apply. - Ordinary answers now enforce declared canonical constraints before persistence. Invalid answers leave the card pending. - Regex validation has a one-second deadline and a four-worker capacity limit. A complex pattern or capacity error leaves the card pending with a validation error. - No database migration or change to company authorization is required. ## Model Used - OpenAI GPT-6 through Codex. The session exposes the GPT-6 model family; its exact runtime model identifier and context window size are not exposed. Used reasoning, tool use, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-02 14:21:11 -05:00
Devin FoleyandPaperclip	4e52463203	fix(daytona): recover output from stalled log streams (#14889 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Daytona driver streams sandbox command output to the host. > - A log socket can stop delivering bytes without closing or rejecting. > - Missing permission requests and cancellation results can leave a run active and block queued work. > - This pull request switches an idle log stream to saved-output polling for the same command. > - The host can receive the missing output without another command dispatch. ## Linked Issues or Issue Description What happened? The driver waits for the SDK log-stream promise before it can start recovery. If that promise never settles, the host can miss new output that is already in the provider's saved logs. A run can remain active after a watch completes. Interrupt can then time out with “Execution is still stopping; termination has not been verified.” Expected behavior Recover command observation when the live stream stalls. Forward new permission requests and cancellation results. Require a recorded command exit before reporting completion. Keep quiet commands running under the caller's existing lifetime controls. Steps to reproduce 1. Start a session command and leave the callback log promise pending. 2. Put new output in the snapshot API without invoking the stream callback. 3. Keep the command running until the host receives that output, then expose its cancellation output and exit code. 4. Verify that the host receives each byte once and dispatches the command once. Paperclip version or commit Base commit `479554120d`. Agent adapter(s) involved Daytona session commands, including sandbox ACP agent sessions. Related: #14799 handles closed streams; this change handles sockets that never close. #14485 handles input delivery retries. #13262 adds permission diagnostics. ## What Changed - After 15 seconds with no stdout or stderr, switch directly to the existing status and log-snapshot polling path. - Ignore callbacks and delayed failures from the abandoned stream. Clear its idle timer on every exit path. - Preserve byte-offset deduplication, one command dispatch, and independent timeouts for recovery reads. - Add regressions for an initial stream stall, a stall after UTF-8 output, cancellation output, late callbacks, hung recovery reads, and quiet commands that outlive an operation timeout. - Update the provider documentation and keep hour-long healthy-stream coverage active with periodic output. ## Verification - `pnpm vitest run packages/plugins/sandbox-providers/daytona/src/plugin.test.ts`: 245 passed. - `pnpm exec vitest run --project @paperclipai/plugin-daytona`: 339 passed; 14 gated live tests skipped. - Both new stalled-stream regressions fail on the unchanged base driver because it never starts snapshot recovery. Both pass with this change. - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - `pnpm test:run`: the local run did not pass. It was stopped after confirmed local skill-path and macOS skill-cache failures, once complete PR CI was green. Four chat/email tests could not load connector skill files from an ancestor directory outside the checkout. Three company-skills tests hit macOS `EACCES` during cache publication. One unrelated wakeup test timed out in the full run and passed on a focused rerun (`1 passed`, `27 skipped`). No source or test assertions were changed for these failures. This is not a complete local-suite pass. - Complete PR CI on `11ac4e030b`: 53 successful checks, 2 expected skips, no pending or failed checks. The clean CI run includes the full test suite. - Greptile reviewed `11ac4e030b` at 5/5 with no findings or unresolved threads. The branch has no merge conflicts with `master`. - `git diff --check` and a local scan for secrets and private identifiers passed. ## Risks - Quiet healthy commands also switch to polling. Full snapshots can increase bandwidth as output grows; polling remains limited to one snapshot per second. - The SDK exposes no stream cancellation handle. The old socket remains owned by session teardown, and its callbacks cannot publish after fallback. - This change recovers a stalled output stream. It does not claim to identify every cause of an unanswered permission request or change the requirement to verify termination before releasing work. - No schema migration or command replay. ## Model Used OpenAI GPT-6 through Codex, with tool use and code execution. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run local change-specific tests; the full suite passes in CI, with local-suite limitations documented above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-01 21:04:41 -07:00
dependabot[bot]	0f9e9be408	build(deps): bump @codemirror/state from 6.7.2 to 6.7.6 (#13391 ) Bumps [@codemirror/state](https://github.com/codemirror/state) from 6.7.2 to 6.7.6. <details> <summary>Commits</summary> <ul> <li>See full diff in <a href="https://github.com/codemirror/state/commits">compare view</a></li> </ul> </details> <br /> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2026-10-01 13:45:48 -07:00
Devin FoleyandPaperclip	0d3e7bf6ac	fix(daytona): keep commands alive after log stream closure (#14799 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Sandbox providers run agent processes and send their output to the host. > - The Daytona SDK can close a log socket while the remote command still runs. > - The driver treated a clean socket close as completion before it had a command exit code. > - This pull request recovers log observation for the same command and waits for a recorded exit. > - The host keeps receiving new output without a second command dispatch. ## Linked Issues or Issue Description What happened? A clean close of the Daytona session log WebSocket resolves the SDK callback promise. If the command still runs, the driver returned `exitCode: null` with `timedOut: false` after a short status check. A streamed ACP bridge can then report a process disconnect. Expected behavior A log socket close must not complete a running command. Recovery must preserve new output, the caller's lifetime controls, and one command dispatch. Steps to reproduce Use the callback form of `getSessionCommandLogs`. Let that promise resolve while `getSessionCommand` still has no exit code. Keep the command running, then expose its final logs and exit code. The new regressions exercise this sequence, including streams that run longer than the provider operation timeout. Related: #11021, #11049. #14485 covers input delivery retries, which are a separate transport path. ## What Changed - Require a recorded command exit after a clean log-stream close. Reconnect once, then read status and full log snapshots at most once per second. - Forward new output during recovery. Remove replayed prefixes and reconcile a final snapshot so bytes written after socket close are retained. - Hold a trailing UTF-8 replacement suffix until replay or completion resolves it. This handles the SDK decoder flush when a socket closes in the middle of a character. - Preserve healthy initial and reconnected stream lifetimes. Bound each recovery read. Preserve the existing fallback timeout budget after rejected stream attempts. - Retain partial output on timeout. A log-observation timeout reports an unconfirmed result and preserves any observed exit code in metadata; it does not synthesize successful completion. ## Verification - `pnpm exec vitest run --config packages/plugins/sandbox-providers/daytona/vitest.config.ts`: 335 passed, 14 gated tests skipped. - `pnpm exec vitest run --project @paperclipai/plugin-daytona`: 335 passed, 14 gated tests skipped. - `pnpm exec tsc --noEmit -p packages/plugins/sandbox-providers/daytona/tsconfig.json`: passed. - `pnpm exec tsc -p packages/plugins/sandbox-providers/daytona/tsconfig.json`: passed. - `pnpm --workspace-concurrency=1 -r typecheck`: passed. - `CARGO_BUILD_JOBS=2 pnpm --workspace-concurrency=1 -r build`: passed. - `pnpm test:run`: exited with failure after 845.77 seconds. The server phase reported 43 failed files, 529 passed, and 163 skipped; 14 failed tests, 9,122 passed, and 5,599 skipped. All failure entries were traced to local PostgreSQL startup/cleanup errors or ten macOS skill-cache rename errors. The package helper restored 17 missing PostgreSQL library links; the four sequencing/migration tests and fourteen native-workspace-finalizer tests then passed. The ten cache failures match clean-base evidence with identical source/test blobs. This is not a full-suite pass; later local test groups did not run. PR CI supplies the complete check result. - Regressions cover clean and rejected stream recovery, two hour-long streams with a five-minute operation budget, live fallback output, one dispatch, delayed final output, stale snapshots, SDK UTF-8 decoding, bounded observation, and timer cleanup. - `git diff --check` and a local diff scan for secrets and private identifiers passed. - Greptile reviewed head `293c4dbe67` at 5/5. Both prior findings are fixed, both threads are resolved, and no new actionable findings remain. ## Risks - The SDK returns full snapshots with no offset API. After both stream attempts end, polling bandwidth grows with retained output. The one-second cadence limits request frequency. - The SDK callback stream has no cancellation handle. Existing caller stop logic and provider/session teardown still own its lifetime. Late callbacks from a settled stream are ignored. - A successful status read with no exit code keeps recovery active under the existing caller guard. A failed read stops recovery; it is not retried indefinitely. - No command replay, provider API change, schema migration, or runtime timeout policy change is included. ## Model Used OpenAI GPT-6 through Codex, with tool use and independent code review. The exact serving model identifier is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-30 18:37:54 -07:00
DottaandPaperclip	b54b2dc35c	fix: preserve warm Codex turns with incremental managed file checkpoints (#14735 ) ## Thinking Path > - Paperclip manages AI agents and keeps their instructions and files durable. > - Native Codex runners can keep a process alive between compatible turns. > - Managed file collection stopped that process after each turn, which defeated warm reuse. > - Agent folders can contain large images and other files, so full copies on every turn are expensive. > - This change keeps one managed directory for the live session and saves only file changes after each turn. > - Ownership, authorization, instruction changes, and process retirement still control when reuse is safe. ## Linked Issues or Issue Description Related: #13710 introduced native warm session reuse. This fixes managed file collection that still forced those sessions to stop. No duplicate open PR or issue was found. What happened? With managed instructions and warm native Codex enabled, consecutive turns reused a Daytona sandbox but started a new runner process each time. The managed directory collector required process termination before saving files. Expected behavior Compatible turns keep the same process and managed `AGENT_HOME`. Each completed turn saves added, changed, and deleted files before the next turn starts. Unchanged large files do not transfer again. Steps to reproduce 1. Use a native Codex agent with managed instructions and a reusable Daytona environment. 2. Enable warm session reuse and run three turns on the same task. 3. Write a large binary on the first turn, edit a small note on each turn, and delete a file on the second turn. 4. Compare process identity across turns and read the canonical files through the public agent-files API. Paperclip version or commit Reproduced on `d30b03bd8c17604cdab1533eeeeb087aba30e8b1`. Deployment mode Local server with remote Daytona execution; cloud native runner uses the same path. ## What Changed - Retain the managed directory only for the verified owner of a live native Codex session. - Checkpoint each completed turn before releasing the session for reuse. Retry unstable captures, then stop and collect when a warm checkpoint cannot be validated. - Compare metadata and cached hashes, stream only changed file payloads, record deletions, and validate path, content, quota, and authorization before saving. - Rotate sessions when canonical files, loaded instructions, credentials, or launch policy change. Fence stale collection and cleanup callbacks from later owners. - Keep cleanup and recovery aware of the current session owner. Recheck canonical files under the writer lock at handoff, attach the successor collector before fallible bookkeeping, and emit one final save receipt on checkpoint fallback. Preserve storage warnings across unchanged checkpoints. - Add regression coverage and a three-turn Daytona test with independent public API file checks, an unchanged 8 MiB binary, deletion checks, and strict process identity checks. - Document checkpoint consistency, lifecycle behavior, and local run-log counters. - Replace a timing assumption in the Daytona teardown test with explicit transfer-arrival gates after CI exposed an unset release callback. ## Verification - Full local `pnpm -r typecheck` and `pnpm build` passed. Server checks were repeated after the final storage-warning fix. - Runner E2E typecheck and 749 runner E2E unit tests passed. - Focused file checkpoint, directory ownership, instruction collection, native session, and merge tests passed. After review fixes, the managed-directory and native-session suites passed 550 tests, including intervening canonical edits, same-run fresh restore, failed handoff collection, and one-call fallback collection. Server typecheck passed again. The Daytona plugin suite passed 218 tests. The quota-warning regression failed before the fix and passed afterward. - Three real Daytona campaigns passed before the final handoff review fixes. The latest kept PID 547 across all three turns. The first checkpoint copied 8,388,635 bytes; the next two copied 36 and 54 bytes. Public API reads verified the binary, note contents, and deletion after every turn. Test cleanup deleted the sandbox. - The final head was also deployed to an isolated cloud staging instance and passed three UI-triggered native Codex turns with managed instructions. All three retained the same process ID/start time, native session, provider session, runner instance, and Daytona sandbox. Checkpoints copied 8,388,643 bytes on turn 1, then only 52 and 78 bytes on turns 2 and 3; those warm captures also hashed only 52 and 78 bytes. Independent canonical API reads verified every byte of the unchanged 8 MiB binary and the exact note contents after every turn; the deleted file returned 404 after turns 2 and 3. After restoring the original lifecycle and agent-auth configuration, removing the temporary secret, pausing the test agent, and deleting both test sandboxes, independent canonical API reads still verified the entire binary, the final 78-byte three-line note, and the deletion. The native runner flag remained enabled and the final serving revision remained the PR head. - Two earlier staging attempts are preserved as failures and are excluded from the acceptance result: a saved ChatGPT login failed with a provider routing 401, and its subsequent stopped-sandbox retry failed before provider startup with a closed-lease admission error. The successful campaign used a fresh sandbox and a temporary encrypted API-key binding. The stopped-lease retry remains unexplained; this campaign does not establish recovery of that failed sandbox. - All [Paperclip CI gates](https://github.com/paperclipai/paperclip/actions/runs/36750397355) pass on `26ef2ef56a389259246809805c0b34a4747eb86b`, including full test partitions, build, typecheck, runner verification, E2E shards, and the Canary clean public-npm install. Greptile reviewed that exact head at 5/5 with no unresolved review threads or outstanding findings. - Full local repository coverage used the existing CI partitions, but the 40,000-file Git streaming stress test timed out and its local retry was interrupted by macOS thermal emergency sleep; this is not a green full local suite claim. The exact stress test passed on the final head in [CI server shard 2/12](https://github.com/paperclipai/paperclip/actions/runs/36750397355/job/110008294290), in 111.9 seconds. - Repeat the live test with configured credentials and a Linux runner artifact: `pnpm test:e2e:runner -- --id daytona-warm-continuity.runner-codex.daytona.warm-three-turn`. ## Risks - This is a file-level checkpoint, not an atomic snapshot of the whole folder. Background writes after a capture are saved by the next checkpoint or final stopped collection. - Metadata scans still visit all paths. Modified files transfer in full; unchanged files do not rehash or transfer. - Incorrect ownership or reuse could collect the wrong directory. Run ownership fences, current authorization, stable capture validation, and stopped collection fallbacks are covered by tests. - Warm reuse remains opt-in. No database migration or fleet default changes. ## Model Used OpenAI GPT-6 through Codex, with reasoning, code editing, tool use, and test execution. The exact serving model ID and context-window size are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-30 13:33:37 -05:00
DottaandPaperclip	14795136f5	fix(runtime): finalize and recover sandbox workspace exports safely (#14402 ) Serialize native workspace finalization, validate streamed archives within bounded limits, and quietly recover unsafe exports from saved results. Preserve exact allocations for exhausted transient failures and provide export-only retry without rerunning the provider. Consolidates #14314, #14315, #14329, and #14334 while preserving the already-merged finalization label changes. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-09-28 10:57:32 -05:00
Devin FoleyandPaperclip	ffa32373bc	fix(runtime): honor provider acquisition timeout defaults (#14097 ) Declare provider acquisition budgets so slow Daytona creation does not hit the host’s 30-second fallback. Bound creation and setup to one deadline and preserve scoped cleanup ownership after timeout. Legacy drivers retain their original call shape. Validated by 238 provider/manifest tests, focused database and heartbeat regressions, root and standalone provider typecheck/build, and full CI. Local broad tests also expose recorded Mac baseline limitations. Greptile 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-09-26 00:47:21 -07:00
DottaandPaperclip	1dbffb4c04	fix(daytona): clean up sandbox allocation after create failure (#13979 ) ## Thinking Path > - Paperclip manages AI agents and their execution environments. > - The Daytona provider creates sandboxes before it returns a lease. > - Daytona can allocate a sandbox, then time out while waiting for it to start. > - The SDK throws without returning the allocated sandbox handle. > - Paperclip previously lost the resource identity and could leave a running sandbox behind. > - This pull request keeps an exact creation identity and deletes a matching failed allocation. ## Linked Issues or Issue Description Related: #13882. Searches for Daytona create cleanup and orphan fixes found no duplicate for this path. What happened? In [Grok qualification campaign 36080870743](https://github.com/paperclipai/paperclip/actions/runs/36080870743), Daytona sandbox creation exceeded 300 seconds. No native runner or provider session started. The SDK threw, but an ownership-filtered provider lookup found the allocated sandbox still running. The disposable resource was then deleted separately. The original attempt and its failure remain retained. Expected behavior A failed create must retain enough identity to clean up an allocated sandbox. Cleanup must never delete another attempt's resource. If the provider cannot confirm cleanup during sandbox lease acquisition, the host must retain a durable cleanup record and retry the exact owned allocation after restart. Steps to reproduce 1. Have Daytona allocate a sandbox for an environment acquisition. 2. Make the SDK throw while waiting for startup, before it returns the sandbox handle. 3. Before this fix, acquisition fails without deleting the allocated sandbox. 4. The regression tests reproduce this failure and verify exact-owner deletion. Paperclip version or commit Observed at `1b0551bb7c8de3c54f4bee64dbe2c88328b3645e`. The same creation path exists on master; this fix starts from `efce9356b`. ## What Changed - Assign each cold create a unique provider name and creation-attempt label. - On create failure, look up that exact name and verify every ownership label before deletion. - Wait for provider deletion with bounded lookup and deletion calls. - Preserve the original create error after successful cleanup. Report both errors and the provider name when cleanup cannot be confirmed. - Treat a missing lookup after an uncertain create as unconfirmed cleanup: a delayed provider request might still create the sandbox. - Transfer only validated ownership fields through the acquire-lease RPC error; provider exceptions and arbitrary error data are not serialized. - Persist a pending-cleanup lease before the host retries deletion. Reuse the existing cleanup sweep and durable spool, retaining the original environment scope after its row is deleted. - Fence retries by company, environment, run, provider account, unique creation name, and every ownership label. Missing provisional-name lookups remain unresolved. - Journal an ownership-verified provider ID before host-driven deletion. Its absence then confirms cleanup after a lost deletion reply or final database update; failed journal writes block deletion. - Keep ownership labels separate from the mutable input Daytona modifies during create. - Report confirmed host-side deletion accurately while still rejecting the failed acquisition. - Add protocol, malformed-evidence, cross-scope denial, controller-restart, and deleted-environment regressions alongside the original bounded cleanup tests. ## Verification - SDK suite: 92 passed. Daytona plugin suite: 277 passed; six opt-in live tests skipped. Environment-runtime suite: 103 passed, then all four focused observation/recovery variants passed after adding a foreign-observation case. SDK and provider TypeScript checks and `git diff --check` pass. - Regressions failed before the respective fixes: missing durable ownership, misleading successful-cleanup message, and SDK mutation of the ownership labels. - Controlled real-Daytona proof at `d2ac1c95b32ca64daf039c0427137657f351c0e0`: create a real sandbox; inject an SDK failure and a failed immediate lookup; persist the validated ownership envelope; have a child journal the observed provider ID before deleting; deliberately drop its successful deletion receipt; reconcile absence from another fresh process and confirm it with a separate provider read. Passed, with no manual cleanup and zero model calls. The live proof uses an owner-only envelope file; actual host database/spool recovery is tested separately in the environment-runtime suite. - The preceding controlled attempt failed because the real SDK mutated the labels object. That failure is retained. The harness deleted its exact owned sandbox, and a fresh lookup confirmed absence. The fix copies the SDK input labels separately from the ownership snapshot. - [Repository CI](https://github.com/paperclipai/paperclip/actions/runs/36095925399) passes on this exact head, with a clean 5/5 review. The unchanged Cursor remote-command test passed its one bounded rerun after a 10-second timeout; the same test had already passed on the combined tree. The original failure is retained. Earlier `199f0852` CI and immediate-deletion proof are retained, without being promoted to the new source. - Combined-source verification uses temporary PR #13990 against the Grok feature branch. No Docker or Rust build runs on the developer laptop. ## Risks Creation failures now add up to ten seconds for lookup and fifteen seconds for deletion confirmation. The SDK still owns the initial creation timeout. If a request materializes only after the failed lookup, its durable pending record remains eligible for later cleanup; a never-observed creation name is never reported deleted from a 404. A previously observed provider ID can be reconciled as deleted. The host and SDK must be deployed together. Durable handoff applies to plugin-backed sandbox lease acquisition used by heartbeat and runner login. Probe and custom-image interactive setup retain immediate cleanup and explicit failure reporting, but do not use this lease-recovery path. A worker crash before delivering the failure envelope still cannot be recovered through this mechanism. A cleanup error never returns a lease. Successful creation behavior is unchanged except for the provider name and ownership label. ## Model Used OpenAI GPT-6 through Codex, with repository tools and code execution. The exact serving identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-25 10:06:57 -05:00
Devin FoleyandPaperclip	f3a214fe77	Fix relative symlinks in secondary sandbox repositories (#13953 ) ## Thinking Path > - Paperclip manages agents and their task workspaces. > - A task can use several independent Git repositories. > - Sandbox staging copies secondary repositories from temporary clones. > - The copy changed relative symlinks into absolute host paths. > - Those links broke skill discovery and made workspace restore fail. > - This change preserves link targets while keeping extraction checks intact. ## Linked Issues or Issue Description What happened? Staging a secondary repository rewrites a link such as `.claude/skills/demo -> ../../skills/demo` to an absolute path in a temporary Git clone. That clone is then removed. The link is broken in the sandbox, and Daytona refuses the outbound archive during workspace restore. An agent can finish its turn but still have its run fail during restore. Expected behavior Repository links keep their original targets after staging. Links within a repository remain usable, and changes return to the local checkout. Unsafe outbound archive links still fail before extraction. Steps to reproduce 1. Create a project with a primary repository and a secondary repository. 2. Commit a relative skill directory link in the secondary repository. 3. Stage the workspace for sandbox execution and inspect the copied link. 4. Restore that repository through Daytona. Before this fix, the link points at a removed host temporary directory and restore rejects it. Paperclip version/commit Reproduced on `0f8750627f11d855552abce9a837d7f3b67c9ddf` in the multi-repository sandbox path. Related: #13442 introduced multi-repository provisioning. #13882 adds native Grok but keeps legacy adapters; #12991 addresses Grok instruction isolation and leaves skill staging unchanged. Searches of open/closed PRs and open issues found no direct fix for this copy behavior. ## What Changed - Set `verbatimSymlinks: true` when copying secondary Git clones. This [Node option](https://nodejs.org/api/fs.html#fspromisescpsrc-dest-options) preserves the stored link target instead of resolving it against the temporary source. - Test directory, file, chained and dangling links after temporary-clone cleanup. Assert the copied Git checkout remains clean. - Cover skill-link reads, edits and restore in fresh, warm-adoption and durable-seed workspace modes. - Extend Daytona checks for valid relative directory links and rejected absolute targets. - Document the staging behavior. ## Verification - Four regression cases fail without the source fix: one clone test and three staging modes. - `pnpm exec vitest run packages/adapter-utils/src/git-workspace-sync.test.ts packages/adapter-utils/src/sandbox-managed-runtime.test.ts packages/plugins/sandbox-providers/daytona/src/plugin.test.ts`: 301 passed. - `pnpm -r typecheck` and `pnpm build`: passed. - `pnpm test:run` was started locally, then stopped after the full Linux CI suite passed. It did not finish locally; this is not a full local-suite pass. - Full PR CI: 53 checks passed, two conditional skips, on `07b30ff298baab327a4e60708ade899935688a3f`. Three server shards were interrupted by runner shutdowns; the unchanged mobile repository test timed out waiting for a disabled Save changes button. One same-commit failed-job rerun passed. Original attempts remain in [run 36036538369](https://github.com/paperclipai/paperclip/actions/runs/36036538369). - Greptile: 5/5 on the same head, no review threads or actionable findings. - Diff scanned for secrets and private identifiers; no matches. ## Risks Low risk: the production change is one copy option. It preserves symlinks instead of following or materializing their targets. Daytona extraction guards, workspace exclusions, authentication and database behavior do not change. This prevents corruption in newly staged snapshots. It does not rewrite an already corrupted warm workspace or durable seed; those need fresh staging from the source checkout. No live provider run or customer-task replay was performed. The tests use real Git, filesystem and tar operations with mocked provider transport. ## Model Used OpenAI GPT-6 through Codex, with tool use and code execution. The exact serving model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-24 10:58:49 -07:00
Devin FoleyandPaperclip	e4237c45f3	fix(ui): prevent organization title flicker during plugin loading (#13854 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The sidebar identifies the current organization. > - An optional plugin can replace this navigation surface. > - The built-in title appears before discovery and module loading finish. > - This pull request reserves the trigger until its owner is known. > - The organization name appears once, while failures retain built-in navigation. ## Linked Issues or Issue Description Refs #13832. Searched related pull requests and issues; no duplicate fix found. What happened? The organization switcher renders a provisional built-in title before an installed replacement loads. Unrelated plugin imports can also affect its loading state. Expected behavior Reserve the trigger with a neutral placeholder, then show the resolved navigation surface. Keep the built-in menu on failed or absent contributions. Steps to reproduce Install an organization-switcher contribution. Delay session, company, contribution, and module responses. Reload the page and watch the trigger through each stage. ## What Changed - Reserve the trigger through account, company selection, slot discovery, and module loading. - Distinguish failed session lookup from pending lookup so errors retain usable navigation. - Load and await only contributions matching the requested slots. Observe completion of imports started by another consumer. - Document loading behavior and add regression coverage for loading, failures, unrelated modules, and identity transitions. ## Verification - `pnpm -r typecheck` passed, including Rust checks. - `pnpm build` passed. - All 629 UI test files passed: 6,593 tests. The 42 focused UI/API/plugin tests also passed. - `pnpm check:token-gates` and `git diff --check` passed. - `pnpm test:run` was also attempted. The broad local server run was stopped after recording skill-cache/channel fixture failures outside this diff (for example, runtime skill source status `missing` instead of `available`). The original cause is not established. All latest-head Linux CI gates pass; the complete UI suite and affected local checks pass. - Desktop (1440px) and mobile (390px) Chromium checks passed with real host components, dynamic module loading, and the built Account bundle. Delayed fixture responses produced exactly two title states: empty placeholder, then the resolved name. A slow refresh preserved the title and trigger dimensions; absent/failed plugin fallback and Escape dismissal passed, with zero uncaught browser errors. This is browser component integration, not a live signed-in tenant test. ## Risks A cold load displays a neutral placeholder until discovery completes. Absent, ambiguous, failed, and invalid contributions still use the built-in menu. No migrations or authorization changes. Scoped module loading changes when an unrelated contribution is imported; each surface loads its own matching modules. ## Model Used OpenAI Codex, GPT-6, with reasoning, code execution, and browser verification. The exact deployment ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-22 20:33:29 -07:00
Devin FoleyandPaperclip	7badae6981	feat(plugins): add an optional organization switcher slot (#13832 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Plugins can add UI surfaces to the board. > - Organization navigation is still fixed in the host sidebar. > - A distribution needs a supported way to supply its own organization menu. > - This pull request adds one optional React slot with host-owned navigation controls. > - The built-in menu stays available when the optional contribution cannot render. ## Linked Issues or Issue Description Subsystem affected Plugin SDK, server capability validation, and sidebar UI. Problem or motivation An installed plugin cannot replace the organization switcher without editing the host menu. Existing sidebar and overlay slots do not provide this replacement surface. Proposed solution Add an `organizationSwitcher` slot that requires `ui.sidebar.register`. Pass display state, an icon renderer, and navigation/logout callbacks. Keep the built-in menu for absent, ambiguous, missing, failed, or unsupported contributions. Roadmap alignment This keeps distribution UI in plugins and adds a small host contract. It does not add an account system or change company authorization. The maintainer requested this extension. Related prior menu changes: #12788, #10917, and #10850. No duplicate replacement-slot PR was found. ## What Changed - Add the slot to shared validation, SDK types, and server capability checks. - Wrap both sidebar menu variants with the optional replacement. - Resolve selection against the current account query before mounting, and reset replacement state on account or company changes. - Add host-specific component props and a fallback to `PluginSlotMount`. - Document the React-only contract and its trust boundary. - Report runtime-supervisor fixture startup details when CI readiness fails. ## Verification - `pnpm -r typecheck` and `pnpm build` passed. - `pnpm check:token-gates` passed. - `pnpm exec vitest run --project @paperclipai/ui`: 6,580 tests passed. - Targeted manifest, replacement, and built-in menu tests: 26 passed, including the incoming-account selection regression. - Installed a local test contribution into an isolated server. CLI inspection reported `ready`. Browser checks covered the loaded production UI, keyboard dismissal, current-organization selection, and an expired remote session. Remote account responses were fixtures. - `pnpm test:run` was attempted. Its first server phase passed 8,323 tests but failed in 36 files due to embedded PostgreSQL startup and filesystem permission errors on this Mac. Later phases did not run. The current Linux CI run is green: 54 checks passed and two were skipped, including build, typecheck, and browser gates. See https://github.com/paperclipai/paperclip/actions/runs/35796722770. - The earlier runtime-supervisor readiness failure did not reproduce locally. The complete affected shard passed locally: 58 files and 843 tests. The six supervisor tests also passed on Node 24.21.0 with CI flags. Added fixture startup diagnostics for the selected Node executable and listener port. The affected Linux shard then passed all 843 tests. The original root cause remains unconfirmed; no production runtime behavior or timeout was changed. ## Risks - Plugin UI remains trusted same-origin code. Display props do not authorize account requests. - A replacement can change navigation behavior. The host retains the built-in menu when discovery or rendering fails and keeps logout/session cleanup host-owned. - No database migration. Existing menus and portfolio behavior remain available. - Local full-suite verification is limited by the environment failures listed above. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository inspection, code execution, and browser testing. The runtime does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-22 16:39:11 -07:00
Devin FoleyandPaperclip	1f3ff75d33	Recognize confirmed container loss when resuming Daytona leases Daytona can retain a sandbox API record after its container disappears. Confirm the exact missing-container response with a bounded fresh read, then let the host apply its existing replacement and backup policy. Preserve unknown failures, recoverable errors, and identity mismatches. Verified 251 provider tests, 93 host lifecycle tests, plugin typecheck and build. Added 15 regressions. Read-only inspection confirmed the provider response and an existing verified backup without changing live state. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-09-21 20:35:36 -07:00
Dotta Paperclip OpenAI GPT-6	3790ca2f13	fix(runner): repair approval and Stop races and eval infrastructure (#13750 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Runner tasks must continue after approval and stop when the user presses Stop. > - Live evals found races at approval delivery and provider startup. > - Browser readiness and CI setup errors also hid the actual task results. > - This pull request fixes those races and the related test infrastructure. > - Regression tests and saved live reports show which cases now pass. ## Linked Issues or Issue Description Companion eval definitions PR: https://github.com/paperclipai/paperclip-evals/pull/25 (AgentCore paused and provider/environment infrastructure). Related: #13741 now supplies the late-startup Stop fence and warm-attachment recovery; this PR retains that fence and extends startup tracking and regression coverage to both native backend paths. #13539 introduced queued approvals during active runs. #13738 fixes child assignment, task replies, and warm process continuity and is already in the base. #13291 concerns automatic continuation of interrupted legacy sandbox runs; this PR fixes native startup cancellation and does not change that recovery policy. What happened? An accepted service approval could wait after its source run stopped. Stop could return success before the provider handle existed. Work could then start after Stop, or a cancelled run could be recorded as failed. Some E2E tests also failed on unloaded browser content or irrelevant reply wording. Runner CI could fail before model work because of dependency or sandbox setup. Expected behavior Deliver each settled approval once after its source run stops. Do not start work after an acknowledged Stop. Preserve the audited cancellation. Test the intended product behavior with a ready browser and verified runtime dependencies. Steps to reproduce 1. Approve a service request while its source run is active. Let the run finish. Check that its result starts one continuation. 2. Delay provider startup. Press Stop before its handle is available. Check cancellation, then submit `/new`. 3. Run the browser, warm-workspace, and Stop-and-redirect cases from the linked report. Paperclip version or commit The branch includes master at `9d19f98b5`. The report records the original source for each focused attempt. Deployment mode Isolated local development instances and disposable Daytona sandboxes. ## What Changed - Deliver settled tool-action results for the exact company and source run during final cleanup. Keep the existing idempotent receipt and periodic recovery sweep. - Wait for startup to hand off its provider handle before acknowledging Stop. Reject first-turn admission after cancellation. Preserve a matching audited pending or acknowledged cancellation. - Wait for mounted task history and connector controls in browser tests. Record failure evidence. Grade workspace contents and process continuity separately from exact reply wording. Require each warm-turn marker once and in order, allowing surrounding prose. - Stop-and-redirect now checks that the source file exists and work is active before Stop. - Resolve target dependency locks in an uncredentialed CI job. Verify the lock artifact hash. Keep orchestration and publication on the trusted workflow revision. - Materialize the pinned OpenCode executable and configure the exact Codex executable's user-namespace profile before provider credentials are available. - Compress Daytona directory uploads with gzip. Preserve files, executable modes, symlinks, empty directories, and confinement checks. - Classify file-transfer RPC deadlines as infrastructure. Keep unrelated runner RPC failures visible. ## Verification - [Focused live report with screenshots and original attempts](https://pages.paperclip.ing/runner-reliability-20260921/): 14 of 15 selected Product E2E cases pass across the recorded revisions. Claude and Codex Stop → `/new`, Claude service approval, delegation, both hiring/reuse cases, and native Daytona warm continuity pass. - Two credentialed Runner smoke cases pass. These are not full protocol coverage. - E2E harness after the master merge: 429 tests pass. E2E and server TypeScript checks pass. - Daytona plugin: 239 tests pass, 6 skipped. Plugin TypeScript build passes. The compression test fails against the old code and passes with the change. - Runner backend/runtime regression group: 161 tests pass. Cancellation/startup selection: 26 tests pass. Approval delivery: 34 real-database tests pass. - Workflow security: 7 tests pass. Both edited workflows pass actionlint. Runner TypeScript and Rust builds pass. - After merging master, all 389 native executor tests pass, including both native backend paths and late startup after the Stop deadline. - Post-merge `pnpm -r typecheck` and `pnpm build` pass. The monolithic local `pnpm test:run` was interrupted to integrate master and is inconclusive. The [hosted CI test partitions](https://github.com/paperclipai/paperclip/actions/runs/35620461738) pass on `50a3e43822bcba1e0d07b1b45b0be91cbf9312da`. An unchanged sandbox callback schema test initially received HTTP 503. It passed five isolated local runs, its full local test file, and one failed-job CI retry. No assertion was weakened. ## Risks - Stop can wait for the bounded startup handoff. If it cannot settle, the existing pending-recovery state remains instead of a false acknowledgement. - Immediate approval delivery must remain idempotent across cleanup and recovery sweeps. Tests cover duplicate delivery and company/run boundaries. - The workflow changes still need hosted Linux verification. They retain the trusted workflow and credential boundaries. - Gzip reduces the observed provider upload from about 1.8 GB to 663 MB. It does not yet fix the remaining Claude Daytona transfer timeout. That recovery test never reached Claude, so recovery remains unverified. Use a matching image with the verified provider package preinstalled for the next recovery test; retain cold-upload coverage separately. - The report preserves diagnostic runs with missing source metadata and marks them as such. It does not claim a new full-suite pass. - This PR adds no new prompt policy or historical status reconciliation. ## Model Used OpenAI GPT-6 through Codex performed the primary implementation and review. The exact primary backend model ID is not exposed in this session. OpenAI `gpt-5.6-luna` assisted with bounded infrastructure work and verification. The agents used repository tools, code execution, and browser tests. The exact backend revision and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: OpenAI GPT-6 <noreply@openai.com>	2026-09-21 12:50:50 -05:00

1 2 3 4 5