1265 Commits

Include renames

Author	SHA1	Message	Date
Devin FoleyandPaperclip	dfe7df0015	fix(native): expire settled native workspace sync folders (#17358) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Native runs sync their workspace with a remote sandbox through a per-run folder under `<instance>/native-workspace-sync/<run>/` > - The folder holds a descriptor, a manifest, and workspace and git seed archives, and a seed can be more than a gigabyte > - Only a few finalization commit paths delete the folder, so many runs leave it on disk forever > - Instance disks fill up with folders that no code will read again > - This pull request adds a bounded, deadline-driven retention worker that removes a folder only when no reader can still need it > - The benefit is that instance disks stop growing with stale sync folders, and recovery and export paths keep their files ## Linked Issues or Issue Description **What happened?** `cleanupNativeWorkspaceSync(runId)` is the only code that deletes `native-workspace-sync/<run>/`. It runs only when finalization commits through specific paths. Runs that end in `terminal_failure`, retained recovery, reconstruction, superseded runs, commits through other paths, and server crashes leave the folder forever. On one instance, 22 folders older than 7 days held 6.8 GB. Single seed archives were up to 1.4 GB. **Expected behavior** A run's sync folder is removed after a retention window, once the run and its finalization are settled and no recovery or export path can still read it. The shared `generations` seed cache stays. **Steps to reproduce** 1. Run a native agent with a remote sandbox and let the run end in `terminal_failure` (or crash the server during finalization). 2. Wait more than 7 days. 3. Look in `<instance>/native-workspace-sync/`. The run's folder and its seed archives are still there. **Paperclip version or commit** `master` at 837add0caa **Deployment mode** Any deployment that runs native agents with remote sandboxes. **Agent adapter(s) involved** Native runtime with remote sandbox workspace sync (for example Codex on Daytona). ## What Changed - Added `server/src/services/native-runtime/native-workspace-sync-retention.ts`. It removes `native-workspace-sync/<name>/` when `<name>` is a UUID, the entry is a real directory, its newest file is older than the window, and one of these holds: - no `heartbeat_runs` row has that id (orphan); - the run has no valid `nativeWorkspaceSync` reference and is settled: terminal status, `finished_at` before the window, no live recorded process (see below), no live controller lease, no `nativeWorkspaceFinalizationOwner` receipt; - the run is settled as above, its `native_run_finalizations` row (if any) has no `lease_owner` and no `next_attempt_at` and is `committed` (any recovery state) or `terminal_failure` with no or `blocked` recovery state, no active/escalated export-cause or owner-unverified recovery action names it, no lease carries `nativeWorkspaceExportResume` or `pendingCleanupInFlight`, retained recovery cannot still use it (referenced lease destroyed, or no scheduled/running/awaiting_evidence system recovery and no queued/recovering workspace recovery), and historical recovery adoption (`adoptHistoricalWorkspaceRecoveries`) has no pending candidate for it. - Each folder is decided in one transaction that holds `pg_try_advisory_xact_lock` on the run's finalization ownership key. The rule is evaluated while the lock is held. A held lock skips the folder. The folder is renamed to `.expired-<uuid>` in that transaction and deleted after commit. The next sweep removes leftover `.expired-*` folders. `generations`, non-UUID names, symlinks and files are never touched. Each tick tries at most 50 leftovers, then checks at most 200 folders and removes at most 50. Leftovers and folders each continue from their own cursor, so undeletable leftovers do not block run folders. A leftover that cannot be deleted schedules the 15-minute retry. Deadline discovery reads run state 200 IDs at a time and stops when asked to. It supports dry run and `shouldContinue`. - A recorded `process_pid` / `process_group_id` blocks removal only while it can still be the run's own process. This reuses the rule from `conversation-continuation.ts`: PIDs are ignored when every lease of the run is remote (they are sandbox PIDs), and a local PID or process group blocks only while it exists on this host and its start time matches `process_started_at`. An unreadable start time keeps the folder. A recycled PID does not. - Separate finding, not changed here: native runs do not clear `process_pid` when they finish. It is cleared only by the status arbiter's `release_run_resources` effect (cancellations and commits) and by the recovery paths. A failed run gets a `schedule_retry` effect instead, so its PID stays set. For remote targets the stored value is the sandbox PID. Other readers already treat these PIDs as liveness hints only, so this PR does not change that write path. - Exported `nativeWorkspaceFinalizationLockKey` from `native-workspace-finalization-ownership.ts` and used it in `withNativeWorkspaceFinalizationOwnership` and the retention worker, so both use the same key derivation. - Exported `nativeWorkspaceSyncStateRoot()` from `native-workspace-sync.ts`; `descriptorDirectory` now uses it. - Registered the worker on the shared delivery-work coordinator (`DELIVERY_QUEUES.nativeWorkspaceSyncRetention`) in `server/src/index.ts`, next to the git restore point worker. It follows `rules/sleep-aware-background-work.mdc`: no interval. It runs at startup and at real deadlines: the earliest settled run's finish (or an orphan's newest file) plus the window; a recheck one window later for folders it kept; and a 15-minute retry after a failure or a held lock. With no folders, `nextRunAt` is `null`. The run folder is now created through one helper in `native-workspace-sync.ts` that notifies process-local listeners; the worker wakes on that signal only while it has no deadline, because any scheduled deadline is never later than a new folder's own deadline. `index.ts` unsubscribes the listener on shutdown. It pauses passive maintenance during idle drain. - Added `PAPERCLIP_NATIVE_WORKSPACE_SYNC_RETENTION_ENABLED` (default on unless `false`), `PAPERCLIP_NATIVE_WORKSPACE_SYNC_RETENTION_DRY_RUN` (default off) and `PAPERCLIP_NATIVE_WORKSPACE_SYNC_RETENTION_DAYS` (default `7`, minimum `1`) in `server/src/config.ts`. Documented them in `doc/DEVELOPING.md` and added a "Native workspace sync folders" section to `doc/recoverable-disk-cleanup.md`. ## Verification - `pnpm exec vitest run src/__tests__/native-workspace-sync-retention.test.ts src/__tests__/native-workspace-sync-retention-config.test.ts` in `server/`: 62 tests pass. They cover: - deletes a settled committed run (also with a left-over `resuming_session` recovery state), a blocked `terminal_failure` with no recovery owner, an orphan, and an unreferenced settled run; - keeps runs whose local process or process group is still alive, or whose live process start time cannot be read; keeps a running run with a PID; deletes terminal runs whose recorded local process exited, whose PID was recycled, or whose PID came from a destroyed sandbox; - keeps recent files, recently finished runs, unsettled runs, a live controller lease, an owner receipt, `observed` and `retryable_failure` phases, a live `lease_owner`, a scheduled `next_attempt_at`, and each non-blocked recovery state on `terminal_failure`; - keeps runs with an open export-cause or owner-unverified recovery action, `nativeWorkspaceExportResume` or `pendingCleanupInFlight` lease metadata, retained recovery in flight on a lease that was not destroyed, or pending historical recovery adoption; deletes when the retained lease was destroyed; - never touches `generations`, non-UUID names, symlinks or files; dry run deletes nothing; leftover `.expired-*` folders are removed; an advisory lock held by another transaction makes it skip; ticks are bounded and resume from a cursor; - `nextDeadlineAt` returns `null` with no folders, now with leftovers, the settled finish plus the window, the orphan mtime plus the window, and one window later for kept folders; the registered worker goes quiet with no folders, retries after a held lock or an undeletable leftover, and the real delivery-work coordinator wakes again when the real producer (`writeDescriptor`) creates a run folder; leftover removal is bounded and undeletable leftovers do not block run folders. - Mutation checks: I removed each guard one at a time (advisory lock, owner receipt, process pid, finished_at window, lease owner, next attempt, phase, recovery state, export recovery, owner-unverified, export resume, retained recovery, backfill pending, symlink/real-directory check, dry run, and both mtime checks together, plus the folder-creation wake, the process liveness rule (restoring the old `process_pid IS NULL` rule, the sandbox exemption, the process group check, the recycled-PID check, the unreadable start time), the leftover batch limit, the leftover cursor and the leftover retry delay). Each mutation made at least one test fail. I restored the code after each one. Removing only the first mtime check does not fail a test, because the second check inside the transaction still keeps the folder. - `pnpm exec vitest run` on related suites in `server/`: `native-workspace-sync.test.ts`, `native-workspace-sync-history.test.ts`, `native-workspace-finalizer.test.ts`, `native-workspace-finalizer-failure.test.ts`, `native-harness-backup-expiration*.test.ts`, `git-restore-point-ledger*.test.ts`, `delivery-work-coordinator.test.ts`, `chat-delivery-work.test.ts`. All pass. - `pnpm run typecheck` in `server/` passes. - Greptile review: earlier rounds; all findings fixed with tests; final score 5/5 with no open threads. - Not run: the full server test suite and an end-to-end run against a real instance. I did not measure disk savings on a live instance. ## Risks - Deleting a folder that a reader still needs would make that path fall back or fail. Same-run resume and export run under the finalization ownership lock or before the run settles, and the worker checks the rule while it holds the same lock. Retained workspace recovery already handles a missing folder (ENOENT) by snapshotting the sandbox. The rule keeps the folder for every in-flight or reopenable recovery state listed above. - A committed finalization row with a stale recovery state and no lease owner is treated as settled. Nothing reads the recovery state of committed rows. - The worker is on by default. Operators can set `PAPERCLIP_NATIVE_WORKSPACE_SYNC_RETENTION_DRY_RUN=true` first to see candidates in the log, or turn it off with `PAPERCLIP_NATIVE_WORKSPACE_SYNC_RETENTION_ENABLED=false`. - The first sweep on a large instance does more filesystem work. It is bounded to 200 folders checked and 50 removed per tick. - The folder-creation wake is process-local. A folder created by another server process sharing the same instance directory is picked up at the next deadline or restart. - A stale `process_pid` no longer blocks removal. If a run's local process is still alive with its recorded start time, the folder stays. Live work is also still guarded by the run status, controller lease, finalization lease and ownership lock checks. - No schema migration. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file edits, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 14:05:55 -07:00
DottaandPaperclip	4d90634f74	feat(connections): add 1Password service-account connector (#17249) ... ## Thinking Path > - Paperclip lets people manage AI agents and govern their access to outside services. > - Apps connections keep provider credentials in the instance vault and mediate every action through grants, policies, and audit. > - Agents currently have no Apps connection for reading or managing 1Password items with an operator-supplied service-account token. > - 1Password supports this through its official SDK; its documentation MCP server does not expose vault operations. > - This pull request adds a one-field service-account connection and registers eight reviewed SDK actions through the existing native-provider catalog. > - Operators can select vault permissions in 1Password and agent/action access in Paperclip without installing a CLI or handing durable credentials to agents. ## Linked Issues or Issue Description Related: #16144 adds 1Password as Paperclip's secret-storage backend. This PR adds the separate Apps/tool-access flow with a per-connection credential; it does not change secret providers or use a server-wide ambient token. **Subsystem affected** Apps catalog, native tool-provider registry, and shared connection setup. **Problem or motivation** People need to paste their own `OP_SERVICE_ACCOUNT_TOKEN`, find official service-account setup instructions, and authorize agents to use the selected vaults through ordinary Paperclip governance. **Proposed solution** Add `1password/service-account`, with a password field labeled **Paste your 1Password Service Account token** and a link to [1Password's creation instructions](https://www.1password.dev/service-accounts/get-started#create-a-service-account). Use the pinned official `@1password/sdk` 0.5.0 for vault/item listing, item and secret-reference reads, and item create/update/archive/delete. **Alternatives considered** A remote MCP card would not provide vault access. A local CLI or a third-party MCP package adds installation and credential surfaces that the official SDK avoids. ## What Changed - Added the source and generated catalog definition, permission review, official light/dark artwork, documentation, and capability-map references. - Added a provider-owned SDK bridge with strict schemas, account-domain validation, contract hashes, fixed errors, bounded output, and disposable workers with cancellation/deadlines. - Registered the provider through master's native-provider catalog; existing setup, health, discovery, gateway, policy, and audit owners remain in place. - Labeled the single password field **Paste your 1Password Service Account token**. The shared formatter preserves explicit credential prompts; ordinary key labels retain their existing presentation. - Added credential-ownership, gateway read/write, policy/revocation, worker, setup UI, and interactive Storybook coverage. ## Verification - Final review follow-up: added two deterministic worker-entry tests for nested exact-token redaction and oversized-output rejection. Removing each guard caused exactly its regression test to fail. The focused provider suite passed **15 tests**, and the official server typecheck passed after rebuilding stale workspace SDK artifacts. Current head `569cb7f9bd57db791fb13a6034ce69bf50878cb1` passed the full [CI run](https://github.com/paperclipai/paperclip/actions/runs/40016166260), including typecheck, build, unit/integration tests, and all eight browser shards. The required `ci / verify` and `ci / e2e` gates passed. [Greptile](https://github.com/paperclipai/paperclip/pull/17249#issuecomment-6353731805) and [Paperclip Review](https://github.com/paperclipai/paperclip/pull/17249#issuecomment-6372165895) each rated this exact head **5/5**, with no open findings. Paperclip Review completed with a neutral check because its scoped reviewer could not inspect CI; the full CI result was verified separately. Other neutral checks and intentionally skipped Storybook jobs contain no actionable findings. - CI recovery on the same commit: the first attempt lost six jobs. Five logged simultaneous runner shutdowns at 18:49 UTC on 2026-10-23; the sixth stopped logging before a test result and its root cause was not established. No assertion failure appeared in the retrieved logs. All six jobs and the aggregate gates passed one failed-job retry. [CI run](https://github.com/paperclipai/paperclip/actions/runs/40005754936). - Copy follow-up: the catalog, provider, and setup UI suites passed **275 tests**. The governed gateway lifecycle passed **3 tests** (one live-only test skipped), including create, update, archive, and delete. UI token gates and map checks passed. The branch was rebased onto `9c442cf48e` to resolve additive catalog conflicts while retaining the Vercel and Supabase entries. The existing live write proof below remains applicable; this follow-up changes presentation only. - Five affected suites, including the existing Daytona consumer, passed **290 tests** (one opt-in live test skipped in fixture mode). Full `pnpm -r typecheck` and `pnpm build` passed. The local broad-suite attempt could not complete because PostgreSQL semaphore exhaustion prevented a competing integration suite from starting; the complete affected-file retry passed after disposable-process cleanup. The prior commit `f208a8aff6dc28859cc3f343497e18d85f124579` passed the complete CI matrix. All CI build, typecheck, test, and browser gates also passed on the earlier commit `9e2f3d3140368a08da43b004b2e8d3a0f4c0fa4d`. The separate Paperclip Review reported a worker-result regression-test gap on that head; the follow-up above addresses it. Its review thread is resolved, and both fresh current-head reviews are clean as recorded above. - Token gates, module boundaries, brand validation, map checks, and Storybook catalog checks passed. - Direct SDK create/archive/delete and rejected-call coverage passed (13 provider tests); the Storybook Permissions/Test play completed successfully in Chromium. Server and UI typechecks passed after those fixes. Greptile passed prior commit `9e2f3d3140368a08da43b004b2e8d3a0f4c0fa4d` at **5/5**. Both reviewers also passed the worker-test follow-up at **5/5** as recorded above. - Live 1Password proof on 2026-10-22: connect, eight-action discovery, vault listing through an actual run-scoped gateway, refresh, reconnect preserving identity, another read, removal, and denial through the retained session passed. Credential absence was asserted in responses/audit; an exact-token scan of inspected logs and changed files passed. - Real browser: Connectors → 1Password → token entry → personal access limited to one disposable agent → completion → Permissions → Test `onepassword_list_vaults` succeeded. The disposable connection was removed afterward. - Compiled-worker smoke initially exposed inherited Node launch flags. Workers now use explicit empty `execArgv`; the compiled worker then authenticated successfully and listed one vault. - Live write proof on **2026-10-23**: the replacement Keychain token passed all eight actions through the actual run-scoped gateway for both personal and shared credentials. The test created, listed, read, updated, resolved a secret from, archived, and deleted two synthetic items. Both were absent from subsequent item lists. Refresh/reconnect, identity preservation, removal, retained-session denial, and credential absence passed. The live suite passed **3 tests** (one fixture-only test skipped) in 116.65 seconds. The expanded fixture suite passed **3 tests** (one live-only test skipped); the 13 provider tests and server typecheck passed. An exact-token scan passed over 64 source/log files. - Failure history: the original token denied creation on 2026-10-22 and created no item. The first 2026-10-23 attempt stopped before provider calls because host PostgreSQL semaphore limits prevented test-database startup. A temporary ten-connection test profile allowed the complete retry to pass without changing shared services or host limits. A new delete-result assertion also exposed an outdated fixture response (`success` instead of `deleted`); the fixture was corrected and the complete fixture suite passed. Official contract sources: [service accounts](https://www.1password.dev/service-accounts/get-started), [SDKs](https://www.1password.dev/sdks), [item management](https://www.1password.dev/sdks/manage-items), [official JavaScript SDK](https://github.com/1Password/onepassword-sdk-js). ## Risks - Authorized item/secret reads intentionally return secret values to the agent and ordinary tool-result history. Select trusted agents and use restricted service-account vaults; the connection token itself is redacted. - New connections retain existing Paperclip action defaults, including Allowed writes/destructive actions for selected agents. 1Password still enforces the token's permissions. - Service-account access cannot be changed in place. Create a replacement account and reconnect to change vault permissions; read-only keys can connect but cannot write. - SDK authentication happens per action in a disposable worker. Calls have a 30-second maximum; cancellation stops local work, while a mutation already accepted by 1Password can still finish. Writes are never automatically retried. - Additive dependency and catalog change; no database migration. Disable/remove the connection to withdraw access, or remove the provider from catalog visibility for rollback. ## Model Used OpenAI Codex, based on GPT-6, with code editing, shell/browser tools, and live provider verification. A more specific runtime model ID and context-window limit were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 13:27:33 -07:00
Devin FoleyandPaperclip	ef9e17304c	perf: narrow runtime progress database reads (#17355) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Active runs publish workspace and provider progress. > - Each update must check the current run status before it publishes. > - This check fetched the full run, including prompts and runner configuration. > - This pull request reads only the four fields that progress publication uses. > - The benefit is less database transfer while each update still checks durable state. ## Linked Issues or Issue Description Related: #17348 narrows authorization reads. This PR applies a separate projection to runtime progress. GitHub searches for runtime progress and run-status read PRs found no duplicate. **What existing behavior does this improve?** The database read before a runtime progress update. **Current behavior** Each update calls `getRun`. This reads large run context and runner configuration even though progress publication only uses the run ID, company ID, agent ID, and status. **Proposed behavior** Use a fresh four-column read for each update. Keep the existing active-status check, event fields, error propagation, and late-progress suppression. **Reason and benefit** Transfer progress can emit many updates for one run. A small status check avoids repeatedly transferring payloads that it does not use. **Breaking changes** None. No API response, schema, scheduling, or cancellation policy changes. ## What Changed - Add `getRunStatus` to the run-state owner. The full reader remains available to consumers that need execution data. - Wire the narrow reader into the lifecycle progress check. Remove the unused context field from that check's input contract. - Cover the exact selected fields, missing runs, database errors, large stored payloads, terminal transitions, and the public heartbeat service wiring. Document the read boundary. ## Verification - Run-state and lifecycle suites: 44 tests passed before the additional public-service regression. - Lifecycle, public-service wiring, and native cancellation suites: 36 tests passed. - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - The [full PR workflow](https://github.com/paperclipai/paperclip/actions/runs/40013092548) passed on `95bf496f689049812646b5ccddcdf2cca21efad0`. All 54 reported checks passed; two conditional checks were skipped. This includes test shards, browser checks, Runner verification, production bundle boot, and clean npm installation. - `GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=init.defaultBranch GIT_CONFIG_VALUE_0=master pnpm test:run`: an additional local all-in-one run is still in progress; no full-local-pass claim. The process-scoped Git default matches existing master-only fixtures and does not change user configuration. - The initial description check failed because the dedup-search checkbox was missing. Searches are complete and the template checkbox is now included. The rerun passed on the same source commit. - Greptile Apex: 5/5 on `95bf496f689049812646b5ccddcdf2cca21efad0`, no findings or review threads. - No hosted deployment or bill reduction is claimed. CI and Apex are complete; this PR is merge-ready. ## Risks - Low scope: the query selects fewer columns and preserves the existing status read on every progress callback. It adds no worker, cache, timer, or idle query. - A run can still finish after the status read and before publication, as it could before this change. This PR does not change that concurrency boundary. - No migration or operator configuration is required. ## Model Used OpenAI Codex (GPT-6 family). The runtime does not expose the exact model ID or context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites above; full suite is still running) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 12:50:36 -07:00
Devin FoleyandPaperclip	837add0caa	fix(recovery): deliver review-path repair instructions to agent prompts (#17353) ... ## Thinking Path > - Paperclip keeps agent tasks moving through owned execution and durable waiting paths. > - A consumed review path gets one repair wake before board escalation. > - That wake records an instruction to save a new monitor or another real review path. > - The canonical wake builder omitted that instruction from the agent prompt. > - This pull request delivers it through the existing disposition-repair section. > - The repair limit and execution authority stay unchanged. ## Linked Issues or Issue Description **What happened?** An agent opened a pull request and promised to watch CI and external review. It ended without scheduling a monitor. Its bounded review repair also ended without a maintained path, so the task became blocked for a board decision. The saved repair context contained the instruction, but the canonical wake payload and rendered prompt did not. **Expected behavior** The repair prompt tells the agent to save a one-shot monitor before promising another check. Both fresh and resumed sessions receive that instruction and the one-attempt limit. **Steps to reproduce** 1. Create a review-path recovery with `decideIssueReviewPathRecovery` for a task whose review path was consumed. 2. Pass its recorded context through `buildPaperclipWakePayload` and wake serialization. 3. Render a fresh or resumed prompt. Before this fix, it has the wake reason but no repair instruction. **Paperclip version or commit** Base: `9c442cf48e6`. Current head: `f38d65891c8d9f2b38eb76a8578e870debc2b1e6`. **Deployment mode** Observed on managed sandbox execution with the legacy Claude ACP adapter. The corrected canonical payload serves the shared adapter prompt path. Refs #15764 introduced the bounded exhausted-repair disposition. This change preserves that limit. Open monitor-retry work such as #17171 addresses an already scheduled check; this defect occurs before a check exists. Searches found no duplicate for this missing prompt field. ## What Changed - Build the review-path repair disposition in its owning recovery module, using the server instruction and fixed one-attempt bound. - Project it into the existing canonical `dispositionRepair` field. No new adapter field or execution protocol is added. - Ignore arbitrary replacement instructions and attempt-limit values. A pending repair keeps its instruction across a connection-pool retry that changes the wake reason. Ordinary wakes without the typed repair marker ignore raw instruction text. Existing legacy disposition repairs retain their values. - Add a regression across the real recovery decision, wake builder, serializer and fresh/resumed prompt selectors. Update execution guidance and the affected recovery maps. ## Verification - Verified red before the source fix: two regression cases failed because the instruction was absent; the sibling repair case passed. The same cases pass after the fix. - Passed 194 focused and safety tests across wake preparation, review recovery, adapter prompt handling, and database-backed liveness escalation. The connection-pool retry case also failed before its correction and passes after it. - Replayed the recorded incident wake through the fixed boundary. The old canonical prompt lacked its saved instruction. The corrected serialized prompt includes it with a 1/1 bound. This replay did not execute a provider turn or change the source task. Private evidence stays outside the repository. - Passed `pnpm -r typecheck` after the pool-retry correction. Passed `pnpm build` on final head `f38d65891c8d9f2b38eb76a8578e870debc2b1e6`. Map checks and `git diff --check` pass. - [Full CI](https://github.com/paperclipai/paperclip/actions/runs/40005890857) passed on that head: 54 checks passed and two optional checks were skipped. This includes the full test shards, typecheck, build, and browser suites. The local `pnpm test:run` attempt was stopped after the CI test shards passed. No complete local full-suite pass is claimed. - [Apex review](https://github.com/paperclipai/paperclip/pull/17353#issuecomment-6372160694) is 5/5 on the final head, with no open findings. The first review found the pool-retry case. Its regression and correction are included. The branch has no merge conflicts. - Test setup first required the plugin SDK build, then an empty credential-history database boundary. Those setup errors are not the claimed red. The temporary recorded-wake probe was removed before final validation. - Hosted follow-up acceptance is not run. The source task remains blocked and needs a retry after rollout. This PR does not claim that prompt delivery itself schedules a monitor or that every model will follow the instruction. ## Risks - This fixes instruction delivery. The agent must still save a valid monitor or another waiting path. The existing bounded escalation remains if it does not. - The wake stays a normal-model repair. It does not become status-only recovery and can continue useful work. Existing ownership, approval, budget, pause, and monitor admission gates remain in place. - No schema, credential, telemetry, API, model, or retry-budget change. ## Model Used OpenAI Codex, based on GPT-6, assisted with diagnosis, code editing, shell execution and verification. The exact serving model ID and context-window limit are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (194 focused and safety cases; full coverage passed in CI) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 12:07:07 -07:00
DottaandPaperclip	9c442cf48e	feat(connections): deploy and configure Supabase and Vercel (#17217) ... Add governed Supabase and Vercel deployment/configuration connectors with vaulted credentials, agent access policies, approvals, audit, and bounded provider operations. Deliver reviewed REST actions to native agents and guard Vercel preview intent on empty projects. Preserve secret-safe request identity and durable retry behavior. Verified by current-head CI, fresh Greptile 5/5 review, and real-agent deployment/configuration walkthroughs on disposable provider resources. Test resources and temporary provider tokens were removed after verification. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-23 11:37:29 -07:00
Devin FoleyandPaperclip	f406b32d08	perf: narrow run authorization reads (#17348) ... ## Thinking Path > - Paperclip lets people manage and observe AI agents at work. > - Live events and run history must enforce current task permissions. > - Some permission checks load the complete run record for each event or request. > - Run context, runner configuration, results, and output can make those records large. > - This pull request reads only the persisted fields each access check needs. > - The checks still read current permissions and fail closed. ## Linked Issues or Issue Description Related: #17251, #9155, and #16229. Those changes cover other run projections, recovery polls, and task history. This change covers live-event delivery, workspace-operation run checks, and event-list access. **What happened?** Live-event delivery loaded every run column for each subscribed viewer, although the permission helper used only the company, run scope, and task binding. Workspace-operation checks did the same. The run-event endpoint also loaded complete run records before returning separate event rows. Large contexts and runner configurations therefore increased database transfer for ordinary access checks. **Expected behavior** Read only the stored access binding. Keep company checks, permission freshness, deleted-task tombstones, and response bodies unchanged. **Steps to reproduce** 1. Create an issue-bound run with large context, runner configuration, and output fields. 2. Subscribe an authorized viewer to company live events and publish run updates. 3. Read the run events and a linked workspace operation. 4. Inspect the run SELECT projections. The previous queries load fields that the access checks do not use. **Paperclip version or commit** The defect is present on the current master base, `62c78e56e53`. **Deployment mode** Authenticated server with PostgreSQL. The defect is not adapter-specific. ## What Changed - Select only company, scope, and issue bindings for live-event and workspace-operation run authorization. - Add a four-field run-access read for event-list requests. Reuse those columns in the existing log-access projection. - Preserve per-delivery and per-request permission checks, company predicates, and event responses. - Add real PostgreSQL and WebSocket coverage with large run fixtures. Cover grant revocation, grant restoration, deleted tasks, missing and foreign-company runs, and failed reads. - Document the bounded access reads in the existing polling-load guide. ## Verification - Six focused suites pass: 40 tests. They cover the changed paths, run-state reads, run logs, disconnect retries, and native event replay. - The new WebSocket test uses the production delivery path and a real database. It verifies that the selected SQL omits large run fields while authorization changes take effect on the same open socket. - The event-route test preserves the returned event content and denies the next request after grant revocation. - Local recursive typecheck, full build, and module-boundary checks pass. [Exact-head CI](https://github.com/paperclipai/paperclip/actions/runs/39997370119) completed successfully, covering the full grouped unit suites, typecheck, build/runtime boot, and E2E shards. The separate serial local `pnpm test:run` is still running and is not claimed as completed. - A later CI shard failed an unchanged email test expecting exactly one task after permission grants. The unchanged-base test passes at normal speed; adding only a 2.2-second pause before granting access reproduces the extra tasks because previously denied deliveries become due for retry. No email code or assertions are changed in this PR. The original failure is retained. The single shard rerun passed: 92 suites and 1,489 tests passed, with one suite and two tests skipped by the existing gates. - CI initially found a route-test double missing the new access helper. The follow-up fixes that fixture and adds event reads to malformed-ID and missing/cross-company checks. All 122 tests across the four changed-path suites pass; a final rerun of the expanded route suite passes all 92 tests. - The first CI run also timed out in the signoff changes-requested handoff while waiting for a reviewer run after a cancelled execution blocker. The focused test passes locally on this source. The same focused test passes three times on unchanged master. Controller-lease cancellation messages appear on both versions. The original CI timeout is preserved as an intermittent, unreproduced failure; its exact cause remains unknown. The signoff shard passes in the final exact-head CI run. - Initial fresh-checkout tests could not start because workspace SDK output and native PostgreSQL library links were missing. Dependency builds and the package's native-link setup corrected those prerequisites. The subsequent focused run passed. Focused command: ```sh pnpm --filter @paperclipai/server exec vitest run src/__tests__/live-events-run-read.test.ts src/__tests__/heartbeat-run-privacy-routes.test.ts src/services/heartbeat/run-state.test.ts src/__tests__/native-observer-replay.test.ts src/__tests__/heartbeat-run-log.test.ts src/__tests__/heartbeat-run-privacy-disconnect-retry.test.ts ``` ## Risks - A missing access field could cause an incorrect allow or denial. Tests exercise current grants, revocation, company scope, and deleted-task bindings through the real consumers. - No successful permission decision is cached. Every delivery and request still checks current database state. - No schema, migration, polling interval, API response, or deployment configuration changes are included. - This reduces bytes for the affected reads. It does not establish a whole-account savings percentage. Hosted transfer measurements remain separate validation after staging deployment. ## Model Used OpenAI Codex, a GPT-6 based coding agent, used repository tools, code execution, and database-backed tests. The session does not expose the exact backend model identifier or context-window size; those values are not guessed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 11:01:06 -07:00
Nicky LeachandPaperclip	a708a13336	Separate plugin workspace completion from maintenance inspection (#17329) ... ## Thinking Path > - Paperclip manages AI agents and their work. > - Storage plugins own remote task workspaces. > - Run completion currently requests the same inspection used for storage cleanup. > - That inspection can fetch remote refs and scan files after the agent has finished. > - Completion needs retained workspace identity and durable writes, not permission to delete files. > - This change adds an optional completion operation and keeps maintenance inspection separate. ## Linked Issues or Issue Description **What happened?** A successful plugin-backed run can fail during completion because the storage provider performs a slow remote-recoverability inspection. Dirty or unpublished output does not need a deletion-safety proof to remain valid task output. **Expected behavior** Completion verifies the assigned workspace and confirms durable writes through the provider. Cleanup still requires the existing full inspection. **Steps to reproduce** Run a task with plugin-owned storage whose inspection fetches remote refs. Let the agent finish, then observe completion waiting for the maintenance inspection. Related context: #16428 added finalization observability. The duplicate search found no existing PR for this provider contract. ## What Changed - Add optional `workspaceProvider.finalizationVersion: 1` and strict `finalize` / `finalized` schemas. - Pass original run and agent context through live completion and recovered completion. - Give the new operation a 30-second deadline and keep ownership checks and lifecycle serialization. - Retain the inspection fallback for providers without the capability. Keep maintenance inspection and removal unchanged. ## Verification - SDK build passed. - Targeted SDK and completion unit tests: 7 passed. - Added integration coverage for capability selection, legacy fallback, maintenance separation, and failure propagation. - Local integration setup timed out starting PostgreSQL; the integration suite passed in GitHub CI. - Full build/typecheck and broader tests passed in GitHub CI (55 successful checks, 2 skipped). Local checks remained targeted as requested. - Diff reviewed for secrets and operational identifiers. ## Risks The capability is opt-in. Providers must not advertise it until their completion implementation is available. An unavailable provider or failed durability barrier remains retryable. No schema or stored-binding migration is required. Test-file size exception: the provider integration suite grows from 973 to 1,002 lines. The 29-line case follows the same prepared binding through legacy completion, opted-in completion, and maintenance inspection using the existing single-connection database fixture. Keeping it there avoids a second embedded-Postgres lifecycle for one transition test. The completion helper unit tests are in a separate focused file. ## Model Used OpenAI GPT-6 Codex, with reasoning and code/test tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 09:07:35 -07:00
Devin FoleyandPaperclip	0fb152ee94	fix: save stale remote-tracking commits in nested restore points (#17289) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Git cache expiration removes closed issues' worktrees once their remote, or a restore point, can reproduce them > - A nested `.paperclip-repositories` checkout's remote-tracking refs can point at commits the remote has since dropped, such as a branch deleted after a squash merge > - Strict expiration rightly keeps such a checkout, because it may hold the only copy; but restore point mode also kept it forever, because there was nothing else to save > - This pull request saves those commits in the checkout's restore point, so the workspace can be reclaimed without losing them > - The benefit is that long-lived multi-repository workspaces are reclaimed in restore point mode ## Linked Issues or Issue Description Refs #17257. Bug report described in-PR (bug report template). **What happened?** With restore points enabled, four closed workspaces on one instance were kept with `unpushed_managed_checkout`, after a dry run had counted them as candidates. Each nested `paperclip` checkout was clean, on `main` at a commit that is on GitHub `main`. But it held about 1,000 remote-tracking refs, and between 28 and 41 of them pointed at branches the remote has since deleted or force-pushed (147 to 203 commits it no longer has). The relaxed check passed, so the checkout needed no restore point beyond those commits. Capture saved only `HEAD` history, so it found nothing to save and stored no restore point. The final check before deletion then ran the strict check, which keeps stale remote-tracking commits, and rolled back. The dry run never reaches that final check, so it over-reported. Together these workspaces held about 2.4 GB. **Expected behavior** In restore point mode, commits that exist only in remote-tracking refs are saved, and the workspace expires. **Steps to reproduce** 1. In a nested checkout, fetch a remote branch, so `refs/remotes/origin/<branch>` exists. 2. Delete that branch on the remote and garbage-collect it. 3. With restore points enabled, close the issue and run expiration: the workspace is kept with `unpushed_managed_checkout`. **Paperclip version or commit:** `master` at 412914cfa7. **Deployment mode:** any. **Agent adapter(s) involved:** not adapter-specific (core). ## What Changed - **Strict mode is unchanged.** An earlier revision of this pull request ignored remote-tracking refs. Review pointed to `does not trust a stale remote-tracking ref`, which deliberately protects those commits, so this pull request keeps that behavior and the test is unchanged. - **Relaxed nested check** (`assertReproducibleManagedCheckout(..., { restorePoint: true })`): - reads all reflogs in one bounded call (`--format=%H %gD`), deduplicates them by commit, and splits them into local and remote-tracking entries before applying the tip budget; - local entries still have to be covered or upstream, as before; - remote-tracking tips and remote reflog entries that the remote no longer has are returned as `preserve` (new `tipsMissingUpstream`, sharing `commitsMissingUpstream` with the existing check). - **Capture:** the proof is shared as `provenManagedCheckoutHistory`. `storeGitRestorePoint({ resolvePreserveCommits })` calls it after taking the history digest and before capture, then compares the digest again after capture. A fetch before the digest is included in the preserved commits, and one after it changes the digest and aborts capture. The commits flow into `captureRestorePoint`. They are bundled as history-only commits, within the existing limits (50 preserved tips, 1,000 commits, 256 MB bundle). The checkout therefore gets a restore point, and the final check uses the relaxed check for it. - **Final check:** `restorePointHistoryDigest` now also covers remote-tracking refs and their reflogs, so a fetch, ref change or reflog-only change after capture rolls expiration back. - **Restore:** the commits are imported as objects; the stale remote-tracking refs themselves are not recreated. - **Docs:** `doc/recoverable-disk-cleanup.md`. No code comments were added. ## Verification - `pnpm exec vitest run src/__tests__/git-workspace-expiration.test.ts src/__tests__/git-reproducible-managed-checkouts.test.ts -t "stale remote-tracking|remote-tracking refs change"` (in `server/`): 5 passed. - The existing strict test passes unchanged. - A clean nested checkout whose remote-tracking ref points at a deleted, garbage-collected branch is kept strictly, expires with a restore point, and reopens on its commit with the stale commit present. - A remote-tracking ref created after capture rolls expiration back, and so does a reflog-only change to one. - 6,000 repeated reflog entries for one commit do not hit the tip budget. - **Mutation checks:** each of these fails a case: - dropping the preserved commits reproduces the instance's symptom (`unpushed_managed_checkout`); - dropping remote-tracking refs or their reflogs from the digest; - removing deduplication (`managed_checkout_local_tip_budget`). - `pnpm exec vitest run git-restore-point git-workspace-expiration git-workspace-contents git-workspace-restoration git-reproducible-managed-checkouts execution-workspaces heartbeat-git-restore-point-resume` (in `server/`): 261 passed. - `pnpm run typecheck` (in `server/`): pass. - Not run: server build and boot smoke. ## Risks - **Only with restore points enabled.** Strict expiration is unchanged. - **Retention:** saved stale commits are kept until the restore point expires (90 days after the tree closes by default), then deleted with it. - **Limits:** checkouts with more than 50 stale tips, or bundles over the limits, skip capture and stay kept, as before. - **Spurious rollbacks:** a fetch during the short window between capture and deletion rolls back; the next sweep retries. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file edits, tests, read-only inspection of a live instance). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 08:24:53 -07:00
Devin FoleyandPaperclip	6877760904	feat: save ignored files in git restore points (#17290) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Git cache expiration removes closed issues' worktrees after saving a restore point of anything the remote cannot reproduce > - Restore points left ignored files out, so any worktree with an ignored file that was not a known regenerable directory was kept > - On one instance that was the most common reason a closed worktree was still kept, mostly build caches and agent evidence files > - This pull request saves those ignored files in the restore point too > - The benefit is that more closed worktrees are reclaimed, and nothing an agent left behind is lost ## Linked Issues or Issue Description Refs #17248, #17257. Bug report described in-PR (bug report template). **What happened?** With restore points enabled, one instance kept 15 of 43 inspected closed worktrees with `unique_ignored_files`. Among 19 such worktrees on disk: - 9 held `ui/tsconfig.tsbuildinfo`; - others held build output outside the regenerable directory names (`server/ui-dist/`, `test-results/`); - 2 held materialized skill copies under `.claude/skills/`; - about 5 held small files agents kept out of git: patches, evidence archives, recovery receipts. **Expected behavior** Ignored files that are not regenerable are saved with the restore point and restored on reopen, so the worktree can be removed. **Steps to reproduce** 1. In a closed issue's worktree, leave an ignored file outside `node_modules`, `dist` and the other regenerable directories (for example `ui/tsconfig.tsbuildinfo`). 2. With restore points enabled, run expiration: the worktree is kept with `unique_ignored_files`. **Paperclip version or commit:** `master` at 412914cfa7. **Deployment mode:** any. **Agent adapter(s) involved:** not adapter-specific (core). ## What Changed **Capture** (`server/src/services/git-restore-point.ts`): - `restorePointSavesIgnoredFile` defines which ignored files are saved. It leaves out: - `.env` and `.env.*` files; - `.paperclip-repositories/` (nested checkouts have their own restore points); - `.claude/settings.local.json` (saved with the workspace contents); - anything inside a directory named in `REGENERABLE_WORKSPACE_DIR_NAMES`. - Those files are scanned with the same limits and checks as untracked files, and snapshotted into a separate `ignored` tree, committed on top of `HEAD`. Its ref travels in the bundle. - They are hashed with `hash-object --no-filters` (symlinks by their raw target bytes), so `.gitattributes` conversions such as `text=auto` never change their bytes. Every path given to `--stdin-paths` is C-quoted, so git always reads the literal workspace file, even for names that start with `"`. - An ignored nested repository skips capture, which keeps the worktree. - A worktree whose only extra files are ignored is no longer "nothing to save". **Format:** a payload with an ignored tree uses `paperclip.git-restore-point-payload.v2`, and one without stays `v1`. Decoding requires `ignored` exactly for `v2`, so older servers reject the new payloads instead of restoring without the files. **Restore:** `applyRestorePoint` writes the ignored files directly from their blobs, read in one `cat-file --batch` call, so no filter is applied. - It checks every target before writing, refuses any path that would leave the worktree, and never overwrites an existing file: a conflict fails the restore closed. - If a write fails partway, it removes the files it wrote, so a retry starts clean. The post-apply verification, the before-deletion snapshot check and `restorePointSnapshotMatches` all include the ignored tree. The stored reference (`GitRestorePointRef.ignored`) is validated and checked against the decoded manifest. **Expiration** (`git-workspace-restoration.ts`): in restore point mode, the reproducibility check accepts the ignored files a restore point saves. Strict mode is unchanged. **Docs:** `doc/recoverable-disk-cleanup.md`. No code comments were added. ## Verification - `pnpm exec vitest run src/__tests__/git-restore-point.test.ts` (in `server/`): 27 passed. New cases: - **Round trip:** ignored files round-trip as a `v2` payload with their modes, while `dist/`, `node_modules/` and `.env` stay out. - **Conflict:** an existing conflicting file fails the restore closed and is left as it was. - **Nested repository:** an ignored nested repository skips capture. - **Forged payload:** a `v1` payload claiming an ignored tree is rejected. - **Line endings:** an ignored CRLF file in a `* text=auto` repository restores byte for byte. - **Partial failure:** a restore that fails partway leaves nothing behind, and a retry succeeds. - **Quoted names:** a crafted ignored path that looks like a quoted absolute path saves and restores the workspace file, never the outside file it names. - **Symlinks:** a symlink target with bytes that are not valid UTF-8 restores exactly. - `pnpm exec vitest run src/__tests__/git-restore-point-expiration.test.ts` (in `server/`): new cases. - A closed worktree whose only extra files are an ignored build cache and evidence file is kept without restore points, expires with one, and reopens with those files and without `node_modules`. - An ignored file changed after capture rolls back. - **Mutation checks:** each of these fails at least one case: - not relaxing the reproducibility check; - not capturing ignored files (the end-to-end case then catches the loss); - leaving the ignored tree out of the comparison; - not writing the files on apply; - hashing with filters; - dropping the partial-restore rollback; - passing unquoted paths; - decoding symlink targets as UTF-8. - `pnpm exec vitest run git-restore-point git-workspace-expiration git-workspace-contents git-workspace-restoration execution-workspaces workspace-compaction heartbeat-git-restore-point-resume src/services/heartbeat/environment-selection` (in `server/`): 296 passed. - `pnpm run typecheck` (in `server/`): pass. - Not run: server build and boot smoke. ## Risks - **Only with restore points enabled.** Strict expiration is unchanged. - **Bigger restore points.** Ignored files count toward the existing limits (20,000 files, 64 MB per file, 256 MB total). Larger caches outside the regenerable names make capture skip, which keeps the worktree as before. - **Rollback:** older servers cannot restore `v2` restore points; they fail closed. Disable restore points before downgrading. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file edits, tests, read-only inspection of a live instance). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 08:24:31 -07:00
DottaandPaperclip	3f1ad6b25b	fix(chat): resume task Retry after retained-source containment (#17317) ... ## Thinking Path > - Paperclip manages agent work through task and execution ownership. > - A task Retry must retain the user's request while the system verifies an old executor. > - Connected conversations also retain their external principal and accepted source batch. > - Chat Retry admission required containment before it could save a wait for that recovery. > - A saved chat receipt could also remain deferred after containment became available. > - This pull request lets the exact recovery-bound request wait and resume through its chat owner. > - Normal admission still checks containment, current access and policy before one successor starts. ## Linked Issues or Issue Description **What happened?** An ordinary task Retry on a connected conversation failed with `failed_chat_retry_not_authorized` while its retained remote executor still needed verification. The new recovery generation existed, but the chat action failed before it could save a dispatch wait. This occurred after the repairs in #17267 were confirmed live. The local regression found a second gap. A processed chat action with a deferred wake was not reconsidered after containment. Re-entering normal admission also tried to insert its existing durable receipt again. A historical hold for the same failed run could also hide the current Retry's workspace strategy, so the request still waited after the old executor was contained. Preparation could then read that sibling instead of the recovery record that owned the fresh successor, losing the fresh-workspace decision. **Expected behavior** Save the exact authorized Retry while its matching recovery generation verifies the executor. Once containment is recorded, reconsider the same chat receipt through normal admission. Start one successor with the original source batch. Keep publication, principal, pause and budget checks. **Steps to reproduce** 1. Admit a GitHub or Telegram request and record a failed native run with a retained remote lease. 2. Leave the executor's termination unconfirmed and exhaust its prior recovery generation. 3. Use ordinary task Retry to create a new recovery generation. 4. Process the connected Retry. The old admission code fails instead of retaining a wait. 5. With a deferred receipt, record exact containment and reconcile again. The old dispatch path leaves it stranded. **Paperclip version or commit** Reproduced from `412914cfa78d5abefcc34d3eb8d4bb86e461caac`. The branch is rebased onto current master. **Deployment mode** Hosted tenant for the reported failure. Isolated local PostgreSQL and the assembled heartbeat/chat services for the regression. Refs #17267 and #16965. This change does not add a Cloud dependency or change the absence protocol. ## What Changed - Permit a saved admission wait only for the exact current task Retry and active source recovery generation. - Keep promotion and execution dependent on independent containment proof. - Bind a Retry to active automatic recovery without replacing its generation, attempt count or deadline. - Share the current exact Retry strategy across historical holds for the same run. Retain each hold's stop and effect checks. - Select the canonical recovery record that proves the committed fresh successor during workspace preparation. - Let task receipt reconciliation return the same deferred action to its chat owner. - Recheck current board and external authority, then update the existing wake atomically with its successor. - Preserve the exact chat source batch. Keep generic board-comment filtering unchanged. - Add GitHub and Telegram cases for waiting, containment, one successor, stale generations, source edits, endpoint pause and supersession. - Update recovery documentation and capability maps. ## Verification - Red: real PostgreSQL tests reproduced the admission rejection for GitHub and Telegram. Six active-recovery cases then failed on the generation-ID assumption. Two same-run historical-hold cases remained deferred after containment on the old strategy lookup. A preparation regression then returned no readiness when the historical hold was stored first. - Green: 73 selected chat integration tests and 394 continuation, task Retry, workspace recovery, recovery-state and workspace-preparation tests passed. The chat tests use the real chat admission and heartbeat queue. Unrelated capacity stays occupied so no real provider starts. They assert no executor before containment, one successor afterward, unchanged source leases while waiting, preserved recovery budgets, and denial after authority changes. A separate source hold still blocks work. Both sibling insertion orders pass through the committed-readiness lookup and real workspace preparation: the new directory has independent Git storage and files, and unpublished old work remains intact. - `pnpm -r typecheck`, `node scripts/check-maps.mjs` and `git diff --check` passed after the review fixes. The source build passed on `f6660efa48e37a80cd9857217a537606de85b5c8`. - The local host reached its PostgreSQL semaphore limit. Successful runs used a test-only loader to reduce disposable cluster connection and worker limits. No repository database helper or application setting changed. - The full local `pnpm test:run` began before the final review edits, failed in its first server phase and was stopped after those failures. Later phases did not run. This is not a full local pass. The transient Retry-notice case passes after fixture isolation; the four Discord close-owned publication cases also pass in isolation. Other broad-run failure causes are not established. - Exact-head CI and security checks pass on `f6660efa48e37a80cd9857217a537606de85b5c8`; Greptile is 5/5 with zero unresolved review threads. Optional Paperclip Review completed neutral with a 5/5 static assessment of all 11 changed files and no findings; it did not run tests or hosted acceptance. All 57 current-head checks are terminal: 54 success, two skipped and the optional neutral review, with no failures or pending checks. - The real hosted workflow confirmed that the deployed absence repair recorded containment. A later ordinary UI Retry was accepted and saved a durable request, but it remains behind the historical source hold. The useful existing review remains unchanged. This follow-up is not deployed; a live successful successor and final task disposition remain unverified. No missing files are claimed recovered. ## Risks - The existing task reconciliation sweep can restage a deferred chat action. This reuses its receipt and generation; it does not grant new execution or publication authority. - Dispatch must keep the board Retry identity separate from the connected request's principal. The regression checks these bindings and preserves current chat authorization at each phase. - Historical holds reuse a strategy only from the locked same-run recovery that names the current Retry intent. A different run, malformed recovery record or independent source hold remains fenced. - Unknown executors without a matching active recovery still fail admission. Promotion and execution require containment even when admission saved a wait. - No schema or API shape changes. No credential changes, manual deployment or service restart. ## Model Used OpenAI GPT-6 through Codex, with repository tools, code execution and browser verification. The exact served model variant and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 07:51:34 -07:00
DottaandPaperclip	3bd709df8b	Operationalize Product E2E accounting, tracing, and scheduling (#17009) ... ## Thinking Path > - Paperclip manages agents and the work they perform. > - Product E2E tests exercise real agents, tools, setup and cleanup. > - Repeated runs need retained costs and traces, including failed attempts. > - A passing retry must not hide earlier spending or feedback. > - Recurring work also needs admission limits, private reports and failure detection. > - This change supplies those controls while keeping paid activation behind existing gates. ## Linked Issues or Issue Description Refs #16996 and #17158. This is the consolidated landing of the reviewed eval operations stack: #17019, #17024, #17025, #17027, #17030, #17033, #17035, #17038, #17043, #17044, #17048, #17050, #17051, #17092, #17096, #17100, #17107, #17109, #17112, #17115, #17117, #17121, #17123, #17127, #17130, #17133, #17134, #17137, #17140, #17143, #17154. The repository permits squash merges only. A local rehearsal showed that squashing each slice into master causes repeated conflicts in dependent slices. This PR preserves their commits and lands their combined implementation in one reviewed integration. Superseded slices will close after this merge. ## What Changed - Retain every attempt's cost receipt, including setup, failures, interruptions and retries. Separate reported amounts, estimates, missing costs and reserved capacity in reports and dashboards. - Add bounded request admission, exact model pricecards, gateway error accounting, cache policy and reconciliation evidence. Refuse unsupported routes and requests that cannot fit the allowance. - Integrate Braintrust gateway traces and OTLP correlation, with secret filtering and experiment imports. Record phase timing, including setup, feedback capture and cleanup. - Collect complaint and suggestion evidence after each attempt, including recovery after failures. Retain explicit review dispositions independently from the product grade. - Add gated daily and weekly selections, measured forecasts, setup/readiness checks, wall-time limits and retained monitor notifications. Include declared non-model overhead in admission while reporting the existing subscription separately. - Keep recurring history private, bound uploads and retention, and use isolated archive environments. Keep public publication an explicit operation. Report and publication jobs install only the committed lockfile, with scripts disabled and isolated stores; stale pins fail closed. - Integrate current master, preserve its Product E2E journey billing and privacy checks, and test that shared setup receipts do not double-count estimated agent usage. Preserve upstream fixture repairs. ## Verification - Complete integration baseline `0649a5ce7544ab20825c2183d2f421157db49f24` includes master `a3e1d915a7110a4e115b46396a3f79e6dfb4d05e` and every original stack head as ancestors. - On that integration baseline, `pnpm test:e2e:runner:unit`: 4,012 Vitest tests passed, two skipped, across 193 files; all 137 Node source-contract tests passed. - Focused billing and Product E2E journey contract checks: 24 tests passed. They cover estimates, unknown receipts, single-owner setup costs, browser-only execution and private publication guards. - Monitor and workflow wiring: all 96 Node tests passed. `node scripts/check-token-gates.mjs`: all gates clean. - `pnpm test:e2e:runner:typecheck` passes after the standard Runner TypeScript build refreshed local declarations. The first attempt rejected the new master `runner_prepared_launch` union member because this checkout retained an older generated declaration; the source union already contained that member. - Current head `492b0708a1c531a023d9ac10f0f3dc6383cc1754` removes dependency re-resolution from archive-authorized reporting jobs after the security scan flagged it. All 15 workflow-security tests pass after updating the old two-install assertion. - Hosted [CI run 39965460180](https://github.com/paperclipai/paperclip/actions/runs/39965460180) passes on the current repaired head, including typecheck, build, general tests, Runner checks, browser shards and canary dry run. Greptile is 5/5 on this exact head with no unresolved threads. The security scan passes after the fixed lockfile-resolution thread was resolved. - Paperclip Review returned a terminal neutral check but an incomplete 0-file assessment on the repaired head: its platform blocked both diff responses as prompt-injection content. This is not a successful code review. No bypass or fabricated pass was attempted; current-head Greptile, security and hosted CI are independent gates. - No paid test, reservation release or recurring-run activation accompanies this landing. Live route qualification and invoice reconciliation remain separate evidence. <details> <summary>Retained earlier validation failures</summary> Earlier broad local runs had two 15-second test timeouts (CLI poll cadence and Retry validation), one embedded-database setup timeout, and later PostgreSQL semaphore exhaustion. The exact timed-out tests passed separately without larger timeouts; the broad-run causes were not established. Those local failures remain distinct from subsequent hosted CI results. The independent #17158 fix records its Muse fixture collision, its UUID correction and hosted verification. Local Muse execution was also blocked by macOS embedded-PostgreSQL startup. No historical failure is treated as a successful paid qualification. </details> ## Risks - Admission is a local control, not a provider invoice cap. Missing provider receipts or resource charges retain unknown amounts and holds; estimates do not become billed dollars. - The bounded path admits only documented local suites and provider routes. Ordinary legacy or separate Runner protocol campaigns have different controls. Merging this code does not qualify additional routes. - Recurrence still needs the existing enable gates and current qualification/allowance evidence. Archive permissions and resources must match the documented private-environment contract. - Legacy reports may lack complete attempt or phase evidence. Reports preserve that limitation and do not infer zero usage from missing data. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools and code execution. The exact deployment variant and context-window size are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (targeted checks; full-suite host limitation disclosed above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 07:18:37 -07:00
DottaandPaperclip	4173059081	fix(inbox): keep advisory GitHub review waits quiet (#17318) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Mine shows a person's participation and requests that need their response. > - Automatic GitHub reviews should remain quiet until the assigned work needs human input. > - Review bots used completion requests to keep tasks in review while CI or a PR was still open. > - Those requests correctly entered Mine, despite the automatic-work filter from #16356. > - This pull request gives published reviews an existing provider-event wait in the in-review state and withdraws superseded completion requests. > - Real input requests and separate approval gates keep their current behavior. ## Linked Issues or Issue Description Refs #16356 Refs #13314 An advisory score below 5/5, pending or failed CI, or an unmerged PR does not by itself require a person's approval in Paperclip. The bot can track those conditions without adding a completion card to Mine. ## What Changed - Explain the quiet review lifecycle in the GitHub review instructions. Use a yielded provider-event wait, or an existing saved monitor, after publication. Report done when the configured completion conditions are met. - Keep a confirmed, published review in `in_review` during that wait. Reuse existing authorization and monitor delivery. Add no schema, timer, or polling loop. - Recognize the committed external wait in review attention and recovery. Shared execution receives a verified publication flag; GitHub receipt lookup stays with the GitHub module. Other chat waits keep their current status. - Withdraw an older system-created completion request only when the same bot has published both reviews and its new report has no attention request. Keep explicit cards, separate approval gates, and requests without the prior review evidence. Finish validation stays read-only; withdrawal commits atomically with the new status after checking current governance, the admitted result, and both low-risk contracts. - Update the product rule, capability maps, and regression coverage. ## Verification - Final commit `7c502118eec36028d1b4537bb8797ef893940bf0`: 54 GitHub checks passed, two intentionally skipped, none pending or failed. Greptile is 5/5, the security scan passed, and all review threads are resolved. - Passed: `pnpm -r typecheck` and `pnpm build`. Rechecked server typecheck and build after the final authorization guard. - Passed: 195 tests across `external-chat-wait.integration.test.ts`, `status-arbiter.test.ts`, `review-instructions.test.ts`, and `issue-liveness.test.ts`. The database suite used an isolated PostgreSQL 18 container through `PAPERCLIP_TEST_DATABASE_URL`. - Acceptance coverage: 3/5 stays in review outside Mine; 5/5 with pending completion conditions stays in review; 5/5 with satisfied conditions finishes quietly; a real repository-access request enters Mine until resolved. Also covers obsolete-card withdrawal, transaction rollback, rejected reports, separate approval gates, multiple reviewed heads, saved monitors, and revoked endpoints, repositories, or membership. - Passed: `node scripts/check-maps.mjs`, `node scripts/check-module-boundaries.mjs`, and `git diff --check`. - The initial acceptance run exposed a missing authenticated user in the test fixture. Added the user required by the existing addressee authorization check. All 154 tests in that database suite then passed, including the final rerun. - Review found premature withdrawal during validation, stale repository access, and a missed receipt when a run reviewed multiple heads. The revision fixes all three and includes acceptance regressions. The saved-monitor commit guard now accepts the verified published-review state. - `pnpm test:run` was attempted but could not complete on this Mac. Embedded PostgreSQL failed to allocate semaphores (`semget: No space left on device`); a direct initialization probe confirmed the host resource limit. The isolated database run proves the affected suite, not the full repository suite. - These tests exercise persisted review receipts, finalization, and Mine queries. They do not prove a live model followed the new instructions or submit a live GitHub review. ## Risks - The bot must follow the new lifecycle instructions. A real explicit attention request still enters Mine. - Existing obsolete cards are withdrawn when the same bot publishes a later review and reports no human dependency. This change does not run a bulk cleanup of historical inbox entries. - Review attention now recognizes an external response path. It requires an applied current status decision, a successful bound run, and current conversation, repository, and principal authorization. - No migration or UI change. The full local test suite is unverified because of the host database limit; CI passes on the final commit. ## Model Used OpenAI Codex, GPT-6, with reasoning, repository inspection, code editing, and test execution. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (195 focused tests; full-suite limitation above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 07:14:40 -07:00
DottaandPaperclip	5ef385f7e6	Run reusable Product E2E journeys across deployment targets (#17262) ... ## Thinking Path > - Paperclip manages agents and their work. > - Product E2E tests check complete user journeys. > - The current harness prepares self-hosted instances. > - Hosted workspaces need separate authentication, setup, and cleanup. > - The same in-app story should run on either deployment. > - This pull request adds deployment targets and explicit prerequisites. > - The private Cloud adapter owns fleet access and lifecycle operations. ## Linked Issues or Issue Description **Subsystem affected** Product E2E harness, fixture registry, and reports. **Problem or motivation** Self-hosted test setup cannot qualify a deployed Cloud workspace. Copying each story into a second harness would make its assertions diverge. **Proposed solution** Add explicit deployment targets, setup recipes, and ordered campaigns. Reuse a no-model task-properties story on both deployments. Let fresh Cloud onboarding provide verified context for that same story. Keep Cloud lifecycle and credentials behind a private target provider. **Roadmap alignment** This supports Cloud deployment qualification and the existing agent eval system. It adds no application endpoint or scheduler. Related work: #17112. The separate onboarding gateway UI prerequisite is #17253. ## What Changed - Add explicit target configuration and a credential-free preparation preview. Keep historical selectors and the current `--all` matrix unchanged. - Resolve requested fixture prerequisites, including a run-owned backlog task prepared before measurement. Reuse verified onboarding context without counting recipe setup as onboarding coverage. - Add task assignment persistence and the actual onboarding parent/child first-task journey, with an independent artifact oracle and native trace requirement. - Preserve failed boundaries and blocked dependents, deployment identity, costs, phase outcomes, and private evidence in the existing dashboard. - Validate success assertions and accounting before successful cleanup. Bound collection by the campaign deadline and cleanup/evidence hooks by a separate ten-minute grace, with cancellation and stop-only fallback. - Stage dashboard rendering separately so a timed-out renderer cannot overwrite retained failure results. Add evidence finalization hooks so the private provider can save evidence before target deletion and lease release. - Reject private journey bundles in the public history publisher. Browser-only results have no model or provider. ## Verification - The unchanged task-properties story passed on managed self-hosted and a registered testing Cloud workspace. Authentication, setup, journey, collection, and cleanup passed. Model calls: zero. Earlier attempts remain private and retained. - Final journey and report tests: 96 passed in eight files. Catalog compatibility: 39 passed. The wider focused run exposed three stale master catalog expectations; these now match the already shipped case-handoff v4 metadata and four-cell matrix. The default 385-cell selection is unchanged. - Review regressions reproduced premature success for a zero-request receipt and a missing required assertion. Both now fail before success cleanup while retaining cost receipts. The initial lifecycle fixes passed 33 checks. A subsequent review exposed a late renderer write that could erase saved failures; rendering now writes into isolated staging and publishes HTML only after it settles. Combined campaign, fixture-contract, and retained-report checks: 67 passed, including late writes, delayed timer callbacks, stalled collection, release, stop, evidence hooks, and browser close. Harness typecheck passed after the fixes. - A broader harness unit attempt completed 3,601 passing cases, two failures, and two skips. The failures exposed a stale Pi count and an omitted active Copilot fixture in the Daytona image input closure. The focused compatibility fixes passed 83 tests. After all source fixes, the full harness support suite passed: 160 files, 3,621 tests, two skips, plus 128 native-source checks. The earlier failures remain recorded. - Harness typecheck, repository typecheck, repository build, map checks, and Storybook catalog checks passed on the rebased branch. - The required full `pnpm test:run` attempt timed out after 35 minutes. It had 1,195 completed passing cases and one chat integration failure before its first suite completed. That case and nine adjacent cases pass in isolation. The original failure remains unresolved and retained; this is not a full-suite pass. - Current-head remote checks (`39b2a1ed8c49222fabd1ba8e225cac81bff5c6f7`): 53 passed, two skipped, none pending or failing. Greptile completed successfully at 5/5 with no unresolved review threads. - Fresh paid Cloud onboarding, dependent reuse, and disposable cleanup have not been qualified live. They require deployment of the separate UI prerequisite and actual Cloud gateway budget admission. The trusted Cloud workflow remains disabled. - Final live verification at `39b2a1ed8c`: the unchanged shared story passed on managed self-hosted and the registered testing Cloud stack using the prepared backlog-task recipe. Both cleaned up successfully with zero model calls. Independent private S3 checks confirmed final evidence and released QA identity/stack leases. Cloud evidence remains private. ## Risks - A deployed UI or Cloud control-plane change can invalidate selectors or provider contracts. Version drift fails the attempt. - Cloud evidence and lifecycle recovery are private-provider responsibilities. A pending cleanup or evidence upload must keep the target unavailable for reuse. - No release is deployed or pinned by this change. No automatic story retry is added. The initial pilot retains its explicit $10, one disposable workspace, and 30-minute bounds. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, code execution, and delegated agents. The runtime does not expose a more specific deployment model identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 06:27:49 -07:00
Nicky Leach	412914cfa7	Read native plugin task output files on the execution host (#17285)	2026-10-22 23:38:09 -07:00
Devin FoleyandPaperclip	d69631b01b	feat: reopen a fresh workspace with a warning when its restore point expired (#17269) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - #17248 and #17257 let git cache expiration remove closed issues' worktrees after saving encrypted restore points > - Restore points are kept for 90 days after an issue tree closes, then deleted > - Reopening a workspace whose restore point had expired failed, so the issue could not get a workspace back > - This pull request starts a fresh workspace on the same branch instead, and warns that the saved work is gone > - The benefit is that old issues can always be resumed, with a clear record of what was lost ## Linked Issues or Issue Description Refs #17248, #17257. Bug report described in-PR (bug report template). **What happened?** After `PAPERCLIP_GIT_RESTORE_POINT_RETENTION_DAYS` passes, the retention worker marks the restore point expired and deletes it. Reopening that workspace then fails with the git cache restoration error, every time. **Expected behavior** Reopening succeeds with a fresh workspace on the issue's branch, and the issue says that unsaved work from before the cleanup is no longer available. **Steps to reproduce** 1. Enable restore points and let a closed issue's worktree expire with unsaved work. 2. Let its restore point expire. 3. Reopen the issue: the workspace rebuild fails. **Paperclip version or commit:** `master` at 650581b2f0. **Deployment mode:** any. **Agent adapter(s) involved:** not adapter-specific (core). ## What Changed - **Detection:** `gitRestorePointExpired(recipe)` is true when the main restore point or any nested checkout's restore point has `expiredAt`. - **Reopen** (`execution-workspaces.ts`): - `restoreGitCacheBase(..., { freshStart: true })` prepares the base without the saved commit or the bundle. - The saved recipe is left out of the metadata handed to the runtime, and dropped from the stored metadata only when the reopen succeeds, so a failed attempt can be retried and still warns. The runtime recreates the worktree on its branch: the branch tip if the repository still has it, otherwise a new branch from the base ref. When the expired recipe had nested checkouts, the project's repositories are prepared with `prepareProjectRepositoryWorkspaces` (without the expired saved state) right after the worktree is recreated and before the provision command runs, with the same copy identity and access rechecks as other reopen copies. - The result carries `warning`, the `execution_workspace.reopened` activity records `restorePointExpired` and the warning, and a system comment on the issue posts the warning in the same transaction. - **Retention:** `markGitRestorePointExpired` now also marks nested checkouts' restore points, matched by object key. - **Docs:** `doc/recoverable-disk-cleanup.md` describes the fresh start. No code comments were added. ## Verification - `pnpm exec vitest run src/__tests__/git-restore-point-expiration.test.ts` (in `server/`): 13 passed. New case: an expired restore point reopens fresh with the warning and the system comment, on the original branch, even after the base lost the branch and its unpushed commit; the recipe is gone. A first attempt that fails keeps the expired recipe, and the retry still warns and comments exactly once. - New nested case in `git-workspace-expiration.test.ts`: an expired nested checkout restore point is marked on the checkout entry and reopens fresh with the warning, with the nested repository cloned again at its pushed commit and without its expired uncommitted files. - **Mutation checks:** each of these fails a case: - removing the fresh base path (reopen fails); - removing the system comment; - handing the runtime the old recipe; - dropping the stored recipe before the rebuild succeeds; - marking only the main restore point expired; - skipping nested repository preparation on a fresh start. - `pnpm exec vitest run git-workspace-expiration git-workspace-contents git-restore-point execution-workspaces` plus the environment selection and heartbeat resume suites (in `server/`): 257 passed. - `pnpm run typecheck` (in `server/`): pass. - Not run: server build and boot smoke. ## Risks - **Only with restore points enabled:** recipes get `expiredAt` only from the restore point retention worker, so nothing changes otherwise. - **Lost work is intended:** the fresh workspace has no uncommitted files and, when the repository no longer has the branch, no unpushed commits. The warning says so. - **Recipe dropped:** after a fresh start the workspace no longer carries the expired recipe, so later expirations start from a clean slate. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file edits, tests). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-22 22:20:52 -07:00
Nicky Leach	2cd6eb27c7	fix(runner): record the plugin-prepared working directory (#17243)	2026-10-22 20:42:26 -07:00
DottaandPaperclip	4c57da3708	fix(recovery): continue Retry after verified sandbox absence (#17267) ... ## Thinking Path > - Paperclip manages AI agents and their work. > - Task Retry must reconcile a stopped native execution before starting another one. > - A retained sandbox can disappear while workspace preservation is pending. > - A tenant-key error does not prove absence, especially after credential rotation. > - This pull request accepts only a short-lived Cloud observation bound to the exact source and current preservation owner. > - Normal Retry admission can then continue in a fresh workspace without claiming that missing files were recovered. ## Linked Issues or Issue Description **What happened?** A native run stops with workspace preservation pending. Its Daytona allocation no longer exists. Recovery tries to reopen it, then cannot confirm a stop. The ordinary Retry remains deferred behind cleanup and repeats a transient failure. **Expected behavior** Cloud must independently confirm absence in the recorded provider organization. Core must verify that proof and settle executor containment under the current physical owner. Normal Retry admission must enforce all existing gates and create at most one successor. The old run must remain failed and missing files must remain unavailable. **Steps to reproduce** 1. Persist a failed native run with a retained Daytona lease and an exhausted workspace-recovery generation. 2. Remove the exact provider allocation so reopen and stop cannot confirm it. 3. Request ordinary task Retry. 4. Observe that recovery reports `provider_unavailable` and leaves the Retry deferred. **Paperclip version/commit** Reproduced on source based on `6d44b80a5edbcc9f69bfc3d520994e70aa9fab6d` with the prior preservation and Retry repairs present. **Deployment mode** Cloud tenant with native Runner and a retained Daytona source. The local acceptance uses real Core, Cloud, PostgreSQL, signatures, native Runner and UI, with deterministic model and provider HTTP boundaries. Refs #16962, #16965, #17053. These repairs cover completion proposals, durable Retry and retained-source reopening. They do not establish absence of a missing provider allocation. Related #16809 preserves orphan sandbox leases; this change requires independent proof before settling an absent allocation. ## What Changed - Add an HTTPS client for a purpose-separated, host-bound Cloud sandbox-absence capability. - Verify signature, audience, stack, expiry, source identities and current physical-owner token. - Settle only executor absence with a metadata and owner compare-and-set. Preserve source protection and record that workspace contents are unavailable. - Leave successor creation, approval, pause, budget and uncertain-effect checks to normal Retry admission. - Document the protocol and update the recovery maps. ## Verification - Red: the new persisted-state regression returned `provider_unavailable` and scheduled recovery on the unfixed code. Expected: `source_unavailable` with a contained old execution. - Green: 384 focused tests pass across workspace recovery, signed absence, explicit continuation, ordinary Retry, workspace preparation and provider-owned fresh workspaces. - Signature tests reject forged, expired and mismatched proofs. HTTP failures or missing configuration grant no authority. - Two missing allocations are both reconciled before Retry. A simulated interruption reuses the first durable absence receipt. - Concurrent Retry admission produces one queued fresh successor. It records no successful old result and no recovered-file claim. - Browser acceptance at this change: one ordinary Retry, two provider absence reads for two retained sources, one fresh native execution, useful confirmation and Done after reload. The old run remains failed. Existing saved review and newer local file bytes remain unchanged. The deterministic model does not perform external publication; this is local protocol acceptance, not live provider recovery. - `pnpm typecheck`, `pnpm build`, and `node scripts/check-maps.mjs` pass. - A complete local `pnpm test:run` pass was not obtained. The first run hit a 15-second timeout in an unchanged status-card test. It was stopped to apply review fixes before the remaining phases ran. The unchanged status-card suite then passed 25/25 in isolation with the repository short temporary-path convention. - CI exposed a chat nonduplication assertion that compared unordered database rows. The assertion now sorts both complete row sets by run ID. Its focused test passes. - All 58 current-head checks are terminal on `1e2e881f6b6b1ae1fbfe1f2b2922657ce46e1470`: 55 success, two unaffected Storybook skips, and one neutral advisory review. Required CI, build, E2E and security checks pass. Greptile is 5/5 with no unresolved threads. The optional Paperclip review is neutral advisory 5/5 with nine-file source coverage and no findings; it did not independently run tests. - One unchanged native-runtime test timed out at five seconds on its first CI attempt. Its suite passed 33/33 in isolation; a single retry of that lane passed without code changes. ## Risks - The companion Cloud endpoint must ship before this consumer. Older or unavailable Cloud versions return no proof and retain the current hold. - A valid absence receipt permits a fresh workspace attempt. It cannot restore missing remote files, establish task success or authorize repeating an uncertain external effect. - This PR has not been deployed. The original task remains unrecovered pending rollout and reconciliation of its existing Retry. - No schema, credential, provider-resource or fleet changes are included. ## Model Used OpenAI GPT-6 through Codex, with reasoning, source inspection, code execution and browser tools. The exact backend variant and context-window size are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused scope above; full local run limitation recorded) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-22 22:31:52 -05:00
Devin FoleyandPaperclip	5ef783d971	feat: expire idle open issues' worktrees with restore points (#17271) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - #17248 and #17257 let git cache expiration remove a closed issue tree's worktree after saving an encrypted restore point > - Issues that stay open but go untouched for weeks still keep their worktrees forever, so disks keep filling > - This pull request expires those idle worktrees the same way, and has the next run restore its workspace automatically > - The benefit is bounded disk use for stalled open work, with nothing lost when work resumes ## Linked Issues or Issue Description Refs #17248, #17257. Bug report described in-PR (bug report template). **What happened?** With restore points enabled, expiration only considers worktrees whose whole issue tree has been `done` or `cancelled` for a while. A worktree whose issue stays `todo`, `in_progress`, `blocked` or `in_review` with no activity for weeks is never removed. **Expected behavior** An open issue's worktree that has been idle long enough is saved to a restore point and removed, and the next run on that issue gets it back exactly. **Steps to reproduce** 1. Enable git cache expiration and restore points. 2. Leave an issue open with an isolated worktree and no activity for weeks. 3. The worktree is never removed. **Paperclip version or commit:** `master` at 650581b2f0. **Deployment mode:** any. **Agent adapter(s) involved:** not adapter-specific (core). ## What Changed **Expiration** (`git-workspace-expiration.ts`): - `isGitWorkspaceExpirationEligible(..., idleCutoff)`: in idle mode the closed-tree conditions are replaced by idle ones. No issue in the tree, and no other issue bound to the workspace, was updated after the cutoff, and no relevant run was created, started or finished after it. Runs of bound issues now count as relevant runs in both modes. Every other check stays, including no live run, recovery action, unsettled finalization, environment lease or running service. - `gitWorkspaceLastActivityAt` returns the latest of those timestamps, sharing the same tree and run scope. - `expireReconstructableGitWorkspace` takes `idleCutoff` and passes it to all three eligibility checks. **Reaper** (`execution-workspaces.ts`): - For an open tree's isolated git worktree, when restore points and git cache expiration are on and `idleMs` is set, the reaper computes the idle deadline (last activity + `PAPERCLIP_GIT_RESTORE_POINT_IDLE_DAYS`). - Before the deadline, it records the deadline, so the next sweep wakes then. Open trees cause no recurring checks before that. - At the deadline, it runs expiration with a restore point. A kept workspace is retried after an hour, like terminal ones. - An expiration interrupted after archiving (`completed: false`) is resumed by later sweeps even while the tree is open, with a retry scheduled until cleanup finishes. **Runs** (`heartbeat/environment-selection.ts`): - When a run's issue is bound to a workspace that git cache expiration removed (archived, `workspaceRemoved`), the run reopens it as the agent through `reopenClosedIsolatedExecutionWorkspaceForIssue`, with the restore point applied. - It then clears the reopen fence, since the live run now holds the workspace, and continues with the reopened row. - A failed restore fails the run instead of silently starting a new workspace. - Comments and agent updates on the issue already reopened closed workspaces through the issue routes. **Config:** `PAPERCLIP_GIT_RESTORE_POINT_IDLE_DAYS` (default `7`, `0` turns idle expiry off). Docs in `doc/recoverable-disk-cleanup.md` and `doc/DEVELOPING.md`. No code comments were added. ## Verification - `pnpm exec vitest run src/__tests__/git-restore-point-expiration.test.ts -t "idle open issues"` (in `server/`): 7 passed, with real repositories and embedded Postgres. - **Round trip:** an open issue's worktree, idle for 8 days with unsaved work, is skipped without idle mode, expires with a restore point in idle mode, and reopens exactly. - **Kept:** issue activity 3 days ago, or a run finished 2 days ago, keeps the worktree. - **Interrupted:** an idle expiration interrupted after archiving is finished by the reaper while the issue stays open, and reopens exactly. - **Shared workspace:** a recent update on another issue bound to the workspace keeps it, and moves the last activity time. - **Reaper:** before the deadline the sweep skips the worktree and schedules exactly that deadline; at the deadline it expires it with an active restore point. With idle expiry off, nothing is scheduled or expired. - `pnpm exec vitest run src/services/heartbeat/environment-selection.test.ts` (in `server/`): 14 passed. New cases: a run reopens its expired workspace as the agent, clears the fence with the reopen's generation and binds the reopened row; a failed restore fails the run; a workspace archived for other reasons is left alone. - `pnpm exec vitest run src/__tests__/heartbeat-git-restore-point-resume.test.ts` (in `server/`): 1 passed. Assembled heartbeat journey with no mocked restore: a real worktree with an unpushed commit and uncommitted notes is idle-expired, a queued run executes, the agent runs in the original worktree path with the original workspace id, the commit and notes are back, the workspace is no longer archived with no reopen fence left, and the issue stays bound. Disabling the run-side reopen fails it. - `git-restore-point-expiration-config.test.ts`: the default is 7 and `0` turns it off. - **Mutation checks:** each of these fails a case: - dropping the idle deadline; - dropping the idle cutoff; - dropping the run activity condition; - not failing the run on a failed restore; - not re-reading the reopened workspace; - not resuming a pending expiration on an open tree; - ignoring bound issues' activity. - `pnpm exec vitest run git-restore-point git-workspace-expiration git-workspace-contents execution-workspaces src/services/heartbeat/environment-selection src/services/heartbeat/workspace-preparation` plus `heartbeat-git-restore-point-resume` (in `server/`): 312 passed. - `pnpm run typecheck` (in `server/`): pass. - Not run: server build and boot smoke. ## Risks - **Off unless enabled:** requires `PAPERCLIP_GIT_CACHE_EXPIRATION_ENABLED=true` and `PAPERCLIP_GIT_RESTORE_POINTS_ENABLED=true`. `PAPERCLIP_GIT_RESTORE_POINT_IDLE_DAYS=0` keeps restore points for closed trees only. - **First run after idle expiry is slower:** it waits for the restore, about as long as a reopen. - **Failed restore blocks runs:** a restore that fails closed (for example a missing restore point) fails each run on that issue until an operator fixes or rebinds the workspace. This is deliberate, so work is never silently continued in an empty workspace. - **Kept idle workspaces are rechecked hourly**, matching the existing behavior for closed trees that cannot be expired. - **Retention:** restore points of open trees are not deleted while the tree stays open, as before. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file edits, tests). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-22 20:02:21 -07:00
Devin FoleyandPaperclip	c5ca29dbef	fix(claude): prepare compatible sandbox runtimes during onboarding (#17259) ... ## Thinking Path > - Paperclip connects agent accounts and prepares their execution environments. > - A managed sandbox image can outlive the model defaults shipped by the app. > - Claude setup skipped installation whenever a CLI was already on PATH. > - The selected model then failed its minimum-version check after sign-in succeeded. > - This pull request prepares a compatible private CLI before testing or running it. > - Cloud onboarding also stops asking customers to run inaccessible shell commands. ## Linked Issues or Issue Description **What happened?** A new managed Cloud stack completed Claude subscription sign-in. Its environment test then failed because the snapshot contained Claude Code 2.1.263, while the selected `claude-opus-5-5` model requires 2.1.280. The screen displayed Manual debug commands that a hosted customer cannot run. **Expected behavior** Paperclip prepares its managed runtime before testing the selected account. A stale image CLI does not require the customer to upgrade a server package. Failed preparation still blocks hiring. Cloud does not display manual shell debugging. **Steps to reproduce** 1. Start a managed sandbox with Claude Code 2.1.263 on PATH. 2. Connect a Claude subscription and select `claude-opus-5-5` on the CLI lane. 3. Connect the agent. Installation is skipped because the command exists, then the version gate blocks the hello test. **Paperclip version or commit** Source baseline: `6204239cfc8c3be79fc185cf3f468ebaa0ab1e3e`. **Deployment mode** Authenticated managed Cloud with a Daytona named snapshot. The correction also applies to the default CLI in other managed sandbox transports. Related work: Refs #14993 sets model/runtime requirements; Refs #15993 changes native defaults on qualified targets. Neither prepares an outdated default sandbox CLI on this path. Searches found no open duplicate. ## What Changed - Check the actual default CLI against the selected model's existing minimum version. - When a sandbox's default command is outdated, install the exact required npm package into a private versioned workspace prefix. Reuse a compatible private installation. Keep the image's global CLI intact. - Verify the private executable and use that same command for the account hello test and normal CLI runs, with matching command metadata. Probe effort support after preparing that executable so the chosen effort is retained. Keep the model, account, engine and permissions unchanged. - Return safe preparation checks to environment tests and normal run logs, distinguishing installation failure causes without returning raw installer output or credential-bearing URLs. - Exclude provider and Paperclip credentials from the installer environment. Preserve the existing failing version gate if installation or verification cannot succeed. Local, SSH and explicit custom commands remain operator-managed. - Hide Manual debug and legacy operator check hints on Cloud and while deployment metadata is unknown; preserve check messages and details. Retain self-hosted help and the detailed failed checks. Cloud copy asks for a retry instead of asking the customer to repair the environment. - Add subprocess-backed runtime controls, an account-probe regression and mounted Cloud/self-hosted wizard controls. Update development guidance and the affected capability maps. ## Verification - Passed all Claude source suites and the server Claude execution suite on head `c941ae44c22d8dafec0df033a129e39ce4ebd966`: **412 passed**, one existing opt-in test skipped. These include the new command/effort ordering, safe installer diagnostics, environment forwarding and run-log regressions. - On final head `0ecae0e9304f495a57524158b3700b8e9bc21cb8`, passed **208 tests** across all 174 mounted onboarding wizard tests and 34 Claude execution tests. The failure fixture includes the real command-bearing hint, and execution asserts both metadata command fields agree. - Passed `pnpm -r typecheck`, `pnpm build`, `pnpm check:token-gates`, `node scripts/check-maps.mjs` and `git diff --check`. Full typecheck and build were repeated successfully on final head `0ecae0e9304f495a57524158b3700b8e9bc21cb8`. [Latest-head CI](https://github.com/paperclipai/paperclip/actions/runs/39879208694) is green: 54 checks passed and two skipped, including browser E2E and serialized database suites. - The new runtime controls execute real subprocesses with a controlled installer. They cover an old CLI, an already compatible CLI, failed or ineffective installation, local/custom commands, and an unknown version. The probe regression asserts that the hello call uses the private executable. - A separate non-root local subprocess check installed the real npm `@anthropic-ai/claude-code@2.1.280` package. Its private command reported 2.1.280, the simulated image CLI remained 2.1.263, and a second preparation reused the private copy. This ran on macOS ARM64. It did not use a provider account, send a model prompt, or prove live Daytona behavior. - Attempted `pnpm test:run`. Embedded PostgreSQL initialization failed after five attempts on this Mac with `Postgres init script exited with code null`. The unchanged native-session database cases then failed. Stopped only that full-suite process tree after recording the repeated startup failure. No full local-suite pass is claimed; Linux CI must cover those tests. - The first new subprocess fixture had a quoting syntax error. It was corrected before the passing focused runs. The one-off real-package script first required an ESM extension correction, then passed. Logs retain both attempts. A new execution assertion initially inspected the final sandbox cleanup shell instead of the launched command; it now checks execution metadata. A broader directory selection also picked up generated `dist` tests whose Markdown fixtures are not copied by the build (two ENOENT failures). The explicit source-suite run above passes; no generated-test pass is claimed. - Apex review: **5/5**, reviewed final head `0ecae0e9304f495a57524158b3700b8e9bc21cb8`, all five earlier findings addressed and resolved, no new findings. - Merge, deployment and a fresh live Cloud subscription test are still required. No customer account or production runtime was changed by this branch. ## Risks - An outdated sandbox needs npm registry access and up to 120 seconds for one bounded installation attempt. A failed or unverified install continues to block execution. - The versioned package uses sandbox workspace storage. Compatible cached copies avoid repeat installation within that workspace. Existing runtime cleanup and sandbox lifecycle still apply. - The compatible version comes from the adapter's existing model-version contract. This does not add a model fallback or change the selected execution engine. - No schema, credential, permission, workflow or lockfile change. ## Model Used OpenAI Codex, based on GPT-6, assisted with implementation and verification. The exact serving model ID and context-window limit are not exposed in this session. Capabilities used: code editing, shell execution, API inspection and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-22 18:44:53 -07:00
Devin FoleyandPaperclip	650581b2f0	feat: give nested managed checkouts their own restore points (#17257) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - #17248 lets git cache expiration remove a closed issue's worktree with unsaved work after saving an encrypted restore point > - Workspaces with nested `.paperclip-repositories` checkouts were still kept whenever a checkout had unpushed or uncommitted work > - On one instance that was the second most common reason expiration skipped a workspace > - This pull request gives each nested checkout its own restore point and restores it when the workspace comes back > - The benefit is that multi-repository workspaces can be cleaned up as safely as single-repository ones ## Linked Issues or Issue Description Refs #17238, #17241, #17248. Bug report described in-PR (bug report template). **What happened?** With restore points enabled, a workspace whose nested managed checkout has unpushed commits or uncommitted files is still skipped with `unpushed_managed_checkout` or `dirty_or_untracked_files`. On one instance, `unpushed_managed_checkout` accounted for 32 of the last 111 expiration checks. **Expected behavior** A nested checkout's unsaved work is saved like the main worktree's, and the workspace can expire and later be restored exactly. **Steps to reproduce** 1. Use a project with an extra repository, so the task worktree contains `.paperclip-repositories/<name>`. 2. Commit in that checkout without pushing, then close the issue. 3. With restore points enabled, the workspace is never expired. **Paperclip version or commit:** `master` at 6d44b80a5e. **Deployment mode:** any. **Agent adapter(s) involved:** not adapter-specific (core). ## What Changed **Relaxed checkout check** (`assertReproducibleManagedCheckout(..., { restorePoint: true })`, used only in restore point mode): - **Allowed:** the checked-out branch's unpushed commits, plus its uncommitted, untracked and history-only work. - **Still blocks:** - another local branch or tag whose tip the remote lacks and `HEAD` does not contain (`unpushed_managed_checkout_branches`), because restore points do not recreate other branches; - commits in any reflog that are neither on the remote nor in the bundled history (`HEAD`'s reflog and `ORIG_HEAD`), such as work left in another branch's reflog; - objects reachable from nothing; - a `COMMIT_EDITMSG` that matches no commit in the checkout's history. - **`COMMIT_EDITMSG`:** a nested checkout is a full clone, so its base git storage check now accepts a `COMMIT_EDITMSG` that matches a commit in its history, in strict and relaxed mode. It is read only if it is a regular file of at most 64 KB, opened without following symlinks and without blocking; anything else keeps the workspace (`unsupported_commit_message`). The main worktree's existing `COMMIT_EDITMSG` read uses the same bounded reader. A clean checkout with only an unpushed commit therefore reports `unpushed_managed_checkout`, which restore point mode can relax, instead of `unknown_base_git_files`. **Capture** (`git-workspace-expiration.ts`): - **Expiration flow:** strict inspection still runs first. `unpushed_managed_checkout` is now relaxable. The relaxed prover records which checkouts need restore points. - **Storage:** each checkout's restore point is stored under `.../<generation>/checkouts/<name>/<id>.pcrp`, against a remote-verified boundary from that checkout. It is recorded on its `contents.checkouts[]` entry. - **Activation:** all restore points of the generation are activated together in the archive transaction. Any nested restore point makes the recipe `paperclip.git-cache-restoration.v2`, so older servers reject it. - **Schema:** migration `0374` adds `checkout_name` to `execution_workspace_restore_points` and includes it in the live-generation unique index. - **Final check:** under the expiration locks, each checkout with a restore point gets the relaxed check and must match its own snapshot. Checkouts without one get the strict check and must still be clean. Every restore point, the main worktree's included, records a digest of its `HEAD` reflog and `ORIG_HEAD` at capture, and that history must be unchanged, so a commit followed by a reset after capture rolls back. Rollback drops every restore point reference, returns the recipe to v1, and supersedes the stored restore points. **Restore** (`git-workspace-contents.ts`): - **Re-cloned checkouts:** re-cloning a checkout imports its bundle before the saved commit is needed. After checking out the saved commit, it applies and verifies the restore point. - **Existing checkouts:** the same helper as the main worktree marks a matching checkout applied, and fails closed instead of overwriting a diverged one. **Validation:** `readGitCacheRestoration` and `savedGitWorkspaceRestoration` validate nested restore point references against their checkout names. Contents comparison ignores the reference. **Docs:** `doc/recoverable-disk-cleanup.md` describes nested checkouts. No code comments were added. ## Verification - `pnpm exec vitest run src/__tests__/git-workspace-expiration.test.ts -t "restore points for nested checkouts"` (in `server/`): 10 passed, with a real nested checkout and embedded Postgres. The cases: - **Round trip:** an unpushed commit and an untracked file in the nested checkout survive expiration and reopen. The recipe is v2, the restore point lives on the checkout entry, and a ledger row carries the checkout name. - **Another branch:** unpushed work on another local branch keeps the workspace. - **Change after capture:** a nested change after capture rolls back, keeps the files, returns the recipe to v1, and supersedes the restore point. - **Only an unpushed commit:** a clean nested checkout with only an unpushed commit (and its `COMMIT_EDITMSG`) expires and restores. - **Reflog-only history:** unpushed history held only in another branch's reflog keeps the workspace. - **History after capture:** a commit followed by a reset after capture rolls back, in a nested checkout and in the main worktree. - **Unsafe `COMMIT_EDITMSG`:** a named pipe, an oversized file, or a symlink keeps the workspace without reading it unsafely. - **Mutation checks:** each of these fails a case: - removing the other-branch check, the per-checkout snapshot check, the rollback cleanup, or the checkout apply; - rejecting a saved `COMMIT_EDITMSG`; - dropping reflog coverage; - dropping the history digest check; - removing the size cap. Not covered by a dedicated case: a workspace with two nested checkouts where only one has a restore point. - `pnpm exec vitest run git-workspace-expiration git-workspace-contents git-restore-point execution-workspaces` (in `server/`): 232 passed. - `pnpm run check:migrations` (in `packages/db`): pass. - `pnpm run typecheck` (in `server/`): pass. - The server builds, the runtime bundle builds, and `scripts/server-boot-smoke.sh node server/dist-runtime/index.mjs` reaches healthy. ## Risks - **Off by default:** nothing changes unless `PAPERCLIP_GIT_RESTORE_POINTS_ENABLED=true`. - **Other local branches:** a nested checkout's other local branches are not saved. Any of them with commits the remote lacks keeps the workspace instead. - **Migration:** adds a defaulted column and replaces one partial unique index on a table only written when restore points are enabled. - **Rollback:** disable the setting. Workspaces already expired with nested restore points keep v2 recipes that require this code to restore. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file edits, tests, local server builds and boots, read-only measurements on live instances). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-22 18:42:22 -07:00
Nicky LeachandPaperclip	989ee668e0	Support provider-owned fresh retry workspaces (#17252) ... ## Thinking Path > - Paperclip coordinates AI agents and retains their work across failures. > - A retry can require a new workspace while preserving the failed attempt. > - Core rejects this recovery strategy for provider-owned storage. > - Providers should implement the required outcome using their own storage design. > - This pull request adds a semantic fresh-preparation contract and delegates recovery preparation. > - The benefit is that a supported provider can run a fresh retry without losing prior work. ## Linked Issues or Issue Description **What happened?** A retry that selects `fresh_isolated_workspace` fails with `fresh_workspace_storage_unsupported` for provider-owned storage. **Expected behavior** Core asks a capable provider to prepare new storage and attach verified recovery files. The failed workspace remains intact. **Steps to reproduce** Run a task in a task-capable plugin environment. Let the task fail without a verified workspace preservation receipt. Retry the task so Core selects a fresh attempt. **Paperclip version or commit** 6204239cf (rebased on current master). **Deployment mode** Self-hosted server with a plugin task environment and managed workspace provider. Search found no duplicate implementation. This branch includes the default-provider changes from #17247 and preserves their source selection behavior. ## What Changed - Define an opt-in fresh-preparation contract with a stable attempt identity and verified recovery file descriptors. - Describe retained-storage and recovery-attachment guarantees without requiring a Git storage layout. - Reserve replacement task bindings with revision checks and keep the failed workspace intact. Projectless and project-backed task environments both replay their stable attempt reservation. - Reconcile an interrupted preparation against its reserved workspace and stable operation identity. - Stage verified recovery archives at stable transfer paths. Delegate attachment materialization to the provider. - Preserve projectless attempt directory identities through later turns and pin the intent into each execution lease. - Protect fresh intent and projectless ownership from caller metadata replacement or injection. ## Verification - SDK workspace-provider tests: 4 passed. - Targeted reservation and task-dispatch tests: 31 passed. - Recovery transfer staging and lifecycle focus: 7 passed. The separate cache-cleanup failure reconciliation case also passed. - After rebasing on current master, 7 focused preparation and ownership cases passed. These cover inherited provider defaults, fresh preparation, stable projectless identity, and authorization denial. - The focused metadata authority regression passed, including forged intent, replacement, and null metadata patches. - Three assembled interrupted task-reservation cases passed: projectless, project with a saved source, and project with a newly selected default source. - Targeted tests use worktree SDK sources. No full local build or typecheck ran. - Local checks remain targeted as requested. Latest-head GitHub CI passed: 54 successful checks, including typecheck, build, server suites, Runner validation, and E2E shards; 2 unrelated Storybook checks were skipped. - Greptile reviewed commit 73595b0c61 at 5/5 with zero unresolved threads. - Live VM E2E has not run. ## Risks - Existing providers retain their existing behavior. Fresh attempts require explicit version support. - The companion provider implementation must materialize recovery snapshots safely and preserve the old workspace. - Fresh retries require the companion provider implementation. Providers without the version marker fail closed. ## Model Used - OpenAI GPT-6.1 (`gpt-6.1-sol`), reasoning and coding tools. The runtime does not expose the context-window limit. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-22 17:46:27 -07:00
Devin FoleyandPaperclip	35bd7f6059	Reduce encrypted event retries and GitHub run-query transfer (#17251) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its Cloud connector delivers encrypted GitHub events to an instance. > - One failed inner event currently rejects the whole authenticated batch. > - Healthy events then remain unacknowledged and are downloaded again. > - GitHub response and check reconciliation also reads full run records when it needs only ownership and status. > - This pull request isolates inner failures and narrows those run projections. > - Valid events can finish while failed events remain available for recovery. ## Linked Issues or Issue Description **What happened?** An inner App event encrypted with an unavailable key causes `leaseEvents` to reject the complete batch. No sibling can finish or acknowledge. Repeated retries download the same encrypted payloads. Separately, GitHub response and check reads load prompts, runtime profiles, results, and log excerpts that their callers do not use. **Expected behavior** Authenticate the outer batch and lease before handling individual events. Process valid siblings independently. Retain failed events without acknowledging them. Read only the current run fields needed for response ownership and status checks. **Steps to reproduce** 1. Seal one event for an old instance key and another for the current key. 2. Put both events in a correctly authenticated outer lease. 3. Observe that the old client rejects the whole lease. 4. With this change, the valid event processes, the failed ID is reported locally, and only completed events acknowledge. No public issue exists. The change does not add a provider integration or duplicate a roadmap feature. ## What Changed - Keep outer authentication, environment checks, and lease binding strict. - Isolate inner decryption and route-binding failures. Return their authenticated IDs only to the local poller and count them as failures. - Leave failed events unacknowledged in the broker. Continue processing and acknowledging healthy and duplicate siblings. - Read only run identity, status, and required request context for GitHub responses. Read only run ID and status for check reconciliation. - Cover stale keys, swapped routes, invalid outer bindings, all-failed batches, large unused run payloads, and cross-company/task refusals. ## Verification - Connector cryptography suite: 47 tests passed. - Final connector suite: 50 tests passed, including cryptography and three real PostgreSQL connection-event tests. These cover acknowledgment isolation, all-failed batches, and the exact safe warning fields. - GitHub workflow regression suite after the master refresh: 307 tests passed, including the large-payload/company/task test (1,084 unrelated tests skipped by the selection). - Server typecheck passed after the final source changes. - `pnpm -r typecheck` and `pnpm build` passed after the latest conflict-free master refresh. - The broad local `pnpm test:run` was stopped when the branch was refreshed; it has no complete passing result. All CI gates passed on the final commit, including general and serialized server suites, runner verification, build, typecheck, and end-to-end tests. Apex is 5/5 with all review comments resolved. The last change removes source comments and expands docs; both edited TypeScript files emit identical JavaScript before and after that change. - A read-only sample compared full run records with the new field sets: response data decreased by about 96%, and check data by over 99%. This is serialized query output, not a claim of realized billing savings or CPU reduction. - An initial database test setup failed because the local dependency install omitted native-library symlinks. The package's standard setup script repaired them. The unchanged test timeout then passed. - Added lines checked for credentials, customer data, private links, and internal issue references. ## Risks Failed inner events remain pending and retry under the broker's existing lease and retention policy. This change does not recover a lost original private key or introduce a dead-letter deletion path. Outer authentication failures still reject the complete batch. Run projections must retain request provenance and task binding fields. Existing predicates, ordering, identity validation, and publication authority remain unchanged. The change has no schema migration and adds no background timer or new provider call. ## Model Used OpenAI GPT-6 through Codex, with reasoning and shell/GitHub CLI tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used with version and capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs - [x] I have described the issue in-PR following the bug report template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket ID - [x] Focused local regressions and all CI gates pass - [x] I have added and updated tests - [x] I have updated relevant documentation - [x] I have documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-22 17:40:29 -07:00
Devin FoleyandPaperclip	6d44b80a5e	feat: expire closed git worktrees with unsaved work behind restore points (#17248) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Git cache expiration removes a closed issue's worktree only when the remote can reproduce it exactly > - Most closed worktrees hold unpushed commits or uncommitted files, so they are kept forever and disks fill up > - #17238 added restore points (an encrypted save of exactly what the remote cannot reproduce) and #17241 added their lifecycle > - This pull request lets expiration remove those worktrees after saving a restore point, and restores them exactly when the work resumes > - The benefit is bounded disk use for closed work without losing anything a user could need, behind a setting that is off by default ## Linked Issues or Issue Description Refs #17238, #17241. Bug report described in-PR (bug report template). **What happened?** With `PAPERCLIP_GIT_CACHE_EXPIRATION_ENABLED=true` and dry run off, one instance's last 111 expiration checks removed nothing: | Skip reason | Checks | |---|---| | `dirty_or_untracked_files` | 39 | | `unpushed_managed_checkout` | 32 | | `live_or_unresolved_consumer` | 21 | | `unique_ignored_files` | 9 | | commit not on the remote | 10 | Without expiration, the terminal workspace reaper archives only merged, clean worktrees. Closed issues' worktrees therefore accumulate without bound. **Expected behavior** A closed worktree whose only obstacle is unsaved work can be removed after that work is saved, and restored exactly if the issue is reopened or work resumes. **Steps to reproduce** 1. Close an issue whose worktree has an unpushed commit or an uncommitted file. 2. Enable git cache expiration with dry run off. 3. The worktree is never removed. **Paperclip version or commit:** `master` at 12bb7e1976. **Deployment mode:** any. **Agent adapter(s) involved:** not adapter-specific (core). ## What Changed **Expiration** (`server/src/services/git-workspace-expiration.ts`): - **When it applies:** with `PAPERCLIP_GIT_RESTORE_POINTS_ENABLED=true` (default `false`), a worktree that fails the reproducibility check for a relaxable reason is reconsidered once its issue tree has been closed for `PAPERCLIP_GIT_RESTORE_POINT_TERMINAL_DELAY_DAYS` (default `3`). The relaxable reasons are uncommitted or untracked files, a commit not on the remote, reflog or `ORIG_HEAD` history, and ignored `.env` files. A leftover `COMMIT_EDITMSG` still blocks expiration unless it matches a commit in the worktree's reflog, because restore points do not save draft messages. - **What still blocks:** a relaxed inspection (`inspectGitCacheRecipeForRestorePoint`) keeps every other check. Nested managed checkouts, submodules, special git state, local config and other unique ignored files still prevent expiration. - **Capture:** inside the existing source and lifecycle locks, a restore point is captured with the #17238 library against a remote-verified boundary (the `HEAD` itself, or its merge base with `origin/HEAD`), then encrypted, uploaded, read back and hash-checked (`git-restore-point-storage.ts`). - **Commit before touching disk:** it is activated in the #17241 table in the same transaction that archives the workspace with a `paperclip.git-cache-restoration.v2` recipe referencing it, still before anything on disk is touched. - **Final check:** under the file locks, the worktree must still match the captured snapshot (`HEAD`, index, working tree, modes). Otherwise the existing rollback runs. The rollback now also drops the restore point reference and returns the recipe to v1, so a later restore can never apply a stale restore point over the live worktree; the stored restore point is marked superseded. - **Failure paths:** the `pending` row is inserted in a transaction that first registers a `git-restore-points` notification. If the upload or read-back fails, the row is marked `superseded` so retention deletes any uploaded object. Nothing is deleted from disk. - **Encryption key:** restore points are refused (`restore_point_key_not_durable`) on `s3` storage unless the secrets master key comes from `PAPERCLIP_SECRETS_MASTER_KEY`. - **Dry run** reports `restorePoint: true` candidates without storing anything. **Restore** (`git-workspace-restoration.ts`, `workspace-runtime.ts`, `execution-workspaces.ts`): - **Recipe versions:** `readGitCacheRestoration` and `savedGitWorkspaceRestoration` accept v1 without a restore point, or v2 with a valid one. Older servers reject v2 instead of restoring the commit without the saved work. - **Before the saved commit is needed:** reopening (`restoreGitCacheBase`) and the runtime path import the bundle when the saved commit is missing locally. - **After the worktree is re-added:** the runtime always imports the bundle, so history-only commits are present again as objects even when the base was rebuilt (they are not recreated as branches), then applies the restore point, verifies the result, and records `appliedAt`. Reopen persists that through a callback, as it already does for saved contents. - **Never overwriting:** a worktree that already matches is only marked applied. A worktree that diverged from the saved commit's clean state is never overwritten; the restore fails closed and can be retried. Missing, corrupt or expired restore points also fail closed. **Retention:** the #17241 worker's expiry hook now marks the recipe's restore point expired before its object is deleted. **Config and docs:** - `server/src/config.ts` adds the two settings; `doc/recoverable-disk-cleanup.md` and `doc/DEVELOPING.md` describe them. - No code comments were added. **Not in this PR (follow-ups):** - workspaces with nested `.paperclip-repositories` checkouts; - removing idle worktrees of issues that are still open; - starting a fresh workspace with a warning when a restore point has expired. ## Verification - `pnpm exec vitest run src/__tests__/git-restore-point-expiration.test.ts` (in `server/`): 12 passed, with real repositories and embedded Postgres. The cases: - **Round trip:** an unpushed commit, staged and unstaged edits of one file, a deletion, and an untracked file are restored exactly on reopen, `.env` is dropped, and the restore point is active. Restoring through the runtime path works the same way, and a later run never re-applies over newer work. - **Kept:** a worktree that is not yet due, one with the feature disabled (today's skip reason), and one whose encryption key is not durable. - **Change after capture:** rolls back, keeps the worktree, returns the recipe to v1 and supersedes the restore point. - **Missing restore point:** fails closed, keeps the row archived and unapplied, and fails closed again on retry. - **Dry run:** stores nothing. - **Reproducible worktrees:** expire as before, with a v1 recipe. - **History-only commits:** reopen imports them even after they are pruned from the base. - **Draft messages:** a draft commit message keeps the worktree. - **Upload failure:** a failed upload marks the pending restore point for cleanup and keeps the worktree. - **Mutation checks:** each of these fails at least one case: - removing the pre-deletion snapshot check; - keeping the restore point reference on rollback; - removing the runtime apply; - removing the re-apply guard (the divergence check then fails closed instead of overwriting); - skipping the import for a matching worktree, relaxing the draft message guard, or not marking a failed upload's row. - `pnpm exec vitest run src/__tests__/git-restore-point-expiration-config.test.ts`: 5 passed. - `pnpm exec vitest run git-workspace-expiration git-workspace-contents git-restore-point workspace-runtime execution-workspaces` (in `server/`): 529 passed on the first revision. Two `workspace-runtime.test.ts` cases about remote base refs fail, and they also fail on unmodified `master` on this machine. After the review fixes, the expiration, contents, restore point and execution workspace suites passed again: 222 tests. - `pnpm run typecheck` (in `server/`): pass. - The server builds, the runtime bundle builds, and `scripts/server-boot-smoke.sh node server/dist-runtime/index.mjs` reaches healthy. ## Risks - **Off by default:** nothing changes unless both `PAPERCLIP_GIT_CACHE_EXPIRATION_ENABLED=true` and `PAPERCLIP_GIT_RESTORE_POINTS_ENABLED=true` are set. Roll out with `PAPERCLIP_GIT_CACHE_EXPIRATION_DRY_RUN=true` first to see which worktrees would receive restore points. - **Data at rest:** restore points contain source and uncommitted files, encrypted with keys wrapped by the instance secrets key. Losing that key makes them unrecoverable, which is why `s3` storage requires an environment-provided key. Untracked `.env` files are not stored. - **Interrupted apply:** a crash in the middle of applying a restore point leaves a worktree that neither matches the restore point nor is clean. The restore then fails closed rather than overwriting it, and an operator must reset that worktree to retry. - **Rollback:** disable the setting. Already-expired worktrees keep their v2 recipes and restore points, so reverting the code would leave those recipes unreadable until restored by a server with this change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file edits, tests, local server builds and boots, read-only measurements on live instances). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-22 16:50:15 -07:00
Dotta	46760cc588	fix(chat): honor dismissal for all composer requests (#17218) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task conversations show requests in the feed and above the message composer. > - Earlier fixes kept dismissed questions in history, but excluded completion reviews and other request kinds. > - Closing those forms left a pending badge, and the server could still demand a response. > - This pull request saves human dismissal and removes the request from composer reminders and ordinary completion gates. > - People can continue the conversation and reopen the request from history. ## Linked Issues or Issue Description Related work: #15229 and #15392. **What happened?** Dismissing a completion review closed its form but left it counted as pending. It could return after reload, block a retry, or cause the agent to demand another response. Raw runtime requests could also lose their only usable control when dismissed. **Expected behavior** Dismissal leaves an accessible request in the feed and clears the composer form and pending count. It persists after reload. It does not submit an answer, approve an action, or wake the agent. **Steps to reproduce** 1. Open a task with a pending completion review or another composer request. 2. Dismiss the request, then reload the task. 3. Check that the composer stays clear and the feed entry can reopen the request. ## What Changed - Add nullable dismissal fields and a human-only dismissal endpoint with company, private-task access, resolver, and locked review-policy checks. - Persist saved-request dismissal on the server. Keep browser persistence for raw runtime requests and existing question dismissals. - Keep confirmations, choices, item verdicts, suggested tasks, and runtime requests accessible in the feed. - Exclude dismissed ordinary requests from completion and retry gates. Tell agents not to repeat deferred requests. - Preserve tool, secret, connection, plan, company-policy, and configured review authorization gates. Expire deferred non-question requests when their task completes. - Preserve the selected form when another dismissal finishes saving. Keep reopened reviews visible through delayed comment refreshes. - Add a completion-review Storybook test drive and update design, API, database, and capability documentation. ## Verification - `pnpm -r typecheck`: passed. The final UI changes also passed the UI typecheck. - `pnpm build`: passed. The final UI changes also passed the UI production build. - Focused UI and transcript tests: 550 passed across nine files after merging the current base branch. - Focused database, completion, and prompt tests: 105 passed across three files, including two review-policy regressions that failed before the fix. The tests reject a review requester excluded by current policy, preserve the pending gate on denial, and permit an independent reviewer to defer the request. - Route regression suite: 81 passed, including denial before reading or mutating a private-task request. - `pnpm check:token-gates`: passed. - Map and Storybook structural checks: passed; 30 map/catalog script tests passed. - Browser: used the production task-thread components in the completion-review fixture. Dismissed, reloaded, reopened from the feed, and dismissed again. No pending badge or composer takeover remained. Story callbacks simulate API saves; separate server tests cover persistence and permissions. - `pnpm test:run`: stopped after the general-server group reported one failure in `workspace-runtime.test.ts`: "returns a bounded structured conflict and exposes only same-company workspace references". The same test passed in isolation (1 passed, 178 skipped). The cause of the full-suite failure is not established. The full PR CI suite subsequently passed on e243cbb001; its two authorization review findings are fixed in 9e1b4478b3. Fresh [full CI](https://github.com/paperclipai/paperclip/actions/runs/39871944846) passes on 9e1b4478b3, including the previously failing server and browser shards. All 54 executed checks passed; two optional Storybook deployment/visual jobs were skipped. The legacy security status also passed. Fresh Greptile review is 5/5, the security scan passes, and no review threads remain open. This full CI run settles the merge gate; the earlier local workspace-suite failure remains recorded above because its cause was not established. - Additional verification: the original failing browser journey passed locally after the delayed-refresh fix. The expanded page-reload case also passes on 9e1b4478b3 (52.7s): dismiss, reload, reopen, approve, and receive the actual tool failure in the continuing task. It exposed a fixture reading an accepted request before tool execution finished; the fixture now waits for a stored execution result. The first reload attempt stopped on a still-loading page at the default five-second assertion; the reload now uses the suite’s navigation allowance. A controlled server-restart run timed out waiting for server health before the UI assertions. Local route-suite runs timed out during concurrent module loading; sequential imports passed all 81 tests, including the privacy regression. - Initial verification failures: a missing SDK build and a story fixture type error were fixed. PostgreSQL bootstrap then failed because the host had exhausted its semaphore allowance. After explicit operator approval, 64 verified abandoned PostgreSQL semaphore sets were reclaimed; no live process was stopped. The database regressions then passed. The raw-runtime regression exposed a missing feed control; the implementation now preserves that entry and the regression passes. ## Risks - Apply the additive database migration before the new server code. It adds two nullable columns and does not change existing decisions. - A dismissal is shared task attention state. Only a human with task read access and current resolver/review authority can persist it. Review policy is checked under the task lock. The operation is idempotent and records the dismissing user. - Raw runtime dismissal remains specific to a browser. Saved interaction dismissal persists across browsers. - A deferred review is not an approval. Tests retain protected-action and configured review gates. - The browser walkthrough uses a fixture. This branch has not been deployed to a live instance. ## Model Used OpenAI Codex, GPT-6, with code execution and browser tools. The exact backend model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-10-22 18:37:36 -05:00
DottaandPaperclip	042051904d	fix(codex): deliver configured service secrets to shell tools (#17216) ... ## Thinking Path > - Paperclip lets operators assign tools and credentials to agents. > - Legacy Codex starts provider processes and builds a separate shell environment policy. > - Both ACP and CLI removed every variable whose name contained KEY, SECRET, or TOKEN. > - This also removed service credentials that the operator explicitly assigned, such as SLACK_BOT_TOKEN. > - This pull request uses the resolved configuration to admit assigned service secrets by name. > - Every Codex launch keeps a bounded shell allowlist, including launches without an agent identity. > - Assigned scripts can use their credentials without adding unassigned host secrets to shell tools. ## Linked Issues or Issue Description Related: #16255. That PR fixes project-level exclude patterns. This PR fixes the removal of explicitly assigned service secrets and the fallback to an inherited policy when no agent identity is present. **What happened?** A Codex task could not run a Slack posting script. SLACK_BOT_TOKEN was present in its resolved launch environment, but Paperclip removed its name from shell_environment_policy.include_only. A live removal check also showed that an identity-free launch without an explicit policy could inherit an unassigned host token. **Expected behavior** An explicitly assigned service credential reaches shell commands. Removing that binding removes its shell access on the next launch. Unassigned host credentials stay outside the allowlist. Codex login credentials and observability headers keep their separate provider paths. **Steps to reproduce** 1. Configure a Legacy Codex agent with a synthetic SLACK_BOT_TOKEN value. 2. Run a shell command that checks whether the variable exists without printing its value. 3. Before this fix, the variable is absent. Repeat through ACP and CLI. 4. Remove the binding and run a second turn. Check that neither the removed token nor an unassigned host token reaches the command. **Paperclip version or commit** Reproduced in the executor regressions before the fix on master at 4dfcb4c9b8. The final change is rebased on 25f2d05282. **Deployment mode** Source checkout. Local macOS arm64 provider acceptance with Codex CLI 0.160.0 and Codex ACP 1.6.2, gpt-5.6-sol, and an existing ChatGPT login. ## What Changed - Added one shared Codex shell-name policy for Legacy ACP and CLI execution. - Admit explicitly resolved service-secret bindings. Keep Codex login keys, observability credentials, and private runtime transport tokens outside that opt-in. - Reject glob-shaped environment names so one configured name cannot match other credentials. - Always emit the bounded policy. Do not fall back to host inheritance after a binding is removed. - Updated executor regressions, the identity contract, and the paired verification maps. ## Verification - Red before the fix: executor regressions failed because SLACK_BOT_TOKEN was absent in ACP and CLI, with and without agent identities. - Green: 267 tests passed across the two executor suites and the shared shell-policy suite. Coverage includes host/provider isolation, unresolved and removed bindings, glob names, identity delivery, and names-only metadata. - The complete Codex adapter source suite passed after the CI follow-up: 571 tests passed, 1 skipped across 31 files. Its typecheck also passed. - The first CI run caught four stale expectations in ACP config and SSH launch tests. Those now assert the bounded policy and the existing command/resume behavior. No production code changed in that follow-up. - Live CLI and ACP acceptance: run a Python presence check through each real adapter, remove the synthetic Slack binding, and run again. The assigned token was present only on the first turn. The unassigned host token and OpenAI API key were absent on both turns. CLI retained its session; ACP correctly invalidated its session after the configuration changed. Workspace proof files persisted. - An earlier live CLI check failed after binding removal because the no-policy fallback exposed the synthetic host token. The final change removes that fallback. Both live checks then passed. - Full pnpm -r typecheck and pnpm build passed. The changed packages were compiled again after the safety correction. - node scripts/check-maps.mjs and git diff --check passed. - The local pnpm test:run attempt reported failures in native warm-session tests and was interrupted. That full-suite result is not a pass; PR CI must establish the final repository-wide gate. One named native failure passed alone, and the complete native executor file then passed separately (774 tests). The original broad-run failure cause remains unconfirmed. - After rebase, one Gemini skill-root case timed out at five seconds while the broad run was active. The case passed alone, and the full focused set then passed again at the PR head (267 tests). The earlier timeout remains part of the verification record. - An initial broad adapter run failed because sandbox restrictions denied loopback listeners. Running the same suites with loopback access passed. ## Risks - Explicitly assigned service secrets now reach shell commands. This is the intended behavior change: assigning a credential authorizes the run to use it, and a model-directed command can read it. This is not a broker that hides assigned credentials from the model. Operators must assign only credentials and service permissions authorized for the run. Configuration and secret-access authorization remain with their existing owners. - Superagent initially flagged this intended assigned-credential exposure as a P2. The inline reply records the trust-model rationale and the documentation states it explicitly. The security scan on the final documented revision passed without any check override. - The launch policy contains names, not values. Log redaction remains in place. Provider login credentials and observability headers remain excluded from the service-secret opt-in. - Identity-free Codex runs now also receive a bounded policy. This removes reliance on the operator's inherited shell policy. - Existing explicit project exclude patterns retain their behavior; #16255 addresses that separate case. - The change has not been deployed to a shared instance. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, and code execution. The exact served model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with Fixes / Closes / Refs OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run the focused tests locally and they pass; the full-suite limitation is recorded above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-22 18:17:34 -05:00
Devin FoleyandPaperclip	12bb7e1976	feat: track git workspace restore points and apply their retention (#17241) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Closed issues' git worktrees pile up on disk because they can only be removed once the remote reproduces them exactly > - #17238 adds restore points: encrypted saves of the unpushed and uncommitted parts of a worktree, so the worktree itself can be removed > - Those saves live in file storage and must be deleted on a schedule, and promptly when their issue or company is deleted > - This pull request adds the table that tracks restore points and a deadline-driven retention worker, with nothing creating restore points yet > - The benefit is a reviewable lifecycle foundation before capture is wired into expiration ## Linked Issues or Issue Description Refs #17238 (restore point library). Bug report described in-PR (bug report template). **What happened?** Closed issues' worktrees accumulate without bound. For example, one instance's last 111 git cache expiration checks removed nothing, because each worktree held unpushed commits or uncommitted files. Restore points (#17238) will let those worktrees be removed safely. That needs a durable record of every stored restore point and a retention policy for deleting them. **Expected behavior** - A restore point is kept while its work can still resume. - It is deleted a fixed time after its issue tree closes. - It is deleted promptly when its issue or company is deleted. **Steps to reproduce** Not applicable on its own: this adds the lifecycle that the capture integration will use. **Paperclip version or commit:** `master` at def4726bea. **Deployment mode:** any. **Agent adapter(s) involved:** not adapter-specific (core). ## What Changed **New table `execution_workspace_restore_points`** (migration `0372`, schema `packages/db/src/schema/execution_workspace_restore_points.ts`): - **Columns:** storage key, sha256, size, generation, source issue, `status`, closing time, review/deletion time (`delete_after`), a `review_generation` counter and last error. - **Statuses:** `pending`, `active`, `expiring`, `superseded`, `delete_requested`, `deleted`. - **No foreign keys:** a row outlives the deletion of its workspace, issue or company long enough to remove the stored object. - **Indexes:** - the storage key is unique; - live generations are unique per workspace; - an index on status and deadline serves the worker, and a partial index on active rows' source issue serves the trigger. **Issue tree trigger** (same migration): `paperclip_restore_point_tree_changed` runs after an issue's `status` or `parent_id` changes, and after an issue is deleted. It walks up from the issue and from its old and new parents (at most 64 levels). For every active restore point whose source issue is in that set, it makes the row due now (keeping an earlier due time) and increments `review_generation`. This happens in the same transaction, so a child's deletion or move still reaches its former parent's tree. **New `server/src/services/git-restore-point-ledger.ts`:** - **`gitRestorePointRetentionService(db, { storage, retentionDays, beforeExpire })`.** Each `tick` handles only due work, in batches of 50: - **Review:** an active restore point is checked when its `delete_after` time arrives. The due check compares against the later of the worker's clock and the database clock, so a row the trigger just marked is never skipped. At review: - **Tree still open:** clear `delete_after`. The trigger marks it due again on the next relevant issue change, so open trees cause no recurring checks. - **Tree `done` or `cancelled`:** expire `PAPERCLIP_GIT_RESTORE_POINT_RETENTION_DAYS` (default `90`, minimum `1`) after the tree's latest close. Expiry moves the row to `expiring`, then runs an optional `beforeExpire` hook, which the capture integration will use to mark the workspace no longer restorable. Only then does it request deletion. An `expiring` row's object is never deleted: after a restart, or if the hook fails, the hook is retried (after 15 minutes when it failed). - **Source issue gone:** delete. - **Concurrency:** a review result is written only if `review_generation` is unchanged since the worker read the row. If the trigger fired in between (for example, the tree closed or reopened), the write is skipped and the row stays due for the next run. Expiry and orphan deletion claim the row under the same check before the expiry hook runs. - **Abandoned captures:** a `pending` row older than one hour is deleted. - **Deletion:** `superseded` and `delete_requested` rows have their object removed. They become `deleted` only once `headObject` reports the object gone. Failures, including a local unlink error the provider swallows, are recorded in `last_error` and retried after 15 minutes. - **`nextDeadlineAt`** returns the earliest real deadline, or `null` when nothing is due. - **`requestRestorePointDeletion(tx, { sourceIssueId } | { companyId })`** registers a delivery-work notification and marks the matching `pending`/`active` rows `delete_requested` in the caller's transaction. **Deletion hooks:** - `issueService.remove` and `deleteCompany` call `requestRestorePointDeletion` in their transactions, so a deleted issue's or company's objects are removed promptly. - Child issues cannot be deleted with their parent (the parent link restricts deletion), so each issue removal goes through this hook. **Scheduling** follows `rules/sleep-aware-background-work.mdc`: - `registerGitRestorePointWork` registers a `git-restore-points` queue with the shared delivery-work coordinator. - The worker keeps no in-memory inputs. Due work is recorded in the table, by the trigger or by deletion requests. Each run handles it and schedules only `nextDeadlineAt`, so a failed run loses nothing and is retried by the coordinator. - It runs at startup, at real deadlines, and when terminal-workspace notifications settle after issue changes, but only while restore points exist. With nothing due, it schedules nothing. A full batch continues immediately. There is no timer. **Config and docs:** - `server/src/config.ts` adds `PAPERCLIP_GIT_RESTORE_POINT_RETENTION_DAYS`. - `doc/recoverable-disk-cleanup.md` gains a "Git workspace restore points" section; `doc/DEVELOPING.md` documents the variable. - `drizzle-kit generate` pruned `meta/0367_snapshot.json`, following the repository's snapshot-retention convention. - No code comments were added. ## Verification - `pnpm exec vitest run src/__tests__/git-restore-point-ledger.test.ts` (in `server/`): 26 passed against embedded Postgres, stable across repeated runs. The cases: - **Review outcomes:** an open tree clears its deadline and schedules nothing; a closed tree is scheduled 90 days after its close; a long-closed tree expires and is deleted; a missing source issue is deleted. - **Expiry durability:** a failing hook keeps the row `expiring` with its object until a retry succeeds; an `expiring` row left by a restart resumes the hook before deletion; an `expiring` row's object is never deleted while its hook fails. - **Trigger:** the last open child closing, a child being deleted, a child moving to another parent, and the source issue's own status change all mark the restore point due. Unrelated issues, inactive rows and already-due rows are untouched. End to end, an open tree's restore point waits without a deadline, then is scheduled 90 days after its tree closes. - **Races:** closing the tree, or reopening it, between the worker's read and write leaves the row due, and the next run resolves it correctly. A reopened tree's restore point is not deleted. - **No rescanning:** 60 retained restore points with future review times are not evaluated. A backlog of 60 due ones is processed as 50, then 10, then nothing. - **Captures and deletion:** abandoned pending rows are deleted while recent ones are kept; superseded rows are deleted; an object still present after deletion is not marked deleted; a failed deletion is retried only after its 15-minute deadline. - **Deadlines:** the next deadline is correct for each obligation, and `null` when idle. - **Deletion hooks:** `issueService.remove` marks only that issue's restore points, and `deleteCompany` marks all of the company's; the worker then deletes their objects. - **Worker:** schedules nothing when idle, and only the next deadline otherwise; wakes on issue changes only while restore points exist. - **Mutation checks:** each of these fails at least one case: - removing the issue hook or the company hook; - making the trigger ignore former parents; - removing the worker's generation check or the trigger's generation bump; - letting storage cleanup delete `expiring` rows, or not resuming the hook; - ignoring open trees; - removing the deletion verification; - evaluating rows that are not due; - dropping the retry deadline. - `pnpm exec vitest run company-deletion issue-delete issues-service delivery-work issue-tree` (in `server/`): 190 passed. - **CI note:** one earlier `Typecheck + Release Registry` run was cancelled when the GitHub runner received a shutdown signal; it is not a code failure. - `pnpm exec vitest run` (in `packages/db`): 211 passed. One run had a single failure in `issue-comment-derived-attribution-migration.test.ts`, an older migration's test unrelated to this change. It passed alone and on the full re-run. - `pnpm exec vitest run src/__tests__/git-restore-point-ledger-config.test.ts`: 5 passed. - `pnpm run check:migrations` (in `packages/db`): numbering and safety checks pass. - `pnpm run typecheck` (in `server/`): pass. ## Risks - **No behavior change today:** nothing creates restore points yet. The worker runs once at startup, finds no rows and schedules nothing. The capture integration follows in a separate PR. - **Trigger cost:** every issue status change, parent change or deletion now runs a short ancestor walk, plus an `UPDATE` on the new table that matches only active restore points through a partial index. With no restore points, the `UPDATE` matches nothing. - **Missed wakes:** if a wake is ever missed, for example because the process exits after commit, the due row stays in the table. The worker's startup run or next deadline processes it. - **Deletion paths:** issue and company deletion each gain one `UPDATE` on the new table inside their existing transactions. - **Migration:** `0372_mature_the_initiative.sql` creates one empty table, its indexes, and the trigger function with two triggers on `issues`; nothing is rewritten. - **Rollback:** revert. The table can stay, since nothing else reads it. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file edits, tests, read-only measurements on live instances). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-22 15:52:04 -07:00
TonioandClaude Fable 5.1	341b622604	feat(ui): Artifacts V1 behind an experimental setting — Artifacts page, panel tab and task thread (#17228) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents produce artifacts: documents, files, pictures, video and links. The app shows them on the Artifacts page, in the right panel's Artifacts tab and in the task thread > - The artifact directions lab (#17213) settled a presentation for those surfaces, MVP V1, drawn from the PCLP-Core mocks. It is a reference, not product code > - Shipping it means product components on product data, token-only, behind a switch, one surface at a time, so the current presentation stays the default until the new one has been used > - This pull request adds the switch (Settings > Experimental > Artifacts V1), the `--artifact-*` tokens, the V1 primitives as product components, and the V1 presentation on all three surfaces behind the switch: the Artifacts page, the panel's Artifacts tab and the task thread > - The benefit is the redesign in the product, reversible by one setting, with every existing behaviour of those surfaces kept ## Linked Issues or Issue Description No public issue exists. This section follows the feature request template; the plan is `doc/plans/2026-10-22-artifacts-mvp-v1.md`. **Subsystem affected** The web UI (`ui/`): the Artifacts page, the right panel's Artifacts tab and the task thread. The experimental settings contract in `packages/shared` and `server` gains one boolean. **Problem or motivation** The three artifact surfaces grew separately and their artifact presentation is dated and inconsistent. The lab produced a measured direction; nothing in the product uses it yet. **Proposed solution** An experimental setting, `enableArtifactsV1`, default off, with the same plumbing as the other experimental settings. Product primitives under `ui/src/components/artifacts/v1` (kind helpers and adapters, the sheet and print artwork, the stack, the page tiles with the Paper Mono pill, the row the panel and thread will share), driven by `--artifact-*` tokens in `index.css` with light and dark values. With the setting on: the Artifacts page shows one row of views (Tasks, All files, Images, Videos, Documents, Text, Files) and the V1 tiles, a task with several artifacts as a stack whose newest artifact is the first page, everything else as single tiles; the panel's Artifacts tab is one flat list of V1 rows, newest first, with the expandable rows (Markdown review surface, document annotations, deep links) keeping their behaviour behind the row as their face; the thread shows a message's files as rows under its footer and registered deliverables and documents as one row each. The hover movement respects reduced motion. **Alternatives considered** Replacing the current cards outright: no fallback and three surfaces to land at once. Mounting the lab's components: they are typed on lab fixtures and keep lab-local tokens, which the token gates reject. The primitives were ported instead and the lab stays a reference. **Roadmap alignment** Supports the "Artifacts & Work Products" theme in `ROADMAP.md`. Presentation only; no data model change. **Additional context** No new API fields: the tiles show title, identifier, count and "Updated …"; rows take sizes from the work-product and attachment payloads; a picture's aspect ratio comes from the loaded image, 4:3 until it loads. The plan records what each surface keeps and what it drops (run sections in the panel; the "Images · N / Files · N" labels and the four-up grid in the thread). ## What Changed - Add the `enableArtifactsV1` experimental setting: shared type, validator and feature catalog, server defaults and managed keys, the `useArtifactsV1Enabled` hook, and the Settings > Experimental toggle (in its alphabetical place). - Add the `--artifact-*` tokens and the Artifacts V1 component rules to `ui/src/index.css`: light values on `:root`, dark overrides, and the hover movement under `prefers-reduced-motion: no-preference`. - Add `ui/src/components/artifacts/v1/`: `kinds.ts` (kind treatment, labels, glyphs, the server's media-kind rule, adapters from `CompanyArtifact`), `geometry.ts` (the measured constants), `when.ts`, `ArtifactSheet`, `ArtifactPrint`, `ArtifactStack`, `ArtifactFront`, `ArtifactTile` (stack and single tiles, caption, grid), `ArtifactRow` (the row and its inline group), `ArtifactsV1Filters`, with the mock's SVG assets. - Wire the Artifacts page: with the setting on, the toolbar's filter row becomes the V1 views and the grid renders V1 tiles; the toolbar takes an optional `filters` row for this. The old cards and kind filters stay the default. - Wire the panel's Artifacts tab (`IssuePropertiesArtifactsTab`): with the setting on, one flat list of V1 rows, newest first; `DocumentRow` and `MarkdownWorkProductRow` take a `face` render prop so the V1 row fronts their unchanged expansion, review and deep-link behaviour; `PanelFileRow` opens attachments and attachment-backed work products the way the shipped card does (gallery, text tab, new tab) with open and download beside the name; other work products are a link row with their status. - Wire the task thread: `TaskChatBubble` hangs a message's files under its footer as V1 rows inline (pictures and video to the gallery, text to a text tab, files to a new tab) in place of the media grid and file chips; `TaskChatProtocolCard`'s resource rows render deliverables, documents and attachments as one V1 row each, opening where the card did. - `ArtifactRow` keeps trailing controls beside its own control (no nested interactive elements) and can be a link, an in-app anchor, a new-tab anchor or a button; adapters read work products, documents, attachments and message references. - Add a catalogued Storybook story, `Components/Artifact V1 primitives`, under `documents-artifacts`, on fixtures. - Add tests: the primitives (kinds, dates, print fitting, tiles, rows), the hook, the Artifacts page, the panel's Artifacts tab, the task-chat bubble and the protocol card, each with the setting on and off; the settings page and server tests list the new key. - Add `doc/plans/2026-10-22-artifacts-mvp-v1.md`. ## Verification - Run `cd ui && pnpm typecheck`: the Storybook catalog accepts the new story, and `tsc -b` is clean apart from the five pre-existing `PluginOrganizationSwitcher` errors. - Run `pnpm check:token-gates` and `node scripts/check-maps.mjs`: clean. - Run `cd ui && npx vitest run src/components/artifacts src/components/issue-properties src/components/task-chat/TaskChatBubble.test.tsx src/components/task-chat/TaskChatBubble.artifactsV1.test.tsx src/components/task-chat/TaskChatProtocolCard.test.tsx src/components/task-chat/TaskChatProtocolCard.artifactsV1.test.tsx src/pages/Artifacts.test.tsx src/hooks/useArtifactsV1Enabled.test.ts src/pages/InstanceExperimentalSettings.test.tsx`: the suites pass (151 tests in the last run). - Run `cd packages/shared && npx vitest run src/feature-catalog.test.ts` and `cd server && npx vitest run src/__tests__/instance-settings-service.test.ts`: pass. - Manual: open Settings > Experimental, turn on Artifacts V1. On the Artifacts page the view row reads Tasks, All files, Images, Videos, Documents, Text, Files; tasks with several artifacts show as stacks, hovering fans the pages and lifts the artwork (not under reduced motion); Images shows single tiles with bordered prints. Open a task's Artifacts tab: one list of rows, newest first; a Markdown work product still expands into its review surface, a document still opens in its tab. In the thread, a message's files sit under its footer as rows. Turn the setting off: everything returns to the current presentation. - Storybook: `Tasks and conversations / Documents, attachments, and work products / Components / Artifact V1 primitives`. - Not run: the full `pnpm test:run` and `pnpm build` (CI runs them), and a browser pass against a dev instance with real artifacts on the three surfaces; that pass is the remaining step before this leaves draft. ## Risks - Off by default; with the setting off, the only changes on a product path are an optional `filters` prop on the toolbar, an optional `face` prop on two panel rows, a read of the setting in the bubble and the protocol card, and the new token declarations. - With the setting on, the Artifacts page's view tabs set two URL parameters (`groupBy`, `kind`) at once; the group-by menu and search are unchanged. - A picture with unknown dimensions renders at 4:3 until it loads, then the print refits once; the frame's height is fixed so nothing below it moves. - About thirty new `--artifact-*` tokens, namespaced and documented in place. ## Model Used - Claude Fable 5.1 by Anthropic, model ID `claude-fable-5-1`, used through Claude Code in the desktop app. Extended reasoning was on. Capabilities used: tool use (shell, file edits, a headless Playwright browser for the lab's measurements, the Figma MCP server for the mocks during the lab work) and image input. The context window is the Claude Code session's; its exact size is not surfaced to the session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above (#17213 is the lab this builds on) - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (the suites listed above; CI runs the full set) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (the plan document; no command or product behaviour changes outside the setting) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>	2026-10-22 15:14:58 -07:00
Devin Foley	059ae49cab	feat(connections): add Unikraft Cloud connection (#17139) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Apps connections give agents governed access to external services through vault credentials, grants, action policies, the tool gateway, and audit. > - Unikraft Cloud runs unikernel micro-VMs, services, and volumes across regional metros, and it offers a sandbox plugin for commands and files. > - Unikraft is not in the catalog. It has no hosted MCP server, so the generic "Connect your own MCP server" flow cannot reach it. > - Daytona (#16483) added a native REST pattern for this case: a catalog generated from the provider's published OpenAPI specifications and executed through the gateway. > - This pull request adds Unikraft on that pattern and moves the shared native-provider code into one registry, so the shared connection code does not name each provider. > - The benefit is governed agent access to all published Unikraft actions, with the token kept in the vault and sent only to published Unikraft hosts. ## Linked Issues or Issue Description Refs #16483 **Problem or motivation** Teams that run workloads on Unikraft Cloud want their agents to list instances, read logs, deploy and scale services, and run commands in sandboxes. Unikraft is not in the Apps catalog. Unikraft has no hosted MCP server, so operators cannot connect it through the remote MCP flow. **Proposed solution** Add a native Unikraft connection. Generate its actions from Unikraft's official OpenAPI specifications for the platform API and the sandbox plugin. Pin the specifications to a reviewed commit. The operator connects an API token. The token stays in the vault and goes only to the five published metro hosts. Every action takes an optional metro. Actions without a metro use the connection's default metro, which the operator can change under Advanced. **Alternatives considered** A catalog-only AppDefinition was not possible because no hosted MCP endpoint exists. Wrapping the `unikraft` CLI was rejected because it would expose host filesystem behavior and CLI login state. A second copy of the Daytona bridge was rejected; the shared transport and catalog contract are now one module with a provider registry. **Roadmap alignment** This extends the existing Apps and Connections provider coverage. It uses the existing credential, grant, policy, gateway, and audit model. It does not overlap planned core work. ## What Changed - Added `scripts/generate-unikraft-tools.mjs` and its tests. The generator reads the platform and sandbox plugin specifications at commit `5dd107ea1cd8716b83cb86a83ed2a6ee46f99b5b`, resolves shared parameters, records source SHA-256 hashes, and writes `server/src/services/unikraft/operations.generated.json`. - The catalog has 105 actions: 89 platform operations and 16 sandbox plugin operations. The server-sent audit event stream is recorded in the inventory and is not advertised as an action. - Classification: GET operations and sandbox reads and waits are reads. Delete, stop, suspend, detach, unpin, and sandbox command execution, signals, and stdin are destructive. All other operations are writes. - Added `server/src/services/connectors/unikraft/client.ts` and its generated catalog. It validates arguments against the generated schemas, accepts only published metros, rejects path traversal, sends JSON bodies on GET and DELETE as the specification requires, and verifies the token with `GET /v1/users/quotas`. A sandbox wait that times out (documented HTTP 408) returns a still-running result instead of a failure. - Moved the bounded, credential-redacting HTTP sender and the catalog contract helpers from `daytona.ts` to `native-rest-client.ts`. Added `native-provider-catalogs.ts`. `tool-access.ts`, the `tool-access/catalog` and `token-broker` modules, `connectors/tool-policy.ts`, and `tool-gateway.ts` now use this registry instead of Daytona-specific checks. Daytona behavior does not change. - Added the Unikraft definition source, the generated definition, the brand mark and manifest entry, and the permission review (`liveProof: not-run`). - Added `doc/connections/UNIKRAFT.md` with a sequence diagram, endpoints, setup, actions, and limits. Updated the connections README, the permission audit, and both capability maps. - Tests: definition shape, database-backed gateway reads, writes, deletes, and sandbox commands for personal and shared grants, default and per-call metros, rejection of unpublished metros and traversal, Off policies, revocation, stale-contract blocking, credential export denial, schema validation for all actions, pinned-transport tests for GET and DELETE bodies, and a browser setup check. ## Verification - `pnpm exec vitest run packages/shared/src server/src/__tests__/unikraft-connection.test.ts server/src/__tests__/unikraft-http-boundary.test.ts server/src/__tests__/daytona-connection.test.ts server/src/__tests__/daytona-http-boundary.test.ts server/src/__tests__/tool-access-service.test.ts server/src/__tests__/tool-gateway-service.test.ts server/src/__tests__/tool-gateway.test.ts ui/src/pages/apps/AppsConnect.test.tsx ui/src/pages/apps/Browse.test.tsx ui/src/lib/app-brand-assets.test.ts` plus `connector-tool-policy` and `tool-catalog-publication`: all passed after the rebase onto the split tool-access modules. - `node --test scripts/generate-unikraft-tools.test.mjs scripts/generate-daytona-tools.test.mjs scripts/app-brand-validation.test.mjs`: 20 passed. - `node scripts/check-app-brand-assets.mjs`, `node scripts/check-maps.mjs`, and `git diff --check`: passed. - `pnpm --filter @paperclipai/shared typecheck`, `pnpm --filter @paperclipai/server typecheck`, and `pnpm --filter @paperclipai/ui typecheck`: clean. - `npx playwright test --config tests/e2e/playwright.config.ts tests/e2e/unikraft-connector.spec.ts tests/e2e/daytona-connector.spec.ts`: 2 passed. - `node scripts/generate-unikraft-tools.mjs` regenerates the committed catalog with the recorded hashes. - Manual: on a local instance, open Connectors, search for Unikraft, and select **Add key**. Confirm the icon, the password field, the key guidance, and the default metro select under **Change**. - Not verified: a real Unikraft account. Live provider read and write proof is pending. ## Risks - A Unikraft token acts for the whole organization in every metro. Unikraft offers no narrower token permissions. Assign only trusted agents and review action permissions. - Sandbox actions run arbitrary commands inside the selected instance. They are classified as destructive. - A timed-out write can still complete at the provider. The bridge does not retry writes automatically. - New metros fail closed until the specification is regenerated and reviewed. Legacy `*.kraft.cloud` hosts are not reachable. - The Daytona refactor moves code without behavior changes. All Daytona tests pass without edits. - The generated JSON snapshot is large because it holds the provider input contracts. No migration or lockfile change is required. ## Model Used - Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, 1M context window, used through Claude Code with tool use (shell, file editing, web research, browser tests). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-10-22 15:14:26 -07:00
DottaandPaperclip	4dfcb4c9b8	refactor(tools): isolate connection access and removal (#17167) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Connection grants and installations control which identities and agents can reach tools. > - Removal must close those access paths and stop runtimes before cleaning up owned secrets. > - These responsibilities still shared the setup coordinator after earlier extractions. > - This pull request gives grants, installations, credential validation and removal explicit owners. > - The benefit is a smaller coordinator and focused access-lifecycle modules with existing security ordering intact. ## Linked Issues or Issue Description Depends on #17165. Earlier stack: #17144 → #17146 → #17147 → #17155 → #17157 → #17159 → #17161 → #17162 → #17165 → this PR. **What existing behavior does this improve?** Maintenance of connection audiences, delegation, installation and removal. **Current behavior** The 6,419-line service still owns grant mutation, installation reach and resumable credential cleanup alongside app setup. **Proposed behavior** A grant owner manages audiences and delegation. An installation owner manages reach and its profile bindings. A removal owner keeps access revocation, runtime shutdown, secret classification and cleanup together. The coordinator delegates through typed APIs. **Reason and benefit** The entry point falls to 4,821 lines / 198,065 bytes, meeting the requested under-5,000-line target. Across the full stack it falls from 21,525 lines / 829,519 bytes. App setup remains together in the coordinator. ## What Changed - Extract credential validation (75 lines), grants (603), installations (350) and removal (658). - Preserve company boundaries, audience advisory locks, membership locks, assignment locks and audit order. - Keep the existing runtime supervisor and secret client. Retain live HTTP options for provider shutdown. - Keep Slack credential-mutation leasing and resumable owned-secret cleanup with removal. ## Verification - Removal, grant-resolution and the stronger real HTTP OAuth fixture pass before extraction (13 cases); the existing service and publication baselines also pass. - The original combined targeted run passed 525 cases across seven suites. After replaying onto master, 601 cases pass across nine suites, including Braintrust lifecycle and connector-policy coverage, with one database worker. - Full repository typecheck and build passed before merge integration. After integration, server typecheck, module-boundary and whitespace gates pass; fresh CI covers the full repository gates. - All 94 pre-existing function/method bodies match by normalized syntax comparison. A full-stack TypeScript symbol audit preserves all 18 exports and 89 service methods. - The earlier local full-suite run was interrupted after 79 minutes and remains unverified. All current-head CI gates pass on `d5307fcea2365e51a75d49db82deeed236d19951`, including the full sharded test matrix, browser suites, typecheck, build and canary dry run. Greptile is 5/5 with no open review threads after merge integration. - GitHub canceled duplicate CI run [39716841632](https://github.com/paperclipai/paperclip/actions/runs/39716841632) during the atomic stack update. Replacement run [39716843243](https://github.com/paperclipai/paperclip/actions/runs/39716843243) passed on the same head; no failing test was retried. ## Risks - Audience replacement must not erase an inactive member marker and widen authority. Its advisory and membership locks remain in the grant owner. - Uninstall must remove only its own profile bindings. Existing operator-authored bindings keep their current behavior. - Removal must remain fail-closed and retryable after provider failures. Its order, shared-secret retention, supervisor and cleanup references are unchanged and covered by removal tests. - No schema, public API, policy or scheduling change is intended. The first eight PRs have landed; this PR follows the updated catalog extraction. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-21 22:39:00 -05:00
DottaandPaperclip	f7210cd31f	refactor(tools): isolate catalog ownership (#17165) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents receive reviewed actions discovered from their connections. > - Transport, discovery, risk classification, publication and account inventory lived in the same service. > - Their ownership differs: transport guards requests, discovery reads providers, and publication changes the catalog under locks. > - This pull request gives those phases separate modules with explicit capabilities. > - The benefit is a visible discovery-to-publication path that preserves permission and quarantine behavior. ## Linked Issues or Issue Description Builds on merged #17162 and preserves the connector policy from #17141. Earlier stack: #17144 → #17146 → #17147 → #17155 → #17157 → #17159 → #17161 → #17162 → this PR. **What existing behavior does this improve?** Maintenance of tool discovery, catalog publication, health and upstream account inventory. **Current behavior** The 8,988-line service handles HTTP guards, MCP sessions, provider probes, risk classification, quarantine, cached reads and account sync. **Proposed behavior** Discovery returns descriptors through the existing credential and transport owners. Publication owns catalog writes and default access. A reader owns refresh deduplication. Health and aggregator inventory have separate owners. **Reason and benefit** The entry point falls to 6,419 lines, a reduction of 2,569. Provider discovery no longer shares a closure with connection setup and teardown. ## What Changed - Extract guarded remote HTTP, descriptor policy, discovery, publication, default access, cached reads, health, attention and aggregator account sync. - Keep the public descriptor/risk exports and existing service methods. - Preserve database locks, transaction-scoped default-access writes, quarantine decisions, current grant checks and failure classification. - Preserve per-service catalog flights and aggregator jobs, lease ownership and live remote-request options. - New modules range from 39 to 492 lines. ## Verification - Before and after the move, 470 public service/HTTP integration cases and 44 publication, HTTP rebinding and aggregator sync cases pass. - Server typecheck, module-boundary gate and whitespace checks pass. - All 157 pre-existing function/method bodies match by normalized syntax comparison. - Before merge integration, the completed stack passed 525 targeted cases, repository-wide typecheck and build. Its stronger rotating-token fixture is included. - After replaying onto master, the integrated stack passes 601 cases across nine suites, including Braintrust lifecycle and connector-policy coverage. Server typecheck and module-boundary checks pass. All 157 moved bodies still match the current master versions. - The earlier local full-suite run was interrupted after 79 minutes and remains unverified. All current-head CI gates pass on `a21595f20809a514b7619d8a8748c56f8e546e2f`, including the full sharded test matrix, browser suites, typecheck, build and canary dry run. Greptile is 5/5 with no open review threads after merge integration. ## Risks - Discovery must keep its current grant and session behavior; publication must retain its locks and quarantine rules. Existing authority, schema-drift and publication-race tests exercise these paths. - Aggregator account observations must not become authorization. Lease, freshness, failure and credential-fingerprint behavior stay unchanged. - Live transport options must not be snapshotted. The HTTP owner retains the same options object. - No schema, public API, access-policy or scheduling change is intended. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-21 22:38:59 -05:00
DottaandPaperclip	ba9e6e4bc5	refactor(tools): isolate authorization lifecycle (#17162) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Browser authorization binds a consenting identity to a company connection. > - Direct OAuth, managed-cloud claims and Vercel authorization have different state and persistence rules. > - Keeping all of them in the general tool service obscures those rules and their cleanup owners. > - This pull request separates state, initiation, provider completion and explicit identity promotion. > - The benefit is reviewable authorization flows with their existing authority checks and ordering preserved. ## Linked Issues or Issue Description Depends on #17161. Earlier stack: #17144 → #17146 → #17147 → #17155 → #17157 → #17159 → #17161 → this PR. **What existing behavior does this improve?** Maintenance of browser authorization and credential-sharing lifecycle. **Current behavior** The 12,019-line service combines provider callbacks, actor-bound states, authorization URLs and personal-to-company credential promotion. **Proposed behavior** Separate owners handle authorization state, start, direct/cloud/Vercel callbacks, catalog completion and explicit access finalization. Shared setup policy and identity checks have typed APIs used by the coordinator and authorization owners. **Reason and benefit** The entry point falls to 8,997 lines, a reduction of 3,022. Provider-specific state consumption and durable-write rules are visible in their own modules. ## What Changed - Extract authorization flows and their constraints; the largest new module is 700 lines. - Preserve direct/Vercel terminal state consumption and managed-cloud retryability until claim persistence commits. - Keep membership and grant locks, secret transaction clients, interaction resolution and catalog ordering unchanged. - Keep partial secret acquisition and cleanup together in access finalization. - Give shared organization-grant persistence, personal identity checks, setup policy and response projections named owners. ## Verification - All 470 public service and real local HTTP/PostgreSQL OAuth cases pass before and after extraction. - Coverage includes same-actor state binding, replay/denial, scope selection, membership revocation, provider failures, refresh concurrency and persisted usable credentials. - Server typecheck, module-boundary gate and whitespace checks pass. - All 182 pre-existing function/method bodies match by normalized syntax comparison. - The completed stack passes 525 targeted cases, full repository typecheck and build. The earlier interrupted local full suite remains unverified; all current-head CI gates pass; Greptile is 5/5 with no open review threads. - Original CI failure retained: [server shard 1/12](https://github.com/paperclipai/paperclip/actions/runs/39710536789/job/119600621627), `workspace-runtime.test.ts` case “replaces a reused Paperclip dev runtime whose 2xx health payload is unhealthy”. The fixture failed its initial health readiness with `ECONNREFUSED` after 97 probes / 10 seconds. The fixture and runtime implementation are unchanged and do not import this service. The isolated case passes without changing its timeout or assertions. One retry of only that shard passed at the same commit. All CI gates are now green; the original failure remains recorded here. ## Risks - Provider flows intentionally consume authorization state at different points. This extraction preserves each flow rather than sharing its completion algorithm. - Promotion must remove partially created secrets on failure and retire source secrets only after persistence. Both phases remain in the same owner with unchanged ordering. - Local HTTP fixtures prove assembled application behavior, not live third-party qualification. - No schema, public API or authorization policy change is intended. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-21 22:21:43 -05:00
DottaandPaperclip	4fdef9457f	refactor(tools): isolate credential ownership (#17161) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Connections need durable credentials that survive authorization and token rotation. > - Storage, registration, refresh coordination and credential selection lived in the general tool service. > - Each has a different ownership boundary: secrets, process-wide registration, database leases and per-service refresh work. > - This pull request gives those responsibilities named modules and passes their narrow capabilities to consumers. > - The benefit is visible credential ownership while retaining current concurrency and failure behavior. ## Linked Issues or Issue Description Depends on #17159. Earlier stack: #17144 → #17146 → #17147 → #17155 → #17157 → #17159 → this PR. **What existing behavior does this improve?** Maintenance of OAuth credential persistence, refresh and provider credential access. **Current behavior** The 14,877-line service owns registration flights, refresh leases, secret writes and provider selection alongside app setup. **Proposed behavior** Dedicated owners handle credential storage, client registration, refresh leases, connection/grant refresh, GitHub continuity and Vercel credential access. Credential selection composes those owners. Authorization callbacks remain in the coordinator for the next extraction. **Reason and benefit** The entry point falls to 12,019 lines, a reduction of 2,858. Lease acquisition and cleanup can be reviewed together, independently of callback completion. ## What Changed - Extract credential owners and shared single-flight/connection-link primitives. The largest new module is 650 lines. - Preserve process-wide client-registration flights, per-service refresh maps and GitHub sweep state. - Preserve live cloud-connector lookup, options and environment reads; use the existing secret client and transaction-scoped clients. - Extend the real local HTTP/PostgreSQL OAuth fixture with concurrent refresh across two service instances and authenticated use of rotated credentials. ## Verification - The two extended HTTP integration cases pass before extraction for public and client-secret-basic clients. - After extraction, all 470 public service and HTTP integration cases pass, including refresh failure, abandoned leases, secret redaction and membership revocation cases. The fixture was then strengthened after review: a second expiry must use the persisted rotated refresh token, and the provider rejects the old token. Both cases pass before and after extraction; server typecheck also passes. - Server typecheck, module-boundary gate and diff whitespace checks pass. - All 233 pre-existing function/method bodies match by normalized syntax comparison. - Full typecheck/build will run at the stack handoff. The earlier interrupted local full suite remains unverified; all current-head CI gates pass, with Greptile 5/5 and no open review findings. - Prior-head CI failure retained: run 39708550066, general server shard 2/12, `native-session-executor.test.ts` case “preserves a real listening process and its marker when a health RPC times out” raised `runner_remote_process_identity_unavailable` / `process_identity_changed`. Both the fixture and runner implementation are unchanged. The isolated case passes without changing assertions or timeouts. The fresh full CI run passes for the test-only review fix; the prior failure remains recorded here. ## Risks - Changing state lifetime could permit duplicate refreshes or registrations. Existing concurrency cases and the real HTTP fixture exercise these paths. - A transaction-scoped secret write must stay inside its authority lock. Function bodies, await ordering and client arguments are preserved. - The fixture proves assembled local HTTP/database behavior, not live third-party qualification. - No schema, public API, policy or scheduling change is intended. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-21 22:21:43 -05:00
DottaandPaperclip	ffcd113761	refactor(tools): isolate runtime and activity ownership (#17159) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Operators inspect tool execution, review pending actions and control local runtimes. > - These paths shared a large service with token exchange and app authorization. > - Runtime state and execution history need explicit owners and stable public responses. > - This pull request separates runtime controls, health summaries, activity reads and the review queue. > - The benefit is focused modules that preserve the current supervisor and approval behavior. ## Linked Issues or Issue Description Depends on #17157. Earlier stack: #17144 → #17146 → #17147 → #17155 → #17157 → this PR. **What existing behavior does this improve?** Maintenance of tool runtime administration and execution activity views. **Current behavior** The 16,100-line service contains runtime slot controls, health calculations, usage history, invocation decisions and action review queries. **Proposed behavior** Runtime modules own slot persistence, operator controls and health summaries. Activity modules own history, response projections and the review queue. The public service delegates to these owners. **Reason and benefit** The entry point falls to 14,877 lines, a reduction of 1,223 lines. The coordinator no longer needs execution-history joins or runtime alert calculations. ## What Changed - Extract five modules totaling 1,282 lines. The largest is 426 lines. - Pass the existing supervisor to slot controls. Pass the same live deployment options, clock and host getter to runtime health. - Keep activity reads scoped as before. Keep unsigned-request grace and invalid-signature cancellation in the review-queue owner. - Preserve public methods, response projections, query order and audit writes. ## Verification - The public service and unsigned review-queue tests pass before and after the move. The combined post-move run passes 469 cases with one database worker. - Existing cases exercise actual local stdio stop/restart, route permissions, stale-slot and audit-failure health, activity attribution, run decisions and unsigned pending requests. - Server TypeScript check, module-boundary gate and `git diff --check` pass. - Normalized syntax comparison finds all 256 pre-existing function/method bodies unchanged. - Full typecheck/build will run at the stack handoff. The earlier interrupted local full suite remains unverified; all current-head CI gates pass, with Greptile 5/5 and no open review threads. ## Risks - Creating another supervisor would split process ownership. The new module receives the same instance; it does not construct one. - Copying deployment settings could freeze live values. The health reader holds the existing options object and getter. - Review-queue reads can cancel invalid requests. Those writes and the unsigned-request grace period are preserved and covered by the existing regression test. - No schema, public API, authorization policy or background scheduling change is intended. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-21 22:21:42 -05:00
DottaandPaperclip	a42f213244	refactor(tools): isolate connection token broker (#17157) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents can request short-lived tokens for connections that allow token exchange. > - Admission, scope checks, exchange protocols and issuance records lived inside the general tool service. > - The security flow needs one visible owner while provider payloads need a separate boundary. > - This pull request extracts the broker into admission, configuration, endpoint, exchange and issuance modules. > - The benefit is a focused issuance flow with its locks, authority checks and audit order intact. ## Linked Issues or Issue Description Depends on #17155. Earlier stack: #17144 → #17146 → #17147 → #17155 → this PR. **What existing behavior does this improve?** Maintenance and verification of connection token issuance for agent runs. **Current behavior** A 17,752-line service contains broker configuration, endpoint policy, credential exchange, admission and issuance. **Proposed behavior** A dedicated broker coordinates the same admission and transaction flow. The exchange owner returns normalized token material. It receives the existing secret client, including the transaction-scoped client at issuance time. **Reason and benefit** The entry point falls to 16,100 lines, a reduction of 1,652 lines. Reviewers can see the responsible-user lock and hash-only issuance write without navigating app setup or OAuth callbacks. ## What Changed - Extract six broker modules. The largest is 755 lines; together they contain 1,721 lines. - Give tool-access audit persistence a named owner shared by the broker and coordinator. - Keep current-state checks, policy decisions, rate limits, secret resolution and transaction order unchanged. Private-host configuration is still read live. - Add two public-route integration cases with a real local HTTP server and disposable PostgreSQL. Cover Pages and RFC 8693 success and upstream failure. ## Verification - Baseline: all 466 existing public service tests passed before this move. The two new real HTTP cases passed on the original implementation before extraction. - After extraction: 495 tests pass across the public service and HTTP rebinding suites, with one database worker. This includes the two new cases, membership revocation ordering, scope denials, disabled broker behavior and hash-only persistence. - Server TypeScript checking, the module-boundary gate and `git diff --check` pass. - Normalized syntax comparison finds all 294 pre-existing function/method bodies unchanged. - Full typecheck and build passed for the preceding batch. Its interrupted local full-suite result remains unverified. All current-head CI gates pass; Greptile is 5/5 with no open review threads. ## Risks - A new closure could use the wrong secret client or move exchange outside the membership lock. The existing transaction still passes `secretService(tx)` to the exchange owner after taking that lock. - Endpoint validation must preserve live configuration and fail before credential disclosure. Existing rebinding and private-host tests cover this path. - The local fixture proves real HTTP and persisted application behavior. It does not qualify a live third-party broker. - No schema, API or access policy change is intended. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-21 22:21:42 -05:00
DottaandPaperclip	233753e577	refactor(tools): isolate profile ownership (#17155) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Tool profiles select the actions each agent can use. > - Profile reads, review, mutation and assignment resolution shared a large OAuth and connection service. > - Those paths have their own company checks, lock order and database projections. > - This pull request gives profiles explicit owners while preserving the existing transactions and public API. > - The benefit is a smaller coordinator and a reviewable access boundary. ## Linked Issues or Issue Description Depends on #17147. Stack: #17144 → #17146 → #17147 → this PR. These PRs remain open. **What existing behavior does this improve?** Maintenance of tool profile management and effective agent assignments. **Current behavior** The service entry point contains 19,293 lines. Profile behavior is spread across its closure and returned object. **Proposed behavior** Profile management accepts only a database and clock. Assignment resolution accepts three named capabilities. Projection and shared lock functions have explicit homes. Public methods and responses remain unchanged. **Reason and benefit** Reviewers can follow profile transactions, lightweight reads and precedence without navigating OAuth setup. The entry point falls to 17,752 lines, a reduction of 1,541 lines in this PR. ## What Changed - Extract profile management, projections and effective assignment resolution into three modules. Their sizes are 886, 268 and 216 lines. - Move shared actor/transaction types, identifiers and assignment locks into named modules. Preserve the public lock export. - Keep all database reads, transaction boundaries, lock order and method signatures. Add no cache or background work. - Record the next ownership cuts toward an entry point below 5,000 lines. ## Verification - Before and after: 466 cases in `tool-access-service.test.ts` and 33 cases across profile entry validation, effective assignments and catalog publication. All 499 cases pass with one database worker. - Server TypeScript check and `node scripts/check-module-boundaries.mjs` pass. `git diff --check` passes. - Normalized syntax comparison finds all 329 pre-existing function/method bodies unchanged. Wiring passes the unchanged public service tests. - The previous stack passed full typecheck and build. Its full local test run was interrupted and remains unverified; its complete CI matrix passed. This PR will record its own current-head CI result below. - Current-head CI is green. Current review: Greptile is 5/5 at `0f5c85905e`, with no inline review threads. - CI failure preserved: [general server shard 1](https://github.com/paperclipai/paperclip/actions/runs/39704932041/job/119582710174) failed the unchanged workspace-runtime test that replaces an unhealthy local process. The replacement refused connections for its original 10-second readiness budget. The fixture and runtime service are unchanged and do not import tool-access. One isolated run passed without changes or timeout increases. The failure log is preserved; the cause is not reproduced. Only that failed job was retried; it passed at the same head without code changes. The original failure remains recorded above. ## Risks - Moving profile mutations could change transaction clients or lock order. Both are unchanged and the existing database tests cover publication and assignment behavior. - Effective assignment resolution still uses the same installed-connection loader and full catalog projection. The lightweight path still omits large schema and metadata fields. - No schema, public API or access policy change is intended. Existing tests provide characterization coverage; no new test mirrors the extraction. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-21 22:21:42 -05:00
DottaandPaperclip	4413bb262f	refactor(tools): isolate OAuth protocol boundaries (#17147) ... ## Thinking Path > - Paperclip manages AI agents and their access to external tools. > - Tool connections use OAuth to obtain credentials from remote providers. > - The tool-access service mixed provider protocol parsing with database and secret writes. > - This made protocol changes hard to review without reading the full coordinator. > - This PR gives discovery, endpoint policy, and client exchange explicit owners. > - The coordinator keeps authorization, leases, persistence, and audit decisions. > - A real local provider fixture checks the boundary through public routes. ## Linked Issues or Issue Description **What existing behavior does this improve?** The ownership of OAuth discovery, registration, token exchange, and client metadata. **Current behavior** The tool-access service contains over 20,000 lines. Provider responses, endpoint policy, and stateful operations share the same closure. **Proposed behavior** Small protocol modules accept settings and named request capabilities. They return normalized results. The service owns durable state and existing public exports. This is part 3 of the stack: #17144, then #17146, then this PR. It targets #17146's branch. Review only this PR's diff. ## What Changed - Extract endpoint policy, metadata parsing/discovery, registration/token exchange, provider error normalization, and the public client metadata document. - Validate remote registration responses in the protocol module. Return client material to the coordinator for secret and connection writes. - Remove the unused raw provider response from the internal token result. Keep every consumed field. - Preserve process-wide registration single-flight, service-local maps, refresh leases, membership checks, callback checks, and transaction ownership. - Reuse connection-method URL normalization. Keep the endpoint setting and clock callbacks live. - Add a real local HTTP OAuth/MCP route fixture for public and Basic-authenticated clients. - Reduce `tool-access.ts` from 20,917 lines / 808,777 bytes to 19,293 lines / 749,158 bytes. The seven OAuth modules total 1,697 lines; the largest has 582 lines. The stack removes 2,232 lines from the original service. - Record the next profile-resolution and token-broker boundaries in the dated plan. ## Verification - Baseline: 466 service cases and 80 protocol/provider-policy cases pass. The new two-case HTTP fixture passes before moving production code. - After: `pnpm exec vitest run server/src/__tests__/tool-access-oauth-protocol.integration.test.ts server/src/__tests__/tool-access-service.test.ts server/src/__tests__/remote-mcp-connectors.test.ts server/src/__tests__/remote-http-rebinding.test.ts server/src/__tests__/oauth-client-metadata.test.ts` — 508 tests pass with one worker and disposable PostgreSQL. - The fixture drives connect, metadata discovery, DCR, PKCE code exchange, callback, catalog review, and finish. It checks persisted active state, secret references, issuer/auth-method binding, and wire-level authentication. - `pnpm -r --workspace-concurrency=2 typecheck` passes. The sandbox-only attempt could not open a local IPC socket; the same command passed with local process permissions. - `node scripts/check-module-boundaries.mjs` and `git diff --check` pass. - AST comparison confirms 37 moved bodies are unchanged. Registration protocol and persistence statements match their original halves. Token exchange only drops its unused raw result. Preflight uses explicit defaults and a clock callback. - Initial fixture authoring used a nonexistent catalog enabled field; the corrected assertion checks persisted status and reviewer identity. An initial typecheck found optional defaults at the new boundary; the caller now passes its already-validated defaults. - Final-stack local verification: `pnpm -r --workspace-concurrency=2 typecheck` and `pnpm build` pass. Remote CI is green at `3dc46c132408556ecb31038a9b9621ac20aacefc`. Greptile is 5/5 with no actionable findings or open inline threads. - Full local-suite limitation: `pnpm test:run` was interrupted after 79 minutes during the serial general-server phase, after CI had completed the full test matrix. Before interruption, it flagged the unchanged `company-search-service.test.ts` suite (26 database cases skipped). One isolated rerun passed all 28 cases without code changes. The original log and a pre-interruption copy are preserved. The original failure cause was not reproduced. The full local suite and its later groups are not claimed as passed; the complete CI matrix is green on this stack. ## Risks - OAuth endpoint checks and issuer/resource binding are security boundaries. Existing denial, redirect, scope, DCR, secret-redaction, and refresh tests cover the preserved paths. - A moved closure could capture stale policy or split registration/refresh ownership. The new modules hold no mutable state. Policy is read through the existing callback, and state stays in the service. - The local fixture proves real HTTP and database behavior. It does not claim a live third-party provider sign-in. - No schema, API, authorization policy, or persisted-data migration changes are intended. ## Model Used OpenAI Codex, GPT-6 family, with code execution and repository tools. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-21 22:21:41 -05:00
DottaandPaperclip	8e95caf9ae	refactor(tools): isolate connection method projection (#17146) ... ## Thinking Path > - Paperclip manages AI agents and their access to external tools. > - Connection methods define operator settings and managed tool arguments. > - Setup, catalog discovery, and gateway invocation must use the same projection rules. > - These rules were embedded in a large database service. > - This PR gives them one module with a narrow settings contract. > - Existing provider policy modules still decide their provider-specific restrictions. ## Linked Issues or Issue Description **What existing behavior does this improve?** Connection method configuration and tool input projection. **Current behavior** The tool-access database service owns settings validation and pure input/schema transforms. Callers import these through the service. **Proposed behavior** A connection-method module owns these transforms and accepts only connection settings. Existing imports remain compatible. This PR depends on #17144 and targets its branch. Review only this PR's diff. It is part 2 of the three-PR stack. #17147 follows this PR. ## What Changed - Move method selection, settings validation, and header/schema/argument projection into `tool-access/connection-methods.ts`. - Replace database row parameter types with required config and transport settings. No new validation is added inside typed calls. - Keep Honcho, Google Chat, and Gmail policy in their existing modules. Preserve their call order. - Retain the service's public exports for routes, gateway consumers, and tests. - Reduce `tool-access.ts` from 21,237 lines / 820,116 bytes to 20,917 lines / 808,777 bytes. The new module is 359 lines / 12,665 bytes. ## Verification - Baseline: all 466 tool-access service tests and 80 protocol/provider-policy cases pass before the move. - After: `pnpm exec vitest run server/src/__tests__/tool-access-service.test.ts server/src/__tests__/memory-connectors.test.ts server/src/services/google-chat-tool-policy.test.ts server/src/services/gmail-tool-governance.test.ts` — 506 tests pass with one worker and disposable PostgreSQL. - `pnpm --filter @paperclipai/server exec tsc --noEmit` passes after the existing workspace prerequisites were built. - AST body comparison confirms the 11 moved functions and private record parser are unchanged. `git diff --check` passes. - The first typecheck found optional config in the public DTO. The new input type now states the existing required-config invariant directly. - Final-stack local verification: `pnpm -r --workspace-concurrency=2 typecheck` and `pnpm build` pass. Greptile is 5/5 at `2d0bb86b5830c0306580e3ddc8637c4994b2a75a` with no actionable findings. - CI passed all non-browser gates. [The first browser attempt](https://github.com/paperclipai/paperclip/actions/runs/39675461620/job/119484152569) failed in the unchanged Cases read-only test: the status filter remained All statuses while the announcement-dismiss locator handler interrupted the assertion. The same test passed on #17144. The failure log is preserved. One retry of only the failed jobs passed without code changes; all CI gates are now green on the same head. - Full local-suite limitation: `pnpm test:run` was interrupted after 79 minutes during the serial general-server phase, after CI had completed the full test matrix. Before interruption, it flagged the unchanged `company-search-service.test.ts` suite (26 database cases skipped). One isolated rerun passed all 28 cases without code changes. The original log and a pre-interruption copy are preserved. The original failure cause was not reproduced. The full local suite and its later groups are not claimed as passed; the complete CI matrix is green on this stack. ## Risks - A projection-order change could alter managed arguments or advertised schemas. Function bodies and call order are unchanged, and tests exercise setup, discovery, and gateway consumers. - Legacy Gmail method aliases, GitHub toolset headers, missing unrelated settings, and disabled method filtering retain their previous behavior. - The module has no database dependency, new network work, or mutable state. ## Model Used OpenAI Codex, GPT-6 family, with code execution and repository tools. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-21 22:21:41 -05:00
DottaandPaperclip	9db3a5726e	refactor(tools): isolate GitHub access discovery (#17144) ... ## Thinking Path > - Paperclip manages AI agents and their access to external tools. > - Tool connections discover repositories through GitHub account grants. > - The tool-access service mixes GitHub response parsing with grant authorization and persistence. > - These responsibilities have different inputs and failure rules. > - This PR puts GitHub discovery in one provider module. > - The service keeps current authority checks and stored grant updates. ## Linked Issues or Issue Description **What existing behavior does this improve?** The ownership and review of GitHub access discovery in the tool-access service. **Current behavior** The 21,525-line service contains GitHub pagination, account metadata validation, and grant lifecycle code. **Proposed behavior** A GitHub module returns normalized access metadata. Existing service exports remain compatible. This is the first PR in a three-part refactor stack. No behavior change is intended. A search found no existing tool-access refactor PR or issue. Stack order: this PR, #17146, then #17147. Each PR targets its predecessor. ## What Changed - Move six functions into `tool-access/github-access-discovery.ts`. Their bodies are unchanged. - Keep grant selection, credential resolution, refresh retry, membership locks, revision checks, and webhook updates in the service. - Add public-entry characterization tests for HTTP error classification and invalid pagination. - Record five candidate boundaries and the initial three-PR scope in the dated plan. - Reduce `tool-access.ts` from 21,525 lines / 829,519 bytes to 21,237 lines / 820,116 bytes. The new module is 294 lines / 9,722 bytes. ## Verification - Before and after: `pnpm exec vitest run server/src/__tests__/github-grant-metadata.test.ts server/src/__tests__/project-repositories.test.ts` — 13 tests pass. - Before and after: `pnpm exec vitest run server/src/__tests__/tool-access-service.test.ts -t 'GitHub|github'` — 11 selected tests pass with disposable PostgreSQL; 455 cases are outside this selection. - `pnpm --filter @paperclipai/server typecheck` passes. - Exact text comparison confirms all six moved bodies match the base. `git diff --check` passes. - The initial baseline attempt failed because the plugin SDK had not been built. The normal workspace prerequisite build resolved it before behavioral validation. - Final-stack local verification: `pnpm -r --workspace-concurrency=2 typecheck` and `pnpm build` pass. CI is green at `07e78dd1b1c26e0bd4234006766444c71e234451`. Greptile is 5/5 with no actionable findings. - Full local-suite limitation: `pnpm test:run` was interrupted after 79 minutes during the serial general-server phase, after CI had completed the full test matrix. Before interruption, it flagged the unchanged `company-search-service.test.ts` suite (26 database cases skipped). One isolated rerun passed all 28 cases without code changes. The original log and a pre-interruption copy are preserved. The original failure cause was not reproduced. The full local suite and its later groups are not claimed as passed; the complete CI matrix is green on this stack. ## Risks - Import wiring could break existing consumers. Existing imports remain supported and all four production call sites are unchanged. - Provider requests retain their ordering, timeout, fixed GitHub origin, and response redaction. The new module has no database dependency or shared mutable state. - Tests simulate GitHub responses. They do not qualify live GitHub access. ## Model Used OpenAI Codex, GPT-6 family, with code execution and repository tools. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-21 22:21:40 -05:00
DottaandPaperclip	734c265339	Default new Pi agents to Paperclip Runner (#17142) ... ## Thinking Path > - Paperclip manages AI agents and their work. > - Users choose a harness during agent setup. > - Claude, Codex, and OpenCode already receive their qualified Runner default. > - Pi has a qualified native integration but new Pi agents still use the direct adapter. > - This change gives new Pi agents the native default on supported targets. > - Saved agents keep their runner and compatible settings. Legacy remains available in Advanced. ## Linked Issues or Issue Description Refs #17120, #14921, #14922, #14923, #14924, and #14956. Related generic default work: #15422. This PR is the separate Pi follow-up to the merged OpenCode change. **Current behavior** Choosing Pi for a new agent saves the direct Pi adapter. The native Pi integration remains behind the generic experimental flag. **Proposed behavior** Default new Pi agents to ACPX Pi on macOS ARM64/x64 and GNU/Linux x64 with supported glibc. Keep unsupported targets and explicit Legacy selections on the direct adapter. Require an explicit provider/model before test or hire. Permit model-free inactive setup drafts. **Reason and benefit** Users choose Pi once and receive its qualified runner. Saved agents retain their identity and settings. Conversions reject unsupported legacy thinking and launch settings instead of dropping them. ## What Changed - Add Pi to the shared platform/default contract and server creation selection. - Admit native Pi without the generic experimental flag. - Preserve provider models, credentials, workspace settings, exact thinking, and saved native policies. - Add Pi runner controls, branding, setup resume, onboarding, built-in configuration, and import handling. - Add managed Pi OpenRouter routing and matching native provider API-key accounts. Keep subscription and custom gateway exclusions. Clear alternate credentials and reject custom Pi provider overrides. - Keep Pi draft models empty instead of applying Claude's default model. - Verify setup against the pinned Pi runtime. For native Pi on the official OpenRouter route, reverify the API key live without requiring a global Legacy Pi CLI. - Match Pi’s bounded controller close window to its native snapshot cleanup and lifetime verification, preserving explicit caller deadlines and the mandatory drain/suspension proof. - Bind Pi sidecars to the package that owns the pinned native assets, including source and vendored server layouts. Other providers retain their existing package authority. - Keep protocol-owned finish/block tools out of the ACPX forwarded catalog. The trusted bridge installs these tools itself; the complete Rust authorization catalog and terminal receipt ledger remain intact. This fixes actual Pi session startup and applies to other ACPX consumers. - Advance Copilot’s shared-source attestation to v58 because it pins the shared files touched by Pi. Preserve immutable v57 bytes and historical decoding; Pi’s own release identity is unchanged. - Add regression coverage and update the paired maps and developer documentation. ## Verification - Passed: full repository typecheck and full production build on final head 656d304783, including the generated attestation repair. - Passed: focused shared/server/UI suites covering default selection, platform fallback, conversion preservation, inactive drafts, managed account compatibility, credential isolation, OpenRouter routing/model IDs, exact thinking controls, and saved-account selection. Relevant batches include 375 server/conversion tests, 267 account/routing tests, and 110 UI setup/account tests. - Passed: 179 attestation, historical-profile, qualified-profile, and native-execution contract tests after CI identified stale Copilot source hashes. The identity generator check passes; the only delta after the passing Pi browser run is this Copilot attestation/history repair. - Passed: 145 managed-account tests, including actual service preparation and API-key adoption/rejection for native Pi without the Legacy CLI; 6 pinned-installation probe tests. - Red then green: an actual task failed session.open because its 54-operation catalog included the bridge-owned terminal tools. The regression with those tools failed before the filter; all 399 Rust library tests pass after it. The full product retry exposed an additional package-authority mismatch. Four authority tests failed before that repair; 11 authority checks and 14 installation/asset checks pass afterward. Direct real Pi session initialization with the corrected package root and catalog succeeds. No model prompt was sent by that diagnostic. - Red then green: real Pi posted Hello from Pi but its controller closed before native cleanup could finish. The cleanup-budget test failed with the old default and passes with the bounded Pi window. All 46 selected suspension, drain, and package-authority tests pass after building their native fixture. - Passed: token gates, paired map checks, Rust formatting, and git diff --check. - Passed real browser acceptance on macOS ARM64, generic native runner flag disabled, source a433748a12 (Pi code unchanged afterward): the UI selects Paperclip Runner by default, retains Pi and the selected model/account, and verifies the pinned runtime plus live OpenRouter key. A fresh UI-created task, assigned to the UI-created native Pi agent, posted exactly `Hello from Pi` and reached Done without retry or restart. The native run and workspace finalization succeeded; the response and Done persisted after Reload. - Evidence: 8 preserved browser attempts, including the final passing 18-step journey with all screenshots. Prior failures remain recorded. The earlier failed checkpoint run recovered on server restart; that recovery is separate from the fresh final task’s clean success. - Preserved browser failures: missing account control/inherited Claude draft model; incompatible managed-account/model-prefix handling; generic npm probe for Pi; Legacy CLI hello check; duplicate terminal tools and wrong provider-package authority at actual native session.open; a real Hello from Pi comment followed by premature checkpoint cleanup timeout. Each failure has retained screenshots/logs and a corresponding focused repair. The fresh final task proves these repairs end to end. - The exact Runner Vitest shard was also run locally after the attestation fix: the stale-hash failure is gone; 1,833 tests passed, 10 skipped, and two Codex retained-control settlement scenarios failed with `native_cleanup_maintenance_unproven` during terminal ACK/suspension replay. Both scenarios passed in an isolated rerun (2/2), and the same 19-test file passed in the preceding Linux CI run. The precise full-shard timing cause is not established. These Codex scenarios are outside the changed Pi paths; the original local failures are retained; no complete local Runner-shard pass is claimed. The corresponding Runner shard now passes on the final head in hosted Linux CI. All 60 final-head check runs are now terminal and clean (55 successful, 1 neutral, 4 skipped), and the legacy Snyk status is successful. - Local full repository tests were attempted but not completed: provisioning briefly confused a workspace symlink, then a later run was stopped before browser-driven repairs. No full local-suite pass is claimed. Final-head hosted CI is green and fresh Greptile review is 5/5 with all review threads resolved; earlier jobs canceled by runner shutdown are not counted as a pass. ## Risks - Pi cleanup has a finite 45-second default window, accounting for the native 30-second snapshot cleanup and 5-second lifetime fence. Other providers and explicit caller deadlines retain their prior behavior. - Pi requires explicit operator setup with `paperclipai runtime setup pi` on the execution host. This PR does not add automatic downloads. - Linux ARM64, Windows, musl, and unqualified targets retain Legacy. - Legacy thinking levels minimal, medium, and xhigh cannot be translated exactly. Native max thinking, warm lifecycle, or restricted permissions cannot be dropped during a return to Legacy. - Copilot’s bound profile advances to v58 with exact current launch admission; v57 remains decodable for recovery. No historical identity is rewritten. - Existing agents and immutable run descriptors are not migrated. Cloud Pi availability is unchanged. ## Model Used OpenAI GPT-6 through Codex with reasoning, code editing, command execution, and browser use. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-21 17:10:00 -05:00
DottaandPaperclip	92cfd701c3	feat: add governed Braintrust app connections (#17141) ... Add hosted US and EU Braintrust OAuth and API-key connections through the existing governed MCP system. Keep reviewed tool risks and catalog review policy in connector modules and a narrow registry. Preserve credential ownership, operator decisions, regional setup and changed-tool quarantine. Include official artwork, setup guidance, focused regressions and sanitized qualification evidence. US OAuth has bounded live read/write, gateway, reconnect and cleanup proof. Remaining account-method and token-lifecycle qualification gaps are documented. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-21 16:19:58 -05:00
DottaandPaperclip	efc2ddc7bf	fix: stop idle login cleanup polling (#17106) ... **Dependency landed:** PR #17129 is merged. This branch is rebased onto master `0f15f82b19` and preserves its provider-owned completion contract. The shared save path calls `notifyLoginSessionChanged(tx)`; the cleanup module invalidates both existing sandbox cleanup workers without selecting a provider. ## Thinking Path > - Paperclip manages work for AI agents. > - Subscription login can allocate a temporary sandbox. > - Existing cleanup workers scan the database every heartbeat tick, even with no login work. > - Empty scans keep idle databases active. > - This PR uses committed session changes and saved expiry times to schedule cleanup. > - Existing rows still drive recovery after failure and restart. ## Linked Issues or Issue Description **What happened?** On master `898144f4b3`, the enabled heartbeat scheduler runs both login reapers every 30 seconds by default. An instance with no pending logins still queries sessions and leases. **What did you expect to happen?** After startup reconciliation, idle login cleanup should make no recurring SQL or provider calls. New sessions, expiry, failed cleanup and restart must still recover. **Steps to reproduce** Start a claimed instance with heartbeat scheduling enabled and no pending login sessions. Observe repeated adapter-login and setup-token cleanup queries at the heartbeat interval. Related sleep migration: #17094. Searched open issues and PRs. Related setup-token cleanup work: #15774, #14236 and #16648. This PR does not change their release-result or null-lease behavior. ## What Changed - Register separate adapter-login and setup-token workers with the existing delivery coordinator and scheduler. - Notify before writes in the owning stores, the lease-tag write, and AI connection save. Shared save uses a provider-neutral notification; each cleanup worker checks its own durable predicates. The shared transaction protocol retains intent until the outer transaction settles. - Reconstruct session expiry, promotion expiry and concrete cleanup retries from existing rows. Preserve configured heartbeat spacing to bound scan bursts. - Recover tagged orphan leases even with terminal or missing sessions. Protect live acquisitions that have not attached their lease yet. - Exclude local-subscription rows from sandbox reapers. Their existing local-login worker owns cleanup. - Preserve deferred work across drain and shutdown. Add no tables, Cloud alarms or host-sleep exemptions. - Update the inventory and document producer coverage, writer scope and recovery boundaries. ## Verification - Updated head: `1d57e13606`. Seven focused suites passed (282 tests): AI connections, login cleanup, device-login service/reaper, login-session runtime, and setup-token session/reaper. `pnpm -r typecheck` and `pnpm build` passed. The duplicate local `pnpm test:run` was interrupted after CI passed all server shards; it is not counted as a complete local pass. Full-suite evidence comes from the exact-head CI run. Greptile reviewed this exact head at 5/5 with no findings or unresolved threads. [Full CI run 39669285577](https://github.com/paperclipai/paperclip/actions/runs/39669285577) passed on attempt 2. Both aggregate gates passed; no checks remain failed or pending. - Extended the real PostgreSQL AI connection tests for both Anthropic and OpenAI. The tests check that neither cleanup observer wakes inside the completion callback, both wake after commit, and an idempotent save does not emit another wake. - Existing cleanup tests cover idle/history states, producer wake, expiry, promotion, cancellation, retry/restart, orphan recovery, nested commit/rollback, drain, concurrency, and shutdown. - Current-head CI attempt 1: the unchanged `sandbox-callback-bridge.test.ts` HTTP/2 preface test failed because its child process could not bind `127.0.0.1:34033` (`EADDRINUSE`). Its single unchanged rerun passed. The exact test also passed locally without changing code or timeouts. This establishes recovery from the bind collision; it does not establish which process owned the port. No login code ran in that package test. - Previous-head failures remain recorded: the login-runtime transaction fixture was corrected; a workspace-runtime readiness port-bind failure passed in isolation and fresh CI, but its first failure cause remains unconfirmed. An earlier combined AI suite timed out in local-login isolation and passed alone; its timing cause remains unconfirmed. An earlier local full suite was interrupted and was not counted as passing. ## Risks - Process-local signals cover one active server with the instrumented shared database handle. Independent SQL writers or replicas need their own signal integration. - Provider resources that never became persisted tagged leases remain the responsibility of the unchanged general lease/spool recovery. Setup-token acquisition retains its required provider expiry. - Cleanup starts are bounded by the configured heartbeat interval. Writes during a sweep can cause a final follow-up scan before quiet. - Existing login and resource blockers still prevent unsafe host sleep. This is worker polling removal, not provider sleep qualification. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools and test execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes:` / `Closes` / `Refs` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal/instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-21 15:39:23 -05:00
DottaandPaperclip	0f15f82b19	feat: offer existing Claude and Codex agents a runner upgrade (#17122) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Each agent stores its harness, model, account and runtime configuration. > - New Claude and Codex agents now use Paperclip Runner on qualified targets. > - Existing agents keep their legacy runner and need a clear upgrade action. > - The existing configuration API already translates settings and records revisions. > - This pull request adds an explicit upgrade action that uses that API. > - Operators can retain supported settings and correct rejected settings in place. ## Linked Issues or Issue Description **What existing behavior does this improve?** Existing Claude and Codex agent configuration. Related defaults: #15683 and #15993. **Current behavior** Existing agents retain their legacy runner. The operator must find the runner selector in Advanced to change it. **Proposed behavior** Show Upgrade runner when Paperclip Runner is the qualified default for the saved environment. One click saves pending edits and requests the existing server conversion. Show any rejected settings without changing saved configuration. **Reason and benefit** Operators can adopt the current runner while retaining their harness, model, account bindings, instructions and history. Unsupported settings remain visible and editable. **Breaking changes** None. Visiting configuration and saving unrelated changes do not upgrade an agent. The server API, authorization and conversion rules are unchanged. ## What Changed - Add an upgrade notice, pending state, saved result and retry feedback to the production configuration form. - Include pending profile, runtime and environment-variable edits in the upgrade request. - Bind asynchronous upgrade feedback and save receipts to their original agent. Navigating away cannot show success on another agent or redirect back. - Keep the notice absent for native agents, other harnesses, unavailable targets and failed discovery. - Add configuration/API regressions and database session-reset checks for Claude and Codex. - Add 13 production-page Storybook states with browser checks in both themes and on mobile. - Update the product contract and feature/access maps. ## Verification - Configuration form and original-agent save receipts: 135 tests passed. Existing agent route/rename helpers also pass. - Runner selection, configuration API and database migration regressions: 220 tests passed. - Storybook browser checks: 26 passed. Interactive cases wait for Storybook's visible Pass report. Static cases check the rendered production form. These use simulated HTTP boundaries. - Full typecheck, build, map checks and token gates passed on the current head. Storybook build and public preview verification passed. - Published fixture preview: https://storybook.paperclip.page/storybook/branches/codex~2Fagent-runner-upgrade/?path=/story/agents-runner-upgrade--legacy-codex - All 58 current-head PR checks passed, including the full test shards, browser suites, build and aggregate gates. The duplicate local pnpm test:run was stopped after 34 minutes once the same-head full CI suite passed; it had not finished its general-server phase. No local full-suite pass is claimed. - Real disposable local test drive: Codex used gpt-5.4 and Claude used claude-sonnet-4-6. Both completed one task on Legacy, upgraded through the browser button, and completed another task on Paperclip Runner with the same model and API credentials. Independent file checks confirmed saved instructions and environment bindings before and after conversion. - Reopened both configurations and compared identity, model, credential references, instruction hashes, working directories, run policies and configuration revisions. - A Claude Chrome setting produced a field-specific rejection. The saved agent stayed Legacy. Clearing Chrome in the form and clicking Upgrade runner saved the pending correction and completed conversion. ## Risks - The existing translator rejects settings that the new runner cannot honor. The notice shows that error and leaves the form available for correction. - Converted agents start fresh sessions. Prior runs remain available. - Real task execution was checked on local macOS with API credentials. SSH, sandbox targets and subscription credentials were not exercised in this change. - Storybook responses simulate persistence. The real test drive and database tests provide separate persistence evidence. ## Model Used - OpenAI GPT-6 through Codex. The exact serving variant and context-window size were not exposed in this session. Used reasoning, code editing, shell tools and browser interaction. No subagents. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-21 15:13:42 -05:00
DottaandPaperclip	39e44a9d9b	Default new OpenCode agents to Paperclip Runner (#17120) ... Default new OpenCode agents to Paperclip Runner on qualified macOS and Linux x64 targets. Preserve saved runner choices, explicit Legacy selection, and unsupported-target fallback. Pi remains a separate change. Verified the complete browser path with a real OpenRouter connection: create the new agent, assign a task, observe its response and Done status, and reload to confirm persistence. CI passes and Greptile is 5/5 with no unresolved findings. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-21 15:09:15 -05:00
DottaandPaperclip	0a5f1546ee	feat(cases): add reusable prompt workflows (#17114) ... ## Thinking Path > - Paperclip manages AI agents and their work. > - Cases keep facts, documents, and one primary task across agent handoffs. > - Each case currently needs its own procedure text. > - Agents need to reuse procedures and move cases between them without replacing the task. > - This change adds company workflow definitions with a name and Markdown prompt. > - Existing handoff and document transactions preserve authority, revisions, and recovery. > - Cases remain behind `experimental.enableCases`. ## Linked Issues or Issue Description Refs #16959. Related proposal: #3951 uses task graphs and a visual editor. This change extends existing Cases with prompt definitions and keeps its current handoff model. **Problem or motivation** Agents cannot share a reusable procedure across cases or inspect a case's movement between procedures. **Proposed solution** Add name-and-prompt workflow definitions. Adopt a prompt into the primary task's revisioned workflow document through a handoff. Keep phases as free text. **Alternatives considered** Defer graphs, workflow versions, run entities, role schemas, authoring screens, and Spec Kit integration. **Roadmap alignment** This extends the existing Cases experiment and same-task review flow. It adds no work queue or new execution engine. The maintainer approved this scope in the implementation plan. ## What Changed - Add company-scoped definition storage, shared request schemas, REST routes, and four optional Cases tools. - Accept a workflow ID at kickoff. Agent handoffs can adopt, refresh, preserve, or detach a workflow. - Capture prompt text and the document base revision at acceptance. Apply workflow, phase, and document revision together after outgoing execution settles. - Hold conflicting document edits for recovery. Preserve cancellation, replay, and restart behavior. - Show current workflow names and captured transition names/phases in Work, history, and Audit. - Update instructions, API/tool contracts, capability maps, and the isolated workflow journey. - Pin bounded workflow prompt reads to a content fingerprint; continuation conflicts on edits and can restart safely. - Remove experimental Cases/workflow/Mermaid guidance from the always-on native prompt. Extract runtime-context prompt strings into a CODEOWNERS-protected constants file while preserving the public imports. - Refresh Copilot’s pinned source identity to v57, including the extracted prompt file and retaining historical identities. ## Verification Local checks: repository `pnpm -r typecheck` and `pnpm build` pass on the prompt-extraction tree (`88222d45d8`). Full repository coverage previously completed through the supported isolated shard groups (1,082 general server suites, 161 serialized suites, and all three workspace groups). The earlier default serial run was interrupted and remains incomplete. The first sharded run retained the failures described below; targeted corrections and separate limited-concurrency reruns pass. All CI jobs, security checks, and the 5/5 Greptile review passed on `fc1bd79c28`. Fresh CI and review are pending for the prompt extraction in `88222d45d8`. Focused verification passes: real-database handoff recovery and reuse; 2,002 native-tool/API contract checks; 144 Runner catalog/discovery checks; 27 Runner context/coverage checks; 37 case/Audit UI checks; and Product E2E oracle calibration. Token gates, migration safety/numbering, module boundaries, capability maps, and generated tool/Storybook catalogs pass. The final route/context corrections pass 49 API/native-tool tests, 90 real-database handoff tests, and server typecheck; 142 Copilot installation/sidecar tests pass with the refreshed identity. Greptile’s first review found that successive prompt pages could mix definitions after an edit. The fix returns a content fingerprint, requires it for continuation, and rejects edits with 409. Strict live/catalog schemas pass (171 Runner checks). The 2,005 API/OpenAPI tests pass; the initial native regression caught compression’s weak ETag prefix, and all 23 native integration tests pass after handling it. The regression covers same-length edits and a four-byte restart window. No stored version fields or entities were added. The prompt extraction passes all 462 checks across six focused suites, including persisted-context recovery and provider transport. The first local transport attempt lacked the required debug Runner binaries (102 failures, retained); after building the normal prerequisites, all 283 tests in that suite pass without source or assertion changes. Existing saved snapshots retain their recorded prompt text, and new executions use core-only prompt v7. The extracted prompt source is now covered by both CODEOWNERS and Copilot’s source fingerprint. Live-provider evidence is separate from those simulated checks: the isolated API-key campaign `reusable-workflows-api-final-2026-10-21` passed with local code matching `a1d73b4652`. Four actual Codex API runs read the adopted task documents, moved one case A → B → A on the same task, completed normally, and passed all nine durable checks. Browser assertions verified the current workflow/phase, historical names/phases, five handoff records including terminal completion, and the completed original task. Cleanup passed. The local report remains under `tests/runner-e2e/results/reusable-workflows-api-final-2026-10-21/`; no credentials or screenshots are committed. CI initially caught an omitted route-inventory entry and a stale pinned instruction-source hash; both are corrected and separately verified. Local serialized coverage also recorded an unrelated environment-route socket disconnect and activity-route timeout; their separate 11-test and 17-test suites passed without code or timeout changes. An unchanged chunked-webhook check also passed its separate four-test rerun. The initial adapter/database workspace groups hit time and resource limits under default worker counts; complete reruns with `VITEST_MAX_WORKERS=2` pass, including all 55 database files and 211 tests. The UI/CLI groups passed 837 files and 9,442 tests in the original run. Original failures remain retained and are not regraded. Earlier failed live campaigns remain unchanged: subscription HTTP 429 before agent execution; an obsolete exact-four-row oracle that rejected the valid terminal fifth row; and a chat-marker expectation that did not account for the existing explicit-comment presentation policy. The final fixture requires terminal completion and places the marker in its authoritative comment. Positive/negative oracle tests and the fresh live campaign pass. ## Risks - The additive migration adds a table and nullable case/handoff fields. Existing rows keep null workflow metadata; no assignments are inferred. - Adoption can wait on an intervening task-document edit. Recovery must cancel the old intent and accept a new one after readback. - Shared definition edits affect future adoptions. Existing case copies keep their revisions. - The feature remains experimental. This PR does not merge or deploy the change. - Live acceptance covers native Codex before the prompt extraction; this cleanup has contract/transport verification rather than a new live-provider campaign. The refreshed Copilot identity has static compatibility evidence, not new live-provider qualification. ## Model Used OpenAI Codex, configured model ID `gpt-6-astra`, xhigh reasoning. Used terminal execution, file editing, and tool calls. The active context-window size is not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-21 15:00:12 -05:00
DottaandPaperclip	159d006bb4	refactor: let login flows own session completion (#17129) ... ## Thinking Path > - Paperclip manages AI agents and their connected accounts. > - Login flows bind verified credentials to an owned connection. > - The shared save service currently uses the provider name to choose a session ID column and completion state. > - A provider can support more than one login flow. Provider identity does not define session identity or state. > - This PR gives every flow the same internal session ID contract and lets the setup-token owner complete its own claim. > - Credential writes, session binding, and completion remain in one transaction. ## Linked Issues or Issue Description Refs #17106. This is an independent prerequisite for that sleep cleanup PR. It contains no scheduler or cleanup-worker changes. **Current behavior** The connection save service looks up Anthropic sessions by public ID and all other sessions by internal ID. It also marks every Anthropic session as stored. Local Anthropic login works because its public and internal IDs happen to match, and its caller overwrites that intermediate state. **Proposed behavior** Every caller passes an internal session ID. A small setup-token completion service resolves the public ID within its owner scope and supplies a transaction callback. That callback uses the existing conditional stored transition. Device and local login flows keep their own final states. **Reason and benefit** Adding or changing a login flow no longer requires provider branches in the shared session binding path. The service still owns membership checks, row locking, target validation, credential writes, and idempotent binding. The flow owns only public-ID resolution and its completion transition. **Breaking changes** The internal TypeScript save signature changes. All production callers are updated. Public HTTP payloads, database schema, and local session URLs stay unchanged. ## What Changed - Replace the ambiguous session ID argument with an internal session reference and an optional completion callback. - Run the callback inside the existing save transaction, only for the first successful binding. - Move public-ID resolution and the stored transition to setup-token-ai-connection.ts. - Update setup-token, device, and local callers. - Add real PostgreSQL tests for ID selection, owner and target checks, one-time completion, and rollback after a rejected completion. - Exercise local Anthropic and OpenAI HTTP completion with distinct public and internal IDs. - Document the ownership contract in AI-CONNECTIONS.md. ## Verification - Passed: pnpm -r typecheck. - Passed: pnpm exec vitest run server/src/__tests__/ai-connections.test.ts server/src/__tests__/agent-device-login-routes.test.ts server/src/__tests__/agents-claude-oauth-binding.test.ts — 233 tests. - Passed: pnpm build. - Passed: full CI, including test shards, typecheck, build, and E2E, on d98ce6e504: https://github.com/paperclipai/paperclip/actions/runs/39654210168. - The additional local pnpm test:run was stopped after the full CI pass to avoid duplicating the completed run. It is not reported as a completed local full-suite pass. - Greptile: 5/5 on d98ce6e504, with no actionable findings or unresolved threads. - The first AI suite run had two new fixture failures: a duplicate environment name across provider cases, and an assertion that ignored existing shared connections. The fixtures now use distinct names and query the attempted connection directly. The corrected affected suites passed. The first CI run passed all 1,420 assertions in server shard 11/12 but failed on an unhandled child-process rejection in `workspace-runtime-exposure.test.ts` (the synthetic EADDRINUSE case). Both the test and runtime implementation are unchanged from this PR's base. The two affected parameter cases passed locally unchanged (2 passed, 29 skipped). Inspection shows a possible timing window before the early-exit promise joins the readiness race; that cause has not been reproduced conclusively. The single CI retry passed on the same commit. This demonstrates recovery; the original timing cause was not conclusively reproduced. Original job: https://github.com/paperclipai/paperclip/actions/runs/39654210168/job/119410225203. ## Risks - A completion callback failure must roll back the credential and session binding. The real-database regression test proves this for a rejected setup-token claim. - The callback is internal, awaited inside the existing transaction, and must use the supplied transaction handle. It adds no network work or provider SDK dependency. - Setup-token resolution adds one indexed lookup per completion. The shared save then locks and validates the row again. No recurring work is added. - The setup-token owner now uses the existing conditional completion guard, including the unconsumed-claim and commit-time expiry checks. Invalid claims fail closed. - Live provider sign-in was not exercised; tests use fixture credentials and real PostgreSQL. No schema migration is needed. ## Model Used OpenAI GPT-6 through Codex, with reasoning, source inspection, code editing, and test execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-21 14:42:38 -05:00
Devin Foley	a1e2173ceb	feat(apps): add Ditto connection (#16472) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents reach external services through the Apps catalog. Each catalog entry is a reviewed `AppDefinition` that connects a provider's hosted MCP server to Paperclip's shared vault, grants, policies, gateway, and audit trail. > - Ditto (`dittowords.com`) is a single source of truth for product copy. Its hosted MCP server lets agents read style guide rules, search and reuse approved text, and create, edit, translate and delete text, components, variants and style guides. > - Ditto is not in the catalog. Operators must use the generic "Connect your own MCP server" flow. That flow has no branding, no guidance about Ditto's `token` API-key scheme, and no warning about permanent deletes. > - The generic risk classifier also reads several Ditto mutations as reads: `translate_*` replaces variant text, and `link_*` and `unlink_*` replace developer IDs. > - This pull request adds the Ditto definition, its artwork, the research and permission-review ledger rows, documentation, deterministic tests, and one reviewed risk-classification rule. > - The benefit is a branded, governed Ditto connection with browser sign-in, an API-key option that uses the correct header, and correct risk levels for every documented tool. ## Linked Issues or Issue Description **Problem or motivation** Product and engineering teams use Ditto to keep UI copy consistent. They want their Paperclip agents to follow the Ditto style guide and reuse approved strings when they write code. Ditto is not in the Apps catalog. Operators must paste the MCP URL into the generic remote-MCP flow. That flow gives no branding and no method guidance. It also does not tell the operator that Ditto API keys use `Authorization: token <key>`, not `Bearer`, and it does not warn that delete tools are permanent. The generic classifier also marks five Ditto mutations as reads, so the Permissions screen shows them with the wrong risk level. **Proposed solution** Add a catalog-only Ditto connection built from the connector playbook. Browser sign-in uses dynamic client registration on Ditto's Auth0 authorization server and requests the reviewed scopes `openid`, `profile`, `email` and `offline_access`. An optional method sends a customer-created workspace API key with the `token` scheme. Both methods share one `write` capability profile, so the delete and link warning shows before sign-in. Add a reviewed `ditto` rule to `classifyRisk`: `list_*`, `get_*` and `search_*` tools, `suggest_edit`, and tools that Ditto marks read-only are reads; `delete_*` tools are destructive; all other tools are writes. **Alternatives considered** A plugin was not needed because no custom UI, tables, workers, or webhooks are involved. Requesting no scopes was not chosen, because Auth0 issues a refresh token only for `offline_access`, so the connection would expire with its first access token. Ditto's separate documentation MCP server (`developer.dittowords.com/mcp`) was not added, because it only searches Ditto's public docs. **Roadmap alignment** This extends the existing self-serve remote-MCP connection catalog and does not overlap planned core work. ## What Changed - Added the `ditto` provider to `scripts/ingest-app-definitions.mjs` (category `content`, `token` API-key placement, guidance, warnings, shared capability profile, description) and regenerated `packages/shared/src/app-definitions/ditto.json` and the generated registry. - Added the Ditto row to the self-serve MCP research ledger with `dcr_or_api_key` auth and risk tier S4, because the normal tool catalog includes permanent deletes. Widened `SelfServeMcpResearchEntry.wave` to include `4`, which existing entries already use. - Added permission reviews for `ditto/mcp-oauth` (explicit scopes `openid`, `profile`, `email`, `offline_access`) and `ditto/mcp-api-key` (provider key), with evidence links. - Added a reviewed `ditto` rule to `classifyRisk` in `server/src/services/tool-access.ts`. - Added Ditto's mark (`ui/public/brands/apps/ditto.png`, the avatar of Ditto's official GitHub organization) and the brand manifest entry. - Added gallery copy for the Ditto card. - Added `doc/connections/DITTO.md` (sign-in sequence diagram, provider and Paperclip endpoints including `/api/tools/oauth/callback`, scopes, administrator setup, capabilities and policy, manifest, brand provenance, validation hook) and linked it from the connections README and the permission audit. - Tests: Ditto definition shape, store visibility and artwork, URL recognition, reviewed scopes, `token` header projection with the key kept out of connection config, risk classification of every read-looking Ditto mutation, the connect form's sign-in default, the warning before sign-in, API-key gating, and the pinned catalog counts. ## Verification - `pnpm exec vitest run packages/shared/src/app-definitions.test.ts packages/shared/src/app-definitions-url.test.ts ui/src/pages/apps/AppsConnect.test.tsx ui/src/pages/apps/Browse.test.tsx ui/src/lib/app-brand-assets.test.ts ui/src/pages/apps/AppLogo.brand-assets.test.tsx server/src/__tests__/tool-access-service.test.ts` — 765 passed. - `node scripts/check-app-brand-assets.mjs` and `node --test scripts/app-brand-validation.test.mjs` — passed. - `pnpm --filter @paperclipai/shared typecheck`, `pnpm --filter @paperclipai/server typecheck`, `pnpm --filter @paperclipai/ui typecheck` — clean. - Manual: on a local instance, open Apps → Browse and confirm the Ditto card and icon. Open `/apps/connect?source=ditto` and confirm that sign-in is the default, that the warning shows before sign-in, and that the API-key method is under Advanced. - Live metadata probe: an unauthenticated `initialize` on `https://api.dittowords.com/v2/mcp` returns 401 with `resource_metadata`. The protected-resource and authorization-server documents return the recorded endpoints. Dynamic client registration succeeds, and the authorize endpoint redirects to Ditto's sign-in page with the reviewed scopes and the `resource` parameter. - Each of the 41 documented tool names was run through `classifyRisk` with and without the new rule. Without the rule, `translate_text_items`, `translate_library_components`, `link_figma_nodes_to_text_item`, `link_text_items_to_component` and `unlink_text_items_from_component` were reads. ## Risks - Low risk to existing providers: the catalog change is additive, and the new `classifyRisk` branch runs only for the `ditto` source template. - Ditto delete tools are permanent, and linking a text item to a component replaces its developer ID. Both methods warn the operator before connecting, and the API-key helper text tells operators to set delete and link actions to Ask first. - Ditto API keys are not scoped, reach the whole workspace, and act as the user who created them. Paperclip cannot narrow an issued key. - The permission-review ledger records live proof for both methods as not run. The lifecycle checklist in `doc/connections/DITTO.md` still needs a documented pass with a Ditto workspace, including confirmation that Auth0 issues a refresh token that `/v2/mcp` accepts. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`, 1M context) in Claude Code, with tool use (shell, file editing, web research). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-10-21 11:29:48 -07:00
3 people	6dda8cc2b7	fix: preserve Stop authority and qualify complete recovery journeys (#17105) ... ## Thinking Path > - Paperclip lets people manage agents that do useful work. > - Users need complete tasks, clear questions, retained context, and dependable recovery. > - Separate run tests do not prove that the complete user experience works. > - Successful retries can hide earlier failures and missing evidence. > - This PR extends the existing Product E2E harness with three explicit journeys. > - It adds a qualification checkpoint over the full retained result pool. > - Independent artifacts, state, rendered conversations, and delivery receipts establish each outcome. ## Linked Issues or Issue Description Refs #17088, #17089, and #17046. This PR repairs two Stop lifecycle defects and combines the qualification changes previously reviewed in #17095, #17090, and #17099. Daily operations remain with #17051. It adds no second scheduler or recovery engine. **What happened?** A completed run can still leave a user without useful output, an understandable question, or a working continuation. Existing isolated tests and best-attempt reports do not establish the combined behavior. **Expected behavior** A necessary question includes its explanation and resumes once after an answer. Stop prevents automatic continuation until the user resumes. Restart during finalization preserves the accepted provider result and delivers it once. Qualification requires independent evidence for six journeys on both agents through Slack and Paperclip. **Steps to reproduce** List the necessary-question, followup-stop-resume, and finalization-recovery suites using the documented selectors. Run one exact cell with the configured model and target. Supply a private qualification manifest to the existing report command to grade all retained attempts. Missing proof and incomplete coverage must fail. ## What Changed - Wake existing accounting in the same committed native Stop closure. A receipt that arrives before closure must still settle once; incomplete receipts remain held. - Require a later user message before native Stop can promote earlier saved input. Preserve those earlier messages for the authorized resumption. - Add explicit necessary-question cells. Verify the actual prompt and context, answer, one bound continuation, and the delivered document link and revision. - Add explicit follow-up, Stop, and later resumption cells. Verify a live owned provider turn, cancellation acknowledgement, at least five seconds without an automatic path, retained direction and workspace context, and an actual browser download. - Add explicit Daytona finalization-recovery cells. Stop the owned controller after public result acceptance and during workspace finalization. Require the original result, restored file hash, resumed operation, one provider turn, and one receipt-bound delivery. - Share the receipt-based delivery observer across both recovery journeys. Keep model prose separate from the authoritative delivered comment identity. - Add a 24-cell qualification checkpoint over every retained attempt. Require three fresh candidate passes per cell and later staging proof with exact source, runtime, model, settings, instructions, and independent evidence. - Reject absent or unsafe manifest evidence. Preserve failed attempts and incomplete observations in the normal reports. - Provide four explicit rotating daily selections. Refuse daily selection until all required live drivers exist. Keep scheduling and spending with the existing operations owner. - Document the exact selectors, evidence contract, limitations, and remaining staging work. ## Verification - Local Product E2E support: 3,554 passed with two existing skips; source prerequisites: 128 passed. E2E typecheck passes. The actual file-chip and Download-control browser regression passes. - Two real-database regressions fail before the production repair. The stopped coordinator leaves accounting asleep after a complete receipt; the saved queue can restart from pre-Stop input. The repaired settlement and cancellation cohort passes 66 checks. The complete 256-case continuation suite passes with stronger two-phase Stop expectations, and 66 native executor accounting/Stop safety cases pass. Server typecheck passes. - All required CI, repository build/typecheck/tests, and fresh Greptile 5/5 pass at the final head 232d3c52d0, with no unresolved findings. - Both actual-model necessary-question development cells passed on the earlier recorded c4e409d819 source. This is supporting evidence for that revision, not current qualification. - Both fresh Stop development cells at 6036cee2f4 stopped, remained idle, resumed once, reached Done, and settled original and successor accounting. Their original machine grades remain failed: the observer counted the generated file-preparation carrier as another final reply. The corrected observer independently correlates the applied preparation receipt with the actual attachment/comment binding. Retained-wire diagnostic replay finds one final delivery for each model without changing either grade. Those original failures remain retained: https://github.com/paperclipai/paperclip/actions/runs/39644220458. - Both fresh actual-model Stop/resume cells pass all five checks at 232d3c52d0: https://github.com/paperclipai/paperclip/actions/runs/39647010372. Independent inspection confirms exactly one cancelled original and one committed successor on the same task, 5.091/5.513-second quiet intervals before the user resumes, retained workspace/instructions/tool identities, and one receipt-bound final response. Each visible JSON chip opens in a tab and its Download control yields verified bytes. The rendered reference and changed label independently reconstruct the registered file hash and the original context hash. Both cleanups pass. These remain local API-key/per-turn development results, not the required staging/Slack qualification. - Other retained Stop failures include missed active boundaries, the real accounting admission failure, and a preparation observer that required nativeIssueId before runtime selection. They remain recorded at https://github.com/paperclipai/paperclip/actions/runs/39636796645 and https://github.com/paperclipai/paperclip/actions/runs/39642060165. The preparation observer now validates all present task IDs and still requires selected native execution at the active checkpoint. - The remote finalization attempt failed during image export with no disk space before a provider started. Remote crash acceptance remains unverified. - Negative calibration rejects wrong or absent ownership/evidence, stale settings, unsafe files, repeated admissions, unacknowledged Stop, premature resumption, incomplete receipts, lost context, genuine duplicate final replies, and missing delivery. A prepared-looking body cannot suppress a comment; absent, rejected, foreign, or mismatched file receipts and attachment bindings also cannot suppress it. - Core staging/Slack drivers, 72 fresh candidate passes, deployed staging proof, and recovery of the original user-visible task remain incomplete. No measured reliability percentage or completed qualification is claimed. ## Risks These are explicit development cells. Their API-key and per-turn runs do not qualify deployed subscription authentication, warm reuse, or Slack. A fast provider can miss the Stop boundary; that attempt fails. The remote crash boundary depends on independently observed public state and an owned controller. The checkpoint is opt-in through a private manifest and cannot replace the missing live observers. The two production changes affect native Stop settlement and saved-input promotion. They use the existing transaction signals and admission path. A complete receipt can settle without another provider turn. Partial accounting and all ownership, cleanup, pause, and budget gates remain authoritative. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The runtime does not expose its exact deployment ID or context-window size. Those values are not inferred. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Paperclip <paperclip@paperclip.ing>	2026-10-21 13:02:27 -05:00
Nicky LeachandPaperclip	42715d459f	Support authenticated WebSocket routes and agent views in plugins (#17111) ... ## Thinking Path > - Paperclip manages agents and their work. > - Plugins provide optional views and connect to external runtimes. > - A live runtime view needs a binary browser connection. > - The current plugin API supports JSON routes but cannot proxy that connection. > - Agent detail tabs are declared in the plugin contract but are not mounted on agent pages. > - This PR completes those two host capabilities and keeps upstream credentials on the server. ## Linked Issues or Issue Description **Subsystem affected** Plugin scoped API routes and agent detail views. **Problem or motivation** Plugins cannot expose a binary or text stream through a browser session without exposing an upstream credential or adding a separate browser gateway. Agent-scoped detail contributions also have no mount on the agent page. **Proposed solution** Allow declared board GET routes to select a server-only WSS upstream. Run upgrades through the existing HTTP authentication pipeline and trusted Origin check. Add agent company resolution with the existing `agent:read` decision. Mount contributed detail views in agent navigation. **Alternatives considered** A separately hosted viewer needs its own browser credentials and cross-origin policy. JSON or SSE transport adds encoding and buffering to a binary protocol. **Roadmap alignment** This extends the existing plugin API and detail-tab contract. It adds no desktop provider to Core. I checked ROADMAP.md and searched related open PRs; I found no equivalent transport and agent view change. ## What Changed - Add declared WebSocket routes requiring board authentication, GET, and `http.outbound`. Reject body-based company resolution because browser upgrades cannot send a body. - Keep worker-selected upstream URLs and headers out of browser JSON responses. Strip browser cookies and Origin. Preserve binary frames and socket backpressure. - Close both sockets on disconnect or worker shutdown. Reauthorize on reconnect after a five-minute connection lifetime. Preserve HTTP idle-work accounting for the stream lifetime. - Resolve agent scope through `agent:read` and mount generic agent plugin detail views, with current agent/view breadcrumbs and browser titles. ## Verification - Final head `9d8d269040acf9dc7bb65524876406af522dacb5`: CI passes (including Runner Rust, build, and browser E2E); Greptile 5/5; no unresolved review threads; no merge conflicts. Verified 2026-10-21. - Core server TypeScript check: `node node_modules/typescript/bin/tsc -p server/tsconfig.json --noEmit`. - Focused server suites: 17 passing tests (`server/src/__tests__/plugin-scoped-api-routes.test.ts` and `server/src/http/server.test.ts`), including arbitrary build-output and metrics channels with real local text and binary WebSocket round trips, disconnect cleanup, denied actors and origins, credential exclusion, agent-read denial, and cross-company 404 equivalence. The test replaces TLS with local HTTP while retaining the real upgrade and binary stream. - UI suites: 9 passing tests for independent Insights and Notes contributions, removed contributions, built-in route preservation, sidebar links, direct plugin entry and heading updates after tab navigation. These contributions do not require the WebSocket transport. - Shared validator suite: 42 passing tests, including rejecting body-based WebSocket resolution while retaining ordinary HTTP body resolution and accepting query/entity scopes. - UI token gates pass. - UI TypeScript check: `node ../scripts/storybook-catalog.mjs` then `tsc -b`. - Shared and plugin SDK builds pass. - Full Runner build reached its unrelated Rust step and stopped because cargo is unavailable. Runner TypeScript build was run separately. No live remote VM was used. ## Risks - This adds a browser ingress path for trusted installed plugins. The plugin must select its upstream from a trusted binding. HTTP requests cannot retrieve the upstream descriptor. - Membership or binding changes are rechecked on reconnect. Connections last at most five minutes, and worker shutdown is checked every five seconds. - Initial CI caught a missing cross-company 404 guard and an old test that assumed exactly one upgrade listener. Both were corrected. Greptile's body-resolution and stale-heading findings are fixed with regression coverage; latest-head CI passes and Greptile scored 5/5 with no unresolved findings. - No schema migration. Rollback restores the previous Core image; plugins using this transport must be disabled or rolled back first. ## Model Used OpenAI Codex, based on GPT-6, with reasoning and code/tool execution. The exact runtime model ID and context-window size are not exposed in this session. ## Checklist - [x] Thinking path and issue description included - [x] Model described - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] No internal issues, private links, or runtime credentials included - [x] Descriptive branch name - [x] Focused tests and relevant typechecks run - [x] Tests and documentation updated - [x] Risks documented - [x] All CI gates green - [x] Greptile 5/5 with no unresolved findings - [x] I will address review findings before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-21 10:10:00 -07:00
3 people	507a0d7098	test: verify repository delivery after real runner conversion (#17089) ... ## Thinking Path > - Paperclip manages agents that perform useful work. > - Repository setup and Runner selection must preserve that work context. > - A successful provider turn cannot prove configuration retention or usable delivery. > - The repaired setup chain needs independent product checks with the recorded models. > - This PR adds explicit repository and conversion journeys to the existing Product E2E harness. > - The observer checks source bytes, Git identity, saved configuration, and downloaded output. > - Development and staging qualification remain separate. ## Linked Issues or Issue Description Refs #17088 and #17047. The complete repository setup repair in #17088 is merged. No competing conversion journey was found in the open PR search. **What happened?** Existing checks do not prove that a real legacy agent retains its configured context when switched to Paperclip Runner and then delivers verified repository work. The projectless repair check also uses different default models from the recorded staging models. **Expected behavior** A configured repository task delivers correct, accessible output. A real legacy-to-native conversion preserves configuration, instructions, environment, and equivalent permission policy before doing subsequent work. A projectless task continues through setup without supervision. **Steps to reproduce** List the configured-repository, legacy-conversion, and automatic-repository-configured-models suites. Run one exact ID from the README. These are explicit paid development checks, excluded from the default selection. ## What Changed - Add four configured-repository cells and four real legacy-conversion cells for Codex gpt-6.1-sol and Claude claude-opus-5-5 on local and Daytona environments. - Seed a fresh reference only in an owned disposable Git checkout. Compare the delivered JSON with its original bytes, hash, and commit. - Switch the same real legacy agent through the public Runner selection API. Compare allowlisted configuration and instruction digests before and after conversion. - Require one committed native run with the configured provider, model, and environment, a visible registered download after reload, Done, and no pending question. - Add four projectless replay cells with the recorded staging models and the unchanged strict continuation oracle. - Use production hire instructions and existing sandbox profiles, budgets, report pipeline, and cleanup. Record API-key and per-turn lifecycle differences explicitly. - Retain the required allowlisted API snapshot after successful automatic setup. Preserve lifecycle IDs, state, ordering, and receipt references without raw provider content. Verify both admissions retain the configured provider and model. ## Verification - Product E2E support after integrating the setup repair: 3,353 passed, two existing skips; source prerequisites: 128 passed. - Product E2E typecheck passes. Discovery lists all twelve new explicit cells. - Independent oracle calibration rejects wrong configuration, instructions, provider, model, source bytes, hash, commit, downloaded output, delivery, and pending questions. - The first full support run exposed a catalog count mismatch and an unprovisioned new profile ID. Both are fixed; the successful run uses the existing trusted sandbox profile IDs without changing workflow authority. - Review found fixture prerequisite defects: optional CLI-only Codex arguments and platform qualification after Runner selection. Both are corrected; the disposable Claude baseline also excludes its unsupported optional CLI turn limit. Seven new setup checks cover probe ordering, absent qualification, immutable images, local conversion, and uncertain probe accounting. The final focused cohort passes 86 checks and E2E typecheck passes. - The first qualification-regression run also had an incorrect test assumption about the legacy timeout; the corrected red reaches the unsupported extraArgs setting. No production conversion restriction was relaxed. - A retained four-cell development campaign delivered independently verified ordinary Codex output and real Codex/Claude conversion results with retained configuration. All three attempts nevertheless failed the required evidence contract because api-state.json was missing. The flow now saves allowlisted task/run lifecycle fields; it does not export prompts, comments, interactions, configuration contents, or provider outputs. The projectless cell delivered valid bytes but used a monitor fallback and failed the unchanged continuation oracle; its driver correction belongs to #17088. All four attempts remain failures; fresh exact-candidate acceptance is pending. These API-key/per-turn checks do not establish subscription authentication, warm reuse, Slack delivery, or the complete milestone. - Four fresh local ordinary-repository and conversion cells passed at `c4e409d819cc0a703736d3f1ed3552064ca690b8`. Independent downloaded artifacts and saved configuration evidence agree. - The repaired automatic setup chain passed all six journey checks for both models, but first failed because the success path omitted api-state.json. The separate capture repair now passes the fresh two-cell [campaign](https://github.com/paperclipai/paperclip/actions/runs/39633381139) at `7a7c26f6688ec6b0e0454562a43fd48ab1a81c97`. Both required API files and their run/intent/dispatch bindings were independently checked. Earlier failures remain failures. - After merging current master, 114 workspace/admission/continuation checks across six suites pass. The provider/model and evidence privacy cohort passes 65 checks, and E2E typecheck passes. The final model-binding assertion has unit calibration; fresh actual-model acceptance of that assertion is still pending. - Exact-head CI/review and complete staging acceptance remain pending. Local development passes do not establish the 24-cell milestone. ## Risks The real provider may expose additional conversion or setup defects. Failures remain evidence and cannot be replaced by catalog registration or a later retry. The conversion uses the production API and does not inject a native result. Only synthetic data is seeded in a resolved fixture-owned directory. Configuration evidence contains digests, not raw environment values. This PR changes evaluation code, not production behavior. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The runtime does not expose its exact deployment ID or context-window size. Those values are not inferred. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Paperclip <paperclip@paperclip.ing>	2026-10-21 11:51:50 -05:00
3 people	eece07a747	fix: repair the complete projectless repository setup chain (#17088) ... ## Thinking Path > - Paperclip lets people manage agents that do useful work. > - A projectless task can prepare an authorized repository in its retained workspace. > - Setup must hand work to a later run without user supervision. > - Real task attempts exposed gaps across admission, completion, and queue dispatch. > - A successful provider exit did not prove that the requested result was delivered. > - This PR repairs the existing lifecycle and checks the complete repository journey. ## Linked Issues or Issue Description Refs #17047. This PR includes the preserved repairs from #17049, #17052, and #17082. **What happened?** A projectless native task could stop after requesting a repository. It could also continue through a timer or generic recovery path and lose its saved conversation lineage. The latest real Codex and Claude attempts delivered correct files, but the native continuation had been folded into the finishing predecessor. Both attempts remain failed measurements. **Expected behavior** Prepare the authorized repository, retain the original context, and dispatch one saved continuation after its predecessor finishes. Deliver the verified result in the originating conversation. Stop, current access, approval, and real blocker rules remain authoritative. **Steps to reproduce** Run the explicit automatic-repository suite for native Codex or Claude. Inspect the downloaded result, original context artifact, both committed run records, and continuation receipts. The suite uses one attempt and does not widen the default paid selection. ## What Changed - Resolve the retained local-board identity through current company membership and workspace authority. - Prepare required source dependencies before the isolated server imports native tool authority. - Explain the saved repository continuation in production completion tools. Correct only the exact pending-checkout blocker backed by a live receipt. Reject an unfinished response wait without a real response source. - Preserve the persisted task-session workspace source and its explicit absence of root repository context. Permit Git's exact no-repository result only for a validated plain task directory. Keep other Git errors and linked metadata strict. - Revalidate server-resolved same-agent continuation values against current ownership and the saved kind, summary, and key. Carry this shape through the internal bridge schema. Keep provider-facing schemas restrictive. - Preserve terminal result authority after a correlated successful controller response. Controller rejection creates no result authority. Bind the shared guidance and transport source in Copilot v55 and preserve historical identities. - Admit native status wakes with the existing no-coalescing contract. The existing deferred queue retains a distinct future dispatch instead of retagging the finishing run. - Recover deferred conversation lineage through the actual admitted dispatch. Accept a null outbox run binding only with matching decision, effect, dispatch actor, idempotency, task, company, agent, and requester evidence. Reject a conflicting nonnull binding. - Independently grade retained random context, actual checkout bytes and HEAD, accessible output, distinct continuation receipts, and admission order. ## Verification - Real database and Git regressions fail before their repairs. Prior focused cohorts cover local-board authority, optional repository waits, plain task directories, and first/reused workspace provenance. - The new valid decision/effect-backed assembled-heartbeat regression reproduces coalescing into its active predecessor before the queue repair. With the repair, it dispatches one successor after the predecessor finishes. Duplicate sweeps create no extra run. - The deferred-outbox conversation regression fails before its repair. The repaired test also rejects a conflicting predecessor binding. - The current queue, continuation, publication, cancellation, stale-admission, and domain cohort passes 344 checks across 11 suites. Server and Product E2E typechecks pass. The strengthened automatic-repository oracle passes 34 calibration checks. - Three omitted/null/resolved forms pass through the compiled Rust runner and controlled provider. The prior shared bridge cohort passes 492 TypeScript checks and 399 Rust core checks. Runner source identity, TypeScript, Rust formatting, and locked Rust typechecks pass. - Six real local development cells passed ordinary repository, conversion, and necessary-question journeys. Independent artifacts and rendered question/result evidence agree. They do not qualify staging or Slack. - Both latest automatic-repository attempts delivered correct files but failed the strict continuation check. They remain failures. A fresh replay of this queue repair is pending. - On the combined candidate `380047f4b923807a8c4f7ef6016e4fe130ab4a3a`, `pnpm -r typecheck` and `pnpm build` pass. Product E2E support passes 3,525 checks with two existing skips, plus 128 source checks. The complete local `pnpm test:run` is running and is not yet a pass. - The first replay selection failed catalog validation due to an operator profile-ID error. A second selection used the wrong default-model suite and was cancelled before any provider job existed. Both setup attempts remain recorded. The corrected campaign selects the recorded GPT-6.1 and Opus 5.5 models plus the local Stop/resume journey. - Exact-head CI, review, and deployment acceptance remain pending. GitHub reports delayed PR updates and webhook delivery; the current head and checks have now propagated. No earlier-head green badge establishes acceptance of this commit. ## Risks This changes several boundaries of repository setup. Full product and deployment acceptance remain required. The queue uses the existing deferred admission and delivery contracts. Batch deduplication remains in place. The provenance repair requires the actual bound dispatch and does not grant publication authority. Current authorization is still checked at publication. No accepted provider result is rewritten, and completed provider work is not rerun to repair persistence or delivery. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The runtime does not expose its exact deployment ID or context-window size. Those values are not inferred. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Paperclip <paperclip@paperclip.ing>	2026-10-21 11:21:55 -05:00