PaperClipAI

0 Fork 0

mirror of https://github.com/paperclipai/paperclip.git synced 2026-10-06 10:48:12 +02:00

Author	SHA1	Message	Date
Devin Foley	f9173782cd	feat(release): add smoke-gated nightly channel and lane-separated Docker tags (#11006 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The release subsystem publishes the `paperclipai` npm package set and the Docker images on two lanes: canary on every master push, and stable on manual promotion > - There is no middle ground between those lanes. Users must track every merge or wait weeks for a stable. Docker `:latest` also tracks master, so Docker users have no stable image at all > - A calm prerelease lane needs to exist, and it must never ship a build that failed its checks > - This pull request adds the nightly channel: a scheduled job that selects the newest master commit with a green canary publish, runs the full release smoke suite against that exact published canary, and only then republishes it as the nightly. It also separates Docker tags by lane, so `:latest` finally means stable > - The benefit is that users can follow prereleases at a nightly cadence with a smoke-tested guarantee, and Docker users get real `:canary`, `:nightly`, and stable image tags ## Linked Issues or Issue Description Subsystem affected Release automation: `scripts/release.sh`, `scripts/release-lib.sh`, `.github/workflows/release.yml`, `.github/workflows/docker.yml`, `.github/workflows/release-smoke.yml`. Problem or motivation The project publishes only `canary` (every master push) and `latest` (manual stable). Users who want prereleases without per-merge churn have no option. Docker has a second problem: master builds overwrite `:latest`, and CI-published stables never produced Docker images, because tags pushed with `GITHUB_TOKEN` do not fire the `v` tag trigger in `docker.yml`. No stable-versioned image exists in ghcr today. Proposed solution* Add a `nightly` channel. A scheduled job selects the newest canary-tagged master commit, smoke-tests that exact published canary, and republishes the same commit as `YYYY.MDD.P-nightly.N` under the `nightly` dist-tag. Separate Docker tags by lane (`:canary` for master, `:nightly` for nightly tags, `:latest` plus version tags for stable tags only), and have the release jobs dispatch `docker.yml` at the new tag so lane images actually build. Alternatives considered Moving the `nightly` dist-tag to the existing canary version without a republish. Rejected: the version string would say `canary` while the user is on nightly, which breaks at-a-glance lane identification in bug reports and `--version` output. ## What Changed - `scripts/release-lib.sh`: channel-parameterized `next_prerelease_version` and `prerelease_tag_name` helpers (canary helpers delegate to them), a `require_channel_tag_at_head` guard, and the no-provenance retry for Sigstore transparency-log duplicates now covers the `nightly` dist-tag as well as `canary` - `scripts/release.sh`: new `nightly` channel. It requires HEAD to carry a `canary/v` tag, publishes the full public package set as `YYYY.MDD.P-nightly.N` under dist-tag `nightly`, and tags the source commit `nightly/vYYYY.MDD.P-nightly.N` - `.github/workflows/release.yml`: scheduled nightly chain (09:00 UTC) — select candidate, smoke it via `release-smoke.yml`, publish on green under the existing `npm-canary` environment, push the tag, dispatch `docker.yml`. New `channel` dispatch input (default `stable`, so existing stable dispatches are unchanged) with `nightly_source_version` and `dry_run` support for forced runs. The stable path now also dispatches `docker.yml` at the new `v` tag - `.github/workflows/docker.yml`: lane tag mapping for both image jobs — master pushes publish `:canary` and no longer move `:latest`; `nightly/v` tags publish `:nightly`; only stable `v` tags publish `:latest` and the versioned tags. New `workflow_dispatch` trigger for the release-job dispatches. Build-version stamping uses the exact nightly version on nightly tag builds - `.github/workflows/release-smoke.yml`: `nightly` added to the dispatch choice list - `doc/CHANNELS.md` (new): user-facing guide to the channels - `doc/RELEASING.md`: nightly lane documentation, Docker tag mapping table, and a nightly failure playbook - `doc/RELEASE-AUTOMATION-SETUP.md`: note that nightly reuses `npm-canary` and needs no npm trusted-publisher changes - Tests: channel-parameterized version helper coverage in `scripts/release-registry-versions.test.mjs`, and nightly flow coverage (publish identity, notes not required, canary-tag guard) in `scripts/__tests__/release-dry-run-notes.test.mjs` ## Verification - `node --test` on the release script suites: 68 pass, including 6 new tests. The only failure, `acpx-patch-packaging.test.mjs`, needs installed `node_modules` and fails identically on a pristine checkout of master in the same environment - `bash -n` on both shell scripts and YAML parse of all three workflows - Live fail-path check: `./scripts/release.sh nightly --print-version` from a master tip with no canary tag fails with `HEAD has no canary/v* tag` - Live success-path check: the same command from the `canary/v2026.806.0-canary.7` commit prints `2026.806.0-nightly.0` - Live selection check: the candidate-selection shell logic run against the real repository selects the commit of `canary/v2026.806.0-canary.7`, which matches the current npm `canary` dist-tag exactly - After merge: dispatch `release.yml` with `channel: nightly` and `dry_run: true` to preview, then a real forced run to validate end to end before the first scheduled run ## Risks - Docker `:latest` changes meaning from "latest master build" to "latest stable release". This is deliberate and will be announced. Users who want the old behavior pull `:canary`. Until the first stable release after this change, `:latest` stays at its current (master-built) image - The nightly is a rebuild of the same source commit, not the byte-identical canary artifact that was smoked. The lockfile pins dependencies, and the publish path's registry-visibility and clean-prefix install gates still run on the nightly artifacts - All npm publishing must stay inside `release.yml` because npm trusted publishing pins that workflow file per package. The nightly jobs were added to `release.yml` for exactly that reason; this constraint is now documented in `RELEASING.md` - The stable-lane Docker dispatch fails gracefully (a warning with manual instructions) when the source ref predates `docker.yml`'s `workflow_dispatch` trigger ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic) in Claude Code, with extended thinking and full tool use (repository exploration, local test execution, live registry and git verification). All code, tests, and docs in this PR were model-authored under human direction. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green (pending — will confirm before merge) - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (pending — will confirm before merge) - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-10 12:16:55 -07:00
Devin Foley	75acc4650f	feat(sandbox-providers): pre-fill environment form with default sizing and image values (#11004 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent runs execute inside sandbox environments. Sandbox provider plugins (Daytona, Modal, exe.dev, and others) declare a JSON-Schema `configSchema`. The Environment configuration form renders from that schema. > - The sizing and image fields open empty. Users must guess working values. The Modal form cannot submit at all until the user types an app name and an image by hand. > - The form renderer already pre-fills every field that declares a JSON-Schema `default`. The manifests do not use this mechanism for sizing or image fields. > - This pull request adds optional `default` values to the Daytona, Modal, and exe.dev manifest schemas. > - The benefit is a form that opens with known-good values. Users can create a working environment without provider research. ## Linked Issues or Issue Description Current behavior The Environment configuration form opens with empty sizing and image fields for the Daytona, Modal, and exe.dev sandbox providers. Users must find working values in provider documentation. Modal declares `appName` and `image` as required with no default, so the form blocks submission until the user invents both values. Proposed behavior The provider manifests declare JSON-Schema `default` values. The existing form renderer pre-fills them: - Daytona: CPU `4`, memory `4` GiB, disk `10` GiB, image `daytonaio/sandbox:0.8.0` - exe.dev: CPU `4`, memory `4GB`, disk `20GB` - Modal: app name `paperclip`, image `node:22` Secret-ref fields (API keys, tokens) get no defaults on purpose. The form persists a raw string in a secret-ref field as a company secret on save. A placeholder default would become a stored secret with a bogus value. Each plugin test suite now guards this invariant. Reason and benefit New users can create a working sandbox environment without guessing. The defaults stay optional: users can clear or change every value, and the schema marks no new field as required. The Modal image default `node:22` satisfies the sandbox runtime contract in `SANDBOX-REQUIREMENTS.md` (`node`, `sh`, and `tar` on PATH). Subsystem affected Sandbox provider plugins (`packages/plugins/sandbox-providers/`): environment driver `configSchema` manifests. Breaking changes* None. Defaults only seed the create-mode form. Saved environments keep their stored config. E2B, Novita, Cloudflare, and Kubernetes manifests do not change: E2B and Novita already default to their base templates, and the Cloudflare bridge and Kubernetes cluster fields have no sensible universal value. ## What Changed - Add `default` values for `cpu`, `memory`, `disk`, and `image` in the Daytona manifest. Trim the memory description to match. - Add `default` values for `cpu`, `memory`, and `disk` in the exe.dev manifest. - Add `default` values for `appName` and `image` in the Modal manifest. Extend the image description with the runtime-contract rationale. - Add manifest tests in all three plugins: defaults match expected values, defaults satisfy their own schema constraints, and no secret-ref field declares a default. - Bump plugin versions: daytona and modal `0.1.0` → `0.1.1`, exe-dev `0.1.1` → `0.1.2`. ## Verification - Run `pnpm test` in `packages/plugins/sandbox-providers/daytona`, `.../modal`, and `.../exe-dev`. The new `* manifest form defaults` suites pass. - Run `./node_modules/.bin/tsc --noEmit` in each of the three packages. Typecheck passes. - Manual: rebuild the plugins (`pnpm build` in each package), let the plugin dev-watcher refresh the manifest, then open Environments → New environment. The Daytona form shows CPU 4, Memory 4, Disk 10, and image `daytonaio/sandbox:0.8.0`. The Modal form shows `paperclip` and `node:22`. The API key fields stay empty. - Verified live on a local instance: the served `configSchema` in the plugin registry carries the new defaults, and existing environments are unchanged. ## Risks - Low risk. The change touches only manifest schema metadata and tests. No runtime code path changes. - New environments created with untouched forms now request 4 CPU / 4 GiB / 10 GiB from Daytona instead of provider minimums. This can raise cost per sandbox for users who previously saved empty fields. - The Daytona image default pins `daytonaio/sandbox:0.8.0`. The default needs a manual bump when Daytona ships new sandbox images. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Fable 5 (Anthropic, model ID `claude-fable-5`), via the Claude Code CLI, with extended thinking and tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-06 12:04:33 -07:00
Devin Foley Claude Paperclip	00a24d7e8f	ci: split general-server tests into five shards with refreshed durations (#10925 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The PR workflow runs the server vitest suite across sharded runners because the suite is pinned to one worker. > - In successful PR run 30930345729 (2026-08-04), shard `server (3/4)` took 311 seconds of wall time and was the slowest check in the run. > - The suite has grown to about 946 seconds of serial vitest time, but the duration manifest was last sampled on 2026-08-01 at about 882 seconds. > - This pull request refreshes the per-suite duration manifest from that run's logs and splits the lane into five shards. > - The benefit is a shorter PR critical path: each shard carries about 196 seconds of suite time, level with the other lanes. ## Linked Issues or Issue Description Refs #10663 (previous split of this lane into four shards). Related: #10923 splits the separate serialized-suites lane into five shards. Both PRs touch `.github/workflows/pr.yml` in different matrix blocks; whichever merges second needs a trivial rebase. What existing behavior does this improve? The `general-server` vitest lane runs in four shards with a duration manifest sampled on 2026-08-01. Current behavior In PR run 30930345729, shard 3/4 ran for 311 seconds (273 seconds in the test step) and was the longest check in the run. The suite now totals about 946 seconds of serial vitest time. Proposed behavior Run the same suite set in five shards, balanced with a per-suite duration manifest refreshed from that run's shard logs (279 suites measured by diffing consecutive completion timestamps). Reason and benefit The refreshed LPT partition balances at about 196 seconds of suite time per shard (about 240 seconds per job), level with the other PR lanes. No test coverage is lost. Breaking changes None. The change only alters the CI partition size and the duration manifest. ## What Changed - Bump the `general-server` shard matrix in `.github/workflows/pr.yml` from four to five shards. - Refresh `scripts/general-server-shard-durations.json` from the 2026-08-04 run's shard logs. - Update `SHARD_COUNT` in `scripts/__tests__/run-vitest-stable-shard.test.mjs` to five. ## Verification - `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs` — 9/9 pass, including the complete non-overlapping partition proof and the duration-balance check. - `node --test scripts/__tests__/release-verify-workflow.test.mjs` — 2/2 pass. - `node --test scripts/__tests__/e2e-shard.test.mjs` — 7/7 pass. - A 5-way dry-run partition covers all suites exactly once with equal projected weights. ## Risks - Low risk. The change only alters CI partition size and duration weights; the suite set is unchanged. - One more runner is used per PR run for this lane. - Stale duration weights degrade gracefully: suites missing from the manifest get the median weight. ## Model Used - Claude (Anthropic), Claude Code CLI, model ID `claude-fable-5`, extended thinking with tool use enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (workflow comments explain the new shard math) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Claude <claude@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-08-05 16:44:29 -07:00
Devin FoleyandDevin Foley	b6e58019f2	ci: split serialized tests into five shards (#10923 ) ## Thinking Path > - Paperclip uses CI to keep control-plane changes safe and mergeable. > - The PR workflow splits serialized server tests across isolated runners. > - A recent successful run spent 305 seconds in serialized shard 2/4. > - That job was the slowest check in the run. > - The four shards reported about 739 seconds of Vitest suite time. > - This pull request adds a fifth serialized shard and keeps release verification aligned. > - The benefit is a shorter PR critical path with no loss of test coverage. ## Linked Issues or Issue Description What existing behavior does this improve? The PR and release verification workflows run serialized server tests in four shards. Current behavior Successful PR run 30876682788 spent 305 seconds in `Verify serialized server suites (2/4)`. The test step used 256 seconds and made this job the slowest check. Proposed behavior Run the same serialized suite set in five complete and non-overlapping shards. Reason and benefit The measured suites reported about 739 seconds of total Vitest time. Five runners reduce the expected average suite time from about 185 seconds to about 148 seconds before setup overhead. Breaking changes None. The change only alters CI partition size. ## What Changed - Split serialized server tests into five shards in the PR workflow. - Apply the same five-shard layout to release verification. - Add a partition test that proves complete and non-overlapping serialized coverage. - Update release workflow coverage tests for five shards. ## Verification - `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs scripts/__tests__/release-verify-workflow.test.mjs` - `git diff --check` ## Risks - Low risk. CI uses one additional runner for the serialized lane. - Round-robin partition weights can still vary as suite timings change. > This change does not overlap with planned core work in `ROADMAP.md`. Related PR #10663 optimized the separate general-server lane. ## Model Used - OpenAI Codex, GPT-5, agentic coding with reasoning, tool use, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Devin Foley <139239+devinfoley@users.noreply.github.com>	2026-08-05 16:29:49 -07:00
Devin Foley	ffd62a4cbb	fix(adapter-utils): carry the workspace origin remote into transported git workspaces (#10873 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - When an agent runs on a different host (sandbox or SSH), the adapter transport copies the local git execution workspace to that host and syncs changes back after the run > - The transport materializes the remote copy with `git init` plus a depth-1 or bundle fetch, so the copy has no `origin` remote and its head reads as a parentless snapshot commit > - An agent asked to publish its branch (push it, open a pull request) sees "no remote, root snapshot" and must hand the publish step back to a human operator, even when the branch base is a commit the upstream remote already holds > - This pull request carries the workspace's `origin` URL (credential-scrubbed) onto the transported copy as metadata > - The benefit is that branches produced in transported workspaces stay publishable by any actor with credentials, while the transport itself still never fetches or pushes ## Linked Issues or Issue Description No public issue exists. Description follows the enhancement template: What existing behavior does this improve? The workspace transport in `@paperclipai/adapter-utils` already copies a git workspace to the execution host and back. This change improves the fidelity of that copy: the transported repo keeps the workspace's `origin` remote instead of losing it. Subsystem affected Adapter utilities — the sandbox transport (`withShallowGitWorkspaceClone` in `packages/adapter-utils/src/git-workspace-sync.ts`) and the SSH transport (`importGitWorkspaceToSsh` in `packages/adapter-utils/src/ssh.ts`). Current behavior The transported copy is built with `git init` plus a depth-1 (sandbox) or bundle (SSH) fetch. It has no remotes. `git remote -v` is empty and the head commit reads as a root snapshot with no visible ancestry. Agents and operators inside the execution host cannot fetch real ancestry or push a branch, even when the branch base is a commit the upstream remote already holds. Proposed behavior The transport reads the source workspace's `origin` URL, scrubs credentials from it, and configures it on the transported copy. The sandbox path adds the remote to the fresh clone. The SSH path sets or adds the remote in the remote setup script, which also covers reused workspace directories. A workspace with no `origin` transports exactly as before. Reason and benefit A branch committed in a transported workspace becomes publishable in place: the shallow boundary commit already exists on the remote, so a push pack closes without full local ancestry (a new test locks in this property). Fetching real ancestry also becomes possible for whoever holds credentials. Without this, agents must describe their change in a handoff document and a human must reconstruct the branch by hand. Breaking changes None. The URL copy is best-effort and metadata-only. The transport never fetches from or pushes to the remote. The no-remote-git contract holds: sync-back through the local cwd stays the only cross-run persistence path, and `packages/adapters/AUTHORING.md` gains a paragraph that makes the carried-remote nuance explicit. ## What Changed - `packages/adapter-utils/src/git-workspace-sync.ts`: new `sanitizeGitRemoteUrl` (strips http(s) userinfo, where tokens can be embedded; scp-like/ssh forms and filesystem paths pass through) and `readSanitizedOriginRemoteUrl`; `withShallowGitWorkspaceClone` configures the scrubbed `origin` on the fresh clone, best-effort. - `packages/adapter-utils/src/ssh.ts`: `importGitWorkspaceToSsh` sets or adds the scrubbed `origin` in the remote setup script, non-fatal under `set -e`. - `packages/adapter-utils/src/git-workspace-sync.test.ts`: four new integration cases (remote copied, credentials scrubbed, no-origin unchanged, push from the shallow clone to an origin that holds the base commit) plus `sanitizeGitRemoteUrl` unit tests. - `packages/adapters/AUTHORING.md`: documents that a transported copy may carry a credential-scrubbed `origin` as metadata, and why this does not weaken the no-remote-git contract. ## Verification - `npx vitest run packages/adapter-utils/src/git-workspace-sync.test.ts` — 12/12 pass (4 new integration cases + sanitizer unit tests). - `npx vitest run packages/adapter-utils/src/sandbox-managed-runtime.test.ts` — 24/24 pass. - `npx vitest run packages/adapter-utils/src/ssh-fixture.test.ts` — 16/16 pass, including the `no-remote-git contract` case (a workspace without `origin` still round-trips with no remote introduced at any point). - `node scripts/check-no-git-push.mjs` — passes; this change adds no push or fetch to adapter/runtime code. - `pnpm typecheck` in `packages/adapter-utils` — clean. ## Risks - Low risk. The change is additive metadata on the transported copy only; failure to record the remote never fails the transport. - Credential exposure is the real hazard and is handled: http(s) userinfo is stripped before the URL leaves the host. Non-http forms (scp-like, `ssh://`) carry no secret in the URL and pass through. - A reused SSH workspace whose project `origin` changed now gets the current URL via `set-url` instead of keeping a stale one. ## Model Used Claude Fable 5 (`claude-fable-5`), Anthropic — extended thinking, agentic tool use via Claude Code CLI. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-05 10:24:01 -07:00
Devin Foley	f0b06d2de9	feat(claude): environment-aware test-environment probe and claude-local CI coverage (#10833 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The claude_local adapter runs Claude Code on sandbox execution targets, and operators verify an agent's configuration with the test-environment probe before running it > - Real runs merge the selected environment's env vars (secret refs included) under the agent's adapter config env, but the probe built its config from the adapter config alone — so environment-level auth worked in runs while the Test button reported missing auth, and a dropped secret binding passed silently > - The claude env-test hints also did not recognize `CLAUDE_CODE_OAUTH_TOKEN` even though the CLI accepts it, and a hello probe that hit the subscription usage limit reported a hard failure although authentication worked > - Separately, the claude-local package test suites were absent from the CI project list, so two suites drifted broken without notice > - This pull request makes the probe resolve the same layered env as a real run, adds the missing auth hint, classifies usage-limit probe results as a warning, repairs the drifted suites, and turns the claude-local project on in CI > - The benefit is a Test button that tells the truth about environment-level configuration, and a test suite that actually gates the claude-local adapter ## Linked Issues or Issue Description No public issue exists; related open PRs: Refs #9488 (recognizes CLAUDE_CODE_OAUTH_TOKEN in environment checks — overlaps with the auth-hint portion of this PR via a differently named check; it does not cover the environment-envVars probe merge, the usage-limit classification, or the CI coverage), Refs #9933 (live credential validation in environment checks — complementary, no file-level conflict with the route change). The underlying problem, following the enhancement template: Current behavior The test-environment route builds the probe config from the agent's adapterConfig only. Real runs merge the selected environment's envVars under the agent env, so environment-level env vars (including auth such as `ANTHROPIC_API_KEY` or `CLAUDE_CODE_OAUTH_TOKEN` bound as environment secrets) work in runs while "Test environment" cannot see them, and a missing secret binding passes silently. The claude env-test hints do not recognize `CLAUDE_CODE_OAUTH_TOKEN`. A hello probe that hits the subscription usage limit reports a hard `claude_hello_probe_failed`. The claude-local package test suites do not run in CI, and two of them are stale. Proposed behavior The probe resolves the selected environment's envVars (environment-consumer secret bindings included) and merges them under the agent config env with the run-path precedence; missing bindings surface as an explicit error check that fails the test. The env-test emits a `claude_oauth_token_configured` info check when that variable is set. Usage-limit probe results classify as a `claude_hello_probe_usage_limited` warning because auth works and only the usage window is spent. The claude-local suites run in CI. Docs state the resulting facts. Reason and benefit The Test button should tell the truth: it previously contradicted run behavior for environment-level configuration and hid broken secret bindings. Enabling the package suites in CI prevents further silent drift — two suites were already broken on master without anyone noticing. Breaking changes None. Runs are unchanged. The probe route only adds env layers and checks; setups without environment envVars behave exactly as before. ## What Changed - `server/src/routes/agents.ts`: the test-environment route resolves the selected environment's envVars (forbidden keys stripped, environment-consumer secret context) and merges them under the agent adapterConfig env, mirroring `resolveExecutionRunAdapterConfig` precedence. Missing secret bindings are skipped, reported as an `environment_env_binding_missing` error check, and fail the test — matching the `ConfigurationIncompleteFailure` a real dispatch would raise. - `packages/adapters/claude-local/src/server/test.ts`: new `claude_oauth_token_configured` info hint between the API-key warning and the subscription fallback; hello-probe classification gains a `claude_hello_probe_usage_limited` warning for provider-quota results (previously a hard `claude_hello_probe_failed`). - `scripts/run-vitest-stable.mjs`: add `@paperclipai/adapter-claude-local` to `nonServerProjects` so CI runs the package suites. - `packages/adapters/claude-local/src/server/execute.remote.test.ts`: assert both runtime asset syncs (skills and mcp-config); the suite predated the mcp-config asset. - `packages/adapters/claude-local/src/server/test.probe.test.ts`: usage-limit fixture now expects the usage-limited warning; new fixture covers the genuine transient path (529 overloaded); new tests cover the token hint and API-key precedence. - `server/src/__tests__/agent-test-environment-routes.test.ts`: new tests for the env merge (agent wins on conflict, forbidden key filtered), missing-binding reporting, and the no-execution-target fallback path. - `docs/adapters/claude-local.md`, `docs/adapters/overview.md`: state the auth-input facts (API key or oauth token wins over stored logins; snapshot-owns-auth applies when neither is configured) and describe the environment-aware Test behavior. ## Verification - `npx vitest run --project @paperclipai/adapter-claude-local` — 131 tests pass (both drifted suites repaired; they fail on master today). - `npx vitest run server/src/__tests__/agent-test-environment-routes.test.ts` — 7 tests pass. - `node --test ./scripts/__tests__/run-vitest-stable-shard.test.mjs` — passes with the added project. - `pnpm typecheck` in `server/` and `packages/adapters/claude-local/` — clean. ## Risks - Low risk. The run path is untouched; the probe route change is additive and inert when the environment has no envVars. - The probe now performs environment-consumer secret resolution at test time; access is authorized per binding exactly as at run time, and the audit consumer is the environment (as before for adapter-config resolution). - Enabling the claude-local project in CI adds about 2 seconds of vitest wall time to the general workspaces group and could surface future regressions in that package — which is the point. ## Model Used Claude Fable 5 (`claude-fable-5`), extended thinking enabled, agentic tool use via Claude Code (CLI). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-04 10:31:14 -07:00
Devin Foley	cb52f0b750	db: env-configurable client options; parallelize attention feed queries (#10795 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The server stores all state in PostgreSQL through Drizzle and the postgres.js driver > - Self-hosted installs run Postgres on localhost, so per-query latency is near zero; hosted installs often attach Postgres over a network, sometimes through a transaction-mode pooler > - The DB client passes no options to the driver, so operators cannot disable prepared statements or tune the pool without a source edit, and the deploy docs told them to edit `client.ts` > - The attention feed also runs its related-data lookups one after another, so its latency grows as queries × network round trip > - This pull request adds optional environment configuration for the DB client and batches the independent attention-feed lookups with `Promise.all` > - The benefit is that network-attached deployments get correct pooler support and a much faster attention feed, while self-hosted behavior does not change ## Linked Issues or Issue Description No public issue exists for this; description follows the bug report template: What happened? On deployments where PostgreSQL is network-attached (managed providers, pooled endpoints), the attention feed endpoint is slow: `attentionService.list()` awaits ~15–20 queries strictly in sequence, so a 70ms round trip turns into more than one second of pure network wait per call. Separately, connecting through a transaction-mode pooler (pgbouncer, Supavisor port 6543, Neon `-pooler` hosts) requires disabling prepared statements, and the only documented way was to hand-edit `packages/db/src/client.ts` — which `doc/DATABASE.md` itself tells operators not to do. Expected behavior The DB client is configurable from the environment (prepared statements, pool size, timeouts) with driver defaults when unset, and hot read paths do not multiply network latency by issuing independent queries sequentially. Steps to reproduce 1. Run the server with `DATABASE_URL` pointing at a Postgres instance with ~70ms round-trip latency. 2. Open the attention feed (`GET /companies/:companyId/attention`) and measure response time — it exceeds one second even with little data. 3. Try to connect through a transaction-mode pooler: there is no supported configuration to disable prepared statements. ## What Changed - `packages/db/src/client.ts`: `createDb` accepts a `DatabaseClientOptions` argument and reads optional env config — `DATABASE_PREPARED_STATEMENTS`, `DATABASE_POOL_MAX`, `DATABASE_IDLE_TIMEOUT_SECONDS`, `DATABASE_CONNECT_TIMEOUT_SECONDS`. When nothing is set, no option is passed to the driver and behavior is identical to the previous bare `postgres(url)`. - `packages/db/src/client-options.test.ts` (new): env parsing and driver-option mapping tests, including malformed-value rejection. - `server/src/services/attention.ts`: the independent related-data lookups in each feed section now run under `Promise.all` (issue summary/image/plan-document maps, decision bundle titles, blocked-issue maps, the newer-runs scan). Section order, item assembly, and query shapes are unchanged. - `doc/DATABASE.md` and `docs/deploy/database.md`: the edit-source pooling instruction is replaced with the env toggle, plus a short client-tuning reference. ## Verification - `pnpm --filter @paperclipai/db exec vitest run src/client-options.test.ts` — 6 tests pass. - `pnpm --filter server exec vitest run src/__tests__/attention-service.test.ts` — 22 tests pass. - `pnpm --filter server exec vitest run src/__tests__/decisions-service.test.ts src/__tests__/decision-training.test.ts` — 45 tests pass; this covers the call path that runs `attentionService.list()` inside `db.transaction`, where postgres.js serializes queries on the reserved connection. - `tsc` reports no errors in the changed files. ## Risks - Low risk for self-hosted installs: with no env vars set, `postgres(url, {})` receives an empty options object, which postgres.js treats the same as no options — driver defaults throughout. - The `Promise.all` batches only group queries that had no data dependency on each other; on the transaction call path the driver still executes them one at a time on the reserved connection, so transactional semantics are unchanged. - Malformed env values now fail fast at startup with a clear message instead of being silently ignored; this is intentional and only affects operators who set the new variables. ## Model Used Claude Fable 5 (`claude-fable-5`), Anthropic — via Claude Code CLI, extended thinking enabled, tool use (test execution, live latency measurement against a network-attached Postgres to size the problem). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above (searched "prepared statements", "pgbouncer", "pool", "attention feed", "lockfile" — closest matches are #10573/#10787 lockfile chores, unrelated to this change) - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-04 06:30:36 -07:00
Devin Foley	2c90cf0f2c	fix(server): serialize managed-checkout materialization and stop misattributing clone failures (#10723 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Repo-only project workspaces are materialized by a server-side managed `git clone` into a per-project directory (#10720 added credentials for private repos) > - Two issues on the same project routinely wake seconds apart, and both runs race the same clone target > - The loser fails with "destination path already exists", and its failure cleanup removes the directory out from under the winner's in-progress clone — both runs then fail every round > - This pull request serializes materialization per target directory and makes the clone land atomically via a temp sibling + rename, so the shared target is never partial and never removed > - The benefit is that concurrent runs on the same project converge: one clone happens, everyone adopts it, and unrelated failures no longer blame the GitHub credential ## Linked Issues or Issue Description What happened? With isolated workspaces enabled on a project whose only workspace is repo-only, unblocking two issues at once produced lockstep mutual destruction (observed live, two consecutive rounds): both runs called the managed-checkout materialization concurrently; one clone created the target directory, the other's `git clone` failed with `fatal: destination path '…' already exists and is not an empty directory`, and that run's failure cleanup deleted the directory while the first clone was still writing into it (`fatal: could not set 'core…'`). Both runs failed `workspace_validation_failed`; their staggered retries could race again. The failure message also wrongly claimed the GitHub credential "was rejected or lacks access" — the collision had nothing to do with auth. Expected behavior Concurrent materializations of the same project checkout share one clone; a completed checkout is never removed by a failing sibling; the credential is only blamed for auth-shaped failures. Steps to reproduce 1. Project with a repo-only workspace (private repo, isolated workspaces on). 2. Move two issues on that project to `todo` at the same time so both runs start within seconds. 3. Both runs fail workspace validation with "destination path already exists" / "could not set 'core…'" instead of one clone succeeding. Paperclip version or commit `master` (`75f6256b76`). ## What Changed - `ensureManagedProjectWorkspace` serializes in-flight materializations per target cwd (a module-level promise map): concurrent callers share one attempt. - The clone lands in a `<target>.clone-XXXXXX` temp sibling created with `mkdtemp`, then moves into place with an atomic `rename`. Clone failure removes only the temp directory; the shared target is never created partially and never deleted. If the target appears between the emptiness check and the rename (another process won), the completed checkout is adopted instead of failing the run. - `describeGitAuthFailure` attributes the GitHub credential only when the error matches the auth-failure pattern; unrelated failures (path collisions, network errors) no longer claim the token was rejected. ## Verification - `cd server && npx vitest run src/__tests__/heartbeat-managed-clone-credentials.test.ts src/__tests__/git-credentials.test.ts src/__tests__/heartbeat-workspace-session.test.ts` — 169 tests pass, including new cases: concurrent materializations of the same checkout succeed with one shared result and no temp litter; failed clones leave neither target nor temp directories; an authenticated clone failing for non-auth reasons does not blame the credential. - `pnpm --filter @paperclipai/server typecheck` — clean. ## Risks - Low risk. The serialization is in-process and keyed by exact target path; the temp+rename pattern stays on the same filesystem (sibling path) so the rename is atomic. Single-run behavior is byte-identical apart from the temp-dir intermediate. - The rename-conflict adoption path accepts a checkout another materialization completed; the pre-existing `gitDirExists` adoption semantics are unchanged. ## Model Used Claude Fable 5 (`claude-fable-5`, extended thinking, agentic tool use via Claude Code CLI). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-02 22:14:58 -07:00
Devin Foley	75f6256b76	fix(server): resolve duplicate scrubGitCredentialText declaration on master (#10722 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - #10719 and #10720 both hardened credential scrubbing for server-side git operations > - #10719 defined `scrubGitCredentialText` locally in `heartbeat.ts`; #10720 imported the identical function from the new `git-credentials.ts` > - The two merged cleanly textually, but together they leave `heartbeat.ts` with an import that conflicts with a local declaration (TS2440) > - This pull request keeps the `git-credentials.ts` copy as canonical, drops the heartbeat-local duplicate, and re-exports the import so existing importers are unchanged > - The benefit is that master typechecks, builds, and produces Docker images again ## Linked Issues or Issue Description What happened? After #10719 and #10720 merged, master fails typecheck, the server build, and both Docker image builds with `src/services/heartbeat.ts(76,3): error TS2440: Import declaration conflicts with local declaration of 'scrubGitCredentialText'`. The canary release run for `e0c2448267` failed for the same reason, so no `@paperclipai/db` canary carrying migrations 0201/0202 can publish. Expected behavior Master typechecks and builds; one canonical `scrubGitCredentialText` lives in `git-credentials.ts`. Steps to reproduce `pnpm --filter @paperclipai/server typecheck` on `e0c2448267`. Paperclip version or commit `master` (`e0c2448267`). ## What Changed - Removed the heartbeat-local `scrubGitCredentialText` definition (byte-identical to the `git-credentials.ts` copy). - Re-exported the imported function from `heartbeat.ts` so existing importers, including `heartbeat-workspace-session.test.ts`, keep working. ## Verification - `pnpm --filter @paperclipai/server typecheck` — clean. - `cd server && npx vitest run src/__tests__/heartbeat-workspace-session.test.ts src/__tests__/git-credentials.test.ts src/__tests__/heartbeat-managed-clone-credentials.test.ts` — 167 tests pass. ## Risks Low risk — deletes one of two identical implementations and preserves the public import surface via a re-export. ## Model Used Claude Fable 5 (`claude-fable-5`, extended thinking, agentic tool use via Claude Code CLI). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-02 21:06:53 -07:00
Devin Foley	e0c2448267	feat(server): authenticate server-side git clone and fetch with a company-secret GitHub token (#10720 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Repo-only project workspaces are materialized by a server-side `git clone`, and isolated `git_worktree` runs refresh their base ref with server-side `git fetch` > - Both operations run outside the agent process with no credentials, so private GitHub repositories can never be cloned or refreshed — agent-scoped credential env bindings do not reach them > - The company secret store already has a well-known GitHub token convention (`GITHUB_TOKEN` / `GH_TOKEN` / `PAPERCLIP_GITHUB_TOKEN`, consumed by the external-object provider for API reads), but nothing server-side consults it for git > - This pull request resolves that token per run and authenticates the managed clone and every base-ref refresh with it through an ephemeral credential helper > - The benefit is that isolated workspaces work on private repositories with one company secret, while public repositories and self-hosted ambient git configuration keep working unchanged ## Linked Issues or Issue Description Subsystem affected Server workspace materialization (`server/src/services/heartbeat.ts`) and execution-workspace realization (`server/src/services/workspace-runtime.ts`). Problem or motivation A project workspace configured with only a private GitHub `repoUrl` cannot be used for isolated `git_worktree` runs: the managed `git clone` runs with a sanitized, credential-less environment, and plain git cannot consume a bare token env variable without a credential helper. There is no way to give the server a git credential — storing a `GH_TOKEN` company secret has no effect on server-side git, and a credential-less private clone hangs on a terminal prompt until the ten-minute clone timeout. Base-ref refreshes (`git fetch`) during worktree realization have the same gap. Proposed solution A `git-credentials` module resolves a token per run — company secret by well-known name (`GITHUB_TOKEN`, `GH_TOKEN`, `PAPERCLIP_GITHUB_TOKEN`), then `GITHUB_TOKEN`/`GH_TOKEN` in the server process environment for self-hosted deployments, then none — and builds a git invocation that authenticates via an inline credential helper. The token travels in an env variable; it never appears in argv, URLs, or on disk. Only `https://github.com` remotes are authenticated; everything else keeps ambient behavior. The provider is a single factory seam so a future brokered credential source can replace it without touching call sites. Alternatives considered - A GitHub OAuth "connect your account" flow: heavier product surface, needs app registration and callback custody; out of scope for a server credential and better served by a dedicated connector later. The provider seam keeps that path open. - `gh auth setup-git`: writes helper configuration to disk and requires a global token env; rejected in favor of per-invocation config with no persistent state. - Embedding the token in the clone URL: leaks into argv, error messages, and `.git/config`; rejected. ## What Changed - New `server/src/services/git-credentials.ts`: `createGitRemoteAuthProvider` (memoized per run, one secret resolution and one audit event), `buildGitAuthInvocation` (helper-reset + inline helper, `x-access-token` username, `GIT_TERMINAL_PROMPT=0`), `isGitHubHttpsRemoteUrl` host gating (rejects ssh/GHES/http/other hosts/userinfo URLs), `describeGitAuthFailure`, and the canonical `scrubGitCredentialText`. Secret resolutions pass a `system` consumer access context so they are recorded as secret access events. - `ensureManagedProjectWorkspace` (now exported) accepts an optional auth provider; the clone env spreads the token after `sanitizeRuntimeServiceBaseEnv` (which strips `PAPERCLIP_*`), always sets `GIT_TERMINAL_PROMPT=0`, distinguishes "credential rejected" from "no credential configured — add a GITHUB_TOKEN or GH_TOKEN company secret" in the error, and removes the partially created directory on clone failure so a timeout-killed clone cannot be adopted as a broken checkout by the next run. - `refreshRemoteTrackingBaseRef` (now exported) captures the remote URL it already looked up, asks the provider for an invocation, and attributes failed authenticated fetches to the credential in a scrubbed warning. The optional provider threads through `detectDefaultBranch`, `resolveAuthoritativeBaseRef`, `inspectExecutionWorkspaceBaseDrift`, `realizeExecutionWorkspace`, and `ensurePersistedExecutionWorkspaceAvailable`; heartbeat builds one provider per run for both the anchor-resolution clone path and workspace realization/restore. - `github-external-object-provider.ts` imports the shared secret-name list; `isGitHubDotCom` is exported from `github-fetch.ts`. - Docs: "Private repositories and repo-only project workspaces" section in the execution-workspaces guide, cross-linked from the secrets deploy doc. ## Verification - `cd server && npx vitest run src/__tests__/git-credentials.test.ts` — resolution chain order and precedence, env fallback, memoization, audited access context, host-gating matrix, invocation shape (token absent from argv), scrubber, failure descriptions, and a real-git `git credential fill` round trip that proves the helper executes and answers with the env-carried token (no network). - `cd server && npx vitest run src/__tests__/heartbeat-managed-clone-credentials.test.ts` — clones behave byte-identically with no provider or a null-returning provider (local repos, no network), authenticated-failure errors name the credential, non-auth failures do not mention credentials, partial clone directories are removed, pre-existing non-git directories keep the "Using it as-is" path, and the sanitizer spread order keeps the token env alive. - `cd server && npx vitest run src/__tests__/workspace-runtime.test.ts` — new `refreshRemoteTrackingBaseRef` cases: provider offered the remote URL and null keeps behavior identical; failed authenticated fetch warning names the credential; unauthenticated failure warning stays credential-free. - `pnpm --filter @paperclipai/server typecheck` is clean. - Manual (optional, networked): store a `GH_TOKEN` company secret, configure a repo-only project workspace pointing at a private GitHub repository, run an isolated-workspace issue — the managed clone succeeds and the worktree run proceeds. ## Risks - Every new parameter is optional; with no provider the git invocations are byte-identical to before. Public repos and ambient credential helpers keep working whenever no token resolves. - Precedence change when a token exists: a stored company secret now wins over ambient helpers for `https://github.com` remotes (the helper list is reset for that invocation). The rejected-credential error names the secret so an operator can fix or remove it. - `GIT_TERMINAL_PROMPT=0` on the managed clone is the one always-on change: a credential-less private clone now fails fast with a clear message instead of hanging until the ten-minute timeout (it could only ever "succeed" interactively on a TTY dev server). - The token is scoped to the git process env for one invocation; it is never written to agent env, run context, disk, or logs, and error text is scrubbed of URL userinfo. - No migrations, no image changes (git ships in the image). ## Model Used Claude Fable 5 (`claude-fable-5`, extended thinking, agentic tool use via Claude Code CLI). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-02 20:27:09 -07:00
Devin Foley	97590ff8c4	feat(dev): add pnpm dev:mobile and dev:both for prebuilt UI preview (#10718 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The board UI is a React SPA served by the paperclip server; the standard local dev flow is `pnpm dev`, which runs vite in dev mode with HMR and an unbundled module graph > - The unbundled dev bundle is hundreds of MB of JS across many requests, which is fine on a local machine but unusable from a phone or tablet on slow/lossy links (airplane wifi, mobile data, distant tailnet peers) > - Contributors who want to iterate on the board from a mobile device today have no supported way to preview a small production-shaped bundle without stopping the dev server and running a one-off `vite preview` with manual proxy plumbing > - This pull request adds `pnpm dev:mobile` — build the UI and serve `ui/dist` via `vite preview` on port 3101, with `/api` proxied to the running dev server on 3100 — plus `pnpm dev:both` to run both flavors together > - The benefit is a supported second flavor of the dev server for phones/tablets that runs alongside the normal one, without touching the primary `pnpm dev` flow ## Linked Issues or Issue Description Subsystem affected ui/ — React + Vite board UI Problem or motivation The vite dev server serves an unbundled module graph, which is fine on localhost but unusable from a phone or tablet on a slow link. Contributors testing responsive behavior on mobile devices have no supported way to serve a small production-shaped SPA against the running dev API. Running `vite preview` directly does not work either — the server's board mutation guard checks that the browser's Origin matches the request Host, and a preview on a second port would fail every mutation. Proposed solution Add two root scripts: - `pnpm dev:mobile` — build `ui/dist` and serve it via `vite preview` on port 3101, with `/api` proxied to the API server on 3100. - `pnpm dev:both` — run `pnpm dev` and `pnpm dev:mobile` together in a single terminal with prefixed output and shared signal handling. The vite preview config binds `0.0.0.0`, sets `allowedHosts: true` so it accepts arbitrary hostnames (LAN, tailnet, ngrok, etc.), and the shared `/api` proxy forwards the client's original Host header as `x-forwarded-host`. The paperclip server's mutation guard already prefers `x-forwarded-host` over `host` when computing trusted origins, so the browser's Origin becomes trusted automatically. Alternatives considered - Bespoke node proxy script — works but duplicates what vite preview already does. - Loosen the mutation guard to accept arbitrary origins — reduces security for the primary server for the sake of a dev-only workflow. - Second server config that binds a second port from the paperclip server itself — much larger change and mixes runtime concerns with a dev-tooling convenience. ## What Changed - New `pnpm dev:mobile` script — build UI then run `vite preview` on port 3101. - New `pnpm dev:both` script — run `pnpm dev` and `pnpm dev:mobile` together via `scripts/dev-both.mjs`, which prefixes each child's output, propagates SIGINT/SIGTERM, and exits when either child exits. - `ui/vite.config.ts` — add a `preview` block (port 3101, host `0.0.0.0`, `allowedHosts: true`, shared `/api` proxy). - New `ui/src/lib/vite-api-proxy.ts` — extracts the `/api` proxy factory shared by dev and preview, and forwards the client Host as `x-forwarded-host` (plus `x-forwarded-proto`). - New unit test `ui/src/lib/vite-api-proxy.test.ts` covering the header-injection behavior and the pass-through when no Host is present. ## Verification - `pnpm --filter @paperclipai/ui exec vitest run src/lib/vite-api-proxy.test.ts` — 3 tests pass. - `pnpm --filter @paperclipai/ui typecheck` — clean. - `pnpm --filter @paperclipai/ui build` — clean. - Manual: ran `vite preview` against an echo listener and confirmed the request arrives with `x-forwarded-host` set to the client Host header and `x-forwarded-proto: http`. Then ran `pnpm dev:mobile` against the live dev server and verified board mutations (mark issue read, resolve recovery action, run routine) succeed from a second-port browser session that previously 403'd. ## Risks Low risk. Changes are limited to dev tooling — no runtime code paths, no server changes, no schema/migrations. The `apiProxy` refactor is a no-op behaviorally for the existing dev server (same target, same `ws: true`); the only new behavior is the two `x-forwarded-*` headers, and the server side already prefers those headers when trusting origins. `dev:mobile` and `dev:both` are additive; existing `pnpm dev` is untouched. ## Model Used Claude Opus 4.7 (1M context), extended thinking, tool use (bash, file edits). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-02 20:26:03 -07:00
Devin FoleyandPaperclip	185515c97b	fix(external-objects): refresh PR status labels (#10704 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The issue properties panel can show external objects such as GitHub pull requests. > - Those objects are resolved by external-object providers and then displayed as compact status labels. > - A GitHub pull request could remain in the fallback `unknown` state and appear as `Not yet resolved`. > - That label is confusing when the object is known but has not been refreshed yet. > - This pull request refreshes due external objects from the heartbeat scheduler and improves the unknown-status copy. > - The benefit is a properties panel that moves from pending refresh to the real pull request state without a manual refresh. ## Linked Issues or Issue Description No public GitHub issue exists for this bug. I searched for related public issues and pull requests using the terms `Not yet refreshed`, `external objects refresh`, and `external PR status`, and did not find a duplicate implementation. What happened? The issue properties panel could show a GitHub pull request as `Not yet resolved` even when the referenced pull request was valid. The object stayed stale unless a manual refresh path ran. Expected behavior A known external object should show pending-refresh copy while it waits for provider data. When the scheduler refreshes it, the properties panel should show the provider status such as open, merged, or closed. Steps to reproduce 1. Create or view an issue that references a GitHub pull request. 2. Open the issue properties panel. 3. Observe the external object row before a manual refresh has run. Paperclip version or commit Current `master` before this pull request. Deployment mode Local dev and self-hosted server. Installation method Built from source. Agent adapter(s) involved Not adapter-specific. Database mode Not database-related. Access context Board view. Privacy checklist I reviewed this description and did not include logs, credentials, private URLs, internal issue IDs, or PII. ## What Changed - Added a heartbeat scheduler tick that refreshes due external objects for active companies. - Kept manual external-object refresh behavior on the same service path. - Changed display copy so known provider objects use liveness labels such as `Not yet refreshed`, while fresh unknown provider statuses show `Status unavailable`. - Added server and UI tests for scheduled refresh and label behavior. ## Verification - `corepack pnpm install --frozen-lockfile` - `pnpm check:token-gates` - `pnpm exec vitest run server/src/__tests__/external-objects-service.test.ts server/src/__tests__/server-startup-feedback-export.test.ts ui/src/components/ExternalObjectPill.test.tsx ui/src/components/IssueProperties.test.tsx ui/src/lib/external-objects.test.ts` - `pnpm --filter @paperclipai/ui typecheck` - `pnpm --filter @paperclipai/server typecheck` - `pnpm --filter @paperclipai/server build` - `pnpm --filter @paperclipai/ui build` - `pnpm run typecheck:build-gaps` - GitHub PR checks passed on head `0e7fcd30` - Greptile reported 5/5 on head `0e7fcd30` with no unresolved review threads Notes: - I ran recursive typecheck and build first. Both hit container resource limits with exit 137 during concurrent package work, so I reran the affected server and UI targets separately. - An unrelated workspace-runtime auto-port test fails in this container with a PID ownership mismatch. It is outside the files changed here. ## Risks Low to medium risk. The scheduler does more periodic external-object work, so the main risk is extra provider refresh load. The implementation bounds the work to active companies, due non-terminal objects, and 50 objects per company per tick. The path also stays behind the external-objects experimental setting. ## Model Used OpenAI GPT-5 Codex in the Codex execution environment, with shell and GitHub CLI tool use. The runtime did not expose a more specific internal model ID or context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-08-02 20:24:52 -07:00
Devin Foley	7a3815eb9a	fix(heartbeat): surface the real cause when a git_worktree base cannot be materialized (#10719 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Heartbeat runs prepare an execution workspace for each issue; with isolated workspaces (or low-trust runs), the `git_worktree` strategy needs a real git checkout as its base > - For repo-only project workspaces, the server materializes the checkout with a managed `git clone`; when that clone fails, the resolver drops the error and silently falls back to the agent home directory with `source: "project_primary"` > - The pre-dispatch guard then reports `git_worktree_base_not_git_checkout`, which hides the real cause; if the fallback directory happens to be a git checkout, the run silently builds worktrees off the wrong repository > - This pull request records materialization failures on the resolved workspace, marks the fallback explicitly, and fails the guard with a truthful `git_worktree_base_materialization_failed` reason that carries the clone error > - The benefit is that operators see the real cause (a failed clone) in the run error, the blocked-issue comment, and the recovery next action, instead of a misleading symptom ## Linked Issues or Issue Description What happened? An issue configured for isolated `git_worktree` execution on a project whose only workspace is repo-only (a `repoUrl` with no local path) fails every run with `workspace_validation_failed` and reason `git_worktree_base_not_git_checkout`, pointing at the agent home directory. The message does not mention that the managed `git clone` of the project repository failed (for a private repository the clone can never succeed without credentials). The recovery flow then blocks the issue with the same misleading explanation. Run warnings claim "Project workspace has no local cwd configured" even though the workspace is configured and the clone failed. Expected behavior The run failure, the blocked-issue comment, and the recovery next action should state the real cause: the project workspace checkout could not be prepared, including the clone error, so the operator can repair the repository URL, clone access, or configured local cwd. A fallback directory that happens to be a git checkout must not let the run proceed against the wrong repository. Steps to reproduce 1. Create a project whose primary workspace has a `repoUrl` pointing at a private GitHub repository and no local path. 2. Enable the Isolated Workspaces experimental setting (or use a low-trust run, which forces isolation). 3. Run any issue in that project. 4. The run fails with `git_worktree_base_not_git_checkout` on the agent home directory; the clone failure appears nowhere. Paperclip version or commit `master` (`bd86dbe41b`). ## What Changed - `resolveAnchorWorkspaceForRun` collects every failed project-workspace materialization attempt (previously the error was dropped unless the row was the preferred workspace) and returns two new fields on `ResolvedWorkspaceForRun`: `baseCwdFallback` and `materializationFailures`. The `source` label is unchanged because session migration keys off `source === "project_primary"`. - `assertGitWorktreeBaseWorkspaceReady` accepts the anchor facts and fails with the new reason `git_worktree_base_materialization_failed` — checked before the git-checkout probe, so a fallback directory that happens to be a git repo can no longer host worktrees for the wrong repository. The message carries the first scrubbed clone error and remediation, and lands in `run.error`, the persisted `workspaceValidation` payload, and the blocked-issue comment. - New `scrubGitCredentialText` masks URL userinfo (a `repoUrl` can legitimately embed credentials today) before errors reach warnings or persisted payloads. - Fallback warning assembly moved into the pure helper `buildAnchorFallbackWorkspaceNotes`; a clone failure now produces "Failed to prepare the project workspace checkout: …" instead of the false "no local cwd configured", with the existing warning texts preserved byte-for-byte when nothing failed to materialize. - The workspace-validation recovery comment and the recovery service's next action explain the new reason specifically. ## Verification - `cd server && npx vitest run src/__tests__/heartbeat-workspace-session.test.ts` — new cases: the new reason takes precedence over the git-checkout probe (fallback cwd is a real git repo), payload carries the scrubbed failures, anchor-absent legacy behavior unchanged, scrubber unit tests, and warning-assembly unit tests that pin the existing texts. - `cd server && npx vitest run src/__tests__/issue-recovery-actions.test.ts src/__tests__/heartbeat-process-recovery.test.ts src/__tests__/heartbeat-retry-scheduling.test.ts` — recovery surfaces still pass. - `pnpm --filter @paperclipai/server typecheck` is clean. ## Risks - Additive persisted-payload fields and a new reason string; recovery reason handling falls through to generic text for unknown reasons, and no UI consumes the `git_worktree_base_*` strings. - Intentional behavior change: a repo-only project whose clone fails and whose agent-home fallback happened to be a git checkout previously ran in that unrelated repository; it now fails with the truthful reason. A test locks this. - Runs without isolated workspaces (the default) never reach the guard; their fallback behavior is unchanged. ## Model Used Claude Fable 5 (`claude-fable-5`, extended thinking, agentic tool use via Claude Code CLI). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-02 20:23:15 -07:00
Devin FoleyandPaperclip	799973f26a	fix(ui): hide empty inbox search sections (#10700 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Inbox helps operators scan issues that need attention. > - Inbox search can add supplemental sections for archived matches and other matches. > - The supplemental search section builder still sent empty sections into the grouped render path. > - That made the Archived and Other results dividers appear even when those sections had no rows. > - This pull request drops empty supplemental sections before rendering. > - The benefit is a cleaner near-empty inbox search view. ## Linked Issues or Issue Description No public GitHub issue exists for this report. Public GitHub search found no duplicate or related open issues or pull requests for this inbox search behavior. What happened? Inbox search could show Archived and Other results divider headers even when those supplemental sections had no rows. Expected behavior Empty supplemental search sections should not render divider headers. Steps to reproduce 1. Open the Inbox. 2. Search in a near-empty inbox with no archived matches and no outside-inbox matches. 3. Observe that empty supplemental divider headers can appear. Paperclip version or commit `master` before this change. Deployment mode Built from source. ## What Changed - Dropped empty supplemental inbox search sections before they reach the grouped inbox render path. - Added a unit regression test for empty Archived and Other results sections. - Refreshed the branch against current `master` to clear the merge conflict. ## Verification - `git diff --check origin/master...HEAD` passed. - Public diff is limited to `ui/src/lib/inbox.ts` and `ui/src/lib/inbox.test.ts`. - Local focused Vitest could not run in this execution checkout because dependencies are not installed and `corepack pnpm exec vitest ...` reports `Command "vitest" not found`. - Pull request CI is green for typecheck, build, server tests, e2e, security checks, policy checks, canary dry run, and aggregate verify. - Greptile Review passed on commit `dc2e224` with confidence score 5/5 and no comments. ## Risks Low risk. The Inbox change only filters empty supplemental search sections. Normal inbox sections and non-empty archived or other search results keep their current behavior. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, GPT-5, reasoning-enabled with terminal tool use and code execution. The runtime context-window size is not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-08-02 19:05:38 -07:00
Devin FoleyandCody	bd86dbe41b	fix(codex-local): detect server-visible Codex credentials in ACP environment test (#10703 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The codex-local adapter runs Codex agents through the ACP lane and offers a "Test" button on the agent configuration page to validate the environment > - The environment test used its own ad-hoc credential probe, while real dispatch uses the shared `evaluateCodexCredentialReadiness` predicate in `codex-home.ts` > - The two paths disagreed: a user with valid Codex subscription auth in the shared, server-visible Codex home still saw "No Codex ACP credentials were detected" > - The old warning also suggested `codex login` without explaining that a `/login` in a separate Codex or chat session does not authenticate the Paperclip server process > - This pull request makes `testCodexAcpEnvironment` use the same shared readiness predicate as real dispatch and rewords the warning to name the server credential boundary > - The benefit is that the Test button now agrees with what dispatch will actually do, and the warning tells the user exactly which process needs the credentials ## Linked Issues or Issue Description No public GitHub issue exists for this bug. Related credential-handling work, not duplicates: Refs #10160 (classifies OpenAI invalid-key 401s at dispatch time) and Refs #9598 (classifies Codex refresh auth failures). Both cover dispatch-time failures; this PR fixes the pre-dispatch environment test. Description follows the bug report template: What happened? A user configured a Codex ACP agent and had already authenticated Codex (subscription auth present in the shared Codex home visible to the Paperclip server). Clicking "Test" on the agent configuration page still reported: `warn: No Codex ACP credentials were detected. Hint: Set OPENAI_API_KEY or run codex login before starting a Codex ACP agent.` Expected behavior The environment test should detect the same credentials that real agent dispatch would use. When shared managed Codex auth is available, the test should pass with an informational check instead of warning. When credentials really are missing, the warning should explain that the Paperclip server process is the one that needs them. Steps to reproduce 1. Run the Paperclip server as an OS user whose shared Codex home contains valid subscription `auth.json` (no `OPENAI_API_KEY` in the adapter env or server env). 2. Configure an agent with the codex-local adapter using the ACP engine. 3. Click "Test" on the agent configuration page. 4. Observe the `codex_acp_credentials_missing` warning even though dispatch would succeed. Agent adapter(s) involved Codex Additional context The confusion was amplified by the hint: users had run `/login` in a Codex chat session and assumed the server was authenticated. That login lives in a different process and home directory, so the server never saw it. ## What Changed - `testCodexAcpEnvironment` (packages/adapters/codex-local/src/server/acp.ts) now calls the shared `evaluateCodexCredentialReadiness` predicate from `codex-home.ts` instead of a local ad-hoc `hasCodexNativeCredentials` probe, so the Test button and real dispatch agree. - An explicit empty `OPENAI_API_KEY` in the adapter config env no longer falls through to the server environment key. - An externally managed `CODEX_HOME` override is now reported as its own informational check (`codex_acp_external_home_configured`). - The `codex_acp_credentials_missing` warning now says the credentials must be visible to the Paperclip server, and the hint explains that a `/login` in a separate Codex or chat session does not authenticate the server. - Removed the now-unused `hasCodexNativeCredentials` helper. - Added two regression tests: shared managed Codex auth is detected (no false warning), and the missing-credentials warning carries the new server-boundary wording. ## Verification - `pnpm --filter @paperclip/adapter-codex-local test -- src/server/acp.test.ts` — the two new tests cover the shared-home detection branch and the new warning wording; the existing ACP lane tests cover the API-key and remote-target branches. - Manual: with subscription auth in the server-visible shared Codex home and no `OPENAI_API_KEY`, the agent configuration Test now reports `codex_acp_native_auth_detected` (info) instead of `codex_acp_credentials_missing` (warn). ## Risks - Low risk. The change only affects the environment test path, not dispatch. The readiness predicate is the same one dispatch already uses, so drift between the two paths is now structurally prevented. - Behavioral shift: an explicit empty adapter `OPENAI_API_KEY` no longer silently falls back to the server env key in the test result. This matches dispatch behavior and is intentional. ## Model Used - Implementation authored by OpenAI Codex (gpt-5.5) running through the Codex ACP lane with tool use. - PR preparation, rebase onto master, and review fix-up by Anthropic Claude (Claude Code CLI agent, extended thinking, tool use). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Cody <noreply@paperclip.ing>	2026-08-02 16:55:53 -07:00
Devin Foley	8b83d69e3c	feat(heartbeat): serialize shared-workspace issue runs with bounded busy deferrals (#10699 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The heartbeat service dispatches agent runs, and issues in a project can share one project workspace (one working tree on disk). > - Two runs can execute in the same shared workspace at the same time. Each run mutates the same uncommitted files and branches, and the runs corrupt each other's state. > - Multi-agent projects hit this as soon as two issues in one project become active together, so the platform needs to serialize shared-workspace execution instead of relying on luck. > - This pull request adds a pre-dispatch gate: a run whose issue targets a busy shared workspace is deferred with a bounded scheduled retry instead of dispatched. > - The benefit is that concurrent issue runs in one project take turns in the shared working tree, while isolated-workspace runs and unrelated workspaces stay fully parallel. ## Linked Issues or Issue Description Fixes #10645 ## What Changed - `server/src/services/heartbeat.ts`: - New pre-dispatch gate in the run executor. Before adapter dispatch, when the run's issue has a `projectWorkspaceId` and the effective execution workspace mode is `shared_workspace`, the executor looks for a holder: another `running` run whose context issue shares the same project workspace. The gate covers every run shape that reaches adapter dispatch with issue context — assignee execution runs, comment/mention interaction wakes, and review-participant runs. - When a holder exists, the run throws `WorkspaceBusyDeferral` instead of dispatching. The outer catch recognizes the deferral: it cancels the run with `errorCode: "workspace_busy"` (contention is not a failure), cancels its wakeup, schedules a retry through the existing `scheduleBoundedRetryForRun` primitive (`workspace_busy` reason, 60–120 s jittered delay), and returns the agent to idle. The issue execution lock transfers to the scheduled retry run, so the issue keeps an active execution path and stranded-issue recovery does not fire. - An adapter never dispatches alongside a live holder: deferral has no attempt ceiling, so a deferred run keeps rescheduling until the workspace frees. Deadlock safety comes from holder liveness, not a counter — a holder silent past `ACTIVE_RUN_OUTPUT_SUSPICION_THRESHOLD_MS` (recovery's own "suspicious silence" bar, 1 h) stops counting as a holder, so a zombie run can only delay work, never park it forever, and recovery's silent-run escalation is already reaping it in parallel. If no retry can be scheduled (agent paused, issue reassigned), the deferral releases the issue execution lock so the issue does not strand. - Holder detection honors isolation: when the isolated-workspaces experiment is enabled, holders whose issue settings select `isolated_workspace` / `operator_branch` (or the legacy `isolated` alias) are not counted, because they never touch the shared tree. A NULL or `agent_default` mode counts as a holder — over-serializing is the safe direction. - Non-assignee deferrals survive replay: the deferral stamps `workspaceBusyDeferredWhileAssignee` into the run context (inherited by the scheduled retry), and both the retry promotion gate and the claim-time staleness check exempt a non-assignee `workspace_busy` retry from the reassignment cancellation — for such a retry an assignee mismatch is the expected state, not a reassignment race. An assignee run's retry keeps the full protection: if the issue is reassigned while the retry pends, it still cancels with `issue_reassigned`. - `server/src/__tests__/heartbeat-workspace-busy.test.ts` (new): embedded-Postgres coverage of the full lifecycle plus unit coverage of the delay window. ## Verification - `cd server && pnpm vitest run src/__tests__/heartbeat-workspace-busy.test.ts` — 10 tests: - a run whose issue targets a busy shared workspace is cancelled with `workspace_busy`, its adapter never executes, a `scheduled_retry` run exists with the 60–120 s window, the issue execution lock points at the retry run, the holder run is untouched, and the agent returns to idle; - after the holder finishes, `promoteDueScheduledRetries` + `resumeQueuedRuns` execute the retry run to success; - a non-assignee comment-mention wake defers, does not touch the issue execution lock, and its retry promotes, survives the claim-time staleness check, and executes despite the assignee mismatch; - an assignee retry is still cancelled with `issue_reassigned` when the issue is reassigned while the retry pends; - a holder issue with `executionWorkspaceSettings.mode = "isolated_workspace"` does not cause deferral; - a running run in a different project workspace does not cause deferral; - a holder silent past the staleness threshold does not cause deferral (the run executes); - a retry with ten prior deferrals still defers again — never dispatches — while the holder is live; - delay jitter stays inside the base-to-base-plus-jitter window and clamps out-of-range random sources. - `cd server && pnpm vitest run src/__tests__/heartbeat-` — full heartbeat suite sweep. - `cd server && pnpm run typecheck`. ## Risks - Behavioral shift: shared-workspace runs that used to start immediately now wait for the workspace to free. Against a long-running live holder the wait is unbounded by design — the alternative is dispatching into a held working tree, which is the corruption this PR removes. Every deferral is visible in the run timeline (lifecycle event with the holder run, issue, and attempt number), and the wake is parked, never dropped. - A zombie holder (a `running` row whose process died) delays contending runs by up to the 1 h staleness threshold before it stops counting. Recovery's silent-run escalation targets the same run on the same clock, so this window matches what the system already tolerates for silent active runs. - The holder check and the dispatch are not atomic; two runs that pass the gate in the same instant can still race. The gate closes the common window (a second run waking while the first is mid-execution); the pre-existing sync-conflict handling remains the backstop for the rare simultaneous start. - No schema change, no API change, no new configuration. ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic) via Claude Code — extended thinking, tool use, full repository access; implementation, tests, and verification runs. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-02 12:12:28 -07:00
Devin Foley	6008482aab	fix(decisions): remove clock race from sweep-expiry tests (#10701 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The decisions service lets agents propose decisions with a TTL, and a sweep expires them. > - Four sweep tests create decisions that expire 5 ms in the future, then race the service's own clock read. > - On a loaded CI runner more than 5 ms routinely elapse before validation, so `create` itself rejects the decision and the test fails on unrelated PRs. > - This pull request makes the expiry deterministic: create with a comfortable future TTL, then move `expiresAt` into the past directly in the store. > - The benefit is that the decisions suite stops failing intermittently and stops blocking unrelated PRs. ## Linked Issues or Issue Description No open issue exists; the defect is described here following the bug template. What happened? `decisions-service.test.ts` fails intermittently in CI with `expiresAt must be within 30 days` in `bounds expiration work to the configured batch size` and `falls back to the default sweep batch size for invalid configuration`. The failure hits unrelated PRs — for example the `PR` workflow runs for #10699 failed three times on this suite while the same suite passes locally. Steps to reproduce 1. Run `pnpm vitest run src/__tests__/decisions-service.test.ts` on a machine under load (or add a ~10 ms delay inside `decisionService.create` before the expiry validation). 2. The test builds `expiresAt: new Date(Date.now() + 5)`; by the time `create` validates, `expiresAt.getTime() <= Date.now()` is true. 3. `create` throws `expiresAt must be within 30 days` (the past-expiry branch of the validator) and the test fails before the sweep runs. Expected behavior The sweep tests exercise expiry deterministically and never depend on fewer than 5 ms elapsing between two clock reads in different modules. Paperclip version master (`717684ad8f`); the tests landed with the decisions desk workflow in #10672. Deployment mode Not deployment-specific — CI and local test runs. ## What Changed - `server/src/__tests__/decisions-service.test.ts`: added two helpers — `nearFutureExpiry()` (a 60 s TTL that passes validation with a wide margin) and `expireDecisionNow(id)` (moves the stored `expiresAt` into the past). The four affected tests create decisions with the future TTL, force-expire them through the store, and drop the 10 ms sleeps. The sweep observes the same expired state as before with no scheduler-timing dependence. ## Verification - `cd server && pnpm vitest run src/__tests__/decisions-service.test.ts` — five consecutive local runs, 31/31 passing each. - No production code changed; the diff is test-only. ## Risks - Low risk: test-only change. The force-expire helper writes the store directly, which is the same technique other TTL suites use to avoid sleeping through real time. ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic) via Claude Code — extended thinking, tool use; diagnosis, fix, and verification runs. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-02 12:11:35 -07:00
Devin Foley	2ffebd4836	Test adapters in the environment a run would actually use (#10698 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents execute in environments — local, SSH, or sandboxes — resolved at run time as agent environment → instance default → local > - The Configuration page has a Test button that probes the adapter (working directory, command, a model call) in the environment it will run in > - But the Test sent only the agent's own environment id, with no instance-default fallback, so agents relying on the instance default were probed on the Paperclip host instead > - A sandbox image carrying an extra CLI then fails the Test with "command not found" even though every real run would resolve to the sandbox and succeed — the Test lies about a working setup > - This pull request mirrors the run-time resolution in the Test call via a small shared helper with tests > - The benefit is that the Test button reports the truth about where the agent actually runs ## Linked Issues or Issue Description No existing public issue — inline description following the bug report template: What happened? With the instance default environment set to a sandbox (whose image includes the adapter CLI) and an agent that leaves its environment unset ("use instance default"), the Configuration page's Test fails with `command not found` for that CLI. Expected behavior The Test probes the environment a real run would use — here the instance-default sandbox, where the CLI exists — and passes. Steps to reproduce 1. Set the instance default environment to a sandbox whose image carries an adapter CLI not installed on the Paperclip host (e.g. `grok`). 2. Create a `grok_local` agent without selecting an environment. 3. Press Test on the agent's Configuration page → `command not found`, while a real heartbeat run resolves to the sandbox and works. Paperclip version or commit Reproduced on `sha-53bcf38-cloud`-era master; root-caused in `ui/src/components/AgentConfigForm.tsx` (`environmentId = currentDefaultEnvironmentId \|\| null`) versus the server's `resolveExecutionWorkspaceEnvironmentId` (agent → instance default → local). ## What Changed - New `ui/src/lib/adapter-test-environment.ts`: `resolveAdapterTestEnvironmentId` — agent environment first, else instance default, else null (host probe) — documented as the mirror of the server's run-time resolution. - `AgentConfigForm` uses it in the Test mutation. The raw agent environment id is now sent even when it points at the local environment: the server already resolves the driver and probes the host for local, so explicit-local behavior is unchanged, and the test-environment route's remote paths (SSH/sandbox lease + custom-image template) engage exactly as they do for the fallback environment. - Tests pin the fallback (agent wins; instance default when agent unset; null when neither). Deliberately untouched: the onboarding wizard's adapter test still sends no environment — during onboarding an instance default frequently doesn't exist yet, and changing that flow deserves its own look. ## Verification - `vitest run` on the new helper suite plus both `AgentConfigForm` suites — 19 tests pass; `tsc` clean in `ui/`. - Root cause verified against a live deployment: an agent with `default_environment_id = NULL`, instance default = sandbox environment; the Test posted `environmentId: null` and probed the host (no `Probing inside environment: …` check in the result), which lacks the CLI that the sandbox image carries. ## Risks - Low. The change only widens which environment the Test probes, matching run-time reality. Sandbox-backed tests boot a throwaway sandbox (existing route behavior — lease, custom-image template, archive-on-release), so Tests for instance-default-sandbox agents now take sandbox-boot time instead of failing fast and wrongly. ## Model Used Claude Fable 5 (`claude-fable-5`, extended thinking, via Claude Code with tool use and code execution); diagnosis included live inspection of a deployed instance's agent/environment configuration. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (helper doc-comment carries the rationale; no user-facing doc covers the Test button) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-02 11:23:50 -07:00
Devin FoleyandPaperclip	8540ce2973	ci: shard general-server tests 4 ways (#10663 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Pull request CI must give contributors fast and stable feedback. > - The `general-server` Vitest lane runs many single-worker server suites. > - A recent completed PR run showed this lane as the slowest completed check. > - Three shards still left one runner with the largest share of work. > - This pull request splits that lane into four duration-balanced shards. > - The benefit is a shorter critical path for the same server test coverage. ## Linked Issues or Issue Description No public GitHub issue exists for this CI maintenance change. Pre-submission checklist - I confirmed this improves existing behavior. It does not add a new command, endpoint, or concept. - I searched open public issues and pull requests for related CI sharding work. What existing behavior does this improve? The pull request workflow's `general-server` Vitest lane. Subsystem affected Cross-cutting. This affects GitHub Actions CI and the Vitest shard duration manifest. Current behavior The `general-server` lane uses three shards. The server suites now total about 880 seconds of serial Vitest wall time. The slowest shard was about 313 seconds in the measured run. Proposed behavior The `general-server` lane uses four shards. Each shard receives about 220 seconds of predicted suite weight from the refreshed duration manifest. Reason and benefit The slowest PR check controls how soon a reviewer can trust the PR. Four balanced shards reduce the slowest `general-server` shard while keeping the same suite selection rules. Breaking changes None. This only changes CI partitioning and duration data for existing test suites. Additional context Related public searches found no exact open issue or pull request for this `general-server` sharding change. ## What Changed - Split the `general-server` CI matrix from three shards to four shards. - Refreshed `scripts/general-server-shard-durations.json` with wall-time weights from a recent completed PR run. ## Verification - `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs` - `git diff --check origin/master...HEAD` - Dry-ran the four `general-server` shards locally during implementation. The partition covers 300 unique suites with about 220.56 seconds of predicted weight per shard. - Ran a local sensitive-data scan before push. It found only test filenames that contain words such as `secret` or `token`, not credential values. ## Risks Low risk. The main risk is that the duration manifest becomes stale as suite costs move. Missing suites fall back to the median weight, so the lane still runs if the manifest is incomplete. ## Model Used OpenAI Codex, GPT-5, with tool use and local command execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-08-01 19:03:28 -07:00
Devin Foley	e4b0152ca3	fix(server): keep the agent invokable when a run fails on a workspace sync conflict (#10660 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents on the same project can share one `shared_workspace` clone, and each sandbox run reconciles its git history back into it > - When histories written by different runs genuinely diverge, reconciliation can hit a real merge conflict — a property of the workspace, not of whichever agent happened to run last > - That agent nonetheless finalized into a sticky `error` state, removing a healthy agent from rotation while the workspace stayed broken — and on a shared workspace this serially knocks out every agent that touches it > - This pull request classifies workspace-reconciliation failure signatures as workspace-scoped, so the run still fails with the full message but the agent stays invokable > - The benefit is that one bad workspace state no longer disables agents one by one ## Linked Issues or Issue Description Refs #10645 — this addresses the sticky-agent-error clause of that issue. Workspace run serialization / per-agent worktrees remain tracked there (design sketch on the issue). ## What Changed - New exported `isWorkspaceSyncConflictFailure(message)` matching the reconciliation failure signatures: `merge-tree` conflict ("Failed to merge concurrent remote git histories"), integrate-retry exhaustion ("Failed to integrate concurrent remote git history"), and bundle prerequisite failures ("did not send all necessary objects", "lacks these prerequisite commits"). - Both run-failure finalization paths (adapter returned a failed result; adapter threw) pass `keepIdleOnFailure` for these signatures — the same mechanism already used for provider-quota failures — so the agent finalizes to `idle` instead of `error`. The run itself still fails and carries the full message; nothing about run reporting changes. ## Verification - `pnpm vitest run server/src/__tests__/heartbeat-workspace-session.test.ts` — new signature matrix (4 positive signatures, negatives for unrelated adapter failures and null/empty); 121 tests total. - `cd server && pnpm run typecheck`. ## Risks - Low. The only change is which failure families put the agent into `error`; behavior for every other failure is untouched. A workspace stuck in conflict still fails every run against it (visible on the runs surface) — it just no longer takes agents down with it. ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic) via Claude Code — extended thinking, agentic tool use. No other models involved. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-01 17:53:20 -07:00
Devin Foley	592cade5a6	feat(server): trigger the push-capability preflight from the issue's stated PR deliverable (#10659 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Runs that must push to GitHub have a pre-dispatch credential preflight (`push_write_credential_missing`) so the missing token surfaces as a configuration-incomplete blocker instead of a late runtime failure > - The preflight only triggers when the issue mentions the GitHub PR workflow skill — routine-created issues and agent-to-agent handoffs rarely do, even when their text literally says "push the branch and open a PR" > - In practice the credential gap then surfaced only after implementation and review were complete, stranding finished work > - This pull request adds a conservative, verb-anchored text heuristic over the issue title and description as a second preflight trigger > - The benefit is that the credential ask reaches the human before any work is burned ## Linked Issues or Issue Description Fixes #10644 (completes the prevention set with #10648, #10650, #10658) ## What Changed - `issueTextImpliesPrDeliverable(text)`: matches verb-anchored deliverable statements — "open/create/raise/submit a (draft) pull request/PR", "push … branch/remote/origin/upstream". Verb anchoring deliberately ignores passing mentions ("the PR merged yesterday", "PR feedback addressed"). - `requiresPushCapabilityPreflight` takes the issue's title+description and ORs the text heuristic with the existing skill-mention trigger; adapter-type and issue gating are unchanged. The run-dispatch call site threads the already-loaded issue text — no extra query. ## Verification - `pnpm vitest run server/src/__tests__/heartbeat-workspace-session.test.ts` — new heuristic matrix (4 positive, 6 negative including null/empty) and preflight-by-text cases (text triggers, passing mention does not, no issue → no preflight); 122 tests total. - `cd server && pnpm run typecheck`. ## Risks - A false positive turns into a configuration-incomplete blocker asking for a GitHub token on an issue that didn't need one — the heuristic is intentionally conservative (verb-anchored) to keep that rare, and the blocker names the exact remediation. - No behavior change for issues that neither mention the skill nor state a PR deliverable. ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic) via Claude Code — extended thinking, agentic tool use. No other models involved. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-01 17:43:42 -07:00
Devin Foley	ddbcf53e31	fix(server): refuse agent delegation cycles back to an open ancestor's creator (#10658 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents decompose work by creating child issues assigned to other agents > - When two agents each lack a capability the other assumed (e.g. neither can push to GitHub), each can "resolve" its blocker by delegating the same step to the other: A creates a child for B, B creates a grandchild back for A > - Nothing detects the cycle; the chain of blocked issues grows and no signal reaches the human who could actually fix the capability gap > - This pull request refuses agent-initiated child creation when the child's assignee is the creator of a still-open ancestor in the same chain — a mechanical, semantics-free cycle signal > - The benefit is that the hot-potato dies at creation time with an actionable error instead of growing a dead chain ## Linked Issues or Issue Description Fixes #10642 (write-time counterpart: #10648 refuses assignment to paused agents; the credential-gap preflight side is tracked separately in #10644) ## What Changed - `issueService.findOpenAncestorCreatedByAgent(parentIssueId, agentId, {maxDepth})`: bounded walk up the parent chain looking for a still-open (not done/cancelled) ancestor created by the given agent. - Agent-initiated issue creation with a parent (both the create-with-`parentId` route and `POST /issues/:id/children`) now refuses with a structured 409 (`code: delegation_cycle`, naming the ancestor) when the new child would be assigned to the agent that created a still-open ancestor: that agent delegated the work into this chain, so assigning it back is a cycle. The message states the alternatives — complete the work, leave the child unassigned, or escalate to a board operator. - Deliberately unaffected: human actors (deliberate re-routing is their call), closed ancestors (re-engaging the creator of finished work is normal), and accepted-plan decomposition (its children come from a human-approved plan). ## Verification - `pnpm vitest run server/src/__tests__/issue-assignee-invokability-routes.test.ts` — cycle refused with 409 and no create call; the same child allowed when no open ancestor matches; board actors never consult the guard. - `pnpm vitest run server/src/__tests__/issues-service.test.ts` — new embedded-Postgres coverage: ancestor found through the chain, closed ancestors ignored, depth bound honored (114 total). - `pnpm vitest run server/src/__tests__/issue-create-deduplication-routes.test.ts server/src/__tests__/issue-agent-mutation-ownership-routes.test.ts` — unchanged (79). - `cd server && pnpm run typecheck`. ## Risks - Low-to-moderate: a new 409 for a creation shape that previously succeeded. The blocked shape (agent assigns new work to the creator of an open ancestor) is the cycle signature; the legitimate "hand a subtask to the parent's assignee" pattern is unaffected because it keys on assignee, not creator. Watchdog and plan-decomposition flows are exempt or unaffected as described. - The walk adds at most `maxDepth` (10) single-row lookups per agent child creation with an assignee. ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic) via Claude Code — extended thinking, agentic tool use. No other models involved. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-01 17:43:09 -07:00
Devin Foley	3dec88ce90	feat(agents): warn when an agent's escalation path routes to a paused manager (#10657 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents escalate work up the org chart (`reports_to`), and operators pause agents — notably, instance imports pause every agent by default > - A paused manager does not invalidate the chain (subordinates stay invokable), so nothing surfaces when an operator unpauses workers but leaves their manager paused > - Escalations then dead-letter silently: agent-created issues assigned to the paused manager sit in a queue nothing will ever run > - This pull request computes paused ancestors in the existing org-chain health model and surfaces a non-blocking warning on the agent read models and detail page > - The benefit is that the operator learns their escalation paths are dead before work vanishes into them ## Linked Issues or Issue Description Fixes #10647 (companion to #10648, which refuses agent-initiated assignment to paused agents at write time — this PR makes the standing hazard visible) ## What Changed - `AgentOrgChainHealth` gains two additive, optional fields: `pausedAncestors` (paused agents in the `reports_to` chain) and `escalationWarning` (human-readable, only set when the agent itself can work — a paused/terminated agent's escalation path is moot). Chain validity, invokability, and assignability are byte-identical. - No server route changes needed: the fields flow through every existing agent read model (list, detail, org chart) since they ride the same `getAgentWorkEligibility` computation. - Agent detail page shows an amber "Escalation path is paused" banner (same visual language as the invalid-chain banner, but non-blocking) with the warning text naming the paused manager and the two remedies. ## Verification - `pnpm vitest run packages/shared/src/agent-eligibility.test.ts` — 5 new cases: paused direct manager warns; paused grandparent through a healthy manager warns; the agent itself paused → no warning (but ancestors still reported); fully active chain → no warning, empty list; terminated ancestor keeps the invalid-chain classification without double-counting as paused. - Full `@paperclipai/shared` suite (392 tests) and `agent-eligibility-routes` (54) unchanged. - `tsc --noEmit` in shared, server, and ui. ## Risks - Low. Purely additive fields plus one UI banner; no behavior gates on the new data. ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic) via Claude Code — extended thinking, agentic tool use. No other models involved. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-01 17:42:42 -07:00
Devin Foley	a0dbe21045	fix(server): stand down recovery while an operator-cancelled run is the latest activity (#10656 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Operators can cancel a running agent from the board when a run is unwanted — most acutely while cleaning up a runaway loop > - The recovery machinery treats a cancelled run like any other unsuccessful terminal run: the stranded-issue sweep classifies the issue as stranded, creates a recovery action, and wakes the agent again > - So cancelling runs to stop a loop fed the loop: each operator cancel spawned a recovery action that re-woke the agent the operator had just stopped > - This pull request stamps board-initiated cancellations with operator attribution and makes the sweep stand down while such a run is the issue's latest activity > - The benefit is that an operator's cancel is final until something new happens, instead of being fought by automation ## Linked Issues or Issue Description Fixes #10646 ## What Changed - `POST /heartbeat-runs/:runId/cancel` (board-only) now cancels with an explicit reason ("Cancelled by a board operator") and stamps `resultJson.cancelledByActorType: "user"` / `cancelledByUserId`. - `reconcileStrandedAssignedIssues` gains an early stand-down: when the issue's latest run is operator-cancelled (the new stamp, or the existing `operator_interrupted` error code from interrupt-by-comment), the issue is skipped entirely — no recovery action, no wake — and counted in a new `operatorCancelExempted` result field. The exemption is inherently self-limiting: any newer run or wake supersedes it because the gate only looks at the latest run. - System cancellations without operator attribution (lease expiry, assignee changes, terminal-status cancels, pause holds) keep today's recovery behavior unchanged. ## Verification - `pnpm vitest run server/src/__tests__/issue-recovery-actions.test.ts` (embedded Postgres) — 3 new cases: a stamped operator cancel produces zero recovery actions and zero wakes; an `operator_interrupted` cancel likewise; an unattributed system cancel still flows into pre-existing recovery (wake observed), proving the stand-down is scoped to operator attribution. - `pnpm vitest run server/src/__tests__/heartbeat-process-recovery.test.ts server/src/__tests__/issue-scheduled-retry-routes.test.ts` — unchanged (109 tests). - `cd server && pnpm run typecheck`. ## Risks - Low. The only suppressed behavior is recovery of runs a human explicitly cancelled from the board; everything else is byte-identical. If an operator cancels and walks away, the issue stays quiet until any new activity — which is the intent (the operator owns the next step). ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic) via Claude Code — extended thinking, agentic tool use. No other models involved. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-01 16:51:13 -07:00
Devin Foley	0f12721ee9	fix(server): let board users cancel issues with an active review stage (#10655 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The server enforces an issue execution policy. It gates status changes while a review or approval stage is active. > - A board user could not cancel a task while an agent reviewer held the active stage. The API returned "Only the active reviewer or approver can advance the current execution stage". > - Board users own the board. They must always be able to edit and cancel any task. > - This pull request adds a board override to the execution stage transition. A board cancel clears the pending stage state and proceeds instead of raising an error. > - The benefit is that board users can always stop work, even while a review is pending or the stored stage state has drifted. ## Linked Issues or Issue Description No public GitHub issue exists for this bug. Description follows `bug_report.yml`: What happened? A board user set a task to `cancelled` while the task had an active reviewer stage held by an agent. The PATCH failed with "Task Update Failed... only the assigned approver or reviewer...". The same failure occurred when the stored stage state had drifted: the server silently forced the task back to `in_review` instead of honoring the cancel. Expected behavior A board user can always edit and cancel any task. A board cancel must clear the pending review stage and apply the requested status. Steps to reproduce 1. Create a task assigned to agent A with agent B configured as reviewer in the execution policy. 2. Let agent A hand the task off so the review stage becomes active. 3. As the board user, set the task status to Cancelled. 4. The update fails with the reviewer-only error. Related work: #5487 touches the execution-policy approver UI. It does not address the board cancel path. ## What Changed - `server/src/routes/issues.ts`: the issue PATCH route now passes `allowBoardOverride` when the actor is a board user. - `server/src/services/issue-execution-policy.ts`: when `allowBoardOverride` is set and the requested status is not `in_review` or `in_progress`, the transition clears `executionState` and proceeds. This applies both while a stage decision is pending and when the stage state has drifted, so a board cancel is no longer rejected or silently flipped back to `in_review`. - Reviewer gating is unchanged for everyone else: a board user who is the active participant still uses the normal approve / request-changes flow, and non-participant agents still receive the 422 guard. - Assignee-only board updates on an `in_review` task keep the stage state coherent: reassigning to an eligible stage participant re-pends the stage with them as the current participant, while reassigning to a non-participant (or unassigning) dissolves the review back to `in_progress` instead of persisting an `in_review` issue with no execution state or an ineligible participant. - New unit tests and route tests cover board cancellation of an active review stage and of a drifted pending review, plus reviewer swap, non-participant reassignment, and unassignment during an active review. ## Verification - In `server/`: `pnpm exec vitest run src/__tests__/issue-execution-policy.test.ts src/__tests__/issue-execution-policy-routes.test.ts` — 2 files, 73/73 tests pass on top of current `master`. - In `server/`: `pnpm run typecheck` passes. ## Risks - Low risk. The override branch runs only for board actors and only for target statuses other than `in_review` and `in_progress`. Cancelling clears `executionState`, so a later reopen starts from a fresh stage state. Agent-facing flows and reviewer gating are unchanged. ## Model Used - Claude Fable 5 (Anthropic), model ID `claude-fable-5`, running in Claude Code (Claude Agent SDK) with extended thinking and agentic tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-01 15:43:13 -07:00
Devin Foley	ada47be764	fix(server): refuse agent-initiated issue assignment to paused agents (#10648 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents can create and assign issues to other agents, and commonly escalate to their org-chart manager (`reports_to`) when they hit something outside their authority > - Issue assignment already refuses terminated and pending-approval assignees, but accepts paused assignees from any actor > - A paused agent never runs, so agent-initiated escalations to a paused manager become invisible dead letters — accepted silently, never picked up, never surfaced > - This pull request refuses paused assignees when the assigning actor is an agent, at the single normalization helper all four assignment paths flow through > - The benefit is that agent-routed work can no longer silently vanish into a paused agent's queue ## Linked Issues or Issue Description Fixes #10641 ## What Changed - `normalizeIssueAssigneeAgentReference` (used by issue create, both child-create routes, and issue update) now throws a 409 when an agent actor assigns to a paused agent, with a message naming the alternatives: assign an invokable agent, leave the issue unassigned, or escalate to a board operator. - Board/user actors are unchanged and may still assign to paused agents deliberately — the pause state is visible in the UI, and staging work for a later unpause is a legitimate workflow. Terminated / pending-approval / invalid-org-chain refusals are unchanged for all actors. - This matches the existing precedent for watchdogs ("Cannot assign watchdog to an agent that is not invokable") using the same conflict-error shape. ## Verification - `pnpm vitest run server/src/__tests__/issue-assignee-invokability-routes.test.ts` — new coverage: agent PATCH → paused assignee 409 (no update call), agent child-create → paused assignee 409 (no create call), agent assignment to an invokable agent still 200, board assignment to a paused agent still 200. - Neighboring suites unchanged: `issue-update-comment-wakeup-routes`, `issue-agent-mutation-ownership-routes`, `issue-create-deduplication-routes`, `issue-watchdogs-routes` (97 tests). - `cd server && pnpm run typecheck`. ## Risks - Low. The only behavior change is a new 409 for agent actors assigning to paused agents — previously a silent dead-letter. Agents that relied on this (escalation flows) now get an actionable error instead; human workflows are untouched. ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic) via Claude Code — extended thinking, agentic tool use. No other models involved. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-01 15:05:16 -07:00
Devin Foley	27f8c8dbcf	feat(server): cap agent review rounds and escalate exhausted reviews to the responsible human (#10650 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Execution policies let one agent implement and another review, cycling through changes-requested → addressed rounds > - Nothing bounds that cycle: no round counter, no escalation, no termination signal — two agents can ping-pong indefinitely, especially when the review's success criteria drift to something the implementer cannot satisfy > - On a real multi-agent instance this produced 6+ unattended rounds (~8 runs) that continued even after the human had merged the PR under review > - This pull request counts consecutive agent-initiated changes-requested rounds and, at a configurable cap, hands the still-pending review to the responsible human instead of bouncing back to the implementer > - The benefit is that unattended review loops terminate in a human decision instead of burning runs forever ## Linked Issues or Issue Description Fixes #10643 ## What Changed - `IssueExecutionState.changesRequestedCount` (schema + type, default 0): consecutive agent-initiated changes-requested rounds on the current stage. Carries through executor resubmissions, resets to 0 on approval, and resets when a human makes the changes-requested decision — the cap targets unattended agent↔agent ping-pong, never human review. - `IssueExecutionPolicy.maxReviewRounds` (optional, 1–50, default null → server default `DEFAULT_MAX_REVIEW_ROUNDS = 3`). - At the cap, the transition records the reviewer's changes-requested decision as usual but keeps the stage pending with the responsible human (`responsibleUserId`, falling back to `createdByUserId`) as the participant: the issue is assigned to that human and the pending review surfaces through the existing attention/review UI. The human then approves, requests changes (resetting the counter and handing back to the implementer), or re-scopes. - The escalated hold is sticky: transitions from anyone other than the escalated human no longer re-select a configured agent participant for the stage (which would have silently undone the escalation on the next unrelated PATCH). The escalated human's own decisions flow through the normal participant decision branch. - Issues with no responsible human keep today's hand-back behavior; the counter still accumulates so operators can see the churn. ## Verification - `pnpm vitest run server/src/__tests__/issue-execution-policy.test.ts` — 8 new cases: round counting on hand-back, count carried through resubmission, escalation at the default cap, sticky hold across unrelated transitions, human changes-requested resets the counter, human approval completes the stage, no-responsible-human fallback, and a `maxReviewRounds: 1` policy override. - `pnpm vitest run server/src/__tests__/issue-execution-policy-routes.test.ts` and the full `@paperclipai/shared` suite (387 tests) — schema additions are backward compatible (both fields optional with defaults; persisted states without the counter parse as 0). - `pnpm --filter @paperclipai/shared exec tsc --noEmit` and `cd server && pnpm run typecheck`. ## Risks - Behavior change: an agent-only review loop that previously ran forever now escalates to a human after 3 agent rounds by default. Instances that want longer loops can set `maxReviewRounds` per policy. Flows where a human participates are unaffected (human decisions reset the counter). - Escalation requires a `responsibleUserId`/`createdByUserId` on the issue; without one, behavior is unchanged. - Persisted execution states from before this change parse with `changesRequestedCount: 0` — no migration needed. ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic) via Claude Code — extended thinking, agentic tool use. No other models involved. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-01 15:04:56 -07:00
Devin FoleyandPaperclip	8444e5735c	ci: pass e2e shard specs without separator (#10640 ) ## Thinking Path > - Paperclip uses pull request CI to test changes before merge. > - The e2e PR lane runs Playwright specs in a shard matrix. > - Each shard builds a list of spec files for its matrix entry. > - The workflow passed that list after a literal `--` separator. > - Playwright did not receive the list as file filters. > - This pull request removes the separator and adds a guard test. > - The benefit is that each e2e shard runs only its assigned specs. ## Linked Issues or Issue Description Refs #10629. What happened? The e2e shard step used `pnpm run test:e2e -- $specs`. The shard spec list was not applied as Playwright file filters. Expected behavior Each e2e shard should pass only its selected specs to Playwright. Steps to reproduce 1. Inspect `.github/workflows/pr.yml` at the merge commit for #10629. 2. Find the `e2e_shards` command that invokes `pnpm run test:e2e`. 3. See the literal `--` before `$specs`. Paperclip version or commit `86767951` Deployment mode GitHub Actions PR CI. ## What Changed - Removed the literal `--` from the e2e shard `pnpm run test:e2e $specs` invocation. - Added a regression test that checks the workflow passes `$specs` without that separator. ## Verification - `node --test scripts/__tests__/e2e-shard.test.mjs` ## Risks Low risk. This changes one CI command and one workflow guard test. The main risk is shell argument handling in the workflow, and the guard now covers the expected command shape. ## Model Used OpenAI GPT-5 through Codex. The run used shell and GitHub CLI tool access. The runtime did not expose a context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-08-01 15:04:21 -07:00
Devin FoleyandCody	8676795188	ci: split e2e PR lane into three shards (#10629 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The pull request workflow protects changes with a Playwright e2e lane. > - That lane already uses a weighted file partition so slow specs do not cluster by test count. > - Recent green PR runs showed the two e2e shard jobs were slower than the next slow required lane. > - The largest spec is indivisible, so a third shard lets that spec run alone and lets the rest split by duration. > - This pull request changes only the PR e2e shard matrix and the guard test. > - The benefit is a shorter expected PR critical path while the required `e2e` aggregate check name stays stable. ## Linked Issues or Issue Description Refs #9923 What existing behavior does this improve? The `pull_request` workflow Playwright e2e lane. Subsystem affected Cross-cutting: GitHub Actions CI and test scripts. Current behavior The PR workflow runs the weighted Playwright e2e partition across two jobs. Recent green runs showed those jobs as the slowest required checks. Proposed behavior The PR workflow runs the same e2e spec set across three weighted jobs. The aggregate required check stays named `e2e`. Reason and benefit The third shard lets the slow smoke-lab spec run alone while the rest of the catalog stays balanced. This should shorten the PR critical path. The win is bounded by fixed per-job setup time. Breaking changes None. The required aggregate check contract is preserved. ## What Changed - Change the PR e2e shard matrix from two entries to three entries. - Update the shard guard test to expect three shards. - Floor the balance bound at the largest single spec weight. - Assert that the workflow does not define more shard indexes than `SHARD_COUNT`. ## Verification - `node --test ./scripts/__tests__/e2e-shard.test.mjs` passes with 6 tests. - The recorded-weight partition is complete and non-overlapping: 168.0s, 116.5s, and 114.4s. - I checked `ROADMAP.md` and found no overlapping roadmap-level core feature. - I searched public GitHub PRs and issues for related e2e shard work. I found related PR #9923 and no open duplicate for this branch or change. ## Risks - This adds one extra GitHub Actions runner to the PR e2e lane. - The wall-clock win is bounded by fixed per-job setup. - Behavior risk is low because the aggregate required check remains named `e2e`. ## Model Used OpenAI Codex, GPT-5, tool-enabled coding agent in this repository. The runtime did not expose the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Cody <noreply@paperclip.ing>	2026-08-01 10:10:22 -07:00
Devin Foley	6401f4f78c	fix(sandbox): bundle git copy-back against the merge-base so diverged/reset workspaces still import (#10601 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - An agent that runs in a sandbox has its workspace copied back to the host when the run ends, so its work persists — the copy-back ships a git bundle of the sandbox's commits > - The bundle is created as a thin delta, `git bundle create HEAD --not <baseSha>`, which records `baseSha` (the host workspace HEAD captured at export) as a prerequisite the host must already hold > - That assumption breaks when the sandbox HEAD has diverged from `baseSha`, or when a shared host workspace no longer holds `baseSha` at import time — then `git fetch` on the host hard-fails and the entire run is lost even though the agent finished its work > - This pull request bundles against the merge-base of `baseSha` and the sandbox HEAD (with a full-bundle fallback), which the host can satisfy in those cases > - The benefit is that copy-back no longer discards a completed run's work over a base the host can't reconcile ## Linked Issues or Issue Description What happened? A sandbox agent run completed its work, then failed during workspace finalize: ``` git -C <host workspace> fetch --force <git-delta.bundle> refs/…/export:refs/…/imported error: Could not read <baseSha> fatal: revision walk setup failed error: git-delta.bundle did not send all necessary objects ``` The run is reported as `adapter_failed` even though the agent produced output. The copy-back bundle names the host workspace's recorded HEAD (`baseSha`) as a prerequisite, but the host cannot satisfy it. Steps to reproduce Two independent triggers, both reproduced in tests: 1. The sandbox's HEAD has diverged from `baseSha` — e.g. the sandbox carries a local-only branch that forked from an older commit than the host's current HEAD. 2. The shared host workspace no longer holds `baseSha` at import time (it was reset / re-realized between export and import). In either case `git fetch` of the thin bundle fails with a missing prerequisite. Expected behavior Copy-back imports the sandbox's work as long as the host holds any common ancestor, instead of hard-failing and discarding the run. Paperclip version Current `master`. Deployment mode Any deployment running agents in sandbox environments with workspace sync (notably shared-workspace clones and custom images that carry a local-ahead branch). ## What Changed - `buildRemoteGitDeltaBundleScript` now computes `bundle_base = git merge-base <baseSha> HEAD` and bundles `HEAD --not <bundle_base>`. The merge-base is an ancestor of `baseSha`, so any host that holds `baseSha` (or an ancestor of it — e.g. after a reset) can satisfy the prerequisite, and the bundle stays a delta rather than a full-history transfer. - When `baseSha` is absent from the sandbox, or no merge-base exists, it falls back to a full, self-contained bundle (no prerequisites) so the import can always complete. - The existing empty-bundle no-op (no new commits) and the ordinary fast-forward path are unchanged; the `cat-file` base check no longer aborts the script under `set -e`. ## Verification - `pnpm vitest run packages/adapter-utils/src/git-workspace-sync.test.ts packages/adapter-utils/src/sandbox-managed-runtime.test.ts` — new cases: a diverged sandbox HEAD imports when the host holds only the merge-base (not `baseSha`), and the full-bundle fallback imports into a host that shares no history; existing thin-delta and empty-bundle cases still pass. - `pnpm --filter @paperclipai/adapter-utils exec tsc --noEmit`. - Standalone shell repro confirmed the old thin bundle fails with "Repository lacks these prerequisite commits" in both trigger cases, and the merge-base bundle imports successfully. ## Risks - Low. For the common case (sandbox HEAD descends from `baseSha`) the merge-base is `baseSha`, so the bundle is byte-for-byte the same delta as before. The change only alters behavior when the old code would have hard-failed. - This makes the copy-back import succeed on a diverged base; the subsequent reconciliation of divergent histories (`integrateImportedGitHead`) is unchanged and still owns how the imported head is merged into the host branch. Where a workspace's history has genuinely diverged (e.g. a stale custom image carrying a local-only branch), a clean re-clone/re-capture is still the right operational fix — this change prevents work loss, it does not reconcile intentional divergence. ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic) via Claude Code — extended thinking, agentic tool use (file edits, shell repro, vitest/tsc runs). No other models involved. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-31 23:41:42 -07:00
Devin Foley	ee9d907d01	fix(codex): do not inject a duplicate --skip-git-repo-check for sandbox runs (#10595 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A Codex agent runs `codex exec`, and the adapter assembles its argument vector from the agent's config plus execution-context options > - For sandbox execution the adapter injects `--skip-git-repo-check`, because a headless remote workspace has no git trust prompt to answer > - The adapter also appends the operator's `extraArgs` verbatim, so an agent that already lists `--skip-git-repo-check` in its config gets the flag twice on a sandbox run > - `codex exec` rejects a repeated `--skip-git-repo-check` and exits with code 2, which the adapter surfaces as `adapter_failed` before any work runs > - This pull request skips the sandbox injection when the operator's args already carry the flag > - The benefit is that a common, harmless-looking config no longer crashes every sandbox run ## Linked Issues or Issue Description What happened? A `codex_local` agent configured with `extraArgs: ["--skip-git-repo-check"]` fails on every sandbox run: ``` error: the argument '--skip-git-repo-check' cannot be used multiple times Usage: codex exec [OPTIONS] [PROMPT] ``` The adapter reports `stopReason: "adapter_failed"` (Codex exited with code 2). The flag appears twice in the argv: once injected by the adapter for sandbox execution, once from the operator's `extraArgs`. Steps to reproduce 1. Configure a `codex_local` agent with `extraArgs: ["--skip-git-repo-check"]` (or the legacy `args` field). 2. Point it at a sandbox environment. 3. Start a run — `codex exec` aborts immediately on the duplicate flag. Expected behavior The run launches with a single `--skip-git-repo-check`. An operator listing the flag the adapter already injects should be a no-op, not a hard failure. Paperclip version Current `master`. Deployment mode Any deployment running Codex agents in sandbox environments. ## What Changed - `buildCodexExecArgs` no longer pushes the sandbox `--skip-git-repo-check` when the resolved args (`extraArgs`, or the legacy `args` fallback) already contain it. The operator's copy stands; the argv carries the flag exactly once. Non-sandbox runs and configs without the flag are unchanged. ## Verification - `cd packages/adapters/codex-local && pnpm vitest run src/server/codex-args.test.ts` — new cases: `extraArgs` already carrying the flag (single occurrence), the legacy `args` field carrying it (single occurrence), and the operator's flag preserved when the sandbox injection is not requested. Existing "adds --skip-git-repo-check when requested" case unchanged. - `cd packages/adapters/codex-local && pnpm vitest run` — full package suite (218 tests). - `pnpm run typecheck` in the package. ## Risks - Low. The change only suppresses a duplicate of a single, idempotent flag; it never removes an operator-supplied argument and never adds one that was not already going to be present. ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic) via Claude Code — extended thinking, agentic tool use (file edits, vitest/tsc runs). No other models involved. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-31 20:37:51 -07:00
Devin Foley	c0b875c46c	fix(codex): let sandbox runs use the sandbox image's own Codex login (#10582 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Codex agents can run inside sandbox environments, and operators can bake a Codex login into the sandbox image during interactive image setup > - Two credential gates (the control plane's pre-dispatch configuration-incomplete gate and the adapter's execute-time fail-fast) required host-side Codex credentials — a usable `auth.json` in the managed home or a configured `OPENAI_API_KEY` — regardless of where the run executes > - On managed cloud hosts a local Codex login never exists, so every sandbox run of a Codex agent failed immediately with "configuration incomplete: no Codex credentials available for managed home …", even though the adapter's inbound auth merge already supports the image-login case end to end > - This pull request makes the execute-time gate probe the sandbox for its own `~/.codex/auth.json` before failing, and exempts sandbox-destined runs from the pre-dispatch host check > - The benefit is that a sandbox image signed in to Codex is a first-class credential source, matching what the auth-merge, precedence-warning, and copy-back machinery were already built for ## Linked Issues or Issue Description What happened? Running a `codex_local` agent in a sandbox environment whose image carries a Codex login failed instantly with `configuration incomplete: no Codex credentials available for managed home "…/codex-home". Sign in to Codex on the host with a ChatGPT subscription, or bind a per-agent OPENAI_API_KEY secret for this agent.` The host has no Codex login and never will on a managed cloud deployment; the sandbox's own login was never consulted. Steps to reproduce 1. Configure a sandbox environment and capture a custom image after signing in to Codex inside the interactive image setup. 2. Create a `codex_local` agent that uses that environment, on a host with no Codex login and no `OPENAI_API_KEY` bound. 3. Start a run: it fails pre-dispatch with the configuration-incomplete blocker above. Expected behavior The run launches and Codex authenticates with the sandbox image's own login, the same way the adapter's host↔sandbox auth merge already keeps the sandbox credential when the host ships none. A run should only fail fast when neither the host, a bound `OPENAI_API_KEY`, nor the sandbox has credentials. Paperclip version Current `master` (cloud image deployments). Deployment mode Managed cloud stacks (any deployment where the server host has no local Codex login). ## What Changed - Extracted the adapter's execute-time gate into `assertCodexCredentialsLaunchable`: when host readiness fails and the target is a sandbox, it probes `~/.codex/auth.json` in the sandbox (same command the auth-precedence warning uses) and proceeds with a log line naming the credential source; when the sandbox has no login either, the error now names all three remediation options (sandbox image sign-in, per-agent `OPENAI_API_KEY`, host sign-in). Non-sandbox targets keep today's strict behavior byte-for-byte. - The control plane's pre-dispatch gate in `resolveExecutionRunAdapterConfig` now takes the selected environment's driver and skips the host-credential check for sandbox-destined runs — only the adapter can probe the sandbox once it is up, so the execute-time gate is the authority there. Non-sandbox runs keep the early, well-attributed configuration-incomplete blocker. - The codex Test flow needed no change: it already seeds host credentials only when they exist and otherwise leaves the sandbox's `CODEX_HOME` alone; this aligns the run path with it. ## Verification - `cd packages/adapters/codex-local && pnpm vitest run` — 210 tests, including new gate cases: sandbox login present (proceeds + logs source), sandbox and host both credential-less (fails with the extended message), non-sandbox target (strict host requirement kept, no sandbox probe), per-agent API key (no probe at all). - `cd server && pnpm vitest run src/__tests__/heartbeat-project-env.test.ts src/__tests__/codex-local-adapter-environment.test.ts` — includes the new sandbox-exemption case next to the existing blocker tests. - `pnpm run typecheck` in `server` and `packages/adapters/codex-local`. ## Risks - Sandbox-destined misconfigurations (no credentials anywhere) now surface at adapter execute time instead of pre-dispatch, so they read as an adapter failure with a precise message rather than a configuration-incomplete blocker. The trade-off is deliberate: the sandbox must be up to know whether credentials exist, and the failure message names the exact remediations. - The sandbox probe adds one short (5s-capped) shell command to sandbox runs whose host has no credentials; runs with host credentials or a bound key are untouched. - Self-hosted behavior is unchanged for local and SSH targets. ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic) via Claude Code — extended thinking, agentic tool use (file edits, vitest/tsc runs). No other models involved. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-31 18:36:36 -07:00
Devin Foley	90ead239a8	feat(ui/server): name cross-company environment secret refs instead of calling them missing (#10577 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Environment configs (sandbox providers, SSH) can bind stored company secrets through `format: "secret-ref"` fields, picked in the environment editor's secret picker > - Environments are instance-scoped and shared by every company on an instance, but the picker lists only the current company's secrets, so a ref pointing at another company's secret renders as "Missing secret (…)" in destructive styling > - That state is indistinguishable from a genuinely deleted secret, so operators "fix" a healthy binding by creating a duplicate secret in their own company — the exact sequence that used to corrupt bindings before #10576 > - This pull request adds an instance-gated metadata endpoint for an environment's secret refs and teaches the picker to name a cross-company secret and its owner honestly > - The benefit is that operators can tell a healthy cross-company binding from a broken one, and stop creating duplicate secrets ## Linked Issues or Issue Description Is your feature request related to a problem? Please describe. In the environment editor, a secret-ref field that points at a secret owned by a different company shows "Missing secret (22095402…)" in red, with "The previously selected secret is no longer available. Pick another or remove the binding." The binding is actually healthy — the current company's picker just cannot list the other company's secrets. Operators react by creating a duplicate secret and re-pointing the field. Describe the solution you'd like The editor should know the referenced secret's name, status, and owning company (metadata only, never the value) and present a cross-company ref neutrally, a deleted secret as deleted, and only an unknown id as missing. Related: #10576 (fixes the binding corruption this UI state used to trigger). ## What Changed - New `GET /environments/:id/secret-refs` returns `{ refs: [{ configPath, secretId, name, status, companyId, companyName }] }` for the environment's config-derived secret refs. Values are never returned. The route sits behind `assertCanAccessInstanceEnvironments`, the same gate as environment editing. - New `secretService.describeSecretRefs` loads that metadata across companies; unknown ids are omitted. - `SecretBindingPicker` reads an optional `SecretRefHintsContext` (keyed by secret id). With a hint, a ref the company list cannot show renders as `NAME — Owning Company` with neutral styling and the note "Owned by the … company. The binding keeps working; selecting a secret from this list re-points it here." A hint with `status: "deleted"` reports the secret as deleted. Without hints, behavior is byte-identical to before — agent editors and other picker users are unaffected. - `CompanyEnvironments` fetches descriptors for the environment being edited and provides them through the context. ## Verification - `cd server && pnpm vitest run src/__tests__/environment-routes.test.ts src/__tests__/secrets-service.test.ts` — new endpoint happy path, agent 403 (descriptors never computed), and embedded-Postgres coverage proving cross-company names resolve and unknown ids drop out. - `cd ui && pnpm vitest run src/components/SecretBindingPicker.test.tsx src/components/JsonSchemaForm.test.tsx src/pages/CompanyEnvironments.test.tsx` — hinted cross-company rendering, hinted deleted secret, and unchanged no-hint fallback. - `pnpm run typecheck` in `server` and `ui`. - Manual: edit an environment whose secret-ref field references another company's secret; the field names the secret and its owning company instead of "Missing secret". ## Risks - The endpoint exposes secret names and company names across companies to instance-level environment editors. Those actors already manage instance-shared environments (and instance admins are implicit members of every company), so this reveals no secret material and no new reach; the service method documents that callers must sit behind an instance-level gate. - UI change is additive and context-gated; pickers without a provider render exactly as before. ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic) via Claude Code — extended thinking, agentic tool use (file edits, vitest/tsc runs). No other models involved. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-31 16:46:26 -07:00
Devin Foley	f51cba33fa	fix(server): keep environment secret bindings consistent when re-pointing config secrets (#10576 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents can run inside environments (SSH boxes, sandbox providers); a sandbox environment's config can reference stored company secrets (for example a provider API key) through `format: "secret-ref"` fields > - Environments are instance-scoped and shared by every company on an instance, but `company_secret_bindings` rows are company-scoped, and the environment routes synced config-derived bindings under one guessed "context company" resolved from the environment's existing bindings > - When a save re-pointed a secret-ref field at a secret owned by a different company, the binding sync threw after the config row had already been persisted: the config referenced the new secret, the binding still pointed at the old one, every later lease acquisition failed with `Secret is not bound to environment:<id> at apiKey`, and the stale cross-company binding made every later save fail with a company-context conflict — with no route-level way to recover > - This pull request makes config-derived bindings follow the company that owns each referenced secret, and makes the environment write and its binding syncs atomic > - The benefit is that environment saves can no longer strand an environment in a half-updated state that breaks all of its runs ## Linked Issues or Issue Description Refs #10577 (companion UX change: the editor state that nudges operators into this sequence). What happened? Saving an environment whose secret-ref config field points at a secret owned by a different company than the environment's existing binding partially applied: the config row updated, the binding sync failed server-side, and the environment was left referencing a secret it has no binding for. Every run that leased the environment then failed with `lease_acquire_failed: ... Secret is not bound to environment:<id> at apiKey`, and every later save of the environment returned 409 `Environment secret bindings already use a different company context.` — with no route-level way to recover. Steps to reproduce 1. On an instance with two companies, create a sandbox environment from company A with a picker-bound API-key secret owned by A (the binding lands in A). 2. From company B, create a new secret and re-point the environment's API-key field at it, then save. 3. The save persists the config but the binding sync throws, so no binding for B's secret exists. 4. Run any agent that uses the environment, or try to save the environment again. Expected behavior The save either fully applies (config and bindings consistent) or fully fails. Re-pointing a config secret ref to a secret owned by another company moves the binding with the secret. Paperclip version Reproduced on current `master` (also present on recent release images). Deployment mode Multi-company server deployment (any mode with more than one company on the instance). ## What Changed - New `secretService.replaceSecretRefsForInstanceTarget`: writes each config-derived binding under the company that owns the referenced secret, replaces all non-`env.` bindings of the target across every company, and validates every ref (secret exists, not deleted, config-path and projection-class rules) before any row is written. `env.` env-var bindings stay company-scoped and untouched. - The environment create and update routes now run the environment write and its binding syncs inside one `db.transaction`, threading the transaction through new optional executor seams on `environmentService.create/update` and the existing `SecretBindingDb` seam pattern, so an invalid ref rolls the whole save back instead of leaving a half-updated environment. - `resolveEnvironmentSecretContextCompanyId` no longer lets existing bindings veto the caller's context (the 409s above); it now only picks where new raw-pasted secrets are created and how env-var bindings and probes resolve: explicit route/query company first, then the single company the bindings live in, then the actor's company. ## Verification - `cd server && pnpm vitest run src/__tests__/environment-routes.test.ts src/__tests__/environment-instance-routes.test.ts src/__tests__/secrets-service.test.ts src/__tests__/environment-custom-image-routes.test.ts` (165 tests, includes new coverage below) - New embedded-Postgres tests prove: a re-point moves the binding to the new secret's company and deletes the stale row; refs across several companies each bind under their own secret's company; an unknown secret ref rejects without touching existing bindings; `env.*` rows survive config-ref replacement. - New route tests prove: a cross-company re-point that previously 409'd now saves, with the update and binding replacement on the same transaction executor; a failing ref surfaces as 422. - `cd server && pnpm run typecheck` ## Risks - Behavioral shift: environment saves no longer 409 on a company-context mismatch between the caller and existing bindings; bindings follow the referenced secret's company instead. Environment routes are instance-admin gated, and instance admins already had access to every company's secrets by passing the company explicitly, so this removes an ordering trap rather than widening access. - Runtime lease resolution is unchanged: a run still resolves environment secrets under the run's own company, so an environment referencing company B's secret still only leases for company B runs (fail-closed as before). - The delete route's per-company binding cleanup is unchanged. ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic) via Claude Code — extended thinking, agentic tool use (file edits, vitest/tsc runs). No other models involved. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-31 16:19:25 -07:00
Devin Foley	ea0dd3917e	build: make image layer caching actually hit (#10571 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Published container images are the deployable unit for self-hosted and managed instances, so merge-to-image latency bounds every deploy iteration > - The docker workflow configures BuildKit caching, but builds still ran ~12+ minutes essentially cold > - Two causes: the most expensive layer (four CLI toolchains + apt) is ordered after the always-changing app copy so it can never cache, and the type=gha cache's 10GB repo cap means the two multi-arch mode=max jobs evict each other > - This pull request reorders the tool layer above the app copy (with a weekly epoch so @latest tools keep advancing) and switches both jobs to registry-backed cache in ghcr > - The benefit is that warm builds shrink to roughly the app build + push, targeting the sub-5-minute range together with the amd64-only cloud variant ## Linked Issues or Issue Description No existing public issue — inline description following the feature request template: Subsystem affected CI / release publishing (docker workflow, Dockerfile) Problem or motivation Despite `cache-from/cache-to` being configured, image builds run effectively cold: (1) the production stage installs four CLI toolchains + apt packages after `COPY --from=build /app /app`, and since the app copy changes every commit, that most-expensive layer rebuilds every build, per arch; (2) the `type=gha` BuildKit cache is capped at 10GB per repository, and two multi-arch `mode=max` jobs overflow and evict each other's entries. Proposed solution Order the tool/OS layer before the app copy (it references nothing from `/app`), refresh it weekly via a `CLI_TOOLS_CACHE_EPOCH` build arg so the `@latest` tools don't freeze in the cache, and move both jobs to registry-backed BuildKit cache (`:buildcache` / `:buildcache-cloud` refs in ghcr, no size cap, separate refs so the parallel jobs don't clobber each other). Alternatives considered Pinning CLI tool versions instead of the weekly epoch — more deterministic, but adds a version-bump chore; the weekly epoch preserves current freshness semantics with bounded staleness. Keeping type=gha with `mode=min` — smaller cache but loses intermediate-stage reuse, which is where most of the win is. Roadmap alignment Not on ROADMAP.md; CI/publishing speed improvement only. ## What Changed - `Dockerfile`: the production stage's tool/OS `RUN` (npm --global CLIs, apt, `/paperclip` setup) moves above `COPY --from=build /app /app`; new `CLI_TOOLS_CACHE_EPOCH` arg consumed by that layer. The `cloud` stage is unaffected — it only layers plugin dists on top of the finished production stage. - `.github/workflows/docker.yml`: both jobs stamp the ISO week into `CLI_TOOLS_CACHE_EPOCH`, and both switch `cache-from/cache-to` from `type=gha` to `type=registry` with per-job refs. - Includes the one-line amd64-only cloud-variant commit from #10570 so the two PRs can't conflict; if #10570 merges first, this PR rebases down to a single commit automatically. ## Verification - Image content is unchanged by layer reordering: the moved `RUN` references nothing from `/app`, and Docker layer ordering only affects caching, not the final filesystem (tool installs and app copy touch disjoint paths). - The cache ref is written only by this workflow — `docker.yml` runs on master/tag pushes, never on PRs — so the workflow's existing "no shared caches into build inputs" supply-chain stance is unchanged (BuildKit layer cache was already accepted via type=gha; the registry backend has the same writer trust). - Runtime proof lands with the first two master builds after merge: the first warms the cache, the second should show the tool layer and deps/build stages as CACHED in the build log, with wall clock dropping accordingly. I'll be watching those as part of managed-deploy work. ## Risks - Low. Worst case the registry cache misses (cold-build behavior, same as today). The weekly epoch means CLI tools update at most a week late inside images; a release built mid-week ships the tools from that week's first build. Cache refs add two small artifacts to ghcr. ## Model Used Claude Fable 5 (`claude-fable-5`, extended thinking, via Claude Code with tool use and code execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (no test-affecting changes) - [x] I have added or updated tests where applicable (n/a — build config and layer ordering only) - [x] I have updated relevant documentation to reflect my changes (in-file comments document both mechanisms) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-31 14:47:25 -07:00
Devin Foley	ce40343b9c	build: publish the cloud image variant for amd64 only (#10570 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Published container images are how both self-hosted users and managed-deployment hosts run it > - The repository publishes two variants: the self-hosted image and a cloud variant for managed deployments > - The cloud variant was built for amd64+arm64, but its only consumers are managed-deployment hosts, which run amd64 > - The QEMU-emulated arm64 half dominates the build's wall clock, delaying every merge-to-deployable-image cycle > - This pull request drops arm64 from the cloud variant only, keeping the self-hosted image multi-arch > - The benefit is roughly halving the time from merge to a deployable cloud image, with no change for any actual consumer ## Linked Issues or Issue Description No existing public issue — inline description following the feature request template: Subsystem affected CI / release publishing (docker workflow) Problem or motivation The cloud image variant builds for `linux/amd64,linux/arm64`, but the arm64 half runs under QEMU emulation and dominates the job's wall clock — while no consumer of the cloud variant runs arm64 (managed-deployment hosts are amd64). Every deploy iteration pays ~double the necessary build time. Proposed solution Build the cloud variant amd64-only. The self-hosted image keeps `amd64+arm64` so ARM users (Apple Silicon, ARM servers) are unaffected. Alternatives considered Keeping multi-arch but building arm64 on native arm64 runners with a manifest merge — faster than QEMU and worth doing for the self-hosted image if its build time becomes a pain point, but unnecessary complexity for a variant with no arm64 consumers. Roadmap alignment Not on ROADMAP.md; CI/publishing speed improvement only. ## What Changed - `.github/workflows/docker.yml`: the `build-and-push-cloud` job's `platforms` is now `linux/amd64` (with a comment explaining why). The self-hosted `build-and-push` job is untouched. ## Verification - Build-config-only change; the workflow runs on merge to master. The published `-cloud` manifest will be amd64-only, which its consumers already pull. - No test changes: nothing at runtime differs on any platform that actually runs the image. ## Risks - Low. If an arm64 consumer of the cloud variant ever appears (e.g. local `docker run` on Apple Silicon for debugging), it would fall back to emulation on the consumer's machine or need this reverted — a one-line change. The self-hosted image's platform matrix is unchanged. ## Model Used Claude Fable 5 (`claude-fable-5`, extended thinking, via Claude Code with tool use and code execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (no test-affecting changes) - [x] I have added or updated tests where applicable (n/a — CI platform matrix only) - [x] I have updated relevant documentation to reflect my changes (in-workflow comment documents the rationale) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-31 13:41:44 -07:00
Devin Foley	521271ebb7	build: bake the build commit into published images (#10566 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Instances commonly run from published container images, and operators need to observe which build a container actually serves > - `/api/health` now reports the running build commit, and server-info already falls back to `PAPERCLIP_BUILD_COMMIT` when git is unavailable > - But published images carry no `.git` and never received `PAPERCLIP_BUILD_COMMIT`, so containers report `commit: null` — verified live against a current image > - That leaves the new deployment-verification field inert exactly where it matters most: containerized deploys > - This pull request bakes the exact build commit into both image variants at build time, mirroring how `PAPERCLIP_BUILD_VERSION` is already stamped > - The benefit is that containers report their true commit on `/api/health`, so deploy tooling can verify a rollout actually shipped ## Linked Issues or Issue Description Companion to #10563 (which exposed the `commit` field on `/api/health`). Inline description following the bug report template: What happened? A container from a published image responds to `GET /api/health` with `"commit": null`. The image has no `.git` directory and the `PAPERCLIP_BUILD_COMMIT` fallback that `server-info` supports is never provided at build time, so git metadata resolves as unavailable. Expected behavior A container reports the commit it was built from, the same way it already reports its build version via the baked `PAPERCLIP_BUILD_VERSION`. Steps to reproduce Run any published image (e.g. `ghcr.io/paperclipai/paperclip:sha-c4f6264-cloud`) and `curl /api/health` — `commit` is `null` even though the build commit is known at image-build time. Paperclip version or commit `sha-c4f6264-cloud` (first image containing #10563). ## What Changed - `Dockerfile`: new `PAPERCLIP_BUILD_COMMIT` build arg, exported as an ENV in the production stage (the `cloud` stage inherits it), directly parallel to `PAPERCLIP_BUILD_VERSION`. Empty for local `docker build`, which keeps the normal fallbacks. - `.github/workflows/docker.yml`: both build jobs pass `PAPERCLIP_BUILD_COMMIT=${{ github.sha }}`. ## Verification - Reviewed the plumbing end-to-end: `build-commit.ts` reads `PAPERCLIP_BUILD_COMMIT` (validated as a full SHA), `server-info.ts` `readGitInfo` falls back to it when the git CLI fails, producing `available: true, fullSha` — which `/api/health` surfaces as `commit`. - Verified live that a current published image reports `commit: null`; this change repairs that on the next build. Post-merge, the first master image should report its commit — I'll be verifying that as part of managed-deploy validation. - No test changes: the fallback path is already covered by existing server-info tests; this PR only supplies the env at image build. ## Risks - Low. Two build-time stamps; no runtime code changes. A wrong SHA would only mislabel the build (same failure mode `PAPERCLIP_BUILD_VERSION` already carries), and `${{ github.sha }}` is the exact commit the workflow builds. ## Model Used Claude Fable 5 (`claude-fable-5`, extended thinking, via Claude Code with tool use and code execution); diagnosis included live probes of a running container's `/api/health`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (no test-affecting changes; server suites unaffected) - [x] I have added or updated tests where applicable (n/a — build-time stamps only) - [x] I have updated relevant documentation to reflect my changes (Dockerfile comments document the arg) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-31 13:01:52 -07:00
Devin Foley	51bb41c7e3	Expose the running build commit on the unauthenticated health response (#10563 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Instances run self-hosted or under hosting/deploy tooling, and operators need to observe what build a server is actually running > - `/api/health` carries the git SHA only inside `serverInfo`, which is gated to board/agent actors — anonymous callers get a redacted body with no version signal at all > - Deploy tooling that manages instances from outside (fleet rollouts, hosting providers, upgrade scripts) therefore cannot ground-truth that a deploy actually shipped without holding credentials > - A build commit is a plain git SHA of this public repository — it is not a secret, and gating it buys no security while blocking legitimate verification > - This pull request surfaces the running build commit as a top-level `commit` field on every `/api/health` response, including the redacted anonymous one > - The benefit is credential-free deploy verification: any operator or tool can confirm which commit an instance serves, while the fuller `serverInfo` block stays access-controlled as before ## Linked Issues or Issue Description No existing public issue — inline description following the feature request template: Subsystem affected Server (API, runs, routes) Problem or motivation An anonymous `GET /api/health` returns a redacted body with no version information; the running git SHA exists only in `serverInfo.git.fullSha`, which requires a board/agent actor. External deploy tooling (fleet rollouts, hosting providers, upgrade scripts) therefore cannot verify that an instance is actually serving the build it was just upgraded to — a rollout that silently keeps running the old image is indistinguishable from a successful one at the health endpoint. Proposed solution Surface the running build commit as a top-level nullable `commit` field on every `/api/health` response shape, including the redacted anonymous one, while keeping the fuller `serverInfo` block access-controlled as before. A build commit is a plain git SHA of this public repository — exposing it costs nothing and enables credential-free deploy verification, like the `version` endpoints on most server software. Alternatives considered Authenticating deploy tooling as a board actor to read `serverInfo` — rejected: it forces credential plumbing into infrastructure that only needs a public SHA, and adds a whole class of auth-misconfiguration failure to deploy verification. Roadmap alignment Not on ROADMAP.md; a small operational observability improvement, no overlap with planned core work. ## What Changed - `server/src/routes/health.ts`: derive `commit` from the server info snapshot (`serverInfo.git.fullSha` when git metadata is available, else `null`) and include it as a top-level field on every `/api/health` response shape — the redacted anonymous body, the full-details body, the no-db body, and the 503 database-unreachable body. - `serverInfo` itself remains gated to full-details responses exactly as before; only the bare commit is newly public. - `server/src/__tests__/health.test.ts`: updated exact-shape assertions to include `commit`, and added an assertion that `commit` is `null` (not omitted) when git metadata is unavailable. The redacted-response tests now pin that anonymous callers receive the commit. ## Verification - `pnpm vitest run src/__tests__/health.test.ts` in `server/` — 13 tests pass, including the redacted-anonymous shapes (which now pin the `commit` field) and the git-unavailable `null` case. - `tsc -p server/tsconfig.json --noEmit` — clean. - Manual: `curl -s https://<instance>/api/health` as an anonymous caller returns `"commit": "<full sha>"` alongside the existing redacted fields. ## Risks - Version disclosure: anonymous callers can now fingerprint the exact running commit. This is a deliberate trade-off: the builds are of a public repository (the SHA reveals no private code), the endpoint already responds to anonymous callers, and the operational value — verifying deploys actually shipped — outweighs the marginal fingerprinting surface. Operators who consider this sensitive are typically fronting `/api` with their own access controls already. - Otherwise low risk: no behavioral change to any gated field, no schema or API-surface removal; `commit: null` keeps the field shape stable when git metadata is absent (e.g. non-git installs). ## Model Used Claude Opus 4.8 (`claude-opus-4-8`, extended thinking, via Claude Code with tool use and code execution) authored the change and tests; finalized and PR'd under Claude Fable 5 (`claude-fable-5`). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (none needed beyond code comments — health endpoint has no standalone doc) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-31 11:46:34 -07:00
Devin Foley	dd1a7f5290	Ensure app-home ownership before the privilege drop, not only on remap (#10530 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Docker image persists all instance state (project checkouts, worktrees, run logs, uploads) under `PAPERCLIP_HOME`, and deployments mount a volume there for durability > - The entrypoint starts as root and drops privileges to the `node` user, but it fixes `PAPERCLIP_HOME` ownership only when it remaps the user's UID/GID > - A freshly mounted volume arrives root-owned and shadows the image's build-time `chown`, so a default-UID boot drops privileges onto an unwritable home and the server crashes on its first `mkdir` > - This pull request makes the entrypoint probe the home's ownership and chown whenever it does not match the runtime user, before the privilege drop > - The benefit is that the image works out of the box on any platform-managed volume, with the common already-correct boot staying chown-free ## Linked Issues or Issue Description No public issue exists — describing the bug inline (per the bug report template). What happened? Running the image with a freshly created volume mounted at `/paperclip` (a Docker named volume, a Kubernetes PV, or any platform-managed volume) and the default `USER_UID`/`USER_GID` crashes on boot: `Error: EACCES: permission denied, mkdir '/paperclip/instances/default/logs'`. Expected behavior The container boots and initializes its instance tree on the mounted volume, exactly as it does when `/paperclip` is the image's own (build-time chowned) directory. Steps to reproduce 1. `docker volume create paperclip-data` 2. `docker run -v paperclip-data:/paperclip ghcr.io/paperclipai/paperclip:<any current tag>` 3. Observe the EACCES crash on the first `mkdir` under `/paperclip`. Root cause `scripts/docker-entrypoint.sh` chowns `/paperclip` only inside its UID/GID remap branch (`changed=1`). A fresh volume mount is root-owned and shadows the image's build-time `chown node:node /paperclip`; with the default 1000:1000 no remap happens, so no chown happens, and `gosu node` drops onto an unwritable home. Paperclip version or commit: reproduces on `master` and any published image. Deployment mode: any; observed on managed-cloud volume mounts and reproducible with plain Docker named volumes. Installation method: Docker image (`ghcr.io/paperclipai/paperclip`). Related PRs (dedup search): no open or merged PR touches the entrypoint ownership logic; the entrypoint's privilege-handling tests were added previously and this extends them. No duplicate found. ## What Changed - `scripts/docker-entrypoint.sh`: the remap-conditional `chown` is replaced by an ownership probe — after any UID/GID remap, the entrypoint stats `PAPERCLIP_HOME` (default `/paperclip`) and runs `chown -R node:node` only when the owner does not match the runtime user, before `exec gosu node`. Covers fresh root-owned mounts and trees written under a previous UID mapping; the already-correct boot performs no chown. The unprivileged (non-root start) branch is unchanged. - `server/src/__tests__/docker-entrypoint.test.ts`: `stat` stub added to the harness; new cases for the fresh root-owned mount with default UID/GID and for `PAPERCLIP_HOME`-relative probing; the remap case now models the post-remap ownership mismatch. ## Verification - `pnpm vitest run server/src/__tests__/docker-entrypoint.test.ts` — 7 passed (5 existing behaviors unchanged, 2 new). - Live on a managed deployment: a container that crash-looped with the EACCES above boots cleanly once the home is chowned before the drop (the same effect this entrypoint change produces; forced there by a UID remap as an interim workaround). ## Risks - Low. Behavior changes only for boots where `PAPERCLIP_HOME` exists with mismatched ownership — exactly the boots that crash today. `chown -R` on a large previously-mismatched tree adds one-time boot latency; correctly-owned homes skip it entirely. Kubernetes restricted / OpenShift non-root starts keep the existing exec-directly path untouched. ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic; Claude Code CLI with extended thinking and tool use; tests executed locally via Vitest). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above (no duplicates; extends the existing entrypoint privilege tests) - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-30 21:34:45 -07:00
Devin Foley	075951f6bd	Fix import completion UX: inbox flood, false-failure message, stale company list (#10538 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Company Import/Export (#10507, hardened in #10523 and #10531) now imports a large company end to end via an async job > - A real 1,418-issue import succeeded, but three rough edges showed up in that success > - Imported issues flooded the inbox, a completed import surfaced a false "failed" message after its in-memory result expired, and the new company didn't appear in the switcher until a manual refresh > - This pull request keeps imported issues out of the inbox, treats an expired-but-completed import as success, and refreshes the company list on completion > - The benefit is that a successful import looks and feels successful, and doesn't bury the user's inbox in historical tasks ## Linked Issues or Issue Description - Refs #10507 / #10523 / #10531 (Import/Export and its hardening). No open issue; three post-import bugs described above. ## What Changed - Imported issues no longer flood the inbox. The inbox "mine" tab is a query: an issue is "touched" if the user authored a comment on it, and import re-attributes bundled user comments to the importing user — so every imported issue appeared. Import now seeds a per-user `issue_inbox_archives` row for each imported issue (via a batched `issues.archiveImportedInbox`), the exact table the inbox visibility query excludes. Gated on an actor user id, so agent/system imports and normal issue creation are untouched; genuine new activity still resurfaces the issue. - A completed import no longer shows a false failure. The in-memory job's terminal retention was 5 minutes, so a poll after that 404'd and the UI showed "failed." Retention is extended to 60 minutes — the real mitigation for a user who steps away during a long import. `watchImportJob` additionally treats a server-confirmed success whose full result is no longer retained (a `succeeded` status carrying only the compact summary — a cloud tenant job, or a board job whose full in-memory result aged out) as a soft success ("import completed — open the company"), navigating by the summary's company id. A 404 while the job is still being watched is not treated as success: a running job is never dropped by the retention sweep, so its disappearance means a restart mid-import that may not have finished, and it surfaces the honest "may have restarted while the import ran" error. A first-poll 404 (the id never existed) is likewise a real error. - The imported company appears without a refresh. `onSuccess` now invalidates the companies/switcher query unconditionally (covering both the full-result and expired-but-completed paths) and navigates by the job's company id. ## Verification - shared/server/ui typechecks clean; 15 UI tests in the touched spec green, plus the embedded-Postgres import batching and portability-routes suites. - New tests: embedded-Postgres test that imported touched issues are archived for the actor and excluded from the inbox query while a normally-created issue still appears; job resolvable at the old window+1 and only 404s past 60 min; UI soft success on a server-confirmed `succeeded` job without a retained full result (no error, list invalidated, navigates by company id), a running-then-gone job → honest error (restart mid-import), and a first-poll 404 → error. ## Risks - Low and import-scoped: the inbox archive only affects imported issues for the importing user; normal issue creation and non-user (agent/system) imports are unchanged. Retention extension is a constant; the async job store remains in-memory by design. A restart mid-import still 404s and is surfaced honestly as a possible failure (never masked as success); only a server-confirmed success whose full result has expired is reported as a soft success. ## Model Used - Implementation: Claude Fable 5 (`claude-fable-5`, Anthropic). Review hardening (the confirmed-success narrowing): Claude Opus 4.8 (`claude-opus-4-8`, Anthropic). Both via the Claude Code CLI with extended thinking + tool use; root-caused against the live import. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-30 21:33:02 -07:00
Devin Foley	5ec7ce76e5	Upload company import packages as compressed zip uploads (fix large-company imports through Cloud) (#10531 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Company Import (#10507, hardened in #10523) lets a user upload a company package on the Import page > - The page expanded the user's `.zip` into a files map and POSTed it as ONE inline JSON body — ~40MB for a real company because attachment blobs get base64-inflated > - On Paperclip Cloud that body travels browser → harness proxy → tenant, where it truncated in transit → body-parser 400 → the browser saw "Failed to fetch", and nothing imported > - Two compounding causes: the giant inline body itself, and the board async opt-in riding an `x-paperclip-cloud-` header that the Cloud harness strips as anti-spoofing (so async never engaged and the import held one fragile synchronous connection) > - This pull request uploads the raw compressed `.zip` as a multipart request (about a third the size, already compressed) parsed server-side into the same bundle the importer consumes, and moves the async opt-in to a proxy-safe `?async=1` > - The benefit is that a large-company import actually completes through Cloud: a small compressed upload, a real async job that survives dropped connections ## Linked Issues or Issue Description - Refs #10507 / #10523 (Import/Export and its hardening). No open issue; problem described above (large-company browser import through a proxy: inline JSON body truncates → 400 → "Failed to fetch"; async opt-in header stripped by the front door → async never engages). ## What Changed - Multipart zip transport.* The Import page uploads the raw `File` as `multipart/form-data` (field `package`, import options in a JSON `meta` field); the server unzips it into `{ rootPath, files }` and runs the exact existing preview/import logic. The `application/json` inline path is byte-identical for CLI/programmatic callers. Bare `application/zip` (meta via `?meta=`) is also accepted for programmatic use. - Shared node zip reader. `packages/shared/src/portability-zip.ts` (node-only subpath, not re-exported to the browser bundle — same pattern as `portability-hash.ts`); the CLI's `zip.ts` becomes a thin re-export. Identical codec (STORE + DEFLATE via `inflateRawSync`, rejects data descriptors/zip64). - Proxy-safe async signal. `wantsAsyncImport` = `?async=1` (board browsers, survives the harness) OR the existing `x-paperclip-cloud-async-import` header (cloud tenants, set server-side). The UI async client now uses `?async=1`. Backward compatible. - Size + preflight. New `PORTABLE_ZIP_UPLOAD_LIMIT_BYTES = 128MB`; the inline 56MB preflight no longer gates the zip path (it shows the compressed size instead). Async submit/poll/resume, the duplicate-guard fingerprint (now over the resolved bundle), pause-on-import, progress/error panels, and activation all apply to the multipart path. - OpenAPI documents json + multipart + zip bodies and the `async` query param. ## Verification - Full typecheck chain (shared, server, ui, cli) clean. - 152 tests across 8 files: new `portability-zip.test.ts` (STORE/DEFLATE/base64-blob byte-exact round-trip, truncation throws, data-descriptor rejection); `company-portability-routes.test.ts` +7 (multipart import+preview equals the inline bundle; async multipart 202→poll→success; board async via `?async=1` with no cloud header; cloud-tenant async via header; sync fallback with neither; truncated-zip 400, nothing imported); `CompanyImport.test.tsx` asserts the local zip sends the raw File and the inline preflight no longer blocks; `openapi-routes.test.ts` green. - NOT yet measured: the end-to-end browser upload through the live Cloud harness — verified on staging after deploy before closing out. ## Risks - Import semantics unchanged — only transport changed; the JSON inline path is byte-identical, the cloud-tenant header async path untouched. Multipart parsing is server-side (memory-bound: a ~13MB zip → ~30MB files map, fine on the server). - The bare `application/zip` path is programmatic-only and covered by content-type dispatch but not a dedicated route test (the multipart path is). ## Model Used - Claude Fable 5 (`claude-fable-5`, Anthropic), Claude Code CLI, extended thinking + tool use; root-caused against live logs/DB and the harness proxy source. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-30 18:48:22 -07:00
Devin Foley	276ae3a75d	Harden company import: durable UI, async jobs, integrity guard, batched inserts (#10523 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Company Import/Export (#10507) moves whole companies between instances as portability bundles > - Real-world use on a large company (1,418 issues, ~10.6k comments) surfaced a cluster of related failures: the import took hours and the browser connection died while the server kept running, a retry silently produced a second partial import, the progress/error UI gave no durable signal, and a cloud-tenant user couldn't even open the companies afterward > - Root cause of the slowness: importBundle inserted every issue, comment, and document as a separate round-trip to a network Postgres — an N+1-over-network pattern > - This pull request hardens the whole import path: durable progress/error UI, an async server-side job so imports survive dropped connections (with a duplicate-submit guard), a fail-closed guard against incomplete payloads, and batched inserts that cut a large import from hours to minutes > - The benefit is that migrating a real, large company actually completes, is legible while it runs, and can't half-import twice ## Linked Issues or Issue Description - Refs #10507 (the Import/Export feature this hardens). Supersedes #10513 (the progress/error-UI piece, folded in here). No open issue; problem described above (large-company import: slow, connection-fragile, silently duplicable, opaque UI). ## What Changed - Batched inserts (perf): importBundle pre-generates entity ids in JS and inserts in chunked multi-row statements, so children no longer wait on parents' generated ids. A 1,418-issue import drops from ~15,600 insert statements to 82 (190×); benchmark below. Import semantics — collision handling, pause-on-import, label/blocker/monitor/attachment/embedded-asset handling, blob sha verification — are unchanged (full portability suite green). - Async import jobs for board sessions: the existing cloud-tenant async job path opens to board sessions with per-actor job keys; the import page submits, polls, and resumes watching after a reload or dropped connection instead of holding one fragile request. A non-terminal job blocks a duplicate submit (409 returns the running job), preventing the double-import. - Fail-closed completeness guard: an optional `expectedFileCount` on inline imports; the server rejects (422 `import_payload_incomplete`) a body carrying fewer files than declared, so a re-framed/short payload fails loudly instead of half-importing. - Durable progress/error UI (was #10513): persistent progress panels with size-aware copy, persistent error panels with retry guidance, and inline explanation when the preview button is disabled; request-lifecycle guards so stale previews/imports can't publish or detach. ## Verification - `pnpm -r` typechecks (shared, server, ui) clean. - `company-portability.test.ts` (76) + `company-portability-routes.test.ts` (30) green — the import correctness net — plus new `CompanyImport.test.tsx` async/resume/409 coverage and a new batching regression test (a 50-issue import issues <50 issue-insert statements; rows land unchanged). - Batching benchmark (embedded Postgres): at 1,418 issues × 7 comments × 1 doc — 82 insert statements vs ~15,598 one-per-row (190×), ~1s wall-clock; a row-verifying run at that scale imports all 1,418 issues / 9,926 comments / 1,418 documents with unique identifiers and no warnings (no rows dropped by chunking). Over a network DB the round-trip reduction is the hours→minutes lever. - What is NOT directly measured here: wall-clock against a real network Postgres (that happens on a staging deploy); the local timing is network-free. ## Risks - Batching is the load-bearing change: it rewrites the import write path. Mitigated by the unchanged 106-test correctness suite, a new scale/row-integrity test, and per-writer transactions (a failure rolls back its table group; not a single outer transaction across writers — noted, correctness preserved). - Async jobs are in-memory (lost on server restart → pollers 404 and can resubmit); matches the pre-existing cloud-tenant job semantics. - `expectedFileCount` is optional (older callers unaffected); over-count is allowed, only under-count fails closed. ## Model Used - Claude Fable 5 (`claude-fable-5`, Anthropic), Claude Code CLI, extended thinking + tool use; implementation across Fable 5 subagents with live diagnosis against a running instance. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-30 16:52:58 -07:00
Devin Foley	1c52f02d34	Let cloud tenant sessions reach companies they hold memberships in (#10524 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - On Paperclip Cloud, each stack authenticates its users to the tenant app through trusted headers (`resolveCloudTenantActor`), which seed a primary company for the stack > - That actor was pinned to exactly one company — the seeded primary — regardless of any other companies the user actually holds a membership in > - Companies created later (via the import flow, or company creation) write real membership rows for the user, but the pinned actor ignored them, so those companies showed up in listings yet returned "User does not have access to this company" when opened > - This pull request unions the pinned primary with the user's own active membership rows, exactly as a locally authenticated session already does > - The benefit is that a Cloud user can reach every company they belong to — most visibly, a company they just imported ## Linked Issues or Issue Description - Refs #10507 (Import/Export — imported companies were unreachable on Cloud stacks). No open issue; bug described above (companies visible in listing but unreachable; expected: reachable when the user holds an active membership). ## What Changed - Extracted the session path's own active-membership query into `loadActiveUserCompanyMemberships(db, userId)` (single-sourced; the session path now calls it too). - `resolveCloudTenantActor` unions its result with the pinned primary: `companyIds = [primary, ...others]`, memberships likewise, primary first. Strictly per-user; a membership-read failure degrades to primary-only (mirrors the existing fail-closed owner-elevation pattern). No change to owner instance-admin elevation, grant seeding, the stale instance-admin purge, or trusted-header validation. - Grants are seeded at membership creation across all flows (company create, invite/join, import), not per request — so no extra seeding was added here. ## Verification - `@paperclipai/server` typecheck clean. - `cloud-tenant-actor.test.ts` (+ union / other-user-excluded / inactive-excluded / no-rows-identical cases), `auth-session-route.test.ts` (route-level: trusted headers reach a unioned company through `assertCompanyAccess`), plus agent-auth, authz-company-access, cross-company-authz, portability-routes — 83 tests green. ## Risks - Low and tightly scoped: only widens a Cloud actor's reachable companies to those it already holds active memberships in; users without extra memberships, other users' rows, and owner elevation are all unaffected. Read failure fails closed to primary-only. ## Model Used - Claude Fable 5 (`claude-fable-5`, Anthropic), Claude Code CLI, extended thinking + tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-30 16:36:50 -07:00
Devin Foley	187a90b7bc	Add opt-in in-flight run-log mirroring with graceful-shutdown flush (#10512 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The run-log store records each agent run's output and can mirror completed logs to S3-compatible object storage > - The mirror uploads only on finalize, so a server restart mid-run loses the whole in-flight log > - Deployments and crashes are routine on ephemeral hosts, and lost run output makes failed runs impossible to debug > - This pull request adds an opt-in throttled mirror for still-running logs plus a graceful-shutdown flush > - The benefit is that a restart mid-run keeps the log tail up to the last mirror interval, and an orderly restart keeps everything ## Linked Issues or Issue Description No public issue exists — describing the feature inline (per the feature request template). Subsystem affected server/ — REST API & orchestration services Problem or motivation `RUN_LOG_S3_BUCKET` gives finished run logs durability, but the mirror uploads only on finalize. A run that is still writing when the server restarts leaves nothing in object storage. On hosts with ephemeral disks the local file is gone too, so the run's output is lost end to end and failed runs cannot be debugged. Proposed solution Mirror the in-flight log to the same object key on a throttled cadence (`RUN_LOG_S3_INFLIGHT_MIRROR_SECONDS`), and flush dirty tails during graceful shutdown. Keep it opt-in so existing deployments see zero new upload traffic unless they ask for it. Alternatives considered Per-append uploads (rejected: one PUT per output chunk is hostile to S3 endpoints and run latency). Chunked part objects with read-time stitching (rejected: complicates the read path, and S3 multipart minimum part sizes do not fit small tails). Persistent volumes (rejected upstream already: the data dir is deliberately an emptyDir in hardened cloud_tenant deployments). Roadmap alignment Not on ROADMAP.md; extends the existing run-log durability mirror without changing any default behavior. Additional context Ranged reads already serve partial objects like a live tail, so the read path needs no change; finalize overwrites the mirror with the complete file. Related PRs (dedup search): the finalize-only S3 mirror landed previously and this extends it; no duplicate or competing PR found for in-flight run-log mirroring. ## What Changed - `server/src/services/run-log-store.ts`: new opt-in `inflightMirrorMs` on the S3 options (`RUN_LOG_S3_INFLIGHT_MIRROR_SECONDS` env). When set, appends schedule at most one upload of the current file per interval, to the same key finalize uses. Ranged reads already serve that key, so a partial object behaves like a live tail and needs no read-path change. Finalize retires the in-flight bookkeeping and waits out an upload already on the wire, so a stale partial can never overwrite a finalized log. Upload failures warn, re-mark the tail dirty, and retry at most once per interval. - `server/src/services/run-log-store.ts`: new `flushInflightMirrors()` on the store and a module-level `flushInFlightRunLogMirrors()` for the shutdown path. Both are no-ops when the mirror is off. - `server/src/index.ts`: graceful shutdown flushes dirty in-flight tails after the heartbeat run drain, so runs the drain did not finalize (timeouts, the hot-restart skip path) still persist their output. - `server/src/services/run-log-store.test.ts`: five new tests — off-by-default (no uploads before finalize), tail preserved after a wipe without finalize, throttle coalescing with a single flush upload, finalize superseding the in-flight mirror and retiring its timer, and upload failures never breaking appends with recovery on the next flush. ## Verification - `pnpm vitest run server/src/services/run-log-store.test.ts` — 13 passed (8 existing + 5 new). - `pnpm vitest run server/src/__tests__/heartbeat-run-log.test.ts server/src/__tests__/heartbeat-active-run-output-watchdog.test.ts` — 21 passed (consumers of the store, unchanged behavior). - `pnpm -C server run typecheck` — clean. - Self-hosted behavior is unchanged unless `RUN_LOG_S3_INFLIGHT_MIRROR_SECONDS` is set: with the variable unset there are zero new uploads and the finalize-only mirroring is byte-identical (asserted by the off-by-default test). ## Risks - Low. The feature is opt-in; unset env preserves today's behavior exactly. When enabled, worst case is one extra PUT per interval per active run, and every upload is best-effort — a failing endpoint warns and never breaks appends, finalization, or shutdown. The finalize path awaits any in-flight upload before writing the complete file, closing the only overwrite race the design introduces. Timers are `unref`ed so the mirror never keeps the process alive. ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic; Claude Code CLI with extended thinking and tool use; tests executed locally via Vitest). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above (no duplicates found for in-flight run-log mirroring; the finalize-only mirror landed previously and this extends it) - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-30 14:01:02 -07:00
Devin Foley	916c13501f	Replace host-to-host Cloud Sync with full-fidelity company Import/Export (#10507 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A company accumulates real state — issues, labels, blockers, documents, work products, monitors, attachments, agents, routines — and people need to move that state between instances: self-hosted to cloud, cloud back to self-hosted, or plain backups > - The experimental, flag-gated Cloud Sync transport (#6548) tried to solve this host-to-host: the source pushed into a receiver over HTTPS with a cross-instance consent/token handshake, which required the destination to be publicly reachable and broke for common self-hosted topologies (plain-HTTP LAN/VPN origins); the receiver half never landed upstream at all > - Meanwhile the portability bundle and the existing export/import pages already move companies offline with none of those networking constraints — but silently dropped labels, blockers, issue documents, work products, monitors, and every attachment > - This pull request removes the host-to-host transport and makes Import/Export the single data-movement path: the pages become first-class company-settings destinations, exports declare exactly what they do not carry, and bundle schemaVersion 6 now carries all of the above, with attachments as content-addressed sha256 blobs verified before a single row is written > - The benefit is a migration and backup flow that works between any two instances with no reachability requirements, no cross-instance auth, and no silent data loss ## Linked Issues or Issue Description - Refs #6548 — the original Cloud Sync sender this PR supersedes and removes. - Related, not duplicates: #1697 (goals in the portability manifest — orthogonal field addition), #954 (an earlier import/export + skill-visibility proposal predating the current portability bundle). - No open issue describes this directly, so in brief (feature-request shape): Problem — moving a company between instances silently lost labels (imports with label references actually hard-failed), blocker relations, issue documents, work products, monitor state, and all attachments, and the alternative Cloud Sync transport required the destination to be publicly reachable over HTTPS plus a consent handshake, which failed for typical self-hosted setups. Desired behavior — one Import/Export flow in company settings that produces a portable bundle carrying all of that data, tells the operator up front what it cannot carry, imports with automations paused, and offers real one-click activation afterwards. ## What Changed - New export fidelity report (`GET /api/companies/:companyId/export/fidelity`) + an "Export fidelity" panel on the Export page listing anything a bundle will not include (now only: approvals, cost history, activity history) - Imports accept `pauseAutomations`; imported agents and routines land paused, the import result reports created routines, and the Import page ends in an activation panel that actually resumes selected agents/activates routines - Export and Import pages promoted into the company-settings nav; the Cloud Upstream wizard, ux-lab page, and API client removed; the old settings route redirects to Export - Host-to-host transport removed: upstream-sync/receiver-client routes and services, CLI `cloud connect`/`cloud push` + keypair store, the shared upstream transfer contract, and the `enableCloudSync` flag; migration `0196` drops the two experimental `cloud_upstream_` sender tables - Bundle schemaVersion 6: labels (definitions + per-task names, remapped by name on import), blocker relations (`blockedBy` slugs, cycle-tolerant), issue documents (`tasks/<slug>/documents/<key>.md`), work products (system refs nulled), monitors (notes/scheduledBy restored, imported un-armed) - Attachments travel as content-addressed `blobs/<sha256>` entries (deduped; comment-scoped attachments re-link via comment index); every blob is hash-verified before any write, so a corrupted bundle cannot leave a partially imported company; both zip codecs now round-trip extensionless/binary entries byte-exactly; the Import page preflights the inline body limit and offers continue-without-attachments - v5 (and older) bundles still import, with an informational warning; bundles newer than v6 are rejected cleanly - Docs: board-operator import/export guide, CLI README, README/ROADMAP updated ## Verification - `pnpm -r` typechecks (shared, db incl. migration numbering/safety checks, server, ui, cli) and `pnpm check:token-gates` — clean - Vitest: full server + shared sweep 4,888 passed / 1 skipped, with the only 3 failures being pre-existing on `master` (2× heartbeat-workspace-branch-containment, 1× workspace-runtime auto-port; reproduced identically with this change stashed); ui + cli suites green; the embedded-Postgres export-fidelity suite applies the full migration chain including the new `0196` against a fresh database - Live end-to-end on a scratch instance: seeded a company with labels, a blocker pair, an issue document, a work product, a monitor, an agent, a routine, and two binary attachments (one comment-scoped) → export → import into a fresh company → labels remapped to new ids, blocker edge and document restored, monitor un-armed with notes intact, attachments byte-identical (sha256-compared through the API), agents/routines paused → activation panel resumed them; a v5-shaped bundle imported with only the info warning; flipping one byte in a blob made the import 422 with zero* rows created - Reviewer repro: create a company with a labeled issue + attachment → Settings → Export → download → Settings → Import on another company/instance → watch the preview, apply with "start paused", then activate ## Risks - Migration `0196` drops `cloud_upstream_connections`/`cloud_upstream_runs` — experimental tables behind a default-off flag; their connection/run history is intentionally discarded - Breaking removals are all of experimental, flag-gated surface: `/api/upstream-sync/` + `/api/cloud-upstreams/` routes, `paperclipai cloud connect\|push`, and the `enableCloudSync` flag (stale keys in stored instance settings parse harmlessly) - Import remains non-atomic on mid-apply errors generally (pre-existing behavior); the new blob verification specifically moved ahead of all writes so tampered bundles cannot create partial state - GitHub-sourced imports do not fetch `blobs/*` and skip attachments with a warning ## Model Used - Claude Fable 5 (`claude-fable-5`, Anthropic), via Claude Code CLI with extended thinking, tool use, and subagent orchestration; implementation and review split across Fable 5 subagents, with live end-to-end verification against a running instance ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-30 11:37:00 -07:00
Devin FoleyandPaperclip	5c5366d0c1	fix(server): honor explicit plugin RPC timeouts (#10460 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Plugin workers connect Paperclip to external runtimes and sandbox providers. > - Some adapter heartbeats run a full sandbox session inside one `environmentExecute` RPC. > - The worker manager limited every RPC timeout to 15 minutes, even when the caller gave a longer timeout. > - This pull request keeps the normal default timeout behavior but honors explicit caller timeouts. > - The benefit is that long sandboxed agent sessions can continue past 15 minutes while other safety guards still bound hung work. ## Linked Issues or Issue Description No public GitHub issue exists for this bug. ### Bug Report Pre-submission checklist: - Searched existing open and closed issues and did not find a duplicate. - Confirmed the bug is reproducible on `master` from the current source tree. - Confirmed the error starts in Paperclip timeout handling, not in an adapter provider or local configuration. What happened? - A sandbox-backed adapter heartbeat can run a full agent session inside one `environmentExecute` plugin RPC. - The plugin worker manager capped every RPC timeout at 15 minutes. - The cap also applied when the caller passed a longer explicit timeout for an execute-style call. - A long sandbox command could fail before the adapter budget expired. Expected behavior: - Ordinary plugin RPC calls should keep the normal 30-second default timeout. - The default timeout path should still have a 15-minute maximum. - A caller-supplied positive finite timeout should be honored, including values above 15 minutes. Steps to reproduce: 1. Use a plugin environment driver that calls `environmentExecute` with an explicit timeout above 15 minutes. 2. Run a command that stays active longer than 15 minutes and remains inside the adapter budget. 3. Observe that the worker manager times out the RPC at 15 minutes before this fix. 4. Run the same path after this fix and observe that the explicit timeout is used. Paperclip version or commit: - Reproduced from the current `master` line before this change. Deployment mode: - Local dev or self-hosted server with sandbox-backed execution. Installation method: - Built from source. Agent adapter(s) involved: - Codex. - Custom or external plugin adapter. - Core plugin worker timeout handling. Database mode: - Not database-related. Access context: - Agent execution context. Node.js version: - Not version-specific. Operating system: - Not OS-specific. Relevant logs or output: ```shell RPC call "environmentExecute" timed out after 900000ms ``` Relevant config: - Not config-related. Additional context: - Execute-style sandbox calls already have adapter inactivity monitors, platform silent-run checks, and provider command timeouts. This PR removes the unintended worker-manager clamp only for explicit positive finite caller timeouts. Privacy checklist: - Reviewed all pasted output for PII, user paths, API keys, tokens, company names, and internal instance links. Duplicate search: - Searched open PRs and open issues in `paperclipai/paperclip` for `environmentExecute timeout`, `MAX_RPC_TIMEOUT_MS`, and `plugin-worker-manager timeout`. - Searched the same terms in `HenkDz/paperclip`. - Found no matching open PRs or issues. - Compared this patch-id against my open PRs in `paperclipai/paperclip`; no match was found. ## What Changed - Added `resolveRpcCallTimeoutMs()` to keep explicit positive finite timeouts intact. - Kept the 15-minute maximum only for the default timeout path. - Updated `callInternal()` to use the new resolver. - Added unit tests for explicit long timeouts, default timeout clamping, fractional values, and invalid explicit values. - Clarified why notification invocation scopes still use the 15-minute TTL. ## Verification - `corepack pnpm install --frozen-lockfile` - `corepack pnpm --filter @paperclipai/plugin-sdk ensure-build-deps` - `corepack pnpm --filter @paperclipai/server exec vitest run src/__tests__/plugin-worker-manager.test.ts` - `corepack pnpm --filter @paperclipai/server exec tsc --noEmit` - `git diff --check `9574cad3e8` HEAD` - GitHub PR checks on `bf7bfd0d`: all passing; Storybook visual regression skipped by workflow. - Greptile on `bf7bfd0d`: 5/5, no blocking failure remains; prior P2 thread resolved. ## Risks Low risk. The change affects RPC timeout resolution in the plugin worker manager. Ordinary plugin calls still use the 30-second default and the default path is still capped at 15 minutes. Callers that pass explicit long timeouts now own that budget. Adapter inactivity monitors and platform silent-run safety checks still bound hung runs. ## Model Used OpenAI Codex, GPT-5-based coding agent, with shell and GitHub CLI tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-29 14:27:23 -07:00
Devin FoleyandPaperclip	7a5a217d60	fix(server): gate sandbox/ssh execution targets by shared remote-managed adapter capability (#10459 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Paperclip can run an agent in a remote-managed environment, such as a sandbox provider or an SSH host. > - The server resolves an execution target for each run. The resolver kept its own hardcoded list of allowed adapters. > - The shared capability metadata in `packages/shared/src/environment-support.ts` already defines which adapters support remote-managed environments. The environment selector and the capabilities API use it. > - The two lists drifted. The UI offered sandbox environments to Grok Build (`grok_local`) agents, but the resolver refused them at run time. > - This pull request makes the resolver use the shared capability check for both the sandbox gate and the SSH gate. > - The benefit is one source of truth. The UI and the runtime now agree on which adapters can use remote-managed environments. ## Linked Issues or Issue Description No public GitHub issue exists for this bug. Inline description per the bug report template: What happened? A Grok Build (`grok_local`) agent was assigned a sandbox environment (a Daytona provider). The UI allowed the assignment. Every run and primary-model test then failed with the warning: `Adapter "grok_local" is not allowed in "<environment>" environments.` Expected behavior An adapter that the environment selector offers for a sandbox environment must also pass the runtime gate. The Grok Build run must start in the sandbox. Steps to reproduce 1. Create a sandbox environment (for example, with a Daytona provider plugin). 2. Create an agent that uses the `grok_local` adapter. 3. Set the agent's environment to the sandbox environment. The UI accepts this. 4. Run the agent, or run the primary-model test. The run fails with the adapter-not-allowed warning. Paperclip version or commit Reproduced on `master` at `0edb742f8d`. Deployment mode Local instance with a remote sandbox provider plugin. The same gate also applies to SSH environments. ## What Changed - `resolveEnvironmentExecutionTarget` in `server/src/services/environment-execution-target.ts` now gates the sandbox path with the shared `adapterSupportsRemoteManagedEnvironments()` helper. Before, it used a hardcoded six-adapter list that did not include `grok_local`. - The SSH path in the same file now uses the same shared helper. - New regression tests in `server/src/__tests__/environment-execution-target.test.ts`: sandbox target resolution for every remote-managed adapter (including `grok_local`), SSH target resolution for `grok_local`, and the null path for an adapter without remote-managed support. ## Verification - Run `node_modules/.bin/vitest run server/src/__tests__/environment-execution-target.test.ts`. All 10 tests pass, including the 3 new ones. - Confirm `grok_local` is in the `REMOTE_MANAGED_ADAPTERS` set in `packages/shared/src/environment-support.ts`. The resolver now reads the same set. - On a live local instance with this fix, a `grok_local` agent assigned to a Daytona sandbox environment no longer produces the adapter-not-allowed warning. ## Risks Low risk. The change routes two hardcoded checks through existing shared capability metadata. Behavior changes only where the lists had drifted: `grok_local`, and any future adapter added to the shared set, can now resolve sandbox and SSH execution targets. Adapters outside the shared set still return `null`. ## Model Used Claude Fable 5 (`claude-fable-5`) by Anthropic, with extended thinking and tool use, running in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-29 12:59:51 -07:00
Devin FoleyandPaperclip	78f8c6c3d4	Recover managed bundled plugin workers on demand (#10429 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Managed deployments can auto-provision bundled sandbox-provider plugins so cloud or remote execution environments appear in the board UI > - In a multi-service deployment, several server processes can share one database and boot concurrently > - A sibling process can create a bundled plugin row while the web process sees it before it reaches `ready` > - The web process correctly avoids clobbering the existing row, but its startup `loadAll()` can miss the plugin and never start that worker locally > - The environments capabilities route then filters out the sandbox provider because the plugin is ready in the database but not running in the web process > - This pull request adds a narrow managed-bundle recovery path that lazily starts the missing worker when the capabilities route sees a ready managed bundled plugin > - The benefit is that the sandbox provider becomes visible after the install finishes, without requiring a web-process restart ## Linked Issues or Issue Description - No public GitHub issue found for this exact deployment race. - Related broad plugin runtime context: Refs #432. Bug description: - What happened: in a managed multi-service deployment with shared database state and bundled plugin auto-install enabled, the API-serving process can skip a plugin row while it is still `installed`, run startup plugin loading before that row becomes `ready`, and then permanently omit the sandbox provider from environment capabilities. - Expected behavior: once the managed bundled plugin row reaches `ready`, the API-serving process should be able to start the plugin worker and include its sandbox provider without a restart. - Steps to reproduce: boot a web process and a sibling worker process concurrently; have the sibling create the bundled plugin row and transition it to `ready` after the web process has already skipped auto-install and run `loadAll()`. - Deployment mode: managed multi-service deployment with shared database state and `plugins.autoInstall` configured. ## What Changed - Added a managed bundled plugin worker recovery helper that single-flights lazy `loadSingle()` starts and only allows configured managed bundled plugin keys. - Passed the managed recovery hook into the environments capabilities route. - Updated `listReadyPluginEnvironmentDrivers()` to attempt bounded recovery for ready managed bundled plugins whose worker is missing in the current process, and only for plugins that actually declare a `sandbox_provider` environment driver. - Made request-time recovery use `loadSingle(id, { markErrorOnFailure: false })` so a local activation failure in one process never transitions the shared plugin row to `error` (a sibling process may be running the plugin successfully). - When error writes are suppressed and activation fails after the worker was spawned, the loader now tears down the partially-registered local runtime (scheduler registration, event subscriptions, agent tools, worker process) instead of leaving a half-activated worker lingering; the teardown steps are factored out of `unloadSingle()` into a shared helper. - A failed recovery attempt now discards the crashed/stopped handle it left registered in the worker manager (a worker that dies during initialize is killed without a scheduled restart), so later capability requests can retry recovery instead of being blocked by the handle-presence gate until a process restart. Handles in starting/running/backoff states are left to the worker manager's own lifecycle; recovery only ever starts when no handle existed, so no pre-existing worker can be affected. - Added a regression test suite covering the installed-to-ready race, allowlist behavior, the driver-kind gate, existing worker handles, concurrent single-flight recovery, bounded slow recovery attempts, suppressed shared error-state writes, partial-runtime teardown on late activation failure, and retry after a dead handle is discarded. ## Verification - `pnpm vitest run src/__tests__/plugin-environment-driver-ready-recovery.test.ts` (in `server/`) passed: 10 tests. - `pnpm --filter @paperclipai/server typecheck` passed. ## Risks - Low risk for self-hosted single-process deployments because lazy recovery is only wired when managed plugin auto-install config is present; with no managed config the capabilities route takes the exact pre-change code path. - The capabilities route can wait briefly while attempting recovery; the attempt is bounded and defaults to 2 seconds. - Failed recovery keeps the prior behavior of omitting the provider until a later successful worker start, and now also cleans up any partially-started local worker so retries begin from a clean slate. ## Model Used - Initial implementation: OpenAI GPT-5 via Codex local coding agent, with repository tool use and command execution. - Review-feedback follow-ups (driver-kind gate, partial-runtime teardown, expanded regression tests): Claude Fable 5 (claude-fable-5) via Claude Code, with repository tool use and command execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-29 12:50:16 -07:00
Devin FoleyandPaperclip	4c8d92f086	fix(ui): leave agent detail after termination (#10451 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The agent detail page lets operators inspect one agent and run lifecycle actions from that context > - Terminated agents are removed from the normal active-agent surface, so the current detail route may no longer be fetchable after termination > - Before this change, terminating an agent from its detail page invalidated agent queries while leaving the browser on the now-stale detail route > - That refetch could surface an "Agent not found" error even though the terminate action itself succeeded > - The shared action button already owns the terminate mutation, so it can notify detail-page callers when termination succeeds > - This pull request redirects the detail page back to the agents list after a successful terminate action > - The benefit is that operators land on a valid route and the Back button does not return them to the stale terminated-agent detail route ## Linked Issues or Issue Description No direct public GitHub issue or PR was found for this detail-page termination flow. ### What happened? After terminating an agent from its detail page, the UI could remain on that agent's detail route and show an "Agent not found" error after query invalidation/refetch. ### Expected behavior Once termination succeeds, the operator should leave the now-stale detail page and land on a valid agents view. ### Steps to reproduce 1. Open a non-built-in agent detail page. 2. Use the overflow actions menu to terminate the agent. 3. Observe the post-termination route/error state. ### Paperclip version or commit Current `master` before this PR. ### Deployment mode Browser UI behavior, independent of a specific deployment mode. Duplicate search: searched public GitHub issues and PRs for `agent not found terminate`, `terminate agent detail`, and `Agent not found`; no direct duplicate or viable in-flight PR was found. ## What Changed - Added an optional `onTerminateSuccess` callback to `AgentActionButtons`, fired only after the shared terminate mutation succeeds. - Wired `AgentDetail` to replace-navigate to `/agents/all` after successful termination. - Extended `AgentActionButtons` coverage for the terminate success path, including API args, callback payload, and query invalidations. ## Verification - `corepack pnpm exec vitest run ui/src/components/AgentActionButtons.test.tsx` - `corepack pnpm check:token-gates` - `git diff --check origin/master..HEAD` - Local diff scan for obvious tokens, credential filenames, and email addresses found no matches. ## Risks Low risk. The new callback is optional, only fires for successful terminate actions, and preserves existing behavior for other `AgentActionButtons` callers. ## Model Used OpenAI Codex, GPT-5-based coding agent (`gpt-5`), tool use enabled for repository inspection, editing, local verification, and GitHub CLI operations. Context window details were not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-29 12:26:07 -07:00
Devin Foley	dc12197cce	fix: prevent duplicate built-in agents and self-heal reconciliation (#10223 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Every company is auto-provisioned a set of built-in agents (e.g. the Summarizer), and a startup reconciler keeps that set correct across every company on boot. > - Provisioning marks these agents with `metadata.paperclipBuiltInAgent.key`, but nothing in the database enforced one active agent per `(company, key)` — `provision()`/`ensure()` did a check-then-insert with no guard. > - Two concurrent server processes (e.g. a `tsx watch` double-boot) could both read "no summarizer exists" and both insert, leaving a company with duplicate built-in agents plus paired orphan pending `hire_agent` approvals. > - That data blemish then became a recurring outage: `findSingleAgent` throws on >1 marked row, and because the throw escaped `reconcileBuiltInAgentsOnStartup`'s sequential loop, every company after the affected one was silently skipped on each boot — no auto-provisioning, no default grants — until manual DB surgery. > - This pull request closes the race at the database level and makes reconciliation self-healing and fault-isolated. > - The benefit is that concurrent provisioning can no longer create duplicates, and even pre-existing duplicates are resolved automatically instead of bricking startup reconciliation for unrelated companies. ## Linked Issues or Issue Description - [x] I searched the GitHub PR list (open and recently closed) for similar PRs and confirmed this is not a duplicate. No public GitHub issue exists; describing the bug in-PR (bug-report shape): What happened A dev instance booted with two concurrent server processes. Both ran built-in agent provisioning for the same company at the same time, and the check-then-insert in `provision()`/`ensure()` (`server/src/services/built-in-agents.ts`) let both writers see "no summarizer exists" and each create one — the company ended up with two identical Summarizer agents (identical `paperclipBuiltInAgent` markers) plus two paired pending `hire_agent` approvals. From then on, every server boot logged: ``` ERROR: startup reconciliation of built-in agents failed Multiple built-in agents found for summarizer (built_in_agent_duplicate_instance) ``` because `findSingleAgent` throws on >1 marked row rather than resolving the duplicate. Worse, `reconcileBuiltInAgentsOnStartup` loops companies sequentially and the throw escaped the loop, so every company after the affected one was silently skipped on every boot. Expected behavior 1. Concurrent provisioning must not create duplicate built-in agents (there was no DB uniqueness constraint on the marker key per company). 2. Reconciliation should be resilient: if duplicates exist anyway, self-heal (keep the oldest row, terminate the newer dupe, cancel its orphan pending `hire_agent` approval), and never let one bad company abort reconciliation for the rest. Steps to reproduce - Race two `provision(companyId, "summarizer")` calls for a company with board approval for new agents enabled (or simulate a double-boot); both insert. - Restart the server → startup reconciliation error fires, companies later in the loop are never reconciled. ## What Changed Part 1 — stop creating duplicates - Migration `0192_built_in_agent_unique_marker` adds a partial unique index on `(company_id, metadata->'paperclipBuiltInAgent'->>'key')` where the marker exists and `status != 'terminated'`. It first resolves any pre-existing duplicates (keep oldest by `created_at`, terminate newer dupes, cancel their orphan pending `hire_agent` approvals, revoke their API keys) so the index can be created on already-affected instances. - `provision()`/`ensure()` now catch the losing race's `23505` unique violation (walking the driver's wrapped cause chain) and re-resolve to the winning row instead of surfacing the error. Part 2 — resilient reconciliation - `findSingleAgent` self-heals: keeps the oldest marked row, terminates the newer duplicates, and cancels each one's orphan pending `hire_agent` approval (idempotent) instead of throwing. - `reconcileBuiltInAgentsOnStartup` isolates per-company failures in both loops so one bad company can't abort reconciliation for the rest; it surfaces a `companyFailures` count in the startup log. - Adds `approvalService.cancel()` for system-initiated cancellation of an orphan approval. ## Verification - `pnpm --filter @paperclipai/db run check:migrations` → numbering + safety checks pass. - `packages/db` migration test (real embedded Postgres) — seeds pre-index duplicate state, runs the migration, asserts dupes resolved + index enforced: 1 passed. - `server` `built-in-agents.test.ts` — self-heal, concurrent races (plain and board-gated), and startup self-heal-without-aborting-later-companies: 34 passed. ``` pnpm --filter @paperclipai/db exec vitest run src/built-in-agent-unique-marker-migration.test.ts pnpm --filter @paperclipai/server exec vitest run src/__tests__/built-in-agents.test.ts ``` ## Risks - Migration safety: the migration mutates data (terminates duplicate rows, cancels their orphan pending approvals, revokes their API keys) before creating the index. It keeps the oldest row per `(company, key)` and only touches non-terminated marked rows; the destructive step is covered by the migration test and the safety-check baseline. On a clean instance it is a no-op cleanup followed by `CREATE UNIQUE INDEX IF NOT EXISTS`. - Otherwise low risk: the unique index is partial (excludes terminated rows, so re-provisioning after a termination stays possible), and the conflict handling degrades gracefully to re-resolving the existing winner. ## Model Used Claude Opus 4.8 (`claude-opus-4-8`), 1M context window, extended thinking, with tool use.	2026-07-28 11:12:58 -07:00
Devin Foley	9f5af4ea5d	fix(server): accept secret_ref binding objects in sandbox provider environment config (#10355 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents execute in environments; sandbox provider plugins (Daytona, Modal, e2b, …) declare their config via a JSON schema, with credentials marked `format: "secret-ref"` > - The environments UI renders those fields with a secret picker that submits `{ type: "secret_ref", secretId, version }` binding objects, while the server-side environment config paths only understood raw string values and bare secret-id strings > - The binding object reached the plugin worker's `environmentValidateConfig` untouched; plugins parse non-string config values as absent, so saving or testing an environment with a picker-bound secret always failed validation (e.g. "Daytona sandbox environments require an API key in config or DAYTONA_API_KEY.", "Modal sandbox environments require tokenId and tokenSecret.") > - Worse, an environment first saved with raw pasted values becomes uneditable: the stored value is a secret reference, the edit form re-submits it as a binding object, and every subsequent save fails the same way > - This pull request canonicalizes binding objects to the bare secret id before plugin validation, and teaches the persistence/runtime/probe secret-ref resolvers to accept the object shape defensively > - The benefit is that picker-bound secrets work for every schema-driven sandbox provider — create, edit, and Test — with no plugin changes required ## Linked Issues or Issue Description Fixes #10105 The same failure reproduces with the Daytona provider: Settings → Instance settings → Environments → New, driver sandbox, provider daytona, bind Api Key to an existing secret via the picker → Save fails with "Daytona sandbox environments require an API key in config or DAYTONA_API_KEY." ## What Changed - `server/src/services/json-schema-secret-refs.ts`: new `parseSecretRefBindingObject()` that recognizes the `{ type: "secret_ref", secretId, version? }` shape the secret picker submits (version defaults to `"latest"`; malformed objects return null). - `server/src/services/plugin-environment-driver.ts`: `validatePluginSandboxProviderConfig()` now canonicalizes binding objects at the driver schema's `format: "secret-ref"` paths to the bare secret id (the persisted shape) before invoking the plugin worker's `environmentValidateConfig`. Pinned numeric versions are rejected with a clear 422, since sandbox provider references always resolve the latest version — silently resolving a different version would be worse. - `server/src/services/environment-config.ts`: the persistence, runtime, and probe secret-ref resolvers plus `collectEnvironmentSecretRefs()` accept the binding-object shape defensively, so any previously persisted object-shaped refs (from providers whose validation tolerated them) resolve instead of being silently skipped; the missing-companyId runtime guard also now fails closed for object-shaped refs. ## Verification - `npx vitest run server/src/__tests__/json-schema-secret-refs.test.ts server/src/__tests__/plugin-sandbox-provider-config-validation.test.ts server/src/__tests__/environment-routes.test.ts server/src/__tests__/environment-config.test.ts` — 82 tests pass, including new coverage: binding-object canonicalization before plugin validation, pinned-version rejection, raw-string pass-through, and a route-level create with a picker-submitted binding object persisting the bare secret id without minting a duplicate secret. - `npx vitest run server/src/__tests__/environment-runtime.test.ts` — 24 tests pass against embedded Postgres, including a new test that persists an object-shaped ref and verifies runtime resolution produces the plaintext credential for the plugin worker. - `pnpm typecheck` in `server/` — clean. ## Risks - Low. The canonical persisted shape (bare secret-id string) is unchanged, so existing saved environments and lease-resume fingerprints are unaffected; raw pasted values and bare-id strings take exactly the same code path as before. - New behavior only triggers where a save/probe previously failed 422 (binding objects at secret-ref paths) or where an object-shaped ref was previously skipped silently at runtime (now resolved, or failed closed without a companyId). - Pinned binding versions at sandbox-provider paths are now an explicit 422 instead of an accidental validation failure; no UI submits pinned versions today (`allowVersionSelector={false}`). ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic) with extended thinking and agentic tool use (Claude Code harness): source diagnosis, fix, and tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (no doc surface changed) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-28 10:50:13 -07:00
Devin FoleyandPaperclip	c274f10abc	feat(server): computed owner instance-admin elevation for cloud-managed instances, behind platform floors (#10343 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Cloud-managed instances authenticate tenant users through a trusted-header path (`resolveCloudTenantActor`) that deliberately never grants `instance_admin`, so every tenant user is company-scoped > - On a dedicated (single-owner) managed instance that leaves the paying owner unable to administer their own instance: instance settings, the environments admin surface, and the custom sandbox image flow are all instance-admin gated (the environments UI can't even show the provider/image of the platform sandbox because the restricted read view blanks `config` entirely) > - Re-granting the old blanket `instance_user_roles` row would repeat the mistake the shared-pool hardening fixed: DB rows go stale, resurrect via restores, and elevate through every auth path > - This pull request elevates only the stack `owner`, computed per request at the trusted-header boundary behind a new managed-tier feature flag, and ships that elevation together with code floors on the platform-owned surfaces an instance admin must not control on a managed instance > - The benefit is that dedicated-stack owners can administer their own instance while platform credentials, execution policy, backups, and runtime code-install stay platform-owned, and self-hosted behavior is unchanged ## Linked Issues or Issue Description No public GitHub issue exists for this change; the underlying issue is described here following the feature-request template. ### Problem or motivation - On cloud-managed instances, tenant users resolved from trusted headers are always company-scoped. For dedicated instances with a single paying owner, the owner cannot reach any instance-admin surface of their own instance (instance settings, environments administration, custom image setup), and the restricted environment read view hides even structural fields like the sandbox provider and image. - The previous hardening intentionally removed blanket elevation (and purges stale `instance_user_roles` rows on every trusted-header authentication). That protection must not regress for shared multi-tenant pools. ### Proposed solution - Owner-only, computed, flag-gated elevation plus code floors on platform-owned surfaces, in one PR so the elevation can never ship without the floors. ### Alternatives considered - Re-inserting an `instance_user_roles` row for owners (the pre-hardening model): rejected — DB rows go stale, survive restores, and elevate through every auth path; #7525 removed exactly this. - Widening only the environments read view without any elevation: rejected — it fixes one screen but still leaves a dedicated-instance owner unable to administer instance settings or custom images. - Elevating additional stack roles (`member`/`admin`/`support`): rejected — only the owner has an ownership claim over the whole instance; other roles stay company-scoped. ### Roadmap alignment - Extends the shipped "Cloud deployments" roadmap work (multi-tenant isolation, company-scoped cloud tenants, managed-instance bootstrap) without overlapping planned core items, and leaves self-hosted behavior unchanged. ## What Changed - New feature key `enableOwnerInstanceAdmin` (`packages/shared`): boolean flag in `instanceExperimentalSettingsSchema`, catalog tier `managed`, `cloudDefault: true`, `selfHostedDefault: false`. Inert on self-hosted instances — the elevation path only exists behind the cloud tenant trust token. - Computed elevation (`server/src/middleware/auth.ts`): `resolveCloudTenantActor` now returns `isInstanceAdmin: true` only when the trusted-header stack role is `owner` and the flag is enabled. The flag is resolved through the instance-settings service so the managed-config overlay applies (the control plane can disable elevation fleet-wide without touching tenant databases; a DB row edit or restore cannot resurrect it). Resolution fails closed on settings read errors. The `instance_user_roles` never-insert and the per-request stale-row purge are byte-identical. `member`/`admin`/`support` stack roles stay company-scoped. - Authorization guard split (`server/src/services/authorization.ts`): the blanket-allow now trusts the actor's computed `isInstanceAdmin` flag (only the attested resolver can set it for `cloud_tenant` actors) while keeping the `instance_user_roles` DB lookup excluded for `cloud_tenant` — a stale or hand-inserted role row still elevates nothing. - Floor F1 — platform environment credentials (`server/src/routes/environments.ts`): on cloud-managed instances, platform-provisioned environment rows (`managedByPaperclip` marker, plus the legacy managed-Kubernetes marker) use a single floored view for every reader on all environment routes (list, get, create, update, delete responses): `envVars` are never echoed and credential-shaped `config` keys (reusing the managed-config `SECRET_LIKE_CONFIG_KEY_PATTERN`) are dropped — for all actors including instance admins — while structural config (provider, image, template, region, …) and the managed markers stay visible. This also fixes the environments UI for managed sandboxes, which previously lost the provider/image entirely in the restricted view. The floor also covers writes: `PATCH /environments/:id` and `DELETE /environments/:id` on a platform-provisioned row are rejected (403, `environment_platform_managed`) for every actor including instance admins, and the guard binds to the persisted row's markers so a patch cannot strip the managed marker to lift the floor. The one recovery path is a metadata-only PATCH that solely clears the marker keys (null/false), for rows stamped through the old unrestricted API before the markers became reserved — and it never applies to a row whose slot markers are live platform state: the single local row (`environments_local_driver_idx`), which `ensureLocalEnvironment` adopts and stamps on cloud-managed instances from every caller (company creation, the heartbeat, run orchestration), and the single marked sandbox row (`environments_managed_sandbox_idx`) while a managed-sandbox bootstrap path is configured (managed-config `environments` section or `PAPERCLIP_EXECUTION_MODE=kubernetes`) and the provisioner therefore adopts and refreshes it on every boot. Clearing a live slot row's markers would let the next write reclassify it as tenant-managed and bypass the floor; conversely, when no sandbox provisioning path is configured the platform holds no claim on any sandbox row, so a platform marker there is stale by definition and the recovery patch applies. Every marker outside a live slot is clearable, so no legacy row is ever locked permanently. Custom-image setup and probes on the platform sandbox stay available to instance admins — those are the owner-facing flows this elevation exists for. The marker keys themselves are reserved: client create/update payloads that set `managedByPaperclip` or `managedKubernetesSandbox` are rejected (422, `environment_platform_marker_reserved`) on cloud-managed instances, so a tenant row can never be stamped platform-provisioned through the API and self-locked behind the write floor (the provisioner writes markers at the service layer, not through these routes). Tenant-created environments are otherwise unaffected. - Floor F2 — executionMode (`server/src/routes/instance-settings.ts`): on cloud-managed instances, `PATCH /instance/settings/general` rejects writes that would change `executionMode` (403, `execution_mode_platform_managed`). Same-value echoes pass so settings forms that submit the full general-settings object keep working. The boot-time execution-policy bootstrap path is untouched (it calls the service directly). - Floor F3 — manual database backups (`server/src/routes/instance-database-backups.ts`): floored off on cloud-managed instances (403, `database_backups_platform_managed`); backups are platform-owned there, and the result would also echo a server-side filesystem path. - Floor F4 — adapter code install (`server/src/routes/adapters.ts`): `POST /adapters/install` and `POST /adapters/:type/reinstall` are floored off on cloud-managed instances (403, `adapter_install_platform_managed`). Adapter packages execute in the server process, so a runtime install would let an instance admin read the platform trust anchors out of the process environment. This mirrors the existing bundled-only plugin install floor; adapter code on managed instances comes bundled with the platform image. ## Instance-admin surface audit Before widening who can hold `isInstanceAdmin`, every instance-admin-gated surface in `server/src` was enumerated and reviewed for whether its response or side effects could echo process environment values or platform credentials (tenant trust token, JWT signing keys, database connection strings, provider API keys): 29 distinct gate definitions covering ~90+ call sites, in four groups — sole instance-admin gates (12), instance-admin-or-company-permission gates (10), response-shaping/scope-widening sites (6), and the central `allow_instance_admin` short-circuit in the authorization service (58 `decide()` call sites). Findings and dispositions: - Environment read/write responses exposed platform sandbox `envVars`/credential-shaped config to instance admins → closed by floor F1. - Manual backup trigger echoed a server filesystem path and triggers a platform-owned operation → closed by floor F3. - Adapter install/reinstall loads externally fetched code into the server process (indirect, complete env exposure) → closed by floor F4. The sibling plugin-install path already had a bundled-only floor on managed instances and needed no change. - Token-minting surfaces (gateway tokens, custom-image terminal/connection tokens) mint credentials scoped to the instance's own resources, not platform trust anchors → acceptable for an owner-admin of a dedicated instance; unchanged. - All remaining gated surfaces return ordinary instance-scoped business data; none echo `process.env` or platform secrets directly. OAuth client secrets are referenced by env-var name only; SSH private keys are stored as secret refs before persistence and are not echoed. Operational note for managed platforms: this model assumes the process environment of a managed instance holds only that instance's own credentials. Platform operators should keep provider credentials per-instance (never fleet-shared) since an instance admin ultimately controls in-process code on their own instance. ## Verification - `pnpm vitest run server/src/middleware/cloud-tenant-actor.test.ts` — resolver matrix: owner × flag on/off, flag via managed overlay (on-over-DB-off and off-over-DB-on), member/admin/support × flag on, no-token self-hosted, fail-closed settings read, purge still runs and no role row is ever inserted (14 tests). - `pnpm vitest run server/src/__tests__/authorization-service.test.ts` — computed flag elevates a `cloud_tenant` actor; a stale `instance_user_roles` row still never does; `session` actors unchanged (full suite, embedded Postgres). - `pnpm vitest run server/src/__tests__/environment-routes.test.ts` — F1: no secret echo to admins on get/list, structural config visible to restricted readers, platform-row PATCH/DELETE rejected for admins (including a marker-stripping patch), marker-clear recovery allowed for stale legacy rows and for a marked sandbox row when no provisioning path is configured, but refused on the managed local row and on the sandbox slot row under a managed-config `environments` entry or the forced kubernetes execution mode, client marker-stamping creates/patches rejected, tenant rows still readable and writable, self-hosted read+write regression (60 tests). - `pnpm vitest run server/src/__tests__/environment-service.test.ts` — `ensureLocalEnvironment` adopts a pre-existing local row on cloud-managed instances (marker stamped, other metadata preserved, idempotent — no rewrite on re-ensure) and leaves self-hosted rows untouched (22 tests, embedded Postgres). - `pnpm vitest run server/src/__tests__/instance-settings-routes.test.ts server/src/__tests__/instance-database-backups-routes.test.ts` — F2 change-vs-echo matrix incl. self-hosted regression; F3 floor for both admin shapes (32 tests). - `pnpm vitest run server/src/__tests__/adapter-routes-authz.test.ts` — F4 floor; self-hosted install/reinstall behavior unchanged (existing cases). - `pnpm vitest run server/src/__tests__/first-admin-claim.test.ts server/src/__tests__/bootstrap-claim-routes.test.ts server/src/__tests__/managed-config.test.ts server/src/__tests__/health.test.ts server/src/__tests__/instance-settings-managed-overlay.test.ts server/src/services/managed-environments.test.ts server/src/services/execution-policy-bootstrap.test.ts` — first-admin bootstrap gate and managed-config behavior unchanged (91 tests). - `pnpm vitest run packages/shared/src/feature-catalog.test.ts` — catalog/schema sync tests cover the new key (selfHostedDefault must equal the schema default). - `pnpm run typecheck` — all 31 workspace projects clean. ## Risks - Self-hosted behavior is unchanged: every floor binds to `isCloudManagedInstance()` (tenant trust token present), the new flag defaults off with no elevation path, and regression tests pin the self-hosted branches. - The elevation is fail-closed and stateless: turning the flag off (managed overlay or DB) de-elevates on the next request; there is no role row to clean up and restores cannot resurrect elevation. - On a cloud-managed instance a pre-existing unmarked local row is adopted (stamped `managedByPaperclip`) by the next ensure and becomes platform-owned — the intended managed-product semantic: the platform owns the single local slot. Self-hosted instances are untouched. - F1 widens restricted readers' view of platform-provisioned rows from fully blanked `config`/`metadata` to structural-only `config` plus markers. Platform-delivered config is guaranteed secret-free by the managed-config contract (secret-shaped keys fail startup), and the floor re-drops secret-shaped keys defensively. - One extra instance-settings read per trusted-header request for owner-role actors (the resolver already performs several queries per request). ## Model Used Claude Fable 5 (Anthropic) — model id `claude-fable-5`, extended thinking enabled, agentic tool use via Claude Code; read-only explore subagents on the same model were used for the surface audit sweep. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-27 18:58:31 -07:00
Devin Foley	273315a4d0	feat(server): provision managed sandbox environments from the managed config (#10324 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Hosted/managed deployments configure instances entirely from the control plane: `PAPERCLIP_MANAGED_CONFIG` already delivers feature flags and `plugins.autoInstall` (bundled sandbox provider plugins), parsed fail-closed at boot > - A plugin alone is not usable for execution: runs need an instance-level `driver: "sandbox"` environment row pointing at the provider, and today only Kubernetes has a boot path for that (`PAPERCLIP_EXECUTION_MODE` → `ensureKubernetesEnvironment`); every other provider requires a manual product-API call the control plane cannot make on a managed instance > - Adding one `ensureXxxEnvironment` per provider would multiply near-identical boot hooks and env-var surfaces > - This pull request generalizes the existing Kubernetes machinery: the managed-config document gains an optional `environments` section that declares a sandbox environment for any bundled provider, ensured idempotently at boot by a provider-agnostic service function (the Kubernetes hook becomes a thin wrapper over it) > - The benefit is that a managed fleet can provision any sandbox provider (Daytona, Modal, E2B, …) purely from configuration — no per-provider code, no manual API calls, no secrets in the document — while self-hosted behavior is untouched ## Linked Issues or Issue Description No public issue exists. Refs #10157 (the cloud image variant that bundles sandbox provider plugins — this PR is the configuration half that makes an installed provider usable). Problem (feature-request shape): on a managed instance the control plane can auto-install a bundled sandbox provider plugin via `PAPERCLIP_MANAGED_CONFIG.plugins.autoInstall`, but cannot create the environment row that makes the provider schedulable. The only boot-time environment provisioning is Kubernetes-specific (`PAPERCLIP_EXECUTION_MODE=kubernetes` + `PAPERCLIP_K8S_`). A generic, config-driven path is needed so any bundled provider can be provisioned without per-plugin code or manual API calls. ## What Changed - `server/src/services/managed-config.ts`: optional `environments` top-level section — `[{ name, description?, provider, config? }]` — validated fail-closed: unknown keys, more than one entry (the DB permits exactly one Paperclip-managed sandbox row, `environments_managed_sandbox_idx`), a `provider` not present in `plugins.autoInstall`, `config.provider`, or secret-looking config keys at any depth (`api_key`/`token`/`secret`/`password`/`credential`) all refuse startup. Absent section ⇒ `environments: []`, so pre-section documents keep booting newer builds. - `server/src/services/environments.ts`: new provider-agnostic `ensureManagedSandboxEnvironment({ name, description?, provider, config?, extraMetadata? })` — idempotently owns the single managed sandbox row: refreshes name/description/config each call, adopts the slot across provider switches (dropping the stale `managedKubernetesSandbox` marker), adopts a same-name unmanaged sandbox row (stamping it managed) instead of colliding on `environments_name_idx` every boot, and falls back to keeping the current name if the desired name belongs to a different row. `ensureKubernetesEnvironment` is now a thin wrapper that pins `provider: "kubernetes"` and stamps the legacy marker. - `server/src/services/managed-environments.ts` (new): `applyManagedEnvironments` boot step — no-op for self-hosted/empty; throws (fail startup) when `PAPERCLIP_EXECUTION_MODE` is also set, since both would own the same managed sandbox row; otherwise ensures each declared environment fail-safe per entry (log + continue boot, matching bundled-plugin provisioning posture). - `server/src/index.ts`: runs the new boot step right after the execution-policy bootstrap, before the heartbeat resumes queued runs. - `server/src/services/index.ts`: exports `applyManagedEnvironments` and `ManagedEnvironmentSpec`. - Secrets stay out of the document by construction: provider credentials reach managed instances only as process env vars (each provider's documented fallback, e.g. `DAYTONA_API_KEY` for the Daytona plugin). ## Verification ```sh cd server pnpm exec tsc --noEmit -p tsconfig.json pnpm exec vitest run \ src/__tests__/managed-config.test.ts \ src/services/managed-environments.test.ts \ src/services/execution-policy-bootstrap.test.ts \ src/__tests__/environment-service.test.ts \ src/__tests__/environment-instance-routes.test.ts \ src/__tests__/environment-routes.test.ts \ src/__tests__/plugin-install-guard.test.ts \ src/__tests__/environment-execution-target.test.ts \ src/__tests__/instance-settings-managed-overlay.test.ts \ src/__tests__/bundled-plugins.test.ts ``` All pass locally (typecheck clean; environment-service suite runs against embedded Postgres and exercises the refactored Kubernetes wrapper plus the new generic ensure: create/refresh, provider switch, unmanaged-row adoption, name-conflict fallback). New tests cover the parser (12 cases incl. secret-key rejection at depth) and the boot step (no-op, mutual exclusion, pass-through, fail-safe). ## Risks - Self-hosted: none intended.* Without `PAPERCLIP_MANAGED_CONFIG` nothing new executes; the `PAPERCLIP_EXECUTION_MODE=kubernetes` path is regression-covered by the existing bootstrap/service/route suites (all green). - Behavioral shift in `ensureKubernetesEnvironment` (deliberate): it now also refreshes `name`/`description` to their managed defaults each boot (desired-state semantics, same as config today) and adopts a `managedByPaperclip` sandbox row that lacks the Kubernetes marker — previously that state made the ensure throw every boot. - New startup failure modes are all explicit misconfigurations (malformed section, provider not auto-installed, secret in config, execution-mode conflict) and fail with precise errors; DB-side ensure failures never block boot (fail-safe per entry, logged). - No migrations; no API surface changes. ## Model Used Claude Fable 5 (Anthropic, model ID `claude-fable-5`) with extended thinking and tool use, driving the change end-to-end inside a Claude Code / agent-harness session (code, tests, and verification runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (the managed-config module header is the contract doc) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-27 12:00:21 -07:00
Devin FoleyandPaperclip	d1b9448b57	fix(server): stamp the real build version into images instead of the package.json placeholder (#10257 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work; it ships as a Docker image that self-hosters and managed deployments run. > - The server resolves its own version at runtime in `server/src/version.ts` (`resolveServerVersion()`), which feeds analytics and the server debug panel. > - That resolver derives the real version from `git describe`, and falls back to `server/package.json`'s `version` when git isn't available. > - But `server/package.json`'s version is a static placeholder — CI only stamps the real CalVer at publish, so in source it is never the real version (currently `0.3.1`). > - A Docker image has no `.git` (it's dockerignored), so `git describe` can't run inside it. Every image therefore falls back to the placeholder and reports `0.3.1` in analytics and the debug panel, regardless of which commit it was built from. > - This PR computes the real version once on the CI build runner (where `.git` and tags exist), bakes it into the image, and has `resolveServerVersion()` prefer that stamp when `git describe` is unavailable. > - The benefit: self-hosted and cloud images report their true version instead of a misleading placeholder, with no change to dev checkouts, `git describe`-based resolution, or local `docker build`. ## Linked Issues or Issue Description No public issue exists — describing the bug inline (per the bug report template). What happened? Docker images built from `master` (and release tags) report the server version as the `0.3.1` placeholder in analytics and the server debug panel, instead of the real version of the commit the image was built from. Expected behavior An image reports the real version of its build commit (e.g. `2026.722.0+51.git.<sha>`), so operators can tell which build is running. Steps to reproduce 1. Build the server Docker image from any `master` commit (the `Docker` workflow, `production` target). 2. Run the image and open the server debug panel (or inspect the version reported to analytics). 3. Observe the version is `0.3.1` rather than the commit's real version. Root cause `resolveServerVersion()` derives the real version from `git describe`, but the image has no `.git` (dockerignored), so it falls back to `server/package.json`'s `version` — a static placeholder CI only replaces with the real CalVer at publish time. Nothing bakes the real version into the image. Paperclip version or commit: reproduces on `master` (`4c55f0d8`) and any published image. Deployment mode: self-hosted and managed (both the `production` and `-cloud` images). Installation method: Docker image (`ghcr.io/paperclipai/paperclip`). Related PRs (dedup search): #9103 (merged — added the `git describe`-based source-install resolution this builds on) and #9637 (closed). Neither bakes a version into the image; this PR closes that gap. No duplicate found. ## What Changed - `.github/workflows/docker.yml` — checkout with full history + tags (`fetch-depth: 0`), and a new `Compute build version` step that runs `git describe --tags --match 'v' --long --dirty` on the pristine runner checkout. The result is passed as a `PAPERCLIP_BUILD_VERSION` build-arg to both the `production` and `-cloud` image builds. - `Dockerfile`* — the `production` stage takes an `ARG PAPERCLIP_BUILD_VERSION` (default empty) and bakes it into the runtime `ENV`; the `cloud` stage inherits it via `FROM production`. - `server/src/build-version.ts` (new) — `readBuildVersion()` / `parseBuildVersion()`, mirroring `build-commit.ts`: reads `PAPERCLIP_BUILD_VERSION` (or a `.paperclip-build-version` file) as a single-token stamp. - `server/src/version.ts` — `resolveServerVersion()` prefers the baked build version when `git describe` is unavailable, parsing it with the same rules as a live checkout (`parseGitDescribeVersion`), and falling through to the existing `build-commit` stamp and package version when unset. A live checkout's `git describe` still wins over any stamp. - Tests for the new behavior and the precedence. ## Verification - `pnpm --filter @paperclipai/plugin-sdk ensure-build-deps && tsc --noEmit` in `server/` — clean. - `vitest run server/src/__tests__/version.test.ts server/src/__tests__/build-version.test.ts` — 23 tests pass, covering: stamped version used when git describe fails, stamp parsed to real CalVer, stamp preferred over the build-commit fallback, on-tag stamp collapses to the release version, a pre-resolved stamp used verbatim, and a live git describe still winning over a stamp. - `git describe --tags --match 'v*' --long` for this commit → `v2026.722.0-51-g<sha>`, which `resolveServerVersion()` reports as `2026.722.0+51.git.<sha>` — no longer `0.3.1`. - Not run locally: the full multi-arch image build (CI-only). The workflow change is verified by inspection; the version is computed on the pristine checkout before any lockfile refresh, so it carries no spurious `-dirty`. ## Risks Low. Additive and image-only: - No runtime behavior changes for dev checkouts (git describe still primary and wins over any stamp) or for local `docker build` (empty arg → server keeps its existing fallbacks). - Not a breaking change; no schema or API surface. The stamp is informational (version reporting only). - `fetch-depth: 0` makes the release-image checkout fetch full history/tags — a modest cost on a workflow that already runs at release cadence with a 60-minute budget. - Rollback: revert the commit; images simply return to reporting the placeholder. ## Model Used Claude Opus 4.8 (`claude-opus-4-8`, 1M-context variant), extended thinking, with tool use / code execution — agentic edits, `tsc` + `vitest` runs, and a `git describe` resolution check. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work (bugfix, not core feature work) - [x] I have searched GitHub for duplicate or related PRs and linked them above (#9103, #9637 — related, not duplicates) - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (`fix/build-version-stamp`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (no user-facing docs affected; behavior is documented inline in `version.ts` / `build-version.ts` and the workflow/Dockerfile) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-25 10:06:28 -07:00
Devin Foley	965a827ee7	feat(docker): publish a cloud image variant with built bundled plugins (#10157 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Managed (cloud-hosted) deployments configure instances through `PAPERCLIP_MANAGED_CONFIG`, including a `plugins.autoInstall` key list that the boot-time installer resolves against the bundled plugin catalog > - The installer requires each bundled plugin's `dist/manifest.js` (`server/src/services/bundled-plugins.ts`), but the published image only ships the sandbox providers' source — they are intentionally excluded from the pnpm workspace, and the Dockerfile never builds them > - Every managed auto-install therefore logs `bundled plugin bundle not present; skipping auto-install` and no sandbox provider can be provisioned through managed config > - Baking built plugins into the single published image would fix it but makes every self-hosted pull carry the providers' `node_modules` for a managed-only mechanism > - This pull request adds a `cloud` Dockerfile target extending `production` with built bundled plugins — parameterized by build arg and currently just `daytona` — published alongside the default image with a `-cloud` tag suffix > - The benefit is working plugin auto-provisioning for managed deployments while the self-hosted image stays byte-identical and the cloud variant only carries what is actually deployed ## Linked Issues or Issue Description Fixes #10158 (filed for this problem; no prior issue existed — searched for duplicate/related PRs and issues around bundled plugins, docker image variants, and auto-install). Summary: What happened: on a managed instance with `plugins.autoInstall: ["daytona"]` delivered via `PAPERCLIP_MANAGED_CONFIG`, boot logs `bundled plugin bundle not present; skipping auto-install` with `pluginPath: /app/packages/plugins/sandbox-providers/daytona`, and the plugin is never installed. Expected: the advertised bundled-catalog keys are installable from the published image. Why: the image ships plugin source without `dist/` — nothing in the Dockerfile builds the workspace-excluded sandbox providers. ## What Changed - `Dockerfile`: new `cloud-plugins` stage (based on `build`, so devDependencies are available for `tsc`) that installs and builds each provider named in the `CLOUD_BUNDLED_PLUGINS` build arg standalone (`pnpm install --ignore-workspace --no-lockfile && pnpm build`, exactly as the providers' READMEs prescribe), asserting `dist/manifest.js` exists per plugin and failing loudly on unknown names; new `cloud` stage = `production` + the built plugin tree. The arg defaults to `daytona` — the only provider managed deployments auto-install today; every entry adds its `node_modules` to the image, so the list grows only with actual need (a one-line workflow change). - `.github/workflows/docker.yml`: the existing build step is pinned to `target: production` (without this, the new trailing stage would silently become the default build target — this pin is what keeps the self-hosted image identical); new metadata + build-push steps publish the `cloud` target (with `CLOUD_BUNDLED_PLUGINS=daytona`) under the same tag set with a `-cloud` suffix (`sha-<short>-cloud`, `latest-cloud`, `<version>-cloud`), same schema labels, reusing the GHA layer cache ## Verification - All seven sandbox providers build standalone from a clean checkout with the exact commands the new stage runs, each producing `dist/manifest.js` — so the current `daytona` default works and future list additions are known-good - The stage's shell loop was dry-run against the checkout (directory existence + per-plugin assertion logic) - Workflow YAML lints clean - Not run: a full multi-arch `docker build` (no local docker daemon). The `cloud` stage is additive and the default target is pinned, so the risk is contained to the new build step; the first master build after merge proves it end-to-end ## Risks - Self-hosted behavior: unchanged. The default image build is pinned to the `production` target, which produces the same layers as before this change; the `cloud` stages run only for the new build step. - The plugin installs in the `cloud-plugins` stage use `--no-lockfile` (the providers are workspace-excluded and lockfile-less by design), so plugin dependency resolution is not pinned at image-build time. This mirrors the existing Plugins-page install path, which resolves from npm at install time. - CI cost: one additional build-push per master push. It reuses the layer cache from the production build, so the marginal work is the single plugin's build layers. - An unknown name in `CLOUD_BUNDLED_PLUGINS`, or a provider that stops producing `dist/manifest.js`, fails the cloud build loudly rather than publishing a broken variant. ## Model Used Claude (Anthropic), model ID `claude-fable-5[1m]` via Claude Code CLI — extended thinking and tool use (code edits, standalone plugin build verification, workflow lint). ## Checklist - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] Self-hosted behavior unchanged (default build target pinned to `production`) - [x] One clear change: publish a cloud image variant with built bundled plugins	2026-07-24 08:22:34 -07:00
Devin FoleyandPaperclip	14f20be92b	ci: harden Docker image build workflow against lockfile drift and runner disk exhaustion (#10142 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Docker image publish workflow (`.github/workflows/docker.yml`) builds and pushes the multi-arch `ghcr.io` image on every master push, so users pulling the container get the latest code > - The two newest master runs of that workflow failed, so no images have been published past a recent master commit > - The failures had two distinct causes: run [30054330748](https://github.com/paperclipai/paperclip/actions/runs/30054330748) hit `ERR_PNPM_LOCKFILE_CONFIG_MISMATCH` (committed `pnpm-lock.yaml` drifted from `patchedDependencies` in package metadata), and run [30050197392](https://github.com/paperclipai/paperclip/actions/runs/30050197392) hit `no space left on device` during the multi-arch buildx export > - This pull request hardens the publish job against both failure modes: it refreshes the lockfile (lockfile-only, guarded) before the build, and frees runner disk space before buildx setup > - The benefit is that image publishing keeps working through routine lockfile drift and the growing multi-arch build footprint, so `ghcr.io` images stay current with master ## Linked Issues or Issue Description - Refs #8286 — same class of Docker-build lockfile mismatch failure - Refs #8827 — pnpm 9.15.x pin / lockfile regeneration discussion - Note: the immediate lockfile drift on master was fixed by #10132; the refresh step here prevents the next drift from breaking image publishing again ## What Changed - Added a pnpm + Node setup and a "Refresh lockfile for Docker build context" step to the image job in `.github/workflows/docker.yml`: runs `pnpm install --lockfile-only --ignore-scripts --no-frozen-lockfile`, exits cleanly if nothing changed, and fails the job if anything other than `pnpm-lock.yaml` was modified by the refresh - Added a "Free runner disk" step (before buildx setup) that prunes the pnpm store, apt caches, preinstalled toolchains (`/usr/share/dotnet`, Android SDK, Swift, Boost, PowerShell, GHC, CodeQL/PyPy/Ruby toolcache), and dangling Docker state, logging `df -h` before/after - No changes outside the workflow file (54 added lines, nothing removed) ## Verification - Pulled the logs of both failed master runs and matched each failure to the step that addresses it: [30054330748](https://github.com/paperclipai/paperclip/actions/runs/30054330748) failed with `ERR_PNPM_LOCKFILE_CONFIG_MISMATCH`, [30050197392](https://github.com/paperclipai/paperclip/actions/runs/30050197392) failed with `no space left on device` during the buildx export - Confirmed pnpm `9.15.4` in the new setup step matches the repo `packageManager` field and the version used in the Dockerfile, so the refreshed lockfile is generated by the same pnpm the image build consumes - Validated the workflow YAML parses cleanly - The workflow triggers on master pushes / manual dispatch; the definitive check is the first master run after merge — reviewers can also `workflow_dispatch` it from this branch if desired ## Risks - The lockfile refresh runs with `--ignore-scripts` and a guard that aborts on any non-lockfile change, so it cannot silently pull unexpected code into the image; worst case it fails the job with a clear diff - The published image could be built from a refreshed lockfile that differs from the committed one when drift exists — that keeps publishing alive but can mask drift on master, which still needs the committed lockfile fixed (as #10132 did) - Disk cleanup removes preinstalled toolchains only on the ephemeral runner for this job; other jobs/workflows are unaffected - Low risk overall: additive steps in a single workflow file ## Model Used - Claude (Anthropic) — `claude-fable-5` (Claude Code agent harness, extended thinking, tool use). Used to diagnose the failing CI runs from logs, author the workflow changes, and prepare this PR. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (no runtime code touched; workflow YAML validated — see Verification) - [ ] I have added or updated tests where applicable (n/a — CI workflow change) - [x] I have updated relevant documentation to reflect my changes (none needed) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green (pending — will confirm once checks run) - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (pending review pass) - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-23 21:19:30 -07:00
Devin Foley	176a9e8230	fix(built-in-agents): allow first-time setup of a needs_setup built-in under board-approval policy (#10129 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Companies can require board approval for new agents; built-in agents (e.g. the Reflection Coach / Briefs) are provisioned through the `built-in-agents` service `provision()` > - Some built-in agents are auto-provisioned as a hire that, once approved, resolves to an idle agent row whose `adapterConfig` is still empty — status `needs_setup` > - When the board operator then opens that agent's setup dialog and submits the adapter config, `provision()` saw `adapterType`/`adapterConfig` on an already-existing row and classified it as a reconfiguration, throwing a dead-end 409: "Built-in agent adapter changes require board approval before they can be applied." > - The operator is the board, so there was no one left to grant an approval they already implicitly hold — setup could never be completed > - This pull request treats first-time adapter setup of a `needs_setup` built-in as the first-time configuration it actually is, applying it directly while still gating genuine reconfiguration of a live agent > - The benefit is the board can finish setting up an auto-provisioned built-in agent without hitting an unsatisfiable approval wall ## Linked Issues or Issue Description <!-- No public GitHub issue exists; describing the underlying bug in-PR following the bug_report template. --> What happened? With "require board approval for new agents" enabled, completing the adapter setup of an auto-provisioned but unconfigured built-in agent (status `needs_setup`, e.g. the Reflection Coach) failed with a 409 — "Built-in agent adapter changes require board approval before they can be applied." — even for the board user. Because the operator is the board, no additional approver existed, so setup was permanently blocked. Root cause: in `builtInAgentService.provision()`, any request carrying `adapterType`/`adapterConfig` against an existing row was treated as a reconfiguration and gated, regardless of whether that row had ever completed its initial adapter setup. An auto-provisioned hire resolves to an idle row with an empty `adapterConfig` (`needs_setup`), so its very first configuration was misclassified. Expected behavior The board can complete first-time setup of an already-sanctioned built-in agent without a fresh approval, matching the behavior when board approval is not required. Genuine reconfiguration of an already-configured (`ready`/`paused`) agent should still require approval. Steps to reproduce 1. In a company with `requireBoardApprovalForNewAgents` enabled, have a built-in agent auto-provisioned so its row exists but its adapter is unconfigured (status `needs_setup`). 2. As the board user, open that agent's setup dialog and submit an adapter type + config. 3. Observe the 409 "Built-in agent adapter changes require board approval before they can be applied." with no way for the board to grant the approval. Deployment mode Local single-instance / self-hosted (server `built-in-agents` service). ## What Changed - `server/src/services/built-in-agents.ts`: In `provision()`, when the existing built-in row has not yet completed adapter setup (`!hasCompleteAdapterConfig(...)`, i.e. `needs_setup`), first-time adapter configuration now applies directly via `ensure()` — the same path used when board approval is not required. The hire that created the row was already sanctioned, so no fresh approval is required. - Reconfiguration of an already-configured (`ready`/`paused`) built-in agent stays gated behind board approval exactly as before, and `pending_approval` rows are handled before the new branch. - `server/src/__tests__/built-in-agents.test.ts`: Added a regression test — under `requireApproval: true`, completing first-time setup of a `needs_setup` built-in returns `approval: null`, transitions the agent to `ready`, and creates no approval row. ## Verification ```bash cd server npx vitest run src/__tests__/built-in-agents.test.ts # Test Files 1 passed (1) # Tests 31 passed (31) ``` - New test `completes first-time setup of a needs_setup built-in without a fresh board approval` passes. - Full `built-in-agents.test.ts` suite (31 tests) passes, including existing tests that assert genuine reconfiguration of a configured agent remains gated. ## Risks Low risk. The change narrows an over-broad approval gate: it only opens the direct-apply path for rows that have never completed adapter setup (`needs_setup`), determined by the existing `hasCompleteAdapterConfig` predicate that already drives `deriveBuiltInAgentStatus`. Already-configured (`ready`/`paused`) agents, and `pending_approval` rows, are unaffected and still gated. No schema or migration changes. ## Model Used Claude Opus 4.8 (`claude-opus-4-8`), 1M context, extended thinking, with tool use / code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above (searched my open PRs and compared patch-ids — no duplicate exists) - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-23 14:53:15 -07:00
Devin Foley	429792f1f3	fix(interactions): stop wedging confirmation accept on a terminal workspace_finalize (#10099 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - When an agent finishes work in an execution workspace, the board can confirm the result through an issue-thread interaction (e.g. the "Merged" / mark-done confirmation button on a `request_confirmation`). > - That accept action is gated: it must not race a worktree sync-back (`workspace_finalize`) that is still copying the agent's commits out of the sandbox, or the board could act on a base that hasn't received them yet. > - The gate (`runWorkspaceIsFinalized`) treated the sync-back as "settled" only when the latest `workspace_finalize` op was `succeeded` — so a run whose finalize reached a terminal `failed` state, or died leaving a stale `running` op, was treated as "still syncing" forever. > - Users hit a permanent, misleading `... has not finished syncing its workspace` error and could never click "Merged", even though nothing was syncing and the run had long since ended. > - This PR fixes the settle semantics so the gate blocks only while a sync-back is genuinely pending or in flight, and treats any terminal (or stale-orphaned) finalize as done. > - The benefit is that a failed or abandoned sync-back no longer wedges the human confirmation, while a genuinely in-flight sync-back on a live run still blocks correctly. ## Linked Issues or Issue Description No public GitHub issue exists for this. Describing the bug in-PR (bug report): What happened Clicking the "Merged" / mark-done confirmation at the bottom of an issue thread returns an error that the workspace "has not finished syncing its workspace" — but nothing is actually syncing, and the run that created the interaction has already ended. The confirmation is permanently stuck; the only workaround is to merge and mark the task done manually. Expected behavior Once the source run's worktree sync-back has finished — whether it succeeded, failed, or was skipped — the confirmation should be acceptable. The gate should block only while a sync-back is genuinely still running on a live run. Steps to reproduce Have an agent run reach `workspace_finalize` and end without a `succeeded` finalize (e.g. the sync-back fails, or the run process dies mid-finalize leaving a `running` op). Then attempt to accept the `request_confirmation` interaction it created → 409 "... has not finished syncing its workspace" with no way to proceed. Paperclip version or commit Reproduced on the current `master` line (server service); root cause is in `runWorkspaceIsFinalized` in `server/src/services/issues.ts`. Deployment mode Local / self-hosted instance (server service). Root cause `runWorkspaceIsFinalized` returned `true` only when the latest `workspace_finalize` operation was `succeeded`. A terminal `failed` finalize (the sync-back ran and failed; it will not retry within that run) and a `running` finalize left behind by a dead run both left the gate closed forever. ## What Changed - `runWorkspaceIsFinalized` (server/src/services/issues.ts) now treats a sync-back as settled when the latest `workspace_finalize` op reached any terminal status (`succeeded`, `failed`, or `skipped`), instead of only `succeeded`. - A `workspace_finalize` still marked `running` blocks only while its owning run is alive; a `running` record left behind by a terminal/missing run is treated as stale (settled), so a dead run can no longer wedge the gate. - Preserved existing behavior for the other cases: no operations recorded at all → settled; earlier phases recorded but no `workspace_finalize` yet → still blocks (the sync-back hasn't been attempted). - Extracted the run-liveness check into a shared exported helper `heartbeatRunIsTerminalOrMissing` and reused it from the existing `isTerminalOrMissingHeartbeatRun` closure (no behavior change there). - Added a short comment at the confirmation-accept gate (server/src/services/issue-thread-interactions.ts) documenting the relaxed settle semantics. - The dependency-readiness / blocker barrier (`listPendingFinalizeBlockerIssueIds`) is deliberately left unchanged: an automated dependent must not proceed onto a base that never received a blocker's synced-back commits, so a failed finalize keeps that gate closed. Only the human-driven confirmation accept is relaxed. - Added regression tests for: failed finalize, stale `running` finalize on a dead run, and a genuinely `running` finalize on a live run (must still block). ## Verification - `cd server && node_modules/.bin/vitest run src/__tests__/issue-thread-interactions-service.test.ts -t "accept"` → 17 passed (includes the 3 new regression tests), 21 unrelated tests skipped by the name filter. - Manual reasoning walkthrough of `runWorkspaceIsFinalized` for each op-history shape (no ops / earlier-phase-only / terminal finalize / running-on-dead-run / running-on-live-run) confirms the intended block-vs-settle outcome. ## Risks - Low risk and narrowly scoped to the human confirmation-accept gate. The only behavioral change is that a terminal (`failed`/`skipped`) or stale-orphaned `running` finalize now settles the gate instead of blocking forever. - A genuinely in-flight sync-back on a live run still blocks (covered by a regression test), so the accept cannot race commits that are actively being synced back. - The blocker/dependency barrier for automated dependents is unchanged, so no dependent will be advanced onto a base missing a failed blocker's commits. ## Model Used - Provider/model: Claude (Anthropic), Opus 4.8, model ID `claude-opus-4-8`, 1M context window. - Capabilities used: extended thinking, tool use (repo inspection, local test execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-23 10:34:47 -07:00
Devin FoleyandPaperclip	0ef3b320c7	Harden `POST /plugins/install`: canonicalize `localPath` for all instances; bundled-only floor for managed instances (#10067 ) Builds on #10058 — managed detection keys off the presence of the `PAPERCLIP_MANAGED_CONFIG` env var that PR introduces, deliberately never its parsed body. Summary. Two layered hardenings of the plugin install route. (1) For all instances: `localPath` installs previously skipped the package-name validation entirely; the path is now null-byte-checked, resolved absolute, `realpath`'d (collapsing `..` traversal and symlinks), and required to be an existing directory before the loader ever sees it. (2) For instances running under a managed hosting control plane (detected by the presence of `PAPERCLIP_MANAGED_CONFIG` — deliberately never its body, so a corrupted document cannot widen the surface): registry/npm installs return 403, and `localPath` installs must canonicalize to inside the bundled plugin catalog root (`packages/plugins`) — a positive allowlist enforced in code at the route, independent of any flag value. Self-hosted behavior is otherwise unchanged. ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The plugin system lets instance admins install plugins from a registry or from a local filesystem path, and plugin installation is code execution on the host > - The `localPath` branch of `POST /plugins/install` skips the validation applied to registry installs; the raw path reaches the plugin loader without canonicalization > - Separately, instances operated by a managed hosting control plane must constrain installs to the bundled plugin catalog, because there the host belongs to the operator, not the tenant > - This pull request canonicalizes and validates `localPath` for all instances, and adds a bundled-only install floor for managed instances > - The benefit is a smaller install-route attack surface everywhere, and a positive code-enforced allowlist where the operator owns the machine ## Linked Issues or Issue Description No public issue exists; `bug_report` template fields for the validation gap this PR fixes: - What happened: `POST /plugins/install` with `localPath` set bypasses the package-name validation entirely; the un-canonicalized path (relative segments, symlinks, no existence check) is handed straight to the plugin loader. - Expected behavior: path installs are validated like registry installs — null-byte-checked, resolved absolute, `realpath`'d, and required to be an existing directory before the loader sees them. - Steps to reproduce: as an instance admin, call `POST /plugins/install` with a `localPath` containing `..` traversal or a symlink pointing outside any plugin directory; observe the loader receives the raw path. Exploitability is bounded (the route already requires instance admin), so this is hardening of an admin-only surface rather than an open exploit. - Version: current `master`. The managed-instance bundled-only floor layered on top is new behavior (motivation: on managed hosting, arbitrary plugin install is arbitrary code execution on operator infrastructure), aligned with the in-progress "Cloud deployments" milestone in `ROADMAP.md`. ## What Changed - New `server/src/services/plugin-install-guard.ts` — three pure primitives: managed detection (presence-based), path canonicalization (null-byte check → absolute resolve → `realpath` → must be an existing directory), and segment-based containment in the bundled plugin catalog root. - Route enforcement in `server/src/routes/plugins.ts`: npm/registry installs return 403 on managed instances; `localPath` installs are canonicalized on every instance and, on managed instances, must land inside the bundled catalog root. - The plugin loader now receives the canonical path instead of the raw request string. ## Verification - 15 guard unit tests (`server/src/__tests__/plugin-install-guard.test.ts`): traversal, symlink escape, null byte, file-vs-directory, string-prefix sibling root. - 13 route security tests (`server/src/__tests__/plugin-install-route-security.test.ts`): 403 matrix on managed instances + self-hosted happy paths. - 36 existing plugin route authz tests green (`server/src/__tests__/plugin-routes-authz.test.ts`). - Server `tsc --noEmit` clean. ```bash cd server pnpm vitest run src/__tests__/plugin-install-guard.test.ts src/__tests__/plugin-install-route-security.test.ts src/__tests__/plugin-routes-authz.test.ts pnpm exec tsc --noEmit ``` ## Risks - Managed instances: npm/registry installs and out-of-catalog `localPath` installs now return 403 — intended new behavior, enforced in code rather than configuration. - All instances: `localPath` installs that previously pointed at nonexistent paths or non-directories now fail with 400 before reaching the loader (previously the loader failed later, less safely). Symlinked deployment layouts are handled by canonicalizing both sides of the containment check. - Self-hosted npm install path is unchanged. Low residual risk. ## Model Used Claude Fable 5 (`claude-fable-5`), extended thinking, agentic tool use; independently peer-reviewed by a second AI agent before push. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-22 20:48:41 -07:00
Devin Foley	04e070bf45	docs(skill): require agents to claim only monitors they actually scheduled (#10064 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work, and the shared `skills/paperclip/SKILL.md` is the behavioral contract every managed agent follows each heartbeat. > - Issue continuation between heartbeats depends on real, persisted state: an issue only auto-resumes when it has a scheduled issue monitor (`monitorNextCheckAt` + an execution-policy `monitor` block) that the server's `tickDueIssueMonitors` scheduler polls and re-wakes via `issue_monitor_due`. > - A run/heartbeat is an ephemeral execution window — nothing keeps "watching" after it exits — but the skill never said this, so agents narrated a "watcher in this run" as if a live subscription existed. > - That gap produced a concrete user-facing failure: an agent claimed "a watcher in this run wakes me when CI + Greptile complete," then the run ended with no monitor scheduled and nothing ever resumed, leaving the user unsure whether a watcher existed at all. > - This PR closes the gap by documenting what a monitor actually is and adding hard rules so agents only claim a watcher they have actually scheduled, describe it in checkable terms, and never imply a live watcher on a task they mark `done`. > - The benefit is that agent narration stays consistent with the disposition guard and recovery classifier that already enforce these paths in state, so users get accurate expectations about whether and when a task will resume. ## Linked Issues or Issue Description This is a documentation-only change to a shared agent skill, so no code issue is required. The underlying problem it addresses: Problem or motivation — Agents were telling users that a "watcher in this run" would wake them when external checks (CI, Greptile) finished, when no persisted issue monitor had been scheduled. Because a heartbeat is ephemeral, no such watcher exists after the run exits, so the task silently never resumed and the user was left confused about what would happen next. Proposed solution — Document, in the shared skill, exactly what an issue monitor is (durable `monitorNextCheckAt` + execution-policy `monitor` block, polled by `tickDueIssueMonitors`, re-woken via `issue_monitor_due`) and add rules that agents may only claim a watcher/monitor after actually scheduling one, must describe it in checkable terms (kind / next check / timeout / attempts), and must never imply a live watcher on a task being marked `done`. Alternatives considered — Enforcing purely in server state (the disposition guard and recovery classifier already reject `in_review`/parked issues without a real wake path). That enforcement exists but does not stop an agent from narrating a non-existent watcher in a comment; aligning the skill guidance with the existing state enforcement is the missing piece. ## What Changed - Added a "Monitors and Watchers (say only what you actually scheduled)" subsection to `skills/paperclip/SKILL.md` explaining that a watcher does not live inside a run, and that only a persisted issue monitor can auto-resume an issue (with the concrete fields and the `tickDueIssueMonitors` / `issue_monitor_due` polling path). - Added three behavioral rules: only claim a monitor after scheduling one (and how to schedule/confirm it via `PATCH /api/issues/{id}` and `monitor/check-now`); describe monitors in checkable terms; never imply a live watcher on a task marked `done`. - Cross-referenced the rule from the Critical Rules list. - Tightened the final-disposition checklist so `in_review` / `in_progress` continuation requires a real, non-null `monitorNextCheckAt` rather than a merely described one. ## Verification - Docs-only change to `skills/paperclip/SKILL.md`; no code paths are affected. - Confirmed every identifier referenced in the new text is real in the codebase: `monitorNextCheckAt`, `monitorScheduledBy`, `executionPolicy.monitor`, `tickDueIssueMonitors`, and the `issue_monitor_due` wake reason. - Rendered the Markdown to confirm the new subsection and the Critical Rules bullet display correctly and links resolve within the document. - `git diff` confirms the change is limited to the single skill file (14 insertions, 2 deletions). ## Risks Low risk. This is guidance text in a shared agent skill with no runtime or schema impact. Worst case is stylistic wording that can be refined in a follow-up; it cannot break builds, migrations, or behavior. It strengthens (never loosens) the existing disposition guarantees. ## Model Used Claude Opus 4.8 (model id `claude-opus-4-8`, 1M-context variant) running in an agent harness with extended thinking and tool use (file edit, shell, git, GitHub CLI). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [ ] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-22 20:47:52 -07:00
Devin Foley	62e367c1b2	fix(ui): rewrite recovery and blocked-notice copy in plain language (#10065 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The issue-detail UI shows recovery cards and blocked/parked notices when a task loses its next step — a run finished with no disposition, a task is stranded, work is blocked behind other tasks, or an assigned item sits in the backlog > - That copy was written in the scheduler's internal vocabulary — "Corrective wake queued", "Graph Liveness", "lost a live action path", "the responsible" — which describes Paperclip's internals rather than the user's situation > - Users seeing these cards report having no idea what the card means or what they are supposed to do > - This pull request rewrites the user-facing copy in plain language and adds explicit calls to action that match the options in the card's Resolve menu > - The benefit is that a non-expert operator can read a recovery or blocked notice and immediately understand what happened and which action to take next ## Linked Issues or Issue Description No public GitHub issue exists for this; describing the problem here (bug-report format): - What happened: Recovery action cards and blocked notices render internal jargon, e.g. the headline "Paperclip detected this task lost a live action path. A recovery owner needs to act.", the chip "Corrective wake queued", the kind label "Graph Liveness", and phrases like "Comments still wake the responsible". Status values also appear as raw code literals (`in_progress`, `todo`). - Expected behavior: These notices should tell a normal user, in plain language, what happened and what to do next (retry the task, mark it done, send it for review, or record a blocker). - Impact: Operators stall on tasks that only need a simple disposition because the UI doesn't tell them that's what is being asked. Related prior work: #9417 (merged) made the reopen-suppressed blocked message explicit; this PR extends the same plain-language treatment to the rest of the recovery and blocked-notice copy. ## What Changed - Recovery card headlines for `missing_disposition`, `stranded_assigned_issue`, and `issue_graph_liveness` now say what Paperclip found and name the concrete next steps ("try the task again, mark it done, or send it for review") matching the card's Resolve menu. - The `issue_graph_liveness` kind label "Graph Liveness" is now "Task Needs Next Step", and the "Wake" metadata row is now "Follow-up". - Wake-policy chips describe actual behavior: "An agent will be asked to choose the next step" (was "Corrective wake queued"), "Board will decide", "Manual follow-up needed", "Repair needed before retry", "Check scheduled". - Blocked/waiting/parked notices say "the assignee" instead of "the responsible" / "responsible agent", and "notify" instead of "wake". - The still-needs-a-next-step notice drops raw `in_progress` code literals and keeps a plain-language option list (mark done or cancelled, send for review, record what is blocking it, delegate follow-up). - Parked-backlog notice renders "To do / In progress" as plain labels instead of code literals. - Component tests updated to pin the new copy and the successful-run example options. ## Verification - `cd ui && npx vitest run src/components/IssueRecoveryActionCard.test.tsx src/components/IssueBlockedNotice.test.tsx src/components/IssueAssignedBacklogNotice.test.tsx src/components/IssueChatThread.test.tsx` — 4 files, 126 tests, all passing. - Copy-only review: the diff touches display strings, one label map entry, and test assertions; no control flow, props, or identifiers change. ## Risks - Low risk — user-facing strings and test updates only. No behavior, API, or schema changes. The only functional surface is that anything keying off the displayed text (e.g. screenshots, external docs) will show the new wording. ## Model Used - Claude (Anthropic) — Claude Fable 5, model ID `claude-fable-5`, extended thinking enabled, running in Claude Code with agentic tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (no docs reference this copy) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-22 20:47:00 -07:00
Devin FoleyandPaperclip	0f898f95af	Render managed experimental settings as locked, with a 'Managed by Paperclip Cloud' badge (#10061 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The experimental settings page renders one interactive toggle per feature from the settings API > - On managed instances some values are enforced by the hosting control plane, and the API now reports those keys as managed > - Rendering enforced values as live toggles misleads users: the click appears to work, and the value silently snaps back > - This pull request renders managed keys as locked toggles with a "Managed by Paperclip Cloud" badge and guards the handlers so no PATCH can be emitted > - The benefit is UI honesty on managed instances, with self-hosted responses rendering exactly as before ## Linked Issues or Issue Description Builds on #10058 — renders the per-key `managedKeys` metadata #10058 adds to settings responses (typing shared from #10058 at rebase). No public issue exists; `feature_request` template fields: - Problem or motivation: on managed instances users see fully interactive toggles for settings the control plane enforces; changes appear to apply and never do, with no explanation. - Proposed solution: disabled toggle + badge + guarded handler driven by the settings response's managed-key metadata; the ~17 uniform setting cards are extracted into one shared component with copy, aria-labels, and patch payloads preserved verbatim. - Alternatives considered: hiding managed settings entirely (users lose sight of the effective value and why it is fixed); tooltip-only hints on still-active toggles (doomed PATCHes are still emitted and stripped server-side). - Roadmap alignment: supports the in-progress "Cloud deployments" milestone in `ROADMAP.md`. ## What Changed - When the settings API reports a feature key as managed (`managedKeys` from the managed-config overlay), the experimental settings page renders that toggle disabled with a badge and a guarded handler, so a click can never emit a PATCH. Previously, managed-instance users saw fully interactive toggles they could never actually change. Self-hosted responses (no `managedKeys`) render exactly as before. - The ~17 copy-pasted uniform setting cards are extracted into one `ExperimentalToggleCard` component with titles, descriptions, footnotes, aria-labels, and patch payloads preserved verbatim; the two bespoke cards get inline managed handling (the managed auto-recovery toggle also cannot open its preview dialog). - `ui/src/api/instanceSettings.ts` response typing now uses the shared `InstanceExperimentalSettingsWithManaged` / `ManagedSettingMetadata` types from #10058; `ui/src/pages/InstanceExperimentalSettings.tsx` locked rendering + card extraction; tests. ## Verification - 24 page tests (20 existing unmodified + 4 new: locked badge with no PATCH while unmanaged keys stay editable; managed auto-recovery opens no dialog; an open recovery preview closes with no PATCH when a refresh marks auto-recovery managed; self-hosted unaffected): `pnpm --filter @paperclipai/ui exec vitest run src/pages/InstanceExperimentalSettings.test.tsx` - `pnpm --filter @paperclipai/ui typecheck` clean ## Risks - Low risk. UI-only change; no server or API behavior changes. Self-hosted responses carry no `managedKeys`, so the page renders exactly as before there. The card extraction preserves copy, aria-labels, and patch payloads verbatim, covered by the 20 pre-existing page tests passing unmodified. ## Model Used Claude Fable 5 (`claude-fable-5`), extended thinking, agentic tool use; independently peer-reviewed by a second AI agent before push ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-22 20:46:00 -07:00
Devin FoleyandPaperclip	c4cdcc4826	Generalize bundled plugin provisioning: `ensureBundledKubernetesPlugin` → `ensureBundledPlugins` (#10063 ) Builds on #10058 — reads `plugins.autoInstall` from the parsed managed-config contract #10058 introduces (the interim `readManagedPluginAutoInstall` shim is retired at rebase). Summary. Boot-time bundled-plugin provisioning becomes catalog-driven. A new bundled-plugin catalog lists the sandbox providers shipped in-tree (keys like `kubernetes`, `daytona` → plugin key + path under the catalog root). Managed instances read `plugins.autoInstall` from `PAPERCLIP_MANAGED_CONFIG`; unknown keys or paths escaping the catalog root (symlinks resolved) throw before listen — a managed instance refuses to start rather than boot half-provisioned. Installation keeps today's mechanism: an in-process, fail-safe `loader.installPlugin({ localPath })` under a system actor — no HTTP route, no user, no role widening. Self-hosted boot is unchanged (kubernetes bundle only, existing env override honored, install failures still log-and-continue). Semantics. A plugin already present in any non-uninstalled state is skipped, so an operator-disabled plugin is never silently re-enabled; managed mode reinstalls soft-uninstalled bundles (the control plane owns provisioning); removal from the autoInstall list never auto-uninstalls. ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Sandbox-provider plugins ship in-tree, but boot-time provisioning is hard-coded to exactly one of them (Kubernetes) via a bespoke function > - On managed hosting, tenant users have no install privileges, so any bundled plugin that is not provisioned at boot is unusable > - Widening install routes or granting roles to fix that would trade a provisioning gap for a security regression > - This pull request generalizes the existing boot installer into a catalog-driven `ensureBundledPlugins`, fed by `plugins.autoInstall` from `PAPERCLIP_MANAGED_CONFIG` > - The benefit is that managed tenants get working bundled plugins out of the box, through the same in-process, role-free mechanism the codebase already trusts, while self-hosted boot is unchanged ## Linked Issues or Issue Description No public issue exists; `feature_request` template fields: - Problem or motivation: on managed instances tenant users cannot install plugins (by design they never hold instance admin), so even plugins shipped with the product are unusable; boot provisioning currently knows only the Kubernetes bundle. - Proposed solution: a bundled-plugin catalog plus `ensureBundledPlugins(keys)` driven by the managed config; same in-process `loader.installPlugin({ localPath })` under a system actor; unknown keys or catalog-escaping paths fail startup; already-present plugins are skipped so operator-disabled plugins are never silently re-enabled. - Alternatives considered: granting tenant users install privileges (widens secrets/adapters/settings access to solve a one-button problem); a separate non-admin install route for bundled plugins (new authz surface; provisioning removes the need for any install action at all). - Roadmap alignment: supports the in-progress "Cloud deployments" milestone and builds on the shipped sandbox-provider milestone in `ROADMAP.md`. Refs #10058. ## What Changed - New `server/src/services/bundled-plugins.ts`: the bundled-plugin catalog, the fail-to-start resolver (`resolveBundledPluginInstalls`, positive allowlist + catalog-root containment with symlinks resolved), and the fail-safe installer (`ensureBundledPlugins`). - `server/src/app.ts`: replaces the hard-coded `ensureBundledKubernetesPlugin` boot hook with resolver + installer wiring, with test hooks (`managedPluginAutoInstall`, `bundledPluginCatalogRoot` options). - `server/src/index.ts`: passes `plugins.autoInstall` from the single fail-closed `PAPERCLIP_MANAGED_CONFIG` startup parse (#10058) into `createApp`; absent env means self-hosted and changes nothing. ## Verification - 24 new tests in `server/src/__tests__/bundled-plugins.test.ts` (catalog resolution, containment incl. symlink and `..` escapes, skip/reinstall matrix, self-hosted invariants, installer error paths) — all green. - 85 adjacent startup/plugin-route/auto-build/managed-config tests green (`managed-config`, `instance-settings-managed-overlay`, `plugin-install-autobuild`, `plugin-routes-authz`, `server-startup-feedback-export`). - Server `tsc --noEmit` clean. ```bash cd server npx vitest run src/__tests__/bundled-plugins.test.ts npx vitest run src/__tests__/managed-config.test.ts src/__tests__/instance-settings-managed-overlay.test.ts src/__tests__/plugin-install-autobuild.test.ts src/__tests__/plugin-routes-authz.test.ts src/__tests__/server-startup-feedback-export.test.ts npx tsc --noEmit ``` ## Risks - Managed instances with a malformed or unknown `plugins.autoInstall` entry now refuse to start (fail closed, by design) instead of booting half-provisioned; harness misconfiguration surfaces as a precise startup error. - Self-hosted behavior is unchanged (kubernetes bundle only, `PAPERCLIP_KUBERNETES_PLUGIN_PATH` honored without containment, install failures log-and-continue), so the default deployment path carries low risk. - No uninstall path exists in this module; removal from the autoInstall list can leave a previously provisioned plugin installed (intentional v1 semantics, documented in code). ## Model Used Claude Fable 5 (`claude-fable-5`), extended thinking, agentic tool use; independently peer-reviewed by a second AI agent before push. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-22 20:35:51 -07:00
Devin FoleyandPaperclip	216d3d2680	Managed-instance config: fail-closed PAPERCLIP_MANAGED_CONFIG parsing and read-time settings overlay (#10058 ) Builds on. #10055 — the `catalogVersion` this config document pins is the feature-catalog artifact #10055 emits. Summary. Instances operated by a managed hosting control plane can now receive instance configuration through a single environment variable, `PAPERCLIP_MANAGED_CONFIG` (versioned JSON: `mode`, `catalogVersion`, `features`, `plugins.autoInstall`). When the variable is absent the instance is self-hosted and nothing changes. When present, parsing is strict and fail-closed: blank value, malformed JSON, unknown feature key, a feature key this build's feature catalog does not mark tier `managed`, missing required section, or unsupported version refuses startup with a precise error — a typo that silently does nothing is how a security control quietly fails. Managed feature values are overlaid at read time inside the instance settings service (never persisted), so a DB restore or manual row edit cannot resurrect a disabled capability; responses expose per-key `managedKeys` metadata (`managed: true`, `managedBy`) so clients can render locked state. ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Paperclip runs both self-hosted and under managed hosting, where an operator's control plane owns instance configuration > - Today instance feature settings live only in the tenant database; a hosting control plane has no way to enforce a configuration that tenant-side writes or restores cannot undo > - Managed configuration will carry security posture, so delivery must be atomic and parsing must fail closed — a typo that silently does nothing is how a security control quietly fails > - This pull request adds strict parsing of one `PAPERCLIP_MANAGED_CONFIG` env var and overlays its feature values at read time inside the settings service, never persisting them > - The benefit is a minimal, auditable managed-hosting contract: absent var ⇒ self-hosted instances are byte-for-byte unchanged; present ⇒ deterministic, locked configuration surfaced to clients via per-key managed metadata ## Linked Issues or Issue Description Refs #966 — this PR delivers that issue's "managed config injection" hook, via a strict env-var contract rather than the config-file path it sketches; the issue's other hooks (identity header, health, usage webhook, lifecycle, external secrets, IAM auth) are out of scope, so the PR refs rather than closes it. Mechanism differs from #966's proposal, so the `feature_request` fields are also filled in: - Problem or motivation: managed hosting deployments need to centrally enable/disable instance features; DB-stored settings can be edited, restored, or migrated back to permissive values, and nothing marks a value as operator-enforced. - Proposed solution: one versioned JSON env var; fail-closed parse at startup; read-time overlay in the settings service (precedence: managed value over stored value over schema default); `managedKeys` metadata in settings responses so clients can render locked state. - Alternatives considered: per-feature env vars (non-atomic across a half-updated env set, unbounded env surface); seeding the DB at boot (persisted values can be edited or restored over, and cannot express "forced"); lenient warn-and-drop parsing (fails open — unacceptable for a security-bearing control). - Roadmap alignment: supports the in-progress "Cloud deployments" milestone in `ROADMAP.md`. ## What Changed - New `server/src/services/managed-config.ts` (pure parser over the env record) - Startup parse ordered before the first `instanceSettingsService` construction in `server/src/index.ts` - Read-time merge + `managedKeys` in the settings service - Shared validator updates ## Verification - 29 parser/overlay tests (fail-closed matrix incl. blank/whitespace env, missing sections, catalog-tier mismatch, empty-section happy path): `pnpm vitest run src/__tests__/managed-config.test.ts src/__tests__/instance-settings-managed-overlay.test.ts` (from `server/`) - 40 existing settings route/service tests green: `pnpm vitest run src/__tests__/instance-settings-routes.test.ts src/__tests__/instance-settings-service.test.ts` (from `server/`) - 15 shared validator tests: `pnpm vitest run src/validators/instance.test.ts` (from `packages/shared/`) - Server `tsc --noEmit` clean: `pnpm typecheck` (from `server/`) ## Risks - Self-hosted instances (no `PAPERCLIP_MANAGED_CONFIG` set) are byte-for-byte unchanged — the parser only runs when the variable is present. - For managed instances, a malformed document now refuses startup by design (fail-closed). This is an intentional behavioral guarantee, not a regression: the control plane owns the variable and a precise startup error is the contract. - Overlay values are never persisted, so no migration or data-shape risk. ## Model Used Claude Fable 5 (`claude-fable-5`), extended thinking, agentic tool use; independently peer-reviewed by a second AI agent before push. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-22 19:48:43 -07:00
Devin FoleyandPaperclip	ad74fb5450	Add a feature catalog build artifact derived from the experimental settings schema (#10055 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Instances expose ~23 experimental feature settings, all declared in one shared zod schema and toggled per instance > - Deployment tooling and hosting control planes have no machine-readable list of those feature keys for a given release — the schema is only reachable from code that imports the package > - Any external system that references feature keys therefore does so as free text, and typos drift silently > - This pull request derives a versioned `feature-catalog.json` build artifact from the schema, with a compiler-checked metadata map so the schema stays the single source of truth > - The benefit is a stable contract external tooling can validate feature-key references against, with zero runtime behavior change ## Linked Issues or Issue Description No public issue exists; `feature_request` template fields: Problem or motivation: External deployment tooling cannot enumerate or validate an instance's feature keys per release; free-text references fail silently when keys are renamed or removed. Proposed solution: A metadata map keyed by the settings schema's own keys (compiler flags drift) plus a build step emitting `feature-catalog.json` (keys, tiers, defaults, `catalogVersion`) as a release artifact. Alternatives considered: A hand-maintained catalog file (drifts from the schema); serving the schema from a runtime API (requires a running instance at validation time — a build artifact works offline and pins to a release). Roadmap alignment: Supports the in-progress "Cloud deployments" milestone in `ROADMAP.md`. ## What Changed Adds a metadata map (title, description, tier, cloud/self-hosted defaults) keyed by the keys of `instanceExperimentalSettingsSchema`, so the schema stays the single source of truth and the compiler flags any drift. A new build step (`build:feature-catalog --version <v>`) emits `feature-catalog.json` — all 23 feature keys, their tiers, and a `catalogVersion` — as a release artifact that managed-hosting control planes can validate feature-flag writes against. No runtime behavior changes. - New `packages/shared/src/feature-catalog.ts`: per-flag metadata map keyed by a type derived from the settings schema (adding/removing/renaming a flag without updating the map is a compile error), plus `featureCatalogArtifactSchema` and `buildFeatureCatalogArtifact`/`renderFeatureCatalogArtifact` for the artifact - New `scripts/generate-feature-catalog.ts` wired as `pnpm build:feature-catalog --version <v>` - `scripts/create-github-release.sh` generates the artifact and uploads it as a GitHub Release asset (with a dry-run preview line) - Tests in `packages/shared/src/feature-catalog.test.ts` ## Verification - `vitest run packages/shared/src/feature-catalog.test.ts` — 9 tests: schema-key coverage, drift detection, artifact shape - `pnpm --filter @paperclipai/shared typecheck` - Artifact generation run end-to-end: `pnpm build:feature-catalog --version 0.0.0-test` emits 23 keys with `catalogVersion` ## Risks Low risk — no runtime behavior changes; the change is metadata, a build script, and a release-artifact emission step only. ## Model Used Claude Fable 5 (`claude-fable-5`), extended thinking, agentic tool use; independently peer-reviewed by a second AI agent before push ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-22 18:12:56 -07:00
Devin FoleyandPaperclip	1944c86153	fix(ci): preserve required e2e check for sharded runs (#9923 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The pull request workflow is the main merge gate for changes to that app. > - The Playwright e2e lane is expensive because every spec shares one isolated server and runs serially. > - Splitting that lane across runners shortens the critical path, but the public required-check contract still needs a check named exactly `e2e`. > - This pull request shards the real e2e work while preserving a fast aggregate `e2e` job for branch protection. > - The benefit is a faster PR workflow without making otherwise-good PRs unmergeable because a legacy required check disappeared. ## Linked Issues or Issue Description No public GitHub issue exists for this CI follow-up. Related prior CI work: - Refs #8360 - Refs #9168 - Refs #9516 Bug report: ### What happened? Sharding the PR e2e lane directly at the workflow job level changes the emitted check names to shard-specific names, while existing branch protection expects a check named exactly `e2e`. ### Expected behavior The PR workflow should be able to run e2e specs across multiple runners while still emitting a stable aggregate check named `e2e`. ### Steps to reproduce 1. Open a PR against `master`. 2. Run the PR workflow with the e2e lane split only as a matrix job. 3. Observe that the shard checks complete, but a required check named exactly `e2e` never appears. ### Paperclip version or commit Current `master`. ### Deployment mode GitHub Actions pull request workflow. ## What Changed - Added `scripts/e2e-shard.mjs`, which partitions default Playwright e2e specs by recorded per-spec duration. - Added `scripts/e2e-shard-durations.json` with measured e2e spec durations so the slow smoke-lab spec does not dominate one runner. - Split the PR workflow e2e lane into two `e2e_shards` matrix jobs and added a fast aggregate job named exactly `e2e`. - Added `scripts/__tests__/e2e-shard.test.mjs` to lock the shard partition, ignored-spec sync, manifest coverage, and aggregate required-check contract. ## Verification - `node --test scripts/__tests__/e2e-shard.test.mjs` - `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs` - `git diff --check upstream/master..HEAD` - Searched GitHub for duplicate or related e2e-shard / required-check PRs and issues before opening this PR; no direct duplicate was found. ## Risks Low risk. The main risk is that the duration manifest can drift as specs are added or runtimes change; missing specs fall back to the median known duration, and the focused shard test catches empty, overlapping, or badly imbalanced partitions. ## Model Used OpenAI GPT-5 via Codex CLI coding agent, with shell/tool execution and repository inspection. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-20 16:40:19 -07:00
Devin Foley	2f42a4968d	Treat cloud-managed instances as bootstrapped in the health gate (#9912 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Paperclip instances can be self-hosted, or provisioned and managed by a cloud control plane that authenticates users through trusted headers validated against `PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN` (`resolveCloudTenantActor`) > - In `authenticated` deployment mode, the health route reports `bootstrapStatus: bootstrap_pending` until at least one `instance_admin` exists, and the UI locks everyone out at the "waiting on its first admin" claim screen until then — correct for self-hosted instances, where a human operator must claim the instance > - But the cloud-tenant trust middleware, by deliberate security hardening, never grants `instance_admin` and actively purges legacy grants — so a cloud-managed instance can never leave `bootstrap_pending`: the gate demands a role the middleware forbids > - Every control-plane-provisioned instance is therefore permanently locked at the claim screen even though its users and memberships exist > - This pull request makes the gate cloud-aware: when the tenant server token is configured, the instance is considered bootstrapped, because the control plane owns identity and there is no operator claim step > - The benefit is that cloud-managed instances become usable while self-hosted behavior stays byte-for-byte identical, now pinned by a previously missing regression test ## Linked Issues or Issue Description Refs #2927 (introduced the browser-native first-admin bootstrap flow this gate feeds). No existing public issue for the deadlock; inline description per the bug report template: - What happened?: an instance configured with `PAPERCLIP_DEPLOYMENT_MODE=authenticated` and `PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN` reports `bootstrapStatus: bootstrap_pending` forever. All users — including ones created via the trusted-header path with owner-level company membership — are locked out at the "This Paperclip is waiting on its first admin" screen. - Expected behavior: a control-plane-managed instance has no first-admin claim step; users arriving with control-plane identity should reach the app. - Steps to reproduce: 1. Run the server with `PAPERCLIP_DEPLOYMENT_MODE=authenticated` and a `PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN` set 2. Create users only through trusted cloud headers (the middleware upserts them but never grants `instance_admin`, and purges any legacy grants) 3. `GET /api/health` → `bootstrapStatus` stays `bootstrap_pending`; the UI shows the claim screen for every visitor, and no supported path exists to create the `instance_admin` the gate requires - Paperclip version or commit: reproducible on `master` as of 2026-07-20; present since the cloud-tenant `instance_admin` purge hardening landed. ## What Changed - `server/src/middleware/auth.ts`: new exported `isCloudManagedInstance()` predicate beside the trust middleware that defines the tenant-token contract. - `server/src/routes/health.ts`: the authenticated-mode first-admin gate is skipped when the instance is cloud-managed; `bootstrapStatus` reports `ready`. - `server/src/__tests__/health.test.ts`: two new tests — authenticated without the token → `bootstrap_pending` (previously untested regression baseline), and with the token → `ready` despite zero instance admins. ## Verification - `pnpm vitest run src/__tests__/health.test.ts` in `server/` — 13/13 - `pnpm vitest run src/middleware/cloud-tenant-actor.test.ts` — 6/6 - Manual: with the env vars from the repro steps set, `GET /api/health` now returns `bootstrapStatus: "ready"`; without the token, behavior is unchanged ## Risks - None for self-hosted deployments: without the env var the gate is the prior behavior, now pinned by the new regression test. - For cloud-managed instances the claim screen and `bootstrapInviteActive` flow no longer appear — intended; browser-based claim was already disabled in that configuration. ## Model Used - Claude (Anthropic) — model id `claude-fable-5`, via the Claude Code CLI harness with tool use (shell, file edits, test execution). Diagnosis and change agent-assisted, human-directed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (n/a — behavior documented in code comments and pinned by tests) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-20 15:59:08 -04:00
Devin Foley	fa4f900d1a	ci: label Docker images with their bundled schema migration set (#9908 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Paperclip publishes Docker images from this repo that self-hosters and orchestration tooling deploy; the server refuses to start when its database is missing any schema migration the build bundles (`ensureMigrations`) > - Anything that deploys these images therefore needs to know an image's schema expectations before deploying it — today that requires pulling the image or checking out the matching commit, both heavyweight for tooling that just wants to answer "will this image boot against a database migrated to N?" > - Getting this wrong is expensive: an image ahead of the applied schema crash-loops at startup and fails healthchecks after deployment resources are already created > - This pull request labels every published image with its bundled migration set (last migration file and count), computed at build time from `packages/db/src/migrations` in the same tree the image is built from > - The benefit is image/schema compatibility verification with two cheap registry requests (manifest + config blob), no pull, and no drift risk between label and image contents ## Linked Issues or Issue Description No existing public issue; inline description per the feature request template: - Subsystem affected: Docker image publishing (`.github/workflows/docker.yml`), `packages/db` migrations - Problem or motivation: deployment tooling cannot cheaply determine which schema migrations a published image expects; the only options are pulling the image or checking out the matching commit. Deploying an image whose bundled migrations exceed the applied schema makes the server refuse to start, so this check is needed before resources are created. - Proposed solution: OCI labels (`io.github.paperclipai.schema.last-migration`, `io.github.paperclipai.schema.migration-count`), computed from the migrations directory at build time via the existing `docker/metadata-action` step. Keys use org-based reverse-DNS (the GitHub org) so the label contract survives product-domain migrations. - Alternatives considered: a schema manifest published beside the image (second artifact to keep in sync — rejected); encoding schema info in tags (tags already carry semver/sha meaning — rejected). - Roadmap alignment: checked `ROADMAP.md` — no overlap with planned core work; this is build metadata only. ## What Changed - `.github/workflows/docker.yml`: a `Compute schema migration labels` step (`ls` + `sort` over `packages/db/src/migrations/.sql`) feeding two custom labels into the existing `docker/metadata-action` step. ## Verification - Workflow YAML validated locally. - The label-computation commands run against the current tree produce `last=0181_decision_training_retention_policy.sql`, `count=180`. - After merge, verify with: fetch the image config blob for a fresh `sha-` tag from ghcr and confirm both `io.github.paperclipai.schema.*` labels are present. ## Risks - Low. Labels are metadata only; no change to image contents. If the migrations directory ever moves, the label step fails the workflow loudly (`ls` exits non-zero) rather than publishing wrong labels. ## Model Used - Claude (Anthropic) — model id `claude-fable-5`, via the Claude Code CLI harness with tool use (shell, file edits). Change authored and verified agent-assisted, human-directed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (YAML validation; label commands — no app code changed) - [x] I have added or updated tests where applicable (n/a — CI metadata only) - [x] I have updated relevant documentation to reflect my changes (n/a — workflow comment documents the labels) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-20 15:54:31 -04:00
Devin FoleyandClaude	ce7dedf33d	perf(ci): balance general-server test shards by recorded suite duration (#9516 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Its PR CI runs the general-server vitest lane pinned to `maxWorkers=1` and sharded across 3 runners (introduced in #8360) > - Suites were assigned to shards round-robin by sorted file index, so shard test time was unbalanced: a recent PR run split 73s / 153s / 115s, and the heaviest shard made "General tests (server 2/3)" the slowest check in the whole workflow at 314s wall > - The slowest shard sets the lane's wall time, so unbalanced partitions waste the other two runners and stretch the PR critical path > - This pull request replaces the round-robin assignment with a deterministic longest-processing-time partition weighted by a checked-in per-suite duration manifest > - The benefit is near-even shard weights (projected 113s / 113s / 113s with the current manifest), taking roughly 40s off the PR critical path with no reduction in coverage ## Linked Issues or Issue Description - Refs #8360 (introduced the 3-way general-server sharding this PR rebalances) - No public issue exists. Problem: the general-server test lane's round-robin shard assignment ignores per-suite duration, so one shard can carry multiple 30s+ suites while another finishes in half the time; the slowest shard alone determines the check's wall time. ## What Changed - `scripts/general-server-shard.mjs` (new): manifest loader and deterministic LPT (longest-processing-time) partitioner; suites missing from the manifest get the median recorded weight, and a missing or malformed manifest degrades to uniform weights so the lane never fails on stale data - `scripts/general-server-shard-durations.json` (new): per-suite duration manifest sampled from a real PR run (240 suites); the `$comment` field documents how to regenerate it - `scripts/run-vitest-stable.mjs`: both shard-selection sites (run and `--dry-run`) now use the balanced partition instead of index round-robin - `scripts/__tests__/run-vitest-stable-shard.test.mjs`: 6 new tests covering skew-balance vs round-robin, determinism, median fallback for unlisted suites, malformed-manifest degradation, manifest coverage of the current suite set, and real-partition balance - `server/src/__tests__/heartbeat-issue-rewake-throttle.test.ts`: hardened the `afterEach` sweep — post-run bookkeeping (run-event records, follow-up wake scheduling) can still insert rows briefly after a run reaches a terminal status, and a late insert landing between the `agent_wakeup_requests` and `agents` deletes failed teardown with a foreign-key violation on the first CI attempt of this PR; the sweep now retries so a late background write cannot take down the shard - `release-verify.yml` shares the same runner script and inherits the balancing with no workflow change ## Verification - `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs` — 9/9 pass (run against current master) - `npx vitest run src/__tests__/heartbeat-issue-rewake-throttle.test.ts` — 6/6 pass against embedded Postgres with the hardened teardown - `node --test scripts/__tests__/release-verify-workflow.test.mjs` — 2/2 pass - `node scripts/run-vitest-stable.mjs --dry-run` with each shard flag shows every suite assigned exactly once across the 3 shards, with projected weights ~113s each ## Risks - Low risk: partition changes which runner executes which suite, not what runs; a completeness test asserts every suite is assigned to exactly one shard - The duration manifest will drift as suites are added/changed; unlisted suites get the median weight and a coverage test flags when the manifest covers less than half the suite set, so drift degrades balance gracefully rather than breaking the lane > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Fable 5 (`claude-fable-5`, Anthropic), extended thinking enabled, agentic tool use (file edits, shell, test execution) via Claude Code ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude (Paperclip SWE) <noreply@paperclip.ing>	2026-07-13 12:25:54 -07:00
Devin FoleyandCody	b49d178c46	fix(ui): experiments auto-recovery dialog leaves UI dimmed and locked after enabling (#9513 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Operators tune instance behavior through Settings → Experiments, where experimental features are toggled on and off > - The task graph liveness auto-recovery experiment shows a confirmation dialog (preview of what would be recovered) before it is enabled > - After confirming with "Enable only" or "Enable and run", the dialog's Radix overlay and the `pointer-events: none` body lock were left behind, dimming the page and blocking all interaction until a refresh > - The dialog was unconditionally mounted and only closed inside the mutation's `onSuccess`, so the overlay teardown depended on the mutation outcome and could race or never happen > - This pull request closes the dialog before the mutation fires in both confirm flows, clears the pending preview alongside the open flag, and mounts the dialog conditionally so its overlay fully unmounts > - The benefit is that enabling an experiment behaves like every other settings change: the dialog goes away, the page stays interactive, and errors surface in the page-level error banner instead of a dead UI ## Linked Issues or Issue Description No public GitHub issue exists for this bug; description follows the bug report template. Refs #4587 (the PR that introduced the configurable liveness auto-recovery controls this dialog belongs to). What happened? In Settings → Experiments, toggling on "Task graph liveness auto-recovery" and confirming via "Enable only" left the whole UI dimmed and unclickable. The dialog content disappeared, but the modal overlay and the `pointer-events: none` lock on `<body>` remained until a full page refresh. Expected behavior: Confirming (or dismissing) the auto-recovery dialog should close it completely and return the page to a fully interactive state, with the toggle reflecting the new setting. Steps to reproduce: 1. Open Settings → Experiments. 2. Toggle on "Task graph liveness auto-recovery"; the confirmation dialog with the recovery preview appears. 3. Click "Enable only". 4. The dialog content disappears but the page stays dimmed and nothing is clickable; refreshing restores the UI and shows the setting was applied. Paperclip version or commit: master @ `634ae12` · Deployment mode: local instance · Area: UI only (`ui/src/pages/InstanceExperimentalSettings.tsx`). ## What Changed - Added a `closeRecoveryPreview()` helper that resets both `previewDialogOpen` and `pendingPreview` together, and used it everywhere the dialog closes (confirm flows, run-mutation success, and user dismissal). - "Enable only" and "Enable and run" now close the dialog before firing the mutation, so overlay teardown no longer depends on the mutation outcome; mutation errors roll back the optimistic toggle and surface in the existing page-level error banner. - The `RecoveryPreviewDialog` is now conditionally mounted (`previewDialogOpen ? <RecoveryPreviewDialog … /> : null`), guaranteeing the Radix overlay and body pointer-events lock are fully removed when closed. - Added a regression test that walks the real flow — toggle on → preview dialog appears → "Enable only" — and asserts the update payload, that the dialog text and `[data-slot="dialog-overlay"]` element are gone, and that the toggle reads enabled. Credit: the implementation commit was authored by Cody — thanks! This PR packages that fix for upstream review. ## Verification - `pnpm vitest run src/pages/InstanceExperimentalSettings.test.tsx` in `ui/` — 17/17 tests pass, including the new regression test. - Independently re-verified beyond the committed assertions: with a temporary assertion (not committed), confirmed `document.body.style.pointerEvents` is `none` while the dialog is open and released after "Enable only" — the actual "can't interact with anything" symptom, not just overlay DOM removal. - Manual check: open Settings → Experiments, toggle the auto-recovery feature, click "Enable only" — the dialog closes, the page stays interactive, and the toggle shows enabled without a refresh. ## Risks - Low risk: change is confined to one page component's dialog lifecycle; no API, schema, or shared-package changes. - Behavioral shift: the dialog now closes immediately on confirm instead of staying open with a pending spinner until the mutation resolves. Errors are still surfaced via the page-level error banner, and the optimistic toggle rolls back on failure. ## Model Used - Implementation commit authored by the AI coding agent "Cody" (Anthropic Claude-based agent). Review, independent verification, and PR preparation by Claude (Anthropic), model ID `claude-fable-5`, extended thinking with tool use (code execution, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Cody <noreply@paperclip.ing>	2026-07-13 11:52:59 -07:00
Devin Foley	49d1abc458	fix(ui): detail the env unsaved-changes banner and guard against draft loss (#9391 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Instance settings include an Environments section where operators configure execution environments, each with an environment-variables editor for run-time bindings > - The editor showed a bare "Unsaved changes" banner that never said which variables changed, sometimes appeared the moment a saved config was opened (a lossy round-trip through the editor's emit rules made clean values look dirty), and the environment form let you navigate away without any confirmation, silently dropping the draft > - Operators could not tell what was unsaved, distrusted the phantom banner, and lost half-finished environment edits to a stray click — the agent configuration page already confirms before discarding, so environments behaved inconsistently > - This pull request lists the new/edited/removed variable names under the banner, normalizes both sides of the dirty comparison so saved values no longer look dirty on open, and confirms before cancel, in-app navigation, or tab unload while the form has unsaved changes > - The benefit is that the banner is trustworthy and specific, and unsaved environment edits can no longer be lost without an explicit confirmation ## Linked Issues or Issue Description Related (not fixed by this PR): #8930 introduced the current environment-variables editor and its unsaved-changes banner; #9386 moved environment create/edit from a modal to routed pages, which this PR's navigation guard builds on. No existing public issue for the defects themselves; described per the bug report template: What happened? The environment-variables editor in Environments settings showed a bare "Unsaved changes" banner with no indication of which variables changed. For some saved configurations (names with surrounding whitespace, incomplete secret references, duplicate names differing only by whitespace) the banner appeared immediately on opening the edit form, before any user input. Navigating away from the environment form — cancel, an in-app link, or closing the tab — silently discarded the draft with no confirmation. Expected behavior The banner should say which variables are new, edited, or removed; a freshly opened saved configuration should show no banner; and leaving the form with unsaved changes should require an explicit confirmation, consistent with the agent configuration page. Steps to reproduce 1. Open Settings → Instance settings → Environments and edit an environment whose saved config round-trips lossily (e.g. an env var name stored with trailing whitespace) — the "Unsaved changes" banner appears with no user edits. 2. Add or edit a variable — the banner gives no hint of what is unsaved. 3. With a dirty draft, click any in-app link or Cancel — the draft is dropped with no confirmation. Deployment mode Self-hosted (local development instance), reproducible on `master`. ## What Changed - The unsaved-changes banner in `EnvironmentVariablesEditor` now renders a change summary line — `New: … · Edited: … · Removed: …` — showing up to three names per group with a `+N more` overflow and the full list in a `title` tooltip. A rename shows as one addition plus one removal. - Dirty detection normalizes both the committed value and the draft through the same rules the editor uses when emitting values (trimmed names, incomplete secret refs dropped, last-writer-wins on trimmed duplicates), so a saved config that round-trips lossily no longer shows a phantom banner on first open. - The editor exposes an `onDirtyChange` callback and warns via `beforeunload` while its local draft is dirty. - The environment create/edit page (`CompanyEnvironments`) tracks a payload-level baseline fingerprint of the form as initialized and treats the page as having unsaved changes when the current form differs from it or the editor draft is dirty. While dirty it confirms ("Discard unsaved environment changes?") on Cancel, intercepts same-origin in-app link clicks, and warns on tab unload. ## Verification - `node_modules/.bin/vitest run ui/src/pages/CompanyEnvironments.test.tsx ui/src/components/environment-variables-editor/EnvironmentVariablesEditor.test.tsx` — 48 tests pass, including new coverage for: the change-summary banner text, no phantom banner for lossy round-trip values, beforeunload only while dirty, cancel confirmation on the edit page, and unload/link-click warnings after edits are staged into the form. - `tsc -b` in `ui/` passes. - Manual: edit an environment, add/edit/remove variables, observe the summary line; click Cancel or an in-app link and observe the confirmation; save and observe navigation proceeds without prompting. ## Risks - Low risk, UI-only. The click interceptor is scoped to same-origin anchor navigation while the environment form page has unsaved changes and is removed on cleanup; modified-key/middle-button clicks and external links are left alone. - The dirty-normalization intentionally ignores differences the editor could never persist (incomplete secret refs, untrimmed duplicate names); those were previously reported as unsaved changes that could not be saved away. ## Model Used - Claude Fable 5 (`claude-fable-5`, Anthropic) via Claude Code, with extended thinking and tool use (code editing, test execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-10 17:14:28 -07:00
Devin Foley Cody Cody	9cde4e128c	feat: run ACP sessions in sandbox execution targets (#9390 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent adapters (Claude, Codex, Gemini) default to the ACP engine lane, which needs a live bidirectional stdio session with the agent process > - Sandbox execution targets only exposed one-shot command execution, so every ACP-capable adapter refused remote targets and fell back to the CLI lane with a "supports only the local Paperclip host" warning > - Running agents in sandboxes is a core deployment mode, and losing ACP there means losing streaming updates, structured events, and default-lane parity with local runs > - This pull request adds a provider-agnostic process-session bridge that relays the ACP stdio session into the sandbox over the existing sandbox runner contract, and updates the adapters to use it > - The benefit is that the default ACP lane now behaves the same on the local host and in any sandbox provider, with CLI fallback reserved for targets that genuinely cannot host a bidirectional session ## Linked Issues or Issue Description No existing public issue covers this; inline description following the feature request template: Problem or motivation Configuring an ACP-capable adapter (e.g. Claude) with a sandbox environment made every run fall back to the CLI lane with the warning "Claude ACP currently supports only the local Paperclip host, but this run targets a remote environment." The ACP engine only knew how to spawn a local subprocess, while sandbox providers only expose one-shot command execution — so there was no way to hold the bidirectional stdio session ACP requires. Proposed solution Add a process-session bridge in `adapter-utils`: a local ACPX-spawnable proxy script connects to a token-authenticated loopback TCP server, which relays JSON-framed stdin/stdout/stderr events to and from a small relay script executed inside the sandbox via the provider's ordinary runner. Claude/Codex/Gemini adapters now treat sandbox targets with a runner as ACP-capable, resolve agent commands against the remote target, and fall back to CLI only when the sandbox exposes no bidirectional path. The sandbox callback bridge injects a run-scoped API endpoint and bridge token so the agent inside the sandbox can reach Paperclip (including work-product handoffs) without ever receiving the host run JWT. Alternatives considered A provider-specific lane was prototyped first: Daytona minting SSH access metadata at lease time, converted into an SSH execution target. It was dropped because it only worked for providers able to advertise SSH, added per-provider surface area, and left every other sandbox provider on the CLI fallback. The merged design rides the one-shot runner contract all providers already implement; a regression test pins that sandbox targets stay on the bridge lane even when lease metadata advertises SSH access. Roadmap alignment Directly advances the "Cloud / Sandbox agents" roadmap item — agents running in remote and sandboxed environments keep the same control-plane behavior as local ones. No overlap with other planned core work. ## What Changed - `packages/adapter-utils/src/execution-target.ts`: new `startAdapterExecutionTargetProcessSessionBridge()` plus helpers — writes a token-authenticated local proxy script (spawnable by ACPX) and a remote relay script synced into the sandbox, with a loopback TCP server streaming JSON-framed stdio between them; events emitted before the ACP client attaches are buffered so none are lost. - `packages/adapter-utils/src/acpx-engine/execute.ts`: the ACP engine can execute against remote sandbox targets through the bridge instead of requiring a local subprocess, including remote cwd/env shaping. - `packages/adapter-utils/src/sandbox-callback-bridge.ts`: sandbox-scoped API bridging extended to allow work-product handoffs; the sandbox payload env carries a bridge token, never the host run JWT. - `packages/adapters/claude-local`, `codex-local`, `gemini-local` (`src/server/acp.ts`): default-lane selection no longer rejects all remote targets; command resolution is remote-aware (`ensureAdapterExecutionTargetCommandResolvable`, `resolveAdapterExecutionTargetCwd`); the fallback reason is now scoped to sandboxes that expose only one-shot execution. - `server/src/__tests__/environment-execution-target.test.ts`: pins that sandbox targets resolve to the bridge lane, including when lease metadata advertises SSH access. - Non-sandbox remote targets (e.g. SSH) keep the CLI lane: the ACP engine's remote transport is sandbox-only, so default-lane selection falls back for those targets across all three adapters, and tests covering CLI-specific remote behavior pin `engine: "cli"` explicitly. - The bridge authenticates loopback connections before they can own the session or receive buffered output (token required, idle unauthenticated peers dropped), and remote event writes are serialized so the exit event always lands after stdout/stderr have drained. - Daytona plugin: formatting-only residue from the earlier iteration; no functional change. ## Verification - `vitest run` over the touched suites — `packages/adapter-utils/src/acpx-engine/execute.test.ts`, `packages/adapter-utils/src/execution-target-sandbox.test.ts`, `packages/adapter-utils/src/sandbox-callback-bridge.test.ts`, the three adapter `acp.test.ts` files, and `server/src/__tests__/environment-execution-target.test.ts` — 102 tests pass. - End to end: with a Claude agent configured on a Daytona sandbox environment, the primary-model test now selects the default ACP lane (no fallback warning), and the full round trip (wake → sandbox execution → API bridge → comment post) was exercised twice from inside a live sandbox. ## Risks - Behavioral shift: adapters that previously always fell back to CLI on sandbox targets now default to ACP there; `engine=cli` still pins the CLI lane explicitly. - The bridge relays stdio as JSON lines over loopback TCP guarded by a per-session random token; the remote relay runs inside the sandbox under the provider's runner. Providers with slow one-shot execution will see higher session startup latency — the CLI fallback remains for genuinely incapable targets. - No schema or migration changes. ## Model Used - Claude Fable 5 (`claude-fable-5`, Anthropic) — extended thinking enabled, agentic tool use via the Claude Agent SDK harness; implementation iterated with local Vitest verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (no shipped docs describe the old local-only ACP limitation) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Cody <noreply@paperclip.ing> Co-authored-by: Cody <cody@paperclip.local>	2026-07-10 17:13:53 -07:00
Devin Foley Cody Paperclip	0f9b1d399c	fix(ui): make environment edit a routed page (#9386 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Instance settings include an Environments section where operators configure sandbox/SSH/local execution environments, including interactive custom-image setup sessions with a browser terminal > - The environment create/edit form was rendered inside a modal dialog, so pressing Escape anywhere — including inside the embedded SSH terminal while capturing a snapshot — closed the whole modal and destroyed the in-progress session > - Environment editing is a heavyweight, long-lived flow; losing it to a reflexive Escape keypress is destructive and surprising > - This pull request converts environment create/edit from a modal into routed standalone pages, so Escape no longer dismisses the form > - The benefit is that terminal sessions and half-completed edits survive Escape, and the flow gets shareable URLs and normal back/forward navigation ## Linked Issues or Issue Description No existing public issue; described per the bug report template: What happened? While editing an environment's sandbox snapshot in the embedded SSH terminal, pressing Escape (e.g. to exit a mode inside the terminal) closed the entire environment edit modal, discarding the setup session and any unsaved form state. Expected behavior Escape inside the terminal or form should not dismiss the environment editor. A heavyweight flow like environment configuration should be a standalone page where Escape behaves as expected within the focused widget. Steps to reproduce 1. Open Instance settings → Environments and edit a sandbox environment 2. Start a custom image setup session and focus the browser terminal 3. Press Escape 4. The modal closes and the session context is lost ## What Changed - Converted the environment create/edit dialog in `CompanyEnvironments.tsx` into routed pages at `/company/settings/instance/environments/new` and `/company/settings/instance/environments/:environmentId/edit` - Registered the new routes in `App.tsx` and wired breadcrumbs for the list/create/edit states - Form state now initializes from the route (create vs edit) instead of dialog open/close state, and successful saves navigate back to the environments list - Updated `CompanyEnvironments.test.tsx` and `CompanySettings.test.tsx` to render through a router with the new routes and assert against the routed form page instead of a dialog ## Verification - `pnpm vitest run ui/src/pages/CompanyEnvironments.test.tsx ui/src/pages/CompanySettings.test.tsx` — 22/22 passing - `tsc --noEmit` on the `ui` package — clean - Behavioral coverage: the updated tests exercise the routed create/edit pages end to end (open edit via the list, interact with the setup-session controls on the form page, save navigates back to the list); with the form no longer in a dialog there is no Escape-close handler to trigger ## Risks - Low risk; UI-only routing change. Deep links into the old modal state do not exist (the modal had no URL), so no redirects are needed - The edit page resolves the environment from the route param; a stale/unknown id falls back to the environments list ## Model Used - Claude (Anthropic), model ID `claude-fable-5` (Fable 5), extended thinking enabled, agentic tool use via Claude Code ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Cody <cody@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-10 15:36:03 -07:00
Devin FoleyandPaperclip	ac66fd65cb	Fix Cody default model adapter test config (#9365 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent configuration includes adapter-specific model settings and a built-in adapter test action so operators can verify runtime configuration before saving changes. > - Cody/Codex-style local adapters can use an adapter default model when the user clears the explicit model field. > - The adapter test path still passed an object containing `model: undefined` in some create/edit flows, which is different from omitting the model and can break default-model behavior. > - The previous fix was reverted because it also included an unrelated skill documentation edit. > - This pull request reapplies only the UI default-model test-config fix, with no doc or skill changes. > - The benefit is that testing Cody/Codex adapter settings with the default model follows the same contract as saving default model settings: no explicit model key is sent. ## Linked Issues or Issue Description Bug report: - Summary: Testing a Cody/Codex local agent after selecting the default model could send an adapter config with an undefined model value instead of omitting the model key. - Expected behavior: Clearing the model to use the adapter default should test with `adapterConfig: {}` unless another model is explicitly selected. - Actual behavior: The UI test-config path could preserve `model: undefined`, causing the adapter test to fail instead of exercising the default model. - Related PRs: Reapplies the UI-only portion of #9361 after #9363 reverted the original PR. ## What Changed - Exported and reused `omitUndefinedEntries` so adapter test config payloads drop undefined adapter config entries before calling the test endpoint. - Hardened the current model display value so create-mode values that are nullish or non-string do not crash the model selector/test flow. - Added render coverage for editing a Codex agent back to the default model and for testing a create form with the default model. ## Verification - `pnpm exec vitest run ui/src/components/AgentConfigForm.render.test.tsx` - `pnpm check:token-gates` - Confirmed `git diff origin/master --name-only` contains only: - `ui/src/components/AgentConfigForm.render.test.tsx` - `ui/src/components/AgentConfigForm.tsx` - `ui/src/lib/agent-config-patch.ts` ## Risks Low risk. The change only removes `undefined` adapter config entries from the UI adapter-test payload and adds focused render coverage. Explicit model values and other adapter config fields are preserved. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, GPT-5 coding agent, tool-use enabled. Context window size not exposed in this runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-10 14:49:28 -07:00
Devin FoleyandClaude	d1f6a6850a	Fix agent detail URL after agent rename (#9340 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The agent detail page uses route references that can be based on an agent's URL key. > - Renaming an agent can change that URL key while the browser is still on the old route. > - After save or rollback, refetching the stale route reference can render an "Agent not found" state even though the agent still exists. > - This pull request redirects the detail page to the updated canonical route when the saved agent's route reference changes. > - The benefit is that agent renames keep users on the same configuration workflow without landing on a stale URL. ## Linked Issues or Issue Description - Refs #1848 - Related public search performed for agent rename/not-found issues and PRs; no closer in-flight PR was found. - Bug context: after saving a renamed agent or rolling back to a revision with a different name-derived URL key, the agent detail page could continue using the old URL and show "Agent not found". ## What Changed - Added a small route-sync helper that compares the previous and updated agent route refs after mutations. - Redirects the agent detail page with `replace: true` when a save or rollback changes the canonical route ref. - Removes the stale detail-query cache entry so the old route reference is not refetched after a rename. ## Verification - Local outgoing patch scan for common secrets, private paths/emails, and internal issue/link references: no matches. - `corepack pnpm install --frozen-lockfile` - `corepack pnpm --dir ui run typecheck` - `corepack pnpm --dir ui exec vitest run src/pages/AgentDetail.progress.test.ts src/App.test.tsx` - `corepack pnpm check:token-gates` ## Risks - Low risk: the redirect only runs when the updated agent resolves to a different route ref than the current agent. - If a future mutation response omits both URL key and name, the existing route-ref fallback behavior still applies. ## Model Used OpenAI Codex, GPT-5 coding agent via the local Codex adapter, with tool-assisted repository inspection, shell execution, and GitHub API use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude <noreply@paperclip.ing>	2026-07-10 14:45:42 -07:00
Devin Foley	17dde9d3f2	fix(sandbox): keep custom-image snapshots applied to config tests, probes, and saves (#9385 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Sandbox environments can capture reusable custom images (provider snapshots) so agents boot with pre-installed tools and CLI logins > - The custom-image runtime fingerprint check included provider secret-ref paths (e.g. the Daytona `apiKey`), while capture-time fingerprinting excluded them, so any config carrying a credential never matched its captured snapshot > - As a result, agent config tests and environment probes silently booted the provider base image instead of the snapshot, test sandboxes were deleted before operators could inspect them, and any environment save orphaned the snapshot without warning > - The UI compounded the confusion by displaying an internal template id that matches nothing in the provider dashboard > - This pull request aligns runtime fingerprints with capture-time exclusions, re-stamps fingerprints on saves that cannot affect the snapshot (warning when they can), archives test/probe sandboxes instead of deleting them, and surfaces the provider snapshot ref in the UI > - The benefit is that custom images actually apply to config tests and probes, survive unrelated config edits, and are debuggable against the provider dashboard ## Linked Issues or Issue Description No public GitHub issue exists for this; describing it in-PR per the bug template. Related: Refs #9329 (saved-environment probe company context — this branch carries an equivalent fix), Refs #8794 (introduced reusable sandbox custom images). What happened? With a Daytona environment whose provider config stores the API key as a secret reference and an active captured custom-image snapshot: - Agent config tests and environment probes booted the provider base image (`daytonaio/sandbox:0.8.0`) instead of the captured snapshot, so CLI upgrades/logins baked into the snapshot were missing and the probe reported "login required" and an outdated CLI. - The environment card showed an internal template id (e.g. `b5be03e1-ca5…`) that does not correspond to any snapshot name in the provider dashboard, making the active image impossible to correlate. - Test/probe sandboxes were deleted immediately after the run, so the sandbox a test used could not be inspected afterwards. - Saving the environment config (even fields unrelated to the image) changed the stored fingerprint, silently detaching the snapshot with no warning. Expected behavior Config tests and probes boot the captured snapshot when one is active; the UI shows the provider-facing snapshot/template ref; test sandboxes stay inspectable for a short window; unrelated config edits keep the snapshot linked, and edits that genuinely invalidate it produce an explicit warning. Steps to reproduce 1. Configure a sandbox environment on Daytona with the API key stored as a company secret reference. 2. Capture a custom image snapshot from the environment page and mark it active (e.g. after installing/logging into a CLI in the setup sandbox). 3. Run the agent config test or an environment probe: the sandbox boots the base image, not the snapshot, and the sandbox is deleted immediately after the test. 4. Save the environment config with an unrelated field change: the snapshot silently stops applying. Paperclip version or commit `master` at the merge-base of this branch. Deployment mode Self-hosted local instance (macOS, pnpm dev server) with the Daytona sandbox provider plugin. ## What Changed - Runtime custom-image fingerprint checks now exclude provider secret-ref paths, matching capture-time exclusions, so configs carrying credentials match their captured snapshots (`environment-custom-image-runtime.ts`). - Agent config tests and saved-environment probes force fresh, non-reused sandboxes and pass company context so lease-backed probes can resolve company secrets and boot the real snapshot (`environment-probe.ts`, `routes/agents.ts`, `routes/environments.ts`). - Test/probe sandboxes are released by archiving (stop + 60-minute provider-side auto-delete) instead of immediate deletion, so operators can inspect the exact sandbox a test used (Daytona plugin). - On environment PATCH save, changes that cannot affect the captured snapshot re-stamp the template's source fingerprint so the snapshot stays linked; boot-source or provider-identity changes (new manifest field `templateIdentityPaths`) mark the template detached and the save response reports it (`environment-custom-images.ts`, shared plugin types/validators). - The custom-image overview exposes `activeTemplateMatchesConfig`; the environments UI shows the provider snapshot/template ref (internal id moved to a tooltip), warns via toast when a save detaches the snapshot, and shows a persistent "Not in use" warning when the active template no longer matches the saved config (`CompanyEnvironments.tsx`, `api/environments.ts`). ## Verification - `pnpm vitest run server/src/__tests__/environment-custom-images-service.test.ts server/src/__tests__/environment-probe.test.ts server/src/__tests__/environment-routes.test.ts server/src/__tests__/agent-test-environment-routes.test.ts` — server coverage for fingerprint exclusions, re-stamp/detach on save, probe company context, and fresh-sandbox test behavior. - `pnpm vitest run packages/plugins/sandbox-providers/daytona/src/plugin.test.ts` — archive-on-release and snapshot ref handling. - `pnpm vitest run ui/src/pages/CompanyEnvironments.test.tsx` — snapshot ref display, detach toast, and "Not in use" warning. - Manually verified end-to-end on a live self-hosted instance against real Daytona: config test boots the captured snapshot (CLI login and version persist), the test sandbox remains visible in the provider dashboard as archived, and saving unrelated fields keeps the snapshot applied. ## Risks - Fingerprint exclusion widening: a provider credential rotation alone no longer detaches a captured snapshot; that is the intended behavior (the snapshot content does not depend on the credential), and provider-identity fields (e.g. Daytona `apiUrl`) still detach via `templateIdentityPaths`. - Archived test sandboxes consume provider-side resources for up to their auto-delete window instead of being freed immediately; bounded (60 minutes) and only for test/probe sandboxes. - New optional manifest field `templateIdentityPaths` is backward-compatible; providers that omit it keep current matching behavior. ## Model Used - Claude Fable 5 (`claude-fable-5`), extended thinking, agentic tool use via Claude Code / Claude Agent SDK. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-10 14:32:47 -07:00
Devin Foley	e84731af70	Revert "Fix default model adapter test config" (#9363 ) Reverts paperclipai/paperclip#9361	2026-07-10 08:55:23 -07:00
Devin FoleyandCody	ebd62ca5ae	Fix default model adapter test config (#9361 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The board UI lets operators create and edit agent adapter configuration, including a primary model field and an adapter test action. > - For Cody and similar adapter forms, selecting the default model means the model value is intentionally unset so the adapter can use its default. > - The adapter test path still allowed `model: undefined` to survive in the generated adapter config, which could send an invalid test payload instead of omitting the field. > - This pull request normalizes create/edit adapter test config so default-model selections omit `model` entirely. > - The benefit is that testing an agent configured to use the adapter default model exercises the same clean config shape that should be saved and run. ## Linked Issues or Issue Description No public GitHub issue was found for this local UI bug, so the problem is described inline. Bug description: - What happened: using the adapter test action after choosing the default model could include `model: undefined` in adapter config and surface a UI/runtime error instead of testing with the adapter default. - Expected behavior: choosing the default model should omit the `model` field from adapter config so the adapter default is used. - Steps to reproduce: edit a Codex/Cody-style agent with a concrete model, switch the model selector to Default, then run the adapter Test action. - Paperclip version/commit: current `master` before this PR. - Deployment mode: board UI, deployment-mode independent. ## What Changed - Sanitized adapter test config assembly so undefined adapter config entries are omitted before the test request is sent. - Made create-mode current model display resilient when the model is unset for adapter defaults. - Added regression coverage for editing an existing agent from a concrete model back to Default and testing it. - Added regression coverage for create-mode testing with an unset/default model. - Hardened the developer skill wording used by the existing server skill utility contract test. ## Verification - `pnpm --filter @paperclipai/ui exec vitest run src/components/AgentConfigForm.render.test.tsx` - `pnpm exec vitest run server/src/__tests__/paperclip-skill-utils.test.ts` - GitHub PR checks on this branch are green, including Typecheck + Release Registry, Build, General tests, e2e, verify, security scans, and Greptile Review. ## Risks - Low risk: this only removes undefined values from adapter test config payloads, which aligns with the existing persisted patch behavior. - Low risk: default-model display now treats unset create-mode model values as an empty string. - No database, API schema, migration, auth, or adapter runtime contract changes. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected - check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, GPT-5 coding agent, tool-enabled software-engineering session. Exact context window size was not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Cody <cody@paperclip.ing>	2026-07-10 08:54:20 -07:00
Devin Foley	a4993a72a6	Fix live run streaming text readability (#9330 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The issue thread UI renders live agent output from adapter run logs and transcript parsing. > - Some adapter streams emit many small or repeated token chunks, and live UI updates can expose partial words, duplicated slices, or transient markdown placeholders. > - That makes active run updates look like gibberish even when the underlying agent output is valid. > - The fix needs to preserve raw logs while making the live thread view stable, readable, and ordered. > - This pull request adds monotonic run-log sequencing, safer live transcript dedupe/order handling, markdown placeholder hiding, and readable live text stabilization. > - The benefit is a live issue thread that updates smoothly without showing confusing partial parser artifacts. ## Linked Issues or Issue Description No public GitHub issue exists yet, so this PR includes the bug details inline. ### What happened? Live run updates in the issue thread can show confusing repeated or partial text while an adapter is streaming. The visible text appears to lose parsing boundaries during active updates, especially with ACP-style token deltas, so the live output can briefly render duplicated chunks, incomplete words, or HTML-comment placeholders. ### Expected behavior Live text should remain readable while preserving the underlying run output for raw inspection. ### Steps to reproduce 1. Start a live agent run whose adapter emits small stdout token deltas. 2. Watch the issue thread while the run is still active. 3. Observe transient duplicated chunks, incomplete words, or markdown placeholder artifacts in the live rendered text. ### Paperclip version or commit Reproduced against current `master` before this PR branch. ### Deployment mode Local dev issue-thread UI with live local adapter runs. ### Additional context GitHub PR search for `live run streaming text markdown transcript` found one broad merged PR, `#252` (“Dotta updates - sorry it's so large”), but no targeted duplicate for this live streaming readability bug. ## What Changed - Added per-run monotonic sequence numbers to persisted and live run-log chunks. - Dedupe and order live transcript chunks by sequence before falling back to timestamp ordering. - Hide markdown HTML comment placeholder text from rendered markdown output. - Smooth live issue-thread text updates so partial additions reveal at readable word boundaries and sliding-window removals do not produce gibberish. - Added coverage for run-log ordering/deduping, markdown comment hiding, live issue-thread stabilization, and Greptile-reviewed edge cases where overlap rewrites could synthesize text or no-boundary additions could stay hidden. ## Verification - `pnpm --filter @paperclipai/ui exec vitest run src/lib/issue-chat-messages.test.ts src/components/MarkdownBody.test.tsx src/components/transcript/useLiveRunTranscripts.test.tsx` passed before the review fix: 3 files, 86 tests. - `pnpm --filter @paperclipai/ui exec vitest run src/lib/issue-chat-messages.test.ts` passed after the review fix: 1 file, 30 tests. - `pnpm --filter @paperclipai/ui exec vitest run src/lib/issue-chat-messages.test.ts src/components/transcript/useLiveRunTranscripts.test.tsx` passed after the final Greptile overlap fix: 2 files, 40 tests. - `pnpm check:token-gates` passed. - Local PII/secret scan of touched files found only expected code/test words such as `secret`, `token`, and redaction-related strings; no literal credentials found. - `pnpm -r typecheck` passed after restoring declared dependencies with `CI=1 pnpm install --frozen-lockfile` and running with a short `TMPDIR` because `tsx` IPC sockets fail under the long sandbox temp path. - `pnpm build` passed with existing Vite CSS/font/chunk warnings. - GitHub PR checks passed on head `4c052dfe86aecb5feb73504e6b48843f68fce813`: build, typecheck/release registry, server and workspace test shards, serialized server suites, e2e, canary dry run, policy, review, Socket, Superagent, Snyk, and verify. - Greptile review passed on head `4c052dfe86aecb5feb73504e6b48843f68fce813` with confidence score 5/5 and no blocking issues found. - `pnpm test:run` failed in unrelated server workspace tests on this macOS local environment: - `server/src/__tests__/heartbeat-workspace-branch-containment.test.ts`: two assertions compare `/tmp/...` with `/private/tmp/...`. - `server/src/__tests__/heartbeat-worktree-suppression.test.ts`: expected one heartbeat run but observed two, followed by cleanup fallout in the full run. - Isolated rerun of those two server suites reproduced the same three failures. ## Risks - Low product risk for the UI changes: the readable smoothing only affects active live-run display stabilization, not stored comments or raw run logs. - Moderate verification risk: local full Vitest did not pass because of unrelated server workspace tests. Targeted tests for this change, typecheck, token gates, and build passed. - Run-log sequence fields are optional for compatibility with older log rows that do not include `seq`. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5-based coding agent, tool-using local workspace execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I searched the GitHub PR list for similar PRs and confirmed this is not a duplicate - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-10 08:11:52 -07:00
Devin Foley	05973b2073	Enable sandbox environments for Grok local adapter (#9338 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Local CLI adapters can run against Paperclip-managed execution environments instead of only the host filesystem. > - The environment picker and environment capability API derive sandbox support from shared adapter capability lists. > - The Grok Build adapter is implemented as a local CLI adapter, but it was missing from those shared environment capability lists. > - That made Grok agents look local-only even when sandbox environments were configured. > - This pull request registers `grok_local` in the shared adapter constants and remote-managed environment support path. > - The benefit is that Grok Build agents can select the same local, SSH, and sandbox environment overrides as other local CLI adapters. ## Linked Issues or Issue Description No public GitHub issue exists for this bug. Inline bug report follows. ### Pre-submission checklist - [x] I have searched existing open and closed issues and this is not a duplicate. - [x] I am on `master`. - [x] I have confirmed the error originates in Paperclip itself, not in the Grok adapter provider or local configuration. ### What happened? When configuring a Grok Build local agent in the board UI, Paperclip did not expose configured sandbox environments as selectable environment overrides. The shared environment capability helper treated `grok_local` as local-only because it was missing from the remote-managed local adapter allowlist. ### Expected behavior Grok Build should behave like other local CLI adapters: when environments are enabled and a runnable sandbox environment exists, the agent configuration form should show the environment override selector and allow the sandbox to be selected. ### Steps to reproduce 1. Enable environments in instance experimental settings. 2. Configure at least one runnable sandbox environment. 3. Open the agent configuration form for a Grok Build local agent. 4. Observe that the sandbox environment is not offered as an override before this fix. ### Paperclip version or commit `master` before this PR. ### Deployment mode Local dev (`pnpm dev`). ### Installation method Built from source (`pnpm dev` / `pnpm build`). ### Agent adapter(s) involved - Grok Build local adapter. - Core bug in shared environment capability logic. ### Database mode Not database-related. ### Access context Board human operator. ### Relevant logs or output No runtime error is emitted; the issue is a missing UI option caused by shared capability metadata. ### Relevant config No secret-bearing config required. Reproduction only needs environments enabled and a runnable sandbox environment configured. ### Privacy checklist - [x] I have reviewed all pasted output for PII, usernames, file paths, API keys, tokens, company names, and redacted where necessary. ## What Changed - Added `grok_local` to the shared built-in adapter type list. - Added `grok_local` to the remote-managed adapter set used by environment capability helpers. - Added shared regression coverage for Grok local sandbox provider and driver support. - Added a UI render regression test that confirms Grok Build agents show the environment override when a runnable sandbox exists. ## Verification - `git diff --check` - Changed-file secret scan with `rg` for common token/key patterns. - `pnpm --filter @paperclipai/shared exec vitest run src/environment-support.test.ts` - `pnpm --dir ui exec vitest run src/components/AgentConfigForm.render.test.tsx` ## Risks - Low risk. This expands environment support for an existing local adapter to match the local CLI adapter behavior already used by Claude, Codex, Gemini, OpenCode, Cursor, and Pi. - Operators still need at least one configured runnable sandbox environment before a Grok agent has a sandbox option to select. - This PR was created from a Paperclip execution workspace branch whose name is runtime-provided; the PR body intentionally avoids internal issue identifiers or instance-local links. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5 coding agent, tool-enabled terminal/code execution environment. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-09 20:46:46 -07:00
Devin FoleyandPaperclip	bc85b456a1	fix(ui): keep agent names visible on mobile agents index (#9236 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Agents index (`ui/src/pages/Agents.tsx`) is the main roster view, with both a list layout and an org-tree layout > - On mobile viewports the roster could render rows with no visible agent names because shrink-resistant trailing controls competed with the fixed-width title cell > - A roster where names are unreadable is unusable on phones, and the org-tree view had a related narrow-width overflow risk for deeply indented names > - This pull request lets the list title flex below `xl`, hides nonessential row controls on mobile, keeps Join/Leave reachable, and truncates org-tree names safely > - The result is a readable agents index on mobile while preserving desktop meta-column alignment and truncation ## Linked Issues or Issue Description No existing public GitHub issue; describing the bug per the bug report template: What happened? On a mobile-width viewport, the agents index showed rows where agent names could become unreadable or disappear because trailing controls consumed the available row width. In the org view, deeply indented names could overflow the row. Expected behavior Agent names remain visible on every viewport, Join/Leave stays reachable, and desktop rows continue to align and truncate as before. Steps to reproduce Open Paperclip in a browser at a narrow viewport, navigate to the Agents index, and observe rows with long names or left-membership controls. Paperclip version or commit Reproducible on `master` prior to this fix. Deployment mode Local dev instance. The bug is viewport-width dependent, not deployment-mode dependent. ## What Changed - `EntityRow` now lets callers control title text, subtitle text, and the meta spacer classes while keeping the default truncation behavior unchanged. - Agents list rows now use `flex-1 xl:flex-none xl:w-56`, so names get mobile width while desktop meta columns keep their aligned fixed title column. - Long list-view names/subtitles wrap below `xl` but return to truncation at `xl` and above. - Join/Leave stays visible on mobile; run/status/star row controls stay hidden on mobile to avoid squeezing names. - Org-tree rows use `min-w-0 truncate`, so deep indentation shortens names with an ellipsis instead of overflowing. - Regression tests cover mobile name visibility, left-membership dimming, responsive title/meta behavior, and mobile Join/Leave reachability. ## Verification - `pnpm --filter @paperclipai/ui exec vitest run src/components/EntityRow.test.tsx src/pages/Agents.test.tsx` — 2 files, 18 tests passed. - `pnpm check:token-gates` — all gates clean. - `git diff --cached \| rg -n "(AKIA\|ASIA\|SECRET\|TOKEN\|PASSWORD\|PRIVATE KEY\|BEGIN RSA\|BEGIN OPENSSH\|api[_-]?key\|bearer\|paperclip_api_key\|DATABASE_URL\|postgres://\|sk-[A-Za-z0-9]\|xox[baprs]-)"` — no matches before push. ## Risks - Low risk: UI-only change scoped to the agents index. The main behavior change is that nonessential row controls remain hidden on mobile while Join/Leave remains available there and all controls remain available on larger screens/detail pages. ## Model Used - OpenAI GPT-5 via Codex (`codex_local` adapter), agentic code editing and local/GitHub verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-09 13:37:17 -07:00
Devin FoleyandPaperclip	eedc7ddef2	Make ACP the default engine for local adapters (#9238 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Adapter packages are the bridge between the control plane and local agent harnesses such as Claude Code, Codex, and Gemini CLI. > - ACP support was concentrated in a separate `acpx_local` adapter, which made ACP feel like a separate agent choice instead of an execution capability of the harness adapters. > - Claude, Codex, and Gemini now have ACP-capable harnesses, so the native adapter should own ACP selection, fallback, config, transcript parsing, and environment diagnostics. > - The standalone ACPX adapter still needs a compatibility path for existing rows, but it should not be offered as an active adapter for new agents. > - This pull request moves the shared ACP runtime into `@paperclipai/acpx-engine`, wires Claude/Codex/Gemini local adapters to prefer ACP when prerequisites are available, and retires `acpx_local` to a tombstone. > - The benefit is one adapter per harness, richer ACP transcripts by default where possible, and a migration path for existing Claude/Codex ACPX agents. ## Linked Issues or Issue Description Closes #5932 — the broken default `acpx_local` Claude path is replaced by native `claude_local` ACP support, existing Claude/Codex ACPX rows migrate to native adapters, and new agents no longer choose the standalone ACPX adapter. Refs #4893 — original merged ACPX local adapter runtime that this PR replaces with native per-harness ACP engines. Refs #6590 — prior ACPX-Claude seamlessness work folded into the new native Claude ACP path. Refs #197 — related open generic ACP/Kiro adapter work; this PR does not close it because Kiro/custom generic ACP remains a separate adapter decision. Refs #7018 — related Kimi-specific `acpx_local` shell failure; this PR retires the built-in standalone adapter but does not add a native Kimi adapter. Refs #8864 — related ACPX prompt/API guidance PR; this PR moves runtime guidance into the shared/native ACP engine path instead of the old standalone adapter. Refs #8881 — related `acpx_local` POSIX shell failure from the old `acpx` pin; this PR updates ACP dependencies but does not claim custom/OMP ACP support as a first-class native adapter. Refs #8964 — related open `acpx_local` stderr cleanup PR; this PR makes the old runtime path obsolete for new agents but keeps it as a non-closing reference. Problem description: - The standalone `acpx_local` adapter duplicates Claude/Codex agent choices that already have first-class local adapters. - ACP should be an execution engine capability of each harness adapter when the underlying harness supports ACP. - Existing `acpx_local` agents should either migrate to native harness adapters or fail with an explicit retirement message instead of silently falling back to the process adapter. ## What Changed - Added `@paperclipai/acpx-engine` as the shared ACP execution, session-codec, CLI formatter, and UI parser package. - Wired `claude_local`, `codex_local`, and `gemini_local` to auto-select ACP by default when prerequisites pass, with `engine=cli` opt-out and `engine=acp` strict mode. - Added ACP config schema/UI fields, environment checks, session-codec preservation, transcript parsing, and adapter capability metadata for the native adapters. - Retired `acpx_local` to a server tombstone, removed its UI/package/runtime image surface, and added a migration for existing Claude/Codex ACPX agents. - Updated package manifests, lockfile, release tooling, docs, Kubernetes sandbox defaults, and tests. ## Verification - `corepack pnpm --filter @paperclipai/acpx-engine typecheck` - `corepack pnpm --filter @paperclipai/adapter-claude-local typecheck` - `corepack pnpm --filter @paperclipai/adapter-codex-local typecheck` - `corepack pnpm --filter @paperclipai/adapter-gemini-local typecheck` - `corepack pnpm --filter @paperclipai/acpx-engine exec vitest run` - `corepack pnpm --filter @paperclipai/adapter-claude-local exec vitest run src/server/acp.test.ts src/server/execute.acp-fallback.test.ts src/ui/build-config.test.ts` - `corepack pnpm --filter @paperclipai/adapter-codex-local exec vitest run src/server/acp.test.ts src/ui/build-config.test.ts` - `corepack pnpm --filter @paperclipai/adapter-gemini-local exec vitest run src/server/acp.test.ts src/ui/build-config.test.ts src/ui/parse-stdout.test.ts` - `corepack pnpm --filter @paperclipai/plugin-sdk ensure-build-deps && corepack pnpm --filter @paperclipai/server exec tsc --noEmit` - `corepack pnpm --filter @paperclipai/server exec vitest run src/__tests__/adapter-routes.test.ts src/__tests__/adapter-session-codecs.test.ts src/__tests__/adapter-models.test.ts` - `corepack pnpm --filter @paperclipai/ui typecheck` - `corepack pnpm --filter @paperclipai/ui exec vitest run src/adapters/metadata.test.ts src/adapters/adapter-display-registry.test.ts src/components/AgentConfigForm.test.ts src/components/AgentConfigForm.render.test.tsx src/components/transcript/RunTranscriptView.test.tsx` - `node --test scripts/bootstrap-npm-package.test.mjs scripts/release-package-map.test.mjs scripts/verify-release-registry-state.test.mjs` Note: the server typecheck script calls `pnpm` internally; this dev shell exposes pnpm through Corepack only, so I ran the two script steps manually with `corepack pnpm`. ## Risks - Migration changes existing `acpx_local` Claude/Codex agents to native adapter types and clears old ACPX task sessions/runtime state. - Custom ACP commands remain on the retired tombstone and will need a separate future adapter/plugin path. - ACP auto-selection depends on local Node and ACP server command prerequisites; remote and unsupported environments fall back to CLI unless `engine=acp` is explicit. - `@paperclipai/acpx-engine` is a new public package and needs npm trusted-publishing bootstrap before release automation can publish it. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 via Codex coding agent. Exact hosted model build and context-window size are not exposed in this runtime. Tool use included shell execution, repository editing, GitHub CLI operations, and local test/typecheck execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-08 19:05:03 -07:00
Devin FoleyandPaperclip	cc17f29e7e	fix(timeouts): raise sandbox wall-clock backstop to 4h and make acpx_local timeouts self-describing (#9232 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent runs execute through adapters (e.g. `acpx_local`), which can run locally, over SSH, or inside sandbox execution targets, each with a wall-clock execution timeout > - Sandbox-backed runs defaulted to a 30-minute wall-clock backstop (`DEFAULT_REMOTE_SANDBOX_ADAPTER_TIMEOUT_SEC = 1800`), which kills healthy long agent runs that are still making progress — long before the recovery watchdog's 4h critical threshold would even consider them stuck > - On top of that, `acpx_local` resolved its timeout directly from `adapterConfig.timeoutSec` instead of the shared execution-target resolver, and its timeout failures surfaced as a bare `Timed out after Ns` — giving operators no clue which timer fired or which knob raises it > - This pull request raises the sandbox backstop to 4h (aligned with the recovery watchdog), routes `acpx_local` through the shared timeout resolver, logs the effective timeout and its source at run start, and makes every timeout error message self-describing > - The benefit is that long-running sandbox agent runs no longer die at 30 minutes, and when a wall-clock timeout does fire, the run log states exactly which timer fired and how to configure it ## Linked Issues or Issue Description Refs #4535 (related: wall-clock execution timeouts killing agent runs that are still making progress — that issue covers a different hardcoded 600s timer, but the operator pain is the same). No exact public issue exists for this one, so describing it in-PR: Bug: A long sandbox-backed `acpx_local` agent run was killed with a bare `Timed out after 1800s` even though the agent was actively working. - What happened: The run hit the 30-minute `DEFAULT_REMOTE_SANDBOX_ADAPTER_TIMEOUT_SEC` backstop. `acpx_local` never consulted the shared execution-target timeout resolution (it read `adapterConfig.timeoutSec` directly, default 0), so on sandbox targets the sandbox-provider default applied with no adapter-level say. The resulting error named neither the timer that fired nor the knob that controls it. - Expected: Healthy long runs should not be killed by a 30-minute wall-clock backstop when the recovery watchdog only treats runs as critically stuck after 4h of output silence; and any timeout error should say which timeout fired and how to raise it. - Impact: Long, legitimate agent runs in sandboxes fail mid-work; operators waste time reverse-engineering which of several timers produced "Timed out after Ns". ## What Changed - `packages/adapter-utils/src/execution-target.ts` - `DEFAULT_REMOTE_SANDBOX_ADAPTER_TIMEOUT_SEC` raised from `1_800` to `14_400` (4h), with a comment explaining it intentionally matches the recovery watchdog's `ACTIVE_RUN_OUTPUT_CRITICAL_THRESHOLD_MS` (4h) so the adapter backstop never fires before the watchdog path. Output-inactivity monitors remain the primary hang detectors. - New `resolveAdapterExecutionTargetTimeout(target, configuredTimeoutSec)` returns `{ timeoutSec, source }` where `source` is `configured` / `sandbox_default` / `unlimited`. The existing `resolveAdapterExecutionTargetTimeoutSec` is preserved as a thin wrapper, so current callers are unaffected. - New `formatAdapterExecutionTimeoutErrorMessage(resolution)` and `formatAdapterExecutionTimeoutStartLogLine(resolution)` produce self-describing messages that name the timer that fired and the `adapterConfig.timeoutSec` knob that controls it. - `packages/adapters/acpx-local/src/server/execute.ts` - `buildRuntime` now resolves the wall-clock timeout through the shared resolver: sandbox targets default to the 4h backstop, local/SSH keep the historical "0 = no adapter timeout", and a configured `adapterConfig.timeoutSec` always wins. - The executor logs the effective timeout and its source at run start (`[paperclip] Adapter execution timeout: …`), so a later timeout is diagnosable from the run log alone. - All three bare timeout messages (timer cancel reason, turn result `errorMessage`, catch-path `messageOverride`) now use the self-describing format. - `packages/adapters/acpx-local/src/index.ts` — the adapter configuration doc for `timeoutSec` states the sandbox default and that the output-inactivity monitor remains the primary hang detector. - Tests: `packages/adapter-utils/src/execution-target-sandbox.test.ts` and `packages/adapters/acpx-local/src/server/execute.test.ts` (see Verification). ## Verification - `pnpm --filter @paperclipai/adapter-utils typecheck` — passes - `pnpm --filter @paperclipai/adapter-acpx-local typecheck` — passes - `npx vitest run packages/adapter-utils/src/execution-target-sandbox.test.ts packages/adapters/acpx-local/src/server/execute.test.ts` — 2 files, 40 tests, all pass - New/updated test coverage: - sandbox default resolves to 4h (and the constant is asserted to be `4 * 60 * 60`) - `resolveAdapterExecutionTargetTimeout` reports `configured` / `sandbox_default` / `unlimited` sources with the correct precedence (configured > sandbox default; local/SSH stay unlimited) - exact wording of the self-describing error message and the start-of-run log line - `acpx_local` runtime picks up the sandbox default into `timeoutMs`, keeps the unlimited local default, honors configured-over-default precedence, emits the start-of-run log line, and surfaces the self-describing `errorMessage`/cancel reason when the wall-clock timer kills a turn ## Risks - Behavioral shift: sandbox-backed adapter runs that previously hit the 30-minute backstop now run up to 4h before the adapter kills them. Genuinely hung runs are still caught much earlier by the adapters' output-inactivity monitors and by the recovery watchdog; the wall-clock timer is a last-resort kill switch. Operators who relied on the 30-minute default can restore it explicitly via `adapterConfig.timeoutSec`. - Error-message consumers: any tooling that pattern-matched the exact `Timed out after Ns` string from `acpx_local` will see the new self-describing message instead. - No API or schema changes; `resolveAdapterExecutionTargetTimeoutSec` keeps its exact signature and behavior (modulo the raised sandbox default). > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude (Anthropic) via Claude Code CLI — model ID `claude-fable-5`, extended thinking enabled, agentic tool use (file edits, shell, test execution) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-08 13:50:00 -07:00
Devin FoleyandClaude	e68ee09809	perf(release): batch npm registry version queries (#9202 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The release subsystem publishes the public workspace packages and also powers release-related CI validation. > - The release flow currently asks npm for package versions one package at a time in multiple places. > - That serial registry latency slows the PR Canary Dry Run path and real release invocations even though the checks are independent. > - This pull request batches npm registry version lookups with bounded concurrency and reuses the result for version calculation. > - The benefit is shorter non-build release-script time while preserving the fresh target-version existence check before publishing. ## Linked Issues or Issue Description - No public GitHub issue exists for this release-script performance cleanup. ### Problem or motivation Release validation spends avoidable time on repeated serial `npm view` calls across the public package set. The slow path affects PR release validation and real release invocations because version discovery waits on independent registry reads one at a time. ### Proposed solution Fetch package version maps concurrently with bounded parallelism, reuse that map for stable/canary version calculation, and keep a fresh parallel absence check for the target publish version. ### Alternatives considered Keeping the existing serial shell loop is simpler, but it preserves the CI latency cost. Caching the final target-version existence check was rejected because release publish safety should still query npm freshly before publishing. ### Roadmap alignment This is a small release-tooling performance improvement. It does not duplicate any planned core product work found in `ROADMAP.md`. ## What Changed - Added `scripts/release-registry-versions.mjs` to fetch npm package version maps and assert target-version absence with bounded parallelism. - Updated `scripts/release.sh` to prefetch package versions once and to batch the final target-version absence check. - Updated `next_stable_version` and `next_canary_version` to use the prefetched version map when present, with the existing per-package npm fallback preserved. - Added release-registry helper coverage and included it in `pnpm run test:release-registry`. - Hardened the release publish helper tests so their fake `pnpm`/`npm` fixture PATH is preserved under non-login shell execution. ## Verification - `node --test scripts/release-registry-versions.test.mjs` - `pnpm run test:release-registry` - `bash -n scripts/release.sh scripts/release-lib.sh` - `git diff --check` - Safety scan before push: searched changed files for common key/token/password patterns and PII markers; only benign script-name text matched (`secrets:migrate-inline-env`). - Remote PR checks on the latest head passed, including `Typecheck + Release Registry`, `Canary Dry Run`, build, tests, e2e, policy, security scans, and commitperclip review. - Greptile reviewed the latest head with Confidence Score 5/5 and no blocking issues. ## Risks - Low risk. The release version helpers keep their original npm fallback when no prefetched version map is supplied. - The existence check remains fresh and uncached before publish, but now reports all matching package/version pairs from a parallel check. - If npm has transient failures during the prefetch step, missing or failed packages still map to an empty version list, matching the old helper behavior. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex coding agent using GPT-5, with shell/tool execution in the local repository. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude <noreply@paperclip.ing>	2026-07-08 08:04:39 -07:00
Devin FoleyandPaperclip	bdffd26ad3	fix(ui): pass company context to custom image setup (#9028 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Environment settings let board users configure sandbox execution targets for agent runs > - Sandbox custom-image setup is company-scoped because the backend resolves provider secrets and access through company context > - The browser UI already knows the selected company, but the custom-image setup calls were not passing it to routes whose OpenAPI contract includes `companyId` > - In multi-company authenticated deployments, the backend cannot safely infer company context and returns a `companyId query parameter is required` error > - This pull request passes the selected company id through the custom-image overview, setup, rollback, and disable UI paths > - The benefit is that custom-image setup works consistently in multi-company instances while preserving backend company-boundary checks ## Linked Issues or Issue Description - No public issue is filed for this regression. - Related prior work: #8911 - Bug summary: after the browser SSH terminal custom-image setup flow shipped, opening custom-image setup in an authenticated multi-company instance could show `companyId query parameter is required for environment customImage setup` instead of starting the setup session. - Expected behavior: the environment settings UI should send the selected company context to company-scoped custom-image endpoints. - Reproduction shape: run Paperclip in authenticated/private mode with more than one company, open a sandbox environment's edit dialog, and use the custom-image setup controls. ## What Changed - Added company-id query construction for custom-image overview, setup, rollback, and disable calls in the UI environment API wrapper. - Passed the selected company id into the custom-image panel used by the environment edit dialog. - Updated the environment page tests so the regression fails if company context is dropped again. ## Verification - `pnpm exec vitest run ui/src/pages/CompanyEnvironments.test.tsx` - `pnpm --filter @paperclipai/ui typecheck` - `git diff --check` - Also applied the same patch to a local dev checkout serving port 3100 and confirmed `/api/health` still returns `ok` after the dev watcher reload. ## Risks - Low risk: this only adds the selected company id to UI calls for endpoints that already declare or require company context. - If the selected company id is stale or invalid, the existing server-side company access checks still reject the request. ## Model Used - OpenAI Codex CLI using GPT-5, with tool-enabled repository inspection, editing, shell command execution, and local test execution. Context window size is not exposed in this environment. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-07 12:02:27 -07:00
Devin FoleyandPaperclip	a328ec953a	Fix inherited workspace reuse fallback (#8963 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent heartbeats provision execution workspaces before invoking local or sandboxed adapters. > - Some follow-up issues intentionally request `reuse_existing` so they continue in an inherited execution workspace. > - The heartbeat provisioning path treated missing or archived workspace rows as if no explicit reuse request existed. > - That could silently realize and persist a fresh project/default workspace over an explicit inherited-workspace binding. > - This pull request keys explicit reuse off the issue preference and workspace id, then either restores that workspace or fails with a structured workspace validation error. > - The benefit is that intentional workspace inheritance remains auditable and does not silently degrade into unrelated fallback workspaces. ## Linked Issues or Issue Description Refs #8058 Refs #6036 Refs #2203 This fixes a narrower heartbeat provisioning bug around explicit `reuse_existing` issue runs: if the target inherited execution workspace is missing, archived, or fails restore, provisioning now reports the reuse failure instead of replacing the issue's workspace binding with a freshly realized fallback. ## What Changed - Added explicit helpers for resolving workspace reuse requests and deciding whether reuse should restore, refresh metadata, or keep prior replacement-class drift visible. - Changed heartbeat workspace provisioning so explicit `reuse_existing` requests go through restore-or-fail behavior instead of falling back to `realizeExecutionWorkspace` when the stored workspace row is unavailable. - Added structured `workspace_validation_failed` details for inherited workspace reuse failures. - Added regression coverage for replacement-class drift, restore errors, missing rows, archived rows, and restore misses. ## Verification - `pnpm install --frozen-lockfile` - `pnpm --filter @paperclipai/plugin-sdk ensure-build-deps` - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/heartbeat-workspace-session.test.ts` - `pnpm --filter @paperclipai/server typecheck` - `git diff --check origin/master...HEAD` - Scanned the branch diff and commit messages for credentials, tokens, private URLs, PII-style values, and internal issue links before pushing; no unsafe hits remained. ## Risks - Explicit reuse requests whose stored workspace cannot be restored now fail the run instead of opportunistically creating a replacement workspace. That is intentional, but it may surface stale or archived workspace rows as visible provisioning failures that require repair. - Non-reuse workspace provisioning still uses the existing realization path, so the behavior shift is scoped to issues that explicitly request existing workspace reuse. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 via Codex local agent, with shell/tool use enabled for repository inspection, code editing, verification, git, and GitHub CLI operations. Runtime context-window details were not exposed by the adapter. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-04 06:35:06 -07:00
Devin FoleyandPaperclip	85e36aaefb	Show heartbeat progress in run logs (#8965 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The agent detail page is where operators inspect heartbeat runs and watch live execution output. > - Heartbeat runs can publish progress events while longer operations are happening. > - The live log viewer already appended streamed log and structured run events, but it ignored progress events for the same run. > - That made useful progress text invisible in the run log until another event type arrived or the operator inspected other surfaces. > - This pull request renders run progress events as system log lines in the live agent run viewer. > - The benefit is clearer live feedback during long-running heartbeat operations without changing the backend event contract. ## Linked Issues or Issue Description No public GitHub issue exists, so this PR describes the issue inline following the bug report template. ### What happened The agent detail run log subscribed to company live events and handled `heartbeat.run.log` plus structured `heartbeat.run.event` payloads, but it ignored `heartbeat.run.progress` events for the active run. ### Expected behavior When a heartbeat run emits a progress message, the active run log should show that message immediately as operator-visible system output. ### Steps to reproduce 1. Open an agent detail page for a live heartbeat run. 2. Trigger a run operation that emits `heartbeat.run.progress` events with a `message` and optional `phase`. 3. Watch the live log viewer. Before this change, the progress event was ignored by the log viewer. After this change, it appears as a system log line, prefixed by `[phase]` when a phase is present. ### Paperclip version / deployment mode Current `master`; local development and normal board UI deployments. ### Related work search Searched public GitHub issues and PRs in `paperclipai/paperclip` for `heartbeat.run.progress AgentDetail` and `run progress log viewer`; no duplicate issue or PR was found. ## What Changed - Added live handling for `heartbeat.run.progress` events in `AgentDetail`'s run `LogViewer`. - Render progress messages as `system` log lines for the matching run. - Include the optional progress phase in the displayed line as `[phase] message`. - Prefer the event's `updatedAt` timestamp when provided, falling back to the live event timestamp. - Added a replay key for progress log lines so WebSocket reconnect replay does not duplicate the same rendered progress line. - Added focused formatter/key tests covering phased progress, unphased progress, empty messages, and replay-key output. ### Visual output example The rendered log text is covered by the new formatter test: ```text [workspace] Syncing issue history Preparing workspace ``` No layout or styling changes are included; this PR only makes existing log-line UI receive one more live event type. ## Verification - `pnpm install --frozen-lockfile` — completed; emitted non-fatal bin-link warnings for the unbuilt plugin SDK dev CLI. - `pnpm --filter @paperclipai/ui typecheck` — passed. - `pnpm --filter @paperclipai/ui exec vitest run src/pages/AgentDetail.progress.test.ts src/context/LiveUpdatesProvider.test.ts` — passed, 2 files / 26 tests. - `git diff --check` — passed. - Local sensitive-content scan over the PR diff using patterns for API keys, tokens, secrets, passwords, auth headers, private keys, localhost/private paths, internal ticket ids, agent links, and tailnet markers — no findings. ## Risks Low risk. This is a UI-only live-event handling change for an existing event type. The replay guard is intentionally scoped to progress lines and uses the rendered timestamp, stream, and chunk as the key, so repeated progress events with distinct timestamps or messages still appear. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, GPT-5 coding agent with repository tool use, shell execution, GitHub CLI access, and local test execution. Context window size was not exposed in this runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-04 06:33:49 -07:00
Devin FoleyandPaperclip	bcac517f3b	Add browser SSH terminal for custom image setup (#8911 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Environment sandboxes already support custom image creation and refresh through a temporary SSH setup session. > - The existing workflow makes operators copy an SSH command into an external terminal before they can install packages or make image changes. > - That extra context switch is slower, easier to get wrong, and less integrated with the setup session Paperclip already tracks. > - This pull request adds an embedded browser SSH terminal for custom image setup, so operators can start working in the target sandbox directly from the environment configuration flow. > - The implementation uses short-lived websocket attachment tokens, session-lifetime SSH host-key pinning, and server-managed terminal cleanup so the feature fits the existing setup-session boundary. > - The benefit is a smoother custom image creation and refresh experience without asking users to leave Paperclip for routine sandbox setup work. ## Linked Issues or Issue Description No public GitHub issue exists. ### Subsystem affected Cross-cutting: `server/` custom image setup APIs and websocket handling, `ui/` environment configuration UI, and shared custom image contracts. ### Problem or motivation Custom image creation and refresh require an operator to open a separate SSH client, paste the command shown by Paperclip, perform setup work, then return to the browser to finish the image flow. This is functional but awkward for a setup process that already starts and tracks a temporary sandbox session. ### Proposed solution Embed an SSH terminal in the custom image setup UI. When a setup session exposes an SSH payload, Paperclip should open a browser terminal backed by a server-side websocket session, let the operator run setup commands in-place, and then close the terminal when setup is finished, cancelled, expired, or disconnected. ### Alternatives considered - Keep the existing copy/paste SSH command workflow. This remains a fallback, but it does not streamline the common path. - Put SSH credentials directly into websocket URLs. This was avoided so terminal authentication can happen in an explicit first websocket auth frame rather than in logged URLs. - Trust the SSH host blindly for every reconnect. This PR instead pins the observed host-key fingerprint for the setup-session lifetime. ### Roadmap alignment This fits the roadmap theme of making agent workspaces usable in more remote and sandboxed environments while preserving Paperclip's control-plane model. ### Additional context Public GitHub search did not find a duplicate issue or PR for `custom image terminal ssh` in `paperclipai/paperclip`. ## What Changed - Added server-side terminal session tracking for custom image setup sessions, including connect-token issuance, websocket attachment, expiry, resize, input, and shutdown handling. - Added an embedded browser terminal to the custom image creation and refresh flow when a setup session provides SSH connection details. - Moved terminal token authentication out of the websocket URL and into the first websocket JSON auth frame. - Added SSH host-key SHA-256 pinning for each terminal session and documented the provider convention for username-embedded SSH credentials. - Updated the custom image environment API and UI so the setup terminal can open, reconnect, show status, authenticate, resize, and remain active for the setup-session lifetime once attached. - Kept custom image setup routes company-scoped and closed active terminal sessions on setup finish/cancel. - Added focused unit/integration/UI coverage for token expiry, setup-session expiry, websocket close paths, host-key pinning, and terminal session lifecycle behavior. - Removed the generated lockfile delta from the PR; CI owns temporary lockfile regeneration for manifest-changing PRs. ## Verification - `pnpm exec vitest run server/src/__tests__/server-startup-feedback-export.test.ts server/src/__tests__/environment-custom-image-terminal-ws.test.ts server/src/services/environment-custom-image-terminal-sessions.test.ts server/src/__tests__/environment-custom-image-routes.test.ts packages/shared/src/environment-custom-images.test.ts ui/src/pages/CompanyEnvironments.test.tsx` - 6 test files passed - 58 tests passed - `pnpm --filter @paperclipai/server typecheck` - `pnpm --filter @paperclipai/ui typecheck` - `pnpm --filter @paperclipai/server build` - `pnpm --filter @paperclipai/ui build` - `pnpm run typecheck:build-gaps` - `git diff --check` - Local sensitive-content scan over the PR diff using patterns for API keys, private keys, private hostnames, local paths, token fields, and credential-like strings. - Findings were limited to removed URL-token code and synthetic test placeholders such as `ssh-token-secret` and `terminal-token-terminal-token-123456`. - No real credentials, private hostnames, local filesystem paths, or instance-local links were found. - Remote PR checks were green after the implementation commit, including Build, Typecheck + Release Registry, General tests, serialized server suites, e2e, verify, Socket, Snyk, Superagent, and Greptile 5/5. - Post-merge PR hardening on July 3, 2026: merged `origin/master` at `47448721e` into the branch, resolved the `CompanyEnvironments.tsx` import conflict, reran focused tests, server/UI typechecks, server/UI builds, `pnpm run typecheck:build-gaps`, and `git diff --check`, scanned the final diff for sensitive content, pushed `4b43558cc`, and confirmed all remote checks plus Greptile 5/5 were green. - PR metadata correction on July 3, 2026: changed the title/body framing from bug-fix language to feature-request language. No source files changed for this metadata-only update. ## Risks - Moderate surface area because this adds websocket routing, setup-session runtime state, package dependencies, and a new custom image UI path. - New websocket attachments still require valid short-lived tokens; established terminal sessions remain bounded by setup-session expiry, explicit finish/cancel, client close, or server shutdown. - The terminal-session store is in-memory, so active terminal websocket tokens and host-key pins do not survive server restarts. - SSH host-key verification uses session-lifetime TOFU pinning because the current provider payload does not expose a trusted host-key fingerprint. - The external SSH command remains important as a fallback if a browser, proxy, or network environment cannot sustain the websocket terminal. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, GPT-5 coding agent with shell/tool execution. Context window size was not exposed in this runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-03 16:44:21 -07:00
Devin FoleyandPaperclip	c48feee190	Improve live agent feedback during sandboxed runs (#8915 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A core part of that experience is watching active agent runs without dropping into raw logs first > - Local and sandbox-backed adapters already record useful run output, progress, and tool activity > - But active issue threads could sit visually stale while the agent was syncing workspaces, tailing sandbox output, or emitting incremental tool-call updates > - Operators need timely, human-readable progress while preserving the raw transcript underneath > - This pull request streams sandbox run-log progress into runtime status, keeps visible issue threads refreshed, and folds repeated ACPX tool updates into stable transcript cards > - The benefit is that long-running agent work becomes easier to supervise without changing the task/comment control-plane model ## Linked Issues or Issue Description No public GitHub issue exists for this exact change. Problem/motivation: - During long-running sandboxed agent work, the issue UI can appear idle even though the agent is actively syncing, running tools, or producing incremental output. - Operators need realtime feedback at the issue-thread layer, not only after opening raw logs or waiting for the final heartbeat result. - Related public context: #1808 previously added live-run status dots to Projects; #4362 touches heartbeat wakeup behavior but is not a duplicate of this runtime/UI feedback change. ## What Changed - Added sandbox run-log streaming support and defaulted sandbox-capable local adapters into the richer live-feedback path. - Surfaced environment/sandbox sync progress through heartbeat runtime status with bounded, redacted snippets. - Added live issue-thread cache patching so visible active runs update as progress events arrive. - Folded repeated ACPX `tool_call` updates into one transcript card instead of stacking duplicate cards. - Updated adapter docs and added focused regression coverage for sandbox log streaming, runtime status, ACPX parsing, live updates, transcript rendering, and issue chat messages. ## Verification - `pnpm install --frozen-lockfile` - `pnpm exec vitest run ui/src/context/LiveUpdatesProvider.test.ts` - `pnpm exec vitest run server/src/services/heartbeat-run-runtime-status.test.ts server/src/__tests__/heartbeat-runtime-state.test.ts ui/src/context/LiveUpdatesProvider.test.ts` - `pnpm exec vitest run packages/adapter-utils/src/execution-target-sandbox.test.ts packages/adapter-utils/src/sandbox-managed-runtime.test.ts server/src/services/heartbeat-run-runtime-status.test.ts server/src/__tests__/agent-live-run-routes.test.ts server/src/__tests__/heartbeat-runtime-state.test.ts packages/adapters/acpx-local/src/ui/parse-stdout.test.ts ui/src/context/LiveUpdatesProvider.test.ts ui/src/components/transcript/RunTranscriptView.test.tsx ui/src/lib/issue-chat-messages.test.ts ui/src/components/IssueChatThread.test.tsx` - GitHub PR workflow on head `8397953e7b41ccd42e5d9457ee7e4dfb996e4ec5`: `verify`, build, typecheck/release-registry, e2e, general shards, serialized server shards, and canary dry run passed. - Greptile Review on head `8397953e7b41ccd42e5d9457ee7e4dfb996e4ec5`: Confidence Score 5/5, no unresolved review threads. ## Risks - Live issue-thread cache patching could miss an edge case for a route shape not covered by tests. - Surfacing active-run snippets needs continued care around redaction; this PR keeps snippets bounded and adds redaction-focused coverage. - More frequent active-run UI refreshes could expose performance issues on very large issue threads, though updates are scoped to visible run/query caches. ## Model Used OpenAI GPT-5 via Codex, operating as a tool-enabled coding agent with shell, git, and repository-editing capabilities. Context window size is not exposed in this runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-02 22:21:56 -07:00
Devin FoleyandPaperclip	bf982c8c83	Normalize adapter display labels (#8913 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Adapter names are part of the board-facing agent setup and management experience. > - The product now treats adapters as harnesses, while execution environments are modeled separately. > - Several built-in adapter labels still carried legacy local wording from the older harness-by-environment model. > - That wording makes the UI noisier and implies a distinction users no longer need to reason about. > - This pull request normalizes adapter display labels while keeping persisted adapter type identifiers unchanged. > - The benefit is clearer adapter selection and management copy without a database migration. ## Linked Issues or Issue Description No public GitHub issue was found for this exact cleanup. Related public PRs: - Supersedes #8910, an earlier branch for the same cleanup that did not include the later docs/gateway/Cursor alignment. - Refs #8819, which is related display-registry work for external multi-segment adapter labels, but not a duplicate of this built-in label cleanup. Feature request details: - Subsystem affected: Cross-cutting (`ui/`, `packages/adapters`, and docs). - Problem or motivation: user-facing adapter names include legacy local qualifiers even though adapters map to harnesses and environments are first-class elsewhere. - Proposed solution: remove the legacy local wording from built-in display labels, keep machine-readable adapter type ids unchanged, and keep gateway disambiguation where it is useful. - Alternatives considered: changing persisted adapter type ids was ruled out because it would create migration and compatibility risk; one-off UI replacements were ruled out because the display registry is already the correct central label boundary. - Roadmap alignment: this is small adapter UX polish, not a new roadmap-level core feature. ## What Changed - Updated the adapter display registry so known adapter labels are final and no built-in local adapter renders a legacy local suffix. - Preserved clean derived labels for unknown plugin local types while keeping gateway disambiguation for unknown gateway types. - Updated `AdapterManager` to prefer registry labels when the server reports raw adapter type ids for built-ins. - Removed legacy local wording from built-in adapter metadata labels in UI and adapter packages. - Aligned Cursor adapter metadata with the central display registry label. - Updated adapter docs and Storybook fixtures to match the new display names. - Added focused registry coverage for built-in labels and unknown plugin suffix behavior. ## Verification - `pnpm check:tokens` - `git diff --check origin/master...fix/adapter-display-labels` - Patch-addition scan for added secrets, private paths, and internal links: no matches. - GitHub duplicate search for open adapter-label/local-suffix issues and PRs; #8910 was identified as the older superseded public PR. - `pnpm exec vitest run ui/src/adapters/adapter-display-registry.test.ts` - `pnpm --filter @paperclipai/ui typecheck` - Stale-label scan found no remaining user-facing display-label suffixes; remaining local wording is operational/test terminology such as adapter ids, docs about running locally, and test descriptions. ## Risks Low risk. The change is display-label and documentation focused, and adapter type ids remain unchanged. The main risk is ambiguous gateway naming, mitigated by keeping explicit gateway labels where variants need disambiguation. ## Model Used OpenAI GPT-5 via Codex, tool-enabled coding agent in a local repository workspace. Context window size is not exposed by this environment. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-02 21:32:50 -07:00
Devin FoleyandPaperclip	936687ca55	fix(workspace): restore clean branch drift on finalize (#8914 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent runs can execute inside reusable, runtime-created git worktree execution workspaces. > - Those managed worktrees record the expected branch so later dispatches do not accidentally run an agent in the wrong checkout. > - Successful run finalization already checked branch coherence, but it treated every unrecorded branch switch as fatal. > - A common publishing flow can briefly switch a clean worktree to a PR/publish branch that points at the same commit as the recorded issue branch, leaving no divergent work to protect. > - This pull request keeps the strict finalization guard for unsafe drift, but lets finalization restore the recorded branch when same-commit repair is provably safe. > - The benefit is fewer false failed runs after harmless branch switches while preserving hard failures for divergent or dirty worktrees. ## Linked Issues or Issue Description No public issue exists for this exact finalization failure. Related public worktree-recovery context: #3087 and #3056, but those address different worktree realization/reuse recovery paths rather than successful-run finalization branch repair. Bug report details: What happened? When an adapter run succeeded after switching a managed git worktree from its recorded issue branch to a publish/PR branch, finalization failed with a managed worktree branch mismatch even when the publish branch and recorded branch pointed at the same commit and the worktree was clean. Expected behavior Finalization should restore the recorded branch only when it can prove the worktree is clean, registered, and the recorded branch points at the current `HEAD`. If the actual branch has different commits or unsafe state, finalization should continue to fail with bounded validation evidence. Steps to reproduce 1. Create a runtime-managed `git_worktree` execution workspace for an issue run. 2. During the adapter run, create and check out a new publish branch without committing new changes. 3. Return adapter success and let heartbeat finalization run. 4. Before this change, finalization records a failed branch check and fails the run even though the branches point at the same commit. 5. With this change, finalization records the repair operation, restores the recorded branch, and records a successful finalize row. 6. Repeat with a commit on the publish branch; finalization still fails because the branch heads differ. Paperclip version or commit Reproduced against `master` at `bac7307ec`; fixed by this PR at `64ec605cf`. Deployment mode Local dev / built from source. Agent adapter(s) involved Not adapter-specific. This is core heartbeat/workspace finalization behavior. Database mode Embedded test Postgres in the focused server test. Access context Agent run finalization. Node.js version `v25.6.1` Operating system `Darwin 24.6.0 arm64` Relevant logs or output The new focused test intentionally exercises both outcomes: ```text Test Files 1 passed (1) Tests 3 passed (3) ``` Relevant config Runtime-created `git_worktree` execution workspace. Additional context The unsafe divergent branch case still fails with `workspace_validation_failed` and `git_worktree_branch_incoherence` evidence. Privacy checklist Reviewed; this description avoids internal task links, local workspace paths, credentials, and instance-specific URLs. ## What Changed - Reused the existing guarded branch-coherence repair helper during heartbeat finalization when the final branch inspection finds clean same-commit branch drift. - Recorded repair metadata in the `workspace_finalize` operation so reviewers/operators can audit whether finalization repaired branch drift. - Preserved failure behavior for divergent branch heads and surfaced the bounded workspace validation evidence from the repair helper. - Added focused server coverage for safe finalization repair and unsafe divergent branch failure. - Updated execution semantics docs to describe the narrower finalization rule. ## Verification - `pnpm exec vitest run server/src/__tests__/heartbeat-workspace-finalize-branch.test.ts` - `pnpm --filter @paperclipai/server typecheck` - `git diff --check` ## Risks Low to medium risk. The change affects successful-run finalization for runtime-created git worktree execution workspaces. The repair path is constrained to clean, registered, same-commit branch drift, and the focused test confirms divergent branch heads still fail instead of being restored silently. ## Model Used OpenAI Codex, GPT-5-based coding agent. Exact hosted model ID was not exposed in the runtime; tool use and local shell execution were enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-02 20:58:26 -07:00
Devin FoleyandPaperclip	bac7307ec4	fix(adapter-utils): improve sandbox restore failure diagnostics (#8903 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent adapters share managed-runtime helpers from `@paperclipai/adapter-utils` so local and sandboxed runs can prepare, execute, and restore workspaces consistently. > - The sandbox managed runtime syncs workspaces in both directions by creating tar archives on the local host and inside the remote sandbox. > - A prior fix avoided archiving `.` during upload because tar self-entries can force chmod/utime on a directory the command user does not own. > - The restore/download path still archived the remote workspace as `.`, and failed managed commands only surfaced stderr. > - This pull request applies entry-based tar creation to the sandbox restore path and includes stdout in managed command failure diagnostics. > - The benefit is that restore failures become visible to operators and sandbox workspace restore avoids the same directory-metadata failure class already fixed for upload. ## Linked Issues or Issue Description No existing public issue found; describing in-PR (bug). - What happens: sandbox workspace restore can fail while creating `workspace-download.tar` from the remote workspace when the tar command includes a `.` self-entry and the command user cannot update metadata on the workspace directory. If the failing command writes its diagnostic to stdout, the managed runtime error can collapse to a generic failed shell command without the useful tar message. - Expected behavior: restore should archive the workspace entries without a `.` self-entry, and failed managed runtime commands should include useful stdout/stderr diagnostics. - Where: `packages/adapter-utils/src/sandbox-managed-runtime.ts` restore/download path and `packages/adapter-utils/src/command-managed-runtime.ts` command error formatting. - Related public context: #7836 fixed the upload side of the same tar self-entry failure class. ## What Changed - Added stdout-aware failed-command formatting in the command managed runtime, keeping diagnostics bounded to the tail of stdout/stderr. - Added remote workspace tarball creation that names top-level entries explicitly instead of archiving `.` during sandbox restore. - Preserved empty-workspace restore support by creating a valid empty tarball when the remote workspace has no entries. - Added regression coverage for stdout diagnostics, restore tar members, and empty workspace restore tarballs. ## Verification - `pnpm vitest run packages/adapter-utils/src/command-managed-runtime.test.ts packages/adapter-utils/src/sandbox-managed-runtime.test.ts` — 2 files, 17 tests passed. - `pnpm --filter @paperclipai/adapter-utils typecheck` — passed. - `git diff --check origin/master..HEAD` — passed. - Local public-hygiene/PII scan of the committed diff checked for internal ticket refs, local/private URLs, common token/key patterns, private-key blocks, and email-like values — passed. - GitHub duplicate/related search found no public issue or PR for `workspace-download.tar Permission denied` or `sandbox restore tar permission denied`; #7836 is linked as related prior work. - PR CI on commit `3ad4c8d` — all GitHub Actions lanes, security scans, and aggregate `verify` passed. - Greptile Review on commit `3ad4c8d` — Confidence Score 5/5; the prior P2 thread is resolved with no open P2s, recommendations, or follow-ups. ## Risks Low. This is limited to shared adapter runtime error formatting and sandbox restore archive construction. Archive contents should remain equivalent apart from the removed `.` self-entry, and the new diagnostics are bounded to avoid dumping unbounded command output. ## Model Used OpenAI Codex, GPT-5, tool-enabled coding agent with shell and GitHub CLI access. Context window size was not reported by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (n/a; internal adapter-runtime behavior only) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-02 17:06:34 -07:00
Devin FoleyandPaperclip	b4815bf964	Scope environment custom images to instance environments (#8850 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Environments are now managed as instance-level runtime resources rather than per-company rows > - The custom environment image setup tables were introduced with their own `company_id` columns and route query parameters > - That split made one saved environment image state depend on an extra company context even though the environment itself is the durable owner > - It also made saved-environment probes harder because applying the active custom image template could require a company context when no secret-backed config needed one > - This pull request scopes custom image templates and setup sessions directly to the saved environment > - The benefit is that reusable environment images follow the same instance-scoped model as environments while secret resolution still uses company context only when secrets require it ## Linked Issues or Issue Description No matching public GitHub issue was found. Bug report: ### What happened? saved environment custom-image routes and persistence required a `companyId` even though environments are instance-scoped, and saved sandbox probes did not opt into active custom-image template application unless a company context was present. ### Expected behavior custom-image templates and setup sessions should be owned by the saved environment, and saved sandbox probes should apply the active template while still requiring a company context only for secret-backed runtime config. ### Steps to reproduce 1. Configure an instance-scoped sandbox environment with custom-image setup support. 2. Start or inspect a custom-image session or template for that saved environment. 3. Probe the saved environment without a custom-image-specific `companyId` query parameter. ### Paperclip version or commit current `master` after the environment custom-image template migration. ### Deployment mode Local dev (pnpm dev) or authenticated local Paperclip instance. ### Installation method Built from source (pnpm dev / pnpm build). ### Agent adapter(s) involved Not adapter-specific (core bug). ### Database mode Embedded PGlite/Postgres dev database. ### Access context Board human operator. ### Privacy checklist No logs, secrets, tokens, private URLs, or local machine paths are included. Duplicate search performed: - `gh search prs "environment custom image companyId repo:paperclipai/paperclip" --state open --limit 20` - `gh search prs "custom image environment scoped repo:paperclipai/paperclip" --state open --limit 20` - `gh search issues "environment custom image repo:paperclipai/paperclip" --state open --limit 20` The returned results were unrelated adapter, Docker, auth, or stale-workspace items. ## What Changed - Removed redundant `company_id` columns from environment custom-image templates and setup sessions. - Added migration `0127_environment_custom_images_instance_scoped` to collapse duplicate active rows per environment before dropping the old company-scoped indexes/columns. - Updated custom-image services, route handlers, shared validators, and UI API/query keys to use environment-scoped custom-image state. - Kept runtime secret resolution company-aware only when secret refs or bindings require a company context. - Made saved sandbox environment probes opt into active custom-image template application. - Updated DB, shared, server, and UI tests for the new environment-scoped contract. ## Verification - `pnpm --filter @paperclipai/db run check:migrations` - `pnpm exec vitest run packages/db/src/environment-custom-images-schema.test.ts packages/shared/src/environment-custom-images.test.ts server/src/__tests__/environment-custom-image-routes.test.ts server/src/__tests__/environment-custom-images-service.test.ts server/src/__tests__/environment-routes.test.ts ui/src/pages/CompanyEnvironments.test.tsx` - `pnpm -r typecheck` - `pnpm test:run` before rebasing onto latest `master`; after the rebase only the migration number changed, and the migration check plus focused suite, typecheck, and build were rerun. - `pnpm build` ## Risks - Migration safety: the migration supersedes duplicate active templates per environment and fails duplicate active setup sessions before adding environment-only unique indexes. Operators with duplicate historical active rows should review which active template is kept. - Behavior shift: plugin custom-image setup calls now receive `companyId: "instance"` when no secret binding determines a concrete company context. - Secret-backed configs still require an explicit or uniquely inferable company context; environments with secret bindings spread across multiple companies continue to fail fast. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex via the `codex_local` adapter, GPT-5-based coding model with tool-enabled repository inspection, editing, testing, git, and GitHub CLI access. Exact context-window metadata was not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-01 23:57:31 -07:00
Devin FoleyandPaperclip	2eba718bef	Fix sandbox bridge credentials and stalled review recovery (#8844 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The local adapter and heartbeat recovery systems decide whether an agent has a real control-plane mutation path. > - Sandboxed local adapters split execution between the trusted host process and the sandbox shell/tool surface. > - A host-side adapter can still reach Paperclip while the sandbox shell surface cannot, which leaves agents thinking no endpoint or credentials are configured even though the host can still post comments. > - Execution-policy review stages can also remain pending after a reviewer run finishes without recording a decision. > - This pull request makes the sandbox bridge available to the actual shell mutation surface and adds bounded recovery for terminal-but-still-pending review participants. > - The benefit is that agents get a real reachable Paperclip API path where they need it, and stalled review stages become visible recovery work instead of silently drifting. ## Linked Issues or Issue Description No exact public GitHub issue matched this combined failure. I searched for exact and related terms including `cannot reach the Paperclip control plane`, `execution_review_participant_recovery`, `sandbox callback bridge`, `review participant in_review`, and `control plane sandbox`. Related public issues: - Refs #8482 for `in_review` liveness invariant recovery. - Refs #863 for prior agent API-key reachability confusion. - Refs #248 for the broader sandboxed agent execution model. Bug summary: - What happened: a sandboxed local-adapter run could have host-side Paperclip access while the sandbox Bash/tool surface lacked a reachable API endpoint or usable run credentials. Separately, a reviewer run could finish while its execution-review stage remained pending, leaving the source issue in `in_review` with no decision and no live participant run. - Expected behavior: the mutation surface that agents actually use should receive a run-scoped Paperclip bridge, and pending review participants should get one bounded normal-model recovery wake before moving to explicit blocked/source-scoped recovery. - Steps to reproduce: run a sandbox-backed local adapter that needs Bash/curl/tooling to call Paperclip from inside the sandbox, or finish an execution-policy reviewer run without submitting the pending review decision. - Deployment mode: local/authenticated private development instance with sandbox-backed local adapters. ## What Changed - Changed sandbox callback bridge startup so bridge credentials are passed through the sandbox runner environment instead of embedded in the visible `nohup env ...` command string. - Added adapter-utils coverage proving the sandbox shell can call Paperclip through the bridge, forwards the host run JWT with `X-Paperclip-Run-Id`, and does not leak host or bridge tokens into stdout/stderr, runner command text, or runtime files. - Added one bounded execution-review participant recovery path for terminal reviewer runs whose `executionState` remains pending. - Escalated exhausted or non-invokable review participant recovery to blocked/source-scoped recovery with dedicated evidence, activity, and next-action text. - Documented the mutation-surface reachability contract in `doc/execution-semantics.md` and updated the Paperclip skill authentication guidance for sandbox bridge env vars. ## Verification - `pnpm exec vitest run packages/adapter-utils/src/execution-target-sandbox.test.ts` - `pnpm exec vitest run server/src/__tests__/heartbeat-process-recovery.test.ts --no-file-parallelism --maxWorkers=1` - `pnpm --filter @paperclipai/adapter-utils typecheck` - `pnpm --filter @paperclipai/server typecheck` - `git diff --check` - `curl -fsS $PAPERCLIP_API_URL/api/health` returned `status: ok` on the local instance. ## Risks - Medium behavioral risk: more `in_review` issues with terminal-but-pending reviewer runs will now be retried once and then blocked explicitly instead of remaining quiet. - Low sandbox bridge risk: credential delivery moved from command text to the runner environment, which is less leaky but depends on sandbox providers honoring the env payload for startup commands. - No database migration is included. - Full repo build and CI were not run locally before opening the PR; targeted server/adapter tests and typechecks passed. ## Model Used OpenAI GPT-5 via the Codex local agent, with repository tool use and shell-based code execution. The runtime did not expose a precise context-window value to the agent. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-01 21:38:01 -07:00
Devin Foley Cody Cody	d68c34f2cc	Fix managed workspace branch coherence recovery (#8826 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Managed issue workspaces are part of the control-plane runtime boundary: the server records which git worktree and branch an agent run is allowed to use. > - Existing reuse checks validated the worktree path and cleanliness, but did not fully validate that the actual checked-out branch still matched the recorded execution workspace branch. > - That gap let an agent run switch a managed worktree onto a publishing branch without updating the execution workspace record, then later reuse or finalize the workspace as though it were coherent. > - The runtime needs a bounded repair path for provably safe mismatches and a hard validation failure for dirty, divergent, or unrecorded branch transitions. > - This pull request adds branch coherence to managed git worktree validation, records explicit recovery evidence, and prevents finalize success when a run silently changes branches. > - The benefit is that branch drift becomes either safely repaired or visibly recoverable instead of silently corrupting managed workspace state. ## Linked Issues or Issue Description No public GitHub issue exists for this bug. Bug report: - What happened: a managed agent workspace could be recorded for one branch while the underlying git worktree was actually checked out on another branch. Reuse and finalization could still treat the workspace as healthy. - Expected behavior: managed git worktrees should verify the actual branch against the recorded execution workspace branch. Safe same-HEAD clean mismatches may be repaired, while dirty, divergent, or unrecorded branch transitions should fail into explicit workspace validation recovery. - Reproduction outline: create a runtime-managed issue worktree, switch its checkout to another branch without updating the execution workspace record, then attempt reuse or run finalization. - Deployment mode: local/self-hosted Paperclip server using managed git workspaces. - Related public work: Refs #7644 and #7579. Related but not duplicate: #8275 and #5851. ## What Changed - Added managed git worktree branch inspection, formatted validation evidence, and safe same-HEAD repair logic to the workspace runtime service. - Validated recorded managed workspace branch state before reuse and during heartbeat setup. - Added finalization-time branch guards so runs that silently switch branches fail with `workspace_validation_failed` instead of recording a successful finalize. - Added recovery fingerprints and evidence for `git_worktree_branch_incoherence`, including manual-repair next actions for unsafe branch drift. - Documented branch coherence as part of runtime-created git worktree workspace coherence. - Added focused tests for safe branch repair, dirty/divergent recovery evidence, heartbeat setup validation, and finalize failure/success paths. ## Verification - `pnpm install --frozen-lockfile` - `git diff --check origin/master...HEAD` - `pnpm exec vitest run server/src/__tests__/workspace-runtime.test.ts server/src/__tests__/heartbeat-workspace-session.test.ts server/src/__tests__/issue-recovery-actions.test.ts server/src/__tests__/heartbeat-workspace-finalize-branch.test.ts` - `pnpm -r typecheck` - `pnpm test:run` - `pnpm build` Notes: - An initial full `pnpm test:run` attempt hit a transient `socket hang up` in one `plugin-routes-authz` case. The exact case passed when rerun directly, the full `plugin-routes-authz` file passed, and the subsequent full `pnpm test:run` passed. - `pnpm build` still emits existing Vite CSS pseudo-element and chunk-size warnings unrelated to this change. ## Risks - This intentionally changes behavior for managed runs that switch branches without recording the transition: they now fail during workspace validation/finalization instead of silently proceeding. - The automatic repair path is intentionally narrow. It only repairs clean branch mismatches when both branches point at the same commit; dirty or divergent worktrees require manual recovery. - Recovery fingerprints now include workspace-validation evidence, so duplicate recovery-action grouping is more precise for branch-incoherence failures. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, GPT-5 Codex CLI/API coding agent, with shell/git/test execution and reasoning mode enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Cody <noreply@paperclip.ing> Co-authored-by: Cody <cody@paperclip.ing>	2026-07-01 13:35:58 -07:00
Devin Foley Paperclip Cody	a8f0ebaa80	Refresh run config before reusing workspaces (#8797 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent runs are assembled by the heartbeat service from agent config, project workspaces, environment config, secret bindings, skills, and runtime session state. > - The heartbeat service intentionally reuses adapter sessions, execution workspaces, and sandbox leases when that preserves useful state. > - Reuse becomes incorrect when the effective next-run config changes after a saved session, workspace, or lease was created. > - Stale reuse can make a later run appear pinned to old agent, environment, secret, instruction, or workspace settings. > - This pull request records non-sensitive fingerprints for the effective session, workspace, and lease config at run boundaries. > - When those fingerprints drift, Paperclip refreshes persisted runtime config or starts fresh execution instead of reusing stale state. > - The benefit is predictable next-run config freshness without storing raw secret values, full env maps, provider credentials, or private path details. ## Linked Issues or Issue Description - Refs #8058 - Related PRs checked during dedup search: #4968, #4155, #84, #8480. These cover nearby workspace/session routing or model-config freshness areas, but do not duplicate this effective run config fingerprinting path. ## What Changed - Added effective run config fingerprinting for session, workspace, and lease reuse decisions, with canonicalization that ignores generated runtime noise and redacts sensitive values. - Updated heartbeat reuse logic to compare stored and next-run fingerprints, reset stale saved sessions, refresh persisted workspace config snapshots, replace stale reused workspaces when required, and avoid stale sandbox lease reuse. - Included plain environment value drift via value hashes, without storing the raw env values. - Root-bound instruction content hashing so legacy direct absolute instruction paths are represented but not read for config fingerprints. - Batched secret/version metadata lookups for environment lease fingerprinting. - Added workspace operation/run result freshness metadata so operators can inspect non-sensitive decision categories. - Surfaced config freshness labels and next-run copy in the UI and docs. - Added focused coverage for fingerprint redaction, session reset decisions, workspace refresh/replace behavior, environment lease drift, and persisted workspace restoration. ## Verification - `git diff --check` - Sensitive-data scan before push: - `git diff --unified=0 origin/master...HEAD \| rg -n --pcre2 "(AWS_ACCESS_KEY_ID\|AWS_SECRET_ACCESS_KEY\|ghp_[A-Za-z0-9_]{20,}\|github_pat_[A-Za-z0-9_]{20,}\|sk-[A-Za-z0-9]{20,}\|-----BEGIN (RSA \|OPENSSH \|EC \|DSA )?PRIVATE KEY-----\|AKIA[0-9A-Z]{16})"` - `git diff --unified=0 origin/master...HEAD \| rg -n --pcre2 "[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\\.[A-Za-z]{2,}"` - `pnpm exec vitest run server/src/__tests__/effective-run-config-fingerprints.test.ts server/src/__tests__/heartbeat-workspace-session.test.ts server/src/__tests__/environment-runtime.test.ts` - `pnpm --filter @paperclipai/server typecheck` - `pnpm -r typecheck` - `pnpm --filter @paperclipai/db clean` - `pnpm test:run` - `pnpm build` - UI screenshots from Cutter: - https://artifacts.cutter.sh/8797/run-2f4827c-2026-06-30T18-57-25/preview/change-01.png - https://artifacts.cutter.sh/8797/run-2f4827c-2026-06-30T18-57-25/preview/change-02.png - https://artifacts.cutter.sh/8797/run-2f4827c-2026-06-30T18-57-25/preview/change-03.png ## Risks - Medium: overly broad fingerprints could start fresh sessions, workspaces, or sandbox leases more often than necessary. - Medium: missing a config category would allow stale reuse to persist for that category. - Medium: legacy direct absolute instruction paths are no longer content-hashed unless they are paired with an absolute managed instructions root. - Low data risk: fingerprint metadata stores hashes and category names, not raw secrets, raw env values, provider credentials, or private path details. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 via Codex CLI / Codex coding agent, tool-enabled with shell, Git, GitHub CLI, local test execution, and code editing. The exact deployed model variant and context window are not exposed by this environment. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Cody <cody@paperclip.ing>	2026-06-30 14:50:25 -07:00
Devin FoleyandPaperclip	8d9f9fd240	Add reusable sandbox custom images (#8794 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A growing part of that work runs in sandboxed environments rather than on the operator's local machine. > - Today sandbox providers can start fresh workspaces and run probes, but they do not have a shared contract for capturing and reusing prepared sandbox state. > - Operators need a way to set up tools, credentials, and project dependencies once, then reuse that prepared image for later agent runs. > - This pull request adds reusable sandbox custom images across the provider contract, server runtime, and board UI. > - It also keeps probes and sandbox copy flows aligned with pre-authenticated/custom-image environments. > - The benefit is faster, more reliable sandbox runs without repeatedly rebuilding the same environment setup. ## Linked Issues or Issue Description No public GitHub issue was found for this change. Inline feature request follows. ### Problem or motivation Sandboxed agents need reusable prepared runtime state so repeated runs do not require manual setup every time. Operators often need system packages, CLIs, SDKs, dependency caches, credentials, and project tooling available before an agent can work productively. ### Proposed solution Add a provider-level custom-image capability, server-side setup/capture lifecycle, Daytona/fake provider support, and board UI controls for creating, testing, selecting, and deleting custom images. ### Alternatives considered Leaving this as provider-specific setup outside Paperclip would keep the control plane blind to image state and would not give agents consistent environment metadata. Re-running setup commands for every lease is simpler, but slower and less reliable for interactive or credentialed setup. ### Roadmap alignment Checked `ROADMAP.md`; this aligns with the Cloud / Sandbox agents roadmap area and does not duplicate any related public issue or PR found by search. Additional context: - Subsystem affected: cross-cutting (`packages/db`, `packages/shared`, `packages/plugins`, `server`, `ui`). - Duplicate search: searched GitHub for `sandbox custom image` and `sandbox template environment`; no related public issues or PRs were found. ## What Changed - Added custom-image shared types, validators, constants, API paths, and database schema/migration. - Added server services/routes for custom-image templates and setup sessions, including runtime cleanup and provider metadata handling. - Extended plugin/sandbox provider capabilities for interactive setup, template capture, and template deletion. - Implemented custom-image support in the fake sandbox provider and Daytona provider. - Updated environment runtime/config handling so active custom images flow into leases, probes, and agent execution. - Added board UI controls and API client support for custom-image setup, capture, selection, status, and error states. - Hardened sandbox copy/probe behavior for insecure clipboard contexts and pre-authenticated sandbox images. - Added targeted coverage across shared validators, DB schema, server routes/services, provider plugins, adapter probes, and UI flows. ## Verification - `pnpm install --frozen-lockfile --ignore-scripts` - `pnpm vitest run packages/adapters/claude-local/src/server/test.probe.test.ts packages/adapters/claude-local/src/server/test.ts packages/adapters/codex-local/src/server/test.remote.test.ts packages/adapters/codex-local/src/server/test.ts` - `pnpm --filter @paperclipai/adapter-claude-local typecheck` - `pnpm --filter @paperclipai/adapter-codex-local typecheck` - `pnpm vitest run packages/db/src/environment-custom-images-schema.test.ts packages/shared/src/environment-custom-images.test.ts packages/shared/src/validators/plugin.test.ts server/src/__tests__/environment-custom-images-service.test.ts server/src/__tests__/workspace-runtime.test.ts packages/plugins/sandbox-providers/daytona/src/plugin.test.ts ui/src/pages/CompanyEnvironments.test.tsx ui/src/pages/CompanySettings.test.tsx` - `pnpm -r typecheck` - `pnpm build` - `rm -rf packages/db/dist && pnpm test:run` - Public-safety scan of the final diff found no internal Paperclip issue links, private instance URLs, or real secret patterns. ## Risks - Adds a database migration and new environment runtime tables, so migration ordering and rollback need care. - Provider implementations may differ in how reliably they can capture/delete images; unsupported providers surface capability-gated UI states. - Custom-image state can contain operator-prepared tooling and credentials inside the provider image, so providers must enforce their own access controls and cleanup semantics. - Broad surface area across shared contracts, server runtime, plugins, adapters, and UI means CI and Greptile review should be watched closely. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 (`gpt-5`) via Codex CLI with tool use and code execution. Assisted with branch cleanup, conflict resolution, local verification, and PR preparation. Earlier branch implementation work was assisted by Paperclip-managed Claude/Codex agents. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-06-30 09:57:49 -07:00
Devin Foley	b61c2852fb	build(ui): drop redundant tsc type-check from ui build (#8781 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - CI runs a set of required PR checks; the Canary Dry Run job is the slowest (~4m) > - Its dominant cost is `release.sh` Step 2 `pnpm build` (~133s), whose critical path is the `ui` package > - In `ui`, `tsc -b` runs ~46s before `vite build` (~20s), but `ui/tsconfig.json` sets `noEmit: true`, so `tsc -b` is a pure type-check that emits nothing — vite produces all artifacts > - The PR workflow already runs a parallel Typecheck job (`typecheck:build-gaps`) that auto-type-checks any workspace package whose build script omits `tsc` > - This pull request drops `tsc -b` from the `ui` build so the type-check moves off the build/release critical path onto the slack-rich Typecheck job > - The benefit is ~46s shaved off the slowest PR check with no loss of type coverage ## What Changed - `ui/package.json`: `build` script changed from `tsc -b && vite build` to `vite build`. - Type coverage is preserved automatically: `scripts/run-typecheck-build-gaps.mjs` already type-checks any workspace whose `build` script omits `tsc` and defines a `typecheck` script. With this change, `@paperclipai/ui` (whose `typecheck` is `tsc -b`) is now picked up by the parallel Typecheck job. No script edit was required — the gap-detection is generic. ## Verification - `pnpm --filter @paperclipai/ui build` produces `ui/dist` via vite alone (no `tsc` invocation). - `pnpm typecheck:build-gaps` now type-checks `@paperclipai/ui` and exits 0. - A type error in `ui` still fails the parallel Typecheck job, so type-error coverage is unchanged. ## Risks - Low. The type-check is relocated, not removed. This is build-tooling only — no migration, no runtime behavior change. If `scripts/run-typecheck-build-gaps.mjs` ever stopped covering `ui`, a type error could slip past the build job — but its gap-detection is generic (any package with a tsc-less build plus a `typecheck` script), and `ui` qualifies. ## Model Used Claude (Anthropic), `claude-opus-4` family, extended-thinking mode with tool use, operating as an automated CI-health agent.	2026-06-30 08:55:14 -07:00
Devin FoleyandPaperclip	0c2ec7deb4	Fix sandboxed Claude and Codex probe behavior (#8775 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Local adapters are the bridge between Paperclip's control plane and provider CLIs such as Claude Code and Codex. > - Those adapters can run either on the host machine or inside a remote/sandbox execution target. > - Sandbox probes need to validate the same auth/config path that real sandbox execution will use. > - The previous probe paths could surface misleading Claude errors, rely on host-only Codex state, or upload far more Codex home state than the probe needed. > - This pull request fixes the Claude and Codex sandbox probe/runtime behavior together while keeping provider-specific sandbox image work out of scope. > - The benefit is faster, clearer adapter health checks that better match real sandbox execution. ## Linked Issues or Issue Description No public GitHub issue was found for this exact bug during duplicate search. Bug report: What happened? Sandboxed Claude/Codex adapter tests could diverge from real runtime auth/config behavior. Claude sandbox probes could show the leading stream init line instead of the real final error, and Codex sandbox probes could upload full managed home state or mask a sandbox-local login with an empty uploaded `CODEX_HOME`. Expected behavior Sandbox probes should exercise the remote runtime contract, preserve useful sandbox credentials, avoid relying on unrelated host state, and report actionable probe failures. Steps to reproduce 1. Configure a remote/sandbox execution target for `claude_local` or `codex_local`. 2. Run the environment Test/probe path where host credentials differ from the sandbox's runtime credentials or the managed Codex home contains session history. 3. Observe that probe behavior can differ from the actual sandbox runtime path or surface an unhelpful Claude stream initialization line. Paperclip version or commit Current `master` before this PR, based on `4a2447da3`. Deployment mode Local development/control-plane deployment with remote sandbox execution targets. Related search performed: - Public issues: `Claude sandbox probe`, `Codex CODEX_HOME sandbox` returned no matches. - Public PRs: `Claude Codex sandbox probe`, `codex home sandbox`, `claude auth sandbox` returned no matches. ## What Changed - Made Claude sandbox Test probes materialize the same Paperclip-managed Claude config seed path used by sandbox execution. - Preserved sandbox-local Claude credentials when materializing remote Claude config and expanded auth-required detection for `/login` API-key failures. - Improved Claude hello-probe diagnostics so the final result/error is surfaced instead of the unhelpful stream init event, with transient upstream failures downgraded to warnings. - Changed Codex probe behavior to upload only minimal auth/config files instead of the full managed `CODEX_HOME`. - Let Codex sandbox probes leave `CODEX_HOME` unset when the host has no credentials, so pre-authenticated sandbox images can be tested directly. - Excluded bulky host-local Codex session/shell state from sandbox runtime home uploads. - Switched the Codex local default model away from the ChatGPT-unsupported `gpt-5.3-codex` option. - Added regression coverage for Claude parsing/probe paths, Codex adapter metadata/argument/probe behavior, and server-level Claude sandbox environment behavior. ## Verification Passed locally: - `pnpm install --frozen-lockfile` - `pnpm vitest run packages/adapters/claude-local/src/server/parse.test.ts packages/adapters/claude-local/src/server/test.probe.test.ts server/src/__tests__/claude-local-adapter-environment.test.ts` - `pnpm vitest run packages/adapters/codex-local/src/index.test.ts packages/adapters/codex-local/src/server/codex-args.test.ts packages/adapters/codex-local/src/server/test.remote.test.ts` - `pnpm --filter @paperclipai/adapter-claude-local typecheck` - `pnpm --filter @paperclipai/adapter-codex-local typecheck` - `pnpm --filter @paperclipai/server typecheck` - `git diff --check` ## Risks - Adapter configuration behavior is sensitive to local vs sandboxed execution mode, so review should focus on environment detection, argument construction, and any state written during probe/test runs. - The Codex default-model change may affect newly created agents that rely on the adapter default instead of an explicit model. - Excluding Codex session/shell state from sandbox uploads should be safe for fresh sandbox runs, but reviewers should confirm no runtime resume path depends on that host-local state. - Provider-specific setup/capture behavior is intentionally left to separate work. ## Model Used OpenAI GPT-5 Codex via Paperclip `codex_local`; tool-enabled local coding session with terminal access. Context window size was not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-06-30 08:41:50 -07:00
Devin FoleyandPaperclip Agent	f019f54bb3	Fix active heartbeat run reaping (#8776 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Heartbeat monitoring is the subsystem that keeps agent execution visible and recovers work only when execution continuity is genuinely lost. > - Routes and scheduler paths can construct separate heartbeat service instances inside the same server process. > - Active adapter execution tracking was scoped to each service instance, so the periodic orphan reaper could miss a run that another service instance was actively executing. > - Remote or sandbox adapters are especially exposed because they may not persist a local process PID/group and may go quiet while the remote command is still alive. > - This pull request makes active in-process adapter execution tracking shared across heartbeat service instances and adds a regression for the cross-instance reaper case. > - The benefit is fewer false `process_lost` failures for long-running or quiet sandbox/remote agent runs. ## Linked Issues or Issue Description No public GitHub issue exists. This PR describes the bug inline using the bug report template fields. ### What happened? An actively executing heartbeat run could be finalized as `process_lost` by the orphan reaper when adapter execution was active through one `heartbeatService()` instance but the reaper ran through another instance in the same server process. ### Expected behavior The orphan reaper should skip runs that are still actively executing in-process, regardless of which `heartbeatService()` instance is doing the reaping. ### Steps to reproduce 1. Create two `heartbeatService()` instances in the same process. 2. Start an adapter run through the first instance. 3. Backdate the run row enough for orphan reaping to consider it stale. 4. Run orphan reaping through the second instance while the first instance is still awaiting adapter execution. 5. Observe that the old instance-local tracking can mark the live run as `process_lost`. ### Paperclip version or commit Reproduced against `master` before commit `44ba6d8bb4f7ae1ca3715697f750844d770d83a3`. ### Deployment mode Self-hosted/local server process with route and scheduler code paths constructing separate heartbeat service instances. Remote or sandbox adapters are the highest-risk case because they may not have local PID metadata and can be quiet while still running. ## What Changed - Moved active adapter execution tracking from the `heartbeatService()` closure to module-level process state shared by heartbeat service instances. - Added a regression test that starts a run through one heartbeat service instance and runs orphan reaping through another, proving the active run is not reaped and can finish normally. ## Verification - `pnpm exec vitest run server/src/__tests__/heartbeat-process-recovery.test.ts` passes: 61 tests. - `pnpm -r typecheck` passes. - `pnpm build` passes; existing UI build warnings remain for `::highlight(...)`, large chunks, and a mixed static/dynamic import. - `pnpm test:run` does not fully pass in this local environment: 1 unrelated existing failure in `server/src/__tests__/workspace-runtime.test.ts` for `auto-detects the default branch via symbolic-ref when origin/HEAD is set`. The fixture command fails with `git push -u origin main master` because the temp repo has no `master` ref. - Reran the isolated failing test with `pnpm exec vitest run server/src/__tests__/workspace-runtime.test.ts -t "auto-detects the default branch via symbolic-ref when origin/HEAD is set"`; it reproduces the same missing-`master` ref failure. ## Risks - Low risk for single-process Paperclip servers: this only broadens in-process active run tracking across service instances. - Multi-process deployments still need persisted or distributed execution liveness to coordinate reaping across processes; this PR does not claim to solve cross-process recovery. - A run could be skipped by the reaper while its adapter promise is active, but the existing `finally` path removes the active marker after execution settles. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 via Codex, with tool use and local command execution. The runtime did not expose a separate context-window value. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass — targeted regression, typecheck, and build pass; full `pnpm test:run` has the unrelated missing-`master` ref fixture failure documented above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes — no docs change needed for this internal bug fix - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip Agent <noreply@paperclip.ing>	2026-06-30 00:40:56 -07:00