## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Isolated task workspaces are Git worktrees on the server disk, and
the terminal workspace reaper keeps that disk usage bounded
> - #17369 moves old workspaces off their private per-issue clones onto
the project's shared clone and then deletes the private clone
> - On one instance, the first live sweep moved 12 workspaces but
deleted none of their clones (about 600 MB each), and 3 workspaces could
not move because their restore point was "too large"
> - The kept clones are not referenced by any workspace, so no other
cleanup removes them
> - This pull request replaces the clone check with a real
reproducibility check, sweeps the orphaned clones, lifts the restore
point limits for local transfers, and fixes a false
`source_identity_changed`
> - The benefit is that the disk space #17369 was meant to reclaim is
reclaimed, without deleting unique work
## Linked Issues or Issue Description
Refs #17369. Bug report described in-PR (bug report template).
**What happened?**
On one instance, with `PAPERCLIP_ISOLATED_SOURCE_REHOME_ENABLED=true`
and dry run off, the first sweep re-homed 12 workspaces, skipped 5 with
`live_or_unresolved_consumer` and 3 with `restore_point_too_large`. No
clone was removed: 11 logged `cloneKeptReason:
"clone_checkout_has_files"` and 1 logged `"source_identity_changed"`.
Each clone's own checkout was clean (`git status --porcelain
--untracked-files=all` empty). Its only history that the shared clone
lacked was remote branches and release tags that the shared clone never
fetched, which exist on the remote. An operator removed 11 clones by
hand after verifying them.
**Expected behavior**
A re-homed clone whose checkout is clean and whose local history is in
the shared clone or on the remote is deleted. A clone with real unique
work is kept. Workspaces whose changes exceed the upload limits still
move, because the transfer is local.
**Steps to reproduce**
1. Use a project repository whose `.gitignore` (or the clone's
`info/exclude`) contains `.paperclip/`, with a release tag on a commit
that is not on any branch.
2. Re-home an isolated workspace of that repository with #17369.
3. The old worktree is removed, the empty `.paperclip/worktrees`
directory stays, and `git ls-files --others --ignored --directory` lists
`.paperclip/`. The clone is kept with `clone_checkout_has_files`.
Without the ignore line, the local release tag keeps it with
`unique_clone_history`.
4. Add a file larger than 64 MiB to a workspace. The move skips with
`restore_point_too_large`.
**Paperclip version or commit:** `master` at aa0037a137 (rebased on
ee29e2850d, after #17407).
**Deployment mode:** any.
**Agent adapter(s) involved:** not adapter-specific (core).
## What Changed
- New clone reproducibility check (`cloneDisposable` in
`server/src/services/isolated-repository-rehome.ts`), used before
deleting a re-homed clone:
- `git worktree prune`, then only the clone's own worktree may remain
(`base_has_worktrees`);
- no tracked changes and no untracked files
(`clone_checkout_has_files`); the only untracked exception is
Paperclip's compaction trash;
- ignored files only inside regenerable directories (`node_modules`,
`dist`, ...) or Paperclip's compaction trash. Ignored entries are listed
with `--no-empty-directory`, and a non-regenerable ignored directory is
expanded file by file, so empty directories such as the old worktree's
`.paperclip/worktrees` parent no longer count;
- only `refs/heads`, `refs/tags` and `refs/remotes` (no stash, notes or
replacement refs: `unsupported_clone_refs`);
- every commit reachable from local branches, local tags, non-remote
reflogs, `ORIG_HEAD` and `HEAD` (so a detached checkout counts) must be
on a remote-tracking ref, present in the shared clone, or on the remote.
The remote check reuses `hasCommitsMissingUpstream` (now exported),
which fetches the remote's branches and tags into a temporary store, and
it runs only when some commit is in neither place. A local tag is
accepted when its commit is on the remote. Otherwise the clone is kept
(`unique_clone_history`), or kept with
`managed_checkout_upstream_unavailable` when the remote cannot be read;
- existing 5,000 budgets for refs, reflog entries and unique commits
(`clone_history_budget`), plus at most 1,000 ignored directories outside
regenerable ones, read in one Git call (`clone_file_budget`);
- the log entry now includes `cloneKeptPaths`: up to five paths,
worktrees, refs or commits that kept the clone.
- Cleanup also checks that the clone's `origin` still matches the
workspace `repoUrl` before deleting it.
- Old worktree ownership (#4): it is proven by `git rev-parse
--git-common-dir`, or, when that fails, by the worktree's `.git` pointer
into `<clone>/.git/worktrees/`.
- Orphan sweep (`sweepOrphanedIsolatedRepositorySources`): at the first
reaper rotation after server start (feature on), it walks
`isolated-repository-sources/<company>/<issue>/checkout`. It holds the
isolated source lock and skips clones that a workspace still needs (an
open workspace with matching `isolatedRemoteSource`, a closed one that
is not expired with its worktree removed, a project workspace or a
running service), clones with a pending move, and clones whose re-homed
workspace still has its own cleanup pending. The repository URL comes
only from the database: the issue's workspaces must record exactly one
URL and the clone's `origin` must equal it (`unproven_repository` or
`source_identity_changed` otherwise), so a rewritten `origin` is never
fetched. It then applies the same check (using the shared clone of a
workspace of the same issue and repository when one exists), renames to
trash and deletes. It logs `event:
"execution_workspace.isolated_source_orphan_cleanup"` and removes
leftover trash directories from interrupted removals. It inspects at
most eight issue directories per reaper sweep and resumes from the saved
position in the next sweep, so slow remotes cannot hold up other
cleanup. When a full walk kept any orphan, it registers a deadline one
day later in the reaper's existing deadline set. Otherwise it registers
nothing until the next server start. No new timer.
- Restore point limits for re-home (#3):
`ISOLATED_REHOME_RESTORE_POINT_LIMITS` (500,000 files, 1 GiB per file
and in total, 1.5 GiB bundle, 1,000,000 commits) replaces the upload
defaults for this local transfer. Verification is unchanged (exact
snapshot equality). In `git-restore-point.ts`, `applyRestorePoint` now
streams `git diff --binary` into `git apply` instead of buffering it
(and skips the pipe when there is no diff, so no new Git version is
needed). On a failure it kills both processes and waits for both. The
ignored-file read sizes its buffer from the tree instead of the default
limit.
- Docs: `doc/recoverable-disk-cleanup.md`.
### What I found for `source_identity_changed` (#4)
In #17369, cleanup ran `git rev-parse --git-common-dir` in the old
worktree and used `""` when the command failed. Then it compared that to
the clone's `.git` path, so any Git failure in the old worktree was
reported as `source_identity_changed`. Git fails there when the old
worktree's registration in the clone (`<clone>/.git/worktrees/<name>`)
is gone while the directory still exists. Its `.git` file then points to
a missing directory. I reproduced this by deleting the registration
after a re-home: the old code keeps the clone with
`source_identity_changed`, the new code removes the old worktree and the
clone. I could not inspect the instance, so the exact event that removed
that one registration is not confirmed. The check was a false positive
in that case, and the fix proves ownership from the `.git` pointer
instead. A worktree whose pointer leads outside the clone is still kept.
## Verification
- `pnpm exec vitest run
src/__tests__/isolated-repository-rehome.test.ts` in `server/`: 28
passed (real Git repositories, embedded Postgres). New cases:
- a clean clone whose only extra history is a remote branch and a
release tag on a commit outside any branch, both absent from the shared
clone, with `.paperclip/` ignored in the clone: deleted;
- a clone with a local branch commit that is neither upstream nor in the
shared clone: kept (`unique_clone_history`, the commit is in
`cloneKeptPaths`);
- a clone with a tracked change, then an ignored non-regenerable file
next to an ignored `node_modules`, then an untracked `build/notes.txt`:
each kept (`clone_checkout_has_files` with the path), then deleted after
cleanup;
- a clone with a second registered worktree: kept
(`base_has_worktrees`);
- a clone whose detached `HEAD` holds the only copy of a commit (reflogs
expired): kept (`unique_clone_history`);
- an old worktree whose registration was pruned: old worktree and clone
removed;
- an orphan whose `origin` was rewritten is kept with
`source_identity_changed`, and with `unproven_repository` once no
workspace records its repository;
- 1,001 ignored directories keep a clone (`clone_file_budget`);
- orphan sweep: a batch of one returns a resume position and the next
batch finishes the walk; a dry run changes nothing. Then an orphan whose
history is in the shared clone or on the remote is deleted, and an
orphan with no workspace rows and an unpushed commit is kept, with the
next deadline one day later;
- a workspace with 65 MiB untracked and ignored files (default capture
refuses them with `restore_point_too_large`) moves with an exact
snapshot match.
- After rebasing on #17407 (nested clone migration, which reuses helpers
from the re-home module): `isolated-repository-rehome`,
`nested-clone-migration`, `execution-workspaces-service`,
`execution-workspaces-routes`, `execution-workspace-reopen`,
`execution-workspace-compaction`, `execution-workspace-policy`,
`execution-workspace-runtime-control-conflict`,
`execution-workspace-runtime-lease-route`, `git-workspace-expiration`,
`git-restore-point`, `git-restore-point-expiration`: 405 passed;
typecheck passes.
- Related suites in `server/` (before the rebase):
`git-workspace-expiration`, `git-restore-point`,
`git-restore-point-expiration`, `execution-workspace-reopen`,
`execution-workspaces-service`, `execution-workspace-compaction`,
`heartbeat-git-restore-point-resume`, `heartbeat-project-repositories`,
`git-reproducible-managed-checkouts`,
`isolated-remote-workspace-source`, with the new suite: 398 passed.
- `pnpm run typecheck` in `server/` passes.
- Mutation checks, each failing a test:
- remote proof replaced by "always unique": the unfetched-history and
orphan tests fail;
- remote proof replaced by "never unique": the local-commit and orphan
tests fail;
- `.git` pointer fallback removed: the pruned-registration test fails;
- re-home limits removed: the large-file test fails;
- orphan sweep removed from the reaper: the orphan test fails;
- worktree count check removed: the second-worktree test fails;
- regenerable-directory exception applied to untracked files: the
dirty-checkout test fails;
- orphan batch limit removed: the orphan test fails;
- orphan URL taken from the clone again: the rewritten-origin test
fails;
- ignored directory budget removed: the many-directory test fails;
- `HEAD` left out of the checked commits: the detached-HEAD test fails.
- Dropping only `--no-empty-directory` does not fail a test, because the
per-directory expansion also accepts an empty ignored directory. The two
mechanisms are redundant on purpose.
- Not run: the full server suite, or the change on a live instance.
## Risks
- Deleting a clone now depends on reading the remote when some local
commit is neither on a remote-tracking ref nor in the shared clone. If
the remote cannot be read, the clone is kept.
- Commits that only remote-tracking refs hold (for example a remote
branch deleted upstream after the clone fetched it) are treated as not
unique, as before. They were pushed once.
- The orphan sweep deletes clones that no workspace refers to. A closed
workspace that is expired with its worktree removed does not count as a
user, because reopening it rebuilds from the shared clone or clones
again.
- Re-home transfers can now use up to about 1.5 GiB of memory for a
single restore point payload, so a very large workspace uses more memory
during its move than before.
- The orphan sweep runs once per server start and then only while it
keeps an orphan, so a clone that becomes orphaned later waits for the
next restart.
- No migration. No schema change.
## Model Used
Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code
with tool use (shell, file editing, Git, test runs).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agent environments can leave pending teardown or unverified provider
resources.
> - Recovery must preserve those resources until ownership and workspace
safety allow cleanup.
> - Empty maintenance scans keep an idle instance and its database
active.
> - The existing rows, transaction signals, and scheduler can represent
the actual recovery obligation.
> - This pull request schedules pending cleanup and historical
verification only while work remains.
> - An empty or fully verified instance stops these scans without losing
cleanup recovery.
## Linked Issues or Issue Description
**What happened?**
The server scanned pending environment cleanup on its maintenance tick.
The run reaper also scanned it even when it recovered no runs or active
leases. Historical resource verification scanned every five minutes,
including when all history already had valid cleanup proof.
**What did you expect?**
Writes should wake cleanup. Pending rows, deferred orphan buffers, and
unverified history should retain recovery deadlines. When none remain,
these workers should make no periodic database or provider calls.
**Steps to reproduce**
Start a configured instance with no pending leases and fully verified
history. Leave it idle. Observe repeated pending-cleanup and
historical-verification reads from the maintenance paths.
Related: #17106 supplies the preceding login-cleanup migration. #13523
improves lease-reaper query indexes. #16809 addresses reusable-sandbox
recovery authority. This PR keeps those authority checks in their
existing services.
## What Changed
- Register two workers with the existing delivery coordinator and shared
scheduler. Remove their unconditional maintenance-tick calls. Run the
reaper's inline pending cleanup only after it recovers actual work.
- Add commit-aware signals to the audited lease and custom-image
writers. Existing transactions notify directly; other writers use one
small transaction helper. No tables or migrations are added.
- Derive pending cleanup deadlines from the existing row backoff and
retry or claim metadata. Retain a historical retry only while an
unverified candidate exists.
- Register unwritten orphan buffers across runtimes that share the
database handle. Buffer registration wakes recovery without a database
connection. Load the existing spool at startup and retry failed or
capacity-limited loads while needed.
- Isolate spool failures per runtime. Attempt all buffers and persisted
pending rows, then retain a retry for any failed spool.
- Document producer coverage, idle behavior, and remaining blockers.
Update the sleep inventory.
- Repair the inherited CSS-recovery E2E test: the current build no
longer emits task-page CSS. Exercise the actual lazy environment
stylesheet, preserving real network failure, recovery, and bounded
reload assertions.
## Verification
October 24 integration update: merged current master and resolved the
runtime/spool conflicts. Preserve serialized ingestion of records
appended after startup. Plugin replacement failures now register
deferred recovery with an idle hold; concurrent callers share one
persistence attempt. The producer audit includes this new writer.
- Focused cleanup selection: 270 tests now pass across five suites. The
initial run lacked a built plugin SDK and executed no tests; building
the SDK fixed setup. The new concurrent-flush assertion then reproduced
duplicate rows; a single-flight guard fixed it and all 18 plugin
integration tests passed on rerun.
- Server typecheck passed. After review fixes, all 211 tests across the
cleanup worker, real orphan spool, environment runtime, and plugin
integration suites passed. These overlap the earlier focused selection.
The rate-limit test initially expected a write after twenty idle minutes
to wait; corrected it to assert that subsequent writes cannot accelerate
the five-minute cadence.
- Review fixes preserve retries for unreadable spool
directories/records, keep historical provider checks off the startup
wait while joining them at shutdown, and restore their independent
five-minute minimum spacing. The complete [CI
matrix](https://github.com/paperclipai/paperclip/actions/runs/40108905033)
passed on final commit `79ae1589c3c7b924db4bedb337d0c7ed3d95020b`,
including all E2E shards, server/workspace tests, runner checks,
typecheck, build/public-install verification, and security checks.
- CI found a stale E2E asset matcher: `issue-detail-*.css` is no longer
emitted. Both corrected browser tests passed against a fresh production
UI build and disposable local instance. Fresh Greptile review is 5/5 on
that same final commit, with all review threads resolved.
Previous implementation verification:
- `pnpm -r typecheck` and `pnpm build` passed.
- New PostgreSQL worker suite: 12 tests passed. Covers quiet configured
state, verified history, writes, outer commit and rollback, retry
deadlines, restart, drain, deferred orphan buffers, spool failure,
pagination, provider uncertainty, shutdown, and the idle run-reaper
path.
- Worker and environment runtime suites after review fix: 183 tests
passed. A new assertion reproduced buffer starvation before the fix.
Updated transaction fault injection and isolated simulated-instance
database handles. Server typecheck also passed after the fix.
- Run-recovery suites: 58 tests passed. Pending-cleanup suite: 38 tests
passed after updating the logging fixture to seed real orphan recovery;
an idle reaper no longer invokes cleanup.
- Stop and workspace-recovery integration selection: 268 tests passed;
two live-provider tests skipped by their environment gate. This overlaps
the worker suite and is not an additional unique-test count.
- Started the full local `pnpm test:run`, then stopped that older
duplicate run after CI found the fixture mismatch. The corrected 38-test
suite passed locally. The full sharded [CI
matrix](https://github.com/paperclipai/paperclip/actions/runs/39678174479)
passed on final commit `fc0ff3c4554ca4190a8b0b81ec06ef7065b8ad2c`,
including build, typecheck, runner, server, workspace, and E2E checks.
This is not a claim that the local full run completed.
## Risks
- Missing a future producer signal could delay discovery until restart.
The producer audit is documented in `doc/idle-environment-cleanup.md`.
Independent SQL writers and another server process are outside the
existing process-local notification contract.
- Historical candidates blocked by ownership or provider uncertainty
keep a five-minute retry until they can be verified. Provider
destruction, claim fencing, Stop retention, and workspace safety remain
controlled by their existing services.
- This does not remove orphan-active discovery or legacy shutdown
settlement from run recovery. Active custom-image expiry is also still
periodic. External wake coverage does not change.
- Local tests prove these workers become quiet and recover. They do not
establish deployed instance suspension or fleet cost savings.
## Model Used
OpenAI Codex (GPT-6), with code execution and reasoning. The exact
serving model ID and context size were not exposed.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Product E2E campaigns retain a request ledger and send native traces
and gateway logs to Braintrust.
> - Native trace verification did not prove that each charged gateway
request had a stored log row.
> - A hosted probe showed that `x-bt-span-id` identifies the row `id`,
while the stored `span_id` differs. Propagated campaign metadata was
absent.
> - This pull request joins retained request receipts to the correct
Braintrust rows during scheduled finalization.
> - Operators can see missing request coverage without changing costs,
releasing holds, or rerunning agents.
## Linked Issues or Issue Description
Refs #17354 and #17371.
**Describe the bug**
A campaign could pass native trace and experiment indexing checks while
gateway request logs remained unverified. The runbook also claimed that
propagated metadata made a request searchable after cancellation before
response headers. The hosted gateway did not preserve that metadata in
retained evidence.
**Steps to reproduce**
Inspect a retained gateway response header and query project logs by its
`x-bt-span-id`. The matching record uses that value as `id`, not
`span_id`. Compare every model reservation with its saved response
receipt, including setup and canceled work.
**Expected behavior**
Verify each request against the stored row and report absent or
conflicting evidence. Keep trace coverage separate from final usage and
invoice reconciliation.
## What Changed
- Added a bounded, read-only gateway index verifier. It matches row IDs,
provider and model, and retains the actual span and trace IDs.
- Included setup, failed and canceled requests in coverage. Missing
receipts or row IDs, duplicate attribution, conflicting records and
evidence changes prevent verification.
- Added a separate scheduled gateway coverage gate and private receipt.
Native trace and experiment checks remain independent.
- Shared the existing bounded Braintrust query reader. Queries select
only identity fields, with at most 100 IDs per batch, three rounds and a
45-second deadline.
- Corrected the runbook's metadata guarantee and documented the
historical `gatewaySpanId` receipt field.
## Verification
- `pnpm test:e2e:runner:unit`: 4,120 passed, two existing skips; all 137
Node contracts passed.
- Final repair: 54 focused tests passed, including
scheduled-finalization coverage and all five optional identity fields.
Each conflicting-field regression failed before the fix; absent or
matching partial metadata remains supported.
- `pnpm test:e2e:runner:typecheck`: passed after rebuilding stale local
plugin SDK declarations. The initial stale-declaration failure remains
recorded in the validation logs.
- `git diff --check`: passed.
- Read-only live verification of retained evidence: all ten local Codex
requests and the one canceled drain-probe request matched stored rows.
The hosted Claude campaign matched eight of ten; its two pre-header
cancellations correctly remained incomplete.
- The first hosted lookup exceeded the 1 MiB read limit when selecting
full metadata. The final identity-only projection passed the same
lookup.
- Original journals stayed byte-identical. These checks made zero model
calls and did not change provider data or release holds.
- Full repository CI passed on
`957ad0b684b80267d230fc4fb56d4f002ef089ad`: [run
40105676630](https://github.com/paperclipai/paperclip/actions/runs/40105676630),
including typecheck, tests and build. Both current-head automated
reviews scored 5/5 with no open findings.
- The superseded initial CI run was canceled after the repair push to
release the queue; its completed results remain available. No failed job
was retried on the final commit.
## Risks
Missing gateway IDs now fail scheduled trace coverage. A failed or
canceled call may have no stored row, so this can require investigation
even if product assertions pass. Indexing can lag; bounded query retries
do not retry model work. Stored identity does not prove final billing or
gateway span nesting under native attempt traces. Private identifiers
remain in private evidence only. Recurrence and billing settings are
unchanged.
## Model Used
OpenAI Codex, GPT-6 family. The runtime does not expose a more specific
model ID or context-window size in this session. Used reasoning,
repository tools, local code execution and read-only API verification.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - People use New task to send work to an agent.
> - The board currently downloads the task dialog code after the first
click.
> - This shows a loading window before people can enter their request.
> - The form already loads agent and project choices after it opens.
> - This pull request loads the task dialog code with the board.
> - People can start writing immediately and keep their text as choices
arrive.
## Linked Issues or Issue Description
Related change: #16849. Commit
`fa50cab5685b9cdbb26ad9c610cc5b6c373d3ff4` added the deferred dialog and
its loading window on October 19, 2026.
**What happened?**
The first New task click shows “Loading new task…” while the browser
downloads the dialog code.
**Expected behavior**
The form opens on click. Agent and project choices load in place. Text
entered during those requests stays in the form.
**Steps to reproduce**
1. Open the board in a fresh browser session.
2. Select New task before its code is cached.
3. Observe the loading window before the form appears.
**Paperclip version or commit**
Source baseline: `8b4ab65b1c3d863d282752e83307943720a5c85c`.
**Deployment mode**
The browser code applies to local and hosted boards. Verification used a
source checkout.
## What Changed
- Import NewIssueDialog with the board. Keep the existing mount and
draft lifecycle.
- Add a first-open test that fails if the form waits for an asynchronous
render.
- Cover text entry during delayed agent and project requests and
defaults that arrive later.
- Document the behavior and its verification recipe.
## Verification
- Regression proof: the first-open test failed before the import change
and passed after it.
- All 125 tests in BoardDialogs, DeferredOverlay, NewIssueDialog and
Layout passed.
- The production UI build and UI TypeScript check passed. The typecheck
passed with `NODE_OPTIONS=--max-old-space-size=2400` after the competing
test process was stopped.
- Token gates, map validation and Git diff checks passed.
- The first UI typecheck could not resolve the unbuilt local plugin SDK.
The SDK was then built.
- Full typecheck and build attempts hit the sandbox's 4 GB memory limit
when run with the full test suite. Two later UI typecheck attempts also
hit that limit during other validation. The cgroup recorded four OOM
kills in total. The full repository test attempt and a broader UI test
attempt were stopped. No complete suite result is claimed. Serial
repository typecheck reached the missing Rust `cargo` executable. Full
CI passed on head `9dde4bc`; local limits remain recorded here.
- First CI run at `e5a99b8`: typecheck, build, all unit-test shards and
seven browser shards passed. Browser shard 2 failed at
`chat-adapters-ui-providers.spec.ts:185`, waiting for Slack setup
Success after reload. The saved page showed the working connection and
Continue button. The cause is not confirmed; this failure is retained
for follow-up. [Failed
job](https://github.com/paperclipai/paperclip/actions/runs/39522214527/job/118968542923).
- Removed the added import comment after Greptile review. Head `9dde4bc`
changes only that comment. The review thread is resolved. Fresh CI
passed on head `9dde4bc`. Greptile reviewed that exact head at 5/5 with
no open findings. All eight browser shards passed, including the earlier
Slack failure. The earlier failure remains unexplained; the only
intervening code change removed a source comment. A passing retry
establishes recovery, not its cause.
- Component tests use mocked directory requests. No deployed latency or
browser persistence claim is made.
Latest CI: [run
39523376437](https://github.com/paperclipai/paperclip/actions/runs/39523376437).
Reviewed head: `9dde4bc0caa3556e315a2fc197268b7c9879efa4`.
## Risks
- Task dialog code is part of board startup again. This adds its
dependencies to the initial code graph. No startup latency benchmark is
claimed.
- Other creation dialogs retain their existing lazy imports and loading
behavior.
## Model Used
OpenAI Codex, GPT-6, with reasoning, repository tools and code
execution. The runtime does not expose a more precise model ID or
context-window size.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Existing Claude and Codex agents can upgrade to Paperclip Runner.
> - A Daytona computer needs platform qualification before Runner
becomes its default.
> - The upgrade notice required that default, so it hid the action that
could start qualification.
> - Saved Claude turn limits also caused conversion to fail even though
Runner can run without them.
> - This pull request verifies the computer on click and ignores
obsolete Claude options with a warning.
> - Agents retain their model, account, environment, instructions, and
permission policy.
## Linked Issues or Issue Description
Related: #17122.
**What happened?**
A legacy Claude agent on a Daytona snapshot had no upgrade button until
its environment had platform qualification. A saved `maxTurnsPerRun:
1000` then caused conversion to return HTTP 422.
**Expected behavior**
The upgrade action verifies a supported Daytona computer and converts
the agent. Unsupported tuning options produce a visible warning and do
not block execution.
**Steps to reproduce**
1. Create a legacy Claude agent with `maxTurnsPerRun: 1000` on an active
Daytona snapshot.
2. Open Runtime before the environment has platform qualification.
3. Try to upgrade to Paperclip Runner.
**Paperclip version or commit**
After #17122, reproduced against `9c442cf48e`.
## What Changed
- Offer upgrade on probe-capable Daytona snapshots and immutable sandbox
images. Perform the public environment probe only after a click. Save
after fresh adapter discovery confirms support.
- Resolve the actual computer for explicit, instance-default, and hidden
managed environments. Preserve pending edits and reject stale probe
results.
- Ignore Claude `maxTurnsPerRun`, `chrome`, `warmHandleIdleMs`, and
`acpWarmHandleIdleMs`. Return structured warnings from create, hire, and
update. Show warnings in Runtime and after a save.
- Keep validation for commands, credentials, filesystem, network, and
permission settings.
- Retain the legacy turn limit in the existing live conversion fixture.
Update regression tests, upgrade stories, and capability maps.
## Verification
- Red: the old route returns HTTP 422 for the saved turn limit; the old
form hides the Daytona upgrade notice.
- Green: 417 focused server and form tests pass. A further 188 tests
pass after the review fixes. Qualification coverage includes failed
probes, unsupported targets, changed settings, managed defaults, unsaved
environment variables, and forced Kubernetes targets.
- `pnpm -r typecheck` and `pnpm build` pass. Token gates, map checks,
map tests, and Storybook catalog checks pass.
- Real local Claude Opus 5.5 conversion and task pass, with the
downloaded artifact checked against seeded source bytes, SHA-256, and
Git commit.
- Real browser upgrade and full Claude Opus 5.5 task on a disposable
named Daytona snapshot pass at `b81cec7d8c`. The saved agent retains its
model, high effort, credentials, timeout, and grace settings while
dropping the turn limit with a visible warning. Exactly one successful
run completed the task. The downloaded artifact matched independently
seeded source bytes, SHA-256, and Git commit; the source was unchanged,
no interactions remained pending, and the delivery link survived a
browser reload. The snapshot used current runtime assets built from
public source. The supplied Daytona account could not access the
original staging snapshot, so this verifies an equivalent local fixture
rather than that exact snapshot. Disposable test resources were cleaned
up.
- Earlier live attempts remain recorded: an image-based sandbox creation
failed, and the first named-snapshot probe returned failure with
unresolved provisional cleanup identity. Later named-snapshot probing
and the task succeeded; the initial creation failures are not attributed
to this PR or declared resolved. A separate attempt using the Oct 18
cached runtime failed the current provider-pack compatibility check;
rebuilding the runtime from matching public source removed that
mismatch. Owned resources were identified and cleaned up, with
diagnostic databases retained.
- 54 checks passed at `b81cec7d8c`, with no failed checks; Greptile and
Paperclip Review both scored 5/5, with both review threads resolved.
Master advanced during the live proof, so the branch was rebased with
both sides of additive import/test conflicts retained. On the rebased
head `2b74222b94`, 472 focused tests, UI typecheck, and UI build pass.
All required CI gates pass; Greptile and Paperclip Review both score 5/5
on this head, with no unresolved review threads. The PR is mergeable.
The serial local `pnpm test:run` was stopped after the CI test lanes
passed; that local run is not claimed as a pass.
## Risks
- An explicit upgrade can allocate a temporary probe sandbox and take
several minutes. Probe failures leave the agent on its existing runner.
- Ignored options no longer enforce a turn cap, Chrome integration, or
legacy idle-time tuning. The UI warns about each active ignored option.
- Unsafe or ambiguous execution settings still fail validation. This
change does not weaken environment qualification or runtime artifact
checks.
## Model Used
OpenAI Codex, GPT-6, with reasoning, code execution, and browser tools.
The exact deployment model ID and context-window size are not exposed in
this session.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Product E2E checks real provider behavior under a campaign budget.
> - Its request and trace limits also determine the operating-cost
forecast.
> - The initial limits greatly exceed the measured marker and feedback
cases.
> - This pull request lowers those defaults while keeping the same
selected cases and full model context limits.
> - The benefit is a smaller bounded resource envelope with explicit
quota failures and readable historical evidence.
## Linked Issues or Issue Description
Refs: #17365
Refs: #17371
**What existing behavior does this improve?**
The resource ceilings for bounded Product E2E campaigns and their OTLP
relay.
**Current behavior**
Daily and weekly campaigns allow 60 and 120 model requests. The native
relay defaults to 256 MiB of expanded source and 16,384 records per
attempt. Both October 23 two-case qualification campaigns used ten model
requests. The largest native attempt used 6,531,801 expanded source
bytes and 2,455 records.
**Proposed behavior**
Use 30 daily and 60 weekly model requests. Default native exports to 16
MiB of expanded source and 4,096 records. Preserve the older explicit
relay maxima so prior receipts remain valid and deliberate controller
overrides still work.
**Reason and benefit**
The smaller limits leave headroom for the measured cases and reduce the
maximum source traffic in the operating forecast. They do not change the
daily or weekly dollar cap, model context ceiling, per-request
reservation, selected cases, or attempt policy.
**Breaking changes**
A new bounded attempt that exceeds the smaller default trace allowance
receives a quota error. It cannot silently omit required evidence.
Existing receipts remain readable. No paid recurrence is enabled by this
change.
## What Changed
- Set scheduled model request caps to 30 daily and 60 weekly.
- Add smaller native relay defaults while preserving the existing
explicit maximum and historical receipt validation.
- Exercise default count and expanded-source exhaustion through the HTTP
relay. Keep a regression for larger historical receipt policies.
- Document the measured basis, default capacities, and failure behavior.
## Verification
- `pnpm test:e2e:runner:unit`: 4,095 Vitest tests passed, two existing
skips; 137 Node contract tests passed.
- `pnpm test:e2e:runner:typecheck`: passed.
- `git diff --check`: passed.
- The relay test accepts two 3-MiB log payloads, rejects the third
before upstream dispatch, and retains its quota receipt.
- Read-only replay of the prior real hosted campaign preserved all five
ingestion receipts and 123,044 payload bytes exactly. Its accounting
journal remained byte-for-byte unchanged; zero provider calls.
- Full repository typecheck, tests, build and reviews are pending hosted
CI. No paid model call was made for this change. Live qualification of
the smaller defaults remains open.
## Risks
- More verbose future cases may exhaust the new defaults. Such a failure
remains visible and requires evidence before increasing a limit.
- Source-byte ceilings are not Braintrust invoice units. Operating-cost
qualification and reconciliation remain separate requirements.
- Manual controller overrides can still use the earlier larger maxima;
their cost audit must account for the limits actually recorded in the
receipt.
## Model Used
OpenAI Codex, GPT-6 family, with reasoning, code execution and tool use.
The host does not expose an exact model build identifier or context
window size for this session.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The server keeps attachments and other files in a storage provider.
Hosted instances use the S3 provider.
> - Sentry received `SignatureDoesNotMatch` errors from the S3 provider
on two staging hosts.
> - The events have only AWS SDK stack frames. They do not show which S3
call failed, the HTTP status, or the S3 request ID.
> - Without these facts, we cannot find the cause or make a safe fix.
> - This pull request records a small, fixed-shape failure record on the
S3 error and sends it with the Sentry event.
> - The benefit is that the next event shows the failed operation and
the provider request ID, and no private data goes to Sentry.
## Linked Issues or Issue Description
**What happened?**
The S3 storage provider threw `SignatureDoesNotMatch` four times on
staging. The Sentry events do not identify the S3 operation or the
caller.
**Expected behavior**
A storage provider failure in Sentry shows which storage call failed and
the provider's correlation fields.
**Steps to reproduce**
1. Configure the S3 storage provider against an endpoint that rejects
the request signature.
2. Read, write, check, or delete a stored file.
3. Look at the Sentry event. It has no first-party stack frame and no
operation name.
No public GitHub issue exists. I searched open and closed pull requests
for `SignatureDoesNotMatch` and storage failure diagnostics. No pull
request covers this.
## What Changed
- Add `server/src/storage/failure-diagnostics.ts`. It keeps a frozen,
fixed-shape record for each failed storage error in a `WeakMap`.
- Add `server/src/storage/s3-failure.ts`. It wraps each S3 call and
records the operation, error code, HTTP status, request ID, and SDK
attempt count. It reads only data properties. A value that does not
match its bounded format becomes `unknown` or `null`.
- Wrap `PutObject`, `GetObject`, `HeadObject`, `DeleteObject`,
`CreateMultipartUpload`, `UploadPart`, and `CompleteMultipartUpload`.
- In `captureException`, add the `storage_operation` and
`storage_error_code` tags and a `storage_failure` context to the event
of that error only. The original error and its default grouping do not
change.
- Document the new context in `doc/observability.md`.
## Verification
- `pnpm exec vitest run
src/__tests__/storage-s3-failure-diagnostics.test.ts
src/__tests__/sentry.test.ts` (in `server/`): 37 passed, 5 skipped. The
skipped tests are the existing real-SDK tests, which need an environment
flag.
- `pnpm exec vitest run src/__tests__/storage-s3-multipart.test.ts
src/__tests__/storage-local-provider.test.ts`: 8 passed.
- The new tests use a local HTTP server that returns a real S3
`SignatureDoesNotMatch` XML body with a `StringToSign` field. The tests
make sure that the record has the operation, code, status and request
ID, and that it does not contain the private string.
- I typechecked the changed source files with a scoped `tsc` project.
They have no errors. I could not run the full `pnpm -r typecheck` and
`pnpm build` in my sandbox: it has a 4 GiB memory limit and no `cargo`
for the runner binary. CI runs the full checks.
## Risks
- Low risk. The change adds a `try`/`catch` that records data and throws
the same error again. Storage behavior, not-found handling, and HTTP
responses do not change.
- Secret-sensitive routes replace the error before reporting, so those
events do not get the record. This is intended.
- The record never contains the bucket, object key, endpoint,
credentials, error message, or provider response body.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic. Used through Claude
Code with tool use (shell, file edit) and extended reasoning.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Before a run starts, Paperclip prepares the execution workspace (for
example `git worktree add`). Each step is a workspace operation. The
workspace operation recorder writes it as a row and as a log file on
disk.
> - When an operation fails, the recorder writes the error to the log
file, then sets the row to `failed`.
> - If that log write also fails (for example ENOSPC when the disk is
full), the recorder throws before it updates the row.
> - The row stays `running` with no `finishedAt`. The log write error
also replaces the real error of the operation.
> - This pull request makes the failure-path log write best-effort, so
the row always reaches `failed` and callers get the correct error.
> - The benefit: on a full disk, the operator sees a terminal operation
with the real failure reason. Operations do not stay "running" for ever.
## Linked Issues or Issue Description
No public issue exists. Bug description:
**What happened?**
A run failed during workspace setup on a host with a full disk (`ENOSPC:
no space left on device, write`). The stack shows the error in the
workspace operation log append inside `recordOperation`. In the failure
path, the recorder calls the log append before it updates the row.
Because the append threw again, the row stayed `running` with no
`finishedAt`. When the operation itself failed (for example a git
error), callers got the ENOSPC log error and not the git error.
**Expected behavior**
A failed workspace operation always reaches `failed` with `finishedAt`
set. Callers get the operation's own error. A log write failure must not
hide that error.
**Steps to reproduce**
1. Make the operation log append throw ENOSPC (the new tests stub
`fs.appendFile` for the log root).
2. Record an operation whose `run` throws, or whose output append
throws.
3. Before this fix: the row stays `running`, and the caller gets the
ENOSPC error and not the operation's error.
**Paperclip version or commit**
`ee29e28` (current `master`).
**Agent adapter(s) involved**
Not adapter-specific (core bug).
Related open PRs with disk-full handling in other places (no overlap
with this code path): #5882 (run-log rotation), #16773 (backup disk
use).
## What Changed
- `server/src/services/workspace-operations.ts`: in the
`recordOperation` failure path, the stderr log append is now best-effort
(`.catch(() => undefined)`). The stderr excerpt still records the
message, because the excerpt updates before the file write.
`logStore.finalize` was already best-effort in this path.
- `server/src/__tests__/workspace-operations-reconciliation.test.ts`:
two regression tests with log appends that fail with ENOSPC:
- The output append fails after a successful run. The row reaches
`failed` with `finishedAt` and the ENOSPC message in the stderr excerpt.
The caller gets the ENOSPC error.
- The operation itself throws. The row reaches `failed` and the caller
gets the operation's own error, not the log write error.
## Verification
- Red before the fix: both new tests failed on `master`. The row stayed
`running`, and the caller got the ENOSPC error and not the git error.
- Green after the fix: `pnpm exec vitest run
src/__tests__/workspace-operations-reconciliation.test.ts
src/__tests__/workspace-runtime-control-recovery.test.ts` passes (17
tests).
- Sibling suites that use the recorder (26 files, 945 tests): all pass
except 4 tests in `workspace-runtime.test.ts`. Those 4 tests also fail
on unchanged `master` in the same sandbox, because they need the local
pnpm and worktree CLI. This change does not cause them.
- Typecheck: `tsc --noEmit` on `workspace-operations.ts` passes. The
full server `tsc` was not run locally because the 4 GB sandbox ran out
of memory. CI runs the full typecheck.
## Risks
- Low risk. The change affects only the failure path, and only the log
file write. The success path does not change: if a log write fails after
a successful run, the operation still fails with the write error.
- The log file for a failed operation can lose its last stderr line on a
full disk. The row's stderr excerpt still has the message.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edit, test execution).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Devin Foley <139239+devinfoley@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
