Commit Graph
5240 Commits
Author SHA1 Message Date
DottaandPaperclip efcc8a7d89 test: bound command snapshot forwarding checks (#16421)
Keep the command snapshot option test focused on forwarding and real file copyback. Use inline delegates in that test, and retain the separate real-worker suites and original deadlines.

Validation: 124 affected tests, full typecheck/build, exact-head CI/E2E/Superagent and Apex 5/5. Negative controls reject lost command and restore forwarding. Failed earlier merge-readiness evidence remains in the PR.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-15 06:57:38 -05:00
DottaandPaperclip 250429942e fix(agents): report actionable lifecycle failure steps (#16418)
## Thinking Path

> - Paperclip helps people manage AI agents for work.
> - New agents must pass setup and execution checks before they can run.
> - The lifecycle worker saved the same retry message for every failure.
> - Users could not identify the failed step or correct its
configuration.
> - The UI also described failed work as work still in progress.
> - This pull request names the failed step and gives a safe corrective
action.
> - Users can identify configuration problems and understand automatic
retries.

## Linked Issues or Issue Description

**What happened?**

A new hire showed a paused status and “The lifecycle step failed. Retry
the operation.” Repeated retries did not explain the failure. A
successful retry cleared the original error. This change cannot recover
an error that an older server already cleared.

**Expected behavior**

Name the failed setup step. Give a useful action for known execution
test failures. Explain automatic retry. Keep provider output and
credentials out of lifecycle errors.

**Steps to reproduce**

1. Create a disposable agent with an unavailable model or invalid
execution command.
2. Open its profile while verification fails.
3. Inspect the error, correct the configuration, and select Retry now.
4. Confirm that the failure disappears after the agent becomes ready.

Related change: https://github.com/paperclipai/paperclip/pull/16155
shows pending setup progress in creation and assignment. This change
improves the cause and action shown when setup fails.

## What Changed

- Save the failed lifecycle phase and step in the existing lifecycle
error field.
- Classify authentication, model, command, directory, timeout, and
response failures from execution test check codes.
- Use fixed error messages. Do not retain raw provider messages, check
details, or credentials.
- Share the allowed failure codes with the lifecycle log.
- Show failed-state labels and automatic retry guidance. Rename Retry to
Retry now.
- Extend worker, database, and UI regression coverage. Update the agent
verification maps.

## Verification

- Passed 105 focused tests across lifecycle diagnostics, database
lifecycle commands, saved harness verification, and
AgentLifecycleStatus.
- Passed server TypeScript compilation with `pnpm --filter
@paperclipai/server exec tsc --noEmit`.
- Passed UI typecheck and production build.
- Passed module boundary checks, token gates, character sync, map
validation, and `git diff --check`.
- Used Playwright to inspect the production component with a simulated
model failure. Confirmed the failure text, retry guidance, retry
callback, and recovered state. Captured a screenshot. This component
check does not prove a live provider run.
- Attempted `pnpm -r typecheck` and `pnpm build`. Both require Rust
cargo, which is not installed in the local execution environment.
- Started `pnpm test:run` locally. No final local suite result was
recorded before handoff. The complete PR CI workflow passed.
- Passed all latest-commit CI gates, including typecheck, build, general
and serialized tests, Rust checks, and all eight browser test shards:
https://github.com/paperclipai/paperclip/actions/runs/38726198774
- Greptile scored the latest commit 5/5 with no findings. There are no
review threads to resolve.

## Risks

- Failure classification depends on existing adapter check codes.
Unknown codes use a safe fallback message.
- A known code can describe only the checks reported by the adapter.
- No database migration is required. The error field remains a string.
- Retry schedules, leases, readiness, approvals, and budget controls
keep their current behavior.
- Full build verification requires a runner with Rust cargo.

## Model Used

- OpenAI GPT-6 through Codex. Used reasoning, code editing, browser
checks, and tool execution. The runtime does not expose the exact
deployment ID or context window.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-15 06:46:29 -05:00
DottaandPaperclip 7fc2dc8b32 fix: isolate workspace snapshots from controller work (#16414)
Move native sandbox snapshot captures off the controller callback queue while preserving the existing capture and restore contract. Keep legacy adapters inline, bound captures to two workers, and verify cleanup and reordered worker completion.

Validation: 461 affected cases, full typecheck/build, exact-head CI/E2E/Superagent and Apex 5/5. The full local canonical suite remains running; no full local pass or under-30-second staging response is claimed.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-15 06:31:59 -05:00
Devin Foley 8365872743 fix(codex-local): retry ACP model capacity failures as transient upstream (#16407)
Classify typed Codex ACP `service` failures about model capacity or server overload as `codex_transient_upstream` / `transient_upstream`, so the existing bounded transient retry and continuation recovery apply. In `unexpected` Sentry mode, skip only runs carrying that full classified evidence; `all` mode still reports them.
2026-10-15 04:28:02 -07:00
DottaandPaperclip 13e5128164 docs: preserve runtime and verification lessons (#16379)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Its control plane shares execution capacity across requests and
agent orchestration.
> - A dependency can return a small value while doing slow or repeated
work underneath.
> - Tests can check that value while mocks hide the behavior that
affects other work.
> - A later passing retry can also leave an earlier material failure
unexplained.
> - This pull request records general rules for dependency work,
verification, and acceptance.
> - The benefit is a durable review habit that applies beyond the runner
incident.

## Linked Issues or Issue Description

Refs #15683. Refs #16335.

The staging runner incident exposed a gap in how we establish the
operational cost of a dependency call. The original platform check read
a small field from a process report. The call could block shared
execution, and value-focused mocks did not exercise that behavior. The
fix replaced that probe with bounded asynchronous work. This proposal
addresses the reasoning and evidence gap, without assigning blame or
prescribing one API or timeout for future work.

The acceptance guidance also preserves material failures after a retry
passes. It does not assert that every earlier test failure had the same
cause as this incident.

Related proposal checked: #14275 covers release sign-off templates. This
PR covers implementation assumptions and verification evidence.

## What Changed

- Add a scoped rule for dependency work in server, adapter, and runner
execution paths. Cover blocking, repetition, cost growth, cancellation,
and the effect on unrelated work.
- Require evidence that can disprove the consequential assumption
introduced by a change. Identify behavior removed by mocks and exercise
the relevant boundary.
- Register the rule in the Greptile manifest with matching path scope.
Existing `AGENTS.md` instructions route coding agents to matching rules.
- Add failure preservation to the verification skill. Link it from the
repository-wide PR review rule. Keep recovery and resolution distinct.
- Include examples and exceptions so cosmetic edits and ordinary pure
transformations do not acquire unnecessary test requirements.

## Verification

- Passed `git diff --check` and the repository PR template and
duplicate-search validators.
- Passed a local Python structural check: manifest JSON parses; rule
globs and Greptile scopes match; seven representative runtime paths
match; five unrelated paths do not; the evidence link and heading
resolve; existing agent and reviewer loading instructions are present.
- Manually applied the rule to a network-filesystem configuration
getter, a repeated remote permission check, and an already-loaded map
lookup. The first two need evidence about their new dependency risk. The
last does not require fault testing by default.
- Attempted `pnpm -r typecheck`, `pnpm test:run`, and `pnpm build`. The
installed pnpm launcher could not fetch registry data to verify the
requested pnpm version. Using the cached project version reached
typecheck, which failed because workspace dependencies are absent
(`@types/node` missing). Tests and build remain unverified locally.
- Greptile reviewed commit `e56d1ec70f3ebffe697b35cc637975b8d8198681` at
5/5 with no unresolved review threads. CI is still running. These
structural checks establish instruction wiring, not future compliance by
an agent or reviewer.

## Risks

- The main risk is adding ceremony to low-risk edits. The rule limits
requests to concrete risks introduced by changed behavior and explicitly
exempts unrelated work.
- Instructions depend on contributors and reviewers reading and applying
them. This change adds no automated runtime regression gate.
- No application behavior, schema, dependency, lockfile, or workflow
changes.

## Model Used

OpenAI Codex, GPT-6 family. The exact serving model ID, context window,
and configured reasoning effort are not exposed in this session. Used
reasoning, repository inspection, shell tools, and patch editing.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details available in this session)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable (guidance-only
change; structural validation described above)
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-15 06:21:39 -05:00
Devin FoleyandPaperclip 72ae600742 test(server): reuse comment wakeup route modules (#16410)
## Thinking Path

> - Paperclip manages AI agents and their work.
> - Task routes decide which agent wakes after a comment or assignment
change.
> - The comment-wakeup suite reloads the large route module before each
test.
> - The worker exhausts its memory before all 34 cases finish.
> - This PR loads the module once and resets mock behavior between
cases.
> - The suite keeps every assertion and runs within the existing memory
limit.

## Linked Issues or Issue Description

Refs #16017. Related fixes: #16396 and #16404 cover other route suites.

The [failed serialized
shard](https://github.com/paperclipai/paperclip/actions/runs/38706528509/job/116333628474)
completes 26 of 34 cases before a 4 GiB heap OOM. It also logs
`runner.select is not a function` when real external-object sync reaches
the empty transaction fixture.

A duplicate search found no open PR for this suite's memory failure.

## What Changed

- Move module reset, mock registration, and initial route import into
`beforeAll`.
- Mock external-object sync at its direct service import.
- Reset mock implementations between cases, then restore the fixture
defaults.
- Keep all 34 test cases and every assertion unchanged.

## Verification

- Node 24.21.0, matching the failed CI job.
- Unchanged suite with `NODE_OPTIONS=--max-old-space-size=4096`: worker
receives SIGKILL after 31/34 cases in this 4 GiB container.
- Fixed suite under the same heap limit: 34/34 pass in 5.06 seconds;
peak child RSS is 654 MiB.
- Fixed suite with a 1 GiB heap and shuffled order (seed 394): 34/34
pass.
- Compared all test bodies and assertions against master; they are
byte-for-byte unchanged.
- `git diff --check` passes.
- Workspace typecheck reaches the Rust stage, then stops because this
container has no `cargo`.
- Master has a stale lockfile. Local setup used `pnpm install
--no-lockfile`; Vitest resolved to 5.0.4 and Supertest to 7.3.1.
- Local build stops because Node.js headers are missing for the
SO_PEERCRED addon.
- The local full-suite command was bounded to three minutes and did not
finish.
- [Greptile gives this exact head
5/5](https://github.com/paperclipai/paperclip/pull/16410#issuecomment-6188812731),
with no findings or review threads.
- [CI run 38712230280, attempt
2](https://github.com/paperclipai/paperclip/actions/runs/38712230280/attempts/2)
passed on this head. All nine serialized shards, general tests,
typecheck, build, Rust checks, and e2e checks passed.
- The first attempt hit an unrelated legacy-recovery timestamp flake.
Its [single retry
passed](https://github.com/paperclipai/paperclip/actions/runs/38712230280/job/116349967086).
- Current check inventory: 53 successful check runs, two skipped
Storybook checks, and a successful Snyk commit status.

Reproduce the focused check with:

```sh
NODE_OPTIONS=--max-old-space-size=4096 pnpm exec vitest run --project @paperclipai/server server/src/__tests__/issue-update-comment-wakeup-routes.test.ts --pool=forks --isolate
```


## Risks

Low risk. Only test setup changes. Production routes, workflows, and the
lockfile are unchanged. The shuffled run checks that mock behavior does
not leak between tests.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository tools, and code
execution. This runtime does not expose a more specific model ID or
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-15 02:55:49 -07:00
Devin FoleyandClaude Opus 5.5 11d69d04c7 fix(native-runtime): report remote recovery labels thrown before coordinator save (#16406)
Restart reattachment can fail in the executor before the native coordinator saves its result. The run failure report now reads the closed remote recovery stage and reason from the caught error when the saved result lacks them. The saved result keeps priority and the same closed-vocabulary validation applies.

Add a regression for thrown, wrapped-transport, saved-priority, and out-of-vocabulary cases. Full current-head CI and Greptile 5/5 passed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
canary/v2026.1015.0-canary.11
2026-10-15 02:30:58 -07:00
Devin FoleyandPaperclip 307c27e3d8 fix(native): reclaim task execution before same-run recovery
Restore the original task execution binding atomically during native same-run recovery. Preserve current owners, stops, pauses, assignments, and dependency waits.

Isolate candidate failures, bound deferral writes, prioritize undated retries, and allow recovery ahead of queued waiters without premature queue promotion. Add PostgreSQL attachment, ownership, contention, fairness, and queue regressions.

Verification: 99 focused tests, workspace typecheck, build, full current-head CI, and Apex 5/5.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
canary/v2026.1015.0-canary.10
2026-10-15 01:53:26 -07:00
Devin FoleyandPaperclip cd77c1c4dc test(server): reuse remaining serialized route modules
Load route modules once per suite while keeping every test case and assertion.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-15 01:25:57 -07:00
fd057f6ac4 fix(runner): keep OpenCode turns alive through oversized SSE events
Retain bounded OpenCode SSE events without restarting the event stream. Preserve canonical tool identity, status, output tails, original UTF-8 sizes, and truthful truncation markers. Bound id-less event deduplication with hashes.

Cover fragmented and complete large frames, exact-limit split separators, bulk input retention pressure, truncation-record exhaustion, and unpaired surrogate byte counts. Full current-head CI and Apex 5/5 passed. The unrelated local post-tool answer assertion was reproduced on unchanged baseline source.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Paperclip <noreply@paperclip.ing>
nightly/v2026.1015.0-nightly.0 canary/v2026.1015.0-canary.9
2026-10-15 01:02:28 -07:00
DottaandPaperclip af4191f45e fix: keep workspace notices from suppressing final replies (#16400)
Keep stored system notices visible without letting them occupy the final reply slot. Read presentation metadata in live and recovery selection, and cover explicit replies, progress receipts, company boundaries, and native replay.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-15 02:39:07 -05:00
8d9d2f9ddf fix(recovery): preserve deliberate stops and dependency waits
Preserve deliberate assignee blocks and operator stops during dependency and run-release recovery. Attribute native waits to their bound owner. Ignore unrelated edits and shared-workspace exports from another issue. Continue after genuine dependency changes while preserving independently authorized user input.

Keep the original contributor's implementation and review history. Add native ownership, unrelated-edit, later readiness, and shared-workspace regressions. The shared-workspace loop was reproduced before the fix.

Co-Authored-By: Devin Foley <devin@paperclip.ing>
Co-Authored-By: Paperclip <noreply@paperclip.ing>
canary/v2026.1015.0-canary.8
2026-10-15 00:18:23 -07:00
DottaandPaperclip c68a483cd9 fix: replay prepared warm seeds when generation caching is unavailable (#16399)
Replay this preparation’s verified private workspace seed after failed warm validation, including when the generation cache refuses publication. Preserve host-current semantics and rebuild if the private seed is missing or changed.

Verified 89 focused cases, full typecheck/build, exact-head CI/E2E/security and Apex 5/5. Live PAP latency acceptance remains separate.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-15 01:35:18 -05:00
Devin FoleyandPaperclip 5f463dd625 test(server): reuse route modules across test cases (#16396)
## Thinking Path

> - Paperclip uses server routes to manage agents and their work.
> - Route tests check permissions, login sessions, skills, and file
downloads.
> - Four suites reload large module graphs before every test.
> - Their workers exhaust the heap and block dependency updates.
> - This pull request loads each route graph once per suite.
> - Each test still gets a fresh app and resets its fixture state.

## Linked Issues or Issue Description

Refs #16017. Related: #12987 addresses repeated logger transports
through a different production change.

**What happened?**

CI run 38293835699, attempt 2, aborted four server test workers. The
affected suites cover attachments, device login, agent skills, and setup
tokens. Three failed log tails show heap exhaustion. The separate
`db.transaction` warning came from the document-annotation suite, which
passed.

**Expected behavior**

All route cases complete within the worker heap limit and keep
independent test state.

**Steps to reproduce**

Run the unchanged agent-skills suite from master with
`NODE_OPTIONS=--max-old-space-size=1536` and `--logHeapUsage`. The
worker aborted after 66 of 90 cases, at a reported heap of 1,532 MB.

**Paperclip version**

The fix is based on master commit
`444d5854b0f53b19d138f060a267cfd10171c7b3`. The original dependency PR
head is `7b74c3f248731943154547385b1de5eb6562bf01`.

**Deployment mode**

Local tests and CI.

## What Changed

- Move module resets and mock registration from `beforeEach` to
`beforeAll` in four route suites.
- Load the agent-skills route modules during suite setup.
- Keep each test's app, fixture resets, and login cleanup.
- Remove two comments that describe the old reset behavior.

## Verification

All 193 tests in the four affected suites pass with a 1.5 GiB heap
limit:

```sh
NODE_OPTIONS=--max-old-space-size=1536 pnpm exec vitest run --project @paperclipai/server \
  server/src/__tests__/agent-skills-routes.test.ts \
  server/src/__tests__/agent-device-login-routes.test.ts \
  server/src/__tests__/issue-attachment-routes.test.ts \
  server/src/routes/setup-token-route.test.ts --reporter=verbose --logHeapUsage
```

- Local runtime: Node 25.9.0, Vitest 5.0.4, Supertest 7.3.1, Express
5.2.1.
- Master has an existing manifest/lockfile mismatch. Local installation
used `--no-lockfile`. Express was restored to its locked version in an
untracked test dependency directory. The tracked lockfile is unchanged.
- The agent-skills suite stays below 400 MB of reported heap after the
fix.
- `git diff --check` passes.
- Repository typecheck reaches the Runner Rust check and stops because
this environment has no `cargo`.
- The repository build and normal server typecheck also stop at the
missing Rust toolchain. A direct server `tsc --noEmit` attempt ends with
SIGKILL.
- [CI run
38686964537](https://github.com/paperclipai/paperclip/actions/runs/38686964537)
passes on `c744774bbada4e35ce3acf5991ff19cf50405f2f`, including all
general and serialized server shards, typecheck, build, Runner checks,
and e2e.
- GitHub reports 53 successful check runs, two skipped Storybook checks,
and a successful Snyk status. No checks are pending or failing.
- [Greptile
review](https://github.com/paperclipai/paperclip/pull/16396#issuecomment-6185194537)
gives 5/5 on the same commit, with no actionable issues and zero review
threads.
- The full repository test suite was verified in CI; the local container
has a 4 GiB memory limit.

## Risks

Module state now lasts for one suite. Each case still constructs its own
router and resets its mocks and fixtures. The affected suites pass
together locally, and all applicable CI gates pass on the published
commit.

The change contains four test files. It changes no production code,
workflow, manifest, or lockfile. Merge this separate fix before asking
Dependabot to rebase #16017.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository tools, and code
execution. The runtime does not expose a more specific model ID or
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-14 23:21:16 -07:00
DottaandPaperclip 5bc4e878e7 fix: reuse verified seeds after warm workspace fallback (#16398)
Preserve the verified company/workspace seed-cache scope when a warm sandbox requires host-current staging. Keep remote identity checks, archive digests and rebuilding for missing or corrupt cache generations.

85 focused cases and full repository typecheck/build pass. Exact-head full CI/E2E, Superagent, legacy Snyk and Apex 5/5 pass with no unresolved findings. No staging response-time claim is made.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-15 01:09:07 -05:00
Devin FoleyandPaperclip 79c34058cd fix(native): retain unexported workspaces after execution failure (#16397)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Native agents can edit files in a remote sandbox before they submit
a result.
> - A process identity or startup failure can end the run before file
export.
> - Cleanup can then delete the sandbox and its unpublished files.
> - This pull request retains the exact source and records a file repair
action before cleanup.
> - Operators can recover the files without replaying the failed turn or
changing the task owner.

## Linked Issues or Issue Description

**What happened?**
A native remote process had completed local edits when process identity
recovery failed. The run still had a prepared workspace reference and no
accepted result. Cleanup deleted the source sandbox. No file recovery
card appeared. A restarted run can hit the same path during setup,
before the native executor starts.

**Expected behavior**
Retain the exact unexported source. Show the existing file recovery
card. Require source stop and file recovery evidence. Keep task
ownership, status, and external action reconciliation separate from file
repair.

**Steps to reproduce**
1. Run a native agent in a remote ephemeral workspace and make
unpublished edits.
2. Fail process identity recovery before result acceptance or file
export.
3. Allow failure handling and lease cleanup to run.
4. Before this change, cleanup can destroy the source without a file
repair action.

Refs: #16341 for related process identity diagnostics. Refs: #16349 for
startup ownership fencing. This change addresses file retention after
failure.

## What Changed

- Record source retention and a file recovery action in the terminal
native failure transaction.
- Cover restarted preparation failures before the terminal setup writer
and cleanup run.
- Require the exact company, run, lease, provider allocation, and
workspace binding. Require the exact live recovery claim to settle an
owned preparation failure. Refuse stale or competing claims.
- Reuse the existing stop-only cleanup and file-only reconciliation
path. Leave accepted results and scheduled same-run recovery on their
existing paths.
- Document the retention rule and add database and runtime boundary
regressions.

## Verification

- All 1,242 tests passed across the full heartbeat recovery suite,
native executor, native source retention, runtime teardown, run
completion, and legacy file recovery.
- The integrated missing-descriptor regression fails before the
inner-handler fix and passes after it. The runtime retention and owned
restart regressions also reproduce their prior failures.
- The tests cover exact source identity, current and stale claims,
atomic rollback, cancellation, idempotence, and file-only repair on a
completed task.
- `pnpm -r typecheck` passed on the final code.
- `pnpm build` passed on the final code.
- Full repository CI passed on
`3945eb2ba5c228c5d4992da02b765dcf2e634cb8`: 54 successful checks and two
skipped Storybook checks.
- Apex scored 5/5 on that exact commit. Both earlier findings are fixed
and all review threads are resolved.
- The unsharded local `pnpm test:run` is still in progress. It is not
counted as a local full-suite pass; complete suite coverage is verified
in CI.

## Risks

- Failed runs can retain a sandbox that needs operator cleanup after
file recovery. This uses the existing retention policy and recovery
controls.
- This does not recover files from allocations that were already
deleted.
- No schema migration, UI change, or telemetry change is required.

## Model Used

- OpenAI GPT-6 through Codex. Tool use, code execution, and reasoning
assisted this change. The environment identifies the model as GPT-6; it
does not expose a more specific serving snapshot or context window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-14 23:07:15 -07:00
DottaandPaperclip 074c67fea8 fix(grok): avoid routine wakes for accepted work (#16394)
## Thinking Path

> - Paperclip lets people manage AI agents and their work.
> - Grok Bot receives assignments through routine webhooks and uses
agent MCP tools.
> - Each accepted assignment belongs to the current external
interaction.
> - A pending tool result or comment could start another routine while
that work was still active.
> - Quiet MCP traffic cannot prove that Grok has stopped working.
> - This pull request keeps receipts and feedback durable for the
current interaction to read.
> - New task routines start only for offers that the Bot has not
accepted.

## Linked Issues or Issue Description

Refs #16219 and #16305. The duplicate search also found #12991, which
covers instruction files for local Grok runs.

**What happened?**

The dispatcher could start another routine for an unread result after a
period without a routine reply. It could also wake accepted work for a
task comment. A queued delivery could still start after the Bot had
claimed the offer.

**Expected behavior**

Accepted work stays in its current interaction. The Bot polls the
original operation receipt and reads queued feedback at a safe boundary.
Acknowledgements of a sent wake remain valid after acceptance.

**Steps to reproduce**

1. Accept a Grok assignment and leave it active.
2. Add an unread comment, or complete a tool operation whose call
returned pending.
3. Run the dispatcher after its quiet period.
4. Observe another webhook call. Also queue an offer wake, accept the
offer before sending, and observe that delivery still starts.

**Paperclip version or commit**

Observed during real-client qualification on Core
ff0f976de1. The three new regression cases
also failed on current master before the correction.

## What Changed

- Stop automatic result and feedback wakes for accepted work. Keep both
available through their existing read paths.
- Require an unaccepted offer before dispatch or an explicit delivery
retry. Serialize the final HTTP handoff with acceptance receipts. Cancel
obsolete unsent wakes. Preserve late acknowledgements and immediate Stop
authority.
- Return current Grok polling instructions on assignment reads. Clarify
pending and unknown outcomes in setup guidance.
- Add protocol, receipt-polling, queued-delivery and
late-acknowledgement regressions. Update the operation guide and
readiness plan.

## Verification

- Three new regression cases fail before the fix because a webhook
starts.
- The latest 31 Grok protocol checks and 47 existing Dot Runner checks
pass. New tests force acceptance during secret lookup and the final send
check. They also prove that acknowledgements and Stop remain free during
pending delivery.
- The final-head Grok, Dot, MCP and question-delivery suite passes all
212 checks across six files. Dedicated onboarding UI checks passed six.
- Dedicated Runner driver checks use the real local Rust daemon. Built
the debug daemon required by that package; all 17 checks then passed.
- Repository typecheck and build pass. Token gates, the 2,731-story
structural catalog and the Storybook build pass.
- Public synthetic marker for the post-deployment native source-read
check: `grok-native-read-b45be4eff22cb15a31d0af512f437b8d`.
- Final-head CI passes all 55 checks, with two optional Storybook skips.
Greptile is 5/5 with no open threads. The first chat-shard attempt
failed one Telegram SQL-quiet timing assertion; all five related cases
pass locally and the single targeted CI rerun passes.
- After deployment, repeat actual Grok work through an answered
question, receipt polling, a saved report and file, canonical completion
and cleanup. This correction is not production qualification by itself.

## Risks

A Bot that does not poll a pending receipt can wait without another
routine wake. Current assignment reads now supply polling instructions
to existing connections. Task comments stay queued until the Bot reads
them or Runner admits the next continuation. The real client must verify
that behavior before production enablement. The observed second file
version does not prove which provider interaction created it. Acceptance
can wait for the bounded webhook response. Stop and authenticated
acknowledgements use independent row locks and remain available.

## Model Used

OpenAI GPT-6 in Codex, with reasoning, repository editing, shell tools
and browser tools. The exact serving model ID, snapshot and context
window are not exposed by this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-15 00:54:15 -05:00
DottaandPaperclip 854ab294cc fix: restore repository admission and Claude startup on Linux ARM64 (#16372)
Batch retained workspace admission reads while preserving individual privacy and trust decisions. Preserve bounded saved-policy projections when integrating current master, and qualify the pinned Claude executable on Linux ARM64.

Verified exact head 4d468f998b with full CI/E2E, Superagent, Snyk and Apex 5/5, with no unresolved findings. Local typecheck/build and 209 server plus 74 integrity cases pass; full local suite remains pending.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-15 00:33:09 -05:00
DottaandPaperclip 444d5854b0 test(adapter-utils): consolidate outbound restore span coverage (#16115)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-15 00:23:28 -05:00
DottaandPaperclip a15798e752 test(server): reuse seeded delivery workspace fixture (#16114)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-15 00:12:27 -05:00
DottaandPaperclip 1315aa22c2 test(ui): retire duplicate company settings checks (#16113)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-15 00:10:50 -05:00
DottaandPaperclip 86816ff965 test(adapters): consolidate Grok stale-auth promotion coverage (#16112)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-15 00:09:21 -05:00
DottaandPaperclip 982ff62949 test(shared): consolidate ZIP conversion observations (#16118)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-15 00:06:50 -05:00
DottaandPaperclip 5e62927a8f test(adapters): consolidate Codex staging permission guards (#16111)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-15 00:06:35 -05:00
DottaandPaperclip 1897080a6d test(scripts): retire duplicate provisioning fallback launch (#16110)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-15 00:02:38 -05:00
DottaandPaperclip 5c65525219 test(scripts): consolidate stable shard selection cases (#16109)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-15 00:02:23 -05:00
DottaandPaperclip a05052f77d test: retire duplicate heartbeat disposition recovery executions (#16102)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-15 00:00:51 -05:00
DottaandPaperclip 67e45e869e test(ui): retire duplicate CompanyExport render checks (#16108)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-14 23:59:17 -05:00
DottaandPaperclip 662a20281d test(cli): retire duplicate ZIP codec owner tests (#16116)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-14 23:57:43 -05:00
DottaandPaperclip 0de90e25bf test(adapter-utils): share pre-READY work-bound bridge execution (#16098)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-14 23:56:07 -05:00
DottaandPaperclip 1ff08a53cb test(tailscale): retire duplicate substituted-listener withdrawal (#16106)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-14 23:51:52 -05:00
DottaandPaperclip 522619744b test(ui): retire duplicate IssueChatThread render cases (#16092)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-14 23:51:37 -05:00
DottaandPaperclip ae6e701821 test(ui): share repeated IssueDetail render proofs (#16091)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
canary/v2026.1015.0-canary.7
2026-10-14 23:47:38 -05:00
DottaandPaperclip 997ea5fd17 test(server): retire duplicate process-loss recovery execution (#16088)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-14 23:47:22 -05:00
DottaandPaperclip 3b6d1452f2 test(daytona): retire duplicate pack span execution (#16099)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-14 23:45:50 -05:00
DottaandPaperclip aa6910939b test(runtime): share the projectless local lease acquisition (#16086)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-14 23:41:48 -05:00
DottaandPaperclip d6e4b36c8c test(server): move Git redaction proof to its owner (#16081)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-14 23:35:43 -05:00
76164b53cb fix(heartbeat): retry a legacy run that lost its lease before dispatch (#16358)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The heartbeat runner executes legacy agent runs. A controller lease
watchdog keeps durable ownership of each legacy run, and it aborts the
run when it loses the lease.
> - #16084 records a lease loss as a `controller_lease` cancellation
when the adapter returns or throws. It does not cover a lease loss
during setup, before the adapter starts.
> - In setup, the abort reason goes to the setup catch. The catch
records an opaque `setup_failed` and sends the raw
`LegacyControllerLeaseLostError` to Sentry with no cause label.
> - Issue continuation recovery treats `setup_failed` as non-retryable.
The task moves to `blocked`, but no provider work started.
> - This pull request records a lease loss before dispatch as
`controller_lease_lost`, with the same `controller_lease` evidence.
Recovery retries it as transient infrastructure, and so does the
accepted-interaction (plan approval) continuation retry.
> - The benefit is that a host lease loss during setup no longer strands
the task, and the next Sentry event shows which renewal failure caused
the loss.

## Linked Issues or Issue Description

Refs #16084
Refs #16355
Refs #16359

**What happened?**
A staging server reported `LegacyControllerLeaseLostError: Legacy
controller lease lost` from the heartbeat. The run failed during setup
with `setup_failed`. Continuation recovery then said "non-retryable
failure on this issue's continuation run (`setup_failed`)" and moved the
task to `blocked`. A scanner had to move the task back to `in_progress`
by hand.

**Expected behavior**
A lease loss before adapter dispatch is a host infrastructure stop. The
run records the `controller_lease` cause, like the adapter-phase path.
The run keeps `providerWorkStarted: false`. Recovery retries the
continuation with its bounded transient budget, and it does not block
the task.

**Steps to reproduce**
1. Start a legacy run that has a controller lease.
2. Before the adapter dispatch, make the lease renewal fail: the update
matches no row, the query errors, or the lease deadline passes.
3. `controllerLease.assertOwned()` throws the abort reason in workspace
preparation or before dispatch.
4. Before this change: the run is `failed`/`setup_failed`, `resultJson`
has no `cancellation`, and continuation recovery blocks the task. After
this change: the run is `failed`/`controller_lease_lost` with
`cancellation.source: "controller_lease"` and a closed cause, and
continuation recovery classifies it as `transient_infra`.

**Additional context**
#16355 handles a different path: a restart orphan that the next boot
reaps as `process_lost`. This change handles a live controller that
loses its own lease during setup. This change does not find the root
cause of the lease loss. The closed cause label in the next Sentry event
gives that fact.

## What Changed

- `server/src/services/heartbeat/run-completion.ts`: `failRunSetup`
receives the execution abort signal. When the abort reason is a
`LegacyControllerLeaseLostError`, it writes error code
`controller_lease_lost`, adds the `controller_lease` cancellation
evidence, and keeps the existing bootstrap recovery facts. The agent
goes back to idle, not `error`, because the stop is a host fault. This
applies only before provider dispatch: `failRunSetup` also receives
`providerDispatchStarted`, and a setup-path error after a legacy adapter
or native dispatch began keeps `setup_failed`, so started work is never
retried.
- `server/src/services/legacy-controller-lease.ts`: add the
`CONTROLLER_LEASE_LOST_FAILURE_CODE` constant.
- Merged with #16359 (lease-loss timings): the setup failure keeps its
`resultJson.controllerLeaseLoss` timings and adds this PR's
`controller_lease` cancellation from the abort signal. #16359's setup
test now aborts the signal first, as `assertOwned()` does, and expects
`controller_lease_lost`.
- `server/src/services/heartbeat.ts`: pass the execution abort signal
and `providerDispatchStarted` (`legacyAdapterEntered ||
nativeDispatchStarted`) to `failRunSetup`.
- `server/src/services/recovery/service.ts`: add `controller_lease_lost`
to the transient-infrastructure continuation codes (3 bounded attempts
with backoff).
- `server/src/services/heartbeat/retries.ts`: add
`controller_lease_lost` to the accepted-interaction continuation retry
allowlist, next to `process_lost`. Without it, a plan-approval
continuation that lost its lease before dispatch was escalated and
blocked before generic recovery could see it.
- `server/src/services/heartbeat/run-lifecycle.ts`: add
`controller_lease_lost` to the pre-adapter setup codes, so the comment
policy does not queue a missing-comment retry.
- `doc/run-log-events.md`: document the new error code.
- Tests: setup lease loss when the abort reason is thrown, and when a
setup step throws a different error after the abort. A unit test shows a
lease loss after dispatch keeps `setup_failed`. A classifier test shows
`controller_lease_lost` is transient and `setup_failed` stays
non-retryable. Two assembled heartbeat tests
(`heartbeat-process-recovery.test.ts`) lose the real lease during
workspace preparation: one for a plain issue continuation (one
`transient_failure` retry) and one for an accepted plan-approval
interaction continuation (one `interaction_continuation_infra_retry`
retry). Both check the saved run, no adapter dispatch, and the issue
still `in_progress`.

## Verification

- `cd server && npx vitest run
src/services/heartbeat/run-completion.test.ts`: 25 passed (embedded
Postgres), including the new post-dispatch test.
- `cd server && npx vitest run src/services/run-cancellation.test.ts
src/services/recovery/provider-failure-classification.test.ts`: passed
on the first head (29 with run-completion; 25 classifier).
- `cd server && npx vitest run
src/services/legacy-controller-lease.test.ts`: 13 passed (needs `pnpm
--filter @paperclipai/plugin-sdk... build` first).
- After merging master (head `2148b27`): `run-completion`,
`legacy-controller-lease`, `run-cancellation`,
`provider-failure-classification` and `run-failure-report` suites, 310
passed; `heartbeat-process-recovery.test.ts -t "lease|dispatch"`, 31
passed.
- Head `c52825c`: `retries`, `run-completion`,
`heartbeat-retry-scheduling`, `legacy-controller-lease` and
`provider-failure-classification` suites, 134 passed;
`heartbeat-process-recovery.test.ts -t "lease|dispatch|interaction"`, 41
passed.
- Negative controls: with the lease-loss branch in `run-completion.ts`
disabled, the plain-continuation assembled test fails (`setup_failed`
instead of `controller_lease_lost`). With the `retries.ts` allowlist
change reverted, the accepted-interaction assembled test fails because
no retry is scheduled.
- Not run locally: full `tsc` and `pnpm build`. The sandbox has a 4 GB
memory limit and the OOM killer stops `tsc`. CI is the typecheck and
build gate.

## Risks

- Low risk. The change applies only when the execution abort reason is a
`LegacyControllerLeaseLostError` in the setup catch before dispatch.
Other setup failures keep their current codes.
- The retry uses the existing continuation recovery budgets and
admission checks (agent state, budget, pause). The
interaction-continuation retry keeps its existing 2-attempt budget. The
retry is a new run. The revoked controller cannot dispatch, because its
renewal compare-and-set fails.
- A run that the reaper settles first keeps the reaper outcome.
`setRunStatusIfRunning` makes the first terminal write win.
- Sentry still gets this unexpected stop, with the closed
`controllerLeaseLossCause` label. If the lease loss is frequent, the
event volume does not change.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with
tool use (shell, file edits, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass (targeted suites pass;
typecheck/build not possible in this sandbox, see Verification)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Devin Foley <139239+devinfoley@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 21:31:42 -07:00
DottaandPaperclip 3ea39f75c1 test: consolidate duplicate owner proofs (#16075)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-14 23:17:13 -05:00
DottaandPaperclip 8541dfad29 test(e2e): share initial Settings sidebar browser proof (#16083)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-14 23:14:20 -05:00
DottaandPaperclip 9076329149 test: consolidate duplicate runtime credential proofs (#16079)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-14 23:13:45 -05:00
DottaandPaperclip 5f160421e3 test(server): run status corpus fixtures once (#16078)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-14 23:13:31 -05:00
DottaandPaperclip 0e1bb3748b test(db): retain owner nullability in the active-slot keeper (#16076)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-14 23:13:15 -05:00
DottaandPaperclip ccbfefef5f test(ui): retire 33 duplicate executions and share render setup (#16072)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-14 23:12:34 -05:00
DottaandPaperclip f64d05adf4 test: consolidate repeated ACPX startup metadata scenarios (#16068)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
canary/v2026.1015.0-canary.6
2026-10-14 23:02:09 -05:00
DottaandPaperclip 422728fc36 test: retire seven repeated ACPX trace executions (#16061)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-14 22:50:25 -05:00
8b27af77db fix(heartbeat): wait for a live holder of a reused shared execution workspace (#16371)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The heartbeat pre-dispatch gate decides if a run can use a shared
workspace in its execution environment.
> - A child task can inherit the shared execution workspace of its
parent. The parent run is often still live when the child run starts.
> - On a sandbox environment, the gate failed the child run with
`configuration_incomplete`. It did this for every concurrency policy,
also for the default `auto` policy.
> - Recovery then blocked the task and told the operator to bind missing
secrets. That was not the real cause. An operator had to select a
different workspace by hand.
> - Shared project workspaces already use the `auto`/`serialize` policy
to wait through the `workspace_busy` retry path.
> - This pull request applies the same policy to a reused shared
execution workspace.
> - The benefit: the child task waits for the parent run and then
continues automatically. It does not fail and become blocked.

## Linked Issues or Issue Description

**What happened?**
A parent agent created a child task. The child inherited the parent's
shared execution workspace. The child agent used a sandbox environment.
The child run started about 1.5 seconds after the parent created the
task, while the parent run was still live. The run failed before the
adapter started with this error: "This environment cannot place
concurrent agents in one physical shared folder. Use local/SSH sharing
or isolated task workspaces." Recovery set the task to `blocked` with
the message "required secret/env bindings are missing".

**Expected behavior**
With the default `auto` policy, the child run waits for the parent run
(`workspace_busy`) and then continues automatically. This is the same
behavior as for a shared project workspace on a sandbox.

**Paperclip version or commit**
Staging release `8ffc5748`. The same code is on `master`.

**Deployment mode**
Hosted cloud (staging), agent on a sandbox execution environment.

**Steps to reproduce**
1. Use an agent with a sandbox environment.
2. Bind two issues to the same `shared_workspace` execution workspace.
Set `executionWorkspacePreference: reuse_existing` on the second issue.
3. Keep a live run on the first issue.
4. Wake the agent on the second issue. Before this change, the run fails
with `configuration_incomplete`. After this change, the run is deferred
with `workspace_busy`.


## What Changed

- `environment-selection.ts`: When a reused shared execution workspace
has a live holder on a non-local/non-SSH environment, the gate now reads
`sharedWorkspaceConcurrency`. With `auto` or `serialize`, it throws
`WorkspaceBusyDeferral`. Only `allow` still fails with
`shared_realization_unsupported`.
- `retries.ts`: Added `findExecutionWorkspaceHolder` next to
`findSharedWorkspaceHolder`. It keeps the old match rule. A running
holder that has had no output for more than one hour (the existing
staleness cutoff) is still returned, with `stale: true`. A live holder
is preferred over a stale one.
- `environment-selection.ts`: A stale holder still owns the folder,
because silence does not prove that the run stopped. The child run
therefore fails with `configuration_incomplete`
(`shared_realization_unsupported`, `holderStale: true`). It does not
share the folder with the holder, and it does not wait without limit.
This is the same result as before this pull request, with a more
specific message.
- `retries.ts`: `WorkspaceBusyDeferral` can now carry an
`executionWorkspaceId`. The run result, the run event, and the
cancellation reason name the execution workspace when it applies.
- `stranded-notice.ts`: Added a specific recovery notice for
`shared_realization_unsupported`, and a second notice for a stale
holder. They no longer tell the operator to bind missing secrets.
- `doc/experimental/issue-worktree-support.md`: Documented that `allow`
is only concurrent on local/SSH, and that a reused shared execution
workspace follows the same rules.

## Verification

- `cd server && npx vitest run
src/__tests__/heartbeat-workspace-busy.test.ts -t "reused shared
execution workspace"` covers 4 tests:
- `auto` on a sandbox with a live holder: the run is cancelled with
`workspace_busy`, a retry is scheduled, no Sentry run failure is
reported, and the issue does not become blocked.
- After the holder finishes, the scheduled retry is promoted and runs.
The adapter gets the inherited shared folder, and the issue keeps its
execution workspace.
- `allow` on a sandbox: the run still fails with the shared-folder
message and no retry is scheduled.
- A holder that is still running but silent past the staleness cutoff:
the run fails with `holderStale: true`, the adapter does not start, no
retry is scheduled, and the holder is not touched.
- `npx vitest run src/services/recovery/stranded-notice.test.ts
src/services/heartbeat/environment-selection.test.ts
src/services/heartbeat/retries.test.ts` covers the notice and the
selection unit tests.
- First commit (`d0b7a092`): the 3 original integration tests and the
unit suites passed locally, and all CI checks passed.
- Follow-up commit (`3c7e515`, which addresses the Greptile review):
these tests were **not** run locally. Dependency install was killed
twice by the sandbox's 4 GB memory limit. CI is the first run of the new
and changed tests.
- Known local failure: `does not hide ambiguous, malformed, provider, or
transport cancellations behind workspace_busy` fails in this sandbox
also on unchanged `master` (checked with `git stash`). This change does
not cause it.
- Not run locally: the full server `tsc --noEmit`, for the same memory
reason. CI runs the typecheck.

## Risks

- Behavior change: before, a run with a live holder of a reused shared
execution workspace on a sandbox failed at once. Now it waits with
`auto`/`serialize`. A long parent run can delay the child. If the holder
has no output for more than one hour, the child fails as before instead
of waiting without limit.
- An explicit `allow` policy keeps the old failure.
- Local and SSH environments do not change.
- No schema or migration change.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code
execution in the Paperclip agent runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass (first commit only; see
Verification)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Devin Foley <139239+devinfoley@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 20:45:29 -07:00
DottaandPaperclip 66724b8ce0 perf(auth): project only required run context fields (#16380)
Project only the required JSON fields for saved-run policy and direct-parent report authorization. Preserve current reads, JSON types, identity checks, permission decisions and legacy task-key precedence.

Validation: 186 affected cases; 17 database transfer-budget cases after the shared startup-timeout correction; full typecheck/build; exact-head full CI, E2E, Superagent and Apex 5/5.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-14 22:45:08 -05:00
DottaandPaperclip 1d5e2c89f0 test(server): retire duplicate guards and share issue seed fixtures (#16059)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Server tests protect task scheduling, session continuation,
transport decisions, and issue data.
> - Four cases repeat the same registered inputs and assertions as
existing owner cases.
> - Issue service tests also repeat 103 database seed statements.
> - Each distinct input, assertion, lifecycle, and security contract
must keep its proof.
> - This pull request removes the four exact duplicates and shares only
exact seed statements.
> - The benefit is 584 fewer test lines and four fewer registered
executions.

## Linked Issues or Issue Description

Refs #16047. This draft is stacked on
`codex/test-audit-round-six-delivery` at
`e868e6645ce0d598d756d3458f0b3d71afd6db4a`.

**What existing behavior does this improve?**

Maintenance of server owner tests. Production behavior does not change.

**Current behavior**

Four registered cases repeat identical executable bodies. The issue
service suite repeats 80 company rows, ten project rows, and thirteen
agent rows.

**Proposed behavior**

Keep each contract in its existing owner suite. Use three direct async
seed helpers for the exact repeated database rows. Preserve each
caller's database instance, IDs, awaited insert position, field order,
and fresh object allocations.

**Reason and benefit**

Remove 584 net test lines and four registered executions. No measured
wall-time improvement is claimed. No production, dependency, workflow,
telemetry, or skip change is included.

## What Changed

- Share exact company, Workspace project, and CodexCoder rows in
`issues-service.test.ts`. All 137 registered cases remain.
- Retire `heartbeat-scheduling-suppression.test.ts` → `suppresses
heartbeat scheduling while a task drain is active`. Keeper:
`heartbeat-task-drain.test.ts` →
`start_task_drain_suppresses_admission`.
- Retire `heartbeat-timer-wake-session-reset-pf4.test.ts` → `does not
reset for issue_commented (preserve continuation context)`. Keeper:
`heartbeat-workspace-session.test.ts` → `does not reset for comment
wakes`.
- Retire `setup-token-session.test.ts` → `denies a direct non-loopback
HTTP receive-token request, so it delivers no token (SR-6)`. Keeper in
the same describe: `denies a direct non-loopback HTTP request (SR-6)`.
- Retire `setup-token-session.test.ts` → `denies a spoofed forwarded
HTTPS receive-token request under a broad proxy setting (SR-7)`. Keeper
in the same describe: `denies a spoofed X-Forwarded-Proto when no
trusted proxy is configured (SR-6)`.

The retired receive-token cases only call
`evaluateConfidentialTransport`. They neither request a route nor
configure `TRUST_PROXY`. The keepers protect the exact guard inputs and
output. The route now uses the guard to select a transport advisory;
this PR makes no route-denial or token-delivery proof claim.

## Verification

- Complete matched six-file baseline: **414/414 pass**, no failures or
skips. Candidate: **410/410 pass**, no failures or skips.
- Run: `pnpm exec vitest run --project @paperclipai/server
server/src/__tests__/issues-service.test.ts
server/src/__tests__/heartbeat-scheduling-suppression.test.ts
server/src/__tests__/heartbeat-task-drain.test.ts
server/src/__tests__/heartbeat-timer-wake-session-reset-pf4.test.ts
server/src/__tests__/heartbeat-workspace-session.test.ts
server/src/services/setup-token-session.test.ts`.
- Expand the actual three helper bodies at all 103 call sites. The
complete issue-service AST is identical to the original after expansion.
Independent review checked the exact inputs, lexical bindings, awaited
order, and fresh nested configs.
- Deliberate regressions disabled task-drain suppression, forced comment
wakes to reset, and allowed the final insecure transport branch. All
four retained keepers failed at their intended equality assertions. Each
production file was restored byte for byte against HEAD.
- An optional project-inheritance mutation was ineffective because
another workspace path still derives project identity. It passed and is
excluded from accepted preservation evidence.
- `git diff --check` passes. Independent cross-review approved the exact
source and proof before this draft.
- Broad typecheck, full test, and build gates passed in PR CI on
`a9713432b1e498e023b4828ffa283a351abe856f`. All 52 check-runs concluded
success. Greptile completed on that exact head with confidence 5/5 and
no findings.

## Risks

- Low risk: only test code changes. Four exact duplicate bodies are
removed.
- The fixture helpers preserve all original IDs and DB operations. They
do not cache data or generate assertions.
- Same-named explicit-project and derived-project inheritance cases
remain because their inputs differ. Separate login PTY and duplex
lifecycle cases also remain.
- No wall-time speed gain is claimed from fixture line reduction.

## Model Used

- OpenAI GPT-6.1-Sol, exact model ID `gpt-6.1-sol`, through Codex. Tool
use, code execution, and reasoning assisted the audit. The runtime does
not expose its context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-14 22:38:16 -05:00
DottaandPaperclip d73d7a8651 perf(skills): batch repository dependency reads (#16377)
Batch dependency Markdown reads through the bounded Git snapshot reader and queue newly discovered dependencies once. Preserve package contents, limits, authorization, audits, and reference order.

Validation: 63 affected cases; full typecheck/build; exact-head full CI, E2E, Superagent and Apex 5/5. Local Git comparison: 502 individual reads reduced to three batches with identical entries.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-14 22:31:38 -05:00