Commit Graph
118 Commits
Author SHA1 Message Date
8f1cb11a03 fix(workspaces): name the owner action when a shared source is restricted (#16545)
A reused task workspace in a shared project checkout keeps its source
denial when another task's restricted worktree shares that checkout
(#16095 deliberately does not clone over existing workspaces). The run
then failed as an opaque `setup_failed` 403 and the task was blocked
with a generic "no live execution path" notice.

Convert that exact denial into a `configuration_incomplete` failure
with reason `workspace_source_restricted`, give the task notice and the
recovery action the owner step, and keep this pre-dispatch owner action
out of the unexpected Sentry run-failure stream. The reason and
predicate live in a dependency-free module. Access checks are unchanged.

Co-Authored-By: Paperclip <Paperclip-Paperclip@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 01:33:53 -07:00
8d9d2f9ddf fix(recovery): preserve deliberate stops and dependency waits
Preserve deliberate assignee blocks and operator stops during dependency and run-release recovery. Attribute native waits to their bound owner. Ignore unrelated edits and shared-workspace exports from another issue. Continue after genuine dependency changes while preserving independently authorized user input.

Keep the original contributor's implementation and review history. Add native ownership, unrelated-edit, later readiness, and shared-workspace regressions. The shared-workspace loop was reproduced before the fix.

Co-Authored-By: Devin Foley <devin@paperclip.ing>
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-15 00:18:23 -07:00
76164b53cb fix(heartbeat): retry a legacy run that lost its lease before dispatch (#16358)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The heartbeat runner executes legacy agent runs. A controller lease
watchdog keeps durable ownership of each legacy run, and it aborts the
run when it loses the lease.
> - #16084 records a lease loss as a `controller_lease` cancellation
when the adapter returns or throws. It does not cover a lease loss
during setup, before the adapter starts.
> - In setup, the abort reason goes to the setup catch. The catch
records an opaque `setup_failed` and sends the raw
`LegacyControllerLeaseLostError` to Sentry with no cause label.
> - Issue continuation recovery treats `setup_failed` as non-retryable.
The task moves to `blocked`, but no provider work started.
> - This pull request records a lease loss before dispatch as
`controller_lease_lost`, with the same `controller_lease` evidence.
Recovery retries it as transient infrastructure, and so does the
accepted-interaction (plan approval) continuation retry.
> - The benefit is that a host lease loss during setup no longer strands
the task, and the next Sentry event shows which renewal failure caused
the loss.

## Linked Issues or Issue Description

Refs #16084
Refs #16355
Refs #16359

**What happened?**
A staging server reported `LegacyControllerLeaseLostError: Legacy
controller lease lost` from the heartbeat. The run failed during setup
with `setup_failed`. Continuation recovery then said "non-retryable
failure on this issue's continuation run (`setup_failed`)" and moved the
task to `blocked`. A scanner had to move the task back to `in_progress`
by hand.

**Expected behavior**
A lease loss before adapter dispatch is a host infrastructure stop. The
run records the `controller_lease` cause, like the adapter-phase path.
The run keeps `providerWorkStarted: false`. Recovery retries the
continuation with its bounded transient budget, and it does not block
the task.

**Steps to reproduce**
1. Start a legacy run that has a controller lease.
2. Before the adapter dispatch, make the lease renewal fail: the update
matches no row, the query errors, or the lease deadline passes.
3. `controllerLease.assertOwned()` throws the abort reason in workspace
preparation or before dispatch.
4. Before this change: the run is `failed`/`setup_failed`, `resultJson`
has no `cancellation`, and continuation recovery blocks the task. After
this change: the run is `failed`/`controller_lease_lost` with
`cancellation.source: "controller_lease"` and a closed cause, and
continuation recovery classifies it as `transient_infra`.

**Additional context**
#16355 handles a different path: a restart orphan that the next boot
reaps as `process_lost`. This change handles a live controller that
loses its own lease during setup. This change does not find the root
cause of the lease loss. The closed cause label in the next Sentry event
gives that fact.

## What Changed

- `server/src/services/heartbeat/run-completion.ts`: `failRunSetup`
receives the execution abort signal. When the abort reason is a
`LegacyControllerLeaseLostError`, it writes error code
`controller_lease_lost`, adds the `controller_lease` cancellation
evidence, and keeps the existing bootstrap recovery facts. The agent
goes back to idle, not `error`, because the stop is a host fault. This
applies only before provider dispatch: `failRunSetup` also receives
`providerDispatchStarted`, and a setup-path error after a legacy adapter
or native dispatch began keeps `setup_failed`, so started work is never
retried.
- `server/src/services/legacy-controller-lease.ts`: add the
`CONTROLLER_LEASE_LOST_FAILURE_CODE` constant.
- Merged with #16359 (lease-loss timings): the setup failure keeps its
`resultJson.controllerLeaseLoss` timings and adds this PR's
`controller_lease` cancellation from the abort signal. #16359's setup
test now aborts the signal first, as `assertOwned()` does, and expects
`controller_lease_lost`.
- `server/src/services/heartbeat.ts`: pass the execution abort signal
and `providerDispatchStarted` (`legacyAdapterEntered ||
nativeDispatchStarted`) to `failRunSetup`.
- `server/src/services/recovery/service.ts`: add `controller_lease_lost`
to the transient-infrastructure continuation codes (3 bounded attempts
with backoff).
- `server/src/services/heartbeat/retries.ts`: add
`controller_lease_lost` to the accepted-interaction continuation retry
allowlist, next to `process_lost`. Without it, a plan-approval
continuation that lost its lease before dispatch was escalated and
blocked before generic recovery could see it.
- `server/src/services/heartbeat/run-lifecycle.ts`: add
`controller_lease_lost` to the pre-adapter setup codes, so the comment
policy does not queue a missing-comment retry.
- `doc/run-log-events.md`: document the new error code.
- Tests: setup lease loss when the abort reason is thrown, and when a
setup step throws a different error after the abort. A unit test shows a
lease loss after dispatch keeps `setup_failed`. A classifier test shows
`controller_lease_lost` is transient and `setup_failed` stays
non-retryable. Two assembled heartbeat tests
(`heartbeat-process-recovery.test.ts`) lose the real lease during
workspace preparation: one for a plain issue continuation (one
`transient_failure` retry) and one for an accepted plan-approval
interaction continuation (one `interaction_continuation_infra_retry`
retry). Both check the saved run, no adapter dispatch, and the issue
still `in_progress`.

## Verification

- `cd server && npx vitest run
src/services/heartbeat/run-completion.test.ts`: 25 passed (embedded
Postgres), including the new post-dispatch test.
- `cd server && npx vitest run src/services/run-cancellation.test.ts
src/services/recovery/provider-failure-classification.test.ts`: passed
on the first head (29 with run-completion; 25 classifier).
- `cd server && npx vitest run
src/services/legacy-controller-lease.test.ts`: 13 passed (needs `pnpm
--filter @paperclipai/plugin-sdk... build` first).
- After merging master (head `2148b27`): `run-completion`,
`legacy-controller-lease`, `run-cancellation`,
`provider-failure-classification` and `run-failure-report` suites, 310
passed; `heartbeat-process-recovery.test.ts -t "lease|dispatch"`, 31
passed.
- Head `c52825c`: `retries`, `run-completion`,
`heartbeat-retry-scheduling`, `legacy-controller-lease` and
`provider-failure-classification` suites, 134 passed;
`heartbeat-process-recovery.test.ts -t "lease|dispatch|interaction"`, 41
passed.
- Negative controls: with the lease-loss branch in `run-completion.ts`
disabled, the plain-continuation assembled test fails (`setup_failed`
instead of `controller_lease_lost`). With the `retries.ts` allowlist
change reverted, the accepted-interaction assembled test fails because
no retry is scheduled.
- Not run locally: full `tsc` and `pnpm build`. The sandbox has a 4 GB
memory limit and the OOM killer stops `tsc`. CI is the typecheck and
build gate.

## Risks

- Low risk. The change applies only when the execution abort reason is a
`LegacyControllerLeaseLostError` in the setup catch before dispatch.
Other setup failures keep their current codes.
- The retry uses the existing continuation recovery budgets and
admission checks (agent state, budget, pause). The
interaction-continuation retry keeps its existing 2-attempt budget. The
retry is a new run. The revoked controller cannot dispatch, because its
renewal compare-and-set fails.
- A run that the reaper settles first keeps the reaper outcome.
`setRunStatusIfRunning` makes the first terminal write win.
- Sentry still gets this unexpected stop, with the closed
`controllerLeaseLossCause` label. If the lease loss is frequent, the
event volume does not change.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with
tool use (shell, file edits, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass (targeted suites pass;
typecheck/build not possible in this sandbox, see Verification)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Devin Foley <139239+devinfoley@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 21:31:42 -07:00
8b27af77db fix(heartbeat): wait for a live holder of a reused shared execution workspace (#16371)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The heartbeat pre-dispatch gate decides if a run can use a shared
workspace in its execution environment.
> - A child task can inherit the shared execution workspace of its
parent. The parent run is often still live when the child run starts.
> - On a sandbox environment, the gate failed the child run with
`configuration_incomplete`. It did this for every concurrency policy,
also for the default `auto` policy.
> - Recovery then blocked the task and told the operator to bind missing
secrets. That was not the real cause. An operator had to select a
different workspace by hand.
> - Shared project workspaces already use the `auto`/`serialize` policy
to wait through the `workspace_busy` retry path.
> - This pull request applies the same policy to a reused shared
execution workspace.
> - The benefit: the child task waits for the parent run and then
continues automatically. It does not fail and become blocked.

## Linked Issues or Issue Description

**What happened?**
A parent agent created a child task. The child inherited the parent's
shared execution workspace. The child agent used a sandbox environment.
The child run started about 1.5 seconds after the parent created the
task, while the parent run was still live. The run failed before the
adapter started with this error: "This environment cannot place
concurrent agents in one physical shared folder. Use local/SSH sharing
or isolated task workspaces." Recovery set the task to `blocked` with
the message "required secret/env bindings are missing".

**Expected behavior**
With the default `auto` policy, the child run waits for the parent run
(`workspace_busy`) and then continues automatically. This is the same
behavior as for a shared project workspace on a sandbox.

**Paperclip version or commit**
Staging release `8ffc5748`. The same code is on `master`.

**Deployment mode**
Hosted cloud (staging), agent on a sandbox execution environment.

**Steps to reproduce**
1. Use an agent with a sandbox environment.
2. Bind two issues to the same `shared_workspace` execution workspace.
Set `executionWorkspacePreference: reuse_existing` on the second issue.
3. Keep a live run on the first issue.
4. Wake the agent on the second issue. Before this change, the run fails
with `configuration_incomplete`. After this change, the run is deferred
with `workspace_busy`.


## What Changed

- `environment-selection.ts`: When a reused shared execution workspace
has a live holder on a non-local/non-SSH environment, the gate now reads
`sharedWorkspaceConcurrency`. With `auto` or `serialize`, it throws
`WorkspaceBusyDeferral`. Only `allow` still fails with
`shared_realization_unsupported`.
- `retries.ts`: Added `findExecutionWorkspaceHolder` next to
`findSharedWorkspaceHolder`. It keeps the old match rule. A running
holder that has had no output for more than one hour (the existing
staleness cutoff) is still returned, with `stale: true`. A live holder
is preferred over a stale one.
- `environment-selection.ts`: A stale holder still owns the folder,
because silence does not prove that the run stopped. The child run
therefore fails with `configuration_incomplete`
(`shared_realization_unsupported`, `holderStale: true`). It does not
share the folder with the holder, and it does not wait without limit.
This is the same result as before this pull request, with a more
specific message.
- `retries.ts`: `WorkspaceBusyDeferral` can now carry an
`executionWorkspaceId`. The run result, the run event, and the
cancellation reason name the execution workspace when it applies.
- `stranded-notice.ts`: Added a specific recovery notice for
`shared_realization_unsupported`, and a second notice for a stale
holder. They no longer tell the operator to bind missing secrets.
- `doc/experimental/issue-worktree-support.md`: Documented that `allow`
is only concurrent on local/SSH, and that a reused shared execution
workspace follows the same rules.

## Verification

- `cd server && npx vitest run
src/__tests__/heartbeat-workspace-busy.test.ts -t "reused shared
execution workspace"` covers 4 tests:
- `auto` on a sandbox with a live holder: the run is cancelled with
`workspace_busy`, a retry is scheduled, no Sentry run failure is
reported, and the issue does not become blocked.
- After the holder finishes, the scheduled retry is promoted and runs.
The adapter gets the inherited shared folder, and the issue keeps its
execution workspace.
- `allow` on a sandbox: the run still fails with the shared-folder
message and no retry is scheduled.
- A holder that is still running but silent past the staleness cutoff:
the run fails with `holderStale: true`, the adapter does not start, no
retry is scheduled, and the holder is not touched.
- `npx vitest run src/services/recovery/stranded-notice.test.ts
src/services/heartbeat/environment-selection.test.ts
src/services/heartbeat/retries.test.ts` covers the notice and the
selection unit tests.
- First commit (`d0b7a092`): the 3 original integration tests and the
unit suites passed locally, and all CI checks passed.
- Follow-up commit (`3c7e515`, which addresses the Greptile review):
these tests were **not** run locally. Dependency install was killed
twice by the sandbox's 4 GB memory limit. CI is the first run of the new
and changed tests.
- Known local failure: `does not hide ambiguous, malformed, provider, or
transport cancellations behind workspace_busy` fails in this sandbox
also on unchanged `master` (checked with `git stash`). This change does
not cause it.
- Not run locally: the full server `tsc --noEmit`, for the same memory
reason. CI runs the typecheck.

## Risks

- Behavior change: before, a run with a live holder of a reused shared
execution workspace on a sandbox failed at once. Now it waits with
`auto`/`serialize`. A long parent run can delay the child. If the holder
has no output for more than one hour, the child fails as before instead
of waiting without limit.
- An explicit `allow` policy keeps the old failure.
- Local and SSH environments do not change.
- No schema or migration change.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code
execution in the Paperclip agent runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass (first commit only; see
Verification)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Devin Foley <139239+devinfoley@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 20:45:29 -07:00
Devin FoleyandPaperclip 35b3345033 fix(server): support issue monitor deadlines in idle sleep reports (#16325)
## Thinking Path

> - Paperclip manages work for companies and AI agents.
> - Hosted workspaces can sleep when their required work has a safe wake
path.
> - The current idle check treats every open issue as background work.
> - Backlog, unassigned and human-waiting records can therefore keep an
idle process running.
> - Future automated monitors still need their saved check time and
execution claims.
> - This PR adds a signed host capability for passive issues and
verified monitor deadlines.
> - The host can save one alarm for the earliest required routine or
monitor time.

## Linked Issues or Issue Description

**What happened?**

A quiet workspace with a backlog or unassigned todo issue reports
background work as present. A future issue monitor also prevents sleep
even when the host can save a durable wake alarm.

**Expected behavior**

Proven passive records can sleep. A validated future monitor can sleep
only after the host saves its alarm and validates the same owned report.
Active, unknown and unresolved work stays awake.

**Steps to reproduce**

1. Create a quiet workspace with an unassigned todo issue and no other
work.
2. Acquire a bounded, signed Cloud idle hold.
3. Request the routine-only safety report with `scheduledWake=1`. The
issue blocks sleep.
4. Request the new capability with `scheduledWake=2`. A passive issue
can now report no work.
5. Add a valid future monitor to an assigned in-progress issue. The
second capability returns its saved deadline for host acknowledgement.

**Paperclip version or commit**

Base: `c30fb7d2b75ea30f72d63656b2a7cfff7fe0929d`.

**Deployment mode**

Cloud-managed authenticated instance. Local scheduling remains in the
existing scheduler.

Related protocol:
[#16215](https://github.com/paperclipai/paperclip/pull/16215). Open
scheduler and monitor PRs were checked for overlap. This change leaves
monitor admission, dispatch and quota recovery with their existing
owners. The roadmap already includes scheduled work; this is a focused
idle-safety extension.

## What Changed

- Add signed scheduling capability `2`; capability `1` retains the
routine-only rules.
- Allow untouched onboarding prompts with idle assignees to sleep using
the same awaiting-user predicate as recovery; typing, answering the
opening card or a prior run removes the exemption.
- Inspect passive issue state and future monitors in the same bounded,
read-only snapshot as other work.
- Check ownership, company scope, known states, claims, policy/deadline
agreement, attempts, timeout and dependency repair.
- Combine routine and monitor deadlines into a known proof kind and
catalog generation.
- Preserve other sleep blockers, including task watchdogs,
conversations, live runs, external credentials and recovery.
- Update the protected route, OpenAPI contract, documentation and
PostgreSQL regression tests.

## Verification

- Full workspace typecheck and build passed. Final server typecheck and
full build passed after the final guards.
- Initial safety, scheduling, protected-route and control-authentication
checks: 326 tests passed.
- Final safety and heartbeat scheduling checks: 253 tests passed.
Includes reporting a future deadline, restoring scheduling from
PostgreSQL and one dispatch across competing schedulers.
- Local general server checks: 20,260 passed; UI: 8,460 passed; CLI: 534
passed, including isolated reruns of two embedded-PostgreSQL startup
failures. All remaining workspace groups and 158 serialized suites
passed. The suite was completed in groups after the isolated
infrastructure retries; the final onboarding change also passed the
focused 674-test recovery/safety run. Earlier unchanged Git fixtures
failed under Xcode's system `init.defaultBranch=main`, reproduced on the
clean base and passed with a test-scoped fixture default; no source or
machine configuration was changed.
- Rebased issue, protected-route and scheduler checks: 115 tests passed;
server typecheck passed.
- The expanded change passed 674 idle-safety, scheduling and recovery
regression tests, including six onboarding admission/invalidation cases.
Final server typecheck and build passed.
- Current-head Apex is 5/5 with no unresolved threads. All 54
current-head checks passed, with two expected skips. Four jobs were
interrupted by runner shutdowns or lost communication on their first
attempts; their targeted reruns passed. Live hosting qualification
awaits the verified master release and matching host capability.

## Risks

This intentionally covers a limited set of passive states and valid
future monitors. Assigned active tasks without a proved waiting path,
unknown states, claimed monitors and quota recovery still block sleep.
It does not make every open issue safe to sleep.

A matching host must recognize `routines_and_issue_monitors`,
acknowledge the durable alarm and revalidate the exact owner, generation
and deadline before stopping. The report alone grants no stop authority.
Older hosts retain capability `1`. Independent database writers remain
outside the admission contract. Existing scheduler claims and execution
authority are unchanged. No schema migration is required.

Roll back host negotiation to capability `1` to restore conservative
admission, while retaining alarm delivery until outstanding alarms have
been handled.

## Model Used

- OpenAI GPT-6 (Codex), with repository editing, command execution and
GitHub tools. The exact deployment ID and context-window metadata are
not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-14 15:39:46 -07:00
DottaandPaperclip adb69e8f09 feat: default new Codex agents to Paperclip Runner on Linux and macOS (#15683)
## Thinking Path

> - Paperclip is the open source control plane for agent companies.
> - People choose a harness when creating an agent.
> - The server should resolve that harness to a qualified runner.
> - Codex is the first harness in the staged default rollout.
> - Saved agents must retain their execution choices.
> - New Codex agents therefore use Paperclip Runner on qualified
macOS/Linux targets, with an explicit Legacy override.

## Linked Issues or Issue Description

Refs #16028, #15555, #15673, #15422, #15631, #15926, #15932, #16149.

The runtime/install prerequisite and release repair are merged. This
slice owns Codex creation, setup, configuration and preservation. Other
harness defaults remain unchanged. It adds no database migration;
incoming master migrations, required maps and regression coverage are
retained.

## What Changed

- Resolve `runner: auto | paperclip | legacy` at the common creation
boundary across onboarding, API/CLI, hires, built-ins, plugins and
imports. Freeze pending hires and export resolved execution choices.
- Preserve existing runners on upgrades, unrelated edits and import
updates, including compatible accounts, workspace settings, Codex effort
and intentional clears.
- Use the underlying harness picker and branding; put the runner
override in Advanced. Use the shared dropdown and update
production-component stories and creation contracts.
- Support qualified Apple Silicon, Intel Mac, Linux x64 and Linux ARM64.
Linux requires GNU libc 2.39+. Disabled, external, unsupported and
unqualified combinations retain their applicable Legacy execution.
- Test the selected runtime/account/model/environment. Ordinary Connect
probes eligible immutable managed targets and refreshes scoped
discovery. Creation/discovery do not provision. Auth, dependency,
incompatible-setting and cleanup failures remain actionable without
changing runners.
- Keep setup probes read-only, preserve local/SSH isolation and resolve
packaged assets before staging credentials. Bound cleanup reads by the
remaining absolute deadline; physically close every owned ingress socket
before endpoint release. Unconfirmed cleanup retains recovery state.
- Preserve validated completed-onboarding callbacks through startup
recovery. First file-producing work belongs to the child task and uses
the existing dependency/block handoff and task-scoped registration.
- Reuse existing E2E fixtures, artifact checks, reporting and controls.
Keep Docker/Cargo/native compilation remote. Paperclip tarballs contain
no Codex payloads; official optional npm platform dependencies remain.

## Verification

Current tested head: `1c03a8815672d82c264a9b08d4ea1984e8fae107`.
**Merge-ready: required CI, fresh review, package qualification, Mac
core acceptance and Test task/follow-up acceptance pass.** The existing
attachment and native-continuity oracles independently verify both Test
downloads, run attribution and unchanged native/provider/workspace
identities. The maintainer authorized a code-owner-review exception for
this PR only. The first Test setup failure and normal Board
completion-review friction are retained below. Actual post-merge
packages and private Cloud distribution remain separate qualification
gates.

- [Current
CI](https://github.com/paperclipai/paperclip/actions/runs/38564241625):
all 48 ordinary jobs and required verify/E2E pass. Attempt 1 retained
six simultaneous host-shutdown cancellations. One failed-only retry
passes those six lanes and two aggregates, preserving 40 successes.
Actual integration checkout `53104c1` combines this head with master
`c80`; it is not relabeled as the branch tree. Fresh exact-head Greptile
5/5, 30 resolved threads, no actionable findings.
- [Exact-source package
qualification](https://github.com/paperclipai/paperclip/actions/runs/38565287239):
nine applicable jobs pass, nine conditional jobs skip. Four remote
native producers; Linux x64 clean install; Intel Mac, Apple Silicon and
Linux ARM64 native startup/physical cleanup controls. All 18
tarballs/four daemon source/hash/manifest bindings and 343 migrations
through 0344 verify. No Codex payload is packed. These hosted tarballs
are not a published npm canary.
- Normal Apple Silicon installation and credential-free native
startup/cleanup pass using the packaged daemon, with no source build or
provider turn. Fresh installed DB initialization passes. Upgrade
651a→1c03 preserves all three complete execution configs, accounts, ten
recorded run profiles, paused budgets, active incidents, cost summaries
and two historical unpriced events. Historical recovery state is
retained.
- Cleanup regressions: four delayed-read baseline failures become 15
passing deadline/authority controls. No-acknowledgement and prior-error
socket ownership regressions become 12 passing controls. Direct
runtime/server types, maps: 35 areas / 176 IDs, diff checks and
independent reviews pass. Permission/auth/reconnect policy and the
10-second setup budget remain intact.
- Existing source coverage includes creation/selection/imports, pending
approvals, recovery, account/lifecycle handling, mounted forms,
dropdowns and E2E support. Full typechecks/tests/build/token and
applicable Storybook checks pass in the recorded current CI; no
duplicate suite or weakened grader.
- Literal 9546 live Mac proof: ordinary Auto Codex onboarding, useful
document, real completed-child callback/reply, seven successful native
runs, manual follow-up and independently downloaded1,101-byte child
file. Existing Legacy title edit/reload preserves execution; two real
Legacy runs produce independently checked15/34-byte files. These remain
historical source records.
- Current literal 1c03 Mac core acceptance passes independent checks:
ordinary Auto native Codex onboarding without runner override; only
timeout 180 seconds/concurrency 1 saved; six successful native runs and
six serialized turns (maximum overlap 1); both same-task continuity
checks and five mechanical callback checks; actual rendered parent reply
and exact child downloads. The useful handoff advances from revision 1
to 2. Both files are 48 bytes, SHA-256
`8b742bd4e8123a7c078f0a8d9f3449fbfa8a320d7d90be641a2818db3c53d92c` and
`f42db9774fa06e804ab7d596fc8b922a2b638921c418b87d0111151d290a6afe`;
exactly one byte changes and old history remains. The agent is paused
with zero active runs/errors. Both artifact work products have
`isPrimary=false`, and descriptive titles fail a stricter title-helper
assumption; these retained results are scoped editorial follow-up, not
new merge gates. Six source-bound screenshots are uploaded and verified
in the private primary QA report.
- [Literal 9546 managed
proof](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-38485488621-1/index.html):
original 19 assertions and completed judge pass, four native turns,
child-bound240-byte file and session/workspace continuity. Checkout
reports a verified generated target-lock mutation; full final porcelain
was not retained. Signed execution-image identity remains separate from
Core source.
- Actual 651a Test deployment succeeds on one owned/pinned target. One
ordinary Auto API-key Connect then fails native cleanup before hire.
Controller turn/drain/suspend completion and service-recorded
environment cleanup are observed; they do not certify physical deletion
or native authority retirement. Failure/recovery/unknown holds are
retained. No Legacy switch or unchanged paid retry. The two source
repairs above require actual treatment.
- [Current immutable Core
preview](https://github.com/paperclipai/paperclip/actions/runs/38570365725)
and matching DB/shared artifacts pass. One owned, pinned Test target
serves exact `1c03`; identical migration history correctly skips the
migrator. Ordinary Auto Codex Connect succeeds and hires native; saved
native selection and QA bounds survive reload. The initial child
preparation attempt fails before provider work, with actual stdout
identifying `renewal_deadline`; its trigger remains unknown and the
failure is retained. Normal Board **Try again** and **Resume** recover
the child to Done, producing a browser-verified 44-byte file, SHA-256
`0a8458b1efcc2d7a0d815689458c4caab039a46c7c7a3e5805de6bc8b1a96ecc`. The
actual parent callback reports the completed child document/file. The
existing validator requires a parent-owned attachment, so recovery
enters normal Review; after byte checking, normal **Approve completion**
reaches parent Done. No forced status or duplicate parent file is used.
The validator is unchanged from integrated and current master.
Autonomous initial callback-to-Done is not proven. The same-child
follow-up completes document revision 2 and publishes a second
independently downloaded 44-byte file, SHA-256
`22022257300fae762784632d08d3bbe0bab61e2ce81f91864a14dcd7aa464591`; only
the revision digit changes. The unchanged existing native-continuity and
attachment oracles pass. All ten recorded runs are terminal: nine native
successes and the retained pre-provider setup failure. The QA agent is
paused with zero active runs. The pre-existing parent evidence scope
mismatch is a separate follow-up, with ordinary completion review
exercised rather than a forced status or relaxed validation. This
unsigned Core preview remains distinct from post-merge private Cloud
composition.

## Risks

- Saved agents are not migrated. Explicit runner changes use normal
configuration/session rules; historical runs retain their recorded
execution.
- Native readiness depends on verified assets and server-owned target
qualification. A Legacy probe, HTTP200 or successful account save cannot
prove native readiness.
- Recovery exceptions remain restricted to validated
completed-onboarding reports; process-loss, ownership and terminal-state
safeguards remain active. Unexecuted live journeys stay open.
- Linux ARM paid tasks, Windows, Alpine and 32-bit ARM remain deferred.
Cold Cargo cache misses are recorded; no warm-cache speedup is claimed.
- Historical invoices/physical cleanup remain unconfirmed and held.
After merge, exact published npm canary, signed standard image/migrator,
private composition and serving/live acceptance still require
qualification. Production deployment/default promotion is not
authorized.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, tools, code execution and
parallel agents. Exact serving snapshot and context-window size are not
exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used with available version and
capability details
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have linked public issues or PRs above
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change and contains no internal
ticket id
- [x] I have run the stated focused tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation
- [x] I have considered and documented risks
- [x] All Paperclip CI gates are green on the current candidate
- [x] Greptile is 5/5 with no open findings on the current candidate
- [x] I will address all reviewer comments before requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-14 11:33:02 -05:00
83e13db8fa fix(inbox): keep automatic recovery cleanup out of personal participation (#16063)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The personal inbox uses issue activity to identify a person's
participation.
> - Issue reads can cancel stale recovery actions and record that
cleanup as the reader's action.
> - This adds unrelated tasks to the inbox and refreshes their activity
timestamps.
> - A workspace owner hold can also alternate between creation and
cancellation after a task finishes.
> - This pull request corrects cleanup attribution, excludes historical
read cleanup, and retains one exact owner hold.
> - Agents can read authorized issue activity to diagnose the same
problem themselves.

## Linked Issues or Issue Description

Refs #15742 excludes logs by the execution-recovery actor from activity
timestamps. This PR covers reader-attributed stale recovery cleanup,
personal inbox participation, exact workspace owner holds, and sandbox
issue activity access. I searched open issues and PRs for recovery inbox
and activity bridge fixes. I found no duplicate for this defect.

**What happened?**

Loading issue lists cancels stale recovery actions under the reader's
identity. The personal inbox treats those rows as user participation.
One completed issue had hundreds of such rows and no edits or comments
by the affected reader. A retained native workspace owner repeatedly
creates a verification action that read projections then cancel. Sandbox
agents cannot inspect the issue activity because the reverse bridge
rejects the route.

**Expected behavior**

Automatic cleanup belongs to the system and does not add an issue to a
reader's inbox. Historical cleanup rows remain auditable but do not
count as participation or recent activity. An unresolved physical
workspace owner keeps one stable verification action. Agents can inspect
activity only for issues their identity may read.

**Steps to reproduce**

1. Finish a task whose native run retains an unverifiable workspace
owner receipt.
2. Repeat workspace recovery checks and GET requests for issue detail,
compact lists, and recovery actions.
3. Observe new recovery action IDs and reader-attributed cancellations
in the activity history.
4. Query the personal inbox for that reader and observe the unrelated
task.
5. From a sandbox agent, request GET /api/issues/:id/activity and
observe Route not allowed.

**Paperclip version or commit**

Observed at f5346c05b9. Regression tests
use current master behavior.

**Deployment mode**

Authenticated hosted instance with native and legacy sandbox agents. The
inbox defect is independent of the adapter.

## What Changed

- Attribute automatic stale recovery cleanup to the recovery system
actor.
- Exclude exact historical read cleanup rows from personal inbox
membership, unread counts, and both activity timestamp queries. Keep all
audit rows and explicit user recovery decisions.
- Retain owner verification holds only while the exact company, run,
issue, and owner fingerprint still match. Apply the same rule to reads
and background recovery.
- Make repeated checks for the same owner hold leave its identity,
attempts, and timestamps unchanged. Keep physical takeover blocked.
- Allow GET issue activity through the sandbox bridge. Keep the existing
company and issue-read authorization and reject broader audit routes.
- Add regressions and update the runtime API and recovery documentation.
Refresh generated source references for the documentation edits. Update
the paired activity, inbox, and recovery maps.

## Verification

- Focused Vitest verification: 315 tests pass across five files. The
updated terminal owner hold regression also passes. The files cover
issue service, recovery routes, activity authorization, sandbox callback
bridge, and native workspace finalization.
- `pnpm -r typecheck` and `pnpm build` pass locally. The full general
and serialized unit test matrix passes in CI on the latest commit. The
duplicate local `pnpm test:run` was stopped after about 30 minutes once
that full CI coverage passed; it did not report a local full-suite pass.
- `node scripts/check-maps.mjs` passes for all paired areas and
capability IDs.
- Apex is 5/5 on commit 44c3787888. Its
access map finding is fixed and resolved. All CI checks are green on
this commit.
- The new regression repeats reads and background sweeps for done and
cancelled tasks. It checks one unchanged hold, zero physical copyback,
zero personal inbox participation, and system cleanup after the exact
receipt is cleared.
- Historical cleanup tests preserve explicit recovery decisions,
nullable metadata, unread counts, and the original audit rows.
- Authorization tests permit an authorized agent and reject
cross-company or issue-boundary reads. Bridge tests reject write
methods, extra path segments, and company-wide audit routes.

## Risks

Existing inbox items can disappear if their only participation evidence
was automatic read cleanup. No data migration or audit deletion is
needed. An unresolved workspace owner remains visible after task
completion; this preserves the physical writer fence. A stale or cleared
owner does not retain the hold. Issue activity access expands only the
sandbox transport allowlist and still uses existing controller
authorization.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository inspection, code
editing, shell execution, and test tools. The session identifies the
model family as GPT-6. A more specific runtime model ID and context
window size are not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Amy (Opus) <noreply@anthropic.com>
2026-10-12 19:25:15 -07:00
DottaandPaperclip a7dd5284b5 feat: add company fast-response acknowledgements (#15748)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Human messages start agent work through task, chat, and
external-channel intake.
> - Agent startup can delay the first visible response.
> - A small direct model call can acknowledge the request while normal
work starts.
> - This response must remain separate from evidence that the agent
answered or completed work.
> - This pull request adds optional company-wide fast responses with
durable receipts, access checks, and cost accounting.
> - Users receive prompt acknowledgement without changing the agent
execution path.

## Linked Issues or Issue Description

**Subsystem affected**

Cross-cutting: database, shared contracts, server orchestration,
connections, and UI.

**Problem or motivation**

Users can wait for an agent to start before they receive a contextual
response. Generic progress messages do not confirm what the user asked.
An acknowledgement must not falsely complete work or satisfy an
unanswered turn.

**Proposed solution**

Add a disabled-by-default experimental fast-response model using an
existing shared API connection. Generate one short acknowledgement for
each accepted human turn. Use the existing agent identity and external
publication outboxes. Retain platform provenance and account for the
inference independently.

**Alternatives considered**

Running the acknowledgement through the agent runtime adds startup
latency. Restricting implementation to OpenRouter leaves other existing
API connections unsupported. This change supports all existing API
connections and uses OpenRouter for live verification.

**Roadmap alignment**

This extends existing agent communication and connection settings.
ROADMAP.md and open pull requests were checked. No duplicate
fast-response implementation was found. The maintainer supplied the
feature plan directly.

## What Changed

- Add Experimental fast responses to instance settings. Keep the feature
off by default. Enforce the flag at intake, dispatch, and publication.
- Move Decision model and Fast response setup to company Settings →
Connections when the experiment is enabled. Place Connections below
Environments. Separate the model sections with clear headings and a
divider. Keep Decision model in General when the experiment is off.
Route Costs setup links to the available settings page.
- Make the acknowledgement prompt a multiline template string. Ask for a
brief, positive response in the user’s language.

- Add settings, connection/model selection, sponsorship, fixed-sample
testing, and Costs history.
- Add a Vercel AI SDK text service for existing native, routed, Bedrock,
and local API connections.
- Add durable per-turn requests, a bounded worker, a three-second
admission deadline, and one inference attempt.
- Integrate task creation, task and agent chat, steering, queued
follow-ups, experimental board chat, shared chat intake, and email.
- Preserve receipt provenance and exclude receipts from completion,
recovery, interaction resolution, and agent wakeups.
- Recheck authorization, source validity, real replies, budgets, and
destination binding before publication. Preserve accounting for
suppressed or uncertain outcomes.
- Add provider contracts, database race tests, channel outbox tests, UI
tests, Stories, and documentation. Keep decision-model execution
unchanged.

## Verification

- Local `pnpm -r typecheck`, `pnpm build`, and UI token gates pass after
rebase. The UI typecheck and build also pass after the final navigation
fixes. All 37 focused navigation tests pass. The first full local `pnpm
test:run` invocation reported seven chat failures. All seven passed in
isolated reruns after fixture updates and host cleanup. This invocation
was stopped after the full final-head CI suite passed.
- Initial local database tests hit macOS shared-memory exhaustion. A
diagnostic confirmed `shmget: No space left on device` with the system
limit of 32 IDs. Three abandoned, unattached PostgreSQL IPC headers were
reclaimed. The isolated rerun passed all 31 fast-response and channel
tests. The two additional Slack regression cases also passed in a
separate rerun. No active database was stopped.
- UI token gates pass. Provider contract tests cover all supported API
connection types. Fast-response tests cover uniqueness, authorization,
budgets, expiry, cancellation, restart, real-reply races, and the
experimental flag.
- The test drive verified that disabling the experiment hides
Connections and redirects its direct URL. Enabling it restores the saved
OpenRouter setup. After restart on the final commit, the OpenRouter
settings sample returned “Sure, I’ll take care of fixing the border
around the settings panel.” in 428 ms. It used 260 input and 31 output
tokens and recorded $0.0000576 in reported cost.
- Earlier live OpenRouter verification produced a receipt in 1.013
seconds for a new task. The real agent then supplied its substantive
answer. A separate settings test took 1.34 seconds, used 246 input and
23 output tokens, and recorded $0.0000361.
- Live Slack verification on final commit
`24016fa525961ebc7a4acffdc5019ce123970549` passed for a fresh initial
task and a follow-up in a disposable bot DM. The initial acknowledgement
appeared 2.405 seconds after the Slack message and 1.413 seconds after
durable acceptance; the follow-up appeared in 2.322 seconds and 1.160
seconds respectively. The recorded generation/settlement durations were
898 ms and 697 ms. The initial call used 291 input / 39 output tokens
and cost $0.00006705; the follow-up used 281 / 47 and cost $0.00007035.
- The Slack acknowledgements retained the assigned agent identity and
`origin: fast_response`, source-turn references, and no fabricated run
ID. A status snapshot after the initial receipt showed the task still
`in_progress` with its real execution run. The agent then read the CSV
fixture and correctly reported two ready items, two blockers, and their
owners. Both replies stayed in the originating DM, and no duplicate
generic working message followed the delivered contextual
acknowledgements. The `new` command started a fresh conversation session
without generating a fast response.
- An earlier eligible Slack turn generated a response but suppressed
publication. Its reported cost was retained, generic progress remained,
and the real agent completed the requested result. This is recorded
separately from the two delivered acknowledgements; the history did not
expose a precise suppression reason.
- Slack verification used real signed provider events, the existing
intake/publication paths, and a user-approved temporary webhook-only
HTTPS tunnel. Subscriptions were limited to direct-message events. The
test bot was paused and the tunnel stopped afterward. Live
channel/group-thread and GitHub journeys remain unverified; their mocked
tests cover intake, destination binding, publication, and continued
underlying work.
- Greptile reviewed final commit
`24016fa525961ebc7a4acffdc5019ce123970549` at 5/5. Both new review
findings are fixed and all threads are resolved. [Final-head CI
run](https://github.com/paperclipai/paperclip/actions/runs/38060015730)
passed. This includes grouped server/chat/workspace tests, serialized
suites, runner checks, browser journeys, typecheck, build, release
registry, policy, and the canary dry run.
- Manual check: enable Experimental fast responses. Open company
Settings → Connections. Select a shared API connection and model. Save
and use Test response. Submit a task message and confirm that its
receipt does not complete the task.

## Risks

- The migration adds receipt and configuration tables plus accounting
references. It uses idempotent statements and follows the current
migration sequence.
- A slow provider, authorization change, or real-agent reply can
suppress an acknowledgement. Existing progress feedback remains
available.
- Unknown provider billing retains a reservation until reconciliation.
External deliveries with uncertain outcomes must not be blindly
repeated.
- Live Slack initial-task and follow-up DM journeys pass. Live Slack
channel/group-thread and GitHub journeys remain unverified. The PR
remains a draft and records these gaps. Final-head repository CI and
automated review pass.

## Model Used

OpenAI Codex, based on GPT-6, with reasoning, repository tools, code
execution, and browser interaction. The runtime does not expose an exact
model snapshot ID or context-window size; these are not asserted.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-10 13:09:17 -05:00
Devin FoleyandPaperclip 3389bea708 fix(recovery): surface exhausted review-path repair (#15764)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - An issue in review needs a saved path to its next action.
> - A consumed review path starts one automatic repair run.
> - The repair run can finish with only a promise to check later.
> - The old code skips another wake but records no recovery action.
> - This change records a board decision when that repair also leaves no
saved path.
> - The issue lock protects a new monitor, a changed owner, or newer
work from stale finalization.

## Linked Issues or Issue Description

**What happened?**

An agent waiting for an external build said it would check again soon.
Its one-shot monitor had already fired. The automatic review-path repair
also finished without saving another monitor. The task stayed in review
with no future wake or recovery card.

**Expected behavior**

The repair must save a monitor or another review path. If it cannot do
so, Paperclip must show an explicit recovery action and stop automatic
retries.

**Steps to reproduce**

1. Leave an agent-owned issue in review after consuming its last monitor
or interaction.
2. Let the source run finish without saving another review path.
3. Let the single automatic repair run also finish without saving a
path.
4. Before this change, the task stays in review without a scheduled
follow-up. After this change, it is blocked with a board-owned recovery
action and an inline notice that links the repair run.

Related work: Refs #14757 (preserve active review during stale
recovery), Refs #11925 (review-path liveness). This PR handles exhausted
repair wakes; it does not change path aging.

## What Changed

- Return an explicit exhausted result after the single review-path
repair attempt.
- Record the recovery action, blocked state, notice, and activity in one
transaction. Preserve the original assignee. Keep implicit service reads
on the transaction connection.
- Recheck the company, owner, saved review paths, and latest issue run
before escalation. Publish activity after commit.
- Tell the repair agent to save a one-shot monitor and report its check
time when an external check remains pending.
- Retry missed escalation from the saved terminal repair run during the
stranded-issue sweep. Cover restart recovery and rollback with a
one-connection pool.
- Preserve valid due and claimed monitors for dispatch while the owner
permits on-demand wakes. Surface overdue monitors after those wakes are
disabled. Keep the future-only liveness classifier unchanged.
- Cover monitor repair, stale finalization, company boundaries,
duplicate finalization, and failed repair. Document the behavior.

## Verification

- `pnpm exec vitest run
server/src/services/recovery/review-path-recovery.test.ts
server/src/__tests__/heartbeat-issue-liveness-escalation.test.ts
server/src/__tests__/issue-liveness.test.ts` — 62 tests passed,
including PostgreSQL integration tests.
- `pnpm --filter @paperclipai/server exec tsc --noEmit` — passed.
- `pnpm -r typecheck` and `pnpm build` — passed.
- `pnpm test:run` — 18,144 passed, with three unrelated local fixture
failures. Two Git fixtures pass when run with their expected initial
branch (`master`); this machine defaults to `main`. The native same-task
replacement fixture also fails on an unchanged checkout of master. No
unrelated production behavior was changed to accommodate those fixtures.
- Apex review — 5/5 on commit `3aca7059848`, with no unresolved review
comments.
- Latest-head CI — all required checks passed, including test shards,
typecheck, build, browser checks, and canary dry run. No merge
conflicts.
- Repeat the steps above. Confirm that the issue has one recovery
notice, one board-owned action, and no second automatic repair wake.
Save a monitor during repair and confirm that the issue stays in review.

## Risks

- A review repair that exhausts its one attempt now moves the task to
blocked and asks the board to act. This is an intentional behavior
change.
- The check uses the current review-path classifier. A path that the
classifier treats as healthy still suppresses escalation.
- No schema, API, or UI contract changes are required. Existing recovery
controls render the action and notice.

## Model Used

OpenAI GPT-6 through Codex, with tool use and code execution. The
runtime does not expose a more specific model ID or context window.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (62 tests for this change;
full-suite fixture limitations are listed above)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-09 18:56:20 -07:00
DottaandPaperclip b6b3cb3761 Fix unavailable workspace branches with an inline repair and retry (#15726)
## Thinking Path

> - Paperclip manages AI agents and their work.
> - Isolated workspaces must resolve a starting Git ref before an agent
starts.
> - A task can explicitly select a branch that the repository does not
have.
> - The current recovery notice calls this a missing secret and offers
no branch repair.
> - This pull request explains the branch failure and adds a task-scoped
repair action.
> - Operators can correct the branch and request a guarded retry from
the task thread.

## Linked Issues or Issue Description

Related: Refs #11892 added unresolved-ref recovery. Refs #15719 added
bounded failure diagnostics. This change uses that recovery path and
adds its operator repair UI.

**What happened?**

A task that selects `main` in a repository with only `master` stops
before the adapter starts. The task shows a missing-secret message and a
generic retry. Repeating the run cannot fix the stored branch setting.

**Expected behavior**

Explain that the starting ref could not be resolved. Offer a verified
repository default, or let the operator enter a branch. Save only the
task's starting branch and retry its recorded owner through the existing
admission gates.

**Steps to reproduce**

1. Create an isolated-workspace task in a repository with `master` and
no `main`.
2. Set the task's base ref to `main` and assign an agent.
3. Wait for the configuration failure.
4. Use the inline branch repair and confirm the agent completes in a
worktree based on `master`.

## What Changed

- Replace the missing-secret diagnosis for unresolved workspace refs.
Keep uncertain fetch/access failures described as uncertain.
- Read the remote's advertised default branch with the existing Git
credentials and a bounded timeout. Do not guess or silently change the
branch.
- Add an atomic branch repair to the existing recovery resolution
endpoint. Check company access, board runtime permission, incident
identity, current owner, task/project state, dependencies, interactions,
approvals, budget, and active execution before requesting a retry.
- Preserve the effective inherited workspace setup when saving the task
branch override, and attach one repair notice per failed run even when
the incident is reused.
- Bind the shared repair card to the exact recovery notice and failed
run in both task-thread renderers. Validate branch syntax and prevent
duplicate submissions.
- Add Storybook examples, regression tests, and a documented local
acceptance fixture with a real Git repository and process adapter.

## Verification

- Red/green regressions cover the repair endpoint and preservation of
inherited agent setup. The regression failed before each fix and passed
afterward.
- Focused integration suite: 162 tests passed; task-thread/card suite:
230 passed. After review refinements, all 13 repair-route tests, 10 card
tests, 77 recovery-action tests, and the repeated-failure heartbeat
regression passed locally.
- Real browser acceptance passed for suggested-branch and custom-entry
repairs. Invalid syntax left the task blocked. Both valid repairs ran
the authenticated worker in a real isolated worktree and reached Done;
reload preserved the result.
- Repeated failures reused the recovery incident and retained an
actionable notice on the latest run. The inherited-setup journey
verified the provisioned file, agent branch template, worktree parent
directory, and actual Git commit before completing. [Before/after
screenshots](https://github.com/paperclipai/paperclip/pull/15726#issuecomment-6088734666).
- Full local typecheck and build passed; server typecheck passed again
after review fixes. Token gates passed.
- The monolithic local Vitest run was stopped after resource-contention
timeouts and stale assertions. Targeted regressions were rerun; the full
CI matrix passed on final revision
`26c543ff490bec29ddc77a49ee532c7f84fb96dc`. One signoff E2E
worker-cancellation failure passed on a rerun without code changes. The
two opt-in Storybook workflow jobs were skipped; the Storybook journey
was checked manually.
- Repeat the live journey with `tests/e2e/workspace-base-ref/README.md`.
Storybook examples use simulated callbacks and are not the live
acceptance proof.

## Risks

- Default-branch discovery can add up to 10 seconds to an unresolved-ref
failure. Lookup failure leaves manual entry available.
- A valid branch name can still be absent or inaccessible. The retried
workspace preparation remains authoritative and can block again with new
evidence.
- Retry remains subject to normal admission gates. The repair changes
only the task override; it does not alter project or agent defaults.
- No database migration or new dependency.

## Model Used

OpenAI GPT-6 through Codex, with tool execution, repository analysis,
and browser testing. The exact serving variant and context-window size
are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-09 15:56:14 -05:00
Devin FoleyandPaperclip 7a58591721 fix(recovery): preserve accepted workspace finalization during reaping (#15718)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - A native run saves its accepted result before it copies workspace
changes back.
> - A restart can leave that copyback under an unverified physical
owner.
> - The generic orphan reaper can mistake the exited provider for lost
work and release the source lease.
> - Recovery must preserve the accepted result and let the existing
finalizer settle its remaining work.
> - This pull request checks that authority at the conditional
process-loss write and preserves the matching recovery hold.

## Linked Issues or Issue Description

Related: #15475 added bounded process-loss evidence. The earlier #9687
addresses dead-run finalize barriers with synthetic receipts. This
change preserves an existing accepted native result and requires real
workspace settlement.

**What happened?**

After a server restart, the native finalizer required operator
verification of unfinished copyback. The generic orphan reaper then
marked the run `process_lost` and released its source lease. A new
execution path also made the recovery reconciler clear the copyback hold
before physical ownership was verified.

**Expected behavior**

An accepted result bound to the same company, issue, and run keeps its
unfinished workspace suffix under native finalization. The generic
reaper must not discard its source. The existing finalizer must still
complete the work, apply its bounded failure policy, or require operator
verification.

**Steps to reproduce**

1. Accept a native result and leave its coordinator in
`workspace_finalizing`.
2. Stop the controller before copyback completes, leaving its physical
owner receipt.
3. Start recovery, then run the generic orphan and lease cleanup sweeps.
4. Before this change, the run can fail with `process_lost` despite its
accepted result and explicit copyback hold.

**Paperclip version or commit**

Reproduced against `011e5669`; the same recovery code remains at base
`1602512820`.

**Deployment mode**

A hosted server with PostgreSQL and native runner execution. The tests
use only isolated local PostgreSQL and synthetic fixtures.

## What Changed

- Check the exact accepted result and `workspace_finalizing` coordinator
before generic process-loss cleanup.
- Repeat the check in the conditional status write. Compare the
heartbeat tuple version to reject a result publisher that commits while
the update waits on its row lock.
- Preserve nonterminal copyback holds when another execution path
appears. Resolve only the matching run's hold after verified
finalization commits.
- Add fourteen real PostgreSQL service regressions and document the
recovery boundary.

## Verification

- `pnpm -r typecheck` passed.
- `pnpm exec vitest run
server/src/__tests__/heartbeat-process-recovery.test.ts -t 'native
finalization'`: 14 passed. The test filter excluded 383 existing tests.
- Tests cover startup, repeated reaper and cleanup sweeps, malformed and
foreign owners, an expired old controller, exact database bindings,
legacy and unsupported states, verified settlement, and unchanged
terminal-task dispositions.
- The concurrent publisher test uses the real accepted-result
transaction and proves the reaper update is blocked by its row lock
before the publisher commits.
- The local `pnpm test:run` aggregate was stopped after setup timeouts
in the chat and tool-access files and a damaged-epoch Runner test
failure. It is not a full local pass.
- The chat file passed all 1,145 tests when run alone. The tool-access
file then passed 419 of 420; one case returned `socket hang up`. That
single case passed on its isolated rerun. The cause of the socket
failure is not established here.
- The damaged-epoch Runner case still fails locally at its existing
completion assertion. The same assertion failed on untouched baseline
`556daf9a` before this patch; its test and Runner sources are unchanged.
Exact-head CI passes this coverage. No workaround for that local fixture
is included.
- `pnpm build` passed.
- Independent review approved this exact commit. All 14 new PostgreSQL
regressions and 181 existing recovery/finalizer cases passed. One
existing suite needed an isolated rerun after a setup timeout.
- Independent negative controls reproduced the original bug. Removing
only the tuple-version guard reproduced the blocked-row race. Removing
only the hold exclusion cleared the safety hold too early. Restoring
this patch passed all three checks.
- GitHub CI passed all 53 applicable checks. The two Storybook jobs were
skipped as expected. Greptile scored this commit 5/5 with no unresolved
review threads.
- Local runtime: Node 25.6.1. GitHub CI supplies the repository's Node
24 checks.
- `git diff --check`, Gitleaks, and a local diff scan for secrets and
private data passed.

## Risks

- PostgreSQL tuple versions are used only as an optimistic fence for
`process_lost`. Both accepted-result publishers lock and update that
heartbeat in their result transaction.
- The guard requires the bound accepted result and the durable
coordinator state. A heartbeat phase or arbitrary owner JSON alone
cannot suppress ordinary process loss.
- Existing finalizer retries, ownership verification, explicit
cancellation, and terminal-task dispositions remain unchanged. A
physical owner can still require operator verification.
- This change does not replay provider work or restore a source already
destroyed by an older controller. No migration or runtime operation is
included.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, repository tools, code
execution, and independent agent review. The exact service model ID and
context-window size are not exposed in this worker session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-09 12:28:16 -07:00
Nicky LeachandPaperclip 2dd9811e8b Own agent hiring and termination in a lifecycle module (#15631)
## Thinking Path

> - Paperclip manages AI agents and their work.
> - Agent creation, approval, pause, and termination change whether an
agent can work.
> - These changes currently use separate service and database paths.
> - Some agents need setup after their record exists and before their
harness can run.
> - This PR adds one module that owns those state changes and their
transactions.
> - The module records progress and retries incomplete work after a
server restart.

## Linked Issues or Issue Description

**Problem or motivation**

A hire can become active before its execution environment is ready.
Direct agent state writes also bypass one common lifecycle boundary.
Termination does not have a durable step for resource cleanup.

**Proposed solution**

Use agent lifecycle commands for hiring, approval, pause, resume, and
termination. Keep proposed hires pending until approval. Prepare each
approved hire, test its saved harness configuration, and then permit
work. Wait for required plugin work before completing a step.

**Alternatives considered**

A general database commit hook would expose transaction details to
callers. A timer alone would delay each new hire. This module owns each
command transaction and starts its worker after commit. A periodic scan
recovers work after a process stops.

**Roadmap alignment**

This change adds the lifecycle boundary for the existing agent and
plugin systems. It does not add a user interface or a provider
implementation. It replaces the lifecycle delivery approach in #15391.
Related proposals include #11475 and #13099; this change does not add
their external executor protocols.

## What Changed

- Add durable approval, preparation, verification, pause, resume, stop,
and cleanup states.
- Keep agent creation, lifecycle state, configuration writes, and agent
record deletion behind `modules/agent-lifecycle`. Keep validation,
credential rules, reads, onboarding, invitations, general approvals, and
company data deletion in their services.
- Commit a hire before the remaining onboarding or invitation work.
Serialize requests with the existing dedicated database connection
helper. Reuse the hire after a failed attempt.
- Keep accepted work in the run queue during setup. Require execution
and cleanup evidence before completing pause or termination.
- Retain the original credential owner when approving migrated hires.
- Start lifecycle work after each command commits. Use database leases
and revision checks for retries and late results.
- Add a generic required plugin method and capability. Store the
required plugin IDs for each agent. Track host and plugin completion
separately.
- Keep harness verification and credential rules in services. Supply
their results to lifecycle through explicit ports. Reject failed logins
and incomplete probes.
- Remove reverse imports from lifecycle adapters into services. Share
record queries and validation below both layers.
- Use one hire decision path for agent IDs and approval IDs. Resolve the
approval and agent change in one transaction. Expose policy
reconciliation on the command object.
- Save configuration and verification invalidation through one module
operation. Ordinary commands own their transaction. Restrict the shared
transaction integration to its listed callers. Reject identity, company,
accounting, timestamp, and unknown fields.
- Require completed termination before deleting agent records. The
company deletion service calls the module through a separate deletion
interface.
- Save an approved secret before applying its agent binding. A failed
binding leaves the secret approved. A retry reuses it.
- Remove old permission and metadata export files. Move hire
configuration rules out of lifecycle persistence.
- Report failed worker steps and scans with operation context and known
error codes. Omit provider messages, stacks, and configuration values.
Document ordinary application use and extension rules.
- Recover expired native owners only after execution stops. Retry
cleanup of failed retained leases. Wait for setup in the live smoke
script.
- Keep manual, budget, and company pause reasons separate. Guard
execution status writes with the ready state.
- Add a migration, boundary checks, regression tests, and a lifecycle
guide. Keep the current UI.
- Limit periodic policy checks to 100 eligible agents per minute.
Continue from the last agent ID. Exclude pending hires and agents in
termination or a final state. Keep explicit company and budget updates
immediate.

## Verification

- Latest head `ccc3c86b0` aligns the shared status update schema with
the PATCH route. All 15 shared tests, 104 server route/OpenAPI tests, 2
CLI tests, and shared/server/CLI TypeScript checks pass. The Greptile
comment is fixed. CI is running; no new Apex review was requested for
this small fix.

- Rebase `75708ac94` rebases onto master at `2eb5187e4`. The backup test
uses the shared 90-second timeout from master; the older timeout commit
was removed. The affected backup test passes.
- Agent budget edits now pass the target agent ID through enforcement.
All 50 affected tests, server TypeScript, and 11 boundary tests pass.
Both configuration update paths have regression coverage. CI is running
on the latest head. No new Apex review was requested for these small
fixes.

- Rebase `9b568666a` preserves the run-control extraction and
delivery-work startup changes from master. All 559 affected tests,
server TypeScript, and 11 boundary tests pass. CI and Apex review are
running on the final head.
- The policy scan fix passes 56 lifecycle and diagnostic tests, 24
policy and caller tests, and 11 boundary tests. A regression test checks
the page limit, interval, excluded states, and cursor wraparound.

- Rebase `fd4698750` passes all 611 tests in eight affected lifecycle,
route, and heartbeat suites. Server TypeScript and all 11 module
boundary tests pass. The company deletion rollback and concurrent hire
tests are included.

- Follow-up `00aac7646` fixes the failed decision deletion test and the
public update schema. All 126 affected server tests, 8 shared tests, 2
CLI tests, and 11 boundary tests pass. Shared, server, CLI, and UI
TypeScript checks pass.

- Server TypeScript passes: `pnpm --filter @paperclipai/server exec tsc
--noEmit`.
- All 11 boundary tests pass: `node --test
scripts/check-module-boundaries.test.mjs`. The repository boundary scan
also passes.
- All 68 lifecycle, failure diagnostic, and harness verification tests
pass. Tests reject forbidden fields through the root command, combined
transition command, and restricted transaction integration. They also
check invalid fields returned by configuration preparation.
- Failure tests cover host work, plugin work, claims, renewals,
deferrals, and scans. Log tests check known error codes and exclusion of
private error contents.
- All 355 tests in 10 existing caller suites pass. These tests cover
secret approval, pairing, pending hires, credentials, company import,
built-in agents, managed agents, instructions, and skills.
- The branch is rebased onto `origin/master` at `2eb5187e4`. The queue
guards and cancellation lock fix use the extracted heartbeat queue.
Agent cancellation uses the extracted run-control file. The source diff
scan found no secrets or private URLs.
- Earlier checks cover approval retries, rollback, late verification
results, policy holds, deletion rollback, concurrent company deletion,
and queue admission during setup.
- Before the rebase, CI passed on `00aac7646` and Greptile Apex rated it
5/5 with no open comments. CI and a new Apex review are pending for the
rebased commit. The branch has no merge conflicts.
- The full local build and native Dot suite were not run. The host has
no Cargo. No live provider test was run.

## Risks

- Background policy recovery takes more than one minute when more than
100 agents are eligible. Explicit company and budget updates still
reconcile immediately.

- The restricted configuration integration keeps an agent reference and
its dependent records in one transaction. It cannot change a budget or
invoke a lifecycle transition. A periodic scan starts pending setup
after that transaction commits.
- Secret approval and binding use separate transactions. A failed
binding can leave an approved secret with no agent binding.
- New approved hires remain unavailable until setup and verification
complete. Existing clients see `paused` during these steps. This PR does
not change the UI.
- Migration `0320_daily_onslaught.sql` maps existing statuses and pause
reasons. It does not create external resources or claim that an existing
harness passed verification.
- Use a coordinated server update for this migration. Older server
versions can write legacy status without the new lifecycle checks.
- A failed onboarding or invitation request can leave a committed hire.
A retry reuses that hire and completes the remaining records.
- Deletion requires completed termination. Company deletion can require
another request while cleanup is pending.
- Required plugins must make repeated calls safe and reject older
revisions. A disabled or removed required plugin blocks completion until
it is restored.
- A new hire can run a second harness test after the existing UI test.
Some adapters can charge for this test.
- No live provider test was run. Provider implementation and resource
backfill remain separate work.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, code tools, and local test
execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have described the issue in this PR with the feature request
field labels
- [x] I have not referenced internal or instance-local issues or links
- [x] My branch name describes the change
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation
- [x] I have considered and documented risks
- [x] Targeted local tests pass
- [x] All Paperclip CI gates are green on the new commit
- [x] Greptile Apex is 5/5 with no open comments on the new commit
- [x] I will address review comments before requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-09 09:05:12 -07:00
Devin FoleyandPaperclip f1c44b7b56 Preserve remote work when required workspace restore fails (#15479)
Persist exact source-retention obligations before run finalization and protect them across cancellation, restart and task changes. Require board-authorized repair evidence without replaying old work or changing current task ownership, state or locks. Hide an unavailable retry action and document operator recovery and retention costs.

Validated with 674 scoped regressions, 224 combined integration tests, full typecheck/build, independent safety reviews and green CI with Greptile 5/5. No historical file recovery is claimed.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-07 13:29:44 -07:00
DottaandPaperclip 3a726e676f feat: enforce private task permissions across execution and data (#10633)
Enforce private task and project access across direct reads, search, execution, files, plugins, live delivery, and sharing mutations. Preserve downward-only sharing, current responsible-user authorization, and audited emergency access. Bind historical draft assets with migration 0314.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-07 07:06:40 -05:00
Devin FoleyandPaperclip 799e4d556f fix: make accounting durable and synchronize cost reporting (#14997)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-06 22:26:48 -07:00
DottaandPaperclip a6306ba606 feat(runner): consolidate Cursor production integration (#15075)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The native Runner keeps provider sessions under company authority,
approvals, budgets and durable recovery.
> - Cursor work was spread across candidate branches. The published
branch lacked later plan, permission and cleanup fixes.
> - Production also needs public installation and matching runtime
assets for local and Daytona execution.
> - This pull request consolidates Cursor onto current mainline recovery
behavior and completes that installation path.
> - The installed v11 release passed focused local and Daytona
qualification after the generic mode and lifecycle cleanup. The later
model-selection correction and current mainline merge produce v14
artifacts that need matching release qualification.
> - Cursor admission is enabled in source; publish only an artifact
combination with matching qualification. Native AskQuestion and complete
per-run dollar accounting remain excluded.

## Linked Issues or Issue Description

Refs: #14435, #14631, #14669, #14699, #14724.

This completes the Cursor implementation by @cryppadotta from combined
source `22c78242a4e0c2369fecf0c2dc4e7600fbad6706`. It preserves newer
mainline recovery, completion and warm-directory behavior. Pi and
Copilot remain gated.

## What Changed

- Generate named Rust and TypeScript ACPX release profiles from one
manifest. Share runtime pins with packaging and server verification.
Preserve vendor runtime versions; bind the updated ACPX patch to Cursor
profile v14 and reject stale generated declarations at build/typecheck.
- Remove ACPX model allowlists, including the former Codex and Pi
restrictions and the duplicate developer test-drive gate. Send any
explicit model ID unchanged to its provider and verify the effective
selection before prompting. The bundled ACPX package forwards unlisted
IDs, rejects mismatched acknowledgements, and restores the exact
selection after session load. It does not expand Cursor model aliases.
Provider rejection, mismatch, or missing model controls fails without a
fallback. Model examples live in evaluation fixtures, outside runtime
declarations.

- Add pinned Cursor execution, contained instructions, exact model
verification and Agent/Plan/Ask modes.
- Carry an opaque generic `mode` identifier in shared native execution,
sidecar, Rust and recovery contracts. The provider adapter owns
supported modes, defaults, native translation and acknowledgement.
- Keep native RPC recognition, accepted-plan interpretation and
permission evidence behind provider adapters. Shared settlement and
recovery verify normalized facts and their committed evidence.
- Replace the Cursor-only warm-attachment branch with a runner-owned
capability. Only Cursor opts into it. Move profile compatibility and
optional usage parsing into provider metadata and adapters.
- Write generic plan-wait receipts. Read exact historical Cursor
receipts through a separate compatibility decoder. Reject mixed formats
and preserve existing authority checks.
- Carry native plans, semantic questions, todos, child activity,
permission identities and partial usage diagnostics through the Runner.
- Preserve durable response delivery, cancellation, warm ownership and
process retirement.
- Finish accepted planning runs successfully. Keep their tasks open for
explicit direction. Acceptance does not start implementation.
- Ship `paperclipai runtime setup cursor` and its provisioner through
the public package. npm installation does not download Cursor. Setup
uses the OS account's closure-keyed cache so system-wide npm packages
can remain read-only. Run it as the Paperclip service account.
- Include Cursor in normal provider packs and Daytona images for macOS
ARM64/x64 and Linux x64.
- Reject stale release packs by source revision and current ACPX/Cursor
pins before assembly writes files. Verify current Cursor
version/profile/closure again at runtime.
- Ship all three daemon targets and the expected Linux image-pack
identity. A macOS controller uses its packaged Linux daemon for Daytona.
Image mismatches fail before provider launch.
- Use the vendored Runner boundary for installed readiness probes.
Verify the actual installed Cursor probe.
- Verify compiled public Daytona plugins and their release versions in
installed smokes.
- Record exact artifacts, the acceptance matrix, retained failures,
supported capabilities and rollback behavior in the [readiness
report](https://github.com/paperclipai/paperclip/blob/codex/cursor-production-readiness/doc/plans/2026-10-03-cursor-production-readiness.md).

## Verification

- Current head `f7ec5cc1f0e30c62a829c016ff2013a2a9d79037` merges
mainline `faa8e452c73bae5e044dd6379179a00106abb131`. It keeps Cursor
plan and cancellation guards alongside mainline historical-question
filtering. The evaluation catalog includes both Cursor and expanded
adapter accounting cases (683 total). Recursive typecheck, full build,
696 lifecycle/recovery tests, 45 fixture tests and fixture typecheck
passed. Current-head CI passed: 56 successful checks, one neutral and
four skipped. [Complete
CI](https://github.com/paperclipai/paperclip/actions/runs/37557996535).
The fresh Base Greptile review is 5/5 on this exact head, with 304 files
reviewed, zero new comments and zero unresolved threads. The user
authorized overriding the CODEOWNER review gate after checks passed; no
failing checks are overridden. Prior results below retain their own head
identities.
- Corrective head `3d2b168366258036f6b6a6fccb382c49138cc601` fixes the
post-merge Apex finding. Automatic-review and new-evidence
reconciliation preserve pending child results and recheck delivery under
the status lock before completing. Account repair now excludes unrelated
secret consumers and requires the failed agent's identity. Regression
coverage includes the commit race, delivery statuses,
current-run/current-intent exclusions, repeated reconciliation, both
database reconciliation paths, and credential consumer boundaries. All
184 affected tests, server typecheck and server build passed.
Current-head Base Greptile review is 5/5, with 304 files reviewed, zero
new comments and zero unresolved threads. Current-head CI passed: 56
successful checks, one neutral and four skipped. [Complete
CI](https://github.com/paperclipai/paperclip/actions/runs/37535994724).
This Base review is distinct from the earlier Apex review.
- Merge head `5957c257a` reconciles mainline `b508a05c4`. It preserves
both accepted-plan waits and pending-child-completion checks, current
provider selectors, task-creation response identities, and mainline ACPX
missing-file handling. The combined patch is bound to Cursor profile
v14; historical records keep their original identities.
- Merge head `5957c257a` passed recursive typecheck, full build, 43
installed ACPX/package contracts, 107 provider UI and plan/recovery
tests, 593 database-backed lifecycle tests, 49 profile/native contract
tests, 45 Product E2E fixture tests, fixture typecheck, token gates,
three provider-free browser task-creation cases, and Runner
conformance/replay checks. Its complete CI passed (55 successful checks,
one neutral and four skipped), while Apex returned 2/5 with a
child-delivery finding addressed below.
- The local full-suite attempt again failed the unchanged Git streaming
test (360-second timeout) and was stopped. The concurrent local Rust
attempt failed four unchanged Codex process/deadline tests; all four
passed serially without code changes in 7.29 seconds after removing the
competing test load. These failed commands are retained and are not
reported as full-suite passes; the fresh Linux CI runs are tracked
separately.
- The previous head `907bdb2a2778c7ffeb4a662a91460c9d1ddfc9c5` earned
Apex 5/5 with zero comments after fixing all three findings: per-user
install cache, stale release-pack rejection, and public Linux smoke
account/home handling. Its real built installer passed from read-only
public packages on macOS ARM64 and Linux x64. All 137 release-registry
checks and 64 ACPX package contracts passed. That review does not cover
this mainline reconciliation.
- Prior `beadd3654` passed the full CI matrix; its one unchanged chat
test failure and successful single retry remain in the [CI
history](https://github.com/paperclipai/paperclip/actions/runs/37521449327).
Historical results below remain attributed to their original builds.

- Fixture follow-up `dd59d7e82b103a88b7cbd7d2c38b612c0fbbff7a` removes
provider-specific model choices from generic offline ACPX tests. The
fake sidecar preserves the model and session identity selected at open
through suspension. Affected verification passed: 106 Rust tests and 73
TypeScript tests. This commit changes test code only; the
production-code checks below retain their recorded identities. Its CI
and Greptile review later passed; those results belong to that
historical head.
- Model-selection cleanup `9a070808b48960a41fdfd369ae0636b95af82459`:
252 focused Runner tests passed (six platform skips), covering all six
ACPX agents, native model acknowledgement, rejected selections,
installation integrity and recovery identity. The merged branch passed
recursive typecheck, full build, token gates, server admission (19
tests), and the Product E2E catalog (45 tests). The acceptance catalog
passed all four tests. The full Rust suite passed: 643 tests, 2 ignored.
It verifies sidecar acknowledgement of unlisted models and rejection of
model mismatches. The final commits only update Rust tests; production
sources match the verified build at
`65ec3279ac50185e3cda109b5cfd9b4f56105de0`. No new paid provider calls
were made.
- The merge preserves both Cursor and the new mainline public-MCP
fixture cases. Auto-merge remains disabled; the latest follow-up status
is recorded above. The local `pnpm test:run` attempt hit the unchanged
Git streaming test's 300-second timeout and was interrupted before
merging mainline. The broad Runner attempt found obsolete single-model
assertions plus three macOS fixture-path failures caused by a
`/private/tmp` override. The assertions are corrected; affected
TypeScript checks passed with the standard macOS temporary directory,
and the complete Rust suite passed. Neither interrupted command is a
full-suite pass.
- Earlier declaration-cleanup head `6f4a5e9e2` passed recursive
typecheck, build, Rust and focused tests. Its CI later exposed a test
expecting duplicated Grok digest literals. The current source fixes that
assertion to compare launcher bytes with the shared manifest. Historical
successes and failed attempts are retained; no new live provider
qualification is claimed.
- Previous head `e75fde6098b0ddd8cec765bfb6ecaeecb88a26a6` passed
complete CI (56 successful checks, one neutral, four skipped) and
Greptile 5/5. [Historical complete
CI](https://github.com/paperclipai/paperclip/actions/runs/37489112305).
Those results are not claimed for the cleanup head.
- Frozen live application: `d7b696f9b8f79095233e9e3d56d23e6a6018dd48`.
Public package version: `0.0.0-cursor-verify.3d0c9b7761c6`. The
declaration cleanup preserves release pins and does not relabel that
tested artifact as a build of the new source. Mainline through
`e34abee670` was reconciled while preserving accepted-plan waits,
provider-capacity handling, and both Cursor and public-MCP fixtures.
- Clean normal installation, explicit Cursor setup and daemon resolution
passed on macOS ARM64, macOS x64 under Rosetta, and Linux x64. npm
lifecycle hooks ran without silently downloading Cursor.
- Historical v11 live matrix: **18/18 passed with cleanup** (nine local,
nine Daytona) after the generic mode and lifecycle cleanup. The campaign
has 23 attempts; all five failures and their diagnoses remain recorded.
Exact case identities, hashes and limits are in the readiness report.
All provider calls are real, use the explicit Luna model and
company-bound credentials, and run without qualification or
runtime-asset overrides.
- The immutable Daytona image is
`ghcr.io/paperclipai/paperclip-daytona-runner@sha256:d6259b6bba094702c13fc2283bd85550849c1c53145b656fb2746778f9fa1747`.
The public Daytona plugin is installed independently and its version is
checked.
- Recursive typecheck, full build, token gates and Runner
contract/conformance/replay checks passed on the frozen application. Its
complete Linux CI suite passed. The duplicate local full-suite command
was incomplete after timing failures; affected repeats passed, but that
command is not reported as a clean pass.
- Qualification fixtures passed typecheck, 1,675 Vitest tests (one
skip), 128 Node checks, three provider-free browser tests, and 150
focused lifecycle tests after the final diagnostic correction. The
affected legacy Cursor command file also passed all five tests after
removing its shorter 10-second override; it now inherits the suite’s
standard 15-second timeout. Greptile is 5/5 on `e75fde609` with no
unresolved review threads. CI results above are recorded separately from
historical build results.

## Risks

- Cursor v14 includes the updated ACPX dependency patch and release
identity. The v11 live matrix and image below remain historical
evidence. They do not certify new v14 package/image artifacts.

- ACPX accepts models beyond the qualification fixtures. Availability
and entitlement depend on the provider. Successful configuration is not
a claim of live qualification for every model.
- Shared mode is an opaque identifier. Provider adapters own its
meaning. Incompatible historical sessions remain fenced; exact committed
plan waits and task history remain inspectable.
- Native AskQuestion is excluded. Paperclip semantic questions are
supported. Authoritative per-run dollar accounting is unavailable;
partial counters remain diagnostics and unknown cost is not zero.
- Image input, detailed native diffs, deeper child transcripts and
native plan-file export remain follow-ups.
- macOS x64 has clean-install and daemon-startup proof under Rosetta,
not a separate live campaign on Intel hardware.
- Release only the tested package/image combination. Merging this PR
does not publish npm packages or deploy that image. Later builds need
their own release verification. Rollback disables new Cursor admission
while preserving records and recovery inspection.
- A model can fail an exact instruction: one cancelled-plan attempt
returned the wrong summary marker despite correct cancellation. The
unchanged repeat passed; both results remain in the report.

> ROADMAP.md was checked. This completes existing native Runner/Cursor
work; it does not add an independent core feature proposal.

## Model Used

OpenAI Codex, GPT-6. The exact serving variant and context window are
not exposed in this session. The agent used reasoning, repository
inspection, code execution, protocol tests and browser-backed Product
E2E tools. Cursor acceptance uses the explicit
`gpt-5.6-luna[context=272k,reasoning=medium,fast=false]` model. That is
the evaluated provider model.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass — affected suites passed;
full CI and the retained local failed attempts are recorded separately
above.
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green — 56 successful checks, one
neutral and four skipped on `f7ec5cc1f0e30c62a829c016ff2013a2a9d79037`
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups —
fresh Base review passed on `f7ec5cc1f0e30c62a829c016ff2013a2a9d79037`;
zero new comments and no unresolved threads. The earlier Apex finding
remains fixed.
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-06 20:48:15 -05:00
Devin FoleyandPaperclip 1d5a439e2b Diagnose native model rejections and preserve repairable reviews (#15304)
## Thinking Path

> - Paperclip manages agent execution and recovery.
> - A native provider can return a failed turn with an accepted result.
> - This path replaced a structured model rejection with a generic
failure.
> - Finalization could also queue the same failed request for recovery.
> - This change retains a bounded model and authentication diagnosis.
> - Workers wait for a configuration fix; reviews retain their
assignment.

## Linked Issues or Issue Description

**Describe the bug**

A native Codex turn can report HTTP 400 because the selected model is
not supported by its ChatGPT connection. When the turn also returns a
failed semantic result, the run shows `Native session failed` and
`adapter_failed`. The native failed-result finalizer can create a retry
intent before the adapter error is saved.

**To Reproduce**

Use a native Codex session that returns a structured
`invalid_request_error` model/ChatGPT rejection and an accepted failed
result. The regression tests reproduce this with a stub provider and a
real isolated PostgreSQL database. No provider account or live request
is needed.

**Expected behavior**

Show an actionable error and stop automatic retries of the rejected
request. Block the current worker for a configuration fix. Preserve a
pending review so its reviewer can explicitly retry after repair and
resolve the original assignment. A stale run must not block a new task
owner or override a completed review.

**Version and deployment mode**

The affected source path is present at `858094ba81`. This change applies
to native Codex execution in local and remote environments.

Related work: #13853 covers CLI protocol compatibility, #14942 refreshes
runtime/model pins, and #12767 proposes legacy adapter model validation.
This change concerns accepted native failed results and does not replace
those changes.

## What Changed

- Recognize only a bounded structured HTTP 400 provider error for the
exact selected model and failed turn.
- Save a fixed actionable error and four fixed diagnostic fields.
Exclude raw provider text and model names from the new diagnostic.
- Re-derive finalization evidence from the pinned execution and
committed runner event, including its canonical hash and identity
bindings.
- Block only the current worker. For a current pending reviewer,
preserve the original review decision and status version, release
execution, and create no automatic recovery action. Recheck execution
ownership under the issue lock. Preserve superseded and completed
reviews.
- Keep unknown failures, accepted results, cleanup, and ownership fences
intact. Advance the status policy version to `phase6-v8`.
- Route this exact failure code through the existing configuration
recovery path. Permit an authenticated Board retry of this native
failure only for its saved, still-pending review; discard caller review
markers and revalidate at dispatch.

## Verification

- Node 24: `pnpm -r typecheck` and `pnpm build` passed.
- Independent focused validation: 222 tests passed across the new
observer and PostgreSQL integration suites, status arbiter, diagnostic
redaction, recovery classification, wake policy, and wake use cases.
- The integration tests use actual result persistence, finalization, and
the wake dispatcher. They cover zero retry dispatch for an authorized
rejection, repeated reconciliation, pending/superseded/resolved reviews,
reassignment, and a same-agent successor at an unchanged status version.
- Final-commit verification: full typecheck and build passed;
independent route/integration/arbiter/review validation passed 136 tests
with no remaining source findings. This includes authenticated retry
refusals, actual heartbeat retry admission, atomic queued-review claim,
and acceptance of the original review.
- The local aggregate test run was stopped to avoid duplicating the
canonical Linux CI aggregate on this development host. Its partial log
showed no failures; no full local-suite pass is claimed. The full hosted
Linux CI aggregate passed on the final commit.
- No live provider calls, runtime changes, or schema changes were made.

## Risks

The classifier is intentionally narrow. A changed provider response
format remains an unknown failure. No global model restriction is
introduced. The worker blocker requires current task authority and does
not replace newer assignments or completed review decisions. A pending
review remains in review and requires an explicit retry after
configuration repair; it is not rebound to a new status decision.

The retry decision is durable. There remains a small existing gap
between status/effect commit and diagnostic metadata persistence; a
crash there can leave a generic diagnostic while preserving the blocked
decision. No schema migration is required.

## Model Used

OpenAI Codex, GPT-6, with reasoning, tool use, and code execution. The
exact serving model suffix and context-window size are not exposed in
this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#123` / `Refs #123` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub references)
- [x] My branch name describes the change and contains no internal
ticket id or instance-derived details
- [x] I have run focused tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-05 20:57:33 -07:00
Devin FoleyandPaperclip 2e67ea8dc7 fix: renew explicit continuation authorization for bounded retries (#14987)
Preserve exact user-authorized continuation receipts across bounded retries and retain the task claim through owned retry admission. Stop, reassignment, superseding input, and active cleanup continue to block unsafe continuation.

Validation: exact-head Greptile 5/5, passing CI, no unresolved review threads, and a clean merge. Detailed verification and limitations are recorded in the pull request.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-05 10:57:57 -07:00
DottaandPaperclip 9ae3d8db3d fix: keep pre-dispatch review waits out of execution recovery (#15024)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The queued-run gate can cancel a continuation that must wait for
review.
> - This cancellation happens before execution authority or a provider
starts.
> - Recovery currently treats the gate receipt as unknown provider
execution.
> - That mistake blocks a conversation after a successful reply and
hides Retry.
> - This pull request recognizes only the recorded, unclaimed
review-wait state.
> - Review waits keep their normal disposition path, and new user input
can recover older false holds.

## Linked Issues or Issue Description

Refs #15015, #15020, and #15022. Related #11614 narrows the review
posture that causes cancellation; this change corrects recovery after a
valid cancellation.

**What happened?**

After a successful agent reply, the queued-run gate cancelled an
automatic continuation with `issue_continuation_waiting_on_review`. The
gate retained `timeoutSource: stale_queued_run_gate` and a matching stop
reason. No execution authority or provider started. Recovery still
created an unknown-action hold, moved the task to Blocked, and hid
Retry.

**Expected behavior**

Use normal review-wait disposition repair for this recorded state. Allow
Retry, a new user message, or saved undelivered input to recover an
older false hold after ordinary admission checks pass. Preserve the
cancelled run and do not replay its input.

**Steps to reproduce**

1. Finish an agent turn on an open task that has a real review target.
2. Let the automatic continuation reach the queued-run review gate.
3. Refresh after the cancelled run is checked by recovery.
4. Confirm a review wait is handled as a wait rather than unknown
provider work.
5. Reproduce an older false hold for the same receipt, then request
Retry or send a new message.
6. Confirm only one fresh turn starts, and contradictory execution or
cleanup evidence retains the hold.

**Paperclip version or commit**

Reproduced on `215586d127`.

**Deployment mode**

Authenticated private self-hosted server, built from source.

## What Changed

- Recognize the exact review-wait dispatch receipt only while all
execution claims remain unset.
- Exempt recovery only after checking retained launch and provider
events, coordinators, and environment cleanup. Keep the synchronous
classifier conservative without that database proof.
- Apply the verified classification to automatic recovery, heartbeat
retries, and stranded-queue release, so saved user input starts once.
- Reuse guarded startup admission for Retry, new input, and saved input
on older false holds.
- Show a precise review-wait notice and keep continuation guidance
consistent with Retry availability.
- Verify provider events, launch events, coordinators, and cleanup
before admission.
- Add five initial red regressions, three additional red recovery
evidence regressions, concurrent saved-input coverage, and negative
evidence checks.
- Document the review-wait contract.

## Verification

- Red: five regressions fail on unchanged master. Existing review-wait
and contradictory-evidence cases still pass.
- Workspace `pnpm -r typecheck` passes.
- All 752 affected recovery, continuation, queue, classification, and
retry-scheduling tests pass.
- Three additional review regressions failed before the database proof
was added; all 12 focused review-wait cases then pass.
- Workspace `pnpm build` passes.
- Saved-input promotion and recovery notice regressions fail before
their fixes and pass afterward.
- Current-head CI has 54 passing checks and 2 skipped optional Storybook
checks. Greptile reviewed `0bf1437110e1617ae4544afa41f4319eac763dba` at
5/5 with zero open findings. The complete suite runs in sharded CI. No
complete local monolithic pass is claimed; an earlier long run was
stopped, and its unrelated failing case passed in isolation.

## Risks

The error code alone cannot establish that no provider started. This
exception also requires the server gate receipt, matching stop reason,
and null execution authority fields. User admission separately checks
coordinator, launch and provider events, environment cleanup, pending
decisions, ownership, task holds, budget, and active execution. No
migration or dependency change.

## Model Used

OpenAI Codex, an agent based on GPT-6. The exact runtime model ID and
context window are not exposed in this session. Used reasoning,
repository tools, code execution, and browser inspection.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-03 01:00:42 -05:00
Devin FoleyandPaperclip 22cea6b2e6 fix: bound sandbox bridge waits and flag silent runs sooner (#14979)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Sandbox agents exchange input and output through bridge control
commands.
> - A provider can stop responding to a command even when it receives a
timeout.
> - These small commands can inherit a four-hour agent lifetime and
block input or teardown.
> - The board also calls a silent run healthy for the first hour.
> - This pull request bounds bridge control waits and surfaces silence
sooner.

## Linked Issues or Issue Description

**What happened?**

A sandbox run can remain active when a bridge control command never
returns. The shared helper passes a timeout to the provider but does not
enforce it on the host. It also accepts the agent's hours-long timeout.
Output silence remains `ok` for an hour and becomes `critical` only
after four hours.

**Expected behavior**

Bound short bridge operations even if the provider never settles. Report
failed input delivery through the existing shutdown path. Warn after
five silent minutes and escalate after fifteen. Keep normal agent
command limits and require verified termination before releasing
execution ownership.

**Steps to reproduce**

1. Use a sandbox runner whose bridge read or input-upload promise never
settles.
2. Set its configured timeout to four hours.
3. Observe that the old queue client never returns or rejects.
4. Inspect a running task with 35 minutes of output silence. The old
summary still reports `ok`.

**Paperclip version or commit**

Base commit `d6d88b9de2`.

**Deployment mode**

Self-hosted server with sandbox execution.

**Agent adapter(s) involved**

Shared command-managed sandbox bridge, including Codex ACP sessions. The
informational silence thresholds apply to active runs across adapters.

Related: #14889 recovers stalled Daytona output streams; #14485 retries
explicit gateway failures during input delivery. This change bounds
short control operations whose provider promises never settle. It does
not add tool replay or automatic cancellation for output silence. #6297
proposes configurable per-agent silence thresholds; this patch only
changes the existing defaults.

## What Changed

- Enforce at most 30 seconds per bridge control shell command on the
host and provider, including callback startup and shutdown,
process-session launch, and payload setup. Preserve shorter configured
deadlines and launch environments.
- Keep the long-lived agent command outside this deadline. Use a fixed
timeout diagnostic without command payloads.
- Surface suspicious output silence after five minutes and critical
silence after fifteen minutes.
- Decouple the shared-workspace holder cutoff from warning thresholds
and preserve its existing one-hour value.
- Add regressions for hung reads, a late upload response, failed input
delivery, exact warning boundaries, and fresh output clearing warnings.
- Update the adapter guide and execution contract.

## Verification

- The three new queue-client regressions fail on the unchanged base and
pass with this patch.
- Final callback bridge and sandbox session suites: 214 passed. These
cover hung reads, writes, startup, shutdown, process-session launch,
payload setup, and the separate long-running agent limit.
- Stdin ordering and shutdown suite: 56 passed after the lifecycle
change.
- Daytona and watchdog coverage passed in the earlier focused runs.
Across the focused suites, 602 distinct tests pass.
- `pnpm -r typecheck` and `pnpm build`: passed. Server and adapter
typecheck/build also passed after their respective follow-up changes.
- `pnpm test:run`: attempted and stopped after known local failures.
Four chat/email cases used an external ancestor skill path, three
skill-cache cases failed on macOS, and one wakeup case timed out. The
wakeup case passes alone (1 passed, 27 skipped). This run spanned the
workspace-cutoff follow-up and also failed its new holder case; a fresh
final-head workspace suite passes all 19 tests. The interrupted run is
not a full local-suite pass or final-head verification.
- A filesystem queue-drain test failed once during the lifecycle rerun
and passed on the complete two-suite rerun. It uses the filesystem
client, outside the changed command-runner path.
- Complete CI on `ff2212c235`: 53 successful checks and two expected
skips, including the full test suite and canary packaging dry run. No
failed or pending checks.
- Greptile reviewed `ff2212c235` at 5/5. All review findings are
addressed, no threads remain unresolved, and the branch has no merge
conflicts with `master`.
- `git diff --check` and a scan of added text for secrets and private
identifiers passed.

## Risks

- A bridge control operation that needs more than 30 seconds now fails,
even if the caller selected a longer run lifetime. Agent commands retain
their own limits.
- Timing out a provider promise does not cancel the remote operation or
prove it stopped. Existing execution settlement still owns termination
verification. No uncertain tool action is replayed.
- Quiet healthy runs display warnings sooner. Existing snooze, continue,
and false-positive dismissal controls still apply. Silence alone does
not cancel a run, create review work, or change assignments.
- No schema or API shape change.

## Model Used

OpenAI GPT-6 through Codex, with tool use and code execution. The exact
serving model ID and context window are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run change-specific tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-02 15:29:41 -07:00
DottaandPaperclip 7a52dcdc74 fix: repair MCP validation and cancelled execution recovery (#14951)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The tool gateway gives agents access to connected services. Recovery
controls what happens when a run stops.
> - Generated tool names can exceed the provider limit after the MCP
client adds its prefix.
> - The same invalid definition can fail each automatic retry. A
cancelled run can also hold saved messages without showing its cause.
> - This pull request bounds tool names, stops configuration retries,
and retains cancellation evidence.
> - It shows the stopped run and admits saved input only after the
existing safety checks pass.
> - The benefit is a clear recovery path that preserves operator Stop
and prevents duplicate message delivery.

## Linked Issues or Issue Description

**What happened?**

A long connected MCP tool name makes the provider reject the entire
request. Automatic recovery repeats the invalid request. Separately,
unexpected legacy cancellations can leave saved input behind a recovery
hold. The notice does not identify the stopped run or its cause.

**Expected behavior**

Complete MCP names fit the provider limit. Tool-definition errors
require configuration repair. Cancelled runs retain their source and
reason. The recovery notice shows the cause and saved-message count.
Verified unexpected cancellations can start a fresh turn through the
existing admission checks.

**Steps to reproduce**

1. Assign an App gallery connection with a long application key and tool
name to a Claude agent.
2. Start a run. The provider rejects a name over 128 characters,
including its MCP prefix.
3. For cancellation recovery, stop a legacy provider turn without an
operator Stop request and send a user message while the recovery hold is
active.
4. Inspect the recovery notice and the deferred message queue.

**Paperclip version or commit**

Rebased onto master at `cf8ad63c806685bfd7c48e3ed4a919d61a7c55f1`.

**Deployment mode**

Hosted or self-hosted server with legacy Claude or Codex execution.

Related public work:

- Refs #14017. That PR caps name segments. This PR preserves existing
short names and uses stable hash aliases for long complete names. It
also covers classification and recovery.
- Refs #4510. That PR adds a cancellation-source column. This PR records
bounded evidence in the existing run result, without a migration.
- Refs #12552 and #4506. Those PRs suppress recovery after operator
cancellation. This PR preserves operator intent and uses the existing
continuation gates.

## What Changed

- Bound gateway names with the full provider prefix in the 128-character
budget. Retain the original upstream tool name for dispatch and
permissions.
- Classify invalid tool definitions as configuration failures before
diagnostic redaction. Stop automatic retries and continuation attempts
for that error code.
- Persist cancellation source, expectedness, initiator, reason, and
time. Preserve recorded Stop intent when adapter results arrive. Report
unexpected started cancellations with closed diagnostic labels.
- Show the run cause, saved-message count, and Inspect run link. Offer
Continue for eligible unexpected cancellations. Require verified
provider stop, empty tool inventory, ownership, and the existing pause,
budget, approval, and dependency gates. Use the existing queue for
single delivery.
- Add regression coverage and update the execution, MCP gateway, and
run-log documentation.

## Verification

- `pnpm -r typecheck` and `pnpm build` passed.
- `pnpm check:token-gates` passed.
- Ran `pnpm test:run` and completed its workspace and serialized groups.
Initial resource and timing failures passed on isolated reruns. All 149
serialized route suites passed.
- Reran the changed server, adapter, and UI suites after the rebase.
Coverage includes long-name upstream dispatch, configuration retry
suppression, cancellation evidence retention, privacy labels, oversized
run projection, and concurrent saved-message delivery.
- `pnpm test:e2e tests/e2e/legacy-failure-continuation.spec.ts` passed
all six browser scenarios. The recovery notice shows the run cause and
inspection link, and each recovery entry point reaches one new response.
- Added database-backed checks for active, removed, paused, unavailable,
and disabled chat connections. The final continuation and
recovery-notice suites passed 167 tests. Externally bound chats hide
board Continue and show a usable next action.
- All 55 GitHub checks passed on
`42afbf1371dcaeb72646e3d8f65c19ff7cddf8de`. Two unrelated Storybook jobs
were skipped by their normal conditions. Greptile reviewed that commit
at 5/5 with no findings and no open review threads.

## Risks

- Long tool names change to aliases. Existing short names stay
compatible. The original connection and upstream name remain the
dispatch authority.
- Invalid tool definitions no longer get automatic retries. An operator
must repair the configuration before a new attempt.
- Continuation changes apply only to positively identified unexpected
legacy cancellations with complete empty tool inventory. Operator Stop,
unknown historical cancellations, outstanding tools, and unverified
provider termination keep their holds.
- No database migration. The added projection fields are optional.
Cancellation reason and initiator IDs remain local run evidence; Sentry
receives only closed source and initiator-type labels and expectedness.

## Model Used

- OpenAI GPT-6 through Codex, with reasoning, repository editing, shell
execution, and GitHub tool use. The runtime does not expose the exact
model variant or context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-02 13:47:59 -05:00
DottaandPaperclip 2f6fa3b6dc fix: recover provider authentication inside tasks (#14629)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents need a working model provider connection to run a task.
> - A provider can reject a stored credential after the task starts.
> - The failed run must ask the responsible user to repair that
connection.
> - This pull request adds that request directly to the task and reuses
Connections sign-in.
> - The user can choose an API key or subscription, then continue the
task with a fresh session.

## Linked Issues or Issue Description

**What happened?**

A run that ended with `acpx_auth_required` or another known provider
authentication error did not immediately offer an inline way to connect
the provider. A repair form could also lock the user to the failed
account's sign-in method.

**Expected behavior**

Show a provider connection card in the task as soon as the
authentication failure is saved. Allow the responsible user to connect
or repair the provider with any supported sign-in method. Keep the
connection name automatic and resume the task after successful setup.

**Steps to reproduce**

1. Run a task with a supported provider and an expired or invalid
credential.
2. Let the run fail with a provider authentication error.
3. Open the task and attempt to repair the connection.

Related work: Refs #13724 and #13726. This change adds the inline task
repair flow and method choice.

## What Changed

- Classify provider authentication failures and create one connection
request for the current task. A persisted blocked classification
suppresses automatic retries only after the repair card is created;
unsupported providers retain their existing recovery path.
- Mark only the attributed, unchanged credential as needing sign-in.
Preserve credentials that were refreshed after the failed run started.
- Reuse the provider sign-in controls inside the task. Allow API key and
subscription choices for Claude, Codex, and Grok. Keep names hidden and
generate a default from the user, provider, and method.
- Keep the existing account when reconnecting with the same method.
Create and select another account when the method changes. Validate
updates to explicit agent bindings through the normal agent save path.
- Require explicit adoption for legacy agent authentication. Validate in
the agent environment, then commit the binding, connection install,
audit, and card completion in one transaction. Keep failed setup and
account selection visible and retryable.
- Add regression tests and update the specification and Connections
documentation.

## Verification

- Fresh local verification: 199 tests passed across the inline form,
provider method selector, default naming, authentication and recovery
classifiers, run liveness, OpenAPI routes, database adoption/rollback,
and Cursor execution suites. The adoption database suite also passed
against disposable Docker PostgreSQL.
- Full repository `pnpm build` and `pnpm -r typecheck` passed on the
latest commit. Token gates are clean.
- Embedded browser: opened real task cards from seeded authentication
failures; switched Claude from API key to subscription and back;
switched Codex from subscription to API key; confirmed the name field
stays hidden. Provider sign-in was not completed with real credentials.
- The broad local `pnpm test:run` started before review fixes and was
interrupted after the working tree changed; it is not counted as a
passing full run. Fresh focused tests passed. CI supplies the full test
and browser suite results for the current commit.
- CI is green on commit `4b97a4e447045ff3d7516525a187a5d1d21e0d4c`: 54
checks passed and two Storybook checks were skipped by their path rules.
The workspace preview job passed on one rerun after a local-server
startup timeout; its rerun passed 835 tests.
- Greptile is 5/5 on the same commit with no actionable findings and no
unresolved review threads.

## Risks

- Incorrect authentication classification could prompt for a connection
unnecessarily. Tests exclude tool authorization, quota, and unrelated
runtime failures.
- A method change selects the new personal provider default, which also
applies to other agents that use that user's default. Explicit account
bindings use the existing permission and runtime validation path.
- Credential invalidation must not race with refresh or reconnect. The
code compares the saved credential generation and grant update time
under locks.
- No database migration or new credential storage format is required.

## Model Used

OpenAI GPT-6 through Codex. The exact model ID and context window size
were not exposed in this session. Capabilities used: reasoning,
repository editing, shell commands, database tests, and embedded-browser
interaction.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (focused tests listed
above)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-30 06:39:37 -05:00
DottaandPaperclip 14795136f5 fix(runtime): finalize and recover sandbox workspace exports safely (#14402)
Serialize native workspace finalization, validate streamed archives within bounded limits, and quietly recover unsafe exports from saved results. Preserve exact allocations for exhausted transient failures and provide export-only retry without rerunning the provider.

Consolidates #14314, #14315, #14329, and #14334 while preserving the already-merged finalization label changes.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-09-28 10:57:32 -05:00
Devin FoleyandPaperclip be43c23e2d fix(recovery): preserve pending native result finalization (#14219)
Preserve native coordinator ownership while accepted results await workspace
copy-back, assessment, or arbitration. Check that ownership in the terminal
update so a coordinator recorded after the liveness read is also protected.
Keep terminal task authority and exhausted-retry cleanup unchanged.

Validation: 28 focused recovery tests, local typecheck/build, 52 passing CI
checks, and Greptile 5/5 with no open findings.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-09-27 00:22:49 -07:00
DottaandPaperclip 96bf004a79 fix: use persisted state for lifecycle continuation and retry budgets (#13888)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Its control plane decides when a task can continue, wait, stop, or
complete.
> - Legacy continuation could change when an agent changed its wording
without changing task state.
> - Shared attempt counts also let repair and infrastructure retries
affect each other's limits.
> - This pull request uses persisted state and separate, bounded
allowances for these decisions.
> - If automatic repair stops, the task explains what happened and
offers a guarded retry.
> - Paired tests and real-provider evaluations verify that Stop,
approvals, ownership, and spending limits remain authoritative.

## Linked Issues or Issue Description

Related work: Refs #13761, Refs #11126, Refs #13610. These cover
obsolete continuation dispatch and retry storms. Open and closed issues
and PRs were searched for related lifecycle, continuation, and retry
work.

**What happened?**
Legacy continuation depended on English wording and progress heuristics.
Repair, failure retry, and productive continuation could consume shared
counts. When bounded repair stopped, the task showed a technical
recovery message without a clear next action.

**Expected behavior**
Persisted disposition and owned execution paths determine the next
action. Missing disposition prompts bounded agent repair. Explicit work
mode determines planning mode. Narrative changes and raw activity counts
cannot replenish allowances. An exhausted repair shows a readable
notice. An explicit retry checks current controls and preserves the
assigned agent.

**Steps to reproduce**
Run `pnpm test:lifecycle-baseline`. The paired probes keep structured
state constant while varying completion, planning, blocker, and progress
prose. Run the explicit `lifecycle-baseline` and
`continuation-accounting` Product E2E suites for real-provider coverage.
In Storybook, open **Design previews / Recovery notice** to inspect the
production component's normal, pending, acknowledged, unavailable,
failure, and mobile states.

## What Changed

- Hide the image attachment button, icon, and drop/paste hint in answer
composers. Image paste and drop support remains available.
- Merge current master and retain both browser regression sets. Use a
production-stamped service worker in the offline recovery browser
fixture.
- Share one state-based legacy continuation decision across immediate,
delayed, and recovered dispatch. Bind bounded repairs to their source
run and episode.
- Remove title and description wording from work-mode authority. Agents
can still write requested plans in execution mode.
- Persist separate failure-retry and productive-continuation counters.
Disposition repair and resource waits cannot consume or reset those
allowances.
- Validate delayed repair identity, then recheck current gates before
provider dispatch. Fence native startup cancellation.
- Show **Agent needs attention**, a plain-language explanation, **Retry
agent**, and expandable details in both task interfaces. Report request
progress, acknowledgement, and errors inline.
- Store typed recovery notice metadata. Recognize older active notices
only through exact stored action and run IDs. Notice text never grants
retry authority.
- Use the existing recovery-action endpoint for retry. Recheck current
action, status, owner, agent availability, dependencies, active runs,
pending questions and confirmations, approvals, pause controls, and
budget. Duplicate requests do not wake twice.
- Add component, page, route, database, contract, and Storybook
coverage. Keep the scenario inventory and executable evals here.
Historical reports and snapshots live in the [commit-pinned
paperclip-evals
archive](https://github.com/paperclipai/paperclip-evals/blob/ce3e5afcd4a1184650f586a2b5b8be5874c66c8b/experiments/2026-09-lifecycle-authority/README.md).
- Preserve unsaved project fields while the same project URL changes to
its canonical alias. Do not reuse data across projects or companies.
This separate fix addresses the repeated repository-editor browser
failure without changing the browser test.
- Keep the development service worker from intercepting Vite module
reloads. Update the connection-intent browser fixture to record progress
and completion through the agent API.

## Verification

Merge preparation on September 25, commit
`c1e8e4b7ddd9fbc4913ed55ce21b8e12906c2f97`:

- Merged master `bd2030932` and resolved the browser test-list conflict
by keeping both sets of regressions.
- Deterministic lifecycle baseline: 1,090/1,090 assertions passed; no
failures, skips, or missing selected evidence. Unit 423, runner 184,
database integration 397, grading 86.
- Browser support: 17/17 passed. The offline recovery test first failed
with an unstamped development worker, then passed with the production
stamp. Its assertions are unchanged.
- Focused interaction UI and offline fallback tests: 19/19 passed.
Verified the custom-answer composer in Storybook: no attachment controls
or hint; entering an answer enables Next.
- Recursive typecheck, production build, token gates, and diff checks
passed. The worktree is clean. No new real-provider campaign was run.
- Current CI and review: [Current PR CI
passed](https://github.com/paperclipai/paperclip/actions/runs/36166011243):
55 successful checks and two optional Storybook skips. Greptile scored
this exact commit 5/5. Hiding the question attachment controls is an
intentional UI change; paste/drop remains available.

Earlier recovery UI verification, commit
`21be0fec0e90e86b6d662b8ee4831847cd041cdb`:

- Recursive typecheck, production build, token gates, and diff checks
passed.
- Focused UI coverage: 338 tests passed across six suites (336 before
the interaction guard, with the two affected suites rerun at 149 passed
after it). Covers both task interfaces, the real page mutation,
pending/error acknowledgement, stale state, and unavailable controls.
- Recovery database integration: 352 tests passed before the interaction
guard. The complete recovery-action and mutation-route suites passed 181
tests after it. The two new pending question/confirmation regressions
failed before the fix and passed afterward, including
resolved-interaction controls. Shared validator suite: 31 passed. E2E
catalog suites: 34 passed.
- Browser inspection passed for light/dark themes, mobile layout,
expandable details, pending retry, acknowledgement, failure, and
disabled retry. Storybook renders the production component; its request
is simulated.
- The broad local run hit two chat callback-order wait failures and was
stopped after all CI unit/database/runner shards passed. Both local
failures passed when rerun without the competing full-suite process.
- CI exposed a repeated project-repository draft-loss race during
canonical redirects. A new unit regression failed before the fix; all
nine project-page tests now pass, including controls for other projects
and companies. Both unchanged repository browser tests passed against a
fresh local server. UI typecheck, production UI build, and token gates
passed after this fix.
- [Earlier PR CI
passed](https://github.com/paperclipai/paperclip/actions/runs/36072486798)
on `21be0fec0e90e86b6d662b8ee4831847cd041cdb`: 55 successful checks, two
optional Storybook skips, and no failed or pending checks. The
repository browser shard passed with the production fix. Greptile is 5/5
on this exact commit with no unresolved review threads. The PR is
mergeable.

Historical, source-qualified lifecycle evidence:

- Lifecycle baseline: 1,074 assertions. Native session coverage: 447
tests. Product E2E support: 515 tests. Browser support: 11 tests. Full
earlier verification is retained in the archive.
- [Real-provider campaign: 8/8 passed, zero
retries](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-35881382080-1/index.html),
source `e88d210417280140b44a36449027290adcb1aeaa`. Evidence and cleanup
checks passed. This includes deliberately exhausted repair cases that
correctly remain blocked; it does not mean every task finished Done.
This campaign predates the recovery UI change.
- Archive migration verified all 16 original JSON files byte-for-byte
and all 24 checksum entries. App tests do not need private archive
access. [Archive PR
#27](https://github.com/paperclipai/paperclip-evals/pull/27) is merged.

## Risks

- Agents that omit durable disposition receive at most two repair
attempts by default. Prose-only completion exposes missing state rather
than silently changing scheduling.
- A retry is an explicit board action. The server rechecks current
controls. A successful response confirms the task returned to To do; it
does not claim that the provider has already started.
- Existing notice metadata remains valid. Only older active notices with
matching structured evidence receive the new UI. Historical notices
without that evidence keep their existing rendering. No schema migration
is required.
- Old run records require conservative retry accounting. Tests cover old
counters, alternating retry lanes, restarts, and exhausted repairs.
- Historical snapshots require private `paperclip-evals` access. The app
index retains public campaign links. Live campaigns qualify specific
sources and scenarios; no new real-provider campaign has run for the
recovery UI commit.

> This fixes existing lifecycle and recovery behavior and does not
duplicate planned core work.

## Model Used

OpenAI GPT-6 through Codex assisted implementation, reasoning, code
execution, and review. The exact serving model ID and context window are
not exposed in this task. Historical real-provider evaluations used
Codex model `gpt-5.6-sol`, separately from the implementation assistant.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-25 15:28:11 -07:00
Devin FoleyandPaperclip 6bc830b62b fix(recovery): escalate an issue whose interrupted run has spent its retry budget (#14046)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The stranded-issue sweeper is what puts an assigned issue back on a
live path when its run dies.
> - A failed or interrupted run gets a bounded transient retry; when
that budget is spent, the retry scheduler queues nothing and reports the
exhaustion.
> - The sweeper treated that "nothing queued" like every other "nothing
queued" and skipped the issue, on every tick, forever: `in_progress`, no
run, no path, no notice.
> - Three server restarts in a row (a deploy storm) are enough to spend
the budget, so this is reachable in ordinary operation.
> - This pull request makes the sweeper escalate a spent budget to a
board-owned recovery action, the same visible `blocked` state its other
dead ends use.
> - The benefit is that no issue can sit assigned and silent after its
retries run out.

## Linked Issues or Issue Description

No existing issue found. Related work: #14028 (this branch's
predecessor: queue drain-time wakes, retry interrupted corrective runs)
fixed two neighbouring gaps but not this one.

**What happened?**

A routine-created issue's run was interrupted by a graceful server
shutdown, and both bounded transient retries were interrupted by the
next two shutdowns. The retry scheduler logged "Bounded retry exhausted
after 2 scheduled attempts; no further automatic retry will be queued"
and stopped. On every tick after that, `reconcileStrandedAssignedIssues`
reached the generic continuation lane, `enqueueStrandedIssueRecovery`
delegated to `scheduleRecoveryRetry`, which returned null (exhausted),
and the sweeper counted the issue as `skipped`. The issue stayed
`in_progress` with no run, no recovery action, and no comment for hours
until a person woke the agent by hand. The same hole exists in the
assigned-`todo` dispatch lane.

**Expected behavior**

When the transient retry budget for an interrupted or failed run is
spent, the sweeper should treat it as the dead end it is: escalate the
issue to `blocked` with a board-owned recovery action and a notice,
exactly as it does when a continuation retry chain or an assignment
retry chain is exhausted. Leaving the budget spent across restarts is
correct and unchanged; leaving the issue silent is not.

**Steps to reproduce**

1. Assign an agent using a conversation adapter (its interrupted runs
carry `conversationContinuation`, so legacy reconciliation does not
terminalize them) an issue and let a run start.
2. Interrupt the run with a graceful shutdown, let the transient retry
start, interrupt it, and repeat once more so `scheduledRetryAttempt`
reaches 2.
3. Run the stranded-issue sweep. Before this change: `skipped` every
tick, issue `in_progress`, no run, no recovery action. After:
`escalated`, issue `blocked`, one active board-owned
`issue_recovery_actions` row, a "No live execution path" notice.

**Paperclip version or commit**

master at bd6caf51bb (2026-09-25).

**Deployment mode**

Managed cloud instance restarted by fleet deploys; the code path is the
same for any operator whose server restarts more often than the retry
budget allows.

## What Changed

- `recoveryService` gains an optional `transientRetryBudgetSpent(run)`
dependency; `heartbeatService` wires it as
`executionFailureRetryCount(run) >=
BOUNDED_TRANSIENT_HEARTBEAT_RETRY_MAX_ATTEMPTS`, the same check
`scheduleBoundedRetryForRun` applies.
- `enqueueStrandedIssueRecovery` takes an optional `outcome`
out-parameter and sets `retryExhausted` when the failed predecessor's
retry returned nothing **because** the budget is spent. A null return
without it still means another authority owns the run (native runtime,
legacy reconciliation) and the caller leaves it alone; the deliberate
"failure recovery cannot fall through into the continuation queue" rule
is unchanged.
- The generic `in_progress` continuation lane and the assigned-`todo`
dispatch lane escalate on `retryExhausted` via
`escalateStrandedAssignedIssue` with a "No live execution path" notice
(danger tone), which creates the board-owned source-scoped recovery
action and moves the issue to `blocked`. Every other
`enqueueStrandedIssueRecovery` caller is unchanged.
- Tests (`heartbeat-process-recovery.test.ts`): an `in_progress` issue
with a spent budget escalates (blocked, one active board-owned action,
notice, no successor run, idempotent on the next sweep); an interrupted
run with budget remaining still gets its transient retry; an assigned
`todo` issue with a spent dispatch budget escalates. The existing guard
"does not reset an exhausted incident budget on server restart" keeps
its no-successor-run assertion and now expects the board escalation
instead of nothing, with a comment on why.

## Verification

```
pnpm -r --filter './packages/**' build
cd server
npx vitest run src/__tests__/heartbeat-process-recovery.test.ts \
  src/__tests__/issue-recovery-actions.test.ts \
  src/services/recovery/successful-run-handoff.test.ts \
  src/__tests__/heartbeat-task-drain-admission-release.test.ts \
  src/__tests__/attention-service.test.ts \
  src/__tests__/heartbeat-comment-wake-batching.test.ts
npx tsc --noEmit -p tsconfig.json
```

Live reproduction: the exact stuck state (issue `in_progress`, latest
run `interrupted` with `scheduledRetryAttempt` 2, the "Bounded retry
exhausted" lifecycle event, sweeper `skipped` every tick) was observed
on a managed instance running current master before this change was
written.

## Risks

- Behavior change is limited to runs whose transient budget is already
spent, which previously produced no action at all. Nothing new is
retried; the change only adds the escalation, so no retry loop can be
introduced.
- The board-owned action spawns no run. Resolving it (restore to the
owner) re-dispatches through the existing recovery-action routes, the
same flow as every other stranded escalation.
- Native-runtime and legacy-reconciliation predecessors are untouched:
they return before the exhaustion check.

## Model Used

Claude (Anthropic) — `claude-fable-5-1`, extended thinking, tool use
(Claude Code CLI). The incident diagnosis and the choice to escalate
rather than re-dispatch were steered by the maintainer.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-25 13:17:48 -07:00
Devin FoleyandPaperclip 5b09d66183 fix(heartbeat): keep drain-time wakes queued and retry interrupted disposition handoffs (#14028)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The server's heartbeat scheduler queues, admits, and recovers agent
runs.
> - A control plane arms an instance task drain before it restarts the
process, so no new run starts mid-restart.
> - Two recovery paths treated that transient hold as a permanent
verdict: a wake that arrived during a drain was written as `skipped` and
never replayed, and a corrective disposition run that the restart
interrupted counted as an exhausted attempt, so the issue went to
`blocked`.
> - Both outcomes leave an assigned issue with no run and no path until
a person notices.
> - This pull request keeps drain-time wakes in the durable queue and
gives an interrupted corrective run the same bounded transient retry any
interrupted run gets.
> - The benefit is that a graceful restart never strands or blocks an
issue by itself.

## Linked Issues or Issue Description

No existing issue found. I searched open PRs that touch the task drain
(#13527 adds opt-in termination of active runs; #13965 handles deferred
wakes after a stale cancellation). Neither replays drain-time wakes or
changes the corrective-handoff escalation.

**What happened?**

1. An operator accepted a plan confirmation while the instance was in a
task drain (a control plane armed it before a deploy restart).
`enqueueWakeup` wrote the assignee's continuation wake with `status:
"skipped"` and `heartbeatSkip.reason: "task_drain"`. The drain is
process-local and cleared on the restart seconds later, but nothing
replays skipped wakes. The issue stayed in `todo` with no run for 20
hours until a person retried it by hand. The stranded-issue sweeper did
not help: a `todo` issue whose latest run succeeded is treated as
deliberately handed back.
2. On another issue, the watchdog correctly raised
`successful_run_missing_state` and queued the single corrective handoff
run. A graceful server shutdown (the same deploy restart) interrupted
that run with `server_shutdown_interrupted`. On the next boot,
`reconcileStrandedAssignedIssues` saw a corrective run at
`handoffAttempt >= maxHandoffAttempts` and escalated the issue to
`blocked` with a board-owned `missing_disposition` recovery action. The
agent never got to finish one corrective attempt.

**Expected behavior**

A task drain holds admission, not the request. A wake that arrives
during a drain must run once the drain lifts or the process restarts. An
interrupted corrective run is not evidence that the agent could not
choose a disposition; it should be retried like any interrupted run, and
escalate only when that retry budget is spent or a finished attempt
still leaves no disposition.

**Steps to reproduce**

1. Assign an agent an issue and `POST /api/instance/task-drain` with a
Cloud control assertion (or call `startTaskDrain({})` in a test).
2. Trigger any wake for that agent (assignment, comment, or an accepted
`request_confirmation`).
3. Observe the `agent_wakeup_requests` row: `status = skipped`, `reason
= heartbeat.scheduling_suppressed`, `payload.heartbeatSkip.reason =
task_drain`. Stop the drain or restart: no run is ever created for that
wake.
4. For the second case: let a `finish_successful_run_handoff` run be
interrupted by SIGTERM during a graceful shutdown, then boot. The issue
moves to `blocked` on the startup sweep without any retry.

**Paperclip version or commit**

master at e2f1a66aa7 (2026-09-25).

**Deployment mode**

Managed cloud instance behind a control plane that arms task drains
before deploy restarts; the code paths are the same for any operator who
uses the drain endpoint.

## What Changed

- `enqueueWakeup` no longer writes a wake as `skipped` when the only
suppression is `task_drain`. The wake and its queued run land in the
durable queue; `startNextQueuedRunForAgent` and `executeRun` already
refuse to admit work while the drain is active, and the boot-time
`resumeQueuedRuns` pass picks it up after the restart.
`worktree_instance` and `database_restore_in_progress` suppression still
write `skipped`: those holds are not transient restarts.
- `reconcileStrandedAssignedIssues`: when the latest run is a corrective
successful-run handoff at its attempt cap **and** that run is
`interrupted`, the sweeper schedules the bounded transient retry (via
`enqueueStrandedIssueRecovery` → `scheduleRecoveryRetry`) instead of
escalating. The retry keeps the handoff context, so it is still the
corrective run. If the retry budget is spent, or a finished attempt
still leaves no disposition, escalation proceeds exactly as before.
Failed corrective runs (adapter or provider failures) are unchanged:
they still escalate immediately.
- New sweep counter `successfulRunHandoffRetried`, included in the
startup and periodic recovery log lines.
- Tests: `heartbeat-task-drain-admission-release.test.ts` gains a case
that a wake during a drain is queued, held while the drain is active
(quiescence still reports true), and runs to completion once the drain
lifts. `heartbeat-process-recovery.test.ts` gains two cases: an
interrupted corrective run is retried with its handoff context and the
issue stays `in_progress`; an interrupted corrective run with a spent
transient budget escalates to `blocked` with the usual recovery action
evidence.

## Verification

```
pnpm -r --filter './packages/**' build
cd server
npx vitest run src/__tests__/heartbeat-process-recovery.test.ts            # 296 passed
npx vitest run src/__tests__/heartbeat-task-drain-admission-release.test.ts \
  src/__tests__/heartbeat-worktree-suppression.test.ts \
  src/__tests__/heartbeat-scheduling-suppression.test.ts \
  src/__tests__/heartbeat-task-drain.test.ts \
  src/__tests__/instance-settings-routes.test.ts \
  src/services/recovery/successful-run-handoff.test.ts \
  src/__tests__/issue-recovery-actions.test.ts \
  src/__tests__/heartbeat-comment-wake-batching.test.ts \
  src/__tests__/attention-service.test.ts                                    # 226 passed
npx tsc --noEmit -p tsconfig.json                                           # clean
```

Live reproduction of the first defect: after this change was written, a
board wake issued against a draining managed instance (running current
master) was again recorded as `skipped` with `heartbeatSkip.reason =
task_drain`, which is the exact row the new test asserts no longer
appears.

## Risks

- A wake that lands during a drain now waits in the queue instead of
being dropped. `getTaskDrainStatus().pendingWakes` counts only in-flight
enqueue promises, so quiescence is unchanged and a control plane waiting
for the drain is not held longer. If a drain is stopped without a
restart, the queued run starts on the next scheduler tick.
- The interrupted-handoff retry reuses the existing transient retry
budget (two attempts) and the existing `hasActiveExecutionPath` skip, so
a sweep cannot double-schedule. Native-runtime corrective runs still
return `null` from the recovery enqueue and escalate as before.
- Self-hosted instances that never call the drain endpoint see no change
on the first path; the second path only changes behavior for corrective
runs interrupted by a graceful shutdown.

## Model Used

Claude (Anthropic) — `claude-fable-5-1`, extended thinking, tool use
(Claude Code CLI). Human-directed: diagnosis of the two production
incidents, the fix design, and review were steered by the maintainer.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-25 11:28:54 -07:00
Devin FoleyandPaperclip 110d176fc9 fix: preserve chat message bindings in review recovery (#13818)
## Thinking Path

> - Paperclip manages AI agents and their work.
> - External chat messages can start work on tasks that remain in review.
> - Paperclip queues one recovery run when a task loses its review path.
> - That recovery keeps the chat source but loses the admitted message IDs.
> - The authorization check then rejects the recovery before execution starts.
> - This change retains the message IDs so the existing check can verify current access.

## Linked Issues or Issue Description

Refs #13809.

**What happened?**

A successful external-chat run can leave an active task in review without a maintained review path. Its automatic recovery then fails with `reviewed_chat_execution_binding_not_authorized`. The recovery context retains `chat:slack` but drops `wakeCommentIds`.

**Expected behavior**

An eligible recovery should retain its admitted message references and pass a new authorization check. Missing or revoked access must still prevent execution.

**Steps to reproduce**

1. Finish a chat run whose task remains in review with no maintained review path.
2. Build the bounded review recovery from that run's context.
3. Dispatch the recovery through the reviewed-chat authorization check.

The new regression tests fail before this patch. Related PR #13809 handles answered conversations that become idle. This patch handles recovery when the conversation remains active.

## What Changed

- Retain the admitted message batch for external-chat review recovery. Derive the current comment reference from that batch.
- Share the existing supported-provider selector between recovery and run-bound chat authorization. AgentMail remains on its separate email inbox path.
- Keep the existing authorization check. Do not copy prior checkout, authorization, session, or prompt state.
- Test Slack and Discord recovery, missing batches, revoked access, and changed task or company bindings.
- Document the recovery authorization contract.

## Verification

- The new tests reproduced the missing-message failure before the fix.
- Six focused suites passed: 257 tests covering review recovery, reviewed-chat authorization, issue liveness, Slack lifecycle, comment-wake batching, and external-chat waits. PostgreSQL integration tests ran against a temporary local PostgreSQL database.
- Focused test files: `review-path-recovery.test.ts`, `heartbeat-reviewed-chat-binding.integration.test.ts`, `heartbeat-issue-liveness-escalation.test.ts`, `slack-conversation-lifecycle.test.ts`, `heartbeat-comment-wake-batching.test.ts`, and `external-chat-wait.integration.test.ts`.
- Server TypeScript check passed with scratch configuration that resolves this checkout's workspace packages. Existing dependency links point to another checkout; the default check reports stale shared-type errors.
- `node scripts/check-module-boundaries.mjs` and `git diff --check` passed.
- `git diff | gitleaks stdin --redact --no-banner` passed. The diff was also checked for private identifiers and user data.
- [Full PR CI](https://github.com/paperclipai/paperclip/actions/runs/35760757895) passed on the latest commit, including build, workspace typecheck, general and serialized tests, Rust checks, and all eight browser shards. The unchanged local-service readiness test and agent-chat page-load assertion passed when their failed shards were retried. The local-service suite also passed locally (6 tests). The first, superseded run lost a chat runner; its stuck browser job was cancelled to unblock the current run.
- Full local workspace typecheck, tests, and build were not run. The machine has about 2 GiB free, and those commands include Rust builds. The full PR CI checks passed before merge.

## Risks

Small context-construction change. Dispatch still checks current execution ownership and access for every admitted message. The recovery remains bounded to one attempt per consumed path. No schema changes. Existing failed runs are not retried by this patch.

## Model Used

OpenAI GPT-6 (Codex), with tool use and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context to this change
- [x] I have specified the model used (with version and capability details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before requesting merge

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-09-22 10:46:13 -07:00
DottaandPaperclip 8725d6ce09 fix: make answered Slack conversations idle (#13809)
Settle published, successful Slack turns as Idle; resume the same conversation on an admitted message. Preserve unfinished work, delivery errors, and explicit dispositions.

Verified through focused lifecycle/API/UI tests, full CI, and a real staging Slack conversation in the embedded browser.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-09-22 10:35:33 -05:00
Devin FoleyandPaperclip d54b750111 Preserve Claude ACP quota classification and reset time (#13651)
Typed Claude ACP quota failures lost their recovery classification and reset
time when the runtime reduced provider metadata to a generic category error.
Inspect terminal metadata in memory and retain only safe recovery labels and
a parsed reset timestamp. Preserve the existing handling of other limits.

Verified real child processes on both pinned ACPX runtimes, adapter and
server recovery regressions, all PR CI gates, and Greptile 5/5. Also isolate
a pre-existing chat regression from unrelated fixtures’ retry work.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-09-18 19:42:28 -07:00
DottaandPaperclip f4cdc7b231 fix: recover transient workspace bootstrap scans (#13481)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The control plane prepares task workspaces before it starts an
agent.
> - Workspace preparation reads Git state so it can preserve edits and
exclude private files.
> - A failed scan was treated as a non-Git folder and lost its actual
failure code.
> - The resulting generic setup failure could not recover, even when the
cause was temporary.
> - This pull request keeps the cause and uses the existing bounded
retry schedule before provider startup.
> - Tasks can recover without human intervention, while permanent
failures and exhausted retries stop with useful guidance.

## Linked Issues or Issue Description

**What happened?**

A Git scan error during managed repository preparation became
`Configured repository folder is not a Git checkout`, followed by
generic `setup_failed`. The agent never started. Generic recovery could
not distinguish a temporary timeout from a bad workspace configuration.

**Expected behavior**

Keep the closed scan error code. Retry temporary timeouts and queue
saturation under the existing shared budget. Preserve edits, exclusions,
ownership, and pause gates. Stop permanent failures and exhausted
retries with a specific explanation. Do not replay historical generic
setup failures.

**Steps to reproduce**

1. Configure a task project with a local Git source that must be copied
into its managed repositories.
2. Make the ignored-file scan exceed its timeout before the agent
starts.
3. Before this fix, the snapshot returns null and the run ends as
non-retryable `setup_failed`.
4. Use the disposable browser fixture in
`tests/e2e/workspace-bootstrap/README.md` to inject real timeouts and
test the full recovery path.

**Paperclip version or commit**

Reproduced against `4510bf7c9e2fcbeb043445850928b5dcb79908ca`.

**Deployment mode**

Built from source. The defect is in core workspace setup, not a specific
model provider.

Related work: Refs #13442 (managed repository preparation), Refs #11572
(bounded Git scheduler), Refs #12997 (separate adapter startup retry
work), Refs #13469 (separate terminal-workspace scan performance work).

## What Changed

- Return the non-Git fallback only for repository discovery. Propagate
failed scans of a confirmed repository.
- Replace full ignored status output with an ignored-only directory
listing. Preserve NUL-delimited paths and exclusions.
- Preserve typed, sanitized scan errors through workspace preparation
and persist pre-provider failure details.
- Retry only timeouts and queue saturation, using the existing durable
two-retry budget and issue gates. Prevent generic recovery from adding
another budget.
- Show workspace-specific failure copy and actionable exhausted-recovery
notices.
- Add red-green unit tests, real-database restart and retry-boundary
tests, and opt-in browser acceptance fixtures with real Git subprocess
timeouts.
- Document the recovery contract and browser verification procedure.

## Verification

- Red: injected scan failures returned null instead of rejecting; setup
lost the timeout code; task-thread and recovery notices had generic
copy.
- Green: 119 focused adapter/backend tests, 20 recovery-boundary tests,
and 136 task-thread tests.
- `pnpm -r typecheck` — passed.
- `pnpm build` — passed on the final production code.
- `pnpm check:token-gates` — passed.
- The initial local `pnpm test:run` overlapped source edits and was
interrupted after two late-added assertions saw pre-fix behavior; it is
not counted as a green full run. A fresh final-head run passed all 229
tests across the six affected adapter/backend/UI suites. The clean
latest-head CI full test matrix passed: all five general-server shards,
all five serialized-server shards, and all three general-workspace
shards.
- Latest-head CI also passed all three browser shards and their
aggregate gate, typecheck and release registry, build, runner
verification, canary dry run, policy, Docker context integrity, and
security gates. Greptile: 5/5, with the review thread resolved.
- Browser: created a task in a disposable instance. A real Git timeout
scheduled recovery, the next run completed through the run-scoped API
without manual Retry, and Done survived reload. The deterministic
process worker checked preserved source edits and excluded private
files; no model calls were made.
- `WORKSPACE_BOOTSTRAP_TEST_URL=<disposable-instance-url> pnpm exec
playwright test --config
tests/e2e/workspace-bootstrap/playwright.config.ts` — 2 passed (3.6
minutes). The persistent case made exactly three failed attempts, never
started the worker, showed the cause-specific notice, stayed stopped for
another scheduler tick, and retained Blocked after reload.
- Extra red-green coverage: 50 recovery tests passed after fixing an
exhausted-bootstrap classification that incorrectly implied unknown
provider actions. Missing or uncertain evidence still retains the safety
hold.
- Verified the documented Git executable override during repository
seeding.

## Risks

- A confirmed repository scan failure now fails closed instead of
falling back to directory sync. This prevents unfiltered copying but
makes previously hidden errors visible.
- Temporary host problems can create up to two additional setup
attempts, 30 seconds apart. Permanent scan errors do not auto-retry.
Generic recovery cannot reset this budget.
- The durable retry path still enforces ownership, pause, and work
eligibility. Integration tests cover restart, duplicate promotion,
pause, exhaustion, and non-retryable categories.
- No schema migration, new runtime setting, new retry budget, production
deployment, or historical task replay.

## Model Used

OpenAI Codex, GPT-5-based coding agent, with reasoning, repository
tools, shell execution, and browser testing. The exact deployment model
ID and context-window size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-15 12:37:21 -05:00
cceeb0aa66 test(runner): add everyday workflow evaluation harness (#13474)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The native runner must support project work, delegation, hiring, and
service access.
> - Browser tests exposed lost connection access, rejected helper
events, and stalled recovery.
> - Some eval failures also came from incorrect fixtures and decision
controls.
> - This pull request fixes those paths and adds eight everyday workflow
stories.
> - The tests retain observed failures and verify delivered files
independently.
> - The benefit is repeatable evidence for common user tasks and their
remaining gaps.

## Linked Issues or Issue Description

Related work: #13404 contains earlier workflow fixes. #13300 and #13470
changed the CI contracts used by the harness security tests. Merged
companion:
[paperclip-evals#22](https://github.com/paperclipai/paperclip-evals/pull/22).

**What happened?**

Native ACPX sessions did not receive the assigned connection gateway.
Codex helper events could arrive before their spawn receipt and fail
thread validation. A parent continuation could take a shared workspace
before its child retried. A failed native continuation could leave the
task status without a clear recovery blocker. The eval harness also
confused tool approvals with new connection requests and could reject a
valid delegated download.

**Expected behavior**

Keep assigned gateway access and its approval checks. Verify helper
lineage before accepting helper progress. Let a waiting child proceed
before automatic parent recovery. Preserve a failed task's recovery
ownership. Grade the actual requested workflow and its delivered files.

**Steps to reproduce**

Run the everyday workflow suite with the native Codex and Claude
profiles. Exercise service approval, connection refusal, delegated
project work, and teammate reuse. The commands and case requirements are
in `tests/runner-e2e/EVERYDAY-WORKFLOWS.md`. Use `pnpm
test:runner-recovery` for controlled crash and replacement cases.

## What Changed

- Pass the scoped connection gateway binding through the native ACPX
host and sidecar.
- Recognize Codex helper lineage from parent metadata and spawn
receipts. Verify early helper events with `thread/read`. Keep helper
events separate from root completion authority.
- Guide agents to use persistent hiring, child tasks, dependency
records, and a blocked handoff while waiting for a child.
- Defer automatic parent recovery while a child has an active execution
path in the same shared workspace. Allow parent recovery when the child
needs review.
- Record Blocked status and recovery evidence when a failed native
continuation needs reconciliation, including existing active or
escalated incidents. Preserve their owner and retry budget.
- Add eight browser-driven workflow cases. Use real decision controls,
explicit child feedback delivery, managed hiring credentials, and
independent ZIP checks inside a bounded Docker sandbox. Verify sandbox
availability before task creation. Record screenshot SHA-256 at capture.
- Keep runner crash probes in controlled recovery tests. Preserve the
original failure when cleanup also fails.
- Display missing accounting and replay revisions as unavailable. Align
harness security assertions with the approved CI changes.
- Make the channel-rejection browser fixture bind its file after the
send captures its payload. This prevents live refresh from removing the
file before the simulated race.

## Verification

- Full workspace `pnpm -r typecheck` passed after merging current
master.
- Runner E2E typecheck passed. Harness unit tests passed: 216/216.
- Wake-queue database tests passed: 55/55. The two added
existing-incident tests failed before the fix and pass after it.
- Docker artifact calibration passed: 12/12. Host-file and host-loopback
isolation tests failed before the fix and pass after it. Read-only
delivery and output limits are also verified.
- Full `pnpm build` passed. Targeted recovery tests passed: 83/83.
- The channel-rejection browser test passed five consecutive runs after
fixing the fixture race found in CI.
- Local general-server (12,351 tests), UI (6,250), CLI (485), and
workspace package groups passed. The monolithic run stopped at an
unchanged lock-heartbeat fixture race; the isolated workspace group
passed on rerun (shared: 747/747). A separate local serialized run
passed 97 files before two socket errors in the unchanged issue-list
route suite; that suite passed 15/15 on isolated rerun. These local full
commands did not finish uninterrupted; the complete CI matrix below
covers the remaining suites.
- Final head `0fb293733fe307be7e6667ae8f1364077d0c6455`: **34 successful
checks, 2 expected skips**, including every server/workspace shard,
browser shard, native runner verification, build, and typecheck. [Final
CI
run](https://github.com/paperclipai/paperclip/actions/runs/34989136700).
- Greptile reviewed this exact head at **5/5**; all review threads are
resolved. Both Superagent security checks are successful.
- ACPX credential-boundary tests passed: 118/118. Superagent accepted
the runner/sidecar versus provider-environment trace and cleared its
finding.
- The latest paid local campaign on source
`f6a2fdf7ac2af859826a2ae627ff4125a5478529` passed 22/24 cases: Sol 8/8,
Claude 7/8, Mini 7/8. These results predate the merge with current
master.
- The two remaining failures are in `hire-reuse`: Claude exceeded the
attempt deadline during final review; Mini made invalid deliverable tool
calls and remained Blocked.
- Six Daytona cases were not run because the matching immutable runner
image was unavailable. This PR does not claim new remote model results.

## Risks

The changes affect connection admission, helper identity, and recovery
scheduling. Assigned gateway grants and user approval still govern
service calls. The workspace admission gate still exists; the broader
folder-sync design is separate work. Provider behavior can still cause
the two recorded hiring failures. No database migration is required.
Paid cases are opt-in and have bounded attempt deadlines. Project
stories now require Docker and the documented pinned Python image on the
harness host.

## Model Used

OpenAI `gpt-6-astra` performed implementation, diagnosis, and
substantive review. OpenAI `gpt-5.6-luna` assisted with verification, PR
preparation, and review tracking. Both used repository tools and code
execution. Context-window sizes were not recorded.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (focused checks and
isolated reruns; full-run limitations are documented above)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: OpenAI GPT-5.6 Luna <noreply@openai.com>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-15 11:04:16 -05:00
DottaandPaperclip a2e7ffdc34 fix(runtime): validate sandbox paths and preserve live controller leases (#13432)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents can run in remote sandboxes.
> - Connection checks must use the selected execution target.
> - Recovery must respect the controller that owns an active run.
> - A host path or PID does not describe a remote sandbox.
> - This pull request checks sandbox paths on the target and preserves
live controller leases.

## Linked Issues or Issue Description

**What happened?**

Selecting an AI account for a sandbox agent could fail because the
Claude ACP environment check tried to create the sandbox directory on
the Paperclip host. The recovery sweep could also interrupt a sandbox
run while its controller lease was still valid. It treated a PID absent
from the local host as proof that the run had stopped.

**Expected behavior**

ACP checks directories on the selected execution target. Recovery leaves
a run with a live controller lease alone. Its final database write
rejects a stale snapshot after renewal, a claim, a controller change, or
a runtime change.

**Steps to reproduce**

1. Test a Claude ACP sandbox agent with a directory that cannot be
created on the host. The check fails before this fix.
2. Give a running sandbox task a valid controller lease and a PID absent
from the host. Run the stale-lock sweep without an in-memory handle. The
sweep interrupts the run before this fix.
3. Renew or replace the controller between the sweep's read and write.
The old snapshot must not end that controller's run.

**Paperclip version or commit**

Rebased onto `origin/master` at
`0e9b24c8216171c26c8358ba387d77858e02c7a9`. All seven regression cases
still fail against this base.

Refs #13438, which supplies the managed hiring and task-connection
behavior, and #13433, which preserves non-assignee subscription comment
wakes. This PR preserves both upstream changes and addresses the two
remaining sandbox failures.

## What Changed

- Resolve and create Claude ACP test directories through the
execution-target helpers.
- Preserve active legacy controller leases during stale-lock recovery,
including finalization after a task becomes terminal.
- Recheck the controller, lease, runtime mode, and native ownership in
the terminal database write.
- Add two sandbox-directory cases and five database-backed
controller-lease cases.
- Document the target used for ACP directory checks.

## Verification

- Red: all seven new cases fail against `0e9b24c82` without these two
implementation changes.
- Before the final upstream sync, 192 focused tests passed. Full `pnpm
test:run` coverage completed using the repository's group/shard runner:
all general server and workspace groups passed, and all 147 serialized
server suites passed across the initial run and isolated continuations.
Five cold-import timeout suites passed with
`--experimental.fsModuleCache`; their assertions and deadlines were
unchanged.
- The hiring routes, default-selection service, and upstream hiring
tests match `origin/master` exactly. The two remaining fixes are
unchanged by the final rebase.
- On final head `bd28d5cefbdf7084acc3759199cd9661907e7a26`, all 372
focused tests pass across 16 suites covering both upstream changes and
these fixes. Two timeouts in the combined run (database setup and an
existing ACP case) pass in isolated reruns with fresh test homes and
temporary directories. `pnpm -r typecheck` and `pnpm build` also pass on
this head.
- All 32 active checks pass on final head `bd28d5cef`, including the
full test matrix and browser shards, in [CI run
34904204849](https://github.com/paperclipai/paperclip/actions/runs/34904204849).
Two Storybook checks are skipped by path filters.
- Greptile reviewed final head `bd28d5cef` at 5/5 with no findings or
unresolved review threads.

## Risks

- A failed remote directory check still blocks connection adoption.
- A live controller retains finalization authority after its task
becomes terminal. Cleanup waits for ownership to expire and must pass
the final ownership check.
- No schema or credential-storage changes.

## Model Used

OpenAI GPT-6 in Codex, with reasoning, repository inspection, code
execution, and API tools. The exact serving model ID and context-window
size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-14 22:13:57 -05:00
DottaandPaperclip 47ded8bf97 feat: manage AI runtime credentials through Connections (#13247)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agent runs need credentials for a specific provider and sign-in
method.
> - Connections already owns accounts, grants, and access permissions.
> - AI authentication should use those same boundaries.
> - This pull request adds the storage, API, adoption, and runtime
foundation.
> - Legacy agents keep their authentication until they explicitly adopt
a managed connection.

## Linked Issues or Issue Description

**Problem or motivation**
AI credentials are configured separately from Connections. Agents cannot
consistently reuse a responsible user's account or a permitted shared
account.

**Proposed solution**
Manage AI accounts with the existing Connections grants and permissions.
Keep model and harness selection independent from credential selection.
Preserve legacy authentication until validated adoption.

**Alternatives considered**
A separate credential registry would duplicate ownership and access
policy. Automatic fallback would risk using the wrong account.

**Roadmap alignment**
This extends the shipped Apps, multi-user, secrets, and agent-runtime
capabilities. The maintainer requested the feature and reviewed the UI.
Related groundwork: #11899 (connection permissions), #10910 (connection
wizard), #11692 (Claude subscription profiles), and #11854 (Codex
account rotation).

## What Changed

- Add AI-purpose/runtime-auth contracts and an additive, idempotent
migration.
- Add Claude, OpenAI, OpenRouter, and Grok provider capabilities and
catalog entries.
- Store credentials on grants. Resolve responsible-user defaults or
explicit permitted grants.
- Isolate managed credentials and provider sessions across accounts.
Block missing credentials without ambient fallback.
- Keep imported legacy secrets unchanged during reconnect. Use
independent local Codex/Grok sign-in attempts for rotating credentials.
- Add authorization, migration, concurrent refresh, retry, cancellation,
and legacy-compatibility tests.

This is part 1 of a two-PR stack. The app UI follows in #13248. Merge
the foundation first.

## Verification

- Updated against master `04e364236`, preserving upstream provider login
and connector workflows.
- Full workspace typecheck, production build, Storybook build, and token
gates passed on the integrated branch. Final local-login changes passed
59 focused tests; new-agent and inbox regression suites passed 63 tests.
- Browser checks verified automatic local Claude account detection,
resumable Codex login commands, retry, focus restoration, and
desktop/phone layouts. Commands create their isolated directory before
invoking the CLI.
- All current-head CI checks passed on `2a996560a`, including all
server/workspace tests, browser shards, runner verification, typecheck,
build, and canary dry run. Greptile reviewed that commit at 5/5 with no
unresolved threads. Earlier local full-suite attempts hit the Mac
PostgreSQL shared-memory limit; the complete suites passed in CI.
- Renumbered the additive AI migration to `0276` after upstream
migrations and regenerated its snapshot. Existing legacy agents retain
their configuration.
- Added local login status checks, owner-scoped retry, managed OpenCode
remote homes, credential-aware model discovery, and task
connection-repair delivery.

## Risks

- Managed credential failures intentionally block execution. They do not
restore legacy fallback.
- Preview-era copied Codex/Grok subscriptions require independent
reconnect.
- The integrated branch has live provider acceptance coverage. This
update verifies local Claude detection and Codex API-key task repair; it
does not add a new subscription authorization/refresh or Daytona stress
pass.
- Runtime-auth connections must stay excluded from tool and channel
handling.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository tools, code
execution, and browser testing. The exact runtime model identifier and
context-window size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-12 16:30:10 -05:00
DottaandPaperclip ab15aff390 feat: add experimental persistent agent chat (#13284)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Conversations must use the same tasks, controls, and execution
history.
> - Users need an ongoing chat with an agent without managing task
properties.
> - Agents should clarify and plan work, then hand execution to assigned
project tasks.
> - This pull request combines the reviewed Agent Chat stack for one
squash merge.
> - The benefit is persistent conversation with normal task governance
and shared UI.

## Linked Issues or Issue Description

**Subsystem affected**

Task lifecycle, agent runtime tools, shared task UI, and browser/paid
runner tests.

**Problem or motivation**

Users need one persistent conversation with each agent. A separate chat
store or renderer would duplicate task behavior and bypass existing
controls.

**Proposed solution**

Use a task-backed chat per company, user, and agent. Reuse the task
composer and transcript. Clarify and plan in chat, then create assigned
project tasks with the relevant plan. Keep Agent Chat behind its own
disabled-by-default experimental setting.

**Roadmap alignment**

This implements the task-backed direction in [CEO
Chat](https://github.com/paperclipai/paperclip/blob/master/ROADMAP.md#-ceo-chat).
Related proposals: #2504 and #9693. Related request: #7981. The
maintainer requested one squash merge of the complete stack.

Consolidates the reviewed runtime
[#13281](https://github.com/paperclipai/paperclip/pull/13281), backend
[#13282](https://github.com/paperclipai/paperclip/pull/13282), and UI
[#13283](https://github.com/paperclipai/paperclip/pull/13283) layers
with this PR's E2E coverage. All four layers passed CI and received
Greptile 5/5 before consolidation. This PR targets master and includes
the complete feature.

## What Changed

- Add personal canonical chat tasks with ordinary company visibility,
immutable identity, idempotent first sends, and an idle waiting state.
- Process `/new` in queue order. Preserve history, release a chat pause,
and fence old provider context and delayed writes.
- Keep chat lifecycle rules across recovery, finalization, assignment,
task lists, and rollups.
- Support research and plan revision in chat. Hand plans to ordinary
assigned project tasks before execution starts. Reject new chat
subtasks.
- Add repository-aware project creation and discovery tools, including
multiple repository IDs and GitHub URLs, authorization, idempotency, and
durable project-created cards.
- Reuse task UI components for chat, with starred/recent agent
navigation and a separate `enableAgentChat` experimental flag.
- Add deterministic browser tests and 24 paid chat cells across four
Codex/Claude profiles, with validated reports and screenshots.
- Integrate current master recovery, controller lease, queued-message,
and task UI changes. Gate chat interruption and deferred promotion on
ownership/feature policy. Guarantee lease renewal and active controls
are stopped even if teardown fails.
- Preserve master's migration 0273 and generate chat migration 0274 with
idempotent replay for development databases.

## Verification

- Prior exact heads of all four PRs passed Linux CI, including build,
typecheck, general/serialized tests, and browser E2E. Each had Greptile
5/5 and no unresolved findings.
- Integrated local verification passed: full repository typecheck and
production build, Storybook build, token gates, 340 focused UI tests,
all 20 deterministic chat browser tests, two migration replay tests, 88
focused chat/queue/native/controller tests, and provider/session
regressions including real lease expiry. These include the three
lifecycle regressions for the final admission/teardown fixes; server
typecheck also passes. Current head
`1268eda16cc2af892055917e7292f068820be135` has Greptile 5/5 with no
unresolved findings and passing security scans. All final-head CI gates
passed: build, full Runner verification, typecheck/release registry,
canary, all general/serialized test shards, and all browser E2E shards
([CI
run](https://github.com/paperclipai/paperclip/actions/runs/34696739927)).
Local PostgreSQL startup contention required serialized retries; skipped
fixtures do not count as passing coverage.
- The earlier paid campaign passed all 24 chat cells and retained 32
screenshots:
[report](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-34648511170-1/index.html?report=agent-chat#suite-agent-chat).
It tested `abacbdfd2f660709ec37312cdb758284c8399d04`; it is prior
evidence, not a paid run of this integrated head.
- Manual check: enable Agent Chat in Experimental settings, open an
agent, clarify and revise a plan, then hand off to an assigned project
task. Stop a reply, send `/new`, and verify fresh context with retained
history. Disable the setting and verify agent shortcuts/new chat turns
are blocked.

## Risks

- Queue/session integration can affect retries and delayed writes. Tests
cover ownership, cancellation, reset boundaries, idle recovery, and
ordinary task behavior.
- Migration 0274 adds conversation fields and constraints. Replay is
idempotent and preserves existing development chat history.
- This combines the previously reviewed stack at the maintainer's
request. Agent Chat remains off by default and is separate from
Conference Room.

## Model Used

OpenAI Codex, GPT-6 Astra (`gpt-6-astra`), with reasoning, code
execution, browser tools, and parallel review. The exact context-window
size is not exposed in this session. Codex and Claude also ran as test
subjects in the linked paid campaign.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-12 08:56:04 -05:00
DottaandPaperclip 3bafac12f7 refactor: remove automatic productivity reviews (#13263)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Its recovery loop keeps assigned work moving after execution
failures.
> - Productivity review used run counts, comment counts, and elapsed
time to create management tasks.
> - Infrastructure failures could satisfy those rules and create more
tasks without evidence that the source work needed management review.
> - This pull request removes that detector and its continuation holds.
> - Bounded recovery, budgets, explicit blockers, and normal review
stages remain in place.
> - Existing task records stay readable and unchanged.

## Linked Issues or Issue Description

Refs #5897. That request describes unwanted automatic productivity
reviews and asks to preserve existing tasks. This change retires the
feature instead of adding another configuration switch.

Related prior approaches: Refs #9191, Refs #12489. Those changes
excluded infrastructure failures or bounded review creation. This
removal replaces the detector rather than tuning its thresholds.

## What Changed

- Delete the scheduled detector, automatic task creation, evidence
refresh, and productivity continuation holds.
- Remove computed productivity fields, special attention items, badges,
and Storybook fixtures.
- Retain historical origin values, decision compatibility, and recovery
recursion exclusions. Add no migration and change no existing task data.
- Update the execution contract. Replace feature tests with regressions
for legacy task reads, ordinary attention, and bounded continuation in
the presence of an old review.

## Verification

- Targeted attention, issue-route, startup, and UI tests: 4 files and
101 tests passed.
- Updated issue-route and UI tests: 2 files and 61 tests passed.
- Bounded continuation regression: 2 cases passed, including a legacy
review plus pre-dispatch cancellation churn.
- `pnpm check:token-gates`: all four gates passed.
- `git diff --check`: passed.
- `pnpm build-storybook`: passed.
- Greptile: 5/5 on `a5a612eea`, with no actionable findings.
- Scheduler and historical recovery regressions: 2 files and 28 tests
passed.
- Repository `pnpm -r typecheck` and `pnpm build`: passed.
- The complete `pnpm test:run` suite passed across the CI server,
serialized-server, and workspace shards on `a5a612eea`. Stopped the
duplicate local monolithic run after the full CI suite passed; no
completed local full-suite result is claimed. The targeted local suites
above passed.
- CI serialized shard 5 initially hit a 10-second timeout in the first
interaction-route test. The complete file passed locally (78 tests),
then the single CI rerun passed.
- All CI gates are green, including the build and end-to-end suites.
- A local merge check against current `master` (`ce09ea40b`) completed
without conflicts.

## Risks

- API responses no longer include the computed `productivityReview`
field. Consumers must stop using it.
- The scheduler no longer creates management work from elapsed time, run
counts, or missing comments. This is the intended behavior change.
- Existing review tasks and explicit dependencies remain in place.
Historical origins still prevent recursive recovery treatment. No task
cleanup or data migration occurs.
- The native review handoff repair is separate from this removal.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, tool use, and code
execution. The exact runtime model identifier and context-window size
are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-11 15:46:35 -05:00
DottaandPaperclip d10cbde815 fix(recovery): reject stale productive continuation wakes (#13173)
## Thinking Path

> - Paperclip manages agent work through tasks and runs.
> - Recovery continues assigned work when no live execution path
remains.
> - A recovery sweep can read an in-progress task before its run
completes.
> - The sweep can then observe the successful run after completion has
changed the task status.
> - This pull request checks current status and assignment under the
existing enqueue lock.
> - A stale continuation leaves a skipped wake receipt and creates no
run.
> - Task chat also omits an empty continuation cancelled before it
started because its task had become terminal.

## Linked Issues or Issue Description

Related public work: #10779 and #8419. Those older open changes address
terminal disposition across other recovery paths. This change uses the
existing scheduler guard for productive successful-run continuation and
adds real database lock contention coverage.

**What happened?**

Recovery could combine an old in-progress task snapshot with a newer
successful run. It queued an automatic continuation after the task was
done. Dispatch cancelled that run before it started, but task chat
displayed “Couldn't start” below the successful answer. This can happen
after the native runner's finish result has already been accepted. It
does not require a missing comment.

**Expected behavior**

Productive continuation must remain eligible when enqueueing acquires
the task lock. Completion, cancellation, reassignment, or a move away
from in-progress must prevent creation of the run. Actual execution
stops must remain visible.

**Steps to reproduce**

1. Let recovery select an assigned in-progress task whose latest run
succeeded with productive progress.
2. Hold the task row lock in another transaction and change the task to
done.
3. Let recovery attempt to enqueue while that transaction holds the
lock.
4. Commit completion. Before this fix, recovery creates a redundant run
from the stale snapshot.

**Paperclip version or commit**

Reproduced against master at `4042eb1c4` with deterministic integration
tests.

**Deployment mode**

Built from source with PostgreSQL. The bug is in core recovery and is
not adapter-specific.

## What Changed

- Pass the existing status-and-assignee guard for productive terminal
continuation recovery.
- Preserve a skipped wake receipt with the expected and actual task
state, without creating a run.
- Test actual PostgreSQL lock contention for native and legacy
completion, cancellation, backlog, review, blocked state, and
reassignment.
- Omit empty redundant pre-start cancellations from native and legacy
task chat. Preserve stop markers for runs that started.
- Document recovery eligibility at enqueue time.

## Verification

- All seven new race cases failed before the guard was connected.
- `pnpm -r typecheck` passed.
- `pnpm build` passed.
- `pnpm check:token-gates` passed.
- Task chat suite: 98 tests passed.
- Recovery integration suites: 290 tests passed, including 31
stale-queue tests.
- Full CI verification passed on `7ea71f04d`: all 31 active checks
succeeded, including all test shards, browser tests, build, typecheck,
and canary release dry run. Storybook visual regression was skipped by
its path filter.
- Greptile reviewed this commit at 5/5 with no review threads.
- The local `pnpm test:run` aggregate reported a setup failure in the
unchanged `tool-access-service.test.ts` suite. Its isolated rerun passed
all 231 tests without edits. The duplicate aggregate was stopped after
the complete CI matrix passed; it is not counted as a successful local
full-suite run.

## Risks

Low risk. The backend guard applies only to productive successful-run
recovery. It requires the task to remain in-progress with the same
agent. Other wake sources keep their current policy. The UI change only
suppresses empty redundant cancellations; run records remain available.
No schema change or migration is required.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository inspection, code
edits, and local test execution. The exact deployment snapshot and
context window are not exposed by this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-11 08:51:50 -05:00
DottaandPaperclip 889947c238 feat: add experimental native chat connectors (#13038)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - People also ask agents for work in their existing chat tools.
> - Each external conversation needs one task and a current authorized
source.
> - Retries, Stop, and provider failures must not duplicate work or
expose private data.
> - The first chat PR establishes the opt-in provider and data
contracts.
> - This PR adds experimental channel integration and its durable
control plane.
> - Users can request work from connected channels and inspect delivery
in Paperclip.

## Linked Issues or Issue Description

Refs #13100 and #13092. This is the second of exactly two chat PRs.
Foundation #13100 is merged and changed 143 files. Runner prerequisite
#13092 is also merged. This PR changes 400 files against master, below
the 500-file review limit. It contains no wireframe images or HTML
galleries.

## What Changed

- Add native Slack, GitHub, Microsoft Teams, Telegram, and Discord chat
connections. Keep chat disabled unless the operator enables experimental
chat connectors. Preserve the production GitHub tool connection and its
normal setup path.
- Bind each provider bot identity to one immutable Paperclip agent. Bind
each admitted external conversation to one task. Paperclip owns tasks,
runs, permissions, and audit records.
- Add durable admission, per-conversation queues, questions, task
controls, progress, final replies, images, files, and delivery receipts.
Board comments remain internal unless explicitly sent to the channel.
- Check current identity, provider reach, resource access, credentials,
runtime generation, and exact source before provider effects. Keep
private responses private. Never send raw reasoning, private logs,
credentials, or tool arguments.
- Hold uncertain sends for explicit audited resolution. Make Board
Send-to-channel atomic and idempotent. Keep reconnect and setup
credentials in Paperclip secret storage.
- Preserve current native-runner authority across retries, lost
acknowledgements, and recovery. Keep immutable input and completion
contracts separate from newer user input. Receipt reconciliation cannot
launch a provider.
- Reconcile chat close/new ordering and provider-effect lock order.
Audit resource access changes in the same transaction. Submit only the
selected resource from each UI toggle so stale pages cannot undo
unrelated access changes.
- Drain Codex stdout before certifying process exit. Bound the drain
with the existing shutdown grace. Preserve observed terminal authority
without treating an undrained process as successful or reusable.
- Incorporate master `018ca5da` with its ACP Stop, mobile task layout,
runner packaging, and official lock changes. Preserve dedicated
chat-answer continuations in both directions when ordinary queued
comments are adopted after Stop.
- Fence late adapter readiness behind an earlier Stop for the same run.
Preserve verified cleanup for registered adapters. Handle single Stop,
agent pause, duplicate Stops, and failure release without creating a
false cancellation receipt.
- Incorporate master's `6dd48cad4` wake-queue extraction. Preserve exact
failed-chat retry authorization and lineage, retired question-source
suppression, and the block on generic recovery that would discard the
admitted source. Fresh deferred input retains its separate promotion
path.
- Incorporate master `2a05b5ed3` and its queue-admission extraction,
simplified transaction ports, and separate runner CI job. Preserve exact
durable receipts, actor separation, and dedicated-answer isolation
through the new module. A failed receipt insert rolls back the
accompanying deferred-wake merge.

## Verification

Current head: `afe19299d06253cb628eb398e91d1200ea9f412a`, incorporating
master `2a05b5ed3457ea33efd6895520447d1d97fe98d8`. The conflicts are
resolved. This successor fixes two test-harness boundaries exposed by
CI: per-case route-module preparation and actual durable-save completion
before intentional runner termination. Production code and all existing
test/turn deadlines are unchanged. [Exact-head Greptile
review](https://github.com/paperclipai/paperclip/pull/13038#issuecomment-5587250594)
is **5/5**, completed September 10 at 13:20:55 UTC, with no actionable
findings or open review threads. [Fresh exact-head
CI](https://github.com/paperclipai/paperclip/actions/runs/34481724341)
passes **all 24 jobs**, including Build and both required aggregates.
Normal exact-head guarded merge was attempted and rejected by the
remaining branch approval policy: CODEOWNER review is required and no
human approval is present. Normal **squash auto-merge is enabled** as of
September 10 at 13:36:26 UTC. Requested CODEOWNERS have been notified;
no approval bypass or self-approval was used. Earlier-head results below
remain historical evidence, not qualification of this successor.

- Final exact-head Linux evidence: 995/995 chat integration cases; 36/36
agent-skills routes; 35/35 runner live-session cases, including real
process kill/resume; 1948 runner Vitest cases with three existing
benchmark/platform guards; 870/870 API-authority cases; and 104 browser
cases with four existing optional skips. Rust, conformance/replay, full
repository build, typecheck, canary, all server/workspace shards, and
both required aggregates pass with normal CI concurrency. Earlier failed
attempts remain recorded below.

- Latest test-only qualification: 141/141
route/permissions/authentication cases pass in separate cold forks, with
plain server types and independent review clear. The real-runner suite
passes 35/35, with plain runner types and independent review clear. A
controlled premature-save acknowledgement fails as expected; matching
ownership/effect/process evidence, rejected saves, real turn outcome,
test abort, and pre-kill liveness are covered. No local reproduction of
the original CI scheduling failure is claimed. The preceding [CI
run](https://github.com/paperclipai/paperclip/actions/runs/34479680858)
passes 21/24 jobs, including all 995 Linux chat cases and browser
aggregate (104 passed, four existing optional skips); only Build, the
skills serialized shard, and the required verification aggregate fail.
Its exact-head Greptile review was 5/5. Both failed job logs are
retained.

- Final fixture qualification: all eight focused Discord cases and all
995 chat integration cases pass. The exact modal statement/PID is
observed before taking the real connection lock; the test then proves
its actual blocking relationship before mutation. Original SQL
execution, provider behavior, negative assertions, and 1s/15s timeouts
remain unchanged. Independent review is clear and test/production hashes
remain frozen. The preceding [CI
attempt](https://github.com/paperclipai/paperclip/actions/runs/34477184777)
passed 22 jobs, including Build/runner, typecheck, canary, all other
test shards, and browser aggregate (104 passed, four existing optional
skips); the two fixture failures and failed verification aggregate
remain recorded, not relabeled as a pass.

- Current queue-module composition: 308/308 recovery/batching/queue/Stop
tests; 995/995 full chat integration; 89/89 module tests, including real
PostgreSQL receipt-insert rollback; 24/24 workflow/module-boundary
tests; plain server and UI types. All four actual local process/ACP
browser paths pass in 1.4 minutes. Fresh databases, no skips or retries,
stable reviewed source hashes. The initial boundary failure is retained;
its no-op service wrapper was removed without changing recovery context
or weakening the check. An exploratory standalone test-directory
typecheck fails because its new upstream transformation config is not a
standalone typechecking project; standard CI/build does not invoke it,
and no configuration was weakened to suppress those diagnostics.

- The preceding head `e02a63d462ce5d47433b0aeb632bb6fd20aab1ba` passed
[all 24 CI
jobs](https://github.com/paperclipai/paperclip/actions/runs/34436462958)
and exact-head Greptile review at 5/5. Required CODEOWNER review
prevented its normal merge before master advanced again.

- Final extracted-module composition: 307/307 recovery, batching, queue
and Stop-control tests; 995/995 full chat integration; 49/49 module
tests including eight PostgreSQL adapter cases; and 19/19 issue-update
tests. Plain server types pass. All four actual local process/ACP
browser paths pass in 1.3 minutes. Fresh databases, no skips or retries
in these cohorts, frozen source hashes, and independent review clear.

- The preceding head `3e4e1c1c` passes [all PR CI
jobs](https://github.com/paperclipai/paperclip/actions/runs/34415826820),
including Build and required `ci / verify` and `ci / e2e`. Both the
original Rust failure and the previously load-sensitive lineage fixture
pass with unchanged Linux concurrency. Master advanced afterward and
required this reconciliation.
- Final master composition: 448/448 focused UI tests, 186/186 adapter
tests, 24/24 queue/control tests, and 11/11 packaging tests. Plain UI,
server, shared, and adapter types pass. Token gates and diff checks
pass. Independent server and UI reviews are clear.
- Stop-registration regression: both real-service cases fail against
exact `a95` source and pass with the fix. The full corrected
recovery/control suite passes 265/265. Duplicate-owner and failed-Stop
controls also pass. Plain server types pass. The readiness barrier
prevents provider startup without adding an acknowledgment to an already
terminal run.
- Final qualification strengthens terminal-field equality and repeats
both affected cases successfully on a fresh database. All four actual
local process/ACP browser paths pass again in 1.3 minutes, without skips
or retries. The final screenshot shows Cancelled, a paused subtree,
retained input, and no error toast.
- Two new actual-service regressions fail before the merge fix. They
prove that queued-comment adoption could consume a dedicated chat answer
or add unrelated input to that answer. The fixed four-case cohort
passes, including ordinary upstream continuation and adapter Stop
controls. Full recovery passes 257/257. All four actual local
process/ACP Stop browser flows pass in 1.4 minutes, without skips or
retries, on a fresh database.
- The unchanged runner artifact was qualified with 171/171 transport
tests, 870/870 API-authority tests, conformance 1/1, and replay 11/11.
Six controlled reader tests prove the exit/drain repair. Its local
serial Rust workspace passed 546 top-level cases plus two invoked
helpers; the later passing Linux CI supplies default-concurrency
evidence.
- Prior exact-source full chat integration passes 995/995. Settings
regressions cover concurrent stale pages, 501 destinations, pending
state, rejected updates, and explicit retry. These deterministic tests
do not prove live provider behavior.
- Retained failed attempts and their causes are in the [qualification
log](https://github.com/paperclipai/paperclip/blob/afe19299d06253cb628eb398e91d1200ea9f412a/doc/plans/chat-adapters/2026-09-08-chat-queue-and-webhook-repair.md).
The first merge adapter run timed out while macOS slept for 290 seconds.
Its unchanged repeat passed with a temporary sleep guard. No assertion,
deadline, or CI gate was weakened.

Review commands include `pnpm --filter @paperclipai/server exec vitest
run src/__tests__/heartbeat-process-recovery.test.ts
src/__tests__/issue-queued-comments-routes.test.ts` and `pnpm exec
playwright test --config tests/e2e/playwright.config.ts
tests/e2e/acp-stop-continuation.spec.ts`. Database suites require fresh
disposable databases. See the [browser
runbook](https://github.com/paperclipai/paperclip/blob/afe19299d06253cb628eb398e91d1200ea9f412a/doc/plans/chat-adapters/2026-09-04-chat-adapters-browser-e2e-runbook.md)
for provider setup and separate live acceptance steps.

## Risks

- This remains experimental. Deterministic tests and bounded live
evidence do not establish every provider feature, tenant, permission
layout, or media shape. Teams work-tenant qualification is still open.
- Failed and uncertain provider effects remain visible and can require
operator action. A transport receipt does not prove recipient
visibility.
- Native controller and runner artifacts must remain compatible.
Preserve lease ownership, terminal authority, source binding, and
quarantine during future changes.
- Access and audit rows commit together, but activity notifications
remain best-effort. This is not a new durable event outbox.
- The PR operation does not deploy a live server, replace its runner, or
change provider permissions. Remaining live qualification is documented
in the [temporary
handoff](https://github.com/paperclipai/paperclip/blob/afe19299d06253cb628eb398e91d1200ea9f412a/doc/plans/chat-adapters/2026-09-08-open-qualification-followups.md).

## Model Used

OpenAI Codex assisted with implementation, tool execution, testing, and
review. The work records `gpt-6-astra` assistance. The environment does
not report a context-window size. No private reasoning traces are
included.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-10 10:06:45 -05:00
DottaandPaperclip 018ca5daaf fix: verify ACP Stop and preserve safe continuation (#13119)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Task controls coordinate provider execution and queued user
messages.
> - Stop could finish before an embedded ACP provider stopped its tools.
> - A later request could be held for reconciliation without a clear
task response.
> - A restored provider could also retain the stopped run's API
credential.
> - This pull request verifies provider termination and preserves safe
session continuation.
> - Operators can continue known-safe work and see why uncertain work
cannot start.

## Linked Issues or Issue Description

**What happened?**

Stop could leave an embedded ACP provider running. A queued follow-up
followed by “go” could fail before it reached the provider. Task chat
could show a generic missing-response message. Even a restored session
could use the previous run's credential and fail its task update.

**Expected behavior**

Stop waits for confirmed provider termination. A later explicit wake
continues the same compatible session only when recorded actions have
known outcomes. It carries pending comments and the current run's
environment. Uncertain actions retain a visible reconciliation hold.
Composer Stop preserves the existing pause rule: conversation can
continue while paused, but task work requires Resume.

**Steps to reproduce**

1. Start an embedded ACP task.
2. Send a second request while the provider is running.
3. Interrupt the run, then send “go”. Also test composer Stop followed
by Resume work.
4. Check that the request is delivered once and that the provider can
complete the task through the current run's API credential.
5. Repeat with an unfinished write. Confirm that the write stops and
that further execution stays blocked with a visible reason.

**Paperclip version or commit**

Built from source on master at `3bc60dd8b` plus this branch.

**Deployment mode**

Local source build with an isolated embedded PostgreSQL instance.

Refs #11183. Refs #12552. Those changes address recovery after operator
cancellation. This change also covers embedded ACP termination, session
proof, pending-comment delivery, and task feedback.

## What Changed

- Propagate Stop into embedded ACP and wait for bounded adapter cleanup
and provider exit. Retain the actual ChildProcess object for forced
termination on all platforms; never signal a recycled numeric PID.
- Preserve interrupted checkpoints only for acknowledged, local,
persistent sessions with settled reads or no tools. Keep writes,
incomplete actions, and forced termination blocked.
- Restore the same compatible provider session with the current run's
environment. Reject fresh-session fallback for an interrupted
checkpoint.
- Adopt pending comments on the next explicit wake. Stop alone does not
dispatch them.
- Share the execution-blocker rule across dispatch, Resume, and task
detail. Show Stopped or Couldn't start with the recorded reason. Resolve
the stopped agent for the run link, including reviewer runs.
- Keep execution reconciliation holds intact when generic recovery sees
queued comments or healthy child tasks.
- Add process, service, component, and browser regression coverage. Fix
disposable database cleanup and React test settling exposed by the full
suite.

## Verification

- Passed `pnpm -r typecheck`, `pnpm build`, and `pnpm
check:token-gates`.
- Passed all three `acp-stop-continuation.spec.ts` browser journeys.
They use an actual ACP child process and require task completion through
the agent API.
- Passed 165 adapter execution, operator-stop, and child-process control
tests, 17 queued-comment route tests, and 65 tests in the two adjusted
UI suites. Earlier focused recovery, heartbeat, and task-control tests
also passed.
- Manually used the browser to queue a request, Stop, send “go” while
paused, and Resume. The same session answered once and moved the task to
Done with the current run's credential.
- Manually interrupted an unfinished write. Its file size stayed fixed
for five seconds. “Go” showed the reconciliation reason and did not
start another provider prompt.
- Separate live Claude ACP smoke checks confirmed that Stop ended a
disposable local write and that a no-tool interruption could resume the
exact provider session. The browser fixture does not call Drive or
another external app.
- Passed all 5,615 UI tests and 3,090 other workspace tests. The CLI and
general server groups pass with targeted retries: two transient server
failures passed together on retry, and two embedded-database startup
failures passed after removing abandoned shared-memory segments from
this task's completed browser fixtures. All 144 serialized server suites
completed, with 2,189 tests passing after two transient HTTP socket
failures passed on retry.
- Passed all 135 heartbeat process/recovery tests, including a
deterministic regression that failed before the recovery-sweep fix.
- Passed 18 dispatch integration tests, including stopped-reviewer
links, company boundaries, and malformed run IDs.
- Greptile is 5/5 on `7dd170d83`, with zero unresolved review threads.
The security scan and all required CI gates pass for the same commit.

## Risks

- Safe continuation depends on complete tool reporting and a restorable
local provider session. Unknown outcomes remain blocked and require
reconciliation.
- Provider cleanup can take time. A timeout does not grant replay
permission.
- The change adds optional adapter context fields and an optional issue
projection. It does not change the database schema or require a
migration.
- Test cleanup truncates company data only in a disposable test
database.

## Model Used

OpenAI GPT-6, running as Codex with repository tools, code execution,
and browser interaction. The runtime does not expose a more specific
model deployment ID or context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-09 22:06:06 -05:00
DottaandPaperclip 2991a59b17 fix(adapters): prevent engine fallback and preserve usable runtime defaults (#13105)
## Thinking Path

> - Paperclip manages agents that must write work and report task
outcomes through its API.
> - Local adapters select an execution engine and its permission
settings.
> - A higher ACP Node requirement can make an unchanged installation
lose access to its default engine.
> - The adapter then silently selects CLI, which can change permissions
and block API access.
> - This pull request keeps the engine choice fixed and reports missing
prerequisites before work starts.
> - It also gives explicit Codex CLI runs usable defaults and keeps
managed services on a supported Node runtime.

## Linked Issues or Issue Description

Refs #12215. Related changes: #11792 raised the Node requirement; #13094
addressed separate runner networking behavior. This change fixes the
engine-selection and managed-launcher paths.

**What happened?**

An unchanged agent could switch from ACP to CLI after an upgrade. Codex
CLI then used read-only permissions with networking disabled. The run
could finish without updating its task. Repeated recovery attempts used
the same unavailable setup. Managed updates also skipped the Node check
and did not refresh old launchers.

**Expected behavior**

An unavailable engine must fail with a clear setup error. It must not
silently select another engine. Explicit CLI runs must be able to write
workspace files and call the API unless the operator configures stricter
settings. Managed updates must validate Node and keep child tools on
that runtime.

**Steps to reproduce**

1. Run an ACP-default agent under Node 22 after the ACP minimum rises to
24.11.
2. Leave the engine unset and disable the approval/sandbox bypass.
3. Observe the old adapter select CLI and fail to write task disposition
through the API.
4. Start a managed service with an old launcher and a supervisor PATH
that selects a different Node for child tools.

## What Changed

- Remove automatic engine fallback for Codex, Claude, Gemini, and Kimi.
Check prerequisites for default and explicit ACP selections.
- Return a configuration error with proof that provider work did not
start. Stop automatic continuation retries for this error.
- Enable Codex ACP workspace networking at the actual turn boundary.
Upstream mode presets otherwise force it off even when config.toml
enables it. Preserve explicit network denial and read-only mode.
- Set workspace-write and network access defaults for explicit Codex CLI
runs. Preserve explicit sandbox modes, profiles, and network
restrictions.
- Pin the validated Node directory in managed launcher PATH. Refresh
legacy launchers during installs and npm/Git updates.
- Reject updates on unsupported Node. Keep update checks, dry runs, and
rollback available.
- Synchronize the qualified Codex ACP executable identity across server,
TypeScript runner, Rust runner, and provider-pack launch paths.
- Add regression tests and update engine and installation documentation.

## Verification

- [Full CI passed on the final
head](https://github.com/paperclipai/paperclip/actions/runs/34387099695):
typecheck, build/native runner verification, all general and serialized
test shards, all browser shards, release registry, canary dry run, and
policy checks.
- Greptile: 5/5 on `2c1d6e2815830a5cd39e36c8a082cc0c4441b6c0`, with no
unresolved review findings. Security gates are green.
- Full workspace typecheck and build also passed locally. The final
deployed Linux build passed.
- Full Codex, Claude, Gemini, and Kimi source test suites: 804 passed, 2
skipped. Installer, updater, and launcher tests: 47 passed. Installed
ACP turn-boundary tests: 3 passed. ACP packaging tests: 14 passed.
Focused recovery classification tests also passed.
- Real Linux Codex CLI runs, both fresh and resumed, wrote a workspace
file and reached the control-plane health API with the new defaults.
- Explicit read-only and network-disabled control probes retained those
restrictions.
- A real ACP run on the final deployed Linux build wrote a file and
reached the control-plane API with HTTP 200, without engine fallback.
The same probe failed DNS before the turn-policy patch.
- Executable-identity and installed-policy contracts: 12 passed.
Affected native server tests: 197 passed. Runner factory tests: 21
passed. Rust qualification and native provider integration tests: 11
passed.
- Deployed the production changes to a Linux service on Node 24.20 after
a verified database backup. Health, bootstrap readiness, static UI,
executable/cwd identity, and guarded restart checks passed. The restart
lost no runs.
- Corrected stale Kimi skill-default and Gemini remote-archive fixtures;
both suites pass.

## Risks

- Default or legacy auto engine settings now fail when ACP is
unavailable. Operators who intend to use CLI must select it explicitly.
- Codex CLI now permits workspace writes and networking by default, and
ACP workspace-write turns permit networking by default. Explicit
operator sandbox settings remain authoritative.
- Old managed launchers keep their pinned Node until they are
reinstalled under a supported runtime. An old updater cannot repair
itself; the documentation gives the current installer command.
- Custom service wrappers and global/source installations must configure
their runtime PATH. No database migration is required.

## Model Used

OpenAI Codex, based on GPT-6, with reasoning, repository inspection,
shell execution, and test tools. The exact serving model identifier and
context-window size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-09 13:27:24 -05:00
DottaandPaperclip 35fdc0c66b fix: make task recovery durable and preserve current requests (#13075)
Make task recovery durable and preserve the latest user request across native and legacy continuations. Keep routine recovery quiet and prevent replay when action outcomes are uncertain.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-09-09 09:14:25 -05:00
5acf56658b feat(onboarding): first task opens as a chat with a chief of staff (#13068)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Onboarding ends by handing a new user to their first agent on a
seeded first task
> - Today the wizard asks for a mission up front, the UI composes what
the agent is told, and the agent starts running before the user says
anything
> - New users get a cold, ticket-shaped start, and nobody can edit the
agent's brief or persona without a code change
> - This pull request makes the first task a short chat: a four-step
wizard, a chief-of-staff persona, a greeting plus a two-option opening
card, server-owned markdown texts, and no run until the user answers
> - It also gives question cards one consistent action row (Cancel /
Skip / Next), makes agent hires idempotent within a run, and turns the
Paperclip Runner flag on by default for self-hosted instances
> - The benefit is a first run the user steers, with texts a board
operator can edit as markdown

## Linked Issues or Issue Description

No public GitHub issue exists for this change. The feature request
fields follow.

Related PRs and issues:

- Refs #11043 — an earlier draft of the first-task onboarding
experience. This PR supersedes it.
- Refs #11280 — a report about the onboarding first-task route test.
This PR extends that test file.

### Subsystem affected

Onboarding wizard, the seeded first task and its texts, task-chat
question cards, agent hiring, and the instance experimental settings.

### Problem or motivation

The onboarding wizard collects a mission through two extra steps and a
questionnaire. The UI then composes the first agent's instructions and
the first task description from those answers. The first task wakes the
agent at once, so the agent runs and posts before the user types a word.
Board operators cannot change the greeting, the brief, or the persona
without editing TypeScript. Question cards in chat behave differently
per adapter, and a single-select pick submits on click. A misread hire
response could create a duplicate agent that the creating agent cannot
remove.

### Proposed solution

Reduce the wizard to four steps and stop the UI from authoring agent
texts. Move the greeting, the brief, the chief-of-staff persona, and the
opening question into markdown and JSON files that the server loads at
runtime. Seed the persona onto the first agent through an explicit hire
marker. Do not wake the first task until the user answers the opening
card or types. Give every question card the same Cancel / Skip / Next
actions. Add an experimental toggle that switches the single-task
proposal between one confirmation card and a plan document with a
checkbox card. Make agent hires idempotent within a run.

### Alternatives considered

- Keep the mission questionnaire and feed it into the brief. Rejected:
the agent asks better questions in chat, and the wizard gets shorter.
- Keep the first task open-ended with a plain composer. Rejected: a
two-option card gives the user a clear first move.
- Derive the plan-document behaviour from the user's intent only.
Rejected in favour of an explicit experimental toggle so operators can
choose.
- Key the "pick does not submit" behaviour off the presence of a submit
label. Rejected: several adapters set a submit label on single-select
cards, and their cards would change behaviour.

### Roadmap alignment

`ROADMAP.md` lists no planned core work on onboarding or the first task.
This change refines the existing flow and does not duplicate planned
work.

## What Changed

- Wizard: four steps (Name your organization, Create your first agent,
Connect a model, Review). The front door and both mission steps are
removed with their state and saved-progress keys. The UI no longer
composes the first agent's instructions or the first task description.
- Server-owned texts: the greeting, the brief with two proposal
variants, the chief-of-staff persona, the opening question, and a README
live in `server/src/onboarding-assets/first-task/` and load at runtime.
The create route stores the assembled brief and ignores any client
description.
- Persona seed: an `onboardingFirstAgent` marker on the hire lets the
server seed the chief-of-staff persona over the first agent's entry
file. Board-authored hires only. The persona tells the agent the hire
response shape and to list agents before it acts on an unclear result.
- No auto-run: the first task does not queue an assignment wake. The
stranded-assignment reconciler leaves it idle until a user comment or an
answered card exists.
- Opening card: the server seeds an `ask_user_questions` card right
after the greeting with two options: "Interview me and propose a plan
and an agent team to execute it." and "I have a task in mind" with free
text. Answering wakes the agent.
- Experimental toggle `enableFirstTaskPlanProposal` (default off): the
single-task proposal is one confirmation card, or a plan document plus a
checkbox card when on.
- Question cards: every `ask_user_questions` card renders Cancel, Skip,
and Next (the submit label on the last question). Skip hides on required
questions. Picking an option no longer advances or submits by itself.
- Wizard guards: the dashboard's agentless offer ignores a cached empty
agent list while a refetch is in flight. The hire step adopts an agent
that already carries the typed name instead of hiring "Name 2".
- Agent hires are idempotent within a run: a retry of the identical
request under the same run id returns the existing agent with `200` and
`idempotent: true`. The fingerprint covers the whole validated request,
so a corrected payload is a new hire. Lookup, create, and activity
record run under one lock per company and run, so overlapping retries
cannot both create.
- The Paperclip Runner experimental flag defaults to on for self-hosted
instances. Cloud keeps its declared default: a managed instance whose
tenant row and managed overlay omit the flag resolves it to off.
- Question cards: a send that finds an earlier required answer missing
returns to that question with a message instead of failing silently.
- The two onboarding e2e specs follow the new wizard: the front door and
growth intake shots are gone, and the planning-mode spec dismisses the
opening card before it reads the composer.
- Docs: `docs/board-operator/editing-first-task-texts.md` explains how
to edit the texts and the toggle.

## Verification

Commands, run from the repo root:

```
pnpm -r --filter './packages/*' --filter '!@paperclipai/paperclip-runner' build
pnpm --filter ./packages/shared typecheck
pnpm --filter ./ui typecheck
pnpm --filter ./server exec tsc --noEmit
pnpm check:token-gates
pnpm --filter ./ui exec vitest run OnboardingWizard onboarding QuestionForm InteractionCard ProtocolCard TaskChatComposer Dashboard feature
PAPERCLIP_IN_WORKTREE=false pnpm --filter ./server exec vitest run onboarding-first-task heartbeat-process-recovery agent-hire-idempotency instance-settings agent-skills-routes issue-onboarding onboarding-greeting --testTimeout=90000
```

Results on this branch:

- Typecheck is clean for shared, ui, and server.
- Token gates: 4 of 4 clean.
- UI: 344 tests pass across 23 files.
- Server: all suites pass. The first test in `agent-skills-routes` has
its own 10 s cap and needs about 15 s on my laptop for the app cold
start. It passes with a longer cap. This PR does not change that cap.

Manual steps on a dev instance:

1. Open `/onboarding`. Confirm four steps: Name your organization,
Create your first agent, Connect a model, Review.
2. Finish the wizard. Confirm the first task shows the chief-of-staff
greeting and the opening card with two options. Confirm no run starts.
3. Pick "Interview me…". Confirm no run starts. Press Continue. Confirm
a run starts and an interview card of 3–4 questions arrives.
4. On a fresh organization, pick "I have a task in mind", type a task,
and press Continue. Confirm a proposal arrives as one confirmation card.
5. Turn on Settings → Experimental → "First task: propose with a plan
document" and repeat step 4. Confirm a plan document and a checkbox card
arrive.
6. Visit the dashboard after the hire. Confirm the wizard does not
reopen and one agent exists.
7. Open any question card. Confirm Cancel returns the plain composer
with the card still pending, Skip advances an optional question, and
Next moves to the next question.

Design reference with flow diagrams, chat mock-ups, and live captures:
https://pages.paperclip.ing/first-task-flow/proposed/

## Risks

- `pnpm dev` now builds the runner daemon because the Paperclip Runner
flag is on by default. Developers without a Rust toolchain must set
`PAPERCLIP_RUNNER_BINARY` or turn the flag off. Self-hosted instances
that never set the flag now let qualified agents use the runner.
- The wizard drops the mission steps and their saved-progress keys. A
user who is mid-wizard on an older build restarts at step 1 after an
upgrade. Existing organizations are not touched.
- The first task no longer runs on its own. A user who neither answers
the card nor types sees no agent activity. This is intended.
- The persona seed applies only to hires that carry the marker from the
wizard. API hires are unchanged.
- Hire idempotency is scoped to one run id and to the exact request.
Retries across runs, or with a changed payload, still create a second
agent. The lock is per server process, which matches how an instance
serves its API.
- Single-select question cards no longer submit on pick. Users of
adapters that relied on that behaviour now press Next.
- No database migrations.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude (Anthropic) through Claude Code. `claude-fable-5-1` with
extended thinking, tool use, and code execution wrote most commits.
`claude-opus-4-8` wrote the toggle, texts, wizard, and idempotency
commits, as the `Co-Authored-By` trailers show.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-08 20:19:14 -07:00
DottaandPaperclip 7ed122911b Add end-to-end session goals to Paperclip Runner
Add capability-aware slash-goal controls, durable provider goal state, PRP v2 negotiation, autonomous goal execution, and safe local session recovery. Integrate with current master, preserve provider session identity, and verify the browser goal/chat/replacement/clear workflow and unsupported-agent rejection.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-09-08 16:18:47 -05:00
Jannes StubbemannandClaude Fable 5.1 5752d6bd93 fix(heartbeat): block runs on a stuck sandbox plugin and re-enable errored bundled plugins at boot (#12957)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents run inside environments. A sandbox environment gets its
sandbox from a provider plugin (for example the bundled
`paperclip.kubernetes-sandbox-provider`), and every run starts by
acquiring a lease through that plugin.
> - When a plugin activation fails once (on a hosted deployment: one
`RPC call "initialize" timed out after 15000ms`), the loader calls
`markError`. That persists `status = error` on the plugin row and
switches off worker auto-restart. Boot activation (`loadAll`), the
bundled-plugin bootstrap and the lazy worker recovery all consider only
`ready` plugins, so the plugin stays in `error` across restarts until an
operator enables it by hand.
> - Every run that needs the provider then fails before dispatch with
`Sandbox provider "kubernetes" is installed via plugin "...", but that
plugin is currently error.` That message matches neither the retryable
classifier (`... but its worker is not running`) nor any configuration
classifier, so the run is recorded as a plain `setup_failed`, the issue
is released, and the scheduler dispatches the same failing run again on
the next tick. On the hosted deployment one company produced about
11,300 identical failed runs, one every 30 seconds, for a week (#12953
is a customer's report of the same condition).
> - Two gaps cause this: the heartbeat treats a condition that only an
operator can change as a transient setup failure, and the bundled-plugin
bootstrap never gives a plugin in `error` another chance even though the
bundle ships with the release image.
> - This pull request classifies the "installed but not ready" lease
failure as `configuration_incomplete`, so the existing recovery path
moves the issue to `blocked` with one recovery action and an actionable
notice; and it re-enables a bundled plugin found in `error` once per
boot, so the next server restart heals the plugin.
> - The benefit is that a stuck provider plugin surfaces as one blocked
issue per task with clear next steps, instead of an endless stream of
identical failed runs, and a restart repairs the plugin without an
operator having to know the plugin API.

## Linked Issues or Issue Description

- Refs #12953 — hosted report: "that plugin is currently error" on every
run for six days, including runs that were retried by hand. This PR
stops the retry loop (issue goes to `blocked`) and makes a server
restart re-activate the bundled plugin. It does not change how a managed
Kubernetes environment is provisioned for a company, which the same
report also mentions.
- Related PR: #9760 pauses the agent for the permanent `Adapter "..." is
not in the configured adapter registry` setup failure. This PR handles a
different permanent condition (plugin not `ready`) and routes it through
the existing `configuration_incomplete` recovery path (issue-level block
with a recovery action) rather than an agent-level pause, because the
gap is on the plugin, not on the agent. The two do not overlap in code
paths.
- No existing issue covers the bundled-plugin re-enable. Bug
description:

**What happened**

A bundled sandbox provider plugin went to `status = error` after one
failed activation. It stayed in `error` across every later server
restart. Every run for every agent on that provider failed lease
acquisition in under a second with `... but that plugin is currently
error.` (`setup_failed`), and the heartbeat kept dispatching new runs
that failed the same way.

**Expected behavior**

A run that fails because its provider plugin is not `ready` is recorded
as a configuration gap and the issue is moved to `blocked` with a notice
that names the plugin and its status, so no further runs are dispatched
until an operator acts. A bundled plugin left in `error` gets a fresh
activation attempt on the next boot.

**Steps to reproduce**

1. Install a sandbox provider plugin and create a sandbox environment
that uses it; make it an agent's default environment.
2. Set the plugin row's status to `error` (or make its worker fail
`initialize` once so the loader does it).
3. Assign an issue to the agent and let the heartbeat run it.
4. Observe: the run fails with `... but that plugin is currently error.`
as `setup_failed`, the issue is released, and the next tick dispatches
another run that fails the same way. Restart the server: the plugin is
still `error`.

**Paperclip version**

master at 856813ba3 (`fix(connections): distinguish local setup from
provider handoff (#12947)`).

**Deployment mode**

Hosted (Kubernetes, bundled kubernetes sandbox provider plugin). The
heartbeat behavior is the same in self-hosted mode.

## What Changed

- `server/src/services/heartbeat.ts`
- New exported `parseSandboxProviderPluginNotReadyFailureMessage()`
recognises environment-runtime's `not_ready` lease message (`... is
installed via plugin "<key>", but that plugin is currently
error|disabled|upgrade_pending`) and returns the provider, plugin key
and status. It does not match the transient `... but its worker is not
running` message (still retried) or the permanent "not installed"
message (unchanged).
- In the setup-failure catch, a matched message sets `errorCode =
configuration_incomplete` and records (independently of whether the
agent lookup succeeded) a `configurationIncomplete` payload with
`reason: "sandbox_provider_plugin_not_ready"`, the provider,
`pluginKey`, `pluginStatus`, and a `fingerprint` of
`sandbox_provider_plugin:<key>:<status>`, so repeated failures on the
same stuck plugin reuse one recovery action. The existing recovery flow
then blocks the issue, skips the infra retry, and posts one notice.
- The two places that build the configuration-incomplete notice now pass
the run's payload so the notice can name the specific gap.
- `server/src/services/recovery/stranded-notice.ts`:
`buildConfigurationIncompleteRecoveryNoticeSeed` takes the optional
payload. For `sandbox_provider_plugin_not_ready` the body names the
plugin and its status and gives status-specific guidance
(`sandboxProviderPluginRemedy`): review and approve the upgraded
capabilities before enabling for `upgrade_pending`, enable again for an
operator `disabled`, enable or restart for `error`. Other reasons keep
the secret/env-binding wording. Exports
`SANDBOX_PROVIDER_PLUGIN_NOT_READY_REASON`.
- `server/src/services/recovery/service.ts`: the recovery action's
`nextAction` for this reason uses the same status-specific guidance
instead of "bind the missing secret(s)". Small refactor:
`readConfigurationIncompletePayload` backs the existing fingerprint
reader.
- `server/src/services/bundled-plugins.ts`
- `ensureBundledPlugins` no longer skips a present bundled plugin whose
status is `error`. It logs at `warn` with the row's `lastError`, resets
the row to `ready` with `lastError` cleared through
`registry.updateStatus` (a plain status reset, not `lifecycle.enable()`,
so no `plugin.enabled` event fires before the worker runs; the startup
`loadAll()` that follows does the activation and its events), and
continues boot on failure. This runs once per boot by construction; if
activation fails again the loader marks `error` again and nothing
retries until the next boot.
- `installed`, `ready`, `disabled` and `upgrade_pending` rows are still
skipped, so an operator's `disabled` stays untouched.
- `BundledPluginProvisionerDeps` gains `registry.updateStatus` and
`logger.warn`; `app.ts` already passes objects that have both.
- `doc/plugins/PLUGIN_SPEC.md`: one bullet in 12.4 Failure Policy about
the once-per-boot re-enable of bundled plugins.
- Tests
- `server/src/__tests__/bundled-plugins.test.ts`: re-enables an `error`
row exactly once with the `lastError` in the warn log and no reinstall;
continues boot and provisions later entries when `enable` throws; still
skips `installed`/`ready`/`disabled`/`upgrade_pending` without calling
`enable`.
- `server/src/__tests__/heartbeat-process-recovery.test.ts` (embedded
PostgreSQL): a plugin row in `error` plus a sandbox environment produce
a run with `errorCode = configuration_incomplete` and the expected
payload, the adapter is never dispatched, no retry or second run is
created, the issue is `blocked`, the recovery action is
`configuration_validation` with the plugin next action, and the notice
names the plugin key and status. Plus a unit case for the message parser
(positive for the three statuses and a wrapped message, negative for
both other sandbox messages).
- `server/src/services/recovery/stranded-notice.test.ts`: the
plugin-specific body, and the unchanged secret-binding body for other
reasons.

## Verification

- `cd server && pnpm typecheck` — passes.
- `cd server && pnpm exec vitest run
src/__tests__/bundled-plugins.test.ts` — 29 tests pass.
- `cd server && pnpm exec vitest run src/services/recovery/` — 77 tests
pass (includes the stranded-notice and classification suites).
- `cd server && pnpm exec vitest run
src/__tests__/heartbeat-process-recovery.test.ts -t "sandbox
provider|retryable pattern|secret ref has no binding"` — 5 tests pass:
the two new cases, the existing transient worker-restart retry, the
existing non-retryable "not installed" escalation, and the existing
secret-binding `configuration_incomplete` block (embedded PostgreSQL).
- Manual check for a reviewer: set a sandbox provider plugin row to
`status = 'error'`, run an agent on that provider, and confirm the issue
moves to `blocked` with a "Configuration incomplete" notice that names
the plugin, and that no second run appears. Restart the server and
confirm the boot log shows `bundled plugin is in error status from a
previous activation; re-enabling it for this boot` followed by normal
activation.

## Risks

- Behavior change: a run against a plugin in `error`, `disabled` or
`upgrade_pending` now blocks the issue instead of failing as
`setup_failed` and being re-picked. For `disabled` this is deliberate:
an operator switched the plugin off, and re-dispatching cannot help. The
block is reversible from the issue (retry or reassign) like every other
`configuration_incomplete` block.
- The classifier is anchored on the exact `... but that plugin is
currently <status>` phrase from `environment-runtime.ts`. If that
message changes, the run falls back to the previous `setup_failed`
behavior (no worse than today). A unit test pins the phrase.
- Bundled re-enable: a bundled plugin whose activation fails on every
boot now costs one activation attempt (the `initialize` timeout, 15 s by
default) per boot instead of none. It runs inside the existing
non-awaited bootstrap chain, so boot time is unaffected. Non-bundled
plugins are untouched.
- No migration, no schema change. The `configurationIncomplete` payload
is JSON in `heartbeat_runs.result_json`, read only by the recovery
service.

## Model Used

- Claude Fable 5.1 (`claude-fable-5-1`) via Claude Code, extended
thinking, tool use (file edits, shell, test runs). The change was
produced with the model and reviewed by the submitting human.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_014t3bi2beVNVVHAxK36dmXm

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-08 09:00:10 -07:00
Nicky LeachandPaperclip 3ed5b7c5c8 refactor(server): extract the active-run output watchdog into a feature module (#12853)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The server recovery service monitors active runs and applies
watchdog decisions
> - The watchdog rules and database operations lived in one large
recovery service
> - This structure made the rules harder to test and made company
scoping harder to inspect
> - This pull request moves the watchdog into domain, application, and
adapter layers
> - The benefit is a smaller recovery service, pure policy tests, and
clear company-scoped ports

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The active-run output watchdog that detects silence, suppression, and
terminal evidence.

**Current behavior**

The recovery service contains the watchdog policy, use cases, database
operations, and process control in one file.

**Proposed behavior**

A feature module separates pure policy, use cases and ports, and
Postgres and process adapters. The recovery service delegates its public
watchdog methods to this module.

**Reason and benefit**

The separation makes policy decisions easy to test. Company identifiers
on every reader and writer port make tenant scope clear. Smaller service
methods reduce change risk.

**Breaking changes**

None. The recovery service keeps its public methods and call sites.

Related public watchdog work includes
[#7043](https://github.com/paperclipai/paperclip/pull/7043) and
[#7770](https://github.com/paperclipai/paperclip/pull/7770).

## What Changed

- Add the `server/src/modules/active-run-watchdog/` feature module with
domain, application, and adapter layers.
- Move watchdog policy, use cases, Postgres access, and local process
control into the module.
- Keep the recovery service public methods and delegate them to the
module.
- Add 53 pure module test cases and retain 8 Postgres integration cases.
- Add company scoping and transaction rollback coverage.

## Verification

- Run `vitest run --config vitest.config.ts src/modules` and confirm 3
files and 53 cases pass.
- Run `vitest run --config vitest.config.ts
src/__tests__/heartbeat-active-run-output-watchdog.test.ts` and confirm
1 file and 8 cases pass.
- Run the full server suite in pull request CI.
- Compare the type-check result with a fresh baseline on the same
checkout.

## Risks

The main risk is a behavior change in recovery decisions during the move
across layers. The pure policy tests cover the moved rules. The
integration tests cover database behavior, company scope, and
transaction rollback. Pull request CI runs the full server suite.

## Model Used

OpenAI Codex, GPT-5, runtime-managed context window, tool use, code
execution, and repository review support.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-05 09:15:59 -07:00
Dotta bcc6fe7a44 fix(runner): restore multi-turn remote sessions (#12840)
## Thinking Path

> - Paperclip manages AI agents and their work.
> - The runner executes agent turns on local and remote providers.
> - A remote per-turn session must save its state before Paperclip
releases its sandbox.
> - The session runtime returned after 100 milliseconds while the remote
checkpoint still ran.
> - The next turn also checked the local state path instead of the
verified remote backup.
> - This pull request waits for the bounded remote close and accepts
only a verified suspended backup.
> - The benefit is reliable multi-turn execution without weaker identity
checks.

## Linked Issues or Issue Description

**What happened?**

A successful remote agent turn released its sandbox before the runner
saved the verified continuation backup. The next turn failed with
`runner_state_identity_mismatch`.

**Expected behavior**

Paperclip must finish the bounded remote checkpoint before it releases
the sandbox. A later turn must validate and restore the digest-matched
suspended backup.

**Steps to reproduce**

1. Run a native ACPX Claude Plan test in a non-reusable Daytona sandbox.
2. Reject the first plan to start a second turn.
3. Observe that the second turn fails before provider execution.

**Paperclip version or commit**

The failure reproduced at `13775a90b078ff64872f50961ea1b83d575e7bc6`.

**Deployment mode**

GitHub Actions with a Daytona sandbox.

## What Changed

- Wait for the internally bounded remote runner close and checkpoint
before the host returns.
- Preserve the existing short cleanup bound for other providers.
- Validate remote continuation lifecycle from a complete digest-verified
backup when local runner state is absent.
- Keep corrupt, non-suspended, mismatched, and unverified state
fail-closed.
- Make native Plan completion and accepted-Plan wake prompts
deterministic.

## Verification

- A prior 45-cell local campaign passed 44 cells. The only failure was
the OpenCode Plan prompt variance fixed here.
- A focused OpenCode local Plan rerun passed.
- ACPX Claude Daytona message and question cells passed.
- Focused regressions cover delayed checkpoint close and verified remote
backup lifecycle.
- GitHub Build and the focused ACPX Claude Daytona Plan cell will
validate this exact head.

## Risks

Remote runnerd sessions now wait for their internally bounded
close/checkpoint path before returning; generic provider cleanup retains
the existing 100 millisecond bound. Durable run success still cannot be
reversed. The environment release guard still blocks sandbox destruction
when no verified backup stamp exists.

## Model Used

OpenAI Codex, GPT-5.6, extended reasoning, with code execution and
GitHub Actions inspection.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal task
id
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open findings
- [ ] I will address all Greptile and reviewer comments before
requesting merge
2026-09-05 06:25:06 -05:00
Dotta 7b094724e6 fix(runner): recover native sessions across restarts (#12845)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The Paperclip Runner keeps durable run and provider state outside
one server process.
> - A server restart can leave that runner alive or can interrupt it
after a provider checkpoint.
> - The old startup path used handoff intent and PID evidence, but it
did not reconstruct native ownership.
> - That gap could block the issue, create a replacement run, or start
duplicate provider work.
> - This pull request adds durable same-run recovery for coordinated and
uncoordinated restarts.
> - The benefit is exact recovery of the run, runner, session, provider,
steering, and finalization state.

## Linked Issues or Issue Description

Refs #9628. That pull request added earlier local-adapter hot-restart
work. This change adds native PRP authority reconstruction and same-run
provider resume.

Refs #10935. That pull request handles missing hot-restart snapshots.
This change also supports hard restarts with no snapshot.

Refs #11624. That pull request prevents unsafe retry after an adopted
legacy process exits. This change reconciles native terminal evidence
before provider recovery.

Refs #12070. That pull request improves process liveness checks. This
change also binds recovery to a process-start fingerprint and fails
closed on ambiguity.

**What happened?**

The server could record hot-restart intent, but startup did not rebuild
native runner ownership. A live runner could not re-register its PRP
authority. A dead runner could not resume the exact native and provider
session on the same heartbeat run. Generic recovery could then block the
issue or create replacement work.

**Expected behavior**

A live native runner must reconnect with the same PID and logical
identities. A dead runner must resume the same durable session and
heartbeat run with only a new operating-system PID. A proposed or
terminal result must finalize once before any provider turn starts.
Ambiguous process or session evidence must stay blocked without a signal
or duplicate spawn.

**Steps to reproduce**

1. Start a Paperclip Runner heartbeat and wait for an active provider
turn.
2. Restart only the Paperclip server, with or without a hot-restart
marker.
3. Observe that the old startup path does not reconstruct the native
control-plane authority.
4. Kill both the server and runner after a provider checkpoint.
5. Observe that the old path cannot resume the exact native session on
the original heartbeat run.

**Paperclip version or commit**

The defect was reproduced from commit
`1991f31fd53e7f7794d5c2e4b93be384ade2b41d`. This branch is rebased onto
the current `master`.

**Deployment mode**

Local development and self-hosted server deployments that use the local
Paperclip Runner.

## What Changed

- Added correlated hot-restart requests and version-compatible native
handoff fields.
- Added controller boot identity, process-start identity, controller
generation, recovery state, request id, and bounded history to the
native finalization ledger.
- Added transactional recovery claims for live-runner reattach,
dead-runner resume, and incomplete bootstrap.
- Added fail-closed ownership takeover rules and process identity
validation.
- Added live runner adoption to the local runner transport without a
duplicate spawn.
- Added same-run provider checkpoint resume and legacy retry-row
compatibility.
- Reconciled proposed and terminal results before runner or provider
recovery.
- Bound the HTTP and PRP listener before startup recovery and delayed
scheduling and generic reapers until classification completes.
- Added restart-aware health diagnostics, run-log recovery transitions,
durable runner diagnostics, and bounded shutdown finalizer draining.
- Moved restart-survivable diagnostics into runner-owned, pre-redacted
bounded writes; raw stdout and stderr are never persisted.
- Added process-start fencing for controller, runner, and provider PIDs;
startup classifies every candidate without an implicit cap.
- Added crash-recoverable, contention-safe development restart-request
coordination and failed-startup listener cleanup.
- Added a credential-free real-process restart suite for eight restart,
scale, and identity scenarios.
- Documented native restart operation, persistence, diagnostics, and
verification.

## Verification

- The documented native restart commands passed. They ran eight
real-process/database recovery scenarios and the live runner adoption
transport test.
- Native executor tests passed: 111 tests.
- Heartbeat recovery tests passed: 124 tests.
- Hot restart, health, and shutdown tests passed: 52 tests.
- The broader affected server suite passed: 350 tests.
- Focused native recovery and startup tests passed: 49 tests.
- Runner transport and control-plane tests passed: 63 tests.
- Runner-owned diagnostic tests passed for write-time bounding,
credential redaction, private file modes, and raw stream
non-persistence.
- Development restart coordination tests passed: 11 tests.
- Database migration checks and the partial-application/replay
regression test passed.
- Server, database, and Paperclip Runner typechecks passed.
- `git diff --check` passed.
- Full Paperclip PR CI passed, including build, canary, all five general
server shards, all five serialized server shards, all three browser E2E
shards, workspace suites, and release-registry verification.
- Greptile completed at 5/5 with no outstanding findings,
recommendations, follow-ups, or open review threads.

## Risks

- Moderate risk. This changes startup ordering and ownership transfer
for active native runs.
- The migration adds nullable columns and does not rewrite existing
rows.
- Recovery fails closed when process or durable session identity is
incomplete or contradictory.
- The first implementation supports the local Paperclip Runner. Remote
targets keep their existing behavior.
- The real-process suite covers cleanup and asserts that no runner or
provider process survives each test.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex with GPT-5. The runtime did not expose a more specific
model revision or context-window size. Repository editing, shell
execution, database tests, and real-process test execution were enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-04 15:03:53 -05:00
Nicky LeachandPaperclip 184b014c25 feat(telemetry): add the agent.task_run event and emit it at every terminal run transition (#12809)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Paperclip records agent run outcomes through telemetry and run
lifecycle services
> - Terminal run transitions need one consistent event for outcome
analysis
> - The current paths do not report every terminal transition through
one event
> - This pull request adds the agent.task_run event and emits it at each
terminal transition
> - The benefit is complete run outcome data without exposing raw task
identifiers

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Paperclip telemetry reports agent activity, but it does not report every
terminal task run through one event.

**Subsystem affected**

Cross-cutting (multiple of the above): packages/shared telemetry and
server run lifecycle services.

**Current behavior**

Several run paths write a terminal status without a matching
agent.task_run telemetry event.

**Proposed behavior**

Each terminal run transition emits one agent.task_run event. The event
records the terminal state and uses the existing pseudonym helper for
the optional task identifier.

**Reason and benefit**

Complete terminal-run data helps operators measure agent outcomes. The
pseudonym helper prevents the raw task identifier from leaving the
installation.

**Breaking changes**

None. The change adds an event and keeps existing event behavior
compatible.

## What Changed

- Add the agent.task_run telemetry contract and client helper.
- Reuse the existing pseudonym helper for the task identifier. The
helper hashes the identifier with a per-installation salt and returns 16
hexadecimal characters. The raw identifier never leaves the
installation. Existing identifiers do not move.
- Emit one event from each legacy, native, recovery, and issue terminal
transition.
- Keep emissions outside database transactions and make delivery
best-effort.
- Add regression tests for event shape, hashing, terminal transitions,
and emission failures.
- Document the event and its privacy rule in the telemetry data
contract.

## Verification

- `npx tsc --noEmit` in `server/` passes at the submitted commit.
- The pull-request CI suite must pass. CI is the authority because local
Vitest has a known dependency artifact.
- The added regression tests cover event output shape, per-installation
hash divergence, raw identifier handoff, omitted identifiers, and
non-throwing emits.

## Risks

- A missed terminal path could reduce event coverage.
- Telemetry delivery remains best-effort and cannot change run
finalization.
- The pseudonym helper uses installation-specific state, so identifiers
differ between installations.

## Model Used

OpenAI Codex, GPT-5, tool use and code execution. Context window details
were not provided.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I have addressed all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-04 08:21:32 -07:00
MagnusandAndrew Aymeloglu b98badb246 fix(recovery): exclude hidden issues from stranded recovery and continuation wakes (#5648)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The recovery subsystem watches assigned issues and re-wakes an agent
whose run ended without finishing the work
> - Intake can hide a duplicate issue by setting `hiddenAt` while
leaving its status and assignee in place
> - The stranded-issue query and the terminal-run cleanup both ignore
`hiddenAt`, so a hidden issue is re-woken on every cycle
> - Nothing on the board shows the hidden issue, so the repeated wakes
have no visible cause
> - This pull request adds a hidden-issue guard to both predicates and a
test for each
> - The benefit is that hiding an issue stops recovery work on it, with
no other change in behavior for visible issues

## Linked Issues or Issue Description

**What happened?**

When intake marks an issue as a duplicate it sets `hiddenAt` but leaves
the status at `todo` or `in_progress` with the agent still assigned. The
stranded-issue recovery timer selects that issue on every tick and
queues an `issue_continuation_needed` wake for it. The agent's run on
the hidden issue fails or is cancelled, the terminal-run cleanup queues
immediate recovery for the same issue, and the cycle repeats
indefinitely. Hidden issues are invisible on the board, so nothing a
person can see explains the wakes.

**Expected behavior**

A hidden issue is never a recovery candidate. Stranded-issue
reconciliation skips it, and a failed, timed-out or cancelled run on it
releases the issue without queuing a continuation.

**Steps to reproduce**

1. Assign an issue to an agent and leave it `in_progress`.
2. Hide the issue (set `hiddenAt`, for example by marking it a duplicate
through intake) without changing its status or assignee.
3. Let a run on that issue fail, or wait for the stranded-issue recovery
timer.
4. Observe a new `issue_continuation_needed` heartbeat run queued for
the hidden issue on every cycle.

**Paperclip version or commit**

Reproduced on `master` when this PR was opened (May 2026). The two
predicates are unchanged on current `master`; this branch is rebased
onto it.

**Deployment mode**

Not deployment-specific: both guards are in the server's recovery and
heartbeat services and apply in every mode.

## What Changed

- `server/src/services/recovery/service.ts`: `isNull(issues.hiddenAt)`
added to the `reconcileStrandedAssignedIssues` candidate query, so
hidden issues never enter the stranded set.
- `server/src/services/heartbeat.ts`: `!issue.hiddenAt` added to
`issueNeedsImmediateRecovery`, so terminal-run cleanup releases a hidden
issue instead of queuing a continuation.
- `server/src/__tests__/heartbeat-process-recovery.test.ts`: one test
per guard. A failed run on a hidden issue queues no recovery run, and a
hidden stranded issue is left out of reconciliation.

## Verification

- `heartbeat-process-recovery.test.ts` covers both guards; CI runs it
against embedded Postgres.

## Risks

Low. Both changes narrow an existing predicate to exclude rows that
already carry `hiddenAt`; visible issues take exactly the path they take
today. A hidden issue that genuinely needs recovery would have to be
unhidden first, which matches how hidden issues behave everywhere else
in the board.

## Model Used

The original two-line fix was authored by @im0xMagnus. The rebase onto
current `master`, the two regression tests, and this description were
produced with Claude (claude-fable-5-1, extended thinking, tool use)
driven by a Paperclip maintainer through Prospector's triage flow.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Andrew Aymeloglu <aaymeloglu@gmail.com>
2026-09-03 22:05:20 -05:00
DottaandPaperclip 4b6de5327e Remove cheap model profiles (#12683)
## Thinking Path

> - Paperclip manages agents that use different model providers and
adapters.
> - Paperclip must keep agent execution rules clear and predictable.
> - The cheap-model profile added a second execution mode across
adapters, task recovery, APIs, and the UI.
> - That mode increased configuration and recovery complexity.
> - This pull request removes the cheap-model profile as a product
feature.
> - The benefit is one model-selection path for normal work and recovery
work.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

This change simplifies model selection across agent configuration, task
execution, recovery, and adapter capabilities.

**Current behavior**

Paperclip exposes cheap-model profiles in adapter metadata, agent
runtime configuration, task overrides, recovery rules, APIs, and the
board UI. Recovery work can select a different model profile from the
agent's configured model.

**Proposed behavior**

Paperclip uses the agent's configured model for normal work and recovery
work. Status-only recovery stays limited to coordination work. The API
rejects legacy model-profile configuration. A migration removes stored
model-profile values from existing agent, issue, and historical revision
records.

**Reason and benefit**

One model path reduces configuration, API, UI, and recovery complexity.
It also prevents status recovery from becoming a separate product-level
model-routing feature.

**Breaking changes**

This change removes model-profile fields and adapter capability
metadata. Existing stored model-profile values are removed by an
idempotent migration. The validators reject new legacy profile values
with clear errors.

## What Changed

- Removed model-profile types, adapter capabilities, API fields, and
model selection logic.
- Removed cheap-model controls from agent and task UI surfaces.
- Kept status-only recovery limited to coordination context while normal
continuations use the configured agent model.
- Added an idempotent migration that removes stored model-profile values
from agents, issues, and configuration revisions without changing issue
update timestamps.
- Updated tests and product documentation for the single-model behavior.

## Verification

- `pnpm check:token-gates` passes.
- `pnpm -r typecheck` passes.
- `pnpm build` passes.
- `pnpm test:run` completed with 5,607 passing tests and 8
environment-sensitive failures in unrelated fixed-port and
database-deadlock suites. The same failures repeated in an isolated
rerun. CI is the final clean-room result.

## Risks

- This is an intentional breaking change for clients that send
model-profile fields.
- The migration changes legacy agent, issue, and configuration-revision
JSON. It is idempotent and preserves unrelated fields and issue update
timestamps.
- The change is cross-cutting because the removed feature existed in
adapters, shared contracts, the server, plugins, and the UI.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex with `gpt-5`. Reasoning and tool use were enabled. The
runtime did not expose the context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-01 14:57:38 -05:00