Commit Graph
5 Commits
Author SHA1 Message Date
DottaandPaperclip 338bf75761 fix(chat): admit task Retry after GitHub reconnect (#17733)
## Thinking Path

> - Paperclip manages agents and their tasks.
> - Task Retry must preserve saved input and enforce current access.
> - A GitHub reconnect invalidates old runtime callbacks.
> - Retry reused that old callback authority and rejected valid new
intent.
> - Retry also replayed an older failed request when a follow-up was
already saved.
> - This change admits a deliberate Retry under current authority and
transfers saved input once.

## Linked Issues or Issue Description

Refs #16965, #17053 and #17317. These repairs preserve work and contain
old executors. This change fixes the subsequent chat admission boundary.
A search found no overlapping open reconnect/admission fix.

**What happened?**

Ordinary task Retry returned `chat_failed_run_retry_not_authorized`
after the same GitHub App reconnected. The endpoint was healthy, but the
saved request still carried its old runtime generation. A newer queued
follow-up also made Retry reject the older failed request instead of
continuing the follow-up. No successor was admitted.

HTTP qualification found two related defects. Concurrent clicks could
omit the canonical chat action from a response. Signed-in board dispatch
also passed an incomplete task resource to the permission resolver,
which rejected a permitted mutation.

**Expected behavior**

A deliberate Retry checks current authority for the same App, sender,
repository, conversation and task. It continues saved input once. A
later reconnect, changed App, revoked access or changed source prevents
dispatch. Old runtime callbacks remain invalid.

**Steps to reproduce**

1. Admit a GitHub request and record a failed native run.
2. Save a follow-up while execution is blocked.
3. Reconnect the same GitHub App.
4. Click ordinary task Retry.
5. Before this change, admission returns 409. Without reconnect, the
newer input alone also causes 409.

**Paperclip version or commit**

Baseline source: `41c42895fa862c586ff841f8c50200d93ffc83a4`.

**Deployment mode**

Source checkout on macOS. Regressions use real PostgreSQL transactions
and the assembled chat, HTTP route and heartbeat admission services.
Provider boundaries are simulated. No hosted recovery is claimed.

## What Changed

- Record the admitting App identity for new GitHub deliveries.
Historical deliveries can use their original working-comment binding.
Reauthorize deliberate Retry under a pinned current generation and
credential-reference fingerprint.
- Select the complete saved-input queue from the failed run's actual
conversation before replaying the older failed batch. Keep exact-run
Retry bounded while ordinary task Retry supports larger queues. Transfer
its receipts with the dispatch transaction. Recover the same ownership
after restart or a lost response. Release unadmitted input after a
definitive rejection.
- Preserve canonical chat action IDs across concurrent clicks. Supply
current task status and assignment fields to board mutation checks.
- Keep exact-run Retry, stale webhook, containment, source integrity,
publication, pause and budget checks. Update recovery contracts and
verification recipes.

## Verification

- Red: all three new PostgreSQL regressions failed on unchanged
production code. Reconnect failed at chat authorization. Newer input
failed at the obsolete-source guard. The combined case also failed.
- Green: 61 selected chat integration cases passed. These include 27 new
cases and 34 existing cases for containment, retained-source recovery,
exact batches and source revocation.
- A further red/green PostgreSQL regression reproduces the 51-message
overflow identified during review. All 51 messages reach one successor,
including a later Retry of that successor. A separate 52-message
consumed-batch regression also passes.
- A competing conversation retains its own deferred message. Distinct
automatic policy contexts are checked independently; revoking only the
later policy blocks the whole unadmitted batch. Batch validation still
rejects a later message with a changed App binding or deleted source. A
real PostgreSQL query capture reduced the 51-message preparation from
1,103 calls to fewer than 180.
- All 16 durable task Retry tests passed.
- The real HTTP route returns 202 and admits one queued successor for
concurrent deliveries. Fault injection covers rollback during input
handoff, response loss after dispatch commit, and permission loss when
resuming an existing durable wait. A wait only adopts input after
current admission succeeds; a one-shot settlement marker keeps status
projection read-only. A new deliberate Retry works after another
reconnect or a failed successor.
- Current-App, repository, sender, source edit/delete, reassignment,
endpoint pause and board-permission controls passed. Historical
deliveries remain unchanged.
- Final-head CI: 55 checks passed, three expected skips. The unchanged
minute-bucket rate-limit assertion passed locally against PostgreSQL and
on its one CI rerun. Greptile reviewed the final commit at 5/5 with no
unresolved threads.
- Workspace typecheck and full build passed. Map alignment, module
boundaries, UI token gates and `git diff --check` passed.
- Hosted task recovery and a live provider turn remain unverified. The
local tests hold agent capacity to inspect admission without starting a
model or publishing a GitHub response.

## Risks

- A deliberate GitHub Retry can admit old saved input under current
connection authority. It requires the same App binding and current
access. It pins that admission; it does not rewrite the historical
delivery or make stale callbacks valid.
- The durable chat queue has an optional admission callback. It
transfers input only when the transaction contains a valid queued
execution or saved wait. Other chat consumers have no callback and
retain their behavior.
- No migration, Cloud dependency, deployment, configuration change or
workspace cleanup is included.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, source inspection, code
execution and GitHub tools. The exact serving variant and context-window
size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-26 10:26:45 -05:00
DottaandPaperclip 3f1ad6b25b fix(chat): resume task Retry after retained-source containment (#17317)
## Thinking Path

> - Paperclip manages agent work through task and execution ownership.
> - A task Retry must retain the user's request while the system
verifies an old executor.
> - Connected conversations also retain their external principal and
accepted source batch.
> - Chat Retry admission required containment before it could save a
wait for that recovery.
> - A saved chat receipt could also remain deferred after containment
became available.
> - This pull request lets the exact recovery-bound request wait and
resume through its chat owner.
> - Normal admission still checks containment, current access and policy
before one successor starts.

## Linked Issues or Issue Description

**What happened?**

An ordinary task Retry on a connected conversation failed with
`failed_chat_retry_not_authorized` while its retained remote executor
still needed verification. The new recovery generation existed, but the
chat action failed before it could save a dispatch wait. This occurred
after the repairs in #17267 were confirmed live.

The local regression found a second gap. A processed chat action with a
deferred wake was not reconsidered after containment. Re-entering normal
admission also tried to insert its existing durable receipt again. A
historical hold for the same failed run could also hide the current
Retry's workspace strategy, so the request still waited after the old
executor was contained. Preparation could then read that sibling instead
of the recovery record that owned the fresh successor, losing the
fresh-workspace decision.

**Expected behavior**

Save the exact authorized Retry while its matching recovery generation
verifies the executor. Once containment is recorded, reconsider the same
chat receipt through normal admission. Start one successor with the
original source batch. Keep publication, principal, pause and budget
checks.

**Steps to reproduce**

1. Admit a GitHub or Telegram request and record a failed native run
with a retained remote lease.
2. Leave the executor's termination unconfirmed and exhaust its prior
recovery generation.
3. Use ordinary task Retry to create a new recovery generation.
4. Process the connected Retry. The old admission code fails instead of
retaining a wait.
5. With a deferred receipt, record exact containment and reconcile
again. The old dispatch path leaves it stranded.

**Paperclip version or commit**

Reproduced from `412914cfa78d5abefcc34d3eb8d4bb86e461caac`. The branch
is rebased onto current master.

**Deployment mode**

Hosted tenant for the reported failure. Isolated local PostgreSQL and
the assembled heartbeat/chat services for the regression.

Refs #17267 and #16965. This change does not add a Cloud dependency or
change the absence protocol.

## What Changed

- Permit a saved admission wait only for the exact current task Retry
and active source recovery generation.
- Keep promotion and execution dependent on independent containment
proof.
- Bind a Retry to active automatic recovery without replacing its
generation, attempt count or deadline.
- Share the current exact Retry strategy across historical holds for the
same run. Retain each hold's stop and effect checks.
- Select the canonical recovery record that proves the committed fresh
successor during workspace preparation.
- Let task receipt reconciliation return the same deferred action to its
chat owner.
- Recheck current board and external authority, then update the existing
wake atomically with its successor.
- Preserve the exact chat source batch. Keep generic board-comment
filtering unchanged.
- Add GitHub and Telegram cases for waiting, containment, one successor,
stale generations, source edits, endpoint pause and supersession.
- Update recovery documentation and capability maps.

## Verification

- Red: real PostgreSQL tests reproduced the admission rejection for
GitHub and Telegram. Six active-recovery cases then failed on the
generation-ID assumption. Two same-run historical-hold cases remained
deferred after containment on the old strategy lookup. A preparation
regression then returned no readiness when the historical hold was
stored first.
- Green: 73 selected chat integration tests and 394 continuation, task
Retry, workspace recovery, recovery-state and workspace-preparation
tests passed. The chat tests use the real chat admission and heartbeat
queue. Unrelated capacity stays occupied so no real provider starts.
They assert no executor before containment, one successor afterward,
unchanged source leases while waiting, preserved recovery budgets, and
denial after authority changes. A separate source hold still blocks
work. Both sibling insertion orders pass through the committed-readiness
lookup and real workspace preparation: the new directory has independent
Git storage and files, and unpublished old work remains intact.
- `pnpm -r typecheck`, `node scripts/check-maps.mjs` and `git diff
--check` passed after the review fixes. The source build passed on
`f6660efa48e37a80cd9857217a537606de85b5c8`.
- The local host reached its PostgreSQL semaphore limit. Successful runs
used a test-only loader to reduce disposable cluster connection and
worker limits. No repository database helper or application setting
changed.
- The full local `pnpm test:run` began before the final review edits,
failed in its first server phase and was stopped after those failures.
Later phases did not run. This is not a full local pass. The transient
Retry-notice case passes after fixture isolation; the four Discord
close-owned publication cases also pass in isolation. Other broad-run
failure causes are not established.
- Exact-head CI and security checks pass on
`f6660efa48e37a80cd9857217a537606de85b5c8`; Greptile is 5/5 with zero
unresolved review threads. Optional Paperclip Review completed neutral
with a 5/5 static assessment of all 11 changed files and no findings; it
did not run tests or hosted acceptance. All 57 current-head checks are
terminal: 54 success, two skipped and the optional neutral review, with
no failures or pending checks.
- The real hosted workflow confirmed that the deployed absence repair
recorded containment. A later ordinary UI Retry was accepted and saved a
durable request, but it remains behind the historical source hold. The
useful existing review remains unchanged. This follow-up is not
deployed; a live successful successor and final task disposition remain
unverified. No missing files are claimed recovered.

## Risks

- The existing task reconciliation sweep can restage a deferred chat
action. This reuses its receipt and generation; it does not grant new
execution or publication authority.
- Dispatch must keep the board Retry identity separate from the
connected request's principal. The regression checks these bindings and
preserves current chat authorization at each phase.
- Historical holds reuse a strategy only from the locked same-run
recovery that names the current Retry intent. A different run, malformed
recovery record or independent source hold remains fenced.
- Unknown executors without a matching active recovery still fail
admission. Promotion and execution require containment even when
admission saved a wait.
- No schema or API shape changes. No credential changes, manual
deployment or service restart.

## Model Used

OpenAI GPT-6 through Codex, with repository tools, code execution and
browser verification. The exact served model variant and context window
are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-23 07:51:34 -07:00
DottaandPaperclip 4c57da3708 fix(recovery): continue Retry after verified sandbox absence (#17267)
## Thinking Path

> - Paperclip manages AI agents and their work.
> - Task Retry must reconcile a stopped native execution before starting
another one.
> - A retained sandbox can disappear while workspace preservation is
pending.
> - A tenant-key error does not prove absence, especially after
credential rotation.
> - This pull request accepts only a short-lived Cloud observation bound
to the exact source and current preservation owner.
> - Normal Retry admission can then continue in a fresh workspace
without claiming that missing files were recovered.

## Linked Issues or Issue Description

**What happened?**

A native run stops with workspace preservation pending. Its Daytona
allocation no longer exists. Recovery tries to reopen it, then cannot
confirm a stop. The ordinary Retry remains deferred behind cleanup and
repeats a transient failure.

**Expected behavior**

Cloud must independently confirm absence in the recorded provider
organization. Core must verify that proof and settle executor
containment under the current physical owner. Normal Retry admission
must enforce all existing gates and create at most one successor. The
old run must remain failed and missing files must remain unavailable.

**Steps to reproduce**

1. Persist a failed native run with a retained Daytona lease and an
exhausted workspace-recovery generation.
2. Remove the exact provider allocation so reopen and stop cannot
confirm it.
3. Request ordinary task Retry.
4. Observe that recovery reports `provider_unavailable` and leaves the
Retry deferred.

**Paperclip version/commit**

Reproduced on source based on `6d44b80a5edbcc9f69bfc3d520994e70aa9fab6d`
with the prior preservation and Retry repairs present.

**Deployment mode**

Cloud tenant with native Runner and a retained Daytona source. The local
acceptance uses real Core, Cloud, PostgreSQL, signatures, native Runner
and UI, with deterministic model and provider HTTP boundaries.

Refs #16962, #16965, #17053. These repairs cover completion proposals,
durable Retry and retained-source reopening. They do not establish
absence of a missing provider allocation. Related #16809 preserves
orphan sandbox leases; this change requires independent proof before
settling an absent allocation.

## What Changed

- Add an HTTPS client for a purpose-separated, host-bound Cloud
sandbox-absence capability.
- Verify signature, audience, stack, expiry, source identities and
current physical-owner token.
- Settle only executor absence with a metadata and owner
compare-and-set. Preserve source protection and record that workspace
contents are unavailable.
- Leave successor creation, approval, pause, budget and uncertain-effect
checks to normal Retry admission.
- Document the protocol and update the recovery maps.

## Verification

- Red: the new persisted-state regression returned
`provider_unavailable` and scheduled recovery on the unfixed code.
Expected: `source_unavailable` with a contained old execution.
- Green: 384 focused tests pass across workspace recovery, signed
absence, explicit continuation, ordinary Retry, workspace preparation
and provider-owned fresh workspaces.
- Signature tests reject forged, expired and mismatched proofs. HTTP
failures or missing configuration grant no authority.
- Two missing allocations are both reconciled before Retry. A simulated
interruption reuses the first durable absence receipt.
- Concurrent Retry admission produces one queued fresh successor. It
records no successful old result and no recovered-file claim.
- Browser acceptance at this change: one ordinary Retry, two provider
absence reads for two retained sources, one fresh native execution,
useful confirmation and Done after reload. The old run remains failed.
Existing saved review and newer local file bytes remain unchanged. The
deterministic model does not perform external publication; this is local
protocol acceptance, not live provider recovery.
- `pnpm typecheck`, `pnpm build`, and `node scripts/check-maps.mjs`
pass.
- A complete local `pnpm test:run` pass was not obtained. The first run
hit a 15-second timeout in an unchanged status-card test. It was stopped
to apply review fixes before the remaining phases ran. The unchanged
status-card suite then passed 25/25 in isolation with the repository
short temporary-path convention.
- CI exposed a chat nonduplication assertion that compared unordered
database rows. The assertion now sorts both complete row sets by run ID.
Its focused test passes.
- All 58 current-head checks are terminal on
`1e2e881f6b6b1ae1fbfe1f2b2922657ce46e1470`: 55 success, two unaffected
Storybook skips, and one neutral advisory review. Required CI, build,
E2E and security checks pass. Greptile is 5/5 with no unresolved
threads. The optional Paperclip review is neutral advisory 5/5 with
nine-file source coverage and no findings; it did not independently run
tests.
- One unchanged native-runtime test timed out at five seconds on its
first CI attempt. Its suite passed 33/33 in isolation; a single retry of
that lane passed without code changes.

## Risks

- The companion Cloud endpoint must ship before this consumer. Older or
unavailable Cloud versions return no proof and retain the current hold.
- A valid absence receipt permits a fresh workspace attempt. It cannot
restore missing remote files, establish task success or authorize
repeating an uncertain external effect.
- This PR has not been deployed. The original task remains unrecovered
pending rollout and reconciliation of its existing Retry.
- No schema, credential, provider-resource or fleet changes are
included.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, source inspection, code
execution and browser tools. The exact backend variant and
context-window size are not exposed by this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (focused scope above; full
local run limitation recorded)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-22 22:31:52 -05:00
DottaandPaperclip eaacadeba3 fix(recovery): let preservation owners reopen retained sources (#17053)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Recovery must save retained workspace files before ordinary
execution can resume.
> - The environment runtime blocks access to retained sources to prevent
unsafe reuse or deletion.
> - The recovery worker also used that ordinary resume path.
> - The guard returned no lease, so recovery reported an invalid source
binding before it could save files.
> - This pull request gives the physical preservation owner a separate,
source-bound resume path.
> - Ordinary execution and destructive cleanup keep their existing
guards.

## Linked Issues or Issue Description

Refs #16965.

**What happened?**

A retained workspace with a workspaceRestoreRecovery marker could not
recover through the real environment runtime. The runtime returned null
before calling the provider. Recovery classified this as
source_binding_invalid. Earlier recovery fixtures replaced the runtime
facade, so they did not exercise this guard.

**Expected behavior**

The verified preservation owner can reopen its exact stopped allocation
to save files. Other callers remain blocked. Recovery preserves newer
local files and does not start another agent or repeat a completed
external action.

**Steps to reproduce**

1. Seed a terminal source run with a retained sandbox, a recovery
action, and the workspaceRestoreRecovery lease marker.
2. Give the source a verified stop receipt.
3. Call recoverRetainedWorkspace through environmentRuntimeService.
4. Before this change, the recovery action returns to blocked with
source_binding_invalid. No files are copied.

**Paperclip version or commit**

Reproduced on master at 65f59e339a. The
same failure was observed in a served build containing #16965.

## What Changed

- Add an internal preservation resume method. It validates the exact
company, task, run, action, lease, environment, current physical owner,
and stop-only cleanup claim before provider access.
- Keep ordinary resume and destructive cleanup blocked for protected
sources.
- Run all retained-workspace fixtures through the assembled environment
runtime. Add a failing-before/passing-after protected-source regression
and four rejection cases.
- Use the same runtime path and protection marker in the opt-in Daytona
interruption probe. Allow a verified immutable snapshot UUID as its
input.
- Document the authority boundary and verification recipes.

## Verification

- Red: the protected-source regression failed with expected recovered,
received blocked. It recorded source_binding_invalid before provider
resume.
- Green: 235 tests passed across workspace recovery, durable task Retry,
and environment runtime. These include saved files, newer local files,
bounded retries, competing owners, and no extra run or wake.
- pnpm -r typecheck and pnpm build passed. The initial sandboxed
typecheck could not open a tsx IPC pipe; the same command passed with
local process access.
- node scripts/check-maps.mjs and git diff --check passed.
- Live Daytona qualification passed using the same immutable snapshot ID
as the affected deployment. Export was interrupted after a real
download; a second controller restored the files and verified one
original agent start, one simulated external action, one saved snapshot,
and a stopped retained source. The disposable sandbox was deleted after
verification.
- Greptile reviewed 804dfe24ef at 5/5 with
no actionable findings or unresolved review threads.
- Full current-commit CI passed:
https://github.com/paperclipai/paperclip/actions/runs/39540771747.
Required ci / verify and ci / e2e passed. Superagent, Socket, and Snyk
checks passed.
- Full local pnpm test:run did not pass: 22,992 tests passed, one
managed-run identity test timed out at 60 seconds, and 100 tests skipped
across the first general-server phase. The command stopped before its
later phases. The unchanged agent-identity.test.ts suite then passed all
11 tests in isolation in 15.53 seconds. The original timeout remains
recorded; it is not reclassified as a full local pass. The same commit's
complete CI test matrix passed.
- The optional Paperclip Review completed with a neutral advisory 5/5 on
the same head and no actionable findings. It inspected all eight changed
files and connected code. It did not independently run tests or the
provider probe.
- Hosted ordinary Retry recorded one durable intent and no successor run
while file preservation remained blocked. The existing published review
stayed unchanged. Hosted recovery remains unverified until this
follow-up is deployed. A Retry acknowledgement alone does not prove
recovery.

## Risks

- Recovery can reopen a retained sandbox. The dedicated path requires
the current physical ownership claim and the exact source's stop-only
intent. It does not clear the ordinary protection marker.
- Provider failures retain the source and use the existing bounded
recovery budget. The provider cannot substitute another allocation.
- The live probe uses a disposable sandbox and simulated
agent/external-effect counters. It does not prove a live LLM or GitHub
review journey.
- No schema, public API, permission, budget, pause, or approval changes.

## Model Used

OpenAI GPT-6 through Codex, with repository inspection, code execution,
browser verification, and test tools. The exact served model variant and
context window are not exposed by this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (focused suites; full suite
still running)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-21 07:27:45 -05:00
DottaandPaperclip d33b40a5c3 fix: recover saved work and provide reliable task Retry (#16965)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The control plane must retain work when an execution or workspace
export fails.
> - Retained-workspace holds can leave saved input waiting with no
usable Retry action.
> - Users cannot verify process ownership or repair provider
allocations.
> - This pull request gives system recovery a durable budget and gives
each deliberate Retry a durable intent.
> - Safe recovery can preserve files, settle an accepted result, or
prepare an independent workspace.
> - One task state keeps infrastructure diagnostics out of the ordinary
conversation while preserving real permission and approval gates.

## Linked Issues or Issue Description

**What happened?**

A task with a retained workspace can show recovery, failed-run and
saved-message warnings at the same time. Historical runs without native
workspace descriptors cannot use the native restore path. Old recovery
metadata can prevent a new attempt even when an independent workspace
would be safe.

**Expected behavior**

Paperclip owns bounded infrastructure retries and file preservation. A
user can request another attempt with ordinary Retry. Thinking appears
immediately. The service preserves completed effects and current user
controls. It does not claim that starting fresh recovered the old files.

**Steps to reproduce**

1. Retain a failed run and its workspace after export or process-stop
failure.
2. Save a follow-up while the recovery hold is active.
3. Open the task and try to continue without infrastructure credentials
or operator certification.

Refs #16906, #16835, #16883. This extends their recovery and receipt
contracts. Related work: #16962 and #16963.

## What Changed

- Add task-level Retry with company-scoped authorization, all-status
idempotency and durable admission before physical work. Keep exact-run
Retry compatible.
- Add typed recovery phases, finite persisted backoff, historical
adoption and durable local failure diagnostics.
- Preserve native workspaces or verified historical snapshots. Admit an
independent fresh workspace only after execution containment. Keep
old-source preservation and cleanup obligations separate.
- Carry completed publication and API receipts forward. Fence uncertain
mutation retries across replacement runs. Preserve connected-chat
identity and authorization.
- Re-enter normal dispatch for saved input. Reuse accepted results
through the existing finalizer.
- Use one task interruption projection across the thread, composer and
task list. Show optimistic Thinking, ordinary Retry and genuine user
gates.
- Add PostgreSQL, filesystem, API, browser-fixture and controlled
provider coverage. Document the behavior and update the feature/access
maps.

## Verification

- All GitHub Actions CI jobs pass on
`6d8c6d27d50586d8548b31d03b47f6d73a0ee11d`: typecheck, all
server/chat/workspace and serialized suites, Runner checks, eight
browser shards, canary and build. The required merge gate passes.
- Greptile reviewed the final commit at 5/5, with no actionable findings
or unresolved review threads. Its two earlier P1 findings are fixed.
- Full recursive typecheck and build pass locally on the final commit.
All 16 task Retry cases pass against isolated real PostgreSQL. The
unrelated-success case and both stale Retry cases fail on the previous
code and pass after the fix.
- The corrected gateway fixture passes its uncertain-mutation and
independent-read case. Four real HTTP API cases pass for definite
403/422 rejection, completed-effect reuse, and uncertain-effect fences
across deliberate and automatic retries. These focused gateway/API
checks used a temporary database lifecycle harness on an existing
healthy local PostgreSQL process. The standard repository helper is
unchanged.
- Earlier qualification covers 291 task UI cases, saved-file copyback
and ancestry, current Agent Chat ownership races, nine chat Retry API
browser cases, and six assembled browser recovery fixtures. A real
API/PostgreSQL/native-runner journey showed Thinking in 25 ms despite a
delayed response, admitted one successor in a separate workspace,
retained original files, and survived reload.
- The controlled Daytona test interrupts export, joins the controller,
restarts it in a separate process, and verifies one agent start, one
external sentinel operation, retained file contents and no premature
cleanup. It does not establish abrupt-kill or live LLM/GitHub
acceptance.
- The full local `pnpm test:run` was attempted and was not green: an
OAuth row-lock timing assertion failed, and a later suite exhausted
macOS PostgreSQL semaphores. The ten OAuth timing cases passed on rerun
against a separate database. Later embedded database initialization
failures remain recorded; the final CI matrix passes.
- Security scanning passed on production-code commit
`63953edf069b111c6becc3144dffa6deb52f68b5`. Its only successor changes
one test-fixture identity line. The final scan is neutral/inconclusive
because its pinned merge-base checkout failed or timed out. Both rerun
API requests returned 404, so no fresh successful scan is claimed.
Socket and Snyk checks pass on the final head.
- The optional Paperclip Review remains queued behind an earlier run in
workspace finalization. Its prior completed assessment was incomplete
because a comparison tool was blocked by the platform filter. It is not
counted as a completed code review. GitHub does not require this check
for merge.
- Hosted acceptance remains pending deployment. Local and CI results do
not establish recovery of a hosted task or its retained files.

## Risks

- Recovery changes span admission, physical ownership and dispatch.
Incorrect evidence could permit overlapping work. Existing physical
locks and authoritative stop requirements remain in place.
- The migration adds concurrent indexes. It requires the existing
top-level migration runner and retries incomplete index creation safely.
- A fresh attempt can complete while old-source preservation remains
unresolved. The service retains that obligation and does not report
those files as recovered.
- Unknown tool mutations remain blocked until their authoritative
receipt owner resolves them. Generic upstream tools have no universal
result-query protocol.
- Recovery and diagnostics stay inside the stack. No Cloud incident
endpoint, notification worker or new outbound reporting channel is
included.

## Model Used

OpenAI Codex, GPT-6 family. The runtime does not expose a more specific
model version or context-window size. The model used reasoning, shell
execution, source editing and browser tools. No model subagents were
used for this implementation.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (focused suites; the full
local run limitation is documented above)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-20 19:58:53 -05:00