Commit Graph
3 Commits
Author SHA1 Message Date
DottaandPaperclip 84aa989bf9 Preserve native sessions through legacy workspace enrollment (#18943)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - A native session retains its workspace and provider session across
turns.
> - An ephemeral web server must restore those files without its
predecessor's disk.
> - Legacy enrollment changes the workspace authority and can change
both workspace addresses.
> - The retained provider backup checks those addresses as part of its
identity.
> - This pull request preserves the exact addresses in a verified
migration receipt.
> - The benefit is that the next turn can use the original session after
enrollment.

## Linked Issues or Issue Description

Refs #18932, merged as `612d9931a6`. This pull request builds on that
all-Runner durability change. Related: #18918 fixes concurrent workspace
preservation locks. This change covers identity preservation during
offline enrollment.

## What Changed

- The native workspace migration owner records the selected session
bindings in the successful workspace migration operation.
- Runtime selection retains the original logical address only for the
exact agent, task and session.
- Remote workspace preparation restores the original provider address.
It refuses conflicting targets, forged receipts and occupied unowned
roots.
- The environment owner includes the migration operation in the reusable
lease fingerprint. Warm policy stays warm and receives a fresh
allocation through normal lease cleanup.
- Exact migration evidence permits the existing verified provider backup
to restore into that fresh warm allocation.
- Preserve verified provider allocation files through a normalized,
lease-bound digest receipt. Unknown files and altered markers remain
refused; interrupted publication is retryable.
- Retain legacy managed-instruction session fingerprints using the exact
imported content hash and recorded paths, preserving all other
configuration and credential checks.
- Migrate existing local skill archive/manifest objects through upload
claims, readback verification and receipt compare-and-swap; retain
original objects and IDs.
- Add explicit two-turn browser qualification for every released native
Daytona profile, with warm and per-turn variants. The production
maintenance entry runs offline. The old home is deleted only after a
verified export and successor readiness. The oracle checks provider
identity, native session, addresses, reference bytes, question and final
document.

## Verification

- Full repository typecheck passed at `b12ecd823e`. The production
session fix passed 198 session/credential tests. Full repository build
passed at `94bb917947`. The final Rust-only provider fix passed 57 ACPX
provider tests and the exact Linux release build.
- Workspace and transport checks passed: 117 transport/workspace tests,
six final real PostgreSQL migration boundary cases, 178 environment
tests, 539 Daytona tests (eight skips), and the native warm-backup
admission control.
- Local skill migration passed eight PostgreSQL/S3 tests (one existing
opt-in skip). Failed readback retains the source. Successful readback
still works after deletion of the old home.
- Latest Product E2E typecheck and 38 focused harness tests pass.
Earlier catalog/admission/report checks passed. Twelve unrelated
support-test failures reproduce on unchanged base `1a39dea9aa`; these
are not reported as passing.
- Live Codex per-turn at `2d5bb9a6d1` retained the native and provider
sessions, both workspace paths and exact reference bytes after deletion
of the old home. It completed the task and delivered its saved document
and original reference. Its original failure was an output-count grading
defect. The corrected grader passes all 21 checks against retained,
hashed evidence; the original result remains unchanged.
- Live OpenCode per-turn and Codex warm at harness `94bb917947` pass all
21 product checks. Their launcher cleanup failed because a read-only
diagnostic observer kept its disposable database open. The observer now
closes before ordinary database cleanup; a no-provider reproduction
confirms the fix. All owned sandboxes were independently verified
absent, child keys revoked and buckets removed. Original failed campaign
dispositions remain unchanged.
- Claude at `94bb917947` kept its sessions and reference bytes but
omitted the final clickable document link. That user-delivery failure
remains recorded. The fixture now explicitly asks for the link and
records migration evidence before checking navigation. The fresh Claude
attempt at `b12ecd823e` passed both strict migration checks: exact
provider/native session, both paths, original file bytes and
independently downloaded attachment. Its full browser workflow failed
again: it requested unnecessary confirmation, used the answer sentence
as the reference, and omitted the clickable document link. This is
retained as a failed user-workflow result, not a full pass.
- Earlier live failures exposed and reproduced the skill-object,
fresh-allocation marker and managed-instruction fingerprint bugs. The
fixes retain original identity checks. No provider session reset or
accounting bypass was used.
- The real Grok continuation at `b12ecd823e` failed before provider
execution because its immutable-instruction attach policy rejected
relocated registered file copies. A no-provider regression reproduced
the exact rejection. `b05ba3728a` selects the existing strict
authenticated asset-path policy for Grok; content digests, unknown
context fields, session/workspace identity and same-run mutations remain
enforced. The regression and all 57 ACPX provider controls pass. The
Grok browser migration now passes all 21 checks with the b05 Linux
Runner, including provider identity, exact reference bytes and final
user output; the launcher also reports successful cleanup.
- `24fabf1d81` applies the same policy to released ACPX Codex after a
second no-provider reproduction. All 58 ACPX provider controls and the
final Linux release build pass. Its live browser migration passes all 21
checks on the final binary, including exact original session/file
preservation and completed user output.
- The full native-session executor suite passes all 1,121 tests after
updating its existing lease database fixture for the migration-receipt
join. The prior CI failure was this unsupported test-double method; the
production guard still rejects ordinary unproven recovery. CI and
Greptile 5/5 passed at `ca1e0d5fe6`. The first CI attempt lost several
workers and had one rotated-recovery transport failure; its seven
descriptor cases passed locally, and the single unchanged retry passed.
The original failure remains recorded. Staging enrollment has not
started. Initial offline enrollment and subsequent rolling availability
are separate checks.

- The final review follow-up adds real runtime-selection PostgreSQL
cases for exact, different-session and absent receipts, plus
executor-level warm restoration that copies actual backup files and
verifies provider identity and the new lease stamp. Both complete
affected suites pass (1,172 tests), and full repository typecheck
passes. Removing either production migration connection independently
makes its new positive case fail; both source files were restored
byte-for-byte after these negative controls.

- Final commit `0971d373a3` passes all 63 checks (57 successes, five
expected skips, one neutral); Greptile is 5/5 and every review thread is
resolved.

## Risks

- An identity conflict blocks enrollment and keeps the original files.
It does not silently reset the session.
- A new migration generation retires a reusable lease through its
existing owner. The original sandbox must finish normal cleanup before
reuse proceeds.
- Initial enrollment has an offline step. The separate rolling suites
prove uninterrupted deploy behavior after enrollment.
- This adds no schema migration and changes no provider credentials or
accounting gate.

## Model Used

OpenAI Codex, GPT-6 family. The exact serving model build and context
window are not exposed in this session. The agent used reasoning,
terminal tools, code edits, PostgreSQL/filesystem tests and isolated
live browser qualification.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-04 04:43:56 -06:00
DottaandPaperclip 24a15c5afa test(evals): verify durable waiting across continuation checkpoints (#15564)
## Thinking Path

> - Paperclip manages agents and durable tasks across provider runs.
> - Human questions must preserve task ownership and stop work until a
real answer arrives.
> - The continuation eval checks the final work, but its lifecycle
oracle misses several broken waiting states.
> - A correct final answer can hide a stale execution lock or a lost
intermediate answer receipt.
> - This pull request checks each wait and retains every question and
run identity.
> - A source audit records which waiting operations belong to native
runtimes and which still require legacy API calls.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The existing Product E2E continuation oracle for human question and
approval journeys.

**Current behavior**

A pending interaction can pass the waiting check even when its task has
the wrong status, a stale lock, or a retry. Only the first pending
question and first run receipts are checked at the final checkpoint.

**Proposed behavior**

Require a healthy wait on the same task and assignee. Preserve all
intermediate run receipts and every pending question's answered
identity. Allow a paused native provider question only when its pending
runtime request identifies the running native run and execution lock.

Related: #15548 and #15554 cover earlier bookkeeping slices. #15544
changes production continuation summaries; this PR changes the eval
oracle and does not overlap that fix.

## What Changed

- Check waiting state, execution ownership, and all answer/run
identities in the existing lifecycle oracle.
- Add negative calibrations for broken records and positive coverage for
both semantic waits and paused provider questions.
- Retain task activity at each checkpoint for inspection of successful
persisted mutations.
- Verify 1,000-cent company and agent budget limits before continuation
work. Disable automatic cell rerolls.
- Document runtime ownership, existing coverage, and the remaining
instruction decision. Keep production instructions unchanged.

## Verification

- `pnpm test:e2e:runner:unit`: 1,867 Vitest tests pass, one skips; 128
Node tests pass.
- `pnpm test:e2e:runner:typecheck`: passes.
- `pnpm -r typecheck`: passes.
- `pnpm build`: passes.
- Negative calibration: 27 added cases fail against the prior oracle and
pass with these checks.
- Four existing local continuation cells are selected for a separate
bounded live canary. Live results are pending; no behavioral pass is
claimed here.
- The full local `pnpm test:run` suite was not repeated because embedded
PostgreSQL was unavailable in the preceding workspace verification.
Required Linux PR CI must pass before readiness.

## Risks

The stronger oracle can expose existing product or fixture defects. A
paused provider question and a terminal semantic wait have distinct
valid states. The four-cell canary does not qualify approval/review,
dependency unblock, crash races, remote execution, or general task
quality. Task activity records successful persisted writes, not failed
API attempts; repeated progress comments are not automatically defects.
Historical eval grades remain unchanged. No production scheduling,
prompt, tool, schema or migration changes.

## Model Used

OpenAI Codex, GPT-6-based, with tool use and code execution. The exact
serving model ID and context-window size are not exposed in this
session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-08 08:45:36 -05:00
DottaandPaperclip 43acbcc398 fix(runner): preserve sessions and complete question and approval continuations (#13655)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The native runner connects task state to provider sessions.
> - Follow-up turns must retain provider memory and carry new user
direction.
> - Lost session IDs caused repeated context and extra input tokens.
> - Native question answers and approval races could leave valid work
blocked.
> - This pull request repairs those paths and adds regression coverage.
> - Agents can continue accepted work without repeating the conversation
or losing the user's answer.

## Linked Issues or Issue Description

Refs #13574. That merged PR shortened continuation prompts and moved
question instructions into tool documentation. This change preserves
sessions and fixes failures exposed by broader testing. Related runtime
work: #13408 and #13410.

**What happened?**

Native follow-up turns could lose the provider session ID. Completion
guidance could replace the original task with its latest comment. Claude
native questions could remain pending after the user answered. Approval
during a running tool call could suspend the run before the tool
response arrived. Onboarding and chat handoff instructions also caused
repeated planning or missing plan documents.

**Expected behavior**

Reuse a valid provider session. Send only new events when that session
already has the history. Preserve the task requirements and apply later
user direction. Store the question answer and deliver it to the waiting
run. Finish governed tool responses before suspending. Execute the
accepted plan without asking for the same approval again.

**Steps to reproduce**

Run the continuation, local-session-integrity, first-task, and
agent-chat suites with native Codex and Claude. Include
provider-question-bridge, accept-while-running, and plan-handoff.

**Paperclip version or commit**

This branch is based on master d54b75011. The active full catalog run
tests 4e75881db. Later review fixes have separate regression coverage.

**Deployment mode**

Isolated local instances and Daytona sandboxes in the existing Runner
full-stack E2E harness.

## What Changed

- Retain provider session identity across turns and late usage
snapshots. Send new continuation events on session reuse, with full
context available for a fresh session.
- Preserve task requirements and later direction in completion guidance.
Return the current contract revision after a stale completion
submission.
- Bridge native Claude questions to saved Paperclip cards. Submit
answers through the saved card and resume the same run.
- Delay governed suspension until tool results settle. Add a
deterministic test barrier for approval during an active run.
- Clarify free-text question examples, explicit onboarding plans, and
execution of accepted chat plans.
- Fix continuation readiness, verified output evidence, and declared
screenshot collection.
- Qualify the legacy Claude test CLI at 2.1.277. The old 2.1.19 CLI did
not discover mounted skills. Update the existing workflow pin and
isolated launcher together.
- Refresh the Daytona image lockfile integrity pin after reviewing
master patch updates.
- Carry continuation mode as runtime metadata instead of inferring it
from user-visible text. Install the test Claude CLI without lifecycle
scripts.

## Verification

- Targeted paid verification: 20/20 cases passed across
local-environment campaigns before the rebase.
[Report](https://pages.paperclip.ing/runner-e2e-seven-fixes-35397904249/).
- Harness checks: 379 unit tests passed; harness typecheck passed.
- Latest-head PR checks: 55 passed, two intentionally skipped. Greptile
is 5/5; the security scan passes.
- Review regressions: 350 executor tests and 204 session/driver tests
passed. A script-free Claude install was verified with the actual CLI.
- Full catalog, including the explicit-only everyday suite: [run
35417932353](https://github.com/paperclipai/paperclip/actions/runs/35417932353).
Completed: **164/205 passed; 41 failed**. [Full dashboard and failure
investigation](https://pages.paperclip.ing/runner-e2e-full-catalog-35417932353/).
Includes 204 case artifacts and one pre-case GitHub authorization
timeout; missing evidence is not scored as a pass. The full run tested
`4e75881db`; Final-head metadata/CLI smoke cases both passed. In the
separate [six infrastructure
retries](https://github.com/paperclipai/paperclip/actions/runs/35419769343),
the GitHub timeout case passed and all five Docker preflight failures
repeated. [Follow-up
dashboard](https://pages.paperclip.ing/runner-e2e-full-catalog-35417932353/follow-up/).
- Full local typecheck and build passed on the rebased branch. The full
local unit run completed with 657 passing files, two test timeouts and
one suite setup timeout. All three affected files passed when rerun in
isolation (84 tests). The first full local run was not clean.
- Focused regression coverage includes the live question bridge,
same-run response delivery, UI routing, stale revisions, approval
overlap, and session reuse.

## Full-catalog follow-ups

- Test infrastructure: 14 Claude everyday cells probe an absent host
CLI; six cells failed pre-task GitHub/Docker qualification (GitHub
passes on retry; all five Docker cases repeat; the workflow preflight
allowlist omits their case IDs); five ACPX Codex cells cannot create
sandbox namespaces.
- Runtime: four OpenCode completion-criteria mismatches masked by
shutdown errors, one service-approval suspension failure; three Daytona
recovery failures encounter existing skill files; one duplicate
completion wake.
- Confirmed test defects: question pagination and a noncanonical plan
document key.
- Product/behavior: mismatched visible/required question sets, an
attachment instead of the requested task document, one lone-option
onboarding question, early completion instead of review, and a Codex
Mini completion-schema failure.
- The report job itself fails on trusted master’s stale patch/lock
configuration. The linked report is rebuilt with the shared renderer
from original cell results and public fixture screenshots; it excludes
private snapshots, logs and traces.

These are investigated follow-ups, not silently regraded passes.
First-task passed 51/52. The PR checks are green independently of the
broader catalog’s behavioral/infrastructure failures.

## Risks

- Session reuse depends on a valid provider identity and context
coverage. Fresh-session fallback and reset tests cover this boundary.
- Native question delivery spans saved interaction state and a live
provider run. Tests cover duplicate events, closed runs, and same-run
answers.
- Provider behavior varies. The full paid catalog may expose failures
beyond these targeted fixes; those results will be reported without
relaxing valid approval or output checks.
- The legacy Claude version update is limited to test infrastructure. No
database migration is included.

## Model Used

OpenAI Codex, GPT-6 family, with repository inspection, code execution,
and browser/E2E tools. The exact runtime model identifier and
context-window size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (targeted checks and all
three timeout-file reruns pass; full-run timeout caveat above)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-19 07:42:57 -05:00