mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-08 12:33:05 +01:00
dd889e6a0b55e4e9f33a49cc0953e9710ea3add7
3
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
513c03cb3e |
fix(e2e): configure supported rolling agents with warm policy (#18994)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Rolling tests must exercise the lifecycle they claim to cover.
> - The warm lane selected a warm environment.
> - Its profiles also set an explicit per-turn agent policy, which took
precedence.
> - The test then waited for a live workspace that per-turn cleanup had
correctly retired.
> - This pull request sets warm agent policy for providers admitted by
the production warm capability.
> - The matrix now checks effective lifecycle and preserves per-turn
coverage for every qualified native profile.
## Linked Issues or Issue Description
Builds on merged #18982 for the source controller dependency link. This
PR now targets master.
**What happened?**
The warm rolling fixture built a Codex agent with `lifecycleMode:
per_turn` and an environment with `runnerLifecycleMode: warm`. The
explicit agent policy won. The initial task completed, its disposable
workspace retired, and the fixture waited for a live file before it
reached deployment.
**Expected behavior**
Warm rolling, legacy migration, remote-file and controller-outage cells
must create actual warm agents. Only providers supported by the
production managed warm-session capability belong in that lane. All
qualified native profiles retain per-turn coverage.
**Steps to reproduce**
Build the old warm Codex cell's agent and environment and compare their
lifecycle policies. The new effective-policy regression fails on the old
catalog and passes on this branch. No duplicate exact-scope warm-fixture
PR was found.
## What Changed
- Share the production managed warm capability filter and explicit warm
agent policy across rolling, legacy migration, remote-file and
controller-outage suites.
- Check effective agent and environment policies and update matrix
coverage/counts.
- Read completed warm turns from the exact saved snapshot while
independently requiring the retained, bound workspace location.
- Keep both rolling lanes separate from the outage simulation and
exercise their assembled supervisor requests.
- Version the corrected definitions and keep every qualified native
profile in the per-turn lanes.
## Verification
- Baseline regression: failed against the unchanged catalog; retained
before the fix.
- Fixed catalog/controller selection: 39 checks passed.
- The first real warm-policy attempt completed and saved its workspace
but timed out because the file oracle demanded live content; retained
trace responses prove the API returned the correct saved snapshot with a
retained lease.
- A new regression fails on the previous reader and passes after
separating saved file source from retained versus released location.
Focused catalog, reader and rolling checks: 81 passed.
- The next warm attempt reached replacement and exposed a second fixture
defect: warm also selected a SIGKILL outage, which the supervisor
rejected alongside rolling mode. The failure remains retained; its four
owned sandboxes, temporary key and local bucket were deleted.
- An outage-selection regression failed before the fix; 57
outage/supervisor checks pass afterward, including assembled requests
for both rolling lanes. The fresh warm Codex run passed all 56 checks in
8.5 minutes: original run/session/runner/lease retained, authenticated
takeover, both predecessor homes removed, later native continuation and
saved file readback. Two replacement receipts recorded 742 ordinary
reads and 75 writes, all 75 read back, with zero HTTP probe failures. A
transient `native_workspace_sync_out_failed` phase recovered before
successful finalization and remains retained. This source-lane result
does not claim released-image or hosted Railway qualification.
- The invalid earlier attempt was interrupted before deployment. Its
four archived disposable sandboxes were deleted and verified absent; its
temporary key and local bucket were removed.
- Broad local typecheck reached stale shared database declarations in
this checkout's linked build dependencies. The fixture's own type error
was corrected; clean-head CI remains required. No full build is claimed
locally.
- An incidental report-catalog test selected by a substring filter
failed to generate its normalized output. That failure is retained
separately; the focused catalog/controller checks pass.
- Review follow-up: fixed the same policy conflict in the base
file-storage, legacy migration and controller-outage suites. The roster
now has 822 valid cells; unsupported warm combinations remain covered
per-turn. Updated the explicit supervisor-request control to use a
supported warm provider. Focused catalog, file-reader, migration,
rolling and supervisor checks pass (133 checks).
- Earlier full CI passed at
|
||
|
DottaandPaperclip
|
612d9931a6 |
Support per-turn Runner sessions on ephemeral web servers (#18932)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The web server controls native Runner sessions in remote sandboxes. > - An ephemeral web server must recover without its previous local filesystem. > - The existing admission rule accepts warm sessions. Managed Claude, Grok, and OpenCode sessions resolve to per-turn execution. > - Per-turn execution also needs durable provider-session bytes after the sandbox is retired. > - This pull request adds encrypted backup publication and removes the warm-only admission rule. > - Both lifecycle modes can use the existing native ownership and checkpoint contracts during a web deployment. ## Linked Issues or Issue Description Refs: #18518, #18529, #18826, #18835, #18841, #18864, #18887. **What existing behavior does this improve?** Native Runner execution on web servers with `PAPERCLIP_EPHEMERAL_SERVER=true`. **Current behavior** The admission rule requires a warm Runner lifecycle. Managed provider credentials can resolve an agent to per-turn execution, so enabling the ephemeral deployment profile blocks those turns. A database controller checkpoint does not preserve provider files after the old web home and per-turn sandbox are removed. **Proposed behavior** Admit native execution in either resolved lifecycle. Keep provider and credential choices unchanged. Publish encrypted provider state to shared storage before sandbox cleanup. Restore it from a fresh web home for the next turn. Use the existing controller lease and generation checks for active takeover. **Reason and benefit** Operators can remove the web server volume without excluding per-turn providers or forcing managed accounts into warm sessions. New native Daytona profiles automatically enter both rolling-deployment qualification lanes. **Breaking changes** The additive migration creates a backup publication ledger. Existing volume-backed local-disk installations retain their local backup path. S3 installations publish encrypted backups and require a stable secrets master key. Existing-volume enrollment must import surviving session data before detaching the volume. ## What Changed - Admit native Runner execution independently of its warm or per-turn lifecycle. Keep host-owned legacy execution excluded from ephemeral deployments. - Store private provider state as encrypted objects. Journal uploads before I/O, read them back, and fence publication against the current native owner. Keep the previous published backup when storage or ownership checks fail. - Restore exact provider bytes into a private cache. Require an exact durable backup before sandbox cleanup. Extend the existing bounded retention pass to retire stored objects without adding idle polling. - Import selected legacy controller and provider state in the existing offline maintenance process. Preserve queued work, task pointers, pending recovery, and original files. Prefer canonical database checkpoints. - Preserve already-failed legacy controllers with missing original artifact proof as encrypted archives. These archives do not become execution authority. Successful retained sessions still require proof. Report archival and executable imports separately. - Verify unmanaged Grok API keys through the existing xAI account validator. The live test exposed a readiness gap that kept remote Grok agents paused after platform checks. Preserve failed credentials and the separate runtime warning. - Ingest Grok whole-prompt usage at the owning ACP boundary. Bind the receipt to the exact outbound request/session; reject replay, conflicts, and incomplete authority. Separate turn cost from cumulative session cost in the sidecar schema and Rust normalization. Preserve managed billing identity and identify direct xAI API credentials. - Preserve published workspace directory names across cold allocations, including destroyed-allocation cleanup. Keep unpublished epoch/lease directories isolated and refuse to overwrite an occupied preparation. This fixes the cold-session identity regression exposed by #18922. - Increase bounded checkpoint-header inspection to 1 MiB. Retain the aggregate inspection limit. - Register all nine native Daytona profiles in per-turn and warm rolling-deployment suites. Delete the entire departing server home, exercise ordinary reads and writes, require active ownership transfer, then check a later turn on another fresh server. Permit only exact notification replay after reconnect. Ownership stays with the existing modules: Runner validates its protocol state; Core validates run scope and controller ownership; storage owns object I/O; the maintenance process owns offline conversion; the E2E wrapper owns process replacement and probes. No provider protocol is added to the shared admission rule. Grok receipt parsing stays in its provider module. The shared sidecar gains an explicit cost basis because its existing price field means session-cumulative spend, while Grok reports prompt totals. Existing consumers retain cumulative behavior when the new field is absent. ## Verification - Focused native admission, archive validation, publication fencing, cleanup, readiness, legacy import, and E2E observer tests pass. Real PostgreSQL cases verify that queued intent and historical recovery rows stay unchanged. - Full typecheck and build pass on the rebased branch. The review follow-up passes server typecheck, 29 focused PostgreSQL/archive/readiness tests, and module-boundary checks. Revision fb8d39a7a1 reached 55 successful CI checks and Greptile 5/5 with no unresolved threads. The Grok follow-up passes Runner/server typechecks, 238 ACP runtime/projection/usage tests, 16 sidecar-schema tests, the Rust cost regression, and Core billing regression. The |
||
|
DottaandPaperclip
|
16787e4167 |
Qualify rolling web deployments and preserve Stop through handoff (#18526)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The web server owns request handling and remote execution coordination. > - Deployment must start a ready replacement while the old server still serves. > - An outage test does not prove that two live servers can overlap safely. > - This pull request adds a separate Product E2E cell for that boundary. > - The existing run, file, recovery, and retained-session checks still apply. ## Linked Issues or Issue Description Refs #18518. The runtime prerequisite is merged; this PR qualifies its ephemeral server and deployment contracts. ## What Changed - Add the explicit `rolling-web-deployment` Codex Daytona workflow. - Start independent A/B/C homes and compiled builds behind a test ingress. - Wait for readiness before routing, then drain and delete the old home. - Observe active authority before handoff and use the existing read-only recovery oracle. - Require ordinary reads and saves to succeed throughout replacement. Read back every accepted save exactly once. - Preserve the five-minute outage suites and their existing validators. - Add a model-free supervisor smoke and an unready replacement control. - Require graceful exit before deleting an old home; forced cleanup cannot satisfy the oracle. - Preserve a durable Stop through lease release/expiry until the recovered execution owner acknowledges it. Permit only fenced reattachment of the original execution for Stop cleanup; submit no new turn. - Exclude events saved by A during shutdown from evidence of B making progress. ## Verification - E2E TypeScript check passed. - Affected harness checks: 143 passed. Catalog, fixture and report checks: 120 passed. - Actual supervisor smoke passed two replacements: 436 reads, 45 saves, zero request failures, and confirmed owned resource cleanup. - B became ready in 18.7 seconds; C became ready in 109.2 seconds. A prior C timeout remains failed evidence. - Fresh browser proof: 11 checks passed, 246 reads, 50 acknowledged saves, zero failures, 1,058ms reconnect, both old homes deleted and cleanup confirmed. - Stop lease-gap regression: three PostgreSQL delivery tests passed with a simulated provider edge. Current server TypeScript build/vendoring passed. - Full harness: 5,814 passed, 17 failed, four skipped. Two shutdown races passed the focused file; two report timeouts passed an isolated 42-test rerun. Onboarding/workflow-security/Pi/date-expired fixtures are unchanged on inspected master; the host embedded PostgreSQL fixture remains a setup limitation. This is not a green full local suite. - CI at `172509b8be` passed all gates. Greptile is 5/5 with every review thread resolved. The earlier access-service race failure passed its isolated regression and the subsequent full CI. - Final review corrections: six focused executor cases, three PostgreSQL command-delivery cases, 19 harness cases and server/E2E typechecks passed. - Three live attempts failed before agent creation: two harness setup gaps were fixed, then provider authentication failed. The saved OpenAI key returns 401; a refresh is pending. The Daytona probe reported cleanup uncertain, and no sandbox with that attempt label was found through the authenticated staging manager. No live model execution is claimed. - Live Runner overlap, released-container qualification and hosted deployment proof remain pending. Merging this opt-in qualification lane does not claim those results. ## Risks - The source lane shares workspace dependencies and does not establish container packaging correctness. - The live workflow needs an explicitly owned S3 bucket, PostgreSQL, pinned remote image and provider credentials. - Qualification cleanup stops all owned processes after failure. A failed replacement must not be reported as a pass. ## Model Used OpenAI Codex, GPT-6. Tool use, code execution and repository review. The runtime does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |