mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-08 12:33:05 +01:00
fix(e2e): configure supported rolling agents with warm policy (#18994)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Rolling tests must exercise the lifecycle they claim to cover.
> - The warm lane selected a warm environment.
> - Its profiles also set an explicit per-turn agent policy, which took
precedence.
> - The test then waited for a live workspace that per-turn cleanup had
correctly retired.
> - This pull request sets warm agent policy for providers admitted by
the production warm capability.
> - The matrix now checks effective lifecycle and preserves per-turn
coverage for every qualified native profile.
## Linked Issues or Issue Description
Builds on merged #18982 for the source controller dependency link. This
PR now targets master.
**What happened?**
The warm rolling fixture built a Codex agent with `lifecycleMode:
per_turn` and an environment with `runnerLifecycleMode: warm`. The
explicit agent policy won. The initial task completed, its disposable
workspace retired, and the fixture waited for a live file before it
reached deployment.
**Expected behavior**
Warm rolling, legacy migration, remote-file and controller-outage cells
must create actual warm agents. Only providers supported by the
production managed warm-session capability belong in that lane. All
qualified native profiles retain per-turn coverage.
**Steps to reproduce**
Build the old warm Codex cell's agent and environment and compare their
lifecycle policies. The new effective-policy regression fails on the old
catalog and passes on this branch. No duplicate exact-scope warm-fixture
PR was found.
## What Changed
- Share the production managed warm capability filter and explicit warm
agent policy across rolling, legacy migration, remote-file and
controller-outage suites.
- Check effective agent and environment policies and update matrix
coverage/counts.
- Read completed warm turns from the exact saved snapshot while
independently requiring the retained, bound workspace location.
- Keep both rolling lanes separate from the outage simulation and
exercise their assembled supervisor requests.
- Version the corrected definitions and keep every qualified native
profile in the per-turn lanes.
## Verification
- Baseline regression: failed against the unchanged catalog; retained
before the fix.
- Fixed catalog/controller selection: 39 checks passed.
- The first real warm-policy attempt completed and saved its workspace
but timed out because the file oracle demanded live content; retained
trace responses prove the API returned the correct saved snapshot with a
retained lease.
- A new regression fails on the previous reader and passes after
separating saved file source from retained versus released location.
Focused catalog, reader and rolling checks: 81 passed.
- The next warm attempt reached replacement and exposed a second fixture
defect: warm also selected a SIGKILL outage, which the supervisor
rejected alongside rolling mode. The failure remains retained; its four
owned sandboxes, temporary key and local bucket were deleted.
- An outage-selection regression failed before the fix; 57
outage/supervisor checks pass afterward, including assembled requests
for both rolling lanes. The fresh warm Codex run passed all 56 checks in
8.5 minutes: original run/session/runner/lease retained, authenticated
takeover, both predecessor homes removed, later native continuation and
saved file readback. Two replacement receipts recorded 742 ordinary
reads and 75 writes, all 75 read back, with zero HTTP probe failures. A
transient `native_workspace_sync_out_failed` phase recovered before
successful finalization and remains retained. This source-lane result
does not claim released-image or hosted Railway qualification.
- The invalid earlier attempt was interrupted before deployment. Its
four archived disposable sandboxes were deleted and verified absent; its
temporary key and local bucket were removed.
- Broad local typecheck reached stale shared database declarations in
this checkout's linked build dependencies. The fixture's own type error
was corrected; clean-head CI remains required. No full build is claimed
locally.
- An incidental report-catalog test selected by a substring filter
failed to generate its normalized output. That failure is retained
separately; the focused catalog/controller checks pass.
- Review follow-up: fixed the same policy conflict in the base
file-storage, legacy migration and controller-outage suites. The roster
now has 822 valid cells; unsupported warm combinations remain covered
per-turn. Updated the explicit supervisor-request control to use a
supported warm provider. Focused catalog, file-reader, migration,
rolling and supervisor checks pass (133 checks).
- Earlier full CI passed at dd5aa71 using the built-in GitHub-hosted
fallback after self-hosted runner shutdowns. That result does not
qualify this new head; fresh CI and review are required. The final head
now has a 5/5 review and passing full CI (run 41901807639, attempt 2).
Attempt 1 hit two unrelated port collisions: the unchanged managed Codex
credential lease test and CLI company-server startup on a busy requested
port. Both passed on one failed-job retry. Those failures remain
retained.
## Risks
This changes test setup and coverage. It does not change agent runtime
defaults or provider capabilities. Warm cells use the production
capability boundary, while every qualified native profile keeps its
per-turn cell.
## Model Used
OpenAI Codex, exact model ID `gpt-6.1-sol`, with reasoning, tool use and
code execution. The host does not expose context window size.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either linked existing issues or described the issue in-PR
following the relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change and contains no internal
ticket id or instance-derived details
- [x] I have run the relevant tests locally; failures and limits are
retained above
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
4ea53f9eb5
commit
513c03cb3e
9 files changed
+151
-47
No files matched your search
@@ -57,7 +57,7 @@ describe("rolling controller qualification", () => {
|
||||
const [outage] = selectRunnerExecutions(parseRunnerSelectors(["--suite", "remote-workspace-storage"]));
|
||||
expect(controllerOutageForExecution(outage!, {})).toEqual({ signal: "SIGKILL", holdMs: 300_000 });
|
||||
});
|
||||
it("requires both lifecycle modes for every production-qualified native sandbox profile", () => {
|
||||
it("covers every qualified native profile per turn and each supported warm profile", () => {
|
||||
const executions = ["rolling-web-deployment", "rolling-web-deployment-warm"]
|
||||
.flatMap(suite => selectRunnerExecutions(parseRunnerSelectors(["--suite", suite])));
|
||||
const native = [...runnerProfiles, ...extendedHarnessProfiles, kimiRunnerProfile]
|
||||
@@ -74,13 +74,19 @@ describe("rolling controller qualification", () => {
|
||||
&& (!profile.qualificationCandidate || QUALIFIED_ACPX_PROFILES[profile.qualificationCandidate].qualificationStatus !== "pending"))
|
||||
.map(kind));
|
||||
expect(new Set(native.map(kind))).toEqual(registeredKinds);
|
||||
const warmIds = new Set(["runner-codex", "runner-acpx-copilot"]);
|
||||
expect(new Set(runnerSuites.find(suite => suite.id === "rolling-web-deployment-warm")!.profiles.map(profile => profile.id))).toEqual(warmIds);
|
||||
for (const profile of native) {
|
||||
const cells = executions.filter(execution => execution.profile.id === profile.id);
|
||||
expect(cells).toHaveLength(2);
|
||||
expect(cells.filter(cell => cell.suite.id === "rolling-web-deployment")).toHaveLength(1);
|
||||
expect(cells).toHaveLength(warmIds.has(profile.id) ? 2 : 1);
|
||||
expect(cells.every(cell => !cell.profile.qualificationCandidate)).toBe(true);
|
||||
expect(new Set(cells.map(cell => cell.environment.configurationKey ?? "per_turn")))
|
||||
.toEqual(new Set(["per_turn", "warm-controller-outage-v1"]));
|
||||
.toEqual(new Set(warmIds.has(profile.id) ? ["per_turn", "warm-controller-outage-v1"] : ["per_turn"]));
|
||||
}
|
||||
expect(executions.every(execution => execution.suite.definitionMetadata?.retainedControllerContinuation === true)).toBe(true);
|
||||
for (const execution of executions) {
|
||||
expect(controllerOutageForExecution(execution, {})).toBeUndefined();
|
||||
}
|
||||
});
|
||||
});
|
||||
Reference in new issue
Block a user