feat(evals): register explicit Daytona feedback persistence (#17986)

## Thinking Path

> - Paperclip manages AI agents and their work.
> - Product E2E evals check real server, browser, runner, and provider
behavior.
> - Feedback persistence already checks saved complaints and suggestions
with exact bodies and run attribution.
> - Its catalog registers only local cells, so a Daytona campaign cannot
select the same workflow.
> - This PR registers the existing workflow for the two native Daytona
profiles and tests their gateway bindings.
> - Remote cells stay explicit-only and unqualified until a bounded live
run passes.

## Linked Issues or Issue Description

Refs #17980 and #17975. No duplicate open PR or issue was found in the
Daytona feedback search.

**What existing behavior does this improve?**

The `feedback-persistence` Product E2E suite and its explicit execution
selectors.

**Current behavior**

At master `d54f776ca0`, the suite has only local Claude and Codex cells.
Selecting the matching Daytona ID fails before admission. The gateway
manifest already supports these native remote profiles.

**Proposed behavior**

Register four cells: local and Daytona for each native profile. Reuse
the existing saved-body and completed-tool oracle. Require an immutable
image and Daytona credentials. Keep one attempt, one expected work run,
five-minute local and inherited 15-minute remote case timeouts, and the
existing 60-minute provider creation TTL.

**Reason and benefit**

A bounded remote qualification can check both a simple task and real
feedback persistence. Local evidence remains separate from remote
acceptance.

**Breaking changes**

Explicit `--suite feedback-persistence` now lists four cells instead of
two. `--all` still excludes the suite. Daily and weekly manifests still
select their existing exact local IDs.

## What Changed

- Register the two native Daytona feedback cells and increment the suite
definition version.
- Address the remote cold-start review: inherit the base task's
15-minute Daytona deadline and retain the five-minute local limit. The
regression failed before the fix and now passes.
- Check immutable-image requirements, credentials, lifetime, and shared
gateway connections in existing contract tests.
- Fix the config type in the TTL catalog test added by #17980. Keep its
runtime lifetime assertions.
- Document remote selection and its unqualified status in the existing
eval guides.

## Verification

- Before the catalog change, the registration regression failed: one
failed and 16 passed.
- Focused catalog, feedback, operations, ledger, manifest, and summary
tests: 114 passed across six files.
- Read-only discovery lists exactly the two Daytona feedback IDs.
- The Node test phase passes separately: 137 tests. The retained Vitest
database failure prevents the unit launcher from reaching that phase.
- Full Product support suite: 4,523 passed, three skipped, and one
embedded PostgreSQL bootstrap failure. The same database failure was
reproduced on a clean baseline before this PR.
- Fresh Linux Product typecheck passed at the review-fix commit:
`GOMEMLIMIT=2GiB GOGC=50 GOMAXPROCS=2 node
node_modules/typescript/bin/tsc -p tests/runner-e2e/tsconfig.json`. The
default run and the ineffective JavaScript-heap attempt exited 137 and
remain retained. The compiler launches a native child process; its
memory controls let the unchanged semantic check complete. Host
typecheck still has 15 stale installed DB/Runner export errors; the two
TTL test typing errors are fixed. All 58 latest-head checks are
acceptable (success, neutral, or skipped). Both fresh reviews are 5/5,
with no unresolved review threads.
- The updated credential-free Linux controller discovers both planned
remote IDs and launches Chromium 151.0.7922.34. Runtime build inputs are
unchanged from the prior controller. The Daytona image content ID is
still
`b55f03dc9c633db300b70206a691ff45a8e779e56a3c546f9358e3282d3597c9`.
- No model requests, sandbox allocation, or paid schedule changes.

## Risks

- Selecting the whole explicit suite now includes remote cells and
requires Daytona configuration. Use exact IDs for a bounded paid
campaign.
- Registration and contract tests do not prove live remote behavior or
token accounting. Remote qualification remains pending.
- The shared dependency checkout limits the local host typecheck. The
fresh Linux semantic check passed. Earlier compiler OOM failures and the
database bootstrap failure remain retained.

## Model Used

- OpenAI GPT-6 through Codex, with tool use and code execution. The
exact serving model ID and context window are not exposed in this
session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>

doc/evals.md

@@ -539,5 +539,7 @@ API and download oracles; no live qualification is implied by fixture registrati
 
 The explicit-only [feedback persistence case](../tests/runner-e2e/README.md#feedback-persistence-explicit-only)
 checks real native complaint/suggestion submissions and retained bodies with
-run-attributed tool receipts. It is manual, uses the normal Product E2E report,
+run-attributed tool receipts on local and Daytona execution. Its four native
+Claude/Codex cells are explicit-only. Remote registrations remain unqualified
+until retained live attempts prove them. It uses the normal Product E2E report
 and keeps synthetic feedback distinct from actionable product findings.

tests/runner-e2e/FIXTURES.md

@@ -1021,14 +1021,18 @@ The oracle rejects a missing or pending terminal intent.
 
 ## Feedback persistence (explicit only)
 
-`--suite feedback-persistence` selects two local native Claude/Codex cells.
+`--suite feedback-persistence` selects four native Claude/Codex cells across local and Daytona execution.
 Each creates one standard task through the browser, asks for two distinct
 synthetic feedback items, and requires the real `submit_complaint` and
-`submit_suggestion` tools. The five-minute case expects one task work run and
-disables automatic retries. Company and agent accounted-work budgets are $10.
+`submit_suggestion` tools. The case expects one task work run and disables
+automatic retries. Local attempts have five minutes; Daytona inherits the
+15-minute remote deadline. Company and agent accounted-work budgets are $10.
 Setup probes and any background work must also pass through the separately
 budgeted gateway when using an operations manifest. Local compute is unmetered.
-Cost and duration forecasts remain unmeasured until a retained live attempt.
+Daytona cells require an immutable image and the existing provider credential.
+Their ephemeral sandboxes carry the 60-minute provider lifetime. Historical local
+measurements remain separate; Daytona cost and time are unknown until measured.
+These registrations do not qualify remote behavior or change daily/weekly selections.
 The runner E2E maintainers own this manual case; `--all` excludes it.
 
 After collection and fixture teardown, the oracle reads the retained evidence
@@ -1043,5 +1047,6 @@ quality or survival after an abrupt worker kill.
 
 ```sh
 pnpm test:e2e:runner -- --list --suite feedback-persistence
+pnpm test:e2e:runner -- --list --suite feedback-persistence --environment daytona
 pnpm test:e2e:runner -- --id feedback-persistence.runner-acpx-claude.local.complaint-and-suggestion --max-automatic-retries 0 --max-parallel 1
 ```

tests/runner-e2e/OPERATIONS-COVERAGE.md

@@ -61,6 +61,11 @@ current `pnpm test:e2e:runner -- --list` before choosing work. In particular,
 Everyday Workflows, provider connections, feedback persistence, interruption,
 and case handoffs must not be inferred from a passing `--all` campaign.
 
+The explicit feedback suite also registers native Claude/Codex cells on Daytona.
+Their model usage, duration, feedback persistence and cleanup remain unqualified
+until retained live attempts prove them. These cells do not enter the daily or
+weekly selections, and `--all` continues to exclude the suite.
+
 Keep the broader rotating lane disabled until its exact cells have a metered
 route, complete retained attempts, a measured cost/time distribution, and a
 budget that includes failures. Prefer targeted continuation, approval, and

tests/runner-e2e/OPERATIONS.md

@@ -11,11 +11,13 @@ Eval audit, and remaining activation gaps.
 
 ## Supported admission
 
-The initial path admits local `core-compatibility` and `feedback-persistence` cells for `runner-codex`,
-`runner-acpx-claude`, `legacy-codex`, and `legacy-claude`. Each cell must have one
-provider credential and an exact priced model. Other suites, remote environments,
-subscription profiles, hosted tools, multimodal inputs, premium service modes,
-and separate judges fail admission. This whitelist is implementation scope, not
+The bounded path admits local `core-compatibility` and `feedback-persistence` cells
+for `runner-codex`, `runner-acpx-claude`, `legacy-codex`, and `legacy-claude`.
+Daytona admits the two native profiles in those suites when the manifest declares
+`remoteGateway: "cloudflare"`. Each cell needs one exact priced model and its
+provider credential; Daytona also needs its provider key. Other suites and remote
+routes, subscription profiles, hosted tools, multimodal inputs, premium service
+modes, and separate judges fail admission. This whitelist is implementation scope, not
 evidence that every admitted harness has passed live qualification.
 
 Before creating the journal or requesting credentials, manifest validation checks
@@ -899,8 +901,9 @@ launch manifests must use the adopted cap. The migration cannot extend an expire
 deadline, change an existing cap, or replenish an allowance. An interrupted append
 remains an accounting failure and must be recovered before more paid work.
 
-The explicit local `feedback-persistence` native Claude/Codex cells also accept
-an operations manifest. They use the same metered connection, prices, campaign
+The explicit `feedback-persistence` native Claude/Codex cells on local and Daytona
+execution also accept an operations manifest. Remote cells require the Cloudflare
+gateway declaration and immutable image. They use the same metered connection, prices, campaign
 journal, setup boundary, and durable attempt allowance as core compatibility.
 Collection and the independent saved-evidence oracle require no additional
 model call. The synthetic items need review as test data, not product incidents.

tests/runner-e2e/README.md

@@ -2985,14 +2985,18 @@ from live qualification and lists remaining billing/reporting coverage gaps.
 
 ## Feedback persistence (explicit only)
 
-`--suite feedback-persistence` selects two local native Claude/Codex cells.
+`--suite feedback-persistence` selects four native Claude/Codex cells across local and Daytona execution.
 Each creates one standard task through the browser, asks for two distinct
 synthetic feedback items, and requires the real `submit_complaint` and
-`submit_suggestion` tools. The five-minute case expects one task work run and
-disables automatic retries. Company and agent accounted-work budgets are $10.
+`submit_suggestion` tools. The case expects one task work run and disables
+automatic retries. Local attempts have five minutes; Daytona inherits the
+15-minute remote deadline. Company and agent accounted-work budgets are $10.
 Setup probes and any background work must also pass through the separately
 budgeted gateway when using an operations manifest. Local compute is unmetered.
-Cost and duration forecasts remain unmeasured until a retained live attempt.
+Daytona cells require an immutable image and the existing provider credential.
+Their ephemeral sandboxes carry the 60-minute provider lifetime. Historical local
+measurements remain separate; Daytona cost and time are unknown until measured.
+These registrations do not qualify remote behavior or change daily/weekly selections.
 The runner E2E maintainers own this manual case; `--all` excludes it.
 
 After collection and fixture teardown, the oracle reads the retained evidence
@@ -3007,6 +3011,7 @@ quality or survival after an abrupt worker kill.
 
 ```sh
 pnpm test:e2e:runner -- --list --suite feedback-persistence
+pnpm test:e2e:runner -- --list --suite feedback-persistence --environment daytona
 pnpm test:e2e:runner -- --id feedback-persistence.runner-acpx-claude.local.complaint-and-suggestion --max-automatic-retries 0 --max-parallel 1
 ```
 

tests/runner-e2e/catalog.test.ts

@@ -219,10 +219,10 @@ describe("runner E2E catalog", () => {
     expect(localIntegrityTasks).toHaveLength(2);
     expect(openRouterBreadthTasks).toHaveLength(3);
     expect(runnerSuites.map((suite) => suite.expectedMatrixSize)).toEqual([
-      2, 1, 3, 2, 2, 4, 4, 4, 4, 4, 4, 8, 10, 6, 63, 12, 6, 4, 2, 4, 2, 30, 3, 16, 16, 2, 6, 8, 46, 2, 23, 15, 52, 6, 6, 20, 26, 52, 2, 28, 18, 2, 6, 6, 13, 10, 2, 48, 16, 10, 2, 1, 1, 4, 116, 2, 2,
+      2, 1, 3, 2, 2, 4, 4, 4, 4, 4, 4, 8, 10, 6, 63, 12, 6, 4, 2, 4, 2, 30, 3, 16, 16, 2, 6, 8, 46, 2, 23, 15, 52, 6, 6, 20, 26, 52, 2, 28, 18, 2, 6, 6, 13, 10, 4, 48, 16, 10, 2, 1, 1, 4, 116, 2, 2,
     ]);
-    expect(validateRunnerCatalog()).toHaveLength(769);
-    expect(new Set(runnerMatrix.map((entry) => entry.id)).size).toBe(769);
+    expect(validateRunnerCatalog()).toHaveLength(771);
+    expect(new Set(runnerMatrix.map((entry) => entry.id)).size).toBe(771);
     expect(
       runnerMatrix.filter((entry) => entry.suite.id === "core-compatibility"),
     ).toHaveLength(48);
@@ -259,12 +259,13 @@ describe("runner E2E catalog", () => {
       executionId: "bounded-lifetime", daytonaImage: `runner@sha256:${"a".repeat(64)}`,
       secretRefs: { DAYTONA_API_KEY: { type: "secret_ref", secretId: "22222222-2222-4222-8222-222222222222", version: "latest" } },
     });
-    const lifetimeMs = Number(payload.config?.ttlMinutes) * 60_000;
+    const config = payload.config as Record<string, unknown>;
+    const lifetimeMs = Number(config.ttlMinutes) * 60_000;
     expect(Number.isFinite(lifetimeMs)).toBe(true);
     const cells = runnerMatrix.filter(entry => entry.environment === environment);
     expect(cells.length).toBeGreaterThan(0);
     for (const cell of cells) {
-      expect(lifetimeMs, cell.id).toBeGreaterThan(cell.task.attemptTimeoutMs.daytona + Number(payload.config?.timeoutMs));
+      expect(lifetimeMs, cell.id).toBeGreaterThan(cell.task.attemptTimeoutMs.daytona + Number(config.timeoutMs));
     }
   });
 

tests/runner-e2e/catalog.ts

@@ -1698,8 +1698,8 @@ export const runnerSuites: readonly RunnerSuiteFixture[] = [
     description: "Real complaint and suggestion submissions, durable bodies, and retained tool receipts.",
     groups: ["native"],
     profiles: runnerProfiles.filter(profile => ["runner-codex", "runner-acpx-claude"].includes(profile.id)),
-    environments: [localEnvironment], tasks: [feedbackPersistenceTask(runnerTasks.find(task => task.id === "message-marker")!)], expectedMatrixSize: 2,
-    definitionMetadata: { version: 1, owner: "runner-e2e-maintainers", scheduling: "explicit-only",
+    environments: runnerEnvironments, tasks: [feedbackPersistenceTask(runnerTasks.find(task => task.id === "message-marker")!)], expectedMatrixSize: 4,
+    definitionMetadata: { version: 2, owner: "runner-e2e-maintainers", scheduling: "explicit-only",
       sourceDigest: feedbackPersistenceDigest(runnerTasks.find(task => task.id === "message-marker")!), expectedWorkRuns: 1, automaticRetryPolicy: "single_attempt",
       budgetCents: 1_000, forecast: "unmeasured; gateway manifest required for bounded qualification",
       grading: "retained-exact-bodies-and-run-attributed-completed-tools", feedback: "synthetic-disposable-items" },

tests/runner-e2e/feedback-persistence.test.ts

@@ -55,13 +55,26 @@ describe("feedback persistence oracle", () => {
       expect(gradeFeedbackPersistence({ ...fixture(), evidence }).some(check => !check.passed)).toBe(true);
     }
   });
-  it("registers only two explicit local native cases with no automatic retry", () => {
+  it("registers explicit local and Daytona native cases with no automatic retry", () => {
     const selected = selectRunnerExecutions(parseRunnerSelectors(["--suite", "feedback-persistence"]));
     expect(selected.map(execution => execution.id)).toEqual([
       "feedback-persistence.runner-codex.local.complaint-and-suggestion",
+      "feedback-persistence.runner-codex.daytona.complaint-and-suggestion",
       "feedback-persistence.runner-acpx-claude.local.complaint-and-suggestion",
+      "feedback-persistence.runner-acpx-claude.daytona.complaint-and-suggestion",
     ]);
     expect(selected.every(execution => execution.task.automaticRetryPolicy === "single_attempt")).toBe(true);
+    expect(selected.every(execution => execution.task.attemptTimeoutMs.local === 5 * 60_000)).toBe(true);
+    for (const execution of selected.filter(entry => entry.environment.id === "daytona")) {
+      expect(execution.requiredCredentials).toEqual(expect.arrayContaining([execution.profile.credential, "DAYTONA_API_KEY"]));
+      expect(execution.requiredCredentials).toHaveLength(2);
+      expect(execution.task.expectedRunCount).toBe(1);
+      expect(execution.task.attemptTimeoutMs.daytona).toBe(15 * 60_000);
+      const secretRefs = { DAYTONA_API_KEY: { type: "secret_ref" as const, secretId: "22222222-2222-4222-8222-222222222222", version: "latest" as const } };
+      expect(() => execution.environment.buildEnvironment({ executionId: execution.id, secretRefs, daytonaImage: "runner:latest" })).toThrow("immutable");
+      const payload = execution.environment.buildEnvironment({ executionId: execution.id, secretRefs, daytonaImage: `runner@sha256:${"a".repeat(64)}` });
+      expect(payload).toMatchObject({ driver: "sandbox", config: { provider: "daytona", reuseLease: false, ttlMinutes: 60 } });
+    }
     expect(runnerMatrix).toEqual(expect.arrayContaining(selected));
     expect(selectRunnerExecutions(parseRunnerSelectors(["--all"])).some(execution => execution.suite.id === "feedback-persistence")).toBe(false);
   });

tests/runner-e2e/feedback-persistence.ts

@@ -26,7 +26,7 @@ export function expectedFixtureFeedback(nonce: string) {
 export function feedbackPersistenceTask(base: RunnerTaskFixture): RunnerTaskFixture {
   return {
     ...base, id: "complaint-and-suggestion", label: "Retain submitted feedback", automaticRetryPolicy: "single_attempt",
-    attemptTimeoutMs: { local: 5 * 60_000, daytona: 5 * 60_000 },
+    attemptTimeoutMs: { local: 5 * 60_000, daytona: base.attemptTimeoutMs.daytona },
     buildPrompt(nonce) {
       const bodies = expectedFixtureFeedback(nonce);
       return [

tests/runner-e2e/operations.test.ts

@@ -164,6 +164,8 @@ it.each([
   cell.id,
   "feedback-persistence.runner-codex.local.complaint-and-suggestion",
   "feedback-persistence.runner-acpx-claude.local.complaint-and-suggestion",
+  "feedback-persistence.runner-codex.daytona.complaint-and-suggestion",
+  "feedback-persistence.runner-acpx-claude.daytona.complaint-and-suggestion",
   "core-compatibility.runner-acpx-claude.daytona.message-marker",
 ])("routes %s through a shared connection without ambient provider bindings", async executionId => {
   const selected = runnerMatrix.find(execution => execution.id === executionId)!;