Files
PaperClipAI/tests
DottaandPaperclip 40c1ab36cd feat(evals): register explicit Daytona feedback persistence (#17986)
## Thinking Path

> - Paperclip manages AI agents and their work.
> - Product E2E evals check real server, browser, runner, and provider
behavior.
> - Feedback persistence already checks saved complaints and suggestions
with exact bodies and run attribution.
> - Its catalog registers only local cells, so a Daytona campaign cannot
select the same workflow.
> - This PR registers the existing workflow for the two native Daytona
profiles and tests their gateway bindings.
> - Remote cells stay explicit-only and unqualified until a bounded live
run passes.

## Linked Issues or Issue Description

Refs #17980 and #17975. No duplicate open PR or issue was found in the
Daytona feedback search.

**What existing behavior does this improve?**

The `feedback-persistence` Product E2E suite and its explicit execution
selectors.

**Current behavior**

At master `d54f776ca0`, the suite has only local Claude and Codex cells.
Selecting the matching Daytona ID fails before admission. The gateway
manifest already supports these native remote profiles.

**Proposed behavior**

Register four cells: local and Daytona for each native profile. Reuse
the existing saved-body and completed-tool oracle. Require an immutable
image and Daytona credentials. Keep one attempt, one expected work run,
five-minute local and inherited 15-minute remote case timeouts, and the
existing 60-minute provider creation TTL.

**Reason and benefit**

A bounded remote qualification can check both a simple task and real
feedback persistence. Local evidence remains separate from remote
acceptance.

**Breaking changes**

Explicit `--suite feedback-persistence` now lists four cells instead of
two. `--all` still excludes the suite. Daily and weekly manifests still
select their existing exact local IDs.

## What Changed

- Register the two native Daytona feedback cells and increment the suite
definition version.
- Address the remote cold-start review: inherit the base task's
15-minute Daytona deadline and retain the five-minute local limit. The
regression failed before the fix and now passes.
- Check immutable-image requirements, credentials, lifetime, and shared
gateway connections in existing contract tests.
- Fix the config type in the TTL catalog test added by #17980. Keep its
runtime lifetime assertions.
- Document remote selection and its unqualified status in the existing
eval guides.

## Verification

- Before the catalog change, the registration regression failed: one
failed and 16 passed.
- Focused catalog, feedback, operations, ledger, manifest, and summary
tests: 114 passed across six files.
- Read-only discovery lists exactly the two Daytona feedback IDs.
- The Node test phase passes separately: 137 tests. The retained Vitest
database failure prevents the unit launcher from reaching that phase.
- Full Product support suite: 4,523 passed, three skipped, and one
embedded PostgreSQL bootstrap failure. The same database failure was
reproduced on a clean baseline before this PR.
- Fresh Linux Product typecheck passed at the review-fix commit:
`GOMEMLIMIT=2GiB GOGC=50 GOMAXPROCS=2 node
node_modules/typescript/bin/tsc -p tests/runner-e2e/tsconfig.json`. The
default run and the ineffective JavaScript-heap attempt exited 137 and
remain retained. The compiler launches a native child process; its
memory controls let the unchanged semantic check complete. Host
typecheck still has 15 stale installed DB/Runner export errors; the two
TTL test typing errors are fixed. All 58 latest-head checks are
acceptable (success, neutral, or skipped). Both fresh reviews are 5/5,
with no unresolved review threads.
- The updated credential-free Linux controller discovers both planned
remote IDs and launches Chromium 151.0.7922.34. Runtime build inputs are
unchanged from the prior controller. The Daytona image content ID is
still
`b55f03dc9c633db300b70206a691ff45a8e779e56a3c546f9358e3282d3597c9`.
- No model requests, sandbox allocation, or paid schedule changes.

## Risks

- Selecting the whole explicit suite now includes remote cells and
requires Daytona configuration. Use exact IDs for a bounded paid
campaign.
- Registration and contract tests do not prove live remote behavior or
token accounting. Remote qualification remains pending.
- The shared dependency checkout limits the local host typecheck. The
fresh Linux semantic check passed. Earlier compiler OOM failures and the
database bootstrap failure remain retained.

## Model Used

- OpenAI GPT-6 through Codex, with tool use and code execution. The
exact serving model ID and context window are not exposed in this
session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 00:54:04 -05:00
..