Files
PaperClipAI/tests
DottaandPaperclip 2c70d04bf6 feat(evals): account for explicit Daytona model requests (#17936)
## Thinking Path

> - Paperclip evals need complete model-spend evidence.
> - The bounded Product E2E controller already owns request admission
and token receipts.
> - Its loopback endpoints cannot serve a Daytona agent.
> - Explicit remote cases therefore need a scoped transport to the same
controller.
> - This PR exposes only authenticated model and OTLP endpoints through
one owned quick tunnel.
> - Remote requests keep the same budget, request identity, usage
journal and cancellation drain.

## Linked Issues or Issue Description

Refs #17928, #17929, #17523.

**What happened?**
The operations manifest and fixture reject Daytona. The parent gateway
and native trace relay bind loopback, so normal remote agent traffic
cannot use their cost accounting.

**Expected behavior**
An explicit native marker or feedback case on Daytona can route requests
through the bounded parent. Upstream provider and Braintrust keys stay
on that controller. Every paid request keeps its usage receipt and the
attempt retains ordinary feedback and cleanup evidence.

**Steps to reproduce**
Use the explicit `remoteGateway: "cloudflare"` manifest option for a
native Codex or Claude Daytona marker/feedback case. Without that
option, the remote selection remains rejected. The parent verifies its
public health nonce before fixture setup.

## What Changed

- Add a deadline-limited bridge with exact model/OTLP paths, separate
scoped tokens, streamed responses and owned tunnel cleanup.
- Start one HTTPS quick tunnel with a private home and no provider
credentials or ambient Cloudflare configuration. Do not retry creation
or paid work.
- Admit only explicit native Daytona marker/feedback cells and require
the provisioning credential before opening the journal.
- Bind fixture model/OTLP endpoints to the assigned remote origin. Keep
provisioning access in the fixture/server controller.
- Preserve final token usage when a remote client disconnects during
shutdown.
- Document the manual route. Daily and weekly selections remain local.

## Verification

- Full Product E2E unit suite: 4,370 passed and 2 skipped across 206
files, plus 137 Node checks.
- Focused remote/usage/operations/trace checks: 71 tests passed; final
bridge/drain checks: 15 passed.
- Product E2E typecheck passed.
- Tests cover missing tunnel executable, expiry, authentication, blocked
paths, token separation, missing provisioning credentials, endpoint
assignment and listener cleanup.
- The cancellation regression uses the real gateway and ledger with a
synthetic upstream. It retains the final receipt once after remote
disconnect.
- Latest source and architecture/cleanup checks: 167 passed; eval
typecheck passed. Both reviews are 5/5 and exact-head CI is green. Live
qualification follows merge.

## Risks

The explicit route sends synthetic eval requests and scoped attempt
tokens through Cloudflare. Those tokens expire with the existing
campaign deadline and budget. Upstream keys stay on the controller.
Public readiness has a finite wait; failure stops before fixture setup.
The operator still needs the existing immutable Daytona image and
verified native artifacts. Source and deterministic tests do not
establish live remote acceptance.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, repository inspection and
code execution. The exact serving model ID and context window are not
exposed to this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-27 16:38:23 -05:00
..