Files
PaperClipAI/tests
DottaandPaperclip cff331f6ca fix(evals): retain local controller cost origin (#17990)
## Thinking Path

> - Paperclip manages AI agents and their work.
> - Its eval journals retain model costs and operating reservations
under one cap.
> - The operating policy records the controller's execution origin.
> - The schema currently accepts only GitHub-hosted and AWS Fleet
origins.
> - A local controller must therefore use an inaccurate label or fail
admission.
> - This PR adds the local origin and keeps the hosted-only schedule
gate.

## Linked Issues or Issue Description

Refs #17986. No duplicate open PR or issue was found in the
operating-overhead/local search.

**What existing behavior does this improve?**

The Product E2E operating allowance and its retained journal policy.

**Current behavior**

At master `40c1ab36cd`, an unscheduled local controller cannot set
`runner: "local"`. The generic allowance validator rejects it. Local
qualification can use the same accounting controls but cannot record its
origin correctly.

**Proposed behavior**

Accept the local origin in the generic allowance. Keep all component,
price, cap, expiry, and receipt rules. The scheduled validator still
requires `github-hosted` and rejects a local allowance.

**Reason and benefit**

Local qualification can retain an accurate execution origin without
claiming that its reservations are billed GitHub charges.

## What Changed

- Add `local` to the operating allowance's runner type and validator.
- Exercise local journal creation, model admission, retained overhead,
restart, and reconciliation in the existing boundary test.
- Check that a local allowance fails Product scheduled admission.
- Pin new hosted Runner controller admission to `github-hosted`.
Preserve original labels when reading historical manifests; those reads
cannot authorize a new worker.
- Exercise the hosted-origin rejection and historical/local reads in
both source and copied portable runtimes.
- Document the local label and its accounting limits.

## Verification

- Before the fix: the new local persistence case and schedule-boundary
assertion failed; eight tests passed.
- Focused ledger, overhead, operations, and scheduled finalization
tests: 71 passed across five files.
- Controlled Runner source and portable tests: 86 passed. The new
hosted-origin regression failed before the fix.
- Full Product support suite at the first commit: 4,524 passed, three
skipped, and one retained embedded PostgreSQL bootstrap failure. The
same database failure was previously reproduced on a clean baseline.
- Separate Node test phase on the host: 137 passed.
- Actual Linux controller: database lifecycle tests pass. After
installing its missing `zip` utility, the full Product suite has 4,527
passes and one Copilot parent-version coverage failure. That same
failure reproduces on clean master `40c1ab36cd` with the same Linux
filesystem and dependencies (14 passes, one failure). Copilot files are
unchanged. The initial seven ZIP-related failures and the watcher
failure remain retained.
- The final fresh Linux semantic check passes: `GOMEMLIMIT=2GiB GOGC=50
GOMAXPROCS=1 node node_modules/typescript/bin/tsc --singleThreaded -p
tests/runner-e2e/tsconfig.json`. The parallel compiler OOM is retained.
No semantic checks were omitted.
- Both fresh reviews are 5/5, with the addressed thread resolved. Hosted
Build first failed during unchanged Pi setup prefetch with an operation
timeout. One job rerun recovered; the original failure remains retained.
All 58 latest-head checks are now acceptable.
- No paid model requests, sandbox creation, schedule changes, or
historical journal edits.

## Risks

- The caller supplies this origin. It must describe the actual
controller.
- `local` does not prove zero compute cost, provide a default allowance,
or release a hold.
- Both maintained hosted admission paths require the GitHub-hosted
controller origin. A previously accepted hosted policy with an AWS/local
label now fails new admission. Historical reads preserve the original
evidence.

## Model Used

- OpenAI GPT-6 through Codex, with tool use and code execution. The
exact serving model ID and context window are not exposed in this
session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 02:05:36 -05:00
..