Commit Graph
554 Commits
Author SHA1 Message Date
Dotta 104a23be5f fix(evals): retain completion stability window within original handoff deadline 2026-10-28 11:21:06 -05:00
Dotta c455a206a8 fix(evals): preserve original native observation clocks for eligible historical finals 2026-10-28 11:15:52 -05:00
Dotta 3e1c87117e fix(evals): seed active prerequisite tasks with valid owners 2026-10-28 11:03:18 -05:00
Dotta a1c5811363 fix(evals): retain executable cancellation proof at final boundary 2026-10-28 10:56:32 -05:00
Dotta b740827954 test(evals): refresh pricing and artifact valid observations 2026-10-28 10:33:20 -05:00
Dotta 22dca95926 fix(evals): avoid cancelling consumed privacy response bodies 2026-10-28 10:22:02 -05:00
Dotta cb31ebccf0 test(e2e): preserve secret proposal original observation clock 2026-10-28 09:57:44 -05:00
Dotta e386933481 test(e2e): bind secret proposal review to expanded card 2026-10-28 09:46:16 -05:00
Dotta 8bbf313552 test(e2e): keep long-thread measurements within original clock 2026-10-28 09:46:16 -05:00
Dotta ed24a4a444 Retain certified source SDK evidence for plugin fixtures 2026-10-28 09:46:16 -05:00
Dotta 7170ca4b81 test(e2e): preserve original approval and session clocks 2026-10-28 09:12:28 -05:00
Dotta a477b2c5d8 fix(evals): preserve original task and inbox observation budgets 2026-10-28 08:17:14 -05:00
Dotta efa3382b21 test(e2e): fingerprint executed plugin CLI source and loader 2026-10-28 08:15:28 -05:00
Dotta 250ba62791 test(e2e): bind installed team to selected OpenCode runtime 2026-10-28 08:15:27 -05:00
DottaandPaperclip 1c6021a4e6 fix(evals): bind inline secret helper to actual consumer task
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-28 08:15:27 -05:00
DottaandPaperclip 0e4ebca61c fix(evals): bind approval wait to actual task workspace
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-28 08:15:27 -05:00
DottaandPaperclip dea44755b0 fix(evals): prepare prerequisite people through public membership routes
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-28 08:15:26 -05:00
Dotta 81987af644 Accept lifecycle-ready running workers in catalog readiness 2026-10-28 08:15:26 -05:00
Dotta 8cd9e8bc6b fix(evals): keep fast response initial observation on original clock 2026-10-28 07:26:09 -05:00
DottaandPaperclip 1c7cbdc25b test(evals): preserve artifact outcome without extra task status gate
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-28 07:20:35 -05:00
DottaandPaperclip 3dab66775d fix(evals): bind answer race completion and ordinary draft evidence
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-28 07:20:35 -05:00
Dotta af01c4f6d3 fix(evals): await fast response save before model readback 2026-10-28 07:19:59 -05:00
Dotta 9ea0c8a61d fix(evals): retain original clock for seven prepared human journeys 2026-10-28 02:31:43 -05:00
Dotta d282ae2eee fix(evals): share original observation clock across selected native phases 2026-10-28 02:01:44 -05:00
Dotta 81abf04b21 fix(evals): qualify owned OpenCode post-stop proxy cleanup 2026-10-28 01:32:20 -05:00
Dotta 15fb2318bf fix(evals): preserve human origin on consumed queue probes 2026-10-28 01:32:20 -05:00
Dotta b70439f387 fix(evals): keep deadline boundary timeout strictly positive 2026-10-28 01:32:20 -05:00
Dotta b0eff403b8 fix(evals): share original budget for instance and confirmation waits 2026-10-28 01:32:19 -05:00
Dotta e232200747 test(e2e): pin configurable auxiliary tests to requested model 2026-10-28 00:55:14 -05:00
Dotta 1e686fa0f3 fix(evals): bind routine failure evidence to selected profile 2026-10-28 00:53:18 -05:00
Dotta 78783dda57 fix(evals): bind routine history grading to observed provider 2026-10-28 00:47:54 -05:00
Dotta 765dd51e0f test(e2e): retain context custody across budget handoff 2026-10-28 00:37:55 -05:00
Dotta fa7304ae5e test(e2e): share original budget across company followups 2026-10-28 00:37:55 -05:00
Dotta 6eca7f6bcc test(e2e): verify returned historical artifact copies 2026-10-28 00:25:04 -05:00
Dotta 2c6d746015 fix(evals): retain contracted delivery negation in date grading 2026-10-28 00:25:04 -05:00
Dotta fefc475470 fix(evals): accept exact natural historical delivery dates 2026-10-28 00:25:04 -05:00
Dotta 8b29ca5489 fix(evals): enable real workspace view before lifecycle journeys 2026-10-28 00:21:59 -05:00
Dotta 81bba1dd0d fix(evals): scope design approval claims to contrast clauses 2026-10-27 23:00:30 -05:00
Dotta 89063699af fix(evals): recognize explicit design revision ordering 2026-10-27 23:00:29 -05:00
Dotta 22dd6a1280 test(evals): report effective project-defaults timeout policy 2026-10-27 22:36:30 -05:00
DottaandPaperclip d1f983e395 test(evals): distinguish team pricing from negated per-user charges
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-27 21:56:43 -05:00
DottaandPaperclip 9434442348 test(evals): reject waived required skills without confusing unrelated history
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-27 21:56:43 -05:00
DottaandPaperclip 96dcd15449 test(evals): reject modal contradictions of required plan features
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-27 21:56:43 -05:00
DottaandPaperclip 021c155682 test(evals): ground team requirements in actual skill identities
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-27 21:56:43 -05:00
DottaandPaperclip 913ffeec7c test(evals): accept substantive single-page plan paraphrases
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-27 21:56:43 -05:00
DottaandPaperclip 33f99c6c5a docs(evals): preserve the composed operating section heading
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-27 21:18:36 -05:00
DottaandPaperclip b797e1cc29 test(evals): read delivered task HTML with the authenticated fixture actor
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-27 21:18:00 -05:00
DottaandPaperclip 1ba267de60 test(evals): use production chat timeout within finite cell budgets
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-27 21:18:00 -05:00
DottaandPaperclip 0195fcb016 Merge pinned production for the post-custody chat pilot
Preserve frozen 244-scenario YAML intent and 191-capability coverage. Production baseline: 39e811c9c5.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-27 21:03:19 -05:00
DottaandPaperclip e4b30163bb fix(evals): preserve the complete trusted controller artifact inventory (#17943)
<!-- entire-trail-link-start -->
https://entire.com/gh/paperclipai/-/trails/103/changes/paperclip/103
<!-- entire-trail-link-end -->

## Thinking Path

> - Paperclip manages real agent work.
> - Controlled Runner evaluations record model spend and traces.
> - A trusted build creates one portable controller with a complete file
inventory.
> - GitHub artifact upload omitted one hidden dependency metadata file.
> - The next job correctly refused the incomplete inventory before any
paid worker started.
> - This change preserves all inventoried bytes without relaxing
verification.

## Linked Issues or Issue Description

Refs #17889, #17929.

**What happened?**
[Run
40585138437](https://github.com/paperclipai/paperclip/actions/runs/40585138437)
failed at copied controller verification. Its artifact contains 294
files; its signed inventory expects 295. The only missing file is
`server/node_modules/protobufjs/node_modules/@protobufjs/pool/.npmignore`.
No extra files exist. Every paid worker was skipped and no model request
ran.

**Expected behavior**
The trusted controller artifact preserves its full inventory, including
hidden package metadata. The strict verifier rejects any actual change
or missing file.

**Steps to reproduce**
Build the portable runtime, upload the directory with the action's
default hidden-file exclusion, then verify the downloaded artifact
against the original receipt.

## What Changed

- Enable hidden files only on the fresh trusted controller code upload.
- Download that exact upload and run a real artifact-service round-trip
test before any private preparation. The test fails if the upload drops
hidden files again; it tests the actual transport instead of matching a
YAML flag.
- Keep the strict byte inventory and trusted source/receipt checks.
- Document why this code artifact differs from private attempt evidence,
which still uses the protected archive.

## Verification

- Exact retained artifact comparison: one missing `.npmignore`, zero
extra files.
- `actionlint .github/workflows/runner-protocol-live-evals.yml` passed.
- Protocol workflow security tests: 14 passed.
- The artifact-service integration test is explicit-only locally. The
controlled workflow supplies the real downloaded artifact, trusted
source and receipt digest.
- No product runtime, provider routing or grading code changed. CI runs
broader checks. Fresh controlled workflow qualification follows merge.

## Risks

Hidden files are allowed only in the code-only output of the
provider-credential-free build. Raw private evidence and credentials do
not enter that upload. The artifact verifier still requires every file
and SHA-256 from the trusted receipt.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, repository inspection and
code execution. The exact serving model ID and context window are not
exposed to this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-27 20:55:18 -05:00