mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-09 16:02:36 +01:00
367b6f9b3fe71477dbf7e024e7740133a5f88955
307
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandClaude Opus 5.5
|
2b2004bf50 |
test(runner-e2e): gate completion briefs inside the run's own task directory (#19475)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent Chat returns delegated-task completions to the chat that started them, and the runner E2E `completion-updates` suite checks this with real providers > - The restart cell (`completion-updates.runner-codex.local.handoff-completion-restart`) failed before its server restart with "Timed out waiting for worker waiting while originating chat is idle" > - Since project-optional task workspaces (#15749), a task in a project without a workspace runs in its isolated task directory. A native chat runs in its isolated conversation directory. Native Codex can read only that directory. > - The fixture still wrote its brief wait scripts into the managed project workspace and the agent home. Codex got EPERM on both scripts, wrote the note from the task record, and finished. The worker never reached the gate, so the setup poll timed out. > - This pull request makes the waits name only cwd-relative files and publishes each brief into the run's own isolated task directory > - The benefit is that the restart cell reaches its real boundary again (durable Done, source provider held at the gate, server restart) and tests restart recovery instead of a stale fixture path ## Linked Issues or Issue Description **What happened?** The runner E2E cell `completion-updates.runner-codex.local.handoff-completion-restart` timed out in setup. The run ledger shows both agents report `permission_denied` (EPERM) for `node .../wait-for-brief-*.cjs`. The worker run cwd was `<instance>/isolated-workspaces/<company>/<issue>`. The fixture wrote the script to the managed project workspace and the agent home. **What did you expect to happen?** The worker waits at the brief gate while the chat is idle. Then the cell releases the brief, holds the completion reply at its gate, and restarts the server. **Steps to reproduce** `pnpm test:e2e:runner -- --max-automatic-retries 0 --id completion-updates.runner-codex.local.handoff-completion-restart` on master `6cafcaf89b`. **Classification** Test defect. The failure happens before the restart. Workspace isolation is the intended product behavior. ## What Changed - `tests/runner-e2e/chat-stories.ts`: add `workspaceChatBrief(nonce, timeoutMs)`. It returns a bounded `node -e` command that reads and writes only cwd-relative files, plus `at(dir)` to find the gate and ready files in a workspace. - `tests/runner-e2e/completion-update-flow.ts`: the worker and completion instructions use that command. The fixture reads readiness from, and publishes briefs to, `isolated-workspaces/<company>/<issue>` for each delegated task and the source chat. The setup poll state now records the worker run cwd. Cleanup releases every wait that is still open. - `tests/runner-e2e/chat-stories.test.ts`: regression test. It runs the command from a temporary cwd, checks that the command names no host path, and checks the nonce and timeout bounds. It was red before the helper existed. - `tests/runner-e2e/catalog.ts`: bump the `completion-updates` definition to version 28 and set `workerBriefWorkspace: "run-isolated-task-directory"`. - `tests/runner-e2e/README.md`: describe the new brief location. ## Verification - `npx vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/chat-stories.test.ts tests/runner-e2e/completion-updates.test.ts tests/runner-e2e/catalog.test.ts`: 108 passed. - Live cell, native Codex `gpt-5.6-sol`, local runner: `pnpm test:e2e:runner -- --max-automatic-retries 0 --id completion-updates.runner-codex.local.handoff-completion-restart`. Results are in the Verification comment below. - Two earlier local attempts in this worktree failed in setup with `harness_test_failed`. The worktree did not have a built `paperclip-runnerd`. After `pnpm --filter @paperclipai/paperclip-runner build`, the cell passed. ## Risks - Low risk. The change is in test fixtures only. No product code changes. - The idle, busy, and multiple cells share this flow. They now also use the isolated task directory. That is the same directory the product selects for these tasks. - The agent must run the inline command from its cwd. The instructions say this, and the command fails closed (no gate) if the agent runs it elsewhere. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits). Default context window. Extended reasoning on. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandClaude Opus 5.5
|
a12fcb8f26 |
fix(runner-e2e): find saved slugify source in a workspace project folder (#19471)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The runner E2E everyday `recover-controller` case proves that a task survives a control-plane restart. > - The case waits for an active run with a saved `slugify.py` before it kills the controller. > - The harness looked for `slugify.py` only at the workspace root. > - The task brief asks for `slugify.py` "in the project workspace". It does not ask for the root. ACPX Claude saved it in `slugify-tool/slugify.py`. > - The harness timed out before the restart, so the case did not test restart recovery at all. > - This pull request makes the harness find the saved source in a bounded set of project folders. > - The benefit is that the case reaches its real fault boundary for agents that make a project folder. ## Linked Issues or Issue Description **What happened?** The local cell `everyday-workflows.runner-acpx-claude.local.recover-controller` failed with "Timed out waiting for active run with saved source". The agent finished the task and delivered a correct ZIP. It saved `slugify.py` in `<workspace>/slugify-tool/`. The harness read only `<workspace>/slugify.py`, so it never saw the saved source. It also recorded `saved-source-unavailable-at-final-capture`. The controller restart never happened. **Expected behavior** The harness finds the source that the agent saved in the project workspace. Then it kills the controller while the run is active and grades the recovery. **Steps to reproduce** 1. Run `pnpm test:e2e:runner -- --max-automatic-retries 0 --id everyday-workflows.runner-acpx-claude.local.recover-controller` with a provider that saves the project in a subfolder. 2. See the timeout before the `controller-restarted` timeline entry. **Paperclip version or commit** `master` at `6cafcaf89b` (also reproduced at `d87f6fa4c2`). ## What Changed - Added `readWorkspaceSource` in `tests/runner-e2e/everyday-interruption.ts`. It reads `slugify.py` at the workspace root first. Then it searches subfolders to depth 2, shallowest and alphabetical first. It skips hidden folders, `node_modules`, `__pycache__` and `venv`. - `recordSource` (the stop-redirect and recover-controller boundaries) and the final local source capture in `tests/runner-e2e/everyday-flow.ts` use this helper. - The stop-redirect workspace snapshot (`workspaceFiles`) now includes the same bounded project folders. A stop check therefore still sees edits when the project is below the root. - Added `tests/runner-e2e/everyday-interruption.test.ts`. ## Verification - Red before the fix: `npx vitest run --config tests/runner-e2e/vitest.config.ts everyday-interruption` failed 2 of 4 tests (the nested-folder cases returned `undefined`). - Green after the fix: the same command passes 4 of 4. - `pnpm test:e2e:runner:typecheck` passes. - Live rerun of the cell after the fix (local, native ACPX Claude, `claude-sonnet-5`): the harness observed the active run with saved source, submitted the follow-up, and restarted the controller (`source-saved-before-interruption` with `active: true`, then `controller-restarted`). The original timeout did not occur. - Two valid live reruns after the fix both reached the restart. Both still did not pass. After the restart, the interrupted ACPX turn ended as `native_provider_terminal_failed: Provider turn failed`. In the second rerun the task stayed Blocked with the follow-up comment still queued and a Retry offered. This is a separate product signature. It is not in the scope of this harness fix. A follow-up handles it (see Risks). - One earlier rerun failed before the task started because the new worktree had no built runner binary (`paperclip_runner_binary_missing`). That was local setup only. The runner was then built with `pnpm --filter @paperclipai/paperclip-runner build`. ## Risks - Low risk. The change is test-harness only. - If a workspace has several `slugify.py` files, the harness picks the root copy, else the shallowest copy. A stale copy in another folder could be chosen. The search depth and skip list keep this bounded. - Known remaining failure in the same cell (separate signature): after a hard controller restart, the ACPX runner restore path (`acpx_provider_backend.rs`, `restore_session_if_needed`) closes the in-flight turn with `code: acpx_active_turn_recovery_closed` and no `error` object. The run shows the generic "Provider turn failed". The Codex fix in #15297 gives Codex a recoverable `provider_turn_lost_on_restore` cause and a same-conversation continuation. ACPX does not have this yet. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, used through Claude Code with tool use (shell, file edit) for diagnosis, the regression test, the fix, and live reruns. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (not applicable: test-harness-only change) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandClaude Opus 5.5
|
7e6d882558 |
fix(runner-e2e): require Daytona image lock digest; name expired overhead allowance (#19467)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Runner qualification uses a Daytona runner image and a nightly Runner Full-Stack E2E workflow > - The nightly workflow fails at "Validate catalog and select cells", and a local image build fails at `sha256sum -c` > - Operators cannot build a current Daytona image for restart tests until both failures are understood > - This pull request removes a stale Dockerfile lock digest default, fixes the local recipes, and makes the nightly failure name its real cause > - The benefit is that a local build fails clearly when the digest is missing, and an expired overhead allowance is obvious from the log ## Linked Issues or Issue Description **What happened?** 1. The nightly `Runner Full-Stack E2E` runs 41997383733 and 42190298486 fail in "Validate catalog and select cells". The log shows `digest-mismatch: error`, but that line is only the input list of `actions/download-artifact`. The real error is `Operating overhead allowance needs review before paid admission`. The repository variable `RUNNER_E2E_OPERATING_OVERHEAD_JSON` has `expiresAt: 2026-11-03T19:49:53Z`. `scheduledOperatingOverhead` rejects an allowance that expires within 4 hours. Manual dispatch (run 42046147744) does not use the scheduled overhead path, so it passed this job. 2. `docker/daytona-runner/Dockerfile` set `ARG PAPERCLIP_RUNNER_LOCK_SHA256=342a01e4…`. That value went stale when `pnpm-lock.yaml` changed (#19443). The local recipes in `tests/runner-e2e/README.md` and `docker/daytona-runner/README.md` did not pass the argument, so local builds failed with a confusing checksum mismatch. **Expected behavior** A local build computes and passes the lockfile digest. A missing digest fails by name. An expired overhead allowance tells the operator which allowance expired and which variable to renew. **Steps to reproduce** 1. Run `gh run view 42190298486 --log` and read the "Validate catalog and select cells" job. 2. Follow the old "## Daytona image" recipe in `tests/runner-e2e/README.md` on current master. ## What Changed - `docker/daytona-runner/Dockerfile`: remove the fixed `PAPERCLIP_RUNNER_LOCK_SHA256` default. Reject a missing or non-SHA-256 value by name before `sha256sum -c` and before installation. CI already passes the real value. - `tests/runner-e2e/README.md` and `docker/daytona-runner/README.md`: compute `shasum -a 256 pnpm-lock.yaml` and pass `PAPERCLIP_RUNNER_LOCK_SHA256` in the local build recipes. Replace the text about a "fixed Dockerfile default". - `tests/runner-e2e/daytona-image.test.ts`: new test. It requires a default-free `ARG`, the shape check before the checksum, and the argument in every documented local recipe. - `tests/runner-e2e/operating-overhead.ts`: the scheduled admission error now names the allowance reference, its validity window, the 4-hour margin, and `RUNNER_E2E_OPERATING_OVERHEAD_JSON`. The check itself is unchanged. ## Verification - `node node_modules/vitest/vitest.mjs run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/operating-overhead.test.ts tests/runner-e2e/daytona-image.test.ts tests/runner-e2e/scheduled-operations.test.ts tests/runner-e2e/cost-ledger.test.ts`: 55 passed. - `pnpm test:e2e:runner:typecheck`: passed. - `pnpm verify`: exit 2 (README is a global CI input). Required CI supplies broad coverage. - `tests/runner-e2e/workflow-security.test.ts` has 5 local failures on files that this PR does not touch (`pr-trusted.yml`, `chat-catalog-testops-case.yml`). They are not caused by this change. - I did not run a Docker build or a paid campaign. ## Risks - Low risk. Any external script that builds this Dockerfile without `PAPERCLIP_RUNNER_LOCK_SHA256` now fails with a clear message. Before, it failed with a checksum mismatch whenever the lockfile changed. Both CI build paths already pass the argument. - The Dockerfile is an image content-ID input, so the next image gets a new content tag. Current master needs a new image anyway. - The nightly workflow stays red until an operator reviews and renews `RUNNER_E2E_OPERATING_OVERHEAD_JSON`. This PR does not renew the allowance, because that is a reviewed spend decision. - Scheduled runs set `needs_daytona=false`, so the nightly run does not publish a Daytona image even when the catalog passes. A current image comes from a manual dispatch that selects Daytona cells. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, GitHub CLI). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandClaude Opus 5.5
|
d87f6fa4c2 |
test(runner-e2e): add staging-soak reliability cases to everyday workflows (#19455)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The paid runner E2E suite (`tests/runner-e2e`) drives the real browser, server, native runner, provider and Daytona > - A manual staging soak found about nine reliability bugs (S47–S55, H2, U9). Each fix got unit or integration tests > - The end-to-end journeys that exposed the bugs existed only as ad-hoc browser scripts, so a regression could return without a signal > - These journeys are expensive, so they must not run per PR or in `--all`. They must run on demand before a release > - This pull request adds the journeys to the existing `everyday-workflows` suite as an explicit `reliability` group > - The benefit is one command that runs the whole reliability set before a release or after a risky runner change ## Linked Issues or Issue Description **What is the problem?** The staging-soak scenarios that found S47–S55, H2 and U9 are not repeatable. They were ad-hoc browser scripts on staging. **Why does it matter?** Without a repeatable end-to-end case, a reliability regression (stalled follow-ups, duplicate execution, dead Retry, hangs after silent commands) can return without a signal. **What is the proposed solution?** Add the scenarios to the paid `everyday-workflows` suite as explicit-only cases with independent graders and a selectable `reliability` group. Refs #19370, #19355, #19354, #19407, #19419, #19394, #19421, #19431 (the related fixes). ## What Changed - `everyday-reliability-cases.ts`: seven new cases (`interrupt-redirect`, `reassign-midrun`, `three-turn-revise`, `long-silent-command`, `tree-pause-release`, `queue-steer`, `queued-across-restart`), their soak signatures, prompts, bounds and the explicit matrix. - `everyday-reliability-flow.ts`: browser and public-API drivers for each journey. A shared watcher records `executionBlocker`, `interruption` and execution projections while it waits. It uses an offered Retry at most once (never while an automatic retry is scheduled). - `everyday-reliability.ts`: pure graders. They encode the product rule: interrupted work resumes automatically or offers a Retry that works on first use. A send-a-message blocker, a dead or repeated Retry, a silent stall, or duplicate execution fails. - `types.ts` / `selectors.ts`: new optional `explicitGroup` on a task. Such a task is selected only by its group, `--case`, or an exact `--id`. `--all`, the plain suite and generic selectors skip it. `--list` still shows it. - `catalog.ts`: the suite gets the reliability tasks, the released Grok/Kimi/Pi facades (only for `long-silent-command`), a per-turn Daytona override for `three-turn-revise`, the `reliability` and `slow` groups, definition version 11, and a computed `expectedMatrixSize` (52 → 62). - `everyday-flow.ts`: a narrow hook calls the scenario driver after task creation. Riley is created for the reassign and tree cases. The reliability checks are added after settlement. - `everyday-observations.ts`: an allowed interrupted run may end `operator_interrupted` or `issue_reassigned`. Lifecycle checks accept a declared reassigned parent owner. - `live-fixtures.ts`: reliability cells get the existing 1,000-cent company and lead-agent hard stop. - Tests: new `everyday-reliability.test.ts` (selection exclusion, matrix, credentials, and calibrated pass/fail cases for every grader). Updated counts in `catalog.test.ts`, `everyday.test.ts` and `installed-cli.test.ts`. - Docs: new "Staging-soak reliability group" section in `EVERYDAY-WORKFLOWS.md` (cases, signatures, run commands, cadence, coverage boundary) and a link from `doc/evals.md`. `everyday-flow.ts` was already over 1,000 lines. This PR adds only about 20 lines of wiring there. The scenario logic is in the new modules. Matrix (10 cells, all Daytona, single attempt): one `runner-codex` (gpt-5.6-sol) cell per case, plus `runner-acpx-grok`, `runner-acpx-kimi` and `runner-acpx-pi` for `long-silent-command`. Run command: ```sh pnpm test:e2e:runner -- --suite everyday-workflows --group reliability --max-parallel 2 ``` ## Verification - `pnpm test:e2e:runner:typecheck` passes. - `node node_modules/vitest/vitest.mjs run --config tests/runner-e2e/vitest.config.ts everyday catalog installed-cli selectors support.test`: 15 files, 362 tests pass. - The full runner-e2e Vitest run has the same 14 failures with and without this change (`controller-outage`, `operating-receipts`, `pi-controls-evidence`, `pi-file-evidence`, `workflow-security`). These fail on the base commit in this local environment too. - `pnpm test:e2e:runner -- --list --suite everyday-workflows --group reliability` lists exactly the ten cells. - `--all`, `--suite everyday-workflows` (still 52 cells), profile/environment selectors and the daily/weekly scheduled selections contain no reliability cell. A test asserts this. - Not run: the paid cells. They need provider and Daytona credentials. The first paid run of the group is the acceptance step. ## Risks - The drivers have not run against a live stack. A first paid run can expose harness timing or selector issues (for example, how fast the queued Interrupt control appears). The failure then names the stalled step and keeps `reliability-observations.json`. - `long-silent-command` does not inject a runner connection loss. The named-cause contract is checked only if a loss happens naturally. - The Grok/Kimi/Pi facades run as ordinary released profiles. If one returns to pending qualification, it drops out of the group and the matrix size follows. - No product code changes. Existing everyday cells keep their selection and checks. The suite definition hash changes because the suite definition changed. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edit, code execution). Standard context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
github-actions[bot]andlockfile-bot
|
523818ea9b |
chore(lockfile): refresh pnpm-lock.yaml (#19443)
## Thinking Path > - Paperclip uses a dependency lock for reproducible builds. > - Dependency changes on master need matching lock entries. > - Paid eval reports also pin a reviewed offline correction. > - Reports must install before paid model work starts. > - This automated PR keeps the lock and report proof together. ## Linked Issues or Issue Description **What happened?** Dependency manifests on master changed after the last committed lock refresh. **Expected behavior** Frozen installs must match those manifests. Eval reports must retain matching offline lock proof. **Steps to reproduce** Run a frozen install before and after this lock refresh. ## What Changed - Refresh the dependency lock from the manifests on master. - Include the matching offline report correction and hashes. ## Verification - Pinned pnpm resolution completed with lifecycle scripts disabled. - The report proof refresh accepted the reviewed committed baseline. - The workflow rejected changes outside the lock and report proof files. ## Risks The lockfile bot merges this PR without CI or review when every added package version is at least 3 days old on npm. Otherwise the PR waits for a maintainer, and the daily refresh tries again. The bot merges only the exact commit this workflow generated from manifests already reviewed on master. Unknown report lock baselines remain blocked. This update changes no permissions or paid enable gates. ## Model Used None — generated by pnpm and the deterministic report-lock updater. ## Checklist - [x] Thinking path and problem description included - [x] Generated lock and matching report proof included - [x] Release-age gate decides between bot merge and maintainer review Co-authored-by: lockfile-bot <lockfile-bot@users.noreply.github.com> |
||
|
DottaandPaperclip
|
8836a71113 |
test(testops): pin calibrated secret acknowledgment source (#19428)
## Thinking Path > - Paperclip lets people manage agents and inspect saved work. > - Protected chat evaluations use a fixed source and reviewed case declarations. > - The last four-case run produced one verified PASS and three original FAILURES. > - Review found narrow acknowledgment and partial-installation assertion gaps. > - Their local repairs preserve the original user goals and safety evidence. > - This PR pins the reviewed source for one bounded secret-pair measurement. > - A final uniform-ten measurement on that same source is still required. ## Linked Issues or Issue Description Refs: #19423, #19325, #19416, #19093, #19084, #19112. The core and full-catalog PRs remain open and unmerged. **What happened?** Launch 3310 retains one original PASS and three original FAILURES. Case 53 used the same undismissed form correctly. Company rejected a valid ordered save/ACK card before human entry. Personal rejected a canonical wake-on-accept/After Saving confirmation before private entry. Team rejected a truthful qualified required-skill partial-installation report. No original result is regraded. **What did you expect to happen?** Admit supported saved cards and truthful partial reports while preserving actual current-human save, immutable policy, acceptance, audit, continuation, private ownership, delivery and saved-resource evidence. Declined or canceled actions and false recovery claims must still fail. **Steps to reproduce** Inspect the original selected journeys from source `5f74dc6545ea66ac71255ac8e6010f5215a1f257` against their actual cards, responses and saved state. The local repairs calibrate the assertions; they do not establish that the next native secret journeys will finish within their original clocks. ## What Changed - Pin immutable source `d68c00a3d78e5cd56b77aedb31f1e8a0f6d6f84e` and its exact 242-row descriptor snapshot. Update only Company v17, Personal v16 and Team v10 oracle-version leaves in the 244-row manifest and retained repair metadata. - Append the measured limits after the exact historical condition prefix. Keep every runtime condition and dependency list unchanged. - Keep all 244 YAML identities, queries, contexts, outcomes, permissions, human/native bounds, clocks and model unchanged. Cap 2 and zero retries stay fixed. - Plan only cases 578 and 580 next, with original 480,000 ms and 660,000 ms cells. Team calibration is not a new native PASS. Final uniform ten remains required. ## Verification - All 40 local pin controls passed: `node --test tests/runner-e2e/chat-full-catalog-testops-manifest.test.mjs tests/runner-e2e/chat-full-catalog-testops-adapter.test.mjs tests/runner-e2e/chat-full-catalog-testops-reconcile.test.mjs tests/runner-e2e/chat-completion-testops-cli.test.mjs`. - The exact source passed 352 fresh controls across ten files. These cover current confirmation, independent human accounting/history, contract, Team and all 242 declarations with zero fixture callbacks. Full runner E2E types passed. - SDK506 synchronization passed unchanged with registry digest `efa70621e9fb42aa185f7ee616a1a726aa3eccb367e67e28817cedf26ef2ade8`. No install, native build, SDK build or local model call was needed. - Fresh master `7e3caa7bac3f2c1a560158b28ec42381a153dfff` matches the accepted ten protected helpers, five source-parity files, 67 dependency-graph inputs and report-lock policy. No private helper imports ran during this preparation. - Local qualification used Node `v26.4.0`. An initial pin test import failed before the qualified dependency link existed; that setup failure remains retained. - Remote ref `codex/chat-secret-ack-v17-qualified-d68c00a3-20261106` was read back at the exact source above. Descriptor SHA256 is `3b62889da60db1c8daafcc23f0fefa7cfcda165224ba90214b2ac391215d3be0`. - New-head CI and reviews remain pending at PR creation. No paid evaluation ran here. ## Risks This source is unmeasured. Company used 344,242 ms for its initial native turn and had only 86,070 ms left at the false gate. Measured API handlers do not explain that time; no avoidable latency defect or full-flow fit has been proved. The gate repair does not create a native completion or private-save claim. Personal must preserve the actual canonical card policy through real save, acceptance and independent evidence. Team's local assertion calibration does not regrade result 5242 or qualify runtime migration. Every saved-state, actor, privacy and delivery check remains mandatory. Case 53's verified 241,214 ms / 300,000 ms journey is carried on unchanged code, not claimed as a new measurement or reliability result. All older failures remain retained. Secret journeys require independent semantic safety review before pass credit. Core and full-catalog PRs stay unmerged; the 196/244 gate remains. Only the secret pair is planned next, followed by the required uniform ten on the same source. No merge override or retry is requested by this PR. ## Model Used OpenAI `gpt-6.1-sol` through Codex, with reasoning, tool use and code execution. The context-window limit is not exposed in this session. The native eval model remains `openrouter/deepseek/deepseek-v4.1-flash-1017` and was not run locally. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b400914345 |
test(testops): pin calibrated four-journey fixture source (#19423)
## Thinking Path > - Paperclip lets people manage AI agents and inspect saved work. > - The protected chat evaluation uses a fixed source and a reviewed case inventory. > - The last four selected journeys all reported FAILED. Their originals remain unchanged. > - Review found one fixture attention-state mutation and three narrow observation or assertion errors. > - The source repairs passed model-free controls, but their next native measurement has not run. > - This PR pins that reviewed source without merging its core or full-catalog overlays. > - The benefit is one bounded measurement with honest conditions and unchanged safety evidence. ## Linked Issues or Issue Description Refs: #19418, #19325, #19416. The last two core PRs remain open and unmerged. The older open source pin #18740 targets a different historical candidate. **What happened?** The multipart fixture dismissed a saved form before asking for its walkthrough. The Company guard rejected a bounded secure-page save instruction. The Personal route prefilter rejected a valid task citation beside the setup hint. The Team assertion rejected a truthful current unattached-skill report. Launch 3277 retains zero reported passes from four cases; none is regraded here. **What did you expect to happen?** Measure the same user goals with the original form still undismissed, supported secure-entry directions, independently validated task links, and truthful partial installation reports. Require all real saved-state, actor and privacy evidence. **Steps to reproduce** Run the protected selected journeys from source `58449c4ecd7fd374f21f175411ef3249f55c18a3`. Inspect each original failure against its user-visible result and saved state. The new source is a local calibration, not a claim that these native journeys now pass. ## What Changed - Pin immutable source `5f74dc6545ea66ac71255ac8e6010f5215a1f257`. - Copy its exact 242-row descriptor snapshot. Update eight oracle-version leaves in the 244-row manifest and retained repair metadata. Dependency lists stay fixed. - Update the expected target assertion and append the new limits after the exact historical condition prefix. Retain all existing runtime conditions. - Keep all 244 YAML identities, queries, outcomes, human/native bounds, clocks, model, cap 2, zero retries and trusted private interfaces unchanged. ## Verification - All 40 local manifest, adapter, reconciliation and CLI admission controls passed: `node --test tests/runner-e2e/chat-full-catalog-testops-manifest.test.mjs tests/runner-e2e/chat-full-catalog-testops-adapter.test.mjs tests/runner-e2e/chat-full-catalog-testops-reconcile.test.mjs tests/runner-e2e/chat-completion-testops-cli.test.mjs`. - The exact source passed 375 affected controls across 20 files, including explicit installed-browser canaries and the 242-declaration gate with zero fixture callbacks. Full runner E2E typecheck passed. These component results are not native passes. - The 506-input SDK guard passed without refresh. The launcher, ten protected helpers, five source/controller parity files and 67 dependency-graph inputs match the accepted baseline. No install or build was needed. - Local qualification used Node `v26.4.0`. Earlier alias and declaration-output setup failures remain retained. The normal remote CI configuration is unchanged. - Verify remote ref `codex/chat-four-fixture-v16-qualified-5f74dc65-20261106` resolves to the source above. The snapshot SHA256 is `913090f18cc4e0f7d997b99586618080f1b5c847b2b5d74984d3c480baf5318d`. - New-head CI and reviews must finish before merge. No paid evaluation ran for this PR. ## Risks The source remains unmeasured. Removing the fixture dismissal does not guarantee that the agent will reuse the form: a later human comment still makes it historical, but it remains answerable and the latest human direction governs. The same-form fidelity check stays strict. The Company journey did not reach human secure entry; its later save, approval, binding and consumer delivery remain unproved. Personal setup hints are not trusted URLs; actual navigation and private ownership checks remain required. Team calibration gives no runtime-migration credit. All original failures remain retained. Secret journeys still require independent semantic safety review before pass credit. No native reliability or latency claim is made. The core and full-catalog PRs remain unmerged; the 196/244 gate stays fixed. A later full-ten run is a separate measurement. Required CI and genuine current-head review are pending at creation; no merge override is requested. ## Model Used OpenAI `gpt-6.1-sol` through Codex, with reasoning, tool use and code execution. The context-window limit is not exposed in this session. Native eval configuration remains `openrouter/deepseek/deepseek-v4.1-flash-1017` and was not run locally. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
4d49a31286 |
test(testops): pin reviewed four-case recovery source (#19418)
## Thinking Path > - Paperclip manages AI agents and their work. > - The protected TestOps workflow evaluates a fixed source commit. > - The last five-case attempt retained two passes and three failures. > - Reviewed fixture changes address observed admission and reporting gaps. > - Product denial guidance and closed diagnostics remain candidate overlays. > - This PR updates only the immutable source pin and its public metadata snapshot. > - A bounded four-case measurement can test this candidate without merging catalog or core code. ## Linked Issues or Issue Description Refs #19325, #19416, #19395. **What happened?** The current pin targets the prior five-case source. The next reviewed source changes question fidelity context, Personal menu observation, Company secure-save acknowledgment admission, and Team partial-install reporting. The last attempt's original failures remain unchanged. **Expected behavior** The workflow must admit the exact reviewed source and matching descriptors. It must preserve the selected model, native and human bounds, clocks, credentials, and zero automatic retries. **Steps to reproduce** Compare the current manifest and descriptor snapshot with source `58449c4ecd7fd374f21f175411ef3249f55c18a3`. Run the local manifest, adapter, reconciliation, and CLI admission controls listed below. **Paperclip version or commit** Controller base `90b8190c33e8f8eae44cae1f57721251fe1e1343`; candidate source `58449c4ecd7fd374f21f175411ef3249f55c18a3`. **Deployment mode** Protected AWS Product E2E through the existing TestOps workflow. ## What Changed - Pin the reviewed immutable source and exact descriptor snapshot. - Update four existing oracle-version leaves and Company/Personal source-file provenance. - Append current conditions while preserving the complete historical conditions prefix. - Retain all 244 case identities, YAML requests and outcomes, counts, model, clocks, runtime conditions and policy gates. - Change one expected target literal in the manifest test. No workflow, lock, runtime, SDK or credential changes. ## Verification - 40 local controls pass: `node --test tests/runner-e2e/chat-full-catalog-testops-manifest.test.mjs tests/runner-e2e/chat-full-catalog-testops-adapter.test.mjs tests/runner-e2e/chat-full-catalog-testops-reconcile.test.mjs tests/runner-e2e/chat-completion-testops-cli.test.mjs` - Parsed all 244 rows. Only the four oracle versions and two source-file arrays differ. Runtime conditions are identical. The candidate snapshot matches committed source bytes. - The source qualification includes focused positive/negative controls, full runner E2E types, and unchanged SDK506 registry `efa70621e9fb42aa185f7ee616a1a726aa3eccb367e67e28817cedf26ef2ade8`. Counts from overlapping component sets are not summed. - All 67 controller dependency graph inputs match the accepted frozen-install proof. No install, SDK build or runtime build was repeated. - The existing selector admits exactly cases 53, 578, 580 and 610 in the native lane with cap 2. No workflow or settings change is needed. This local admission control did not submit a run. - Full repository build/typecheck/test commands were not repeated for this metadata-only change. Normal PR CI and fresh review remain required. ## Risks - The candidate has no new native PASS. Launch 3244 retains two original PASSES and three original FAILURES. Older failed and missing-original results remain unchanged. - Case 53 changes factual setup context to identify the existing visible form and adds stricter request fidelity. Its YAML query and original clock are fixed. This is not a latency fix. - Personal v14 treats the inline code path as an imprecise hint. Actual current-company UI links must establish navigation before private entry. Save, owner isolation, consumer delivery and final confirmation remain separate required proofs. - Team v8 fixes truthful partial-report admission. It does not qualify end-to-end runtime migration. - Company v15 accepts completed-entry acknowledgment questions. Actual current-human save, payload, source, continuation and governed-approval proofs remain mandatory. The corrected 403 handoff guidance is a recovery hypothesis. Closed diagnostics do not prove a native repair. - Pinned OpenCode model-free tests preserve the tested finish types in both directions. They do not prove real provider behavior or engine wait authority. - The next measurement selects only 53/578/580/610 once, with native cap 2, zero automatic retries and original bounds. A later uniform ten remains separately unrun. The unchanged 585 pass is carried-forward evidence. - Core #19325 and #19416 must remain open and unmerged. This PR does not merge the held full catalog or change its 196/244 gate. A maintainer will perform any normal merge and actual measurement after required checks. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`), with reasoning, code execution and repository tools. The host did not report a context-window size. No model inference was used for local fixture controls. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b918ee165d |
test(evals): pin coherent chat feature follow-ups (#19411)
## Thinking Path > - Paperclip manages agent work and its permission boundaries. > - The chat catalog runs feature checks from an immutable source. > - The last full-ten attempt had five reported passes, four failures and one missing original. > - Local evidence identified narrow assertion, setup and evidence-retention repairs. > - This PR updates the trusted source pin and its exact descriptor snapshot. > - The next five-case measurement can test the affected features under the same model, clocks and native cap. ## Linked Issues or Issue Description Refs: #19395, #19405, #19093, #19084, #19112. The previous source remains preserved. This PR selects reviewed source `f5692c57dafaf9afbef479c25fcd277815932516` on `codex/chat-full-ten-coherent-qualified-f5692c57-20261106`, tree `b9e302d2ea7b605a60125dec8c81672eac7a3c57`. The full catalog and all core PRs remain unmerged. Core #19405 is included only as a source overlay. ## What Changed - Update the source target, exact descriptor snapshot and the matching pin checks. Preserve historical conditions, frozen source anchors and the prior query clarification. - Record Company ACK handling and Company/Personal negation corrections, plus the skill-update preview negation correction. A denied or unapplied action still fails when the agent claims it completed. - Disclose the Team setup change: a disposable catalog package selects the approved OpenCode runtime and same-company account. Other file bytes and non-runtime sidecar fields stay unchanged. Original and effective package identities and hashes are recorded. A real PostgreSQL install without caller overrides proves the saved runtime. This gives no end-to-end runtime-migration or override-handling credit. - Include defensive failed-turn evidence retirement and generic, schema-valid human-input examples. The old case 53 missing-original ENOENT cause is not diagnosed. The examples are a usability hypothesis, not a schema repair or native-success claim. All 244 YAML definition fields (query, assumed context and expected outcome) remain byte-identical versus source 936c. The Team derived runtime fixture and task setup context change is disclosed above. Native/human bounds, clocks, DeepSeek 1017, credentials, SDK506/efa, launcher, private interfaces, native cap two and zero automatic retries remain unchanged. Only descriptor `sourceFiles` and oracle versions change. The 196/244 gate stays unchanged. ## Verification Pin-local checks: ```sh node --test tests/runner-e2e/chat-full-catalog-testops-manifest.test.mjs tests/runner-e2e/chat-full-catalog-testops-adapter.test.mjs tests/runner-e2e/chat-completion-testops-contract.test.mjs ``` - All 49 pin/admission controls pass locally. They join all 244 identities, historical query overrides, native bounds and the exact 242-row descriptor snapshot. - Source qualification: 447 controls across 32 suites, full E2E TypeScript checks and normal contract generators pass. The separate 21 core contract controls are carried-forward component evidence; they are not added to 447. Local source checks used Node 26.4.0. AWS runtime settings are unchanged. - The selected source descriptor SHA256 is `24fe88af167724c6fb120d3984296aab47c993a65782158e03d80a936fbd94ed`. - The actual source launcher is byte-identical to 936c: SHA256 `2b9a6953ee51f0ea0dbbd475b0491d9a6c39a1d6835eeaa912e0dbb7cf987724`, Git blob `c88df15f509b2ac8638af826647a97d45f16bd1f`. - All 506 registered SDK inputs and the ten trusted helper bytes are unchanged. Five required source-helper joins pass. No SDK renewal or native build is part of this PR. - The current controller's 67 dependency inputs match the qualified frozen reporting graph. No reporting-lock, package, permission or workflow change is included. - Exact head `b833036fb477b55b05cbdf1f4feb2e2c5387b420` is green: CI run 42166216108 attempt 2 succeeded, Windows succeeded, and both current reviews are 5/5 with zero unresolved threads. The check rollup has 56 successes, three configured skips and one neutral result. New representative cases 53, 578, 580, 585 and 610 are unrun. A later uniform-ten regression is separate. - CI attempt 1 remains retained: one cloud-auth browser fixture read a trailing health response after context teardown began, after all task-recovery assertions had passed. One authorized rerun of only that failed job and its dependent checks succeeded on the unchanged head. This does not fix or qualify fixture reliability; no eval or model run was retried. The previous full-ten outcome remains five reported passes, four failures and one missing-original/reporting gap. Case 585 reached only its first preview before the false rejection; its update phase is unqualified. Case 610's old noncompletion and runtime-override journey remain unresolved. All original failures and evidence remain retained. No old result is regraded or replaced. ## Risks The local repairs may not produce a native pass. The new Team setup tests installation with an already selected runtime package; it does not qualify runtime migration. Failed-turn retirement has local regression coverage but does not establish the cause of the old case 53 reporting gap. The core guidance overlay remains unmerged and its native benefit is unproved. Normal protected merge and a new authenticated measurement are still required. ## Model Used OpenAI `gpt-6.1-sol` through Codex. It used code inspection, local test execution and GitHub tooling. The context window size is not exposed. No model inference or provider call was made for this PR's local checks. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
d8b0eae687 |
test(evals): pin completed timeline command account for full-ten measurement (#19395)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The chat catalog needs fixed source and explicit conditions for each measurement. > - The latest timeline run delivered a complete account within its original deadline. > - Its assertion missed ordinary exitCode notation and a command with an omitted subject inside a named worker bullet. > - The reviewed source corrects those bounded textual checks while retaining independent history and command proof. > - This pull request pins that immutable source for one full-ten measurement. > - Original failed grades and unresolved Company timing remain visible. ## Linked Issues or Issue Description Refs: #19392, #19086, #19388. This follows the prior source pin. The latest timeline response correctly described requirements, a real failed command, saved pause, human reassignment, repair and successful verification. Its original grade still failed one narrative assertion. This pin carries the reviewed correction without merging the full catalog or its production overlays. ## What Changed - Pin source `936c3bbe0e83ca15616254f23d9023b2065fc5fb`, tree `e5c64d67935497164b098e31e995f87ba7875d06`. - Advance only the timeline oracle from v11 to v12 in the candidate snapshot and manifest. - Retain all 244 current queries, contexts, expected outcomes, model, credentials, clocks and human/native accounting. - Record one planned full-ten bounded measurement at native cap 2 and zero automatic retries. - Change only the manifest, its target assertion and the exact candidate descriptor snapshot. ## Verification - I verified the immutable source branch readback before I published this pin. - The new narrative control fails on the parent. The fixed source passes 300 timeline, native-proof and declaration controls, including all 242 lazy declarations with zero fixture callbacks. - The response from result 5094 is accepted by the local diagnostic. The original grade is retained as FAILED; this is not a new native PASS. - Fresh full runner E2E types and the SDK guard over 506 inputs pass in the qualified source graph. An earlier isolated worktree type attempt had dependency-alias errors; the clean source qualification supersedes that setup gap. - Controller: `node --test tests/runner-e2e/chat-full-catalog-testops-manifest.test.mjs tests/runner-e2e/chat-full-catalog-testops-adapter.test.mjs tests/runner-e2e/chat-full-catalog-testops-reconcile.test.mjs` — 40 controls pass. - The snapshot equals the committed source descriptor bytes. All 244 rows compare equal to the prior pin except the timeline oracle version. - Source descriptor SHA256: `19185aa5298c93d159c3d8547661fd6ef22185e74be1c18b8497bae2ef8051c7`. - Controller manifest SHA256: `822494b170139a8204ff36b33891e2f4960ac0aba6800e9856f2d25dd24de7c6`. - Launcher, protected evidence wire and the accepted 506 SDK inputs remain unchanged. The source SDK registry stays `efa70621e9fb42aa185f7ee616a1a726aa3eccb367e67e28817cedf26ef2ade8`. - Exact pin head `31f47fcdfb600e6b15b35ed50c2a447d03681f06` has 56 successful checks and three configured skips. Both protected checks pass. Greptile is 5/5 for this head with zero open threads. Paperclip Review completed neutral at 5/5. GitHub reports CLEAN and MERGEABLE. These checks do not qualify the unrun native experiment. ## Risks - Original timeline result 5094 remains FAILED. All three native runs succeeded and the account was complete in 248,876 ms, but the original narrative check failed. Local calibration does not rewrite that grade. - Original Company result 5093 ended without first-CEO completion. Its model/runtime timing cause remains unresolved. This source makes no Company runtime or latency fix claim. - The full-ten measurement is unrun. It is a bounded experiment, not proof of deterministic timing, reliability or an all-ten pass. - Company retains two base human inputs plus at most one independently verified UI approval, minimum two and maximum three native runs, and 480,000 ms. Timeline retains three native runs, one human input and 720,000 ms. No condition is waived. - Secret journeys still require independent semantic safety review. Metadata shape alone does not establish that a proposed name is non-sensitive. - The interaction-accept and API-search overlays remain separate, reviewed, unmerged core PRs. This pin does not merge them or the full catalog. The 196/244 admission gate stays fixed. ## Model Used OpenAI Codex, GPT-6.1 Sol (`gpt-6.1-sol`), with reasoning, repository tools and code execution. The host does not expose an exact context-window size. No model or paid provider was called for local controls. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
7c04adb2d6 |
test(evals): pin company environment and API search repairs (#19392)
## Thinking Path > - Paperclip manages AI agents and their governed actions. > - The TestOps controller runs an immutable product-eval source. > - The last company-secret attempt rejected a safe metadata question before private entry. > - A governed approval also had a redundant server notification and wake path. > - Natural-language API search could hide the resource needed for a timeline answer. > - This pull request pins the reviewed source and its exact descriptor snapshot. > - New native evidence must still prove the affected journeys. ## Linked Issues or Issue Description Refs #19381, #19093, #19387, #19388. This is the next controller pin for those reviewed changes. The core pull requests remain unmerged. ## What Changed - Pin source `2c99c248e839f41b520fc2d7e3702169ac6cb0a2` and its exact descriptor snapshot. - Advance company oracle v12 to v13. Admit bounded environment-name questions without a required authored pattern or exact prompt template. Preserve independent human-answer validation. - Include the reviewed interaction-accept notification and API-search ranking overlays. Record their route and catalog dependencies in the company and timeline provenance. - Retain the existing cap of two, zero retries, DeepSeek1017 model, and all 244 current YAML definitions. Company keeps 480000ms, native2..3, base human2 plus one independently proved approval. Timeline keeps720000ms, native3/human1. ## Verification - Source:396 controls in9 suites,2046 API catalog controls,145 route controls, full runner TypeScript check, and unchanged SDK506 input guard passed. Fresh-master API PR has2059 controls; its other baseline tests are absent from this catalog source. - Controller:133 contract, API, adapter, reconcile, streaming and manifest controls passed. Existing report-lock preparation returned `already-reviewed`. - All current YAML definitions and launcher bytes remain unchanged from the prior source. No model or native eval ran for this pin. - The original company5059 and timeline5060 remain failed. Company private entry, approval and final binding remain unproved on this successor. - Exact-head normal checks are terminal:56 success,3 intentional skips, and one neutral Paperclip review. Greptile and Paperclip both report5/5 with no findings; review threads are empty. The next TestOps/AWS run follows the protected controller merge. ## Risks The new source is not a native feature pass. Its effect on model completion is unproved. The original480000ms company budget may still be too short for three native turns. No budget change or count waiver is included. The full catalog remains outside master and its existing admission gate remains unchanged. The controller reuses byte-identical SDK506 evidence; no compiler or registry renewal is claimed. Actual post-merge helper imports still need qualification against the merged controller lock. The CI virtual merge includes a newer upstream lock than the local frozen dependency proof. ## Model Used OpenAI `gpt-6.1-sol` through Codex, with repository tools and code execution. The host did not report its context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented risks above - [x] All Paperclip CI gates are green - [x] Greptile is5/5 with no open findings - [x] I will address reviewer comments before requesting merge Co-authored-by: Paperclip <paperclip@paperclip.ing> |
||
|
github-actions[bot]andlockfile-bot
|
675f7283dd |
chore(lockfile): refresh pnpm-lock.yaml (#19389)
## Thinking Path > - Paperclip uses a dependency lock for reproducible builds. > - Dependency changes on master need matching lock entries. > - Paid eval reports also pin a reviewed offline correction. > - Reports must install before paid model work starts. > - This automated PR keeps the lock and report proof together. ## Linked Issues or Issue Description **What happened?** Dependency manifests on master changed after the last committed lock refresh. **Expected behavior** Frozen installs must match those manifests. Eval reports must retain matching offline lock proof. **Steps to reproduce** Run a frozen install before and after this lock refresh. ## What Changed - Refresh the dependency lock from the manifests on master. - Include the matching offline report correction and hashes. ## Verification - Pinned pnpm resolution completed with lifecycle scripts disabled. - The report proof refresh accepted the reviewed committed baseline. - The workflow rejected changes outside the lock and report proof files. ## Risks The lockfile bot merges this PR without CI or review when every added package version is at least 3 days old on npm. Otherwise the PR waits for a maintainer, and the daily refresh tries again. The bot merges only the exact commit this workflow generated from manifests already reviewed on master. Unknown report lock baselines remain blocked. This update changes no permissions or paid enable gates. ## Model Used None — generated by pnpm and the deterministic report-lock updater. ## Checklist - [x] Thinking path and problem description included - [x] Generated lock and matching report proof included - [x] Release-age gate decides between bot merge and maintainer review Co-authored-by: lockfile-bot <lockfile-bot@users.noreply.github.com> |
||
|
DottaandPaperclip
|
46414fcfa4 |
test(evals): pin timeline and governed Company approval at native cap two (#19381)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The chat catalog uses a fixed source and explicit execution conditions. > - A correct timeline response failed because its completed edit and attached command result were not recognized. > - Company credential setup can require a real human approval after the secure save. > - The controller must declare that optional approval and its run bounds before it admits the source. > - This pull request pins the reviewed source and caps native catalog overlap at two. > - The next run can test these conditions without treating local controls as native acceptance. ## Linked Issues or Issue Description Refs: #19357, #19086, #18971, #19093. This follows the prior fixed-source catalog pin. It changes only controller metadata, admission, tests, and load policy. The complete catalog and separate core feature PRs remain unmerged. ## What Changed - Pin immutable source `fe547be84826b4611d3ba9ee1a7d7aa0bd0a2843`, tree `a23723ea5244e6ce8cee11e9c4b476acb6978fcd`. - Advance only timeline v10 to v11 and Company v11 to v12. Add the declared Company proposal helper to its source closure. - Declare Company minimum two and maximum three native runs. Retain two base human inputs and permit at most one independently verified UI approval. Retain its original 480000ms attempt. - Carry the literal Company-only approval marker from the descriptor into the manifest. Reject foreign markers, changed bounds, and a mismatch with static source task metadata. - Cap native catalog execution at two workers. Keep the external lane serial and retain zero retries. - Keep all244 requests, contexts, expected outcomes, model, clocks, credentials and other case bounds unchanged. ## Verification - Source:421 focused controls across seven suites pass. Full runner E2E TypeScript check passes. - Explicit model-free approval UI canary passes one test with three local scenarios. It checks accepted approval, wrong human, and failed response. The first sandbox attempt stopped at Chrome launch; the allowed execution passed. - Controller:40 manifest, adapter and reconciliation controls pass. - SDK506 inputs remain byte-identical to the accepted registry `efa70621e9fb42aa185f7ee616a1a726aa3eccb367e67e28817cedf26ef2ade8`. - Launcher and trusted private evidence wire remain byte-identical. Source contract/flow accounting changes are explicit. - Source descriptor SHA256: `50e90d8c02ad57f444e27b522efe694db9734d47bdbc3a058475af77bac8b2ca`. - Controller manifest SHA256: `0447315601ed21ba15d6ea0ecd7fb89bc79ad6262db2594cd12ac774aa8d6c21`. - Review the two changed rows. The next planned experiment selects timeline and Company once at cap2 before a full ten-case run. No native run on this source has passed yet. - Exact controller head `d49dfc59ab7fddfe1a1de7e81bdb9c4e13400f30`: normal CI has56 successful checks and three configured skips. Protected `ci / verify` and `ci / e2e` both pass. Greptile5/5 and zero open threads are verified for this head. ## Risks - Original timeline5056 and Company5057 remain FAILED. Timeline5056 supplied the correct account; the local revised check passes its diagnostic only. Company5057 saved the secret but left the binding approval pending. - Company native3 completion within480s is unproved. This change does not raise its clock or claim to fix latency. - Two-worker execution is a changed load condition. It does not prove a concurrency cause, reliability, or eight-worker qualification. - Secret journeys require independent semantic safety review before pass credit. The documented closing-comma warning and arbitrary metadata-prefill classifier limitations remain follow-ups in the functional secret PR; a native failure is not waived. - Candidate core PRs remain unmerged. The full-catalog196/244 admission gate remains fixed. Local controls do not qualify all244 journeys. - Local checks use Node26.4.0. They do not replace actual AWS runtime qualification. - The separate Paperclip Review app remains queued while its assigned task waits for maintenance. It has no completed assessment. GitHub reports `UNSTABLE` and `MERGEABLE`; this is not an all-checks PASS claim. No review retry or staging mutation was performed. ## Model Used OpenAI Codex, GPT-6.1 Sol (`gpt-6.1-sol`), with reasoning, repository tools and code execution. The host does not expose an exact context-window size. No model or paid provider was called for local controls. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandClaude Opus 5.5
|
3a8f95d928 |
fix(ci): let verified Dependabot lockfiles merge through the reporting-lock gate (#19356)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Only trusted environments may change `pnpm-lock.yaml`: the lockfile refresh workflow and Dependabot. Contributors never commit it. > - Paid evaluation jobs also install a lock only when the reporting-lock proof (`tests/runner-e2e/report-lock.json` and `.patch`) admits it. > - #19283 added a CI test that checks the committed lock against that proof. Only the refresh workflow writes the proof, so every Dependabot lockfile pull request now fails CI. Examples are #17364, #17172, and #17911. > - A merged Dependabot lockfile also breaks the refresh workflow. `prepare-report-lock.mjs --refresh` refuses to renew the proof for a lock that the proof does not already admit. > - This pull request lets a verified Dependabot lockfile pass, and it keeps both lockfile rules strict. It also closes a gap in the contributor rule. > - The benefit is that Dependabot security updates can merge again, and only trusted environments write lock content. ## Linked Issues or Issue Description Refs #19283, #17364, #17172, #17911, #19303. **What happened?** `server/scripts/reporting-lock-policy.test.mjs` fails on each Dependabot pull request that changes `pnpm-lock.yaml`. The error is "Reporting lock changed; review the dependency correction before install". A Dependabot pull request cannot change the proof files, so it can never pass. After a merge, the lockfile refresh also stops at "Committed report lock is not reviewed; refusing to refresh its proof". **Expected behavior** - Contributors cannot change `pnpm-lock.yaml` or the reporting-lock proof. - Dependabot can change `pnpm-lock.yaml`. - The lockfile refresh workflow can change `pnpm-lock.yaml` and the proof. - After a Dependabot lockfile merges, the refresh workflow admits it and renews the proof. **Steps to reproduce** 1. Open a Dependabot pull request that changes `pnpm-lock.yaml`, such as #17364. 2. Let CI run. The `server (9/12)` shard fails on the reporting-lock policy test. **Security gap found during this work** The old rule skipped the lockfile check when `github.head_ref` was `chore/refresh-lockfile`. A contributor could open a pull request from a fork branch with that name and commit a lockfile change. The old rule also exempted any pull request that Dependabot opened, including commits that other people pushed to its branch. ## What Changed - Add `.github/scripts/lockfile-trust.mjs`. It decides which writers may change lock content: - The refresh workflow: the head is `chore/refresh-lockfile` in this repository, and `github-actions[bot]` opened the pull request. - Dependabot: Dependabot opened the pull request from this repository, and the head `pnpm-lock.yaml` matches a Dependabot commit. That commit must have `dependabot[bot]` as its author, GitHub (`web-flow`) as its committer, and a verified signature. The script reads these fields from the GitHub API. - The reporting-lock proof files count as lock content. Only the refresh workflow may change them. - `pr-trusted.yml` policy job: replace the inline `head_ref` and author exemption with `lockfile-trust.mjs pr`. The job runs the script copy from the base commit, so a pull request cannot change the check that judges it. If the base predates the script, the job uses the earlier rule. - `refresh-lockfile.yml`: add `lockfile-trust.mjs merged`. It checks that the last lockfile change on `master` is a GitHub-signed merge of a Dependabot pull request, and that the lockfile at the pull request head matches a verified Dependabot commit. If both checks pass, it gives the lockfile SHA-256 to `prepare-report-lock.mjs --refresh --trusted-lock-sha256`. - `refresh-lockfile.yml` PR step: check `pnpm-lock.yaml` and both proof files before it exits as unchanged. A refresh that renews only the proof now opens a pull request. - `prepare-report-lock.mjs` and its `.d.mts` declaration: `refreshReportLock` accepts a trusted lock hash. If the trusted lock already resolves, the proof admits it exactly with an empty patch. If not, the proof records the resolution as the reviewed correction, as before. - `reporting-lock-policy.test.mjs`: skip on Dependabot pull requests. The policy job verifies those lockfiles. The general test job gets `PAPERCLIP_CI_PR_AUTHOR` for this purpose. This keeps the test, so it still merges cleanly with #19303. - `check-pr-lockfile.mjs` (advisory gate): allow Dependabot lockfile changes, and flag changes to the proof files. - `CODEOWNERS`: add the two proof files. - `doc/RELEASE-AUTOMATION-SETUP.md`: describe the writers, the `@dependabot recreate` path, and the proof renewal after merge. ## Verification - `node --test '.github/scripts/tests/*.test.mjs'`: 707 passed. This includes 17 new tests in `lockfile-trust.test.mjs` and 3 new tests in `check-pr-lockfile.test.mjs`. The new tests run the `pr` command against real git histories with a stub `gh`. - `tests/runner-e2e/prepare-report-lock.test.ts`: 10 passed, including 3 new trusted-lock refresh tests. I ran it with a minimal Vitest config, because the runner-e2e config builds a provider bundle first. - `server/scripts/reporting-lock-policy.test.mjs`: passes normally, and skips with `PAPERCLIP_CI_PR_AUTHOR=dependabot[bot]`. - Live check of the `pr` mode against #17364 with the real GitHub API: - With the real author, it reports "pnpm-lock.yaml matches verified Dependabot commit 83c328c15c…" and `writer=dependabot`. - With a contributor as the author and `chore/refresh-lockfile` as the head ref, it fails. - Live check of the `merged` mode at the real merge commits of #16018, #16020, and #15361. All three report a trusted lock hash. For #16018 and #15361, `master` changed after Dependabot's commit, so GitHub's merge produced different lockfile bytes. An exact-bytes rule would refuse them. That is why the rule trusts GitHub-signed merges of a verified Dependabot head. - Greptile reviewed `513ca2def8` at 4/5 with one P1: a proof-only refresh was never committed, because the PR step checked only `pnpm-lock.yaml`. `edbeb1ea0d` fixes it. A new test in `lockfile-refresh-merge.test.mjs` runs the step's change check on a proof-only change. The test fails without the fix. - CI `Typecheck + Release Registry` failed at `edbeb1ea0d`: `prepare-report-lock.d.mts` did not declare the new options argument (TS2554 twice in `prepare-report-lock.test.ts`). `9139aaad85` updates the declaration. A local `tsc -p tests/runner-e2e/tsconfig.json` shows the 2 errors with the old declaration and none with the new one. - CI at `9139aaad85` failed in six general test shards. None of them run code that this pull request changes, and PR CI loads `pr-trusted.yml` from `master`. The failures: - `heartbeat-plugin-environment.test.ts`: "expected 'running' to be 'succeeded'" - `status-card-evaluation-work.test.ts`: a deadline comparison - `isolated-repository-rehome.test.ts`: timed out after 15 seconds - `chat-channels.integration.test.ts`: GitHub webhook recovery scheduling - `company-import-export-e2e.test.ts`: child server start timeout, which also fails on unrelated branches - `workspaces-b`: every test passed, but an `EPIPE` error came after `acpx-engine/execute.test.ts` I started a rerun of the failed jobs. - Superagent found a P1 at `9139aaad85`: a verified signature proves who committed, not who authored. A collaborator could sign a commit with their own key and set Dependabot as the author. `4fe180d6c1` also requires GitHub (`web-flow`) as the committer. New tests cover the spoofed-author case. #17364, #16018, #16020 and #15361 still verify against the GitHub API. - `git merge-file` merges this test change with #19303 without conflicts. - Not run: the full repository test suite and typecheck. These changes touch only CI scripts, workflows, one helper, and tests. GitHub CI covers full regression. ## Risks - `pr-trusted.yml` loads from `master`, so the new policy takes effect only after merge. This pull request's own CI uses the earlier rule. - A Dependabot pull request that is behind `master` now fails if someone merges `master` into it and the lockfile changes. Use `@dependabot recreate`. The policy job error says this. - Paid evaluation jobs refuse a merged Dependabot lockfile until the refresh pull request merges. The refresh merges without review when it adds no new package versions. If the release-age gate holds it, the wait lasts until a maintainer merges it or the daily refresh runs. During that time, the reporting-lock policy test also fails on other pull requests, which is the current behavior for any unrenewed lock. - The rule trusts GitHub as the signer of Dependabot commits and of merge commits (`web-flow`). Superagent uses the same trust root. - People with write access can still push to `chore/refresh-lockfile`. This was already true. The bot merges only its own head commit (`--match-head-commit`), and code-owner review covers anything else. - A future manual proof renewal, such as #19283, now needs the refresh workflow or an admin merge. A contributor pull request that changes the proof files fails. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution, in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused tests; see Verification) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Dotta
|
b68f69a801 |
test(evals): pin author self-use and credential guard corrections (#19357)
## Thinking Path > - Paperclip manages agents and their work. > - The protected chat catalog selects an immutable evaluated source. > - The measured parent produced seven independently reviewed passes out of ten. > - The authoring observer rejected valid author-only use, and two credential wording guards needed stricter boundaries. > - The reviewed source corrects those observers and declares each affected oracle version. > - This PR pins that source without changing the original questions or run limits. ## Linked Issues or Issue Description Refs #19346, #19095, #19093. **What happened?** The measured authoring response described real saved author-only use. The old observer required zero assignments and wrongly rejected an audited author-only assignment. The credential scanner also needed to retain an unsafe comma-separated destination and mixed reference/password requests. These are evaluation-layer corrections. The parent timeline and company runs did not complete; no causal latency fix is claimed here. **Expected behavior** Select the reviewed immutable observer source. Keep all original scenario identities, questions, conditions, actor/resource proof, counts, clocks, credentials, model, permissions and zero retries unchanged. **Steps to reproduce** Run the source authoring and shared credential consumer controls. Compare the committed candidate descriptor snapshot with source `d1ab241cac0db7f700fadaaf62ffb3d2b51cf0c3`. ## What Changed - Scope the private Company creation observer to the exact Create secret dialog. Its successful close can open a separate details Sheet without making the observer wait for that Sheet. - Admit blank or bounded non-secret metadata defaults only for the Company saved-secret-name question variant. Verify the same canonical payload in independent human accounting. - Pin source `d1ab241cac0db7f700fadaaf62ffb3d2b51cf0c3`, tree `c4efaef62405cbb37236ad01db1187cf8a156868`. - Store exact descriptor SHA256 `60db015337e807ff53a376b6db11c375a239097b224d2253dad68ecd0b0f61f3`. - Update only seven oracle-version leaves: authoring v5, company v11, personal v12, and four shared external consumers v6. Source-file lists remain fixed. - Retain provenance anchors, historical conditions and failures. Update the single expected-target assertion. Exactly three metadata/test files change. ## Verification - 41 manifest, adapter, reconciliation and CLI controls pass on this actual pin projection. - Qualified source: one actual-worker offline browser canary passes all three creation, failed-POST and missing-form mechanisms. Full E2E types pass on this successor. Reused unchanged proof: 286 distinct controls across authoring, Company saved-name question, human accounting and shared-consumer/declaration suites pass. Full E2E typecheck passes. All 242 declarations load with zero fixture callbacks. - SDK506 inputs and registry `efa70621` remain unchanged. Launcher SHA256 `2b9a6953`, private transport interfaces, production/UI/CLI source, lockfile and workflows remain unchanged from the measured source. - All 244 YAML queries, context and expected outcomes are byte-identical to the parent. All descriptor native/human counts, clocks, authentication, setup policy and source-file lists remain fixed. - The parent source has seven measured semantic passes out of ten. The new authoring and credential oracle versions are native-unrun. Existing failed grades remain failed. - The isolated company concurrency diagnostic [run42124035785](https://github.com/paperclipai/paperclip/actions/runs/42124035785) is a separate measurement. This pin does not alter concurrency or claim to fix native noncompletion. - Repo-wide build/tests were not repeated for this three-file metadata update. Normal PR CI remains required. ## Risks - The name character set does not prove secrecy. The saved-key-name variant can admit a credential-shaped synthetic default; this general classifier limit remains unfixed and tracked in #19093. These are finite experimental journeys only. Independent review must reject any actual credential-like or unknown-value prefill without clear public-metadata/proposed-name grounding, even if autograding is green. If safety cannot be established, the journey cannot receive PASS credit. Secure UI save, human answer, canonical payload/hash, audit, binding and byte-leak checks remain mandatory. - Earlier CI includes three explicit runner-shutdown cancellations and one deferred-observer test assertion at line573. Those outcomes remain retained and unresolved; the Company dialog repair makes no claim to fix them. New exact-head CI is required. - The Company dialog mismatch was established from production source and an actual-worker model-free regression before the next secure-save step. The prior isolated attempt did not reach private entry, so it is not evidence of an observed private-save failure. - A valid review warning remains as a documented follow-up in #19093: safe closing comma exclusions such as “only there” can be falsely rejected by the credential wording guard. The pin authorizes bounded experimental journeys only. If a valid actual handoff hits that guard, its failed grade is retained; no regex failure is waived or counted as PASS. Independent semantic review remains required for both secret handoffs. - Isolated Company original5055 remains FAILED. A real succeeded native took255949ms and produced a valid saved-name question, but the observer rejected its proposed non-secret metadata default. Secure entry and the second native were not reached. This successor repairs that assertion boundary only. - The next selected74/466/578 run is an explicit bounded lower-concurrency functional experiment at the existing cap2, same model, original clocks and zero retries. It is not evidence of a latency fix or reliability improvement. - Local observer proof is not a new native PASS. Timeline and company noncompletion remain unresolved, and no clock increase is included. - A secure handoff still requires independent semantic review before PASS credit. Actual saved resources, owner/audit evidence, provider outcomes and privacy gates remain required. - The source catalog remains separate from master. This PR does not merge the core or full catalog feature changes into master. ## Model Used OpenAI Codex, `gpt-6.1-sol`, with repository tools and code execution. The host does not expose the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Dotta
|
390ef983d6 |
test(evals): pin coherent secure-entry and team repairs (#19346)
## Thinking Path > - Paperclip manages AI agents and their work. > - The protected TestOps catalog selects one immutable source revision. > - The current pin predates reviewed secure-entry and Team observation repairs. > - A new run must use the qualified source and its exact descriptor bytes. > - This PR updates only the source pin, retained descriptor snapshot, and expected-target test. > - Existing failures remain failures until a new native run proves the feature. ## Linked Issues or Issue Description Refs: #19333, #19084, #18971, #19112. **What happened?** The current catalog points to the earlier source. It does not include the reviewed personal entry-dialog observer, visible secure-entry guidance, company menu navigation, Team claim attribution, or completion-link guidance changes. **Expected behavior** The protected catalog must select the qualified immutable source and verify its exact descriptor snapshot. Model, original queries, run counts, clocks, private transfer interfaces, and retries must remain fixed. **Steps to reproduce** Read the manifest target and candidate descriptor digest. Compare them with source `7ccfc33bf074e37d8fd901a6e8453d1b45307e82` and its committed descriptor bytes. ## What Changed - Pin source `7ccfc33bf074e37d8fd901a6e8453d1b45307e82`, tree `5afa0cf2e8ad8d17b7563e62378b0994f7777c67`. - Store the exact candidate descriptor snapshot with SHA256 `e1b7de947567569d96fcc9dca607ef5807ffaba4bf994562c0c7ab46d1856df6`. - Update current version and source-file provenance while retaining the original anchors and prior conditions. - Update one expected-target assertion. This PR changes exactly three metadata/test files. The final target-only successor retains the previous pin commit and its CI results; descriptor bytes do not change. ## Verification - 41 model-free manifest, adapter, reconciliation, and CLI controls pass. Another 42 chat-admission controls pass. - Initial source qualification: 268 focused controls, five explicit offline Chromium controls, and full E2E types pass. The first six-path Company/Team amendment passes 233 affected controls and full E2E types. The final reviewed six-path followup passes 271 affected controls, two offline browser canaries and full E2E types. Unchanged source inputs reuse the prior qualification. The accepted completion-feedback core patch has 22 real PostgreSQL controls from its owner. - All 244 scenario YAML definitions and all 242 descriptor run counts, clocks, authentication, and setup policy remain unchanged from the previous source. Seven descriptor rows change only versions or source-file lists. - SDK506 inputs and registry `efa70621` remain unchanged. Launcher SHA256 `2b9a6953` and private transfer interfaces remain fixed. - Fresh-main reporting installation passes with pnpm9.15.4, frozen lockfile, ignore-scripts, and offline mode. The unchanged reviewed report policy returns `already-reviewed`. No lock or workflow change is included. - Repo-wide build/typecheck were not repeated for this three-file metadata update. Normal PR CI remains required. ## Risks - Both secret journeys require independent review of the actual native handoff before any PASS credit. Unsafe value requests cannot count as a feature pass even if an oracle accepts them. Two additional unexercised adversarial wording guards (comma-only plaintext destinations and mixed reference/password requests) are being hardened separately in #19093. They are not included in this source pin. No current native PASS is claimed. - This pin includes reviewed source changes whose separate core/catalog PRs remain unmerged. This PR does not merge those features into master. - New native measurements remain required. The prior personal failure remains failed; its first native took 580602ms, and completion within the unchanged 660s remains unproven. - The plain-comment save acknowledgment fixes a downstream phase that the failed attempt did not reach. Component checks do not prove native completion. ## Model Used OpenAI Codex, GPT-6.1 Sol (`gpt-6.1-sol`), with code execution and repository tools. The runtime does not expose its context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandClaude Opus 5.5
|
b4cc2d30d2 |
fix(e2e): grade only started runs and bound the clarification readback (#19335)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Product E2E journeys grade agent work in the QA staging smoke plan > - On the current QA deployment, two journeys failed after the product had already done the requested work > - protected-actions counted a wake that the control plane retired before it started as a bound run, and failed its runtime binding check > - question-answer-resume waited for its saved document in a view that no longer exists. The wait lasted 27 minutes, until the case deadline, and starved the run queue > - This pull request grades only runs that started provider work, and reads the document where the UI now shows it, with a bounded wait > - The benefit is that both journeys measure the product, and a UI readback failure costs about two minutes instead of a whole worker ## Linked Issues or Issue Description **What happened?** - protected-actions: the task succeeded with two runs. A third automation wake for the same agent and task was cancelled before it started, with the reason "Agent cannot start work while completed runs await accounting". Its `startedAt` was null, its `resultJson.cancellation` had `source: "control_plane"` and `expected: true`, and its `executionRecovery.providerWorkStarted` was false. That run has no runtime binding, so `validateAgentTaskState` failed with `agent_run_binding_mismatch`. - question-answer-resume: the agent saved the requested document and the task was done by 19:32. The journey then navigated to `#document-<key>` and waited until 19:59 for an element that never appeared. With the task chat shell and Artifacts V1, documents render in the properties pane's Artifacts tab. The run reported `journey_operation_failed` and `evidence_collection_timeout`, and the 60-minute job ended before four other cases could run. **Expected behavior** - A wake retired before admission is kept as evidence but is not graded as provider work. - The saved document is read in its current view, and a UI readback failure ends quickly. **Steps to reproduce** QA staging smoke launch 3074 on app |
||
|
github-actions[bot]andlockfile-bot
|
1e98aa76e3 |
chore(lockfile): refresh pnpm-lock.yaml (#19337)
## Thinking Path > - Paperclip uses a dependency lock for reproducible builds. > - Dependency changes on master need matching lock entries. > - Paid eval reports also pin a reviewed offline correction. > - Reports must install before paid model work starts. > - This automated PR keeps the lock and report proof together. ## Linked Issues or Issue Description **What happened?** Dependency manifests on master changed after the last committed lock refresh. **Expected behavior** Frozen installs must match those manifests. Eval reports must retain matching offline lock proof. **Steps to reproduce** Run a frozen install before and after this lock refresh. ## What Changed - Refresh the dependency lock from the manifests on master. - Include the matching offline report correction and hashes. ## Verification - Pinned pnpm resolution completed with lifecycle scripts disabled. - The report proof refresh accepted the reviewed committed baseline. - The workflow rejected changes outside the lock and report proof files. ## Risks The lockfile bot merges this PR without CI or review when every added package version is at least 3 days old on npm. Otherwise the PR waits for a maintainer, and the daily refresh tries again. The bot merges only the exact commit this workflow generated from manifests already reviewed on master. Unknown report lock baselines remain blocked. This update changes no permissions or paid enable gates. ## Model Used None — generated by pnpm and the deterministic report-lock updater. ## Checklist - [x] Thinking path and problem description included - [x] Generated lock and matching report proof included - [x] Release-age gate decides between bot merge and maintainer review Co-authored-by: lockfile-bot <lockfile-bot@users.noreply.github.com> |
||
|
DottaandPaperclip
|
6a66c09f7f |
test(evals): pin secure confirmation and team warning repairs (#19333)
## Thinking Path > - Paperclip manages AI agents and human approval for work. > - The chat catalog measures real user journeys through TestOps. > - The last personal-secret journey reached a valid secure-save confirmation that its observer rejected. > - The team journey reported a missing named skill with a typographic apostrophe that its observer rejected. > - A company-secret request returned body validation before the required agent authorization denial. > - This PR pins one reviewed source that addresses these boundaries without changing the requests, models, or time limits. > - New native measurements must establish the result. Existing failures remain failures. ## Linked Issues or Issue Description Refs: #19084 Refs: #19112 Refs: #19325 Refs: #19261 This is the single successor pin for the reviewed personal confirmation, team warning, and company authorization changes. It does not merge those core or full-catalog branches into master. ## What Changed - Pin immutable source `ab0f5246f765058af702b93d9b6637f2ef5b83cd`, tree `6cf28980b0ab6d1dd8e54efda3996708c53aeb98`. - Bind the exact descriptor snapshot `0b57bba133a5c15860ca77b804f7282b5314d7feb8efbfd6f11fca42cdbdee49` and matching target test. - Advance personal-secret v9 to v10 and team-install v4 to v5. Record the personal confirmation helper and its existing contract/flow inputs in source provenance. - Disclose the unmerged company-secret authorization fix from #19325. It denies scoped agent creation before body validation and gives safe human entry guidance. Board creation remains unchanged. - Retain the original query history, including the single previously approved multipart-form clarification. Preserve all 244 identities, the 196/244 gate, model, clocks, native and human bounds, SDK registry, launcher, and private evidence protocol. ## Verification - Pin-local checks: all **38** manifest, adapter, and reconcile controls passed with `node --test tests/runner-e2e/chat-full-catalog-testops-manifest.test.mjs tests/runner-e2e/chat-full-catalog-testops-adapter.test.mjs tests/runner-e2e/chat-full-catalog-testops-reconcile.test.mjs`. - Exact source qualification: the combined domain run passed 328 controls and skipped one optional canary. Its two local listener failures were sandbox EPERM setup failures. The same two controls then passed with the authorized local listener, in a 73-control run. The eight new route authorization controls also passed. These local runs do not prove native completion. - Full E2E TypeScript passed on exact source ab0 after reuse of the healthy, same-source workspace graph. Earlier missing-alias graph failures are retained. A full local server build or repository-wide suite was not run; #19325 has its own core checks. - The source SDK guard verified all **506** registered input Git blobs unchanged, registry `efa70621e9fb42aa185f7ee616a1a726aa3eccb367e67e28817cedf26ef2ade8`. No SDK build or qualification renewal was needed. - Actual anchored controller dependency check: `prepare-report-lock.mjs` returned `already-reviewed`, then pnpm **9.15.4** completed the frozen, ignore-scripts install in 17 seconds. This exercised the current upstream Claude ACP patch and reporting output `dd5934060dd06b8e731e3beeed49daf918d1a0b2a6c32d3da7815f2e7f0b3691`. It made no tracked lock or policy change. Earlier source-lock and controller-lock evidence remain separate. - Existing failed personal/team attempts remain failed. The company initial noncompletion remains uncorrected as an outcome claim; the authorization fix addresses one proven error boundary only. Team repeat/link checks were not reached in the failed measurement and remain required. No uniform-ten PASS or new native feature PASS is claimed. - Next validation is a controlled TestOps measurement after normal pin merge and fresh actual-controller proof. Any later whole-ten regression is separate. All attempts remain recorded; there is no retry-until-green policy. - Exact head `58d4a9b2d3c90be725e8877873010295122fb9c0`: required CI aggregates `ci / e2e` and `ci / verify` passed in run 42099871883. Windows passed in run 42099871391. Genuine Greptile review is 5/5 on that exact head, with zero unresolved threads. The complete rollup has 56 successes, three configured skips, and one neutral check; no pending or adverse check remains. No native measurement ran as part of these checks. ## Risks - Personal v10 accepts two logical human inputs: the original chat request and a separately verified same-human secure-save confirmation. The raw chat count remains one. This is a disclosed fixture variant, not proof from a new native journey. - The new confirmation fields do not change the trusted publisher schema or its closed enums. Existing private helper and launcher bytes remain unchanged. - Team v5 only normalizes the apostrophe in its warning predicate. Saved state, named-skill truth, repeat idempotency, links, and contradictory-success guards still apply. - The authorization fix is present only in the pinned source. #19325 remains unmerged; its effect on the unfinished native turn is unproven. ## Model Used OpenAI `gpt-6.1-sol` through Codex, with reasoning, repository tools, and local code execution. The context window size is not reported by this runtime. No model inference was used for local tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandClaude Opus 5.5
|
aaaad177eb |
fix(e2e): follow renamed project and Artifacts V1 download controls (#19323)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Product E2E journeys drive the real UI and run in the QA staging smoke plan > - Two UI changes landed after the previous QA deployment: the Projects toolbar action is now "Create project" (#18716), and Artifacts V1 is on for every instance (#19164), so attachment rows offer download as an icon link with an `aria-label` > - The journeys still clicked "Add Project" and required a visible "Download" text, so they timed out before reaching the server > - This pull request accepts both the old and the current controls > - The benefit is that project-task-context, build-revise and stop-redirect measure the product again on current deployments ## Linked Issues or Issue Description **What happened?** On the QA deployment of |
||
|
github-actions[bot]andlockfile-bot
|
b37188ce42 |
chore(lockfile): refresh pnpm-lock.yaml (#19317)
## Thinking Path > - Paperclip uses a dependency lock for reproducible builds. > - Dependency changes on master need matching lock entries. > - Paid eval reports also pin a reviewed offline correction. > - Reports must install before paid model work starts. > - This automated PR keeps the lock and report proof together. ## Linked Issues or Issue Description **What happened?** Dependency manifests on master changed after the last committed lock refresh. **Expected behavior** Frozen installs must match those manifests. Eval reports must retain matching offline lock proof. **Steps to reproduce** Run a frozen install before and after this lock refresh. ## What Changed - Refresh the dependency lock from the manifests on master. - Include the matching offline report correction and hashes. ## Verification - Pinned pnpm resolution completed with lifecycle scripts disabled. - The report proof refresh accepted the reviewed committed baseline. - The workflow rejected changes outside the lock and report proof files. ## Risks The lockfile bot merges this PR without CI or review when every added package version is at least 3 days old on npm. Otherwise the PR waits for a maintainer, and the daily refresh tries again. The bot merges only the exact commit this workflow generated from manifests already reviewed on master. Unknown report lock baselines remain blocked. This update changes no permissions or paid enable gates. ## Model Used None — generated by pnpm and the deterministic report-lock updater. ## Checklist - [x] Thinking path and problem description included - [x] Generated lock and matching report proof included - [x] Release-age gate decides between bot merge and maintainer review Co-authored-by: lockfile-bot <lockfile-bot@users.noreply.github.com> |
||
|
github-actions[bot]andlockfile-bot
|
b4e7b76635 |
chore(lockfile): refresh pnpm-lock.yaml (#19294)
## Thinking Path > - Paperclip uses a dependency lock for reproducible builds. > - Dependency changes on master need matching lock entries. > - Paid eval reports also pin a reviewed offline correction. > - Reports must install before paid model work starts. > - This automated PR keeps the lock and report proof together. ## Linked Issues or Issue Description **What happened?** Dependency manifests on master changed after the last committed lock refresh. **Expected behavior** Frozen installs must match those manifests. Eval reports must retain matching offline lock proof. **Steps to reproduce** Run a frozen install before and after this lock refresh. ## What Changed - Refresh the dependency lock from the manifests on master. - Include the matching offline report correction and hashes. ## Verification - Pinned pnpm resolution completed with lifecycle scripts disabled. - The report proof refresh accepted the reviewed committed baseline. - The workflow rejected changes outside the lock and report proof files. ## Risks The lockfile bot merges this PR without CI or review when every added package version is at least 3 days old on npm. Otherwise the PR waits for a maintainer, and the daily refresh tries again. The bot merges only the exact commit this workflow generated from manifests already reviewed on master. Unknown report lock baselines remain blocked. This update changes no permissions or paid enable gates. ## Model Used None — generated by pnpm and the deterministic report-lock updater. ## Checklist - [x] Thinking path and problem description included - [x] Generated lock and matching report proof included - [x] Release-age gate decides between bot merge and maintainer review Co-authored-by: lockfile-bot <lockfile-bot@users.noreply.github.com> |
||
|
DottaandPaperclip
|
3fca8a1500 |
test(evals): renew the reporting lock after iMessage update (#19283)
## Thinking Path > - Paperclip verifies the selected source and reporting dependencies before it obtains model credentials. > - The reporting helper admits only an exact reviewed lock or its exact reviewed correction. > - Master updated the iMessage dependency and protobuf after the last reporting-policy review. > - The controller lock therefore stopped matching either admitted hash. > - A real five-case launch failed at this guard before any case execution. > - This PR renews the exact hashes and keeps the existing patch and rejection mechanism unchanged. > - A normal-CI repository check now exercises the real helper against the committed lock before paid execution. ## Linked Issues or Issue Description Refs: #19261, #18914. **What happened?** The five selected execution jobs in the retained launch 3002 stopped during “Verify source-bound build and authenticated selection before credentials.” The first observed error was `Reporting lock changed; review the dependency correction before install` in the trusted controller helper. Model-key and original case execution steps were skipped. The failed attempt remains infrastructure evidence; this PR makes no native product-grade claim. **Expected behavior** Admit the exact reviewed current controller dependency graph. Keep rejecting unknown locks, changed correction patches, and wrong output bytes before credentials or provider execution. **Steps to reproduce** Run the reporting helper against the committed current-master lock. The parent policy recognizes 754… and 60f…, while the actual controller lock is 6f252b…. The parent stops at the same pre-credential boundary as AWS. The existing correction patch reverses and reapplies cleanly to that lock. ## What Changed - Renew only the reporting policy input and output SHA256 values. The output is the exact committed current-master lock after `@photon-ai/advanced-imessage` 2.1.0→2.2.0 and `@bufbuild/protobuf` 2.15.0→2.16.0. - Derive input `ea91060774e3289d5d07bc58f72d8953f5a6dcb9db98aa9a1adf202476bf89ac` by reversing the existing correction patch from the current lock. Reapplying that byte-identical patch yields output `6f252b0c8099ed837a297f97f1fc5e14e96bc7530b0239065d0789b33b2ee477` exactly. - Keep patch SHA256 `2adced9a66052ba5752140ec6cc6670f679552a9a8559bd1faea10036afeeb97`, helper code, and guard behavior unchanged. Update the policy basis to describe the review truthfully. - Add one normal server-suite check. It copies the committed lock and reviewed policy into a temporary directory and runs the real helper. The prior focused suite already covered this contract, but its runner-e2e location is outside normal root Vitest projects. `server/vitest.config.ts` already includes `scripts/**/*.test.mjs`; no workflow change is needed. No `pnpm-lock.yaml`, workflow, source pin, descriptor, oracle, model, permission, credential, scanner, or case bound changes. Immutable source 6773 and the failed launch remain unchanged. ## Verification - Parent existing reporting suite: 4 fail and 3 pass because the committed lock is outside the admitted pair. Renewed policy: all 7 pass, including unknown-lock, changed-patch, wrong-output, and refresh-path rejection controls. - New normal-CI repository control fails with the exact paid-step reporting-lock error on the parent policy. The corrected combined run passes 8 controls across 2 suites. - Exact patch reverse/forward roundtrip preserves the full current lock bytes. The actual CLI returns `already-reviewed`; the working lock hash stays 6f252b… and is not modified. - Focused command: `node node_modules/vitest/vitest.mjs run --config ../report-lock-imessage-vitest.config.mts`. The external local config selects only the two affected suites with the existing 30-second test limit. - `git diff --check` passes. No SDK/native/source build, model call, evaluation retry, or private archive download was needed for local proof. Full workspace typecheck/build were not run for this policy-and-test change. Latest-head required CI and genuine review remain pending. ## Risks This policy deliberately rejects the next unreviewed lock change. The normal-CI check should expose that drift before paid fanout. The input is the current lock with the already approved correction reversed; it is not a claim that a new dependency graph was resolved or installed locally. Local guard controls do not prove a repaired AWS execution or any of the five product journeys. The next measured attempt still requires normal protected merge and fresh actual-controller admission. ## Model Used OpenAI `gpt-6.1-sol`, with reasoning, repository tools, and local code execution. The host did not expose a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandClaude Opus 5.5
|
45073d97ae |
fix(evals): keep evidence for wakes retired by a terminal issue before start (#19258)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Product regression TestOps plan runs 20 Product E2E cells, and each cell saves run evidence after grading > - In the final full run, `first-task/clarify-propose-accept` passed every journey check > - The cell still failed while saving evidence: it requested a provider log for a wake that never started > - That wake was retired with `issue_terminal_status` because its child task was already done, so no log exists > - The settle oracle already accepts such a wake (`isTerminalUnstartedWake`), but the evidence collector did not > - This pull request records that wake's log as omitted, with a reason, and keeps its events > - The benefit is that a passing journey is no longer reported as failed because of a log that never existed ## Linked Issues or Issue Description Refs #19073, #19243 **What happened?** In GitHub run https://github.com/paperclipai/paperclip/actions/runs/42046147744, `first-task.runner-codex.local.clarify-propose-accept` had no failed checks in `first-task.json`. The conversational acceptance was recorded through `resolve-from-comment` (200). The `finally` evidence step then threw `GET /api/heartbeat-runs/ef2295c4…/log returned 404: Run log not found`. Run `ef2295c4` is an `issue_status_changed` wake on the completed child task. It is `cancelled` with `issue_terminal_status`, and `startedAt`, `logStore` and `logRef` are all null. **Expected behavior** Evidence capture keeps that wake's events and records why it has no log, as it already does for reset, blocked, reassigned and pre-provider runs. **Steps to reproduce** Run the cell. A child status-change wake may be queued just before the child becomes `done`, and is then retired before admission. ## What Changed - `collectChatRunEvidence` (`tests/runner-e2e/chat-flow.ts`): a run that is `cancelled` with `errorCode: issue_terminal_status` and explicit null `startedAt`, `logStore` and `logRef` gets `log: null` and `logOmissionReason: "issue_terminal_before_start"`. Its events are still collected. - `chat-flow.test.ts`: a test for that case, plus negative controls. Each of the following still fails on a missing log: started, `startedAt` undefined, `failed` status, a different error code, a log store, a log ref. ## Verification - `npx vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/chat-flow.test.ts tests/runner-e2e/first-task.test.ts tests/runner-e2e/continuation.test.ts`: 378 pass. - `tsc -p tests/runner-e2e/tsconfig.json`: no errors in `tests/runner-e2e`. - Retained artifact from run 42046147744 (cell `clarify-propose-accept`): there are no failed checks, and the run ledger shows the retired wake with null `startedAt`, `logStore` and `logRef`. ## Risks - Low. This is an evidence-capture change in the eval harness, and the journey's grading is unchanged. The omission needs explicit positive evidence that the run never started and never had a log store. Any run that did provider work still fails on a missing log. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), via Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
cb8df178c8 |
test(evals): pin the clarified five-case chat repairs (#19261)
## Thinking Path > - Paperclip helps people manage AI agents and their work. > - The TestOps catalog executes one reviewed immutable source with declared model, permissions, counts, clocks, and private evidence gates. > - Recent retained attempts exposed specific question, timeline, secret-handoff, and team-reporting contract gaps. > - Five focused repairs now have one qualified source; their component checks do not establish a native pass. > - This PR binds the trusted catalog lane to that source and records its exact changed definitions. > - One visible-form question is clarified and disclosed; all other catalog requests stay fixed. > - The controller, launcher, private evidence helpers, and execution authority stay unchanged. ## Linked Issues or Issue Description Refs: #19086, #19084, #19093, #19112, #19148, #19188. **What happened?** The retained native attempts include a timeline narrative rejected despite the saved handoff, secure-entry admission failures before downstream verification, and a truthful contextual team caveat rejected after a correct repeat. Multipart attempts4623 and4717 did not complete under the original broad request. All original statuses and evidence remain unchanged. **Expected behavior** Measure the five reviewed repairs on one immutable source with the original model, limits, and privacy gates. A component replay must not rewrite an old result or count as a product pass. The catalog must display and execute the same versioned request. **Steps to reproduce** After normal protected merge and fresh controller admission, use the existing TestOps plan102 and job100 to select the intended five cases on master. Root owns the actual Submit and original-evidence review. No evaluation is dispatched by this PR. ## What Changed - Pin only the catalog lane to `6773b1c128a2e7efa7571ff04b38238f00d06117`, tree `5669b199edc1aed3ab812403407b5930f5127da4`. - Copy its exact descriptor snapshot, SHA256 `aaf15ca6755fcfe550daa98bca1ed263e3d625159b53cd1e1f52fefbf9474778`. Advance only multipart53 v4, timeline466 v10, company-secret578 v8, personal-secret580 v9, and team-install610 v4. Add four declared source paths across the two secret cases. - Record exactly one query clarification through the existing per-case changed-field join. Old: “Help me choose the details for the launch page before you build it.” New: “Walk me through the launch-page question form so I can choose the audience and tone and enter my own headline before you build anything.” Preserve the old query and failed results in history. Retain inventory `0cf530…` and disclose clarified inventory `814bb363…`. All other243 catalog requests are unchanged. - Disclose that the company-secret second human message now uses actual saved non-secret metadata instead of repeating the initial request. Its continuation still requires the actual agent identity. No fabricated agent comment or private value establishes the handoff. - Team recovery uses independent saved canonical skill/version/member proof. No recovery-success phrase is required. A contradictory current denial still fails; a still-missing skill still requires truthful disclosure. - Keep all244 identities, the196/244 admission gate, model/profile, human/native counts, clocks, retries0, cap8, launcher, private protocol, and SDK506/efa qualification unchanged. No product code, full catalog, workflow, permissions, secret, runtime, or helper change lands on master. ## Verification - Source owner qualification:680 affected controls on the accepted parent and213 focused successor controls plus two explicit browser canaries, fresh same-source server declarations, and full E2E TypeScript pass on localNode26.4.0. Registered SDK506 Git inputs, production source, private protocol, launcher, and original limits match the accepted source. The source-only contract/flow additions are not copied into this controller. - Pin-local command: `node --test tests/runner-e2e/chat-full-catalog-testops-manifest.test.mjs tests/runner-e2e/chat-full-catalog-testops-reconcile.test.mjs tests/runner-e2e/chat-full-catalog-testops-adapter.test.mjs` —38 pass. - Exact source descriptor and launcher bytes match the bound hashes. The manifest keeps the original metadata source and source history, joins the new descriptor, and permits only the one explicitly recorded53 query clarification. `git diff --check` passes. - Warm controller admission:10 committed helper byte checks, five required source matches, and four trusted module imports passed with the existing exact-main60f dependency graph. No SDK or native rebuild ran. Actual post-merge master and imports still require a fresh check before Submit. - The53 clarification changes the evaluated question. It does not fix or regrade the original300-second failures. All form validation, back retention, custom answer, persisted-answer, continuation, and visible-summary checks remain required. One targeted post-clarification measurement is planned; any later whole-ten regression is separate, with all attempts retained and no retry-until-green. The existing case53 description must be updated and read back after merge so TestOps displays the clarified request. - All five successor scenarios remain unrun. No new native pass, uniform-ten pass, reliability claim, or master catalog qualification is made. Focused source PRs remain separate and unmerged; their independent CI does not replace this PR's required checks. - Full workspace typecheck/build were not run locally for this metadata-only PR. Current-head required CI, Windows, genuine review, and zero unresolved material threads are required before normal protected merge. ## Risks A wrong source or descriptor would misstate the measured definition. Strict target, snapshot, field, and identity joins cover that risk. The clarified question cannot establish success for the old broad request. Safe guidance admission still needs real human entry, private UI persistence, and worker evidence in the next attempt. Component controls do not prove those outcomes or completion within the original limits. ## Model Used OpenAI `gpt-6.1-sol`, with reasoning, repository tools, and local code execution. The host did not expose a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
TonioandClaude Opus 5.5
|
9e9a628a57 |
feat(ui): turn Artifacts V1 on for every instance and hide its toggle (#19164)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Artifacts show the output of agent work on the Artifacts page, in the panel Artifacts tab, and in the task thread > - #17228 added the Artifacts V1 presentation behind an experimental setting that is off by default > - The presentation is now ready for all users, so an opt-in toggle is no longer necessary > - A new default alone does not reach all instances, because each experimental write persists every flag > - This pull request removes the toggle and turns V1 on for every instance > - The benefit is one artifact presentation for all users, with an off switch for hosting operators ## Linked Issues or Issue Description Refs #17228 Refs #17263 **Is your feature request related to a problem? Please describe.** Artifacts V1 shows only on instances where an admin turned on an experimental setting. Other users see the previous cards. **Describe the solution you would like** Show Artifacts V1 on all instances. Remove the toggle from Experimental settings. **Describe alternatives you have considered** A new default without a change to the hook. This does not work, because many instances have a stored false that nobody chose, and no toggle would remain to change it. ## What Changed - Removed the Artifacts V1 card from Instance Settings, Experimental. - Changed `enableArtifactsV1` to default on in the schema, the feature catalog, and the settings service. - Changed `resolveArtifactsV1Enabled` to ignore a stored false. It honours false only when the value comes from managed config. - Kept the flag and the previous cards, so a hosting operator can hold V1 off. - Kept the key in the Cloud staging list, so a managed false does not hold V1 off on staging. - Updated unit tests, the Artifacts V1 plan, and the directions-lab Today reference. - Moved the attachment and upload e2e specs and the runner download helper to the V1 rows. - Added V1 tests for a Markdown review: expansion, deep link, and open, raw and download actions. ## Verification - `cd ui && npx vitest run`: one test file failed before its fix. That file tests the previous cards, and it now mocks the hook to off. I reran it with the other changed UI tests, and they pass. I did not repeat the full UI run after the fix. - `npx playwright test --config tests/e2e/playwright.config.ts tests/e2e/text-attachment-tabs.spec.ts tests/e2e/artifact-uploads.spec.ts tests/e2e/artifact-tab-arrival.spec.ts`: 10 tests pass on V1. - `tests/e2e/cloud-auth.spec.ts` failed once in CI on a task heading. It does not touch artifacts, and 10 local runs pass. - `cd server && npx vitest run src/__tests__/instance-settings`: 183 tests pass. - `cd packages/shared && npx vitest run src/feature-catalog.test.ts src/validators/instance.test.ts`: 35 tests pass. - `pnpm run check:token-gates` and the `ui` and `server` typechecks pass. I did not run the full repository check. - In the artifact directions lab, with no stored settings, the Today right panel renders the V1 rows. - I did not test this on a live instance. After merge, confirm on staging that the Experimental page has no Artifacts V1 card and that all three surfaces show V1. ## Risks - All users see V1 after the next release, not only staging users. To revert, revert this pull request. - Users cannot turn V1 off. Only a hosting operator can, through managed config, and not on Cloud staging. - An instance with a managed `enableArtifactsV1: false` stays on the previous cards. The cloud harness source has no reference to this key, but I cannot read the live fleet configuration. - V1 surfaces render before settings load. An operator-held instance can briefly show V1 before the previous cards. - Admins who use the API can still write the flag. The write has no effect on unmanaged instances. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
48b57e0c94 |
test(evals): pin held scenario repairs and await Cloud return (#19188)
## Thinking Path > - Paperclip helps people manage AI agents and their work. > - Protected TestOps runs admit one immutable catalog source. > - Three held scenario observers now have reviewed, locally qualified repairs. > - A separate Cloud recovery test started its heading wait before the Cloud return. > - This PR pins the qualified successor and corrects that test boundary. > - It changes no product behavior, model, execution budget, or private evidence protocol. ## Linked Issues or Issue Description Refs: #19155, #19093, #19095, #19112. The next candidate retains the reviewed skill, company-secret, and team-install observer repairs. It adds a narrow timeline v9 correction: a recorded task identifier can be the subject of its actual reassignment. The real timeline attempt remains failed; the diagnostic correction is not a regrade or a new native pass. Earlier failed attempts stay retained; a source pin and component tests do not establish a native pass. The Cloud recovery fixture failed both direct-task variants in the first CI attempt of #19155. Its five-second heading check started before the external Cloud roundtrip ended. The failure image already showed the expected task shortly afterward. One unchanged local direct-task control passed. The single authorized CI recovery also passed; this did not establish a reliable boundary. This PR waits for one handoff and the exact return URL before the original heading check. ## What Changed - Pin source `5aef4bffa2b106f2066bf2485338798f04a702bd`, tree `464be838a448c728d44e2d5593b9d7faea266663`. - Update the two manifest target literals and expected pin test literal. Bind the exact v9 descriptor snapshot and its one changed timeline oracle leaf. - Add the reviewed two-line Cloud return readiness witness. Keep the original five-second heading assertion and existing overall test limit. - Use descriptor `744a566faaa90b8048fff526e0a30b7475b86c2de88d697783f8da9e02f70e20`. Keep the launcher, all private helpers, 244 identities, requests, setup, human/native counts, clocks, profile, model, retry policy, permissions, and the 196/244 gate unchanged. ## Verification - Current infrastructure worktree: **131 pin, admission, selection, streaming, API, and reconciliation controls passed**. - The fixture owner ran the exact patched Cloud spec: **all five real auth cases passed** in 37.4s. A six-second Cloud hold passed the unchanged heading check after return. Missing handoff and missing return controls rejected. This PR's fresh master base is verified by current-head CI, not credited from the earlier run. - Fresh v9 source qualification: **268 focused timeline controls**, the 242-declaration gate, fresh server declarations, and full E2E typecheck passed under local Node **26.4.0**. The preceding held-helper qualification (239 Node controls and one offline Chromium route control) remains evidence for the byte-identical unaffected helper closures. The original browser sandbox failure is retained. Actual AWS Node and model configuration remain unchanged. - All 506 registered SDK input Git blobs and registry `efa70621e9fb42aa185f7ee616a1a726aa3eccb367e67e28817cedf26ef2ade8` equal the prior qualified source. No new SDK build or runtime refresh. - All 75 private protocol files, production trees, YAML, lockfile, launcher, identities, counts, clocks, and sourceFiles declarations equal the prior source. Only the timeline descriptor oracle advances v8 to v9. - Prior head `ce18f2bd52ed00ce750302f63aecc4167a3daff3` finished CI with 43 passed, one expected skip, four originating runner interruptions, and a failed dependent verify aggregate. Two logs show runner shutdown signals; two annotations show lost runner communication. No assertion failure was observed. Those outcomes are retained; no unchanged recovery was requested. - Required fresh current-head CI, genuine review, thread clearance, and Root's exact-diff acceptance are pending. No merge or new paid run is authorized by this PR's publication. - The earlier 3 PASS / 7 FAIL uniform attempt and all later original outcomes remain unchanged. No new native pass or whole-catalog readiness claim is made. ## Risks The source remains a frozen candidate with unmerged catalog repairs. A real measured run is still required. The readiness witness requires both a real handoff and the exact task URL; it does not extend the heading timeout, bypass auth, skip a case, or change production code. The active measurement on the previous pin is not altered. ## Model Used OpenAI GPT-6.1 (`gpt-6.1-sol`), with reasoning, code execution, and repository tools. The host did not expose a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched for and linked related public PRs above - [x] I have linked public PRs and described the issue - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change - [x] Relevant local pin tests passed; source and fixture qualification scope is stated above - [x] I have updated the expected pin test and the observed fixture boundary - [x] Owning source metadata and this description document the change - [x] I have considered and documented the risks - [ ] All current-head infrastructure CI gates are green - [ ] Greptile is 5/5 with no unresolved material findings - [x] I will address reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
088e959d7b |
test(evals): pin qualified chat scenario follow-up repairs (#19155)
## Thinking Path > - Paperclip helps people manage AI agents and their work. > - The protected TestOps workflow admits one fixed catalog source. > - The retained uniform attempt exposed scenario observation and grading gaps. > - Focused source PRs now contain the reviewed repairs and local controls. > - A new immutable candidate combines those changes without a runtime refresh. > - This PR binds that qualified candidate for a future normal TestOps measurement. ## Linked Issues or Issue Description Refs: #19135, #19148, #19086, #19084, #19093, #19095, #19100, #19112. The previous uniform attempt finished **3 PASS / 7 FAIL**. Its original results and the separate failed control result4587 remain retained. The source repairs correct visible multipart summary evidence, canonical skill navigation, bound approval history, timeline attribution, credential guidance, company configuration questions, and team-install observation. Local qualification is component evidence. It does not establish a new passing uniform-ten attempt. ## What Changed - Pin immutable source `2ec214914cdaa11148ab2527d2203303f1485e2f`, tree `f1dc0c47668e1ed1db1b4d1ead4d3d262edefc0a`. - Update eleven existing oracle-version declarations in the manifest and its frozen descriptor snapshot. - Keep all 244 identities, 242 canonical declarations, requests, setup, human/native counts, clocks, model, profile, retries, permissions, and the 196/244 readiness gate. - Preserve historical manifest provenance and prior failure records. - Keep the launcher and all 75 private protocol files byte-identical to the prior source. This PR changes no controller helper or workflow. ## Verification - Existing focused pin, selection, admission, streaming, API, and reconciliation tests: **131 passed**. - Source owner qualification on the exact published candidate: **675 Node controls and 20 hire DOM controls passed**; all six affected domains are included. The 242-declaration gate passed. - Fresh server declarations and full E2E typecheck passed on the exact candidate under local Node **26.4.0**. The temporary Node24/output tree disappeared; the source owner restored exact frozen dependencies and rebuilt only missing identical-source outputs under Root authorization. Actual AWS Node and the model profile remain unchanged. - Existing SDK guard: **506 registered inputs unchanged**, registry `efa70621e9fb42aa185f7ee616a1a726aa3eccb367e67e28817cedf26ef2ade8`; no SDK renewal or runtime build. - Production server, packages, CLI, UI, YAML, and lock bytes equal the prior source. Only eleven oracle-version descriptor leaves change. Query, clock, count, and policy comparisons passed. - Launcher SHA256 stays `2b9a6953ee51f0ea0dbbd475b0491d9a6c39a1d6835eeaa912e0dbb7cf987724`. Descriptor SHA256 is `41038d984f045f593923da53ac036a9a6adefed5edcaedc609e55991e2fa03b7`. - The next controlled acceptance selection is **only [53, 466, 564, 580]**. Their current source reviews and manual contract reviews are clear. Cases 74, 578, and 610 have separate open observation findings and remain held. This pin does not qualify the whole catalog or a uniform-ten result. - Root approved the final three-SHA pin delta. Exact current-head pin review is 5/5 with no unresolved threads, and Windows passed. Normal CI is terminal and blocked: 42 jobs passed, one expected selector skipped, and six failed. Both direct-task Cloud recovery variants failed the existing initial heading check. The native Runner, package Runner, and server1 test leaves lost communication with their self-hosted runners, as the GitHub annotations confirm. Their host cause is unproven; Spot is not established. The dependent E2E and verify aggregates also failed. Original CI results are retained. One same-head failed-jobs-only recovery was accepted after a fresh terminal/head guard. The independent unchanged direct-task control passed locally in 4.9s; the CI trace shows its five-second heading wait began before the Cloud roundtrip, with the expected page visible in the failure screenshot shortly afterward. This supports a test-boundary hypothesis, not a fixed race or reliability claim. Attempt 2 is pending; no second unchanged retry is authorized. Merge remains blocked until all required gates pass. Individual source PRs retain their separate CI and base-policy limitations; this PR does not claim that all source PRs are merge-ready. - No model call, TestOps submission, archive read, permission change, core merge, or catalog merge occurred during this pin work. ## Risks The candidate contains unmerged catalog changes. The protected source pin keeps that boundary explicit. Component controls cannot replace a real product attempt. Prior failures and unknown historical grades remain unchanged. A future measurement must use the admitted source and original bounds. ## Model Used OpenAI GPT-6.1 (`gpt-6.1-sol`), with reasoning, code execution, and repository tools. The host did not expose a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched for and linked related public PRs above - [x] I have linked related public PRs and described the issue - [x] I have not referenced internal or instance-local Paperclip links - [x] My branch name describes the change - [x] I have run the relevant pin tests locally and they pass - [x] I have updated the source pin regression and descriptor snapshot - [x] I have updated the owning source provenance conditions - [x] I have considered and documented the risks - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a0250284e3 |
fix: preserve proposal revision turns and prove chat completion (#19105)
## Thinking Path > - Paperclip manages work done by AI agents. > - A completed delegated task must report its result to the originating chat. > - That report uses a real provider turn and must stay in cost accounting. > - Ordinary first-task proposals must also support the promised chat acceptance path. > - A revision response can reject an older proposal before it creates the replacement. > - The old rejection transition clears that active response's ownership and blocks its next tool. > - This change proves required completion work, guides ordinary proposal policy and preserves the revision turn under the existing locks. ## Linked Issues or Issue Description No separate issue exists. The original failures remain in [the complete Product run](https://github.com/paperclipai/paperclip/actions/runs/41915965118). The independent continuation repair is #19076. Related #12209 concerns prior-run confirmation binding. This change fixes the ownership reset during rejection, rather than changing binding authorization. **What happened?** The complete 20-case run finished with 13 passes and 7 failures. Three delegated-chat cases counted one required `chat_task_completed` provider turn as unexpected work. Two first-task agents chose optional human-only cards, then correctly received 403 when they tried conversational acceptance. A [reviewed two-case candidate run](https://github.com/paperclipai/paperclip/actions/runs/41931445311) then used the changed first-task skill. Both models inherited the ordinary resolver policy. Clarify/propose/accept passed all 12 predicates. Revise/accept failed after the new response rejected its predecessor's proposal. The saved rejection was followed by `paperclip_runner_tool_binding_not_authorized` on its next tools. The final `native_session_lease_lost` label followed those binding failures; this PR does not assume lease expiry caused them. Both cells cleaned up and published their original grades. **Expected behavior** Keep the existing ordinary-turn bounds and account for one independently proved completion notification. Inherit the normal resolver policy for ordinary proposals that accept chat replies. When an active response rejects an older proposal, save the rejection and retain that response's current ownership so it can create the revised proposal. Keep human, current-source and native completion-review rejection behavior and all real approval restrictions. **Steps to reproduce** The original full run used `4740d251`. The two-case candidate used trusted workflow `7d1e9b34` and target `74da56c4`. Run the focused provider-free controls below. The dual-run API regression first fails because rejection changes the issue to `todo` and clears the newer run's execution and checkout fields. ## What Changed - Prove one completion notification only for native Codex/local `multi-repository`, `clarify-reuse` and `plan-handoff`. Keep the ordinary bounds of 2, 3 and 4. Preserve actual totals of 3, 4 and 5, all run IDs and all usage. - Read those declared totals from the current catalog, which now includes the completion reply. Check the total once and prove that exactly one turn is the required notification. Do not add another turn while waiting. The required-server controls use the actual catalog tasks. - Bind the notification to the same company, agent, conversation generation, completed child, single delivery, accounted native run, saved reply and rendered result link. Reject extra work, duplicate deliveries, unknown events and mutations, including native work without a Paperclip tool call. - Collect workspace operations through the real run API for these three proofs. Keep other collector calls unchanged. Keep the proof controls in the existing required server test lane. - Stop idle polling on a permanent completion-proof error. Continue to retry transient observations within the original 240-second window. - Guide ordinary chat-accepting first-task proposals to omit optional `resolverPolicy` and inherit server/company defaults. Preserve explicit and governed human-only restrictions. Do not lower or recreate saved restricted cards. - Under the existing locked issue and interaction rows, preserve the active responding run when it rejects a different source run's proposal. Change only the ordinary requeue predicate. Keep the saved rejection, notifications, activity and existing Telemetry calls. Keep global status, lease and tool-authority guards unchanged. - Extend the existing native API controls with the actual dual-run rejection/revision sequence and policy, provenance, human, current-source and native completion-review countercontrols. No new workflow or test framework is needed. ## Verification - Current combined head is `a8567ded67e5ab811675390e9a5c5ed2194aee2f`, integrated cleanly on `fb96537e28c4c7bfced4939909cb334392320868`. The upstream completion-reply waiter is preserved. [Current required CI](https://github.com/paperclipai/paperclip/actions/runs/41940780852) passes. Actual test checkout `ff93c034d7bb40f3df7bc75cf51828fd07bb0d29` merges this head into `9c6993f0aab7ebb22a26d78f5d227accb35a6ad1`. - Fresh Greptile is 5/5 on exact head `a8567ded`, with no blocking finding. All addressed review threads are resolved. The separate automated reviewer emitted a paused-work response; that response is not a review approval. - Current required server CI actually passed all 11 native confirmation API cases, 46 completion-proof controls and 13 onboarding asset checks. Canonical typecheck, Build, the native package/server suites, all browser and serialized shards, `ci / e2e` and `ci / verify` pass. No current-head CI retry was needed. - Local dual-run RED: the unmodified transition returned HTTP 200 and saved rejection, then changed the issue to `todo` and cleared the newer run's execution/checkout fields. Whole setup and check took 28.95 seconds; the case took 596 ms. - Local treatment: all 11 real native/API cases pass. The same-session response retains execution/checkout ownership, reads task context and persists the recorded-shape revised confirmation. Its stored lease receipt stays intact and one rejection activity is saved. Human-only rejection, current-source rejection, human rejection with/without a newer owner, Plan/Ask, foreign provenance, authentication and formal approval controls pass. - All 59 existing confirmation resolver controls pass in 16.62 seconds. These cover cancellation, lost ownership, company/user scope, policy changes, transactional rollback, concurrent retries and stale revisions. - All 46 completion controls pass in 4.07 seconds. The two added clock controls preserve the original idle deadline and prove immediate permanent-proof failure plus transient observation retry. The existing live collector-to-proof and native mutation negatives remain in this required server file. - The first current-catalog integration check failed 5 controls in 2.70 seconds. The old consumer added the already-declared reply again and could accept an extra ordinary turn. The corrected consumer passes all 46 controls using the actual catalog. The existing full 20-case catalog/reporting compatibility check also passes in 4.93 seconds. Case identities, configured models and reporting match the declared 20. - The API controls used one owned loopback PostgreSQL instance through existing supported test hooks. It was stopped and removed. Earlier missing-library/setup failures and one corrected current-source test expectation remain retained; they are not counted as product failures or passes. - The unchanged first-task skill has SHA-256 `b789090996e7925676590c006418eb8b57f4514e646f6bca18ffd7b2b8e864de`. Both candidate runtime copies matched it. The inherited policy was observed in both models; one complete acceptance journey passed. This does not qualify the new rejection repair. - The three retained real completion streams pass the stronger local proof, including native items, workspace observations and recorded DOM links. Their original failed grades stay unchanged. - Earlier related checks passed 153 hiring/non-execution controls and 13 asset checks. Their source is unchanged. Required discovery includes the completion, API and asset files exactly once. - Prior candidate `74da56c4` passed all required CI, including canonical typecheck and build, in [CI run 41929148958](https://github.com/paperclipai/paperclip/actions/runs/41929148958). Its fresh Greptile was 5/5. This is prior-head evidence. - The intermediate `053150d9` head also received Greptile 5/5. Its [CI run](https://github.com/paperclipai/paperclip/actions/runs/41940014165) and review stay retained as superseded evidence. They do not qualify the final catalog correction. - A clean full local typecheck/build is not claimed because the reused dependency cache has stale adapter-utils/Runner declarations. No source or lock workaround was made. Fresh canonical CI must pass before merge. - The original full-plan result remains 13/20. The separate two-case probe remains 1/2. A fresh complete saved-plan run against delivered source is required for acceptance. No case, assertion, model, budget, timeout or failed grade is changed. Focused commands: ```sh # From server/; database URLs point only to an owned disposable local instance. node ../node_modules/vitest/vitest.mjs run src/__tests__/chat-completion-proof.test.ts PAPERCLIP_TEST_DATABASE_URL="$OWNED_TEST_DATABASE_URL" node ../node_modules/vitest/vitest.mjs run src/services/native-runtime/confirmation-comment-api.integration.test.ts PAPERCLIP_TEST_POSTGRES_ADMIN_URL="$OWNED_TEST_DATABASE_URL" node ../node_modules/vitest/vitest.mjs run src/__tests__/confirmation-comment-resolution.test.ts ``` ## Risks - The ownership exception applies only to an agent acting as the current execution run against a different non-null source run. Human, unowned and current-source requeue behavior must stay intact. Native completion-review handling remains separate. - The completion proof fails closed on new native tools or event shapes. A new shape needs review. - A stored unchanged lease receipt is supporting local evidence. The final real journey must prove full revision, execution and delivery. - The new head still needs normal protected merge approval. The optional queued Paperclip review is not a passing review. No causal repair of unrelated CI intermittency is claimed. ## Model Used OpenAI `gpt-6.1-sol` in Codex, with `max` reasoning effort. The runtime reports a 475,000-token context window. Used reasoning, code editing, shell tools and provider-free controls. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7c66b6b1f6 |
test(onboarding): cover plan, hiring, and execution approval history (#19069)
## Thinking Path > - Paperclip manages agent work and human decisions. > - Onboarding can approve a plan, a hire, and an execution step in separate turns. > - Tests covered single approvals more closely than that full history. > - The repaired gate must retain valid plan approval through later decisions. > - Every consumer must recognize the saved confirmation kind and selected scope. > - This change tests that contract and registers a complete interview-to-execution journey. ## Linked Issues or Issue Description Refs #19043, #17914, #18283. The earlier tests admitted one accepted proposal at a time. Decomposition fixtures used standard confirmations. The direct-task E2E path forbids hiring. Those cases did not cover an accepted checkbox plan followed by hiring and a separate untargeted execution approval. ## What Changed - Add eight PostgreSQL contract cases for standard and checkbox plan approvals. Native tools create the cards. An authenticated human accepts them through HTTP. A later native run reads the approved revision and actual selected scope. - Exercise native task creation, generic REST creation, child REST creation, and accepted-plan decomposition. Check the assigned child and one creation audit event. Check durable replay on the three entry points that support it. A pending execution confirmation still prevents creation. - Register the explicit-only `onboarding-approval-history` suite for local native Codex and ACPX Claude. The browser chooses the interview, selects only drafting in the saved plan, approves one writer hire, and approves a separate Start drafting confirmation. The model must hire the writer and use that writer's own child execution to save the welcome document. - Grade each human decision, the unchanged plan revision and selection, approval timing, one assigned child and worker execution, the exact writer-authored saved revision, no pending requests, and persistence after reload. Calibrate the oracle against missing, forged, premature, duplicate, and changed outcomes. - Preserve synthetic option choices, document/run bindings, and saved revision authorship in the existing public first-task evidence projection. Keep private provider/account data excluded. Document the workflow, evidence, billing, and cleanup bounds. - Align the approval-history fixture with its completion oracle: the onboarding parent must be Done. Test both native profiles against graded completion evidence and reject a completed child whose parent remains In review. Count parent heartbeats separately from the writer: accept one to seven parent records while retaining the twelve-company-run cap and the independent three-approval/one-writer oracle. - Retain the existing 52-cell first-task suite. The new suite has one attempt, seven expected work turns, a twelve-run maximum, a fifteen-minute deadline, and the existing $5 company task budget. It is excluded from `--all`. - Update two stale cleanup assertions to include the existing thirty-second deletion timeout. ## Verification - Red control: temporarily restore the pre-fix admission module and run the new checkbox/native history case. It fails with the expected `First-task proposal requires human acceptance` error after plan, hiring, and execution acceptance. Restore the repaired module after the probe. - The final post-rebase history matrix and existing onboarding safety suite passed together: 74 tests, no skips. - Final focused Runner E2E support passed: 413 Vitest tests and all 370 Node support checks. This covers catalog discovery, journey controls, wrong-outcome calibration, public evidence, and existing first-task behavior. - Review controls: four authorship cases fail before the revision check. A delayed browser-response control fails before the approval-time fix. Both pass after repair. The final oracle requires the exact current document revision, writer ID, native child run ID, matching body, and readback after reload. - Credential-free discovery lists both new native cells. No provider call was made. `OPENAI_API_KEY` and `ANTHROPIC_API_KEY` are absent, so the real provider journey remains unqualified. - The earlier broad credential-free E2E support run reported 6,728 passing cases, three skipped cases, and nine failures in three unrelated files. All nine reproduce with the changed E2E source restored to the unchanged baseline: three expired overhead-fixture cases, one stale Pi profile-version assertion, and five workflow security assertions. This change does not alter those files, paid workflow policy, or provider pins. - Previous full workspace typecheck/build and [Linux CI](https://github.com/paperclipai/paperclip/actions/runs/41921754638) passed on `4f431f5aae99d072cc75027e90d7b847b4c342ca`, including the new eight-case PostgreSQL file. Ponytail subsequently found a fixture/oracle terminal-status mismatch. Both native-cell controls fail on that fixture (`done` observed versus `in_review` expected), while the parent-in-review negative control passes. A second comparison uses the actual `matchesRunCount` predicate: graded evidence with six parent runs and one writer run fails the old exact-seven-parent requirement. Both fixes pass these controls, all 413 focused support tests, all 370 Node checks, and the owning E2E project typecheck. [Fresh CI](https://github.com/paperclipai/paperclip/actions/runs/41936699686) passed on `91c980b110252da451b92eda597c168517de1276`: 48 successful jobs and one expected skip. Greptile reviewed this head at 5/5 with no unresolved findings. Ponytail's terminal-status finding is fixed and its thread is resolved; its current-head re-review remains queued. Live provider cells remain unqualified. ## Risks The production API and approval gate are unchanged. The new paid journey can expose model or product failures at an intermediate approval boundary. It retains those failures and uses the existing diagnostics and cleanup. Registration and deterministic tests do not establish a live provider pass. The unrelated baseline support failures remain explicit. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused tests; unrelated baseline failures are recorded above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
9c6993f0aa |
test(evals): pin corrected chat catalog and restore retained attachments (#19135)
## Thinking Path > - Paperclip uses real Product E2E journeys to verify agent work. > - The trusted TestOps controller must bind the reviewed source and retain inspectable evidence. > - The last uniform attempt exposed bounded observation and evidence-retention defects. Its original outcomes remain unchanged. > - The reviewed successor corrects those boundaries with the original model, requests, counts, and clocks. > - Its private framework report stores canonical attachments once. The controller must validate and restore those references before normal reporting. > - This PR ports that reader support and pins the qualified source in one infrastructure change. It does not merge the full catalog or its core repair PRs. ## Linked Issues or Issue Description Refs #19127, #19083, #19086, #19084, #19093, #19095, #19100, #19112. **What happened?** The last uniform attempt retained real failed outcomes. It also lost the multipart-question original when evidence packaging exceeded its unchanged physical byte allowance. Review found independent boundaries in skill navigation, announcement observation, timeline attribution, hire history, credential questions, and team visibility. **Expected behavior** The next run must execute the same ten requests against one reviewed source. The controller must retain complete original evidence, validate private attachment references, and report the actual outcome. Historical failures and missing-original UNKNOWN grades must remain unchanged. **Steps to reproduce** Use the focused source controls linked above. The private framework control uses a real failed Playwright test and a 140 MiB attachment. Legacy duplicate carriers hit the old cap. Canonical original evidence passes both seals and strict archive restoration. The controller restores derived files and the existing TestOps exporter preserves the failed result. **Paperclip version/commit** Qualified candidate: `155dd1bc1c4b85ae107f6a0cd429720caa281367`. The controller port is based on current master. The full candidate catalog stays on its own immutable branch. **Deployment mode** Existing AWS/TestOps job 100, plan 102, branch master. This PR does not dispatch a run. ## What Changed - Pin the published qualified source and its exact launcher and descriptor fixtures. - Preserve current manifest history. Update twelve declaration fields across eleven cases: oracle versions and skill source-file provenance only. Preserve all 244 identities, 242 canonical declarations, queries, setup, human/native counts, clocks, profile, model, retry policy, and global cap of 8. - Port the exact source-owned compact framework helper and private-transfer reader. Keep the three existing required source/controller protocol helpers byte-identical. - Validate both sealed inventories before restoring registered attachments into the derived selected report tree. Verify exact bytes and hashes. Preserve seals and unregistered evidence. - Keep deep JSON, decoded-body, sensitive-key, path, and original/final status checks. Plain-text bodies and registered plain files fail closed if existing diagnostic redaction would change them; JSON remains under structured inspection. Keep all existing physical byte, entry, ZIP, time, privacy, and archive bounds. - Preserve registered clean metadata and attachment bytes. Reject content that needs privacy redaction instead of silently rewriting immutable evidence. ## Verification - 131 existing Node pin, selection, admission, adapter, reconciliation, and security controls pass. The explicit new target assertion and drift negatives remain active. - 86 current-controller controls pass. They cover legacy reports, privacy and static assets, canonical reference guards, both seals, strict archive restoration, derived files, the actual report normalizer, and the actual TestOps/Allure exporter. The original failed result stays failed; screenshot and trace hashes resolve. - Actual P1 controls reject both an unknown password in a decoded plain-text body and an unknown command token in a registered text file. Parent code fails both controls; corrected code passes. The composed source passes 87 affected controls, fresh server declarations, and full E2E typechecking. The old cfe head completed all 49 CI jobs without adverse conclusions, but its review found this real privacy defect. That CI result is not credited to this successor. - The existing report-lock preflight returns already-reviewed at `60f4c06bca47af3ab9d74281f2ef2e1ff559d566b862ba51c2fbe6c8e5bac35b`. - The existing source SDK guard confirms 506 registered inputs, unchanged registry `efa70621e9fb42aa185f7ee616a1a726aa3eccb367e67e28817cedf26ef2ade8`, and unchanged runtime pins. No registry renewal or SDK build was needed. - Source launcher SHA-256: `2b9a6953ee51f0ea0dbbd475b0491d9a6c39a1d6835eeaa912e0dbb7cf987724`. Descriptor SHA-256: `e026d5c000102f8d50397dc59055d187388304f91dbca9a6ccfbe747004cd16d`. - No model call, evaluation dispatch, evidence download, reporting retry, core merge, or permission change was performed. Full current-head CI and review are pending. Separate source PR failures remain disclosed; local controls do not establish universal source CI readiness. - Seven cases have verified passing historical attempts. A uniform passing ten-case run remains unverified. The old lost multipart-question original remains UNKNOWN. ## Risks This changes a trusted evidence reader and the source selected for future measurements. The new helper is included in the controller import closure. Compact references require matching originals; missing, foreign, changed, unsafe, or redaction-required evidence fails closed. Legacy reporter formats remain valid. A single canonical evidence set can still exceed a bound. Current-head CI and material review must pass before normal protected merge. Actual AWS/TestOps evidence remains required afterward. ## Model Used OpenAI `gpt-6.1-sol`, with reasoning, code editing, and tool use. The host does not expose the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandClaude Opus 5.5
|
ac57b183f0 |
fix(evals): run campaign cells when TestOps selection is skipped (#19053)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Runner Full-Stack E2E workflow runs the paid Product E2E cells, including the Product regression TestOps plan > - #18983 added an optional `testops_selection` job and made `catalog` depend on it > - That job is skipped in every ordinary manual or scheduled campaign > - GitHub's implicit `success()` also fails when an ancestor job was skipped, so every job below `catalog` was skipped > - Since #18983 merged, ordinary campaigns run no cells: the merge reports every selected cell as "No result artifact was uploaded" > - This pull request gives each job below `catalog` an explicit status condition > - The benefit is that ordinary, nightly and branch-targeted campaigns run their cells again, and TestOps campaigns are unchanged ## Linked Issues or Issue Description Refs #18983, #19000 **What happened?** Run https://github.com/paperclipai/paperclip/actions/runs/41914430034 selected 20 Product regression cells with the `id` input. `catalog` passed. `daytona_image`, `build_runner_artifacts`, `build_remote_provider_pack` and every matrix job were skipped. The merge failed with 20 "No result artifact was uploaded" rows. **Expected behavior** An ordinary manual campaign runs its selected cells when `testops_selection` is skipped, as it did before #18983. **Steps to reproduce** Dispatch `runner-full-stack-e2e.yml` on `master` without `testops_case` or `ALLURE_JOB_RUN_ID`. The scheduled daily campaign has the same path. ## What Changed - `daytona_image`, `build_runner_artifacts`, `build_remote_provider_pack`, `test`, `test_recurring` and `test_manual_private` now start with `!cancelled()`. - Each of those jobs now requires `needs.<job>.result == 'success'` for each of its real dependencies. A failed or skipped build still skips the cells, as before. The existing schedule, cadence and evidence-route conditions are unchanged. - A new regression test parses the workflow. It requires every job below `testops_selection` to use a status function and to require each real dependency result. ## Verification - `node --test tests/runner-e2e/product-regression-testops.test.mjs`: 15/15 pass. - Negative control: the new test fails against the current `master` workflow. - `daytona-image.test.ts`, `workflow-target.test.ts` and `workflow-timeout.test.ts` pass. `workflow-security.test.ts` has 5 failures. The same 5 fail on unchanged `master` (they concern `pr-trusted.yml` and `chat-catalog-testops-case.yml`). - A workflow change takes effect only from `master`, because paid campaigns run only from the default branch. After merge, dispatch an ordinary `id=` campaign and confirm the cells run. ## Risks - Low risk. The conditions keep the previous failure propagation: a job runs only when each direct dependency succeeded and the run is not cancelled. - `catalog` success already implies `authorize` and `target_lock` success, so those are not repeated. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), via Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandClaude Opus 5.5
|
23cd123ff8 |
fix(e2e): retain the closed cause code of a journey cleanup failure (#19079)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The Product E2E journey harness runs the QA staging smoke plan and
reports each case to TestOps
> - Each case runs a journey, then releases its owned target resources
(cleanup)
> - Three recent cleanup failures had a passed journey and all checks
passed, but each result only says `target_cleanup_failed`
> - The adapter throws an error with a closed identifier code, and the
harness discards that code, so no one can find the failing cleanup step
> - This pull request keeps the identifier-shaped cause code as an extra
cleanup phase
> - The benefit is that the next cleanup failure is diagnosable from
TestOps without private runner files
## Linked Issues or Issue Description
**What happened?**
QA staging smoke cleanup failures show only `target_cleanup_failed`.
This occurred for `question-answer-resume` (launch 2390), and for
`company-access-isolation` and `create-skill-studio` (launch 2693). In
each case, the journey and all behavioral checks passed. The private
runner report is deleted with the runner, and the adapter error code is
not kept in the retained result. The cause is not known.
**Expected behavior**
A cleanup failure keeps a closed, identifier-shaped cause code (for
example `fixture_cleanup_not_confirmed`). The result never keeps the raw
error message.
**Steps to reproduce**
1. Run a Product E2E journey with a target session whose `release()`
throws an error that has a `code` property.
2. Read `normalized-results.json`.
3. Before this change, the cleanup phase only contains
`target_cleanup_failed`.
**Paperclip version or commit**
master at
|
||
|
DottaandClaude Opus 5.5
|
b1a71044af |
fix(evals): observe chat handoff replies and restarted task pages (#19073)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The Product regression TestOps plan runs 20 Product E2E cells
through `runner-full-stack-e2e.yml`
> - The full 20-case TestOps launch on `master`
|
||
|
Dotta
|
071b4dd5c3 |
fix(evals): preserve continuation restart and approval boundaries (#19076)
## Thinking Path > - Paperclip manages work done by AI agents. > - Product E2E journeys test that work through the UI and public APIs. > - A continuation must retain completed work across a server restart. > - A human-only plan request needs a recorded human decision. > - The current fixtures retain an old Vite client and dismiss the plan request before sending a comment. > - This change uses the existing restart guard and approves the exact current plan through its UI. ## Linked Issues or Issue Description No separate issue exists. The failures are retained in [the original complete Product run](https://github.com/paperclipai/paperclip/actions/runs/41915965118). **What happened?** `completed-action-resume` reached its intended completed-child and pending-question state. Its navigation then hit the original 30-second limit after a server restart. The retained trace shows the old Vite client polling during listener replacement. The existing chat restart wrapper unloads that client, but the continuation callback did not use it. `revision-preserves-approval` saved the revised plan and final output. Its exact current human-only confirmation remained pending. The fixture had clicked Dismiss to enter a plain approval comment. A comment does not resolve that card, so the final pending-work checks correctly failed. **Expected behavior** Unload the old client before the existing restart. Use the real Confirm action on the exact current plan revision. Require its saved human acceptance before the normal continuation settles. Preserve all original final checks and limits. **Steps to reproduce** Run those two existing continuation cases from source `4740d251`. The original run preserves both failures. The focused controls below require no provider credentials. **Paperclip version or commit** Original failure source: `4740d251b7032e714d33039ba66a56948238c38f`. **Deployment mode** Isolated local Product E2E servers in GitHub Actions. ## What Changed - Route only the continuation restart callback through the existing `restartChatServer` wrapper. - Select the one pending confirmation for the exact saved plan revision. Recheck it before the UI action. - Click its scoped Confirm button. Require the same target, accepted outcome, human resolver and timestamp in the saved receipt. - Preserve the generic chat Dismiss behavior, ordinary reply cases, original prompts, models, run bounds, grading and deadlines. - Add focused unit and browser controls for the approval boundary. ## Verification - The original full run retains these two failed grades. Cleanup passed in both; that does not make either case pass. - Focused continuation unit controls: 44 passed across four files. The final new helper controls also passed 3/3. - Three distinct controlled browser checks passed: the existing old-client restart guard, the exact current human-only Confirm action, and rejection of a changed revision without any approval POST. - The browser controls use local HTTP fixtures. They do not prove a fresh provider journey passed. - All six files match the tested handoff hashes. `git diff --check` passes. - Local harness typecheck reached six diagnostics from stale cached dependency declarations. The missing exports are in unchanged adapter-utils and Runner code. No owned-file diagnostic remains. No source or lockfile workaround was used. Fresh CI typecheck, build and tests remain required. - Fresh real Product acceptance has not run for this repair. The original 20-case result remains 13 passed and 7 failed. Current-head CI passes at `335683b741df5a3d0b76d035e105f0b44eda832d`: required `ci / verify` and `ci / e2e` are green. [CI run](https://github.com/paperclipai/paperclip/actions/runs/41922544434). Greptile is 5/5 on this head. Ponytail is 4/5 with a ship recommendation; its checkpoint warning was resolved with the dominating awaited snapshot/push proof. No source changed after these checks. Real changed-source journey acceptance remains pending. ## Risks - The browser controls model the restart and approval boundaries; the original real cases must still run after delivery. - A changed or stale confirmation now fails before an approval. A dismissed card or an agent-authored claim cannot satisfy human acceptance. - This fixture correction grants no new product permission and changes no provider or scenario limit. ## Model Used OpenAI `gpt-6.1-sol` in Codex, with `max` reasoning effort. The runtime reports a 475,000-token context window. Used reasoning, code editing, shell tools and provider-free browser controls. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
52b4b3ccd7 |
test(evals): pin native credential delivery and worker receipt fixes (#19042)
## Thinking Path > - Paperclip must deliver agent work through ordinary human controls. > - The latest three-case run retained three failed originals. > - The personal value was saved, but the native OpenCode child did not receive the configured task environment. > - A worker proof also relied on legacy process identity fields outside that environment contract. > - Timeline and credential checks had independent review defects that could reject truthful outcomes or miss unsafe instructions. > - This PR pins the reviewed fixes together and keeps all prior grades unchanged. > - A new AWS measurement must still prove the real outcome. ## Linked Issues or Issue Description Refs #18963, #18967, #18968, #18971, #18972, #18977, #19027, #19031. Follows merged #18995. This is the next infrastructure pin, not another core implementation PR. The latest launch 2657 retained FAILED originals for timeline, approved-tool replay and personal secret. The personal journey reached secure UI save and a worker wake, but the worker reported no credential and made no endpoint call. These are failed attempts, not new passes. Six of the selected ten cases have independent passes across prior attempts. The uniform final ten and the full catalog remain unqualified. No old result was regraded. ## What Changed - Pin immutable candidate `c6cc59282a45ed3c79880a220f7f60aa3ac1d6a4` on `codex/chat-credential-destination-contract-candidate-20261104`. - Retain the frozen candidate baseline. Add the accepted failed-action completion feedback hint, canonical configured environment delivery to OpenCode, actual native worker output receipt checks, timeline attribution controls, and credential clause safety checks. The feedback change leaves the existing acceptance gate in place. - Declare exactly seven oracle-version changes: timeline v5 to v6; personal v5 to v6; company v4 to v5; secure API key, least-scope delivery, secret proposal and vault reference each external v1 to v2. The four external cases have explicit overrides; the shared default is unchanged. - Sync the manifest rows, explicit changed-field records, descriptor snapshot and expected target. Preserve all 244 identities, original queries, human/native counts, clocks, model, profile, launcher bytes, privacy limits and the existing 196/244 qualification gate. - Include the reviewed retry cancellation readiness fixture repair from `815d47e4b5d7756367d2c5e2da656b640273eac2` in this infrastructure PR only. It awaits the first scheduled batch before polling its state. Its production behavior and assertions are unchanged. This test repair is absent from the model candidate. - Keep the core PRs, including #19027 and #19031, unmerged. A normal merge of this pin does not merge those PRs. ## Verification - Pin-local checks: all 189 existing manifest, TestOps API, adapter, reconciliation and streaming controls passed. The reviewed 815 fixture repair passed its owning real-PG control (one selected test). A temporary delayed-first-batch control failed with the old observer and passed with the new observer. Batch limits, assertions, drain wait and overall budget are unchanged. The original CI cause remains unknown. - Final candidate: 89 credential/receipt consumer controls passed. The exact Runner TypeScript build, 68 driver controls and one owning failed-action PG control passed before the later helper-only changes. Their runtime inputs remain byte-identical; those results are reused explicitly, not claimed as new executions on every source commit. - Real pinned OpenCode 1.18.34 executed a canned local-provider bash call. Synthetic `PUBLISHING_API_KEY` reached its child in a 2.89-second passing control with zero paid provider calls. This proves server-to-bash inheritance; driver spawn controls cover the preceding projection boundary. Actual AWS secure-entry, worker delivery and privacy acceptance remain pending. - **The latest local candidate declaration loader is NOT GREEN.** It exceeded its unchanged 30-second limit in both a combined run and an isolated run. Five other declaration/contract checks passed. Both failures remain recorded. No timeout was widened and no further retry occurred. Descriptor and definition modules match the earlier passing 5aa source; only the transitive credential helper changed. Prior 242-declaration evidence supports unchanged definitions only, not a claim that these failed executions passed. - The snapshot matches the candidate and exactly seven oracle-version leaves differ from source 249. All 244 identities, queries, run/human counts, clocks, model, profile, launcher and privacy limits remain unchanged. The SDK guard reports 506 unchanged inputs, unchanged runtime pins and registry `efa70621e9fb42aa185f7ee616a1a726aa3eccb367e67e28817cedf26ef2ade8`. The driver is outside those SDK roots. No SDK renewal occurred. Reporting-lock and whitespace checks passed. - Genuine exact-head reviews: source 19e3 is 5/5 with no unresolved threads; current pin 988 is 5/5 with no unresolved threads. The prior pin 4/5 and resolved P1 remain historical. Current exact-head normal CI 41917215227 completed successfully: 48 successful jobs and one selector skip, including final verify job 126888275415. Windows 41917215440 succeeded. The full PR rollup is 56 successes and three intentional skips, with no pending or failed checks. Older green heads do not qualify this head. Paperclip has only an earlier-head 5/5 assessment, which is stale for this head and receives no current assessment credit. - Six of ten cases have independent passes across prior attempts. Launch 2657 retains three FAILED originals; no grade was changed. The next measurement is the exact uniform ten, cap eight and zero automatic retries. No uniform-ten or full-catalog acceptance is claimed. - The local-tests checkbox below covers only the 189 pin controls and owning fixture control. It does not claim all candidate or repository tests are green. Full local repository checks were not repeated for this infrastructure diff. Core PRs remain unmerged. No model call, eval dispatch, private archive read, fleet change or merge occurred for this update. ## Risks Environment delivery crosses a real provider process boundary, so secret isolation remains part of the next actual AWS acceptance. A worker receipt must match an actual successful native run and key-authenticated endpoint call; a copied boolean does not qualify. Oracle safety and attribution changes can alter acceptance, so balanced negatives remain required. No current uniform-ten pass is claimed. Normal CI and genuine exact-head review must finish before protected infrastructure merge. Root owns the next TestOps Submit, with the original model, limits, zero automatic retries and maximum eight concurrent case chains. ## Model Used OpenAI `gpt-6.1-sol` through Codex. Capabilities used: source analysis, mechanical integration, local code execution and PR preparation. The host does not expose the context-window limit. The evaluated model remains OpenCode/OpenRouter `deepseek/deepseek-v4.1-flash-1017`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
c99bcd89fe |
fix: preserve Product TestOps evidence and CI readiness (#19000)
## Thinking Path > - Paperclip manages agent work through governed product workflows. > - Product E2E tests check those workflows through the app and Runner. > - A trusted reporter sends retained results to TestOps. > - Free-form metadata and a short upload observer can lose safe, valid evidence. > - UUID navigation can also discard task metadata before the canonical query returns. > - Hermetic CI files start child processes that need CPU capacity within their readiness limits. > - This change closes the reporting contract, retains authenticated core aliases, and reserves CI CPU capacity. > - Original case identities, conditions, assertions, verdicts and deadlines remain intact. ## Linked Issues or Issue Description Refs #18983. This addresses its late [metadata disclosure finding](https://github.com/paperclipai/paperclip/pull/18983#discussion_r4470789114) and [upload readiness finding](https://github.com/paperclipai/paperclip/pull/18983#discussion_r4470790366). **What happened?** The reporter accepted model IDs and matcher names through syntax filters. A secret-shaped string could pass those filters. First-task results also use their top-level model field for an observed alias. Treating it as configured metadata can reject valid retained evidence. The upload observer stopped after 22 seconds of waits. [Required CI on the earlier head](https://github.com/paperclipai/paperclip/actions/runs/41886885193) also found two real browser title failures and five native package readiness failures. A deterministic control reproduced metadata loss during UUID-to-identifier navigation. A separate fixed-resource control reproduced startup deadline failures with four concurrent hermetic fixtures, while one and two fixtures passed. **Expected behavior** Reject unknown transmitted metadata before creating upload files. Preserve every original matcher verdict and attempt receipt. Observe server processing without rerunning the product. Keep authenticated task metadata visible through canonical navigation. Reserve capacity for hermetic child processes without changing the original readiness deadlines. **Steps to reproduce** Use a secret-shaped model ID or cleanup state in retained evidence, return an observed first-task model alias, or let upload processing settle after 40 seconds of virtual time. Hold the canonical core query pending after an authenticated UUID read. For the CPU control, run the same providerless startup/recovery fixture alone, four times concurrently, and twice concurrently under a fixed 4-vCPU quota and its original 8-second readiness limit. ## What Changed - Bind configured model receipts to each exact declared case model. Four first-task cases use captured agent configuration; their observed aliases and fallback text remain digest-only. A missing capture stays null / Not recorded. The other 16 cells use their configured profile model. Reject wrong or unknown configuration before any upload file exists. - Keep the existing model distribution: 18 Codex cells on `gpt-5.6-sol`, configured-repository case 9 on its existing `gpt-6.1-sol` override, and task-workspaces case 54 on `openrouter/deepseek/deepseek-v4-flash-0731`. The repository override exists at the [original canary source](https://github.com/paperclipai/paperclip/blob/86814c6550f52e99ede150a883bdc773cbb540e3/tests/runner-e2e/catalog.ts#L1148). No model configuration changes. - Allow only existing Matcher kinds, closed failure classes, and cleanup states `passed`, `failed`, or `not_started`. Publish each matcher's kind, ordinal, original boolean verdict and SHA256 path receipt. Full conditions remain in the linked original workflow artifact; raw dynamic path bytes do not enter TestOps. - Observe the same upload for at most 55 seconds of poll waits. Product scenarios, provider retries and scenario deadlines do not change. - Retain task core data under only the authenticated response's own UUID and identifier core keys. Preserve existing canonical data. Keep arbitrary references and full-detail keys empty until their own reads. - Retain a first-failure trace for the existing Cloud recovery test. Keep its assertions, timeout, retries, mock data and artifact policy unchanged. This adds diagnosis evidence; it does not claim to fix that separate Cloud title timeout. - Reserve half the available CPUs for Runner package CI workers, with the existing upper cap of four. A 4-vCPU host uses two workers; an 8-vCPU host still uses four. Local defaults and test selection do not change. ## Verification - Reporting controls first reproduced the two original gaps. All 14 Node contract tests now pass. Unknown configuration creates neither a result directory nor an export summary. A model valid for a different selected case is also rejected. Original functional failures and earlier accounting receipts survive. - The catalog compatibility test passes for all 20 selected cells, exact model tuples, real pure grader outputs and actual `projectFirstTaskResult` shapes. It covers observed aliases, fallback text, absent evidence and wrong or secret-shaped configured metadata. These are contract controls, not live journey passes. - Virtual-clock controls accept a correct result that settles at 40 seconds and reject a persistently wrong verdict after 55 seconds. They make no product or provider calls. - Two new core-cache controls failed before repair. The query/cache suites now pass all 8 tests. They hold canonical refresh pending and verify title/company retention, newer canonical data, full-detail enrichment and unrelated keys. UI typecheck and token gates pass. - The unchanged exact UUID cross-company link/upload browser test passed once after repair: 5.8-second scenario, 32.0-second complete command. The unchanged exact Cloud task-entry diagnostic passed once: 5.4-second scenario, 23.3-second complete command. Its original canonical-only timeout remains unexplained. All five Cloud tests still collect after the trace setting. - The providerless Linux control used the retained Runner and compiled controller from source `3a2249dd`, with a fixed 4-vCPU quota and original 8-second readiness limit. Solo passed 1/1; four concurrent fixtures failed 4/4; two concurrent fixtures passed 2/2. At each failure, Rust had accepted command 2 but startup was still pending; checkpoint and published status agreed, one connection was authenticated, no replay backlog existed, and the sidecar had not launched. Passing controls preserved the expected startup failure through controller replacement. No network or credentials were available. - That control uses Linux amd64 emulation and Node 24.21; hosted CI used Node 24.22.1. It supports the worker-cap change, but does not prove that the original `m5n.xlarge` host caused every CI failure. The original red attempts remain recorded. - Actual config-load controls verify 1 CPU → 1 worker, 2 → 1, 4 → 2, 8 → 4 and 16 → 4, with no local, timeout or retry override. The isolated policy change preserves the same 297 test-file inventory; integration of current master `89703c3ff7` retains all 298 current native package files. The original PR package/native command plans remain unchanged. - Final head `877e28da32f2d2f0d12906c89aff085e2c6ea943` passes both required checks, `ci / verify` and `ci / e2e`, in [CI run 41897677652, attempt 2](https://github.com/paperclipai/paperclip/actions/runs/41897677652/attempts/2). The native package passes 4,950 tests with 10 existing skips. All eight browser shards pass; the Cloud shard passes all 41 tests, including all four unchanged recovery cases. - Attempt 1 preserved 46 successful jobs but its one-file heartbeat recovery shard hit the original 20-minute job limit without a named assertion or completion verdict. Its cause remains unidentified. One explicitly bounded debug replay retained those 46 successes and ran only that leaf and its dependent rollup. The unchanged file passed all 412 tests (346.91-second file, 371.34-second Vitest command, 7m11s job). Source, cases and deadlines did not change. This establishes intermittency, not a causal repair. The original timeout and debug pass both remain recorded. - Fresh [Greptile 5/5](https://github.com/paperclipai/paperclip/pull/19000#issuecomment-6637604918) and [Ponytail 5/5](https://github.com/paperclipai/paperclip/pull/19000#issuecomment-6637489454) reviews name exact head `877e28`. The single review thread is resolved. Live TestOps bootstrap and complete 20-case acceptance remain held until reviewed delivery. CI and focused controls do not count as product plan passes. ## Risks - Future model or Matcher kind changes require a reviewed reporting-contract update. - TestOps shows a closed Matcher kind and ordinal. The path digest binds the original condition in retained evidence. - Core alias data retains the existing authenticated query-client and server company boundaries. It never becomes full-detail evidence. - Fewer CI workers can increase one job's duration. The full suite must verify the final policy. The paid evaluation matrix and its parallelism do not change. - The original Cloud title timeout and the first-attempt heartbeat CI hang do not have proven causes. Current unchanged assertions pass, but this does not establish repeatability. The Cloud test now retains its first-failure trace. A successful upload or focused control alone cannot establish a complete product pass. ## Model Used OpenAI Codex with reasoning, tool use and code execution. The host does not expose the exact model ID or context window to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
513c03cb3e |
fix(e2e): configure supported rolling agents with warm policy (#18994)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Rolling tests must exercise the lifecycle they claim to cover.
> - The warm lane selected a warm environment.
> - Its profiles also set an explicit per-turn agent policy, which took
precedence.
> - The test then waited for a live workspace that per-turn cleanup had
correctly retired.
> - This pull request sets warm agent policy for providers admitted by
the production warm capability.
> - The matrix now checks effective lifecycle and preserves per-turn
coverage for every qualified native profile.
## Linked Issues or Issue Description
Builds on merged #18982 for the source controller dependency link. This
PR now targets master.
**What happened?**
The warm rolling fixture built a Codex agent with `lifecycleMode:
per_turn` and an environment with `runnerLifecycleMode: warm`. The
explicit agent policy won. The initial task completed, its disposable
workspace retired, and the fixture waited for a live file before it
reached deployment.
**Expected behavior**
Warm rolling, legacy migration, remote-file and controller-outage cells
must create actual warm agents. Only providers supported by the
production managed warm-session capability belong in that lane. All
qualified native profiles retain per-turn coverage.
**Steps to reproduce**
Build the old warm Codex cell's agent and environment and compare their
lifecycle policies. The new effective-policy regression fails on the old
catalog and passes on this branch. No duplicate exact-scope warm-fixture
PR was found.
## What Changed
- Share the production managed warm capability filter and explicit warm
agent policy across rolling, legacy migration, remote-file and
controller-outage suites.
- Check effective agent and environment policies and update matrix
coverage/counts.
- Read completed warm turns from the exact saved snapshot while
independently requiring the retained, bound workspace location.
- Keep both rolling lanes separate from the outage simulation and
exercise their assembled supervisor requests.
- Version the corrected definitions and keep every qualified native
profile in the per-turn lanes.
## Verification
- Baseline regression: failed against the unchanged catalog; retained
before the fix.
- Fixed catalog/controller selection: 39 checks passed.
- The first real warm-policy attempt completed and saved its workspace
but timed out because the file oracle demanded live content; retained
trace responses prove the API returned the correct saved snapshot with a
retained lease.
- A new regression fails on the previous reader and passes after
separating saved file source from retained versus released location.
Focused catalog, reader and rolling checks: 81 passed.
- The next warm attempt reached replacement and exposed a second fixture
defect: warm also selected a SIGKILL outage, which the supervisor
rejected alongside rolling mode. The failure remains retained; its four
owned sandboxes, temporary key and local bucket were deleted.
- An outage-selection regression failed before the fix; 57
outage/supervisor checks pass afterward, including assembled requests
for both rolling lanes. The fresh warm Codex run passed all 56 checks in
8.5 minutes: original run/session/runner/lease retained, authenticated
takeover, both predecessor homes removed, later native continuation and
saved file readback. Two replacement receipts recorded 742 ordinary
reads and 75 writes, all 75 read back, with zero HTTP probe failures. A
transient `native_workspace_sync_out_failed` phase recovered before
successful finalization and remains retained. This source-lane result
does not claim released-image or hosted Railway qualification.
- The invalid earlier attempt was interrupted before deployment. Its
four archived disposable sandboxes were deleted and verified absent; its
temporary key and local bucket were removed.
- Broad local typecheck reached stale shared database declarations in
this checkout's linked build dependencies. The fixture's own type error
was corrected; clean-head CI remains required. No full build is claimed
locally.
- An incidental report-catalog test selected by a substring filter
failed to generate its normalized output. That failure is retained
separately; the focused catalog/controller checks pass.
- Review follow-up: fixed the same policy conflict in the base
file-storage, legacy migration and controller-outage suites. The roster
now has 822 valid cells; unsupported warm combinations remain covered
per-turn. Updated the explicit supervisor-request control to use a
supported warm provider. Focused catalog, file-reader, migration,
rolling and supervisor checks pass (133 checks).
- Earlier full CI passed at
|
||
|
DottaandPaperclip
|
3bd52a9fa0 |
test(evals): pin task-history discovery and governed handoff repairs (#18995)
## Thinking Path > - Paperclip must complete work through ordinary human and agent controls. > - The paired timeline attempt omitted a real human handoff, although the other ten checks passed. > - The agent used the always-exposed active-task history tool as if its comments were a status audit. > - The tool now states its actual scope and directs chronology and handoff questions to the existing discovery and API tools. > - A separate approved-tool attempt preserved permission and write boundaries, but paired Markdown emphasis caused a false outcome rejection. > - The reviewed oracle normalizes that formatting before the existing outcome checks and retains balanced negatives. > - This PR pins those corrections with the already reviewed personal secure-handoff admission repair for one new representative batch. ## Linked Issues or Issue Description Refs #18962, #18968, #18971, #18972, #18977. Follows merged #18976. The original paired results remain FAILED. Result 4412 omitted the actual human handoff; this is a real answer gap, not a date-oracle excuse. Result 4411 correctly handled the first rejection with no writes or Trust action and rejected agent permission, but the observer misread paired inline emphasis in the truthful outcome. The personal-secret result 4378 remains FAILED at admission; secure save and worker proof were not reached. No prior grade is rewritten and no new native pass is claimed. ## What Changed - Pin immutable candidate `249b487a2d2451763913038cc009f7090580d1f1` on `codex/chat-catalog-human-answer-admission-candidate-20261104`. - Extend exact prior source `b1119630003785c9e26e14c763859125642ad756` with only the accepted timeline-discovery and formatting corrections: 12 source, test and generated files, with 72 additions and ten removals. - Use the canonical `get_task_history` description in the actual exposed semantic catalog. It reads only bounded active-task comments; calling `get_task` does not change that active scope. It is not a status or assignment audit. Progress and handoff questions must discover the company timeline or issue activity, then use the existing API call. - Regenerate semantic catalog and protocol projections on this frozen candidate. Preserve all non-description fields, schemas and authority boundaries. Do not copy newer master catalog or document-schema changes. - Normalize paired inline emphasis before approved-replay outcome checks. Declare only that oracle as app-review v6. Keep timeline v5, personal v5 and company v4. - Retain the reviewed personal completion-context, addressed-human, optional-note and rendered-placeholder safety controls. Actual private save, owner isolation, audit, authorized worker delivery and native continuation remain required. - Update only the existing manifest, expected pin and descriptor snapshot in this infrastructure PR. Preserve all 244 identities, original requests, run/human counts, clocks, model, launcher bytes, privacy limits and qualification holds. ## Verification - The combined candidate passed 542 existing owning controls across 19 suites, including catalog contracts, consistency and 242 lazy declarations with no fixture callbacks. The manifest retains two legacy tasks, for 244 identities. - Candidate Runner checks passed: 15 semantic consumer tests and 12 catalog tests. The owning Runner TypeScript compiler passed after the existing eval-kernel prerequisite built. The first compile reported missing ignored eval-kernel output; the repair used candidate-owned ignored dependency links and output only. No repository type or acceptance rule was changed. - All three generated checks passed: semantic-action catalog, semantic contracts and protocol manifest. Deep comparisons prove each generated catalog differs only in `get_task_history.description`. The seeded fixture changes only its catalog hash, and the protocol manifest changes only the hash of that fixture. The compiled canonical catalog hash matches the fixture. The candidate import conflict added only `getTaskHistoryAction`; no unrelated newer document-schema import was copied. - Both owning fixes cleared fresh exact-head Greptile 5/5 with no actionable threads before this publication: timeline `7f6ba9686b5edba005a49cd15c0d0b52248178dc` and approved replay `0015a79c1e0a3af5292edf4a61aa9bab8f381899`. The personal prerequisite `789bec74a0f43011723fff45596ef9d084bfaba7` also cleared exact-head 5/5 and all 12 threads before its earlier integration. - All 131 existing pin, TestOps contract/API, adapter, reconciliation and streaming controls passed. The descriptor snapshot matches the actual candidate bytes. The only new declaration change from the prior candidate is app-review v5 to v6. Whitespace checks passed. - The SDK source guard reports 506 unchanged inputs, runtime pins unchanged, registry `efa70621e9fb42aa185f7ee616a1a726aa3eccb367e67e28817cedf26ef2ade8`. No SDK registry or runtime certificate renewal occurred. The existing reporting-lock guard returned `already-reviewed` with lock `60f4c06bca47af3ab9d74281f2ef2e1ff559d566b862ba51c2fbe6c8e5bac35b`. - Historical CI results remain recorded. Head 73ea had a server-5 cancellation with exact AWS Spot-reclamation proof. Head 16134 had 44 successful jobs, three failed leaves and dependent verify failure, plus one selector skip. Two failed hosts had exact Spot termination proof; server 11 had two real deadline PostgreSQL failures. After an initial libzstd setup block, the unchanged four-test suite passed locally on exact merge `70e68bff5fb89251f769671ce9ab949ab5bf7fc9` in 33.69 seconds with its original deadlines and assertions. This does not establish the CI cause. One recovery POST returned HTTP403; zero replications were accepted. - Prior head c95 finished with 41 successful jobs, six failures, one cancellation and one selector skip. Four leaves had exact Spot-termination proof; three of their execution logs were unavailable. Server 11 instead had a real `live-recovery-work` next-deadline assertion failure, and server 4 had a real workspace-health connection-refused failure. Dependent verify failed. Windows passed. No replay or claimed test fix erased those results; their two assertion causes remain unresolved. - Exact head `1e89df928f7ccb43bf7af3b814f2c2207d75883a` now has successful normal CI run 41888521175 (attempt 2: 48 successful jobs and one selector skip) and successful Windows run 41888521015. The first attempt retained 46 successes, the failed Typecheck leaf and failed dependent verify, plus one selector skip. Typecheck reported runner shutdown without a TypeScript diagnostic; its Spot request reported no capacity, while the later EC2 termination reason was Client.UserInitiatedShutdown. One authorized unchanged-head targeted recovery reran only Typecheck and dependent verify; both passed. This does not claim a code fix or establish the cleanup caller. All other successful results were carried forward, not fresh remeasurements. - Current exact-head Greptile and Paperclip assessments are both 5/5 with no actionable findings; Paperclip check conclusion is neutral, but its final assessment completed and states Ship. Security checks passed and no review thread is unresolved. Full repository build/typecheck/test were not repeated locally for this three-file metadata change. A fresh actual-serving source, reporting-lock and selection guard remains required after a normal merge and before a new Submit. - No model call, eval dispatch, secret mutation, private archive download, fleet change or merge was performed by this update. The single targeted CI recovery is recorded above. Separate core PRs remain unmerged. ## Risks The discovery description may change tool choice; its effect on the missing handoff remains unproved until a new native attempt. Formatting normalization can affect outcome admission, so existing positive and negative semantics remain required. The personal secure-entry and worker phases are still unproved. The next representative set is timeline, approved-tool replay and personal secret. Company selection remains gated on personal secure-entry and worker proof. The uniform ten-case regression remains pending, and the full 244-case catalog remains unqualified. ## Model Used OpenAI `gpt-6.1-sol` through Codex. Capabilities used: source analysis, isolated mechanical integration and local code execution. The host does not expose the context-window limit. The evaluated model remains OpenCode/OpenRouter `deepseek/deepseek-v4.1-flash-1017`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
3a2249dd57 |
fix(runner): retry proven ingress bootstrap failures (#18981)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native execution must retain its admitted model and execution authority during recovery. > - A sandbox provider can reject Runner endpoint acquisition before the Runner gets a credential. > - The old error path loses the structured HTTP status and leaves that startup failure ambiguous. > - A zero counter in an old checkpoint cannot prove that no credential was issued. > - This pull request adds a versioned producer proof and carries closed provider status through the existing RPC. > - A proved transient startup failure uses the existing retry budget. A terminal proved-unused run can settle its accounting. ## Linked Issues or Issue Description Refs #18018 and #18934. Those changes restore existing checkpoints and ready retained environments. This change covers endpoint acquisition before Runner dispatch. A search of related public PRs found no duplicate implementation. **What happened?** A provider HTTP 502 during Runner endpoint acquisition becomes an untyped worker error. Native recovery treats it as ambiguous and stops without retrying. The run's native usage receipt also stays pending after startup ends without model work. **Expected behavior** Retry a structured transient endpoint failure only when the complete current controller checkpoint proves that this exact run never issued a Runner credential. Preserve authentication, configuration, Stop, ownership and provider-progress gates. Settle zero usage only under the existing terminal accounting owner. **Steps to reproduce** 1. Use the controlled endpoint in the Daytona ingress integration test to return HTTP 502. 2. Register the actual native Runner driver and persist its bootstrap journal through the encrypted checkpoint writer. 3. Observe the old terminal ambiguous disposition instead of bounded startup recovery. **Paperclip version or commit** Reproduced on the pre-change master source. The original regression used the production native driver, a real HTTP fault, a real durable control plane and local Postgres. The new contract additionally tests the pinned Daytona SDK and real worker RPC. ## What Changed - The controller owns a v2 persisted state with retained origin and credential history. Current readers still accept v1. Older writers reject v2. Restore and checkpoint decoding do not add missing history. - The proof validates the complete bootstrap journal, all six identity fields and every delivery, credential and progress control. The server rechecks its fingerprint under the current controller claim before retry or accounting changes. - Daytona and the SDK carry a closed acquisition phase and HTTP status through the existing ingress RPC. Both public error wrappers omit response bodies and causes, including Pino cause serialization. - Only structured HTTP 429 or 5xx ingress failures can use the existing three-attempt startup budget. HTTP 401/403, configuration and unknown errors do not gain retry authority. - A terminal proved-unused run wakes existing accounting and records zero provider work. It does not create a semantic result, prove physical termination or release an unknown process/workspace hold. - The normal automatic second attempt rechecks its exact claim, frozen execution and matching complete local/encrypted journal before admitting absent remote Runner state. It keeps the original pending bootstrap command IDs. A proved unused Codex home can rematerialize sealed skills. Full raw local state is checked before streaming-event or policy projection. - Normal saved-state readers and the existing E2E journal evidence matcher accept the closed v1/v2 set. The separate legacy no-launch witness is unchanged. Technical docs describe the contracts. ## Verification Current head: `cd38f70409cdb8664e14bfbf4efa208f468a0419` (final delta adds only fixture artifact preparation). Production files are unchanged from the independently reviewed `93c8a33520` source. Full repository typecheck and build passed at `96417acb58`; the affected server typecheck also passes after the final two-line fixture delta. Focused protocol, raw-proof, real retry, SDK and RPC acceptance pass on this source. Earlier broad suite counts are listed with their limits below: - Durable Runner protocol: 187 tests. - Native executor, driver, startup/accounting and redaction: 1,211 tests, including real Postgres and encrypted checkpoint persistence. - Normal saved-state restore/projection compatibility: 21 focused cases. - SDK: 209 tests. - Standalone Daytona: 544 tests; 8 opt-in live tests skipped. - Explicit standalone `pnpm -C packages/plugins/sandbox-providers/daytona exec vitest run --config vitest.sdk-ingress.config.ts`: 6 cases with real pinned SDK 0.203.0 and worker RPC ingress acceptance. These include lookup 502/401/403/400, private-preview 502 after successful lookup/refresh, missing credentials and private-response exclusion. - E2E journal evidence: 4 cases across both supported formats, with wrong ownership, missing stimulus and unknown-schema controls. - The exact historical controller source rejects a new v2 checkpoint at decode and restore. A v1 control is accepted. This verifies the downgrade fence with the actual older parser. - Startup controls cover issued and pruned credentials, old/truncated journals, provider progress, exhausted retry accounting, and checkpoint/Stop/controller-claim races. Race cases retain the current execution and pending usage. - New exact admission controls: 19 cases through encrypted persistence and cold restore. They cover current/stale/expired claims, Stop, pause, model/trust/account changes, prior progress, issued/pruned credentials, adopted/old/truncated state, local/canonical fingerprint mismatch, hidden delivery, unsupported work and company scope. Two additional red/green cases prove that raw local stream progress and missing event policy are rejected before lossy checkpoint projection. No case mutates run, claim, result or accounting state. - Actual native executor, scheduler and Rust Runner retry: 2 cases. Codex uses its fake app server. OpenCode uses the real proxy and fake OpenCode HTTP server with the frozen DeepSeek model and allow permission configuration. Each path launches no Runner on attempt 1, starts one Runner and provider on attempt 2, reuses both original command IDs, commits one accepted result, validates the persisted local workspace, records nonzero synthetic native usage and settles accounting once. No zero-work accounting flag survives. The controlled SDK and protocol tests use local source, isolated PostgreSQL, a private HTTPS preview and fake provider servers on macOS arm64 with Node 24.22.2. They create no hosted sandbox and make no model call. The root workspace intentionally omits standalone provider dependencies; the explicit SDK target fails if its pinned dependency is absent and has no conditional skip. They do not qualify hosted rollout or repair an old unmarked checkpoint. The first full local typecheck found missing checkout SDK links from an install with scripts disabled; the repository link helper repaired that setup. Current-head Linux CI typecheck and build pass. Independent final review checked 12 source scopes and six raw-proof controls with no remaining finding. Fresh Greptile review is 5/5 with no unresolved thread. The full Runner CI suite found two older control-peer test doubles missing the new proof method. Both now return no proof, with all 27 permission, ordering and cancellation assertions retained and passing. The new Cargo-dependent native retry suite uses the existing Rust-cached CI lane. All 39 lane coverage controls pass and prove complete, duplicate-free suite coverage. The first full test run stopped at an admitted chat Retry polling failure. CI also found a stale retained-allocation fixture, now repaired on master by #18984, and a canceled runner job. A real SDK suite initially relied on a dependency absent from the root workspace; the explicit standalone target now owns that acceptance. The separate Product E2E support run had 12 failures in four untouched files. All 12 reproduce on the original clean base with independent dependencies, compiled base source and verified workspace resolution. These concern workflow expectations, cleanup timeout arguments, a profile snapshot and operating receipts. They are not passing acceptance. The later full local repository test command stopped at a 15-second timeout in the unchanged GitHub pre-allocation retry `controller` case. That exact case passed in isolation with unchanged assertions. The broad Runner run initially omitted `test:typescript:prep` and lacked debug fake-provider binaries. Required preparation then passed 370 + 44 package contract cases (11 skipped); the prepared full package run reported 4,932 passed, 9 failed and 13 skipped. All failed families then passed with unchanged assertions in isolation: 8 ACPX controller tests and 2 credential controls. The paired original-base ACPX control also passed. The full-run failures remain recorded; they are not passing full-suite evidence. The first Rust-cached native CI run found a missing generated OpenCode proxy in a fresh checkout. The fixture now runs the existing verified provider-entrypoint builder. Removing the generated proxy reproduces the exact ENOENT before the fix; both real retry cases pass after the fix from the same absent-artifact state. All current-head CI gates pass, including the aggregate verification check. Linux native CI passes all 10 suites: 177 tests pass and 14 opt-in cases skip. Both real provider retry cases pass. The full Runner package passes 297 suites and 4,950 tests, with 1 suite and 10 tests skipped. The CLI workspace groups pass 4,762 and 286 tests. The first CI attempt had Runner shutdowns in the Runner package, Canary and three general server jobs. One unchanged CLI shard also hit child-server startup and dense ZIP deadlines. Both CLI files passed unchanged in isolation: 18 tests. One bounded rerun of unsuccessful jobs and their dependents passed. The original failures remain recorded. ## Risks - Older controllers reject newly written v2 state on downgrade. Current controllers retain v1 restore support, but old or unmarked state cannot gain this new proof. - An older plugin that omits structured ingress evidence remains held. The server and provider must use the new producer contract for this recovery path. The Docker `cloud` target builds the SDK, server/Runner and standalone Daytona producer from one source tree. The default `production` target does not build the standalone producer. Hosted rollout must confirm the new image and active image-owned plugin worker; a registry version alone does not prove its source. An npm-installed producer needs an explicit matching artifact upgrade. - This prevents future proved startup failures. It does not release historical accounting based on an old checkpoint, change retry budgets, replace a model or weaken sandbox isolation. - Zero model usage is separate from process termination, saved files and resource release. Those owners keep their existing gates. ## Model Used OpenAI Codex with tool use and code execution. This runtime does not expose the exact backend model ID or context window to the agent. Read-only independent agents reviewed checkpoint compatibility and SDK/RPC error boundaries. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (affected suites; full local run limits are recorded above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8f06bd4e17 |
fix(e2e): retain workspace packages in rolling app copies (#18982)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A rolling deployment must preserve running native agents and their work. > - The source test lane starts separate copies of the compiled app. > - The development Runner entry imports source from the workspace packages directory. > - That directory was absent from each copied app, so its supervisor could not start. > - This pull request preserves the package link beside the existing CLI link. > - The test lane can then exercise real controller replacement. ## Linked Issues or Issue Description **What happened?** The source rolling-deployment supervisor failed with a missing Runner module before starting a paid agent. The copied app had its compiled server, dependencies and CLI link, but lacked the workspace package path used by the development Runner entry. **Expected behavior** The copied app resolves its development dependencies and reaches controller replacement. **Steps to reproduce** 1. Build the server and UI from source with the development Runner entry. 2. Run `tests/runner-e2e/controller-replacement-smoke.ts` with disposable PostgreSQL and S3. 3. The old copy fails to resolve the Runner package. The fixed copy completes the supervisor smoke. No duplicate PR or issue was found in the exact-path search. This is a test-harness fix within the existing rolling-deployment lane. ## What Changed - Link the checkout's `packages/` directory into each copied source app. - Document the shared dependency links and the limits of source-lane evidence. ## Verification - The missing-module failure was reproduced before the link was added. - The same supervisor smoke passed after the fix and confirmed cleanup. - The ten-check non-agent rolling smoke passed with disposable PostgreSQL and S3. - A fresh strict Codex rolling browser case passed. It kept the same running process, native session and sandbox lease, acquired a new controller generation, finalized the workspace, and continued from the saved project. - An earlier paid attempt failed the strict tool oracle because the agent's own soak script omitted the argument delimiter for hyphen-prefixed input. That failure remains recorded; the verifier and task contract were unchanged for the passing attempt. - Server TypeScript compilation passed in the source test checkout. Full repository typecheck, tests and build are required in CI for this clean PR head; they have not been repeated locally in this dependency-free checkout. - Exact head `d8addaa7723af5a013489e5bb6331dd1021dc6be`: all CI checks passed or were intentionally skipped; Greptile 5/5 with no review threads. Earlier cancelled jobs and the old recovery-fixture failure remain retained; rebasing onto its existing master fix passed cleanly. ## Risks Low production risk: only source test setup and its documentation change. Each compiled app still has a separate cache and home. Its workspace dependencies are shared, so this does not qualify a released container or Railway deployment. ## Model Used OpenAI Codex, exact model ID `gpt-6.1-sol`, with reasoning, tool use and code execution. The host does not expose the context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id or instance-derived details - [x] I have run tests locally and they pass, with the earlier material failure retained above - [x] I have added or updated tests where applicable: the existing supervisor and live rolling browser lane exercise this path - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b99dd57c82 |
test(evals): run Product regression from TestOps (#18983)
## Thinking Path > - Paperclip manages agent work through governed product workflows. > - Product E2E tests check those workflows through the real app and runner. > - The saved Product regression plan has 20 imported case identities. > - The plan has no live executor, so historical baseline rows do not qualify its current behavior. > - The existing paid full-stack workflow already owns all 20 required journeys. > - This change binds TestOps selection and reporting to that workflow without changing the product oracle or paid admission. > - Maintainers can run the plan and inspect current case verdicts and retained evidence. ## Linked Issues or Issue Description **What happened?** The standing Product regression plan has no executor. Its dashboard points at a historical 355-result snapshot across several suites and revisions. **Expected behavior** A TestOps plan run must select the intended existing full-stack cells, execute them on GitHub, and report their original case IDs, assertions and source revision. Missing evidence must remain a visible failure. **Steps to reproduce** Open the Product regression plan and inspect Executors. There is no live job. The existing full-stack workflow accepts exact cell IDs but has no TestOps selection or reporting path. **Paperclip version or commit** The branch includes reviewed master correction |
||
|
DottaandPaperclip
|
e968131efa |
test(evals): pin reviewed outcome and native handoff repairs (#18976)
## Thinking Path > - The catalog must measure the original human requests through ordinary product controls. > - The latest four-case run exposed two oracle false negatives and a real suppressed native handoff. > - Preserve those original failures and fix only their demonstrated causes. > - Keep the reviewed company-selector successor and add the accepted timeline, approved-replay and personal-completion corrections. > - Select native standing-chat terminal answers by authority while preserving governed waits and ordinary-task behavior. > - Bind the resulting immutable candidate and its three changed oracle versions through the existing manifest and descriptor snapshot. > - The next personal-secret attempt must prove the complete secure-entry and worker journey before company selection or a final ten-case regression. ## Linked Issues or Issue Description Refs #18971, #18967, #18968, #18972, #18977. Follows #18974. The latest four-case run retained original failures for timeline, approved tool replay and personal-secret setup. Team installation independently passed. The personal native run had actionable final directions, but presentation selected an earlier ordinary comment and suppressed that answer. This pin makes the reviewed corrections available; it does not regrade those attempts or claim a new native pass. ## What Changed - Pin immutable candidate `5f5e2ca8f18063d051c3d948105f0049562fe4ce` on `codex/chat-catalog-four-outcome-repairs-20261104`. - Refresh the existing descriptor snapshot and matching manifest metadata: timeline v5, app-review v5 and personal-secret v4. Company v3 is retained. Preserve the prior reviewed company recipient correction. - Include the reviewed structural standing-chat response selection and finished-turn recovery, preserving earlier comment history, governed interaction ownership and no-final fallback. The superseded prose classifier is absent. - Preserve all 244 identities, original queries, provider/model, human and native counts, clocks, launcher bytes, private evidence limits and qualification holds. The existing SDK registry/compiler proof is unchanged. - The infrastructure change is three metadata files. No product fix is merged by this PR; its owning core PR remains separate. ## Verification - Exact final candidate: 510 owning tests passed across 19 suites, including catalog contract/consistency and lazy declaration loading. The candidate declares 242 cases plus two existing legacy tasks; no fixture callbacks execute during admission. - Candidate response summary suite: 96 tests passed. The owner separately reports 103 summary/notice controls and 14 recovery PostgreSQL controls; these are distinct from candidate results. - Candidate PostgreSQL scope/recovery controls: four passed (three native/ordinary/legacy caller scopes and the new server-finished yielded unknown-channel recovery). The focused filter left 52 unrelated controls unselected. Earlier setup attempts failed before assertions due missing native library links and plugin-sdk output; package-cwd shipped hydration and one owning SDK compiler build enabled this final actual run. No source workaround, timeout change or acceptance weakening was used. - Two deliberate baseline joins preserve the frozen candidate: the caller test adds only its required `issueComments` import; recovery adds only current-conversation matching yielded result/terminal with no owned interaction, retaining all existing succeeded/completed/session/ownership/authorization guards. The test adds only `statusDecisions` and needed fixture options. Fresh-master ordinary `needs_review` behavior and unrelated imports were not copied. - All 131 existing pin, TestOps contract/API, adapter, reconciliation and streaming controls passed. The descriptor snapshot exactly matches the candidate declaration bytes. Diff whitespace checks passed. - SDK source guard: 506 inputs, `changed: false`, `runtimePinsChanged: false`, registry `efa70621e9fb42aa185f7ee616a1a726aa3eccb367e67e28817cedf26ef2ade8`. The local compiler invocation supplied missing ignored test output; it is not a renewed SDK qualification or runtime certificate. - Existing reporting-lock guard returned `already-reviewed`. Fresh protected master `86814c6550f52e99ede150a883bdc773cbb540e3` has admitted lock `60f4c06bca47af3ab9d74281f2ef2e1ff559d566b862ba51c2fbe6c8e5bac35b`. Repeat the existing serving-master guard after merge and before a new Submit. - The earlier pin run exposed a pre-existing remote-failure-retention fixture failure. Normal protected merge #18984 repaired that baseline on master `04a24f4e06da68043378bdf3646f50292efd5316`; its 47 real jobs and aggregates succeeded despite delayed GitHub UI/API status projection. This branch normally merges that actual master. Its three pin metadata files are byte-identical to prior reviewed head `dec78737cc520ce6d489fb8bfccafb460b79e870`, with the same immutable candidate, descriptor bytes and model/count/clock/provenance fields. No baseline fixture patch was duplicated in this PR. - On updated head `bd5402d4688e690ffd9f1f80081577733ceeda25`, the existing 131 controls pass, reporting lock is already reviewed, and existing `selectedCases` admits exactly case 580 without execution. Fresh actual master lock remains admitted. - Exact-head normal CI run 41865757115 has completed its full actual job set: 48 succeeded and one legitimate selector skip, including `ci / e2e` and `ci / verify`. RunsOn completion records and exact GitHub aggregate logs corroborate the results. Windows run 41865757064 completed SUCCESS; its exact log reports all executed Rust test results as passing. GitHub UI/API/check-rollup projections still contain stale queued/in-progress rows, as independently observed during #18984; this is not a claim that those displayed rows are green. A normal protected merge must be accepted by GitHub without override. - Fresh exact-head Greptile is 5/5, the current Paperclip assessment explicitly says 5/5 and Ship, all review threads are resolved (zero threads), and the security scan succeeded. Earlier company-only checks are not used to qualify this head. Full repository build/typecheck/test were not repeated locally for this metadata PR; owning source checks and ordinary hosted CI provide the relevant gates. - No model run, dispatch, retry, cancellation, secret mutation or merge was performed by this update. ## Risks The structural correction changes final-answer selection only for admitted native standing conversations; it still needs a new real secure-entry journey. Oracle corrections can admit previously rejected wording, so retained positive and negative controls remain required. Original failed and unknown attempts are immutable. Company selection remains gated on actual personal-secret secure-entry/worker proof, and all 244 cases remain unqualified as a whole. ## Model Used OpenAI `gpt-6.1-sol` through Codex. Capabilities used: source analysis, isolated integration and local checks. The host does not expose the context-window limit. The evaluated model remains OpenCode/OpenRouter `deepseek/deepseek-v4.1-flash-1017`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes — manifest and expected-pin assertion agree - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green — actual job/aggregate results passed; stale UI/API projection is disclosed above - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
32e5eada06 |
test(evals): pin reviewed attribution and secure question repairs (#18974)
## Thinking Path > - Paperclip must complete work through ordinary human and agent controls. > - The remote chat campaign exposed observer and setup defects as well as product discovery gaps. > - Case 466 rejected a valid handoff that used headings, pronouns and inline HTML. > - Case 487 seeded three daily runs although its declared journey needs six. > - Secure-entry replies used a real bound question card that the observer did not support. > - Team installation omitted its existing strict request schemas from the API search result. > - This PR pins the reviewed coherent repairs and keeps their metadata in sync. ## Linked Issues or Issue Description Refs #18921. Refs #18964. Refs #18967. Refs #18968. Refs #18970. Refs #18971. Refs #18972. The previous launch remains unchanged. Case 53 passed its latest native attempt. Cases 466, 487, 580 and 610 still retain their prior failures. The next representative run must test the repaired conditions. The company-secret case follows only after the personal-secret journey proves real secure entry and worker use. Its separate case-local follow-up is held for a later pin; case 578 is excluded from the next four-case selection and has no pass credit. ## What Changed - Pin immutable candidate `e757e447371c4781a3eff3a0ea9dc9c4eceb060a` and its existing descriptor snapshot. - Record the explicit oracle versions and source-file changes for timeline attribution, review fixture allowance, and bound personal/company secret questions. - Keep all 244 case identities, human requests, original run and action counts, time allowances, provider/model, private evidence limits and broader qualification holds unchanged. - Preserve the actual request-body schema affordance for team preview and installation, existing response links and prior help. ## Verification - The final integrated candidate passed 442 owning checks across 18 suites. All 242 declarations loaded with zero fixture callbacks; the existing manifest joins those declarations and two legacy tasks into 244 cases. - The team service fixture passed 29 tests. The frozen candidate API consumer passed 2045 tests after the three-line schema exposure change. These are component controls, not a new native pass. - The stale inventory expectation reproduced one failure with 16 passes. Its unchanged derivation traces five previously reviewed YAML corrections. Updating only the expected constant passed 17 controls, and the existing definition-drift and catalog-tamper negatives still passed. No scenario YAML changed in this candidate. - The existing SDK guard confirms all 506 inputs are unchanged and the reviewed registry remains `efa70621e9fb42aa185f7ee616a1a726aa3eccb367e67e28817cedf26ef2ade8`. No compiler build or registry renewal was needed. - The existing pin, TestOps contract, adapter, reconciliation and streaming controls passed locally. Current master also passed the existing reporting-lock guard with its actual current lock. - The final source includes the reviewed immediate-actor/consecutive-pronoun controls and checks every rendered question title, header and submit label for unsafe instructions. The remaining fixed scanner reasons are diagnostic only. The combined source passed 442 checks; all 131 pin/contract/API/streaming controls passed. - Fresh exact-head [normal CI](https://github.com/paperclipai/paperclip/actions/runs/41847423336) passed: 48 successful jobs and one selector skip, including both required aggregates. Greptile reports 5/5 for this head, with no actionable findings or unresolved review threads. The current optional Paperclip review is queued and is not credited as an assessment pass; its prior-head 5/5 remains historical. The finite source peer review found no material issue and did not claim native execution. The prior CI attempt is retained. No new native pass is claimed by this PR. ## Risks The new source repairs observation and setup, but it does not prove the next native outcome. The secure-entry accounting records one real chat request and one authenticated UI answer as the two declared human actions; it keeps the raw comment count separate. The question path requires the bound pending card, actual resolver audit and delivery facts. Ordinary cases retain their existing accounting. Private scans and archive limits can still reject evidence. Prior failed and unknown attempts remain immutable. ## Model Used OpenAI `gpt-6.1-sol` through Codex. Capabilities used: source analysis, integration and local checks. The host does not expose the context-window limit. The evaluated model stays OpenCode/OpenRouter `deepseek/deepseek-v4.1-flash-1017`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
github-actions[bot]andlockfile-bot
|
81b01bcc16 |
chore(lockfile): refresh pnpm-lock.yaml (#18961)
## Thinking Path > - Paperclip uses a dependency lock for reproducible builds. > - Dependency changes on master need matching lock entries. > - Paid eval reports also pin a reviewed offline correction. > - Reports must install before paid model work starts. > - This automated PR keeps the lock and report proof together. ## Linked Issues or Issue Description **What happened?** Dependency manifests on master changed after the last committed lock refresh. **Expected behavior** Frozen installs must match those manifests. Eval reports must retain matching offline lock proof. **Steps to reproduce** Run a frozen install before and after this lock refresh. ## What Changed - Refresh the dependency lock from the manifests on master. - Include the matching offline report correction and hashes. ## Verification - Pinned pnpm resolution completed with lifecycle scripts disabled. - The report proof refresh accepted the reviewed committed baseline. - The workflow rejected changes outside the lock and report proof files. ## Risks The lockfile bot merges this PR without CI or review when every added package version is at least 3 days old on npm. Otherwise the PR waits for a maintainer, and the daily refresh tries again. The bot merges only the exact commit this workflow generated from manifests already reviewed on master. Unknown report lock baselines remain blocked. This update changes no permissions or paid enable gates. ## Model Used None — generated by pnpm and the deterministic report-lock updater. ## Checklist - [x] Thinking path and problem description included - [x] Generated lock and matching report proof included - [x] Release-age gate decides between bot merge and maintainer review Co-authored-by: lockfile-bot <lockfile-bot@users.noreply.github.com> |
||
|
DottaandPaperclip
|
84aa989bf9 |
Preserve native sessions through legacy workspace enrollment (#18943)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A native session retains its workspace and provider session across turns. > - An ephemeral web server must restore those files without its predecessor's disk. > - Legacy enrollment changes the workspace authority and can change both workspace addresses. > - The retained provider backup checks those addresses as part of its identity. > - This pull request preserves the exact addresses in a verified migration receipt. > - The benefit is that the next turn can use the original session after enrollment. ## Linked Issues or Issue Description Refs #18932, merged as `612d9931a6`. This pull request builds on that all-Runner durability change. Related: #18918 fixes concurrent workspace preservation locks. This change covers identity preservation during offline enrollment. ## What Changed - The native workspace migration owner records the selected session bindings in the successful workspace migration operation. - Runtime selection retains the original logical address only for the exact agent, task and session. - Remote workspace preparation restores the original provider address. It refuses conflicting targets, forged receipts and occupied unowned roots. - The environment owner includes the migration operation in the reusable lease fingerprint. Warm policy stays warm and receives a fresh allocation through normal lease cleanup. - Exact migration evidence permits the existing verified provider backup to restore into that fresh warm allocation. - Preserve verified provider allocation files through a normalized, lease-bound digest receipt. Unknown files and altered markers remain refused; interrupted publication is retryable. - Retain legacy managed-instruction session fingerprints using the exact imported content hash and recorded paths, preserving all other configuration and credential checks. - Migrate existing local skill archive/manifest objects through upload claims, readback verification and receipt compare-and-swap; retain original objects and IDs. - Add explicit two-turn browser qualification for every released native Daytona profile, with warm and per-turn variants. The production maintenance entry runs offline. The old home is deleted only after a verified export and successor readiness. The oracle checks provider identity, native session, addresses, reference bytes, question and final document. ## Verification - Full repository typecheck passed at `b12ecd823e`. The production session fix passed 198 session/credential tests. Full repository build passed at `94bb917947`. The final Rust-only provider fix passed 57 ACPX provider tests and the exact Linux release build. - Workspace and transport checks passed: 117 transport/workspace tests, six final real PostgreSQL migration boundary cases, 178 environment tests, 539 Daytona tests (eight skips), and the native warm-backup admission control. - Local skill migration passed eight PostgreSQL/S3 tests (one existing opt-in skip). Failed readback retains the source. Successful readback still works after deletion of the old home. - Latest Product E2E typecheck and 38 focused harness tests pass. Earlier catalog/admission/report checks passed. Twelve unrelated support-test failures reproduce on unchanged base `1a39dea9aa`; these are not reported as passing. - Live Codex per-turn at `2d5bb9a6d1` retained the native and provider sessions, both workspace paths and exact reference bytes after deletion of the old home. It completed the task and delivered its saved document and original reference. Its original failure was an output-count grading defect. The corrected grader passes all 21 checks against retained, hashed evidence; the original result remains unchanged. - Live OpenCode per-turn and Codex warm at harness `94bb917947` pass all 21 product checks. Their launcher cleanup failed because a read-only diagnostic observer kept its disposable database open. The observer now closes before ordinary database cleanup; a no-provider reproduction confirms the fix. All owned sandboxes were independently verified absent, child keys revoked and buckets removed. Original failed campaign dispositions remain unchanged. - Claude at `94bb917947` kept its sessions and reference bytes but omitted the final clickable document link. That user-delivery failure remains recorded. The fixture now explicitly asks for the link and records migration evidence before checking navigation. The fresh Claude attempt at `b12ecd823e` passed both strict migration checks: exact provider/native session, both paths, original file bytes and independently downloaded attachment. Its full browser workflow failed again: it requested unnecessary confirmation, used the answer sentence as the reference, and omitted the clickable document link. This is retained as a failed user-workflow result, not a full pass. - Earlier live failures exposed and reproduced the skill-object, fresh-allocation marker and managed-instruction fingerprint bugs. The fixes retain original identity checks. No provider session reset or accounting bypass was used. - The real Grok continuation at `b12ecd823e` failed before provider execution because its immutable-instruction attach policy rejected relocated registered file copies. A no-provider regression reproduced the exact rejection. `b05ba3728a` selects the existing strict authenticated asset-path policy for Grok; content digests, unknown context fields, session/workspace identity and same-run mutations remain enforced. The regression and all 57 ACPX provider controls pass. The Grok browser migration now passes all 21 checks with the b05 Linux Runner, including provider identity, exact reference bytes and final user output; the launcher also reports successful cleanup. - `24fabf1d81` applies the same policy to released ACPX Codex after a second no-provider reproduction. All 58 ACPX provider controls and the final Linux release build pass. Its live browser migration passes all 21 checks on the final binary, including exact original session/file preservation and completed user output. - The full native-session executor suite passes all 1,121 tests after updating its existing lease database fixture for the migration-receipt join. The prior CI failure was this unsupported test-double method; the production guard still rejects ordinary unproven recovery. CI and Greptile 5/5 passed at `ca1e0d5fe6`. The first CI attempt lost several workers and had one rotated-recovery transport failure; its seven descriptor cases passed locally, and the single unchanged retry passed. The original failure remains recorded. Staging enrollment has not started. Initial offline enrollment and subsequent rolling availability are separate checks. - The final review follow-up adds real runtime-selection PostgreSQL cases for exact, different-session and absent receipts, plus executor-level warm restoration that copies actual backup files and verifies provider identity and the new lease stamp. Both complete affected suites pass (1,172 tests), and full repository typecheck passes. Removing either production migration connection independently makes its new positive case fail; both source files were restored byte-for-byte after these negative controls. - Final commit `0971d373a3` passes all 63 checks (57 successes, five expected skips, one neutral); Greptile is 5/5 and every review thread is resolved. ## Risks - An identity conflict blocks enrollment and keeps the original files. It does not silently reset the session. - A new migration generation retires a reusable lease through its existing owner. The original sandbox must finish normal cleanup before reuse proceeds. - Initial enrollment has an offline step. The separate rolling suites prove uninterrupted deploy behavior after enrollment. - This adds no schema migration and changes no provider credentials or accounting gate. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model build and context window are not exposed in this session. The agent used reasoning, terminal tools, code edits, PostgreSQL/filesystem tests and isolated live browser qualification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
612d9931a6 |
Support per-turn Runner sessions on ephemeral web servers (#18932)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The web server controls native Runner sessions in remote sandboxes. > - An ephemeral web server must recover without its previous local filesystem. > - The existing admission rule accepts warm sessions. Managed Claude, Grok, and OpenCode sessions resolve to per-turn execution. > - Per-turn execution also needs durable provider-session bytes after the sandbox is retired. > - This pull request adds encrypted backup publication and removes the warm-only admission rule. > - Both lifecycle modes can use the existing native ownership and checkpoint contracts during a web deployment. ## Linked Issues or Issue Description Refs: #18518, #18529, #18826, #18835, #18841, #18864, #18887. **What existing behavior does this improve?** Native Runner execution on web servers with `PAPERCLIP_EPHEMERAL_SERVER=true`. **Current behavior** The admission rule requires a warm Runner lifecycle. Managed provider credentials can resolve an agent to per-turn execution, so enabling the ephemeral deployment profile blocks those turns. A database controller checkpoint does not preserve provider files after the old web home and per-turn sandbox are removed. **Proposed behavior** Admit native execution in either resolved lifecycle. Keep provider and credential choices unchanged. Publish encrypted provider state to shared storage before sandbox cleanup. Restore it from a fresh web home for the next turn. Use the existing controller lease and generation checks for active takeover. **Reason and benefit** Operators can remove the web server volume without excluding per-turn providers or forcing managed accounts into warm sessions. New native Daytona profiles automatically enter both rolling-deployment qualification lanes. **Breaking changes** The additive migration creates a backup publication ledger. Existing volume-backed local-disk installations retain their local backup path. S3 installations publish encrypted backups and require a stable secrets master key. Existing-volume enrollment must import surviving session data before detaching the volume. ## What Changed - Admit native Runner execution independently of its warm or per-turn lifecycle. Keep host-owned legacy execution excluded from ephemeral deployments. - Store private provider state as encrypted objects. Journal uploads before I/O, read them back, and fence publication against the current native owner. Keep the previous published backup when storage or ownership checks fail. - Restore exact provider bytes into a private cache. Require an exact durable backup before sandbox cleanup. Extend the existing bounded retention pass to retire stored objects without adding idle polling. - Import selected legacy controller and provider state in the existing offline maintenance process. Preserve queued work, task pointers, pending recovery, and original files. Prefer canonical database checkpoints. - Preserve already-failed legacy controllers with missing original artifact proof as encrypted archives. These archives do not become execution authority. Successful retained sessions still require proof. Report archival and executable imports separately. - Verify unmanaged Grok API keys through the existing xAI account validator. The live test exposed a readiness gap that kept remote Grok agents paused after platform checks. Preserve failed credentials and the separate runtime warning. - Ingest Grok whole-prompt usage at the owning ACP boundary. Bind the receipt to the exact outbound request/session; reject replay, conflicts, and incomplete authority. Separate turn cost from cumulative session cost in the sidecar schema and Rust normalization. Preserve managed billing identity and identify direct xAI API credentials. - Preserve published workspace directory names across cold allocations, including destroyed-allocation cleanup. Keep unpublished epoch/lease directories isolated and refuse to overwrite an occupied preparation. This fixes the cold-session identity regression exposed by #18922. - Increase bounded checkpoint-header inspection to 1 MiB. Retain the aggregate inspection limit. - Register all nine native Daytona profiles in per-turn and warm rolling-deployment suites. Delete the entire departing server home, exercise ordinary reads and writes, require active ownership transfer, then check a later turn on another fresh server. Permit only exact notification replay after reconnect. Ownership stays with the existing modules: Runner validates its protocol state; Core validates run scope and controller ownership; storage owns object I/O; the maintenance process owns offline conversion; the E2E wrapper owns process replacement and probes. No provider protocol is added to the shared admission rule. Grok receipt parsing stays in its provider module. The shared sidecar gains an explicit cost basis because its existing price field means session-cumulative spend, while Grok reports prompt totals. Existing consumers retain cumulative behavior when the new field is absent. ## Verification - Focused native admission, archive validation, publication fencing, cleanup, readiness, legacy import, and E2E observer tests pass. Real PostgreSQL cases verify that queued intent and historical recovery rows stay unchanged. - Full typecheck and build pass on the rebased branch. The review follow-up passes server typecheck, 29 focused PostgreSQL/archive/readiness tests, and module-boundary checks. Revision fb8d39a7a1 reached 55 successful CI checks and Greptile 5/5 with no unresolved threads. The Grok follow-up passes Runner/server typechecks, 238 ACP runtime/projection/usage tests, 16 sidecar-schema tests, the Rust cost regression, and Core billing regression. The |
||
|
DottaandPaperclip
|
b4c69ded9f |
test(e2e): attribute native OpenCode skill loads (#18929)
## Thinking Path > - Paperclip lets people manage agents and assign saved company skills. > - Daily QA must prove that an agent loads the saved procedure it uses. > - The saved-skill journey accepts several native source-read paths. > - OpenCode's native proxy changes its built-in skill loader's normalized operation to unknown. > - The grader expected execute and rejected a real completed skill load. > - This PR accepts that one observed native shape with strict source and identity checks. > - Genuine loads can qualify while missing, foreign or incomplete evidence still fails. ## Linked Issues or Issue Description Refs: #18783. **What happened?** A combined QA run on `0884e73b46ee064b3d242e467035dde6c32e5cb5` failed `saved-skill-native-read-attributed`. The agent's successful native skill load exposed the complete saved procedure. The exact immutable assignment and source checks passed. Core recorded the built-in loader as `operation: unknown` and `readOnly: false`. The grader accepted this loader only with `operation: execute`. **Expected behavior** A complete native OpenCode skill load can qualify when its actual run, session, immutable source and paired receipt all match. A generic unknown operation cannot qualify. **Steps to reproduce** Run the saved-skill journey through Core `09162fe893b464dd2ccdafd14bd3ef1611202a86` with OpenCode. Let the agent use its native `skill` loader. The direct TypeScript converter emits execute, but the real proxy forwards a `builtinToolCall` without that operation. Core's native converter records unknown. The former TypeScript-only fixture missed this boundary. ## What Changed - Add one closed native OpenCode skill-loader branch. Require the actual run, runner, native session, item, turn and paired call. - Bind the API assignment, admitted native skill, immutable version map and materialized runtime directory to the same canonical skill. - Require the exact wrapper name, heading and complete saved procedure. Require an absolute local base directory and an empty file inventory. Reject traversal, URI encodings, backslashes, query or fragment markers and all ASCII controls. Keep valid spaces. - Require a successful, complete and untruncated native receipt with the exact output byte count. Reject missing, duplicate, reordered, foreign, failed, dynamic or MCP carriers. - Add 65 loader and boundary controls. Drive the real TypeScript-to-proxy input and use the observed Core native carrier shape. The JavaScript test does not claim to run the Rust converter. - Document the native loader path. Preserve every existing accepted execute, filesystem, first-party and Codex parser path byte-for-byte. Keep all seven checks, one-run limit, 500-cent guard, task prompt, model and ordinary Beta-off Latest assignment unchanged. ## Verification - The sealed original Core receipt reproduces RED: the former grader returns zero reads. Changing only its normalized operation to execute reaches the old accepted branch. The narrow correction returns one attributed read in separate local analysis. The original QA grade remains failed. - The actual TypeScript, proxy and native normalization source regions match between Core091 and the tested baseline0884. The proxy emits `builtinToolCall` without an operation. The native converter supplies unknown. - 334 focused tests pass across nine files, including 65 new loader controls and all existing saved-skill evidence, flow, contract and catalog controls. Run `node node_modules/vitest/vitest.mjs run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/product-journeys/saved-skill-use-native-loader.test.ts` for the narrow regression. - TAB, ESC and DEL controls fail before the review hardening and pass after it. The valid-space positive remains green. - Canonical Runner TypeScript prerequisites, own server declarations and E2E types pass. The final E2E-only check reuses those same-commit declarations after only the two test-scope files changed. The checked candidate blobs match this committed head. - Credential-free discovery lists 25 catalog journeys, including saved-skill use. Registration and selection do not change. - `git diff --check` passes. No Rust build or product-model call ran locally. Full CI [41773295359](https://github.com/paperclipai/paperclip/actions/runs/41773295359) passed on `fb537b598da40176fa1752b49a4f676f468de1f7`: 48 successful jobs and one skip. Fresh Greptile is 5/5 with zero unresolved threads. The earlier same-head canceled run remains retained; no retry was requested. The optional native review ended neutral with a 5/5 source assessment and explicit execution/qualification coverage limits. Fresh staging qualification [41777787338](https://pages.paperclip.ing/evals-qa-smoke-41777787338-1/) passed on this exact head: all seven checks and all five phases, exactly one native OpenCode run, independent saved-document reopen/source preservation, and normal cleanup complete. TestOps recorded a new passed result and the automatic Slack report succeeded. - The failed original stopped before independent document, reload and source-after checks. Those checks remain unqualified for that run. The original failed grade remains unchanged. A separate genuine fresh QA pass now verifies the complete journey before daily re-admission. ## Risks This path depends on OpenCode's observed native wrapper and materialized directory contract. Missing or changed evidence fails closed. The fix adds no runtime service, schema or API behavior. The native-loader branch is covered by strict local regression controls and the original sealed Core receipt; a separate fresh staging run also passed the complete saved-skill journey. ## Model Used OpenAI GPT-6 through Codex with reasoning, tool use and code execution. The runtime does not expose a more specific deployment ID or context-window size. Local checks made no product-model calls. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
c6fe9b143c |
test(evals): pin reviewed chat repairs and restore UI test setup (#18921)
## Thinking Path > - Paperclip agents must describe recorded work and respect human review decisions. > - The catalog must test the user's request without adding an unrequested format rule. > - Timeline case 466 wrongly required date and timezone tokens in every valid answer. > - Missing private evidence and stalled review admission also need bounded failure facts. > - The reviewed changes correct that contract and add diagnostics without changing product behavior. > - A new retained secure-entry reply also exposed an observer false positive for an explicitly excluded chat destination. > - A retained team-install reply also omitted every resource link, so the reviewed service now returns canonical hrefs. > - This PR pins the coherent corrections and fixes the observed UI-test interruption helper. ## Linked Issues or Issue Description Refs #18914. Refs #18923. Refs #18927. Refs #18780. This prepares the next diagnostic selection. It does not change the three-case run already in progress. The prior case-466 failure remains a failure because its answer omitted the required handoff. The prior case-53 original grade remains unknown. ## What Changed - Pin immutable candidate `25cd5e4ebb6b955701778d2d1d702f86a573cfaa`. - Update the existing descriptor snapshot and matching timeline oracle declaration to v3. Only case 466's expected outcome is clarified: respect a timezone when explicitly requested. - Update case 487's existing source-file declaration to include its diagnostic helper. Its oracle, request, run counts, and clocks stay unchanged. - The candidate adds fixed carrier/reason counts for case-53 evidence failures. It also records bounded, issue-bound delivery/wake/run facts only when case 487's existing settled check fails, then rethrows that original error. - Correct the shared chat-credential matcher for secure-form instructions that explicitly exclude chat. Mixed or later chat requests still fail. Give case 580 an explicit personal-secret v2 oracle identity and update its existing snapshot and manifest declaration. - Include the reviewed additive team-install resourceLinks response, its shared type and actual 201 OpenAPI schema. Preserve the full existing partial-install/repeat help and synchronize the existing typed Storybook result. The core PR remains unmerged. - Fix the UI test announcement handler that timed out during the two-tab attachment check. Only a detached target with no current replacement is ignored; real remaining-target errors still fail. The original functional assertions and deadlines remain. This changes no catalog case 550 behavior. - Keep all 244 case identities, original requests, model, deadlines, private scanner limits, and broader runtime qualification holds unchanged. ## Verification - All 47 existing TestOps contract, adapter, and manifest controls passed on this pin. - Integrated timeline and private-diagnostic controls passed: 162 timeline checks, 14 evidence checks, and full 242-declaration admission with zero fixture callbacks. - The source author ran 53 real PostgreSQL/review controls and the declaration-backed runner typecheck on the exact accepted review diagnostic. Those passed. - The integrated replay passed 179 tests. Its three PostgreSQL observer cases initially could not start because the local native package lacked a shared library. After the existing same-version runtime was resolved through owned dependency links, those three real PostgreSQL cases passed. The setup failures are retained; no source, test, timeout, scanner, or assertion was weakened. - The retained secure-entry consumer replay accepts the truthful secure-form/not-chat handoff with the correction. All 25 shared consumer checks and the fresh full 242-declaration admission passed (28 tests total). The earlier failed result is unchanged. This does not yet prove human entry or worker delivery. - A cheap prefreeze check on actual master passed the existing reporting-lock guard with unchanged reviewed lock bytes. - The resource-link type changes exactly one of 506 SDK inputs. The existing owning helper freshly compiled shared and plugin SDK outputs. Measured SDK runtime JavaScript is unchanged; the type declaration, its map and two build receipts changed. The emitted resource-link consumer compiles and a foreign resource kind fails with TS2322. The old registry/default proof rejected the change; the existing generator updated only that input and the new default proof passed. All 51 SDK and 20 input/build custody controls passed. Compiler, parser, Zod and canary certificate pins are unchanged; held runtime cases remain held. - Core service/wire/OpenAPI controls passed (72 tests), plus four real PostgreSQL checks. The typed story synchronization passed UI typecheck, catalog checks and 30 controls. Candidate token gates passed. - The prior CI two-tab check failed in the announcement locator handler. A controlled real-browser disappearance reproduces it before the fix and passes after it; a genuine remaining-target error still propagates. That prior failed check is retained. Fresh normal CI and review run on this actual correction. - The review-requested real detached-ElementHandle control now covers the catch branch and asserts the target is absent. Removing that predicate fails with the real browser detachment error; the restored three controls pass. This is test coverage only; helper behavior and deadlines are unchanged. - Fresh exact-head normal [PR CI](https://github.com/paperclipai/paperclip/actions/runs/41774814211) and [Windows Runner](https://github.com/paperclipai/paperclip/actions/runs/41774814064) passed. Greptile and the completed Paperclip review both report 5/5 with no actionable feedback; all review threads are resolved. The optional security scan is neutral, not a security assessment PASS. No new catalog runtime PASS is claimed for this candidate. ## Risks The timeline contract correction accepts valid answers without extra date labels. It still rejects explicit dates outside the requested window, missing attribution, failed-run misstatements, hidden work, and unsupported claims. Balanced owning controls retain those negatives. The diagnostics add evidence rather than changing delivery behavior. Case 53 can still fail closed at the existing archive limit. Case 487 can still fail its original settled check. The additive resource links are a product affordance and still need a real agent rerun. Earlier failed grades and attempts remain unchanged. The full 244 runtime catalog is not merged into master; its existing 196/244 gate remains. ## Model Used OpenAI `gpt-6.1-sol` through Codex. Capabilities used: source analysis, code integration, and local validation. The host does not expose the context-window limit. The model under evaluation remains OpenCode/OpenRouter `deepseek/deepseek-v4.1-flash-1017`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
cb75c9510c |
test(evals): restore chat setup and pin secure-entry guidance (#18914)
## Thinking Path > - Paperclip manages agent work and keeps credentials under human control. > - Agents discover supported operations through existing API and connection help. > - The measured secret-entry case chose external provider setup instead of native secure entry. > - The reviewed help distinguishes these actions and keeps the same permissions. > - Two related help changes explain recorded progress and partial team installation. > - The first remote setup found stale reporting-lock metadata on trusted master. > - This PR pins the candidate and renews that existing lock correction for narrow remote checks. ## Linked Issues or Issue Description Refs #18909. Refs #18780. The previous measurement did not complete the requested native secret-entry handoff. The next representative is case 580 on the existing TestOps plan 102 and job 100. Case 610 can use the same source for a separate representative. The progress-help change does not claim that case 466 passes. ## What Changed - Pin immutable candidate `b1ea76b286b870bb62a80eebf3fe00feed6f12c1`, which retains the approval-link fix from #18909. - The candidate adds only reviewed secret-entry guidance, three API help changes, and the matching SDK source registry entry. Its delta from the approval-link candidate covers three files. - Update the launcher snapshot target and expected target test. - Record the actual new SDK compilation and its one changed runtime JavaScript file. Preserve the previous qualification as history. - Refresh only the existing reporting-lock policy and patch from the prior reviewed lock to current master. The helper, actual lockfile, credential rules, and candidate source stay unchanged. - Keep all 244 case rows, inputs, clocks, model, private evidence controls, and held runtime cases unchanged. ## Verification - The owning secret help checks passed: 47 server/shared tests and 31 Secrets UI tests. The real discovery search failed before the help change and passed after it. - The owning progress and team-install search regressions each passed with their full 2057-test runner API suite. - In the candidate, the existing `scripts/ensure-plugin-build-deps.mjs` freshly compiled shared and plugin SDK outputs and exited zero. The emitted guidance consumer passed. Measured output changes contain only the guidance JavaScript, its declarations and maps, and two build receipts. - The old registry and default source proof rejected the changed input. After the existing registry generator updated that one entry, the 506-input check and default source proof passed. Runtime, compiler, parser, Zod, scanner, and canary certificate pins did not change. - All 51 affected SDK tests and all 20 input/build custody controls passed. One initial suite failed to load a missing local dependency; that setup failure is retained. The run with the existing package dependency links passed. - The three existing TestOps contract, adapter, and manifest test files passed: 47 tests, zero failures. - The first case-564 attempt stopped before credentials or model execution: the trusted controller lock had moved through normal dependency updates while its reporting-lock metadata remained stale. That attempt has no original native grade and is not a product failure or a PASS. - The existing lock-refresh command generated the exact reviewed correction from lock `54a81b2e` to current master lock `7540075e`. All seven correction, tamper, and refresh tests pass. The actual current-controller frozen install with lifecycle scripts disabled and both reporting/private-transfer imports pass. The frozen candidate still accepts its unchanged own lock. An initial offline install found an uncached package; the regular frozen install passed. - Fresh CI and review for the renewed head are pending. Local help checks do not establish a native agent outcome. No new TestOps PASS is claimed. ## Risks Help can improve discovery without changing the agent's final choice. The representative runs must test that remaining gap. The candidate retains the prior catalog application base plus the reviewed fixes. It does not refresh or qualify the full catalog. Automatic model retries remain zero and the authorized concurrency cap remains eight. ## Model Used OpenAI `gpt-6.1-sol` through Codex. Capabilities used: source analysis, code integration, compilation, and local validation. The host does not expose the context-window limit. The model under evaluation remains OpenCode/OpenRouter `deepseek/deepseek-v4.1-flash-1017`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |