mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-12 08:19:54 +02:00
5ee9e751fbb38787c055ee793c3412ecfc89da2e
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
60c7c9cd1a |
fix(runner-e2e): pass verified lock digest to Daytona image build (#13876)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E campaigns test the native runner in local and Daytona environments. > - Each campaign resolves one target lockfile and verifies its downloaded artifact. > - The Daytona image job did not pass that artifact digest to Docker. > - Docker used an older default digest and stopped before any selected task ran. > - This pull request passes and validates the campaign digest at the image build boundary. > - The image keeps its checksum check and frozen package installation. ## Linked Issues or Issue Description **What happened?** The merged-master [qualification campaign](https://github.com/paperclipai/paperclip/actions/runs/35863582409) stopped in the Daytona image build. The resolved target lock digest was `e0c928a494f90ddad3c00791e83f09315ee8c82df2a0418a809dcf93649a8ab3`. Docker used its default digest, `57b298aceebc48bb94ea0593347348256475da7b2fddb77025d9e57cc8759420`. The checksum check rejected the mismatch. All 12 selected model cells were skipped. This follows the image provenance work in #13814. **Expected behavior** The image build must check the same lockfile artifact that the campaign restored and verified. A changed lockfile must still fail the checksum check. **Steps to reproduce** 1. Start a Product E2E campaign with a Daytona cell on master `7944ed3d976d1a7cc26a2d0cee51f227f3542084`. 2. Resolve a target lockfile whose digest differs from the Dockerfile default. 3. Observe the provider-pack image stage reject the lockfile before model execution. **Paperclip version or commit** `7944ed3d976d1a7cc26a2d0cee51f227f3542084`. **Deployment mode** GitHub Actions Product E2E campaign with a Daytona image build. **Install method** Built from source with the campaign lockfile artifact. **Agent adapter(s) involved** Native Codex and ACPX Claude cells were selected. No model cell ran in this failed campaign. **Database mode** Not involved. The failure occurs during image creation. **Access context** The authorized default-branch paid workflow. The build receives no provider credentials. ## What Changed - Read the image checksum from the existing target-lock job output. - Require a 64-character lowercase hexadecimal digest before image inspection or build. - Pass the digest as the existing Docker build argument. - Add regression checks and document the campaign checksum handoff. ## Verification - The Daytona image regression fails with the original workflow and passes with the fix. - All six Daytona image contract tests pass. - All 450 Product E2E unit tests pass. - Product E2E typecheck passes. - Actionlint passes for the changed workflow. - A context-shaped resolution probe preserves the downloaded lockfile bytes and digest. - All latest-head CI checks passed on `67d41fd9439b2a9a809ddb05765f8617585072c5` ([run](https://github.com/paperclipai/paperclip/actions/runs/35865739359)). - Greptile gave 5/5 on this head; its test-scoping comment is addressed and resolved. - A hosted Daytona image rebuild and the three remote qualification cells remain pending after merge. ## Risks The campaign digest comes from the existing trusted target-lock job. The restored artifact checks, Docker checksum check, frozen install, content identity, image signing, and verification remain in place. The standalone Docker default remains available. This change does not alter task behavior, prompts, credentials, or dependency versions. ## Model Used OpenAI `gpt-6-astra` through Codex performed diagnosis and review with code execution tools. OpenAI `gpt-5.6-luna` assisted with investigation, implementation, and verification. Context window limits are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
be6f49a425 |
feat(runner): refresh shared coding harness runtimes (#13838)
## Thinking Path > - Paperclip runs agents through local adapters and the native runner. > - Both paths must use the same installed provider CLI. > - New models require current harness releases. > - The runner still pins Codex 0.153.4, Claude SDK 0.3.263, and OpenCode 1.18.29. > - Changing the image alone would fail the runner's exact version and executable checks. > - This pull request updates those dependencies, integrity checks, controller checks, and image pins together. > - Shared installations can then run the current models without a task-time download. ## Linked Issues or Issue Description Refs #13829, which updates model choices and reasoning controls. Searches found no open PR that updates these runtime pins. **Current behavior** The shared provider pack ships old CLIs. Claude Code 2.1.263 cannot run Opus 5.5, which requires 2.1.280. Remote controllers reject provider packs whose versions differ from their declared pins. **Proposed behavior** Use Codex 0.156.0, Claude Agent SDK 0.3.280 / Claude Code 2.1.280, and OpenCode 1.18.32 throughout the runner. Keep the reviewed ACP bridge patches and one shared CLI installation per provider. **Reason and benefit** Current harnesses support the new model IDs while preserving executable verification and remote provider-pack compatibility checks. ## What Changed - Update dependency overrides, the Codex ACP package patch, runtime profiles, and remote controller pins. - Verify the new Claude Linux x64 and macOS arm64/x64 executables and Codex Linux x64 executable against integrity-verified npm archives. - Refresh OpenCode version checks, fixtures, and the runner configuration label. - Refresh the eval image's Grok, Gemini, Kimi, Cursor, and GitHub CLI pins and archive hashes. Hermes remains current at 0.19.0. - Refresh the build-time lock digest from clean pnpm 9.15.4 resolution. Leave lockfile commits to repository automation. - Document model compatibility and the separation between CLI runtimes and patched ACP bridges. ## Verification - `pnpm -r typecheck` and `pnpm build` passed. - Rust workspace release tests passed. - Package/patch and OpenCode binary-materialization contract tests: 11 passed. - Real Codex 0.156.0 startup-ownership and paginated session-resume probes passed with isolated synthetic homes and no model turn. - Codex app-server `thread/start` preserved `gpt-6-sol` and `gpt-6-luna`; no `turn/start` was sent. An unauthenticated built-in catalog does not include those account-served entries. - Installed Claude integrity probes passed for `claude-opus-5-5` and `claude-fable-5-1`. - `pnpm --filter @paperclipai/paperclip-runner test:opencode:qualification` passed with the actual OpenCode 1.18.32 executable under Node 24 and Node 25. The loopback provider exercise covers health/version, session creation/read/delete, SSE, and a completed async prompt. - `pnpm check:token-gates` passed. - The targeted runner suite passed 130 tests. Three macOS failures in snapshot module lookup and OpenCode final-message selection also reproduce on the unchanged base; Linux CI will provide the platform check. - [Final Linux CI](https://github.com/paperclipai/paperclip/actions/runs/35798076399): all gates passed. Four jobs needed one retry after their CI workers received shutdown signals. The PR has 55 successful checks, two skipped checks, Greptile 5/5, and no unresolved review threads. - Changed runner configuration UI tests: 5 passed. - Full macOS `pnpm test:run` reached 13,094 passing server tests, 84 skipped, and 18 failures before the wrapper stopped. Failures involved skill-cache publication permissions, missing bundled connector skills in the worktree, and a conversation-reset timing case. The 10 cache permission failures reproduce on the unchanged base; both conversation-reset cases passed on a targeted retry. The wrapper did not reach its later workspace/serialized groups locally; Linux CI covers those groups. - The local Docker daemon did not respond, so no local Docker build was run. No billable model requests were made. ## Risks - Deploy the matching controller and provider pack together. Older controllers enforce their previous exact pins. - Current upstream CLIs can change behavior. Existing protocol tests and isolated real Codex probes cover the integration boundaries; authenticated model inference is not part of these checks. - ACP bridge package versions and executable digests stay unchanged because their executable bytes are unchanged. Only the underlying CLI/SDK dependencies move. - No schema migration. Revert the runtime and image pins together to roll back. ## Model Used OpenAI GPT-6 via Codex, with repository tools, code execution, and web research. The exact serving model ID and context window were not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass for the changed surfaces and real-executable probes; full macOS-suite limitations are listed above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f6a211479f |
fix: share current CLI runtimes across sandbox adapters (#12994)
## Thinking Path - Paperclip Runner needs its runtime preinstalled for fast sandbox startup. - Native and local adapters should launch one current CLI installation per provider. - An older global copy can shadow that installation, and exact native compatibility pins must match it. - Update the qualified releases and binary digests, expose shared CLI entrypoints from the provider pack, and prefer the image-owned bin directory. - Keep dependency installation in the image build; task startup only discovers, links, and verifies artifacts. ## Linked Issues or Issue Description **What happened?** Remote native startup rejected a stale global Codex, while CLI-only images lacked runnerd entirely. **Expected behavior** An image-baked runtime starts without uploading binaries or installing packages. All adapters share the same current provider CLI. **Steps to reproduce** Start a native remote task with the old global Codex and the updated runtime available only under `/opt/paperclip-runner/bin`. **Paperclip version or commit** Discovery behavior at `54a99d884`. **Deployment mode** Docker with a remote sandbox. ## What Changed - Prefer `/opt/paperclip-runner/bin`, then the user's local bin directory, then PATH. Existing metadata and version validation remains in force. - Qualify Codex 0.153.4, OpenCode 1.18.29, and Claude SDK 0.3.263 / CLI 2.1.263. Update binary digests, TypeScript/Rust checks, registry defaults, and the displayed OpenCode version together. - Share Codex and Claude's native executable with the ACP bridges through exact dependency overrides. Preserve the separately qualified ACP bridge implementations and their security patches. - Expose shared provider-pack CLI launchers; fail the pack build if Codex ACP resolves a separate Codex installation. Update the eval image's other agent CLIs to current stable releases and remove duplicate global provider installs. - Document the single-current-CLI policy in source comments and development guidance. Latest stable releases are resolved at review/build preparation and pinned; task startup never auto-updates. ## Verification - Native-session and adapter-registry suites: 158 tests passed. - Provider suites: 88 tests passed, 7 Linux-only checks skipped on macOS. One existing macOS temporary-path alias assertion passed when rerun with canonical `TMPDIR=/private/tmp`. - Package-contract and OpenCode materialization tests: 11 passed. - Full typecheck, build, and token gates passed. Rust native-provider/recovery tests: 19 passed. - Broad local suite: 5,974 passed, 23 failed, 41 skipped. Failures are in unchanged macOS workspace/path/port and connection suites; focused runtime tests pass. All latest-head Linux PR checks passed, including the full test shards, typecheck, build, runner verification, browser suites, and canary dry run. - The standalone fleet image built with one current provider CLI each and passed native Codex/Claude binary-integrity checks. A disposable Daytona sandbox reported ready in 798 ms; its baked runner completed an API-key `gpt-5.6-luna` turn in 2,430 ms and returned the expected marker with a usage receipt. No runtime artifacts were uploaded or installed. - The normal shared `codex exec` entrypoint also completed an API-key `gpt-5.6-luna` turn in 2,321 ms. - Both image builds verify the complete generated lockfile against a reviewed SHA-256 before package installation or lifecycle execution. Root lockfile changes remain CI-owned. Merge and rollout remain on hold for operator review. ## Risks - Updating provider CLIs changes their behavior for all adapters; version probes and live native smoke testing are required before image promotion. - The image-owned directory takes precedence. Its entries must launch the same shared CLI as the global PATH, not a private older/newer copy. - Application qualification pins and the deployed image must move together. No startup fallback installation is added. - No schema or authentication-policy changes. ## Model Used OpenAI GPT-6 (Codex). The session does not expose a more specific model ID or context-window size. Used reasoning, repository inspection, code execution, and browser verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
5716fe907e |
test(runner): add full-stack acceptance and eval gates (#12700)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The runner subsystem executes agent work across local and managed provider backends. > - The lower pull requests restore the task runtime, provider backends, and managed-provider control plane. > - The restored system needs repeatable full-stack checks before it can ship safely. > - Paid live checks also need clear access, cost, and secret controls. > - This pull request adds acceptance, live evaluation, chaos, and release gates for the restored runner stack. > - The benefit is measurable runner parity with safer release decisions. ## Linked Issues or Issue Description **Subsystem affected** Cross-cutting. This change covers runner tests, release workflows, server contracts, and evaluation tools. **Problem or motivation** The runner stack did not have one complete acceptance surface for native Codex, ACPX, Claude Managed, and AWS AgentCore. Release checks could miss provider drift, task-view regressions, cost-policy errors, and destructive cleanup errors. **Proposed solution** Add a 57-cell full-stack catalog, a Daytona image, and opt-in paid workflows. Add live evaluation, chaos, cost-limit, redaction, and release contract checks. Add AWS AgentCore infrastructure and guarded provisioning tools. Keep the native runner experimental flag off by default. **Alternatives considered** We considered manual smoke tests only. They do not give repeatable evidence and they do not protect release branches. We also considered one large pull request. The stacked pull requests keep each review below the Greptile file limit. **Roadmap alignment** This work supports the shipped Cloud / Sandbox agents milestone and the shipped Agent evals & feedback milestone in `ROADMAP.md`. Related stack: - #12699 adds managed provider backends and lifecycle support. - #12691 adds qualified OpenCode and ACPX provider backends. - #12685 restores task runtime rendering and steering. ## What Changed - Add the runner full-stack harness with 57 catalog cells and 60 unit tests. - Add a Daytona runner image with digest-pinned base images and base-aware image-content checks. - Add guarded live evaluation and chaos workflows with a fixed 40-execution matrix; live and full-stack paid schedules now run only on Sundays or by manual dispatch. - Add in-flight reported-usage cost stops, post-turn cost caps, exact-threshold failure classification, secret redaction, retry classification, and actor authorization. - Reattach stream and hard-budget listeners before restart-recovery continuations so restored paid sessions cannot bypass in-flight interruption. - Preserve OpenCode usage and cost across tool-loop messages and turns while exposing an explicit current-run delta to durable accounting. - Keep PNG/WebM evidence in access-controlled artifacts only, reject SVG, and publish only pruned inert structured per-attempt evidence. - Add AWS AgentCore infrastructure, provisioning checks, and smoke tools; reject unsafe model identifiers, require exact stack ownership markers, and make failed-stack replacement explicit. - Add evaluation-session contracts and capability reports. - Add release workflow checks for immutable action pins, frozen dependency installs, exact weekly cron shape, paid-run guards, provider-secret isolation, and chaos test paths. - Reauthorize the original and triggering numeric actor IDs as the first step of every provider-secret job, including partial reruns, before checkout or provider access. - Give each full-stack matrix cell only its matching provider credential, expose Daytona only to Daytona cells, and disable shared dependency caches anywhere paid credentials or OIDC write access are present. - Protect the legacy manual E2E workflow with the same default-branch, allowlist, environment, and per-job authorization boundary. - Rotate live-eval candidates by week and retain 120 days of compatible history so the seven-week trend window remains viable. - Restore the root runner-acceptance commands and reconcile reported snapshots, raw receipts, and terminal usage without double counting or losing late usage. - Mark ACPX token deltas exact only when every budget field is present, keep cumulative cost/request authority separate, reject non-USD cost labeling, and include thought tokens in output-token budgets. - Keep `enableNativeRunner` off by default. The acceptance harness enables it only in its isolated test instance. ## Verification Passed locally: - `pnpm --filter @paperclipai/paperclip-runner typecheck` - `pnpm test:runner-acceptance:typecheck` - `pnpm test:runner-acceptance` (19 tests) - focused OpenCode proxy, driver, runnerd transport, live-session, and turn-stream tests (106 tests) - `pnpm --filter @paperclipai/paperclip-runner exec vitest run src/live/clean-room-server.test.ts` (22 tests) - `pnpm test:e2e:runner:typecheck` - `pnpm test:e2e:runner:unit` (62 tests) - `node --test scripts/__tests__/release-verify-workflow.test.mjs` - `pnpm --filter @paperclipai/paperclip-runner test:runner-workflow-evals` (22 tests) - `pnpm -r typecheck` - `pnpm build` - `node --test packages/paperclip-runner/scripts/aws-agentcore-provisioning.test.mjs` (6 tests) - `git diff --check` - `cargo test --manifest-path packages/paperclip-runner/runner/Cargo.toml -p paperclip-runner-core --lib --locked` (161 tests) - focused ACPX provider-event tests (10 tests) - The rebased PR changes 92 files. `pnpm-lock.yaml` is unchanged. I did not run paid live provider jobs or provision AWS resources. Those checks need credentials and can create cost. ## Risks The paid workflows can create provider cost. They require an allowlisted original and triggering actor, the protected `runner-e2e-paid` environment, explicit opt-in variables, and cost limits. The four provider credentials exist only in that master-only environment, which requires allowlisted reviewer approval and disables administrator bypass; repository and organization Actions scopes contain no copies. Provider usage arrives after a billable request, so the live guard cannot prevent one request from crossing a threshold. It interrupts immediately on the first reported threshold hit and permits no continuation. Visual evidence can contain secrets rendered as pixels. PNG/WebM remain only in access-controlled workflow artifacts; SVG and per-attempt XML are excluded, and S3/Pages receive a pruned structured dashboard. The AWS scripts can create cloud resources. They use explicit commands, least-privilege roles, KMS encryption, saved nonsecret metadata, and explicit teardown. This pull request does not enable the experimental native runner for existing instances. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex with GPT-5. The model used extended reasoning, tool use, code execution, and parallel subagents. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |