Files
PaperClipAI/docker/daytona-runner
DottaandPaperclip a68f3d8e35 fix(runner-e2e): align Daytona image and provider pack provenance (#13814)
## Thinking Path

> - Paperclip is an open source app people use to manage AI agents for
work.
> - The Daytona runner image provides the native runner and its provider
package.
> - The controller also sends a provider package to native Daytona cells
when the image package does not match.
> - A stale image and a package from another source revision caused a
1.8 GB upload before Claude could run.
> - This pull request refreshes the reviewed lock checksum and documents
how to reuse the exact package from an immutable image.
> - The benefit is a reproducible setup path and clear evidence when
image and package provenance do not match.

## Linked Issues or Issue Description

**What happened?**

A Daytona native Claude run used image source revision
`45c99a0d06cbd5b04982b06b79de321149930ac5` with a controller provider
package from revision `294853dc...`. Runtime verification rejected the
image package and staged a large package upload before model execution.
A hosted campaign also failed during image setup because the Dockerfile
expected lock checksum `d7d96cf0...` while the resolved lockfile
checksum was
`4b796c312833ebf2be4c38228babc0292c76774fb40d43bd18b54bd6b205753d`.

Related public work reviewed:
[#12795](https://github.com/paperclipai/paperclip/pull/12795),
[#12862](https://github.com/paperclipai/paperclip/pull/12862), and
[#12887](https://github.com/paperclipai/paperclip/pull/12887).

**Expected behavior**

The Daytona image and controller provider package must come from the
same verified build. A package extracted from the immutable image must
pass the existing manifest, source revision, lockfile, binary, bridge,
and artifact checks before a native Claude run starts.

**Steps to reproduce**

1. Set `PAPERCLIP_E2E_DAYTONA_IMAGE` to the old immutable image digest.
2. Set `PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH` to a package built
from a different source revision.
3. Run a native Claude Daytona cell.
4. Observe provider package verification failure followed by the large
staging upload.
5. Build the image with the stale Dockerfile lock checksum and observe
the checksum failure.

**Paperclip version or commit**

`3b8df3dcd6f99e99277faa45f3351989edb5c239`.

**Deployment mode**

Daytona native runner E2E.

**Install method**

Built from source.

**Agent adapter(s) involved**

Claude Code through the native ACPX runner.

**Database mode**

Not database-related.

**Access context**

Not applicable to the setup failure.

## What Changed

- Refreshed `PAPERCLIP_RUNNER_LOCK_SHA256` in
`docker/daytona-runner/Dockerfile` to the resolved lockfile checksum.
- Added a local guide for extracting the provider package from an
immutable verified image with Docker.
- Documented the required provenance checks and the expected
manifest-matched runtime log.
- Documented that cold package upload coverage must remain separate from
recovery coverage.
- Pinned the preview-service test guest to the test runner’s Node
executable and logged guest startup and bound ports for readiness
diagnostics.

## Verification

- 442 focused E2E tests passed.
- Typecheck passed.
- `pnpm build` passed.
- Five provider-pack reuse tests passed.
- Six Daytona image contract tests passed.
- Fixed Daytona campaign
[35740613581](https://github.com/paperclipai/paperclip/actions/runs/35740613581)
passed.
- The overall hiring campaign
[35739993219](https://github.com/paperclipai/paperclip/actions/runs/35739993219)
failed because of an unrelated Mini metadata failure; its Claude cell
passed.
- Full local `pnpm test:run` was attempted but did not complete. The
isolated Postgres install was repaired and its 15-test probe passed.
- After the fixture change, all seven preview reservation tests passed
locally and CI server shard 6/12 passed on `20234f75f`. This removes
login-shell Node resolution variance; the exact cause of the earlier
CI-only timeout is not established.
- CI run
[35766034635](https://github.com/paperclipai/paperclip/actions/runs/35766034635)
passed on `20234f75f`. All server, runner, browser, build, and typecheck
gates passed.
- The signoff browser case initially failed waiting for an approver run.
All five signoff tests passed locally without changes; the one allowed
CI retry passed all 19 shard tests. This is recorded as an intermittent
failure, not a demonstrated product fix.
- Greptile reviewed `20234f75f`: 5/5, no actionable findings.
- [Follow-up
report](https://pages.paperclip.ing/runner-daytona-hiring-20260922/)
includes timings, evidence links, and the remaining hiring configuration
failure.
- Review the immutable image source revision and extracted
`provider-pack.json` before another paid recovery run.

## Risks

- The Dockerfile checksum gate intentionally fails when the resolved
lockfile changes. A future dependency change must refresh the reviewed
checksum with the image change.
- The local extraction guide requires Docker and a pullable immutable
image.
- An image built from an older source revision can still fail runtime
manifest verification. The guide does not bypass that check.
- The change does not alter runner prompts, approval policy, or recovery
behavior.

## Model Used

OpenAI Codex using the primary GPT-6 backend; the exact backend
deployment ID is not exposed. Repository analysis and code execution
used tool access. Assistance also came from OpenAI gpt-5.6-luna.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-22 13:49:00 -05:00
..

Paperclip Daytona runner image

This image is the Paperclip Cloud fleet sandbox image plus a source-built paperclip-runnerd and immutable provider pack. The pack contains Node 24.11, OpenCode 1.18.29, the compiled OpenCode proxy, ACPX 0.13.1 sidecar, qualified ACP agents, and the production lockfile. Its manifest digests each executable bridge and binds the pack to the runner source revision, avoiding artifact upload and npm installation on every fresh lease.

The fleet pins are intentionally copied from paperclip-cloud/fleet-sandbox-image/Dockerfile. Update both definitions together until the fleet base is published as a stable image that this Dockerfile can extend directly.

Build and verify

The fleet image is currently amd64-only because the pinned Cursor and GitHub CLI checksums cover amd64.

content_id="$(pnpm --silent test:e2e:runner:image-id)"
docker buildx build \
  --platform linux/amd64 \
  --build-arg PAPERCLIP_RUNNER_CONTENT_ID="${content_id}" \
  --build-arg PAPERCLIP_RUNNER_SOURCE_REVISION="$(git rev-parse HEAD)" \
  --tag "paperclip-daytona-runner:e2e-content-${content_id}" \
  --load \
  --file docker/daytona-runner/Dockerfile \
  .

docker run --rm --platform linux/amd64 \
  --entrypoint paperclip-runnerd \
  "paperclip-daytona-runner:e2e-content-${content_id}" \
  --build-metadata

The metadata must advertise dial_ws_loopback, dial_wss, and listen_ws. The explicit entrypoint is needed only for this local probe because Daytona's base image uses its own long-running sandbox entrypoint.

test:e2e:runner:image-id hashes the audited Docker build dependency closure, target platform, the immutable Dockerfile syntax-frontend digest, and every immutable FROM reference. It fails before the paid workflow can build when the frontend or a base is not pinned to a sha256 digest. When updating the syntax version, resolve and review its registry digest and update both values in the first Dockerfile line. Git commits that do not change those inputs reuse the same content tag.

PAPERCLIP_RUNNER_SOURCE_REVISION remains the full Git SHA that built the first published copy and is retained as provenance rather than cache identity.

Use in Paperclip

Publish the image to a registry Daytona can pull, or use the environment editor's Configure image flow to produce a Daytona snapshot. Set the environment image to that immutable tag or snapshot. Paperclip probes the sandbox user's PATH for paperclip-runnerd and codex and checks /opt/paperclip-runner/provider-pack for OpenCode and ACPX. It uses the pack only when its complete manifest matches the controller's build-owned pack; otherwise it stages the pack configured by PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH. Remote OpenCode and ACPX never fall back to host-local processes.

Do not promote paperclip-runner-e2e-20260826-v2 for OpenCode or ACPX. Build a new immutable image or snapshot from a clean committed revision and pass that full Git SHA as PAPERCLIP_RUNNER_SOURCE_REVISION.

Do not bake provider credentials, Paperclip bootstrap tickets, or Daytona preview tokens into this image. They remain per-run secret material.

Provider CLI updates are manifest-only changes: repository CI owns the root lockfile. The image build resolves the complete workspace manifest graph before its frozen install, matching CI when a source commit precedes the lockfile bot. The complete resolved lockfile must match PAPERCLIP_RUNNER_LOCK_SHA256 before package installation or lifecycle execution. Review and refresh that digest with source dependency changes; registry-time resolution drift fails closed. Keep one latest stable CLI installation per provider; refresh exact runtime versions and qualification digests together, never install a private older copy or download dependencies when a task starts.