mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-09 07:10:34 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Runner Full-Stack E2E workflow tests the Runner in Daytona sandboxes from a signed, content-addressed image > - Every change to an audited Runner input needs a new image before a Daytona cell can test it > - The image job took 10 to 16.5 minutes today, and a restart chaos campaign waits on it after each runner-side fix > - Measurement shows two serial waits: a full Rust dependency compile (about 5 of 6.6 minutes) and a 3.5 to 4 minute download of cached base layers that starts only after the builder stages finish > - There is also no way to build only the image: every dispatch reaches a paid gate that the operator must reject > - This pull request removes both serial waits and adds an `image_only` dispatch input > - The benefit is a shorter fix-to-Daytona-test loop, with every image security check unchanged ## Linked Issues or Issue Description **What happened?** The `Publish verified Daytona image` job is on the critical path of the control-plane restart chaos campaign. Builds today took 16.5 min (run 42240434011), 15 min (42253920727), 10.5 min (42259118276) and 16 min (42265637650). Run 42264611573 failed: its RunsOn runner lost communication with GitHub 7.6 minutes into the build step, during the Rust compile. GitHub has no log for that job. **Expected behavior** A Runner source change should not recompile the locked third-party Rust graph. Cached image layers should download while the builder stages compile. An operator should be able to publish only the image, without a paid gate. **Steps to reproduce** Read the BuildKit timeline in the logs of job 127970125495 (run 42240434011). See the stage table below. ## What Changed - `docker/daytona-runner/Dockerfile`: compile the locked Rust dependency graph in its own layer against placeholder targets. The layer depends only on `Cargo.toml`, `Cargo.lock` and the crate manifest. The step deletes the placeholder outputs and fingerprints, so the next step always compiles the real Runner sources. The final stage still receives only the stripped executable. - `docker/daytona-runner/Dockerfile`: move the content-ID / source-revision check above the two `COPY --from` steps. It does not depend on either builder stage, so BuildKit now downloads the cached base and CLI layers in parallel with the compile. Both copies use `COPY --link`. - `.github/workflows/runner-full-stack-e2e.yml`: add the boolean `image_only` dispatch input. `authorize`, `target_lock`, `catalog` and `daytona_image` run unchanged. Every other campaign job gets `&& inputs.image_only != true`. `authorize` rejects `image_only` together with TestOps inputs. - `tests/runner-e2e/workflow-image-only.test.ts` (new): pin the job classification, the top-level guard on each campaign job, the guard on every protected environment, and the TestOps rejection. - `tests/runner-e2e/README.md`: document the layer layout and the `image_only` command. ## Verification Measured on the same RunsOn `m7i-flex.xlarge` (4 vCPU, 16 GiB, native x64) runner class. "Before" is master job 127970125495 (run 42240434011). "After (cold)" is this branch, job 128077687214 (run 42271236140), with no dependency layer in the registry cache yet. Its paid gate was rejected. | Stage | Before | After (cold dependency cache) | |---|---|---| | Whole image job | 16.5 min | **11.6 min** | | Rust dependency graph | 311 s (inside one 399 s step) | 395 s (own layer; reused from cache after a master build) | | `paperclip-runner-core` compile | ~87 s | 65 s | | Cached base and CLI layer download | 236 s, **after** cargo finished | 257 s, **in parallel** with cargo | | Copy runnerd + provider pack | 20 s + 50 s | 9 s (`--link`) | | Provider pack stage | 143 s (parallel) | 152 s (parallel) | | Export + push | 82 s + 10 s | 81 s + 11 s | | Dependency layer size | n/a | `target` 852 MB + registry 333 MB (builder stage only) | With a warm dependency cache, a Runner source change compiles only the 65 s crate step. The critical path then becomes the ~4 min cached-layer download or the provider pack. The expected job time is about 7 to 8 min. I measure this after merge with an `image_only` dispatch (master first, to write the cache, then a Rust-only branch change). Disk: the instance starts with 38.76 GiB free. The dependency layer adds about 1.2 GB uncompressed to builder storage. The final image does not change. - `vitest run tests/runner-e2e/workflow-image-only.test.ts tests/runner-e2e/workflow-security.test.ts tests/runner-e2e/daytona-image.test.ts`: the new test and all image tests pass. Three tests in `workflow-security.test.ts` (`gates every provider-secret job with stable actor IDs`, `keeps provider credentials inside explicitly gated paid workflows`, `isolates private archive trust from workflows that execute target-branch code`) fail on `origin/master` without this change, because of `chat-catalog-testops-case.yml`. This pull request does not change them. - `node --test tests/runner-e2e/product-regression-testops.test.mjs`: 15/15 pass. - No local Docker build. The Dockerfile is measured with a branch-targeted CI dispatch whose paid gate was rejected. ## Risks - The dependency layer keeps the Rust `target` and registry trees in a builder-stage layer, and default-branch builds export it to the registry cache. This adds builder disk and cache upload when `Cargo.lock` changes. The build prints the size (`du -sk target /usr/local/cargo/registry`). - The dependency cache helps only after a default-branch build writes it. Branch builds read it but never write it. This is the existing cache policy. - `image_only` changes the workflow, so it works only after merge to master. - No check is removed. Lockfile verification, cosign sign and verify, the anonymous pull check, the provider-pack manifest match, the default-branch-only cache write and the actor allowlist are unchanged. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code with tool use (shell, GitHub CLI). Extended reasoning on. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
199 lines
12 KiB
Docker
199 lines
12 KiB
Docker
# syntax=docker/dockerfile:1.7@sha256:a57df69d0ea827fb7266491f2813635de6f17269be881f696fbfdf2d83dda33e
|
|
|
|
# Keep the Rust toolchain explicit so the runner artifact is reproducible and
|
|
# is always built for the same platform as the final Daytona image. Base-image
|
|
# digests are the reviewed linux/amd64 manifests; the tags are annotations.
|
|
FROM rust:1.97-bookworm@sha256:408fe88047cef61a2087653b0c5255fa51c0f2d6d94ddedd7a2562a9b91a46f6 AS runnerd-build
|
|
|
|
WORKDIR /workspace/packages/paperclip-runner/runner
|
|
COPY packages/paperclip-runner/runner/Cargo.toml packages/paperclip-runner/runner/Cargo.lock ./
|
|
COPY packages/paperclip-runner/runner/crates/runner-core/Cargo.toml ./crates/runner-core/Cargo.toml
|
|
# Compile the locked third-party dependency graph against placeholder targets
|
|
# so this layer depends only on the manifests and lockfile. Runner source
|
|
# changes then reuse it from the registry cache and recompile one crate instead
|
|
# of the whole AWS/TLS graph (about five of the build's six and a half minutes).
|
|
# The placeholder targets' outputs and fingerprints are deleted, so the next
|
|
# step always compiles the real runner sources against the same lockfile.
|
|
RUN set -eu; \
|
|
mkdir -p crates/runner-core/src; \
|
|
: > crates/runner-core/src/lib.rs; \
|
|
for target in $(grep -Eo 'src/bin/[A-Za-z0-9_-]+\.rs' crates/runner-core/Cargo.toml); do \
|
|
mkdir -p "crates/runner-core/$(dirname "$target")"; \
|
|
printf 'fn main() {}\n' > "crates/runner-core/$target"; \
|
|
done; \
|
|
cargo build --locked --release -j 2 -p paperclip-runner-core --bin paperclip-runnerd; \
|
|
rm -rf crates/runner-core/src \
|
|
target/release/.fingerprint/paperclip-runner-core-* \
|
|
target/release/deps/*paperclip_runner* \
|
|
target/release/paperclip-runnerd target/release/paperclip-runnerd.d; \
|
|
du -sk target /usr/local/cargo/registry
|
|
COPY packages/paperclip-runner/runner/crates ./crates
|
|
COPY packages/paperclip-runner/protocol ../protocol
|
|
COPY packages/paperclip-runner/acpx-profiles.json packages/paperclip-runner/antigravity-distributions.json ../
|
|
# Only the finished executable leaves this stage. The final image never carries
|
|
# Cargo's target or download trees; the dependency layer above keeps them only
|
|
# in this builder stage and the trusted build cache.
|
|
RUN cargo build --locked --release -j 2 -p paperclip-runner-core --bin paperclip-runnerd \
|
|
&& strip /workspace/packages/paperclip-runner/runner/target/release/paperclip-runnerd \
|
|
&& install -m 0755 target/release/paperclip-runnerd /usr/local/bin/paperclip-runnerd \
|
|
&& rm -rf target /usr/local/cargo/registry /usr/local/cargo/git
|
|
|
|
FROM node:24.21.0-bookworm@sha256:5a750d3be5e5c80275f8c9a5367c3aed99c2875656590c8d0701c7ee687f5f0a AS provider-pack-build
|
|
RUN corepack enable && corepack prepare pnpm@9.15.4 --activate
|
|
# Bound build-time memory on hosts that expose many CPUs but little RAM.
|
|
# These settings belong only to this build stage, never the runtime image.
|
|
ENV NODE_OPTIONS=--max-old-space-size=1024 \
|
|
npm_config_child_concurrency=1 \
|
|
npm_config_network_concurrency=4
|
|
WORKDIR /workspace
|
|
COPY package.json pnpm-lock.yaml pnpm-workspace.yaml .npmrc tsconfig.base.json ./
|
|
COPY patches ./patches
|
|
COPY scripts/link-plugin-dev-sdk.mjs ./scripts/link-plugin-dev-sdk.mjs
|
|
# Include the complete workspace manifest graph so source-owned patches apply.
|
|
# Verify the reviewed lockfile as supplied; never resolve dependencies again
|
|
# against the registry during an immutable image build.
|
|
COPY packages ./packages
|
|
COPY server/package.json ./server/package.json
|
|
COPY ui/package.json ./ui/package.json
|
|
COPY cli/package.json ./cli/package.json
|
|
# The complete resolved lock (including transitive integrity hashes) is reviewed.
|
|
# Reject registry-time drift BEFORE installing packages or running lifecycle code.
|
|
# Every build supplies the digest of the reviewed lockfile it was given; there
|
|
# is no default, so a stale fixed value can never masquerade as lockfile drift.
|
|
ARG PAPERCLIP_RUNNER_LOCK_SHA256
|
|
ARG PAPERCLIP_RUNNER_SOURCE_REVISION
|
|
# Optional qualification assets; this never promotes candidate profiles.
|
|
ARG PAPERCLIP_RUNNER_CANDIDATE_PROVIDERS=
|
|
# pnpm 9 subtracts PNPM_WORKERS from available CPUs; it is not a worker count.
|
|
# Subtract all available CPUs to select its minimum (one tarball worker).
|
|
# Install and build in one layer so the disposable workspace and package store
|
|
# do not remain in an earlier snapshot while BuildKit exports the portable pack.
|
|
RUN if ! printf '%s' "${PAPERCLIP_RUNNER_LOCK_SHA256}" | grep -Eqx '[0-9a-f]{64}'; then \
|
|
echo 'PAPERCLIP_RUNNER_LOCK_SHA256 build arg must be the SHA-256 of pnpm-lock.yaml: --build-arg "PAPERCLIP_RUNNER_LOCK_SHA256=$(shasum -a 256 pnpm-lock.yaml | cut -d " " -f 1)"' >&2; \
|
|
exit 1; \
|
|
fi \
|
|
&& test -n "${PAPERCLIP_RUNNER_SOURCE_REVISION}" \
|
|
&& export PNPM_WORKERS="$(node -p 'require("node:os").availableParallelism()')" \
|
|
&& printf '%s pnpm-lock.yaml\n' "${PAPERCLIP_RUNNER_LOCK_SHA256}" > /tmp/provider-lock.sha256 \
|
|
&& sha256sum -c /tmp/provider-lock.sha256 \
|
|
&& pnpm install --frozen-lockfile --filter '@paperclipai/paperclip-runner...' \
|
|
&& pnpm --filter @paperclipai/paperclip-runner build:typescript \
|
|
&& PAPERCLIP_RUNNER_SOURCE_REVISION="${PAPERCLIP_RUNNER_SOURCE_REVISION}" \
|
|
node packages/paperclip-runner/scripts/build-provider-pack.mjs /provider-pack \
|
|
"--candidate-providers=${PAPERCLIP_RUNNER_CANDIDATE_PROVIDERS}" \
|
|
&& chmod -R a+rX /provider-pack \
|
|
&& du -sk /workspace /provider-pack \
|
|
&& rm -rf /workspace /root/.local/share/pnpm/store
|
|
|
|
# Fleet sandbox base image. Keep this section aligned with
|
|
# paperclipai/paperclip-cloud/fleet-sandbox-image/Dockerfile. The only Paperclip
|
|
# runner-specific addition is /usr/local/bin/paperclip-runnerd below.
|
|
FROM daytonaio/sandbox:0.8.0@sha256:eadf88e4391072b7ad4bed27d9cadfc9fe9d8ed375d9219d34c2ccb518f213e3
|
|
|
|
USER root
|
|
|
|
ENV PAPERCLIP_RUNNER_PROVIDER_PACK_ROOT=/opt/paperclip-runner/provider-pack
|
|
ENV PATH=${PAPERCLIP_RUNNER_PROVIDER_PACK_ROOT}/node_modules/.bin:/usr/local/share/nvm/current/bin:/usr/local/python/current/bin:/usr/local/py-utils/bin:${PATH}
|
|
|
|
# Share the pack's latest stable Codex, Claude and OpenCode with every adapter.
|
|
# Never add a second CLI version to work around a runner compatibility pin.
|
|
RUN npm uninstall -g @anthropic-ai/claude-code @openai/codex opencode-ai \
|
|
&& npm install -g \
|
|
@xai-official/grok@1.0.70 \
|
|
@google/gemini-cli@0.65.0 \
|
|
@moonshot-ai/kimi-code@2.3.3 \
|
|
&& npm cache clean --force
|
|
|
|
# Hermes requires Python >=3.11,<3.14; the image's default Python is newer.
|
|
# Use the distro interpreter in a dedicated environment without changing PATH
|
|
# or keeping an older Hermes CLI. Everything is installed before sandbox boot.
|
|
# Debian 13 in the sandbox base already includes Python 3.13 and python3-venv.
|
|
RUN /usr/bin/python3 -m venv /opt/hermes \
|
|
&& /opt/hermes/bin/pip install --no-cache-dir hermes-agent==0.19.0 \
|
|
&& ln -sf /opt/hermes/bin/hermes /usr/local/bin/hermes
|
|
|
|
ARG CURSOR_VERSION=2026.10.22-27dcf5f
|
|
ARG CURSOR_SHA256_AMD64=85296944e4600a83c4b764f578d1502bf9c416db200d33a593e8f3ecabfdc294
|
|
RUN set -eu; \
|
|
arch="$(dpkg --print-architecture)"; \
|
|
[ "$arch" = "amd64" ] || { echo "FATAL: cursor pin only covers amd64, not $arch" >&2; exit 1; }; \
|
|
dir="/opt/cursor/versions/${CURSOR_VERSION}"; mkdir -p "$dir"; \
|
|
curl -fsSL "https://downloads.cursor.com/lab/${CURSOR_VERSION}/linux/x64/agent-cli-package.tar.gz" -o /tmp/cursor.tgz; \
|
|
printf '%s /tmp/cursor.tgz\n' "${CURSOR_SHA256_AMD64}" > /tmp/cursor.sha256; \
|
|
sha256sum -c /tmp/cursor.sha256; \
|
|
tar --strip-components=1 -xzf /tmp/cursor.tgz -C "$dir"; \
|
|
rm -f /tmp/cursor.tgz /tmp/cursor.sha256; \
|
|
ln -sf "$dir/cursor-agent" /usr/local/bin/cursor-agent; \
|
|
ln -sf /usr/local/bin/cursor-agent /usr/local/bin/agent; \
|
|
chmod -R a+rX /opt/cursor
|
|
|
|
ARG GH_VERSION=2.103.3
|
|
ARG GH_SHA256_AMD64=4c806978ffce1cec70853302a56d70ac6da4f19bae7adca4e1a8ab00633dd91d
|
|
RUN set -eu; \
|
|
arch="$(dpkg --print-architecture)"; \
|
|
[ "$arch" = "amd64" ] || { echo "FATAL: gh pin only covers amd64, not $arch" >&2; exit 1; }; \
|
|
curl -fsSL "https://github.com/cli/cli/releases/download/v${GH_VERSION}/gh_${GH_VERSION}_linux_amd64.tar.gz" -o /tmp/gh.tgz; \
|
|
printf '%s /tmp/gh.tgz\n' "${GH_SHA256_AMD64}" > /tmp/gh.sha256; \
|
|
sha256sum -c /tmp/gh.sha256; \
|
|
tar -xzf /tmp/gh.tgz -C /tmp; \
|
|
install "/tmp/gh_${GH_VERSION}_linux_amd64/bin/gh" /usr/local/bin/gh; \
|
|
rm -rf /tmp/gh.tgz /tmp/gh.sha256 "/tmp/gh_${GH_VERSION}_linux_amd64"
|
|
|
|
COPY packages/paperclip-runner/scripts/provision-grok.mjs /tmp/grok-provision/scripts/provision-grok.mjs
|
|
COPY packages/paperclip-runner/src/providers/grok/platforms.json /tmp/grok-provision/src/providers/grok/platforms.json
|
|
RUN node /tmp/grok-provision/scripts/provision-grok.mjs /opt/paperclip/providers/grok/1.0.13/grok && rm -rf /tmp/grok-provision
|
|
|
|
# Keep revision-dependent metadata below the stable agent CLI installation
|
|
# layers. A source-only image miss can then reuse those expensive layers from
|
|
# the trusted registry cache.
|
|
# This step must stay ABOVE the COPY --from steps. It does not wait for either
|
|
# builder stage, so BuildKit downloads the cached base and CLI layers (about
|
|
# four minutes) while runnerd and the provider pack compile, instead of after.
|
|
ARG PAPERCLIP_RUNNER_CONTENT_ID
|
|
ARG PAPERCLIP_RUNNER_SOURCE_REVISION
|
|
RUN test -n "${PAPERCLIP_RUNNER_CONTENT_ID}" \
|
|
&& test -n "${PAPERCLIP_RUNNER_SOURCE_REVISION}"
|
|
|
|
# --link builds each copied tree as an independent layer, so producing it does
|
|
# not wait for the cached layers below to finish downloading.
|
|
COPY --link --from=runnerd-build /usr/local/bin/paperclip-runnerd /usr/local/bin/paperclip-runnerd
|
|
COPY --link --chown=0:0 --from=provider-pack-build /provider-pack /opt/paperclip-runner/provider-pack
|
|
RUN set -eu; for cli in codex claude opencode; do \
|
|
printf '#!/bin/sh\nexec /opt/paperclip-runner/provider-pack/node_modules/.bin/%s "$@"\n' "$cli" > "/usr/local/bin/$cli"; \
|
|
chmod 755 "/usr/local/bin/$cli"; \
|
|
done
|
|
|
|
RUN set -eu; \
|
|
printf '%s\n' 'export PATH=/opt/paperclip-runner/provider-pack/node_modules/.bin:$PATH' \
|
|
> /etc/profile.d/01-paperclip-runner-provider-pack.sh; \
|
|
chmod 0644 /etc/profile.d/01-paperclip-runner-provider-pack.sh; \
|
|
for command_name in acpx claude-agent-acp codex-acp claude codex grok gemini kimi opencode cursor-agent agent hermes gh paperclip-runnerd; do \
|
|
command -v "$command_name" >/dev/null || { echo "FATAL: $command_name not on PATH after build" >&2; exit 1; }; \
|
|
done; \
|
|
metadata="$(paperclip-runnerd --build-metadata)"; \
|
|
/opt/paperclip-runner/provider-pack/node_modules/node/bin/node -e "const [major,minor,patch]=process.versions.node.split('.').map(Number); if (major<24 || (major===24 && (minor<11 || (minor===11 && patch<0)))) process.exit(1)"; \
|
|
test -f /opt/paperclip-runner/provider-pack/provider-pack.json; \
|
|
printf '%s' "$metadata" | grep -q '"dial_ws_loopback"'; \
|
|
printf '%s' "$metadata" | grep -q '"dial_wss"'; \
|
|
printf '%s' "$metadata" | grep -q '"listen_ws"'; \
|
|
echo "all agent CLIs and runnerd transport modes are available"
|
|
|
|
USER daytona
|
|
|
|
RUN /bin/sh -lc 'set -eu; \
|
|
test -r /opt/paperclip-runner/provider-pack/provider-pack.json; \
|
|
/opt/paperclip-runner/provider-pack/node_modules/node/bin/node -e \
|
|
"JSON.parse(require(\"node:fs\").readFileSync(\"/opt/paperclip-runner/provider-pack/provider-pack.json\", \"utf8\"))"; \
|
|
for command_name in acpx claude-agent-acp codex-acp; do \
|
|
command -v "$command_name" >/dev/null || { echo "FATAL: $command_name not on PATH for daytona user" >&2; exit 1; }; \
|
|
done; \
|
|
test "$(codex --version)" = "codex-cli 0.160.0"; \
|
|
test "$(claude --version)" = "2.1.286 (Claude Code)"; \
|
|
test "$(opencode --version)" = "1.18.34"; \
|
|
test "$(acpx --version)" = "0.13.1"; \
|
|
test "$(claude-agent-acp --version)" = "0.73.0"; \
|
|
test "$(codex-acp --version)" = "@agentclientprotocol/codex-acp 1.6.2"'
|
|
|
|
LABEL io.paperclip.runner.content-id="${PAPERCLIP_RUNNER_CONTENT_ID}" \
|
|
org.opencontainers.image.revision="${PAPERCLIP_RUNNER_SOURCE_REVISION}"
|