feat(ci): add the Runner image resolver and a no-rebuild mode for the Runner vendor step (#21306)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The Runner is the part that executes agent work: a daemon
(`paperclip-runnerd`) and a provider pack of agent tools
> - `master` now publishes the Runner alone as the image
`ghcr.io/paperclipai/paperclip-runner`, one time for each set of Runner
inputs (#21204, #21229, #21230)
> - The application image still compiles its own Runner on every build.
The next steps make it copy the published Runner instead
> - Before a build can do that, it must answer one question the same way
as every other build: "does a published Runner exist for exactly this
tree?" It must also fail clearly when the Runner it was given does not
match the tree
> - This pull request adds the script that answers the question, and a
mode of the Runner vendor step that fails instead of rebuilding
> - The benefit is that the later change to the image build is small,
and that a wrong or unreachable answer can only mean "build the Runner
yourself"

## Linked Issues or Issue Description

Refs #21204, #21229 and #21230 (the Runner image). No issue exists for
this change.

**Problem or motivation**

Three builders make images that contain the Runner: `docker.yml`, the
merge-queue build and the release preview build. Each one would have to
repeat three steps to find the published Runner of its tree: check the
lockfile, compute the inputs digest in the build, and look up
`inputs-<digest>`. Three copies can decide differently. A build that
copies a Runner built elsewhere also has no Rust toolchain. If the
copied Runner does not match the tree, `prepare-runner-vendor.mjs`
starts a rebuild, and the build fails late at `cargo` with a message
that hides the cause.

**Proposed solution**

Add `scripts/runner-image-resolve.mjs`, the one place that decides. Add
`PAPERCLIP_RUNNER_VENDOR_REQUIRE_CURRENT=1` to
`prepare-runner-vendor.mjs`. Nothing calls the resolver and nothing sets
the variable in this pull request, so no workflow job changes.

**Alternatives considered**

- One command that builds and looks up. Two commands keep a registry
credential, when a caller has one, out of the step that builds.
- A wait for the publication of the commit. The decision whether the
application build waits is open, so the resolver has no wait mode. It
reports `absent` and stops.
- A build argument that names the expected inputs. No `Dockerfile`
declares one yet. The change that makes the application image copy the
Runner will add both ends together.

**Roadmap alignment**

This is build infrastructure. It adds no product feature and does not
overlap an item in `ROADMAP.md`.

This is part 1 of 4 of a stack ([stack
#21309](https://github.com/paperclipai/paperclip/stack/21309)). Merge
the parts in order:

1. #21306: the resolver and the no-rebuild mode (this pull request)
2. #21307: the report-only job in `docker.yml` and the pull-request
check of the application image's Runner path
3. #21308: reading a retained provider pack from its Runner image
4. #21310: a registry build cache for the Runner image build

## What Changed

- `scripts/runner-image-resolve.mjs` (new, 185 lines), with two commands
that share a directory:
- `inputs --dir <dir>` exports the `runner-artifacts-context` target,
runs the lockfile check on that export, and builds the
`runner-artifacts-identity` target when the lockfile is current. It
reads no credential. A lockfile that is behind stops there: no image is
published for such a tree.
- `resolve --dir <dir>` looks up `inputs-<digest>` and prints `mode`,
`reason`, `inputs_digest`, `image`, `slim_image`, `build_contexts` and
`labels`. It also writes them to `$GITHUB_OUTPUT` and a short report to
`$GITHUB_STEP_SUMMARY`.
- `mode` is `published` only for a complete publication whose full index
names these inputs and both platforms. Both images are pinned by index
digest. The slim digest is the one the full index names in
`io.paperclip.runner.slim-digest`; the slim tag is never read on its
own.
- Every other answer is `inline`, with the last four values empty:

  | `reason` | Meaning |
  | --- | --- |
  | `lock-behind` | `pnpm-lock.yaml` is behind the Runner's manifests |
  | `absent` | no tag exists |
| `incomplete`, `broken` | a publication stopped, or its two tags do not
belong together |
| `mismatch` | the full index names other inputs, or lacks a platform |
| `unknown` | the registry refused, failed or could not be reached, or
the lookup threw |

- The resolver reuses `runnerLockState`, `readRunnerImageManifest`,
`runnerPublicationState` and `runnerImageTags` from
`scripts/runner-image-publish.mjs`. That file now exports
`environmentCredentials`; it has no other change.
- `server/scripts/prepare-runner-vendor.mjs`: with
`PAPERCLIP_RUNNER_VENDOR_REQUIRE_CURRENT=1`, the step reuses a current
build as before. When the build is not current, it prints the reason,
starts no rebuild, leaves the stamp in place and exits with status 1. It
also fails when `PAPERCLIP_RUNNER_VENDOR_REBUILD=1` is set at the same
time. Without the variable the step is the same as before.
- `doc/runner-image-contract.md` has a new section, "Choosing the Runner
for an image build". `doc/DEVELOPING.md` describes the new variable.
- The resolver is not an input of the Runner image, so the inputs digest
does not change.

## Verification

macOS (Node 26.12.0), in `server/`:

- `node node_modules/vitest/vitest.mjs run
scripts/prepare-runner-vendor.test.mjs
src/__tests__/runner-image-resolve.test.ts
src/__tests__/runner-image-publish.test.ts
src/__tests__/runner-image-identity.test.ts
src/__tests__/runner-image-workflow.test.ts
src/__tests__/server-package-build-script.test.ts`: 179 tests pass in 6
files. 27 of them are new resolver tests and 3 are new tests of the
vendor step, whose end-to-end test also has a new case.
- `tsc` on the new test file, with the settings of
`server/src/__tests__/tsconfig.json`: no error.

Linux (`node:24.22.3-bookworm`, GNU tar 1.34, the worktree mounted
read-only): the same files, 179 tests pass.

Real runs, on clean exports of the tree (`git archive`), with no
credential, on a `linux/arm64` Docker host:

- On `master` at `62d673cc3f`: `inputs` gave `lock=current` and
`inputs_digest=0e6662ad9c8c6451556c76bf326723645d4afaab2decc73cd5e0025f7dd8d1ca`.
`resolve` gave `mode=published`,
`image=ghcr.io/paperclipai/paperclip-runner@sha256:84f2c051aa4ed16c5cc138c08e3533fa417e92177f7d15af5f2aa3e0ba467162`
and
`slim_image=ghcr.io/paperclipai/paperclip-runner@sha256:f63ea43666ed708f1b0e5d2ade8bdc002885bf04252b9ed6cee21f7920e54f3c`.
These are the two indexes that `master` published for those inputs.
- The same tree with one line added to a Runner source file: another
digest, `mode=inline`, `reason=absent`.
- On `master` at `bc71c23e0a`, which changed the Runner's inputs, before
its image was published: digest
`88fe2929321b4706c77f11a098572ce990fec4449d505544c62ef3a84c28308a`,
`mode=inline`, `reason=absent`.
- Four deliberate faults in the resolver each make a test fail: no index
check, only `absent` is inline, the identity is built for a lockfile
that is behind, and the slim image is taken by tag.

Never run:

- A lookup with a registry credential. The package is public, so no
caller needs one now.
- Any workflow. No workflow calls the resolver in this pull request.
- `linux/amd64`.
- The full test suite, the full typecheck and `pnpm build`. `pnpm verify
--explain` reports exit 2, "broader verification required", so CI must
supply them.

## Risks

- Low for existing behaviour: nothing calls the resolver, and the vendor
step is the same unless the new variable is set.
- The resolver does not verify the attestation of the image. The
workflow that consumes the image must do that from the digest the
resolver reports.
- The resolver reports both build contexts, `runner-artifacts` and
`runner-artifacts-slim`. A build that reads only one stage gets one
context it does not use.
- A `broken` publication is `inline`, not a failure. The Runner image
workflow fails for that state.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Fable 5.2 (`claude-fable-5-2`), Anthropic, used through Claude
Code with tool use (shell, file edits, local Docker builds).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
This commit is contained in:
Nicky LeachandClaude Fable 5.2 authored and GitHub committed 2026-11-12 22:03:15 -08:00
1 parent 731b52453c
commit f79e2c5d84
7 files changed
+528 -3

No files matched your search

+7
View File
@@ -1637,6 +1637,13 @@ the Runner's first build step and the prepare step remove any existing stamp
before a rebuild starts, so a build that stops partway never leaves a stamp
that vouches for it.
Set `PAPERCLIP_RUNNER_VENDOR_REQUIRE_CURRENT=1` when the Runner under
`packages/paperclip-runner/dist` was built elsewhere and this build must not
build it again, for example because it has no Rust toolchain. The step then
reuses a current build as usual. When the build is not current it prints the
reason and fails, without starting a rebuild and without removing the stamp.
Setting both variables fails.
### Task search evaluation
The task search relevance rubric and regression corpus are documented in
+33
View File
@@ -144,6 +144,39 @@ its own, as in this repository, can take a built image or tree in its place
with `--build-context <stage>=<image or directory>`. The stage is then not
built.
### Choosing the Runner for an image build
`scripts/runner-image-resolve.mjs` decides whether a build of a checkout can
copy the published image. No build uses its answer yet. It has two commands,
so that the one that builds reads no registry credential:
```sh
node scripts/runner-image-resolve.mjs inputs --dir "$dir"
node scripts/runner-image-resolve.mjs resolve --dir "$dir"
```
`inputs` exports the Runner build context, checks the lockfile and computes
the inputs digest in the build, as the publication workflow does. Run it on
the committed files, before any step that rewrites `pnpm-lock.yaml`.
`resolve` looks up `inputs-<digest>` and reports:
- `mode`: `published` or `inline`, and `reason`;
- `image` and `slim_image`: both variants, pinned by index digest. The slim
digest is the one the full index names (see "Pairing");
- `build_contexts`: one `--build-context` value on each line, which replace
the `runner-artifacts` and `runner-artifacts-slim` stages;
- `labels`: `io.paperclip.runner.image` and
`io.paperclip.runner.inputs-digest`, for the image that copies the Runner.
`published` needs a complete publication whose full index names these inputs
and both platforms. Every other answer is `inline`, with the last four values
empty: the lockfile is behind the Runner's manifests (`lock-behind`), the
image does not exist (`absent`), a publication is `incomplete` or `broken`,
the full index does not match (`mismatch`), or the registry refused, failed
or could not be reached (`unknown`). An inline build runs the Runner stages
itself and carries neither label. The command does not wait for a publication
and does not verify the attestation of the image.
## Publication
`.github/workflows/runner-image.yml` publishes on a `master` push in this
+1 -1
View File
@@ -508,7 +508,7 @@ async function output(values) {
if (process.env.GITHUB_OUTPUT) await appendFile(process.env.GITHUB_OUTPUT, `${lines.join("\n")}\n`);
}
function environmentCredentials(env = process.env) {
export function environmentCredentials(env = process.env) {
const username = env.RUNNER_IMAGE_REGISTRY_USERNAME;
const password = env.RUNNER_IMAGE_REGISTRY_PASSWORD;
return username && password ? { username, password } : null;
+185
View File
@@ -0,0 +1,185 @@
#!/usr/bin/env node
// Which Runner an image build uses: the published Runner image of this tree's
// inputs (ghcr.io/paperclipai/paperclip-runner), or the Runner the build makes
// itself, inline. See doc/runner-image-contract.md.
//
// inputs --dir <directory> exports the Runner build context of the
// checkout, checks its lockfile and, when that is current, computes
// the inputs digest in the build. Needs Docker and pnpm, reads no
// credential. Run it on the committed files, before any lockfile
// refresh. The directory must be new or empty
// resolve --dir <directory> looks up `inputs-<digest>` for the result of
// `inputs` and reports `published`, with both variants pinned by
// index digest, or `inline`. Runs no build
//
// Every answer short of a complete publication of exactly these inputs is
// `inline`: a refused, failed or unreachable lookup included. Registry
// credentials are those of runner-image-publish.mjs.
import assert from "node:assert/strict";
import { spawnSync } from "node:child_process";
import { randomUUID } from "node:crypto";
import { appendFileSync, existsSync, mkdirSync, readdirSync, readFileSync, writeFileSync } from "node:fs";
import path from "node:path";
import { fileURLToPath, pathToFileURL } from "node:url";
import { RUNNER_IMAGE_NAME, RUNNER_IMAGE_PLATFORMS } from "./runner-image-identity.mjs";
import {
environmentCredentials, readRunnerImageManifest, runnerImageTags, runnerLockState, runnerPublicationState,
} from "./runner-image-publish.mjs";
export const RUNNER_IMAGE_LABEL = "io.paperclip.runner.image";
export const RUNNER_INPUTS_DIGEST_LABEL = "io.paperclip.runner.inputs-digest";
const INPUTS_FILE = "runner-image-inputs.json";
const HEX_64 = /^[0-9a-f]{64}$/;
const DIGEST = /^sha256:[0-9a-f]{64}$/;
/**
* @param {{
* repoRoot: string,
* directory: string,
* run?: (command: string, args: string[], options: object) => { status: number | null, error?: Error },
* lockState?: typeof runnerLockState,
* }} input
* @returns {{ lock: "current" | "behind", detail: string, inputsDigest: string | null }}
*/
export function readRunnerInputs({ repoRoot, directory, run = spawnSync, lockState = runnerLockState }) {
assert.ok(!existsSync(directory) || readdirSync(directory).length === 0, `${directory} is not empty`);
mkdirSync(directory, { recursive: true });
const build = (target, name) => {
const destination = path.join(directory, name);
// Build output goes to stderr: stdout carries only this command's result.
const result = run("docker", ["buildx", "build", "--target", target, "--output", `type=local,dest=${destination}`, repoRoot],
{ stdio: ["ignore", 2, 2] });
if (result.error) throw result.error;
assert.equal(result.status, 0, `docker buildx build --target ${target} failed (exit ${result.status})`);
return destination;
};
const lock = lockState(build("runner-artifacts-context", "context"));
let inputsDigest = null;
if (lock.state === "current") {
const identity = path.join(build("runner-artifacts-identity", "identity"), "runner-image-identity.json");
inputsDigest = JSON.parse(readFileSync(identity, "utf8")).digest;
assert.match(inputsDigest ?? "", HEX_64, "The build computed no inputs digest");
}
const inputs = { lock: lock.state, detail: lock.detail, inputsDigest };
writeFileSync(path.join(directory, INPUTS_FILE), `${JSON.stringify(inputs, null, 2)}\n`);
return inputs;
}
function indexProblem(bytes, inputsDigest) {
const index = JSON.parse(bytes.toString("utf8"));
const named = index.annotations?.[RUNNER_INPUTS_DIGEST_LABEL];
if (named !== inputsDigest) return `the full index names the inputs ${named ?? "(none)"}`;
for (const platform of RUNNER_IMAGE_PLATFORMS) {
const [os, architecture] = platform.split("/");
const found = Array.isArray(index.manifests) && index.manifests.some((entry) =>
entry?.platform?.os === os && entry.platform.architecture === architecture && DIGEST.test(entry.digest ?? ""));
if (!found) return `the full index has no ${platform} image`;
}
return null;
}
/**
* @param {{ lock: "current" | "behind", inputsDigest: string | null }} inputs
* @param {{ read?: typeof readRunnerImageManifest, credentials?: { username: string, password: string } | null }} [options]
*/
export async function resolveRunnerImage({ lock, inputsDigest }, { read = readRunnerImageManifest, credentials = null } = {}) {
assert.ok(lock === "current" || lock === "behind", "The lockfile state must be current or behind");
const inline = (reason, detail) => ({
mode: "inline", reason, detail, inputsDigest: inputsDigest ?? "", image: "", slimImage: "", buildContexts: [], labels: [],
});
if (lock === "behind") return inline("lock-behind", "pnpm-lock.yaml is behind the Runner's manifests, so no image is published for this tree");
assert.match(inputsDigest ?? "", HEX_64, "A tree with a current lockfile needs an inputs digest");
try {
const { full: fullTag } = runnerImageTags(inputsDigest);
let fullIndex = null;
const publication = await runnerPublicationState(inputsDigest, async (reference) => {
const result = await read(reference, { credentials });
if (reference === fullTag && result.state === "present") fullIndex = result.bytes;
return result;
});
if (publication.state !== "published") return inline(publication.state, publication.detail);
const problem = indexProblem(fullIndex, inputsDigest);
if (problem) return inline("mismatch", `${fullTag}: ${problem}`);
const image = `${RUNNER_IMAGE_NAME}@${publication.full}`;
const slimImage = `${RUNNER_IMAGE_NAME}@${publication.slim}`;
return {
mode: "published", reason: "published", detail: publication.detail, inputsDigest, image, slimImage,
buildContexts: [`runner-artifacts=docker-image://${image}`, `runner-artifacts-slim=docker-image://${slimImage}`],
labels: [`${RUNNER_IMAGE_LABEL}=${image}`, `${RUNNER_INPUTS_DIGEST_LABEL}=${inputsDigest}`],
};
} catch (error) {
return inline("unknown", `the lookup failed: ${error instanceof Error ? error.message : String(error)}`);
}
}
/** @param {string[]} argv */
export function parseResolveArguments(argv) {
const [command, flag, directory, ...rest] = argv;
if (!["inputs", "resolve"].includes(command) || flag !== "--dir" || !directory || directory.startsWith("--") || rest.length > 0) {
throw new Error("Usage: runner-image-resolve.mjs inputs|resolve --dir <directory>");
}
return { command, directory: path.resolve(directory) };
}
/**
* @param {string[]} argv
* @param {{
* env?: NodeJS.ProcessEnv, repoRoot?: string, read?: typeof readRunnerImageManifest, readInputs?: typeof readRunnerInputs,
* log?: (line: string) => void,
* }} [options]
* @returns {Promise<Record<string, string>>}
*/
export async function runResolveCommand(argv, {
env = process.env,
repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), ".."),
read = readRunnerImageManifest,
readInputs = readRunnerInputs,
log = console.error,
} = {}) {
const { command, directory } = parseResolveArguments(argv);
if (command === "inputs") {
const inputs = readInputs({ repoRoot, directory });
return { lock: inputs.lock, inputs_digest: inputs.inputsDigest ?? "" };
}
const inputs = JSON.parse(readFileSync(path.join(directory, INPUTS_FILE), "utf8"));
const resolved = await resolveRunnerImage(inputs, { read, credentials: environmentCredentials(env) });
log(`Runner image: ${resolved.mode} (${resolved.reason}: ${resolved.detail})`);
if (env.GITHUB_STEP_SUMMARY) {
appendFileSync(env.GITHUB_STEP_SUMMARY, [
`Runner for an image build of this tree: **${resolved.mode}** (\`${resolved.reason}\`: ${resolved.detail})`,
`- inputs digest: ${resolved.inputsDigest ? `\`${resolved.inputsDigest}\`` : "not computed"}`,
...(resolved.mode === "published" ? [`- full image: \`${resolved.image}\``, `- slim image: \`${resolved.slimImage}\``] : []),
"",
].join("\n"));
}
return {
mode: resolved.mode,
reason: resolved.reason,
inputs_digest: resolved.inputsDigest,
image: resolved.image,
slim_image: resolved.slimImage,
build_contexts: resolved.buildContexts.join("\n"),
labels: resolved.labels.join("\n"),
};
}
/** @param {Record<string, string>} values written as $GITHUB_OUTPUT takes them, a value of several lines included */
export function formatOutputs(values) {
return Object.entries(values).map(([name, value]) => {
if (!value.includes("\n")) return `${name}=${value}`;
const delimiter = `end-${randomUUID()}`;
return `${name}<<${delimiter}\n${value}\n${delimiter}`;
}).join("\n");
}
if (process.argv[1] && import.meta.url === pathToFileURL(path.resolve(process.argv[1])).href) {
try {
const text = formatOutputs(await runResolveCommand(process.argv.slice(2)));
console.log(text);
if (process.env.GITHUB_OUTPUT) appendFileSync(process.env.GITHUB_OUTPUT, `${text}\n`);
} catch (error) {
console.error(error instanceof Error ? error.message : String(error));
process.exit(1);
}
}
+21 -2
View File
@@ -19,6 +19,11 @@
// output, or an edit that landed while the build ran always rebuilds. Set
// PAPERCLIP_RUNNER_VENDOR_REBUILD=1 to rebuild regardless of the stamp.
//
// PAPERCLIP_RUNNER_VENDOR_REQUIRE_CURRENT=1 is for a build that was handed a
// Runner built elsewhere and has no Runner toolchain of its own: the step
// then never rebuilds. It reuses a current build as usual and fails, with the
// reason, when the build is not current. It leaves the stamp in place.
//
// Usage: node scripts/prepare-runner-vendor.mjs [--repo-root <dir>]
// The repo root defaults to this checkout; the flag exists for tests that
// build a throwaway workspace layout.
@@ -35,16 +40,20 @@ import {
/** The command the server build has always run here; the rebuild path is unchanged. */
export const RUNNER_VENDOR_BUILD_COMMAND = "pnpm --filter @paperclipai/paperclip-runner build";
export const FORCE_REBUILD_VARIABLE = "PAPERCLIP_RUNNER_VENDOR_REBUILD";
export const REQUIRE_CURRENT_VARIABLE = "PAPERCLIP_RUNNER_VENDOR_REQUIRE_CURRENT";
export const TIMING_SCHEMA = "paperclip.runner-vendor.timing.v1";
export function decideRunnerVendorAction({ env, assessment }) {
const requireCurrent = env[REQUIRE_CURRENT_VARIABLE] === "1";
if (env[FORCE_REBUILD_VARIABLE] === "1") {
return { action: "rebuild", code: "forced", detail: `${FORCE_REBUILD_VARIABLE}=1` };
return requireCurrent
? { action: "fail", code: "conflict", detail: `${FORCE_REBUILD_VARIABLE}=1` }
: { action: "rebuild", code: "forced", detail: `${FORCE_REBUILD_VARIABLE}=1` };
}
if (assessment.current) {
return { action: "reuse", code: assessment.code, detail: assessment.detail };
}
return { action: "rebuild", code: assessment.code, detail: assessment.detail };
return { action: requireCurrent ? "fail" : "rebuild", code: assessment.code, detail: assessment.detail };
}
/**
@@ -104,6 +113,16 @@ export function prepareRunnerVendor({
return 0;
}
if (decision.action === "fail") {
log(
`prepare:runner-vendor: ${REQUIRE_CURRENT_VARIABLE}=1 forbids the Runner rebuild this step needs (${decision.detail}). `
+ "packages/paperclip-runner/dist and its build stamp must come from a Runner build of exactly this tree's Runner inputs, "
+ "pnpm-lock.yaml included. Supply that build, or unset the variable to rebuild the Runner in this step.",
);
log(`${TIMING_SCHEMA},action:fail,reason:${decision.code},check_ms:${checkMs},build_ms:0,status:1`);
return 1;
}
log(`prepare:runner-vendor: rebuilding the Runner (${decision.detail})`);
// A build that stops partway must not leave a stamp that vouches for it. The
// Runner's own build removes the stamp in its first step and writes a fresh
@@ -16,6 +16,7 @@ import {
} from "../../packages/paperclip-runner/scripts/build-inputs.mjs";
import {
FORCE_REBUILD_VARIABLE,
REQUIRE_CURRENT_VARIABLE,
RUNNER_VENDOR_BUILD_COMMAND,
TIMING_SCHEMA,
decideRunnerVendorAction,
@@ -104,6 +105,29 @@ describe("decideRunnerVendorAction", () => {
});
});
describe("decideRunnerVendorAction when the build must be current", () => {
const current = { current: true, code: "inputs_unchanged", detail: "unchanged" };
const stale = { current: false, code: "inputs_changed", detail: "changed" };
const required = { [REQUIRE_CURRENT_VARIABLE]: "1" };
it("reuses a current build and fails on a stale one instead of rebuilding", () => {
expect(decideRunnerVendorAction({ env: required, assessment: current }).action).toBe("reuse");
expect(decideRunnerVendorAction({ env: required, assessment: stale })).toEqual({
action: "fail",
code: "inputs_changed",
detail: "changed",
});
expect(decideRunnerVendorAction({ env: { [REQUIRE_CURRENT_VARIABLE]: "0" }, assessment: stale }).action).toBe("rebuild");
});
it("fails when a rebuild is forced as well: the two cannot both hold", () => {
expect(decideRunnerVendorAction({ env: { ...required, [FORCE_REBUILD_VARIABLE]: "1" }, assessment: current })).toMatchObject({
action: "fail",
code: "conflict",
});
});
});
describe("resolvePnpmInvocation", () => {
it("runs the pnpm that is running the script, through Node for a JavaScript entry", () => {
expect(resolvePnpmInvocation({ npm_execpath: "/corepack/pnpm/9.15.4/bin/pnpm.cjs" }, "/usr/bin/node")).toEqual({
@@ -254,6 +278,41 @@ describe("prepareRunnerVendor", () => {
expect(lines[1]).toMatch(/,action:rebuild,reason:forced,/);
});
it("with PAPERCLIP_RUNNER_VENDOR_REQUIRE_CURRENT=1, reuses a current build and never starts a rebuild", () => {
const { repoRoot, packageRoot } = makeRepo();
recordCurrentBuild({ repoRoot, packageRoot });
const stampPath = buildInputsStampPath(packageRoot);
const env = { [REQUIRE_CURRENT_VARIABLE]: "1" };
let runs = 0;
const prepare = (lines) =>
prepareRunnerVendor({
repoRoot,
env,
run: () => {
runs += 1;
return 0;
},
log: (line) => lines.push(line),
});
const reused = [];
expect(prepare(reused)).toBe(0);
expect(reused[1]).toMatch(/,action:reuse,reason:inputs_unchanged,/);
writeFileSync(path.join(packageRoot, "src", "index.ts"), "export const one = 2;\n");
const failed = [];
expect(prepare(failed)).toBe(1);
expect(runs).toBe(0);
expect(failed[0]).toMatch(
new RegExp(`^prepare:runner-vendor: ${REQUIRE_CURRENT_VARIABLE}=1 forbids the Runner rebuild this step needs \\(inputs changed`),
);
expect(failed[1]).toMatch(
new RegExp(`^${TIMING_SCHEMA},action:fail,reason:inputs_changed,check_ms:\\d+,build_ms:0,status:1$`),
);
// The stamp still describes the build it was written for; only a rebuild removes it.
expect(readBuildInputsStamp(stampPath)).not.toBeNull();
});
it.skipIf(process.platform === "win32")(
"end to end: the first server build rebuilds the Runner, the second reuses it, a change or a lost output rebuilds again",
async () => {
@@ -278,6 +337,7 @@ describe("prepareRunnerVendor", () => {
chmodSync(pnpm, 0o755);
const env = { ...process.env, PATH: `${binDir}${path.delimiter}${process.env.PATH ?? ""}` };
delete env[FORCE_REBUILD_VARIABLE];
delete env[REQUIRE_CURRENT_VARIABLE];
// The suite itself may run under pnpm; the script must then take the
// PATH fallback so the stand-in, not the real pnpm, receives the build.
delete env.npm_execpath;
@@ -307,6 +367,15 @@ describe("prepareRunnerVendor", () => {
const fifth = await run();
expect(calls()).toHaveLength(3);
expect(fifth.stdout).toMatch(/action:reuse,reason:inputs_unchanged,/);
// Required to be current: a stale build fails the step and pnpm is not called.
writeFileSync(path.join(packageRoot, "src", "index.ts"), "export const one = 3;\n");
const required = { ...env, [REQUIRE_CURRENT_VARIABLE]: "1" };
await expect(execFileAsync(process.execPath, [prepareScript, "--repo-root", repoRoot], { env: required })).rejects.toMatchObject({
code: 1,
stdout: expect.stringMatching(/action:fail,reason:inputs_changed,/),
});
expect(calls()).toHaveLength(3);
},
);
});
@@ -0,0 +1,212 @@
import { createHash } from "node:crypto";
import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import path from "node:path";
import { afterAll, describe, expect, it } from "vitest";
import { readRunnerImageManifest, runnerImageTags, SLIM_DIGEST_ANNOTATION } from "../../../scripts/runner-image-publish.mjs";
import {
formatOutputs,
parseResolveArguments,
readRunnerInputs,
resolveRunnerImage,
runResolveCommand,
} from "../../../scripts/runner-image-resolve.mjs";
const INPUTS = "1".repeat(64);
const TAGS = runnerImageTags(INPUTS);
const IMAGE = "ghcr.io/paperclipai/paperclip-runner";
const sha256 = (bytes: Buffer | string) => `sha256:${createHash("sha256").update(bytes).digest("hex")}`;
const scratch = mkdtempSync(path.join(tmpdir(), "runner-image-resolve-"));
afterAll(() => rmSync(scratch, { recursive: true, force: true }));
let sequence = 0;
const directory = () => path.join(scratch, String((sequence += 1)));
type Manifest = Awaited<ReturnType<typeof readRunnerImageManifest>>;
const present = (value: unknown) => {
const bytes = Buffer.from(typeof value === "string" ? value : JSON.stringify(value));
return { state: "present" as const, bytes, digest: sha256(bytes) };
};
const platforms = (...architectures: string[]) => architectures.map((architecture) => (
{ digest: sha256(`image ${architecture}`), size: 500, platform: { os: "linux", architecture } }));
const slim = present({ schemaVersion: 2, manifests: platforms("amd64", "arm64") });
const fullIndex = (annotations: Record<string, string>, manifests = platforms("amd64", "arm64")) =>
present({ schemaVersion: 2, manifests, annotations });
const full = fullIndex({ "io.paperclip.runner.inputs-digest": INPUTS, [SLIM_DIGEST_ANNOTATION]: slim.digest });
const registry = (tags: Record<string, Manifest>) => async (reference: string): Promise<Manifest> => tags[reference] ?? { state: "absent" };
const current = { lock: "current" as const, inputsDigest: INPUTS };
const NOTHING = { image: "", slimImage: "", buildContexts: [], labels: [] };
describe("the Runner an image build uses", () => {
it("is the published image, pinned by digest, with the slim image its full index names", async () => {
const resolved = await resolveRunnerImage(current, { read: registry({ [TAGS.full]: full, [TAGS.slim]: slim }) });
expect(resolved).toMatchObject({
mode: "published",
inputsDigest: INPUTS,
image: `${IMAGE}@${full.digest}`,
slimImage: `${IMAGE}@${slim.digest}`,
buildContexts: [
`runner-artifacts=docker-image://${IMAGE}@${full.digest}`,
`runner-artifacts-slim=docker-image://${IMAGE}@${slim.digest}`,
],
labels: [`io.paperclip.runner.image=${IMAGE}@${full.digest}`, `io.paperclip.runner.inputs-digest=${INPUTS}`],
});
expect(Object.keys(resolved).sort()).toEqual(["buildContexts", "detail", "image", "inputsDigest", "labels", "mode", "reason", "slimImage"]);
});
it.each<[string, string, Record<string, Manifest>]>([
["neither tag exists", "absent", {}],
["a publication stopped before the full tag", "incomplete", { [TAGS.slim]: slim }],
["the slim tag is missing", "broken", { [TAGS.full]: full }],
["the slim tag names an image the full index does not", "broken", { [TAGS.full]: full, [TAGS.slim]: present({ other: true }) }],
["the full tag is not an index", "broken", { [TAGS.full]: present("not JSON"), [TAGS.slim]: slim }],
["the registry gives no clear answer", "unknown", { [TAGS.full]: { state: "unknown", detail: "manifest request: HTTP 503" } }],
["the full index names other inputs", "mismatch", {
[TAGS.full]: fullIndex({ "io.paperclip.runner.inputs-digest": "2".repeat(64), [SLIM_DIGEST_ANNOTATION]: slim.digest }), [TAGS.slim]: slim,
}],
["the full index names no inputs", "mismatch", { [TAGS.full]: fullIndex({ [SLIM_DIGEST_ANNOTATION]: slim.digest }), [TAGS.slim]: slim }],
["the full index lacks a platform", "mismatch", {
[TAGS.full]: fullIndex({ "io.paperclip.runner.inputs-digest": INPUTS, [SLIM_DIGEST_ANNOTATION]: slim.digest }, platforms("amd64")), [TAGS.slim]: slim,
}],
])("is inline, with nothing to pass to the build, when %s", async (_label, reason, tags) => {
expect(await resolveRunnerImage(current, { read: registry(tags) })).toMatchObject({ mode: "inline", reason, ...NOTHING });
});
it.each<[string, (url: string) => Promise<Response>]>([
["cannot be reached", async () => { throw new TypeError("fetch failed"); }],
["refuses the token", async () => new Response("", { status: 403 })],
["denies the manifest of a private package", async (url) =>
(url.includes("/token?") ? new Response(JSON.stringify({ token: "t" })) : new Response("", { status: 401 }))],
["keeps failing", async (url) =>
(url.includes("/token?") ? new Response(JSON.stringify({ token: "t" })) : new Response("", { status: 503 }))],
])("is inline when the registry %s", async (_label, fetchImpl) => {
const read = (reference: string) => readRunnerImageManifest(reference, { fetchImpl, sleep: async () => {} });
expect(await resolveRunnerImage(current, { read })).toMatchObject({ mode: "inline", reason: "unknown", ...NOTHING });
});
it("is inline when the lookup throws, and does not look when the lockfile is behind", async () => {
const failing = async (): Promise<Manifest> => { throw new Error("Registry digest does not match the manifest bytes"); };
expect(await resolveRunnerImage(current, { read: failing }))
.toMatchObject({ mode: "inline", reason: "unknown", detail: expect.stringContaining("does not match the manifest bytes"), ...NOTHING });
let reads = 0;
const counting = async (): Promise<Manifest> => { reads += 1; return full; };
expect(await resolveRunnerImage({ lock: "behind", inputsDigest: null }, { read: counting }))
.toMatchObject({ mode: "inline", reason: "lock-behind", inputsDigest: "", ...NOTHING });
expect(reads).toBe(0);
});
it("refuses inputs it cannot name instead of resolving them", async () => {
const read = registry({ [TAGS.full]: full, [TAGS.slim]: slim });
await expect(resolveRunnerImage({ lock: "current", inputsDigest: null }, { read })).rejects.toThrow(/needs an inputs digest/);
await expect(resolveRunnerImage({ lock: "stale" as "current", inputsDigest: INPUTS }, { read })).rejects.toThrow(/current or behind/);
});
});
describe("the inputs of a tree", () => {
type Call = { command: string; args: string[] };
function docker(calls: Call[], { digest = INPUTS, fail = "" } = {}) {
return (command: string, args: string[]) => {
calls.push({ command, args });
const target = args[args.indexOf("--target") + 1];
if (target === fail) return { status: 1 };
const destination = args[args.indexOf("--output") + 1].replace("type=local,dest=", "");
mkdirSync(destination, { recursive: true });
if (target === "runner-artifacts-identity") writeFileSync(path.join(destination, "runner-image-identity.json"), JSON.stringify({ digest }));
return { status: 0 };
};
}
const lock = (state: "current" | "behind") => () => ({ state, detail: `lockfile ${state}` });
it("exports the build context, checks its lockfile, then computes the digest in the build", () => {
const calls: Call[] = [];
const out = directory();
const checked: string[] = [];
const inputs = readRunnerInputs({
repoRoot: "/repo", directory: out, run: docker(calls),
lockState: (context: string) => { checked.push(context); return lock("current")(); },
});
expect(inputs).toEqual({ lock: "current", detail: "lockfile current", inputsDigest: INPUTS });
expect(calls.map((call) => [call.command, ...call.args])).toEqual([
["docker", "buildx", "build", "--target", "runner-artifacts-context", "--output", `type=local,dest=${path.join(out, "context")}`, "/repo"],
["docker", "buildx", "build", "--target", "runner-artifacts-identity", "--output", `type=local,dest=${path.join(out, "identity")}`, "/repo"],
]);
expect(checked).toEqual([path.join(out, "context")]);
expect(JSON.parse(readFileSync(path.join(out, "runner-image-inputs.json"), "utf8"))).toEqual(inputs);
});
it("computes no digest for a lockfile that is behind: no image is published for that tree", () => {
const calls: Call[] = [];
const inputs = readRunnerInputs({ repoRoot: "/repo", directory: directory(), run: docker(calls), lockState: lock("behind") });
expect(inputs).toEqual({ lock: "behind", detail: "lockfile behind", inputsDigest: null });
expect(calls).toHaveLength(1);
});
it("fails, and leaves no result, when a build fails or reports no digest", () => {
for (const options of [{ fail: "runner-artifacts-context" }, { fail: "runner-artifacts-identity" }, { digest: "latest" }]) {
const out = directory();
expect(() => readRunnerInputs({ repoRoot: "/repo", directory: out, run: docker([], options), lockState: lock("current") }))
.toThrow(/failed \(exit 1\)|no inputs digest/);
expect(existsSync(path.join(out, "runner-image-inputs.json"))).toBe(false);
}
});
it("refuses a directory that could hold the result of another tree", () => {
const out = directory();
mkdirSync(out);
writeFileSync(path.join(out, "runner-image-inputs.json"), JSON.stringify(current));
const calls: Call[] = [];
expect(() => readRunnerInputs({ repoRoot: "/repo", directory: out, run: docker(calls), lockState: lock("current") })).toThrow(/is not empty/);
expect(calls).toEqual([]);
});
});
describe("command line", () => {
const quiet = { log: () => {} };
it("reports the inputs, then resolves them with the registry credentials of the environment", async () => {
const out = directory();
const stub = ({ directory: target }: { directory: string }) => {
mkdirSync(target, { recursive: true });
writeFileSync(path.join(target, "runner-image-inputs.json"), JSON.stringify(current));
return { lock: "current" as const, detail: "", inputsDigest: INPUTS };
};
await expect(runResolveCommand(["inputs", "--dir", out], { ...quiet, env: {}, readInputs: stub })).resolves.toEqual({ lock: "current", inputs_digest: INPUTS });
const summary = path.join(out, "summary.md");
const seen: unknown[] = [];
const read = async (reference: string, options?: { credentials?: unknown }) => {
seen.push(options?.credentials);
return registry({ [TAGS.full]: full, [TAGS.slim]: slim })(reference);
};
const env = { RUNNER_IMAGE_REGISTRY_USERNAME: "u", RUNNER_IMAGE_REGISTRY_PASSWORD: "p", GITHUB_STEP_SUMMARY: summary };
const values = await runResolveCommand(["resolve", "--dir", out], { ...quiet, env, read });
expect(values).toEqual({
mode: "published",
reason: "published",
inputs_digest: INPUTS,
image: `${IMAGE}@${full.digest}`,
slim_image: `${IMAGE}@${slim.digest}`,
build_contexts: `runner-artifacts=docker-image://${IMAGE}@${full.digest}\nrunner-artifacts-slim=docker-image://${IMAGE}@${slim.digest}`,
labels: `io.paperclip.runner.image=${IMAGE}@${full.digest}\nio.paperclip.runner.inputs-digest=${INPUTS}`,
});
expect(seen).toEqual([{ username: "u", password: "p" }, { username: "u", password: "p" }]);
expect(readFileSync(summary, "utf8")).toContain(`**published**`);
const inline = await runResolveCommand(["resolve", "--dir", out], { ...quiet, env: {}, read: registry({}) });
expect(inline).toEqual({ mode: "inline", reason: "absent", inputs_digest: INPUTS, image: "", slim_image: "", build_contexts: "", labels: "" });
});
it("writes a value of several lines in the form a workflow output takes", () => {
const text = formatOutputs({ mode: "published", labels: "a=1\nb=2", image: "" });
expect(text).toMatch(/^mode=published\nlabels<<(end-[0-9a-f-]{36})\na=1\nb=2\n\1\nimage=$/);
});
it.each<[string, string[]]>([
["no command", []],
["an unknown command", ["wait", "--dir", "d"]],
["a missing directory", ["resolve", "--dir"]],
["another flag", ["resolve", "--inputs-digest", INPUTS]],
["an extra argument", ["inputs", "--dir", "d", "--wait"]],
])("rejects %s", (_label, argv) => {
expect(() => parseResolveArguments(argv)).toThrow(/Usage/);
});
});