mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-17 13:01:05 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Runner executes agent work in a sandbox. It needs a provider pack of agent tools, and the server uploads that pack when the sandbox does not have it > - A retained session keeps the pack it started with. When that pack is not the server's own, the server downloads the pack's transport archives from a registry (#19779) > - Today the server looks for them in the application image tagged `sha-<commit that built the pack>` > - `master` now publishes the Runner one time for each set of inputs, as `ghcr.io/paperclipai/paperclip-runner`. A pack from that image names the commit that published the image, and few commits get an application image, so that tag often does not exist > - This pull request reads such a pack from its Runner image, and keeps the old source for every other pack and for a Runner image that cannot be read > - The benefit is that a retained session can still get its pack when the pack came out of the published Runner image ## Linked Issues or Issue Description Refs #19779 (the retained pack source), #21228 (the pack field `runnerInputsDigest`), and #21204, #21229, #21230 (the Runner image). No issue exists for this change. **Problem or motivation** `retained-provider-pack-oci.ts` reads `ghcr.io/paperclipai/paperclip:sha-<runnerSourceRevision>`. It needs that image to exist, to carry the pack, and to have one layer with a known build-history line. For a pack from a published Runner image, `runnerSourceRevision` is the commit that published the image. `docker.yml` writes `sha-<commit>` only when its `master` run for that commit completes, and a newer push can replace a waiting run. The restart-chaos harness has the same assumption in another place: it requires the pack of a sandbox image to be built by the commit in the image's `org.opencontainers.image.revision` label. **Proposed solution** A pack that names its Runner image in `payload.runnerInputsDigest` is read from `ghcr.io/paperclipai/paperclip-runner:inputs-<digest>`. A sandbox image can name the commit that built its Runner in a label of its own. No application image carries a pack with that field yet, and no image carries that label, so behaviour today does not change. **Alternatives considered** - Keep the application image as the only source and require an application image for every Runner publication. Publication would then depend on `docker.yml`, which serves one ref at a time. - Take a fixed number of layers ("the image has four, take the last two"). A server in the field cannot be changed when the image gets another layer. The server now uses the rule that holds for any number of layers: the full image is the slim image plus transport layers. - Fall back to the application image for every failure of the Runner image. A Runner image that exists and does not match the pack is a fault, and a second source would hide it. **Roadmap alignment** This is build and runtime infrastructure. It adds no product feature and does not overlap an item in `ROADMAP.md`. This is part 3 of 4 of a stack ([stack #21309](https://github.com/paperclipai/paperclip/stack/21309)). Merge the parts in order: 1. #21306: the resolver and the no-rebuild mode 2. #21307: the report-only job in `docker.yml` and the pull-request check of the application image's Runner path 3. #21308: reading a retained provider pack from its Runner image (this pull request) 4. #21310: a registry build cache for the Runner image build This pull request uses no code from parts 1 and 2. It is stacked on them because all four parts edit `doc/runner-image-contract.md`. ## What Changed - `server/src/services/native-runtime/retained-provider-pack-oci.ts`, for a pack whose `payload.runnerInputsDigest` is 64 hexadecimal characters: - The server reads the full index `inputs-<digest>` of the Runner image. The index must carry `io.paperclip.runner.inputs-digest` equal to the pack's value, `org.opencontainers.image.revision` equal to the pack's `runnerSourceRevision`, and a valid digest in `io.paperclip.runner.slim-digest`. - It reads the slim index by that digest, never by the slim tag, and checks the bytes it gets against the digest. - For the pack's platform, the layer digests of the slim image must be the first layer digests of the full image, in the same order. The layers after them are the transport layers. There must be at least one and at most eight, and their total size has the existing limit. - The server downloads only those layers. It checks each against its descriptor and refuses every entry that is not a transport archive, the receipt or the `provider-pack.archives` directory. A file may not be in two layers. - When the Runner image cannot be read, the server reads the application image as before. That covers a registry error status for the token, for either index or for either manifest: for example 404 for a tag or a slim index that does not exist. Nothing is written before that point. - A Runner image that can be read and does not match fails with `runner_remote_provider_source_invalid`. The server does not try the application image then. - The same file, for every other pack: the old path, unchanged in what it checks. Its code moved into functions that both paths use (`request`, `pullToken`, `platformImage`, `receiveTransportLayer`). - The downloaded layer file is removed as soon as it is unpacked. Before, the importer removed it after its checks, so `retained-provider-pack-source.ts` loses that one line. - The cache record gets an optional `runnerImage` field with the inputs digest and the digests of the transport layers. `transportLayerDigest` is the last of them and `transportLayerBytes` their total. An older server ignores the new field. - `tests/restart-chaos/daytona-image.ts`: when the image has a non-empty label `io.paperclip.runner.source-revision`, the pack must be built by that commit. Otherwise the pack must be built by the commit in `org.opencontainers.image.revision`, as before. The receipt field `providerPackRunnerSourceRevision` records the pack's commit. - `doc/runner-image-contract.md` has a new section, "Reading the transport archives alone". `tests/restart-chaos/README.md` describes the label. One choice needs a reviewer's eye. When the full index can be read and the slim index cannot (a registry error status), the server falls back to the application image; it does not fail. Without the slim manifest the server cannot tell which layers are transport layers, so it has not accepted any byte of the Runner image, and the old source has its own complete checks. A slim index that is read and has other bytes than its digest still fails. ## Verification Linux (`node:24.22.3-bookworm`, GNU tar 1.34, zstd 1.5.4, the worktree mounted read-only): - In `server/`: `retained-provider-pack-source.test.ts` (34 tests, 15 of them new), `provider-pack-archive.test.ts`, `remote-provider-pack-verification.test.ts`, `remote-provider-pack.test.ts`: 127 tests pass in 4 files. - `node node_modules/vitest/vitest.mjs run --config tests/restart-chaos/vitest.config.ts tests/restart-chaos/daytona-image.test.ts`: 25 tests pass, 5 of them new. macOS (Node 26.12.0): - `daytona-image.test.ts`: 25 tests pass. - `retained-provider-pack-source.test.ts`: the 15 new tests pass. Six older tests fail, with and without this change. They run the real uploader, which needs GNU tar, and this host has bsdtar. - `tsc` on the two changed server files and the test, and on the two restart-chaos files, with the settings of their test projects: no error. The new cases: - Read from the Runner image with 6 registry requests (two indexes, two manifests, two layers), none to the application image and none for the slim tag. - Take every layer after the slim image's: one transport layer, and three. - Fall back when the tag does not exist, when the token is refused, and when the slim index cannot be reached. - Refuse, with no request to the application image: an index for other inputs, an index for another revision, no slim annotation, a slim annotation that is a tag, a slim index with other bytes than its digest, a slim image that is not a prefix of the full image, a slim image equal to the full image, nine transport layers, and a transport layer that holds a pack file. - Four deliberate faults in the reader each make a test fail: the slim index bytes are not checked, no prefix check, the slim annotation is not validated, and no bounds on the number of transport layers. A real run, in the same Linux container (`linux/arm64`), with no credential: - The script took `provider-pack/provider-pack.json` of `linux/arm64` from the published image `ghcr.io/paperclipai/paperclip-runner:inputs-0e6662ad9c8c6451556c76bf326723645d4afaab2decc73cd5e0025f7dd8d1ca`. Pack digest `sha256:604c29cb32a181843cc4d425871f88c0e65c8185b0a158c801fda4eb1bfce52f`, `runnerSourceRevision` `25e9a9c2e9277f3be8649be8c809cc81bde6f0c5`. - `resolveProviderPackUploadSource`, with another pack as the server's own, returned `{ archiveOnly: true }` after 23 seconds. That function calls `receiveRetainedProviderPack` and then the importer's checks: the manifest inside the overlay archive is the selected manifest, every archive matches the receipt, and the content of every file archive matches the digest of its file. - The server code made 9 requests: one anonymous token request, the full index, the `arm64` manifest, the slim index `sha256:f63ea436…4f3c` by digest, its `arm64` manifest, and two layers, each with one redirect. No request went to the application image. - The cache record: ```json { "schema": "paperclip.retained-provider-pack-source.v1", "sourceRevision": "25e9a9c2e9277f3be8649be8c809cc81bde6f0c5", "providerPackDigest": "sha256:604c29cb32a181843cc4d425871f88c0e65c8185b0a158c801fda4eb1bfce52f", "imageIndexDigest": "sha256:84f2c051aa4ed16c5cc138c08e3533fa417e92177f7d15af5f2aa3e0ba467162", "imageManifestDigest": "sha256:73dbe864fbab6208616f0aa6f1610a7ece4ef22f68a73d30901cbaea60139589", "imageConfigDigest": "sha256:740e48463d5caf0bfead4b923cdbd8b220acb93d058419c3db9d00c7f1831d65", "transportLayerDigest": "sha256:35433b9f788641666b470f41c37fef9fc9b872262b7edd5bd79f0e288f75bcdf", "transportLayerBytes": 426602696, "runnerImage": { "inputsDigest": "0e6662ad9c8c6451556c76bf326723645d4afaab2decc73cd5e0025f7dd8d1ca", "transportLayerDigests": [ "sha256:6a340088ca2cfa85f10f0385d9c8f5cdccb22c3a2bbb2062620b1d90616ba5b5", "sha256:35433b9f788641666b470f41c37fef9fc9b872262b7edd5bd79f0e288f75bcdf" ] }, "receiptSha256": "2afdd26645845369d9e851b1e6e2884eb6c3884ea7c4866a1eeb2efb9fb230d7", "verifiedTarPaths": true } ``` - `uploadVerifiedProviderPack` then installed the pack from that source through a local stand-in for the sandbox, with no further registry request, in 4 seconds. The installed tree is 1,249,060,841 bytes. Its `provider-pack.json` is the selected manifest, all 7 artifacts of the manifest have the digest the manifest names, and the pack's own Node prints `v24.22.3`. Never run: - The same real run for `linux/amd64`, whose transport layers are about 1 GB. - A session in a real sandbox. The real run used a local stand-in for the sandbox. - The fallback against the real registry. Fixtures cover it. - The full test suite and the full typecheck. CI must supply them. ## Risks - The old path was moved into shared functions. Its checks are the same, and its 19 existing tests pass on Linux, but a review must read that code as changed. - The layer file of the old path is now removed before the importer's checks, not after them. The importer does not read it. - The reader depends on two rules of the image contract: the full index names its slim index, and the full image starts with the slim image's layers. A Runner image that breaks either rule is refused, with no fallback. - The fallback covers a registry error status only. A network error ends the import, as it does today. - The label `io.paperclip.runner.source-revision` is new and no image carries it yet. The change to the sandbox image that sets it must use this name. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Fable 5.2 (`claude-fable-5-2`), Anthropic, used through Claude Code with tool use (shell, file edits, local Docker builds). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
267 lines
14 KiB
Markdown
267 lines
14 KiB
Markdown
# Runner image contract
|
|
|
|
The Runner image is the Runner built alone, as a plain file tree with no
|
|
entrypoint and no operating system. An image build copies paths out of it
|
|
instead of compiling the Runner again. Nothing in this repository consumes it
|
|
yet. Its name is `ghcr.io/paperclipai/paperclip-runner`, and one build makes
|
|
two variants:
|
|
|
|
- full, the `runner-artifacts` target of the `Dockerfile`: every path below;
|
|
- slim, the `runner-artifacts-slim` target: the same without the two paths
|
|
marked "full only", the provider pack's transport archives.
|
|
|
|
## Layout
|
|
|
|
Every path a consumer may copy (`RUNNER_IMAGE_PATHS` in
|
|
`scripts/runner-image-layout.mjs`). Every file is readable by every user.
|
|
|
|
| Path | Content |
|
|
| --- | --- |
|
|
| `/runner-image.json` | Identity and description of this image. |
|
|
| `/paperclip-runnerd` | The daemon. A stripped release binary, mode `0755`. |
|
|
| `/provider-pack/` | The remote provider pack, with `provider-pack.json`. |
|
|
| `/provider-pack.archive.json` | Full only. Receipt of the pack's transport archives. |
|
|
| `/provider-pack.archives/` | Full only. The transport archives the receipt lists. |
|
|
| `/runner-package/package.json` | The Runner package manifest. |
|
|
| `/runner-package/dist/` | The Runner's complete build output. `dist/bin/paperclip-runnerd` is the same file as `/paperclip-runnerd`. |
|
|
| `/runner-package/node_modules/.cache/paperclip-runner-build-inputs.json` | The build stamp the Runner's build recorded for that output. |
|
|
| `/runner-package/scripts/provision-grok.mjs` | Provisions the native Grok executable. Reads `platforms.json` by relative path. |
|
|
| `/runner-package/src/providers/grok/platforms.json` | The Grok release pins. |
|
|
| `/runner-package/src/live/runnerd-codex-transport.ts` | The transport source, as a contract file. It defines `RUNNER_CLIENT_VERSION`. |
|
|
| `/bundles/runnerd-codex-transport.mjs` | The pack's `dist/live/runnerd-codex-transport.js` as one module. It imports without `node_modules`. |
|
|
| `/pnpm-lock.yaml` | The lockfile the image was installed from. |
|
|
|
|
Paths under `/runner-package/` keep their place inside
|
|
`packages/paperclip-runner`. `runner-image.json` describes one platform, and
|
|
is the same file in both variants:
|
|
|
|
- `inputsDigest` (the identity below) and `buildInputsDigest` (the digest in
|
|
the Runner's build stamp);
|
|
- `sourceRevision`: the commit that built the image, also in the pack as
|
|
`payload.runnerSourceRevision`. An image is published once for its
|
|
inputs, so later commits with the same inputs keep this revision;
|
|
- `publication`: `true` only for a build with the build argument
|
|
`PAPERCLIP_RUNNER_IMAGE_PUBLICATION=1`. Only then does the pack carry the
|
|
identity, as `payload.runnerInputsDigest`. Another image build that runs
|
|
the stage inline builds the same Runner but is not this image;
|
|
- `platform` (`linux`, `amd64` or `arm64`) and `libc`: the glibc the daemon
|
|
was compiled on (Debian bookworm, the oldest a consumer runs) and the
|
|
newest glibc version it needs;
|
|
- `providerPack`: the pack's digest and target;
|
|
- `paths` and `sha256`: the table above, and the digest of every single file;
|
|
- `fullImageOnly`: the names in `paths` that only the full image has.
|
|
|
|
## Layers
|
|
|
|
1. The pack's dependency files of 1 MiB or more: provider binaries and Node.
|
|
2. Everything else, with `runner-image.json`.
|
|
3. Full only: the transport archive of each file in layer 1.
|
|
4. Full only: the other transport archives, the overlay archive, the receipt.
|
|
|
|
The slim image is layers 1 and 2. The full image is those same two layer
|
|
blobs, then layers 3 and 4, which hold transport content and nothing else.
|
|
Layers 1 and 3 have the same bytes in every build with the same dependency
|
|
files, so a new image shares them with the one before it. A large file under
|
|
the pack's `dist` is the Runner's own output: it and its archive are in
|
|
layers 2 and 4. An archive is named by its file's place in the receipt, which
|
|
lists the dependency files first, so their names do not depend on `dist`.
|
|
|
|
## Identity
|
|
|
|
Both variants are named by one digest: the variant is not part of the
|
|
identity. The full image has the tag `inputs-<digest>` and the slim image
|
|
`inputs-<digest>-slim`. `scripts/runner-image-identity.mjs` computes the
|
|
digest from:
|
|
|
|
- every file under the paths the build copies from the build context
|
|
(`RUNNER_IMAGE_CONTEXT_INPUTS`, which include `pnpm-lock.yaml`): path,
|
|
executable bit and content, and each symbolic link by its target;
|
|
- the whole `Dockerfile`: the recipe, the base image pinned by digest and the
|
|
toolchain pins. Any edit to it makes a new image;
|
|
- the two platforms, `linux/amd64` and `linux/arm64`.
|
|
|
|
The source revision is not part of it, and neither is the text of
|
|
`.dockerignore`: a rule changes the digest only when it changes which of
|
|
those files the build receives. The build computes the digest for itself:
|
|
|
|
```sh
|
|
docker buildx build --target runner-artifacts-identity --output type=local,dest=/tmp/identity .
|
|
jq -r .digest /tmp/identity/runner-image-identity.json
|
|
```
|
|
|
|
The install is frozen. It fails when the lockfile is behind the manifests the
|
|
build reads (the root one, `packages/paperclip-runner` and
|
|
`packages/paperclip-eval-kernel`). A contributor cannot edit the lockfile, so
|
|
pull-request runs resolve it first. For a local build, run
|
|
`pnpm install --resolution-only --ignore-scripts --no-frozen-lockfile`.
|
|
|
|
## Checks
|
|
|
|
CI builds each variant as an OCI archive and nothing else.
|
|
`node scripts/runner-image-oci.mjs unpack <oci archive> --out <tree>` writes
|
|
the tree of an image from its layers, so the tree CI checks is the bytes of
|
|
the image. It refuses a whiteout, an absolute or `..` path, a path written
|
|
twice and an entry behind a link.
|
|
|
|
`node scripts/runner-image-layout.mjs check <tree> --variant slim|full` runs
|
|
in the build and again on each unpacked tree. The variant has no default: a
|
|
slim tree with the transport paths fails, and so does a full tree without
|
|
them. The check fails unless:
|
|
|
|
- every documented path exists, matches its digest in `runner-image.json`,
|
|
and resolves inside the tree;
|
|
- the build stamp names the inputs the identity names;
|
|
- every artifact in `provider-pack.json` has its digest, the `dist` tree and
|
|
each provider asset tree match their tree digests, and the pack names this
|
|
image's revision and platform;
|
|
- the pack names the inputs digest in a publication build and in no other.
|
|
`--expect-inputs-digest` fails on any other build;
|
|
- in a full image, every transport archive matches its receipt, and the
|
|
archives extract to the pack. The check extracts them as a sandbox does (GNU tar
|
|
with the options in `provider-pack-layers.ts`) and compares paths, content,
|
|
link targets and all twelve permission bits, after the `a+rX` grant the
|
|
build makes once the archives are written.
|
|
|
|
`node scripts/runner-image-oci.mjs check-layers --slim <oci archive> --full <oci archive>`
|
|
checks "Layers" on the two archives, and that no file is in two layers.
|
|
|
|
The build also runs `paperclip-runnerd --build-metadata` on every base image
|
|
in `docker/runner-image/Dockerfile.smoke`. Add a base there before a consumer
|
|
uses it.
|
|
|
|
## Consuming the image
|
|
|
|
Read the image only with `COPY --from`, and pin it by digest:
|
|
|
|
```dockerfile
|
|
FROM ghcr.io/paperclipai/paperclip-runner@sha256:<digest> AS runner-artifacts
|
|
FROM debian:trixie-slim
|
|
COPY --from=runner-artifacts /paperclip-runnerd /usr/local/bin/paperclip-runnerd
|
|
```
|
|
|
|
A Dockerfile with a `runner-artifacts` or `runner-artifacts-slim` stage of
|
|
its own, as in this repository, can take a built image or tree in its place
|
|
with `--build-context <stage>=<image or directory>`. The stage is then not
|
|
built.
|
|
|
|
### Choosing the Runner for an image build
|
|
|
|
`scripts/runner-image-resolve.mjs` decides whether a build of a checkout can
|
|
copy the published image. No build uses its answer yet: on a `master` push,
|
|
the `runner_image_report` job of `.github/workflows/docker.yml` runs it and
|
|
writes the answer to the job summary, and nothing reads that job. The job
|
|
runs on the committed files, holds no credential (the Runner package is
|
|
public, so the lookup is anonymous) and cannot fail the run. The script has
|
|
two commands, so that the one that builds reads no registry credential:
|
|
|
|
```sh
|
|
node scripts/runner-image-resolve.mjs inputs --dir "$dir"
|
|
node scripts/runner-image-resolve.mjs resolve --dir "$dir"
|
|
```
|
|
|
|
`inputs` exports the Runner build context, checks the lockfile and computes
|
|
the inputs digest in the build, as the publication workflow does. Run it on
|
|
the committed files, before any step that rewrites `pnpm-lock.yaml`.
|
|
`resolve` looks up `inputs-<digest>` and reports:
|
|
|
|
- `mode`: `published` or `inline`, and `reason`;
|
|
- `image` and `slim_image`: both variants, pinned by index digest. The slim
|
|
digest is the one the full index names (see "Pairing");
|
|
- `build_contexts`: one `--build-context` value on each line, which replace
|
|
the `runner-artifacts` and `runner-artifacts-slim` stages;
|
|
- `labels`: `io.paperclip.runner.image` and
|
|
`io.paperclip.runner.inputs-digest`, for the image that copies the Runner.
|
|
|
|
`published` needs a complete publication whose full index names these inputs
|
|
and both platforms. Every other answer is `inline`, with the last four values
|
|
empty: the lockfile is behind the Runner's manifests (`lock-behind`), the
|
|
image does not exist (`absent`), a publication is `incomplete` or `broken`,
|
|
the full index does not match (`mismatch`), or the registry refused, failed
|
|
or could not be reached (`unknown`). An inline build runs the Runner stages
|
|
itself and carries neither label. The command does not wait for a publication
|
|
and does not verify the attestation of the image.
|
|
|
|
## Publication
|
|
|
|
`.github/workflows/runner-image.yml` publishes on a `master` push in this
|
|
repository and nowhere else. Other runs build and check, and publish nothing.
|
|
|
|
- **Lockfile.** A `master` push builds from the committed `pnpm-lock.yaml`,
|
|
so the digest is a function of the commit. When that file is behind the
|
|
Runner's manifests, the run writes a notice, publishes nothing and stays
|
|
green. The commit that merges the lockfile refresh publishes the image.
|
|
- **Once.** The build runs only when `inputs-<digest>` does not exist.
|
|
- **Verified bytes.** The build legs cannot write to the registry. The
|
|
publish job pushes the images they checked by digest and joins each
|
|
variant into a multi-platform index under a `candidate-` tag. It attests
|
|
both indexes, and only then writes the two `inputs-` tags. `candidate-`
|
|
tags are internal: one can name an index that is not attested, and the
|
|
next run overwrites it. Do not consume them.
|
|
- **Order.** It writes `inputs-<digest>-slim` first and `inputs-<digest>`
|
|
last, so the full tag marks a complete publication. The full index names
|
|
its slim index in the annotation `io.paperclip.runner.slim-digest`.
|
|
- **Pairing.** Resolve the full tag first. Take the slim digest from that
|
|
annotation of the full index, and pull the slim image by that digest. Never
|
|
resolve `inputs-<digest>-slim` by tag on its own: while a publication is
|
|
incomplete, that tag can name the slim image of another run.
|
|
- **Recovery.** A run that stops between the two tags leaves a slim tag
|
|
alone. Those inputs are not published. The next run for them builds again,
|
|
replaces that tag with its own slim image and then writes the full tag. A
|
|
full tag whose slim tag is missing or names another image fails every run
|
|
for those inputs. An operator must then delete both package versions and
|
|
re-run the workflow run of a `master` push with those inputs.
|
|
- **Create-only.** The workflow never moves a tag of a complete publication,
|
|
but the registry does not enforce that. Pin the index digest and verify it:
|
|
`gh attestation verify oci://ghcr.io/paperclipai/paperclip-runner@sha256:<digest> --repo paperclipai/paperclip --signer-workflow paperclipai/paperclip/.github/workflows/runner-image.yml --source-ref refs/heads/master`
|
|
- **First publication.** GitHub creates the package private. A maintainer
|
|
must make it public once.
|
|
|
|
## Reading the transport archives alone
|
|
|
|
The server needs the transport form of an older pack when a retained session
|
|
names a pack that is not the server's own. A pack with
|
|
`payload.runnerInputsDigest` came out of this image, so the server reads it
|
|
from `inputs-<digest>` (`retained-provider-pack-oci.ts`). It does not count
|
|
layers. It uses the rule of "Layers": the full image is the slim image, then
|
|
layers that hold transport content and nothing else.
|
|
|
|
- The full index must carry the annotations
|
|
`io.paperclip.runner.inputs-digest`, equal to the pack's digest of inputs,
|
|
`org.opencontainers.image.revision`, equal to the pack's
|
|
`runnerSourceRevision`, and `io.paperclip.runner.slim-digest`.
|
|
- The server reads the slim index by that digest, never by the slim tag, and
|
|
checks the bytes against the digest.
|
|
- For the pack's platform, the layers of the slim image must be the first
|
|
layers of the full image, as the same blobs in the same order. The layers
|
|
after them are the transport layers: at least one and at most eight.
|
|
- The server downloads only those layers. It refuses any entry in them that
|
|
is not a transport archive or the receipt, and any file that is in two of
|
|
them.
|
|
- The importer then checks every archive against the pack's manifest, as for
|
|
any other source.
|
|
- The record the server keeps of the source names the digest of inputs and
|
|
every transport layer under `runnerImage`. Its `transportLayerDigest` is
|
|
then the last of those layers and `transportLayerBytes` their total.
|
|
|
|
When the image cannot be read (a tag or the slim index does not exist, or the
|
|
registry refuses a request), the server reads the application image tagged
|
|
`sha-<runnerSourceRevision>`, as it does for a pack without
|
|
`runnerInputsDigest`. An image that can be read and does not match fails.
|
|
|
|
## Known limits
|
|
|
|
- A build without `PAPERCLIP_RUNNER_SOURCE_REVISION` or
|
|
`PAPERCLIP_BUILD_COMMIT` has no pack. It is for local use.
|
|
- The build runs the steps of the Runner's `build` script that write output,
|
|
not its drift checks; repository CI runs those.
|
|
- A build that vendors `/runner-package/` reuses `dist` only when its own
|
|
Runner build-inputs digest equals the stamp's. The image computes that
|
|
digest in the `.dockerignore`-filtered context; a checkout gives another.
|
|
- `# syntax=docker/dockerfile:1.20` is a tag. A new frontend release under it
|
|
does not change the identity.
|
|
- The archive check does not tell a directory with its own archive member
|
|
from one created only as a file's parent.
|
|
- A lockfile that the lockfile refresh cannot bring up to the Runner's
|
|
manifests leaves every `master` run green with only a notice, for as long
|
|
as it lasts. No image is published in that time, and nothing fails.
|