fix(ci): reuse one available Cloud registry cache (#13334)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Cloud needs a verified image for each merged source
commit.
> - Fresh builders restore compiled native dependencies from registry
caches.
> - The current workflow imports up to eleven historical cache manifests
at once.
> - Live builds missed native layers that a fresh builder reused from
one manifest.
> - This PR selects the nearest available cache and tests reuse across
fresh builders.

## Linked Issues or Issue Description

Refs #13329 and #13330. A search of open cache PRs found no duplicate of
this change.

**What existing behavior does this improve?**

Remote Docker cache reuse on fresh Cloud image builders.

**Current behavior**

[Cloud run
34714483272](https://github.com/paperclipai/paperclip/actions/runs/34714483272/job/103609096836)
imported the previous cache manifest successfully but rebuilt
`cargo-chef` and Rust dependencies. The dependency compile took 3m43s.
The preceding image build had already exported those layers.

A controlled [fresh-builder
diagnostic](https://github.com/paperclipai/paperclip/actions/runs/34715336530)
used the same source and registry cache. The single-manifest job reused
both layers immediately. The multiple-manifest job rebuilt them and
failed the cache assertion. Both jobs used GitHub-hosted runners with
read-only access.

**Proposed behavior**

Inspect cache manifests in first-parent order and import only the
nearest available one. Keep full-SHA cache exports, the ten-commit
search bound, and the legacy fallback. If caches cannot be read, permit
a cold build.

**Reason and benefit**

Avoid the observed cache misses without changing image contents or
builder sizes. Expected savings include about four minutes of native
tool/dependency compilation when those inputs are unchanged. The final
merge-to-deployable gain still needs a post-merge measurement.

**Breaking changes**

No image, artifact, deployment, or runner-routing contract changes.

## What Changed

- Select one available ancestor cache after Docker login and Buildx
setup.
- Preserve separate writable cache tags for each full source SHA.
- Test cache ordering, missing caches, registry errors, and workflow
integration.
- Add the selector tests to the existing release-registry suite.
- Export a local test cache, remove the first builder, and verify a
source rebuild on a fresh builder.
- Document cache selection and the stronger Docker check.

## Verification

- Passed 456 focused workflow, routing, readiness, preview-artifact, and
cache-selector tests.
- Passed shell syntax, ShellCheck for the changed probe, actionlint
workflow validation, and `git diff --check`. actionlint's shell checks
were disabled for the workflow validation because unchanged
migration-label commands trigger existing SC2012 notes.
- The fresh-builder registry diagnostic proves the single-cache
behavior. The [permanent two-builder probe
passed](https://github.com/paperclipai/paperclip/actions/runs/34715771048/job/103612624090),
including a changed real binary and dependency-declaration invalidation.
- Passed all 35 latest-head checks (green or intentionally skipped),
including full typecheck, test, build, and browser suites in [PR CI run
34715771217](https://github.com/paperclipai/paperclip/actions/runs/34715771217).
- The real selector CLI inspected registry metadata and chose the
nearest available ancestor cache.
- Fresh Greptile review is 5/5 with no open findings. The PR title was
corrected to meet the source-change naming rule; the review check passed
after that correction.
- Local full-suite runs and Docker builds are unavailable because the
local Docker daemon is unresponsive after disk exhaustion. CI provides
the Linux verification.

## Risks

- Missing or unreadable caches cause a slower cold build. The selector
logs that condition and preserves image publication.
- Inspecting several missing ancestors adds lookup time. Each lookup has
a ten-second timeout and the search is bounded.
- The Docker test now exports a local cache. It removes the first
builder before starting the second to release disk space, then cleans up
its builders and files.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository tools, and code
execution. The exact serving model ID and context window are not exposed
by this environment.

## Checklist


- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Devin FoleyandPaperclip authored and GitHub committed 2026-09-12 13:20:18 -07:00
1 parent 0ce7df2648
commit 44f6312cd8
8 files changed
+208 -45

No files matched your search

+9 -19
View File
@@ -65,24 +65,6 @@ jobs:
id: tools-epoch
run: echo "epoch=$(date -u +%G-W%V)" >> "$GITHUB_OUTPUT"
# Each SHA exports its own cache. Import recent first-parent caches so
# a late older build cannot overwrite a newer build's cache manifest.
# The legacy ref keeps the first builds warm during the transition.
- name: Select cloud cache ancestry
id: cloud-cache
env:
CACHE_IMAGE: ghcr.io/${{ github.repository }}
run: |
set -euo pipefail
{
echo 'sources<<CACHE_SOURCES'
for commit in $(git rev-list --first-parent --max-count=10 HEAD); do
echo "type=registry,ref=$CACHE_IMAGE:buildcache-cloud-$commit"
done
echo "type=registry,ref=$CACHE_IMAGE:buildcache-cloud"
echo 'CACHE_SOURCES'
} >> "$GITHUB_OUTPUT"
- name: Setup pnpm
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
with:
@@ -161,6 +143,14 @@ jobs:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4
# Mixing several historical manifests missed otherwise reusable native
# layers on fresh builders. Import the nearest available complete cache.
- name: Select cloud cache ancestry
id: cloud-cache
env:
CACHE_IMAGE: ghcr.io/${{ github.repository }}
run: node scripts/select-cloud-cache.mjs
# Deployment tooling reads these labels from the registry to verify an
# image's schema expectations against a migrator before deploying it,
# without pulling the image. The server refuses to start when the
@@ -225,7 +215,7 @@ jobs:
# Same-SHA builds serialize above; different SHAs never share a
# writable cache ref. Registry layers are content-addressed and
# shared even when cache manifests have separate tags.
cache-from: ${{ steps.cloud-cache.outputs.sources }}
cache-from: ${{ steps.cloud-cache.outputs.source }}
cache-to: type=registry,ref=ghcr.io/${{ github.repository }}:buildcache-cloud-${{ github.sha }},mode=max
tags: ${{ steps.meta-cloud.outputs.tags }}
labels: ${{ steps.meta-cloud.outputs.labels }}
+1 -1
View File
@@ -32,7 +32,7 @@ jobs:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4
# Compile the real target, then change source in a disposable context.
# Require dependency-layer reuse and changed metadata from the real binary.
# A fresh builder must import dependencies and produce changed binary metadata.
# No registry credentials, external cache, or image publication.
- name: Verify native build and dependency cache reuse
run: bash scripts/check-docker-runner-cache.sh