Jannik/PaperClipAI
1
0 Fork 0
mirror of https://github.com/paperclipai/paperclip.git synced 2026-10-06 10:48:12 +02:00
Code Issues Packages Projects Releases Wiki Activity

ci: reuse dependency caches without per-PR uploads (#13300)

Browse Source 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Cloud releases wait for source verification before deployment.
> - That verification reuses compiled Rust dependencies to finish
sooner.
> - PR jobs save large pnpm stores under separate merge refs and
different lockfile keys.
> - Those copies compete with master build caches for the repository's
10 GB cache limit.
> - This PR makes PR dependency caches restore-only and reuses
master-compatible keys.
> - A separate pin update will activate the reviewed workflow.

## Linked Issues or Issue Description

**What happened?**
PR merge refs accumulated roughly 700 MB copies of the same pnpm store.
Master Rust caches disappeared, and Cloud readiness run
[34656098157](https://github.com/paperclipai/paperclip/actions/runs/34656098157)
rebuilt dependencies after cache misses. The repository currently has a
10 GB limit. GitHub rejected a request for 50 GB; that setting needs
separate organization/billing access.

**Expected behavior**
PR jobs should reuse downloaded packages without evicting post-merge
compilation caches through duplicate uploads.

**Steps to reproduce**
1. Run several PRs while the checked-in lockfile needs policy
regeneration.
2. Compare the setup-node keys in PR jobs and master jobs.
3. List Actions caches by ref, key, and archive size. The PR keys repeat
across merge refs.

**Paperclip version or commit**
f12b647ae, before this change.

**Deployment mode**
GitHub Actions, with GitHub-hosted and allowlisted AWS PR runners.

Refs #13267 (empty pnpm store prevention). Searched open issues and PRs
for pnpm cache duplication and found no duplicate implementation. This
change leaves the paused capacity documentation PR #13280 alone.

## What Changed

- Replace setup-node cache writes with pinned `actions/cache/restore` in
all seven PR install job definitions.
- Restore against the checked-in lockfile before downloading the
regenerated policy artifact. Keep every install frozen against that
artifact.
- Allow an OS/architecture-specific pnpm fallback and disable automatic
setup-node caching.
- Remove dependency-store caching from the resolution-only policy job.
- Add eight regression tests, update the existing stacked-lockfile cache
assertion, and document cache behavior and storage settings.

## Verification

- Passed 505 workflow, routing, cache, and source-verification tests:
`node --test '.github/scripts/tests/*.test.mjs'
scripts/__tests__/e2e-shard.test.mjs
scripts/__tests__/run-vitest-stable-shard.test.mjs
scripts/__tests__/release-verify-workflow.test.mjs
scripts/cloud-source-verification.test.mjs`.
- Passed `actionlint .github/workflows/pr-trusted.yml` and `git diff
--check`.
- The AWS routing gate is unchanged. Author, event sender, and rerun
actor must still be allowlisted.
- This definition PR does not change the active `pr.yml` pin. After
review and merge, authorize its immutable merge SHA additively and
activate it in a separate PR. Verify a populated restore and no cache
uploads in an allowlisted PR.
- All 32 checks passed or were intentionally skipped on
`44b31eca590f61b75cae646de43c491b6c4deae7`, including full native Runner
verification, application build, server/workspace tests, and browser
shards. Current-head Greptile is 5/5 with no findings. No application
code changes in this PR.

## Risks

- New dependencies present only in a PR may download again on each run
until master saves a cache containing them. Frozen installation remains
the source of dependency resolution.
- Missing or expired stores fall back to normal package downloads.
- Existing PR copies remain until expiry or a separate targeted cleanup.
No cache entries are deleted here.
- The workflow only takes effect after the separate immutable pin
rotation. Storage billing settings are not changed by this PR.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository tools, and code
execution. The exact serving model ID and context window are not exposed
by this environment.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Devin FoleyandPaperclip authored and GitHub committed 2026-09-11 17:31:15 -07:00
1 parent fe38085174
commit 44dde2dec4
4 files changed
+188 -53
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.github/scripts/tests/pr-dependency-cache.test.mjs
+39
View File
@@ -0,0 +1,39 @@
import test from "node:test";
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
const workflow = readFileSync(new URL("../../workflows/pr-trusted.yml", import.meta.url), "utf8");
const jobs = [...workflow.matchAll(/^ ([a-z_][a-z_0-9]*):\n([\s\S]*?)(?=^ [a-z_][a-z_0-9]*:\n|$(?![\s\S]))/gm)];
const installers = jobs.filter(([, , body]) => body.includes("run: pnpm install --frozen-lockfile"));
test("PR workflows restore dependency stores without creating branch copies", () => {
assert.equal(installers.length, 7);
assert.doesNotMatch(workflow, /^ +cache: pnpm$/m);
assert.doesNotMatch(workflow, /uses: actions\/cache(?:@|\/save@)/);
for (const [, job, body] of jobs) {
for (const step of body.split(" - name:").filter((step) => step.includes("uses: actions/setup-node@"))) {
assert.match(step, /package-manager-cache: false/, job);
}
}
const policy = jobs.find(([, name]) => name === "policy")[2];
assert.doesNotMatch(policy, /uses: actions\/cache|cache: pnpm/);
});
for (const [, job, body] of installers) {
test(`${job}: reuse master keys before restoring the resolved PR lockfile`, () => {
const locate = body.indexOf(" - name: Locate pnpm store");
const restore = body.indexOf(" - name: Restore pnpm store (read only)");
const artifact = body.indexOf(" - name: Restore regenerated PR lockfile");
const install = body.indexOf("run: pnpm install --frozen-lockfile");
assert.ok(locate >= 0 && locate < restore && restore < artifact && artifact < install);
const cache = body.slice(restore, artifact);
assert.match(body.slice(locate, restore), /pnpm store path --silent/);
assert.match(body.slice(locate, restore), /node -p 'process.arch'/);
assert.match(cache, /uses: actions\/cache\/restore@[a-f0-9]{40}/);
assert.ok(cache.includes("key: node-cache-${{ runner.os }}-${{ steps.pnpm_store.outputs.arch }}-pnpm-${{ hashFiles('pnpm-lock.yaml') }}"));
assert.ok(cache.includes("restore-keys: node-cache-${{ runner.os }}-${{ steps.pnpm_store.outputs.arch }}-pnpm-"));
assert.match(body.slice(artifact, install), /if: needs.policy.outputs.lockfile_regenerated == '1'/);
assert.match(body.slice(artifact, install), /name: pr-lockfile/);
assert.doesNotMatch(body.slice(artifact, install), /continue-on-error/);
});
}
.github/workflows/pr-trusted.yml
+113 -48
View File
@@ -284,6 +284,7 @@ jobs:
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
package-manager-cache: false
- name: Setup pnpm
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
@@ -295,12 +296,6 @@ jobs:
version: 9.15.4
run_install: false
- name: Setup Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
cache: pnpm
- name: Validate migration ordering against target branch
run: >-
node .github/scripts/check-pr-migration-order.mjs
@@ -389,6 +384,7 @@ jobs:
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
package-manager-cache: false
- name: Setup pnpm
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
@@ -399,6 +395,21 @@ jobs:
with:
version: 9.15.4
# Share the checked-in lockfile key with master. PR merge refs must not
# save full copies of the store or evict the post-merge build caches.
- name: Locate pnpm store
id: pnpm_store
run: |
echo "path=$(pnpm store path --silent)" >> "$GITHUB_OUTPUT"
echo "arch=$(node -p 'process.arch')" >> "$GITHUB_OUTPUT"
- name: Restore pnpm store (read only)
uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
with:
path: ${{ steps.pnpm_store.outputs.path }}
key: node-cache-${{ runner.os }}-${{ steps.pnpm_store.outputs.arch }}-pnpm-${{ hashFiles('pnpm-lock.yaml') }}
restore-keys: node-cache-${{ runner.os }}-${{ steps.pnpm_store.outputs.arch }}-pnpm-
- name: Restore regenerated PR lockfile (if policy uploaded one)
if: needs.policy.outputs.lockfile_regenerated == '1'
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
@@ -406,12 +417,6 @@ jobs:
name: pr-lockfile
path: .
- name: Setup Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
cache: pnpm
- name: Install dependencies
run: pnpm install --frozen-lockfile
@@ -485,6 +490,7 @@ jobs:
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
package-manager-cache: false
- name: Setup pnpm
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
@@ -495,6 +501,21 @@ jobs:
with:
version: 9.15.4
# Share the checked-in lockfile key with master. PR merge refs must not
# save full copies of the store or evict the post-merge build caches.
- name: Locate pnpm store
id: pnpm_store
run: |
echo "path=$(pnpm store path --silent)" >> "$GITHUB_OUTPUT"
echo "arch=$(node -p 'process.arch')" >> "$GITHUB_OUTPUT"
- name: Restore pnpm store (read only)
uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
with:
path: ${{ steps.pnpm_store.outputs.path }}
key: node-cache-${{ runner.os }}-${{ steps.pnpm_store.outputs.arch }}-pnpm-${{ hashFiles('pnpm-lock.yaml') }}
restore-keys: node-cache-${{ runner.os }}-${{ steps.pnpm_store.outputs.arch }}-pnpm-
- name: Restore regenerated PR lockfile (if policy uploaded one)
if: needs.policy.outputs.lockfile_regenerated == '1'
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
@@ -502,12 +523,6 @@ jobs:
name: pr-lockfile
path: .
- name: Setup Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
cache: pnpm
- name: Install dependencies
run: pnpm install --frozen-lockfile
@@ -607,6 +622,7 @@ jobs:
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
package-manager-cache: false
- name: Setup pnpm
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
@@ -617,6 +633,21 @@ jobs:
with:
version: 9.15.4
# Share the checked-in lockfile key with master. PR merge refs must not
# save full copies of the store or evict the post-merge build caches.
- name: Locate pnpm store
id: pnpm_store
run: |
echo "path=$(pnpm store path --silent)" >> "$GITHUB_OUTPUT"
echo "arch=$(node -p 'process.arch')" >> "$GITHUB_OUTPUT"
- name: Restore pnpm store (read only)
uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
with:
path: ${{ steps.pnpm_store.outputs.path }}
key: node-cache-${{ runner.os }}-${{ steps.pnpm_store.outputs.arch }}-pnpm-${{ hashFiles('pnpm-lock.yaml') }}
restore-keys: node-cache-${{ runner.os }}-${{ steps.pnpm_store.outputs.arch }}-pnpm-
- name: Restore regenerated PR lockfile (if policy uploaded one)
if: needs.policy.outputs.lockfile_regenerated == '1'
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
@@ -624,12 +655,6 @@ jobs:
name: pr-lockfile
path: .
- name: Setup Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
cache: pnpm
- name: Install dependencies
run: pnpm install --frozen-lockfile
@@ -653,6 +678,7 @@ jobs:
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
package-manager-cache: false
- name: Setup pnpm
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
@@ -663,6 +689,21 @@ jobs:
with:
version: 9.15.4
# Share the checked-in lockfile key with master. PR merge refs must not
# save full copies of the store or evict the post-merge build caches.
- name: Locate pnpm store
id: pnpm_store
run: |
echo "path=$(pnpm store path --silent)" >> "$GITHUB_OUTPUT"
echo "arch=$(node -p 'process.arch')" >> "$GITHUB_OUTPUT"
- name: Restore pnpm store (read only)
uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
with:
path: ${{ steps.pnpm_store.outputs.path }}
key: node-cache-${{ runner.os }}-${{ steps.pnpm_store.outputs.arch }}-pnpm-${{ hashFiles('pnpm-lock.yaml') }}
restore-keys: node-cache-${{ runner.os }}-${{ steps.pnpm_store.outputs.arch }}-pnpm-
- name: Restore regenerated PR lockfile (if policy uploaded one)
if: needs.policy.outputs.lockfile_regenerated == '1'
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
@@ -670,12 +711,6 @@ jobs:
name: pr-lockfile
path: .
- name: Setup Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
cache: pnpm
- name: Install dependencies
run: pnpm install --frozen-lockfile
@@ -727,6 +762,7 @@ jobs:
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
package-manager-cache: false
- name: Setup pnpm
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
@@ -737,6 +773,21 @@ jobs:
with:
version: 9.15.4
# Share the checked-in lockfile key with master. PR merge refs must not
# save full copies of the store or evict the post-merge build caches.
- name: Locate pnpm store
id: pnpm_store
run: |
echo "path=$(pnpm store path --silent)" >> "$GITHUB_OUTPUT"
echo "arch=$(node -p 'process.arch')" >> "$GITHUB_OUTPUT"
- name: Restore pnpm store (read only)
uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
with:
path: ${{ steps.pnpm_store.outputs.path }}
key: node-cache-${{ runner.os }}-${{ steps.pnpm_store.outputs.arch }}-pnpm-${{ hashFiles('pnpm-lock.yaml') }}
restore-keys: node-cache-${{ runner.os }}-${{ steps.pnpm_store.outputs.arch }}-pnpm-
- name: Restore regenerated PR lockfile (if policy uploaded one)
if: needs.policy.outputs.lockfile_regenerated == '1'
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
@@ -744,12 +795,6 @@ jobs:
name: pr-lockfile
path: .
- name: Setup Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
cache: pnpm
- name: Install dependencies
run: pnpm install --frozen-lockfile
@@ -773,6 +818,7 @@ jobs:
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
package-manager-cache: false
- name: Setup pnpm
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
@@ -783,6 +829,21 @@ jobs:
with:
version: 9.15.4
# Share the checked-in lockfile key with master. PR merge refs must not
# save full copies of the store or evict the post-merge build caches.
- name: Locate pnpm store
id: pnpm_store
run: |
echo "path=$(pnpm store path --silent)" >> "$GITHUB_OUTPUT"
echo "arch=$(node -p 'process.arch')" >> "$GITHUB_OUTPUT"
- name: Restore pnpm store (read only)
uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
with:
path: ${{ steps.pnpm_store.outputs.path }}
key: node-cache-${{ runner.os }}-${{ steps.pnpm_store.outputs.arch }}-pnpm-${{ hashFiles('pnpm-lock.yaml') }}
restore-keys: node-cache-${{ runner.os }}-${{ steps.pnpm_store.outputs.arch }}-pnpm-
- name: Restore regenerated PR lockfile (if policy uploaded one)
if: needs.policy.outputs.lockfile_regenerated == '1'
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
@@ -790,12 +851,6 @@ jobs:
name: pr-lockfile
path: .
- name: Setup Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
cache: pnpm
- name: Install dependencies
run: pnpm install --frozen-lockfile
@@ -860,6 +915,7 @@ jobs:
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
package-manager-cache: false
- name: Setup pnpm
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
@@ -870,6 +926,21 @@ jobs:
with:
version: 9.15.4
# Share the checked-in lockfile key with master. PR merge refs must not
# save full copies of the store or evict the post-merge build caches.
- name: Locate pnpm store
id: pnpm_store
run: |
echo "path=$(pnpm store path --silent)" >> "$GITHUB_OUTPUT"
echo "arch=$(node -p 'process.arch')" >> "$GITHUB_OUTPUT"
- name: Restore pnpm store (read only)
uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
with:
path: ${{ steps.pnpm_store.outputs.path }}
key: node-cache-${{ runner.os }}-${{ steps.pnpm_store.outputs.arch }}-pnpm-${{ hashFiles('pnpm-lock.yaml') }}
restore-keys: node-cache-${{ runner.os }}-${{ steps.pnpm_store.outputs.arch }}-pnpm-
- name: Restore regenerated PR lockfile (if policy uploaded one)
if: needs.policy.outputs.lockfile_regenerated == '1'
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
@@ -877,12 +948,6 @@ jobs:
name: pr-lockfile
path: .
- name: Setup Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
cache: pnpm
- name: Install dependencies
run: pnpm install --frozen-lockfile