ci: pin a checkout-independent Rust cache path so PR lanes hit master's cache (#14394)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Paperclip ships a native Runner binary, written in Rust, and seven
CI lanes build it on every pull request
> - `Canary Dry Run` is the slowest check on every green PR run, and
most of its time is `cargo build --release` on third-party crates
> - Master saves a Rust dependency cache for these lanes, but every PR
lane logs `No cache found` and compiles every crate from zero
> - The cache key matches, but GitHub also compares a hash of the
absolute cache paths, and the master writer (RunsOn fleet,
`/home/runner/_work/...`) and the PR readers (GitHub-hosted,
`/home/runner/work/...`) hash different paths
> - This pull request gives both sides a checkout-independent workspace
path, so the hashes match and the PR lanes restore master's cache
> - The benefit is about 2.5 minutes less wall clock per PR run and
about 18 fewer runner-minutes per run

## Linked Issues or Issue Description

No public issue exists for this problem. The description below follows
the enhancement template.

Related prior PRs on the same cache: Refs #13194, Refs #13259, Refs
#13457, Refs #13459, Refs #13500, Refs #13586. None of them pins the
workspace path, so none of them fixes this miss.

**What existing behavior does this improve?**

The `Swatinem/rust-cache` restore step in the PR workflow lanes that
build the Runner: `Canary Dry Run`, `Build`, `Typecheck + Release
Registry`, and the four `Verify Paperclip Runner` lanes.

**Subsystem affected**

CI workflows under `.github/workflows/`, their guard tests under
`.github/scripts/tests/`, and `doc/RELEASE-AUTOMATION-SETUP.md`.

**Current behavior**

Every PR lane logs `No cache found` although master holds an entry with
the exact key. Run 36424309181 computed
`v0-rust-release-runner-v1-Linux-x64-c3a3ca66-a95b0328`, and master
holds a 678 MB entry with that key. GitHub matches a cache entry on the
key and on a version hash of the absolute paths in the cache. The master
writer runs on the RunsOn fleet, where the checkout is
`/home/runner/_work/paperclip/paperclip`. The PR readers run on
GitHub-hosted `ubuntu-latest`, where the checkout is
`/home/runner/work/paperclip/paperclip`. The stored version `5c40870d…`
is the sha256 of the `_work` paths plus `zstd-without-long|1.0`. The
`work` paths hash to `1656e9ee…`. The key can never match, so each lane
compiles every third-party crate again.

**Proposed behavior**

The writer and the readers pass the same checkout-independent path to
`rust-cache`. Both runner layouts then produce the same version hash,
and the PR lanes restore master's cache.

**Reason and benefit**

`Canary Dry Run` takes 533s on a green run. 251s of that is dependency
compilation that a warm cache removes. Seven lanes pay this cost in
every PR run.

**Breaking changes**

None. This change affects CI only.

## What Changed

- Add a `Pin the Runner Rust workspace path` step before `rust-cache` in
the master writer (`release-verify.yml`, typecheck and runner lanes) and
in all four PR readers (`pr-trusted.yml`). The step creates the symlink
`$HOME/paperclip-runner-rust` →
`$GITHUB_WORKSPACE/packages/paperclip-runner/runner` and passes that
path to `rust-cache` as `workspaces: <path> -> target`. `rust-cache`
resolves the input with `path.resolve`, which does not follow symlinks,
so both runner layouts now produce the same cache paths and the same
version hash. `$HOME` is `/home/runner` on both images, which is why the
`~/.cargo` paths already agreed.
- Bump the shared keys `release-runner-v1` → `release-runner-v2` and
`release-typecheck-v1` → `release-typecheck-v2`. The old, unreachable
entries are then visibly orphaned instead of sharing a key with the new
ones.
- Extend the guard tests `pr-runner-rust-cache`, `release-runner-cache`,
and `typecheck-rust-cache`. They now require the pin step in both
workflows with identical text, placed before the cache step, and they
reject a `workspaces:` value that resolves under the checkout. The
`pr-runner-rust-cache` test checks all four PR reader jobs and fails if
a `rust-cache` step appears in a PR job that is not in its reader list.
- Update `doc/RELEASE-AUTOMATION-SETUP.md` to name the
`release-runner-v2` key and to explain the pinned workspace path.

### Expected savings once merged

Measured from run 36424309181. "Removed" is the dependency-compile time
that a warm restore removes, minus about 18s to restore the 680 MB
entry. The fleet writer's own restore shows this cost.

| Lane | Today | Removed | Expected |
|---|---|---|---|
| Canary Dry Run | 533s | ~150s | ~380s |
| Typecheck + Release Registry | 462s | ~155s | ~305s |
| Verify Paperclip Runner (vitest 2/2) | 453s | ~245s | ~210s |
| Verify Paperclip Runner (rust) | 400s | ~175s | ~225s |
| Build | 348s | ~130s | ~220s |
| Verify Paperclip Runner (static checks) | 321s | ~170s | ~150s |
| Verify Paperclip Runner (vitest 1/2) | 346s | ~70s | ~275s |

- Wall clock per PR run: about 533s → about 385s. That is about 2.5
minutes faster to a green check set. `Canary Dry Run` stays the longest
check. The rest is the non-cargo work in `release.sh` (standalone
package builds ~30s, publish-payload preview ~73s).
- Runner time: about 18 runner-minutes saved per PR run across the seven
lanes.
- The first master push after merge compiles from zero once in the fleet
writer (about 4 extra minutes on that one run) and saves the v2 entry.
Later PRs hit it. When a PR changes `Cargo.lock`, the prefix restore key
still gives a partial hit, as before.

## Verification

- Run the guard tests for the three cache lanes:
`node --test .github/scripts/tests/pr-runner-rust-cache.test.mjs
.github/scripts/tests/release-runner-cache.test.mjs
.github/scripts/tests/typecheck-rust-cache.test.mjs`
  Result: 21 pass, 0 fail.
- Run the full guard suite: `node --test
'.github/scripts/tests/*.test.mjs'`. Result: 376 pass, 3 fail. The 3
failures are in `docker-canary-promotion.test.mjs`. They hit a sandbox
temp-file ENOENT and fail the same way on the unmodified branch.
- Run `node --test scripts/__tests__/release-verify-workflow.test.mjs`.
Result: 14 pass.
- Local archive test: create a tar from the `_work` layout through the
symlink (relative `../../../paperclip-runner-rust/target` entries, `tar
-P -C $GITHUB_WORKSPACE`, the same way `@actions/cache` does). Extract
it on the `work` layout. The files land in the real target directory and
the symlink stays intact.
- After merge, open any GitHub-hosted PR run and confirm that the seven
Rust lanes log `Restored from cache key ...release-runner-v2...` in
place of `No cache found`.

## Risks

- Low risk. The change touches CI workflows, their tests, and one doc
page. No product code changes.
- If the pin step fails, `rust-cache` reports a miss and the lane
compiles from zero, as it does today. The build does not break.
- Both runner layouts sit four levels under `/home/runner`, so the
relative `../../../` archive entries line up. The existing
`~/.cargo/registry` and `~/.cargo/git` cache paths already rely on this
property. A future runner image with a different `$HOME` depth would
miss the cache but would not fail the job.
- `rm -rf "$pinned"` acts on the symlink itself (no trailing slash),
never on the checkout behind it. It only matters on a reused runner.
- Squash-merge note: the branch carries commits by `Bender (Fable)`. Add
`Co-Authored-By: Bender (Fable) <bender-fable@paperclip.local>` to the
squash body to keep that authorship.

## Model Used

- Anthropic Claude Fable 5.1 (`claude-fable-5-1`), run through Claude
Code inside a Paperclip agent heartbeat. Extended thinking was on. Tool
use: shell, GitHub CLI, and the GitHub REST API for workflow logs, cache
listings, and PR operations.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [ ] My branch name describes the change and contains no internal
ticket id. The agent execution workspace fixed this branch name, so I
cannot rename it. Squash-merge drops the branch name.
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Bender (Fable) <bender-fable@paperclip.local>

.github/workflows/pr-trusted.yml

@@ -407,6 +407,28 @@ jobs:
       # rebuilds the Runner release binary; without the shared Rust cache that
       # is a ~3m40s cold compile of all third-party crates (run 35036001734,
       # 2026-09-15). Same restore-only contract as Verify Paperclip Runner.
+
+      # rust-cache hashes its absolute cache paths into the entry's version,
+      # and GitHub only serves an entry whose key and version both match. The
+      # target directory sits under the checkout, and the checkout root
+      # differs by runner: /home/runner/_work on the RunsOn fleets that write
+      # this cache against /home/runner/work on GitHub-hosted runners. Every
+      # key input agreed, yet all 25 completed pull requests on 2026-09-28
+      # logged "No cache found" (run 36424309181) and cold-compiled every
+      # crate for ~4 minutes in four lanes. Point rust-cache at the same
+      # directory through a checkout-independent path so both layouts hash
+      # the same version. Keep this block identical in both workflows: the
+      # reader and the writer must agree or the version matches nothing.
+      - name: Pin the Runner Rust workspace path
+        id: runner_rust_workspace
+        run: |
+          set -euo pipefail
+          pinned="$HOME/paperclip-runner-rust"
+          # A symlink here is removed as a link, never followed into the checkout.
+          rm -rf "$pinned"
+          ln -s "$GITHUB_WORKSPACE/packages/paperclip-runner/runner" "$pinned"
+          echo "path=$pinned" >> "$GITHUB_OUTPUT"
+
       - name: Select the pinned Runner Rust toolchain
         working-directory: packages/paperclip-runner
         run: |
@@ -445,8 +467,8 @@ jobs:
       - name: Restore Runner Rust dependencies (read only)
         uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
         with:
-          workspaces: packages/paperclip-runner/runner -> target
-          shared-key: release-runner-v1
+          workspaces: ${{ steps.runner_rust_workspace.outputs.path }} -> target
+          shared-key: release-runner-v2
           # Mirror the master writer: these also feed the cache key.
           cache-workspace-crates: false
           cache-bin: false
@@ -766,11 +788,33 @@ jobs:
 
       # Same restore-only contract as the pnpm store above, for the Rust
       # dependency tree: master's post-merge verification is the sole writer
-      # of release-runner-v1, and PR merge refs must not save branch-scoped
+      # of release-runner-v2, and PR merge refs must not save branch-scoped
       # copies of a ~680MB target directory. Every key input below has to
       # match that writer in release-verify.yml exactly or each PR misses and
       # recompiles all 313 third-party crates in both profiles. A miss is a
       # slow run, never a wrong one.
+
+      # rust-cache hashes its absolute cache paths into the entry's version,
+      # and GitHub only serves an entry whose key and version both match. The
+      # target directory sits under the checkout, and the checkout root
+      # differs by runner: /home/runner/_work on the RunsOn fleets that write
+      # this cache against /home/runner/work on GitHub-hosted runners. Every
+      # key input agreed, yet all 25 completed pull requests on 2026-09-28
+      # logged "No cache found" (run 36424309181) and cold-compiled every
+      # crate for ~4 minutes in four lanes. Point rust-cache at the same
+      # directory through a checkout-independent path so both layouts hash
+      # the same version. Keep this block identical in both workflows: the
+      # reader and the writer must agree or the version matches nothing.
+      - name: Pin the Runner Rust workspace path
+        id: runner_rust_workspace
+        run: |
+          set -euo pipefail
+          pinned="$HOME/paperclip-runner-rust"
+          # A symlink here is removed as a link, never followed into the checkout.
+          rm -rf "$pinned"
+          ln -s "$GITHUB_WORKSPACE/packages/paperclip-runner/runner" "$pinned"
+          echo "path=$pinned" >> "$GITHUB_OUTPUT"
+
       - name: Select the pinned Runner Rust toolchain
         working-directory: packages/paperclip-runner
         run: |
@@ -809,8 +853,8 @@ jobs:
       - name: Restore Runner Rust dependencies (read only)
         uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
         with:
-          workspaces: packages/paperclip-runner/runner -> target
-          shared-key: release-runner-v1
+          workspaces: ${{ steps.runner_rust_workspace.outputs.path }} -> target
+          shared-key: release-runner-v2
           # Mirror the master writer: these also feed the cache key.
           cache-workspace-crates: false
           cache-bin: false
@@ -884,6 +928,28 @@ jobs:
       # shared Rust cache that is a ~3m40s cold compile of all third-party
       # crates (run 35036001734, 2026-09-15). Same restore-only contract as
       # Verify Paperclip Runner.
+
+      # rust-cache hashes its absolute cache paths into the entry's version,
+      # and GitHub only serves an entry whose key and version both match. The
+      # target directory sits under the checkout, and the checkout root
+      # differs by runner: /home/runner/_work on the RunsOn fleets that write
+      # this cache against /home/runner/work on GitHub-hosted runners. Every
+      # key input agreed, yet all 25 completed pull requests on 2026-09-28
+      # logged "No cache found" (run 36424309181) and cold-compiled every
+      # crate for ~4 minutes in four lanes. Point rust-cache at the same
+      # directory through a checkout-independent path so both layouts hash
+      # the same version. Keep this block identical in both workflows: the
+      # reader and the writer must agree or the version matches nothing.
+      - name: Pin the Runner Rust workspace path
+        id: runner_rust_workspace
+        run: |
+          set -euo pipefail
+          pinned="$HOME/paperclip-runner-rust"
+          # A symlink here is removed as a link, never followed into the checkout.
+          rm -rf "$pinned"
+          ln -s "$GITHUB_WORKSPACE/packages/paperclip-runner/runner" "$pinned"
+          echo "path=$pinned" >> "$GITHUB_OUTPUT"
+
       - name: Select the pinned Runner Rust toolchain
         working-directory: packages/paperclip-runner
         run: |
@@ -922,8 +988,8 @@ jobs:
       - name: Restore Runner Rust dependencies (read only)
         uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
         with:
-          workspaces: packages/paperclip-runner/runner -> target
-          shared-key: release-runner-v1
+          workspaces: ${{ steps.runner_rust_workspace.outputs.path }} -> target
+          shared-key: release-runner-v2
           # Mirror the master writer: these also feed the cache key.
           cache-workspace-crates: false
           cache-bin: false
@@ -1094,6 +1160,28 @@ jobs:
       # build:binary step; without the shared Rust cache that is a ~3m40s cold
       # compile of all third-party crates (run 35036001734, 2026-09-15). Same
       # restore-only contract as Verify Paperclip Runner.
+
+      # rust-cache hashes its absolute cache paths into the entry's version,
+      # and GitHub only serves an entry whose key and version both match. The
+      # target directory sits under the checkout, and the checkout root
+      # differs by runner: /home/runner/_work on the RunsOn fleets that write
+      # this cache against /home/runner/work on GitHub-hosted runners. Every
+      # key input agreed, yet all 25 completed pull requests on 2026-09-28
+      # logged "No cache found" (run 36424309181) and cold-compiled every
+      # crate for ~4 minutes in four lanes. Point rust-cache at the same
+      # directory through a checkout-independent path so both layouts hash
+      # the same version. Keep this block identical in both workflows: the
+      # reader and the writer must agree or the version matches nothing.
+      - name: Pin the Runner Rust workspace path
+        id: runner_rust_workspace
+        run: |
+          set -euo pipefail
+          pinned="$HOME/paperclip-runner-rust"
+          # A symlink here is removed as a link, never followed into the checkout.
+          rm -rf "$pinned"
+          ln -s "$GITHUB_WORKSPACE/packages/paperclip-runner/runner" "$pinned"
+          echo "path=$pinned" >> "$GITHUB_OUTPUT"
+
       - name: Select the pinned Runner Rust toolchain
         working-directory: packages/paperclip-runner
         run: |
@@ -1132,8 +1220,8 @@ jobs:
       - name: Restore Runner Rust dependencies (read only)
         uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
         with:
-          workspaces: packages/paperclip-runner/runner -> target
-          shared-key: release-runner-v1
+          workspaces: ${{ steps.runner_rust_workspace.outputs.path }} -> target
+          shared-key: release-runner-v2
           # Mirror the master writer: these also feed the cache key.
           cache-workspace-crates: false
           cache-bin: false

.github/workflows/release-verify.yml

@@ -42,6 +42,27 @@ jobs:
           node-version: 24
           cache: pnpm
 
+      # rust-cache hashes its absolute cache paths into the entry's version,
+      # and GitHub only serves an entry whose key and version both match. The
+      # target directory sits under the checkout, and the checkout root
+      # differs by runner: /home/runner/_work on the RunsOn fleets that write
+      # this cache against /home/runner/work on GitHub-hosted runners. Every
+      # key input agreed, yet all 25 completed pull requests on 2026-09-28
+      # logged "No cache found" (run 36424309181) and cold-compiled every
+      # crate for ~4 minutes in four lanes. Point rust-cache at the same
+      # directory through a checkout-independent path so both layouts hash
+      # the same version. Keep this block identical in both workflows: the
+      # reader and the writer must agree or the version matches nothing.
+      - name: Pin the Runner Rust workspace path
+        id: runner_rust_workspace
+        run: |
+          set -euo pipefail
+          pinned="$HOME/paperclip-runner-rust"
+          # A symlink here is removed as a link, never followed into the checkout.
+          rm -rf "$pinned"
+          ln -s "$GITHUB_WORKSPACE/packages/paperclip-runner/runner" "$pinned"
+          echo "path=$pinned" >> "$GITHUB_OUTPUT"
+
       - name: Select the pinned Runner Rust toolchain
         working-directory: packages/paperclip-runner
         run: |
@@ -56,8 +77,8 @@ jobs:
         if: ${{ github.repository == 'paperclipai/paperclip' && github.event_name == 'push' && github.ref == 'refs/heads/master' && inputs.ref == github.sha }}
         uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
         with:
-          workspaces: packages/paperclip-runner/runner -> target
-          shared-key: release-typecheck-v1
+          workspaces: ${{ steps.runner_rust_workspace.outputs.path }} -> target
+          shared-key: release-typecheck-v2
           # Rebuild workspace code and rerun every check. Cache only compiled
           # dependencies; never restore installed executables from cargo/bin.
           cache-workspace-crates: false
@@ -294,6 +315,27 @@ jobs:
           node-version: 24
           cache: pnpm
 
+      # rust-cache hashes its absolute cache paths into the entry's version,
+      # and GitHub only serves an entry whose key and version both match. The
+      # target directory sits under the checkout, and the checkout root
+      # differs by runner: /home/runner/_work on the RunsOn fleets that write
+      # this cache against /home/runner/work on GitHub-hosted runners. Every
+      # key input agreed, yet all 25 completed pull requests on 2026-09-28
+      # logged "No cache found" (run 36424309181) and cold-compiled every
+      # crate for ~4 minutes in four lanes. Point rust-cache at the same
+      # directory through a checkout-independent path so both layouts hash
+      # the same version. Keep this block identical in both workflows: the
+      # reader and the writer must agree or the version matches nothing.
+      - name: Pin the Runner Rust workspace path
+        id: runner_rust_workspace
+        run: |
+          set -euo pipefail
+          pinned="$HOME/paperclip-runner-rust"
+          # A symlink here is removed as a link, never followed into the checkout.
+          rm -rf "$pinned"
+          ln -s "$GITHUB_WORKSPACE/packages/paperclip-runner/runner" "$pinned"
+          echo "path=$pinned" >> "$GITHUB_OUTPUT"
+
       - name: Select the pinned Runner Rust toolchain
         working-directory: packages/paperclip-runner
         run: |
@@ -325,8 +367,8 @@ jobs:
         if: ${{ github.repository == 'paperclipai/paperclip' && github.event_name == 'push' && github.ref == 'refs/heads/master' && inputs.ref == github.sha }}
         uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
         with:
-          workspaces: packages/paperclip-runner/runner -> target
-          shared-key: release-runner-v1
+          workspaces: ${{ steps.runner_rust_workspace.outputs.path }} -> target
+          shared-key: release-runner-v2
           # Rebuild workspace code and rerun every check. Cache only compiled
           # dependencies; never restore installed executables from cargo/bin.
           cache-workspace-crates: false