mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-16 03:31:44 +01:00
perf(release): build canary daemons during source verification and keep their dependency cache (#21554)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Every master push runs a canary that publishes npm packages and reaches staging > - Only one canary runs at a time, so a canary's duration sets how often merged work ships > - A canary takes about 70 min. Two parts of it are avoidable waiting: the Runner daemons build only after the source proof, and darwin-x64 often rebuilds every dependency crate > - This pull request starts the daemon builds with the canary and keeps their dependency cache across Runner changes > - The benefit is about 15-25 min less per canary, with no change to what must pass before publication ## Linked Issues or Issue Description **What happened?** In successful canaries on 2026-11-13 the critical path was: `Reuse exact-source verification` 11-14 min, then `Release daemon (darwin-x64)` 10-28 min (9 runs: 10, 16, 18, 20, 20, 21, 24, 24, 28), then `publish_canary` 27-37 min. The daemon builds waited for the source proof, although they only build and upload artifacts. The darwin-x64 build was 16.4 min with a cache miss and 3.9 min with a hit. The native cache key included a hash of `packages/paperclip-runner/src/**`, `protocol/**` and `*.json`, but the cache holds dependency crates only (`cache-workspace-crates: false`). So every Runner change caused a full dependency rebuild. **Expected behavior** Daemons build in parallel with the source wait. The dependency cache changes only when dependencies, Cargo configuration, host or target change. Refs #21399, #16028 ## What Changed - `release.yml`: `release_runner_source` no longer needs `verify_canary`. On a canonical master push it starts at once. `publish_canary` still requires `verify_canary.result == 'success'`, so nothing publishes from unverified source. - `release-verify.yml`: the native Rust cache `shared-key` drops the Runner source hash. It keeps the Cargo manifest and configuration, host and target. - `release-verify-workflow.test.mjs`: tests that a canonical master push selects the source without the proof, that a fork, other ref or other event does not, that `publish_canary` still needs the proof, and that the cache key contains no source globs. - `doc/RELEASING.md`: describes both behaviors. ## Verification - `node --test scripts/__tests__/release-verify-workflow.test.mjs`: 73 of 73 pass. - Red: the same tests against master's `release.yml` and `release-verify.yml` fail 2 tests (the source selection and the cache key). - After merge: the next canary shows `Release daemon (*)` starting at the same time as `Reuse exact-source verification`, and darwin-x64 restores its cache. ## Risks - Low. If verification fails, that commit's daemon builds are wasted macOS minutes. Verification failures on master are now rare. - The first canary after merge saves a new cache entry, so it is not faster for darwin-x64. Later canaries are. - Stable, beta and nightly still select their source through their own gates, unchanged. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
83541e89ad
commit
066c8244b5
4 files changed
+27
-7
No files matched your search
@@ -693,7 +693,7 @@ jobs:
|
||||
uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6
|
||||
with:
|
||||
workspaces: ${{ steps.native_rust_workspace.outputs.path }} -> target
|
||||
shared-key: release-native-v2-${{ hashFiles('packages/paperclip-runner/runner/Cargo.toml', '.cargo/config', '.cargo/config.toml', 'packages/.cargo/config', 'packages/.cargo/config.toml', 'packages/paperclip-runner/.cargo/config', 'packages/paperclip-runner/.cargo/config.toml', 'packages/paperclip-runner/runner/.cargo/config', 'packages/paperclip-runner/runner/.cargo/config.toml') }}-${{ matrix.host }}-${{ matrix.target }}-${{ hashFiles('packages/paperclip-runner/protocol/**', 'packages/paperclip-runner/src/**', 'packages/paperclip-runner/*.json') }}
|
||||
shared-key: release-native-v2-${{ hashFiles('packages/paperclip-runner/runner/Cargo.toml', '.cargo/config', '.cargo/config.toml', 'packages/.cargo/config', 'packages/.cargo/config.toml', 'packages/paperclip-runner/.cargo/config', 'packages/paperclip-runner/.cargo/config.toml', 'packages/paperclip-runner/runner/.cargo/config', 'packages/paperclip-runner/runner/.cargo/config.toml') }}-${{ matrix.host }}-${{ matrix.target }}
|
||||
cache-workspace-crates: false
|
||||
cache-bin: false
|
||||
cache-on-failure: false
|
||||
|
||||
@@ -345,10 +345,10 @@ jobs:
|
||||
run: node scripts/cloud-source-verification.mjs "$SOURCE_SHA"
|
||||
|
||||
release_runner_source:
|
||||
needs: [verify_canary, select_nightly, select_beta, preflight_stable]
|
||||
needs: [select_nightly, select_beta, preflight_stable]
|
||||
if: >-
|
||||
!cancelled() && (
|
||||
(github.event_name == 'push' && needs.verify_canary.result == 'success') ||
|
||||
(github.event_name == 'push' && github.repository == 'paperclipai/paperclip' && github.repository_id == '1170821064' && github.ref == 'refs/heads/master') ||
|
||||
(needs.select_nightly.result == 'success' && needs.select_nightly.outputs.proceed == 'true') ||
|
||||
needs.select_beta.result == 'success' || needs.preflight_stable.result == 'success')
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
Reference in new issue
Block a user