Integrate current master Docker cache and run-event compatibility

* codex/work-folders-remote-recovery-refresh:
  ci: cache compiled Docker Rust dependencies separately from source (#13329)
  ci: rebalance serialized tests with current suite durations (#13328)
  fix: preserve NUL characters in run-event payloads (#13325)
  ci: run release Runner protocol and Rust checks in parallel (#13326)

Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip committed 2026-09-12 14:21:35 -05:00
commit 08dd27ecb0
15 files changed
+554 -182

No files matched your search

+24 -8
View File
@@ -342,11 +342,20 @@ Notes:
## Native Runner build cache
The image compiles the native Runner in `runner-build`, before copying the
application source. That stage includes the pinned Rust compiler, the complete
Cargo workspace and lockfile, and the protocol schemas and fixtures embedded
by Rust. Changes to those inputs rebuild the native binary. Ordinary server or
UI changes can reuse it through the existing registry cache (`mode=max`). Each
platform gets its own native build; no cross-architecture binary is reused.
application source. A pinned `cargo-chef` generates a dependency recipe in
`runner-plan`. The separate `runner-deps` stage compiles that recipe with the
package-owned Rust compiler. Both the dependency build and the real binary use
the release profile and locked Cargo dependencies. The recipe stage never
modifies source in the checkout.
Changes to Rust source or embedded protocol inputs rebuild the real binary but
can reuse compiled dependencies when the recipe is unchanged. Dependency
manifests, the Cargo lockfile, target metadata, or compiler changes invalidate
the relevant cache. Ordinary server or UI changes can reuse the entire native
build through the existing registry cache (`mode=max`). Each platform gets its
own native build; no cross-architecture binary is reused. No additional GitHub
Actions cache is created. A cold build also installs the recipe generator and
compiles dependencies, so the savings apply after those layers are available.
The application build inherits that stage and still runs the normal server
build, including Cargo, binary staging, and generated-contract checks. Rust
@@ -357,6 +366,13 @@ directory as before. Cache misses only cost compilation time.
Pull requests that change the Dockerfile, Docker ignore rules, or Runner native
inputs also build the isolated `runner-build` target in `Docker Runner check`.
This compiles against the actual reduced context and catches missing embedded
inputs before the post-merge image build. It uses a GitHub-hosted runner with
read-only repository access and does not publish images or cache artifacts.
The check runs `bash scripts/check-docker-runner-cache.sh` against a disposable
copy of tracked source and the actual Docker ignore rules. It compiles a baseline,
changes a Rust metadata constant, and rebuilds. It requires a cached dependency
build, an unchanged dependency recipe, and changed metadata from the real binary.
It also verifies that a dependency declaration change alters the recipe. The
probe exports only small metadata files, avoiding a large image import into the
Docker daemon. It catches missing embedded inputs before the post-merge build.
It uses a GitHub-hosted runner with read-only repository access and does not
publish images or cache artifacts. Allow up to 20 minutes for its cold build and
source rebuild.
+20 -12
View File
@@ -338,19 +338,27 @@ Check:
## Runner verification dependency cache
`release-verify.yml` caches Cargo dependencies for its `Verify Paperclip Runner`
job using a pinned Rust Cache action. It selects the compiler from the Runner
package's `rust-toolchain.toml` before computing the cache key. Compiler and Cargo
metadata changes select a new cache; the `release-runner-v1` shared key lets
callers of this reusable verification workflow reuse the same dependency cache.
`release-verify.yml` runs `Verify Paperclip Runner` on two independent runners.
The protocol lane runs `check:eval-kernel` and `check:protocol`. The Rust lane
runs `check:runner` and `check:api-authority`. Together they retain every check
in `check:all`; both lanes must pass before Cloud source verification or
readiness can succeed. A failed lane does not cancel the other lane.
Workspace crates and installed Cargo binaries are excluded. Every run still
builds the Runner workspace and runs `check:all`, including the Rust and
TypeScript tests. Only an own-repository master-push run verifying that push's exact
SHA can restore the cache, and only a successful run saves it. PR, tag, and
manual candidate verification compile without this cache. A miss or eviction costs compilation time but does not change the checks.
To discard old dependency caches, increment the shared-key version and let the
next successful master verification warm it again.
Both lanes restore Cargo dependencies with the pinned Rust Cache action. The
compiler comes from the Runner package's `rust-toolchain.toml` before the action
computes its key. Compiler and Cargo metadata changes select a new cache. The
existing `release-runner-v1` shared key avoids separate copies for these lanes.
Only the Rust lane saves this cache. After verification it also runs `build:rust`
to warm the debug dependencies used by the protocol lane; its own tests already
warm release dependencies. The cache writer is shorter than the protocol lane.
Workspace crates and installed Cargo binaries are excluded. Every run rebuilds
workspace code and runs all assigned checks, including on a cache hit. Only an
own-repository master-push run verifying that push's exact SHA can restore the
cache, and only a successful Rust lane saves it. PR, tag, and manual candidate
verification compile without this cache. A miss or eviction costs compilation
time but does not change the checks. To discard old dependency caches, increment
the shared-key version and let the next successful master verification warm it.
The trust boundary is the protected master branch, not the cache-key text.
GitHub does not let master restore caches created by a child branch, sibling
+11
View File
@@ -13,6 +13,17 @@ the PRP `eventType`, source instance, source event ID, source sequence, protocol
schema version, and a SHA-256 digest of the canonical source envelope. Its
payload is `{ "prpEvent": <canonical PRP event> }`.
PostgreSQL JSONB cannot represent NUL (U+0000), which can occur in command
output such as Vite virtual-module paths. The run-event payload column uses a
lossless storage codec for these events: the JSONB projection renders NUL as
the literal `\u0000`, and the reserved `$paperclipRunEventJsonV1` field contains
the original serialized JSON as a doubly escaped string. Ordinary payloads
retain their existing representation. Drizzle reads restore the exact original
payload before replay, hash validation, redaction, or API presentation. SQL
queries can still inspect ordinary routing fields in the projection; raw SQL
readers of the whole payload must apply `decodeRunEventPayload`. The column
remains JSONB and requires no schema migration.
The writer locks the native `heartbeat_runs` row and allocates the existing
per-run `seq` cursor. A byte-equivalent retry reuses the first row; a changed
retry or source-sequence gap is rejected. Company, issue, agent, run, session,