mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-11 13:24:55 +01:00
Integrate current master Docker cache and run-event compatibility
* codex/work-folders-remote-recovery-refresh: ci: cache compiled Docker Rust dependencies separately from source (#13329) ci: rebalance serialized tests with current suite durations (#13328) fix: preserve NUL characters in run-event payloads (#13325) ci: run release Runner protocol and Rust checks in parallel (#13326) Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
commit
08dd27ecb0
15 files changed
+554
-182
No files matched your search
+24
-8
@@ -342,11 +342,20 @@ Notes:
|
||||
## Native Runner build cache
|
||||
|
||||
The image compiles the native Runner in `runner-build`, before copying the
|
||||
application source. That stage includes the pinned Rust compiler, the complete
|
||||
Cargo workspace and lockfile, and the protocol schemas and fixtures embedded
|
||||
by Rust. Changes to those inputs rebuild the native binary. Ordinary server or
|
||||
UI changes can reuse it through the existing registry cache (`mode=max`). Each
|
||||
platform gets its own native build; no cross-architecture binary is reused.
|
||||
application source. A pinned `cargo-chef` generates a dependency recipe in
|
||||
`runner-plan`. The separate `runner-deps` stage compiles that recipe with the
|
||||
package-owned Rust compiler. Both the dependency build and the real binary use
|
||||
the release profile and locked Cargo dependencies. The recipe stage never
|
||||
modifies source in the checkout.
|
||||
|
||||
Changes to Rust source or embedded protocol inputs rebuild the real binary but
|
||||
can reuse compiled dependencies when the recipe is unchanged. Dependency
|
||||
manifests, the Cargo lockfile, target metadata, or compiler changes invalidate
|
||||
the relevant cache. Ordinary server or UI changes can reuse the entire native
|
||||
build through the existing registry cache (`mode=max`). Each platform gets its
|
||||
own native build; no cross-architecture binary is reused. No additional GitHub
|
||||
Actions cache is created. A cold build also installs the recipe generator and
|
||||
compiles dependencies, so the savings apply after those layers are available.
|
||||
|
||||
The application build inherits that stage and still runs the normal server
|
||||
build, including Cargo, binary staging, and generated-contract checks. Rust
|
||||
@@ -357,6 +366,13 @@ directory as before. Cache misses only cost compilation time.
|
||||
|
||||
Pull requests that change the Dockerfile, Docker ignore rules, or Runner native
|
||||
inputs also build the isolated `runner-build` target in `Docker Runner check`.
|
||||
This compiles against the actual reduced context and catches missing embedded
|
||||
inputs before the post-merge image build. It uses a GitHub-hosted runner with
|
||||
read-only repository access and does not publish images or cache artifacts.
|
||||
The check runs `bash scripts/check-docker-runner-cache.sh` against a disposable
|
||||
copy of tracked source and the actual Docker ignore rules. It compiles a baseline,
|
||||
changes a Rust metadata constant, and rebuilds. It requires a cached dependency
|
||||
build, an unchanged dependency recipe, and changed metadata from the real binary.
|
||||
It also verifies that a dependency declaration change alters the recipe. The
|
||||
probe exports only small metadata files, avoiding a large image import into the
|
||||
Docker daemon. It catches missing embedded inputs before the post-merge build.
|
||||
It uses a GitHub-hosted runner with read-only repository access and does not
|
||||
publish images or cache artifacts. Allow up to 20 minutes for its cold build and
|
||||
source rebuild.
|
||||
@@ -338,19 +338,27 @@ Check:
|
||||
|
||||
## Runner verification dependency cache
|
||||
|
||||
`release-verify.yml` caches Cargo dependencies for its `Verify Paperclip Runner`
|
||||
job using a pinned Rust Cache action. It selects the compiler from the Runner
|
||||
package's `rust-toolchain.toml` before computing the cache key. Compiler and Cargo
|
||||
metadata changes select a new cache; the `release-runner-v1` shared key lets
|
||||
callers of this reusable verification workflow reuse the same dependency cache.
|
||||
`release-verify.yml` runs `Verify Paperclip Runner` on two independent runners.
|
||||
The protocol lane runs `check:eval-kernel` and `check:protocol`. The Rust lane
|
||||
runs `check:runner` and `check:api-authority`. Together they retain every check
|
||||
in `check:all`; both lanes must pass before Cloud source verification or
|
||||
readiness can succeed. A failed lane does not cancel the other lane.
|
||||
|
||||
Workspace crates and installed Cargo binaries are excluded. Every run still
|
||||
builds the Runner workspace and runs `check:all`, including the Rust and
|
||||
TypeScript tests. Only an own-repository master-push run verifying that push's exact
|
||||
SHA can restore the cache, and only a successful run saves it. PR, tag, and
|
||||
manual candidate verification compile without this cache. A miss or eviction costs compilation time but does not change the checks.
|
||||
To discard old dependency caches, increment the shared-key version and let the
|
||||
next successful master verification warm it again.
|
||||
Both lanes restore Cargo dependencies with the pinned Rust Cache action. The
|
||||
compiler comes from the Runner package's `rust-toolchain.toml` before the action
|
||||
computes its key. Compiler and Cargo metadata changes select a new cache. The
|
||||
existing `release-runner-v1` shared key avoids separate copies for these lanes.
|
||||
Only the Rust lane saves this cache. After verification it also runs `build:rust`
|
||||
to warm the debug dependencies used by the protocol lane; its own tests already
|
||||
warm release dependencies. The cache writer is shorter than the protocol lane.
|
||||
|
||||
Workspace crates and installed Cargo binaries are excluded. Every run rebuilds
|
||||
workspace code and runs all assigned checks, including on a cache hit. Only an
|
||||
own-repository master-push run verifying that push's exact SHA can restore the
|
||||
cache, and only a successful Rust lane saves it. PR, tag, and manual candidate
|
||||
verification compile without this cache. A miss or eviction costs compilation
|
||||
time but does not change the checks. To discard old dependency caches, increment
|
||||
the shared-key version and let the next successful master verification warm it.
|
||||
|
||||
The trust boundary is the protected master branch, not the cache-key text.
|
||||
GitHub does not let master restore caches created by a child branch, sibling
|
||||
|
||||
@@ -13,6 +13,17 @@ the PRP `eventType`, source instance, source event ID, source sequence, protocol
|
||||
schema version, and a SHA-256 digest of the canonical source envelope. Its
|
||||
payload is `{ "prpEvent": <canonical PRP event> }`.
|
||||
|
||||
PostgreSQL JSONB cannot represent NUL (U+0000), which can occur in command
|
||||
output such as Vite virtual-module paths. The run-event payload column uses a
|
||||
lossless storage codec for these events: the JSONB projection renders NUL as
|
||||
the literal `\u0000`, and the reserved `$paperclipRunEventJsonV1` field contains
|
||||
the original serialized JSON as a doubly escaped string. Ordinary payloads
|
||||
retain their existing representation. Drizzle reads restore the exact original
|
||||
payload before replay, hash validation, redaction, or API presentation. SQL
|
||||
queries can still inspect ordinary routing fields in the projection; raw SQL
|
||||
readers of the whole payload must apply `decodeRunEventPayload`. The column
|
||||
remains JSONB and requires no schema migration.
|
||||
|
||||
The writer locks the native `heartbeat_runs` row and allocates the existing
|
||||
per-run `seq` cursor. A byte-equivalent retry reuses the first row; a changed
|
||||
retry or source-sequence gap is rejected. Company, issue, agent, run, session,
|
||||
|
||||
Reference in new issue
Block a user