Native ingestion experiment helpers

These helpers reproduce the experiment registered on November 3, 2026. They retain its immutable application revisions, harness revision, workload hashes, acceptance checks and failure behavior. They are not a benchmark against an arbitrary current checkout. Measurements and their qualification status belong in the dated plan and evidence report; copying these helpers proves no gain.

Provenance

The source helpers were snapshotted from the following task-owned files. The hashes describe the originals before the small packaging changes below.

Committed file	Original in `/private/tmp/`	Original SHA256
`run-frozen.mjs`	`native-ingestion-run-frozen.mjs`	`09879a9d6a4554b98bd0435cdc222915b0613cb9dfb22b36615ede8794f70435`
`diagnostic-run.mjs`	`native-ingestion-diagnostic-run.mjs`	`06cc3de3b67dcaa97dfa90f57562fd72b4fdf39b4f9e33c2cac543d569087681`
`diagnostic-preload.mjs`	`native-ingestion-diagnostic-preload.mjs`	`2afb5e0b9fff878cb7abc7e8a4e236e1620ac35eede9ca06accfec845756d10a`
`verify-delivery-runtime.py`	`native-ingestion-verify-delivery-runtime.py`	`a2d9d2fe4ed2e2b0bff61b9d264f1678b84266d0407d43e19cf9f4063819a7d4`

Packaging changes: the diagnostic runner imports its renamed adjacent diagnostic-preload.mjs; its dry-run message uses the committed path; the acceptance runner rejects both the original and packaged preload filenames in NODE_OPTIONS. Application, protocol, compiled-anchor and gate hashes are unchanged. The runtime comparison helper and diagnostic preload are exact copies.

The frozen experiment revisions are:

Role	Revision
B0 baseline	`83b5f7d486e70e484acd9a41bc957054f4e10175`
B1 cursor	`7922802f069ab3b39e9f79368c46615ba2201ba9`
B2 cursor and batching	`6afeba4edafd47120e8117f736641d5424b27d4a`
H browser driver	`f5c560d80d9d4ee089297d3b3f300c052db17c9e`

H is a separate test-driver checkout. Folding its test changes into a later delivery commit does not change the recorded H revision. Check out these exact revisions to repeat this experiment; do not update the constants to pass a guard.

Preparation and quiet windows

Use Node 26.4.0 and pnpm 9.15.4, matching the captured experiment environment. Install each measured application independently with pnpm install --frozen-lockfile and run its normal pnpm build. Every application owns its workspace dependencies, TypeScript output and UI output. The build stamp must equal that checkout's exact revision. The original experiment shared only a verified identical Rust target cache; cache sharing is optional.

The historical H setup linked only its root and server node_modules to B1's locked driver tools. H did not supply application source or compiled output to any measured application. The runner records this setup in the evidence's PAPERCLIP_INGESTION_DRIVER_DEPENDENCIES description. Preserve it when reproducing the historical runner. Install the matching Playwright Chromium build first.

Select a disposable quiet host with PostgreSQL and Chromium available. The service harness starts a real embedded PostgreSQL database; browser cases start the compiled application and shipped UI against a disposable database. No cloud account, production data or provider credential is required. Run one stage at a time, with no competing build, test, profiler or benchmark. Setup and warmup are excluded from timed append samples. The production-built browser experiment includes offered-load queueing and terminal backlog drainage.

Set the following shell variables to your own absolute checkout and output paths. TOOLS_CHECKOUT contains this committed helper directory. B1_CHECKOUT, B0_CHECKOUT, B2_CHECKOUT and H_CHECKOUT contain the exact revisions above. Choose a new output directory for every independent experiment; existing evidence is never overwritten. Always supply the roots explicitly because the fallback paths preserve the historical local setup.

node "$TOOLS_CHECKOUT/scripts/native-ingestion/run-frozen.mjs" \
  --root="$B1_CHECKOUT" --b0-root="$B0_CHECKOUT" --b2-root="$B2_CHECKOUT" \
  --harness-root="$H_CHECKOUT" --output-root="$EVIDENCE_DIR"

Without --execute, this prints the registered stages and launches no work. After qualification and release of a quiet window, add --execute and one --stage=<name> from the following sequence:

service-aa-b0, service-b0-b1, service-aa-b1, service-b1-b2, service-b0-b2.
browser-smoke-b0, browser-smoke-b1, browser-smoke-b2, then browser-full-qualification-b0.
browser-aa-b0, browser-b0-b1, browser-aa-b1, browser-b1-b2, browser-b0-b2.

Stop on failure, retain its evidence, and diagnose before continuing. A timeout or cleanup error is incomplete evidence. The process watchdogs bound resources; they do not relax performance gates. Readiness must admit all 100 participants before the common emission epoch in the full browser workload.

Acceptance and interpretation

Service A/A requires five alternating pairs for all nine latency cases: ordered, replay and warm-suffix at 1k, 10k and 100k retained histories. Every case must have at most 20% maximum paired relative difference and 10% median arm drift. Each timed sample contains 16 groups of 32 events. Late-gap closure has a separate structural/correctness check; its one-shot timing is not a latency gate.

Candidate improvement must exceed the observed per-case A/A noise in at least four of five individual pairs and in the median. Cursor comparisons require 10k/100k improvement and 1k nonregression. Browser comparisons preserve sparse streaming, inter-chunk, first-render and Stop nonregression gates. The batching and combined comparisons additionally require improvement for loaded burst512 p95 emission-to-render, p95 intended-to-render and terminal drainage. Every background run requires independent durable delta proof. Browser A/A suitability still requires review of its actual noise; successful execution alone is not a repeatability conclusion.

These are local service/application measurements. They establish neither deployed production improvement nor qualification of a 10,000-agent fleet. The live-provider integration journey is separate. Do not interpret sparse raw transport output as universally durable merely because its wire ACK was sent.

Separate diagnostics

node "$TOOLS_CHECKOUT/scripts/native-ingestion/diagnostic-run.mjs" \
  --root="$B1_CHECKOUT" --b0-root="$B0_CHECKOUT" --b2-root="$B2_CHECKOUT" \
  --harness-root="$H_CHECKOUT" --tree=B2 --output=/tmp/native-ingestion-profile-new

This is also a dry run until --execute is supplied. Start with --smoke to qualify the assembled diagnostic observer; it has no CPU profile and cannot establish acceptance. Full diagnostics require successful marker coverage and all compiled instrumentation anchors to be exercised. Their qualification must be recorded separately from their presence in this directory.

The observer records actual normalized queue depth, postcommit source markers, SQL/transaction attempts without SQL text or parameters, server Node CPU/memory, event-loop delay, and sampled PostgreSQL process CPU/RSS. Full runs also capture an app-only Node CPU profile. The outputs are explicitly diagnosticOnly and ineligible for unprofiled acceptance. Queue observation excludes collector, iterator, in-flight write and raw transport buffers. CPU/SQL window brackets include disclosed sampling overshoot and may include verification reads. PostgreSQL RSS sums double-count shared pages. Server Node CPU excludes Rust, providers, Chromium and the external driver. Preserve these limits with any chart or table derived from the diagnostic outputs.

The diagnostic output must be a new owned directory under /tmp/ or /private/tmp/. The test-only provider wrapper leaves production environment sanitization unchanged. Unknown compiled-module SHA values fail immediately; do not add an unreviewed hash to make a different build run.

Delivery-source comparison

python3 "$TOOLS_CHECKOUT/scripts/native-ingestion/verify-delivery-runtime.py" \
  --repo="$TOOLS_CHECKOUT" \
  --frozen=7922802f069ab3b39e9f79368c46615ba2201ba9 \
  --delivery=3b77c1ca29467fbd742e9b6df4885bccef0acf08

For B2, supply its frozen and delivery commits instead. This is a read-only Git source/input comparison with the two documented test-typecheck script exceptions and genuine CI helpers pinned to 8ab89f41e499032e3e55d261f457a3463d754a1d. It does not compare installed dependencies or compiled artifacts; those require their separate build/equivalence receipts. Later delivery revisions must remain explicit full commit IDs.

Evidence retention

Keep paired samples, exact command manifests, source/protocol hashes, environment, correctness and cleanup results together. Keep diagnostic evidence separate. Retain failed qualifications and their primary errors, record retry linkage and confidence in the diagnosis, and never promote a failed report by rewriting its status. Publish compact JSON and interpretation with the owning PR; link raw artifacts by a durable location and SHA256. Temporary local paths alone are not a reviewable final evidence location. Scan live-provider artifacts for credentials before publishing; these deterministic helpers require none.

9.2 KiB Raw Blame History