Files
PaperClipAI/tests/runner-e2e/history-publish.ts
T
DottaandPaperclip 3bd709df8b Operationalize Product E2E accounting, tracing, and scheduling (#17009)
## Thinking Path

> - Paperclip manages agents and the work they perform.
> - Product E2E tests exercise real agents, tools, setup and cleanup.
> - Repeated runs need retained costs and traces, including failed
attempts.
> - A passing retry must not hide earlier spending or feedback.
> - Recurring work also needs admission limits, private reports and
failure detection.
> - This change supplies those controls while keeping paid activation
behind existing gates.

## Linked Issues or Issue Description

Refs #16996 and #17158. This is the consolidated landing of the reviewed
eval operations stack: #17019, #17024, #17025, #17027, #17030, #17033,
#17035, #17038, #17043, #17044, #17048, #17050, #17051, #17092, #17096,
#17100, #17107, #17109, #17112, #17115, #17117, #17121, #17123, #17127,
#17130, #17133, #17134, #17137, #17140, #17143, #17154. The repository
permits squash merges only. A local rehearsal showed that squashing each
slice into master causes repeated conflicts in dependent slices. This PR
preserves their commits and lands their combined implementation in one
reviewed integration. Superseded slices will close after this merge.

## What Changed

- Retain every attempt's cost receipt, including setup, failures,
interruptions and retries. Separate reported amounts, estimates, missing
costs and reserved capacity in reports and dashboards.
- Add bounded request admission, exact model pricecards, gateway error
accounting, cache policy and reconciliation evidence. Refuse unsupported
routes and requests that cannot fit the allowance.
- Integrate Braintrust gateway traces and OTLP correlation, with secret
filtering and experiment imports. Record phase timing, including setup,
feedback capture and cleanup.
- Collect complaint and suggestion evidence after each attempt,
including recovery after failures. Retain explicit review dispositions
independently from the product grade.
- Add gated daily and weekly selections, measured forecasts,
setup/readiness checks, wall-time limits and retained monitor
notifications. Include declared non-model overhead in admission while
reporting the existing subscription separately.
- Keep recurring history private, bound uploads and retention, and use
isolated archive environments. Keep public publication an explicit
operation. Report and publication jobs install only the committed
lockfile, with scripts disabled and isolated stores; stale pins fail
closed.
- Integrate current master, preserve its Product E2E journey billing and
privacy checks, and test that shared setup receipts do not double-count
estimated agent usage. Preserve upstream fixture repairs.

## Verification

- Complete integration baseline
`0649a5ce7544ab20825c2183d2f421157db49f24` includes master
`a3e1d915a7110a4e115b46396a3f79e6dfb4d05e` and every original stack head
as ancestors.
- On that integration baseline, `pnpm test:e2e:runner:unit`: 4,012
Vitest tests passed, two skipped, across 193 files; all 137 Node
source-contract tests passed.
- Focused billing and Product E2E journey contract checks: 24 tests
passed. They cover estimates, unknown receipts, single-owner setup
costs, browser-only execution and private publication guards.
- Monitor and workflow wiring: all 96 Node tests passed. `node
scripts/check-token-gates.mjs`: all gates clean.
- `pnpm test:e2e:runner:typecheck` passes after the standard Runner
TypeScript build refreshed local declarations. The first attempt
rejected the new master `runner_prepared_launch` union member because
this checkout retained an older generated declaration; the source union
already contained that member.
- Current head `492b0708a1c531a023d9ac10f0f3dc6383cc1754` removes
dependency re-resolution from archive-authorized reporting jobs after
the security scan flagged it. All 15 workflow-security tests pass after
updating the old two-install assertion.
- Hosted [CI run
39965460180](https://github.com/paperclipai/paperclip/actions/runs/39965460180)
passes on the current repaired head, including typecheck, build, general
tests, Runner checks, browser shards and canary dry run. Greptile is 5/5
on this exact head with no unresolved threads. The security scan passes
after the fixed lockfile-resolution thread was resolved.
- Paperclip Review returned a terminal neutral check but an incomplete
0-file assessment on the repaired head: its platform blocked both diff
responses as prompt-injection content. This is not a successful code
review. No bypass or fabricated pass was attempted; current-head
Greptile, security and hosted CI are independent gates.
- No paid test, reservation release or recurring-run activation
accompanies this landing. Live route qualification and invoice
reconciliation remain separate evidence.

<details>
<summary>Retained earlier validation failures</summary>

Earlier broad local runs had two 15-second test timeouts (CLI poll
cadence and Retry validation), one embedded-database setup timeout, and
later PostgreSQL semaphore exhaustion. The exact timed-out tests passed
separately without larger timeouts; the broad-run causes were not
established. Those local failures remain distinct from subsequent hosted
CI results. The independent #17158 fix records its Muse fixture
collision, its UUID correction and hosted verification. Local Muse
execution was also blocked by macOS embedded-PostgreSQL startup. No
historical failure is treated as a successful paid qualification.

</details>

## Risks

- Admission is a local control, not a provider invoice cap. Missing
provider receipts or resource charges retain unknown amounts and holds;
estimates do not become billed dollars.
- The bounded path admits only documented local suites and provider
routes. Ordinary legacy or separate Runner protocol campaigns have
different controls. Merging this code does not qualify additional
routes.
- Recurrence still needs the existing enable gates and current
qualification/allowance evidence. Archive permissions and resources must
match the documented private-environment contract.
- Legacy reports may lack complete attempt or phase evidence. Reports
preserve that limitation and do not infer zero usage from missing data.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository tools and code
execution. The exact deployment variant and context-window size are not
exposed by this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (targeted checks;
full-suite host limitation disclosed above)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-23 07:18:37 -07:00

781 lines
26 KiB
TypeScript

import { validateCampaignAccounting, validateRetainedRunnerResult } from "./result-validation.js";
import { validateEvalOperationsReport } from "./operations-report.js";
import { validateCampaignFeedbackReview } from "./feedback-dispositions.js";
import { createHash } from "node:crypto";
import { execFile } from "node:child_process";
import path from "node:path";
import { promisify } from "node:util";
import {
cp,
copyFile,
mkdtemp,
lstat,
mkdir,
readFile,
readdir,
realpath,
rm,
stat,
writeFile,
} from "node:fs/promises";
import os from "node:os";
import { writePublicCampaignSummaryImage } from "./public-summary-image.js";
import { regenerateRunnerDashboard } from "./dashboard-regenerate.js";
import { renderRunnerHistoryIndex } from "./history-index.js";
import { validateHistoryDestination } from "./history-destination.js";
import { PUBLIC_RUNNER_SCREENSHOT_MARKER } from "./screenshot-policy.js";
import { BoundedHistoryPublisher, checkHistoryPayload, HISTORY_LIMITS } from "./history-budget.js";
import {
campaignHistoryRecord,
emptyRunnerHistory,
mergeRunnerHistory,
} from "./history.js";
import type { RunnerE2ECampaign, RunnerE2EHistoryIndex } from "./types.js";
const execFileAsync = promisify(execFile);
const repositoryRoot = path.resolve(import.meta.dirname, "../..");
const MUTABLE_HISTORY_FILES = new Set([
"history.json",
"latest.json",
"latest-green.json",
]);
const PUBLISH_ROOT_FILES = new Set([
"dashboard.html",
"index.html",
"junit.xml",
"normalized-results.json",
"summary.md",
]);
const MAX_PUBLIC_RASTER_BYTES = 12 * 1024 * 1024;
const EMPTY_PUBLIC_SCREENSHOTS: ReadonlySet<string> = new Set();
const PRIVATE_EVIDENCE_DIRECTORIES = new Set([
"blob-report",
"html-report",
"playwright-output",
]);
function publicEvidencePath(
relative: string,
publicScreenshots: ReadonlySet<string> = EMPTY_PUBLIC_SCREENSHOTS,
) {
const match = relative.match(
/^evidence\/[A-Za-z0-9._-]+\/attempt-[1-9][0-9]*\/(.+)$/,
);
if (!match) return false;
const evidencePath = match[1]!;
const segments = evidencePath.split("/");
if (
segments.some(
(segment) =>
!segment ||
segment === "." ||
segment === ".." ||
PRIVATE_EVIDENCE_DIRECTORIES.has(segment),
)
) {
return false;
}
// A text extension and credential scan do not establish a public projection.
// API snapshots and process logs can still hold hidden reasoning and provider
// session identities. Keep those diagnostics in the retained Actions artifact;
// normalized-results.json already carries the graded result contract.
return (
path.posix.extname(evidencePath).toLowerCase() === ".png" &&
publicScreenshots.has(relative)
);
}
export function publicScreenshotPaths(campaign: RunnerE2ECampaign) {
assertPublicCampaign(campaign);
const screenshots = new Set<string>();
for (const result of campaign.results) {
const files =
result.screenshots
?.filter(
(screenshot) =>
screenshot.publication === PUBLIC_RUNNER_SCREENSHOT_MARKER,
)
.map((screenshot) => screenshot.file) ?? [];
if (files.length === 0) continue;
if (
!/^[A-Za-z0-9][A-Za-z0-9._-]{0,299}$/.test(result.executionId) ||
!Number.isSafeInteger(result.attempt) ||
result.attempt < 1
) {
throw new Error(
`Cannot publish screenshots for unsafe execution identity ${result.executionId}`,
);
}
for (const file of files) {
if (!/^[A-Za-z0-9][A-Za-z0-9._-]{0,199}\.png$/i.test(file)) {
throw new Error(
`Cannot publish unsafe screenshot path for ${result.executionId}: ${file}`,
);
}
screenshots.add(
`evidence/${result.executionId}/attempt-${result.attempt}/${file}`,
);
}
}
return screenshots;
}
export function assertPublicCampaign(campaign: Pick<RunnerE2ECampaign, "results">) {
if (campaign.results.some(result => result.productJourney?.privacy === "private")) {
throw new Error("Private Product E2E campaigns cannot enter the public history publisher");
}
}
export function isHistoricalBundlePathAllowed(
relative: string,
allowPublicSummary = false,
publicScreenshots: ReadonlySet<string> = EMPTY_PUBLIC_SCREENSHOTS,
) {
if (
relative.includes("\\") ||
relative.startsWith("/") ||
relative.includes("..")
) {
return false;
}
if (PUBLISH_ROOT_FILES.has(relative)) return true;
if (allowPublicSummary && relative === "public-images/campaign-summary.png") {
return true;
}
if (
relative === "assets/favicon-32x32.png" ||
relative === "assets/InterVariable.woff2"
) {
return true;
}
return publicEvidencePath(relative, publicScreenshots);
}
function hasPublicPngMagic(content: Buffer) {
return (
content.length >= 24 &&
content
.subarray(0, 8)
.equals(Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a])) &&
content.subarray(12, 16).equals(Buffer.from("IHDR", "ascii"))
);
}
async function validatePublicRaster(absolute: string, relative: string) {
const metadata = await stat(absolute);
if (metadata.size === 0 || metadata.size > MAX_PUBLIC_RASTER_BYTES) {
throw new Error(
`Public screenshot ${relative} exceeds the per-file size boundary`,
);
}
const content = await readFile(absolute);
if (!hasPublicPngMagic(content)) {
throw new Error(
`Public screenshot ${relative} does not match its raster file type`,
);
}
}
async function pruneEvidenceDirectory(
root: string,
current: string,
publicScreenshots: ReadonlySet<string>,
) {
const entries = await readdir(current, { withFileTypes: true });
for (const entry of entries) {
const absolute = path.join(current, entry.name);
if (entry.isDirectory()) {
await pruneEvidenceDirectory(root, absolute, publicScreenshots);
if ((await readdir(absolute)).length === 0) {
await rm(absolute, { recursive: true });
}
continue;
}
const relative = path.relative(root, absolute).split(path.sep).join("/");
if (!entry.isFile() || !publicEvidencePath(relative, publicScreenshots)) {
await rm(absolute, { force: true });
}
}
}
export async function prunePrivateHistoryEvidence(
root: string,
publicScreenshots: ReadonlySet<string> = EMPTY_PUBLIC_SCREENSHOTS,
) {
const evidenceRoot = path.join(root, "evidence");
const metadata = await lstat(evidenceRoot).catch(() => null);
if (!metadata) return;
if (!metadata.isDirectory()) {
throw new Error("Historical evidence root must be a directory");
}
await pruneEvidenceDirectory(root, evidenceRoot, publicScreenshots);
}
export async function stageTrustedHistoryAssets(
root: string,
trustedRoot = repositoryRoot,
) {
const resolveTrustedAsset = async (segments: string[]) => {
const trustedRootReal = await realpath(trustedRoot);
let source = trustedRoot;
for (const [index, segment] of segments.entries()) {
source = path.join(source, segment);
const metadata = await lstat(source);
if (metadata.isSymbolicLink()) {
throw new Error(
`Refusing symbolic link in trusted publisher asset path ${segments.join("/")}`,
);
}
const final = index === segments.length - 1;
if (
(final && !metadata.isFile()) ||
(!final && !metadata.isDirectory())
) {
throw new Error(
`Trusted publisher asset path has an invalid file type: ${segments.join("/")}`,
);
}
}
const sourceReal = await realpath(source);
const relative = path.relative(trustedRootReal, sourceReal);
if (
!relative ||
relative === ".." ||
relative.startsWith(`..${path.sep}`)
) {
throw new Error(
`Trusted publisher asset escapes its checkout: ${segments.join("/")}`,
);
}
return sourceReal;
};
const [faviconSource, fontSource] = await Promise.all([
resolveTrustedAsset(["ui", "public", "favicon-32x32.png"]),
resolveTrustedAsset(["ui", "public", "fonts", "InterVariable.woff2"]),
]);
const assets = path.join(root, "assets");
await rm(assets, { recursive: true, force: true });
await mkdir(assets, { recursive: true });
await Promise.all([
copyFile(faviconSource, path.join(assets, "favicon-32x32.png")),
copyFile(fontSource, path.join(assets, "InterVariable.woff2")),
]);
}
interface BundleManifest {
schema: "paperclip.runner-e2e.bundle/v1";
campaignId: string;
bundleDigest: string;
files: Array<{ path: string; sha256: string; bytes: number }>;
}
function json(value: unknown) {
return `${JSON.stringify(value, null, 2)}\n`;
}
export { validateHistoryDestination };
async function relativeFiles(
root: string,
current = root,
allowPublicSummary = false,
publicScreenshots: ReadonlySet<string> = EMPTY_PUBLIC_SCREENSHOTS,
boundary?: { entries: number },
): Promise<string[]> {
const entries = await readdir(current, { withFileTypes: true });
const files: string[] = [];
for (const entry of entries) {
if (boundary && ++boundary.entries > 1024) throw new Error("Public bundle exceeds the directory-entry boundary");
const absolute = path.join(current, entry.name);
if (entry.isSymbolicLink()) {
throw new Error(`Refusing to publish symbolic link ${entry.name}`);
}
if (entry.isDirectory()) {
files.push(
...(await relativeFiles(
root,
absolute,
allowPublicSummary,
publicScreenshots,
boundary,
)),
);
} else if (entry.isFile()) {
const relative = path.relative(root, absolute).split(path.sep).join("/");
if (MUTABLE_HISTORY_FILES.has(relative)) continue;
if (
!isHistoricalBundlePathAllowed(
relative,
allowPublicSummary,
publicScreenshots,
)
) {
throw new Error(
`Refusing non-allowlisted historical bundle path ${relative}`,
);
}
files.push(relative);
}
}
return files;
}
export async function createBundleManifest(
root: string,
campaignId: string,
allowPublicSummary = false,
publicScreenshots: ReadonlySet<string> = EMPTY_PUBLIC_SCREENSHOTS,
bounded = false,
): Promise<BundleManifest> {
if (!/^[A-Za-z0-9][A-Za-z0-9._-]{0,199}$/.test(campaignId)) {
throw new Error("Campaign ID is unsafe for immutable object storage");
}
const retainedPaths = await relativeFiles(root, root, allowPublicSummary, publicScreenshots, bounded ? { entries: 0 } : undefined);
if (bounded) {
checkHistoryPayload(await Promise.all(retainedPaths.map(async file => ({
path: file, bytes: (await stat(path.join(root, file))).size,
}))), HISTORY_LIMITS.campaignBytes, HISTORY_LIMITS.campaignFiles, "Public campaign");
}
const retained = new Set(retainedPaths);
const missingScreenshots = [...publicScreenshots].filter((relative) => !retained.has(relative));
if (missingScreenshots.length > 0) {
throw new Error(`Declared public screenshots are missing from the historical bundle: ${missingScreenshots.sort().join(", ")}`);
}
const files = await Promise.all(
retainedPaths.sort().map(async (relative) => {
const absolute = path.join(root, ...relative.split("/"));
if (
relative === "public-images/campaign-summary.png" ||
publicScreenshots.has(relative)
) {
await validatePublicRaster(absolute, relative);
}
const [content, metadata] = await Promise.all([
readFile(absolute),
stat(absolute),
]);
return {
path: relative,
sha256: createHash("sha256").update(content).digest("hex"),
bytes: metadata.size,
};
}),
);
const bundleDigest = createHash("sha256")
.update(JSON.stringify(files))
.digest("hex");
return {
schema: "paperclip.runner-e2e.bundle/v1",
campaignId,
bundleDigest,
files,
};
}
export function buildHistoryPointers(history: RunnerE2EHistoryIndex) {
const byCampaign = new Map(
history.campaigns.map((campaign) => [campaign.campaignId, campaign]),
);
const pointer = (campaignId: string | null | undefined) => {
const campaign = campaignId ? byCampaign.get(campaignId) : undefined;
return campaign
? {
campaignId: campaign.campaignId,
generatedAt: campaign.generatedAt,
publicUrl: campaign.publicUrl,
sha: campaign.source.sha,
}
: null;
};
return {
latest: {
schema: "paperclip.runner-e2e.pointer/v1",
updatedAt: history.updatedAt,
overall: pointer(history.latestCampaignId),
suites: Object.fromEntries(
Object.entries(history.latestBySuite).map(([suiteId, campaignId]) => [
suiteId,
pointer(campaignId),
]),
),
},
latestGreen: {
schema: "paperclip.runner-e2e.pointer/v1",
updatedAt: history.updatedAt,
overall: pointer(history.latestGreenCampaignId),
suites: Object.fromEntries(
Object.entries(history.latestGreenBySuite).map(
([suiteId, campaignId]) => [suiteId, pointer(campaignId)],
),
),
},
};
}
function awsObject(bucket: string, key: string) {
return `s3://${bucket}/${key}`;
}
async function objectExists(bucket: string, key: string) {
try {
await execFileAsync("aws", [
"s3api",
"head-object",
"--bucket",
bucket,
"--key",
key,
]);
return true;
} catch (error) {
const detail = String(
(error as { stderr?: string }).stderr ??
(error instanceof Error ? error.message : error),
);
if (/\b(?:404|Not Found|NoSuchKey)\b/i.test(detail)) return false;
throw new Error(
`Unable to inspect historical object: ${detail.slice(0, 400)}`,
);
}
}
async function downloadJson<T>(
bucket: string,
key: string,
destination: string,
bounded?: BoundedHistoryPublisher,
) {
if (bounded) return bounded.downloadJson<T>(bucket, key, destination);
if (!(await objectExists(bucket, key))) return null;
await execFileAsync("aws", [
"s3",
"cp",
awsObject(bucket, key),
destination,
"--only-show-errors",
]);
return JSON.parse(await readFile(destination, "utf8")) as T;
}
async function uploadJson(
bucket: string,
key: string,
file: string,
cacheControl: string,
) {
await uploadFile(bucket, key, file, "application/json", cacheControl);
}
async function uploadFile(
bucket: string,
key: string,
file: string,
contentType: string,
cacheControl: string,
) {
await execFileAsync("aws", [
"s3",
"cp",
file,
awsObject(bucket, key),
"--only-show-errors",
"--content-type",
contentType,
"--cache-control",
cacheControl,
]);
}
export async function uploadImmutableBundle(
bucket: string,
key: string,
directory: string,
manifest?: BundleManifest,
bounded?: BoundedHistoryPublisher,
) {
if (bounded && manifest) {
for (const file of manifest.files) {
const extension = path.extname(file.path).toLowerCase();
const contentType = ({ ".png": "image/png", ".json": "application/json", ".html": "text/html; charset=utf-8",
".woff2": "font/woff2", ".xml": "application/xml", ".md": "text/plain; charset=utf-8" } as Record<string, string>)[extension];
if (!contentType) throw new Error(`Missing public content type for ${file.path}`);
await bounded.upload(bucket, `${key}/${file.path}`, path.join(directory, file.path), contentType,
"public,max-age=2592000,immutable", true);
}
return;
}
const common = [
"--recursive",
"--only-show-errors",
"--cache-control",
"public,max-age=31536000,immutable",
];
await execFileAsync("aws", [
"s3",
"cp",
directory,
awsObject(bucket, key),
...common,
"--exclude",
"history.json",
"--exclude",
"latest.json",
"--exclude",
"latest-green.json",
"--exclude",
"*.png",
]);
await execFileAsync("aws", [
"s3",
"cp",
directory,
awsObject(bucket, key),
...common,
"--exclude",
"*",
"--include",
"*.png",
"--content-type",
"image/png",
]);
}
async function main() {
const reportRoot = path.resolve(
process.env.PAPERCLIP_RUNNER_E2E_REPORT_DIR ??
"runner-e2e-merged-report/normalized",
);
const bucket = process.env.RUNNER_E2E_HISTORY_S3_BUCKET ?? "";
const destination = validateHistoryDestination({
bucket,
prefix: process.env.RUNNER_E2E_HISTORY_PREFIX ?? "runner-e2e",
publicBaseUrl: process.env.RUNNER_E2E_HISTORY_PUBLIC_BASE_URL ?? "",
});
const bounded = process.env.PAPERCLIP_E2E_BOUNDED_HISTORY === "true";
if (bounded && process.env.GITHUB_ACTIONS === "true" && process.env.GITHUB_RUN_ATTEMPT !== "1") {
throw new Error("Bounded history publication requires the original workflow attempt; retain failed publication evidence");
}
const publisher = bounded ? new BoundedHistoryPublisher(path.join(path.dirname(reportRoot), "history-publication-receipt.json")) : undefined;
if (publisher) await publisher.verifyRetention(bucket, destination.prefix);
const campaign = JSON.parse(
await readFile(path.join(reportRoot, "normalized-results.json"), "utf8"),
) as RunnerE2ECampaign;
for (const result of campaign.results ?? []) validateRetainedRunnerResult(result);
if (campaign.accounting) validateCampaignAccounting(campaign.accounting);
if (campaign.operations) validateEvalOperationsReport(campaign.operations);
if (campaign.feedbackReview) validateCampaignFeedbackReview(campaign, campaign.feedbackReview);
assertPublicCampaign(campaign);
if (campaign.schema !== "paperclip.runner-e2e.campaign/v2") {
throw new Error("Historical publishing requires a v2 normalized campaign");
}
const temporary = await mkdtemp(
path.join(os.tmpdir(), "runner-e2e-history-"),
);
const historyKey = `${destination.prefix}/history.json`;
const current =
(await downloadJson<RunnerE2EHistoryIndex>(
bucket,
historyKey,
path.join(temporary, "current-history.json"),
publisher,
)) ?? emptyRunnerHistory();
const history = mergeRunnerHistory(
current,
campaignHistoryRecord(
campaign,
`${destination.publicBaseUrl}/${destination.prefix}`,
),
bounded ? { now: Date.now(), maxAgeMs: HISTORY_LIMITS.indexDays * 86_400_000, maxCampaigns: HISTORY_LIMITS.indexCampaigns } : undefined,
);
if (bounded && !history.campaigns.some(entry => entry.campaignId === campaign.campaignId)) {
throw new Error("Bounded campaign is outside the retained history window");
}
const publicScreenshots = publicScreenshotPaths(campaign);
// Campaign bundles are immutable and must not capture a mutable history
// file left in a reused local directory. The root landing page below is the
// only dashboard that embeds navigation across campaigns. S3 gets its own
// staged copy. Only screenshots declared by normalized results (plus the
// fixed failure screenshot for failed executions) cross the public evidence
// boundary. Raw per-attempt text stays in the retained Actions artifact.
const s3ReportRoot = path.join(temporary, "s3-campaign");
await cp(reportRoot, s3ReportRoot, { recursive: true, errorOnExist: true });
await prunePrivateHistoryEvidence(s3ReportRoot, publicScreenshots);
await stageTrustedHistoryAssets(s3ReportRoot);
await rm(path.join(s3ReportRoot, "public-images"), {
recursive: true,
force: true,
});
const summaryImage = path.join(
s3ReportRoot,
"public-images",
"campaign-summary.png",
);
await writePublicCampaignSummaryImage(campaign, summaryImage);
await regenerateRunnerDashboard({
bundle: s3ReportRoot,
historyFile: null,
});
const manifest = await createBundleManifest(
s3ReportRoot,
campaign.campaignId,
true,
publicScreenshots,
bounded,
);
const campaignPrefix = `${destination.prefix}/campaigns/${campaign.campaignId}`;
const manifestKey = `${campaignPrefix}/bundle-manifest.json`;
const existingManifest = await downloadJson<BundleManifest>(
bucket,
manifestKey,
path.join(temporary, "existing-manifest.json"),
publisher,
);
if (
existingManifest &&
existingManifest.bundleDigest !== manifest.bundleDigest
) {
throw new Error(
`Immutable campaign ${campaign.campaignId} already exists with a different digest`,
);
}
const manifestFile = path.join(temporary, "bundle-manifest.json");
await writeFile(manifestFile, json(manifest), "utf8");
// Pages uses the same declared-screenshot boundary in a sibling stage. The
// downloaded access-controlled report stays intact for this whole job.
const pagesRoot = path.join(path.dirname(reportRoot), "pages");
await rm(pagesRoot, { recursive: true, force: true });
await cp(reportRoot, pagesRoot, { recursive: true, errorOnExist: true });
await prunePrivateHistoryEvidence(pagesRoot, publicScreenshots);
await stageTrustedHistoryAssets(pagesRoot);
await rm(path.join(pagesRoot, "public-images"), {
recursive: true,
force: true,
});
const pointers = buildHistoryPointers(history);
const historyFile = path.join(pagesRoot, "history.json");
const latestFile = path.join(pagesRoot, "latest.json");
const latestGreenFile = path.join(pagesRoot, "latest-green.json");
await Promise.all([
writeFile(historyFile, json(history), "utf8"),
writeFile(latestFile, json(pointers.latest), "utf8"),
writeFile(latestGreenFile, json(pointers.latestGreen), "utf8"),
]);
await regenerateRunnerDashboard({ bundle: pagesRoot, historyFile });
const pagesManifest = await createBundleManifest(
pagesRoot,
campaign.campaignId,
false,
publicScreenshots,
bounded,
);
const landingDirectory = path.join(temporary, "landing");
await regenerateRunnerDashboard({
bundle: s3ReportRoot,
historyFile,
outputDirectory: landingDirectory,
evidenceHrefPrefix: `campaigns/${campaign.campaignId}`,
});
await writeFile(
path.join(landingDirectory, "index.html"),
renderRunnerHistoryIndex(history),
"utf8",
);
if (publisher) {
const mutableFiles = [
{ key: historyKey, file: historyFile, contentType: "application/json" },
{ key: `${destination.prefix}/latest.json`, file: latestFile, contentType: "application/json" },
{ key: `${destination.prefix}/latest-green.json`, file: latestGreenFile, contentType: "application/json" },
...["index.html", "dashboard.html", "normalized-results.json"].map(file => ({ key: `${destination.prefix}/${file}`,
file: path.join(landingDirectory, file), contentType: file.endsWith("json") ? "application/json" : "text/html; charset=utf-8" })),
...["favicon-32x32.png", "InterVariable.woff2"].map(file => ({ key: `${destination.prefix}/assets/${file}`,
file: path.join(s3ReportRoot, "assets", file), contentType: file.endsWith("png") ? "image/png" : "font/woff2" })),
];
const metadata = async (file: string) => ({ path: file, bytes: (await stat(file)).size });
const campaignSize = checkHistoryPayload([...manifest.files, await metadata(manifestFile)],
HISTORY_LIMITS.campaignBytes, HISTORY_LIMITS.campaignFiles, "Public campaign with manifest");
const mutableMetadata = await Promise.all(mutableFiles.map(file => metadata(file.file)));
const mutableSize = checkHistoryPayload(mutableMetadata, HISTORY_LIMITS.mutableBytes, 8, "Mutable history");
const pointerMetadata = await Promise.all([historyFile, latestFile, latestGreenFile].map(metadata));
checkHistoryPayload(pointerMetadata, HISTORY_LIMITS.jsonBytes, 3, "History JSON");
const pagesSize = checkHistoryPayload([...pagesManifest.files, ...pointerMetadata],
HISTORY_LIMITS.pagesBytes, HISTORY_LIMITS.campaignFiles + 3, "Pages bundle");
if (!existingManifest) {
await uploadImmutableBundle(bucket, campaignPrefix, s3ReportRoot, manifest, publisher);
await publisher.upload(bucket, manifestKey, manifestFile, "application/json", "public,max-age=2592000,immutable", true);
}
for (const file of mutableFiles) await publisher.upload(bucket, file.key, file.file, file.contentType, "no-cache");
console.log(JSON.stringify({ campaign: campaignSize, mutable: mutableSize, pages: pagesSize, billedUsd: null }));
return;
}
if (!existingManifest) {
await uploadImmutableBundle(bucket, campaignPrefix, s3ReportRoot);
await uploadJson(bucket, manifestKey, manifestFile, "public,max-age=31536000,immutable");
}
await Promise.all([
uploadJson(bucket, historyKey, historyFile, "no-cache"),
uploadJson(
bucket,
`${destination.prefix}/latest.json`,
latestFile,
"no-cache",
),
uploadJson(
bucket,
`${destination.prefix}/latest-green.json`,
latestGreenFile,
"no-cache",
),
uploadFile(
bucket,
`${destination.prefix}/index.html`,
path.join(landingDirectory, "index.html"),
"text/html; charset=utf-8",
"no-cache",
),
uploadFile(
bucket,
`${destination.prefix}/dashboard.html`,
path.join(landingDirectory, "dashboard.html"),
"text/html; charset=utf-8",
"no-cache",
),
uploadFile(
bucket,
`${destination.prefix}/normalized-results.json`,
path.join(landingDirectory, "normalized-results.json"),
"application/json",
"no-cache",
),
uploadFile(
bucket,
`${destination.prefix}/assets/favicon-32x32.png`,
path.join(s3ReportRoot, "assets", "favicon-32x32.png"),
"image/png",
"public,max-age=86400",
),
uploadFile(
bucket,
`${destination.prefix}/assets/InterVariable.woff2`,
path.join(s3ReportRoot, "assets", "InterVariable.woff2"),
"font/woff2",
"public,max-age=86400",
),
]);
console.log(
`Published immutable campaign ${campaign.campaignId} (${manifest.bundleDigest}) and ${history.campaigns.length} history record(s)`,
);
}
if (
process.argv[1] &&
path.resolve(process.argv[1]) === path.resolve(import.meta.filename)
) {
await main().catch((error) => {
console.error(error instanceof Error ? error.message : String(error));
process.exitCode = 1;
});
}