mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-04 19:28:46 +01:00
## Thinking Path > - Paperclip helps people manage AI agents at work. > - Native Runner owns execution and task authority. > - Operators need an explicit native Cursor Cloud choice. > - Tools/files need company, task, session and run scope. > - This connects setup and lifecycle to Runner Core. > - Legacy agents retain their selected runtime/history. ## Linked Issues or Issue Description Depends on #18068. Managed-only API placement and API-owned Stop containment are in #18310. Refs #5664, #11803, #17299, and #17471. No duplicate native integration was found. ## What Changed - Add explicit native setup, Test, saved settings and dispatch with no-prompt account/catalog/callback checks. - Scope MCP to company, agent, task, session and run through authenticated HTTPS. Register verified artifacts/work products; preserve unpriced accounting, completion and budgets. - Connect Question/Plan continuation, cold handoff and admitted fresh sessions; recheck waiting admission under the task lock. - Present audited Stop intent while preserving physical containment proof; bound typed Stop metadata. Preserve title edits through preference/navigation readiness. - Select one live/history representation per run to prevent duplicate settled cards. Integrate upstream task/read-state/retry/schema contracts and repair two test fixtures. ## Verification Core #18068 merged normally as `3910352c923a`. Product head `5a60f541c982` has passing repository CI/build/browser checks, including required `ci / verify` and `ci / e2e`, plus a fresh exact-head Greptile 5/5 review with zero unresolved threads. Two non-required external app checks remain pending: Paperclip Review is queued waiting for its assigned agent; Superagent Security Scan is reviewing the diff. These are not claimed completed. The normal merge endpoint enforces the repository rules; no policy or status override is used. The dependent Top PR is addressing its fresh deferred-Stop review finding. Product `5a60f541c982` passed 270 UI and 20 selected Cloud/native/disk-full tests, UI/server typechecks, tokens and maps. Top’s clean normal merge passed tokens/maps/diff checks; its matrix is pending. Earlier receipts retain Core’s 47 Rust, 4 compiled-native retirement and 19 settlement controls, normal Runner build, and runtime 9af build/recursive typecheck. Reproduced loading/shutdown CI fixture failures passed 7/7 browser and 121/121 unit tests, including a broken-unsubscribe negative control. No timeout/oracle was relaxed. The actual-parent restack adopts 75 mainline paths, including schema/permission and generic runtime changes, beyond the prior two fixture repairs and client projection fix. The full difference from hosted `9af1a5de42a2` is therefore larger than four files; backend source also differs. The merged config-read checks match Core391, and the Cursor placement/runtime/relay/selection modules match frozen9af; selected affected regressions pass. Paid proof keeps its actual 9af source/image identity. The separately tested live/history card fix is not claimed deployed on that host. On 9af, original saved settings survived reopening and fresh no-prompt Test authenticated catalog/model/binding; no new Save is claimed. Fresh Linux admission passed. With `claude-sonnet-4-5`, Artifact downloaded exact 27 bytes and matched 121 canonical identities in complete S3-served history. First Stop’s strict quiet FAILED on a 10.373-second gap despite independent cancellation/no late artifact. One targeted unchanged-sampler retry retained all 24 samples: 35.076 seconds quiet / maximum gap 4.584 seconds, 67 ordered canonical identities, and separate CANCELLED/artifact-absent status after the original 180-second deadline. The pre-reload duplicate settled card is disclosed. Exact-ID cleanup for the three new provider agents is awaiting authorization, so full grades and the fresh Artifact’s post-cleanup download remain blocked. A historical deleted-provider artifact was newly downloaded on 9af; original F226 Question/Plan/reconnect and cleanup proof keep their identities. Retained failures: stale fixture 132/133; E8 48/135 with 87 reset-hook failures; Mac 17 SQL failures/skips; earlier title/startup/custody/Stop/deletion failures with unknown causes/bounded recoveries; `dea023` Stop’s warning/50 duplicate identities; pre-repair ACK and original Core CI failures. WSP’s 251-assertion, 7/8-suite attempt failed physical PostgreSQL teardown; the unchanged wrapper rerun passed 128. Retry alone establishes no cause or full local-suite pass. All paid attempts remain recorded; vendor dollar cost is UNKNOWN. ## Risks Experimental: Cursor independently acknowledges neither the effective model nor instruction digest before the first prompt. Explicit values are submitted and checked through behavior; unpriced provider dollar cost stays UNKNOWN. Callbacks need authenticated reachable HTTPS. Scope, revocation, budgets, completion and cleanup remain authoritative. Operator Stop intent is separate from physical containment. Saved legacy agents retain their selected runtime. Prior shared title-save/deletion failures remain disclosed. ## Model Used OpenAI Codex, GPT-6 family, assisted with reasoning, repository tools, code execution, browser verification, and delegated code review. The serving model’s exact API ID and context window are not exposed. No value is inferred from the model catalog. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used and the limits of available model metadata - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have linked public issues or described the issue in this PR - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch describes the change and contains no internal ticket ID - [x] I have run the scoped local tests reported above; pending gates are stated separately - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented the risks - [x] All required CI gates pass on the current head - [x] Fresh Greptile is 5/5 with no unresolved findings on the current head - [x] I will address all reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
781 lines
26 KiB
TypeScript
781 lines
26 KiB
TypeScript
import { validateCampaignAccounting, validateRetainedRunnerResult } from "./result-validation.js";
|
|
import { validateEvalOperationsReport } from "./operations-report.js";
|
|
import { validateCampaignFeedbackReview } from "./feedback-dispositions.js";
|
|
import { createHash } from "node:crypto";
|
|
import { execFile } from "node:child_process";
|
|
import path from "node:path";
|
|
import { promisify } from "node:util";
|
|
import {
|
|
cp,
|
|
copyFile,
|
|
mkdtemp,
|
|
lstat,
|
|
mkdir,
|
|
readFile,
|
|
readdir,
|
|
realpath,
|
|
rm,
|
|
stat,
|
|
writeFile,
|
|
} from "node:fs/promises";
|
|
import os from "node:os";
|
|
import { writePublicCampaignSummaryImage } from "./public-summary-image.js";
|
|
import { regenerateRunnerDashboard } from "./dashboard-regenerate.js";
|
|
import { renderRunnerHistoryIndex } from "./history-index.js";
|
|
import { validateHistoryDestination } from "./history-destination.js";
|
|
import { PUBLIC_RUNNER_SCREENSHOT_MARKER } from "./screenshot-policy.js";
|
|
import { BoundedHistoryPublisher, checkHistoryPayload, HISTORY_LIMITS } from "./history-budget.js";
|
|
import {
|
|
campaignHistoryRecord,
|
|
emptyRunnerHistory,
|
|
mergeRunnerHistory,
|
|
} from "./history.js";
|
|
import type { RunnerE2ECampaign, RunnerE2EHistoryIndex } from "./types.js";
|
|
|
|
const execFileAsync = promisify(execFile);
|
|
const repositoryRoot = path.resolve(import.meta.dirname, "../..");
|
|
const MUTABLE_HISTORY_FILES = new Set([
|
|
"history.json",
|
|
"latest.json",
|
|
"latest-green.json",
|
|
]);
|
|
const PUBLISH_ROOT_FILES = new Set([
|
|
"dashboard.html",
|
|
"index.html",
|
|
"junit.xml",
|
|
"normalized-results.json",
|
|
"summary.md",
|
|
]);
|
|
const MAX_PUBLIC_RASTER_BYTES = 12 * 1024 * 1024;
|
|
const EMPTY_PUBLIC_SCREENSHOTS: ReadonlySet<string> = new Set();
|
|
const PRIVATE_EVIDENCE_DIRECTORIES = new Set([
|
|
"blob-report",
|
|
"html-report",
|
|
"playwright-output",
|
|
]);
|
|
|
|
function publicEvidencePath(
|
|
relative: string,
|
|
publicScreenshots: ReadonlySet<string> = EMPTY_PUBLIC_SCREENSHOTS,
|
|
) {
|
|
const match = relative.match(
|
|
/^evidence\/[A-Za-z0-9._-]+\/attempt-[1-9][0-9]*\/(.+)$/,
|
|
);
|
|
if (!match) return false;
|
|
const evidencePath = match[1]!;
|
|
const segments = evidencePath.split("/");
|
|
if (
|
|
segments.some(
|
|
(segment) =>
|
|
!segment ||
|
|
segment === "." ||
|
|
segment === ".." ||
|
|
PRIVATE_EVIDENCE_DIRECTORIES.has(segment),
|
|
)
|
|
) {
|
|
return false;
|
|
}
|
|
// A text extension and credential scan do not establish a public projection.
|
|
// API snapshots and process logs can still hold hidden reasoning and provider
|
|
// session identities. Keep those diagnostics in the retained Actions artifact;
|
|
// normalized-results.json already carries the graded result contract.
|
|
return (
|
|
path.posix.extname(evidencePath).toLowerCase() === ".png" &&
|
|
publicScreenshots.has(relative)
|
|
);
|
|
}
|
|
|
|
export function publicScreenshotPaths(campaign: RunnerE2ECampaign) {
|
|
assertPublicCampaign(campaign);
|
|
const screenshots = new Set<string>();
|
|
for (const result of campaign.results) {
|
|
const files =
|
|
result.screenshots
|
|
?.filter(
|
|
(screenshot) =>
|
|
screenshot.publication === PUBLIC_RUNNER_SCREENSHOT_MARKER,
|
|
)
|
|
.map((screenshot) => screenshot.file) ?? [];
|
|
if (files.length === 0) continue;
|
|
if (
|
|
!/^[A-Za-z0-9][A-Za-z0-9._-]{0,299}$/.test(result.executionId) ||
|
|
!Number.isSafeInteger(result.attempt) ||
|
|
result.attempt < 1
|
|
) {
|
|
throw new Error(
|
|
`Cannot publish screenshots for unsafe execution identity ${result.executionId}`,
|
|
);
|
|
}
|
|
for (const file of files) {
|
|
if (!/^[A-Za-z0-9][A-Za-z0-9._-]{0,199}\.png$/i.test(file)) {
|
|
throw new Error(
|
|
`Cannot publish unsafe screenshot path for ${result.executionId}: ${file}`,
|
|
);
|
|
}
|
|
screenshots.add(
|
|
`evidence/${result.executionId}/attempt-${result.attempt}/${file}`,
|
|
);
|
|
}
|
|
}
|
|
return screenshots;
|
|
}
|
|
|
|
export function assertPublicCampaign(campaign: Pick<RunnerE2ECampaign, "results">) {
|
|
if (campaign.results.some(result => result.productJourney?.privacy === "private" || result.antigravityJourney?.privacy === "private" || result.suiteId === "cursor-cloud")) {
|
|
throw new Error("Private Product E2E campaigns cannot enter the public history publisher");
|
|
}
|
|
}
|
|
|
|
export function isHistoricalBundlePathAllowed(
|
|
relative: string,
|
|
allowPublicSummary = false,
|
|
publicScreenshots: ReadonlySet<string> = EMPTY_PUBLIC_SCREENSHOTS,
|
|
) {
|
|
if (
|
|
relative.includes("\\") ||
|
|
relative.startsWith("/") ||
|
|
relative.includes("..")
|
|
) {
|
|
return false;
|
|
}
|
|
if (PUBLISH_ROOT_FILES.has(relative)) return true;
|
|
if (allowPublicSummary && relative === "public-images/campaign-summary.png") {
|
|
return true;
|
|
}
|
|
if (
|
|
relative === "assets/favicon-32x32.png" ||
|
|
relative === "assets/InterVariable.woff2"
|
|
) {
|
|
return true;
|
|
}
|
|
return publicEvidencePath(relative, publicScreenshots);
|
|
}
|
|
|
|
function hasPublicPngMagic(content: Buffer) {
|
|
return (
|
|
content.length >= 24 &&
|
|
content
|
|
.subarray(0, 8)
|
|
.equals(Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a])) &&
|
|
content.subarray(12, 16).equals(Buffer.from("IHDR", "ascii"))
|
|
);
|
|
}
|
|
|
|
async function validatePublicRaster(absolute: string, relative: string) {
|
|
const metadata = await stat(absolute);
|
|
if (metadata.size === 0 || metadata.size > MAX_PUBLIC_RASTER_BYTES) {
|
|
throw new Error(
|
|
`Public screenshot ${relative} exceeds the per-file size boundary`,
|
|
);
|
|
}
|
|
const content = await readFile(absolute);
|
|
if (!hasPublicPngMagic(content)) {
|
|
throw new Error(
|
|
`Public screenshot ${relative} does not match its raster file type`,
|
|
);
|
|
}
|
|
}
|
|
|
|
async function pruneEvidenceDirectory(
|
|
root: string,
|
|
current: string,
|
|
publicScreenshots: ReadonlySet<string>,
|
|
) {
|
|
const entries = await readdir(current, { withFileTypes: true });
|
|
for (const entry of entries) {
|
|
const absolute = path.join(current, entry.name);
|
|
if (entry.isDirectory()) {
|
|
await pruneEvidenceDirectory(root, absolute, publicScreenshots);
|
|
if ((await readdir(absolute)).length === 0) {
|
|
await rm(absolute, { recursive: true });
|
|
}
|
|
continue;
|
|
}
|
|
const relative = path.relative(root, absolute).split(path.sep).join("/");
|
|
if (!entry.isFile() || !publicEvidencePath(relative, publicScreenshots)) {
|
|
await rm(absolute, { force: true });
|
|
}
|
|
}
|
|
}
|
|
|
|
export async function prunePrivateHistoryEvidence(
|
|
root: string,
|
|
publicScreenshots: ReadonlySet<string> = EMPTY_PUBLIC_SCREENSHOTS,
|
|
) {
|
|
const evidenceRoot = path.join(root, "evidence");
|
|
const metadata = await lstat(evidenceRoot).catch(() => null);
|
|
if (!metadata) return;
|
|
if (!metadata.isDirectory()) {
|
|
throw new Error("Historical evidence root must be a directory");
|
|
}
|
|
await pruneEvidenceDirectory(root, evidenceRoot, publicScreenshots);
|
|
}
|
|
|
|
export async function stageTrustedHistoryAssets(
|
|
root: string,
|
|
trustedRoot = repositoryRoot,
|
|
) {
|
|
const resolveTrustedAsset = async (segments: string[]) => {
|
|
const trustedRootReal = await realpath(trustedRoot);
|
|
let source = trustedRoot;
|
|
for (const [index, segment] of segments.entries()) {
|
|
source = path.join(source, segment);
|
|
const metadata = await lstat(source);
|
|
if (metadata.isSymbolicLink()) {
|
|
throw new Error(
|
|
`Refusing symbolic link in trusted publisher asset path ${segments.join("/")}`,
|
|
);
|
|
}
|
|
const final = index === segments.length - 1;
|
|
if (
|
|
(final && !metadata.isFile()) ||
|
|
(!final && !metadata.isDirectory())
|
|
) {
|
|
throw new Error(
|
|
`Trusted publisher asset path has an invalid file type: ${segments.join("/")}`,
|
|
);
|
|
}
|
|
}
|
|
const sourceReal = await realpath(source);
|
|
const relative = path.relative(trustedRootReal, sourceReal);
|
|
if (
|
|
!relative ||
|
|
relative === ".." ||
|
|
relative.startsWith(`..${path.sep}`)
|
|
) {
|
|
throw new Error(
|
|
`Trusted publisher asset escapes its checkout: ${segments.join("/")}`,
|
|
);
|
|
}
|
|
return sourceReal;
|
|
};
|
|
const [faviconSource, fontSource] = await Promise.all([
|
|
resolveTrustedAsset(["ui", "public", "favicon-32x32.png"]),
|
|
resolveTrustedAsset(["ui", "public", "fonts", "InterVariable.woff2"]),
|
|
]);
|
|
const assets = path.join(root, "assets");
|
|
await rm(assets, { recursive: true, force: true });
|
|
await mkdir(assets, { recursive: true });
|
|
await Promise.all([
|
|
copyFile(faviconSource, path.join(assets, "favicon-32x32.png")),
|
|
copyFile(fontSource, path.join(assets, "InterVariable.woff2")),
|
|
]);
|
|
}
|
|
|
|
interface BundleManifest {
|
|
schema: "paperclip.runner-e2e.bundle/v1";
|
|
campaignId: string;
|
|
bundleDigest: string;
|
|
files: Array<{ path: string; sha256: string; bytes: number }>;
|
|
}
|
|
|
|
function json(value: unknown) {
|
|
return `${JSON.stringify(value, null, 2)}\n`;
|
|
}
|
|
|
|
export { validateHistoryDestination };
|
|
|
|
async function relativeFiles(
|
|
root: string,
|
|
current = root,
|
|
allowPublicSummary = false,
|
|
publicScreenshots: ReadonlySet<string> = EMPTY_PUBLIC_SCREENSHOTS,
|
|
boundary?: { entries: number },
|
|
): Promise<string[]> {
|
|
const entries = await readdir(current, { withFileTypes: true });
|
|
const files: string[] = [];
|
|
for (const entry of entries) {
|
|
if (boundary && ++boundary.entries > 1024) throw new Error("Public bundle exceeds the directory-entry boundary");
|
|
const absolute = path.join(current, entry.name);
|
|
if (entry.isSymbolicLink()) {
|
|
throw new Error(`Refusing to publish symbolic link ${entry.name}`);
|
|
}
|
|
if (entry.isDirectory()) {
|
|
files.push(
|
|
...(await relativeFiles(
|
|
root,
|
|
absolute,
|
|
allowPublicSummary,
|
|
publicScreenshots,
|
|
boundary,
|
|
)),
|
|
);
|
|
} else if (entry.isFile()) {
|
|
const relative = path.relative(root, absolute).split(path.sep).join("/");
|
|
if (MUTABLE_HISTORY_FILES.has(relative)) continue;
|
|
if (
|
|
!isHistoricalBundlePathAllowed(
|
|
relative,
|
|
allowPublicSummary,
|
|
publicScreenshots,
|
|
)
|
|
) {
|
|
throw new Error(
|
|
`Refusing non-allowlisted historical bundle path ${relative}`,
|
|
);
|
|
}
|
|
files.push(relative);
|
|
}
|
|
}
|
|
return files;
|
|
}
|
|
|
|
export async function createBundleManifest(
|
|
root: string,
|
|
campaignId: string,
|
|
allowPublicSummary = false,
|
|
publicScreenshots: ReadonlySet<string> = EMPTY_PUBLIC_SCREENSHOTS,
|
|
bounded = false,
|
|
): Promise<BundleManifest> {
|
|
if (!/^[A-Za-z0-9][A-Za-z0-9._-]{0,199}$/.test(campaignId)) {
|
|
throw new Error("Campaign ID is unsafe for immutable object storage");
|
|
}
|
|
const retainedPaths = await relativeFiles(root, root, allowPublicSummary, publicScreenshots, bounded ? { entries: 0 } : undefined);
|
|
if (bounded) {
|
|
checkHistoryPayload(await Promise.all(retainedPaths.map(async file => ({
|
|
path: file, bytes: (await stat(path.join(root, file))).size,
|
|
}))), HISTORY_LIMITS.campaignBytes, HISTORY_LIMITS.campaignFiles, "Public campaign");
|
|
}
|
|
const retained = new Set(retainedPaths);
|
|
const missingScreenshots = [...publicScreenshots].filter((relative) => !retained.has(relative));
|
|
if (missingScreenshots.length > 0) {
|
|
throw new Error(`Declared public screenshots are missing from the historical bundle: ${missingScreenshots.sort().join(", ")}`);
|
|
}
|
|
const files = await Promise.all(
|
|
retainedPaths.sort().map(async (relative) => {
|
|
const absolute = path.join(root, ...relative.split("/"));
|
|
if (
|
|
relative === "public-images/campaign-summary.png" ||
|
|
publicScreenshots.has(relative)
|
|
) {
|
|
await validatePublicRaster(absolute, relative);
|
|
}
|
|
const [content, metadata] = await Promise.all([
|
|
readFile(absolute),
|
|
stat(absolute),
|
|
]);
|
|
return {
|
|
path: relative,
|
|
sha256: createHash("sha256").update(content).digest("hex"),
|
|
bytes: metadata.size,
|
|
};
|
|
}),
|
|
);
|
|
const bundleDigest = createHash("sha256")
|
|
.update(JSON.stringify(files))
|
|
.digest("hex");
|
|
return {
|
|
schema: "paperclip.runner-e2e.bundle/v1",
|
|
campaignId,
|
|
bundleDigest,
|
|
files,
|
|
};
|
|
}
|
|
|
|
export function buildHistoryPointers(history: RunnerE2EHistoryIndex) {
|
|
const byCampaign = new Map(
|
|
history.campaigns.map((campaign) => [campaign.campaignId, campaign]),
|
|
);
|
|
const pointer = (campaignId: string | null | undefined) => {
|
|
const campaign = campaignId ? byCampaign.get(campaignId) : undefined;
|
|
return campaign
|
|
? {
|
|
campaignId: campaign.campaignId,
|
|
generatedAt: campaign.generatedAt,
|
|
publicUrl: campaign.publicUrl,
|
|
sha: campaign.source.sha,
|
|
}
|
|
: null;
|
|
};
|
|
return {
|
|
latest: {
|
|
schema: "paperclip.runner-e2e.pointer/v1",
|
|
updatedAt: history.updatedAt,
|
|
overall: pointer(history.latestCampaignId),
|
|
suites: Object.fromEntries(
|
|
Object.entries(history.latestBySuite).map(([suiteId, campaignId]) => [
|
|
suiteId,
|
|
pointer(campaignId),
|
|
]),
|
|
),
|
|
},
|
|
latestGreen: {
|
|
schema: "paperclip.runner-e2e.pointer/v1",
|
|
updatedAt: history.updatedAt,
|
|
overall: pointer(history.latestGreenCampaignId),
|
|
suites: Object.fromEntries(
|
|
Object.entries(history.latestGreenBySuite).map(
|
|
([suiteId, campaignId]) => [suiteId, pointer(campaignId)],
|
|
),
|
|
),
|
|
},
|
|
};
|
|
}
|
|
|
|
function awsObject(bucket: string, key: string) {
|
|
return `s3://${bucket}/${key}`;
|
|
}
|
|
|
|
async function objectExists(bucket: string, key: string) {
|
|
try {
|
|
await execFileAsync("aws", [
|
|
"s3api",
|
|
"head-object",
|
|
"--bucket",
|
|
bucket,
|
|
"--key",
|
|
key,
|
|
]);
|
|
return true;
|
|
} catch (error) {
|
|
const detail = String(
|
|
(error as { stderr?: string }).stderr ??
|
|
(error instanceof Error ? error.message : error),
|
|
);
|
|
if (/\b(?:404|Not Found|NoSuchKey)\b/i.test(detail)) return false;
|
|
throw new Error(
|
|
`Unable to inspect historical object: ${detail.slice(0, 400)}`,
|
|
);
|
|
}
|
|
}
|
|
|
|
async function downloadJson<T>(
|
|
bucket: string,
|
|
key: string,
|
|
destination: string,
|
|
bounded?: BoundedHistoryPublisher,
|
|
) {
|
|
if (bounded) return bounded.downloadJson<T>(bucket, key, destination);
|
|
if (!(await objectExists(bucket, key))) return null;
|
|
await execFileAsync("aws", [
|
|
"s3",
|
|
"cp",
|
|
awsObject(bucket, key),
|
|
destination,
|
|
"--only-show-errors",
|
|
]);
|
|
return JSON.parse(await readFile(destination, "utf8")) as T;
|
|
}
|
|
|
|
async function uploadJson(
|
|
bucket: string,
|
|
key: string,
|
|
file: string,
|
|
cacheControl: string,
|
|
) {
|
|
await uploadFile(bucket, key, file, "application/json", cacheControl);
|
|
}
|
|
|
|
async function uploadFile(
|
|
bucket: string,
|
|
key: string,
|
|
file: string,
|
|
contentType: string,
|
|
cacheControl: string,
|
|
) {
|
|
await execFileAsync("aws", [
|
|
"s3",
|
|
"cp",
|
|
file,
|
|
awsObject(bucket, key),
|
|
"--only-show-errors",
|
|
"--content-type",
|
|
contentType,
|
|
"--cache-control",
|
|
cacheControl,
|
|
]);
|
|
}
|
|
|
|
export async function uploadImmutableBundle(
|
|
bucket: string,
|
|
key: string,
|
|
directory: string,
|
|
manifest?: BundleManifest,
|
|
bounded?: BoundedHistoryPublisher,
|
|
) {
|
|
if (bounded && manifest) {
|
|
for (const file of manifest.files) {
|
|
const extension = path.extname(file.path).toLowerCase();
|
|
const contentType = ({ ".png": "image/png", ".json": "application/json", ".html": "text/html; charset=utf-8",
|
|
".woff2": "font/woff2", ".xml": "application/xml", ".md": "text/plain; charset=utf-8" } as Record<string, string>)[extension];
|
|
if (!contentType) throw new Error(`Missing public content type for ${file.path}`);
|
|
await bounded.upload(bucket, `${key}/${file.path}`, path.join(directory, file.path), contentType,
|
|
"public,max-age=2592000,immutable", true);
|
|
}
|
|
return;
|
|
}
|
|
const common = [
|
|
"--recursive",
|
|
"--only-show-errors",
|
|
"--cache-control",
|
|
"public,max-age=31536000,immutable",
|
|
];
|
|
await execFileAsync("aws", [
|
|
"s3",
|
|
"cp",
|
|
directory,
|
|
awsObject(bucket, key),
|
|
...common,
|
|
"--exclude",
|
|
"history.json",
|
|
"--exclude",
|
|
"latest.json",
|
|
"--exclude",
|
|
"latest-green.json",
|
|
"--exclude",
|
|
"*.png",
|
|
]);
|
|
await execFileAsync("aws", [
|
|
"s3",
|
|
"cp",
|
|
directory,
|
|
awsObject(bucket, key),
|
|
...common,
|
|
"--exclude",
|
|
"*",
|
|
"--include",
|
|
"*.png",
|
|
"--content-type",
|
|
"image/png",
|
|
]);
|
|
}
|
|
|
|
async function main() {
|
|
const reportRoot = path.resolve(
|
|
process.env.PAPERCLIP_RUNNER_E2E_REPORT_DIR ??
|
|
"runner-e2e-merged-report/normalized",
|
|
);
|
|
const bucket = process.env.RUNNER_E2E_HISTORY_S3_BUCKET ?? "";
|
|
const destination = validateHistoryDestination({
|
|
bucket,
|
|
prefix: process.env.RUNNER_E2E_HISTORY_PREFIX ?? "runner-e2e",
|
|
publicBaseUrl: process.env.RUNNER_E2E_HISTORY_PUBLIC_BASE_URL ?? "",
|
|
});
|
|
const bounded = process.env.PAPERCLIP_E2E_BOUNDED_HISTORY === "true";
|
|
if (bounded && process.env.GITHUB_ACTIONS === "true" && process.env.GITHUB_RUN_ATTEMPT !== "1") {
|
|
throw new Error("Bounded history publication requires the original workflow attempt; retain failed publication evidence");
|
|
}
|
|
const publisher = bounded ? new BoundedHistoryPublisher(path.join(path.dirname(reportRoot), "history-publication-receipt.json")) : undefined;
|
|
if (publisher) await publisher.verifyRetention(bucket, destination.prefix);
|
|
const campaign = JSON.parse(
|
|
await readFile(path.join(reportRoot, "normalized-results.json"), "utf8"),
|
|
) as RunnerE2ECampaign;
|
|
for (const result of campaign.results ?? []) validateRetainedRunnerResult(result);
|
|
if (campaign.accounting) validateCampaignAccounting(campaign.accounting);
|
|
if (campaign.operations) validateEvalOperationsReport(campaign.operations);
|
|
if (campaign.feedbackReview) validateCampaignFeedbackReview(campaign, campaign.feedbackReview);
|
|
assertPublicCampaign(campaign);
|
|
if (campaign.schema !== "paperclip.runner-e2e.campaign/v2") {
|
|
throw new Error("Historical publishing requires a v2 normalized campaign");
|
|
}
|
|
|
|
const temporary = await mkdtemp(
|
|
path.join(os.tmpdir(), "runner-e2e-history-"),
|
|
);
|
|
const historyKey = `${destination.prefix}/history.json`;
|
|
const current =
|
|
(await downloadJson<RunnerE2EHistoryIndex>(
|
|
bucket,
|
|
historyKey,
|
|
path.join(temporary, "current-history.json"),
|
|
publisher,
|
|
)) ?? emptyRunnerHistory();
|
|
const history = mergeRunnerHistory(
|
|
current,
|
|
campaignHistoryRecord(
|
|
campaign,
|
|
`${destination.publicBaseUrl}/${destination.prefix}`,
|
|
),
|
|
bounded ? { now: Date.now(), maxAgeMs: HISTORY_LIMITS.indexDays * 86_400_000, maxCampaigns: HISTORY_LIMITS.indexCampaigns } : undefined,
|
|
);
|
|
if (bounded && !history.campaigns.some(entry => entry.campaignId === campaign.campaignId)) {
|
|
throw new Error("Bounded campaign is outside the retained history window");
|
|
}
|
|
const publicScreenshots = publicScreenshotPaths(campaign);
|
|
|
|
// Campaign bundles are immutable and must not capture a mutable history
|
|
// file left in a reused local directory. The root landing page below is the
|
|
// only dashboard that embeds navigation across campaigns. S3 gets its own
|
|
// staged copy. Only screenshots declared by normalized results (plus the
|
|
// fixed failure screenshot for failed executions) cross the public evidence
|
|
// boundary. Raw per-attempt text stays in the retained Actions artifact.
|
|
const s3ReportRoot = path.join(temporary, "s3-campaign");
|
|
await cp(reportRoot, s3ReportRoot, { recursive: true, errorOnExist: true });
|
|
await prunePrivateHistoryEvidence(s3ReportRoot, publicScreenshots);
|
|
await stageTrustedHistoryAssets(s3ReportRoot);
|
|
await rm(path.join(s3ReportRoot, "public-images"), {
|
|
recursive: true,
|
|
force: true,
|
|
});
|
|
const summaryImage = path.join(
|
|
s3ReportRoot,
|
|
"public-images",
|
|
"campaign-summary.png",
|
|
);
|
|
await writePublicCampaignSummaryImage(campaign, summaryImage);
|
|
await regenerateRunnerDashboard({
|
|
bundle: s3ReportRoot,
|
|
historyFile: null,
|
|
});
|
|
const manifest = await createBundleManifest(
|
|
s3ReportRoot,
|
|
campaign.campaignId,
|
|
true,
|
|
publicScreenshots,
|
|
bounded,
|
|
);
|
|
const campaignPrefix = `${destination.prefix}/campaigns/${campaign.campaignId}`;
|
|
const manifestKey = `${campaignPrefix}/bundle-manifest.json`;
|
|
const existingManifest = await downloadJson<BundleManifest>(
|
|
bucket,
|
|
manifestKey,
|
|
path.join(temporary, "existing-manifest.json"),
|
|
publisher,
|
|
);
|
|
if (
|
|
existingManifest &&
|
|
existingManifest.bundleDigest !== manifest.bundleDigest
|
|
) {
|
|
throw new Error(
|
|
`Immutable campaign ${campaign.campaignId} already exists with a different digest`,
|
|
);
|
|
}
|
|
const manifestFile = path.join(temporary, "bundle-manifest.json");
|
|
await writeFile(manifestFile, json(manifest), "utf8");
|
|
|
|
// Pages uses the same declared-screenshot boundary in a sibling stage. The
|
|
// downloaded access-controlled report stays intact for this whole job.
|
|
const pagesRoot = path.join(path.dirname(reportRoot), "pages");
|
|
await rm(pagesRoot, { recursive: true, force: true });
|
|
await cp(reportRoot, pagesRoot, { recursive: true, errorOnExist: true });
|
|
await prunePrivateHistoryEvidence(pagesRoot, publicScreenshots);
|
|
await stageTrustedHistoryAssets(pagesRoot);
|
|
await rm(path.join(pagesRoot, "public-images"), {
|
|
recursive: true,
|
|
force: true,
|
|
});
|
|
const pointers = buildHistoryPointers(history);
|
|
const historyFile = path.join(pagesRoot, "history.json");
|
|
const latestFile = path.join(pagesRoot, "latest.json");
|
|
const latestGreenFile = path.join(pagesRoot, "latest-green.json");
|
|
await Promise.all([
|
|
writeFile(historyFile, json(history), "utf8"),
|
|
writeFile(latestFile, json(pointers.latest), "utf8"),
|
|
writeFile(latestGreenFile, json(pointers.latestGreen), "utf8"),
|
|
]);
|
|
await regenerateRunnerDashboard({ bundle: pagesRoot, historyFile });
|
|
const pagesManifest = await createBundleManifest(
|
|
pagesRoot,
|
|
campaign.campaignId,
|
|
false,
|
|
publicScreenshots,
|
|
bounded,
|
|
);
|
|
const landingDirectory = path.join(temporary, "landing");
|
|
await regenerateRunnerDashboard({
|
|
bundle: s3ReportRoot,
|
|
historyFile,
|
|
outputDirectory: landingDirectory,
|
|
evidenceHrefPrefix: `campaigns/${campaign.campaignId}`,
|
|
});
|
|
await writeFile(
|
|
path.join(landingDirectory, "index.html"),
|
|
renderRunnerHistoryIndex(history),
|
|
"utf8",
|
|
);
|
|
if (publisher) {
|
|
const mutableFiles = [
|
|
{ key: historyKey, file: historyFile, contentType: "application/json" },
|
|
{ key: `${destination.prefix}/latest.json`, file: latestFile, contentType: "application/json" },
|
|
{ key: `${destination.prefix}/latest-green.json`, file: latestGreenFile, contentType: "application/json" },
|
|
...["index.html", "dashboard.html", "normalized-results.json"].map(file => ({ key: `${destination.prefix}/${file}`,
|
|
file: path.join(landingDirectory, file), contentType: file.endsWith("json") ? "application/json" : "text/html; charset=utf-8" })),
|
|
...["favicon-32x32.png", "InterVariable.woff2"].map(file => ({ key: `${destination.prefix}/assets/${file}`,
|
|
file: path.join(s3ReportRoot, "assets", file), contentType: file.endsWith("png") ? "image/png" : "font/woff2" })),
|
|
];
|
|
const metadata = async (file: string) => ({ path: file, bytes: (await stat(file)).size });
|
|
const campaignSize = checkHistoryPayload([...manifest.files, await metadata(manifestFile)],
|
|
HISTORY_LIMITS.campaignBytes, HISTORY_LIMITS.campaignFiles, "Public campaign with manifest");
|
|
const mutableMetadata = await Promise.all(mutableFiles.map(file => metadata(file.file)));
|
|
const mutableSize = checkHistoryPayload(mutableMetadata, HISTORY_LIMITS.mutableBytes, 8, "Mutable history");
|
|
const pointerMetadata = await Promise.all([historyFile, latestFile, latestGreenFile].map(metadata));
|
|
checkHistoryPayload(pointerMetadata, HISTORY_LIMITS.jsonBytes, 3, "History JSON");
|
|
const pagesSize = checkHistoryPayload([...pagesManifest.files, ...pointerMetadata],
|
|
HISTORY_LIMITS.pagesBytes, HISTORY_LIMITS.campaignFiles + 3, "Pages bundle");
|
|
if (!existingManifest) {
|
|
await uploadImmutableBundle(bucket, campaignPrefix, s3ReportRoot, manifest, publisher);
|
|
await publisher.upload(bucket, manifestKey, manifestFile, "application/json", "public,max-age=2592000,immutable", true);
|
|
}
|
|
for (const file of mutableFiles) await publisher.upload(bucket, file.key, file.file, file.contentType, "no-cache");
|
|
console.log(JSON.stringify({ campaign: campaignSize, mutable: mutableSize, pages: pagesSize, billedUsd: null }));
|
|
return;
|
|
}
|
|
if (!existingManifest) {
|
|
await uploadImmutableBundle(bucket, campaignPrefix, s3ReportRoot);
|
|
await uploadJson(bucket, manifestKey, manifestFile, "public,max-age=31536000,immutable");
|
|
}
|
|
await Promise.all([
|
|
uploadJson(bucket, historyKey, historyFile, "no-cache"),
|
|
uploadJson(
|
|
bucket,
|
|
`${destination.prefix}/latest.json`,
|
|
latestFile,
|
|
"no-cache",
|
|
),
|
|
uploadJson(
|
|
bucket,
|
|
`${destination.prefix}/latest-green.json`,
|
|
latestGreenFile,
|
|
"no-cache",
|
|
),
|
|
uploadFile(
|
|
bucket,
|
|
`${destination.prefix}/index.html`,
|
|
path.join(landingDirectory, "index.html"),
|
|
"text/html; charset=utf-8",
|
|
"no-cache",
|
|
),
|
|
uploadFile(
|
|
bucket,
|
|
`${destination.prefix}/dashboard.html`,
|
|
path.join(landingDirectory, "dashboard.html"),
|
|
"text/html; charset=utf-8",
|
|
"no-cache",
|
|
),
|
|
uploadFile(
|
|
bucket,
|
|
`${destination.prefix}/normalized-results.json`,
|
|
path.join(landingDirectory, "normalized-results.json"),
|
|
"application/json",
|
|
"no-cache",
|
|
),
|
|
uploadFile(
|
|
bucket,
|
|
`${destination.prefix}/assets/favicon-32x32.png`,
|
|
path.join(s3ReportRoot, "assets", "favicon-32x32.png"),
|
|
"image/png",
|
|
"public,max-age=86400",
|
|
),
|
|
uploadFile(
|
|
bucket,
|
|
`${destination.prefix}/assets/InterVariable.woff2`,
|
|
path.join(s3ReportRoot, "assets", "InterVariable.woff2"),
|
|
"font/woff2",
|
|
"public,max-age=86400",
|
|
),
|
|
]);
|
|
console.log(
|
|
`Published immutable campaign ${campaign.campaignId} (${manifest.bundleDigest}) and ${history.campaigns.length} history record(s)`,
|
|
);
|
|
}
|
|
|
|
if (
|
|
process.argv[1] &&
|
|
path.resolve(process.argv[1]) === path.resolve(import.meta.filename)
|
|
) {
|
|
await main().catch((error) => {
|
|
console.error(error instanceof Error ? error.message : String(error));
|
|
process.exitCode = 1;
|
|
});
|
|
}
|