diff --git a/doc/manual-e2e-private-evidence.md b/doc/manual-e2e-private-evidence.md index 844d3f3cd10..894d35c33c5 100644 --- a/doc/manual-e2e-private-evidence.md +++ b/doc/manual-e2e-private-evidence.md @@ -10,18 +10,18 @@ route never uploads raw cell evidence or a merged report to Actions, S3 public history, or Pages. This stays true on setup failure, launcher failure and report failure. Previously staged files do not enable a public fallback. -## Supported first selections +## Supported selections -The first supported selection is an exact subset of these execution IDs: - -- `chat-features.runner-opencode.local.activity.agent-feedback` -- `chat-features.runner-opencode.local.activity.work-timeline` -- `chat-features.runner-opencode.local.companies.import-preview` -- `chat-features.runner-opencode.local.navigation-preferences.dismiss-announcement` -- `chat-features.runner-opencode.local.tasks.completion-with-gate` -- `chat-features.runner-opencode.local.tasks.private-task` -- `chat-features.runner-opencode.local.secrets.personal-secret` -- `chat-features.runner-opencode.local.agents.choose-primary` +An approved original run selects at most **eight** distinct execution IDs from +one fixed registry of the frozen 244 chat scenarios. All use profile +`runner-opencode` and environment `local`. The 242 catalog tasks retain their +exact `area.case` IDs. The two standalone scenarios retain their existing task +IDs: `tasks.create-backlog` maps to +`chat-features.runner-opencode.local.create-backlog`, and +`tasks.delegate-announcement` maps to +`chat-features.runner-opencode.local.delegate-announcement`. The dotted task +aliases are not registered. The exact registry is in +`tests/runner-e2e/manual-private-retention.ts`. These IDs require the reviewed AWS Fleet route. Broad chat suites, mixed unsupported IDs, reruns and missing approvals fail closed. All chat-feature IDs @@ -30,9 +30,25 @@ Other manual selections keep the existing Actions route only after a trusted per-cell check confirms that the actual selected execution is not restricted. Selector whitespace and exclusivity follow the catalog producer. -A case that needs invitation-capability handling or other new producer support -is not supported merely because restricted storage exists. It needs its own -reviewed source, evidence contract and exact operation approval. +The registry size does not increase the per-run limit or the eight fixed report +read slots. Classification, cell binding, every approval row and session policy +reject a private selection larger than eight before actionable outputs or archive +credentials. Generic nonprivate catalog selection retains its existing semantics. + +A registered ID is not a claim of feature readiness or permission to run it. +Each target still needs the exact reviewed producer contracts, matching helper +bytes, declared private inputs, fresh source fingerprints, an available original +attempt and exact operation approval. Earlier accepted target branches do not +acquire these bytes or approval automatically. Running jobs keep their pinned +sources. No retained grade or historical qualification transfers to a new target. + +The private cell report contract binds the installed pinned Playwright producer +and the selected cell before it accepts canonical static trace files. Default +cell evidence keeps its existing unknown-carrier rules. Excluded installed build +staging is outside the evidence tree; unknown carriers inside standard evidence +paths still fail. A target with stronger ZIP or embedded-report scanning must +retain those checks when it adapts this contract. Installed-target identities +and runtime evidence remain separate from source-preparation metadata. ## Deployment and operator gates diff --git a/tests/e2e/chat-adapters-ui-messaging.spec.ts b/tests/e2e/chat-adapters-ui-messaging.spec.ts index cb752e0c066..9f6fc62b34b 100644 --- a/tests/e2e/chat-adapters-ui-messaging.spec.ts +++ b/tests/e2e/chat-adapters-ui-messaging.spec.ts @@ -218,6 +218,16 @@ test.describe("Board send delivery refresh", () => { page, request, }) => { + await page.addLocatorHandler( + page.getByRole("complementary", { name: "Paperclip announcements", exact: true }) + .getByRole("button", { name: "Dismiss announcement", exact: true }), + async (dismiss) => { + const menuOpen = await page.getByRole("menu", { name: "Add to composer", exact: true }).isVisible(); + if (menuOpen) await page.keyboard.press("Escape"); + await dismiss.click(); + if (menuOpen) await page.getByRole("button", { name: "Add to composer", exact: true }).click(); + }, + ); const selected = await seedCompanyAndAgent(request); const target = await seedCompanyAndAgent(request); const issue = await json<{ id: string; identifier: string }>( diff --git a/tests/runner-e2e/SECURITY.md b/tests/runner-e2e/SECURITY.md index d000473077d..f4b95ea97e2 100644 --- a/tests/runner-e2e/SECURITY.md +++ b/tests/runner-e2e/SECURITY.md @@ -569,3 +569,8 @@ Runner hook performs it. Only synthetic source bytes and proof JSON enter the existing private attempt evidence and registered-artifact verification pipeline. A failed attempt remains retained. These development checks are not staging or Slack qualification, and they do not publish trusted archives or credentials. + + +Report identities preserve legacy chat tasks and exact `area.case` chat catalog +tasks. Additional task segments, path separators and dotted nonchat tasks remain +invalid. This parsing does not qualify a result or grant a new execution. diff --git a/tests/runner-e2e/evidence.ts b/tests/runner-e2e/evidence.ts index 6ba750220c6..97d3d26a94a 100644 --- a/tests/runner-e2e/evidence.ts +++ b/tests/runner-e2e/evidence.ts @@ -6,6 +6,7 @@ import { rm, writeFile, copyFile, + lstat, } from "node:fs/promises"; import path from "node:path"; import { @@ -14,8 +15,11 @@ import { findSecretLeakInJsonValues, redactText, sanitizeJson, + normalizedSecrets, } from "./redaction.js"; +import { assertPrivateCellDirectories, cellTraceAsset, readPrivateCellTraceAsset, type PrivateCellEvidenceOwner } from "./manual-private-report-assets.js"; + const TEXT_EXTENSIONS = new Set([ ".json", ".xml", @@ -69,7 +73,7 @@ export interface EvidencePackageResult { missing: string[]; } -async function walk(root: string, relative = ""): Promise { +async function walk(root: string, relative = "", privateStandardOnly = false): Promise { const directory = path.join(root, relative); const entries = await readdir(directory, { withFileTypes: true }).catch( () => [], @@ -77,7 +81,10 @@ async function walk(root: string, relative = ""): Promise { const files: string[] = []; for (const entry of entries) { const next = path.join(relative, entry.name); - if (entry.isDirectory()) files.push(...(await walk(root, next))); + // Installed preparation may leave a large, deliberately excluded staging + // tree. Never enumerate it for private standard-evidence packaging. + if (privateStandardOnly && !relative && !isStandardEvidencePath(entry.isDirectory() ? path.join(next, "placeholder") : next)) continue; + if (entry.isDirectory()) files.push(...(await walk(root, next, privateStandardOnly))); else if (entry.isFile()) files.push(next); } return files; @@ -142,17 +149,39 @@ export async function packageEvidence(input: { uploadDir: string; secrets: readonly string[]; expectPassScreenshot: boolean; + privateCell?: { owner: PrivateCellEvidenceOwner; layout: "original" | "final-wrapper" }; }): Promise { + if (input.privateCell && !["original", "final-wrapper"].includes(input.privateCell.layout)) throw new Error("Invalid private cell packaging layout"); + if (input.privateCell) await assertPrivateCellDirectories(input.privateCell.owner, input.privateDir, input.uploadDir); await rm(input.uploadDir, { recursive: true, force: true }); await mkdir(input.uploadDir, { recursive: true }); const files: string[] = []; const leaks: EvidencePackageResult["leaks"] = []; - const available = await walk(input.privateDir); + const available = await walk(input.privateDir, "", Boolean(input.privateCell)); + const standard = available.filter(isStandardEvidencePath); - for (const relative of available.filter(isStandardEvidencePath)) { + if (input.privateCell && standard.length > 4096) throw new Error("Private cell evidence exceeds entry allowance"); + const privateSecrets = input.privateCell ? normalizedSecrets(input.secrets).flatMap(value => [value, encodeURIComponent(value), Buffer.from(value).toString("base64"), Buffer.from(value).toString("base64url")]) : []; + let privateBytes = 0; + for (const relative of standard) { const source = path.join(input.privateDir, relative); const extension = path.extname(relative).toLowerCase(); const destination = path.join(input.uploadDir, relative); + if (input.privateCell) { + assertSecretFree(relative, privateSecrets, "private cell evidence path"); + const metadata = await lstat(source); + if (!metadata.isFile() || metadata.isSymbolicLink() || metadata.size > 256 * 1024 * 1024 - privateBytes) + throw new Error("Private cell evidence exceeds byte allowance"); + privateBytes += metadata.size; + if (!Number.isSafeInteger(privateBytes)) throw new Error("Invalid private cell evidence byte accounting"); + const asset = cellTraceAsset(relative, input.privateCell.owner, input.privateCell.layout); + if (asset.reserved) { + const bytes = await readPrivateCellTraceAsset({ ...input.privateCell, relative, file: source, remaining: metadata.size, secrets: input.secrets }); + await mkdir(path.dirname(destination), { recursive: true, mode: 0o700 }); + await writeFile(destination, bytes!, { mode: 0o600 }); files.push(relative); continue; + } + if (!isStandardEvidenceFile(relative)) throw new Error("Unsupported private cell evidence carrier"); + } await mkdir(path.dirname(destination), { recursive: true }); if (ACTIVE_CONTENT_EXTENSIONS.has(extension)) { // SVG can execute script when opened directly from an artifact. Keep the diff --git a/tests/runner-e2e/launch.ts b/tests/runner-e2e/launch.ts index e9605563aba..a60e6a0b691 100644 --- a/tests/runner-e2e/launch.ts +++ b/tests/runner-e2e/launch.ts @@ -33,6 +33,7 @@ import { import { isImmutableDaytonaImage, runnerMatrix } from "./catalog.js"; import { renderRunnerE2EDashboard } from "./dashboard.js"; import { packageEvidence } from "./evidence.js"; +import { privateCellEvidenceOwner } from "./manual-private-report-assets.js"; import { readManualPrivateHandoff, sealManualResultEvidence, preserveOriginalFrameworkResult, sealStandardOriginalEvidence, type ManualPrivateHandoff } from "./manual-private-handoff.js"; import { requiresPrivateManualEvidence } from "./manual-private-retention.js"; import { copySharedEvidence } from "./shared-evidence.js"; @@ -837,6 +838,7 @@ async function runAttempt(input: { uploadDir, secrets: credentials, expectPassScreenshot: result.status === "passed", + ...(manualPrivateHandoff ? { privateCell: { owner: await privateCellEvidenceOwner(manualPrivateHandoff, candidate.id), layout: "original" as const } } : {}), }); const originalEvidence = evidence, originalFeaturePassed = result.status === "passed"; if (evidence.leaks.length > 0 || evidence.missing.length > 0) { @@ -862,6 +864,7 @@ async function runAttempt(input: { uploadDir, secrets: credentials, expectPassScreenshot: false, + ...(manualPrivateHandoff ? { privateCell: { owner: await privateCellEvidenceOwner(manualPrivateHandoff, candidate.id), layout: "original" as const } } : {}), }); } if (manualPrivateHandoff) await sealStandardOriginalEvidence({ context: manualPrivateHandoff, diff --git a/tests/runner-e2e/manual-private-cell-report-assets.test.ts b/tests/runner-e2e/manual-private-cell-report-assets.test.ts new file mode 100644 index 00000000000..e44b126986b --- /dev/null +++ b/tests/runner-e2e/manual-private-cell-report-assets.test.ts @@ -0,0 +1,274 @@ +import { afterAll, afterEach, beforeAll, expect, it, vi } from "vitest"; +import { execFile } from "node:child_process"; +import { createHash } from "node:crypto"; +import { createRequire, stripTypeScriptTypes } from "node:module"; +import { cp, mkdtemp, mkdir, readFile, readdir, realpath, rm, symlink, writeFile } from "node:fs/promises"; +import path from "node:path"; +import os from "node:os"; +import { promisify } from "node:util"; +const reads = vi.hoisted(() => ({ files: [] as string[], directories: [] as string[], oversized: "" })); +vi.mock("node:fs/promises", async original => { + const actual = await original(); + return { ...actual, + readFile: ((...args: Parameters) => { reads.files.push(String(args[0])); return actual.readFile(...args); }) as typeof actual.readFile, + readdir: ((...args: Parameters) => { reads.directories.push(String(args[0])); return actual.readdir(...args); }) as typeof actual.readdir, + lstat: (async (...args: Parameters) => { + const metadata = await actual.lstat(...args); + return String(args[0]) === reads.oversized ? Object.assign(metadata, { size: 256 * 1024 * 1024 + 1 }) : metadata; + }) as typeof actual.lstat, + }; +}); +import { copySharedEvidence } from "./shared-evidence.js"; +import { packageEvidence, isStandardEvidencePath } from "./evidence.js"; +import { readManualPrivateHandoff, sealStandardOriginalEvidence, sealManualResultEvidence, type ManualPrivateHandoff } from "./manual-private-handoff.js"; +import { canonicalGeneratedReportAsset, cellTraceAsset, inspectCellReportProducer, privateCellEvidenceOwner, readPrivateCellTraceAsset, type CellReportProducerIdentity } from "./manual-private-report-assets.js"; +import { validateSealedInventory, validateSealedOriginals } from "./manual-private-transfer.js"; +import { MANUAL_PRIVATE_EXECUTIONS, MANUAL_PRIVATE_PROTOCOL, manualCellKind } from "./manual-private-retention.js"; +import { archivePrivateEvidence, retainPrivateTransfer } from "./private-history.js"; +import { BoundedHistoryPublisher } from "./history-budget.js"; +const hash = (value: string | Buffer) => createHash("sha256").update(value).digest("hex"); +const roots: string[] = []; +const temporaryRoot = await realpath(os.tmpdir()); +let reporter: string, producerRoot: string, producer: CellReportProducerIdentity; +beforeAll(async () => { + producerRoot = await mkdtemp(path.join(temporaryRoot, "synthetic-cell-producer-")); + reporter = path.join(producerRoot, "reporter"); const tests = path.join(producerRoot, "node-tests"); + await Promise.all([mkdir(reporter), mkdir(tests)]); + const require = createRequire(import.meta.url), pkg = require.resolve("@playwright/test/package.json"), entry = require.resolve("@playwright/test"); + await writeFile(path.join(tests, "failure.spec.cjs"), `const {test,expect}=require(${JSON.stringify(entry)});test('synthetic Node-only retained failure',async()=>{expect('actual wrong outcome').toBe('expected outcome');});`); + const config = path.join(tests, "isolated.config.cjs"); + await writeFile(config, `module.exports=${JSON.stringify({ testDir: tests, testMatch: "failure.spec.cjs", workers: 1, outputDir: path.join(reporter, "playwright-output"), use: { trace: "retain-on-failure" }, reporter: [["html", { outputFolder: path.join(reporter, "html-report"), open: "never" }], ["junit", { outputFile: path.join(reporter, "junit.xml") }], ["blob", { outputDir: path.join(reporter, "blob-report") }]] })};`); + const failed = await promisify(execFile)(process.execPath, [path.join(path.dirname(pkg), "cli.js"), "test", "--config", config], { cwd: tests, timeout: 30_000, maxBuffer: 1024 * 1024, env: { PATH: process.env.PATH, CI: "1", NODE_OPTIONS: "", NODE_PATH: "", PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: "1" } }).then(() => false, error => { expect(error.code).toBe(1); return true; }); + expect(failed).toBe(true); + // A real automatic TestTracing attachment, not a manually attached fake trace. + const outputs = await readdir(path.join(reporter, "playwright-output")); + expect(await Promise.all(outputs.filter(name => !name.startsWith(".")).map(name => readdir(path.join(reporter, "playwright-output", name))))).toContainEqual(expect.arrayContaining(["trace.zip"])); + producer = await inspectCellReportProducer(); +}, 30_000); +afterEach(async () => { reads.oversized = ""; reads.files.length = 0; reads.directories.length = 0; await Promise.all(roots.splice(0).map(root => rm(root, { recursive: true, force: true }))); }); +afterAll(async () => { await rm(producerRoot, { recursive: true, force: true }); }); +async function fixture() { + const root = await mkdtemp(path.join(temporaryRoot, "synthetic-private-cell-")); roots.push(root); + const id = MANUAL_PRIVATE_EXECUTIONS[0]!, cell = path.join(root, "capture/cases/activity.agent-feedback"); await mkdir(cell, { recursive: true }); + await copySharedEvidence(reporter, cell, path.join(root, "controller"), []); + await writeFile(path.join(cell, "result.json"), '{"status":"failed","failureClass":"assertion"}'); + const context = { root, private: true, protocol: MANUAL_PRIVATE_PROTOCOL, workflowSha: "a".repeat(40), targetSha: "b".repeat(40), runId: "123", attempt: "1", executionIds: [id], cellReportProducer: producer } as ManualPrivateHandoff; + const owner = await privateCellEvidenceOwner(context, id), prefix = `gha-123-1-${id}/chat-features/runner-opencode/local/activity.agent-feedback/attempt-1`; + return { root, id, cell, context, owner, prefix, upload: path.join(root, "results", prefix) }; +} +function syntheticTransport(objects: Map) { + return async (args: string[]) => { + const operation = args[1]!; + if (operation === "put-object") { + expect(args.slice(args.indexOf("--if-none-match"), args.indexOf("--if-none-match") + 2)).toEqual(["--if-none-match", "*"]); + const key = args[args.indexOf("--key") + 1]!; expect(objects.has(key)).toBe(false); + objects.set(key, await readFile(args[args.indexOf("--body") + 1]!)); + return { stdout: JSON.stringify({ VersionId: `synthetic-version-${objects.size}` }) }; + } + if (operation === "get-object-lock-configuration" || operation === "get-bucket-replication") throw new Error(operation === "get-object-lock-configuration" ? "(ObjectLockConfigurationNotFoundError)" : "(ReplicationConfigurationNotFoundError)"); + const replies: Record = { + "get-bucket-versioning": { Status: "Enabled" }, + "get-bucket-lifecycle-configuration": { Rules: [ + { Status: "Enabled", Filter: { Prefix: "runner-e2e-private/bounded-v1/" }, Expiration: { Days: 30 }, NoncurrentVersionExpiration: { NoncurrentDays: 1 }, AbortIncompleteMultipartUpload: { DaysAfterInitiation: 1 } }, + { Status: "Enabled", Filter: { Prefix: "runner-e2e-private/bounded-v1/" }, Expiration: { ExpiredObjectDeleteMarker: true } }, + ] }, + "get-public-access-block": { PublicAccessBlockConfiguration: { BlockPublicAcls: true, IgnorePublicAcls: true, BlockPublicPolicy: true, RestrictPublicBuckets: true } }, + "get-bucket-ownership-controls": { OwnershipControls: { Rules: [{ ObjectOwnership: "BucketOwnerEnforced" }] } }, + "get-bucket-policy": { Policy: JSON.stringify({ Statement: [{ Effect: "Deny" }] }) }, + }; + if (!(operation in replies)) throw new Error("Unexpected synthetic transport operation"); + return { stdout: JSON.stringify(replies[operation]) }; + }; +} +it("retains the real failed trace/report through actual copy, private package, both seals, trusted validator, strict archive and immutable receipt", async () => { + const f = await fixture(); + const evidence = await packageEvidence({ privateDir: f.cell, uploadDir: f.upload, secrets: [], expectPassScreenshot: false, privateCell: { owner: f.owner, layout: "original" } }); + expect(evidence.leaks).toEqual([]); + for (const name of ["playwright-logo.svg", "manifest.webmanifest", "index.html"]) expect(evidence.files).toContain(`html-report/trace/${name}`); + expect(evidence.files.some(name => name.endsWith(".ttf"))).toBe(true); + const original = await sealStandardOriginalEvidence({ context: f.context, executionId: f.id, source: f.cell, evidence, secrets: [], expectPassScreenshot: false }); + await sealManualResultEvidence(f.context, []); + const originalDir = path.join(f.root, "originals", manualCellKind(f.id)), finalDir = path.join(f.root, "sealed-results/html-report"); + expect((await validateSealedOriginals(f.root, f.id, f.context)).inventorySha256).toBe(original.inventorySha256); + const final = await validateSealedInventory(finalDir, "../result-evidence-inventory.json", undefined, "standard", f.context); + for (const name of ["playwright-logo.svg", "manifest.webmanifest", "index.html"]) { + const bytes = await readFile(canonicalGeneratedReportAsset(`html/trace/${name}`)!); + expect(await readFile(path.join(originalDir, "html-report/trace", name))).toEqual(bytes); + expect(await readFile(path.join(finalDir, f.prefix, "html-report/trace", name))).toEqual(bytes); + expect(final.files).toContainEqual({ path: `${f.prefix}/html-report/trace/${name}`, bytes: bytes.length, sha256: hash(bytes) }); + } + const bundle = path.join(f.root, "bundle"); await mkdir(bundle); + await cp(path.join(f.root, "originals"), path.join(bundle, "originals"), { recursive: true }); + await cp(path.join(f.root, "sealed-results"), path.join(bundle, "sealed-results"), { recursive: true }); + const objects = new Map(), receipt = path.join(f.root, "transport.json"), runner = syntheticTransport(objects); + const transferred = await retainPrivateTransfer({ operation: "upload", kind: manualCellKind(f.id), runId: "123", directory: bundle, bucket: "synthetic-private-bucket", publisher: new BoundedHistoryPublisher(receipt, runner), evidenceCompleteness: { meaning: "available-files-and-transfer-only", doesNotAttestFeatureOutcome: true, cells: [{ executionId: f.id, availableFileInventoryComplete: true, requiredPassEvidenceComplete: null, missingRequiredEvidence: [] }] } }, receipt, () => new BoundedHistoryPublisher(`${receipt}.retention`, runner)); + expect(objects.size).toBe(2); expect(transferred).toHaveProperty("versionId", "synthetic-version-1"); + const retainedReceipt = JSON.parse([...objects.values()][1]!.toString()); expect(retainedReceipt.transfer.sha256).toBe(hash([...objects.values()][0]!)); + expect(retainedReceipt.evidenceCompleteness.doesNotAttestFeatureOutcome).toBe(true); + const zip = path.join(f.root, "retained.zip"), restored = path.join(f.root, "restored"); await writeFile(zip, [...objects.values()][0]!); await archivePrivateEvidence("unpack", zip, restored); + expect(JSON.parse(await readFile(path.join(restored, "originals", manualCellKind(f.id), "result.json"), "utf8")).status).toBe("failed"); + const downloaded = { ...f.context, root: restored }; + expect((await validateSealedOriginals(restored, f.id, downloaded)).complete).toBe(true); + expect((await validateSealedInventory(path.join(restored, "sealed-results/html-report"), "../result-evidence-inventory.json", undefined, "standard", downloaded)).complete).toBe(true); + expect(await readFile(path.join(restored, "sealed-results/html-report", f.prefix, "html-report/trace/playwright-logo.svg"))).toEqual(await readFile(canonicalGeneratedReportAsset("html/trace/playwright-logo.svg")!)); +}, 60_000); +it.each(["wrong-case", "wrong-run", "wrong-attempt", "double-wrapper", "extra-nesting", "watch", "xterm", "tampered-html", "tampered-svg", "shape"])("actual package rejects %s in the reserved cell namespace", async kind => { + const f = await fixture(); let relative = "html-report/trace/index.html"; + if (kind === "watch") relative = "html-report/trace/watch.js"; + else if (kind === "xterm") relative = "html-report/trace/assets/xtermModule.synthetic.js"; + else if (kind === "tampered-svg") relative = "html-report/trace/playwright-logo.svg"; + else if (["wrong-case", "wrong-run", "wrong-attempt", "double-wrapper", "extra-nesting"].includes(kind)) { + relative = `${f.prefix}/html-report/trace/index.html`; + if (kind === "wrong-case") relative = relative.replace("/activity.agent-feedback/", "/activity.work-timeline/"); + if (kind === "wrong-run") relative = relative.replace("gha-123-", "gha-124-"); + if (kind === "wrong-attempt") relative = relative.replace("attempt-1/", "attempt-2/"); + if (kind === "double-wrapper") relative = `html-report/html-report/${relative}`; + else relative = `html-report/${kind === "extra-nesting" ? "extra/" : ""}${relative}`; + } + await mkdir(path.dirname(path.join(f.cell, relative)), { recursive: true }); + await writeFile(path.join(f.cell, relative), kind === "shape" ? "sk-proj-synthetic123456789012345" : "changed or foreign canonical script"); + await expect(packageEvidence({ privateDir: f.cell, uploadDir: f.upload, secrets: [], expectPassScreenshot: false, privateCell: { owner: f.owner, layout: "original" } })).rejects.toThrow(/private cell trace|canonical/); +}); +it.each(["final", "final-wrapper"] as const)("binds exactly the %s layout without prefix stripping", async layout => { + const f = await fixture(), suffix = "html-report/trace/index.html", proper = `${layout === "final-wrapper" ? "html-report/" : ""}${f.prefix}/${suffix}`; + expect(cellTraceAsset(proper, f.owner, layout).canonical).toBe(canonicalGeneratedReportAsset("html/trace/index.html")); + expect(cellTraceAsset(`html-report/${proper}`, f.owner, layout)).toEqual({ reserved: true, canonical: undefined }); + expect(cellTraceAsset(proper.replace("/activity.agent-feedback/", "/activity.work-timeline/"), f.owner, layout).canonical).toBeUndefined(); +}); +it("registered roots remain protected even when equal to genuine public static bytes", async () => { + const f = await fixture(), bytes = await readFile(canonicalGeneratedReportAsset("html/trace/index.html")!); + expect(bytes.toString()).toContain("Playwright"); + await expect(packageEvidence({ privateDir: f.cell, uploadDir: f.upload, secrets: ["Playwright"], expectPassScreenshot: false, privateCell: { owner: f.owner, layout: "original" } })).rejects.toThrow(/Secret leak/); +}); +it.each(["encoded", "unknown-shape"])("private cell original scan still rejects %s contamination in standard evidence", async kind => { + const f = await fixture(), secret = "synthetic-credential-1234567890"; + await mkdir(path.join(f.cell, "snapshots")); + await writeFile(path.join(f.cell, "snapshots/unsafe.txt"), kind === "encoded" ? Buffer.from(secret).toString("base64") : "sk-proj-synthetic123456789012345"); + await expect(sealStandardOriginalEvidence({ context: f.context, executionId: f.id, source: f.cell, evidence: { files: [], leaks: [], missing: [] }, secrets: [secret], expectPassScreenshot: false })).rejects.toThrow(/privacy/); +}); +it.each(["unknown", "oversize", "symlink"])("rejects %s before reading a cell static body", async kind => { + const f = await fixture(), file = path.join(f.cell, "html-report/trace", kind === "unknown" ? "unknown.bin" : "index.html"); + if (kind === "unknown") await writeFile(file, "synthetic"); + if (kind === "oversize") reads.oversized = file; + if (kind === "symlink") { await rm(file); await symlink(canonicalGeneratedReportAsset("html/trace/index.html")!, file); } + reads.files.length = 0; + await expect(readPrivateCellTraceAsset({ owner: f.owner, layout: "original", relative: path.relative(f.cell, file), file, remaining: 256 * 1024 * 1024, secrets: [] })).rejects.toThrow(); + expect(reads.files).not.toContain(file); +}); +it("rejects public or overlapping output roots and unverified private flags before deleting output", async () => { + const f = await fixture(), outside = await mkdtemp(path.join(temporaryRoot, "synthetic-public-destination-")); roots.push(outside); await writeFile(path.join(outside, "keep.txt"), "keep"); + for (const [uploadDir, owner] of [[outside, f.owner], [f.cell, f.owner], [f.upload, { ...f.owner }]] as const) + await expect(packageEvidence({ privateDir: f.cell, uploadDir, secrets: [], expectPassScreenshot: false, privateCell: { owner, layout: "original" } })).rejects.toThrow(); + expect(await readFile(path.join(outside, "keep.txt"), "utf8")).toBe("keep"); +}); +it.each(["missing", "same-version-tamper"])("actual preflight rejects %s producer before a fixture callback", async kind => { + const f = await fixture(), file = path.join(f.root, "preflight.json"); + const wrong = structuredClone(producer); wrong.producers[0]!.sha256 = "0".repeat(64); + await writeFile(file, JSON.stringify({ ...f.context, status: "verified", cellReportProducer: kind === "missing" ? undefined : wrong }), { mode: 0o600 }); + const env = { PAPERCLIP_RUNNER_E2E_PRIVATE_PREFLIGHT: file, RUNNER_TEMP: temporaryRoot, GITHUB_RUN_ID: "123", GITHUB_RUN_ATTEMPT: "1", GITHUB_SHA: f.context.workflowSha, PAPERCLIP_RUNNER_E2E_SOURCE_SHA: f.context.targetSha }; + let callbacks = 0; + await expect(readManualPrivateHandoff(env, [f.id]).then(() => { callbacks++; })).rejects.toThrow(/producer/); + expect(callbacks).toBe(0); +}); +it("does not enable static SVG or unknown carriers in the default public package or default cell seal", async () => { + const f = await fixture(); const publicPackage = await packageEvidence({ privateDir: f.cell, uploadDir: f.upload, secrets: [], expectPassScreenshot: false }); + expect(publicPackage.files).not.toContain("html-report/trace/playwright-logo.svg"); + const noProducer = { ...f.context, cellReportProducer: undefined }; + await expect(sealStandardOriginalEvidence({ context: noProducer, executionId: f.id, source: f.cell, evidence: publicPackage, secrets: [], expectPassScreenshot: false })).rejects.toThrow(/Unsupported/); +}); +it.each(["tampered-html", "foreign-case", "foreign-run", "double-wrapper", "watch"])("trusted validator rejects rehashed %s cell inventory", async kind => { + const f = await fixture(); let relative = `${f.prefix}/html-report/trace/index.html`; + if (kind === "foreign-case") relative = relative.replace("/activity.agent-feedback/", "/activity.work-timeline/"); + if (kind === "foreign-run") relative = relative.replace("gha-123-", "gha-124-"); + if (kind === "double-wrapper") relative = `html-report/html-report/${relative}`; + if (kind === "watch") relative = `${f.prefix}/html-report/trace/watch.js`; + const directory = path.join(f.root, "forged-final"), data = Buffer.from("same standard-extension script with forged inventory"); + await mkdir(path.dirname(path.join(directory, relative)), { recursive: true }); await writeFile(path.join(directory, relative), data); + const files = [{ path: relative, bytes: data.length, sha256: hash(data) }]; + await writeFile(path.join(directory, "inventory.json"), JSON.stringify({ protocol: MANUAL_PRIVATE_PROTOCOL, treeKind: "standard", complete: true, files, sourceBytes: data.length, inventorySha256: hash(JSON.stringify(files)) })); + reads.files.length = 0; + await expect(validateSealedInventory(directory, "inventory.json", undefined, "standard", f.context)).rejects.toThrow(/private cell trace|canonical/); + if (kind !== "tampered-html") expect(reads.files).not.toContain(path.join(directory, relative)); +}); +it.each(["snapshots/unknown.svg", "snapshots/unknown.bin"])("verified private packaging still rejects unowned %s", async relative => { + const f = await fixture(); await mkdir(path.dirname(path.join(f.cell, relative)), { recursive: true }); await writeFile(path.join(f.cell, relative), "synthetic unsupported"); + await expect(packageEvidence({ privateDir: f.cell, uploadDir: f.upload, secrets: [], expectPassScreenshot: false, privateCell: { owner: f.owner, layout: "original" } })).rejects.toThrow(/Unsupported/); +}); + +it("accepts the actual compact portable producer identity before callbacks", async () => { + const f = await fixture(), file = path.join(f.root, "preflight.json"); + const serialized = JSON.stringify({ ...f.context, status: "verified" }); + expect(Buffer.byteLength(serialized)).toBeLessThan(65536); + expect(JSON.stringify(producer)).not.toContain(temporaryRoot); + await writeFile(file, serialized, { mode: 0o600 }); + const env = { PAPERCLIP_RUNNER_E2E_PRIVATE_PREFLIGHT: file, RUNNER_TEMP: temporaryRoot, GITHUB_RUN_ID: "123", GITHUB_RUN_ATTEMPT: "1", GITHUB_SHA: f.context.workflowSha, PAPERCLIP_RUNNER_E2E_SOURCE_SHA: f.context.targetSha }; + expect((await readManualPrivateHandoff(env, [f.id]))?.cellReportProducer).toEqual(producer); +}); + +it("does not silently package final rows using an unsupported unwrapped layout", async () => { + const f = await fixture(); await mkdir(f.upload, { recursive: true }); await writeFile(path.join(f.upload, "keep.txt"), "keep"); + await expect(packageEvidence({ privateDir: path.join(f.root, "results"), uploadDir: path.join(f.root, "bad-layout"), secrets: [], expectPassScreenshot: false, + privateCell: { owner: f.owner, layout: "final" as "original" } })).rejects.toThrow(/layout/); + expect(await readFile(path.join(f.upload, "keep.txt"), "utf8")).toBe("keep"); +}); + +it("rejects a registered public-path match before reading its genuine static body", async () => { + const f = await fixture(), file = path.join(f.cell, "html-report/trace/index.html"); reads.files.length = 0; + await expect(readPrivateCellTraceAsset({ owner: f.owner, layout: "original", relative: "html-report/trace/index.html", file, remaining: 256 * 1024 * 1024, secrets: ["index.html"] })).rejects.toThrow(/Secret leak/); + expect(reads.files).not.toContain(file); +}); + +async function stagingFixture() { + const root = await mkdtemp(path.join(temporaryRoot, "synthetic-excluded-cell-staging-")); roots.push(root); + const id = "chat-features.runner-opencode.local.onboarding.new-instance", source = path.join(root, "capture/cases/onboarding.new-instance"); + await mkdir(source, { recursive: true }); + const owner = await privateCellEvidenceOwner({ private: true, root, runId: "123", attempt: "1", executionIds: [id], cellReportProducer: producer }, id); + return { root, source, owner, uploadDir: path.join(root, "output") }; +} +it("excluded installed staging neither consumes the standard allowance nor gets traversed", async () => { + const f = await stagingFixture(), staging = path.join(f.source, "empty-installed-candidate-synthetic/installed/node_modules/synthetic-package"); + await mkdir(staging, { recursive: true }); + expect(isStandardEvidencePath("empty-installed-candidate-synthetic/installed/node_modules/synthetic-package/file-0.js")).toBe(false); + for (let start = 0; start < 4096; start += 64) await Promise.all(Array.from({ length: 64 }, (_, offset) => writeFile(path.join(staging, `file-${start + offset}.js`), ""))); + await writeFile(path.join(f.source, "result.json"), '{"status":"failed","synthetic":true}'); + reads.directories.length = 0; + const result = await packageEvidence({ privateDir: f.source, uploadDir: f.uploadDir, secrets: [], expectPassScreenshot: false, privateCell: { owner: f.owner, layout: "original" } }); + expect(result.files).toEqual(["result.json", "evidence-manifest.json"]); expect(result.leaks).toEqual([]); + expect(reads.directories.some(directory => directory.includes("empty-installed-candidate-synthetic"))).toBe(false); +}); +it("still rejects more than4096 genuine standard files, including node_modules within approved evidence", async () => { + const f = await stagingFixture(), standard = path.join(f.source, "snapshots/node_modules/synthetic-package"); + await mkdir(standard, { recursive: true }); + for (let start = 0; start < 4097; start += 64) await Promise.all(Array.from({ length: Math.min(64, 4097 - start) }, (_, offset) => writeFile(path.join(standard, `file-${start + offset}.js`), ""))); + reads.files.length = 0; + await expect(packageEvidence({ privateDir: f.source, uploadDir: f.uploadDir, secrets: [], expectPassScreenshot: false, privateCell: { owner: f.owner, layout: "original" } })).rejects.toThrow(/entry allowance/); + expect(reads.files.some(file => file.startsWith(standard))).toBe(false); +}); +it.each(["snapshots/node_modules/package/unknown.bin", "html-report/node_modules/package/unknown.svg"])("pruning staging does not ignore unapproved carrier %s within standard evidence", async relative => { + const f = await stagingFixture(); await mkdir(path.dirname(path.join(f.source, relative)), { recursive: true }); await writeFile(path.join(f.source, relative), "synthetic unsupported"); + await expect(packageEvidence({ privateDir: f.source, uploadDir: f.uploadDir, secrets: [], expectPassScreenshot: false, privateCell: { owner: f.owner, layout: "original" } })).rejects.toThrow(/Unsupported/); +}); + +it.each([0, 1])("actual launcher packaging call %s retains the verified private producer tree", async index => { + const source = await readFile(new URL("./launch.ts", import.meta.url), "utf8"); + const calls: string[] = []; + for (const match of source.matchAll(/await packageEvidence\(/g)) { + let depth = 1, quote = "", escaped = false, end = match.index! + match[0].length; + for (; end < source.length && depth; end++) { + const char = source[end]!; + if (quote) { if (escaped) escaped = false; else if (char === "\\") escaped = true; else if (char === quote) quote = ""; } + else if (char === '"' || char === "'" || char === "`") quote = char; + else if (char === "(") depth++; + else if (char === ")") depth--; + } + calls.push(source.slice(match.index!, end)); + } + expect(calls).toHaveLength(2); + const code = stripTypeScriptTypes(`async function actualCall(manualPrivateHandoff, candidate, casePrivateDir, uploadDir, credentials, result) { return ${calls[index]}; }`, { mode: "strip" }); + const actualCall = Function("packageEvidence", "privateCellEvidenceOwner", `${code}; return actualCall;`)(packageEvidence, privateCellEvidenceOwner) as (context: ManualPrivateHandoff | undefined, candidate: { id: string }, source: string, target: string, secrets: string[], result: { status: string }) => ReturnType; + const f = await fixture(); + const packaged = await actualCall(f.context, { id: f.id }, f.cell, f.upload, [], { status: "failed" }); + for (const name of ["playwright-logo.svg", "manifest.webmanifest", "index.html"]) expect(packaged.files).toContain(`html-report/trace/${name}`); + const publicPackage = await actualCall(undefined, { id: f.id }, f.cell, path.join(f.root, "nonprivate-packaged"), [], { status: "failed" }); + expect(publicPackage.files).not.toContain("html-report/trace/playwright-logo.svg"); +}); diff --git a/tests/runner-e2e/manual-private-handoff.test.ts b/tests/runner-e2e/manual-private-handoff.test.ts index ff7fd0b0d9e..2b0f7f6c4fa 100644 --- a/tests/runner-e2e/manual-private-handoff.test.ts +++ b/tests/runner-e2e/manual-private-handoff.test.ts @@ -4,6 +4,7 @@ import path from "node:path"; import os from "node:os"; import { readManualPrivateHandoff, sealStandardOriginalEvidence, preserveOriginalFrameworkResult, type ManualPrivateHandoff } from "./manual-private-handoff.js"; import { MANUAL_PRIVATE_EXECUTIONS, MANUAL_PRIVATE_PROTOCOL } from "./manual-private-retention.js"; +import { inspectCellReportProducer } from "./manual-private-report-assets.js"; const roots: string[]=[]; afterEach(async()=>{await Promise.all(roots.splice(0).map(p=>rm(p,{recursive:true,force:true})));}); async function fixture() { @@ -38,7 +39,7 @@ it("does not copy an escaped or symlinked original",async()=>{ const f=await fixture();await symlink(path.join(f.source,"snapshots/state.json"),path.join(f.source,"snapshots/link.json"));await expect(sealStandardOriginalEvidence(f.input)).rejects.toThrow(/unsupported/); }); it("requires the complete preflight's exact attempt, owner-only file and target before selecting a private root",async()=>{ - const f=await fixture();const file=path.join(f.out,"preflight.json");await writeFile(file,JSON.stringify({...f.context,status:"verified"}),{mode:0o600}); + const f=await fixture();const file=path.join(f.out,"preflight.json");await writeFile(file,JSON.stringify({...f.context,status:"verified",cellReportProducer:await inspectCellReportProducer()}),{mode:0o600}); const env={GITHUB_SHA:f.context.workflowSha,RUNNER_TEMP:f.root,GITHUB_RUN_ID:"123",GITHUB_RUN_ATTEMPT:"1",PAPERCLIP_RUNNER_E2E_SOURCE_SHA:f.context.targetSha,PAPERCLIP_RUNNER_E2E_PRIVATE_PREFLIGHT:file}; expect((await readManualPrivateHandoff(env,f.context.executionIds))?.root).toBe(await realpath(f.out)); await expect(readManualPrivateHandoff({...env,GITHUB_RUN_ID:"124"},f.context.executionIds)).rejects.toThrow(/exact attempt/); diff --git a/tests/runner-e2e/manual-private-handoff.ts b/tests/runner-e2e/manual-private-handoff.ts index 92143c47852..a7cc8f83666 100644 --- a/tests/runner-e2e/manual-private-handoff.ts +++ b/tests/runner-e2e/manual-private-handoff.ts @@ -3,12 +3,12 @@ import { createHash } from "node:crypto"; import { copyFile, lstat, mkdir, open, readdir, readFile, realpath, rm, writeFile } from "node:fs/promises"; import path from "node:path"; import { packageEvidence, isStandardEvidencePath, isStandardEvidenceFile, type EvidencePackageResult } from "./evidence.js"; -import { canonicalGeneratedReportAsset, isTrustedGeneratedReportAsset } from "./manual-private-report-assets.js"; +import { canonicalGeneratedReportAsset, isTrustedGeneratedReportAsset, assertCellReportProducer, privateCellEvidenceOwner, cellTraceAsset, readPrivateCellTraceAsset, assertPrivateCellDirectories, type CellReportProducerIdentity } from "./manual-private-report-assets.js"; import { assertSecretFree, normalizedSecrets } from "./redaction.js"; import { MANUAL_PRIVATE_PROTOCOL, manualCellKind, type ManualRetentionIdentity } from "./manual-private-retention.js"; export interface ManualPrivateHandoff extends ManualRetentionIdentity { - protocol: typeof MANUAL_PRIVATE_PROTOCOL; private: true; root: string; + protocol: typeof MANUAL_PRIVATE_PROTOCOL; private: true; root: string; cellReportProducer?: CellReportProducerIdentity; } export async function readManualPrivateHandoff(env: NodeJS.ProcessEnv, ids: string[]): Promise { const file = env.PAPERCLIP_RUNNER_E2E_PRIVATE_PREFLIGHT; @@ -26,6 +26,7 @@ export async function readManualPrivateHandoff(env: NodeJS.ProcessEnv, ids: stri if (!root.startsWith(`${temporary}${path.sep}`) || root !== path.dirname(await realpath(file)) || !directory.isDirectory() || directory.isSymbolicLink() || directory.mode & 0o077 || typeof process.getuid === "function" && directory.uid !== process.getuid()) throw new Error("Invalid private evidence staging ownership"); + await assertCellReportProducer(receipt.cellReportProducer); return Object.freeze({ ...receipt, root }); } export async function preserveOriginalFrameworkResult(directory: string) { @@ -41,6 +42,8 @@ export async function sealStandardOriginalEvidence(input: { context: ManualPrivateHandoff; executionId: string; source: string; evidence: EvidencePackageResult; secrets: readonly string[]; expectPassScreenshot: boolean; }) { if (!input.context.executionIds.includes(input.executionId)) throw new Error("Private evidence execution is outside preflight"); + const cellOwner = input.context.cellReportProducer ? await privateCellEvidenceOwner(input.context, input.executionId) : undefined; + if (cellOwner) await assertPrivateCellDirectories(cellOwner, input.source); const secrets = normalizedSecrets(input.secrets).flatMap(value => [value, encodeURIComponent(value), Buffer.from(value).toString("base64"), Buffer.from(value).toString("base64url")]); const out = path.join(input.context.root, "originals", manualCellKind(input.executionId)); const root = await realpath(input.source), info = await lstat(input.source); @@ -68,7 +71,8 @@ export async function sealStandardOriginalEvidence(input: { continue; } if (!isStandardEvidencePath(name)) continue; - if (!isStandardEvidenceFile(name)) throw new Error("Unsupported original evidence carrier cannot be retained"); + const asset = cellOwner ? cellTraceAsset(name, cellOwner, "original") : undefined; + if (asset?.reserved ? !asset.canonical : !isStandardEvidenceFile(name)) throw new Error("Unsupported original evidence carrier cannot be retained"); assertSecretFree(name, secrets, "private evidence path"); const source = path.join(root, name), target = path.join(out, name); if (!(await realpath(source)).startsWith(`${root}${path.sep}`)) throw new Error("Original evidence escaped its owned root"); @@ -76,6 +80,7 @@ export async function sealStandardOriginalEvidence(input: { try { const before = await handle.stat(); bytes += before.size; if (!before.isFile() || !Number.isSafeInteger(bytes) || bytes > 256 * 1024 * 1024) throw new Error("Complete original evidence exceeds byte allowance"); + if (asset?.reserved) await readPrivateCellTraceAsset({ owner: cellOwner!, layout: "original", relative: name, file: source, remaining: before.size, secrets }); await mkdir(path.dirname(target), { recursive: true, mode: 0o700 }); const output = await open(target, "wx", 0o600), hash = createHash("sha256"), buffer = Buffer.alloc(65536); try { @@ -99,7 +104,7 @@ export async function sealStandardOriginalEvidence(input: { const validation = path.join(input.context.root, `validation-${manualCellKind(input.executionId)}`); await mkdir(validation, { mode: 0o700 }); try { - const scan = await packageEvidence({ privateDir: out, uploadDir: validation, secrets, expectPassScreenshot: input.expectPassScreenshot }); + const scan = await packageEvidence({ privateDir: out, uploadDir: validation, secrets, expectPassScreenshot: input.expectPassScreenshot, ...(cellOwner ? { privateCell: { owner: cellOwner, layout: "original" as const } } : {}) }); missingRequiredEvidence = scan.missing; if (scan.leaks.length) { await writeFile(path.join(input.context.root, "contamination.json"), JSON.stringify({ protocol: MANUAL_PRIVATE_PROTOCOL, status: "withheld_credential_contamination", complete: false }), { mode: 0o600, flag: "wx" }); @@ -118,6 +123,8 @@ export async function sealStandardOriginalEvidence(input: { } export async function sealManualResultEvidence(context: ManualPrivateHandoff, secrets: readonly string[], source = path.join(context.root, "results"), treeKind: "standard" | "generated-report" = "standard") { + const cellOwner = treeKind === "standard" && context.cellReportProducer + ? await privateCellEvidenceOwner(context, context.executionIds.length === 1 ? context.executionIds[0]! : "") : undefined; const root = await realpath(source), owned = await realpath(context.root); if (!root.startsWith(`${owned}${path.sep}`)) throw new Error("Packaged evidence escaped its owned root"); const wrapper = path.join(context.root, "result-validation"); @@ -132,14 +139,18 @@ export async function sealManualResultEvidence(context: ManualPrivateHandoff, se assertSecretFree(name, scanSecrets, "private packaged evidence path"); if (entry.isSymbolicLink() || !entry.isFile() && !entry.isDirectory() || name.split("/").length > 24) throw new Error("Invalid packaged evidence entry"); if (entry.isDirectory()) { await visit(name); continue; } + const cellAsset = cellOwner ? cellTraceAsset(name, cellOwner, "final") : undefined; + if (cellAsset?.reserved && !cellAsset.canonical) throw new Error("Unsupported private cell trace owner"); const standard = isStandardEvidenceFile(`html-report/${name}`); const canonical = treeKind === "generated-report" ? canonicalGeneratedReportAsset(name) : undefined; const staticReportOwner = treeKind === "generated-report" && (name.startsWith("html/trace/") || canonical !== undefined); - if (staticReportOwner ? !canonical : !standard) throw new Error("Unsupported packaged evidence carrier"); + if (cellAsset?.reserved ? !cellAsset.canonical : staticReportOwner ? !canonical : !standard) throw new Error("Unsupported packaged evidence carrier"); const file = path.join(root, name), metadata = await lstat(file); if (!metadata.isFile() || metadata.isSymbolicLink()) throw new Error("Invalid packaged evidence entry"); if (metadata.size > 256 * 1024 * 1024 - bytes || records.length >= 4096) throw new Error("Complete packaged evidence exceeds allowance"); - const data = await readFile(file); + const data = cellAsset?.reserved + ? (await readPrivateCellTraceAsset({ owner: cellOwner!, layout: "final", relative: name, file, remaining: 256 * 1024 * 1024 - bytes, secrets: scanSecrets }))! + : await readFile(file); if (staticReportOwner) { if (!await isTrustedGeneratedReportAsset(name, data)) throw new Error("Unsupported packaged evidence carrier"); assertSecretFree(data, scanSecrets, "authenticated static report asset"); @@ -155,7 +166,7 @@ export async function sealManualResultEvidence(context: ManualPrivateHandoff, se await cp(root, path.join(wrapper, "html-report"), { recursive: true, errorOnExist: true, force: false }); const validation = path.join(context.root, "packaged-scan"); try { - const scan = await packageEvidence({ privateDir: wrapper, uploadDir: validation, secrets: scanSecrets, expectPassScreenshot: false }); + const scan = await packageEvidence({ privateDir: wrapper, uploadDir: validation, secrets: scanSecrets, expectPassScreenshot: false, ...(cellOwner ? { privateCell: { owner: cellOwner, layout: "final-wrapper" as const } } : {}) }); if (scan.leaks.length) throw new Error("Packaged evidence contains credential contamination"); for (const record of records) { if (createHash("sha256").update(await readFile(path.join(wrapper, "html-report", record.path))).digest("hex") !== record.sha256) diff --git a/tests/runner-e2e/manual-private-policy.mjs b/tests/runner-e2e/manual-private-policy.mjs index 57bf2862dd4..7244812659f 100644 --- a/tests/runner-e2e/manual-private-policy.mjs +++ b/tests/runner-e2e/manual-private-policy.mjs @@ -1,6 +1,6 @@ import { appendFileSync } from 'node:fs'; import { pathToFileURL } from 'node:url'; -import { canonicalManualIds, MANUAL_PRIVATE_EXECUTIONS, requiresPrivateManualEvidence } from './manual-private-retention.ts'; +import { canonicalManualIds, canonicalPrivateManualIds, MANUAL_PRIVATE_EXECUTIONS, requiresPrivateManualEvidence } from './manual-private-retention.ts'; const values = value => (value ?? '').split(',').map(item => item.trim()).filter(Boolean); export function manualEvidenceRouting(env) { @@ -12,7 +12,7 @@ export function manualEvidenceRouting(env) { return { retention: 'actions', executionIds: [] }; } if (!ids.some(requiresPrivateManualEvidence)) return { retention: 'actions', executionIds: ids }; - const selected = canonicalManualIds(ids); + const selected = canonicalPrivateManualIds(ids); if (selected.some(id => !MANUAL_PRIVATE_EXECUTIONS.includes(id))) throw new Error('Unsupported private manual selection'); if (env.TEST_RUNNER !== 'runs-on/fleet=paperclip-public-pr-x64/env=public-ci') throw new Error('Private manual evidence currently requires the reviewed AWS route'); return { retention: 'private', executionIds: selected }; @@ -20,7 +20,7 @@ export function manualEvidenceRouting(env) { export function authorizeManualCellRoute(retention, executionId, privateIds) { const ids = canonicalManualIds([executionId]); if (retention === 'private') { - if (!MANUAL_PRIVATE_EXECUTIONS.includes(ids[0]) || !canonicalManualIds(JSON.parse(privateIds)).includes(ids[0])) throw new Error('Private cell differs from trusted selection'); + if (!MANUAL_PRIVATE_EXECUTIONS.includes(ids[0]) || !canonicalPrivateManualIds(JSON.parse(privateIds)).includes(ids[0])) throw new Error('Private cell differs from trusted selection'); return 'private'; } if (retention !== 'actions' || requiresPrivateManualEvidence(ids[0])) throw new Error('Sensitive cell cannot enter public manual artifacts'); diff --git a/tests/runner-e2e/manual-private-policy.test.ts b/tests/runner-e2e/manual-private-policy.test.ts index f2d53c28ead..dae7e81c5a6 100644 --- a/tests/runner-e2e/manual-private-policy.test.ts +++ b/tests/runner-e2e/manual-private-policy.test.ts @@ -1,3 +1,6 @@ +import { readFileSync, readdirSync } from "node:fs"; +import { createHash } from "node:crypto"; +import YAML from "yaml"; import { expect, it, vi } from "vitest"; import { manualEvidenceRouting, authorizeManualCellRoute } from "./manual-private-policy.mjs"; import { MANUAL_PRIVATE_EXECUTIONS } from "./manual-private-retention.js"; @@ -28,3 +31,56 @@ it.each(["33-known-ids","duplicate-known-id"])("preserves actual nonprivate cata const route=manualEvidenceRouting({...env,SELECT_ID:ids.join(",")});expect(route).toEqual({retention:"actions",executionIds:ids}); for(const row of selected)expect(authorizeManualCellRoute(route.retention,row.id,"[]")).toBe("actions"); }); + +const newlyEnrolled = ["access.invite-teammate", "activity.routine-audit"].map(id => `chat-features.runner-opencode.local.${id}`); +it.each(newlyEnrolled)("enrollment routes only the exact reviewed OpenCode/local ID %s", id => { + expect(manualEvidenceRouting({ ...env, SELECT_ID: id })).toEqual({ retention: "private", executionIds: [id] }); + expect(authorizeManualCellRoute("private", id, JSON.stringify([id]))).toBe("private"); + expect(() => authorizeManualCellRoute("actions", id, "[]")).toThrow(); + for (const other of [id.replace("runner-opencode", "runner-codex"), id.replace(".local.", ".daytona.")]) + expect(() => manualEvidenceRouting({ ...env, SELECT_ID: other })).toThrow(); +}); +it("enrollment keeps eight mixed cells valid and rejects nine or ten even when the bound cell is supported", () => { + const eight = MANUAL_PRIVATE_EXECUTIONS.slice(0, 8); + expect(manualEvidenceRouting({ ...env, SELECT_ID: eight.join(",") }).executionIds).toEqual([...eight].sort()); + expect(authorizeManualCellRoute("private", eight[0], JSON.stringify(eight))).toBe("private"); + for (const count of [9, 10]) { + const ids = MANUAL_PRIVATE_EXECUTIONS.slice(0, count); + expect(() => manualEvidenceRouting({ ...env, SELECT_ID: ids.join(",") })).toThrow(); + expect(() => authorizeManualCellRoute("private", ids[0], JSON.stringify(ids))).toThrow(); + } + expect(() => manualEvidenceRouting({ ...env, SELECT_ID: "chat-features.runner-opencode.local.onboarding.future-unknown" })).toThrow(); +}); + +function frozenExecutionIds() { + const directory = new URL("./chat-scenarios/", import.meta.url); + const scenarios: string[] = readdirSync(directory).filter(name => name.endsWith(".yaml")).flatMap(name => { + const document = YAML.parse(readFileSync(new URL(name, directory), "utf8")); + return (document.scenarios ?? []).map((row: { id: string }) => row.id); + }); + const legacy: Record = { "tasks.create-backlog": "create-backlog", "tasks.delegate-announcement": "delegate-announcement" }; + return scenarios.map(id => `chat-features.runner-opencode.local.${legacy[id] ?? id}`).sort(); +} +it("full catalog routes every frozen execution through private classification and cell binding", () => { + const ids = frozenExecutionIds(); + expect(ids).toHaveLength(244); + expect(new Set(ids).size).toBe(244); + expect(createHash("sha256").update(JSON.stringify(ids)).digest("hex")).toBe("a215d1c3b010d0cb1b72efd45a564bcb662c2caffd87a34316ff8e657b4f02e5"); + for (const id of ids) { + expect(manualEvidenceRouting({ ...env, SELECT_ID: id })).toEqual({ retention: "private", executionIds: [id] }); + expect(authorizeManualCellRoute("private", id, JSON.stringify([id]))).toBe("private"); + expect(() => authorizeManualCellRoute("actions", id, "[]")).toThrow(); + } + expect([...MANUAL_PRIVATE_EXECUTIONS].sort()).toEqual(ids); +}); +it.each([ + "chat-features.runner-opencode.local.future.unknown", + "chat-features.runner-opencode.local.tasks.create-backlog", + "chat-features.runner-opencode.local.tasks.delegate-announcement", + "chat-features.runner-codex.local.agents.choose-primary", + "chat-features.runner-opencode.daytona.agents.choose-primary", + "chat-features.runner-opencode.local.tasks/../private-task", +])("full catalog rejects unregistered or rewritten execution %s", id => { + expect(() => manualEvidenceRouting({ ...env, SELECT_ID: id })).toThrow(); + expect(() => authorizeManualCellRoute("private", id, JSON.stringify([id]))).toThrow(); +}); diff --git a/tests/runner-e2e/manual-private-report-assets.ts b/tests/runner-e2e/manual-private-report-assets.ts index 73fc61fb3fa..d5d1e2373cd 100644 --- a/tests/runner-e2e/manual-private-report-assets.ts +++ b/tests/runner-e2e/manual-private-report-assets.ts @@ -1,6 +1,8 @@ import { createRequire } from "node:module"; -import { lstat, readFile } from "node:fs/promises"; +import { createHash } from "node:crypto"; +import { lstat, readFile, readdir, realpath } from "node:fs/promises"; import path from "node:path"; +import { assertSecretFree, normalizedSecrets } from "./redaction.js"; const require = createRequire(import.meta.url); const repositoryRoot = path.resolve(import.meta.dirname, "../.."); @@ -23,3 +25,138 @@ export async function isTrustedGeneratedReportAsset(relative: string, bytes: Buf const metadata = await lstat(source).catch((error: NodeJS.ErrnoException) => { if (error.code === "ENOENT") return undefined; throw error; }); return !!metadata && metadata.isFile() && !metadata.isSymbolicLink() && metadata.size === bytes.length && bytes.equals(await readFile(source)); } + +// Portable dependency identity: paths are package-relative, never host paths. +export interface CellReportProducerIdentity { + packages: Array<{ name: string; version: string; sha256: string }>; + producers: Array<{ path: string; sha256: string }>; + staticManifestSha256: string; staticFileCount: number; staticBytes: number; +} +const hash = (data: string | Buffer) => createHash("sha256").update(data).digest("hex"); +const packageNames = ["@playwright/test", "playwright", "playwright-core"]; +const producerNames = ["lib/index.js", "lib/worker/workerProcessEntry.js", "lib/runner/index.js"]; +const sourceAllowance = 256 * 1024 * 1024; +async function boundedSource(file: string, remaining: number) { + if (!Number.isSafeInteger(remaining) || remaining < 0) throw new Error("Invalid canonical report allowance"); + const metadata = await lstat(file); + if (!metadata.isFile() || metadata.isSymbolicLink() || metadata.size > remaining) throw new Error("Invalid canonical report source"); + const bytes = await readFile(file), after = await lstat(file); + if (bytes.length !== metadata.size || after.ino !== metadata.ino || after.dev !== metadata.dev + || after.size !== metadata.size || after.mtimeMs !== metadata.mtimeMs || after.ctimeMs !== metadata.ctimeMs) + throw new Error("Canonical report source changed"); + return bytes; +} +export async function inspectCellReportProducer(): Promise { + const test = require.resolve("@playwright/test/package.json"); + const playwright = require.resolve("playwright/package.json", { paths: [path.dirname(test)] }); + const core = require.resolve("playwright-core/package.json", { paths: [path.dirname(playwright)] }); + let sourceBytes = 0; + const sourceRead = async (file: string, limit = sourceAllowance) => { + const data = await boundedSource(file, Math.min(limit, sourceAllowance - sourceBytes)); sourceBytes += data.length; return data; + }; + const packages: CellReportProducerIdentity["packages"] = []; + for (const [index, file] of [test, playwright, core].entries()) { + const bytes = await sourceRead(file, 65536), parsed = JSON.parse(bytes.toString("utf8")); + if (parsed.name !== packageNames[index] || typeof parsed.version !== "string" || !/^\d+\.\d+\.\d+(?:[-+][A-Za-z0-9.-]+)?$/.test(parsed.version) || parsed.version.length > 100) + throw new Error("Invalid report package identity"); + packages.push({ name: parsed.name, version: parsed.version, sha256: hash(bytes) }); + } + const producers: CellReportProducerIdentity["producers"] = []; + for (const relative of producerNames) producers.push({ path: relative, sha256: hash(await sourceRead(path.join(path.dirname(playwright), relative))) }); + const viewer = path.join(path.dirname(core), "lib/vite/traceViewer"), assets = path.join(viewer, "assets"); + for (const directory of [viewer, assets]) { + const metadata = await lstat(directory); + if (!metadata.isDirectory() || metadata.isSymbolicLink()) throw new Error("Invalid canonical report directory"); + } + const manifest: Array<{ path: string; bytes: number; sha256: string }> = []; + let bytes = 0, visited = 0; + for (const prefix of ["", "assets/"]) { + for (const entry of await readdir(path.join(viewer, prefix), { withFileTypes: true })) { + if (++visited > 4096) throw new Error("Canonical report closure exceeds entry allowance"); + const relative = prefix + entry.name, canonical = canonicalGeneratedReportAsset(`html/trace/${relative}`); + if (!canonical) continue; + if (manifest.length >= 4096 || !entry.isFile() || entry.isSymbolicLink()) throw new Error("Invalid canonical report closure"); + const data = await sourceRead(canonical); bytes += data.length; + manifest.push({ path: relative, bytes: data.length, sha256: hash(data) }); + } + } + if (!manifest.length) throw new Error("Empty canonical report closure"); + manifest.sort((a, b) => a.path.localeCompare(b.path)); + return { packages, producers, staticManifestSha256: hash(JSON.stringify(manifest)), staticFileCount: manifest.length, staticBytes: bytes }; +} +export async function assertCellReportProducer(expected: unknown) { + const actual = await inspectCellReportProducer(); + // JSON equality also closes unknown keys, ordering and arbitrary machine paths. + if (JSON.stringify(expected) !== JSON.stringify(actual)) throw new Error("Private cell report producer differs from trusted preflight"); + return actual; +} + +const verifiedOwners = new WeakSet(); +export interface PrivateCellEvidenceOwner { + readonly root: string; readonly executionId: string; readonly runId: string; readonly attempt: "1"; + readonly producer: CellReportProducerIdentity; +} +export type PrivateCellLayout = "original" | "final" | "final-wrapper"; +/** Only the already verified preflight may enable private cell asset handling. */ +export async function privateCellEvidenceOwner(context: { + private: true; root: string; runId: string; attempt: string; executionIds: string[]; cellReportProducer?: CellReportProducerIdentity; +}, executionId: string): Promise { + if (context.private !== true || !context.executionIds.includes(executionId) || !/^chat-features\.runner-opencode\.local\.[a-z0-9][a-z0-9.-]{0,159}$/.test(executionId) + || executionId.includes("..") || !/^[1-9][0-9]{0,19}$/.test(context.runId) || context.attempt !== "1" || !context.cellReportProducer) + throw new Error("Invalid private cell report owner"); + const root = await realpath(context.root), metadata = await lstat(context.root); + if (root !== path.resolve(context.root) || !metadata.isDirectory() || metadata.isSymbolicLink() || metadata.mode & 0o077 + || typeof process.getuid === "function" && metadata.uid !== process.getuid()) throw new Error("Invalid private cell report custody"); + const producer = await assertCellReportProducer(context.cellReportProducer); + const owner = Object.freeze({ root, executionId, runId: context.runId, attempt: "1" as const, producer }); + verifiedOwners.add(owner); return owner; +} +export function cellTraceAsset(relative: string, owner: PrivateCellEvidenceOwner, layout: PrivateCellLayout) { + if (!verifiedOwners.has(owner) || !["original", "final", "final-wrapper"].includes(layout)) throw new Error("Unverified private cell report owner"); + const segments = relative.split("/"); + const reserved = segments.some((part, index) => part === "html-report" && segments[index + 1] === "trace"); + if (!reserved) return { reserved: false }; + const scenario = owner.executionId.slice("chat-features.runner-opencode.local.".length); + const final = `gha-${owner.runId}-${owner.attempt}-${owner.executionId}/chat-features/runner-opencode/local/${scenario}/attempt-${owner.attempt}/html-report/trace/`; + const prefix = layout === "original" ? "html-report/trace/" : layout === "final" ? final : `html-report/${final}`; + const canonical = relative.startsWith(prefix) ? canonicalGeneratedReportAsset(`html/trace/${relative.slice(prefix.length)}`) : undefined; + return { reserved: true, canonical }; +} +export async function assertPrivateCellDirectories(owner: PrivateCellEvidenceOwner, source: string, destination?: string) { + if (!verifiedOwners.has(owner)) throw new Error("Unverified private cell report owner"); + const root = await realpath(owner.root), metadata = await lstat(root); + if (root !== owner.root || !metadata.isDirectory() || metadata.mode & 0o077 + || typeof process.getuid === "function" && metadata.uid !== process.getuid()) throw new Error("Private cell report custody changed"); + for (const [index, directory] of [source, ...(destination ? [destination] : [])].entries()) { + const resolved = path.resolve(directory); + if (!resolved.startsWith(`${root}${path.sep}`)) throw new Error("Cell report escaped private staging"); + let current = root; + for (const part of path.relative(root, resolved).split(path.sep)) { + current = path.join(current, part); + const info = await lstat(current).catch((error: NodeJS.ErrnoException) => { if (error.code === "ENOENT" && index > 0) return undefined; throw error; }); + if (!info) break; + if (!info.isDirectory() || info.isSymbolicLink()) throw new Error("Invalid private cell report directory"); + } + } + if (destination && (path.resolve(destination) === path.resolve(source) || path.resolve(source).startsWith(`${path.resolve(destination)}${path.sep}`) + || path.resolve(destination).startsWith(`${path.resolve(source)}${path.sep}`))) throw new Error("Overlapping private cell report directories"); +} + +export async function readPrivateCellTraceAsset(input: { + owner: PrivateCellEvidenceOwner; layout: PrivateCellLayout; relative: string; file: string; remaining: number; secrets: readonly string[]; +}): Promise { + const asset = cellTraceAsset(input.relative, input.owner, input.layout); + const secrets = normalizedSecrets(input.secrets).flatMap(value => [value, encodeURIComponent(value), Buffer.from(value).toString("base64"), Buffer.from(value).toString("base64url")]); + assertSecretFree(input.relative, secrets, "private cell trace path"); + if (!asset.reserved) return; + if (!asset.canonical) throw new Error("Unsupported private cell trace owner"); + await assertPrivateCellDirectories(input.owner, path.dirname(input.file)); + const metadata = await lstat(input.file), canonical = await lstat(asset.canonical); + if (!metadata.isFile() || metadata.isSymbolicLink() || metadata.size > input.remaining + || !canonical.isFile() || canonical.isSymbolicLink() || canonical.size !== metadata.size) + throw new Error("Invalid private cell trace asset"); + const bytes = await boundedSource(input.file, input.remaining); + if (!bytes.equals(await boundedSource(asset.canonical, bytes.length))) throw new Error("Private cell trace content differs from canonical producer"); + assertSecretFree(bytes, secrets, "private cell trace bytes"); + return bytes; +} diff --git a/tests/runner-e2e/manual-private-retention.test.ts b/tests/runner-e2e/manual-private-retention.test.ts index 58812848cbd..5dc0588b1d9 100644 --- a/tests/runner-e2e/manual-private-retention.test.ts +++ b/tests/runner-e2e/manual-private-retention.test.ts @@ -1,5 +1,5 @@ import { expect, it } from "vitest"; -import { authorizeManualPrivateEvidence, MANUAL_PRIVATE_EXECUTIONS, MANUAL_PRIVATE_PROTOCOL, manualCellKind, manualPrivateSessionPolicy, requiresPrivateManualEvidence } from "./manual-private-retention.js"; +import { authorizeManualPrivateEvidence, canonicalManualIds, MAX_MANUAL_PRIVATE_CELLS, MANUAL_PRIVATE_EXECUTIONS, MANUAL_PRIVATE_PROTOCOL, manualCellKind, manualPrivateSessionPolicy, requiresPrivateManualEvidence } from "./manual-private-retention.js"; const workflowSha = "a".repeat(40), targetSha = "b".repeat(40); const identity = { workflowSha, targetSha, runId: "123", attempt: "1", executionIds: [MANUAL_PRIVATE_EXECUTIONS[0]!] }; const approval = () => ({ version: 1, approvals: [{ workflowSha, targetSha, runId: "123", attempt: "1", executionIds: identity.executionIds, protocol: MANUAL_PRIVATE_PROTOCOL, launchSha256: "c".repeat(64) }] }); @@ -21,7 +21,7 @@ it.each(["workflow", "target", "ids", "protocol", "extra", "duplicate", "empty", expect(() => authorizeManualPrivateEvidence(i, JSON.stringify(p))).toThrow(); }); it("unknown and mixed manual cases cannot acquire private role authority", () => { - expect(() => authorizeManualPrivateEvidence({ ...identity, executionIds: ["chat-features.runner-opencode.local.access.invite-teammate"] }, JSON.stringify(approval()))).toThrow(); + expect(() => authorizeManualPrivateEvidence({ ...identity, executionIds: ["chat-features.runner-opencode.local.onboarding.future-unknown"] }, JSON.stringify(approval()))).toThrow(); expect(() => authorizeManualPrivateEvidence({ ...identity, executionIds: [...identity.executionIds, "runner.native.local.other"] }, JSON.stringify(approval()))).toThrow(); }); it("different cells have distinct immutable private object identities", () => { @@ -44,11 +44,11 @@ it("the supported private archive kind is collision-free and rejects arbitrary m }); it("keeps the full-eight report inline session policy within STS's 2048-character limit",()=>{ - const policy=manualPrivateSessionPolicy({bucket:"b".repeat(63),runId:"12345678901234567890",executionIds:[...MANUAL_PRIVATE_EXECUTIONS],mode:"report-write"}); + const policy=manualPrivateSessionPolicy({bucket:"b".repeat(63),runId:"12345678901234567890",executionIds:MANUAL_PRIVATE_EXECUTIONS.slice(0, 8),mode:"report-write"}); expect(JSON.stringify(policy).length).toBeLessThanOrEqual(2048); }); -it("all eight exact read/write cell scopes fit STS and never broaden to other cells or wildcards",()=>{ +it("all244 exact read/write cell scopes fit STS and never broaden to other cells or wildcards",()=>{ for(const id of MANUAL_PRIVATE_EXECUTIONS)for(const mode of ["cell-read","cell-write"]as const){ const p=manualPrivateSessionPolicy({bucket:"b".repeat(63),runId:"12345678901234567890",executionIds:[id],mode}); expect(JSON.stringify(p).length).toBeLessThanOrEqual(2048);expect(JSON.stringify(p)).not.toContain("*"); @@ -57,3 +57,36 @@ it("all eight exact read/write cell scopes fit STS and never broaden to other ce } expect(()=>manualPrivateSessionPolicy({bucket:"private-bucket",runId:"123",executionIds:[...MANUAL_PRIVATE_EXECUTIONS],mode:"cell-read"})).toThrow(); }); + +const enrolled = ["access.invite-teammate", "activity.routine-audit"].map(id => `chat-features.runner-opencode.local.${id}`); +function approvedSelection(ids: string[]) { + return { version: 1, approvals: [{ ...approval().approvals[0]!, executionIds: ids }] }; +} +it("the full frozen registry preserves exact run authority while a private run remains bounded to eight", () => { + expect(MANUAL_PRIVATE_EXECUTIONS).toHaveLength(244); + expect(new Set(MANUAL_PRIVATE_EXECUTIONS).size).toBe(244); + expect(MAX_MANUAL_PRIVATE_CELLS).toBe(8); + for (const ids of [...enrolled.map(id => [id]), MANUAL_PRIVATE_EXECUTIONS.slice(0, 8)]) { + expect(authorizeManualPrivateEvidence({ ...identity, executionIds: ids }, JSON.stringify(approvedSelection(ids))).executionIds).toEqual([...ids].sort()); + } + expect(canonicalManualIds(Array.from({ length: 32 }, (_, i) => `ordinary-${i}`))).toHaveLength(32); +}); +it.each([9, 10])("enrollment rejects %s private cells at identity, every approval row and session boundaries", count => { + const ids = MANUAL_PRIVATE_EXECUTIONS.slice(0, count); + expect(() => authorizeManualPrivateEvidence({ ...identity, executionIds: ids }, JSON.stringify(approvedSelection(ids)))).toThrow(); + const policy = approval(); + policy.approvals.push({ ...policy.approvals[0]!, runId: "124", executionIds: ids }); + expect(() => authorizeManualPrivateEvidence(identity, JSON.stringify(policy))).toThrow(); + for (const mode of ["report-write", "cell-read", "cell-write"] as const) + expect(() => manualPrivateSessionPolicy({ bucket: "private-bucket", runId: "123", executionIds: ids, mode })).toThrow(); +}); + +it("full catalog requires exact original-run approval for each registered execution", () => { + for (const id of MANUAL_PRIVATE_EXECUTIONS) { + const selected = [id]; + expect(authorizeManualPrivateEvidence({ ...identity, executionIds: selected }, JSON.stringify(approvedSelection(selected))).executionIds).toEqual(selected); + expect(() => authorizeManualPrivateEvidence({ ...identity, runId: "124", executionIds: selected }, JSON.stringify(approvedSelection(selected)))).toThrow(); + } + const duplicate = [MANUAL_PRIVATE_EXECUTIONS[0]!, MANUAL_PRIVATE_EXECUTIONS[0]!]; + expect(() => authorizeManualPrivateEvidence({ ...identity, executionIds: duplicate }, JSON.stringify(approvedSelection(duplicate)))).toThrow(); +}); diff --git a/tests/runner-e2e/manual-private-retention.ts b/tests/runner-e2e/manual-private-retention.ts index 82af9ed1ef2..94f8215b5eb 100644 --- a/tests/runner-e2e/manual-private-retention.ts +++ b/tests/runner-e2e/manual-private-retention.ts @@ -1,10 +1,252 @@ import { createHash } from "node:crypto"; +export const MAX_MANUAL_PRIVATE_CELLS = 8; export const MANUAL_PRIVATE_PROTOCOL = "paperclip.manual-private-evidence/v1"; export const MANUAL_PRIVATE_EXECUTIONS = Object.freeze([ - "activity.agent-feedback", "activity.work-timeline", "companies.import-preview", - "navigation-preferences.dismiss-announcement", "tasks.completion-with-gate", - "tasks.private-task", "secrets.personal-secret", "agents.choose-primary", + "activity.agent-feedback", + "activity.work-timeline", + "companies.import-preview", + "navigation-preferences.dismiss-announcement", + "tasks.completion-with-gate", + "tasks.private-task", + "secrets.personal-secret", + "agents.choose-primary", + "access.cli-authorization", + "access.invite-teammate", + "access.manage-membership", + "access.other-company", + "access.private-resource", + "access.sign-in", + "activity.audit-change", + "activity.company-overview", + "activity.find-runs", + "activity.routine-audit", + "agent-chat.chat-disabled", + "agent-chat.conference-room", + "agent-chat.multiple-handoffs", + "agent-chat.remember-and-follow-up", + "agent-chat.reopen-conversation", + "agent-chat.track-delegated-work", + "agents.agent-api-key", + "agents.ambiguous-cloud-code", + "agents.change-manager", + "agents.configure-required-agent", + "agents.edit-instructions", + "agents.existing-grok-bot", + "agents.hire-claude-code", + "agents.hire-denied", + "agents.hire-needs-account", + "agents.hire-needs-approval", + "agents.managed-profile", + "agents.pause-agent", + "agents.restore-instructions", + "agents.resume-agent", + "agents.retire-agent", + "app-permissions.ask-before-action", + "app-permissions.grant-one-agent", + "app-permissions.new-quarantined-action", + "app-permissions.read-allowed-write-off", + "app-permissions.revoke-account", + "app-permissions.test-as-agent", + "assistant-connections.connect-external-assistant", + "assistant-connections.consent-read-only", + "assistant-connections.decline-consent", + "assistant-connections.device-login", + "assistant-connections.revoke-personal-grant", + "assistant-connections.use-scoped-client", + "budgets-costs.explain-hard-stop", + "budgets-costs.repair-cost-recording", + "budgets-costs.resolve-budget", + "budgets-costs.set-budget", + "budgets-costs.spend-breakdown", + "budgets-costs.subscription-fees", + "cases.browse-cases", + "cases.case-history", + "cases.cases-disabled", + "cases.close-case", + "cases.create-case", + "cases.handoff-same-task", + "cases.related-work", + "cases.update-case-facts", + "chat-channels.channel-file", + "chat-channels.connect-slack", + "chat-channels.link-sender", + "chat-channels.outbound-email", + "chat-channels.provider-disabled", + "chat-channels.repair-endpoint", + "chat-channels.reply-in-thread", + "chat-channels.reply-to-email", + "cli-operations.choose-cli-context", + "cli-operations.create-from-cli", + "cli-operations.export-document-cli", + "cli-operations.feedback-bundle", + "cli-operations.inspect-run-cli", + "cli-operations.owned-dev-instance", + "companies.agent-observability", + "companies.archive-company", + "companies.configure-decision-model", + "companies.export-company", + "companies.fast-responses", + "companies.prompt-injection-guard", + "companies.rename-company", + "companies.switch-company", + "connection-setup.channel-prerequisites", + "connection-setup.choose-provider", + "connection-setup.notion-agent-unassigned", + "connection-setup.notion-connected", + "connection-setup.notion-declined", + "connection-setup.notion-expired", + "connection-setup.notion-missing", + "connection-setup.notion-wrong-account", + "connection-setup.return-from-login", + "connection-setup.save-and-resume", + "connection-setup.secure-api-key", + "connection-setup.unreachable-mcp", + "connection-setup.wrong-person-resolves", + "developer-labs.component-reference", + "developer-labs.gated-lab", + "developer-labs.interaction-fixture", + "developer-labs.long-thread-performance", + "documents-artifacts.find-artifact", + "documents-artifacts.private-download", + "documents-artifacts.produce-download", + "documents-artifacts.resolve-feedback", + "documents-artifacts.restore-document", + "documents-artifacts.revise-delivery", + "documents-artifacts.write-plan", + "execution-environments.add-computer", + "execution-environments.assign-computer", + "execution-environments.choose-capable-target", + "execution-environments.prepare-image", + "execution-environments.test-computer", + "gateways-profiles.connect-client", + "gateways-profiles.create-gateway", + "gateways-profiles.define-profile", + "gateways-profiles.inspect-gateway-error", + "gateways-profiles.rotate-token", + "goals.align-existing-work", + "goals.create-goal-tree", + "goals.explain-purpose", + "inbox-search.archive-done", + "inbox-search.archive-partial-failure", + "inbox-search.archive-without-grant", + "inbox-search.find-design", + "inbox-search.notification-audience", + "inbox-search.restore-archive", + "inbox-search.show-decisions", + "inbox-search.show-my-work", + "instance-operations.backup-and-restore", + "instance-operations.diagnose-health", + "instance-operations.instance-settings", + "instance-operations.provision-runtime", + "instance-operations.restart-service", + "instance-operations.update-instance", + "navigation-preferences.broken-link", + "navigation-preferences.edit-profile", + "navigation-preferences.find-project-view", + "navigation-preferences.quick-search", + "navigation-preferences.sidebar-order", + "onboarding.choose-first-runtime", + "onboarding.first-work", + "onboarding.new-company", + "onboarding.new-instance", + "onboarding.resume-wizard", + "plugins.configure-plugin", + "plugins.disable-plugin", + "plugins.install-plugin", + "plugins.use-plugin-tool", + "projects.add-repository", + "projects.archive-project", + "projects.create-project", + "projects.private-project", + "projects.project-defaults", + "projects.project-work", + "projects.replace-repository", + "projects.share-project", + "questions-and-approvals.accept-one-proposal", + "questions-and-approvals.ambiguous-yes", + "questions-and-approvals.approve-tool-once", + "questions-and-approvals.decline-tool", + "questions-and-approvals.defer-and-reopen", + "questions-and-approvals.duplicate-answer", + "questions-and-approvals.formal-hire-revision", + "questions-and-approvals.multi-part-question", + "questions-and-approvals.necessary-question", + "questions-and-approvals.reject-and-revise", + "questions-and-approvals.review-plan", + "questions-and-approvals.wrong-reviewer", + "recovery.browser-reconnect", + "recovery.controller-restart", + "recovery.explain-stall", + "recovery.post-stop-message", + "recovery.retry-exhausted", + "recovery.source-action", + "recovery.unknown-email-outcome", + "recovery.workspace-repair", + "routines.create-daily-brief", + "routines.manage-other-owner", + "routines.pause-owned", + "routines.plugin-managed-routine", + "routines.read-history", + "routines.run-now", + "routines.webhook-trigger", + "runs-adapters.inspect-failed-run", + "runs-adapters.list-model-options", + "runs-adapters.reset-session", + "runs-adapters.upgrade-runner", + "runs-adapters.validate-hire", + "secrets.ask-for-plaintext", + "secrets.configure-company-secret", + "secrets.least-scope-delivery", + "secrets.secret-proposal", + "secrets.vault-reference", + "skills.author-skill", + "skills.enable-for-agent", + "skills.explicit-restriction", + "skills.find-and-install", + "skills.fork-read-only", + "skills.review-update", + "skills.studio-test", + "status-cards.archive-card", + "status-cards.create-card", + "status-cards.explain-card", + "status-cards.other-authors-card", + "status-cards.refresh-card", + "steering.approval-delivery", + "steering.discard-queued-message", + "steering.edit-queued-message", + "steering.interrupt-with-direction", + "steering.preserve-draft", + "steering.queue-follow-up", + "steering.steer-active", + "steering.stop-versus-pause", + "task-coordination.block-on-design", + "task-coordination.delegate-with-context", + "task-coordination.independent-review", + "task-coordination.pause-tree", + "task-coordination.reject-dependency-cycle", + "tasks.ambiguous-task", + "tasks.assignment-conflict", + "tasks.attachment-brief", + "tasks.comment-with-reference", + "create-backlog", + "delegate-announcement", + "tasks.edit-properties", + "tasks.organize-open-work", + "tasks.share-private-task", + "tasks.website-task", + "tasks.website-without-project", + "teams.configure-team", + "teams.find-team", + "teams.install-team", + "teams.preview-install", + "workspaces.close-and-reopen", + "workspaces.inspect-git", + "workspaces.locate-files", + "workspaces.prepare-repository", + "workspaces.preview-service", + "workspaces.private-repository-no-access", + "workspaces.recover-lost-sandbox", ].map(id => `chat-features.runner-opencode.local.${id}`)); const sha = /^[0-9a-f]{40}$/; const execution = /^[A-Za-z0-9][A-Za-z0-9._-]{0,249}$/; @@ -18,6 +260,11 @@ export function canonicalManualIds(value: unknown): string[] { throw new Error("Invalid exact manual evidence selection"); return [...value].sort(); } +export function canonicalPrivateManualIds(value: unknown): string[] { + const ids = canonicalManualIds(value); + if (ids.length > MAX_MANUAL_PRIVATE_CELLS) throw new Error("Private manual evidence exceeds the per-run cell limit"); + return ids; +} export function requiresPrivateManualEvidence(id: string) { return id === "chat-features" || id.startsWith("chat-features."); } @@ -27,7 +274,7 @@ export interface ManualRetentionIdentity { export function authorizeManualPrivateEvidence(identity: ManualRetentionIdentity, approvalJson: string) { if (!sha.test(identity.workflowSha) || !sha.test(identity.targetSha) || !/^[1-9][0-9]{0,19}$/.test(identity.runId) || identity.attempt !== "1") throw new Error("Invalid manual private evidence identity"); - const ids = canonicalManualIds(identity.executionIds); + const ids = canonicalPrivateManualIds(identity.executionIds); if (ids.some(id => !MANUAL_PRIVATE_EXECUTIONS.includes(id))) throw new Error("Unsupported private manual evidence selection"); if (Buffer.byteLength(approvalJson) > 65536) throw new Error("Invalid protected manual evidence approval"); let policy: unknown; @@ -38,7 +285,7 @@ export function authorizeManualPrivateEvidence(identity: ManualRetentionIdentity if (!keys(row, ["workflowSha", "targetSha", "runId", "attempt", "executionIds", "protocol", "launchSha256"]) || !sha.test(String(row.workflowSha)) || !sha.test(String(row.targetSha)) || !/^[1-9][0-9]{0,19}$/.test(String(row.runId)) || row.attempt !== "1" || row.protocol !== MANUAL_PRIVATE_PROTOCOL || !/^[0-9a-f]{64}$/.test(String(row.launchSha256))) throw new Error("Invalid protected manual evidence approval"); - const selected = canonicalManualIds(row.executionIds); + const selected = canonicalPrivateManualIds(row.executionIds); return row.workflowSha === identity.workflowSha && row.targetSha === identity.targetSha && row.runId === identity.runId && row.attempt === identity.attempt && selected.join() === ids.join(); }); if (matches.length !== 1) throw new Error("Exact protected manual evidence approval missing or ambiguous"); @@ -52,7 +299,7 @@ export function manualCellKind(id: string) { export function manualPrivateSessionPolicy(input: { bucket: string; runId: string; executionIds: string[]; mode: "cell-write" | "cell-read" | "report-write" }) { if (!/^[a-z0-9][a-z0-9.-]{1,61}[a-z0-9]$/.test(input.bucket) || !/^[1-9][0-9]{0,19}$/.test(input.runId)) throw new Error("Invalid manual private storage identity"); - const ids = canonicalManualIds(input.executionIds); + const ids = canonicalPrivateManualIds(input.executionIds); if (ids.some(id => !MANUAL_PRIVATE_EXECUTIONS.includes(id)) || input.mode !== "report-write" && ids.length !== 1) throw new Error("Private archive sessions require one exact cell or the report writer"); const prefix = `arn:aws:s3:::${input.bucket}/runner-e2e-private/bounded-v1/campaigns/gha-${input.runId}-1/`; diff --git a/tests/runner-e2e/manual-private-transfer.test.ts b/tests/runner-e2e/manual-private-transfer.test.ts index c163340bd9e..95005194848 100644 --- a/tests/runner-e2e/manual-private-transfer.test.ts +++ b/tests/runner-e2e/manual-private-transfer.test.ts @@ -1,13 +1,14 @@ import { spawnSync } from "node:child_process"; +import { gunzipSync } from "node:zlib"; import { createRequire } from "node:module"; import { afterEach, expect, it } from "vitest"; -import { mkdtemp, mkdir, readFile, rm, writeFile, copyFile } from "node:fs/promises"; +import { mkdtemp, mkdir, readFile, rm, writeFile, copyFile, readdir } from "node:fs/promises"; import os from "node:os"; import path from "node:path"; import { createHash } from "node:crypto"; import { validateSealedInventory } from "./manual-private-transfer.js"; import { sealManualResultEvidence, type ManualPrivateHandoff } from "./manual-private-handoff.js"; -import { MANUAL_PRIVATE_PROTOCOL } from "./manual-private-retention.js"; +import { MANUAL_PRIVATE_PROTOCOL, MANUAL_PRIVATE_EXECUTIONS } from "./manual-private-retention.js"; const roots:string[]=[]; afterEach(async()=>{await Promise.all(roots.splice(0).map(root=>rm(root,{recursive:true,force:true})));}); async function fixture(){const root=await mkdtemp(path.join(os.tmpdir(),"manual-transfer-test-"));roots.push(root);await mkdir(path.join(root,"results"));const context={root,protocol:MANUAL_PRIVATE_PROTOCOL,private:true,workflowSha:"a".repeat(40),targetSha:"b".repeat(40),runId:"123",attempt:"1",executionIds:["chat-features.runner-opencode.local.activity.agent-feedback"]} as ManualPrivateHandoff;return{root,context};} @@ -34,16 +35,52 @@ it("rejects a forged inventory that lists an unscannable original carrier",async await expect(validateSealedInventory(dir,"inventory.json","id")).rejects.toThrow(/row/); }); -it.each([false, true])("actual prepare checks new report dependency before credentials (tampered=%s)", async tampered => { +it.each([ + ["activity.agent-feedback", "none"], + ["activity.agent-feedback", "assets"], + ["access.invite-teammate", "none"], + ["activity.routine-audit", "none"], + ["access.invite-teammate", "old-retention"], + ["activity.routine-audit", "old-retention"], +])("actual prepare checks exact protocol for %s before credentials (tampered=%s)", async (caseId, tampered) => { const root=await mkdtemp(path.join(os.tmpdir(),"synthetic-prepare-dependency-"));roots.push(root); const target=path.join(root,"target"),modules=path.join(target,"tests/runner-e2e");await mkdir(modules,{recursive:true}); const trusted=path.resolve(import.meta.dirname,"../.."),launch=Buffer.from("synthetic approved launcher; never executed");await writeFile(path.join(modules,"launch.ts"),launch); for(const name of ["manual-private-handoff.ts","manual-private-retention.ts","manual-private-report-assets.ts"])await copyFile(path.join(import.meta.dirname,name),path.join(modules,name)); - if(tampered)await writeFile(path.join(modules,"manual-private-report-assets.ts"),"synthetic incompatible dependency; never executed"); - const workflowSha="a".repeat(40),targetSha="b".repeat(40),id="chat-features.runner-opencode.local.activity.agent-feedback",output=path.join(root,"outputs.txt"); + if(tampered === "assets")await writeFile(path.join(modules,"manual-private-report-assets.ts"),"synthetic incompatible dependency; never executed"); + // Exact 436 retention postimage (e133bfac...), compressed only to keep this synthetic fixture small. + if(tampered === "old-retention"){ + const oldRetention=gunzipSync(Buffer.from("H4sIAAAAAAAC/61YbW/bNhD+nl/BCMFgAZYSp222OUuHtAi2bGkTxG03zPBmRjrbbGRRIyk7rqv/vuOL3mq7abd9SkQe7+W5u4dHs3nGhSJrEgmgCn6mckYKMhF8TryUx9CPxCpT3Dvd24MHIxrxVCry6vz12/Orv25uL9+dv7nAv9dvrl9eX5Ez4mU0AxElLAvnNM1pEmSCLVB3AAsWQxrB4aKH+j6n7uL3i5dv31xevx6gwuu79xCpcCIAPkBnuEeIRyPFFkytQjqFVAUTgPiORvdet7G15OI+UGwOCUtB70R8ntGUgQyZCRr9Qo9g6XW1ypQu2JQqxlO9PgGhPZVhzOScSRnQNOU5rszRnlamqLyXoVaZgDm0ZGoWoAKw6ux+Gbn+0qckIMwK10FIniI0dsH4rSNBjTPOJWjI5lSsUNcIUcw6LCZnz8k4mlEdLVW5QN9EnqYgAp6hq5iqMOERTcKDNYuLse+f7llo5YwiiId/Do+C72kwGa2fHhUHh+UuPECU6wCszHnwBw0+oOSo/jf8Kxitj7rHT7835yYIgzlwDyvZWdAkhz7J0/uUL9Mu6sswWRD3iVSCpdPhyO8TI0SYJLcQcRH/YLe65annZI2QIQ65SMn+vpX+5huiVhnwiTt9doalxU0peHpz/1wIugqZNH+tHz6qIXrTlUztoR9KzHjHD99zlnZ8o20YhmHp7qi9f7pXlPVZhRvRlKcMEX5lqvoy/jR4v47ZBMQmpLPVS/LxI7FRhgmkUzXTC63v5+TJsYml2pB8Dq4KHCz4sa8xsTY9o7TKZqhAKpT3jbEUlmQAqoKCfQBztmnTYqdmgi+N/IUQXHS8yxSF0BQ8YGMR29CkbGQiIQGDjoeQVSnUwBrVJarb4BTwd86wim9si1hQL5xi9LwE028WhwZAx9zqAxM6i0OpqFDyN+zDTlsg9JoesFSBmFD03tq8BYWdhy5dxvqvWhmDmj0mCV8OZrT05JSggSmo1hL24GVcf1KlYJ6peqHKCNZLXR/bAKG5mnGBqbFuOVwaiFjv+rvc7hKaZYIj8L8gt7ThM6WIRFCWhT0RNoK0RbkpUoVsBZAiepocNEMgJ/Q0JXxywiDi25y4JYeKrdee90ipuSJz3FkXW6nO1pqlLxZLJK4trVnZbibAHNRY4LFmR+3vvH5ClkZJHoM0veRvOv02lXmmEwlx5fEjXaIdeJFP8IoJ71YKrkz7dZrJ85EATp49e3Li70YJpZWhrg1zpSZrDa8nkvGERauKqPSyEljnbgMR/GVw/TrMqJDQduQUh4GIqmiGwv/NFVKUdWhY2VrukqG3wLtQY6PvQCcuvZEpICsUOglTPj1Thm1WdWLV6ebZarFJrY8U4JdCa0twruEBXYYbJicsQa7paDNn9pJrQoDLOv5GE9rBwjWc/jC9ZJCxHaT/bRa0/tbe8ogn+v+EIpvM8PDxsxOHYd3TA0MI2myr8S0YZKdszQCPUUDjkCUBc0B/fkIA5XLpulnfNVE6m+X0cvJ0m71m4H4V0n/OcJlj28EovpVrtAMbNFPdWZ/gbS6wbQys55YW3m3JetnJGYjbMnbJ7ZeYtyTKRZQpY2qORJoXywmIkKKiK1fjZQ+ZRtzCTRdmSHgcXKJHakwd4YLQ+R2b5jyXrRGiPfOvCU4UrLrn2ndqWxQj8LukLKz+rqrqmgw1q6ZPXDm5WIdHo/12WdUXbHmifjN1PGmbzg/zLMbFNo3iM2KqK9abwYNn/DM3RR9pGEdbKl2ZFVsnJYviS0iSX1kab4xGhlC2DX6PXbBVWuo3QPuKdbkYu2dchB4EB+vPBY1226EW490RuQlnAFKz+40hzw5LsxzHpzW5y6N7UDsHrR1zFZnrNysOiNrXpWD4HCMf3Sf6HZsvAeb557aLBozIM+Xzx7198OnT6570inK9Gne0n6F18tHRyMg6Svyq2UcqLvBh+JWjjzH3f849ZpgzWjW8lsXbICKfaN0tftgeqcs6oSKasYUejkz6ZfkgIDwF99rQWdMUoWZArDlizIkmDPqxzh4QiTEVaZ8uZV8+6ff7+ApuZKg4dG9lOIby54jDO3zQxxAHi95hROcZZdNUHk5nNCjPmowVQe9wXJvTPmncdbSNh/nB2vpRHKw32tUvwg8sGzd9LseuNXlnZxss2eOj3nHQOwp63+JNPlDoof6toU+GBsc1ucCJMcJv7zzBCwNlzk0zoYCHEf8E6oUJ1SnUT0L9g4PZuWJ4dBUl8JKnE2RbYX7nqPftSUukVzy63yF2Cxn6bRY/q+gmv0O58yjC1L5IuP1dpmVpibmQM5ahGiV4Ijc0GIC8UZfcguS5iLCnP5vfMSkspeNF0WmUqn0t1t3/Ixl+AZIWibZ5k/hwklD1CtOOA5zO+xD/dnX28W+BJJhwGqOlCFimwvfI/GMcwooRQdUj/0sdNOKPeXmTb/VyQ3W7TVF7XaqO1q2EKVFt2cTp/8tATYwjUpR8Y54VlpzZpBzU/XoUPz56+p3/xRxR9g08RADIfpoXBm8GJGFzplpXlhXUV88/58Wt2uAUAAA=", "base64")); + expect(createHash("sha256").update(oldRetention).digest("hex")).toBe("e133bfacac985d95bbf7812d369569f30709aeebf324a0d37416c65c94ffa98a"); + await writeFile(path.join(modules,"manual-private-retention.ts"),oldRetention); + } + const workflowSha="a".repeat(40),targetSha="b".repeat(40),id=`chat-features.runner-opencode.local.${caseId}`,output=path.join(root,"outputs.txt"); const require=createRequire(import.meta.url),tsx=require.resolve("tsx",{paths:[path.join(trusted,"server")]}); const result=spawnSync(process.execPath,["--import",tsx,path.join(import.meta.dirname,"manual-private-transfer.ts"),"prepare"],{encoding:"utf8",timeout:10000,env:{PATH:process.env.PATH,NODE_OPTIONS:"",NODE_PATH:"",SELECTED_EXECUTION_ID:id,MANUAL_PRIVATE_IDS:JSON.stringify([id]),MANUAL_PRIVATE_APPROVALS_JSON:JSON.stringify({version:1,approvals:[{workflowSha,targetSha,runId:"123",attempt:"1",executionIds:[id],protocol:MANUAL_PRIVATE_PROTOCOL,launchSha256:createHash("sha256").update(launch).digest("hex")}]}),GITHUB_SHA:workflowSha,TARGET_SHA:targetSha,GITHUB_RUN_ID:"123",GITHUB_RUN_ATTEMPT:"1",RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET:"private-bucket",TARGET_ROOT:target,RUNNER_TEMP:root,GITHUB_OUTPUT:output}}); - expect(result.error).toBeUndefined();expect(result.status).toBe(tampered?1:0); - if(tampered){expect(result.stderr).toContain("Restricted manual evidence operation failed");await expect(readFile(output)).rejects.toMatchObject({code:"ENOENT"});} + expect(result.error).toBeUndefined();expect(result.status).toBe(tampered !== "none"?1:0); + if(tampered !== "none"){expect(result.stderr).toContain("Restricted manual evidence operation failed");await expect(readFile(output)).rejects.toMatchObject({code:"ENOENT"});} else expect(await readFile(output,"utf8")).toContain("session_policy="); }); + +it.each([8, 9, 10])("enrollment actual report session emits only eight fixed slots and rejects count %s before outputs", async count => { + const root=await mkdtemp(path.join(os.tmpdir(),"synthetic-report-cardinality-"));roots.push(root); + const ids=count === 8 ? MANUAL_PRIVATE_EXECUTIONS.slice(0,8) : MANUAL_PRIVATE_EXECUTIONS.slice(0,count); + expect(ids).toHaveLength(count); + const workflowSha="a".repeat(40),targetSha="b".repeat(40),output=path.join(root,"outputs.txt"); + const trusted=path.resolve(import.meta.dirname,"../.."),require=createRequire(import.meta.url),tsx=require.resolve("tsx",{paths:[path.join(trusted,"server")]}); + const result=spawnSync(process.execPath,["--import",tsx,path.join(import.meta.dirname,"manual-private-transfer.ts"),"report-session"],{ + encoding:"utf8",timeout:10000,env:{PATH:process.env.PATH,NODE_OPTIONS:"",NODE_PATH:"",MANUAL_PRIVATE_IDS:JSON.stringify(ids), + MANUAL_PRIVATE_APPROVALS_JSON:JSON.stringify({version:1,approvals:[{workflowSha,targetSha,runId:"123",attempt:"1",executionIds:ids,protocol:MANUAL_PRIVATE_PROTOCOL,launchSha256:"c".repeat(64)}]}), + GITHUB_SHA:workflowSha,TARGET_SHA:targetSha,GITHUB_RUN_ID:"123",GITHUB_RUN_ATTEMPT:"1",RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET:"private-bucket",GITHUB_OUTPUT:output}}); + expect(result.error).toBeUndefined();expect(result.status).toBe(count===8?0:1); + if(count===8){ + const lines=(await readFile(output,"utf8")).trim().split("\n"),keys=lines.map(line=>line.split("=",1)[0]); + expect(keys).toEqual(["session_policy","cell_count",...Array.from({length:8},(_,i)=>`read_policy_${i}`)]); + expect(lines).toContain("cell_count=8"); + for(const line of lines.filter(line=>line.startsWith("read_policy_")))expect(JSON.stringify(JSON.parse(line.slice(line.indexOf("=")+1))).length).toBeLessThanOrEqual(2048); + }else{ + expect(result.stderr).toContain("Restricted manual evidence operation failed"); + expect(result.stdout).toBe("");await expect(readFile(output)).rejects.toMatchObject({code:"ENOENT"}); + expect(await readdir(root)).toEqual([]); + } +}); diff --git a/tests/runner-e2e/manual-private-transfer.ts b/tests/runner-e2e/manual-private-transfer.ts index 0dd8271c69b..04370a86ca9 100644 --- a/tests/runner-e2e/manual-private-transfer.ts +++ b/tests/runner-e2e/manual-private-transfer.ts @@ -2,8 +2,8 @@ import { createHash } from "node:crypto"; import { appendFile, cp, lstat, mkdir, mkdtemp, readFile, readdir, realpath, writeFile } from "node:fs/promises"; import path from "node:path"; import { pathToFileURL } from "node:url"; -import { canonicalGeneratedReportAsset, isTrustedGeneratedReportAsset } from "./manual-private-report-assets.js"; -import { sealManualResultEvidence } from "./manual-private-handoff.js"; +import { canonicalGeneratedReportAsset, isTrustedGeneratedReportAsset, inspectCellReportProducer, privateCellEvidenceOwner, cellTraceAsset, readPrivateCellTraceAsset } from "./manual-private-report-assets.js"; +import { sealManualResultEvidence, type ManualPrivateHandoff } from "./manual-private-handoff.js"; import { normalizedSecrets } from "./redaction.js"; import { CREDENTIAL_NAMES } from "./types.js"; import { isStandardEvidenceFile } from "./evidence.js"; @@ -48,9 +48,13 @@ async function verify(env: NodeJS.ProcessEnv) { || identity.runId !== approval.runId || identity.attempt !== "1" || identity.root !== root) throw new Error("Private preflight identity changed"); const publisher = new BoundedHistoryPublisher(path.join(root, "preflight-requests.json")); await verifyPrivateHistory(publisher, bucket); - await writeFile(path.join(root, "preflight.json"), JSON.stringify({ ...identity, status: "verified" }), { mode: 0o600, flag: "wx" }); + const preflight = JSON.stringify({ ...identity, status: "verified", cellReportProducer: await inspectCellReportProducer() }); + if (Buffer.byteLength(preflight) > 65536) throw new Error("Private preflight exceeds its byte allowance"); + await writeFile(path.join(root, "preflight.json"), preflight, { mode: 0o600, flag: "wx" }); } -export async function validateSealedInventory(directory: string, inventory: string, id?: string, treeKind: "standard" | "generated-report" = "standard") { +export async function validateSealedInventory(directory: string, inventory: string, id?: string, treeKind: "standard" | "generated-report" = "standard", context?: ManualPrivateHandoff) { + const cellOwner = treeKind === "standard" && context?.cellReportProducer + ? await privateCellEvidenceOwner(context, id ?? (context.executionIds.length === 1 ? context.executionIds[0]! : "")) : undefined; const receipt = JSON.parse(await readFile(path.join(directory, inventory), "utf8")); if (receipt.protocol !== MANUAL_PRIVATE_PROTOCOL || id !== undefined && receipt.executionId !== id || receipt.complete !== true || !Array.isArray(receipt.files) || !receipt.files.length || receipt.files.length > 4096 @@ -61,15 +65,19 @@ export async function validateSealedInventory(directory: string, inventory: stri if (!row || Object.keys(row).sort().join() !== "bytes,path,sha256" || typeof row.path !== "string" || row.path.split("/").some((part: string) => !part || part === "." || part === "..") || row.path.includes("\\") || expected.has(row.path) || !Number.isSafeInteger(row.bytes) || row.bytes < 0 || !/^[0-9a-f]{64}$/.test(row.sha256)) throw new Error("Invalid original evidence inventory row"); + const cellAsset = cellOwner ? cellTraceAsset(row.path, cellOwner, id === undefined ? "final" : "original") : undefined; + if (cellAsset?.reserved && !cellAsset.canonical) throw new Error("Invalid private cell trace owner"); const standard = isStandardEvidenceFile(id === undefined ? `html-report/${row.path}` : row.path); const canonical = id === undefined && treeKind === "generated-report" ? canonicalGeneratedReportAsset(row.path) : undefined; const staticReportOwner = id === undefined && treeKind === "generated-report" && (row.path.startsWith("html/trace/") || canonical !== undefined); - if (staticReportOwner ? !canonical : !standard) throw new Error("Invalid original evidence inventory row"); + if (cellAsset?.reserved ? !cellAsset.canonical : staticReportOwner ? !canonical : !standard) throw new Error("Invalid original evidence inventory row"); if (row.bytes > 256 * 1024 * 1024 - bytes) throw new Error("Original evidence byte accounting changed"); expected.add(row.path); bytes += row.bytes; const file = path.join(directory, row.path), metadata = await lstat(file); if (!metadata.isFile() || metadata.isSymbolicLink() || metadata.size !== row.bytes) throw new Error("Original evidence changed after its scan"); - const data = await readFile(file); + const data = cellAsset?.reserved + ? (await readPrivateCellTraceAsset({ owner: cellOwner!, layout: id === undefined ? "final" : "original", relative: row.path, file, remaining: row.bytes, secrets: [] }))! + : await readFile(file); if (hash(data) !== row.sha256) throw new Error("Original evidence changed after its scan"); if (staticReportOwner && !await isTrustedGeneratedReportAsset(row.path, data)) throw new Error("Invalid original evidence inventory row"); } @@ -88,8 +96,8 @@ export async function validateSealedInventory(directory: string, inventory: stri if (actual.size !== expected.size) throw new Error("Sealed evidence is incomplete"); return receipt; } -export async function validateSealedOriginals(root: string, id: string) { - const receipt = await validateSealedInventory(path.join(root, "originals", manualCellKind(id)), "original-evidence-inventory.json", id); +export async function validateSealedOriginals(root: string, id: string, context?: ManualPrivateHandoff) { + const receipt = await validateSealedInventory(path.join(root, "originals", manualCellKind(id)), "original-evidence-inventory.json", id, "standard", context); const missing = receipt.missingRequiredEvidence; const allowed = new Set(["final-state.png", "result.json", "junit.xml", "html-report/index.html", "snapshots/fixtures.json", "snapshots/api-state.json", "blob-report/*.zip"]); if (![true, false, null].includes(receipt.requiredEvidenceComplete) || !Array.isArray(missing) || new Set(missing).size !== missing.length @@ -109,11 +117,11 @@ async function uploadCell(env: NodeJS.ProcessEnv) { || context.runId !== approval.runId || context.executionIds.join() !== id || context.root !== root) throw new Error("Private evidence preflight identity changed"); if (await lstat(path.join(root, "contamination.json")).then(() => true, error => { if (error.code === "ENOENT") return false; throw error; })) throw new Error("Contaminated private evidence cannot be uploaded"); - const originals = await validateSealedOriginals(root, id); + const originals = await validateSealedOriginals(root, id, context); const evidenceCompleteness = cellCompleteness(id, originals); const bundle = path.join(root, "sealed-bundle"); await mkdir(bundle, { mode: 0o700 }); const results = path.join(root, "sealed-results"); - await validateSealedInventory(path.join(results, "html-report"), "../result-evidence-inventory.json"); + await validateSealedInventory(path.join(results, "html-report"), "../result-evidence-inventory.json", undefined, "standard", context); await mkdir(path.join(bundle, "originals"), { mode: 0o700 }); await cp(path.join(root, "originals", manualCellKind(id)), path.join(bundle, "originals", manualCellKind(id)), { recursive: true, errorOnExist: true, force: false }); await cp(results, path.join(bundle, "sealed-results"), { recursive: true, errorOnExist: true, force: false }); @@ -146,10 +154,10 @@ async function downloadCell(env: NodeJS.ProcessEnv) { const identity = JSON.parse(await readFile(path.join(directory, "identity.json"), "utf8")); if (identity.targetSha !== approval.targetSha || identity.workflowSha !== approval.workflowSha || identity.runId !== approval.runId || identity.executionIds.join() !== id || identity.protocol !== MANUAL_PRIVATE_PROTOCOL) throw new Error("Private cell archive has a foreign identity"); - const originals = await validateSealedOriginals(directory, id), summary = cellCompleteness(id, originals); + const originals = await validateSealedOriginals(directory, id, { ...identity, root: directory }), summary = cellCompleteness(id, originals); if (hash(JSON.stringify(identity.evidenceCompleteness)) !== hash(JSON.stringify(summary)) || hash(JSON.stringify(receipt.evidenceCompleteness)) !== hash(JSON.stringify(summary))) throw new Error("Private evidence completeness receipt differs from archive"); - await validateSealedInventory(path.join(directory, "sealed-results", "html-report"), "../result-evidence-inventory.json"); + await validateSealedInventory(path.join(directory, "sealed-results", "html-report"), "../result-evidence-inventory.json", undefined, "standard", { ...identity, root: directory }); await cp(path.join(directory, "sealed-results", "html-report"), path.join(selected, manualCellKind(id)), { recursive: true, errorOnExist: true, force: false }); await writeFile(path.join(root, `manual-cell-completeness-${index}.json`), JSON.stringify(summary), { mode: 0o600, flag: "wx" }); } diff --git a/tests/runner-e2e/manual-private-workflow.test.ts b/tests/runner-e2e/manual-private-workflow.test.ts index dbb92264970..341ba066e1c 100644 --- a/tests/runner-e2e/manual-private-workflow.test.ts +++ b/tests/runner-e2e/manual-private-workflow.test.ts @@ -4,6 +4,7 @@ import path from "node:path"; import os from "node:os"; import { spawnSync } from "node:child_process"; import YAML from "yaml"; +import { MAX_MANUAL_PRIVATE_CELLS, MANUAL_PRIVATE_EXECUTIONS } from "./manual-private-retention.js"; const source=await readFile(new URL("../../.github/workflows/runner-full-stack-e2e.yml",import.meta.url),"utf8"); const jobs=YAML.parse(source).jobs; const step=(job:string,name:string)=>jobs[job].steps.find((row:{name?:string})=>row.name===name); @@ -50,7 +51,14 @@ it("requires downloaded complete private cell receipts before retaining merged r it("uses one exact read session per approved cell and a distinct write-only report session",()=>{ const report=jobs.report.steps; - for(let index=0;index<8;index++){ + expect(MAX_MANUAL_PRIVATE_CELLS).toBe(8); + expect(MANUAL_PRIVATE_EXECUTIONS).toHaveLength(244); + const indices=report.filter((row:any)=>/^manual_read_credentials_/.test(row.id??"")) + .map((row:any)=>Number(row.id.replace("manual_read_credentials_",""))); + expect(indices).toEqual(Array.from({length:MAX_MANUAL_PRIVATE_CELLS},(_,i)=>i)); + expect(report.filter((row:any)=>/^Download restricted manual cell /.test(row.name??"")) + .map((row:any)=>Number(row.env.SELECTED_CELL_INDEX))).toEqual(indices); + for(let index=0;indexrow.id===`manual_read_credentials_${index}`); expect(credentials.with["inline-session-policy"]).toBe("${{ steps.manual_report.outputs.read_policy_"+index+" }}"); expect(credentials.if).toContain(`cell_count) > ${index}`); diff --git a/tests/runner-e2e/report-catalog.ts b/tests/runner-e2e/report-catalog.ts index b0b606df2b6..4076ec5e5b5 100644 --- a/tests/runner-e2e/report-catalog.ts +++ b/tests/runner-e2e/report-catalog.ts @@ -29,6 +29,9 @@ export function parseReportExecutionId(id: string) { throw new Error("Report execution identity must be a string"); const journey = /^(product-journeys)\.([a-z0-9][a-z0-9_-]*)\.([a-z0-9][a-z0-9_-]*)\.(local|daytona)\.([a-z0-9][a-z0-9_-]*)$/.exec(id); if (journey && id.length <= 512) return { suiteId: journey[1]!, targetId: journey[2]!, profileId: journey[3]!, environmentId: journey[4]! as "local" | "daytona", caseId: journey[5]! }; + // Only the chat catalog has a two-part area.case task identity. + const chat = /^(chat-features)\.([a-z0-9][a-z0-9_-]*)\.(local|daytona)\.([a-z0-9][a-z0-9_-]*\.[a-z0-9][a-z0-9_-]*)$/.exec(id); + if (chat && id.length <= 512) return { suiteId: chat[1]!, profileId: chat[2]!, environmentId: chat[3]! as "local" | "daytona", caseId: chat[4]!, targetId: undefined }; const match = /^([a-z0-9][a-z0-9_-]*)\.([a-z0-9][a-z0-9_-]*)\.(local|daytona)\.([a-z0-9][a-z0-9_-]*)$/.exec( id, diff --git a/tests/runner-e2e/report-execution-identity.test.ts b/tests/runner-e2e/report-execution-identity.test.ts new file mode 100644 index 00000000000..d2dc58d8dfc --- /dev/null +++ b/tests/runner-e2e/report-execution-identity.test.ts @@ -0,0 +1,28 @@ +import { expect, it } from "vitest"; +import { parseReportExecutionId } from "./report-catalog.js"; +it.each([ + ["chat-features.runner-opencode.local.create-backlog", "create-backlog"], + ["chat-features.runner-opencode.local.goals.create-goal-tree", "goals.create-goal-tree"], + ["chat-features.runner-codex-subscription.local.tasks.create-backlog", "tasks.create-backlog"], +])("retains legacy and dotted chat identity %s", (id, caseId) => { + expect(parseReportExecutionId(id)).toMatchObject({ suiteId: "chat-features", environmentId: "local", caseId }); +}); +it("preserves deployment target and ordinary nonchat identities", () => { + expect(parseReportExecutionId("product-journeys.production.operator.local.task-properties")).toMatchObject({ suiteId: "product-journeys", targetId: "production", caseId: "task-properties" }); + expect(parseReportExecutionId("agent-chat.runner-opencode.local.conversation").caseId).toBe("conversation"); +}); +it.each(["../private", "goals/secret", "goals..create", "goals.create.extra", ".create", "goals.", "goals.create%2fsecret", "goals.create\\secret", "goals.CREATE", "goals.create?token=private"])("rejects malformed chat task %s", task => { + expect(() => parseReportExecutionId(`chat-features.runner-opencode.local.${task}`)).toThrow(); +}); +it("does not enable dotted tasks for unrelated suites or permit excess path length", () => { + expect(() => parseReportExecutionId("other-suite.runner-opencode.local.goals.create")).toThrow(); + expect(() => parseReportExecutionId(`chat-features.runner-opencode.local.goals.${"x".repeat(500)}`)).toThrow(); +}); + +it("parses each supported private execution without changing its retained task identity", async () => { + const { MANUAL_PRIVATE_EXECUTIONS } = await import("./manual-private-retention.js"); + for (const id of MANUAL_PRIVATE_EXECUTIONS) { + const identity = parseReportExecutionId(id); + expect(identity).toMatchObject({ suiteId: "chat-features", profileId: "runner-opencode", environmentId: "local", caseId: id.slice("chat-features.runner-opencode.local.".length), targetId: undefined }); + } +});