Files
PaperClipAI/tests/runner-e2e/manual-private-transfer.test.ts
T
DottaandPaperclip 018c4c9c9b Retain approved manual E2E evidence in private history (#18134)
## Thinking Path

> - Paperclip lets people manage AI agents and their work.
> - The manual AWS E2E workflow records browser and model evidence.
> - Public repository Actions artifacts are not a restricted evidence
store.
> - Some chat evidence needs the existing private-history store.
> - A target flag must not grant archive access or enable a public
fallback.
> - This pull request binds restricted runs to protected exact-source
approval and retains their standard evidence with trusted transfer code.
> - Operators can retain complete evidence without changing scanner
verdicts or scenario outcomes.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Manual AWS E2E evidence retention for exact reviewed chat-feature
selections.

**Subsystem affected**

The runner E2E manual workflow, standard evidence handoff and
private-history transfer.

**Current behavior**

Manual cell and merged evidence use Actions artifacts. In a public
repository, repository readers can download those files. Scheduled
private history does not authorize a manual target to use its role.

**Proposed behavior**

Classify sensitive selections in the trusted workflow. Require one
protected original-run approval. Verify existing private storage before
target setup or model secrets. Retain scanned standard originals and
final packaged results through a workflow-pinned runtime. Deny public
artifact and history fallback on failure.

**Reason and benefit**

Preserve inspectable standard evidence in restricted storage without
treating a private bucket as permission to retain credentials.

**Additional context**

Related source work is separate: #18067 changes paid runner routing,
#18100 changes scoped input delivery, and #17888 changes report catalog
parsing. This pull request does not include or revert those changes.

## What Changed

- Add a trusted manual classification and exact run, attempt, workflow,
target, selection and launcher approval contract.
- Add a protected manual private job. Prepare a second workflow-pinned
checkout and isolated publication dependencies before archive
credentials.
- Seal standard originals and final result bytes with exact inventories.
Reject credential contamination, unscanned carriers and changed files.
- Use existing immutable private-history archives and transfer receipts
for cells and merged reports. Keep the existing bounds.
- Use per-cell exact-key read sessions and a separate report writer.
Check every inline policy against the 2,048-character STS limit.
- Scan and seal fresh generated report logs and files with the actual
report-step credential roots before transfer.
- Require a positive nonprivate route for Actions uploads and public
history. Private failures have no public fallback. Preserve normal
catalog semantics for nonprivate selections with more than 32 IDs or
duplicate IDs.
- Authenticate generated-report fonts and every reserved trace-viewer
static asset against the exact trusted producer path and bytes,
including HTML and JavaScript. Keep standard cell evidence and unknown
carriers closed.
- Document the first eight supported IDs, deployment gates and the
operator trust boundary.

## Verification

- 57 new pure, actual-helper and actual-workflow-shell controls pass.
- 50 affected existing workflow, routing, target and private-history
controls pass. The final committed workflow security controls also pass.
- Targeted helper/test and actual launcher noEmit checks pass.
- `actionlint -shellcheck= -pyflakes=
.github/workflows/runner-full-stack-e2e.yml` passes.
- Parent `2f46336d7` required CI is green. The server 11/12 shard first
failed on database cleanup/test timing and passed one bounded recheck
with unchanged source. The original failure remains recorded.
- The preserved docs-only parent `d721c170f` had 59 terminal,
nonblocking checks and fresh Greptile 5/5. Three later review comments
exposed a genuine static-report asset failure and nonprivate selector
regression; their parent counterexamples remain recorded.
- The reviewed parent passes 76 focused controls across nine files. The
final dependency preflight successor passes all six affected controls,
including two new actual `prepare` subprocess controls, and targeted
helper/test noEmit. Unchanged suites were not repeated. A synthetic
Node-only Playwright HTML/blob/JUnit report and the real dashboard
renderer/brand stager pass the actual sealer, strict archive and
immutable receipt path; only AWS transport is synthetic. No browser or
product setup runs in that test.
- Default cell fonts/SVGs, unknown report files, forged paths, altered
static bytes, recomputed inventory tampering and registered credential
matches remain rejected. Eligibility and remaining-byte checks run
before body reads in both sealer and transfer validator; both
independent parent read-order failures are preserved.
- The pre-credential target comparison also binds the new
generated-report asset helper byte for byte. A genuine
tampered-dependency parent failure is preserved.
- Six additional actual sealer/validator controls expose the
standard-extension trace-owner bypass on the parent. All 34 affected
controls pass on its correction, including the real
producer/archive/receipt test; targeted noEmit passes. Ordinary dynamic
report HTML outside the reserved trace directory is unchanged.
- Final head `43683c4e0` passes both required gates, `ci / verify` and
`ci / e2e`, in normal CI run `40849916302` attempt 1. All 59 rollup
checks/statuses are terminal and nonblocking (55 success, two neutral,
two skipped).
- Fresh exact-head Greptile is 5/5; automated review is terminal and all
technical threads are resolved. Human CODEOWNER approval remains a
separate gate. Earlier source, genuine counterexamples and CI history
remain preserved.
- Full local build and live E2E were not run. This task prohibits local
SDK/native/server builds and paid or AWS operations. Normal pull request
CI must pass before merge readiness.
- No existing artifact bodies, archives, media or provider reasoning
were inspected. Synthetic owned fixtures prove the privacy boundaries.

## Risks

- Deployment still needs the existing private bucket, role and protected
environments. Both environments need the exact original-run approval
variable.
- Targets need the compatible handoff protocol and a separately reviewed
exact launcher digest. Existing target branches do not become ready
automatically.
- Complete duplicated originals and packaged evidence can exceed
existing archive limits. Such a transfer fails; files are not silently
removed.
- Required evidence omissions remain fatal to PASS. Safe failed evidence
is retained with an explicit missing list and incomplete
required-evidence verdict. Credential contamination still blocks all
original uploads. Restricted storage does not waive PNG, HTML, blob or
scanner checks.
- This is operator-approved source execution. Target processes can
inherit the job's OIDC request environment and may re-assume a role.
Inline session attenuation is not isolation from malicious approved
source.
- Operator authorization must bind the actual exact source and original
run. A proven administrator-controlled record may supply it without an
environment reviewer UI. No new approval UI or environment change is
required by this PR.
- Required repository CODEOWNER review is still pending. Commented
reviews and review requests are not approvals. This PR stays open and
unmerged.
- This PR does not deploy infrastructure, change IAM, dispatch
workflows, publish report payloads, change grades or transfer prior
scenario passes.

## Model Used

OpenAI `gpt-6.1-sol` in Codex. Used code execution, source analysis and
tool calls. The host does not expose the context window size.
Independent peer review used the same model family in a separate agent.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-29 08:51:40 -05:00

50 lines
5.5 KiB
TypeScript

import { spawnSync } from "node:child_process";
import { createRequire } from "node:module";
import { afterEach, expect, it } from "vitest";
import { mkdtemp, mkdir, readFile, rm, writeFile, copyFile } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { createHash } from "node:crypto";
import { validateSealedInventory } from "./manual-private-transfer.js";
import { sealManualResultEvidence, type ManualPrivateHandoff } from "./manual-private-handoff.js";
import { MANUAL_PRIVATE_PROTOCOL } from "./manual-private-retention.js";
const roots:string[]=[];
afterEach(async()=>{await Promise.all(roots.splice(0).map(root=>rm(root,{recursive:true,force:true})));});
async function fixture(){const root=await mkdtemp(path.join(os.tmpdir(),"manual-transfer-test-"));roots.push(root);await mkdir(path.join(root,"results"));const context={root,protocol:MANUAL_PRIVATE_PROTOCOL,private:true,workflowSha:"a".repeat(40),targetSha:"b".repeat(40),runId:"123",attempt:"1",executionIds:["chat-features.runner-opencode.local.activity.agent-feedback"]} as ManualPrivateHandoff;return{root,context};}
it("seals genuine final result bytes and rejects an unscanned staged addition before transfer",async()=>{
const f=await fixture();await writeFile(path.join(f.root,"results","campaign.json"),'{"cleanup":"passed"}');
await sealManualResultEvidence(f.context,[]);const dir=path.join(f.root,"sealed-results","html-report");
expect((await validateSealedInventory(dir,"../result-evidence-inventory.json")).complete).toBe(true);
await writeFile(path.join(dir,"staged.db"),"unscanned");await expect(validateSealedInventory(dir,"../result-evidence-inventory.json")).rejects.toThrow(/Unscanned/);
});
it("rejects modification of final cleanup/result bytes after the scan",async()=>{
const f=await fixture();await writeFile(path.join(f.root,"results","campaign.json"),'{}');await sealManualResultEvidence(f.context,[]);
const dir=path.join(f.root,"sealed-results","html-report");await writeFile(path.join(dir,"campaign.json"),'{"later":true}');await expect(validateSealedInventory(dir,"../result-evidence-inventory.json")).rejects.toThrow(/changed/);
});
it("an abnormal fallback cannot retain credential-bearing final data or unsupported staged bytes",async()=>{
const f=await fixture();await writeFile(path.join(f.root,"results","result.json"),'{"originalResult":"synthetic-final-credential-123456"}');
await expect(sealManualResultEvidence(f.context,["synthetic-final-credential-123456"])).rejects.toThrow(/contamination/);
await writeFile(path.join(f.root,"results","result.json"),'{}');await writeFile(path.join(f.root,"results","unknown.bin"),"opaque");
await expect(sealManualResultEvidence(f.context,[])).rejects.toThrow(/Unsupported/);
});
it("rejects a forged inventory that lists an unscannable original carrier",async()=>{
const f=await fixture(),dir=path.join(f.root,"results");await mkdir(path.join(dir,"snapshots"));await writeFile(path.join(dir,"snapshots","unknown.bin"),"secret");
const files=[{path:"snapshots/unknown.bin",bytes:6,sha256:createHash("sha256").update("secret").digest("hex")}];
await writeFile(path.join(dir,"inventory.json"),JSON.stringify({protocol:MANUAL_PRIVATE_PROTOCOL,complete:true,executionId:"id",sourceBytes:6,files,inventorySha256:createHash("sha256").update(JSON.stringify(files)).digest("hex")}));
await expect(validateSealedInventory(dir,"inventory.json","id")).rejects.toThrow(/row/);
});
it.each([false, true])("actual prepare checks new report dependency before credentials (tampered=%s)", async tampered => {
const root=await mkdtemp(path.join(os.tmpdir(),"synthetic-prepare-dependency-"));roots.push(root);
const target=path.join(root,"target"),modules=path.join(target,"tests/runner-e2e");await mkdir(modules,{recursive:true});
const trusted=path.resolve(import.meta.dirname,"../.."),launch=Buffer.from("synthetic approved launcher; never executed");await writeFile(path.join(modules,"launch.ts"),launch);
for(const name of ["manual-private-handoff.ts","manual-private-retention.ts","manual-private-report-assets.ts"])await copyFile(path.join(import.meta.dirname,name),path.join(modules,name));
if(tampered)await writeFile(path.join(modules,"manual-private-report-assets.ts"),"synthetic incompatible dependency; never executed");
const workflowSha="a".repeat(40),targetSha="b".repeat(40),id="chat-features.runner-opencode.local.activity.agent-feedback",output=path.join(root,"outputs.txt");
const require=createRequire(import.meta.url),tsx=require.resolve("tsx",{paths:[path.join(trusted,"server")]});
const result=spawnSync(process.execPath,["--import",tsx,path.join(import.meta.dirname,"manual-private-transfer.ts"),"prepare"],{encoding:"utf8",timeout:10000,env:{PATH:process.env.PATH,NODE_OPTIONS:"",NODE_PATH:"",SELECTED_EXECUTION_ID:id,MANUAL_PRIVATE_IDS:JSON.stringify([id]),MANUAL_PRIVATE_APPROVALS_JSON:JSON.stringify({version:1,approvals:[{workflowSha,targetSha,runId:"123",attempt:"1",executionIds:[id],protocol:MANUAL_PRIVATE_PROTOCOL,launchSha256:createHash("sha256").update(launch).digest("hex")}]}),GITHUB_SHA:workflowSha,TARGET_SHA:targetSha,GITHUB_RUN_ID:"123",GITHUB_RUN_ATTEMPT:"1",RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET:"private-bucket",TARGET_ROOT:target,RUNNER_TEMP:root,GITHUB_OUTPUT:output}});
expect(result.error).toBeUndefined();expect(result.status).toBe(tampered?1:0);
if(tampered){expect(result.stderr).toContain("Restricted manual evidence operation failed");await expect(readFile(output)).rejects.toMatchObject({code:"ENOENT"});}
else expect(await readFile(output,"utf8")).toContain("session_policy=");
});