fix(evals): let an allowlisted manual dispatch authorize its private evidence (#19599)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Chat-feature evals run in the trusted `runner-full-stack-e2e`
workflow and keep their evidence in private history
> - A manual chat run needs authority to write that private evidence
> - Today that authority is a stored approval for the exact run ID in
`RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON`
> - The run ID exists only after dispatch, so every manual run stops
until a person writes the record
> - The dispatcher is already checked against
`RUNNER_E2E_ALLOWED_ACTOR_IDS`, so the second step adds friction without
a new decision
> - This pull request lets an allowlisted manual dispatch authorize its
own private evidence, as the TestOps admission already does
> - The benefit is that an allowlisted operator can start a chat eval
and it runs, while scheduled runs still never touch chat evidence

## Linked Issues or Issue Description

Refs #18134, Refs #18170

**What existing behavior does this improve?**
A manual `runner-full-stack-e2e` run with chat-feature IDs fails at
private evidence transfer unless a person first adds an approval row for
that run ID to a repository variable.

**Subsystem affected**
Runner E2E manual workflow private evidence authorization
(`manual-private-transfer.ts`, `runner-full-stack-e2e.yml`).

**Proposed change**
Derive the evidence authority from the dispatch itself, with the same
checks the TestOps chat admission uses.

## What Changed

- New `tests/runner-e2e/manual-dispatch-authority.ts`:
`authorizeManualDispatch` admits private evidence only when all of these
are true:
- The event is `workflow_dispatch` of `runner-full-stack-e2e.yml` on the
default branch, with the `manual` cadence.
  - The run attempt is `1`.
- The actor is in `RUNNER_E2E_ALLOWED_ACTOR_IDS`, the event sender is
that actor, and the triggering actor is the same person.
  - The trusted cell selection equals the dispatch's own `id` input.
- The authority binds the workflow SHA, the target SHA from the
authorize job, the run ID and the cells. Every job derives the same
identity hash.
- `manual-private-transfer.ts`: uses the TestOps authority first, then
the dispatch authority, then the stored approval. With the dispatch
authority, `prepare` records the target launcher digest instead of
comparing it with an approved digest.
- `runner-full-stack-e2e.yml`: the 14 steps that read
`RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON` now set
`PAPERCLIP_MANUAL_DISPATCH_AUTHORITY: v1` and pass
`RUNNER_E2E_ALLOWED_ACTOR_IDS`.
- `doc/manual-e2e-private-evidence.md`: describes the dispatch
authority. The stored-approval shape stays documented for the TestOps
catalog case workflow.
- The new file is separate on purpose. `manual-private-retention.ts` and
the other protocol files must match each target byte for byte, so
existing candidate branches keep working.

## Verification

- `vitest run tests/runner-e2e/manual-dispatch-authority.test.ts
tests/runner-e2e/manual-private-transfer.test.ts
tests/runner-e2e/manual-private-report.test.ts
tests/runner-e2e/manual-private-report-integration.test.ts
tests/runner-e2e/manual-private-policy.test.ts`: 57 tests passed.
- New tests cover one valid dispatch and rejection of: a scheduled run,
a bounded cadence, a non-allowlisted actor, an empty allowlist, a rerun
by another person, a second attempt, a non-default or different workflow
ref, a sender who is not the actor, a missing target SHA, a missing
authority switch, and cells that differ from the dispatch input. The
eight-cell limit still applies.
- A `prepare` test runs the real transfer script: a non-allowlisted
actor produces no outputs, and an allowlisted dispatch produces the
session policy and an identity that records the target launcher digest.
- Negative control: with the previous `manual-private-transfer.ts`, the
new `prepare` test fails.
- Workflow tests that read this workflow: 111 of 116 passed. The 5
failures in `workflow-target.test.ts` (`IMAGE_ONLY: unbound variable`)
also fail on unchanged `master`. `paid-runner-routing.test.mjs` passed
(7 tests).
- I did not run a live dispatch. The first live proof is a manual
dispatch of chat IDs after merge.

## Risks

- The dispatcher now vouches for the target branch. A target branch with
a changed launcher can write private evidence if an allowlisted operator
dispatches it. The protocol files must still match the trusted copies
byte for byte.
- The eight-cell limit is unchanged. The report job has eight fixed read
slots, and each read policy must fit the 2,048-character session policy
limit. Lifting the limit needs a separate change to the report sessions.
- The `runner-e2e-paid` environment still has required reviewers. That
is a repository setting and this change does not touch it.
- The authority uses `RUNNER_E2E_ALLOWED_ACTOR_IDS`, not `CODEOWNERS`.
Keep that variable aligned with the people who may start paid runs.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DottaandClaude Opus 5.5 authored and GitHub committed 2026-11-06 14:26:31 -06:00
1 parent b55d8a9201
commit b7e28eb2f7
5 files changed
+217 -41

No files matched your search

+28 -14
View File
@@ -1101,7 +1101,8 @@ jobs:
working-directory: .manual-private-trusted
env:
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
PAPERCLIP_MANUAL_DISPATCH_AUTHORITY: v1
MANUAL_DISPATCH_ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
TARGET_ROOT: ${{ github.workspace }}
SELECTED_EXECUTION_ID: ${{ matrix.executionId }}
@@ -1130,7 +1131,8 @@ jobs:
working-directory: .manual-private-trusted
env:
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
PAPERCLIP_MANUAL_DISPATCH_AUTHORITY: v1
MANUAL_DISPATCH_ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
TARGET_ROOT: ${{ github.workspace }}
SELECTED_EXECUTION_ID: ${{ matrix.executionId }}
@@ -1456,7 +1458,8 @@ jobs:
working-directory: .manual-private-trusted
env:
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
PAPERCLIP_MANUAL_DISPATCH_AUTHORITY: v1
MANUAL_DISPATCH_ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
TARGET_ROOT: ${{ github.workspace }}
SELECTED_EXECUTION_ID: ${{ matrix.executionId }}
@@ -1557,7 +1560,8 @@ jobs:
if: needs.authorize.outputs.evidence_retention == 'private'
env:
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
PAPERCLIP_MANUAL_DISPATCH_AUTHORITY: v1
MANUAL_DISPATCH_ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
NODE_OPTIONS: ''
@@ -1595,7 +1599,8 @@ jobs:
if: steps.manual_read_credentials_0.outcome == 'success'
env:
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
PAPERCLIP_MANUAL_DISPATCH_AUTHORITY: v1
MANUAL_DISPATCH_ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
NODE_OPTIONS: ''
@@ -1626,7 +1631,8 @@ jobs:
if: steps.manual_read_credentials_1.outcome == 'success'
env:
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
PAPERCLIP_MANUAL_DISPATCH_AUTHORITY: v1
MANUAL_DISPATCH_ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
NODE_OPTIONS: ''
@@ -1657,7 +1663,8 @@ jobs:
if: steps.manual_read_credentials_2.outcome == 'success'
env:
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
PAPERCLIP_MANUAL_DISPATCH_AUTHORITY: v1
MANUAL_DISPATCH_ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
NODE_OPTIONS: ''
@@ -1688,7 +1695,8 @@ jobs:
if: steps.manual_read_credentials_3.outcome == 'success'
env:
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
PAPERCLIP_MANUAL_DISPATCH_AUTHORITY: v1
MANUAL_DISPATCH_ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
NODE_OPTIONS: ''
@@ -1719,7 +1727,8 @@ jobs:
if: steps.manual_read_credentials_4.outcome == 'success'
env:
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
PAPERCLIP_MANUAL_DISPATCH_AUTHORITY: v1
MANUAL_DISPATCH_ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
NODE_OPTIONS: ''
@@ -1750,7 +1759,8 @@ jobs:
if: steps.manual_read_credentials_5.outcome == 'success'
env:
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
PAPERCLIP_MANUAL_DISPATCH_AUTHORITY: v1
MANUAL_DISPATCH_ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
NODE_OPTIONS: ''
@@ -1781,7 +1791,8 @@ jobs:
if: steps.manual_read_credentials_6.outcome == 'success'
env:
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
PAPERCLIP_MANUAL_DISPATCH_AUTHORITY: v1
MANUAL_DISPATCH_ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
NODE_OPTIONS: ''
@@ -1812,7 +1823,8 @@ jobs:
if: steps.manual_read_credentials_7.outcome == 'success'
env:
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
PAPERCLIP_MANUAL_DISPATCH_AUTHORITY: v1
MANUAL_DISPATCH_ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
NODE_OPTIONS: ''
@@ -1831,7 +1843,8 @@ jobs:
if: steps.manual_report.outcome == 'success'
env:
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
PAPERCLIP_MANUAL_DISPATCH_AUTHORITY: v1
MANUAL_DISPATCH_ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
PRIVATE_REPORT_ROOT: ${{ github.workspace }}
NODE_OPTIONS: ''
@@ -2025,7 +2038,8 @@ jobs:
if: always() && steps.manual_report_credentials.outcome == 'success' && steps.manual_cells_downloaded.outcome == 'success'
env:
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
PAPERCLIP_MANUAL_DISPATCH_AUTHORITY: v1
MANUAL_DISPATCH_ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
NODE_OPTIONS: ''
+34 -23
View File
@@ -24,7 +24,7 @@ aliases are not registered. The exact registry is in
`tests/runner-e2e/manual-private-retention.ts`.
These IDs require the reviewed AWS Fleet route. Broad chat suites, mixed
unsupported IDs, reruns and missing approvals fail closed. All chat-feature IDs
unsupported IDs, reruns and runs without dispatch authority fail closed. All chat-feature IDs
are classified as restricted by trusted code. A target descriptor cannot opt out.
Other manual selections keep the existing Actions route only after a trusted
per-cell check confirms that the actual selected execution is not restricted.
@@ -38,8 +38,8 @@ credentials. Generic nonprivate catalog selection retains its existing semantics
A registered ID is not a claim of feature readiness or permission to run it.
Each target still needs the exact reviewed producer contracts, matching helper
bytes, declared private inputs, fresh source fingerprints, an available original
attempt and exact operation approval. Earlier accepted target branches do not
acquire these bytes or approval automatically. Running jobs keep their pinned
attempt and an allowlisted manual dispatch. Earlier accepted target branches do not
acquire these bytes automatically. Running jobs keep their pinned
sources. No retained grade or historical qualification transfers to a new target.
The private cell report contract binds the installed pinned Playwright producer
@@ -58,30 +58,41 @@ or run. Before a supported run, the operator must:
1. Deploy this workflow and its helpers together on the protected default branch.
2. Keep the existing numeric actor allowlist and protected default-branch checks.
3. Keep `runner-e2e-private-paid` and `runner-e2e-private-history` restricted to
the default branch. Establish administrator-controlled approval for the exact
source and original run. A reviewer UI may supply this control but is not
mandatory. Prove the actual control and runtime contract before admission.
the default branch.
4. Provide the existing private-history role ARN, AWS region and bucket variables
in those environments. The existing role must support the existing bounded
prefix and bucket-policy, ownership, public-access and retention reads.
5. Provide `RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON` through administrator-controlled
configuration available to both protected jobs. Approve the exact resolved
workflow SHA, target SHA, original run ID,
attempt `1`, selected execution IDs and launcher SHA-256.
6. Use a reviewed compatible target. Its private handoff and policy modules must
match the workflow implementation byte for byte. Its launcher digest must
match the protected approval. Earlier target branches do not gain this
protocol automatically.
5. Use a compatible target. Its private handoff and policy modules must match
the workflow implementation byte for byte. Earlier target branches do not
gain this protocol automatically.
Classification happens before the protected job. It grants no archive authority.
The administrator-controlled record must bind the actual original run ID before
target dependency setup or archive credentials. Prove who can change that record
and that each job consumes the exact approved record. An environment reviewer
gate is one way to govern authorization; its UI is not required. A missing or
stale row still fails closed before target dependency setup, archive credentials,
model credentials or fixture secrets. There is no wildcard approval.
### Dispatch authority
The variable has this closed shape (all values below are placeholders):
In `runner-full-stack-e2e.yml`, the manual run itself authorizes its private
evidence. There is no separate per-run approval record. Trusted code in
`tests/runner-e2e/manual-dispatch-authority.ts` admits a cell only when all of
these are true:
- The event is `workflow_dispatch` of this workflow on the default branch, with
the `manual` cadence. Scheduled and bounded-cadence runs never qualify.
- The run is on attempt `1`. A rerun does not inherit authority.
- The dispatching actor is in `RUNNER_E2E_ALLOWED_ACTOR_IDS`, the event sender is
that actor, and the triggering actor is the same person.
- The trusted cell selection equals the dispatch's own `id` input.
The authority binds the workflow SHA, the target SHA resolved by the authorize
job, the run ID and the cells. The dispatcher chooses and vouches for the target
branch, so its launcher digest is recorded in the private identity instead of
compared with a separately approved digest. Classification still happens before
the protected job, and a failed check still stops before target dependency
setup, archive credentials, model credentials or fixture secrets. The per-run
limit of eight cells is unchanged.
### Stored approval records
The TestOps catalog case workflow still passes a protected approval record as
`MANUAL_PRIVATE_APPROVALS_JSON`. That record has this closed shape (all values
below are placeholders):
```json
{
@@ -105,7 +116,7 @@ It prepares pinned publication dependencies with `--ignore-scripts` and a
separate store before archive credentials. Transfers use that checkout's
absolute helper, tsx and archive-script paths. They do not load the target's
node_modules or an ambient Node loader. The target executes only after the
protected approval and private-storage preflight pass.
evidence authority and private-storage preflight pass.
The AWS action returns session credentials as step outputs, not job-wide
environment exports. Only trusted preflight and transfer steps receive those
@@ -0,0 +1,90 @@
import { spawnSync } from "node:child_process";
import { createRequire } from "node:module";
import { copyFile, mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import { authorizeManualDispatch } from "./manual-dispatch-authority.js";
const ids = ["activity.agent-feedback", "agents.choose-primary"].map(id => `chat-features.runner-opencode.local.${id}`);
const workflowSha = "a".repeat(40), targetSha = "b".repeat(40);
const env = (overrides: Record<string, string | undefined> = {}): NodeJS.ProcessEnv => ({
PAPERCLIP_MANUAL_DISPATCH_AUTHORITY: "v1", GITHUB_ACTIONS: "true", GITHUB_EVENT_NAME: "workflow_dispatch",
GITHUB_REPOSITORY: "paperclipai/paperclip", GITHUB_REF: "refs/heads/master",
GITHUB_WORKFLOW_REF: "paperclipai/paperclip/.github/workflows/runner-full-stack-e2e.yml@refs/heads/master",
GITHUB_SHA: workflowSha, TARGET_SHA: targetSha, GITHUB_RUN_ID: "123", GITHUB_RUN_ATTEMPT: "1",
GITHUB_ACTOR: "owner", GITHUB_TRIGGERING_ACTOR: "owner", GITHUB_ACTOR_ID: "42",
MANUAL_DISPATCH_ALLOWED_ACTOR_IDS: "[42,7]", MANUAL_PRIVATE_IDS: JSON.stringify(ids), ...overrides,
});
const event = (overrides: Record<string, unknown> = {}, inputs: Record<string, unknown> = {}) => ({
ref: "refs/heads/master", sender: { id: 42 }, repository: { full_name: "paperclipai/paperclip", default_branch: "master" },
inputs: { cadence: "manual", target_branch: "candidate", id: ids.join(","), ...inputs }, ...overrides,
});
describe("authorizeManualDispatch", () => {
it("authorizes an allowlisted operator's manual dispatch for exactly its own cells", () => {
const authority = authorizeManualDispatch(env(), event());
expect(authority).toMatchObject({ workflowSha, targetSha, runId: "123", attempt: "1", launchBinding: "dispatcher", private: true });
expect(authority.executionIds).toHaveLength(2);
expect(authority.approvalSha256).toMatch(/^[a-f0-9]{64}$/);
// Every job of the run derives the same identity.
expect(authorizeManualDispatch(env(), event()).approvalSha256).toBe(authority.approvalSha256);
});
it.each([
["a scheduled run", env({ GITHUB_EVENT_NAME: "schedule" }), event()],
["a bounded cadence", env(), event({}, { cadence: "daily" })],
["an operator outside the allowlist", env({ GITHUB_ACTOR_ID: "99" }), event({ sender: { id: 99 } })],
["an empty allowlist", env({ MANUAL_DISPATCH_ALLOWED_ACTOR_IDS: "[]" }), event()],
["a rerun by another person", env({ GITHUB_TRIGGERING_ACTOR: "someone-else" }), event()],
["a second attempt", env({ GITHUB_RUN_ATTEMPT: "2" }), event()],
["a non-default workflow ref", env({ GITHUB_WORKFLOW_REF: "paperclipai/paperclip/.github/workflows/runner-full-stack-e2e.yml@refs/heads/feature" }), event()],
["a different workflow", env({ GITHUB_WORKFLOW_REF: "paperclipai/paperclip/.github/workflows/other.yml@refs/heads/master" }), event()],
["a sender who is not the actor", env(), event({ sender: { id: 7 } })],
["a missing target commit", env({ TARGET_SHA: "" }), event()],
["the authority switch unset", env({ PAPERCLIP_MANUAL_DISPATCH_AUTHORITY: undefined }), event()],
])("rejects %s", (_label, environment, payload) => {
expect(() => authorizeManualDispatch(environment, payload)).toThrow();
});
it("rejects cells that differ from the dispatch's own selection", () => {
expect(() => authorizeManualDispatch(env(), event({}, { id: ids[0] }))).toThrow(/differ/);
expect(() => authorizeManualDispatch(env({ MANUAL_PRIVATE_IDS: JSON.stringify([ids[0]]) }), event())).toThrow(/differ/);
});
it("keeps the per-run cell limit", () => {
const many = ["activity.agent-feedback", "activity.work-timeline", "companies.import-preview", "navigation-preferences.dismiss-announcement",
"tasks.completion-with-gate", "tasks.private-task", "secrets.personal-secret", "agents.choose-primary", "access.cli-authorization"]
.map(id => `chat-features.runner-opencode.local.${id}`);
expect(() => authorizeManualDispatch(env({ MANUAL_PRIVATE_IDS: JSON.stringify(many) }), event({}, { id: many.join(",") }))).toThrow(/limit/);
});
});
const roots: string[] = [];
afterEach(async () => { await Promise.all(roots.splice(0).map(root => rm(root, { recursive: true, force: true }))); });
it("actual prepare admits a dispatched cell without a stored approval and records the target launcher", async () => {
const root = await mkdtemp(path.join(os.tmpdir(), "synthetic-dispatch-prepare-")); roots.push(root);
const target = path.join(root, "target"), modules = path.join(target, "tests/runner-e2e"); await mkdir(modules, { recursive: true });
await writeFile(path.join(modules, "launch.ts"), "synthetic candidate launcher; never executed");
for (const name of ["manual-private-handoff.ts", "manual-private-retention.ts", "manual-private-report-assets.ts"]) await copyFile(path.join(import.meta.dirname, name), path.join(modules, name));
const eventPath = path.join(root, "event.json"), output = path.join(root, "outputs.txt");
await writeFile(eventPath, JSON.stringify(event()));
const trusted = path.resolve(import.meta.dirname, "../.."), require = createRequire(import.meta.url), tsx = require.resolve("tsx", { paths: [path.join(trusted, "server")] });
const run = (extra: Record<string, string>) => spawnSync(process.execPath, ["--import", tsx, path.join(import.meta.dirname, "manual-private-transfer.ts"), "prepare"], {
encoding: "utf8", timeout: 10000,
env: { PATH: process.env.PATH, NODE_OPTIONS: "", NODE_PATH: "", ...env(), SELECTED_EXECUTION_ID: ids[0]!, GITHUB_EVENT_PATH: eventPath,
TARGET_ROOT: target, RUNNER_TEMP: root, RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: "private-bucket", GITHUB_OUTPUT: output, ...extra } as NodeJS.ProcessEnv,
});
const denied = run({ GITHUB_ACTOR_ID: "99" });
expect(denied.status).toBe(1);
await expect(readFile(output)).rejects.toMatchObject({ code: "ENOENT" });
const admitted = run({});
expect(admitted.error).toBeUndefined(); expect(admitted.status).toBe(0);
const outputs = await readFile(output, "utf8");
expect(outputs).toContain("session_policy=");
const identityRoot = outputs.split("\n").find(line => line.startsWith("root="))!.slice(5);
const identity = JSON.parse(await readFile(path.join(identityRoot, "identity.json"), "utf8"));
expect(identity).toMatchObject({ launchBinding: "dispatcher", runId: "123", targetSha, executionIds: [ids[0]] });
expect(identity.launchSha256).toMatch(/^[a-f0-9]{64}$/);
});
@@ -0,0 +1,54 @@
import { createHash } from "node:crypto";
import { lstatSync, readFileSync } from "node:fs";
import { canonicalPrivateManualIds, MANUAL_PRIVATE_EXECUTIONS, MANUAL_PRIVATE_PROTOCOL } from "./manual-private-retention.js";
const authority = "paperclip.manual-dispatch-evidence/v1";
const repository = "paperclipai/paperclip";
const workflow = ".github/workflows/runner-full-stack-e2e.yml";
const numericId = /^[1-9][0-9]{0,19}$/;
const values = (value: unknown) => String(value ?? "").split(",").map(item => item.trim()).filter(Boolean);
/**
* Private chat evidence authority for a manual run of the trusted workflow.
*
* The run itself is the approval: a `workflow_dispatch` of the default-branch
* workflow, on its first attempt, started and triggered by an allowlisted
* operator, for the exact cells named in its own `id` input. Scheduled and
* bounded-cadence runs never qualify. The evidence stays bound to this run,
* its workflow commit, its resolved target commit and those cells.
*/
export function authorizeManualDispatch(env: NodeJS.ProcessEnv, event: unknown) {
const payload = event as { ref?: unknown; sender?: { id?: unknown }; repository?: { full_name?: unknown; default_branch?: unknown }; inputs?: Record<string, unknown> };
const branch = payload.repository?.default_branch;
const actor = env.GITHUB_ACTOR_ID;
let allowed: unknown;
try { allowed = JSON.parse(env.MANUAL_DISPATCH_ALLOWED_ACTOR_IDS ?? "null"); } catch { allowed = null; }
if (env.PAPERCLIP_MANUAL_DISPATCH_AUTHORITY !== "v1" || env.GITHUB_ACTIONS !== "true" || env.GITHUB_EVENT_NAME !== "workflow_dispatch"
|| env.GITHUB_REPOSITORY !== repository || payload.repository?.full_name !== repository
|| typeof branch !== "string" || !/^[A-Za-z0-9._/-]+$/.test(branch) || env.GITHUB_REF !== `refs/heads/${branch}` || payload.ref !== env.GITHUB_REF
|| env.GITHUB_WORKFLOW_REF !== `${repository}/${workflow}@refs/heads/${branch}`
|| !/^[a-f0-9]{40}$/.test(env.GITHUB_SHA ?? "") || !/^[a-f0-9]{40}$/.test(env.TARGET_SHA ?? "")
|| !numericId.test(env.GITHUB_RUN_ID ?? "") || env.GITHUB_RUN_ATTEMPT !== "1"
|| !numericId.test(actor ?? "") || payload.sender?.id !== Number(actor)
|| !env.GITHUB_ACTOR || env.GITHUB_TRIGGERING_ACTOR !== env.GITHUB_ACTOR
|| !Array.isArray(allowed) || !allowed.length || allowed.some(v => !Number.isSafeInteger(v) || v <= 0) || !allowed.includes(Number(actor))
|| (payload.inputs?.cadence ?? "manual") !== "manual") throw new Error("Invalid manual dispatch evidence authority");
const requested = canonicalPrivateManualIds(values(payload.inputs?.id));
const selected = canonicalPrivateManualIds(JSON.parse(env.MANUAL_PRIVATE_IDS ?? "null"));
if (!selected.length || selected.join() !== requested.join() || selected.some(id => !MANUAL_PRIVATE_EXECUTIONS.includes(id)))
throw new Error("Manual dispatch cells differ from the trusted selection");
const identity = { workflowSha: env.GITHUB_SHA!, targetSha: env.TARGET_SHA!, runId: env.GITHUB_RUN_ID!, attempt: "1", executionIds: Object.freeze(selected) };
const authoritySha256 = createHash("sha256").update(JSON.stringify({ authority, ...identity, actor })).digest("hex");
// The dispatcher vouches for the target branch, so its launcher is recorded
// rather than compared with a separately approved hash.
return Object.freeze({ ...identity, authority, protocol: MANUAL_PRIVATE_PROTOCOL, launchBinding: "dispatcher" as const,
authoritySha256, approvalSha256: authoritySha256, private: true as const });
}
export function manualDispatchEvidenceAuthority(env: NodeJS.ProcessEnv) {
const eventPath = env.GITHUB_EVENT_PATH;
if (!eventPath) throw new Error("Missing manual dispatch event");
const stat = lstatSync(eventPath);
if (!stat.isFile() || stat.isSymbolicLink() || stat.size > 65536) throw new Error("Invalid manual dispatch event custody");
return authorizeManualDispatch(env, JSON.parse(readFileSync(eventPath, "utf8")));
}
+11 -4
View File
@@ -12,12 +12,17 @@ import { BoundedHistoryPublisher } from "./history-budget.js";
import { privateArtifactKey, retainPrivateTransfer, transferPrivateArtifact, verifyPrivateHistory, type PrivateEvidenceCompleteness } from "./private-history.js";
import { authorizeManualPrivateEvidence, MANUAL_PRIVATE_PROTOCOL, manualCellKind, manualPrivateSessionPolicy } from "./manual-private-retention.js";
import { shippedChatEvidenceAuthority } from "./testops-chat-admission.js";
import { manualDispatchEvidenceAuthority } from "./manual-dispatch-authority.js";
const hash = (bytes: string | Buffer) => createHash("sha256").update(bytes).digest("hex");
function request(env: NodeJS.ProcessEnv) {
const ids = JSON.parse(env.MANUAL_PRIVATE_IDS ?? "null") as string[];
const approval = env.PAPERCLIP_TESTOPS_CHAT_MAIN_ADMISSION !== undefined ? shippedChatEvidenceAuthority(env) : authorizeManualPrivateEvidence({ workflowSha: env.GITHUB_SHA ?? "", targetSha: env.TARGET_SHA ?? "",
runId: env.GITHUB_RUN_ID ?? "", attempt: env.GITHUB_RUN_ATTEMPT ?? "", executionIds: ids }, env.MANUAL_PRIVATE_APPROVALS_JSON ?? "");
const approval: { workflowSha: string; targetSha: string; runId: string; attempt: string; executionIds: readonly string[]; approvalSha256: string;
launchSha256?: string; launchBinding?: "dispatcher" } =
env.PAPERCLIP_TESTOPS_CHAT_MAIN_ADMISSION !== undefined ? shippedChatEvidenceAuthority(env)
: env.PAPERCLIP_MANUAL_DISPATCH_AUTHORITY !== undefined ? manualDispatchEvidenceAuthority(env)
: authorizeManualPrivateEvidence({ workflowSha: env.GITHUB_SHA ?? "", targetSha: env.TARGET_SHA ?? "",
runId: env.GITHUB_RUN_ID ?? "", attempt: env.GITHUB_RUN_ATTEMPT ?? "", executionIds: ids }, env.MANUAL_PRIVATE_APPROVALS_JSON ?? "");
return { ids, approval, bucket: env.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET ?? "" };
}
async function output(name: string, value: string) {
@@ -30,13 +35,15 @@ async function prepare(env: NodeJS.ProcessEnv) {
const target = await realpath(env.TARGET_ROOT ?? ""), trusted = path.resolve(import.meta.dirname, "../..");
if (target === trusted) throw new Error("Trusted transfer code must be separate from target checkout");
const launch = await readFile(path.join(target, "tests/runner-e2e/launch.ts"));
if (hash(launch) !== approval.launchSha256) throw new Error("Target evidence launcher differs from protected approval");
const targetLaunchSha256 = hash(launch);
if (approval.launchBinding !== "dispatcher" && targetLaunchSha256 !== approval.launchSha256)
throw new Error("Target evidence launcher differs from protected approval");
for (const file of ["manual-private-handoff.ts", "manual-private-retention.ts", "manual-private-report-assets.ts"]) {
if (!(await readFile(path.join(target, "tests/runner-e2e", file))).equals(await readFile(path.join(trusted, "tests/runner-e2e", file))))
throw new Error("Target private evidence protocol differs from trusted implementation");
}
const temporary = await realpath(env.RUNNER_TEMP ?? ""), root = await mkdtemp(path.join(temporary, "manual-private-evidence-"));
await writeFile(path.join(root, "identity.json"), JSON.stringify({ ...approval, executionIds: [id], root, status: "pending" }), { mode: 0o600, flag: "wx" });
await writeFile(path.join(root, "identity.json"), JSON.stringify({ ...approval, launchSha256: targetLaunchSha256, executionIds: [id], root, status: "pending" }), { mode: 0o600, flag: "wx" });
await output("root", root); await output("session_policy", JSON.stringify(manualPrivateSessionPolicy({ bucket, runId: approval.runId, executionIds: [id], mode: "cell-write" })));
}
function requireSession(env: NodeJS.ProcessEnv) {