mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-09 07:10:34 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Chat-feature evals run in the trusted `runner-full-stack-e2e` workflow and keep their evidence in private history > - A manual chat run needs authority to write that private evidence > - Today that authority is a stored approval for the exact run ID in `RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON` > - The run ID exists only after dispatch, so every manual run stops until a person writes the record > - The dispatcher is already checked against `RUNNER_E2E_ALLOWED_ACTOR_IDS`, so the second step adds friction without a new decision > - This pull request lets an allowlisted manual dispatch authorize its own private evidence, as the TestOps admission already does > - The benefit is that an allowlisted operator can start a chat eval and it runs, while scheduled runs still never touch chat evidence ## Linked Issues or Issue Description Refs #18134, Refs #18170 **What existing behavior does this improve?** A manual `runner-full-stack-e2e` run with chat-feature IDs fails at private evidence transfer unless a person first adds an approval row for that run ID to a repository variable. **Subsystem affected** Runner E2E manual workflow private evidence authorization (`manual-private-transfer.ts`, `runner-full-stack-e2e.yml`). **Proposed change** Derive the evidence authority from the dispatch itself, with the same checks the TestOps chat admission uses. ## What Changed - New `tests/runner-e2e/manual-dispatch-authority.ts`: `authorizeManualDispatch` admits private evidence only when all of these are true: - The event is `workflow_dispatch` of `runner-full-stack-e2e.yml` on the default branch, with the `manual` cadence. - The run attempt is `1`. - The actor is in `RUNNER_E2E_ALLOWED_ACTOR_IDS`, the event sender is that actor, and the triggering actor is the same person. - The trusted cell selection equals the dispatch's own `id` input. - The authority binds the workflow SHA, the target SHA from the authorize job, the run ID and the cells. Every job derives the same identity hash. - `manual-private-transfer.ts`: uses the TestOps authority first, then the dispatch authority, then the stored approval. With the dispatch authority, `prepare` records the target launcher digest instead of comparing it with an approved digest. - `runner-full-stack-e2e.yml`: the 14 steps that read `RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON` now set `PAPERCLIP_MANUAL_DISPATCH_AUTHORITY: v1` and pass `RUNNER_E2E_ALLOWED_ACTOR_IDS`. - `doc/manual-e2e-private-evidence.md`: describes the dispatch authority. The stored-approval shape stays documented for the TestOps catalog case workflow. - The new file is separate on purpose. `manual-private-retention.ts` and the other protocol files must match each target byte for byte, so existing candidate branches keep working. ## Verification - `vitest run tests/runner-e2e/manual-dispatch-authority.test.ts tests/runner-e2e/manual-private-transfer.test.ts tests/runner-e2e/manual-private-report.test.ts tests/runner-e2e/manual-private-report-integration.test.ts tests/runner-e2e/manual-private-policy.test.ts`: 57 tests passed. - New tests cover one valid dispatch and rejection of: a scheduled run, a bounded cadence, a non-allowlisted actor, an empty allowlist, a rerun by another person, a second attempt, a non-default or different workflow ref, a sender who is not the actor, a missing target SHA, a missing authority switch, and cells that differ from the dispatch input. The eight-cell limit still applies. - A `prepare` test runs the real transfer script: a non-allowlisted actor produces no outputs, and an allowlisted dispatch produces the session policy and an identity that records the target launcher digest. - Negative control: with the previous `manual-private-transfer.ts`, the new `prepare` test fails. - Workflow tests that read this workflow: 111 of 116 passed. The 5 failures in `workflow-target.test.ts` (`IMAGE_ONLY: unbound variable`) also fail on unchanged `master`. `paid-runner-routing.test.mjs` passed (7 tests). - I did not run a live dispatch. The first live proof is a manual dispatch of chat IDs after merge. ## Risks - The dispatcher now vouches for the target branch. A target branch with a changed launcher can write private evidence if an allowlisted operator dispatches it. The protocol files must still match the trusted copies byte for byte. - The eight-cell limit is unchanged. The report job has eight fixed read slots, and each read policy must fit the 2,048-character session policy limit. Lifting the limit needs a separate change to the report sessions. - The `runner-e2e-paid` environment still has required reviewers. That is a repository setting and this change does not touch it. - The authority uses `RUNNER_E2E_ALLOWED_ACTOR_IDS`, not `CODEOWNERS`. Keep that variable aligned with the people who may start paid runs. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
55 lines
3.9 KiB
TypeScript
55 lines
3.9 KiB
TypeScript
import { createHash } from "node:crypto";
|
|
import { lstatSync, readFileSync } from "node:fs";
|
|
import { canonicalPrivateManualIds, MANUAL_PRIVATE_EXECUTIONS, MANUAL_PRIVATE_PROTOCOL } from "./manual-private-retention.js";
|
|
|
|
const authority = "paperclip.manual-dispatch-evidence/v1";
|
|
const repository = "paperclipai/paperclip";
|
|
const workflow = ".github/workflows/runner-full-stack-e2e.yml";
|
|
const numericId = /^[1-9][0-9]{0,19}$/;
|
|
const values = (value: unknown) => String(value ?? "").split(",").map(item => item.trim()).filter(Boolean);
|
|
|
|
/**
|
|
* Private chat evidence authority for a manual run of the trusted workflow.
|
|
*
|
|
* The run itself is the approval: a `workflow_dispatch` of the default-branch
|
|
* workflow, on its first attempt, started and triggered by an allowlisted
|
|
* operator, for the exact cells named in its own `id` input. Scheduled and
|
|
* bounded-cadence runs never qualify. The evidence stays bound to this run,
|
|
* its workflow commit, its resolved target commit and those cells.
|
|
*/
|
|
export function authorizeManualDispatch(env: NodeJS.ProcessEnv, event: unknown) {
|
|
const payload = event as { ref?: unknown; sender?: { id?: unknown }; repository?: { full_name?: unknown; default_branch?: unknown }; inputs?: Record<string, unknown> };
|
|
const branch = payload.repository?.default_branch;
|
|
const actor = env.GITHUB_ACTOR_ID;
|
|
let allowed: unknown;
|
|
try { allowed = JSON.parse(env.MANUAL_DISPATCH_ALLOWED_ACTOR_IDS ?? "null"); } catch { allowed = null; }
|
|
if (env.PAPERCLIP_MANUAL_DISPATCH_AUTHORITY !== "v1" || env.GITHUB_ACTIONS !== "true" || env.GITHUB_EVENT_NAME !== "workflow_dispatch"
|
|
|| env.GITHUB_REPOSITORY !== repository || payload.repository?.full_name !== repository
|
|
|| typeof branch !== "string" || !/^[A-Za-z0-9._/-]+$/.test(branch) || env.GITHUB_REF !== `refs/heads/${branch}` || payload.ref !== env.GITHUB_REF
|
|
|| env.GITHUB_WORKFLOW_REF !== `${repository}/${workflow}@refs/heads/${branch}`
|
|
|| !/^[a-f0-9]{40}$/.test(env.GITHUB_SHA ?? "") || !/^[a-f0-9]{40}$/.test(env.TARGET_SHA ?? "")
|
|
|| !numericId.test(env.GITHUB_RUN_ID ?? "") || env.GITHUB_RUN_ATTEMPT !== "1"
|
|
|| !numericId.test(actor ?? "") || payload.sender?.id !== Number(actor)
|
|
|| !env.GITHUB_ACTOR || env.GITHUB_TRIGGERING_ACTOR !== env.GITHUB_ACTOR
|
|
|| !Array.isArray(allowed) || !allowed.length || allowed.some(v => !Number.isSafeInteger(v) || v <= 0) || !allowed.includes(Number(actor))
|
|
|| (payload.inputs?.cadence ?? "manual") !== "manual") throw new Error("Invalid manual dispatch evidence authority");
|
|
const requested = canonicalPrivateManualIds(values(payload.inputs?.id));
|
|
const selected = canonicalPrivateManualIds(JSON.parse(env.MANUAL_PRIVATE_IDS ?? "null"));
|
|
if (!selected.length || selected.join() !== requested.join() || selected.some(id => !MANUAL_PRIVATE_EXECUTIONS.includes(id)))
|
|
throw new Error("Manual dispatch cells differ from the trusted selection");
|
|
const identity = { workflowSha: env.GITHUB_SHA!, targetSha: env.TARGET_SHA!, runId: env.GITHUB_RUN_ID!, attempt: "1", executionIds: Object.freeze(selected) };
|
|
const authoritySha256 = createHash("sha256").update(JSON.stringify({ authority, ...identity, actor })).digest("hex");
|
|
// The dispatcher vouches for the target branch, so its launcher is recorded
|
|
// rather than compared with a separately approved hash.
|
|
return Object.freeze({ ...identity, authority, protocol: MANUAL_PRIVATE_PROTOCOL, launchBinding: "dispatcher" as const,
|
|
authoritySha256, approvalSha256: authoritySha256, private: true as const });
|
|
}
|
|
|
|
export function manualDispatchEvidenceAuthority(env: NodeJS.ProcessEnv) {
|
|
const eventPath = env.GITHUB_EVENT_PATH;
|
|
if (!eventPath) throw new Error("Missing manual dispatch event");
|
|
const stat = lstatSync(eventPath);
|
|
if (!stat.isFile() || stat.isSymbolicLink() || stat.size > 65536) throw new Error("Invalid manual dispatch event custody");
|
|
return authorizeManualDispatch(env, JSON.parse(readFileSync(eventPath, "utf8")));
|
|
}
|