mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-23 17:33:51 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A provider must work through normal installed packages on local and Daytona environments. > - Candidate-only success does not prove that the shipped package contains the manifest and native runtime needed for Daytona. > - The release must preserve saved-token access and task lifecycle guarantees. > - This pull request will qualify Copilot and include the exact-source remote manifest in normal release archives. > - The benefit is a reviewable part of the Copilot production release. ## Linked Issues or Issue Description Refs #15701, #15703 and #15705. Supersedes qualification work in #15563 and the original candidate in #14434. Base: #15705. ## What Changed - Keep Copilot discovery passive during reconnect reconciliation; refresh on explicit opening, account/environment changes and reconnect. - Retain the selected Product cases, permission/continuity evidence and cleanup checks. - Build the Linux provider manifest from the existing pinned Docker stage in the cloud. - Join all four native daemon receipts with that manifest before server packing. - Reject missing or incompatible remote authority during normal release assembly; retain historical workflow compatibility. - Promote the completed profile47 qualification marker, publish the current qualification record and preserve every failed attempt. - Keep one current release checklist and the shipped-canary gates explicit. ## Verification Runtime #15701 is merged at `d855e550ad8d2a267312983f26457aabf5393c3a`. History #15703 is merged at `df80616d98119cbf5abd40549794d9980a5fabf6`. Remaining source [`42bcbd61299655f7e37aabe6bb5f19bba770ad1d`](https://github.com/paperclipai/paperclip/commit/42bcbd61299655f7e37aabe6bb5f19bba770ad1d) contains the four-file authorization repair. The remaining PR trees preserve six nonoverlapping files from the newer public mainline managed-settings fix and the CI-owned lockfile. No unrelated provider is enabled. The security finding is repaired: existing-agent discovery requires `agent_config:update` before connection selection or token resolution. The saved environment is the default. Authorized editors can select a prospective environment. Both denied board-viewer and agent-reader regressions fail on the original source. All49 route tests and server typecheck pass with the fix. Denied requests perform no connection selection, credential resolution or probe. Current-head security scans pass on both remaining PRs; the repaired thread is resolved. Copilot CLI1.0.88 and profile47 retain digest `sha256:55044aa235e88b3fe36ec8a556ee1e3382590870aab907e4462ef7d64af49cea`. The [qualification record](https://github.com/paperclipai/paperclip/blob/42bcbd61299655f7e37aabe6bb5f19bba770ad1d/doc/plans/2026-10-20-copilot-qualification-v47.json) reports20/20 Product cells,154 assertions and7/7 protocol cases,46 assertions against frozen normal installed source `c2616370800f5f6df45246b0890ba35a7cd2faab`. The exact qualification model is `gpt-5.6-luna`. Every passing Product case confirms cleanup. Original failures remain recorded; automatic behavior retries are disabled. The route-only authorization repair changes no bound runtime/profile bytes and does not relabel the frozen campaign. The ordinary installed remote journey passed saved-token selection, model discovery, normal UI save/reopen, metadata-only Test, completed task, accepted receipt and exact19-byte browser download. An open model catalog caused no additional discovery requests during117seconds. Actual installed runtime smokes passed on macOS ARM64, macOS x64 under Rosetta and native Linux x64 on Daytona. Raw token browser entry was not exercised; encrypted connection creation used the normal API. The first Daytona file attempt timed out during delayed semantic dispatch; its precise stalled layer remains unresolved. Independent ordinary remote delivery and the other19 Product cells passed before one instrumented recheck passed8/8 with unchanged source,prompt,assertions and deadlines. No causal latency repair is claimed. Before the authorization repair, all four final PR heads passed required verify/E2E and fresh Greptile5/5 reviews with zero unresolved threads. Current-head CI/review is running for the two repaired heads. Local focused checks retain129 connection,59 runner,72 shared-eval and88 recovery/workspace passes, plus migration safety and token gates. The additive Copilot migration is361; released master history is preserved. [Cloud normal-package run39414239704](https://github.com/paperclipai/paperclip/actions/runs/39414239704) builds the pre-security source4f3f. All four native producers and Linux package assembly pass; the final clean installed consumers are finishing. All18 retained archives match their hashes;37,863 regular files contain zero selected credentials. That package does not contain the later route repair. The actual final published package must include it and pass the local/Daytona live canaries. The frozen Daytona image remains published and independently verified. No separate full-image scan is claimed. Docker builds run in the cloud. **Remaining:** current-head CI/review, setup and qualification merges, final publication, and ordinary local/Daytona canaries against the actual shipped package. The user explicitly authorizes the maintainer review override; CI remains a merge gate. No production deployment is claimed. Provider USD remains unknown and GitHub-attributed. ## Risks An omitted or mismatched image manifest prevents ordinary Daytona startup. Final normal archives must contain the matching image authority and platform daemons. The completed runtime campaign remains bound to its original source; the final UI-only repair needs its own normal-install setup proof. Credit notices are not measured USD receipts. Native steering, native plan mode and detached work remain outside this release. ## Model Used OpenAI Codex, based on GPT-6. The exact serving model ID and context window are not exposed in this task. Used for code changes, review and tool-assisted verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
337 lines
31 KiB
TypeScript
337 lines
31 KiB
TypeScript
import { readCopilotStopAcknowledgement, type CopilotStopAcknowledgement } from "./copilot-stop-acknowledgement.js";
|
|
import { copilotPendingPermissionCard } from "./copilot-permission-card.js";
|
|
import { createHash, randomBytes, randomUUID } from "node:crypto";
|
|
import { join } from "node:path";
|
|
import { expect, type Page } from "@playwright/test";
|
|
import { pollUntil, type RunnerApi } from "./api.js";
|
|
import { collectRunEvents } from "./run-observations.js";
|
|
import { createTaskThroughUi } from "./user-actions.js";
|
|
import { approveCopilotContextThroughUi, prepareCopilotContext } from "./copilot-context-permission.js";
|
|
import { copilotAttachedFinishReady, copilotFinishAttemptBeforeCommandExit, onlyCopilotAttachedOperations, readCopilotSemanticCompletion } from "./copilot-semantic-evidence.js";
|
|
import { copilotOrigin, readCopilotToolEvidence, type CopilotToolNotice } from "./copilot-evidence.js";
|
|
import { createAttachedCommandFixture, createDeniedTargetFixture, bindDeniedTargetPrompt, exists, observeRunProcesses, retainRunProcessIdentity } from "./copilot-local-fixtures.js";
|
|
import { gradeCopilotAttachedSettlement, gradeCopilotDeniedWrite, type CopilotDeniedWriteEvidence } from "./copilot-protection-cases.js";
|
|
import { validCopilotDenialSettlement, type CopilotDenialSettlement, findCopilotFixtureCommand, observeCopilotPreStop, type CopilotPreStopObservation, readCopilotDeniedEdit, copilotDenialSampleCursor, readCopilotDenialSettlement, observeCopilotFixtureCommand, readCopilotRemoteMarkerAfterRetirement, prepareCopilotRemoteAction, assertCopilotRemoteRetirement, assertCopilotRemoteAttached, copilotRemoteDeniedSample, copilotActionNotices, type CopilotRemoteBootstrap, type CopilotRemoteFixture, type CopilotRemoteSnapshot, countCopilotToolOrigins, countCopilotEditOriginsForTarget } from "./copilot-protection-evidence.js";
|
|
import type { LiveFixtureValues } from "./live-fixtures.js";
|
|
import type { MatrixExecution, RunnerTaskFixture } from "./types.js";
|
|
type Row = Record<string, any>;
|
|
type Check = { id: string; passed: boolean; detail: string };
|
|
/** Each persistence barrier reloads durable rows. Stop cannot overtake the
|
|
* failed edit, and a terminal sample cannot use a pre-terminal event cursor. */
|
|
export async function settleCopilotDeniedRun(input: {
|
|
api: Pick<RunnerApi, "post">; request: CopilotToolNotice; deadlineAt: number;
|
|
load(): Promise<{ events: readonly unknown[]; run: Row; issue: Row; retired: boolean }>;
|
|
afterDeniedEdit(): Promise<void>;
|
|
retainPreStop(receipt: CopilotPreStopObservation): Promise<void>;
|
|
retainStopAcknowledgement?(receipt: CopilotStopAcknowledgement): Promise<void>;
|
|
afterSettlement(): Promise<void>;
|
|
}) {
|
|
const cancellationRequestId = randomUUID();
|
|
const assertBeforeStop = (state: Awaited<ReturnType<typeof input.load>>) => {
|
|
if (state.run.id !== input.request.runId || state.run.status !== "running"
|
|
|| state.run.resultJson?.startupCancellation != null || state.run.resultJson?.nativeCancellation != null) {
|
|
throw new Error("Copilot denial observed an earlier Stop or non-running run");
|
|
}
|
|
};
|
|
let stopSent = false;
|
|
const poll = (label: string, accept: (state: Awaited<ReturnType<typeof input.load>>) => boolean) => pollUntil({
|
|
label, deadlineAt: input.deadlineAt, load: async () => {
|
|
const state = await input.load();
|
|
// pollUntil recognizes this definitive rejection before invoking readers.
|
|
if (["failed", "timed_out"].includes(state.run.status)) throw new Error(`Stopped waiting for ${label}: Copilot provider run failed`);
|
|
if (!stopSent) {
|
|
try { assertBeforeStop(state); }
|
|
catch { throw new Error(`Stopped waiting for ${label}: Copilot denial observed an earlier Stop or non-running run`); }
|
|
}
|
|
return state;
|
|
}, accept, intervalMs: 200,
|
|
});
|
|
await poll("persisted correlated failed native edit", state => {
|
|
readCopilotDeniedEdit({ events: state.events, request: input.request, companyId: state.issue.companyId }); return true;
|
|
});
|
|
await input.afterDeniedEdit();
|
|
// This negative case does not qualify active-turn cancellation. Give natural
|
|
// settlement a fixed observation window inside the unchanged case deadline.
|
|
const observeUntil = Math.min(input.deadlineAt, Date.now() + 2000);
|
|
let preStop: CopilotPreStopObservation;
|
|
while (true) {
|
|
const state = await input.load();
|
|
if (["failed", "timed_out"].includes(state.run.status)) throw new Error("Copilot provider run failed before Stop");
|
|
assertBeforeStop(state);
|
|
preStop = observeCopilotPreStop({ events: state.events, request: input.request, companyId: state.issue.companyId, cancellationRequestId });
|
|
if (preStop.terminal || Date.now() >= observeUntil) break;
|
|
await new Promise(resolve => setTimeout(resolve, Math.min(200, observeUntil - Date.now())));
|
|
}
|
|
if (Date.now() >= input.deadlineAt) throw new Error("Copilot denial deadline reached before Stop");
|
|
// Await the artifact write before dispatch. Database createdAt cannot supply
|
|
// this causal boundary, and a later replay must never manufacture it.
|
|
await input.retainPreStop(preStop);
|
|
if (Date.now() >= input.deadlineAt) throw new Error("Copilot denial deadline reached before Stop");
|
|
let dispatchState: Awaited<ReturnType<typeof input.load>>;
|
|
while (true) {
|
|
dispatchState = await input.load();
|
|
assertBeforeStop(dispatchState);
|
|
if (Date.now() >= input.deadlineAt) throw new Error("Copilot denial deadline reached before Stop");
|
|
const observed = observeCopilotPreStop({ events: dispatchState.events, request: input.request, companyId: dispatchState.issue.companyId, cancellationRequestId });
|
|
if (JSON.stringify([observed.terminal, observed.incompleteRecovery]) === JSON.stringify([preStop.terminal, preStop.incompleteRecovery])) break;
|
|
// Completion or its missing-result recovery row can arrive while the previous receipt is being persisted.
|
|
// Retain that actual pre-dispatch observation, then recheck for an earlier
|
|
// Stop after the write rather than discarding the newly observed terminal.
|
|
preStop = observed;
|
|
await input.retainPreStop(preStop);
|
|
if (Date.now() >= input.deadlineAt) throw new Error("Copilot denial deadline reached before Stop");
|
|
}
|
|
const stopDispatchMonotonicNs = process.hrtime.bigint().toString();
|
|
const stopped = await input.api.post<Row>(`/api/heartbeat-runs/${input.request.runId}/cancel`);
|
|
const stopAcknowledgement = readCopilotStopAcknowledgement(stopped,
|
|
{ companyId: preStop.companyId, issueId: dispatchState.issue.id, runId: input.request.runId, callerUserId: "local-board" }, cancellationRequestId, stopDispatchMonotonicNs, undefined,
|
|
preStop.terminal?.eventType === "turn.completed" ? "provider-completed-before-stop" : "active-provider");
|
|
await input.retainStopAcknowledgement?.(stopAcknowledgement);
|
|
stopSent = true;
|
|
await poll("correlated provider settlement and retired run", state => {
|
|
if (!state.retired) return false;
|
|
readCopilotDenialSettlement({ ...state, request: input.request, preStop, stopDispatchMonotonicNs, stopAcknowledgement }); return true;
|
|
});
|
|
await input.afterSettlement();
|
|
return readCopilotDenialSettlement({ ...await input.load(), request: input.request, preStop, stopDispatchMonotonicNs, stopAcknowledgement });
|
|
}
|
|
|
|
/** The alternative is an evidenced recovery outcome, never an arbitrary unfinished task. */
|
|
export function copilotDeniedIssueOutcome(task: RunnerTaskFixture, issue: Row, settlement: CopilotDenialSettlement) {
|
|
const issueStatus = settlement.runStop.issueStatus;
|
|
if (task.flow !== "copilot_protection" || task.id !== "native-permission-deny-write"
|
|
|| !validCopilotDenialSettlement(settlement)
|
|
|| issue.id !== settlement.runStop.issueId || issue.companyId !== settlement.runStop.companyId
|
|
|| issue.status !== issueStatus || ![task.expectedTerminalState.issue, task.expectedTerminalState.alternativeIssue].includes(issueStatus)
|
|
|| (issueStatus === "blocked" && issue.activeRecoveryAction?.id !== settlement.runStop.incompleteRecovery?.actionId)) {
|
|
throw new Error("Copilot denial task outcome does not match its verified settlement");
|
|
}
|
|
const label = issueStatus === "blocked" ? "Blocked" : "In Progress";
|
|
return { issueStatus, statusControl: { name: `Change status (current: ${label}${issueStatus === "blocked" ? "" : ")"}`, exact: issueStatus !== "blocked" } };
|
|
}
|
|
|
|
export async function runCopilotProtectionFlow(input: {
|
|
page: Page; api: RunnerApi; fixtures: LiveFixtureValues; execution: MatrixExecution; nonce: string; workspacePath: string; deadlineAt: number;
|
|
remoteBootstrap?: CopilotRemoteBootstrap;
|
|
registerBeforeEnvironmentTeardownAssertion?(callback: () => Promise<Check[]>): void;
|
|
observe(issue: Row, runs: Row[]): void; capture(id: string, label: string, file: string): Promise<void>; evidence(name: string, data: unknown): Promise<void>;
|
|
}) {
|
|
const { page, api, fixtures, execution, nonce, workspacePath } = input;
|
|
const remote = execution.environment.id === "daytona";
|
|
if ((!remote && execution.environment.id !== "local") || execution.profile.qualificationCandidate !== "copilot") throw new Error("Copilot protection fixtures require an isolated candidate");
|
|
if (remote && (!input.remoteBootstrap || !input.registerBeforeEnvironmentTeardownAssertion)) throw new Error("Remote Copilot protection requires bootstrap and pre-teardown evidence hooks");
|
|
const deny = execution.task.id === "native-permission-deny-write";
|
|
if (!deny && execution.task.id !== "attached-async-settlement") throw new Error("Unknown Copilot protection case");
|
|
const checks: Check[] = []; let issue: Row = {}, runs: Row[] = [], runEvents: Row[] = [];
|
|
let denialSettlement: CopilotDenialSettlement | undefined;
|
|
let notices: CopilotToolNotice[] = [];
|
|
const processObserver = remote ? undefined : observeRunProcesses();
|
|
let processes: { captured: boolean; live: number[]; identityChanged?: boolean } = processObserver?.sample() ?? { captured: false, live: [] };
|
|
let remoteFixture: CopilotRemoteFixture | undefined, baseline: CopilotRemoteSnapshot | undefined, sealed: CopilotRemoteSnapshot | undefined;
|
|
let remoteCommand: { command: string; commandSha256: string } | undefined;
|
|
const remoteMarker = `${randomBytes(24).toString("hex")}\n`;
|
|
const remoteSnapshots: CopilotRemoteSnapshot[] = [];
|
|
async function sealRemote() {
|
|
if (!remoteFixture || !baseline) throw new Error("Remote Copilot evidence was never armed");
|
|
sealed ??= await remoteFixture.finish();
|
|
assertCopilotRemoteRetirement(sealed, baseline); processes = sealed.processes;
|
|
return sealed;
|
|
}
|
|
const targetName = `copilot-denied-${nonce}.txt`;
|
|
const localTarget = deny && !remote ? await createDeniedTargetFixture(workspacePath, targetName) : undefined;
|
|
const target = localTarget?.targetRelativePath ?? targetName, targetPath = localTarget?.targetPath ?? join(workspacePath, target);
|
|
const fileObservations: CopilotDeniedWriteEvidence["fileObservations"] = [];
|
|
let deniedRequest: CopilotToolNotice | undefined;
|
|
const sample = async (phase: CopilotDeniedWriteEvidence["fileObservations"][number]["phase"]) => {
|
|
const providerCursor = phase === "before-request" ? null : copilotDenialSampleCursor(runEvents, deniedRequest!);
|
|
if (remote) {
|
|
if (!remoteFixture || !baseline) throw new Error("Remote denied target has no baseline");
|
|
const snapshot = sealed ?? (phase === "before-request" ? baseline : await remoteFixture.snapshot(phase));
|
|
remoteSnapshots.push(snapshot); fileObservations.push({ ...copilotRemoteDeniedSample(snapshot, baseline, target, phase), providerCursor });
|
|
} else fileObservations.push({ phase, observedAtMs: Date.now(), exists: await exists(targetPath), providerCursor });
|
|
};
|
|
const watcher = localTarget?.watcher;
|
|
const markerPath = join(workspacePath, `copilot-settlement-${nonce}.txt`);
|
|
const command = deny || remote ? undefined : await createAttachedCommandFixture(markerPath, 4000, true);
|
|
const exactCommand = () => remoteCommand ?? command;
|
|
let watchReceipt: CopilotDeniedWriteEvidence["mutationObservation"] | undefined;
|
|
const check = (id: string, passed: boolean, detail: string) => { checks.push({ id, passed, detail }); expect(passed, detail).toBe(true); };
|
|
async function load() {
|
|
if (issue.id) issue = await api.get<Row>(`/api/issues/${issue.id}`);
|
|
const listed = await api.get<Row[]>(`/api/companies/${fixtures.company.id}/heartbeat-runs?limit=100`);
|
|
runs = await Promise.all(listed.map(r => api.get<Row>(`/api/heartbeat-runs/${r.id}`)));
|
|
if (runs.length > 1) throw new Error("Copilot protection case dispatched an extra run");
|
|
input.observe(issue, runs);
|
|
runEvents = runs[0] ? await collectRunEvents<Row>((afterSeq, limit) => api.get(`/api/heartbeat-runs/${runs[0]!.id}/events?afterSeq=${afterSeq}&limit=${limit}`)) : [];
|
|
notices = runs[0] ? readCopilotToolEvidence(runEvents, runs[0].id) : [];
|
|
const run = runs[0];
|
|
if (processObserver) {
|
|
processes = retainRunProcessIdentity(processes, processObserver.sample(run?.processPid ? { pid: run.processPid, groupId: run.processGroupId, startedAt: run.processStartedAt, runId: run.id } : undefined));
|
|
if (processes.identityChanged) check("owned-process-identity", false, "A captured process identity changed; cleanup cannot be qualified");
|
|
}
|
|
return { issue, runs, runEvents, notices, processes };
|
|
}
|
|
const wait = (label: string, accept: (state: Awaited<ReturnType<typeof load>>) => boolean) => pollUntil({ label, deadlineAt: input.deadlineAt, load, accept, intervalMs: 200,
|
|
reject: state => state.runs.some(r => ["failed", "timed_out"].includes(r.status)) ? "Copilot provider run failed" : undefined });
|
|
try {
|
|
const agent = await api.get<Row>(`/api/agents/${fixtures.agent.id}`);
|
|
await api.patch(`/api/agents/${fixtures.agent.id}`, { adapterConfig: { ...agent.adapterConfig, acpxPermissionMode: deny ? "approve-reads" : "approve-all", timeoutSec: 120, lifecycleMode: "per_turn" } });
|
|
check("per-turn-process-lifecycle", (await api.get<Row>(`/api/agents/${fixtures.agent.id}`)).adapterConfig?.lifecycleMode === "per_turn", "This case explicitly requires a per-turn runner process, never a retained warm daemon");
|
|
const project = await api.post<Row>(`/api/companies/${fixtures.company.id}/projects`, {
|
|
name: `Copilot protection ${nonce}`, executionWorkspacePolicy: { enabled: true, defaultMode: "shared_workspace", sharedWorkspaceConcurrency: "serialize", allowIssueOverride: false, environmentId: fixtures.environment.id, workspaceStrategy: { type: "project_primary" } },
|
|
workspace: { name: "Primary", sourceType: "local_path", cwd: workspacePath, isPrimary: true },
|
|
});
|
|
if (deny && !remote) { await sample("before-request"); check("target-initially-absent", !fileObservations[0]!.exists, "The exact isolated target is absent before dispatch"); }
|
|
const prompt = remote ? input.remoteBootstrap!.prompt(nonce) : `${localTarget ? bindDeniedTargetPrompt(execution.task.buildPrompt(nonce), targetName, target) : execution.task.buildPrompt(nonce)}${command ? `\nThe exact supplied command is:\n${command.command}\nDo not inspect or modify fixture code, fabricate its marker, or launch a substitute command.` : ""}`;
|
|
const createdTask = await createTaskThroughUi({ page, issuePrefix: fixtures.company.issuePrefix!, agentName: fixtures.agent.name, title: execution.task.buildTitle(nonce), prompt, workMode: "standard", projectName: project.name, requireExplicitTitle: true });
|
|
const found = await pollUntil({ label: "browser-created Copilot protection task", deadlineAt: input.deadlineAt, load: async () => (await api.get<Row[]>(`/api/companies/${fixtures.company.id}/issues?limit=100`)).find(r => r.id === createdTask.issueId), accept: Boolean });
|
|
if (!found) throw new Error("Browser-created task was not found"); issue = found;
|
|
await page.goto(`/${fixtures.company.issuePrefix}/issues/${issue.identifier ?? issue.id}`);
|
|
await prepareCopilotContext({
|
|
remoteSetup: remote ? async () => {
|
|
await wait("exact remote bootstrap run", state => state.runs.length === 1 && state.runs[0]?.status === "running");
|
|
const bound = await input.remoteBootstrap!.bindAndRelease({ issueId: issue.id, runId: runs[0]!.id,
|
|
targets: [deny ? target : `copilot-settlement-${nonce}.txt`],
|
|
actionPrompt: async fixture => {
|
|
remoteFixture = fixture;
|
|
const prepared = await prepareCopilotRemoteAction({ fixture, companyId: fixtures.company.id, environmentId: fixtures.environment.id, runId: runs[0]!.id,
|
|
target: deny ? target : `copilot-settlement-${nonce}.txt`, prompt: execution.task.buildPrompt(nonce), ...(deny ? {} : { markerText: remoteMarker }) });
|
|
baseline = prepared.baseline; remoteCommand = prepared.command;
|
|
if (deny) { await sample("before-request"); check("target-initially-absent", !fileObservations[0]!.exists, "The actual remote target is absent before action publication"); }
|
|
await input.evidence("copilot-remote-baseline.json", baseline);
|
|
return prepared.prompt;
|
|
} });
|
|
if (bound !== remoteFixture) throw new Error("Remote Copilot fixture changed during publication");
|
|
input.registerBeforeEnvironmentTeardownAssertion!(async () => {
|
|
try {
|
|
const receipt = await sealRemote();
|
|
if (deny) {
|
|
if (copilotRemoteDeniedSample(receipt, baseline!, target, "after-cleanup").exists) throw new Error("Remote denied target exists after retirement");
|
|
}
|
|
else {
|
|
if (!await readCopilotRemoteMarkerAfterRetirement(remoteFixture!, baseline!, `copilot-settlement-${nonce}.txt`, remoteMarker)) throw new Error("Remote sealed marker changed or disappeared");
|
|
}
|
|
await input.evidence("copilot-remote-pre-teardown.json", receipt);
|
|
return [{ id: "remote-sealed-retirement", passed: true, detail: "Exact remote run root and descendants retired; retained pre-deletion filesystem receipt validated" }];
|
|
} finally { await remoteFixture!.close(); }
|
|
});
|
|
} : undefined,
|
|
approveContext: async () => {
|
|
if (deny) await approveCopilotContextThroughUi({ page, companyId: fixtures.company.id, deadlineAt: input.deadlineAt,
|
|
load: async () => { const state = await load(); return { run: state.runs[0] ?? {}, events: state.runEvents }; }, evidence: input.evidence });
|
|
},
|
|
});
|
|
if (deny) {
|
|
await wait("exact native write permission", s => s.notices.some(n => n.stage === "permission_requested" && n.operation === "edit" && n.target === target && n.declineOffered && s.runEvents.some(r => r.eventType === "runtime_request.created" && r.payload?.prpEvent?.payload?.request?.requestId === n.requestId)));
|
|
const request = notices.find(n => n.stage === "permission_requested" && n.operation === "edit" && n.target === target)!;
|
|
deniedRequest = request;
|
|
const pending = runEvents.filter(r => r.eventType === "runtime_request.created" && r.payload?.prpEvent?.payload?.request?.requestId === request.requestId).map(r => r.payload.prpEvent.payload.request);
|
|
check("one-bound-permission", pending.length === 1 && pending[0].requestKind === "permission_approval" && pending[0].origin?.method === "session/request_permission", "The notice maps to the exact durable native permission card");
|
|
const retired = new Set(runEvents.filter(r => ["runtime_request.resolved", "runtime_request.cancelled", "runtime_request.expired"].includes(r.eventType)).map(r => r.payload?.prpEvent?.payload?.requestId));
|
|
const activeRequests = runEvents.filter(r => r.eventType === "runtime_request.created" && r.payload?.prpEvent?.payload?.request && !retired.has(r.payload.prpEvent.payload.request.requestId));
|
|
check("only-one-pending-native-request", activeRequests.length === 1, "Only the exact denied edit is awaiting a decision");
|
|
await sample("pending"); await page.reload();
|
|
const card = copilotPendingPermissionCard(page, target);
|
|
await expect(card).toHaveCount(1); await expect(card).toContainText(target); await input.capture("permission-pending", "Copilot native write awaiting denial", "permission-pending.png");
|
|
const url = `/api/heartbeat-runs/${request.runId}/runtime-requests/${encodeURIComponent(request.requestId!)}/resolve`;
|
|
const sent = page.waitForRequest(r => new URL(r.url()).pathname === url && r.method() === "POST");
|
|
const clickedAtMs = Date.now(); await card.getByRole("button", { name: "Deny", exact: true }).click();
|
|
const posted = (await sent).postDataJSON();
|
|
check("browser-exact-denial", posted.turnId === request.turnId && posted.requestKind === "permission_approval" && posted.resolution?.action === "decline", "Browser submitted denial for the exact run/request/turn");
|
|
const settlement = await settleCopilotDeniedRun({ api, request, deadlineAt: input.deadlineAt,
|
|
load: async () => {
|
|
const state = await load();
|
|
return { events: state.runEvents, run: state.runs[0]!, issue: state.issue,
|
|
retired: remote || (state.processes.captured && state.processes.live.length === 0) };
|
|
},
|
|
afterDeniedEdit: () => sample("after-decision"),
|
|
retainPreStop: receipt => input.evidence("copilot-pre-stop-observation.json", receipt),
|
|
retainStopAcknowledgement: receipt => input.evidence("copilot-stop-acknowledgement.json", receipt),
|
|
afterSettlement: async () => {
|
|
if (remote) await sealRemote();
|
|
await sample("terminal"); await new Promise(resolve => setTimeout(resolve, 100)); await load(); await sample("after-cleanup");
|
|
},
|
|
});
|
|
watchReceipt = remote ? { startedAtMs: baseline!.observedAtMs, endedAtMs: sealed!.observedAtMs, complete: sealed!.watcher.complete, targetMutationCount: sealed!.watcher.targetMutationCount } : watcher!.finish();
|
|
const toolResult = notices.find(n => n.stage === "tool" && n.toolCallId === request.toolCallId && n.status === "failed")!;
|
|
denialSettlement = settlement;
|
|
await input.evidence("copilot-denial-settlement.json", settlement);
|
|
check("correlated-provider-settlement", true, `Exact provider settlement (${settlement.branch}) and audited run Stop; completed does not cover active-turn cancellation`);
|
|
const terminalAt = settlement.providerTerminal.emittedAtMs;
|
|
const evidence: CopilotDeniedWriteEvidence = {
|
|
expected: copilotOrigin(request), requestId: request.requestId!, expectedRelativePath: target,
|
|
request: { ...request, requestId: request.requestId!, targetRelativePath: request.target!, method: "session/request_permission", offeredActions: request.declineOffered ? ["decline"] : [] },
|
|
decision: { ...request, observedAtMs: clickedAtMs, requestId: request.requestId!, browserRequestId: request.requestId!, action: "decline" },
|
|
deliveredDecision: (() => { const n = notices.find(n => n.stage === "permission_delivered" && n.requestId === request.requestId && n.outcome === "reject_once"); return n ? { ...n, requestId: n.requestId!, outcome: n.outcome! } : null; })(),
|
|
toolResult: { ...toolResult, status: "failed" },
|
|
terminal: { runId: settlement.runId, turnId: settlement.turnId, observedAtMs: terminalAt, status: runs[0]!.status },
|
|
settlement,
|
|
cleanup: { observedAtMs: fileObservations.at(-1)!.observedAtMs, ownedProcessesRemaining: processes.live.length }, fileObservations, mutationObservation: watchReceipt,
|
|
nativeAttemptsForTarget: countCopilotEditOriginsForTarget(notices, target),
|
|
};
|
|
await input.evidence("copilot-denial-proof.json", { evidence, processes, notices });
|
|
const grade = gradeCopilotDeniedWrite(evidence); check("denial-without-side-effects", grade.passed, grade.failures.join(", ") || "Exact browser denial, explicit cancellation and absence through process cleanup");
|
|
check("negative-task-unfinished", copilotDeniedIssueOutcome(execution.task, issue, settlement).issueStatus === issue.status, "The negative test does not claim the task is done");
|
|
check("no-extra-native-operation", countCopilotToolOrigins(copilotActionNotices(notices, notices.find(n => n.stage === "tool" && n.toolCallId === request.toolCallId)!, remoteFixture ? { actionFile: remoteFixture.actionFile, events: runEvents } : undefined, runEvents)) === 1, "No alternate native edit, command or delegated operation is permitted");
|
|
} else {
|
|
let finishOrigin: CopilotToolNotice | undefined;
|
|
await wait("native finish while the exact attached command is live", state => {
|
|
const matched = findCopilotFixtureCommand(state.notices, exactCommand()!.command);
|
|
if (!matched) return false;
|
|
finishOrigin = copilotAttachedFinishReady(state.notices, matched.call);
|
|
return Boolean(finishOrigin);
|
|
});
|
|
const release = remote ? await remoteFixture!.releaseAttachedCommand() : command!.releaseAfterFinish();
|
|
await input.evidence("copilot-attached-release.json", { finishOrigin, release, source: "fixture controller releases only its own finite child after the actual native finish receipt; final grading unchanged" });
|
|
await wait("attached command and task settlement", s => s.issue.status === "done" && s.runs[0]?.status === "succeeded" && (remote || (s.processes.captured && s.processes.live.length === 0)));
|
|
// Preserve independent local proof before any command matcher can abort.
|
|
const { external: localExternal, markerMatches: localMarkerMatches, afterCleanupMarkerMatches: localAfterCleanupMarkerMatches, matched } =
|
|
await observeCopilotFixtureCommand(notices, exactCommand()!.command, command ? { fixture: command, markerPath } : undefined);
|
|
await input.evidence("copilot-attached-command-observation.json", { notices, processes, external: localExternal,
|
|
canonicalCommandSha256: exactCommand()!.commandSha256, commandMatch: matched?.match ?? null,
|
|
markerMatches: localMarkerMatches, afterCleanupMarkerMatches: localAfterCleanupMarkerMatches });
|
|
const call = matched?.call;
|
|
check("single-exact-command", Boolean(matched), "Exactly one native execution matches the fixture command with at most eight leading ASCII SPACE/TAB bytes");
|
|
const semantic = readCopilotSemanticCompletion(runEvents, { companyId: fixtures.company.id, runId: call!.runId, turnId: call!.turnId,
|
|
nativeSessionId: call!.sessionId, command: call!, summary: execution.task.buildVisibleMarker(nonce), requireContextRead: true });
|
|
await input.evidence("copilot-semantic-completion.json", semantic);
|
|
check("accepted-canonical-completion", true, "One exact native finish receipt matches the proposed and control-plane accepted result; transport success alone is insufficient");
|
|
check("no-extra-native-operation", onlyCopilotAttachedOperations(copilotActionNotices(notices, call!, remoteFixture ? { actionFile: remoteFixture.actionFile, events: runEvents } : undefined, runEvents), call!, semantic), "Only attested context/setup reads, the exact attached command/result, and one authoritatively correlated accepted finish are allowed");
|
|
const started = notices.find(n => n.toolCallId === call!.toolCallId && n.shellState === "started");
|
|
const result = notices.find(n => n.commandToolCallId === call!.toolCallId && n.shellState === "completed");
|
|
const terminal = runEvents.find(r => r.eventType === "turn.completed" && r.payload?.prpEvent?.turnId === call!.turnId)?.payload.prpEvent;
|
|
if (remote) await sealRemote();
|
|
const remoteAttached = remote ? assertCopilotRemoteAttached(sealed!, baseline!, terminal ? Date.parse(terminal.emittedAt) : NaN) : undefined;
|
|
const external = remoteAttached ? { ...remoteAttached, childGone: true, clientGone: true,
|
|
commandExit: { ...remoteAttached.commandExit!, ownedProcessIdentityVerified: true, commandSha256: exactCommand()!.commandSha256 } } : localExternal!;
|
|
check("finish-attempt-before-command-exit", copilotFinishAttemptBeforeCommandExit(semantic, external.commandExit?.observedAtMs ?? NaN), "The correlated finish attempt precedes the independently observed command exit");
|
|
check("trusted-command-exit", !external.failure && external.connections === 1 && external.childGone && external.clientGone, "Fixed controller-owned child exited and its native client is gone");
|
|
const markerMatches = remote ? sealed!.targets[`copilot-settlement-${nonce}.txt`]?.sha256 === `sha256:${createHash("sha256").update(remoteMarker).digest("hex")}` : localMarkerMatches!;
|
|
check("native-client-before-terminal", Boolean(terminal) && external.clientExitedAtMs !== null && external.clientExitedAtMs < Date.parse(terminal.emittedAt), "Independent PID/start observation confirms native client retirement before turn completion");
|
|
check("marker-before-terminal", Boolean(terminal) && external.markerWrittenAtMs !== null && external.markerWrittenAtMs < Date.parse(terminal.emittedAt), "The independent fixture wrote its undisclosed marker before turn completion");
|
|
const afterCleanupMarkerMatches = remote ? await readCopilotRemoteMarkerAfterRetirement(remoteFixture!, baseline!, `copilot-settlement-${nonce}.txt`, remoteMarker) : localAfterCleanupMarkerMatches!;
|
|
const grade = gradeCopilotAttachedSettlement({ expected: copilotOrigin(call!), nativeCall: call ? { ...call, operation: call.operation!, mode: call.mode!, detach: call.detach!, commandSha256: call.commandSha256! } : null,
|
|
expectedCommand: exactCommand()!.command, expectedCommandSha256: exactCommand()!.commandSha256, commandMatch: matched!.match, commandExit: external.commandExit,
|
|
expectedShellId: started?.shellId ?? "", nativeShellResult: result ? { ...result, shellId: result.shellId!, commandToolCallId: result.commandToolCallId!, status: result.status!, exitCode: result.exitCode! } : null,
|
|
terminal: terminal ? { observedAtMs: Date.parse(terminal.emittedAt), runId: terminal.runId, turnId: terminal.turnId, status: "succeeded" } : null,
|
|
cleanup: { observedAtMs: remote ? sealed!.observedAtMs : Date.now(), ownedProcessesRemaining: processes.live.length }, terminalMarkerMatches: markerMatches, afterCleanupMarkerMatches });
|
|
await input.evidence("copilot-attached-proof.json", { external, processes, notices, grade, commandSha256: exactCommand()!.commandSha256, commandMatch: matched!.match, markerMatches, afterCleanupMarkerMatches });
|
|
check("attached-settlement-before-terminal", grade.passed, grade.failures.join(", ") || "Owned finite process and native shell settled before the actual turn terminal");
|
|
}
|
|
await load(); check("one-native-run", runs.length === 1 && runs[0]!.runtimeMode === "native", "Exactly one native run was accounted");
|
|
const comments = await api.get<Row[]>(`/api/issues/${issue.id}/comments`), interactions = await api.get<Row[]>(`/api/issues/${issue.id}/interactions`);
|
|
await input.evidence("api-state.json", { issue, run: runs[0], runs, comments, interactions, checks, runEvents, runEventsByRun: [{ runId: runs[0]!.id, events: runEvents }] });
|
|
await page.reload();
|
|
const statusControl = deny ? copilotDeniedIssueOutcome(execution.task, issue, denialSettlement!).statusControl
|
|
: { name: "Change status (current: Done)", exact: true };
|
|
await expect(page.getByTestId("issue-detail-header").getByRole("button", statusControl)).toBeVisible();
|
|
if (!deny) check("exact-completion-marker", comments.filter(c => c.authorAgentId === fixtures.agent.id && c.body?.trim() === execution.task.buildVisibleMarker(nonce)).length === 1, "The successful attached task has exactly one terminal marker");
|
|
await input.capture("final-state", "Copilot protection outcome", "final-state.png");
|
|
return { issue, runs, checks };
|
|
} finally {
|
|
watchReceipt ??= watcher?.finish();
|
|
try { await command?.close(); }
|
|
finally { await input.evidence("copilot-protection-checks.json", { issue, runs, checks, fileObservations, watchReceipt, processes, remoteSnapshots, sealed }); }
|
|
}
|
|
}
|