mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-23 00:53:38 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The native runner must support project work, delegation, hiring, and service access. > - Browser tests exposed lost connection access, rejected helper events, and stalled recovery. > - Some eval failures also came from incorrect fixtures and decision controls. > - This pull request fixes those paths and adds eight everyday workflow stories. > - The tests retain observed failures and verify delivered files independently. > - The benefit is repeatable evidence for common user tasks and their remaining gaps. ## Linked Issues or Issue Description Related work: #13404 contains earlier workflow fixes. #13300 and #13470 changed the CI contracts used by the harness security tests. Merged companion: [paperclip-evals#22](https://github.com/paperclipai/paperclip-evals/pull/22). **What happened?** Native ACPX sessions did not receive the assigned connection gateway. Codex helper events could arrive before their spawn receipt and fail thread validation. A parent continuation could take a shared workspace before its child retried. A failed native continuation could leave the task status without a clear recovery blocker. The eval harness also confused tool approvals with new connection requests and could reject a valid delegated download. **Expected behavior** Keep assigned gateway access and its approval checks. Verify helper lineage before accepting helper progress. Let a waiting child proceed before automatic parent recovery. Preserve a failed task's recovery ownership. Grade the actual requested workflow and its delivered files. **Steps to reproduce** Run the everyday workflow suite with the native Codex and Claude profiles. Exercise service approval, connection refusal, delegated project work, and teammate reuse. The commands and case requirements are in `tests/runner-e2e/EVERYDAY-WORKFLOWS.md`. Use `pnpm test:runner-recovery` for controlled crash and replacement cases. ## What Changed - Pass the scoped connection gateway binding through the native ACPX host and sidecar. - Recognize Codex helper lineage from parent metadata and spawn receipts. Verify early helper events with `thread/read`. Keep helper events separate from root completion authority. - Guide agents to use persistent hiring, child tasks, dependency records, and a blocked handoff while waiting for a child. - Defer automatic parent recovery while a child has an active execution path in the same shared workspace. Allow parent recovery when the child needs review. - Record Blocked status and recovery evidence when a failed native continuation needs reconciliation, including existing active or escalated incidents. Preserve their owner and retry budget. - Add eight browser-driven workflow cases. Use real decision controls, explicit child feedback delivery, managed hiring credentials, and independent ZIP checks inside a bounded Docker sandbox. Verify sandbox availability before task creation. Record screenshot SHA-256 at capture. - Keep runner crash probes in controlled recovery tests. Preserve the original failure when cleanup also fails. - Display missing accounting and replay revisions as unavailable. Align harness security assertions with the approved CI changes. - Make the channel-rejection browser fixture bind its file after the send captures its payload. This prevents live refresh from removing the file before the simulated race. ## Verification - Full workspace `pnpm -r typecheck` passed after merging current master. - Runner E2E typecheck passed. Harness unit tests passed: 216/216. - Wake-queue database tests passed: 55/55. The two added existing-incident tests failed before the fix and pass after it. - Docker artifact calibration passed: 12/12. Host-file and host-loopback isolation tests failed before the fix and pass after it. Read-only delivery and output limits are also verified. - Full `pnpm build` passed. Targeted recovery tests passed: 83/83. - The channel-rejection browser test passed five consecutive runs after fixing the fixture race found in CI. - Local general-server (12,351 tests), UI (6,250), CLI (485), and workspace package groups passed. The monolithic run stopped at an unchanged lock-heartbeat fixture race; the isolated workspace group passed on rerun (shared: 747/747). A separate local serialized run passed 97 files before two socket errors in the unchanged issue-list route suite; that suite passed 15/15 on isolated rerun. These local full commands did not finish uninterrupted; the complete CI matrix below covers the remaining suites. - Final head `0fb293733fe307be7e6667ae8f1364077d0c6455`: **34 successful checks, 2 expected skips**, including every server/workspace shard, browser shard, native runner verification, build, and typecheck. [Final CI run](https://github.com/paperclipai/paperclip/actions/runs/34989136700). - Greptile reviewed this exact head at **5/5**; all review threads are resolved. Both Superagent security checks are successful. - ACPX credential-boundary tests passed: 118/118. Superagent accepted the runner/sidecar versus provider-environment trace and cleared its finding. - The latest paid local campaign on source `f6a2fdf7ac2af859826a2ae627ff4125a5478529` passed 22/24 cases: Sol 8/8, Claude 7/8, Mini 7/8. These results predate the merge with current master. - The two remaining failures are in `hire-reuse`: Claude exceeded the attempt deadline during final review; Mini made invalid deliverable tool calls and remained Blocked. - Six Daytona cases were not run because the matching immutable runner image was unavailable. This PR does not claim new remote model results. ## Risks The changes affect connection admission, helper identity, and recovery scheduling. Assigned gateway grants and user approval still govern service calls. The workspace admission gate still exists; the broader folder-sync design is separate work. Provider behavior can still cause the two recorded hiring failures. No database migration is required. Paid cases are opt-in and have bounded attempt deadlines. Project stories now require Docker and the documented pinned Python image on the harness host. ## Model Used OpenAI `gpt-6-astra` performed implementation, diagnosis, and substantive review. OpenAI `gpt-5.6-luna` assisted with verification, PR preparation, and review tracking. Both used repository tools and code execution. Context-window sizes were not recorded. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused checks and isolated reruns; full-run limitations are documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: OpenAI GPT-5.6 Luna <noreply@openai.com> Co-authored-by: Paperclip <noreply@paperclip.ing>
173 lines
5.3 KiB
TypeScript
173 lines
5.3 KiB
TypeScript
export interface StoryCheck {
|
|
id: string;
|
|
passed: boolean;
|
|
detail: string;
|
|
}
|
|
export interface StoryIssue {
|
|
id: string;
|
|
companyId: string;
|
|
title: string;
|
|
status: string;
|
|
identifier?: string | null;
|
|
description?: string | null;
|
|
createdAt?: string;
|
|
assigneeAgentId?: string | null;
|
|
parentId?: string | null;
|
|
projectId?: string | null;
|
|
executionRunId?: string | null;
|
|
scheduledRetry?: unknown;
|
|
activeRecoveryAction?: unknown;
|
|
}
|
|
export interface StoryRun {
|
|
id: string;
|
|
companyId: string;
|
|
agentId: string;
|
|
status: string;
|
|
runtimeMode?: string;
|
|
runnerInstanceId?: string | null;
|
|
processPid?: number | null;
|
|
processStartedAt?: string | null;
|
|
nativeIssueId?: string | null;
|
|
nativeSessionId?: string | null;
|
|
contextSnapshot?: Record<string, unknown> | null;
|
|
resultJson?: Record<string, unknown> | null;
|
|
usageJson?: Record<string, unknown> | null;
|
|
runnerProfileJson?: Record<string, unknown> | null;
|
|
sessionIdAfter?: string | null;
|
|
startedAt?: string | null;
|
|
finishedAt?: string | null;
|
|
retryOfRunId?: string | null;
|
|
scheduledRetryReason?: string | null;
|
|
runtimeModeResolvedAt?: string | null;
|
|
lastOutputSeq?: number | null;
|
|
errorCode?: string | null;
|
|
error?: string | null;
|
|
}
|
|
export const isActiveStoryRun = (run: StoryRun) =>
|
|
["queued", "running", "scheduled_retry"].includes(run.status);
|
|
export function isStoryWorkspaceDeferral(run: StoryRun) {
|
|
const recovery = run.resultJson?.executionRecovery as
|
|
Record<string, unknown> | undefined;
|
|
const startup = run.resultJson?.startupCancellation as
|
|
Record<string, unknown> | undefined;
|
|
const neverStartedRetry =
|
|
run.errorCode === "cancelled" &&
|
|
run.scheduledRetryReason === "workspace_busy" &&
|
|
typeof run.retryOfRunId === "string" &&
|
|
run.retryOfRunId.length > 0 &&
|
|
run.startedAt === null &&
|
|
run.runtimeModeResolvedAt === null &&
|
|
run.lastOutputSeq === 0 &&
|
|
!run.processStartedAt &&
|
|
!run.sessionIdAfter &&
|
|
!run.usageJson &&
|
|
!run.runnerProfileJson?.nativeExecutionInput &&
|
|
typeof startup?.requestedAt === "string" &&
|
|
Number.isFinite(Date.parse(startup.requestedAt));
|
|
return (
|
|
run.status === "cancelled" &&
|
|
((run.errorCode === "workspace_busy" &&
|
|
recovery?.providerWorkStarted === false) ||
|
|
neverStartedRetry) &&
|
|
!run.runnerInstanceId &&
|
|
!run.nativeSessionId &&
|
|
!run.processPid
|
|
);
|
|
}
|
|
export function storyLifecycleChecks(input: {
|
|
issues: StoryIssue[];
|
|
runs: StoryRun[];
|
|
parentId: string;
|
|
leadId: string;
|
|
allowedInterruptedRuns?: string[];
|
|
}): StoryCheck[] {
|
|
const executed = input.runs.filter((r) => !isStoryWorkspaceDeferral(r));
|
|
const allowed = new Set(input.allowedInterruptedRuns ?? []);
|
|
const check = (id: string, passed: boolean, detail: string): StoryCheck => ({
|
|
id,
|
|
passed,
|
|
detail,
|
|
});
|
|
return [
|
|
check(
|
|
"tasks-done",
|
|
input.issues.length > 0 && input.issues.every((i) => i.status === "done"),
|
|
"Every story task must reach Done.",
|
|
),
|
|
check(
|
|
"settled",
|
|
!input.runs.some(isActiveStoryRun) &&
|
|
!input.issues.some((i) => i.scheduledRetry || i.activeRecoveryAction),
|
|
"No active runs or scheduled recovery remain.",
|
|
),
|
|
check(
|
|
"native-runtime",
|
|
executed.length > 0 &&
|
|
executed.every(
|
|
(r) => r.runtimeMode === "native" && Boolean(r.runnerInstanceId),
|
|
),
|
|
"All executions must prove native runtime and runner identity.",
|
|
),
|
|
check(
|
|
"successful-runs",
|
|
executed.every((r) => r.status === "succeeded" || allowed.has(r.id)),
|
|
"Only explicitly interrupted runs may have a non-success terminal state.",
|
|
),
|
|
check(
|
|
"bounded-work",
|
|
input.runs.length <= 12,
|
|
"No more than twelve executions, including child and recovery turns.",
|
|
),
|
|
check(
|
|
"parent-owned-by-lead",
|
|
!executed.some(
|
|
(r) =>
|
|
(r.contextSnapshot?.issueId === input.parentId ||
|
|
r.contextSnapshot?.taskId === input.parentId) &&
|
|
r.agentId !== input.leadId,
|
|
),
|
|
"A mentioned worker must not execute on the parent.",
|
|
),
|
|
];
|
|
}
|
|
|
|
/** A terminal first turn cannot satisfy a later queued request. */
|
|
export function storyRepliesConsumed(
|
|
runs: StoryRun[],
|
|
commentIds: string[],
|
|
): boolean {
|
|
return commentIds.every((id) =>
|
|
runs.some((run) => {
|
|
if (run.status !== "succeeded" || isStoryWorkspaceDeferral(run))
|
|
return false;
|
|
const input = run.runnerProfileJson?.nativeExecutionInput as
|
|
{ task?: { prompt?: string } } | undefined;
|
|
return input?.task?.prompt?.includes(id) === true;
|
|
}),
|
|
);
|
|
}
|
|
|
|
export function storyParentFinishedAfterChildren(
|
|
runs: StoryRun[],
|
|
parentId: string,
|
|
leadId: string,
|
|
childIds: string[],
|
|
): boolean {
|
|
const scope = (r: StoryRun) =>
|
|
r.nativeIssueId ?? r.contextSnapshot?.issueId ?? r.contextSnapshot?.taskId;
|
|
const completed = runs.filter(
|
|
(r) => r.status === "succeeded" && !isStoryWorkspaceDeferral(r),
|
|
);
|
|
const children = completed.filter((r) => childIds.includes(String(scope(r))));
|
|
if (!children.length) return false;
|
|
const lastChildFinish = Math.max(
|
|
...children.map((r) => Date.parse(r.finishedAt ?? "")),
|
|
);
|
|
return completed.some(
|
|
(r) =>
|
|
r.agentId === leadId &&
|
|
scope(r) === parentId &&
|
|
Date.parse(r.finishedAt ?? "") >= lastChildFinish,
|
|
);
|
|
}
|