fix(e2e): follow proven workspace deferrals during onboarding (#18008)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Product E2E journeys verify the user outcome through the browser and
scoped APIs.
> - An onboarding child can wait while its parent owns their shared
workspace.
> - The server records that no provider work started and schedules a
durable replacement.
> - The journey rejected that expected wait, and feedback counted it as
unknown legacy execution.
> - This pull request follows the exact replacement and records the
distinct no-provider evidence.
> - The child must still finish through native execution and produce the
independently checked download.

## Linked Issues or Issue Description

Refs #17940.

**What happened?**

The fresh-workspace journey failed with `onboarding_first_task_failed`
when a child encountered shared-workspace contention. The cancelled
admission had an explicit `workspace_wait` receipt with
`providerWorkStarted: false`. Its linked replacement was still
scheduled. The runtime had not been selected. The default `legacy` value
did not represent a provider execution.

**Expected behavior**

Keep both rows and their run-limit cost. Follow only the unique
replacement for the same company, agent, and issue. A pending
replacement must not pass. Real failures and contradictory execution
evidence must still fail.

**Steps to reproduce**

1. Start a native onboarding parent and accept its child-task proposal.
2. Observe the child admission while the parent still holds their
workspace.
3. Present the server's no-provider cancellation and its durable
scheduled replacement to the journey and feedback collectors.

## What Changed

- Add a shared predicate for the explicit no-provider workspace-deferral
receipt. Reject runtime, process, output, usage, and cost evidence that
contradicts it.
- Follow exact, unique retry lineage during onboarding. Wait for active
or scheduled replacements without repeating browser actions. Preserve
the campaign deadline, run cap, native artifact attribution, and
download oracle.
- Retain per-run feedback inspection classifications: `native_events`,
`provider_not_started`, or `incomplete`. Require exhaustive event reads.
Keep pending work and uncertain execution partial.
- Add causal regressions and document the evidence contract. Historical
failed attempts keep their original grades.

## Verification

- Red: the new journey regression failed with the observed
`onboarding_first_task_failed` before the repair. The new feedback cases
also failed before the repair.
- Green: 70 journey tests and 74 feedback-family tests passed. The final
affected journey/feedback gate passed 275 tests plus 137 Node checks.
Five retry-lane reset cases failed before the review correction and
passed after it.
- `pnpm test:e2e:runner:typecheck` passed after the retry-counter
correction.
- `pnpm -r typecheck` and `pnpm build` passed before the final rebase.
- The full local `pnpm test:run` and harness unit gate encountered
embedded PostgreSQL startup failures. The harness gate passed 4,574
tests and failed its database-startup test. The final focused
journey/feedback gate and all 137 Node checks passed. CI will provide
independent full-suite results before merge.
- Current-head CI passed 55 checks; two checks skipped and one was
neutral. Greptile is 5/5, with zero unresolved review threads.
- Live Cloud verification will use the reviewed harness and Operations
revisions. This PR does not claim that a fresh Cloud campaign has
passed.

## Risks

A broad cancellation exemption could hide real failures. The exemption
requires the server's explicit receipt, unresolved runtime selection, no
contradictory execution evidence, and exact durable lineage. Missing
successors remain pending until the existing deadline. A successful
native child and valid downloaded bytes remain required. No server
behavior, approval policy, or budget policy changes.

## Model Used

OpenAI Codex, `gpt-6.1-sol`, with reasoning, source inspection, tool
use, and code execution. The host does not expose its configured
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed 2026-10-28 07:58:41 -05:00
1 parent e7ea2e2961
commit 3bd0d81c7b
8 files changed
+420 -10

No files matched your search

+118
View File
@@ -103,6 +103,124 @@ it.each([{ runtimeMode: "legacy", status: "succeeded" }, { runtimeMode: "native"
expect(summary.invocationCollection).toBe("partial");
});
function workspaceDeferral(): Record<string, unknown> {
return {
id: "run", companyId: "company", agentId: "agent", issueId: "child", status: "cancelled", runtimeMode: "legacy",
errorCode: "workspace_busy", startedAt: "2026-10-28T03:35:01.660Z", finishedAt: "2026-10-28T03:35:02.268Z",
runtimeModeResolvedAt: null, processStartedAt: null, processPid: null, processStartTicks: null,
runnerInstanceId: null, nativeSessionId: null, sessionIdAfter: null, lastOutputSeq: 0, costUsd: null,
usageJson: { ledgerScope: { issueId: "child" } },
runnerProfileJson: { adapterDispatch: { adapterType: "paperclip_runner" } },
resultJson: {
executionRecovery: { kind: "workspace_wait", providerWorkStarted: false },
cancellation: { source: "control_plane", expected: true, initiator: { type: "system" } },
workspaceBusy: { holderRunId: "parent-run", holderIssueId: "parent", deferralAttempt: 0 },
},
};
}
async function collectWorkspaceDeferral(run: Record<string, unknown>, events: Record<string, unknown>[] = []) {
let retained: FeedbackEvidence | undefined;
const summary = await collectAttemptFeedback({ companyId: "company", persist: async value => { retained = structuredClone(value); }, api: api(path => {
if (path.includes("agent-commentary")) return { items: [], total: 0 };
if (path.includes("/events?")) return events;
if (path === "/api/heartbeat-runs/run") return run;
return [{ id: "run", status: run.status }];
}) });
return { summary, retained: retained! };
}
it("records complete no-provider coverage for a proven workspace deferral without claiming native execution", async () => {
const { summary, retained } = await collectWorkspaceDeferral(workspaceDeferral(), [
{ seq: 1, eventType: "lifecycle", stream: "system", payload: { retryScheduled: true } },
]);
expect(summary).toMatchObject({ collection: "complete", invocationCollection: "complete", runsInspected: 1,
invocations: { completed: 0, failed: 0, unknown: 0 }, toolAvailability: "unknown" });
expect(retained.runCoverage).toEqual([{ runId: "run", classification: "provider_not_started" }]);
});
it.each([
["pending successor", { status: "scheduled_retry", startedAt: null, finishedAt: null }],
["actual legacy work", { status: "succeeded", errorCode: null, resultJson: {} }],
["provider work", { resultJson: { executionRecovery: { kind: "workspace_wait", providerWorkStarted: true } } }],
["native process", { runnerInstanceId: "runner" }],
["native selection", { runtimeMode: "native", runtimeModeResolvedAt: "2026-10-28T03:35:01.700Z" }],
["local process", { processPid: 123 }],
["usage", { usageJson: { inputTokens: 1 } }],
["charge", { costUsd: "0.01" }],
] as const)("keeps invocation coverage partial for %s despite a workspace marker", async (_label, overrides) => {
const { summary, retained } = await collectWorkspaceDeferral({ ...workspaceDeferral(), ...overrides });
expect(summary.invocationCollection).toBe("partial");
expect(retained.runCoverage).toEqual([{ runId: "run", classification: "incomplete" }]);
});
it.each([
event(1, "tool.execution.started", { name: "submit_complaint", executionId: "call" }),
event(1, "turn.started", {}),
{ seq: 1, eventType: "adapter.invoke", stream: "system", payload: {} },
{ seq: 1, eventType: "stdout", stream: "stdout", payload: {} },
{ seq: 1, eventType: "lifecycle", stream: "system", sourceEventId: "native-event", payload: {} },
{ seq: 1, eventType: "lifecycle", stream: "system", runId: "foreign-run", payload: {} },
{ seq: 1, eventType: "unknown", stream: "system", payload: {} },
])("rejects contradictory or unknown events when classifying a workspace deferral: %o", async contradictory => {
const { summary, retained } = await collectWorkspaceDeferral(workspaceDeferral(), [contradictory]);
expect(summary.invocationCollection).toBe("partial");
expect(retained.runCoverage).toEqual([{ runId: "run", classification: "incomplete" }]);
});
it("requires an exhaustive event scan before recording no-provider coverage", async () => {
let retained: FeedbackEvidence | undefined;
const summary = await collectAttemptFeedback({ companyId: "company", persist: async value => { retained = structuredClone(value); }, api: api(path => {
if (path.includes("agent-commentary")) return { items: [], total: 0 };
if (path.includes("/events?")) throw new Error("unavailable");
if (path === "/api/heartbeat-runs/run") return workspaceDeferral();
return [{ id: "run", status: "cancelled" }];
}) });
expect(summary).toMatchObject({ invocationCollection: "partial", runsInspected: 0 });
expect(retained!.runCoverage).toEqual([{ runId: "run", classification: "incomplete" }]);
expect(retained!.diagnostics).toContain("events_read_failed");
});
it("examines all event pages before accepting no-provider coverage", async () => {
let pages = 0, retained: FeedbackEvidence | undefined;
const summary = await collectAttemptFeedback({ companyId: "company", persist: async value => { retained = structuredClone(value); }, api: api(path => {
if (path.includes("agent-commentary")) return { items: [], total: 0 };
if (path.includes("/events?")) {
pages++;
return pages === 1 ? Array.from({ length: 100 }, (_, index) => ({ seq: index + 1, eventType: "lifecycle", stream: "system" }))
: [event(101, "tool.execution.started", { name: "submit_suggestion", executionId: "call" })];
}
if (path === "/api/heartbeat-runs/run") return workspaceDeferral();
return [{ id: "run", status: "cancelled" }];
}) });
expect(pages).toBe(2);
expect(summary).toMatchObject({ invocationCollection: "partial", invocations: { unknown: 1 } });
expect(retained!.runCoverage).toEqual([{ runId: "run", classification: "incomplete" }]);
});
it("does not complete feedback while a workspace deferral's replacement remains scheduled", async () => {
let retained: FeedbackEvidence | undefined;
const runs = [
{ id: "parent-opening", status: "succeeded", runtimeMode: "native" },
{ id: "parent-resumed", status: "succeeded", runtimeMode: "native" },
workspaceDeferral(),
{ id: "replacement", status: "scheduled_retry", runtimeMode: "legacy", retryOfRunId: "run", startedAt: null },
];
const summary = await collectAttemptFeedback({ companyId: "company", persist: async value => { retained = structuredClone(value); }, api: api(path => {
if (path.includes("agent-commentary")) return { items: [], total: 0 };
if (path.includes("/events?")) return [];
if (path.startsWith("/api/heartbeat-runs/")) return runs.find(run => run.id === path.split("/").at(-1));
return runs;
}) });
expect(summary).toMatchObject({ collection: "partial", invocationCollection: "partial", runsInspected: 4 });
expect(retained!.runCoverage).toEqual([
{ runId: "parent-opening", classification: "native_events" },
{ runId: "parent-resumed", classification: "native_events" },
{ runId: "run", classification: "provider_not_started" },
{ runId: "replacement", classification: "incomplete" },
]);
});
it("captures feedback after cancellation failure but before destructive teardown, and still tears down if persistence fails", async () => {
const calls: string[] = [];
const result = await runCleanupWithObservers({