fix(evals): capture complete durable run event streams (#13977)

## Thinking Path

> - Paperclip manages AI agents and records their durable work outcomes.
> - Product E2E checks those outcomes through the browser and public
API.
> - A successful long run can emit more than 1,000 durable events.
> - The harness read one page and missed the later completion evidence.
> - This pull request reads every page before it checks runtime
invariants.
> - Invalid or incomplete capture still fails. A missing page cannot
produce a pass.

## Linked Issues or Issue Description

Related: #13882 (Grok qualification). A duplicate search found no
existing event-pagination fix.

**What happened?**

The local structured-question case in [campaign
36071063537](https://github.com/paperclipai/paperclip/actions/runs/36071063537)
completed the task and passed its six outcome matchers. It failed native
runtime invariants because the capture contained exactly 1,000 events.
The last captured event preceded the run's completion by more than a
minute. The API caps each response at 1,000 rows. The harness did not
request the next page.

**Expected behavior**

Read the complete durable event stream through the public API before
checking semantic-result and terminal-event counts. Reject incomplete or
malformed evidence.

**Steps to reproduce**

1. Complete a native task that emits more than 1,000 durable events.
2. Place the semantic-result and terminal events after row 1,000.
3. Capture the run with the Product E2E harness.
4. Before this fix, the invariant checker sees only the first page.

**Paperclip version or commit**

Observed at `4196a4cd76db434854b679035e4146c7f69689ce`. The same
single-page capture exists on master. The original failed result remains
unchanged; missing historical tail evidence is not reconstructed or
graded as a pass.

## What Changed

- Add a bounded event collector that advances through the public
`afterSeq` cursor.
- Use it for task success/failure evidence and shared chat run evidence.
- Reject invalid pages, missing or non-increasing sequence numbers,
repeated cursors, failed later requests, and an exhausted page limit.
- Test completion events beyond the first page, exact page boundaries,
and malformed evidence.
- Correct the existing Everyday catalog test from 38 to the maintained
47 cells. The suite stays explicit-only.
- Document the complete-capture requirement and its bound.

## Verification

- Product harness typecheck passes.
- The full credential-free harness suite passed 477 tests. After adding
the chat integration regression, all 34 chat evidence tests pass.
- Seventeen pagination tests cover the valid tail and malformed-evidence
cases.
- `git diff --check` passes.
- [Full repository
CI](https://github.com/paperclipai/paperclip/actions/runs/36080683422)
passes at `aed6f79c089226be79e75dcf390969561ec4f787`: 52 successful
checks and two intentional skips, including Rust, typecheck, build,
server tests, and browser shards. Greptile gives this exact head 5/5
with no findings. No local Docker, Rust build, browser suite, or model
invocation was used for this change.
- [Live Grok
requalification](https://github.com/paperclipai/paperclip/actions/runs/36080870743)
is running on combined source
`1b0551bb7c8de3c54f4bee64dbe2c88328b3645e`, with the runner, scheduler,
and evidence fixes. Its full credential-free harness passes all 493
tests and typecheck. Live results are pending; the original campaign
remains a failed measurement.

## Risks

Long runs need more read-only API requests and larger private evidence
files. Capture stops with an explicit error after 100 full pages. This
changes neither production APIs nor provider behavior. It does not relax
an invariant or change a historical grade.

## Model Used

OpenAI GPT-6 through Codex, with repository tools and code execution.
The exact serving identifier and context-window size are not exposed in
this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>

tests/runner-e2e/README.md

@@ -381,6 +381,12 @@ published with permanent campaign dashboards; fixture authors must therefore
 keep credentials and other private data out of every captured UI state. SVG is
 active content and is rejected from the packaged evidence entirely.
 
+The standard task and chat evidence collectors read durable run events through the paginated
+public API, including completion events beyond the first 1,000 rows. It rejects
+missing, repeated, or out-of-order sequence numbers and fails capture after
+100 full pages instead of grading a truncated stream. Original incomplete
+captures remain failed evidence; qualifying a fix requires a new live attempt.
+
 Every completed local campaign also writes
 `tests/runner-e2e/results/<campaign>/dashboard.html`. The self-contained page
 shows the complete profile/environment grid with screenshot thumbnails.

tests/runner-e2e/chat-flow.test.ts

@@ -224,21 +224,34 @@ describe("chat acceptance contracts", () => {
     );
   });
   it("retains reset events without requesting a provider log, and does not hide missing real logs", async () => {
-    const get = vi.fn().mockResolvedValue([{ type: "session_reset" }]);
+    const get = vi.fn().mockResolvedValue([{ seq: 1, type: "session_reset" }]);
     const reset = { ...run, resultJson: { conversationReset: true } };
     await expect(collectChatRunEvidence({ get }, reset)).resolves.toEqual({
       runId: run.id,
       log: null,
-      events: [{ type: "session_reset" }],
+      events: [{ seq: 1, type: "session_reset" }],
     });
     expect(get.mock.calls).toEqual([
-      [`/api/heartbeat-runs/${run.id}/events?limit=1000`],
+      [`/api/heartbeat-runs/${run.id}/events?afterSeq=0&limit=1000`],
     ]);
     get.mockRejectedValue(new Error("Run log not found"));
     await expect(collectChatRunEvidence({ get }, run)).rejects.toThrow(
       "Run log not found",
     );
   });
+  it("retains the tail of a long chat run for invariant checks", async () => {
+    const firstPage = Array.from({ length: 1000 }, (_, i) => ({ seq: i + 1, eventType: "item.delta" }));
+    const tail = { seq: 1001, eventType: "run.terminal" };
+    const get = vi.fn().mockResolvedValueOnce({ content: "fixture log" })
+      .mockResolvedValueOnce(firstPage).mockResolvedValueOnce([tail]);
+    const evidence = await collectChatRunEvidence({ get }, run);
+    expect(evidence.events).toEqual([...firstPage, tail]);
+    expect(get.mock.calls).toEqual([
+      [`/api/heartbeat-runs/${run.id}/log?limitBytes=1048576`],
+      [`/api/heartbeat-runs/${run.id}/events?afterSeq=0&limit=1000`],
+      [`/api/heartbeat-runs/${run.id}/events?afterSeq=1000&limit=1000`],
+    ]);
+  });
   it("retains events for an unstarted dependency-blocked wake without asking for a nonexistent log", async () => {
     const get = vi.fn().mockResolvedValue([]);
     const suppressed = { ...run, status: "cancelled", errorCode: "issue_dependencies_blocked", startedAt: null };

tests/runner-e2e/chat-flow.ts

@@ -7,6 +7,7 @@ import type {
 import type { LiveFixtureValues } from "./live-fixtures.js";
 import type { MatrixExecution } from "./types.js";
 import { isBlockedUnstartedWake } from "./non-execution-wake.js";
+import { collectRunEvents } from "./run-observations.js";
 import { chatMarker } from "./chat-cases.js";
 import { assertChatRememberedAfterRestart, assertChatStartupStopped, isChatStopReady, runChatHardeningFlow } from "./chat-hardening.js";
 import { enableChatThroughSettings, runChatInterruption, runChatSettingsLifecycle } from "./chat-stories.js";
@@ -239,7 +240,9 @@ export async function collectChatRunEvidence(
     log: isResetRun(run) || isBlockedUnstartedWake({ ...run })
       ? null
       : await api.get(`/api/heartbeat-runs/${run.id}/log?limitBytes=1048576`),
-    events: await api.get(`/api/heartbeat-runs/${run.id}/events?limit=1000`),
+    events: await collectRunEvents((afterSeq, limit) =>
+      api.get(`/api/heartbeat-runs/${run.id}/events?afterSeq=${afterSeq}&limit=${limit}`),
+    ),
   };
 }
 

tests/runner-e2e/everyday.test.ts

@@ -212,7 +212,7 @@ describe("manual everyday workflow catalog", () => {
     const selected = selectRunnerExecutions(
       parseRunnerSelectors(["--suite", "everyday-workflows"]),
     );
-    expect(selected).toHaveLength(38);
+    expect(selected).toHaveLength(47);
     expect(selected.every((e) => e.profile.generation === "native")).toBe(true);
     expect(
       selectRunnerExecutions(

tests/runner-e2e/run-observations.ts

@@ -1,3 +1,28 @@
+/** Read the complete durable stream; never grade a silently truncated page. */
+export async function collectRunEvents<T extends { seq?: number }>(
+  loadPage: (afterSeq: number, limit: number) => Promise<unknown>,
+): Promise<T[]> {
+  const pageSize = 1000;
+  const events: T[] = [];
+  let afterSeq = 0;
+  for (let pageNumber = 0; pageNumber < 100; pageNumber += 1) {
+    const page = await loadPage(afterSeq, pageSize);
+    if (!Array.isArray(page) || page.length > pageSize) {
+      throw new Error("Run event evidence returned an invalid page");
+    }
+    for (const event of page) {
+      const seq = event?.seq;
+      if (!Number.isSafeInteger(seq) || seq <= afterSeq) {
+        throw new Error("Run event evidence has a missing or non-increasing sequence");
+      }
+      afterSeq = seq;
+      events.push(event as T);
+    }
+    if (page.length < pageSize) return events;
+  }
+  throw new Error("Run event evidence exceeded 100 pages; refusing incomplete evidence");
+}
+
 export interface ObservableRunState {
   status?: string | null;
   errorCode?: string | null;

tests/runner-e2e/runner.spec.ts

@@ -21,6 +21,7 @@ import { setupLiveFixtures, type LiveFixtureValues } from "./live-fixtures.js";
 import { evaluateMatcher, persistedFinalRunMessage, type MatcherResult } from "./matchers.js";
 import {
   acceptedPlanSessionResetFailures,
+  collectRunEvents,
   hasTerminalMalformedPlanConfirmation,
   isControlPlaneGovernedResponseWait,
   isNonExecutingReviewFenceRun,
@@ -696,8 +697,8 @@ for (const execution of executions) {
             ),
           ),
           events: await capture(() =>
-            api.get<RunEventRecord[]>(
-              `/api/heartbeat-runs/${candidate.id}/events?limit=1000`,
+            collectRunEvents<RunEventRecord>((afterSeq, limit) =>
+              api.get(`/api/heartbeat-runs/${candidate.id}/events?afterSeq=${afterSeq}&limit=${limit}`),
             ),
           ),
         })),
@@ -1694,8 +1695,8 @@ for (const execution of executions) {
             try {
               return {
                 runId: candidate.id,
-                events: await api.get<RunEventRecord[]>(
-                  `/api/heartbeat-runs/${candidate.id}/events?limit=1000`,
+                events: await collectRunEvents<RunEventRecord>((afterSeq, limit) =>
+                  api.get(`/api/heartbeat-runs/${candidate.id}/events?afterSeq=${afterSeq}&limit=${limit}`),
                 ),
                 error: null,
               };

tests/runner-e2e/support.test.ts

@@ -38,6 +38,7 @@ import {
 } from "./ports.js";
 import {
   acceptedPlanSessionResetFailures,
+  collectRunEvents,
   hasTerminalMalformedPlanConfirmation,
   isControlPlaneGovernedResponseWait,
   isNonExecutingReviewFenceRun,
@@ -48,6 +49,47 @@ import {
 import { runnerE2EWebServerCommand } from "./web-server-command.js";
 
 const cleanupDirectories: string[] = [];
+
+describe("complete run event evidence", () => {
+  const page = Array.from({ length: 1000 }, (_, i) => ({ seq: i + 1, eventType: "item.delta" }));
+  it("reads completion events beyond the first 1000 rows", async () => {
+    const terminal = ["run.result.proposed", "run.result.accepted", "run.terminal"]
+      .map((eventType, i) => ({ seq: 1001 + i, eventType }));
+    const load = vi.fn().mockResolvedValueOnce(page).mockResolvedValueOnce(terminal);
+    const events = await collectRunEvents(load);
+    expect(events).toEqual([...page, ...terminal]);
+    expect(load.mock.calls).toEqual([[0, 1000], [1000, 1000]]);
+  });
+  it("checks for another page even at an exact page boundary", async () => {
+    const load = vi.fn().mockResolvedValueOnce(page).mockResolvedValueOnce([]);
+    expect(await collectRunEvents(load)).toEqual(page);
+    expect(load).toHaveBeenCalledTimes(2);
+  });
+  it.each([
+    null, {}, [{ eventType: "run.terminal" }], [{ seq: 0 }], [{ seq: -1 }],
+    [{ seq: 1.5 }], [{ seq: "1" }], [{ seq: NaN }], [{ seq: Infinity }],
+    [{ seq: 2 }, { seq: 1 }], [{ seq: 1 }, { seq: 1 }], [...page, { seq: 1001 }],
+  ])("rejects malformed evidence page %#", async (malformed) => {
+    await expect(collectRunEvents(async () => malformed)).rejects.toThrow("Run event evidence");
+  });
+  it("rejects a repeated cursor instead of accepting duplicate events", async () => {
+    const load = vi.fn().mockResolvedValue(page);
+    await expect(collectRunEvents(load)).rejects.toThrow("non-increasing sequence");
+    expect(load).toHaveBeenCalledTimes(2);
+  });
+  it("propagates a missing later page without returning partial evidence", async () => {
+    const load = vi.fn().mockResolvedValueOnce(page).mockRejectedValueOnce(new Error("Unavailable"));
+    await expect(collectRunEvents(load)).rejects.toThrow("Unavailable");
+  });
+  it("fails closed when a stream never ends within the bounded capture", async () => {
+    const load = vi.fn(async (afterSeq: number) => page.map((event) => ({ ...event, seq: event.seq + afterSeq })));
+    await expect(collectRunEvents(load)).rejects.toThrow("refusing incomplete evidence");
+    expect(load).toHaveBeenCalledTimes(100);
+  });
+});
+
+
+
 afterEach(async () => {
   vi.unstubAllEnvs();
   vi.unstubAllGlobals();