feat(task): show the agent's context window in task properties (#19273)

## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents run through a harness (Codex app-server, or Claude through ACPX), and the runner turns harness events into Paperclip events > - Both harnesses report how full the model's context window is. Codex sends `modelContextWindow` with its token usage. Claude sends `used` and `size` in ACP `usage_update` > - Paperclip dropped both values. The task view could not show how close an agent is to compaction, and a compaction left no visible mark in the task chat > - This pull request carries the context-window occupancy through the runner, stores it with the existing usage events, and shows it as a "Context" row in the task Properties panel > - It also shows a "Context compacted · 230k → 38k tokens" divider in the task chat when the agent compacts its context > - The benefit is that operators can see when a long task is near its context limit and when the agent lost detail to a compaction ## Linked Issues or Issue Description Refs #19252 (same pattern: a canonical harness event turned into task-chat UI). Refs #19267 (status notices; it turns Claude's compaction banner into a notice, and this PR reads that notice too). **What existing behavior does this improve?** The task view shows no context-window information, and a compaction is invisible in the task chat. **Subsystem affected** Runner usage normalization (Rust and TypeScript transport), the PRP usage schema, a read-only issue API, the task Properties panel and the task-chat transcript. **Current behavior** Codex `modelContextWindow` and ACP `used`/`size` are dropped in the runner. `context.compacted` always has null sizes, Claude compactions emit no `context.compacted` event at all, and the live runner turn hides every marker except "interrupted". **Proposed behavior** `usage.reported` carries an optional `context {usedTokens, windowTokens}`. The task Properties panel shows a Context row ("10% of 258k") with a hover breakdown (tokens used, input and cached input, output, cost, compactions). The transcript shows a compaction divider, with sizes from the usage before and after the compaction. **Reason and benefit** Operators can see when a long task is close to its context limit, and they can see where the agent's earlier context was summarized. **Breaking changes** None. The new schema field is optional, and the new endpoint is read-only. ## What Changed - PRP `usage.schema.json`: add an optional `context {usedTokens, windowTokens}` object. Regenerate the schema bundle and the protocol manifest. - Rust `provider_events.rs`: add `codex_context_usage` (last request `totalTokens` and `modelContextWindow`) and `acpx_context_usage` (ACP `used`/`size`). ACPX `usage_update` now sets `context`. - Rust `provider_backend.rs`: Codex `thread/tokenUsage/updated` sets `context` on live (non-historical) usage. - Rust `provider_events.rs`: emit `context.compacted` for the completed `context_compaction` provider notice that #19267 produces from Claude's compaction banner. Only this path marks a Claude compaction, so the server counts each compaction once. - `runnerd-codex-transport.ts`: `rehydrateRunnerdUsageNotification` keeps a valid `context` on `tokenUsage`, so it reaches the stored `item.completed` usage item. - Server: new `GET /api/issues/:id/context-usage` (issue read access is required) and `issueContextUsage` service. It reads only the assignee's current session (the session of its newest usage report), and returns that session's latest context reading, non-zero totals, cost, and the count of compactions across all of the session's runs. Added to OpenAPI. Shared type `IssueContextUsage`. - DB: migration 0432 adds two partial indexes on `heartbeat_run_events`, built concurrently. One indexes usage items by run and time. The other indexes `context.compacted` events by company and session. `EXPLAIN` shows that both lookups use them. - UI: new `ContextUsageMeter` (bar, percentage, hover card) and `IssueContextUsageRow` in the Execution section of Properties. The row refreshes every 8 s while a run is active, and once more when the run ends. - UI transcript: `native-run-events.ts` maps `context.compacted` to a `context` activity entry with pre/post sizes from neighbouring usage items. `transcript-adapter.ts` turns it into a new `context_compacted` marker variant. The live runner turn keeps that marker as its own timeline row. - Storybook: `task-context-window.stories.tsx` (Properties panel in a task, all states, hover breakdowns, divider). ## Verification - `cargo test -p paperclip-runner-core --lib provider_events` (new tests: ACPX context, Codex context, banner compaction, notice compaction) - `pnpm --filter @paperclipai/paperclip-runner exec vitest run src/live/runnerd-codex-transport.test.ts` - `cd ui && npx vitest run src/components/ContextUsageMeter.test.tsx src/components/transcript src/components/task-chat src/components/issue-properties` - `cd server && npx vitest run src/__tests__/issue-context-usage.test.ts` (embedded Postgres) - `node scripts/typecheck-low-memory.mjs`, `pnpm check:token-gates`, `pnpm --filter @paperclipai/paperclip-runner run check:protocol-types` - Live check on a local worktree instance through the runnerd path: - A Codex `gpt-5.5` task showed "10% of 258k" (27k of 258k, input 77k with 49k cached, output 1.1k). - A Claude Sonnet 4.6 task showed "17% of 200k" (34k of 200k). - Storybook: Tasks and conversations / … / Task chat / Context window. - Not verified live: a real compaction. The divider is covered by unit tests through the same projector path the task chat uses. ## Risks - The schema field is optional, and older runners omit it. The UI renders nothing until a harness reports a window. - Migration 0432 builds two partial indexes concurrently on `heartbeat_run_events`, which is a large table. The indexes cover only usage items and compaction events, so they stay small. They add a little write overhead to those events. - Claude in-progress usage rows still record zero tokens with `requests: 1`. The final turn usage is correct. This PR does not change accounting. - A "Compact now" action is out of scope. It needs a compaction-only run that does not produce a run result. It will come in a follow-up. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, in-app browser for Storybook and the live instance). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 16:02:36 +01:00 · 2026-11-05 16:54:27 -06:00
1 parent b61a815d3d
commit bf10bc695a
@@ -0,0 +1,98 @@
+import { randomUUID } from "node:crypto";
+import { afterAll, beforeAll, describe, expect, it } from "vitest";
+import { agents, companies, createDb, heartbeatRunEvents, heartbeatRuns, issues } from "@paperclipai/db";
+import { issueContextUsage } from "../services/issue-context-usage.js";
+import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
+
+const support = await getEmbeddedPostgresTestSupport();
+const suite = support.supported ? describe : describe.skip;
+suite("issue context usage", () => {
+  let db: ReturnType<typeof createDb>;
+  let temporaryDb: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
+  beforeAll(async () => {
+    temporaryDb = await startEmbeddedPostgresTestDatabase("issue-context-usage-");
+    db = createDb(temporaryDb.connectionString);
+  }, 60_000);
+  afterAll(async () => { await temporaryDb?.cleanup(); });
+
+  function prp(runId: string, sessionId: string, eventType: string, payload: Record<string, unknown>) {
+    return { prpEvent: { schema: "paperclip.prp.event.v1", runId, normalizedSessionId: sessionId, eventType, payload } };
+  }
+
+  it("returns the latest reported context window and counts the session's compactions", async () => {
+    const companyId = randomUUID(), agentId = randomUUID(), issueId = randomUUID(), otherIssueId = randomUUID();
+    await db.insert(companies).values({ id: companyId, name: "Context fixture", issuePrefix: `X${companyId.slice(0, 6)}` });
+    await db.insert(agents).values({ id: agentId, companyId, name: "Codie", role: "engineer", status: "idle", adapterType: "codex_local" });
+    await db.insert(issues).values([
+      { id: issueId, companyId, title: "Context", status: "in_progress", assigneeAgentId: agentId },
+      { id: otherIssueId, companyId, title: "Other", status: "in_progress", assigneeAgentId: agentId },
+    ]);
+    expect(await issueContextUsage(db, companyId, { id: issueId, assigneeAgentId: agentId })).toBeNull();
+
+    const runId = randomUUID(), otherRunId = randomUUID();
+    await db.insert(heartbeatRuns).values([
+      { id: runId, companyId, agentId, scopeKind: "issue", issueId, status: "running" },
+      { id: otherRunId, companyId, agentId, scopeKind: "issue", issueId: otherIssueId, status: "running" },
+    ]);
+    const usage = (usedTokens: number, total = { inputTokens: 76_682, cacheReadTokens: 48_896, outputTokens: 1_089 }) => ({
+      kind: "usage",
+      usage: { total, runDelta: total, context: { usedTokens, windowTokens: 258_400 } },
+    });
+    const base = { companyId, agentId, stream: "system" as const, level: "info" as const };
+    await db.insert(heartbeatRunEvents).values([
+      { ...base, runId, seq: 1, eventType: "item.completed", payload: prp(runId, "session-a", "item.completed", usage(229_500)),
+        createdAt: new Date("2026-11-05T10:00:00Z") },
+      { ...base, runId, seq: 2, eventType: "context.compacted", payload: prp(runId, "session-a", "context.compacted", {
+        schema: "paperclip.context.compacted.v1", compactionId: "c-1" }), createdAt: new Date("2026-11-05T10:00:01Z") },
+      { ...base, runId, seq: 3, eventType: "item.completed", payload: prp(runId, "session-a", "item.completed", usage(38_200)),
+        createdAt: new Date("2026-11-05T10:00:02Z") },
+      { ...base, runId, seq: 4, eventType: "item.completed", payload: prp(runId, "session-a", "item.completed", { kind: "usage", usage: { total: {} } }),
+        createdAt: new Date("2026-11-05T10:00:03Z") },
+      { ...base, runId: otherRunId, seq: 1, eventType: "item.completed", payload: prp(otherRunId, "session-b", "item.completed", usage(5)),
+        createdAt: new Date("2026-11-05T10:00:04Z") },
+    ]);
+
+    const issue = { id: issueId, assigneeAgentId: agentId };
+    expect(await issueContextUsage(db, companyId, issue)).toMatchObject({
+      runId,
+      agentId,
+      sessionId: "session-a",
+      usedTokens: 38_200,
+      windowTokens: 258_400,
+      totals: { inputTokens: 76_682, cachedInputTokens: 48_896, outputTokens: 1_089 },
+      costUsd: null,
+      compactions: 1,
+    });
+    expect(await issueContextUsage(db, randomUUID(), issue)).toBeNull();
+    // Another assignee's session never shows through.
+    expect(await issueContextUsage(db, companyId, { id: issueId, assigneeAgentId: randomUUID() })).toBeNull();
+    expect(await issueContextUsage(db, companyId, { id: issueId, assigneeAgentId: null })).toBeNull();
+
+    // A newer run that starts a fresh session hides the old session's reading
+    // until the new session reports its own context.
+    const freshRunId = randomUUID();
+    await db.insert(heartbeatRuns).values({ id: freshRunId, companyId, agentId, scopeKind: "issue", issueId, status: "running",
+      createdAt: new Date("2026-11-05T11:00:00Z") });
+    await db.insert(heartbeatRunEvents).values({ ...base, runId: freshRunId, seq: 1, eventType: "item.completed",
+      payload: prp(freshRunId, "session-c", "item.completed", { kind: "usage", usage: { total: {} } }),
+      createdAt: new Date("2026-11-05T11:00:01Z") });
+    expect(await issueContextUsage(db, companyId, issue)).toBeNull();
+    await db.insert(heartbeatRunEvents).values({ ...base, runId: freshRunId, seq: 2, eventType: "item.completed",
+      payload: prp(freshRunId, "session-c", "item.completed", usage(12_000)), createdAt: new Date("2026-11-05T11:00:02Z") });
+    expect(await issueContextUsage(db, companyId, issue)).toMatchObject({ sessionId: "session-c", usedTokens: 12_000, compactions: 0 });
+
+    // A session reused across more runs than the lookup window still counts
+    // compactions from its oldest runs.
+    const later = ["2026-11-05T12:00:00Z", "2026-11-05T13:00:00Z", "2026-11-05T14:00:00Z"];
+    for (const [index, at] of later.entries()) {
+      const id = randomUUID();
+      await db.insert(heartbeatRuns).values({ id, companyId, agentId, scopeKind: "issue", issueId, status: "succeeded", createdAt: new Date(at) });
+      await db.insert(heartbeatRunEvents).values({ ...base, runId: id, seq: 1, eventType: "item.completed",
+        payload: prp(id, "session-c", "item.completed", usage(13_000 + index)), createdAt: new Date(at) });
+    }
+    await db.insert(heartbeatRunEvents).values({ ...base, runId: freshRunId, seq: 3, eventType: "context.compacted",
+      payload: prp(freshRunId, "session-c", "context.compacted", { schema: "paperclip.context.compacted.v1", compactionId: "c-2" }),
+      createdAt: new Date("2026-11-05T11:00:03Z") });
+    expect(await issueContextUsage(db, companyId, issue)).toMatchObject({ sessionId: "session-c", usedTokens: 13_002, compactions: 1 });
+  });
+});
@@ -10,6 +10,7 @@ import { sanitizeRecord } from "../redaction.js";
 import { badRequest, forbidden } from "../errors.js";
 import { agentActionAuditService } from "../services/agent-action-audit.js";
 import { logActivity } from "../services/activity-log.js";
+import { issueContextUsage } from "../services/issue-context-usage.js";
 import { canActorReadHeartbeatRun } from "../services/heartbeat-run-privacy.js";
 import { issueReadSqlCondition } from "../services/authorization.js";
 import { caseActivityReadSqlCondition } from "../services/case-activity-access.js";
@@ -387,6 +388,13 @@ export function activityRoutes(db: Db) {
    res.json(result);
  });

+  router.get("/issues/:id/context-usage", async (req, res) => {
+    const issue = await getAccessibleResource(req, res, resolveIssueByRef(req.params.id as string), "Issue not found");
+    if (!issue) return;
+    if (!(await assertIssueReadAllowed(req, res, issue))) return;
+    res.json(await issueContextUsage(db, issue.companyId, issue));
+  });
+
  router.get("/heartbeat-runs/:runId/issues", async (req, res) => {
    assertAuthenticated(req);
    const runId = req.params.runId as string;
@@ -6168,6 +6168,15 @@ registry.registerPath({
  responses: { 200: r.ok(), 400: r.badRequest, 401: r.unauthorized, 403: r.forbidden, 404: r.notFound },
 });

+registry.registerPath({
+  method: "get",
+  path: "/api/issues/{id}/context-usage",
+  tags: ["activity"],
+  summary: "Latest context-window usage for an issue's agent session",
+  request: { params: z.object({ id: z.string() }) },
+  responses: { 200: r.ok(), 401: r.unauthorized, 403: r.forbidden, 404: r.notFound },
+});
+
 registry.registerPath({
  method: "get",
  path: "/api/heartbeat-runs/{runId}/issues",
@@ -0,0 +1,116 @@
+import { and, desc, eq, inArray, or, sql } from "drizzle-orm";
+import type { Db } from "@paperclipai/db";
+import { heartbeatRunEvents, heartbeatRuns } from "@paperclipai/db";
+import type { IssueContextUsage } from "@paperclipai/shared";
+
+/**
+ * Recent runs of the assignee searched for its newest usage report. The
+ * usage lookups use `heartbeat_run_events_usage_item_run_idx`, so they read
+ * only the few usage items of these runs.
+ */
+const RECENT_RUN_LIMIT = 3;
+
+function count(value: unknown): number | null {
+  const parsed = typeof value === "string" ? Number(value) : value;
+  return typeof parsed === "number" && Number.isFinite(parsed) && parsed >= 0 ? parsed : null;
+}
+
+function record(value: unknown): Record<string, unknown> | null {
+  return value && typeof value === "object" && !Array.isArray(value) ? value as Record<string, unknown> : null;
+}
+
+/** Matches the predicate of `heartbeat_run_events_usage_item_run_idx`. */
+const usageItem = sql`${heartbeatRunEvents.eventType} = 'item.completed' and (${heartbeatRunEvents.payload} -> 'prpEvent' -> 'payload' ->> 'kind') = 'usage'`;
+/** Matches the key expression of `heartbeat_run_events_compaction_session_idx`. */
+const sessionIdSql = sql<string | null>`(${heartbeatRunEvents.payload} -> 'prpEvent' ->> 'normalizedSessionId')`;
+
+/**
+ * Reads the most recent provider-reported context occupancy for the current
+ * agent session on a task. The current session is the one behind the
+ * assignee's newest usage report: after a reassignment or a fresh session,
+ * older readings are not returned. Returns null until that session reports a
+ * context window.
+ */
+export async function issueContextUsage(
+  db: Db,
+  companyId: string,
+  issue: { id: string; assigneeAgentId: string | null },
+): Promise<IssueContextUsage | null> {
+  if (!issue.assigneeAgentId) return null;
+  const runs = await db
+    .select({ id: heartbeatRuns.id })
+    .from(heartbeatRuns)
+    .where(and(
+      eq(heartbeatRuns.companyId, companyId),
+      eq(heartbeatRuns.agentId, issue.assigneeAgentId),
+      or(eq(heartbeatRuns.issueId, issue.id), eq(heartbeatRuns.nativeIssueId, issue.id)),
+    ))
+    .orderBy(desc(heartbeatRuns.createdAt), desc(heartbeatRuns.id))
+    .limit(RECENT_RUN_LIMIT);
+  if (runs.length === 0) return null;
+  const runIds = runs.map((run) => run.id);
+
+  const recentUsage = and(
+    eq(heartbeatRunEvents.companyId, companyId),
+    inArray(heartbeatRunEvents.runId, runIds),
+    usageItem,
+  );
+  // The newest usage report names the current session.
+  const [current] = await db
+    .select({ sessionId: sessionIdSql })
+    .from(heartbeatRunEvents)
+    .where(recentUsage)
+    .orderBy(desc(heartbeatRunEvents.createdAt), desc(heartbeatRunEvents.id))
+    .limit(1);
+  const sessionId = current?.sessionId ?? null;
+  if (!sessionId) return null;
+
+  const [latest] = await db
+    .select({
+      runId: heartbeatRunEvents.runId,
+      agentId: heartbeatRunEvents.agentId,
+      createdAt: heartbeatRunEvents.createdAt,
+      usage: sql<unknown>`${heartbeatRunEvents.payload} -> 'prpEvent' -> 'payload' -> 'usage'`,
+    })
+    .from(heartbeatRunEvents)
+    .where(and(
+      recentUsage,
+      sql`${sessionIdSql} = ${sessionId}`,
+      sql`${heartbeatRunEvents.payload} -> 'prpEvent' -> 'payload' -> 'usage' ? 'context'`,
+    ))
+    .orderBy(desc(heartbeatRunEvents.createdAt), desc(heartbeatRunEvents.id))
+    .limit(1);
+  if (!latest) return null;
+  const usage = record(latest.usage);
+  const context = record(usage?.context);
+  const usedTokens = count(context?.usedTokens);
+  const windowTokens = count(context?.windowTokens);
+  if (usedTokens === null || windowTokens === null || windowTokens < 1) return null;
+
+  // Every compaction of the session, across all of its runs.
+  const [compactions] = await db
+    .select({ count: sql<number>`count(*)::int` })
+    .from(heartbeatRunEvents)
+    .where(and(
+      eq(heartbeatRunEvents.companyId, companyId),
+      eq(heartbeatRunEvents.eventType, "context.compacted"),
+      sql`${sessionIdSql} = ${sessionId}`,
+    ));
+
+  const total = record(usage?.total);
+  const inputTokens = count(total?.inputTokens) ?? 0;
+  const cachedInputTokens = count(total?.cacheReadTokens) ?? count(total?.cachedInputTokens) ?? 0;
+  const outputTokens = count(total?.outputTokens) ?? 0;
+  const costUsd = count(total?.providerCostUsd);
+  return {
+    runId: latest.runId,
+    agentId: latest.agentId,
+    sessionId,
+    usedTokens,
+    windowTokens,
+    observedAt: latest.createdAt.toISOString(),
+    totals: inputTokens + outputTokens > 0 ? { inputTokens, cachedInputTokens, outputTokens } : null,
+    costUsd: costUsd && costUsd > 0 ? costUsd : null,
+    compactions: compactions?.count ?? 0,
+  };
+}