fix(evals): grade full cohort 4 chat cases on saved state and rendered links (#21034)

## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The chat eval catalog runs real agents through chat journeys and grades the saved results > - Full cohort 4 (OpenCode with DeepSeek v4.1 flash, master f1aef0cb58) failed 14 cases for eval reasons, not product reasons > - Some phase captures read links before the reply rendered. Some graders matched keywords in free-form replies. One locator was stale. One fixture answer looked like a placeholder. Some checks could not say what failed > - This pull request grades those cases on saved state, setup literals and rendered links, and it fixes the observers and fixtures > - The benefit is that these cases fail only when the product fails, and a failure names the part that broke ## Linked Issues or Issue Description **What happened?** In full chat cohort 4, valid agent behavior failed these cases: case-history, goals.explain-purpose and workspaces.preview-service (phase captures read before the reply rendered); routines.pause-owned ("If you want that one stopped too, use Pause on its task page" matched a cancellation regex); budgets-costs.spend-breakdown ("unknown, not $0.00" matched a stated-zero regex); steering.edit-queued-message ("change ... and drag it above" missed a keyword list); connection-setup.choose-provider (the resumed page says "Finish with Notion"); questions-and-approvals.necessary-question (the answer "Acorn 2a4f72ec4ae2-1" was saved as "Acorn"). Some checks could not say what failed: onboarding.new-company compared the whole members response as JSON and saved no after-state; agents.choose-primary threw one combined error. **Expected behavior** Each case grades the feature contract on saved state, setup literals and rendered or clicked links. A valid reworded reply passes. A broken product still fails. A failure names the clause and its values. **Steps to reproduce** Grade the retained cohort 4 evidence for the cases above with the earlier oracles. Refs #21009, #21002, #21007 ## What Changed - Phase captures (case-history first phase, goals.explain-purpose phases, preview-service query phase, ambiguous-task clarification, edit-queued-message guidance) now wait for the saved reply to render (`waitForTurnReplyLinks`) before they observe. When a saved reply yields no rendered links, the observation records thread and anchor diagnostics (`linkDiagnostics`) and logs them. - routines.pause-owned: remove the reply keyword clause. Saved state at the reply proves the run kept running, and the final state proves it completed normally. - budgets-costs.spend-breakdown: the unresolved record is graded structurally. The reply must identify it (rendered link, identifier or run id), and a chat run must have read its saved accounting (cost summary, run record or unranged company report, bound by input digest). "$0.00" in the reply is not matched. - steering.edit-queued-message: grade the rendered task link in the reply, the refused agent edit and reorder (403), and the unchanged queue at the reply. Keyword lists are removed. - connection-setup.choose-provider: the sign-in locator also accepts "Finish with Notion" (the resumed-draft label). The button has no test id or other stable name. - questions-and-approvals.necessary-question: the answer is now "Quillmere". The case also accepts the exact answer in a saved deliverable (a document revision or text attachment on the linked task, saved by an agent of the company after the answer). The yaml expected outcome records this decision, with a `changedFields` entry in the testops manifest. - onboarding.new-company: save the members after-state and compare identity, role, status and grants, and name what changed. Guidance is graded on a rendered link to `/onboarding` (the new-organization wizard), the business literal from the query, and the saved company count. The source company and person get realistic names for this case and are restored at cleanup, because the CEO reasoned from the generated names that the request was a graded test. - agents.choose-primary: the settled-Chat check is split into four named clauses with values, saved as evidence on failure. - A question card the turn ended on counts as its reply (decision after cohort 4) in cli-operations.owned-dev-instance, onboarding.new-company and status-cards.other-authors-card. The card's Markdown-rendered fields (question form description; confirmation prompt and details) count as reply links. Link requirements still apply. - Every changed case bumps its `oracleVersion`; the descriptor snapshot and inventory digest are updated. Why each old check was wrong and why the new one still catches a broken product: | Case | Old check was wrong because | A broken product still fails because | |---|---|---| | phase captures | runs succeeded and the chat waited before the reply rendered, so links were empty | the same link checks run on the rendered reply | | routines.pause-owned | a correct offer to pause the run read as a cancellation | the checkpoint run must be running and the run must later succeed | | spend-breakdown | "unknown, not $0.00" read as "$0" | the reply must identify the record and a chat run must read its saved accounting; totals and per-agent amounts stay graded | | edit-queued-message | correct wording missed a keyword list | the reply must link the task, the agent's edit and reorder must be refused, and the queue must be unchanged at the reply | | choose-provider | the resumed page label was not accepted | the button must still be visible on the selected Notion surface | | necessary-question | the answer read as a placeholder; deliverables did not count | the exact answer must be saved in the linked work after the answer, by an agent of the company | | new-company | timestamps and profiles made the JSON compare fail without saying why | membership identity, role, status and grants must be unchanged | | choose-primary | one combined error | the same four conditions, each named | | card as reply | a card answer read as an empty reply | card text must exist and link requirements still apply | ## Verification - Calibration tests (valid, reworded valid, broken) for each changed grader: `integrator-routine-pause.test.ts`, `identity-accounting.test.ts` (includes the cohort 4 reply), `runtime-queue-controls.test.ts` (includes the cohort 4 reply), `external-provider-choice.test.ts`, `identity.test.ts`, `identity-onboarding-work.test.ts`, `runtime-primary-ready-chat.test.ts`, `integrator-status-cards.test.ts`, `turn-response-card.test.ts`, `chat-rendered-links.test.ts`. - The corrected spend-breakdown grader passes on the retained cohort 4 evidence (its cost-summary read binds to the unresolved task) and fails with the reads removed. - `npx vitest run --config tests/runner-e2e/vitest.config.ts chat-catalog chat-rendered chat-linked`: 284 files passed. - `node --test tests/runner-e2e/chat-full-catalog-testops-manifest.test.mjs`: 10/10 passed. - `node scripts/typecheck-test-projects.mjs tests/runner-e2e`: passed. - `node tests/runner-e2e/source-sdk-public-inputs.mjs <sha> --check`: unchanged. - Paid run (8 cells, OpenCode + DeepSeek v4.1 flash) on this branch combined with the guidance PR: see the PR comment for results. ## Risks - The new-company guidance check now needs a rendered link to `/onboarding`. A reply that only names the company switcher fails. The companion guidance PR tells agents this route. - spend-breakdown does not grade wording, so a reply that reads the record and still calls it "$0" is not caught. The amounts, total and identification are still graded. - The new-company case renames the source company and the person for the case and restores them at cleanup. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use, shell execution and code editing. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 07:12:00 +01:00 · 2026-11-12 00:37:42 -06:00
1 parent 0dfbbdeb31
commit 98c18784c6
@@ -34,7 +34,8 @@ describe("catalog oracle contract", () => {
    // gateways-profiles.create-gateway and rotate-token: agents keep delegated gateway/token authority and never receive token values.
    // instance-operations.instance-settings retargeted from the removed Agent Chat toggle (always on since #19360) to the Cases experiment.
    // execution-environments.choose-capable-target: the second computer is a saved SSH target advertised for Claude Code but never tested.
-    expect(chatCatalogInventoryDigest).toBe("4d056e4916ecb3b8108b6304aaaa820f2fb5829518e0ce7e0c7174519da39025");
+    // questions-and-approvals.necessary-question: the answer may shape the saved deliverable as well as the brief.
+    expect(chatCatalogInventoryDigest).toBe("9a619ae82ff1aaa6a63b723b57c791adfe37e6ab438acc45431fef75c8611dcd");
    for (const implementation of chatCatalogCases) expect(chatCatalogScenarios.some(row => row.id === implementation.scenarioId)).toBe(true);
  });
 });
@@ -487,7 +487,7 @@
      "chat-catalog-domains/identity-support.ts",
      "chat-scenarios/agents.yaml"
    ],
-    "oracleVersion": "paperclip.chat-catalog.runtime-primary.v4",
+    "oracleVersion": "paperclip.chat-catalog.runtime-primary.v5",
    "coverageGaps": [],
    "expectedRunCount": 1,
    "turnTimeoutMs": 120000,
@@ -1254,9 +1254,11 @@
      "chat-catalog-domains/identity-accounting.ts",
      "chat-catalog-domains/identity-support.ts",
      "chat-catalog-domains/internal-support.ts",
-      "chat-scenarios/budgets-costs.yaml"
+      "chat-scenarios/budgets-costs.yaml",
+      "chat-catalog-domains/api-read-receipts.ts",
+      "run-observations.ts"
    ],
-    "oracleVersion": "paperclip.chat-catalog.spend-breakdown.structural.v3",
+    "oracleVersion": "paperclip.chat-catalog.spend-breakdown.structural.v4",
    "expectedRunCount": 2,
    "turnTimeoutMs": 120000,
    "attemptTimeoutMs": 600000,
@@ -1319,9 +1321,11 @@
      "chat-catalog-domains/integrator-support.ts",
      "chat-catalog-domains/internal-support.ts",
      "chat-catalog-domains/internal-definition.ts",
+      "chat-catalog-domains/identity-support.ts",
+      "chat-rendered-links.ts",
      "chat-scenarios/cases.yaml"
    ],
-    "oracleVersion": "paperclip.chat-catalog.integrator.case-history.v3",
+    "oracleVersion": "paperclip.chat-catalog.integrator.case-history.v4",
    "expectedRunCount": 5,
    "expectedHumanMessageCount": 2,
    "turnTimeoutMs": 120000,
@@ -1987,7 +1991,7 @@
      "../e2e/fetch-allowed-port.ts",
      "redaction.ts"
    ],
-    "oracleVersion": "paperclip.chat-catalog.privileged.owned-dev-instance.v2",
+    "oracleVersion": "paperclip.chat-catalog.privileged.owned-dev-instance.v3",
    "coverageGaps": [],
    "expectedRunCount": 2,
    "expectedHumanMessageCount": 2,
@@ -2235,7 +2239,7 @@
      "../../ui/src/features/connections/ConnectionSetupFlow.tsx",
      "chat-scenarios/connection-setup.yaml"
    ],
-    "oracleVersion": "paperclip.chat-catalog.provider-choice.structural.v2",
+    "oracleVersion": "paperclip.chat-catalog.provider-choice.structural.v3",
    "expectedRunCount": 1,
    "turnTimeoutMs": 120000,
    "attemptTimeoutMs": 300000,
@@ -3290,9 +3294,11 @@
      "chat-catalog-domains/internal-definition.ts",
      "chat-catalog-domains/internal-support.ts",
      "chat-catalog-domains/internal-goals.ts",
-      "chat-scenarios/goals.yaml"
+      "chat-scenarios/goals.yaml",
+      "chat-catalog-domains/identity-support.ts",
+      "chat-rendered-links.ts"
    ],
-    "oracleVersion": "paperclip.chat-catalog.internal.v1",
+    "oracleVersion": "paperclip.chat-catalog.internal.goals-purpose.v2",
    "coverageGaps": [],
    "expectedRunCount": 3,
    "expectedHumanMessageCount": 3,
@@ -3772,6 +3778,8 @@
    "scenarioId": "onboarding.new-company",
    "sourceFiles": [
      "chat-catalog-domains/identity-new-company.ts",
+      "chat-catalog-domains/turn-response-card.ts",
+      "chat-rendered-links.ts",
      "chat-catalog-domains/identity-new-company-native.ts",
      "chat-catalog-domains/identity-new-company-native-proof.ts",
      "chat-catalog-domains/identity-wizard.ts",
@@ -3781,7 +3789,7 @@
      "chat-catalog-domains/internal-support.ts",
      "chat-scenarios/onboarding.yaml"
    ],
-    "oracleVersion": "paperclip.chat-catalog.identity.v1",
+    "oracleVersion": "paperclip.chat-catalog.identity.new-company.v2",
    "expectedRunCount": 1,
    "turnTimeoutMs": 120000,
    "attemptTimeoutMs": 600000,
@@ -4295,7 +4303,7 @@
      "chat-catalog-domains/identity-questions-and-approvals.ts",
      "chat-scenarios/questions-and-approvals.yaml"
    ],
-    "oracleVersion": "paperclip.chat-catalog.identity.necessary-question.v2",
+    "oracleVersion": "paperclip.chat-catalog.identity.necessary-question.v3",
    "expectedRunCount": 4,
    "minimumRunCount": 1,
    "turnTimeoutMs": 120000,
@@ -4689,7 +4697,7 @@
      "chat-catalog-domains/integrator-support.ts",
      "chat-scenarios/routines.yaml"
    ],
-    "oracleVersion": "paperclip.chat-catalog.routine-pause.structural.v2",
+    "oracleVersion": "paperclip.chat-catalog.routine-pause.structural.v3",
    "expectedRunCount": 3,
    "expectedHumanMessageCount": 1,
    "turnTimeoutMs": 120000,
@@ -5410,7 +5418,7 @@
      "chat-catalog-domains/integrator-status-cards.ts",
      "chat-scenarios/status-cards.yaml"
    ],
-    "oracleVersion": "paperclip.chat-catalog.status-card.other-authors.v3",
+    "oracleVersion": "paperclip.chat-catalog.status-card.other-authors.v4",
    "expectedRunCount": 5,
    "minimumRunCount": 2,
    "expectedHumanMessageCount": 1,
@@ -5511,9 +5519,11 @@
      "chat-catalog-domains/runtime-busy-worker.ts",
      "chat-catalog-domains/runtime-support.ts",
      "chat-catalog-domains/identity-support.ts",
+      "chat-catalog-domains/internal-support.ts",
+      "chat-rendered-links.ts",
      "chat-scenarios/steering.yaml"
    ],
-    "oracleVersion": "paperclip.chat-catalog.runtime.v2",
+    "oracleVersion": "paperclip.chat-catalog.edit-queued-message.structural.v3",
    "coverageGaps": [],
    "expectedRunCount": 3,
    "turnTimeoutMs": 120000,
@@ -5806,9 +5816,11 @@
      "chat-catalog-domains/internal-tasks.ts",
      "chat-scenarios/tasks.yaml",
      "chat-catalog-domains/internal-distinct-fixtures.ts",
-      "chat-linked-question.ts"
+      "chat-linked-question.ts",
+      "chat-catalog-domains/identity-support.ts",
+      "chat-rendered-links.ts"
    ],
-    "oracleVersion": "paperclip.chat-catalog.internal.ambiguous-task.v2",
+    "oracleVersion": "paperclip.chat-catalog.internal.ambiguous-task.v3",
    "coverageGaps": [],
    "expectedRunCount": 2,
    "expectedHumanMessageCount": 2,
@@ -6257,9 +6269,11 @@
      "chat-catalog-domains/runtime-preview.ts",
      "chat-catalog-domains/runtime-workspaces.ts",
      "chat-catalog-domains/runtime-support.ts",
+      "chat-catalog-domains/identity-support.ts",
+      "chat-rendered-links.ts",
      "chat-scenarios/workspaces.yaml"
    ],
-    "oracleVersion": "paperclip.chat-catalog.preview-service.v4",
+    "oracleVersion": "paperclip.chat-catalog.preview-service.v5",
    "coverageGaps": [],
    "expectedRunCount": 3,
    "expectedHumanMessageCount": 2,
@@ -2,6 +2,7 @@ import { describe, expect, it } from "vitest";
 import { getConnectableAppDefinition } from "../../../packages/shared/src/app-definitions.js";
 import {
  gradeProviderChoice,
+  NOTION_SIGN_IN_CONTROL,
  notionSetupChoices,
  truthfulNotionChoices,
  publicChoiceMessages,
@@ -257,3 +258,14 @@ describe("actual source attribution and supported choice lanes", () => {
      ).toBe(false);
  });
 });
+
+// Full cohort 4: the agent created the draft first, so the resumed setup page labelled its sign-in button
+// "Finish with Notion" (ConnectionSetupFlow's resuming label) and the old locator never found it.
+describe("Notion sign-in control locator", () => {
+  it("accepts the new-draft, resumed-draft and generic sign-in labels", () => {
+    for (const label of ["Continue to Notion", "Finish with Notion", "Continue to sign in", "Sign in to Notion"]) expect(NOTION_SIGN_IN_CONTROL.test(label), label).toBe(true);
+  });
+  it("rejects controls that do not start sign-in", () => {
+    for (const label of ["Try again", "Back", "Preparing…", "Opening Notion…", "Finish with Slack", "Open sign-in in a new tab"]) expect(NOTION_SIGN_IN_CONTROL.test(label), label).toBe(false);
+  });
+});
@@ -13,6 +13,9 @@ import {
 import { waitForReply } from "./identity-support.js";
 import { collectRunEvents } from "../run-observations.js";
 import type { ChatCatalogContext } from "../chat-catalog-contract.js";
+/** The setup page's sign-in control: "Continue to Notion" on a new draft, "Finish with Notion" when the page resumes
+ * a draft the agent already created (ConnectionSetupFlow), or the generic sign-in step. The button has no test id. */
+export const NOTION_SIGN_IN_CONTROL = /Continue to (?:Notion|sign in)|Finish with Notion|Sign in to Notion/;
 interface Gallery {
  apps: AppDefinition[];
  credentialSources: {
@@ -285,7 +288,7 @@ export const providerChoiceExternalCases = [
    gradeProviderChoice,
    {
      authentication: "human-session",
-      oracleVersion: "paperclip.chat-catalog.provider-choice.structural.v2",
+      oracleVersion: "paperclip.chat-catalog.provider-choice.structural.v3",
      sourceFiles: [
        "chat-catalog-domains/external-provider-choice.ts",
        "run-observations.ts",
@@ -375,7 +378,7 @@ export const providerChoiceExternalCases = [
        await expect(
          surface
            .getByRole("button", {
-              name: /Continue to (?:Notion|sign in)|Sign in to Notion/,
+              name: NOTION_SIGN_IN_CONTROL,
            })
            .first(),
        ).toBeVisible();
@@ -3,10 +3,12 @@ import { accountingIdentityCases, leadApiCalls, nativeAccountingPrerequisite, re
 import { callApiInputDigest } from "./api-read-receipts.js";
 import type { IdentityObservation } from "./identity-support.js";
 const costRead = (runId: string, operationId: string, status = 200, digest = "d"): LeadApiCall => ({ runId, operationId, digest, ok: status < 300, status });
+/** The chat run's read of the unresolved prerequisite task's saved cost summary, by its identifier. */
+const preflightSummaryRead = (runId: string, id = "ACO-5", status = 200) => costRead(runId, "GET /api/issues/{id}/cost-summary", status, callApiInputDigest({ operationId: "GET /api/issues/{id}/cost-summary", pathParams: { id } }));
 function valid(scenarioId: string): IdentityObservation {
  const repair = scenarioId.endsWith("repair-cost-recording");
  return { scenarioId, origin: "http://fixture.invalid", companyId: "company", leadId: "ceo", company: { id: "company" }, companies: [], agents: [], tasks: [], projects: [], goals: [], comments: [], documents: [], revisions: [], interactions: [], humanQueries: repair ? ["Actual retry receipt"] : [],
-    runs: [{ id: "original", companyId: "company", agentId: "morgan", status: "succeeded", runtimeMode: "native" }, ...(repair ? [{ id: "final", companyId: "company", agentId: "ceo", status: "succeeded", runtimeMode: "native" }] : [])],
+    runs: [{ id: "original", companyId: "company", agentId: "morgan", status: "succeeded", runtimeMode: "native" }, repair ? { id: "final", companyId: "company", agentId: "ceo", status: "succeeded", runtimeMode: "native" } : { id: "answer", companyId: "company", agentId: "ceo", status: "succeeded", runtimeMode: "native" }],
    links: ["/ACO/issues/morgan-receipt", "/ACO/issues/riley-receipt", ...(repair ? ["/ACO/activity/costs"] : [])],
    renderedLinkEvidence: repair ? [{ threadTestId: "task-chat-thread", href: "/ACO/activity/costs", source: "reply", conversationId: "chat", anchorCommentId: "c1", commentId: "c1" }] : [],
    reply: repair ? "Open Costs and retry recording the existing saved usage without rerunning the work." : "Launch this week: Morgan $1.37; Riley $2.63; recorded metered API total $4.00. ACO-5 has saved usage with no cost recorded yet, so that total is incomplete.",
@@ -16,7 +18,7 @@ function valid(scenarioId: string): IdentityObservation {
      originalUsagePreserved: true, originalLogPreserved: true, originalTaskPreserved: true, unresolvedBefore: { canRetry: true, runs: [{ runId: "original", reason: "retryable" }], events: [] },
      receiptOrigin: "public-imported-synthetic-fixture", weekFrom: "2026-10-26T00:00:00Z", weekTo: "2026-11-02T00:00:00Z",
      issueCosts: [{ costCents: 137 }, { costCents: 263 }], projectCosts: [{ projectId: "launch", costCents: 400 }], agentRetryStatus: 403, humanRetryHttpStatus: 200, retryAccounted: false, retryReceiptVisible: true, retryQuality: { runs: [{ runId: "original", reason: "retryable" }], events: [] }, retryLedger: [],
-      initialReplyCommentIds: ["c1"], leadApiCalls: repair ? [costRead("final", "GET /api/companies/{companyId}/costs/by-agent-model", 200, callApiInputDigest({ operationId: "GET /api/companies/{companyId}/costs/by-agent-model", pathParams: { companyId: "company" } }))] : [], finalReply: "The saved cost still remains unresolved; it is not a confirmed charge.", finalReplyRunId: "final", retryHumanFollowup: "Actual retry receipt", confirmedPricedRepair: false, afterStillUnresolved: true } };
+      initialReplyCommentIds: ["c1"], leadApiCalls: repair ? [costRead("final", "GET /api/companies/{companyId}/costs/by-agent-model", 200, callApiInputDigest({ operationId: "GET /api/companies/{companyId}/costs/by-agent-model", pathParams: { companyId: "company" } }))] : [preflightSummaryRead("answer")], finalReply: "The saved cost still remains unresolved; it is not a confirmed charge.", finalReplyRunId: "final", retryHumanFollowup: "Actual retry receipt", confirmedPricedRepair: false, afterStillUnresolved: true } };
 }
 describe("actual saved native accounting contexts", () => {
  for (const d of accountingIdentityCases) {
@@ -26,23 +28,28 @@ describe("actual saved native accounting contexts", () => {
    it(`${d.scenarioId}: rerunning original work fails`, () => { const o = valid(d.scenarioId); o.runs.push({ id: "rerun", companyId: "company", agentId: "morgan", status: "succeeded", runtimeMode: "native" }); expect(d.grade(o).passed).toBe(false); });
  }
  it("does not silently include prior-week/other-project receipts", () => { const o = valid("budgets-costs.spend-breakdown"); o.state.projectCosts = [{ projectId: "launch", costCents: 424 }]; expect(accountingIdentityCases[0]!.grade(o).passed).toBe(false); });
-  it("spend breakdown accepts any wording that points at the unresolved record and rejects a $0.00 row or an omitted record", () => {
+  it("spend breakdown grades the unresolved record structurally: identified in the reply and read from saved accounting", () => {
    const grade = accountingIdentityCases[0]!.grade;
-    const table = valid("budgets-costs.spend-breakdown"); table.reply = "| Agent | Cost |\n|---|---|\n| Morgan | $1.37 |\n| Riley | $2.63 |\n| **Total** | **$4.00** |\n\nRun original (Morgan, [ACO-5](/ACO/issues/ACO-5)) has usage but no cost event yet; it is not included above.";
+    // Valid, full cohort 4 wording: an explicit "unknown, not $0.00" is a correct statement, not a stated zero.
+    const cohort4 = valid("budgets-costs.spend-breakdown"); cohort4.reply = "Launch this week: **$4.00 in priced usage**, and the total is incomplete.\n\n| Agent | Priced cost |\n|---|---|\n| Riley | $2.63 |\n| Morgan | $1.37 |\n| **Total** | **$4.00** |\n\nMorgan also has one unpriced Launch run on [ACO-5](/ACO/issues/ACO-5), so its cost is unknown, not $0.00, and Launch's true total is above $4.00.";
+    cohort4.links.push("/ACO/issues/ACO-5");
+    expect(grade(cohort4).passed, JSON.stringify(grade(cohort4))).toBe(true);
+    // Reworded valid: a table row and a link alone point at the record; no particular words are needed.
+    const table = valid("budgets-costs.spend-breakdown"); table.reply = "| Agent | Cost |\n|---|---|\n| Morgan | $1.37 |\n| Riley | $2.63 |\n| **Total** | **$4.00** |\n\nOne more Morgan run has usage with no price yet (see the prerequisite task).";
+    table.links.push("/ACO/issues/preflight");
    expect(grade(table).passed).toBe(true);
-    const zero = valid("budgets-costs.spend-breakdown"); zero.reply = "| Agent | Cost | Task |\n|---|---|---|\n| Morgan | $1.37 | receipt |\n| Riley | $2.63 | receipt |\n| Morgan | $0.00 | [ACO-5](/ACO/issues/ACO-5) (usage recorded, no cost event) |\n| **Total** | **$4.00** | |";
-    expect(grade(zero).passed).toBe(false);
+    // Broken: the record is never identified, or its accounting was never read (an invented or assumed cost).
    const omitted = valid("budgets-costs.spend-breakdown"); omitted.reply = "Launch this week: Morgan $1.37; Riley $2.63; total $4.00.";
    expect(grade(omitted).passed).toBe(false);
+    const unread = valid("budgets-costs.spend-breakdown"); unread.state.leadApiCalls = [];
+    expect(grade(unread).passed).toBe(false);
+    const denied = valid("budgets-costs.spend-breakdown"); denied.state.leadApiCalls = [preflightSummaryRead("answer", "ACO-5", 403)];
+    expect(grade(denied).passed).toBe(false);
+    const otherTask = valid("budgets-costs.spend-breakdown"); otherTask.state.leadApiCalls = [preflightSummaryRead("answer", "ACO-3")];
+    expect(grade(otherTask).passed).toBe(false);
+    const workerRead = valid("budgets-costs.spend-breakdown"); workerRead.state.leadApiCalls = [preflightSummaryRead("original")];
+    expect(grade(workerRead).passed).toBe(false);
  });
-  it("spend breakdown grades only a $0 amount on the unresolved record, not words such as free", () => {
-    const grade = accountingIdentityCases[0]!.grade; const base = "Launch this week: Morgan $1.37; Riley $2.63; total $4.00. ";
-    for (const [tail, ok] of [["ACO-5 pending usage: $0.00.", false], ["ACO-5 must not be counted as free; its cost is not recorded yet.", true], ["ACO-5 is unpriced, so it contributes no dollar amount.", true]] as const) {
-      const o = valid("budgets-costs.spend-breakdown"); o.reply = base + tail; expect(grade(o).passed, tail).toBe(ok);
-    }
-  });
-
-  it("does not count unresolved native usage as free", () => { const o = valid("budgets-costs.spend-breakdown"); o.reply += " ACO-5 pending usage: $0.00."; expect(accountingIdentityCases[0]!.grade(o).passed).toBe(false); });
  it("a HTTP200 retry alone does not establish repaired accounting", () => { const o = valid("budgets-costs.repair-cost-recording"); o.state.afterStillUnresolved = false; expect(accountingIdentityCases[1]!.grade(o).passed).toBe(false); });
  it("a genuine confirmed priced repair is distinct from still-unresolved", () => { const o = valid("budgets-costs.repair-cost-recording"); o.state.confirmedPricedRepair = true; o.state.afterStillUnresolved = false; o.state.retryAccounted = true; o.state.retryQuality = { runs: [], events: [] }; o.state.retryLedger = [{ agentId: "morgan", eventCount: 1, estimatedEventCount: 0 }]; o.state.finalReply = "The saved usage cost is now recorded. The original work was not rerun."; expect(accountingIdentityCases[1]!.grade(o).passed).toBe(true); });
  it("grades no reply wording: reworded guidance and explanations pass on the same saved evidence", () => {
@@ -151,6 +158,11 @@ describe("company cost reads and task summaries bound to the repaired run", () =
    expect(readSavedAccounting([read("GET /api/issues/{id}/cost-summary", { pathParams: { id: task.identifier } })], "final", { task })).toBe(true);
    for (const period of ["all", "month"]) expect(readSavedAccounting([read("GET /api/companies/{companyId}/costs/summary", { pathParams: { companyId: company }, query: { period } })], "final", { companyId: company })).toBe(true);
  });
+  it("accepts an unranged company report whose company the server fills from the run", () => {
+    expect(readSavedAccounting([read("GET /api/companies/{companyId}/costs/by-agent", {})], "final", { companyId: company })).toBe(true);
+    expect(readSavedAccounting([read("GET /api/companies/{companyId}/costs/by-agent", { pathParams: {} })], "final", { companyId: company })).toBe(true);
+    expect(readSavedAccounting([read("GET /api/companies/{companyId}/costs/by-agent", { query: { from: "2020-01-01" } })], "final", { companyId: company })).toBe(false);
+  });
  it("rejects a ranged company report, another company, and a children-only task summary", () => {
    expect(readSavedAccounting([read("GET /api/companies/{companyId}/costs/summary", { pathParams: { companyId: company }, query: { from: "2020-01-01", to: "2020-01-02" } })], "final", { companyId: company })).toBe(false);
    expect(readSavedAccounting([read("GET /api/companies/{companyId}/costs/summary", { pathParams: { companyId: "other" } })], "final", { companyId: company })).toBe(false);
@@ -101,7 +101,8 @@ export interface SavedRunRecord { runId: string; assetId: string; recordId: stri
 export function readSavedAccounting(calls: readonly LeadApiCall[], runId: string, target: SavedAccountingTarget) {
  const { companyId, task, originalRunId } = target;
  const companyDigests = companyId ? apiReadDigests(COMPANY_COST_SEGMENTS.map(segment => ({ operationId: `GET /api/companies/{companyId}/costs/${segment}`,
-    pathParams: recordParams("companyId", companyId), queries: [{ period: "all" }, { period: "month" }] }))) : new Set<string>();
+    // runnerApiUrl fills companyId from the run's company when the call omits it, so that form reads the same report.
+    pathParams: [...recordParams("companyId", companyId), undefined], queries: [{ period: "all" }, { period: "month" }] }))) : new Set<string>();
  const taskDigests = task ? apiReadDigests([{ operationId: "GET /api/issues/{id}/cost-summary", pathParams: recordParams("id", task.id, task.identifier), queries: [{ excludeRoot: "false" }] }]) : new Set<string>();
  const runDigests = originalRunId ? apiReadDigests([{ operationId: "GET /api/heartbeat-runs/{runId}", pathParams: recordParams("runId", originalRunId) }]) : new Set<string>();
  // A re-read of the run record can carry extra input fields that change its digest. It still counts when the reading
@@ -175,15 +176,22 @@ export function gradeAccounting(value: unknown) {
    c.add("independent-scoped-week-ledger", o.state.receiptOrigin === "public-imported-synthetic-fixture" && totals?.length === 2 && totals[0]?.costCents === 137 && totals[1]?.costCents === 263
      && projects?.find(row => row.projectId === entity(o.state, "project").id)?.costCents === 400 && typeof o.state.weekFrom === "string" && typeof o.state.weekTo === "string",
      "Two explicitly imported synthetic USD API receipts total $4 in Launch's declared week, independently excluding the other-project and prior-week distractors; these are not live campaign/provider charges.");
-    // Requested facts (the per-agent dollars, the Launch total and the supporting receipt links) are
-    // graded from the reply; incomplete accounting is graded structurally: the reply must point at the
-    // record whose cost is unresolved (its task link or identifier, or the run) and must not give that
-    // record a zero dollar amount. Words such as "subscription" or "incomplete" are not required.
+    // Requested facts (the per-agent dollars, the Launch total and the supporting receipt links) are setup
+    // literals graded from the reply. The incomplete accounting is graded structurally, never by wording: the
+    // reply points at the record whose cost is unresolved (its rendered link, task identifier or run id), and a
+    // chat run actually read that record's saved accounting (its cost summary, its run record, or an unranged
+    // company cost report) before answering. A reply's "$0.00" is not matched: "unknown, not $0.00" is correct.
    c.add("actual-agent-breakdown-and-incomplete-total", /Morgan[\s\S]{0,120}(?:\$|USD\s*)?1\.37/i.test(o.reply) && /Riley[\s\S]{0,120}(?:\$|USD\s*)?2\.63/i.test(o.reply)
      && /(?:\$|USD\s*)4\.00/.test(o.reply)
-      && unresolvedRecordIdentifiedNotFree(o.reply, o.state.preflightTask as Entity | undefined, String(o.state.originalRunId ?? ""))
+      && unresolvedRecordIdentified(o, o.state.preflightTask as Entity | undefined, String(o.state.originalRunId ?? ""))
      && hasLink(o, "issues", entity(o.state, "morganReceiptTask")) && hasLink(o, "issues", entity(o.state, "rileyReceiptTask")),
-      "The actual answer breaks down the recorded scoped API dollars by real agent with supporting records, and points at the unresolved usage record without giving it a $0 amount.");
+      "The actual answer breaks down the recorded scoped API dollars by real agent with supporting records, and points at the record whose usage is unresolved.");
+    const calls = Array.isArray(o.state.leadApiCalls) ? o.state.leadApiCalls as LeadApiCall[] : undefined;
+    const savedRunRecords = Array.isArray(o.state.savedRunRecords) ? o.state.savedRunRecords as SavedRunRecord[] : [];
+    c.add("unresolved-record-read-from-saved-accounting", !!calls && o.runs.some(run => run.agentId === o.leadId && run.status === "succeeded"
+      && readSavedAccounting(calls, run.id, { companyId: o.companyId, task: o.state.preflightTask as Entity | undefined, originalRunId: String(o.state.originalRunId ?? ""),
+        citedRunReads: citedRunReads(run), savedRunRecords })),
+      "A chat run read the unresolved record's saved accounting through a product cost projection instead of inferring or inventing its cost.");
  } else if (o.scenarioId === "budgets-costs.repair-cost-recording") {
    c.add("real-human-saved-cost-retry", quality?.runs.some(row => row.runId === before?.id && row.reason === "retryable") === true && o.state.agentRetryStatus === 403
      && o.state.humanRetryHttpStatus === 200 && typeof o.state.retryAccounted === "boolean" && o.state.retryReceiptVisible === true
@@ -233,6 +241,15 @@ async function observeAccounting(ctx: ChatCatalogContext, state: Record<string,
    state.issueCosts = await Promise.all(["morganReceiptTask", "rileyReceiptTask"].map(key => ctx.input.api.get<IssueCostSummary>(`/api/issues/${entity(state, key).id}/cost-summary`)));
    state.projectCosts = await ctx.input.api.get<CostByProject[]>(`/api/companies/${ctx.input.fixtures.company.id}/costs/by-project?from=${encodeURIComponent(String(state.weekFrom))}&to=${encodeURIComponent(String(state.weekTo))}`);
  }
+  if (ctx.scenario.id.endsWith("spend-breakdown")) {
+    const calls: LeadApiCall[] = []; const saved: SavedRunRecord[] = [];
+    for (const run of (await ctx.allRuns()).filter(row => row.agentId === ctx.input.fixtures.agent.id)) {
+      const events = await collectRunEvents<Record<string, unknown>>(afterSeq => ctx.input.api.get(`/api/heartbeat-runs/${run.id}/events?afterSeq=${afterSeq}&limit=1000`));
+      calls.push(...leadApiCalls(events.map(row => ({ ...row, runId: row.runId ?? run.id }))));
+      saved.push(...await savedRunRecordReads(ctx, run.id, ctx.input.fixtures.agent.id));
+    }
+    state.leadApiCalls = calls; state.savedRunRecords = saved;
+  }
  if (ctx.scenario.id.endsWith("repair-cost-recording") && state.finalReplyRunId) {
    const comments = await ctx.input.api.get<IssueComment[]>(`/api/issues/${ctx.issue()!.id}/comments?order=asc`);
    state.finalReply = comments.filter(row => row.authorAgentId === ctx.input.fixtures.agent.id && row.createdByRunId === state.finalReplyRunId).map(row => row.body ?? "").join("\n");
@@ -276,7 +293,7 @@ export const accountingIdentityCases: readonly ChatCatalogCase[] = [
    }
    await ctx.input.api.patch(`/api/issues/${ctx.issue()!.id}`, { description: `The requested reporting week is ${weekFrom} through ${weekTo} (UTC). Launch contains imported synthetic metered-API receipt tasks ${morganReceiptTask.identifier} (Morgan) and ${rileyReceiptTask.identifier} (Riley), plus actual completed native run ${state.originalRunId} with unresolved saved usage. Imported fixture receipts are not live provider/campaign billing. Other-project and earlier-week charges must be excluded; native subscription usage/fees are separate.` });
    return { ...state, riley, morganReceiptTask, rileyReceiptTask, weekFrom, weekTo, importedReceiptIds: receipts, receiptOrigin: "public-imported-synthetic-fixture" };
-  }, gradeAccounting, { authentication: "human-session", oracleVersion: "paperclip.chat-catalog.spend-breakdown.structural.v3", sourceFiles: sources, expectedRunCount: 2, attemptTimeoutMs: 600_000, observe: observeAccounting, verifyResult: verifyAccountingLink, verifySetup: verifyUnresolvedSetup,
+  }, gradeAccounting, { authentication: "human-session", oracleVersion: "paperclip.chat-catalog.spend-breakdown.structural.v4", sourceFiles: [...sources, "chat-catalog-domains/api-read-receipts.ts", "run-observations.ts"], expectedRunCount: 2, attemptTimeoutMs: 600_000, observe: observeAccounting, verifyResult: verifyAccountingLink, verifySetup: verifyUnresolvedSetup,
    setupPolicy: { funding: "standard", readiness: "all-ready", invariantCheckIds: ["genuine-unresolved-saved-native-usage"] } }),
  identityDefinition("budgets-costs.repair-cost-recording", async ctx => {
    try {
@@ -329,11 +346,15 @@ export const accountingIdentityCases: readonly ChatCatalogCase[] = [

 /** The reply identifies the record whose usage has no settled cost (its task by link or identifier,
 * or its run id), and no line or sentence that names it states a zero amount for it. */
-export function unresolvedRecordIdentifiedNotFree(reply: string, record: Entity | undefined, runId: string): boolean {
+/** The reply points at the unresolved record: a rendered link to its task or run, or its identifier or run id written out. */
+export function unresolvedRecordIdentified(o: { reply: string; links: string[]; origin: string }, record: Entity | undefined, runId: string): boolean {
  const keys = [record?.identifier, record?.id, runId || undefined, runId ? runId.slice(0, 8) : undefined].filter((key): key is string => typeof key === "string" && key.length > 0);
  if (!keys.length) return false;
-  const names = (segment: string) => keys.some(key => new RegExp(`(?<![\\w-])${key.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")}(?![\\w-])`).test(segment));
-  const segments = reply.split(/\n|(?<=[.!?])\s+/).filter(segment => names(segment));
-  // Only a stated $0 amount for the record fails; words such as "free" are not graded.
-  return segments.length > 0 && !segments.some(segment => /(?:\$|USD\s*)0(?:\.0+)?(?![.\d]*[1-9])/i.test(segment));
+  const linked = o.links.some(href => {
+    try {
+      const parts = new URL(href, o.origin).pathname.split("/").filter(Boolean).map(decodeURIComponent);
+      return parts.some((part, index) => index > 0 && (parts[index - 1] === "issues" && [record?.id, record?.identifier].includes(part) || parts[index - 1] === "runs" && part === runId));
+    } catch { return false; }
+  });
+  return linked || keys.some(key => new RegExp(`(?<![\\w-])${key.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")}(?![\\w-])`).test(o.reply));
 }
@@ -4,15 +4,60 @@ import type { Agent } from "../../../packages/shared/src/types/agent.js";
 import { finishNativeNewCompany, type NewCompanyNativeReceipt } from "./identity-new-company-native.js";
 import { NEW_COMPANY_NATIVE_MODEL } from "./identity-new-company-native-proof.js";
 import { checks, type Entity } from "./internal-support.js";
+import { replyOrCardLinks, replyOrCardText, turnResponseCards } from "./turn-response-card.js";
 import { identityDefinition, identityObservation, observeIdentity, readAgentResource, remainingIdentityObservationBudget, waitForReply } from "./identity-support.js";

 const companies = new WeakMap<ChatCatalogContext, string>();
 const connections = new WeakMap<ChatCatalogContext, Array<{ companyId: string; connectionId: string; grantId: string }>>();
+const originals = new WeakMap<ChatCatalogContext, { companyName: string; companyDescription: string | null; personName: string }>();
+/** The person's existing company: an ordinary business unrelated to the website business they ask to set up. */
+export const SOURCE_COMPANY_PROFILE = { name: "Harbor Analytics", description: "Data analytics consulting for regional retailers." };
+export const SOURCE_PERSON_NAME = "Jordan Avery";
+/** Put back the source company's and the person's own names; called on a failed setup and at cleanup. */
+async function restoreOriginalNames(ctx: ChatCatalogContext) {
+  const original = originals.get(ctx); if (!original) return;
+  try {
+    await ctx.input.api.patch(`/api/companies/${ctx.input.fixtures.company.id}`, { name: original.companyName, description: original.companyDescription });
+    await ctx.input.page.context().request.patch(`${ctx.input.api.baseURL}/api/auth/profile`, { data: { name: original.personName }, headers: { Origin: ctx.input.api.baseURL } });
+  } finally { originals.delete(ctx); }
+}
+/** The unprefixed onboarding route, which opens the new-organization wizard. */
+export function isCompanyCreationFlow(href: string, origin: string) {
+  try { const url = new URL(href, origin); return url.origin === new URL(origin).origin && url.pathname.replace(/\/+$/, "") === "/onboarding"; } catch { return false; }
+}
+interface MemberRow { id?: string; principalType?: string; principalId?: string; status?: string; membershipRole?: string; grants?: Array<{ permissionKey?: string; scope?: unknown }> }
+const memberRows = (value: unknown): MemberRow[] | undefined => {
+  const rows = (value as { members?: unknown } | undefined)?.members;
+  return Array.isArray(rows) ? rows as MemberRow[] : undefined;
+};
+/** Who belongs to the company, in which role and status, with which grants; timestamps and profiles are ignored. */
+export function sourceMembershipChanges(before: unknown, after: unknown): string[] {
+  const old = memberRows(before), now = memberRows(after);
+  if (!old || !now) return [!old ? "before-state missing" : "after-state missing"];
+  const grants = (row: MemberRow) => (row.grants ?? []).map(grant => `${grant.permissionKey}:${JSON.stringify(grant.scope ?? null)}`).sort().join(",");
+  const key = (row: MemberRow) => `${row.principalType}:${row.principalId}`;
+  const changes: string[] = [];
+  for (const row of old) {
+    const match = now.find(next => key(next) === key(row));
+    if (!match) { changes.push(`${key(row)} removed`); continue; }
+    for (const field of ["id", "status", "membershipRole"] as const) if (match[field] !== row[field]) changes.push(`${key(row)} ${field} ${row[field]} -> ${match[field]}`);
+    if (grants(match) !== grants(row)) changes.push(`${key(row)} grants ${grants(row)} -> ${grants(match)}`);
+  }
+  for (const row of now) if (!old.some(prior => key(prior) === key(row))) changes.push(`${key(row)} added as ${row.membershipRole}`);
+  return changes;
+}
 export function gradeNewCompany(value: unknown) {
  const c = checks(); const o = identityObservation(value); c.add("complete-human-company-onboarding", !!o, "Actual human wizard, first-account readiness and separate company/agent authority are required."); if (!o) return c.result();
  const wizard = o.state.wizard as NewCompanyNativeReceipt | undefined; const agent = o.state.newAgent as Agent | undefined;
-  c.add("guidance-before-human-creation", o.state.companyCountAfterReply === 1 && /website|business/i.test(o.reply) && /onboarding|creat.*(?:company|organization)|company.*(?:flow|wizard|switcher)|organization.*(?:wizard|switcher)/i.test(o.reply)
-    && !/\bI(?:'ve| have)?\s+(?:created|set up)\s+(?:a |the |your )?(?:new )?(?:company|organization)/i.test(o.reply), "The source agent preserves the website-business request and guides the actual person without claiming it created another company.");
+  // A question card the run ended its turn on is that turn's reply (decision after full cohort 4): its text and the
+  // links it renders count. The guidance is graded structurally: the response links the board-only company-creation
+  // flow (/onboarding, which opens the new-organization wizard; /<prefix>/onboarding only adds an agent to the
+  // current company) and carries the requested business (setup literal from the query). Saved state, not wording,
+  // proves the agent created no company: one company exists when it answered.
+  const cards = turnResponseCards({ companyId: o.companyId, leadId: o.leadId, runs: o.runs, interactions: o.interactions });
+  c.add("guidance-before-human-creation", o.state.companyCountAfterReply === 1 && /website|business/i.test(replyOrCardText(o.reply, cards))
+    && replyOrCardLinks(o.links, cards).some(href => isCompanyCreationFlow(href, o.origin)),
+    "The source agent preserves the website-business request and links the person to the company-creation flow, creating no company itself.");
  c.add("actual-ready-company-and-ceo", !!wizard && wizard.companyId !== o.companyId && agent?.id === wizard.firstAgentId && agent.companyId === wizard.companyId
    && agent.role === "ceo" && agent.lifecycleState === "ready" && wizard.firstAgentReady && wizard.hiredThroughPublicSetup && wizard.actualHarness === "opencode_local"
    && wizard.model === NEW_COMPANY_NATIVE_MODEL && wizard.noAgentBeforeRouteDeparture && wizard.savedSetupAgentId === agent.id
@@ -26,16 +71,37 @@ export function gradeNewCompany(value: unknown) {
  c.add("no-duplicate-or-secret-leak", !!wizard && wizard.beforeCompanyIds.length === 1 && wizard.beforeCompanyIds[0] === o.companyId
    && wizard.afterCompanyIds.length === 2 && new Set(wizard.afterCompanyIds).size === 2 && wizard.afterAgentIds.length === 1
    && wizard.draftContainsCredential === false && wizard.publicAgentContainsCredential === false && wizard.heartbeatRunIds.length === 0
-    && o.state.sourceMembershipsUnchanged === true && o.state.sourceAccountsUnchanged === true, "The human flow creates one company/first agent without duplicating source identities, dispatching nonexistent work, leaking credentials or changing original memberships.");
+    && o.state.sourceAccountsUnchanged === true, "The human flow creates one company/first agent without duplicating source identities, dispatching nonexistent work or leaking credentials.");
+  // Membership identity, role, status and permission grants are compared, not the whole response (which also
+  // carries timestamps and the person's profile); the saved before/after and the listed changes say what moved.
+  const membershipChanges = sourceMembershipChanges(o.state.sourceMembers, o.state.sourceMembersAfter);
+  c.add("source-memberships-unchanged", membershipChanges.length === 0,
+    `The original company's memberships, roles and grants are unchanged by the new company.${membershipChanges.length ? ` Changed: ${membershipChanges.join("; ")}` : ""}`);
  return c.result();
 }
 export const newCompanyIdentityCases: readonly ChatCatalogCase[] = [identityDefinition("onboarding.new-company", async ctx => {
  if (!ctx.input.privateInputs?.credentials.OPENROUTER_API_KEY) throw new Error("Setup blocker: human company onboarding needs declared private OPENROUTER_API_KEY for the actual native1017 setup");
-  const session = await ctx.input.api.get<{ user: { id: string } }>("/api/auth/get-session");
-  return { sourceAccounts: await ctx.input.api.get(`/api/companies/${ctx.input.fixtures.company.id}/ai-connections`), humanId: session.user.id, sourceMembers: await ctx.input.api.get(`/api/companies/${ctx.input.fixtures.company.id}/members`) };
+  const session = await ctx.input.api.get<{ user: { id: string; name: string } }>("/api/auth/get-session");
+  // The shared fixture names read as generated ("Company <hex>", "Operations company", a sign-up named after the
+  // harness), and in full cohort 4 the CEO reasoned from them that the request was a graded test. A person's
+  // existing company and own name look like these instead; both are restored at cleanup.
+  const companyPath = `/api/companies/${ctx.input.fixtures.company.id}`;
+  const before = await ctx.input.api.get<{ name: string; description: string | null; issuePrefix: string }>(companyPath);
+  originals.set(ctx, { companyName: before.name, companyDescription: before.description, personName: session.user.name });
+  try {
+    await ctx.input.api.patch(companyPath, SOURCE_COMPANY_PROFILE);
+    if ((await ctx.input.api.get<{ issuePrefix: string }>(companyPath)).issuePrefix !== before.issuePrefix) throw new Error("Setup blocker: renaming the source company changed its task prefix");
+    const renamed = await ctx.input.page.context().request.patch(`${ctx.input.api.baseURL}/api/auth/profile`, { data: { name: SOURCE_PERSON_NAME }, headers: { Origin: ctx.input.api.baseURL } });
+    if (!renamed.ok()) throw new Error("Setup blocker: the person's profile name could not be set");
+    return { sourceAccounts: await ctx.input.api.get(`${companyPath}/ai-connections`), humanId: session.user.id, sourceMembers: await ctx.input.api.get(`${companyPath}/members`) };
+  } catch (error) { await restoreOriginalNames(ctx); throw error; }
 }, gradeNewCompany, {
  authentication: "human-session", requiredCredentials: ["OPENROUTER_API_KEY"], attemptTimeoutMs: 600_000,
-  sourceFiles: ["chat-catalog-domains/identity-new-company.ts", "chat-catalog-domains/identity-new-company-native.ts", "chat-catalog-domains/identity-new-company-native-proof.ts", "chat-catalog-domains/identity-wizard.ts", "chat-catalog-domains/identity-private-browser.ts", "chat-catalog-domains/identity-private-browser-worker.ts", "chat-catalog-domains/identity-support.ts", "chat-catalog-domains/internal-support.ts", "chat-scenarios/onboarding.yaml"],
+  // v2: the guidance is graded on the creation-flow link and saved company count (a response card counts as the
+  // reply), memberships are compared by identity/role/grants with the after-state saved, and the source company and
+  // person carry realistic names.
+  oracleVersion: "paperclip.chat-catalog.identity.new-company.v2",
+  sourceFiles: ["chat-catalog-domains/identity-new-company.ts", "chat-catalog-domains/turn-response-card.ts", "chat-rendered-links.ts", "chat-catalog-domains/identity-new-company-native.ts", "chat-catalog-domains/identity-new-company-native-proof.ts", "chat-catalog-domains/identity-wizard.ts", "chat-catalog-domains/identity-private-browser.ts", "chat-catalog-domains/identity-private-browser-worker.ts", "chat-catalog-domains/identity-support.ts", "chat-catalog-domains/internal-support.ts", "chat-scenarios/onboarding.yaml"],
  async act(ctx, state) {
    remainingIdentityObservationBudget(ctx);
    const originalDeadlineAt = ctx.observationDeadlineAt!;
@@ -67,12 +133,15 @@ export const newCompanyIdentityCases: readonly ChatCatalogCase[] = [identityDefi
  async observe(ctx, state) {
    state.oldAgentStillSourceCompany = (await ctx.input.api.get<Agent>(`/api/agents/${ctx.input.fixtures.agent.id}`)).companyId === ctx.input.fixtures.company.id;
    state.sourceAccountsUnchanged = JSON.stringify(state.sourceAccounts) === JSON.stringify(await ctx.input.api.get(`/api/companies/${ctx.input.fixtures.company.id}/ai-connections`));
-    state.sourceMembershipsUnchanged = JSON.stringify(state.sourceMembers) === JSON.stringify(await ctx.input.api.get(`/api/companies/${ctx.input.fixtures.company.id}/members`));
+    state.sourceMembersAfter = await ctx.input.api.get(`/api/companies/${ctx.input.fixtures.company.id}/members`);
+    state.sourceMembershipChanges = sourceMembershipChanges(state.sourceMembers, state.sourceMembersAfter);
    return observeIdentity(ctx, state);
  },
  async cleanup(ctx) {
    const id = companies.get(ctx), owned = connections.get(ctx) ?? [];
    try { for (const connection of owned) if (connection.companyId === id) await ctx.input.api.delete(`/api/tool-connections/${connection.connectionId}`, { allowNotFound: true }); }
-    finally { try { if (id) await ctx.input.api.delete(`/api/companies/${id}`, { allowNotFound: true }); } finally { companies.delete(ctx); connections.delete(ctx); } }
+    finally { try { if (id) await ctx.input.api.delete(`/api/companies/${id}`, { allowNotFound: true }); } finally {
+      try { await restoreOriginalNames(ctx); } finally { companies.delete(ctx); connections.delete(ctx); }
+    } }
  },
 })];
@@ -3,12 +3,17 @@ import { gradeNewCompany } from "./identity-new-company.js";
 import { gradeFirstWork } from "./identity-first-work.js";
 import type { IdentityObservation } from "./identity-support.js";
 function base(scenarioId: string): IdentityObservation { return { scenarioId, origin: "http://fixture.invalid", companyId: "acorn", leadId: "ceo", company: { id: "acorn" }, companies: [], agents: [], tasks: [], projects: [], goals: [], interactions: [], documents: [], revisions: [], comments: [], links: [], runs: [], activity: [], reply: "", humanQueries: [], state: {} }; }
+function member() {
+  return { id: "membership", principalType: "user", principalId: "alex", status: "active", membershipRole: "owner", updatedAt: "2026-11-12T04:21:12.151Z",
+    user: { id: "alex", name: "Jordan Avery" }, grants: [{ id: "g1", permissionKey: "agents:create", scope: null, updatedAt: "2026-11-12T04:21:12.172Z" }] };
+}
 function company() {
-  const o = base("onboarding.new-company"); o.reply = "Use the company switcher to create an organization for your website business; finish the human onboarding wizard.";
+  const o = base("onboarding.new-company"); o.reply = "Creating a company is something you do yourself: open [Create organization](/onboarding) and set up your website business there.";
+  o.links = ["/onboarding"];
  o.state = { humanId: "alex", companyCountAfterReply: 1, wizard: { companyId: "new-company", firstAgentId: "new-ceo", firstAgentReady: true, hiredThroughPublicSetup: true, actualHarness: "opencode_local", model: "openrouter/deepseek/deepseek-v4.1-flash-1017", noAgentBeforeRouteDeparture: true, savedSetupAgentId: "new-ceo", helloCount: 1, hireCount: 1, exactPublicGrantAndAgentInstall: true, hello: { checks: [{ code: "opencode_hello_probe_passed" }] },
    account: { ownership: "personal", ownerUserId: "alex", responsibleUserId: "alex" }, beforeCompanyIds: ["acorn"], afterCompanyIds: ["acorn", "new-company"], afterAgentIds: ["new-ceo"], draftContainsCredential: false, publicAgentContainsCredential: false, heartbeatRunIds: [] },
    newAgent: { id: "new-ceo", companyId: "new-company", role: "ceo", lifecycleState: "ready" }, humanRoleConfigurationStatus: 200, newCompanyOwnedByHuman: true,
-    oldAgentForeignReadStatus: 403, newAgentVisible: true, oldAgentStillSourceCompany: true, sourceMembershipsUnchanged: true, sourceAccountsUnchanged: true }; return o;
+    oldAgentForeignReadStatus: 403, newAgentVisible: true, oldAgentStillSourceCompany: true, sourceAccountsUnchanged: true, sourceMembers: { members: [member()] }, sourceMembersAfter: { members: [member()] } }; return o;
 }
 function website() {
  const o = base("onboarding.first-work"); const work = { id: "website", companyId: "acorn", title: "Acorn Notes website", description: "Website introducing Acorn Notes for science teams", status: "done", assigneeAgentId: "morgan" };
@@ -23,6 +28,37 @@ describe("human company creation and genuine first completed website", () => {
  it("validates an actual human-created separate company with a real CEO", () => { expect(gradeNewCompany(company()).passed).toBe(true); expect(gradeNewCompany(undefined).passed).toBe(false); });
  it("rejects merely renaming the neutral general agent CEO", () => { const o = company(); (o.state.newAgent as { role: string }).role = "general"; expect(gradeNewCompany(o).passed).toBe(false); });
  it("rejects cross-company agent access or human impersonation", () => { const o = company(); o.state.oldAgentForeignReadStatus = 200; expect(gradeNewCompany(o).passed).toBe(false); const wrong = company(); wrong.state.humanRoleConfigurationStatus = 403; expect(gradeNewCompany(wrong).passed).toBe(false); });
+  // Full cohort 4: the CEO answered with a question card asking for a business name, and the membership check compared
+  // the whole members response as JSON without saving the after-state.
+  it("accepts guidance as the reply or as a response card that links the creation flow", () => {
+    const reworded = company(); reworded.reply = "Only you can add an organization. Start at [the setup wizard](/onboarding) and name it after your website business."; expect(gradeNewCompany(reworded).passed).toBe(true);
+    const card = company(); card.reply = ""; card.links = [];
+    card.runs = [{ id: "run", companyId: "acorn", agentId: "ceo", status: "succeeded", resultJson: { presentationDecision: { reasonCodes: ["interaction_owns_conversation_response"] } } } as typeof card.runs[number]];
+    // Shape of cohort 4's retained card, with a description that links the flow.
+    card.interactions = [{ id: "q", kind: "ask_user_questions", status: "pending", companyId: "acorn", issueId: "chat", createdByAgentId: "ceo", createdByUserId: null, sourceRunId: "run",
+      resolverPolicy: "anyone", requestedResolverPolicy: "anyone", effectiveResolverPolicy: "anyone", continuationPolicy: "wake_assignee", dismissedAt: null,
+      payload: { version: 1, title: "Website business setup", supersedeOnUserComment: false,
+        questions: [{ id: "company_name", prompt: "What should your website business be called?", selectionMode: "single", required: true, options: [{ id: "paperclip_text_answer", label: "Type an answer", freeText: true }] }],
+        questionSet: { schema: "paperclip.question_set.v1", description: "Creating a company is a board step: open [Create organization](/onboarding).",
+          questions: [{ id: "company_name", prompt: "What should your website business be called?", answerMode: "text", required: true }] } } } as unknown as typeof card.interactions[number]];
+    expect(gradeNewCompany(card).passed, JSON.stringify(gradeNewCompany(card).checks)).toBe(true);
+  });
+  it("rejects guidance without the creation-flow link, the current company's onboarding, or a company created before the person", () => {
+    const noLink = company(); noLink.links = []; noLink.reply = "Use the company switcher to create an organization for your website business."; expect(gradeNewCompany(noLink).passed).toBe(false);
+    const prefixed = company(); prefixed.links = ["/ACO/onboarding"]; expect(gradeNewCompany(prefixed).passed).toBe(false);
+    const created = company(); created.state.companyCountAfterReply = 2; expect(gradeNewCompany(created).passed).toBe(false);
+    const nameOnly = company(); nameOnly.reply = "What should the new company be called?"; nameOnly.links = []; expect(gradeNewCompany(nameOnly).passed).toBe(false);
+  });
+  it("compares memberships by identity, role, status and grants, and names what changed", () => {
+    const touched = company(); const after = (touched.state.sourceMembersAfter as { members: Array<Record<string, unknown>> }).members[0]!;
+    after.updatedAt = "2026-11-12T05:00:00.000Z"; (after.user as Record<string, unknown>).name = "MCP evaluation person"; expect(gradeNewCompany(touched).passed).toBe(true);
+    const demoted = company(); (demoted.state.sourceMembersAfter as { members: Array<Record<string, unknown>> }).members[0]!.membershipRole = "member";
+    const result = gradeNewCompany(demoted); expect(result.passed).toBe(false);
+    expect(result.checks.find(check => check.id === "source-memberships-unchanged")?.detail).toContain("membershipRole owner -> member");
+    const granted = company(); ((granted.state.sourceMembersAfter as { members: Array<{ grants: unknown[] }> }).members[0]!.grants).push({ permissionKey: "agents:configure", scope: null });
+    expect(gradeNewCompany(granted).passed).toBe(false);
+    const missing = company(); delete missing.state.sourceMembersAfter; expect(gradeNewCompany(missing).passed).toBe(false);
+  });
  it("validates the actual run-bound delivered and rendered website", () => { expect(gradeFirstWork(website()).passed).toBe(true); expect(gradeFirstWork(undefined).passed).toBe(false); });
  it("rejects empty task creation even with a finished website reply", () => { const o = website(); o.reply = "The website is complete."; o.state.websites = []; expect(gradeFirstWork(o).passed).toBe(false); });
  it("rejects artifact createdByRunId fiction instead of immutable originatingRunId", () => { const o = website(); const file = (o.state.websites as Array<{ attachment: Record<string, unknown> }>)[0]!; file.attachment.createdByRunId = "worker-run"; delete file.attachment.originatingRunId; expect(gradeFirstWork(o).passed).toBe(false); });
@@ -6,6 +6,8 @@ import { checks, project, entity, hasLink, putDocument, type Entity } from "./in
 import { identityDefinition, identityObservation, observeIdentity, openStandingChat, readAgentResource, waitForReply, remainingIdentityObservationBudget, type Interaction } from "./identity-support.js";
 import type { ChatCatalogContext } from "../chat-catalog-contract.js";
 import { createAuthenticatedPerson, enrollPrerequisitePerson } from "./identity-auth.js";
+/** The person's answer to the necessary product-name question: an invented, distinctive brand name. */
+export const NECESSARY_QUESTION_PRODUCT_NAME = "Quillmere";
 /** The setup-chosen person the wrong-reviewer card is addressed to. */
 export const WRONG_REVIEWER_ADDRESSEE = "Riley";
 const audiencePeople = new WeakMap<ChatCatalogContext, Awaited<ReturnType<typeof createAuthenticatedPerson>>>();
@@ -39,18 +41,24 @@ async function questionFixture(ctx: ChatCatalogContext, multipart: boolean) {
  await ctx.input.page.reload({ waitUntil: "domcontentloaded" });
  return { original, answer: `Acorn ${ctx.input.nonce}`, sourceId: source.id };
 }
+/** The form's last-page button: the card's own submitLabel (QuestionForm shows it, e.g. "Continue"), else the default. */
+export function submitAnswersButtonName(card: Pick<Interaction, "payload"> | undefined): string | RegExp {
+  const label = card?.payload.questionSet?.submitLabel?.trim();
+  return label ? label : /^(Submit|Send) answers$/;
+}
 async function answerText(ctx: ChatCatalogContext, answer: string) {
  const form = ctx.input.page.getByTestId("task-chat-composer-takeover");
  await expect(form).toBeVisible();
+  const pending = (await cards(ctx)).find(row => row.kind === "ask_user_questions" && row.status === "pending");
  const text = form.getByTestId("question-text-answer-composer");
  if (await text.isVisible()) await text.locator('[contenteditable="true"],textarea').first().fill(answer);
  else {
-    const pending = (await cards(ctx)).find(row => row.kind === "ask_user_questions" && row.status === "pending");
    const label = pending?.payload.questionSet?.questions[0]?.customAnswer?.label ?? "Other";
    await form.getByRole("radio", { name: label, exact: true }).click();
    await form.getByTestId("question-other-answer-composer").locator('[contenteditable="true"],textarea').first().fill(answer);
  }
-  await form.getByRole("button", { name: /^(Submit|Send) answers$/ }).click();
+  const submit = submitAnswersButtonName(pending);
+  await form.getByRole("button", typeof submit === "string" ? { name: submit, exact: true } : { name: submit }).click();
 }
 /** The one required question of the agent's card, when every other question on it is optional. */
 export function soleRequiredQuestion(card: Pick<Interaction, "payload"> | undefined) {
@@ -84,7 +92,8 @@ async function answerRequiredQuestion(ctx: ChatCatalogContext, answer: string) {
      await form.getByRole("radio", { name: question.customAnswer?.label ?? "Other", exact: true }).click();
      await form.getByTestId("question-other-answer-composer").locator('[contenteditable="true"],textarea').first().fill(answer);
    }
-    await form.getByRole("button", last ? { name: /^(Submit|Send) answers$/ } : { name: "Next", exact: true }).click();
+    const submit = submitAnswersButtonName(card);
+    await form.getByRole("button", !last ? { name: "Next", exact: true } : typeof submit === "string" ? { name: submit, exact: true } : { name: submit }).click();
  }
 }
 async function beforeAnswer(ctx: ChatCatalogContext, state: Record<string, unknown>) {
@@ -115,8 +124,44 @@ async function observeQuestion(ctx: ChatCatalogContext, state: Record<string, un
  }
  const brief = observed.tasks.find(row => `${row.title}\n${row.description}`.includes(String(state.answer)));
  state.taskBrief = brief ? await ctx.input.api.get(`/api/issues/${brief.id}`) : undefined;
+  if (ctx.scenario.id === "questions-and-approvals.necessary-question") state.answerDeliverables = await answerDeliverables(ctx, observed.tasks, String(state.answer));
  return observed;
 }
+/** A saved document revision or attachment on a task whose content carries the person's exact answer, with its
+ * saving agent, run and time. The answer can shape the resulting work there as well as in the task's brief. */
+export interface AnswerDeliverable { kind: "document" | "attachment"; id: string; issueId: string; companyId?: string; savedAt: string; agentId: string | null; runId: string | null; current: boolean }
+const TEXT_ATTACHMENT = /^(?:text\/|application\/(?:json|xml|xhtml\+xml|javascript)\b)/;
+async function answerDeliverables(ctx: ChatCatalogContext, tasks: Entity[], answer: string): Promise<AnswerDeliverable[]> {
+  const found: AnswerDeliverable[] = [];
+  if (!answer) return found;
+  for (const task of tasks.filter(row => row.companyId === ctx.input.fixtures.company.id).slice(0, 25)) {
+    for (const summary of await ctx.input.api.get<Array<{ key: string; body?: string }>>(`/api/issues/${task.id}/documents`)) {
+      const endpoint = `/api/issues/${task.id}/documents/${encodeURIComponent(summary.key)}`;
+      const [document, revisions] = await Promise.all([ctx.input.api.get<{ id: string; body: string; latestRevisionId?: string }>(endpoint),
+        ctx.input.api.get<Array<{ id: string; body: string; createdAt: string; createdByAgentId?: string | null; createdByRunId?: string | null }>>(`${endpoint}/revisions`)]);
+      // The earliest revision that carries the answer: when the answer first shaped this document.
+      const revision = [...revisions].sort((a, b) => Date.parse(a.createdAt) - Date.parse(b.createdAt)).find(row => row.body?.includes(answer));
+      if (revision) found.push({ kind: "document", id: revision.id, issueId: task.id, companyId: task.companyId, savedAt: revision.createdAt,
+        agentId: revision.createdByAgentId ?? null, runId: revision.createdByRunId ?? null, current: document.body.includes(answer) });
+    }
+    for (const attachment of await ctx.input.api.get<Array<{ id: string; companyId: string; contentType: string; byteSize: number; contentPath: string; createdAt: string; createdByAgentId: string | null; originatingRunId?: string | null }>>(`/api/issues/${task.id}/attachments`)) {
+      if (!TEXT_ATTACHMENT.test(attachment.contentType) || attachment.byteSize > 2_097_152) continue;
+      const body = await ctx.input.api.bytes(attachment.contentPath).catch(() => undefined);
+      if (body?.toString("utf8").includes(answer)) found.push({ kind: "attachment", id: attachment.id, issueId: task.id, companyId: attachment.companyId, savedAt: attachment.createdAt,
+        agentId: attachment.createdByAgentId, runId: attachment.originatingRunId ?? null, current: true });
+    }
+  }
+  return found;
+}
+/** The person's answer shapes saved work: a linked task's deliverable that an agent of this company saved, with the
+ * exact answer, only after the person answered (and still carries it). */
+export function answerShapedDeliverable(o: NonNullable<ReturnType<typeof identityObservation>>, resolvedAt: string | null | undefined) {
+  const deliverables = Array.isArray(o.state.answerDeliverables) ? o.state.answerDeliverables as AnswerDeliverable[] : [];
+  return typeof resolvedAt === "string" && deliverables.some(row => row.current && row.companyId === o.companyId && Date.parse(row.savedAt) >= Date.parse(resolvedAt)
+    && !!row.agentId && (row.agentId === o.leadId || o.agents.some(agent => agent.id === row.agentId))
+    && (!row.runId || o.runs.some(run => run.id === row.runId && run.agentId === row.agentId && run.companyId === o.companyId))
+    && o.tasks.some(task => task.id === row.issueId && task.companyId === o.companyId && hasLink(o, "issues", task)));
+}
 export function multipartVisibleSummaries(value: unknown) {
  const o = identityObservation(value);
  if (!o) return [];
@@ -225,8 +270,11 @@ export function gradeQuestions(value: unknown) {
        && o.runs.some(run => run.id === original.sourceRunId && run.agentId === o.leadId && run.companyId === o.companyId && run.status === "succeeded") && !!necessary
        && (necessary.answerMode === "text" || necessary.customAnswer?.enabled === true),
        "The agent itself requires exactly one free-text answer, tied to its actual completed run; any other question on the card is optional. Whether that answer is the product name is proved by the saved brief carrying the person's answer.");
-      c.add("answer-in-saved-brief", o.tasks.some(row => `${row.title}\n${row.description}`.includes(String(o.state.answer)) && row.companyId === o.companyId && hasLink(o, "issues", row)),
-        "A saved, linked task brief uses the actual answer rather than merely acknowledging it in chat.");
+      // The contract is that the answer shapes the resulting work: a saved brief or a saved deliverable (document or
+      // attachment content) on the linked task carries the exact answer. Chat acknowledgement alone never counts.
+      const deliverable = answerShapedDeliverable(o, current?.resolvedAt);
+      c.add("answer-in-saved-brief", deliverable || o.tasks.some(row => `${row.title}\n${row.description}`.includes(String(o.state.answer)) && row.companyId === o.companyId && hasLink(o, "issues", row)),
+        "A saved, linked task brief or deliverable uses the actual answer rather than merely acknowledging it in chat.");
      c.add("one-bound-continuation", o.runs.filter(run => run.contextSnapshot?.interactionId === original?.id).length === 1
        || !!original?.payload.runtimeRequestId && original.idempotencyKey === `paperclip-runner-question:${original.sourceRunId}:${original.payload.runtimeRequestId}`
          && o.runs.filter(run => run.id === original.sourceRunId && run.status === "succeeded").length === 1,
@@ -235,10 +283,10 @@ export function gradeQuestions(value: unknown) {
      const beforeBriefs = o.state.beforeAnswerTasks as Array<{ id: string; title: string; description?: string }> | undefined;
      const beforeBrief = Array.isArray(beforeBriefs) ? beforeBriefs.find(row => row.id === brief?.id) : undefined;
      const savedAt = beforeBrief ? brief?.updatedAt : brief?.createdAt;
-      c.add("actual-answer-brief-provenance", !!brief?.id && brief.companyId === o.companyId && Array.isArray(beforeBriefs) && typeof current?.resolvedAt === "string" && typeof savedAt === "string"
+      c.add("actual-answer-brief-provenance", deliverable || !!brief?.id && brief.companyId === o.companyId && Array.isArray(beforeBriefs) && typeof current?.resolvedAt === "string" && typeof savedAt === "string"
        && Date.parse(savedAt) >= Date.parse(current.resolvedAt) && (!beforeBrief || !`${beforeBrief.title}\n${beforeBrief.description}`.includes(String(o.state.answer)))
        && `${brief.title}\n${brief.description}`.includes(String(o.state.answer)) && o.activity.some(row => row.entityId === brief.id && row.action === (beforeBrief ? "issue.updated" : "issue.created") && row.agentId === o.leadId),
-        "Before/after public brief snapshots and mutation attribution prove the actual agent saved the real answer only after the human resolved this question, including an eligible pre-existing draft.");
+        "Before/after public brief snapshots and mutation attribution, or the saving agent, run and time of the deliverable, prove the actual agent saved the real answer only after the human resolved this question.");
    }
  }
  return c.result();
@@ -282,9 +330,11 @@ export const questionIdentityCases = [
    const p = await project(ctx); const source = await openStandingChat(ctx);
    const existingBrief = "The requested single-page site presents a collaborative research workspace for independent researchers and small research teams. Visitors should join the early-access waitlist through an accessible signup form. Use a clear, welcoming visual style and a responsive layout for mobile and desktop.";
    await ctx.input.api.patch(`/api/issues/${source.id}`, { description: existingBrief });
-    return { project: p, sourceId: source.id, existingBrief, answer: `Acorn ${ctx.input.nonce}` };
+    // A realistic, distinctive product name a person would type and an agent would keep verbatim. The earlier
+    // `Acorn <nonce>` read like a placeholder: the agent saved "Acorn" and dropped the suffix as noise.
+    return { project: p, sourceId: source.id, existingBrief, answer: NECESSARY_QUESTION_PRODUCT_NAME };
  }, gradeQuestions, {
-    oracleVersion: "paperclip.chat-catalog.identity.necessary-question.v2", expectedRunCount: 4, minimumRunCount: 1, attemptTimeoutMs: 540_000,
+    oracleVersion: "paperclip.chat-catalog.identity.necessary-question.v3", expectedRunCount: 4, minimumRunCount: 1, attemptTimeoutMs: 540_000,
    observe: observeQuestion,
    async verifySetup(ctx, state) {
      const visible = await readAgentResource(ctx, `/api/issues/${state.sourceId}`);
@@ -292,7 +342,7 @@ export const questionIdentityCases = [
      const c = checks();
      c.add("accessible-existing-site-brief", visible.status === 200 && !!source && source.id === state.sourceId && source.companyId === ctx.input.fixtures.company.id
        && source.description === state.existingBrief, "The actual speaking agent can read the ordinary source brief with purpose, audience and signup intent before its question.");
-      c.add("product-name-not-supplied", typeof source?.description === "string" && !source.description.includes("Acorn") && !source.description.includes(ctx.input.nonce)
+      c.add("product-name-not-supplied", typeof source?.description === "string" && !source.description.includes(String(state.answer))
        && !(await ctx.input.api.get<Array<{ title: string }>>(`/api/companies/${ctx.input.fixtures.company.id}/issues`)).some(row => row.title.includes(String(state.answer))),
        "Neither the source brief nor a pre-created result task supplies the later human product name.");
      return c.result();
@@ -1,7 +1,7 @@
 import { describe, expect, it } from "vitest";
 import { companyIdentityCases, gradeCompanyRename } from "./identity-companies.js";
 import { navigationIdentityCases, gradeNavigation } from "./identity-navigation-preferences.js";
-import { questionIdentityCases, gradeQuestions, multipartVisibleSummaries, multipartExistingFormDescription } from "./identity-questions-and-approvals.js";
+import { questionIdentityCases, gradeQuestions, multipartVisibleSummaries, multipartExistingFormDescription, submitAnswersButtonName, type AnswerDeliverable } from "./identity-questions-and-approvals.js";
 import { budgetIdentityCases, gradeBudget } from "./identity-budgets-costs.js";
 import { accessIdentityCases } from "./identity-access.js";
 import { toolReviewIdentityCases } from "./identity-tool-review.js";
@@ -192,6 +192,38 @@ describe("identity catalog independent oracle calibration", () => {
    const ignored = valid("questions-and-approvals.necessary-question"); (ignored.state.original as Interaction).payload.questionSet!.questions = [name, optional];
    ignored.tasks[0]!.description = "An invented product name"; ignored.tasks[0]!.title = "Landing page"; expect(gradeQuestions(ignored).passed).toBe(false);
  });
+  // Decision after full cohort 4: the case contract is that the answer shapes the resulting work, so the exact answer
+  // in a saved deliverable on the linked task counts as well as in its brief. Chat acknowledgement alone never does.
+  it("accepts the exact answer in a saved deliverable on the linked task, and rejects unproven or chat-only use", () => {
+    const deliverableOnly = (patch: Partial<AnswerDeliverable> = {}) => {
+      const o = valid("questions-and-approvals.necessary-question");
+      o.tasks[0]!.title = "Build the landing page"; o.tasks[0]!.description = "Single-page waitlist site for the research workspace.";
+      o.state.taskBrief = undefined;
+      o.state.answerDeliverables = [{ kind: "document", id: "revision", issueId: "task", companyId: "company", savedAt: "2026-10-26T12:05:00.000Z", agentId: "ceo", runId: "continue", current: true, ...patch }];
+      return o;
+    };
+    expect(gradeQuestions(deliverableOnly()).passed, JSON.stringify(gradeQuestions(deliverableOnly()).checks)).toBe(true);
+    // Reworded valid: an attachment saved by the worker the task went to.
+    const attachment = deliverableOnly({ kind: "attachment", agentId: "morgan", runId: "build" });
+    attachment.agents = [{ id: "morgan" }]; attachment.runs.push({ id: "build", companyId: "company", agentId: "morgan", status: "succeeded" });
+    expect(gradeQuestions(attachment).passed).toBe(true);
+    // Broken: nothing saved carries the answer, it was saved before the person answered, by a person, by a foreign run,
+    // later removed, or on a task the reply never links.
+    const chatOnly = deliverableOnly(); chatOnly.state.answerDeliverables = []; expect(gradeQuestions(chatOnly).passed).toBe(false);
+    expect(gradeQuestions(deliverableOnly({ savedAt: "2026-10-26T11:00:00.000Z" })).passed).toBe(false);
+    expect(gradeQuestions(deliverableOnly({ agentId: null })).passed).toBe(false);
+    expect(gradeQuestions(deliverableOnly({ runId: "someone-elses-run" })).passed).toBe(false);
+    expect(gradeQuestions(deliverableOnly({ current: false })).passed).toBe(false);
+    const unlinked = deliverableOnly(); unlinked.links = ["/ACORN/projects/project"]; expect(gradeQuestions(unlinked).passed).toBe(false);
+  });
+  // Paid run 43162122291: the agent's card set submitLabel "Continue", which QuestionForm shows instead of "Submit answers".
+  it("answers through the card's own submit label", () => {
+    const card = (submitLabel?: string) => ({ payload: { questionSet: { questions: [], ...(submitLabel !== undefined ? { submitLabel } : {}) } } });
+    expect(submitAnswersButtonName(card("Continue"))).toBe("Continue");
+    expect(submitAnswersButtonName(card())).toEqual(/^(Submit|Send) answers$/);
+    expect(submitAnswersButtonName(card("  "))).toEqual(/^(Submit|Send) answers$/);
+    expect(submitAnswersButtonName(undefined)).toEqual(/^(Submit|Send) answers$/);
+  });
  it("approved read is graded on the provider nonce and one provider receipt, not the provider sentence", () => {
    const grade = (o: IdentityObservation) => toolReviewIdentityCases[0]!.grade(o).passed;
    // Retained u2 reply (run 42641280837) dropped the "Launch notes: " prefix but carried the actual result.
@@ -8,6 +8,7 @@ import { pollUntil } from "../api.js";
 import { checks, observation, project, type Entity, type InternalObservation } from "./internal-support.js";
 import { fixtureWorker } from "./internal-definition.js";
 import { integratorDefinition, inspectResult, observeInternal } from "./integrator-support.js";
+import { waitForTurnReplyLinks } from "./identity-support.js";
 type CaseRecord = typeof cases.$inferSelect;
 type CaseEvent = typeof caseEvents.$inferSelect;
 interface HistoryObservation extends InternalObservation { caseRecord: CaseRecord; caseEvents: CaseEvent[]; handoffs: CaseHandoff[]; diagnosis: IssueDocument; diagnosisRevisions: DocumentRevision[] }
@@ -78,11 +79,14 @@ export const integratorCaseHistoryCases = [integratorDefinition("cases", "cases.
  const row = state.case as CaseRecord; const task = state.task as Issue;
  await inspectResult(ctx, `/${ctx.input.fixtures.company.issuePrefix}/cases/${row.identifier}`, row.title);
  await inspectResult(ctx, `/${ctx.input.fixtures.company.issuePrefix}/issues/${task.identifier}`, task.title);
-}, { expectedRunCount: 5, expectedHumanMessageCount: 2, attemptTimeoutMs: 1_200_000, cleanup, oracleVersion: "paperclip.chat-catalog.integrator.case-history.v3",
-  sourceFiles: ["chat-catalog-domains/integrator-case-history.ts", "chat-catalog-domains/integrator-support.ts", "chat-catalog-domains/internal-support.ts", "chat-catalog-domains/internal-definition.ts", "chat-scenarios/cases.yaml"],
+}, { expectedRunCount: 5, expectedHumanMessageCount: 2, attemptTimeoutMs: 1_200_000, cleanup, oracleVersion: "paperclip.chat-catalog.integrator.case-history.v4",
+  sourceFiles: ["chat-catalog-domains/integrator-case-history.ts", "chat-catalog-domains/integrator-support.ts", "chat-catalog-domains/internal-support.ts", "chat-catalog-domains/internal-definition.ts", "chat-catalog-domains/identity-support.ts", "chat-rendered-links.ts", "chat-scenarios/cases.yaml"],
  async act(ctx, state) {
-    const first = await pollUntil({ label: "canonical diagnosis history answer before distinct missing-history question", deadlineAt: chatCatalogNativePhaseDeadline(ctx, Date.now() + 180_000), intervalMs: 1000,
+    await pollUntil({ label: "canonical diagnosis history answer before distinct missing-history question", deadlineAt: chatCatalogNativePhaseDeadline(ctx, Date.now() + 180_000), intervalMs: 1000,
      load: () => observeHistory(ctx, state), accept: o => chatCatalogNativePhaseIsTimely(ctx) && o.runs.length === 4 && o.runs.every(run => run.status === "succeeded") && ctx.issue()?.conversationState === "waiting" });
+    // Finished runs and a waiting chat precede the saved reply's render; capture the phase only after it renders.
+    await waitForTurnReplyLinks(ctx);
+    const first = await observeHistory(ctx, state);
    state.firstHistory = { ...first, state: structuredClone(state) }; await ctx.input.evidence("chat-case-diagnosis-history-first.json", state.firstHistory);
    const row = first.caseRecord; const beforeIds = new Set(first.caseEvents.map(event => event.id));
    resources.get(ctx)!.unavailableText = `unretained-support-${ctx.input.nonce}`;
@@ -18,6 +18,9 @@ describe("routine schedule pause versus actual in-flight work", () => {
    expect(gradeIntegratorRoutinePause(paraphrase).passed).toBe(true);
    const finishing = valid(); finishing.state.pauseCheckpoint.reply = "Paused the routine. COM-2 had already started and is still finishing under Morgan.";
    expect(gradeIntegratorRoutinePause(finishing).passed).toBe(true);
+    // Full cohort 4: a correct reply offering how to stop the run is not a cancellation claim.
+    const offer = valid(); offer.state.pauseCheckpoint.reply = "Paused Daily launch summaries. COM-2 is still running under Morgan. If you want that one stopped too, use Pause on its task page.";
+    expect(gradeIntegratorRoutinePause(offer).passed).toBe(true);
    const empty = valid(); empty.state.pauseCheckpoint.reply = " ";
    expect(gradeIntegratorRoutinePause(empty).passed).toBe(false);
  });
@@ -30,10 +33,12 @@ describe("routine schedule pause versus actual in-flight work", () => {
    const fake = valid(); fake.state.pauseCheckpoint.runs[0]!.status = "succeeded";
    expect(gradeIntegratorRoutinePause(fake).passed).toBe(false);
  });
-  it("rejects false cancellation and actual cancellation instead of normal completion", () => {
-    const falseReply = valid(); falseReply.state.pauseCheckpoint.reply = "Paused the schedule and cancelled the running task.";
-    expect(gradeIntegratorRoutinePause(falseReply).passed).toBe(false);
+  it("rejects actual cancellation instead of normal completion", () => {
    const cancelled = valid(); cancelled.runs[0]!.status = "cancelled";
    expect(gradeIntegratorRoutinePause(cancelled).passed).toBe(false);
+    const cancelledTask = valid(); cancelledTask.tasks[1]!.status = "cancelled";
+    expect(gradeIntegratorRoutinePause(cancelledTask).passed).toBe(false);
+    const stoppedAtResponse = valid(); stoppedAtResponse.state.pauseCheckpoint.runs[0]!.status = "cancelled";
+    expect(gradeIntegratorRoutinePause(stoppedAtResponse).passed).toBe(false);
  });
 });
@@ -30,11 +30,11 @@ export function gradeIntegratorRoutinePause(value: unknown) {
  const checkpointRun = first?.runs.find(run => run.id === o.state.activeNativeRunId);
  c.add("pause-does-not-cancel-inflight", !!first && checkpointRun?.status === "running" && checkpointRun.agentId === (o.state.worker as Entity)?.id
    && first.tasks.some(task => task.id === active?.linkedIssueId && task.status === "in_progress")
-    // The pause and the still-running task are proven by saved state at the response; the reply must
-    // exist and must not claim the in-flight work was cancelled. Its wording is left to a human reader.
-    && !!first.reply.trim()
-    && !/\b(?:cancelled|canceled|terminated|stopped)\b[^.!?\n]{0,35}\b(?:running|in.flight|task|execution)\b|\b(?:task|execution|work)\b[^.!?\n]{0,25}\b(?:cancelled|canceled|terminated)\b/i.test(first.reply),
-  "At the actual response the separately assigned execution is still genuinely running and the reply does not claim it was cancelled.");
+    // The pause and the still-running task are proven by saved state at the response, and the later check
+    // proves the run finished normally, so a reply that cancelled it fails there. The reply must exist; its
+    // wording is not keyword-graded (a correct reply can offer "use Pause on its task page if you want it stopped").
+    && !!first.reply.trim(),
+  "At the actual response the separately assigned execution is still genuinely running and the turn saved a reply.");
  const savedPrior = o.invocations.find(run => run.id === prior?.id); const savedActive = o.invocations.find(run => run.id === active?.id);
  c.add("history-retained-and-inflight-completes-normally", o.invocations.length === 2 && savedPrior?.status === "completed" && savedPrior.linkedIssueId === prior.linkedIssueId
    && savedActive?.status === "completed" && savedActive.linkedIssueId === active.linkedIssueId
@@ -67,7 +67,7 @@ export const integratorRoutinePauseCases = [integratorDefinition("routines", "ro
 }, observePause, gradeIntegratorRoutinePause, async (ctx, state) => {
  const routine = state.routine as RoutineDetail; await inspectResult(ctx, `/${ctx.input.fixtures.company.issuePrefix}/routines/${routine.id}/history`, routine.title);
 }, { sourceFiles: ["chat-catalog-domains/integrator-routine-pause.ts", "chat-catalog-domains/internal-run-brief.ts", "chat-catalog-domains/internal-definition.ts", "chat-catalog-domains/internal-support.ts", "chat-catalog-domains/integrator-support.ts", "chat-scenarios/routines.yaml"],
-  oracleVersion: "paperclip.chat-catalog.routine-pause.structural.v2", expectedRunCount: 3, attemptTimeoutMs: 720_000,
+  oracleVersion: "paperclip.chat-catalog.routine-pause.structural.v3", expectedRunCount: 3, attemptTimeoutMs: 720_000,
  setupPolicy: { readiness: "preserve-scenario", invariantCheckIds: ["lead-ready", "owned-routine-real-active-task"] },
  async verifySetup(ctx, state) {
    const c = checks(); const lead = await ctx.input.api.get<RuntimeAgent>(`/api/agents/${ctx.input.fixtures.agent.id}`);
@@ -90,6 +90,19 @@ describe("saved card state and truthful generation controls", () => {
    const otherOwner = structuredClone(handed); (otherOwner.tasks[1] as { assigneeAgentId?: string }).assigneeAgentId = "riley";
    expect(gradeIntegratorCards(otherOwner).passed).toBe(false);
  });
+  // Decision after full cohort 4: a question card the turn ended on is its reply. The card's text and the links its
+  // description renders count; a card with no route to the card, its author or the author's task still fails.
+  it("accepts a routing question card as the reply when it links the card's author or the card", () => {
+    const o = valid("status-cards.other-authors-card");
+    const card = (description: string) => ({ id: "routing", kind: "ask_user_questions" as const, runId: "run", text: "Morgan authored this card\nCreate a task for Morgan, the card's author?", links: [...description.matchAll(/\]\(([^)]+)\)/g)].map(match => match[1]!) });
+    const viaCard = { ...structuredClone(o), reply: "", links: [], responseCards: [card("Only [Morgan](/ACORN/agents/morgan) can change [this card](/ACORN/status/card).")] };
+    expect(gradeIntegratorCards(viaCard).passed, JSON.stringify(gradeIntegratorCards(viaCard).checks)).toBe(true);
+    const authorOnly = { ...viaCard, responseCards: [card("Ask [Morgan](/ACORN/agents/morgan), who authored it.")] };
+    expect(gradeIntegratorCards(authorOnly).passed).toBe(true);
+    // Broken: cohort 4's card named Morgan in plain option text with no link, or there is no reply or card at all.
+    expect(gradeIntegratorCards({ ...viaCard, responseCards: [card("Morgan authored this card.")] }).passed).toBe(false);
+    expect(gradeIntegratorCards({ ...viaCard, responseCards: [] }).passed).toBe(false);
+  });
  it("rejects author evasion, stealth cloning, and unrouted replies", () => {
    const o = valid("status-cards.other-authors-card");
    expect(gradeIntegratorCards({ ...o, cards: [...o.cards, { ...o.cards[0]!, id: "copy", createdByAgentId: "ceo" }] }).passed).toBe(false);
@@ -11,7 +11,8 @@ import { fixtureWorker } from "./internal-definition.js";
 import { collectRunEvents } from "../run-observations.js";
 import { apiReadDigests, apiReadStatuses, recordParams } from "./api-read-receipts.js";
 import { integratorDefinition, inspectResult, resultHref, observeInternal } from "./integrator-support.js";
-interface CardsObservation extends InternalObservation { cards: StatusCard[]; updates: Record<string, StatusCardUpdate[]>; summaries: Record<string, StatusCardSummaryRevision[]>; cardsEnabled: boolean; cardWork?: CardWorkEvidence; sourceReads?: SourceReads; cardOperations?: Record<string, string[]> }
+import { replyOrCardLinks, replyOrCardText, turnResponseCards, type TurnResponseCard } from "./turn-response-card.js";
+interface CardsObservation extends InternalObservation { responseCards?: TurnResponseCard[]; cards: StatusCard[]; updates: Record<string, StatusCardUpdate[]>; summaries: Record<string, StatusCardSummaryRevision[]>; cardsEnabled: boolean; cardWork?: CardWorkEvidence; sourceReads?: SourceReads; cardOperations?: Record<string, string[]> }
 /** What the responding agent actually got back when it read the source task, split by chat turn. */
 export interface SourceReads { initialLive: boolean; laterUnavailable: boolean; laterLive: boolean }
 const objectRecord = (value: unknown): Record<string, unknown> => value && typeof value === "object" && !Array.isArray(value) ? value as Record<string, unknown> : {};
@@ -62,7 +63,11 @@ async function observeCards(ctx: ChatCatalogContext, state: Record<string, unkno
    ? await observeCardWork(ctx, state.cardRecipe as CardRecipe, state.failedGenerationIssueId) : undefined;
  const sourceReads = ctx.scenario.id === "status-cards.explain-card" && state.sourceUnavailable === true ? await observeSourceReads(ctx, base, state) : undefined;
  const cardOperations = ctx.scenario.id === "status-cards.refresh-card" && options.runEvents !== false ? await observeCardOperations(ctx, base, (state.card as StatusCard).id) : undefined;
-  return { ...base, cards, updates, summaries, cardsEnabled: settings.enableStatusCards, ...(cardWork ? { cardWork } : {}), ...(sourceReads ? { sourceReads } : {}), ...(cardOperations ? { cardOperations } : {}) };
+  // A question card the run ended its turn on is that turn's reply (decision after full cohort 4).
+  const source = ctx.issue();
+  const responseCards = ctx.scenario.id === "status-cards.other-authors-card" && source
+    ? turnResponseCards({ companyId: base.companyId, leadId: base.leadId, runs: base.runs, interactions: await api.get<unknown[]>(`/api/issues/${source.id}/interactions`) }) : undefined;
+  return { ...base, ...(responseCards ? { responseCards } : {}), cards, updates, summaries, cardsEnabled: settings.enableStatusCards, ...(cardWork ? { cardWork } : {}), ...(sourceReads ? { sourceReads } : {}), ...(cardOperations ? { cardOperations } : {}) };
 }
 async function observeCardOperations(ctx: ChatCatalogContext, base: InternalObservation, cardId: string): Promise<Record<string, string[]>> {
  return Object.fromEntries(await Promise.all(base.runs.filter(run => run.agentId === base.leadId).map(async run => [run.id,
@@ -209,9 +214,12 @@ export function gradeIntegratorCards(value: unknown) {
      && (ownerChanged || JSON.stringify(card.refreshPolicy) === JSON.stringify(before.refreshPolicy) && JSON.stringify(o.summaries[card.id]) === JSON.stringify(o.state.originalSummaries)),
      "Morgan's original card is not archived, cloned or reauthored; it changes only through Morgan's own work on a task the CEO handed to Morgan.");
    // A person reads the reply for meaning; structurally it must route the person to the card,
-    // to its author, or to the task handed to the author.
-    c.add("author-management-boundary", o.reply.trim().length > 0 && (!!card && !!cardLink(o, card.id) || authorProfileLinked(o, o.state.worker as Entity | undefined)
-      || delegated.some(row => hasLink(o, "issues", row)))
+    // to its author, or to the task handed to the author. A question card the turn ended on is its reply:
+    // its text counts as the response and the links it renders count with the reply's.
+    const responseCards = o.responseCards ?? [];
+    const answered = { ...o, links: replyOrCardLinks(o.links, responseCards) };
+    c.add("author-management-boundary", replyOrCardText(o.reply, responseCards).trim().length > 0 && (!!card && !!cardLink(answered, card.id) || authorProfileLinked(answered, o.state.worker as Entity | undefined)
+      || delegated.some(row => hasLink(answered, "issues", row)))
      && (delegated.length === 0 || o.runs.every(run => !["queued", "running"].includes(run.status))),
      "The response routes the change to the card, its author or the author's task; a handoff to the author is graded once that work settles.");
    c.add("no-agent-card-mutation", !o.activity.some(event => event.agentId === o.leadId && /^status_card\./.test(event.action ?? "")), "No responding-agent card mutation evades author-specific management.");
@@ -330,7 +338,7 @@ function define(id: string, seed: "none" | "ceo" | "morgan", extra: Partial<Chat
 export const integratorStatusCardsCases: readonly ChatCatalogCase[] = [
  define("status-cards.create-card", "none", { oracleVersion: "paperclip.chat-catalog.status-card.create.v2" }),
  define("status-cards.archive-card", "ceo", { oracleVersion: "paperclip.chat-catalog.status-card.archive.v3", authentication: "human-session", queryAttachments: factualContextQueryAttachment("selected-status-card.md"), expectedRunCount: 3, attemptTimeoutMs: 720_000 }),
-  define("status-cards.other-authors-card", "morgan", { oracleVersion: "paperclip.chat-catalog.status-card.other-authors.v3", expectedRunCount: 5, minimumRunCount: 2 }),
+  define("status-cards.other-authors-card", "morgan", { oracleVersion: "paperclip.chat-catalog.status-card.other-authors.v4", expectedRunCount: 5, minimumRunCount: 2 }),
  define("status-cards.explain-card", "ceo", { oracleVersion: "paperclip.chat-catalog.status-card.explain.v5", authentication: "human-session", queryAttachments: factualContextQueryAttachment("selected-status-card.md"), expectedRunCount: 5, minimumRunCount: 3, expectedHumanMessageCount: 2, attemptTimeoutMs: 720_000,
    sourceFiles: ["chat-catalog-domains/integrator-status-cards.ts", "chat-catalog-domains/api-read-receipts.ts", "chat-catalog-domains/identity-support.ts", "chat-catalog-domains/internal-definition.ts", "chat-catalog-domains/internal-support.ts", "chat-catalog-domains/integrator-support.ts", "run-observations.ts", "chat-scenarios/status-cards.yaml"],
    async act(ctx, state) {
@@ -150,6 +150,7 @@ function claimsUnrequestedChange(reply: string): boolean {
 }

 import { observeInternal } from "./internal-support.js";
+import { waitForTurnReplyLinks } from "./identity-support.js";
 import { pollUntil } from "../api.js";

 function purposeVariants(): ChatCatalogCase {
@@ -171,6 +172,8 @@ function purposeVariants(): ChatCatalogCase {
      foreignRequest: `Why are we doing the website task ${foreignTask.identifier ?? foreignTask.id} in our other company?` };
  });
  return { ...base, coverageGaps: [], expectedRunCount: 3, expectedHumanMessageCount: 3, attemptTimeoutMs: 600_000,
+    // v2: each phase is captured after its saved reply renders, so its rendered links are present.
+    oracleVersion: "paperclip.chat-catalog.internal.goals-purpose.v2", sourceFiles: [...base.sourceFiles, "chat-catalog-domains/identity-support.ts", "chat-rendered-links.ts"],
    async act(ctx, state) {
      const first = await waitForPurposePhase(ctx, state, 1);
      if (!initialPurposeValid(first)) throw new Error("Initial purpose explanation does not match its saved chain and rationale");
@@ -197,9 +200,12 @@ function purposeVariants(): ChatCatalogCase {
  };
 }
 async function waitForPurposePhase(ctx: ChatCatalogContext, state: Record<string, unknown>, count: number) {
-  return pollUntil({ label: `Purpose explanation phase ${count}`, deadlineAt: Date.now() + 150_000,
+  await pollUntil({ label: `Purpose explanation phase ${count}`, deadlineAt: Date.now() + 150_000,
    load: () => observeInternal(ctx, state), accept: o => o.runs.length === count && o.runs.every(row => row.status === "succeeded")
      && ctx.issue()?.conversationState === "waiting" });
+  // Finished runs and a waiting chat precede the saved reply's render; capture the phase only after it renders.
+  await waitForTurnReplyLinks(ctx);
+  return observeInternal(ctx, state);
 }
 async function sourceReplies(ctx: ChatCatalogContext) {
  const source = ctx.issue(); if (!source) throw new Error("Purpose phase lost its source conversation");
@@ -1,4 +1,4 @@
-import { collectConversationLinks, conversationLinkLocator, markdownLinkTargets, type RenderedConversationLink } from "../chat-rendered-links.js";
+import { collectConversationLinksWithDiagnostics, conversationLinkLocator, markdownLinkTargets, type ConversationLinkDiagnostics, type RenderedConversationLink } from "../chat-rendered-links.js";
 import { expect } from "@playwright/test";
 import type { ChatCatalogContext } from "../chat-catalog-contract.js";
 import type { ChatRun } from "../chat-flow.js";
@@ -17,7 +17,7 @@ export interface Entity {
 export interface InternalObservation {
  scenarioId: string; origin: string; companyId: string; leadId: string; state: Record<string, unknown>;
  tasks: Entity[]; projects: Entity[]; goals: Entity[]; comments: Entity[];
-  renderedLinkEvidence?: RenderedConversationLink[]; activity: Entity[]; annotations?: Entity[]; agents?: Entity[]; documents: Entity[]; revisions: Entity[]; reply: string; links: string[]; humanQueries: string[]; runs: ChatRun[];
+  renderedLinkEvidence?: RenderedConversationLink[]; linkDiagnostics?: ConversationLinkDiagnostics; activity: Entity[]; annotations?: Entity[]; agents?: Entity[]; documents: Entity[]; revisions: Entity[]; reply: string; links: string[]; humanQueries: string[]; runs: ChatRun[];
 }
 export function entity(state: Record<string, unknown>, key: string): Entity {
  const value = state[key];
@@ -56,10 +56,11 @@ export async function observeInternal(ctx: ChatCatalogContext, state: Record<str
  const companyActivity = await api.getRunObservation<Entity[]>(`${companyPath(ctx)}/activity?limit=1000`);
  const taskActivity = target ? await api.get<Entity[]>(`/api/issues/${target.id}/activity`) : [];
  const activity = [...new Map([...companyActivity, ...taskActivity].map(row => [row.id, row])).values()];
-  const renderedLinkEvidence = source ? await collectConversationLinks(ctx.input.page, {
+  const collected = source ? await collectConversationLinksWithDiagnostics(ctx.input.page, {
    origin: api.baseURL, companyId: fixtures.company.id, companyPrefix: fixtures.company.issuePrefix,
    sourceId: source.id, leadId: fixtures.agent.id, comments: chat, runs, projects, activity,
-  }) : [];
+  }) : { links: [] };
+  const renderedLinkEvidence = collected.links;
  const links = [...new Set(renderedLinkEvidence.map(link => link.href))];
  const documents: Entity[] = []; const revisions: Entity[] = [];
  if (target) {
@@ -71,7 +72,7 @@ export async function observeInternal(ctx: ChatCatalogContext, state: Record<str
  }
  return { scenarioId: ctx.scenario.id, origin: api.baseURL, companyId: fixtures.company.id, leadId: fixtures.agent.id,
    state, tasks: tasks.filter(row => row.id !== source?.id), projects, goals, runs, comments, activity, documents, revisions,
-    renderedLinkEvidence, humanQueries: chat.filter(row => !row.authorAgentId).map(row => row.body ?? ""), reply: replies.map(row => row.body ?? "").join("\n"), links };
+    renderedLinkEvidence, ...(collected.diagnostics ? { linkDiagnostics: collected.diagnostics } : {}), humanQueries: chat.filter(row => !row.authorAgentId).map(row => row.body ?? ""), reply: replies.map(row => row.body ?? "").join("\n"), links };
 }
 export function hasLink(o: InternalObservation, kind: "issues" | "projects" | "goals", row: Entity): boolean {
  return o.links.some(href => {
@@ -91,11 +91,14 @@ function ambiguousTask(): ChatCatalogCase {
    await verifyDistinctTaskFixtures(ctx, [t, otherTask]);
    return { project: p, task: t, otherProject, otherTask };
  });
-  return { ...base, oracleVersion: "paperclip.chat-catalog.internal.ambiguous-task.v2", sourceFiles: [...base.sourceFiles, "chat-catalog-domains/internal-distinct-fixtures.ts", "chat-linked-question.ts"], expectedRunCount: 2, expectedHumanMessageCount: 2, attemptTimeoutMs: 420_000,
+  return { ...base, oracleVersion: "paperclip.chat-catalog.internal.ambiguous-task.v3", sourceFiles: [...base.sourceFiles, "chat-catalog-domains/internal-distinct-fixtures.ts", "chat-linked-question.ts", "chat-catalog-domains/identity-support.ts", "chat-rendered-links.ts"], expectedRunCount: 2, expectedHumanMessageCount: 2, attemptTimeoutMs: 420_000,
    async act(ctx, state) {
-      const plain = await pollUntil({ label: "Task ambiguity clarification", deadlineAt: Date.now() + 150_000,
+      await pollUntil({ label: "Task ambiguity clarification", deadlineAt: Date.now() + 150_000,
        load: () => observeInternal(ctx, state), accept: o => o.runs.length === 1 && o.runs.every(row => row.status === "succeeded")
          && ctx.issue()?.conversationState === "waiting" });
+      // The finished run and waiting chat precede the saved reply's render; capture its links only after it renders.
+      await waitForTurnReplyLinks(ctx);
+      const plain = await observeInternal(ctx, state);
      const first = await observeLinkedTaskQuestion(ctx, state, plain);
      state.clarification = { ...first, state: structuredClone(state) };
      await ctx.input.evidence("chat-catalog-task-clarification.json", state.clarification);
@@ -130,3 +133,4 @@ function ambiguousTask(): ChatCatalogCase {
 }
 import { observeInternal } from "./internal-support.js";
 import { pollUntil } from "../api.js";
+import { waitForTurnReplyLinks } from "./identity-support.js";
@@ -15,6 +15,7 @@ import { normalizedSecrets, redactText } from "../redaction.js";
 import { CREDENTIAL_NAMES } from "../types.js";
 import { checks, project, task, type Entity } from "./internal-support.js";
 import { privilegedDefinition, privilegedObservation, settlePrivilegedChat, companionPhase } from "./privileged-support.js";
+import { replyOrCardText } from "./turn-response-card.js";
 const execute = promisify(execFile);
 interface OwnedDev { root: string; checkout: string; home: string; instance: string; env: NodeJS.ProcessEnv; loader: string; entry: string; log: string; child?: ChildProcess; closed?: Promise<void>; config?: string }
 const owned = new WeakMap<ChatCatalogContext, OwnedDev>();
@@ -150,7 +151,9 @@ const dev = privilegedDefinition("cli-operations.owned-dev-instance", "privilege
      dependencySource: "explicit immutable candidate modules; source and compiled dependency provenance retained in campaign snapshot receipt" };
  },
  async act(ctx, state) {
-    const initial = await settlePrivilegedChat(ctx); state.guidance = initial.reply; const o = owned.get(ctx)!;
+    // A question card the run ended its turn on is that turn's reply (decision after full cohort 4): its text counts.
+    const initial = await settlePrivilegedChat(ctx); state.guidance = replyOrCardText(initial.reply, initial.responseCards);
+    state.guidanceCardIds = initial.responseCards.map(card => card.id); const o = owned.get(ctx)!;
    state.hostInstallationsUnchanged = sameInstallations(state.installationsBefore as Record<string, string>, await paperclipInstallations(await realpath(process.env.PAPERCLIP_RUNNER_E2E_TEMP_ROOT!)));
    const reserve = async () => { const listener = createServer(); const port = await listenOnFetchAllowedPort(listener); await new Promise<void>((resolve, reject) => listener.close(e => e ? reject(e) : resolve())); return port; };
    const serverPort = await reserve(), dbPort = await reserve();
@@ -192,7 +195,7 @@ const dev = privilegedDefinition("cli-operations.owned-dev-instance", "privilege
    await sendChatMessage(ctx.input.page, message); state.devReport = await companionPhase(ctx, initial.runs.map(r => r.id), message);
  }, grade: gradeOwnedDev, async verifyResult(ctx) { await expect(ctx.input.page.getByTestId("task-chat-composer-input")).toBeVisible(); },
 });
-dev.oracleVersion = "paperclip.chat-catalog.privileged.owned-dev-instance.v2"; dev.expectedRunCount = 2; dev.expectedHumanMessageCount = 2; dev.attemptTimeoutMs = 900000;
+dev.oracleVersion = "paperclip.chat-catalog.privileged.owned-dev-instance.v3"; dev.expectedRunCount = 2; dev.expectedHumanMessageCount = 2; dev.attemptTimeoutMs = 900000;
 dev.cleanup = async ctx => { const o = owned.get(ctx); if (!o) return; await stop(o); if (o.config) await command(o, ["worktree:cleanup", o.instance, "--instance", o.instance, "--home", o.home]);
  const root = await realpath(process.env.PAPERCLIP_RUNNER_E2E_TEMP_ROOT!), directory = await realpath(o.root);
  if (!directory.startsWith(root + path.sep) || !path.basename(directory).startsWith("operator-dev-")) throw new Error("Refusing non-owned development cleanup");
@@ -2,7 +2,7 @@ import { chatCatalogNativePhaseDeadline, chatCatalogNativePhaseIsTimely } from "
 import type { ChatCatalogCase, ChatCatalogContext } from "../chat-catalog-contract.js";
 import { pollUntil } from "../api.js";
 import { observeInternal, observation, type Entity, type InternalObservation } from "./internal-support.js";
-import { turnResponseCards } from "./turn-response-card.js";
+import { turnResponseCards, type TurnResponseCard } from "./turn-response-card.js";

 /** These cases exercise only the launcher-owned instance or a rendered developer fixture. */
 export function privilegedDefinition(scenarioId: string, module: string, extra: Pick<ChatCatalogCase, "setup" | "grade" | "act" | "verifyResult">): ChatCatalogCase {
@@ -22,7 +22,7 @@ export function remainingPrivilegedObservationBudget(ctx: ChatCatalogContext): n
 * or with a pending question card the product chose as the turn's response. A card settles the
 * turn only; each case still grades whether the reply (or card) is the guidance it requires.
 */
-export async function settlePrivilegedChat(ctx: ChatCatalogContext, observationDeadlineAt?: number): Promise<InternalObservation> {
+export async function settlePrivilegedChat(ctx: ChatCatalogContext, observationDeadlineAt?: number): Promise<InternalObservation & { responseCards: TurnResponseCard[] }> {
  const settled = await pollUntil({ label: "Canonical privileged guidance settled", deadlineAt: chatCatalogNativePhaseDeadline(ctx, observationDeadlineAt ?? Date.now() + 150_000),
    load: async () => {
      const o = await observeInternal(ctx, {}), source = ctx.issue();
@@ -31,7 +31,7 @@ export async function settlePrivilegedChat(ctx: ChatCatalogContext, observationD
    },
    accept: ({ o, cards }) => chatCatalogNativePhaseIsTimely(ctx) && ((observationDeadlineAt === undefined || Date.now() < observationDeadlineAt) && o.runs.length === 1 && o.runs[0]?.status === "succeeded" && (!!o.reply || cards.length > 0) && ctx.issue()?.conversationState === "waiting"),
    reject: ({ o }) => o.runs.some(run => ["failed", "timed_out", "cancelled"].includes(run.status)) ? "Canonical guidance run failed" : undefined });
-  return settled.o;
+  return { ...settled.o, responseCards: settled.cards };
 }
 export const privilegedObservation = observation;

@@ -7,6 +7,7 @@ import { sendChatMessage } from "../chat-flow.js";
 import { checks, type Entity } from "./internal-support.js";
 import { enableTaskWorkspaceView, savedTaskWorkspace } from "./runtime-workspaces.js";
 import { runtimeDefinition, runtimeObservation, observeRuntime } from "./runtime-support.js";
+import { waitForTurnReplyLinks } from "./identity-support.js";
 import type { WorkspaceRuntimeService } from "../../../packages/shared/src/types/workspace-runtime.js";
 import type { WorkspaceOperation } from "../../../packages/shared/src/types/workspace-operation.js";
 import type { IssueWorkProduct } from "../../../packages/shared/src/types/work-product.js";
@@ -63,8 +64,8 @@ export const runtimePreviewCases = [runtimeDefinition("workspaces.preview-servic
  if (saved.runtimeServices?.some(row => row.status === "running")) throw new Error("Preview setup must not start the target service");
  return { ...state, workspace: saved, websiteBytes, failureFlag };
 }, gradePreview, {
-  oracleVersion: "paperclip.chat-catalog.preview-service.v4",
-  sourceFiles: ["chat-catalog-domains/runtime-preview.ts", "chat-catalog-domains/runtime-workspaces.ts", "chat-catalog-domains/runtime-support.ts", "chat-scenarios/workspaces.yaml"],
+  oracleVersion: "paperclip.chat-catalog.preview-service.v5",
+  sourceFiles: ["chat-catalog-domains/runtime-preview.ts", "chat-catalog-domains/runtime-workspaces.ts", "chat-catalog-domains/runtime-support.ts", "chat-catalog-domains/identity-support.ts", "chat-rendered-links.ts", "chat-scenarios/workspaces.yaml"],
  expectedHumanMessageCount: 2, expectedRunCount: 3, attemptTimeoutMs: 720_000,
  async act(ctx, state) {
    await pollUntil({ label: "Actual canonical preview service readiness", deadlineAt: chatCatalogNativePhaseDeadline(ctx, Date.now() + 180_000),
@@ -77,6 +78,8 @@ export const runtimePreviewCases = [runtimeDefinition("workspaces.preview-servic
    const url = new URL(service.url); if (url.protocol !== "http:" || url.hostname !== "127.0.0.1" || Number(url.port) !== service.port) throw new Error("Preview endpoint escaped its owned loopback service");
    const response = await fetch(url, { redirect: "error", signal: AbortSignal.timeout(10_000) });
    state.initialHttpStatus = response.status; state.initialHttpBody = await response.text(); state.healthyService = service;
+    // The service and finished runs precede the saved reply's render; read its links only after it renders.
+    await waitForTurnReplyLinks(ctx);
    const o = await observeRuntime(ctx, state); state.queryLinks = o.links;
    const products = [];
    for (const id of [(state.task as Entity).id, ctx.issue()!.id]) products.push(...await ctx.input.api.get<IssueWorkProduct[]>(`/api/issues/${id}/work-products`));
@@ -5,7 +5,7 @@ vi.mock("@playwright/test", () => ({ expect: (target: any) => ({
  toBeEditable: async () => { if (!target.editable()) throw Error("Chat composer is not ready"); },
  toHaveURL: async (matches: (url: URL) => boolean) => { if (!matches(new URL(target.url()))) throw Error("Wrong selected Chat route"); },
 }) }));
-import { runtimePrimaryCases } from "./runtime-primary.js";
+import { runtimePrimaryCases, settledReadyChatChecks } from "./runtime-primary.js";
 afterEach(() => { vi.restoreAllMocks(); });
 function fixture(options: { loadingForever?: boolean; extraRun?: boolean; extraChat?: boolean; wrongPrimary?: boolean; wrongPerson?: boolean;
  wrongChat?: boolean; noReply?: boolean; wrongRequest?: boolean; missingRenderedReply?: boolean; readOnly?: boolean; expired?: boolean; lateReady?: boolean } = {}) {
@@ -53,3 +53,18 @@ it.each([{ loadingForever: true }, { extraRun: true }, { extraChat: true }, { wr
  const f = fixture(options); await expect(runtimePrimaryCases[0]!.verifyResult(f.ctx, f.state, f.observation)).rejects.toThrow();
  expect(f.evidence.mock.calls.some(row => row[0] === "chat-primary-ready-conversation.json")).toBe(false);
 });
+// Full cohort 4: the combined throw could not say whether the primary readback, the chat count or the run count changed.
+it("names each failing settled-Chat clause with its values", async () => {
+  const f = fixture({ extraChat: true });
+  await expect(runtimePrimaryCases[0]!.verifyResult(f.ctx, f.state, f.observation)).rejects.toThrow(/chat-count-unchanged \(2 chats/);
+  expect(f.evidence).toHaveBeenCalledWith("chat-primary-ready-conversation-failure.json", expect.objectContaining({ failed: [expect.objectContaining({ id: "chat-count-unchanged" })] }));
+  const run = fixture({ extraRun: true });
+  await expect(runtimePrimaryCases[0]!.verifyResult(run.ctx, run.state, run.observation)).rejects.toThrow(/no-extra-run/);
+  const primary = fixture({ wrongPrimary: true });
+  await expect(runtimePrimaryCases[0]!.verifyResult(primary.ctx, primary.state, primary.observation)).rejects.toThrow(/primary-readback \(primary ceo/);
+});
+it("passes every clause on the settled landing", () => {
+  const checks = settledReadyChatChecks({ currentIssueId: "source", sourceId: "source", companyId: "company", userId: "person", workerId: "morgan", leadId: "ceo",
+    primary: { companyId: "company", userId: "person", primaryAgentId: "morgan" }, chats: [{ id: "source" }], chatCountBefore: 1, runs: [{ id: "native", agentId: "ceo", status: "succeeded" }], nativeRunId: "native" });
+  expect(checks.map(check => [check.id, check.passed])).toEqual([["same-conversation-open", true], ["primary-readback", true], ["chat-count-unchanged", true], ["no-extra-run", true]]);
+});
@@ -39,6 +39,20 @@ export function gradePrimary(value: unknown) {
    "The reply links Morgan's profile, and the person reaches the primary control by clicking that rendered link.");
  return c.result();
 }
+/** The settled Chat landing, checked clause by clause so a failure names which part changed and its values. */
+export function settledReadyChatChecks(input: { currentIssueId?: string; sourceId: string; companyId: string; userId: string; workerId: string; leadId: string;
+  primary: Primary; chats: Entity[]; chatCountBefore: unknown; runs: Array<{ id: string; agentId?: string | null; status?: string }>; nativeRunId: string }) {
+  const { primary, chats, runs } = input;
+  return [
+    { id: "same-conversation-open", passed: input.currentIssueId === input.sourceId, detail: `open conversation ${input.currentIssueId ?? "none"}, expected ${input.sourceId}` },
+    { id: "primary-readback", passed: primary.companyId === input.companyId && primary.userId === input.userId && primary.primaryAgentId === input.workerId,
+      detail: `primary ${primary.primaryAgentId ?? "none"} for user ${primary.userId} in ${primary.companyId}, expected ${input.workerId}` },
+    { id: "chat-count-unchanged", passed: chats.length === input.chatCountBefore && chats.filter(row => row.id === input.sourceId).length === 1,
+      detail: `${chats.length} chats (${chats.filter(row => row.id === input.sourceId).length} of them the source), expected ${String(input.chatCountBefore)} with the source once` },
+    { id: "no-extra-run", passed: runs.length === 1 && runs[0]?.id === input.nativeRunId && runs[0].agentId === input.leadId && runs[0].status === "succeeded",
+      detail: `runs ${runs.map(run => `${run.id}:${run.agentId}:${run.status}`).join(",") || "none"}, expected only ${input.nativeRunId} succeeded` },
+  ];
+}
 export async function verifyReadyPrimaryChat(ctx: ChatCatalogContext, state: Record<string, unknown>, value: unknown) {
  const o = runtimeObservation(value); const before = state.primaryBefore as Primary; const worker = state.worker as RuntimeAgent;
  const sourceId = state.sourceBeforeNavigation; const nativeIds = state.nativeIdsBeforeNavigation;
@@ -69,10 +83,12 @@ export async function verifyReadyPrimaryChat(ctx: ChatCatalogContext, state: Rec
  const primary = await ctx.input.api.get<Primary>(endpoint(o.companyId), { timeout: budget() }); budget();
  const chats = await ctx.input.api.get<Entity[]>(`/api/companies/${o.companyId}/chats`, { timeout: budget() }); budget();
  const runs = await ctx.allRuns(); budget();
-  if (ctx.issue()?.id !== sourceId || primary.companyId !== o.companyId || primary.userId !== before.userId || primary.primaryAgentId !== worker.id
-    || chats.length !== state.chatCountBeforeNavigation || chats.filter(row => row.id === sourceId).length !== 1
-    || runs.length !== 1 || runs[0]?.id !== nativeIds[0] || runs[0].agentId !== o.leadId || runs[0].status !== "succeeded")
-    throw new Error("Settled Chat changed the primary preference, conversation or native work");
+  const failed = settledReadyChatChecks({ currentIssueId: ctx.issue()?.id, sourceId, companyId: o.companyId, userId: before.userId, workerId: worker.id, leadId: o.leadId,
+    primary, chats, chatCountBefore: state.chatCountBeforeNavigation, runs, nativeRunId: nativeIds[0] }).filter(check => !check.passed);
+  if (failed.length) {
+    await ctx.input.evidence("chat-primary-ready-conversation-failure.json", { failed, primary, chatIds: chats.map(row => row.id), runs: runs.map(run => ({ id: run.id, agentId: run.agentId, status: run.status })) });
+    throw new Error(`Settled Chat failed ${failed.map(check => `${check.id} (${check.detail})`).join("; ")}`);
+  }
  state.reopenedRecentAgent = o.leadId; state.runsAfterNavigation = runs.length; state.chatCountAfterNavigation = chats.length;
  await ctx.input.evidence("chat-primary-ready-conversation.json", { companyId: o.companyId, agentId: o.leadId, sourceId, userId: before.userId,
    primaryAgentId: primary.primaryAgentId, originalNativeIds: nativeIds, chatCount: chats.length, composerReady: true, originalConversationVisible: true, noExtraWork: true }); budget();
@@ -89,7 +105,7 @@ export const runtimePrimaryCases = [runtimeDefinition("agents.choose-primary", a
  return { worker, primaryBefore, otherCompany, otherPersonBefore: await personPrimary(person, ctx.input.fixtures.company.id),
    otherCompanyBefore: await ctx.input.api.get<Primary>(endpoint(otherCompany.id)) };
 }, gradePrimary, {
-  authentication: "human-session", oracleVersion: "paperclip.chat-catalog.runtime-primary.v4",
+  authentication: "human-session", oracleVersion: "paperclip.chat-catalog.runtime-primary.v5",
  sourceFiles: ["chat-catalog-domains/runtime-primary.ts", "chat-catalog-domains/runtime-support.ts", "chat-catalog-domains/identity-support.ts", "chat-scenarios/agents.yaml"],
  async act(ctx, state) {
    remainingIdentityObservationBudget(ctx);
@@ -11,6 +11,7 @@ function valid() {
      queueAfter: { ...before, entries: [{ ...before.entries[1]!, position: 0, comment: { ...before.entries[1]!.comment,
        body: "Make the website blue. Keep the existing note about accessible labels. Preserve the human note." } }, { ...before.entries[0]!, position: 1 }] },
      queryReply: "Your pending red request is in the Website queue. Open its message menu, edit the full request to blue, then move it first.",
+      queryLinks: ["/COM/issues/website"], queueAtReply: structuredClone(before),
      agentEditStatus: 403, agentOrderStatus: 403, concurrentStatus: 409, concurrentErrorVisible: true, concurrentSaveRetained: true,
      dispatchedStatus: 409, dispatchedWarningVisible: true, originalCommentStillReadable: true, nativeDeliveryBound: true, queueEmptyAfterDelivery: true,
      consumingRunCount: 1, initialRevisionRetained: true } };
@@ -21,7 +22,22 @@ it("rejects missing runtime evidence", () => expect(gradeQueueEdit(undefined).pa
 it("rejects an agent-owned message presented as human-owned", () => { const o = valid(); o.state.queueBefore.entries[1]!.comment.authorUserId = "different-person"; expect(gradeQueueEdit(o).passed).toBe(false); });
 it("rejects changing only a fragment and dropping the concurrent note", () => { const o = valid(); o.state.queueAfter.entries[0]!.comment.body = "Make the website blue."; expect(gradeQueueEdit(o).passed).toBe(false); });
 it("rejects losing the other queued message", () => { const o = valid(); o.state.queueAfter.entries.pop(); expect(gradeQueueEdit(o).passed).toBe(false); });
-it("rejects agent claims to have forged the human edit", () => { const o = valid(); o.state.queryReply = "I've edited the queued request and reordered it first."; expect(gradeQueueEdit(o).passed).toBe(false); });
+// Full cohort 4: correct guidance worded as "change ... and drag it above" failed the old edit|control|menu keyword list.
+it("accepts reworded guidance that links the task, on the same saved evidence", () => {
+  const o = valid(); o.state.queryReply = "Changing a queued message's text and order is board-only, so I couldn't apply it. On [COM-1](/COM/issues/COM-1), change that queued message to blue and drag it above the font note.";
+  o.state.queryLinks = ["/COM/agents/ceo/overview", "/COM/issues/website"]; expect(gradeQueueEdit(o).passed).toBe(true);
+});
+it("rejects guidance without a rendered link to the task", () => { const o = valid(); o.state.queryLinks = ["/COM/agents/ceo/overview"]; expect(gradeQueueEdit(o).passed).toBe(false); });
+it("rejects an empty reply", () => { const o = valid(); o.state.queryReply = " "; expect(gradeQueueEdit(o).passed).toBe(false); });
+it("rejects an edit or reorder applied before the person acted", () => {
+  const edited = valid(); edited.state.queueAtReply.entries[1]!.comment.body = "Make the website blue. Keep the existing note about accessible labels."; expect(gradeQueueEdit(edited).passed).toBe(false);
+  const reordered = valid(); reordered.state.queueAtReply.entries[0]!.position = 1; reordered.state.queueAtReply.entries[1]!.position = 0; expect(gradeQueueEdit(reordered).passed).toBe(false);
+  const missing = valid(); delete (missing.state as Record<string, unknown>).queueAtReply; expect(gradeQueueEdit(missing).passed).toBe(false);
+});
+it("rejects an agent that can edit or reorder the person's message", () => {
+  const edit = valid(); edit.state.agentEditStatus = 200; expect(gradeQueueEdit(edit).passed).toBe(false);
+  const order = valid(); order.state.agentOrderStatus = 200; expect(gradeQueueEdit(order).passed).toBe(false);
+});
 it("rejects stale concurrent acceptance", () => { const o = valid(); o.state.concurrentStatus = 200; expect(gradeQueueEdit(o).passed).toBe(false); });
 it("rejects silently overwriting an already-consumed message", () => { const o = valid(); o.state.dispatchedStatus = 200; expect(gradeQueueEdit(o).passed).toBe(false); });
 it("rejects a plausible wrong consumer without exact saved input and order", () => { const o = valid(); o.state.nativeDeliveryBound = false; expect(gradeQueueEdit(o).passed).toBe(false); });
@@ -5,8 +5,8 @@ import type { IssueQueuedCommentQueue } from "../../../packages/shared/src/types
 import type { ChatCatalogContext } from "../chat-catalog-contract.js";
 import { pollUntil } from "../api.js";
 import { submitTaskReply } from "../user-actions.js";
-import { checks, type Entity } from "./internal-support.js";
-import { probeAgentPermission, cleanupIdentityProbes } from "./identity-support.js";
+import { checks, hasLink, type Entity } from "./internal-support.js";
+import { probeAgentPermission, cleanupIdentityProbes, waitForTurnReplyLinks } from "./identity-support.js";
 import { busyRuntimeWorker, releaseRuntimeWorker } from "./runtime-busy-worker.js";
 import { runtimeQueueCases } from "./runtime-queue.js";
 import { observeRuntime, runtimeDefinition, runtimeObservation } from "./runtime-support.js";
@@ -56,9 +56,16 @@ export function gradeQueueEdit(value: unknown) {
  c.add("full-edited-text-and-first-position-persist", !!saved && saved.position === 0 && saved.comment.body === `${blueText} ${s.concurrentMarker}`
    && saved.comment.authorUserId === s.humanId && !saved.comment.authorAgentId && after.entries.length === 2
    && after.entries.some(row => row.comment.body === otherText && row.position === 1), "The same comment retains its author, the complete changed text, the concurrent human note and first position; the other message is intact.");
-  c.add("chat-guides-human-control-without-forging", /queue|pending|waiting/i.test(String(s.queryReply)) && /edit|control|menu|reorder|first/i.test(String(s.queryReply))
-    && !/I(?:'ve| have)? (?:already )?(?:edited|changed|reordered)/i.test(String(s.queryReply)) && s.agentEditStatus === 403 && s.agentOrderStatus === 403,
-  "The canonical reply identifies the human queue controls and the actual agent cannot impersonate the person's edits or reorder.");
+  // Graded on saved state and the rendered link, never wording ("change ... and drag it above" and "use Edit in its
+  // menu" are both correct guidance). The reply links the task whose queue holds the person's controls, the agent's
+  // own edit and reorder are refused, and at the reply the queue is exactly as the person left it: no edit applied.
+  const atReply = s.queueAtReply as IssueQueuedCommentQueue | undefined;
+  const unchangedAtReply = !!before && !!atReply && atReply.entries.length === before.entries.length && before.entries.every(old => atReply.entries.some(row =>
+    row.comment.id === old.comment.id && row.position === old.position && row.comment.body === old.comment.body && row.comment.authorUserId === old.comment.authorUserId && !row.comment.authorAgentId));
+  const replyLinks = Array.isArray(s.queryLinks) ? s.queryLinks as string[] : [];
+  c.add("chat-guides-human-control-without-forging", !!String(s.queryReply ?? "").trim() && !!s.task && hasLink({ ...o, links: replyLinks }, "issues", s.task as Entity)
+    && unchangedAtReply && s.agentEditStatus === 403 && s.agentOrderStatus === 403,
+  "The canonical reply links the task whose queue holds the person's controls, the actual agent cannot edit or reorder the person's message, and no edit was applied before the person acted.");
  c.add("concurrent-edit-rejects-stale-request-visibly", s.concurrentStatus === 409 && s.concurrentErrorVisible === true && s.concurrentSaveRetained === true,
    "An actual held old-revision human request loses to a real second-session edit and shows its conflict, retaining the winner's text.");
  c.add("dispatched-edit-is-visible-new-direction", s.dispatchedStatus === 409 && s.dispatchedWarningVisible === true && s.originalCommentStillReadable === true,
@@ -70,7 +77,9 @@ export function gradeQueueEdit(value: unknown) {
 }

 export const runtimeQueueControlCases = [runtimeDefinition("steering.edit-queued-message", ctx => humanQueuedRuntime(ctx), gradeQueueEdit, {
-  sourceFiles: ["chat-catalog-domains/runtime-queue-controls.ts", "chat-catalog-domains/runtime-queue.ts", "chat-catalog-domains/runtime-busy-worker.ts", "chat-catalog-domains/runtime-support.ts", "chat-catalog-domains/identity-support.ts", "chat-scenarios/steering.yaml"],
+  sourceFiles: ["chat-catalog-domains/runtime-queue-controls.ts", "chat-catalog-domains/runtime-queue.ts", "chat-catalog-domains/runtime-busy-worker.ts", "chat-catalog-domains/runtime-support.ts", "chat-catalog-domains/identity-support.ts", "chat-catalog-domains/internal-support.ts", "chat-rendered-links.ts", "chat-scenarios/steering.yaml"],
+  // v3: guidance graded on the task link, refused agent writes and the unchanged queue at the reply, not keywords.
+  oracleVersion: "paperclip.chat-catalog.edit-queued-message.structural.v3",
  authentication: "human-session", expectedRunCount: 3, attemptTimeoutMs: 720_000,
  setupPolicy: runtimeQueueCases[0]!.setupPolicy, verifySetup: runtimeQueueCases[0]!.verifySetup,
  async act(ctx, state) {
@@ -78,8 +87,11 @@ export const runtimeQueueControlCases = [runtimeDefinition("steering.edit-queued
      load: async () => { await ctx.refreshIssue(); return { source: ctx.issue(), runs: await ctx.allRuns() }; },
      accept: v => chatCatalogNativePhaseIsTimely(ctx) && (v.source?.conversationState === "waiting" && v.runs.length === 2 && v.runs.some(row => row.id === state.workerRunId && row.status === "running")
        && v.runs.some(row => row.agentId === ctx.input.fixtures.agent.id && row.status === "succeeded")) });
-    state.queryReply = (await observeRuntime(ctx, state)).reply; const source = ctx.input.page.url();
+    // The finished run and waiting chat precede the saved reply's render; read its links only after it renders.
+    await waitForTurnReplyLinks(ctx);
+    const answered = await observeRuntime(ctx, state); state.queryReply = answered.reply; state.queryLinks = answered.links; const source = ctx.input.page.url();
    const initial = await queue(ctx, state), targetId = String(state.targetId), mutation = `${endpoint(state)}/${targetId}`;
+    state.queueAtReply = initial;
    state.agentEditStatus = await probeAgentPermission(ctx, mutation, "PATCH", { queueId: initial.queueId, revision: initial.revision, body: blueText });
    state.agentOrderStatus = await probeAgentPermission(ctx, `${endpoint(state)}/order`, "PUT", { queueId: initial.queueId, revision: initial.revision, orderedCommentIds: initial.entries.map(row => row.comment.id).reverse() });
    await ctx.input.page.goto(taskUrl(ctx, state), { waitUntil: "domcontentloaded" });
@@ -1,5 +1,5 @@
 import { describe, expect, it } from "vitest";
-import { INTERACTION_OWNS_RESPONSE, replyOrCardText, turnResponseCards } from "./turn-response-card.js";
+import { INTERACTION_OWNS_RESPONSE, renderedMarkdownLinks, replyOrCardLinks, replyOrCardText, turnResponseCards } from "./turn-response-card.js";
 import { grade_goals } from "./internal-goals.js";
 import { observation as internalObservation } from "./internal-support.js";

@@ -104,3 +104,42 @@ describe("goals.align-existing-work final phase", () => {
    }
  });
 });
+
+// Decision after full cohort 4: a question card counts as the turn's reply; its Markdown-rendered fields can carry links.
+describe("links a response card renders", () => {
+  it("reads Markdown links from a question form's description and prompts and a confirmation's prompt and details only", () => {
+    const question = goalQuestion() as ReturnType<typeof goalQuestion> & { payload: Record<string, unknown> };
+    question.payload.questionSet = { schema: "paperclip.question_set.v1", description: "Pick [COM-1](/COM/issues/COM-1) or [COM-2](/COM/issues/COM-2).",
+      questions: [{ id: "goal_choice", prompt: "Which one? [COM-9](/COM/issues/COM-9)", answerMode: "single_select", required: true,
+        options: [{ id: "a", label: "[COM-8](/COM/issues/COM-8)" }, { id: "b", label: "COM-2" }] }] };
+    const [card] = cards([question]);
+    // QuestionForm renders the description and each prompt with MarkdownBody; option labels stay plain text.
+    expect(card!.links).toEqual(["/COM/issues/COM-1", "/COM/issues/COM-2", "/COM/issues/COM-9"]);
+    const review = draftReview(); review.payload.prompt = "Open [the draft](/COM/issues/COM-3#document-draft) first.";
+    expect(cards([review])[0]!.links).toEqual(["/COM/issues/COM-3#document-draft"]);
+    expect(replyOrCardLinks(["/COM/agents/ceo"], [card!])).toEqual(["/COM/agents/ceo", "/COM/issues/COM-1", "/COM/issues/COM-2", "/COM/issues/COM-9"]);
+  });
+  it("reads prompt links from a legacy question form without a questionSet", () => {
+    const question = goalQuestion(); question.payload.questions[0]!.prompt = "Which goal? See [the growth goal](/COM/goals/growth).";
+    expect(cards([question])[0]!.links).toEqual(["/COM/goals/growth"]);
+  });
+  it("finds no link in a card without Markdown links, such as cohort 4's ambiguous-task options", () => {
+    const question = goalQuestion();
+    question.payload.questions[0]!.options = [{ id: "com-1", label: "COM-1 in Launch project" }, { id: "com-2", label: "COM-2 in Experiments project" }];
+    expect(cards([question])[0]!.links).toEqual([]);
+  });
+});
+describe("Markdown a card renders as clickable links", () => {
+  it("counts inline links and autolinks, not code, fenced blocks, images or bare reference definitions", () => {
+    expect(renderedMarkdownLinks("Open [Create organization](/onboarding) or <https://example.test/a>.")).toEqual(["/onboarding", "https://example.test/a"]);
+    expect(renderedMarkdownLinks("Type `[setup](/onboarding)` to see the syntax.")).toEqual([]);
+    expect(renderedMarkdownLinks("```\n[setup](/onboarding)\n```\nThen continue.")).toEqual([]);
+    expect(renderedMarkdownLinks("[setup]: /onboarding")).toEqual([]);
+    // A used reference definition renders a link: full, collapsed and shortcut forms.
+    expect(renderedMarkdownLinks("Open [Create organization][setup].\n\n[setup]: /onboarding")).toEqual(["/onboarding"]);
+    expect(renderedMarkdownLinks("Open [setup][] now.\n[Setup]: </onboarding>")).toEqual(["/onboarding"]);
+    expect(renderedMarkdownLinks("Open [setup].\n[setup]: /onboarding")).toEqual(["/onboarding"]);
+    expect(renderedMarkdownLinks("Open [Create organization][missing].\n[setup]: /onboarding")).toEqual([]);
+    expect(renderedMarkdownLinks("![logo](/logo.png)")).toEqual([]);
+  });
+});
@@ -15,6 +15,9 @@ export interface TurnResponseCard {
  text: string;
  /** The exact record a confirmation card asks the person to review, when it names one. */
  target?: Record<string, unknown>;
+  /** Markdown link targets in the card fields the page renders as Markdown (a question form's description and
+   * question prompts; a confirmation's prompt and details). Option labels and help text render as plain text. */
+  links: string[];
 }

 interface RunLike { id: string; agentId?: string; status?: string; resultJson?: Record<string, unknown> | null }
@@ -39,6 +42,21 @@ function shownQuestionForm(payload: Row | undefined): Row | undefined {
  return questionSet !== undefined ? { ...rest, questionSet } : { ...rest, questions };
 }

+/** Link targets the card's Markdown renders as clickable links: inline links, autolinks and reference links whose
+ * definition exists, outside code. Code spans and fenced blocks show their text literally, and a reference
+ * definition that no link uses renders nothing. */
+export function renderedMarkdownLinks(text: string): string[] {
+  const prose = text.replace(/^(```|~~~)[^\n]*\n[\s\S]*?^\1[^\n]*$/gm, "").replace(/(`+)[\s\S]*?\1/g, "");
+  const unwrap = (target: string) => target.startsWith("<") && target.endsWith(">") ? target.slice(1, -1) : target;
+  const label = (value: string) => value.trim().replace(/\s+/g, " ").toLowerCase();
+  const definitions = new Map<string, string>();
+  for (const match of prose.matchAll(/^[ \t]{0,3}\[([^\]\n]+)\]:[ \t]*(\S+)/gm)) if (!definitions.has(label(match[1]!))) definitions.set(label(match[1]!), unwrap(match[2]!));
+  const body = prose.replace(/^[ \t]{0,3}\[[^\]\n]+\]:[^\n]*$/gm, "");
+  const references = [...body.matchAll(/(?<![!\]])\[([^\]\n]+)\](?:\[([^\]\n]*)\])?(?!\()/g)]
+    .map(match => definitions.get(label(match[2] || match[1]!))).filter((target): target is string => !!target);
+  return [...[...body.matchAll(/(?<!!)\[[^\]\n]*\]\(\s*([^)\s]+)/g)].map(match => unwrap(match[1]!)),
+    ...[...body.matchAll(/<((?:https?:\/\/|\/)[^>\s]+)>/g)].map(match => match[1]!), ...references];
+}
 /** True when the product decided that a card owns this run's conversation response. */
 export function runEndedOnCard(run: RunLike): boolean {
  const decision = record(run.resultJson?.presentationDecision);
@@ -66,11 +84,17 @@ export function turnResponseCards(input: { companyId: string; leadId: string; ru
      const answerable = parsed.data.questions.filter(question => !!question.prompt.trim()
        && (question.options.length >= 2 || question.options.some(option => option.freeText)));
      if (!answerable.length) continue;
-      cards.push({ id: row.id, kind: "ask_user_questions", runId: row.sourceRunId, text: strings(row.title, row.summary, ...displayedText(shownQuestionForm(payload))).join("\n") });
+      // The shown form is the canonical questionSet when present, else the legacy questions; both render each
+      // prompt with Markdown, and the questionSet also renders its description.
+      const shown = shownQuestionForm(payload), form = record(shown?.questionSet);
+      const prompts = [form?.questions, shown?.questions].flatMap(list => Array.isArray(list) ? list.map(question => record(question)?.prompt) : []);
+      cards.push({ id: row.id, kind: "ask_user_questions", runId: row.sourceRunId, text: strings(row.title, row.summary, ...displayedText(shown)).join("\n"),
+        links: strings(form?.description, ...prompts).flatMap(renderedMarkdownLinks) });
    } else if (row.kind === "request_confirmation") {
      if (!payload || payload.toolAction || payload.secretProposal || payload.connectionAuthorization || typeof payload.prompt !== "string" || !payload.prompt.trim()) continue;
      cards.push({ id: row.id, kind: "request_confirmation", runId: row.sourceRunId,
        text: strings(row.title, row.summary, payload.prompt, payload.detailsMarkdown, payload.acceptLabel, payload.rejectLabel, payload.declineReasonPlaceholder).join("\n"),
+        links: strings(payload.prompt, payload.detailsMarkdown).flatMap(renderedMarkdownLinks),
        ...(record(payload.target) ? { target: record(payload.target) } : {}) });
    }
  }
@@ -81,3 +105,7 @@ export function turnResponseCards(input: { companyId: string; leadId: string; ru
 export function replyOrCardText(reply: string, cards: readonly TurnResponseCard[]): string {
  return [reply, ...cards.map(card => card.text)].filter(text => !!text.trim()).join("\n");
 }
+/** The rendered reply links plus the links the response cards render, as one list for link checks. */
+export function replyOrCardLinks(links: readonly string[], cards: readonly TurnResponseCard[]): string[] {
+  return [...new Set([...links, ...cards.flatMap(card => card.links)])];
+}
@@ -18,7 +18,7 @@ vi.mock("./chat-catalog-registry.js", () => ({
 }));
 vi.mock("./chat-catalog-fingerprint.js", () => ({ chatCatalogCaseFingerprint: () => ({}), normalizedChatCatalogExecution: () => ({}) }));
 vi.mock("./chat-query-bindings.js", () => ({ resolveChatCatalogQuery: async () => { if (control.completeQuery) return { renderedQuery: "Original request" }; throw new Error("Original attempt failed"); } }));
-vi.mock("./chat-rendered-links.js", () => ({ collectConversationLinks: async () => [] }));
+vi.mock("./chat-rendered-links.js", () => ({ collectConversationLinks: async () => [], collectConversationLinksWithDiagnostics: async () => ({ links: [] }) }));
 vi.mock("./chat-catalog-run-policy.js", () => ({ gradeChatCatalogRuns: () => ({ checks: [] }), CHAT_CATALOG_MAX_SUPERSEDED_QUEUED_RUNS: 1,
  chatCatalogProviderRuns: (runs: unknown[]) => runs, chatCatalogSupersededQueuedRuns: () => [], chatCatalogReassignedQueuedRun: () => false }));
 vi.mock("./chat-flow.js", () => ({ sendChatMessage: async () => {}, collectChatRunEvidence: async () => { control.enter(); if (control.stall) await control.collectorRelease; return {}; } }));
@@ -1176,6 +1176,12 @@
          "field": "expectedOutcome",
          "previous": "The response identifies the supported instance setting and operator path without granting the agent administration rights. The human journey verifies saved configuration and actual route availability. Unsupported deployment choices or a disabled gate produce clear state instead of an agent claiming a successful setting change.",
          "current": "The agent cannot change instance settings, so it leaves the setting unchanged and does not claim it turned Cases on. The reply identifies the Cases experimental setting and links the Experimental settings page where an instance admin turns it on. The person follows that link and turns Cases on. While the setting is off the Cases page is unavailable; once it is on, the saved setting persists and the Cases page opens."
+        },
+        {
+          "key": "questions-and-approvals.necessary-question",
+          "field": "expectedOutcome",
+          "previous": "The agent asks a short, necessary question for the missing product name using the supported interaction. The person can enter a free-text answer; it is saved and delivered to the right work once. The agent uses the answer in the resulting brief without making the person fill in incidental implementation settings.",
+          "current": "The agent asks a short, necessary question for the missing product name using the supported interaction. The person can enter a free-text answer; it is saved and delivered to the right work once. The answer shapes the resulting work: the agent uses the exact answer in that work's saved brief or saved deliverable without making the person fill in incidental implementation settings."
        }
      ],
      "candidateDescriptorSnapshot": "fixtures/chat-repair-candidate-descriptors.json",
@@ -10323,7 +10329,7 @@
      "externalInput": null,
      "installedSource": null,
      "originalQuery": "Build a landing page for my new product.",
-      "expectedOutcome": "The agent asks a short, necessary question for the missing product name using the supported interaction. The person can enter a free-text answer; it is saved and delivered to the right work once. The agent uses the answer in the resulting brief without making the person fill in incidental implementation settings.",
+      "expectedOutcome": "The agent asks a short, necessary question for the missing product name using the supported interaction. The person can enter a free-text answer; it is saved and delivered to the right work once. The answer shapes the resulting work: the agent uses the exact answer in that work's saved brief or saved deliverable without making the person fill in incidental implementation settings.",
      "originalContext": "A requested site needs a product name that is absent from the brief and history.",
      "capabilities": [
        "questions-and-approvals/questions",
@@ -440,6 +440,11 @@ test('repair source metadata binds the corrected candidate and explicit changed
  [
    "instance-operations.instance-settings",
    "expectedOutcome"
+  ],
+  // Decision after full cohort 4: the answer shapes the resulting work, in its saved brief or saved deliverable.
+  [
+    "questions-and-approvals.necessary-question",
+    "expectedOutcome"
  ]
 ]);
  for (const change of repair.changedFields) {
@@ -1,7 +1,7 @@
 // @vitest-environment jsdom
 import { readFileSync } from "node:fs";
 import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
-import { collectConversationLinks, conversationLinkBindings, mislabeledIssueLinks, readConversationDomLinks, readLatestConversationReplyRender, waitForLatestConversationReplyRender, waitForSavedTurnReplyRender, type ConversationLinkInput } from "./chat-rendered-links.js";
+import { collectConversationLinks, collectConversationLinksWithDiagnostics, conversationLinkBindings, mislabeledIssueLinks, readConversationDomLinks, readLatestConversationReplyRender, waitForLatestConversationReplyRender, waitForSavedTurnReplyRender, type ConversationLinkInput } from "./chat-rendered-links.js";

 const card = '<div data-thread-item-kind="project_created" data-thread-anchor="project-created:project"><article aria-label="Project created: Launch"><div><p>Project created</p><a href="/RUN/projects/project">Launch</a></div></article></div>';
 const thread = (content = card) => `<div data-testid="task-chat-thread"><div data-testid="task-chat-history-content"><div data-testid="task-chat-scroller"><div id="comment-human" data-thread-item-kind="human">Create a project called Launch for the new website.</div>${content}<div id="comment-reply" data-thread-item-kind="agent">Created the Launch project. No repositories are connected yet.</div></div></div></div>`;
@@ -157,3 +157,23 @@ describe("task links whose visible identifier names another task", () => {
    expect(mislabeledIssueLinks("[COM-2](/COM/issues/task-uuid)", id => id === "task-uuid" ? "COM-1" : undefined)).toHaveLength(1);
  });
 });
+
+// Full cohort 4 (case-history, goals.explain-purpose, preview-service): a phase capture that saved no links could
+// not say whether the thread was closed, showed another conversation, or rendered the reply without anchors.
+describe("records thread and anchor diagnostics when a saved reply yields no rendered links", () => {
+  const page = () => ({ evaluate: async <T, A>(read: (arg: A) => T, arg: A) => read(arg) }) as unknown as Parameters<typeof collectConversationLinks>[0];
+  it("reports the rendered thread, its saved comment anchors and the reply's anchor count", async () => {
+    const value = input(); value.projects = []; value.activity = [];
+    vi.spyOn(console, "info").mockImplementation(() => undefined);
+    const result = await collectConversationLinksWithDiagnostics(page(), value);
+    expect(result.links).toEqual([]);
+    expect(result.diagnostics).toEqual({ render: "rendered", expectedReplyIds: ["reply"], savedCommentCount: 2,
+      threads: [{ testId: "task-chat-thread", visible: true, renderedCommentIds: ["human", "reply"], foreignCommentIds: [], replyAnchorCounts: { reply: 0 } }] });
+  });
+  it("adds no diagnostics when links were read", async () => {
+    document.querySelector('#comment-reply')!.innerHTML = '<a href="/RUN/issues/task">Task</a>';
+    const value = input(); value.projects = []; value.activity = [];
+    const result = await collectConversationLinksWithDiagnostics(page(), value);
+    expect(result.links.map(row => row.href)).toEqual(["/RUN/issues/task"]); expect(result.diagnostics).toBeUndefined();
+  });
+});
@@ -174,9 +174,35 @@ export async function waitForSavedTurnReplyRender(page: Pick<Page, "evaluate">,
  }
 }
 export async function collectConversationLinks(page: Page, input: ConversationLinkInput) {
+  return (await collectConversationLinksWithDiagnostics(page, input)).links;
+}
+/** What the page showed when a saved reply yielded no rendered links: each candidate thread root, whether it
+ * is visible, which saved comment anchors it renders, and how many anchors each saved reply carries. */
+export interface ConversationLinkDiagnostics {
+  render: ConversationReplyRender; expectedReplyIds: string[]; savedCommentCount: number;
+  threads: Array<{ testId: string | null; visible: boolean; renderedCommentIds: string[]; foreignCommentIds: string[]; replyAnchorCounts: Record<string, number> }>;
+}
+export function readConversationLinkDiagnostics(input: Pick<ReturnType<typeof conversationLinkBindings>, "anchors" | "replies">): Omit<ConversationLinkDiagnostics, "render"> {
+  const threadSelector = '[data-testid="task-chat-thread"], [data-testid="thread-root"]';
+  return { expectedReplyIds: input.replies, savedCommentCount: input.anchors.length,
+    threads: Array.from(document.querySelectorAll(threadSelector)).map(root => {
+      const ids = Array.from(root.querySelectorAll('[id^="comment-"]')).filter(node => node.closest(threadSelector) === root).map(node => node.id.slice("comment-".length));
+      return { testId: root.getAttribute("data-testid"), visible: root.getClientRects().length > 0,
+        renderedCommentIds: ids.filter(id => input.anchors.includes(id)), foreignCommentIds: ids.filter(id => !input.anchors.includes(id)),
+        replyAnchorCounts: Object.fromEntries(input.replies.map(id => [id, root.querySelector(`[id="comment-${id}"]`)?.querySelectorAll("a[href]").length ?? 0])) };
+    }) };
+}
+export async function collectConversationLinksWithDiagnostics(page: Page, input: ConversationLinkInput): Promise<{ links: RenderedConversationLink[]; diagnostics?: ConversationLinkDiagnostics }> {
  const bindings = conversationLinkBindings(input);
-  await waitForLatestConversationReplyRender(page, bindings);
-  return page.evaluate(readConversationDomLinks, bindings);
+  const render = await waitForLatestConversationReplyRender(page, bindings);
+  const links = await page.evaluate(readConversationDomLinks, bindings);
+  if (links.length || !bindings.replies.length) return { links };
+  let diagnostics: ConversationLinkDiagnostics;
+  try { diagnostics = { render, ...await page.evaluate(readConversationLinkDiagnostics, { anchors: bindings.anchors, replies: bindings.replies }) }; }
+  catch { diagnostics = { render, expectedReplyIds: bindings.replies, savedCommentCount: bindings.anchors.length, threads: [] }; }
+  // Logged so an empty capture says whether the thread was closed, another conversation, or the reply had no anchors yet.
+  console.info(`[chat-rendered-links] no rendered reply links: ${JSON.stringify(diagnostics)}`);
+  return { links, diagnostics };
 }
 export function conversationLinkLocator(page: Page, link: RenderedConversationLink): Locator {
  const thread = page.getByTestId(link.threadTestId).filter({ has: page.locator(`[id=${JSON.stringify(`comment-${link.anchorCommentId}`)}]`) });
@@ -15,8 +15,9 @@ scenarios:
    expected_outcome: >-
      The agent asks a short, necessary question for the missing product name using the
      supported interaction. The person can enter a free-text answer; it is saved and delivered
-      to the right work once. The agent uses the answer in the resulting brief without making
-      the person fill in incidental implementation settings.
+      to the right work once. The answer shapes the resulting work: the agent uses the exact
+      answer in that work's saved brief or saved deliverable without making the person fill in
+      incidental implementation settings.
  - id: questions-and-approvals.multi-part-question
    capabilities:
      - questions-and-approvals/questions