test(evals): land the chat catalog eval harness on master (#19614)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The chat-feature evals measure whether agents complete real chat
requests through the product
> - The chat catalog eval harness and its repairs live on the
`chat-eval/master-candidate` branch, not on master
> - Today's work made 139 of 242 chat evals pass in verified single-case
live runs. Each product fix landed on master as its own pull request
> - The eval-side changes (setups, graders, fault seams, evidence
capture) still exist only on the candidate
> - This pull request lands the harness on master. Master stays the
source of truth for product behavior
> - The benefit is one tree: evals run against master commits, and
product fixes reach the evals as soon as they merge

## Linked Issues or Issue Description

Refs #19441, Refs #19599, Refs #20270, Refs #20334, Refs #20342, Refs
#20344, Refs #20354, Refs #20360, Refs #20373, Refs #20382, Refs #20409

**What existing behavior does this improve?**
The chat catalog eval harness and the product it evaluates live on a
long-running branch outside master. Eval fixes do not reach master, so
master-based eval runs use old setups and graders.

**Subsystem affected**
`tests/runner-e2e` chat catalog harness, plus a small set of product
changes that exist only on the candidate.

**Proposed change**
Land the harness on master. Every product fix from this eval work
already merged to master as its own pull request. Conflicts keep
master's product behavior.

## What Changed

- Most of the diff (808 of 851 files) is under `tests/runner-e2e/`:
chat-catalog domains, scenarios, descriptors, the TestOps manifest,
private evidence tooling, and their tests.
- Graders now check saved state, setup literals and rendered or clicked
links. Many keyword and regex checks on free-form replies are removed.
Each changed grader has a bumped `oracleVersion` and calibration tests
(a valid reply, a reworded valid reply, and broken variants that must
fail).
- The shared link helper `hasLink` no longer treats a list link (for
example `/COM/issues?view=mine`) as a link to every task without a URL
key.
- Setup fixes build real state through supported product routes: routine
history, agent grants through the board grant route, budget holds with a
real cap, and a storage-layer archive fault for
`inbox-search.archive-partial-failure`.
- Browser boundary journeys (sign-in, chat-disabled, onboarding) now
write the standard pass evidence files.
- Retargeted cases, approved by the maintainer:
`agent-chat.chat-disabled` (Conference Room gate),
`agents.restore-instructions` (no saved instruction history),
`access.manage-membership` (member permission edits stay board-only).
- The private evidence trusted files (`manual-private-handoff.ts`,
`manual-private-retention.ts`, `manual-private-report-assets.ts`) are
byte-identical to master.
- Product files outside `tests/` (43 files, +1,328 −49) carry
candidate-only changes that master does not yet have. Most are tests.
The main code changes are in `request-human-input.ts`,
`heartbeat-run-summary.ts`, `native-completion-feedback.ts`,
`native-run-finalizer.ts`, `openapi.ts`, `secrets.ts` and
`IssueChatLongThreadPerf.tsx`.

## Verification

- `pnpm vitest run chat-catalog-domains chat-catalog-contract.test.ts
chat-catalog-consistency.test.ts unit-launcher.test.mjs
--testTimeout=180000` in `tests/runner-e2e`: 269 files pass, 3,550 tests
pass.
- `node --test chat-full-catalog-testops-manifest.test.mjs`: 10 pass.
- `node tests/runner-e2e/source-sdk-public-inputs.mjs <HEAD> --check`:
no change.
- `tsc` for `server` and `tests/runner-e2e`: clean.
- Live single-case verification: 139 of 242 cases passed on paid runs,
and a person read the saved evidence for each. These are single-case
runs, not one full-cohort run.

## Risks

- Large diff. Most of it is test-only, but the 43 non-test files change
product code paths. Review those files first.
- Some graders are now less strict on reply wording. A person must read
replies for meaning on those cases.
- 42 cases need disposable outside accounts (Notion, Slack, GitHub,
Daytona and others) and stay blocked. `access.invite-teammate` stays
failing until local agent isolation lands, because the runner's durable
state keeps a live invite token.
- The paid workflow compares the trusted private-evidence files with
master. They are unchanged, so this risk is low.

## Model Used

Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code
execution, in Claude Code. Subagents used the same model.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Paperclip <noreply@paperclip.ing>

server/src/__tests__/chat-completion-proof.test.ts

@@ -104,14 +104,15 @@ describe("delegated chat completion provider accounting", () => {
     const workspaceOperations = [{ companyId: "company", heartbeatRunId: input.notification.id,
       phase: "workspace_finalize", status: "succeeded", command: null }];
     const requested: string[] = [];
-    const api = { async get<T>(path: string): Promise<T> {
+    async function read<T>(path: string): Promise<T> {
       requested.push(path);
       const url = new URL(path, "http://fixture.invalid");
       if (url.pathname.endsWith("/workspace-operations")) return workspaceOperations as T;
       if (url.pathname.endsWith("/log")) return { content: "recorded completion notification" } as T;
       if (url.pathname.endsWith("/events")) return recorded.events.filter(e => e.seq > Number(url.searchParams.get("afterSeq"))) as T;
       throw new Error("Unexpected evidence endpoint");
-    } };
+    }
+    const api = { get: read, getRunObservation: read };
     const collected = await collectChatRunEvidence(api, input.notification, false, true);
     expect(verifyChatCompletion(input.runs, { ...input.completionEvidence, runEvidence: [collected] }))
       .toMatchObject({ ordinaryProviderRuns: 2, totalProviderRuns: 3 });

server/src/__tests__/heartbeat-run-summary.test.ts

@@ -387,6 +387,67 @@ describe("resolveHeartbeatRunResponse", () => {
     },
   };
 
+  it.each([
+    "Handling the request to set your personal publishing credential. I will verify which piece is missing (definition vs. value) and direct you to the secure entry surface; personal secret values cannot be entered or read through chat.",
+    "Checking the configuration.", "Looking at the request.", "Fetching the current state.",
+    "I will check again tomorrow; the release is ready at /releases/latest.",
+  ])("delivers the native conversation final without classifying the earlier comment: %s", body => {
+    const summary = "Open Settings → Secrets → My secrets. Select the publishing definition and Set my value. Keep the value out of chat.";
+    expect(resolveHeartbeatRunResponse({
+      resultJson: { nativeResult: { schema: "paperclip.run_result.v1", reportedWorkDisposition: "done", summary } },
+      nativeConversationFinalResponse: true, existingComment: { id: "earlier", body },
+    })).toMatchObject({ text: summary, decision: { chosenSource: "semantic_result_summary", commentAction: "create",
+      reasonCodes: ["accepted_semantic_result_summary", "native_conversation_final_precedence"] } });
+    expect(resolveHeartbeatRunResponse({ resultJson, existingComment: { id: "earlier", body } }))
+      .toMatchObject({ text: body, decision: { commentAction: "reuse", commentId: "earlier" } });
+  });
+
+  it.each(["done", "needs_review", "blocked", "failed"])("presents an accepted %s conversation response and reuses identical text", reportedWorkDisposition => {
+    const summary = "The requested work needs your attention.";
+    const resultJson = { nativeResult: { schema: "paperclip.run_result.v1", reportedWorkDisposition, summary } };
+    expect(resolveHeartbeatRunResponse({ resultJson, nativeConversationFinalResponse: true,
+      existingComment: { id: "earlier", body: "I will check." } })).toMatchObject({ text: summary, decision: { commentAction: "create" } });
+    expect(resolveHeartbeatRunResponse({ resultJson, nativeConversationFinalResponse: true,
+      existingComment: { id: "same", body: summary } })).toMatchObject({ text: summary, decision: { commentAction: "reuse", commentId: "same" } });
+  });
+
+  it("prefers the completed provider final over the conversation semantic summary", () => {
+    expect(resolveHeartbeatRunResponse({ resultJson, nativeConversationFinalResponse: true,
+      existingComment: { id: "earlier", body: "Earlier comment" },
+      finalAgentMessage: { text: "The complete answer", sourceEventId: "final", channel: "final" },
+    })).toMatchObject({ text: "The complete answer", decision: { chosenSource: "final_agent_message", commentAction: "create" } });
+  });
+
+  it("does not let compatibility narration outrank an accepted conversation result", () => {
+    const resultJson = { nativeResult: { schema: "paperclip.run_result.v1", reportedWorkDisposition: "blocked", summary: "The accepted handoff" },
+      finalResponse: { final: true, text: "Adapter compatibility prose" }, summary: "Legacy compatibility prose" };
+    expect(resolveHeartbeatRunResponse({ resultJson, nativeConversationFinalResponse: true,
+      existingComment: { id: "earlier", body: "Earlier comment" },
+      finalAgentMessage: { text: "Unknown-channel narration", sourceEventId: "unknown", channel: "unknown" },
+    })).toMatchObject({ text: "The accepted handoff", decision: { chosenSource: "semantic_result_summary" } });
+  });
+
+  it("keeps the explicit conversation comment when no qualified final exists", () => {
+    for (const resultJson of [{}, { summary: "Unqualified legacy summary" }]) {
+      expect(resolveHeartbeatRunResponse({ resultJson, nativeConversationFinalResponse: true,
+        existingComment: { id: "explicit", body: "The release is ready at /releases/latest." },
+      })).toMatchObject({ text: "The release is ready at /releases/latest.", decision: { commentAction: "reuse" } });
+      expect(resolveHeartbeatRunResponse({ resultJson, nativeConversationFinalResponse: true,
+        finalAgentMessage: { text: "Unknown narration", sourceEventId: "unknown", channel: "unknown" },
+      })).toMatchObject({ text: null, decision: { commentAction: "none" } });
+    }
+  });
+
+  it("keeps a governed interaction in control of the conversation response", () => {
+    for (const input of [{ resultJson, interactionOwnsResponse: true },
+      { resultJson: { nativeResult: { schema: "paperclip.run_result.v1", reportedWorkDisposition: "yielded", summary: "Waiting for an answer" } } }]) {
+      expect(resolveHeartbeatRunResponse({ ...input, nativeConversationFinalResponse: true,
+        existingComment: { id: "earlier", body: "I will check." },
+        finalAgentMessage: { text: "Do not publish this", sourceEventId: "final", channel: "final" },
+      })).toMatchObject({ text: null, decision: { commentAction: "none" } });
+    }
+  });
+
   describe("run-log previews of the final message", () => {
     const complete = "**Open work — 501 tasks**\n\n### Morgan\n" +
       Array.from({ length: 501 }, (_, i) => `RUN-${i + 1}`).join(", ");
@@ -609,6 +670,22 @@ describe("resolveHeartbeatRunResponse", () => {
     });
   });
 
+  it.each([
+    { conversationTurnFinished: true, interactionOwnsResponse: false, channel: "final" as const, visible: true },
+    { conversationTurnFinished: true, interactionOwnsResponse: false, channel: "unknown" as const, visible: true },
+    { conversationTurnFinished: false, interactionOwnsResponse: false, channel: "unknown" as const, visible: false },
+    { conversationTurnFinished: true, interactionOwnsResponse: true, channel: "unknown" as const, visible: false },
+  ])("preserves server-finished conversation authority for a yielded result: %j", ({ visible, ...authority }) => {
+    const { channel, ...flags } = authority;
+    const resolved = resolveHeartbeatRunResponse({ ...flags, nativeConversationFinalResponse: true,
+      resultJson: { summary: "Unqualified legacy summary", nativeResult: { schema: "paperclip.run_result.v1",
+        reportedWorkDisposition: "yielded", summary: "Waiting for the next message." } },
+      finalAgentMessage: { text: "Which project should own this?", sourceEventId: "chat-final-1", channel },
+    });
+    expect(resolved.text).toBe(visible ? "Which project should own this?" : null);
+    expect(resolved.decision.commentAction).toBe(visible ? "create" : "none");
+  });
+
   it("persists a final assistant reply when the server completed a conversation turn", () => {
     expect(resolveHeartbeatRunResponse({
       conversationTurnFinished: true,

server/src/__tests__/issue-thread-interaction-routes.test.ts

@@ -1,4 +1,5 @@
 import express from "express";
+import { probeReviewApproval } from "../../../tests/runner-e2e/chat-catalog-domains/external-app-review-probe.js";
 import request from "supertest";
 import { beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
 import type { TaskWatchdogMutationScope } from "../services/task-watchdog-scope.js";
@@ -50,6 +51,9 @@ const mockInteractionService = vi.hoisted(() => ({
   recordSecretProposalExecutionResult: vi.fn(),
 }));
 
+const mockSecretProposals = vi.hoisted(() => ({ getById: vi.fn(), approve: vi.fn() }));
+vi.mock("../services/secret-proposals.js", () => ({ createSecretProposalsService: () => mockSecretProposals }));
+
 const mockHeartbeatService = vi.hoisted(() => ({
   wakeup: vi.fn(async (_agentId: string, _options: { payload?: unknown; contextSnapshot?: unknown }) => undefined),
   cancelRun: vi.fn(async () => null),
@@ -1705,6 +1709,32 @@ describe("issue thread interaction routes", () => {
     );
   });
 
+  it("uses only the bound continuation for the production secret approval fallback", async () => {
+    const proposalId = "44444444-4444-4444-8444-444444444444";
+    mockSecretProposals.getById.mockResolvedValue({ id: proposalId, kind: "binding", originIssueId: ISSUE_ID,
+      interactionId: "interaction-secret-fallback", configPath: "env.DEPLOYMENT_API_KEY" });
+    mockSecretProposals.approve.mockResolvedValue({ status: "approved" });
+    mockInteractionService.acceptInteraction.mockResolvedValueOnce({ interaction: {
+      id: "interaction-secret-fallback", companyId: "company-1", issueId: ISSUE_ID,
+      kind: "request_confirmation", status: "accepted", continuationPolicy: "wake_assignee",
+      sourceRunId: RUN_1, payload: { version: 1, prompt: "Create binding?", secretProposal: {
+        version: 1, proposalId, configPath: "env.DEPLOYMENT_API_KEY" } },
+      result: { version: 1, outcome: "accepted" },
+    }, createdIssues: [] });
+    const app = await createApp();
+    const res = await request(app).post(`/api/issues/${ISSUE_ID}/interactions/interaction-secret-fallback/accept`).send({});
+    expect(res.status).toBe(200);
+    expect(mockSecretProposals.approve).toHaveBeenCalledWith("company-1", proposalId,
+      expect.objectContaining({ resolvedByUserId: "local-board" }));
+    expect(res.body.result.secretProposal.status).toBe("executed");
+    expect.soft(mockIssueService.addComment).not.toHaveBeenCalled();
+    expect.soft(mockHeartbeatService.wakeup).toHaveBeenCalledTimes(1);
+    expect(mockHeartbeatService.wakeup).toHaveBeenCalledWith(ASSIGNEE_AGENT_ID, expect.objectContaining({
+      contextSnapshot: expect.objectContaining({ issueId: ISSUE_ID, interactionId: "interaction-secret-fallback" }),
+      payload: expect.objectContaining({ secretProposal: expect.objectContaining({ proposalId, executionStatus: "executed" }) }),
+    }));
+  });
+
   it("records a failed secret-proposal execution and posts a thread comment", async () => {
     const proposalId = "55555555-5555-4555-8555-555555555555";
     const approveSecretProposal = vi.fn().mockRejectedValue(new Error("binding failed"));
@@ -3244,6 +3274,42 @@ describe("issue thread interaction routes", () => {
     expect(missingRun.body).toMatchObject({ code: "interaction_run_attribution_required" });
   });
 
+  it.each([false, true])("the catalog approval probe distinguishes missing attribution from governed-action denial (drop header: %s)", async dropHeader => {
+    const approveToolActionRequest = vi.fn();
+    const revokeKey = vi.fn(async () => undefined);
+    mockIssueService.getById.mockResolvedValue(createIssue({ status: "todo" }));
+    mockInteractionService.getForIssue.mockResolvedValue({
+      id: "interaction-tool-probe", companyId: "company-1", issueId: ISSUE_ID,
+      kind: "request_confirmation", status: "pending", createdByAgentId: CREATED_AGENT_ID,
+      sourceRunId: RUN_1, effectiveResolverPolicy: "human_only",
+      payload: { version: 1, prompt: "Approve the action?", toolAction: { version: 1, actionRequestId: "action-probe", toolName: "notes_update" } },
+    });
+    const receipt = await probeReviewApproval({
+      baseURL: "https://fixture.example", companyId: "company-1", agentId: CREATED_AGENT_ID, issueId: ISSUE_ID,
+      interaction: { id: "interaction-tool-probe", status: "pending", sourceRunId: RUN_1 },
+      run: { id: RUN_1, companyId: "company-1", agentId: CREATED_AGENT_ID, status: "succeeded", contextSnapshot: { issueId: ISSUE_ID } },
+      invocation: { companyId: "company-1", agentId: CREATED_AGENT_ID, issueId: ISSUE_ID, runId: RUN_1, status: "awaiting_approval" },
+      privateSecrets: [], createKey: async () => ({ id: "probe-key", token: "synthetic-test-key" }), revokeKey,
+      boundary: async () => ({ interactionStatus: "pending", invocationStatus: "awaiting_approval", actionStatus: "pending", providerWrites: approveToolActionRequest.mock.calls.length, trustRuleIds: [] }),
+      send: async (url, options) => {
+        const headers = new Headers(options?.headers);
+        const actorRun = dropHeader ? undefined : headers.get("x-paperclip-run-id");
+        const app = await createApp({ type: "agent", companyId: "company-1", agentId: CREATED_AGENT_ID, runId: actorRun }, { approveToolActionRequest });
+        const res = await request(app).post(new URL(String(url)).pathname)
+          .set("Authorization", headers.get("authorization")!).send(JSON.parse(options?.body as string));
+        return Response.json(res.body, { status: res.status });
+      },
+    });
+    expect(receipt).toEqual({ status: dropHeader ? 422 : 403,
+      code: dropHeader ? "interaction_run_attribution_required" : "interaction_governed_action_denied",
+      boundaryUnchanged: true, keyRevoked: true });
+    expect(approveToolActionRequest).not.toHaveBeenCalled();
+    expect(mockInteractionService.acceptInteraction).not.toHaveBeenCalled();
+    expect(mockHeartbeatService.wakeup).not.toHaveBeenCalled();
+    expect(mockLogActivity).not.toHaveBeenCalled();
+    expect(revokeKey).toHaveBeenCalledExactlyOnceWith("probe-key");
+  });
+
   it("rejects an invalid run attribution before resolving or waking", async () => {
     mockInteractionService.getForIssue.mockResolvedValueOnce({
       id: "interaction-invalid-run",

server/src/__tests__/secrets-routes.test.ts

@@ -155,6 +155,74 @@ describe("secret routes", () => {
     });
   });
 
+  it.each([undefined, "", "agent-supplied-value"])("denies agent secret creation before validating value %j", async (value) => {
+    const res = await request(createApp({
+      type: "agent", agentId: "agent-1", companyId: "company-1", source: "agent_jwt",
+    })).post("/api/companies/company-1/secrets").send({ name: "Deployment key", value });
+
+    expect(res.status).toBe(403);
+    expect(res.body.error).toContain("Settings → Secrets");
+    expect(res.body.error).toContain("human");
+    expect(res.body.error).toContain("Do not send secret values in chat");
+    expect(res.body).not.toHaveProperty("details");
+    expect(mockSecretService.create).not.toHaveBeenCalled();
+    expect(mockLogActivity).not.toHaveBeenCalled();
+  });
+
+  it("gives a denied agent safe response and saved-wait options without echoing the submitted value", async () => {
+    const value = "private-denial-canary-value";
+    const res = await request(createApp({
+      type: "agent", agentId: "agent-1", companyId: "company-1", source: "agent_jwt",
+    })).post("/api/companies/company-1/secrets").send({ name: "Deployment key", value });
+
+    expect(res.status).toBe(403);
+    expect(res.body.error).toContain("finish the current conversation response");
+    expect(res.body.error).toContain("without claiming that credential setup is complete");
+    expect(res.body.error).toContain("first create a non-governing secure-save acknowledgment");
+    expect(res.body.error).toContain("request_human_input");
+    expect(res.body.error).toContain('interactionKind="confirmation"');
+    expect(res.body.error).not.toContain("using request_confirmation");
+    expect(res.body.error).toContain("never ask for the secret value in that card");
+    expect(res.body.error).toContain("Wait only on the actual saved request");
+    expect(JSON.stringify(res.body)).not.toContain(value);
+    expect(res.body).not.toHaveProperty("details");
+    expect(mockSecretService.create).not.toHaveBeenCalled();
+    expect(mockLogActivity).not.toHaveBeenCalled();
+  });
+
+  it.each([undefined, ""])("validates a board caller's missing or empty managed value %j", async (value) => {
+    const res = await request(createApp()).post("/api/companies/company-1/secrets")
+      .send({ name: "Deployment key", value });
+
+    expect(res.status).toBe(400);
+    expect(res.body.details).toEqual(expect.arrayContaining([expect.objectContaining({ path: ["value"] })]));
+    expect(mockSecretService.create).not.toHaveBeenCalled();
+    expect(mockLogActivity).not.toHaveBeenCalled();
+  });
+
+  it("keeps authorized board creation and its human activity attribution", async () => {
+    mockSecretService.create.mockResolvedValue({ id: "secret-1", name: "Deployment key", provider: "local_encrypted" });
+    const res = await request(createApp()).post("/api/companies/company-1/secrets")
+      .send({ name: "Deployment key", value: "board-supplied-value" });
+
+    expect(res.status).toBe(201);
+    expect(mockSecretService.create).toHaveBeenCalledWith("company-1",
+      expect.objectContaining({ name: "Deployment key", value: "board-supplied-value" }),
+      { userId: "user-1", agentId: null });
+    expect(mockLogActivity).toHaveBeenCalledWith(expect.anything(), expect.objectContaining({
+      companyId: "company-1", actorType: "user", actorId: "user-1", action: "secret.created", entityId: "secret-1",
+    }));
+  });
+
+  it.each([undefined, "board-supplied-value"])("denies cross-company board creation before body validation %j", async (value) => {
+    const res = await request(createApp()).post("/api/companies/company-2/secrets")
+      .send({ name: "Deployment key", value });
+
+    expect(res.status).toBe(403);
+    expect(mockSecretService.create).not.toHaveBeenCalled();
+    expect(mockLogActivity).not.toHaveBeenCalled();
+  });
+
   it("rejects managed secret creation when externalRef is supplied", async () => {
     const res = await request(createApp()).post("/api/companies/company-1/secrets").send({
       name: "OpenAI API Key",

server/src/__tests__/teams-catalog-install-no-overrides.test.ts

@@ -154,4 +154,35 @@ describeEmbeddedPostgres("teams catalog install with no caller adapter overrides
       .map((row) => row.adapterType);
     expect(otherAdapters).toEqual(["paperclip_runner", "paperclip_runner"]);
   });
+  it("derived catalog installs selected native runtime without caller overrides", async () => {
+    const repositoryRoot = path.resolve(import.meta.dirname, "../../..");
+    const temporaryRoot = await fs.mkdtemp(path.join(os.tmpdir(), "team-runtime-derived-pg-"));
+    const oldDir = process.env.PAPERCLIP_TEAMS_CATALOG_DIR, oldDefault = process.env.PAPERCLIP_TEAMS_CATALOG_DEFAULT_ADAPTER_TYPE;
+    try {
+      const helper = await import("../../../tests/runner-e2e/chat-catalog-domains/external-team-runtime-fixture.js");
+      Object.assign(process.env, await helper.teamRuntimeCatalogServerEnvironment({ repositoryRoot, temporaryRoot, executionIds: ["chat-features.runner-opencode.local.teams.install-team"] }));
+      const companyId = await seedEmptyCompany();
+      const { secretService } = await import("../services/secrets.js");
+      const account = await secretService(db).create(companyId, { name: "Derived package account", provider: "local_encrypted", value: "synthetic-model-free-account" });
+      const manifest = JSON.parse(await fs.readFile(path.join(repositoryRoot, "packages/teams-catalog/generated/catalog.json"), "utf8"));
+      const team = manifest.teams.find((row: {slug: string}) => row.slug === "product-engineering");
+      const config = { provider: "opencode", model: "openrouter/deepseek/deepseek-v4.1-flash-1017", timeoutSec: 0, lifecycleMode: "per_turn", opencodePermissionMode: "allow", env: { OPENROUTER_API_KEY: { type: "secret_ref", secretId: account.id, version: "latest" } } };
+      const derived = await helper.normalizeTeamRuntimeCatalog({ repositoryRoot, temporaryRoot, team, adapters: Object.fromEntries(team.agentSlugs.map((slug: string) => [slug, { adapterType: "paperclip_runner", adapterConfig: config }])) });
+      vi.resetModules();
+      const { teamsCatalogService: actualService, getCatalogTeamOrThrow } = await import("../services/teams-catalog.js");
+      expect((await getCatalogTeamOrThrow(team.id)).contentHash).toBe(derived.effectiveContentHash);
+      const svc = actualService(db);
+      const preview = await svc.previewCatalogTeamImport(companyId, team.id, {});
+      for (const member of preview.portabilityPreview.manifest.agents) expect(member.adapterConfig).toMatchObject(config);
+      await svc.installCatalogTeam(companyId, team.id, { collisionStrategy: "rename", include: { projects: false, issues: false } });
+      const saved = await db.select().from(agents).where(eq(agents.companyId, companyId));
+      expect(saved).toHaveLength(3);
+      for (const member of saved) { expect(member.adapterType).toBe("paperclip_runner"); expect(member.adapterConfig).toMatchObject(config); }
+    } finally {
+      if (oldDir === undefined) delete process.env.PAPERCLIP_TEAMS_CATALOG_DIR; else process.env.PAPERCLIP_TEAMS_CATALOG_DIR = oldDir;
+      if (oldDefault === undefined) delete process.env.PAPERCLIP_TEAMS_CATALOG_DEFAULT_ADAPTER_TYPE; else process.env.PAPERCLIP_TEAMS_CATALOG_DEFAULT_ADAPTER_TYPE = oldDefault;
+      await fs.rm(temporaryRoot, { recursive: true, force: true });
+    }
+  }, 30_000);
+
 });

server/src/__tests__/teams-catalog-routes.test.ts

@@ -211,6 +211,17 @@ describe("teams catalog routes", () => {
     expect(mockCatalogModule.listCatalogTeams).not.toHaveBeenCalled();
   });
 
+  it("still rejects unknown preview and install fields before service mutation", async () => {
+    const app = await createApp({ type: "agent", agentId: "agent-1", companyId, runId: "run-1" });
+    for (const operation of ["preview", "install"]) {
+      const res = await request(app).post(`/api/companies/${companyId}/teams/catalog/product-engineering/${operation}`)
+        .send({ parentAgentId: "undocumented-field" });
+      expect(res.status).toBe(400);
+    }
+    expect(mockTeamsCatalogService.previewCatalogTeamImport).not.toHaveBeenCalled();
+    expect(mockTeamsCatalogService.installCatalogTeam).not.toHaveBeenCalled();
+  });
+
   it("previews catalog teams with company access and actor/source policy context", async () => {
     const app = await createApp({
       type: "board",