From b7177db6aada0f4317b078e35503418eead0e44e Mon Sep 17 00:00:00 2001
From: Dotta <34892728+cryppadotta@users.noreply.github.com>
Date: Thu, 19 Nov 2026 11:20:01 -0600
Subject: [PATCH] fix(telemetry): report a Paperclip runner agent's underlying
harness as adapter_type (#22324)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The server sends product telemetry. Two events describe agent work:
`agent.task_run` and `agent.task_completed`.
> - Each event has an `adapter_type` dimension. The generated telemetry
contract gives this dimension a fixed enum.
> - A Paperclip runner agent stores `paperclip_runner` as its adapter
type. The enum does not have this value.
> - The collector and Cloud direct delivery change a value that is not
in the enum to `other`.
> - Thus, as more agents use the runner, the adapter breakdown in
analytics becomes mostly `other`. The data is not lost, but it is in one
bucket.
> - This pull request makes a runner agent report the harness that it
runs, for example `claude_local` or `codex_local`.
> - The benefit is that the adapter breakdown shows the real harness
again.
## Linked Issues or Issue Description
Refs #8818 (this PR added the generated telemetry contract and its
`adapter_type` enum).
I did not find an existing issue. This is the bug description:
**What happened?**
`agent.task_run` and `agent.task_completed` send the stored
`agents.adapter_type`. For a Paperclip runner agent, this value is
`paperclip_runner`. The `adapter_type` enum in
`packages/shared/src/telemetry/generated/paperclip-telemetry.ts` does
not include `paperclip_runner`. The telemetry normalizer changes the
value to `other`. In analytics, almost all runner agent events show
`adapter_type = other`.
**Expected behavior**
A runner agent reports the harness that it runs. The server already
resolves this harness with `agentHarnessType()` from
`@paperclipai/shared`. For example, a runner agent with the Claude ACP
configuration must report `claude_local`.
**Steps to reproduce**
1. Create a Paperclip runner agent (`adapterType: "paperclip_runner"`).
2. Let the agent complete a heartbeat run, or let it set a task to
`done`.
3. Look at the `agent.task_run` or `agent.task_completed` event.
`adapter_type` is `paperclip_runner` before delivery, and `other` after
normalization.
**Paperclip version or commit**
`master` at 1355f93546.
**Agent adapter(s) involved**
Paperclip runner (all harnesses).
## What Changed
- `server/src/services/agent-task-run-telemetry.ts`: new
`telemetryAdapterType(agent)` helper. It returns
`agentHarnessType(agent.adapterType, adapterConfig)`. `agent.task_run`
sends this value as `adapterType`.
- `server/src/routes/issues.ts`: `agent.task_completed` calls the same
helper. Thus, the two events cannot report different adapter types for
the same agent.
- A legacy agent (for example `claude_local`) sends the same value as
before. `agentHarnessType()` returns a legacy adapter type without a
change.
- `server/src/__tests__/agent-task-run-telemetry.test.ts`:
- New tests for the default, Claude, and OpenCode runner configurations.
They expect `codex_local`, `claude_local`, and `opencode_local`.
- New contract test. It resolves every runner provider and every ACPX
agent through `telemetryAdapterType()`, and compares the result with the
generated `adapter_type` enum of both events. The test fails when a
runner harness is missing from the contract and is not in the known-gap
list.
- `server/src/__tests__/issue-telemetry-routes.test.ts`: new test that a
runner agent with the Claude ACP configuration sends `claude_local` on
`agent.task_completed`.
- `packages/shared/src/telemetry/README.md`:
- The `agent.task_run` section tells what `adapter_type` means for a
runner agent, and lists the runner harnesses that the contract does not
have yet.
- The `Emission Boundary` section tells why runner harness resolution is
not an alias map. The receiving layer does not get the agent
configuration, so it cannot derive the harness from `paperclip_runner`.
The emitter sends the resolved value raw and does not fall back to
`other`.
- `tests/runner-e2e/source-sdk-public-inputs.json`: refreshed pin for
the changed README. I ran `node
tests/runner-e2e/source-sdk-public-inputs.mjs
--write`.
## Verification
- Red-green: before the fix, the new emitter tests failed with
`paperclip_runner`. After the fix, they pass.
- `cd server && npx vitest run
src/__tests__/agent-task-run-telemetry.test.ts
src/__tests__/issue-telemetry-routes.test.ts
src/__tests__/posthog-direct-event-shape.test.ts` and the three native
tests that mock the telemetry service: 182 tests pass.
- The contract test fails when I remove one known gap (`muse`) from the
list. Then it passes again when I restore the list.
- `cd server && pnpm typecheck`: passes.
- `node tests/runner-e2e/source-sdk-public-inputs.mjs --check`:
`changed: false`.
- Before the rebase, I sent the old and new values through the real
PostHog normalizer. The default, Claude, and OpenCode runner
configurations changed from `other` to `codex_local`, `claude_local`,
and `opencode_local`.
- I did not run the full local suite. `pnpm verify` asks for broader
coverage because the pin registry is a global CI input. CI supplies that
coverage.
## Risks
- Low risk. The change affects only the telemetry dimension value. It
does not change agent behavior, the database, or an API.
- Eight of the fifteen runner harnesses map to enum values:
`codex_local`, `opencode_local`, `claude_local`, `cursor`,
`cursor_cloud`, `grok_local`, `pi_local`, and `kimi_local`.
- Seven runner harnesses continue to show `other`: `acpx:copilot`,
`antigravity_runtime`, `aws_agentcore`, `claude_managed`, `grok_bot`,
`muse`, and `openai_dot`. The fix for these is a change to the contract
in the telemetry backend.
- After this change, `adapter_type` does not show the difference between
a runner agent and a legacy agent with the same harness. A separate
dimension in the telemetry contract can show this difference later.
- Events from before this change keep `other`.
## Privacy Review Request
`rules/data-paths.mdc` requires a privacy review for a telemetry change.
I request a privacy review from a maintainer. This is my analysis:
- The change does not add an event, a dimension, or an enum value. Thus,
the generated contract does not change. The contract comes from the
telemetry backend, and this repository must not edit it.
- The new value comes from `agentHarnessType()`. On each validated write
path (create, hire, PATCH, config rollback, and company import), the
server accepts only a closed set of runner `provider` and `acpxAgent`
values. Thus, the emitted value is a fixed harness identifier, not user
text.
- An agent row that does not pass these checks (for example, a
plugin-managed hire or a direct database write) can hold a different
`provider` string. Before this change, the emitter already sent the raw
`agents.adapter_type` string, which has the same exposure. The receiving
layer changes any value that is not in the enum to `other` before
analytics.
- The value never contains a prompt, a credential, a path, a model
output, or a person's name.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`) from Anthropic, in Claude Code. It
used tool use (shell, file read and edit, `gh`) and extended reasoning.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5
---
packages/shared/src/telemetry/README.md | 18 +++++++++
.../agent-task-run-telemetry.test.ts | 39 ++++++++++++++++++-
.../__tests__/issue-telemetry-routes.test.ts | 28 +++++++++++++
server/src/routes/issues.ts | 4 +-
.../src/services/agent-task-run-telemetry.ts | 8 +++-
.../runner-e2e/source-sdk-public-inputs.json | 2 +-
6 files changed, 93 insertions(+), 6 deletions(-)
diff --git a/packages/shared/src/telemetry/README.md b/packages/shared/src/telemetry/README.md
index 36b4a57d187..41b9c614005 100644
--- a/packages/shared/src/telemetry/README.md
+++ b/packages/shared/src/telemetry/README.md
@@ -53,6 +53,13 @@ to a stable reporting shape.
Do not add client-side lowercasing, alias mapping, or fallback mapping unless
the generated telemetry contract specifically requires that emitted value.
+The `adapter_type` dimension on `agent.task_run` and `agent.task_completed` is
+not an alias map. For a Paperclip runner agent, the server sends the harness
+that `agentHarnessType()` resolves from the agent configuration. The receiving
+layer does not get that configuration, so it cannot derive the harness from
+`paperclip_runner`. The emitter sends the resolved value raw. It does not fall
+back to `other`.
+
If a dimension is privacy-protected before emission, emit only the protected
value and its matching public marker as defined by the typed helper or generated
contract. Do not emit private source material in telemetry dimensions.
@@ -130,6 +137,17 @@ category decides the code only for the generic `adapter_failed` and
run failure code sets in `server/src/services/continuation-cause.ts`, and a
test fails when a code in those sets has no telemetry code.
+The `adapter_type` dimension on `agent.task_run` and `agent.task_completed`
+reports the harness the agent runs, as `agentHarnessType()` resolves it. A
+Paperclip runner agent therefore reports its underlying harness (for example
+`claude_local` or `codex_local`), not `paperclip_runner`. Some runner harnesses
+are not in the generated `adapter_type` enum yet: `acpx:copilot`,
+`antigravity_runtime`, `aws_agentcore`, `claude_managed`, `grok_bot`, `muse`,
+and `openai_dot`. The receiving layer reports them as `other` until the
+contract lists them. A test in
+`server/src/__tests__/agent-task-run-telemetry.test.ts` fails when a runner
+harness that is not in this list is missing from the contract.
+
## Product Lifecycle Events
`model.connected`, `task.created`, `task.completed`, `task.blocked`,
diff --git a/server/src/__tests__/agent-task-run-telemetry.test.ts b/server/src/__tests__/agent-task-run-telemetry.test.ts
index cbce97e41f8..fa762527e9e 100644
--- a/server/src/__tests__/agent-task-run-telemetry.test.ts
+++ b/server/src/__tests__/agent-task-run-telemetry.test.ts
@@ -8,6 +8,8 @@ import {
} from "./helpers/embedded-postgres.js";
import { scopedTelemetryCompanyId, scopedTelemetryPerson } from "../telemetry-company-scope.js";
import { EXECUTION_RECONCILIATION_CAUSES } from "@paperclipai/shared";
+import { PAPERCLIP_ENUM_DESCRIPTIONS } from "@paperclipai/shared/telemetry";
+import { PAPERCLIP_RUNNER_ACPX_PROFILES, PAPERCLIP_RUNNER_PERMISSION_CAPABILITIES } from "@paperclipai/adapter-utils";
import { PERMANENT_FAILURE_CODES, PERSON_OWNED_FAILURE_CODES, RESTART_CLASS_FAILURE_CODES, SETUP_FAILURE_CODES } from "../services/continuation-cause.ts";
const mockTelemetryClient = vi.hoisted(() => ({ track: vi.fn() }));
@@ -27,7 +29,7 @@ vi.mock("@paperclipai/shared/telemetry", async () => {
};
});
-import { agentRunFailureCause, emitAgentTaskRun, emitAgentTaskRunById } from "../services/agent-task-run-telemetry.ts";
+import { agentRunFailureCause, emitAgentTaskRun, emitAgentTaskRunById, telemetryAdapterType } from "../services/agent-task-run-telemetry.ts";
const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport();
const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip;
@@ -376,6 +378,19 @@ describeEmbeddedPostgres("emitAgentTaskRun", () => {
await expect(emitAgentTaskRun(db, run)).resolves.toBeUndefined();
});
+ it.each([
+ { adapterConfig: {}, adapterType: "codex_local" },
+ { adapterConfig: { provider: "acpx", acpxAgent: "claude" }, adapterType: "claude_local" },
+ { adapterConfig: { provider: "opencode" }, adapterType: "opencode_local" },
+ ])("reports a Paperclip runner agent's underlying harness $adapterType as adapterType", async ({ adapterConfig, adapterType }) => {
+ await seedCompanyAndAgent({ adapterType: "paperclip_runner", adapterConfig });
+ const [run] = await db.insert(heartbeatRuns).values({ companyId, agentId, status: "succeeded" }).returning();
+
+ await emitAgentTaskRun(db, run);
+
+ expect(mockTrackAgentTaskRun.mock.calls[0][1].adapterType).toBe(adapterType);
+ });
+
it("omits adapterType and agentRole when the agent row is absent", async () => {
await seedCompanyAndAgent();
const run = {
@@ -454,3 +469,25 @@ describe("agentRunFailureCause", () => {
expect(unclassified).toEqual([]);
});
});
+
+describe("Paperclip runner adapter_type coverage", () => {
+ it.each(["agent.task_run", "agent.task_completed"] as const)("lists every runner harness in the %s adapter_type contract except the known gaps", (event) => {
+ const contract = Object.keys(PAPERCLIP_ENUM_DESCRIPTIONS[event].adapter_type);
+ const harnesses = [
+ ...Object.keys(PAPERCLIP_RUNNER_PERMISSION_CAPABILITIES)
+ .filter((provider) => provider !== "acpx")
+ .map((provider) => telemetryAdapterType({ adapterType: "paperclip_runner", adapterConfig: { provider } })),
+ ...PAPERCLIP_RUNNER_ACPX_PROFILES.map(({ value }) => telemetryAdapterType({ adapterType: "paperclip_runner", adapterConfig: { provider: "acpx", acpxAgent: value } })),
+ ];
+
+ expect(harnesses.filter((harness) => !contract.includes(harness)).sort()).toEqual([
+ "acpx:copilot",
+ "antigravity_runtime",
+ "aws_agentcore",
+ "claude_managed",
+ "grok_bot",
+ "muse",
+ "openai_dot",
+ ]);
+ });
+});
diff --git a/server/src/__tests__/issue-telemetry-routes.test.ts b/server/src/__tests__/issue-telemetry-routes.test.ts
index 04e75132724..a8888f54054 100644
--- a/server/src/__tests__/issue-telemetry-routes.test.ts
+++ b/server/src/__tests__/issue-telemetry-routes.test.ts
@@ -253,6 +253,34 @@ describe("issue telemetry routes", () => {
await vi.waitFor(() => expect(people).toEqual([expected]));
}, 10_000);
+ it("emits a Paperclip runner agent's underlying harness as the adapter type", async () => {
+ mockAgentService.getById.mockResolvedValue({
+ id: "agent-1",
+ companyId: "company-1",
+ role: "engineer",
+ adapterType: "paperclip_runner",
+ adapterConfig: { provider: "acpx", acpxAgent: "claude", model: "claude-sonnet-4-6" },
+ });
+
+ const app = await createApp({
+ type: "agent",
+ agentId: "agent-1",
+ companyId: "company-1",
+ runId: "22222222-2222-4222-8222-222222222222",
+ });
+ const res = await request(app)
+ .patch("/api/issues/11111111-1111-4111-8111-111111111111")
+ .send({ status: "done" });
+
+ expect(res.status).toBe(200);
+ await vi.waitFor(() => {
+ expect(mockTrackAgentTaskCompleted).toHaveBeenCalledWith(
+ expect.anything(),
+ expect.objectContaining({ adapterType: "claude_local", model: "claude-sonnet-4-6" }),
+ );
+ });
+ }, 10_000);
+
it("does not emit agent task-completed telemetry for board-driven completions", async () => {
const app = await createApp({
type: "board",
diff --git a/server/src/routes/issues.ts b/server/src/routes/issues.ts
index 8f1092dacc0..0da962ebb8e 100644
--- a/server/src/routes/issues.ts
+++ b/server/src/routes/issues.ts
@@ -249,7 +249,7 @@ import { ContextCompactionError, contextCompactionService } from "../services/co
import { issueContextUsage } from "../services/issue-context-usage.js";
import { queueLiveRunnerPrpCommand } from "../realtime/runner-prp-ws.js";
import { questionResponseDeliveryService } from "../services/question-response-delivery.js";
-import { emitAgentTaskRunById } from "../services/agent-task-run-telemetry.js";
+import { emitAgentTaskRunById, telemetryAdapterType } from "../services/agent-task-run-telemetry.js";
import {
createAutomaticGoalCommentSteering,
createQueuedCommentQueue,
@@ -16328,7 +16328,7 @@ export function issueRoutes(
withTelemetryActor({ companyId: issue.companyId, person }, () => trackAgentTaskCompleted(tc, {
agentRole: actorAgent.role,
agentId: actorAgent.id,
- adapterType: actorAgent.adapterType,
+ adapterType: telemetryAdapterType(actorAgent),
model,
taskId: issue.id,
}));
diff --git a/server/src/services/agent-task-run-telemetry.ts b/server/src/services/agent-task-run-telemetry.ts
index c05b7dd1111..ceab7157111 100644
--- a/server/src/services/agent-task-run-telemetry.ts
+++ b/server/src/services/agent-task-run-telemetry.ts
@@ -1,6 +1,6 @@
import { and, eq, sql } from "drizzle-orm";
import { agents, heartbeatRuns, issues, type Db } from "@paperclipai/db";
-import { requiresExecutionReconciliation } from "@paperclipai/shared";
+import { agentHarnessType, requiresExecutionReconciliation } from "@paperclipai/shared";
import { PERMANENT_FAILURE_CODES, RESTART_CLASS_FAILURE_CODES, SETUP_FAILURE_CODES } from "./continuation-cause.js";
import { trackAgentTaskRun } from "@paperclipai/shared/telemetry";
import { parseObject } from "../adapters/utils.js";
@@ -143,6 +143,10 @@ export function agentRunFailureCause(input: {
return PLATFORM_FAILURE_CODES.has(code) ? "platform_error" : "unknown";
}
+export function telemetryAdapterType(agent: { adapterType: string; adapterConfig: unknown }): string {
+ return agentHarnessType(agent.adapterType, parseObject(agent.adapterConfig));
+}
+
function providerFailureCategory(resultJson: unknown): string | null {
const providerFailure = parseObject(parseObject(resultJson).providerFailure);
return typeof providerFailure.category === "string" ? providerFailure.category : null;
@@ -243,7 +247,7 @@ export async function emitAgentTaskRun(db: Db, run: TelemetryRun): Promise
agentId: run.agentId,
state: run.status,
...(agent
- ? { adapterType: agent.adapterType, agentRole: agent.role }
+ ? { adapterType: telemetryAdapterType(agent), agentRole: agent.role }
: {}),
...(model ? { model } : {}),
...(() => {
diff --git a/tests/runner-e2e/source-sdk-public-inputs.json b/tests/runner-e2e/source-sdk-public-inputs.json
index ba5e145aece..51c7cdf0aea 100644
--- a/tests/runner-e2e/source-sdk-public-inputs.json
+++ b/tests/runner-e2e/source-sdk-public-inputs.json
@@ -371,7 +371,7 @@
"packages/shared/src/standing-checks.ts": "7ab71e4b6984e41e1c1dfd1bddf0116041d5abf1",
"packages/shared/src/subscriptions.ts": "ef70e63395ca967bbc30776b71560e96eb354a75",
"packages/shared/src/summary-slot.test.ts": "5f8e222dea425f0d5bff0adb5c683087bd7cf7fb",
- "packages/shared/src/telemetry/README.md": "36b4a57d187218ec141256d7f093c96c793ff901",
+ "packages/shared/src/telemetry/README.md": "41b9c614005d53b74b6a4c9c9a55c715e9144653",
"packages/shared/src/telemetry/client-types.test.ts": "3b630edda4972b929ba4decd7457f226eb9cfc5b",
"packages/shared/src/telemetry/client.test.ts": "5a95f2305a393c5b2e7750b10e0ed65da0f30cf8",
"packages/shared/src/telemetry/client.ts": "d6534ce8aaa6f69fb2dd03e12487db024e250a96",