From 2a43fe3e34445a1e55c86700e1e7275de607e0a7 Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Wed, 18 Nov 2026 10:34:39 -0600 Subject: [PATCH] feat(telemetry): add company_id to Cloud direct events from background work (#22183) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Paperclip Cloud sends server telemetry straight to PostHog (`server/src/services/posthog-direct/`) > - Request events get `company_id` from the request context. Background events have no request. > - So `agent.task_run`, `routine.run`, `connection.invoked`, and other background events arrive with `stack_id` but without `company_id`. On staging, 66 of 66 `agent.task_run` events had a null `company_id`. > - Without `company_id`, product analytics cannot split agent and routine activity by company. > - This pull request lets each background emit site name the company of the entity that it reports on. Only the direct PostHog transport reads that company. > - The benefit is that Cloud background events carry `company_id`, and the self-hosted collector output does not change. ## Linked Issues or Issue Description Refs #22006 **Subsystem affected** Server telemetry: Cloud direct delivery to PostHog (`server/src/services/posthog-direct/`) and the server emit sites for background events. **Problem or motivation** Cloud direct events from background work have no `company_id`. Heartbeat runs, scheduled routines, connection invocations, and interaction updates do not run inside an HTTP request, so the request context gives no company. On paperclip.staging, all 66 sampled `agent.task_run` events had `company_id` null. **Proposed solution** Add a small AsyncLocalStorage scope, `withTelemetryCompany(companyId, fn)`, in `server/src/telemetry-company-scope.ts`. Each background emit site wraps its typed `track*` call with the company id of the row that it already has. The direct transport reads the scope at track time, through the existing `captureEventContext()` seam. A live request company still wins. The collector path never reads the scope. **Alternatives considered** 1. An optional context argument on the shared `track*` helpers. This changes the public helpers in `@paperclipai/shared` for a Cloud-only value, so I did not use it. 2. A scope around each whole work unit (heartbeat run, routine run, invocation). The emit calls are fire-and-forget and are spread across many modules, so it is easy to miss one or to leak a company into other work. A scope at the emit site is narrow and explicit. 3. Let the transport look up the company in the database. The transport must not query the database. **Roadmap alignment** This completes the Cloud direct PostHog work from #22006. It does not change the telemetry contract. ## What Changed - Add `server/src/telemetry-company-scope.ts`. It has `withTelemetryCompany(companyId, fn)` and `scopedTelemetryCompanyId()`. With no company id, it calls `fn` directly. - `capturePostHogEventContext()` uses the scoped company when no live request gives a company. The request company, user, and session behave as before. - Wrap these emit sites with the company of their entity: - `agent.task_run` (`emitAgentTaskRun`, `emitAgentTaskRunById`; `companyId` is now part of the run that the function reads). This covers heartbeat, native runtime, recovery, run-dispatch, and watchdog callers. - `routine.run` (`routineService` dispatch, which covers scheduler, webhook, and manual runs) - `agent.first_heartbeat` (heartbeat agent status) - `connection.created`, `connection.updated`, `connection.invoked` - `interaction.created`, `interaction.resolved` - `agent.task_completed` (issue route; the issue company is the fallback when the request gives no company) - Request-only events (`project.created`, `goal.created`, `agent.created`, `routine.created`, `skill.imported`) keep the request context. `server.started` and `error_handler.crash` are not company-scoped. Dynamic plugin and `question_response.delivery` events are not in the contract, so the direct path drops them. - Update the "Cloud Direct Delivery" section of `packages/shared/src/telemetry/README.md`. Design note on module boundaries: the scope lives beside `server/src/telemetry.ts`, not inside `posthog-direct/`. Emit sites do not import the transport, and the transport does not know which services exist. The shared telemetry package does not change. ## Verification - Red/green: with the source changes removed, these 4 new tests fail. With the changes, they pass. - `agent-task-run-telemetry.test.ts`: `emitAgentTaskRun` and `emitAgentTaskRunById` track in the run company scope. - `routine-run-telemetry.test.ts`: a routine run tracks in the routine company scope. - `posthog-direct-telemetry.test.ts`: on Cloud, background `agent.task_run` and `routine.run` (also after an `await`) carry `company_id`, and an unscoped event does not. - `posthog-direct-telemetry.test.ts`: a live request company wins over the scope, and the scope fills the gap when the request has no company. - Red/green per emit family: the tests for `connection.created`, `connection.updated`, `connection.invoked`, `interaction.created`, `interaction.resolved`, `agent.first_heartbeat`, and `agent.task_completed` assert the company scope at track time. With the emit-site wrappers removed, all 7 assertions fail. - New guard test: on self-hosted, a scoped `agent.task_run` and `routine.run` produce a byte-identical collector envelope with no `company` text. - `cd server && npx vitest run` on 16 telemetry, PostHog, routine, heartbeat status, interaction, and connector test files: 408 passed. - `pnpm --filter @paperclipai/server typecheck`: passed. - Local check with a real `posthog-node` client sent to a local capture server (never real PostHog), embedded Postgres, and a real `routineService.tickScheduledTriggers()` tick plus `emitAgentTaskRun`: - Cloud mode: the captured `routine.run` (`source: schedule`) and `agent.task_run` both had the seeded `company_id`, plus `stack_id` and `environment`. `server.started` had no `company_id`. - Self-hosted mode: the collector body from this branch and from `origin/master` are identical after I normalize random ids and timestamps. - I did not boot a full Cloud-managed server, because it needs S3 and an external Postgres. The check above uses the same server modules and the real PostHog client. ## Risks - Privacy review: this is a Telemetry change under `rules/data-paths.mdc`, so I request a privacy review. Background Cloud events now send the real Paperclip `company_id` to PostHog. #22006 already approved `company_id` as a Cloud direct property for request events. This PR only fills it for background events. It adds no new property, dimension, or event, so the generated contract does not change. `packages/shared/src/telemetry/README.md` is updated in this PR. - Low runtime risk. The self-hosted collector path does not read the scope, and a test proves its output does not change. - On Cloud, more events now have `company_id`. No event loses a field. - If a request with a company tracks an event for an entity in a different company, the request company wins. This is the same as before this change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, and test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 --- packages/shared/src/telemetry/README.md | 7 ++ .../agent-task-run-telemetry.test.ts | 15 ++++ .../heartbeat-process-recovery.test.ts | 6 +- .../__tests__/issue-telemetry-routes.test.ts | 4 + ...ssue-thread-interactions-telemetry.test.ts | 9 ++ .../posthog-direct-telemetry.test.ts | 89 ++++++++++++++++++- .../__tests__/routine-run-telemetry.test.ts | 11 +++ .../tool-access-connector-telemetry.test.ts | 10 ++- .../tool-gateway-connector-telemetry.test.ts | 9 +- server/src/routes/issues.ts | 5 +- .../src/services/agent-task-run-telemetry.ts | 8 +- server/src/services/connector-telemetry.ts | 13 +-- server/src/services/heartbeat/agent-status.ts | 3 +- .../src/services/issue-thread-interactions.ts | 18 ++-- .../posthog-direct/request-context.ts | 20 +++-- server/src/services/routines.ts | 5 +- server/src/telemetry-company-scope.ts | 11 +++ .../runner-e2e/source-sdk-public-inputs.json | 2 +- 18 files changed, 214 insertions(+), 31 deletions(-) create mode 100644 server/src/telemetry-company-scope.ts diff --git a/packages/shared/src/telemetry/README.md b/packages/shared/src/telemetry/README.md index 50ae8992b33..eaa75efee24 100644 --- a/packages/shared/src/telemetry/README.md +++ b/packages/shared/src/telemetry/README.md @@ -143,6 +143,13 @@ exposes the `TelemetryTransport` seam on `TelemetryClient`: `source_ref_hashed` stays `true` for them. - The transport can capture per-event context at track time. That context is never added to the collector envelope. +- Background work (heartbeat runs, routine runs, connection invocations, and + interactions) has no request. The server emit site wraps the track call in + `withTelemetryCompany(companyId, fn)` from + `server/src/telemetry-company-scope.ts`, with the company of the entity it + reports on. Only the direct transport reads this scope, and it uses it only + when no live request supplies a company. The collector envelope and the + dimensions do not change. Cloud is Paperclip's own service under the Cloud terms, so its events carry real identifiers: raw agent and task ids, the stack id, and the company and diff --git a/server/src/__tests__/agent-task-run-telemetry.test.ts b/server/src/__tests__/agent-task-run-telemetry.test.ts index 87dbf7a26a3..abf11fd70fb 100644 --- a/server/src/__tests__/agent-task-run-telemetry.test.ts +++ b/server/src/__tests__/agent-task-run-telemetry.test.ts @@ -6,6 +6,7 @@ import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase, } from "./helpers/embedded-postgres.js"; +import { scopedTelemetryCompanyId } from "../telemetry-company-scope.js"; const mockTelemetryClient = vi.hoisted(() => ({ track: vi.fn() })); const mockTrackAgentTaskRun = vi.hoisted(() => vi.fn()); @@ -105,6 +106,20 @@ describeEmbeddedPostgres("emitAgentTaskRun", () => { } }); + it("tracks agent.task_run in the run's company scope", async () => { + await seedCompanyAndAgent(); + const [run] = await db.insert(heartbeatRuns).values({ companyId, agentId, status: "succeeded" }).returning(); + const scopedCompanies: Array = []; + const recordScope = () => scopedCompanies.push(scopedTelemetryCompanyId()); + mockTrackAgentTaskRun.mockImplementationOnce(recordScope).mockImplementationOnce(recordScope); + + await emitAgentTaskRun(db, run); + await emitAgentTaskRunById(db, { runId: run.id, companyId }); + + expect(scopedCompanies).toEqual([companyId, companyId]); + expect(scopedTelemetryCompanyId()).toBeUndefined(); + }); + it("emits the same event from a small scoped run read without loading execution payloads", async () => { await seedCompanyAndAgent(); const payload = randomBytes(128 * 1024).toString("base64"); diff --git a/server/src/__tests__/heartbeat-process-recovery.test.ts b/server/src/__tests__/heartbeat-process-recovery.test.ts index 1f5d3d40020..a0865338dcf 100644 --- a/server/src/__tests__/heartbeat-process-recovery.test.ts +++ b/server/src/__tests__/heartbeat-process-recovery.test.ts @@ -124,6 +124,7 @@ import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase, } from "./helpers/embedded-postgres.js"; +import { scopedTelemetryCompanyId } from "../telemetry-company-scope.js"; import type { EnvironmentLease } from "@paperclipai/shared"; import type { NativeExecutionInput } from "../vendor/paperclip-runner/index.js"; import { runningProcesses, type AdapterExecutionContext, type AdapterExecutionResult } from "../adapters/index.ts"; @@ -10035,11 +10036,13 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => { }); it("tracks the first heartbeat with the agent role instead of adapter type", async () => { - const { agentId, runId } = await seedRunFixture({ + const { companyId, agentId, runId } = await seedRunFixture({ agentStatus: "running", includeIssue: false, }); const heartbeat = heartbeatService(db); + const scopedCompanies: Array = []; + mockTrackAgentFirstHeartbeat.mockImplementationOnce(() => scopedCompanies.push(scopedTelemetryCompanyId())); await heartbeat.cancelRun(runId); @@ -10050,6 +10053,7 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => { agentId, }), ); + expect(scopedCompanies).toEqual([companyId]); }); it("preserves first-heartbeat telemetry after a timer interval claim", async () => { diff --git a/server/src/__tests__/issue-telemetry-routes.test.ts b/server/src/__tests__/issue-telemetry-routes.test.ts index a12809a58bc..bf29f4f5059 100644 --- a/server/src/__tests__/issue-telemetry-routes.test.ts +++ b/server/src/__tests__/issue-telemetry-routes.test.ts @@ -185,6 +185,9 @@ describe("issue telemetry routes", () => { }, 60_000); it("emits task-completed telemetry with the agent role, adapter type, and model", async () => { + const { scopedTelemetryCompanyId } = await import("../telemetry-company-scope.js"); + const scopedCompanies: Array = []; + mockTrackAgentTaskCompleted.mockImplementationOnce(() => scopedCompanies.push(scopedTelemetryCompanyId())); mockAgentService.getById.mockResolvedValue({ id: "agent-1", companyId: "company-1", @@ -213,6 +216,7 @@ describe("issue telemetry routes", () => { taskId: "11111111-1111-4111-8111-111111111111", }); }); + expect(scopedCompanies).toEqual(["company-1"]); }, 10_000); it("does not emit agent task-completed telemetry for board-driven completions", async () => { diff --git a/server/src/__tests__/issue-thread-interactions-telemetry.test.ts b/server/src/__tests__/issue-thread-interactions-telemetry.test.ts index 78bd4a1ca70..b89287c4b19 100644 --- a/server/src/__tests__/issue-thread-interactions-telemetry.test.ts +++ b/server/src/__tests__/issue-thread-interactions-telemetry.test.ts @@ -20,6 +20,7 @@ import { startEmbeddedPostgresTestDatabase, } from "./helpers/embedded-postgres.js"; import { issueThreadInteractionService } from "../services/issue-thread-interactions.js"; +import { scopedTelemetryCompanyId } from "../telemetry-company-scope.js"; const telemetryMocks = vi.hoisted(() => ({ track: vi.fn(), @@ -47,6 +48,7 @@ describeEmbeddedPostgres("issueThreadInteractionService telemetry", () => { beforeEach(() => { telemetryMocks.track.mockClear(); + telemetryMocks.track.mockImplementation(() => scopedTelemetryCompanyId()); telemetryMocks.getTelemetryClient.mockReturnValue({ track: telemetryMocks.track, hashPrivateRef: vi.fn((value: string) => `hashed:${value}`), @@ -135,6 +137,11 @@ describeEmbeddedPostgres("issueThreadInteractionService telemetry", () => { return calls.at(-1)?.[1] as Record; } + function scopesOf(name: string) { + return telemetryMocks.track.mock.calls.flatMap(([eventName], index) => + eventName === name ? [telemetryMocks.track.mock.results[index]?.value] : []); + } + it("emits deprecated resolver alias use at creation without interaction content", async () => { const { companyId, issueId } = await seedIssue("Deprecated alias telemetry issue title"); @@ -161,6 +168,7 @@ describeEmbeddedPostgres("issueThreadInteractionService telemetry", () => { interaction_kind: "request_confirmation", used_deprecated_resolver_policy_alias: true, }); + expect(scopesOf("interaction.created")).toEqual([companyId]); }); it("emits skipped resolution telemetry for composer Skip", async () => { @@ -197,6 +205,7 @@ describeEmbeddedPostgres("issueThreadInteractionService telemetry", () => { target_type: "none", }); expectNoRawInteractionIds(dimensions); + expect(scopesOf("interaction.resolved")).toEqual([companyId]); }); it("emits accepted suggested-task telemetry with created and skipped task counts", async () => { diff --git a/server/src/__tests__/posthog-direct-telemetry.test.ts b/server/src/__tests__/posthog-direct-telemetry.test.ts index e18bda622a9..b35b0b1b3f6 100644 --- a/server/src/__tests__/posthog-direct-telemetry.test.ts +++ b/server/src/__tests__/posthog-direct-telemetry.test.ts @@ -5,7 +5,7 @@ import path from "node:path"; import express from "express"; import request from "supertest"; import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; -import { trackAgentTaskRun, trackCompanyImported, trackRoutineCreated } from "@paperclipai/shared/telemetry"; +import { trackAgentTaskRun, trackCompanyImported, trackRoutineCreated, trackRoutineRun } from "@paperclipai/shared/telemetry"; const hoisted = vi.hoisted(() => ({ instanceRoot: "", posthogNodeLoads: 0 })); @@ -31,6 +31,7 @@ import { capturePostHogEventContext, postHogRequestContextMiddlewareForEnv, } from "../services/posthog-direct/request-context.js"; +import { withTelemetryCompany } from "../telemetry-company-scope.js"; import { serverVersion } from "../version.js"; const COLLECTOR = "http://collector.test/ingest"; @@ -120,6 +121,44 @@ describe("server telemetry sender selection", () => { ); }); + it("self-hosted: a background company scope leaves the collector envelope byte-identical", async () => { + const client = initTelemetry({ enabled: true }, { env: {} })!; + await withTelemetryCompany("company-bg", async () => { + await Promise.resolve(); + trackAgentTaskRun(client, { agentId: "agent-1", state: "succeeded" }); + trackRoutineRun(client, { source: "schedule", status: "issue_created" }); + }); + await shutdownTelemetry(); + + expect(hoisted.posthogNodeLoads).toBe(0); + expect(fetchMock).toHaveBeenCalledTimes(1); + const body = String((fetchMock.mock.calls[0]![1] as RequestInit).body); + const parsed = JSON.parse(body); + expect(body).toBe( + JSON.stringify({ + app: "paperclip", + schemaVersion: "1", + installId: readState().installId, + version: serverVersion, + events: [ + { name: "server.started", occurredAt: parsed.events[0].occurredAt, dimensions: { os_platform: process.platform } }, + { + name: "agent.task_run", + occurredAt: parsed.events[1].occurredAt, + dimensions: { agent_id: "agent-1", state: "succeeded" }, + }, + { + name: "routine.run", + occurredAt: parsed.events[2].occurredAt, + dimensions: { source: "schedule", status: "issue_created" }, + }, + ], + batchId: parsed.batchId, + }), + ); + expect(body).not.toContain("company"); + }); + it("self-hosted: ignores posthog-js tracing headers", async () => { expect(postHogRequestContextMiddlewareForEnv({})).toBeNull(); expect(postHogRequestContextMiddlewareForEnv({ PAPERCLIP_POSTHOG_PROJECT_TOKEN: "phc_test" })).toBeNull(); @@ -345,4 +384,52 @@ describe("server telemetry sender selection", () => { expect(routine.distinctId).toBe("workspace:stack-123"); expect(routine.properties).not.toHaveProperty("$session_id"); }); + + it("Cloud: background events carry the company in scope", async () => { + const posthog = fakePostHog(); + const client = initTelemetry({ enabled: true }, { env: CLOUD_ENV, postHogFactory: posthog.factory })!; + await withTelemetryCompany("company-bg", async () => { + await new Promise((resolve) => setTimeout(resolve, 1)); + trackAgentTaskRun(client, { agentId: "agent-1", state: "succeeded" }); + }); + withTelemetryCompany("company-bg", () => trackRoutineRun(client, { source: "schedule", status: "issue_created" })); + trackRoutineCreated(client); + await shutdownTelemetry(); + + const run = posthog.captured.find((message) => message.event === "agent.task_run")!; + expect(run.distinctId).toBe("workspace:stack-123"); + expect(run.properties).toMatchObject({ company_id: "company-bg", stack_id: "stack-123", $process_person_profile: false }); + expect(run.properties).not.toHaveProperty("user_id"); + const routineRun = posthog.captured.find((message) => message.event === "routine.run")!; + expect(routineRun.properties).toMatchObject({ company_id: "company-bg", stack_id: "stack-123" }); + const unscoped = posthog.captured.find((message) => message.event === "routine.created")!; + expect(unscoped.properties).not.toHaveProperty("company_id"); + }); + + it("Cloud: a live request's company wins over a background company scope", async () => { + const posthog = fakePostHog(); + initTelemetry({ enabled: true }, { env: CLOUD_ENV, postHogFactory: posthog.factory }); + const app = express(); + app.use(postHogRequestContextMiddlewareForEnv(CLOUD_ENV)!); + app.use((req, _res, next) => { + req.actor = { type: "board", userId: "user-1", source: "session" }; + next(); + }); + app.post("/companies/:companyId/routines", (_req, res) => { + withTelemetryCompany("company-bg", () => trackRoutineCreated(getTelemetryClient()!)); + res.json({}); + }); + app.post("/runs", (_req, res) => { + withTelemetryCompany("company-bg", () => trackRoutineCreated(getTelemetryClient()!)); + res.json({}); + }); + await request(app).post("/companies/company-9/routines").expect(200); + await request(app).post("/runs").expect(200); + await shutdownTelemetry(); + + const routines = posthog.captured.filter((message) => message.event === "routine.created"); + expect(routines).toHaveLength(2); + expect(routines[0]!.properties).toMatchObject({ company_id: "company-9", user_id: "user-1" }); + expect(routines[1]!.properties).toMatchObject({ company_id: "company-bg", user_id: "user-1" }); + }); }); diff --git a/server/src/__tests__/routine-run-telemetry.test.ts b/server/src/__tests__/routine-run-telemetry.test.ts index c14a5ebf46e..ef04907111d 100644 --- a/server/src/__tests__/routine-run-telemetry.test.ts +++ b/server/src/__tests__/routine-run-telemetry.test.ts @@ -21,6 +21,7 @@ import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase, } from "./helpers/embedded-postgres.js"; +import { scopedTelemetryCompanyId } from "../telemetry-company-scope.js"; const mockTelemetryClient = vi.hoisted(() => ({ track: vi.fn() })); const mockTrackRoutineRun = vi.hoisted(() => vi.fn()); @@ -168,4 +169,14 @@ describeEmbeddedPostgres("routine run telemetry", () => { status: "issue_created", }); }); + + it("tracks a routine run in the routine's company scope", async () => { + const { routine, svc } = await seedFixture(); + const scopedCompanies: Array = []; + mockTrackRoutineRun.mockImplementationOnce(() => scopedCompanies.push(scopedTelemetryCompanyId())); + + await svc.runRoutine(routine.id, { source: "manual" }); + + expect(scopedCompanies).toEqual([routine.companyId]); + }); }); diff --git a/server/src/__tests__/tool-access-connector-telemetry.test.ts b/server/src/__tests__/tool-access-connector-telemetry.test.ts index 924787d2328..3fdacfc1fc4 100644 --- a/server/src/__tests__/tool-access-connector-telemetry.test.ts +++ b/server/src/__tests__/tool-access-connector-telemetry.test.ts @@ -26,8 +26,9 @@ import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase, } from "./helpers/embedded-postgres.js"; +import { scopedTelemetryCompanyId } from "../telemetry-company-scope.js"; -const track = vi.fn(); +const track = vi.fn((..._args: unknown[]) => scopedTelemetryCompanyId()); const isRegisteredEventName = vi.fn(() => true); vi.mock("../telemetry.js", () => ({ getTelemetryClient: () => ({ track, isRegisteredEventName }), @@ -44,6 +45,11 @@ const describeEmbeddedPostgres = embeddedPostgresSupport.supported type Db = ReturnType; +function scopesOf(name: string) { + return track.mock.calls.flatMap(([eventName], index) => + eventName === name ? [track.mock.results[index]?.value] : []); +} + function createdEvents() { return track.mock.calls.filter(([name]) => name === "connection.created"); } @@ -235,6 +241,7 @@ describeEmbeddedPostgres("connector lifecycle telemetry (tool-access)", () => { status: "active", enabled: true, }); + expect(scopesOf("connection.created")).toEqual([company.id]); }, 30_000); it("updateConnection emits committed transitions only; metadata saves stay silent", async () => { @@ -266,6 +273,7 @@ describeEmbeddedPostgres("connector lifecycle telemetry (tool-access)", () => { previous_status: "active", status: "active", }); + expect(scopesOf("connection.updated")).toEqual([company.id]); track.mockClear(); await svc.updateConnection(connection.id, { name: "Renamed fixture" }); diff --git a/server/src/__tests__/tool-gateway-connector-telemetry.test.ts b/server/src/__tests__/tool-gateway-connector-telemetry.test.ts index f2d2526a6e5..1674b84e906 100644 --- a/server/src/__tests__/tool-gateway-connector-telemetry.test.ts +++ b/server/src/__tests__/tool-gateway-connector-telemetry.test.ts @@ -29,12 +29,13 @@ import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase, } from "./helpers/embedded-postgres.js"; +import { scopedTelemetryCompanyId } from "../telemetry-company-scope.js"; // Proposal wrappers drop unregistered names inside the real client, so the // integration assertions use a client double that reports the proposed event // as registered. `track` observes exactly what the emitter would hand the // real client after schema adoption. -const track = vi.fn(); +const track = vi.fn((..._args: unknown[]) => scopedTelemetryCompanyId()); const isRegisteredEventName = vi.fn(() => true); vi.mock("../telemetry.js", () => ({ getTelemetryClient: () => ({ track, isRegisteredEventName }), @@ -236,6 +237,11 @@ async function createRemoteMcpTool(db: Db, companyId: string) { return { application: application!, connection: connection!, catalogEntry: catalogEntry! }; } +function scopesOf(name: string) { + return track.mock.calls.flatMap(([eventName], index) => + eventName === name ? [track.mock.results[index]?.value] : []); +} + async function settledEvents(expectedCount: number) { await vi.waitFor(() => { expect( @@ -336,6 +342,7 @@ describeEmbeddedPostgres("gateway connector invocation telemetry", () => { status: "succeeded", origin: "agent", }); + expect(scopesOf("connection.invoked")).toEqual([f.company.id]); }, 30_000); it("emits a single failed event when post-success audit work throws (no contradictory pair)", async () => { diff --git a/server/src/routes/issues.ts b/server/src/routes/issues.ts index ca9217a229a..6a115e1abf5 100644 --- a/server/src/routes/issues.ts +++ b/server/src/routes/issues.ts @@ -208,6 +208,7 @@ import { } from "@paperclipai/shared"; import { trackAgentTaskCompleted } from "@paperclipai/shared/telemetry"; import { getTelemetryClient } from "../telemetry.js"; +import { withTelemetryCompany } from "../telemetry-company-scope.js"; import { isUniqueViolation } from "../db-errors.js"; import type { StorageService } from "../storage/types.js"; import { validate, validateIssueMutationBody } from "../middleware/validate.js"; @@ -16704,13 +16705,13 @@ export function issueRoutes( typeof actorAgent.adapterConfig?.model === "string" ? actorAgent.adapterConfig.model : undefined; - trackAgentTaskCompleted(tc, { + withTelemetryCompany(issue.companyId, () => trackAgentTaskCompleted(tc, { agentRole: actorAgent.role, agentId: actorAgent.id, adapterType: actorAgent.adapterType, model, taskId: issue.id, - }); + })); } } } diff --git a/server/src/services/agent-task-run-telemetry.ts b/server/src/services/agent-task-run-telemetry.ts index d9df8afeeef..a80243d9363 100644 --- a/server/src/services/agent-task-run-telemetry.ts +++ b/server/src/services/agent-task-run-telemetry.ts @@ -4,10 +4,11 @@ import { trackAgentTaskRun } from "@paperclipai/shared/telemetry"; import { parseObject } from "../adapters/utils.js"; import { logger } from "../middleware/logger.js"; import { getTelemetryClient } from "../telemetry.js"; +import { withTelemetryCompany } from "../telemetry-company-scope.js"; type HeartbeatRun = typeof heartbeatRuns.$inferSelect; type TelemetryRun = Pick; + "id" | "companyId" | "agentId" | "status" | "startedAt" | "finishedAt" | "usageJson" | "contextSnapshot">; function readNonEmptyString(value: unknown): string | null { return typeof value === "string" && value.trim().length > 0 @@ -79,7 +80,7 @@ export async function emitAgentTaskRun(db: Db, run: TelemetryRun): Promise readNonEmptyString(parseObject(run.contextSnapshot).issueId) ?? undefined; - trackAgentTaskRun(client, { + withTelemetryCompany(run.companyId, () => trackAgentTaskRun(client, { agentId: run.agentId, state: run.status, ...(agent @@ -116,7 +117,7 @@ export async function emitAgentTaskRun(db: Db, run: TelemetryRun): Promise } : {}), ...(taskId ? { taskId } : {}), - }); + })); } catch (err) { logger.warn( { err, runId: run.id }, @@ -134,6 +135,7 @@ export async function emitAgentTaskRunById( const run = await db .select({ id: heartbeatRuns.id, + companyId: heartbeatRuns.companyId, agentId: heartbeatRuns.agentId, status: heartbeatRuns.status, startedAt: heartbeatRuns.startedAt, diff --git a/server/src/services/connector-telemetry.ts b/server/src/services/connector-telemetry.ts index c1b05b134e3..0f314aa7ba1 100644 --- a/server/src/services/connector-telemetry.ts +++ b/server/src/services/connector-telemetry.ts @@ -8,6 +8,7 @@ import { } from "@paperclipai/shared/telemetry"; import { logger } from "../middleware/logger.js"; import { getTelemetryClient } from "../telemetry.js"; +import { withTelemetryCompany } from "../telemetry-company-scope.js"; type ToolConnectionRow = typeof toolConnections.$inferSelect; type ToolInvocationRow = typeof toolInvocations.$inferSelect; @@ -93,14 +94,14 @@ export function emitConnectionCreated( const client = getTelemetryClient(); if (!client) return; if (!isToolPurpose(connection)) return; - trackConnectionCreated(client, { + withTelemetryCompany(connection.companyId, () => trackConnectionCreated(client, { connector_key: connectorKeyForConnection(connection), transport: connection.transport, auth_kind: connection.authKind, setup_flow: setupFlow, status: connection.status, enabled: connection.enabled, - }); + })); } catch (err) { logger.warn( { err, connectionId: connection.id }, @@ -131,7 +132,7 @@ export function emitConnectionUpdated( ) { return; } - trackConnectionUpdated(client, { + withTelemetryCompany(connection.companyId, () => trackConnectionUpdated(client, { connector_key: connectorKeyForConnection(connection), transport: connection.transport, auth_kind: connection.authKind, @@ -140,7 +141,7 @@ export function emitConnectionUpdated( status: connection.status, previous_enabled: previous.enabled, enabled: connection.enabled, - }); + })); } catch (err) { logger.warn( { err, connectionId: connection.id }, @@ -212,13 +213,13 @@ export async function emitConnectionInvoked( ? Math.max(0, Math.round((completedAtMs - startedAtMs) / 1000)) : undefined; - trackConnectionInvoked(client, { + withTelemetryCompany(connection.companyId, () => trackConnectionInvoked(client, { connector_key: connectorKeyForConnection(connection), transport: connection.transport, status: invocation.status, origin: invocationOrigin(invocation), ...(durationSeconds === undefined ? {} : { duration_seconds: durationSeconds }), - }); + })); } catch (err) { logger.warn( { err, invocationId }, diff --git a/server/src/services/heartbeat/agent-status.ts b/server/src/services/heartbeat/agent-status.ts index 511f1da9909..251b5794a6c 100644 --- a/server/src/services/heartbeat/agent-status.ts +++ b/server/src/services/heartbeat/agent-status.ts @@ -5,6 +5,7 @@ import { agents, heartbeatRuns, type Db } from "@paperclipai/db"; import { and, asc, desc, eq, lte, sql } from "drizzle-orm"; import { trackAgentFirstHeartbeat } from "@paperclipai/shared/telemetry"; import { getTelemetryClient } from "../../telemetry.js"; +import { withTelemetryCompany } from "../../telemetry-company-scope.js"; import { logActivity, publishActivity, type ActivityPublication } from "../activity-log.js"; import { publishLiveEvent } from "../live-events.js"; import { finishedIdleRun, finishedIdleWake, finishedFailedIdleLease } from "../idle-work-proof.js"; @@ -116,7 +117,7 @@ export function createHeartbeatAgentStatus(db: Db) { if (!result?.updated) return; if (result.isFirstHeartbeat) { const client = getTelemetryClient(); - if (client) trackAgentFirstHeartbeat(client, { agentRole: result.updated.role, agentId: result.updated.id }); + if (client) withTelemetryCompany(result.updated.companyId, () => trackAgentFirstHeartbeat(client, { agentRole: result.updated.role, agentId: result.updated.id })); } publishStatus(result.updated, outcome); } diff --git a/server/src/services/issue-thread-interactions.ts b/server/src/services/issue-thread-interactions.ts index 890ba8b47ce..fbaf441f1e7 100644 --- a/server/src/services/issue-thread-interactions.ts +++ b/server/src/services/issue-thread-interactions.ts @@ -147,6 +147,7 @@ import { z } from "zod"; import { logger } from "../middleware/logger.js"; import { conflict, forbidden, notFound, unprocessable } from "../errors.js"; import { getTelemetryClient } from "../telemetry.js"; +import { withTelemetryCompany } from "../telemetry-company-scope.js"; import { authorizationService } from "./authorization.js"; import { isCloudManagedInstance } from "./cloud-instance.js"; import { resolveDeploymentMode } from "../config-file.js"; @@ -1797,7 +1798,7 @@ async function emitInteractionResolvedTelemetry( ? (roleByAgentId.get(interaction.createdByAgentId) ?? undefined) : undefined; - trackInteractionResolved(telemetryClient, { + withTelemetryCompany(interaction.companyId, () => trackInteractionResolved(telemetryClient, { interactionKind: interaction.kind === "connection_intent" || interaction.kind === "suggest_routine" || interaction.kind === "propose_responsibility_bar" || interaction.kind === "propose_automation" @@ -1814,7 +1815,7 @@ async function emitInteractionResolvedTelemetry( }), legacyInheritedRestriction: interaction.resolverPolicyProvenance === "legacy_inherited_restriction", - }); + })); } catch (error) { console.error( "[paperclip] Failed to emit interaction.resolved telemetry", @@ -1823,7 +1824,8 @@ async function emitInteractionResolvedTelemetry( } } -function emitInteractionCreatedTelemetry(args: { +function emitInteractionCreatedTelemetry({ companyId, ...args }: { + companyId: string; interactionKind: IssueThreadInteractionKind; usedDeprecatedResolverPolicyAlias: boolean; }) { @@ -1831,7 +1833,7 @@ function emitInteractionCreatedTelemetry(args: { if (!telemetryClient) return; try { - trackInteractionCreated(telemetryClient, { + withTelemetryCompany(companyId, () => trackInteractionCreated(telemetryClient, { ...args, interactionKind: args.interactionKind === "connection_intent" || args.interactionKind === "suggest_routine" @@ -1839,7 +1841,7 @@ function emitInteractionCreatedTelemetry(args: { || args.interactionKind === "propose_dashboard_changes" ? "other" : args.interactionKind, - }); + })); } catch (error) { console.error( "[paperclip] Failed to emit interaction.created telemetry", @@ -3363,6 +3365,7 @@ export function issueThreadInteractionService( const interaction = hydrateInteraction(created) as ConnectionIntentInteraction; if (inserted) emitInteractionCreatedTelemetry({ + companyId: issue.companyId, interactionKind: "connection_intent", usedDeprecatedResolverPolicyAlias: false, }); @@ -3555,6 +3558,7 @@ export function issueThreadInteractionService( return row!; }); emitInteractionCreatedTelemetry({ + companyId: issue.companyId, interactionKind: "suggest_routine", usedDeprecatedResolverPolicyAlias: false, }); @@ -3881,6 +3885,7 @@ export function issueThreadInteractionService( return { row: row!, replacedIds: replaced.map((entry) => entry.id) }; }); emitInteractionCreatedTelemetry({ + companyId: issue.companyId, interactionKind: "propose_responsibility_bar", usedDeprecatedResolverPolicyAlias: false, }); @@ -4056,6 +4061,7 @@ export function issueThreadInteractionService( return { row: row!, replacedIds: replaced.map((entry) => entry.id) }; }); emitInteractionCreatedTelemetry({ + companyId: issue.companyId, interactionKind: "propose_dashboard_changes", usedDeprecatedResolverPolicyAlias: false, }); @@ -4231,6 +4237,7 @@ export function issueThreadInteractionService( return { row: row!, replacedIds: replaced.map((entry) => entry.id) }; }); emitInteractionCreatedTelemetry({ + companyId: issue.companyId, interactionKind: "propose_automation", usedDeprecatedResolverPolicyAlias: false, }); @@ -5457,6 +5464,7 @@ export function issueThreadInteractionService( } const interaction = hydrateInteraction(created); emitInteractionCreatedTelemetry({ + companyId: issue.companyId, interactionKind: interaction.kind, usedDeprecatedResolverPolicyAlias, }); diff --git a/server/src/services/posthog-direct/request-context.ts b/server/src/services/posthog-direct/request-context.ts index 6c1b3dbc2d8..a037e260594 100644 --- a/server/src/services/posthog-direct/request-context.ts +++ b/server/src/services/posthog-direct/request-context.ts @@ -1,6 +1,7 @@ import { AsyncLocalStorage } from "node:async_hooks"; import type { Request, RequestHandler } from "express"; import { isPostHogDirectDeliveryEnabled, resolveCloudPostHogConfig } from "../cloud-posthog-config.js"; +import { scopedTelemetryCompanyId } from "../../telemetry-company-scope.js"; /** * Request-scoped analytics context for Paperclip Cloud direct delivery. @@ -98,13 +99,18 @@ function knownCompanyId(req: Request): string | undefined { /** Snapshot of the current request's analytics context, taken at track time. */ export function capturePostHogEventContext(): PostHogEventContext | undefined { const store = storage.getStore(); - if (!store || !store.active) return undefined; const context: PostHogEventContext = {}; - const userId = actingUserId(store.req); - const companyId = knownCompanyId(store.req); - if (userId) context.userId = userId; - if (companyId) context.companyId = companyId; - if (store.sessionId) context.sessionId = store.sessionId; - if (store.windowId) context.windowId = store.windowId; + if (store?.active) { + const userId = actingUserId(store.req); + const companyId = knownCompanyId(store.req); + if (userId) context.userId = userId; + if (companyId) context.companyId = companyId; + if (store.sessionId) context.sessionId = store.sessionId; + if (store.windowId) context.windowId = store.windowId; + } + if (!context.companyId) { + const scopedCompanyId = scopedTelemetryCompanyId(); + if (scopedCompanyId) context.companyId = scopedCompanyId; + } return Object.keys(context).length > 0 ? context : undefined; } diff --git a/server/src/services/routines.ts b/server/src/services/routines.ts index fd5b6c0e35c..eaf7ba06a65 100644 --- a/server/src/services/routines.ts +++ b/server/src/services/routines.ts @@ -69,6 +69,7 @@ import { trackRoutineRun } from "@paperclipai/shared/telemetry"; import { HttpError, badRequest, conflict, forbidden, notFound, unauthorized, unprocessable } from "../errors.js"; import { logger } from "../middleware/logger.js"; import { getTelemetryClient } from "../telemetry.js"; +import { withTelemetryCompany } from "../telemetry-company-scope.js"; import { getConfiguredSecretProvider } from "../secrets/configured-provider.js"; import { issueService } from "./issues.js"; import { assertAssignableAgent } from "./agent-assignability.js"; @@ -2083,10 +2084,10 @@ export function routineService( const telemetryClient = getTelemetryClient(); if (telemetryClient) { - trackRoutineRun(telemetryClient, { + withTelemetryCompany(input.routine.companyId, () => trackRoutineRun(telemetryClient, { source: run.source, status: run.status, - }); + })); } return run; diff --git a/server/src/telemetry-company-scope.ts b/server/src/telemetry-company-scope.ts new file mode 100644 index 00000000000..6922fcce34a --- /dev/null +++ b/server/src/telemetry-company-scope.ts @@ -0,0 +1,11 @@ +import { AsyncLocalStorage } from "node:async_hooks"; + +const storage = new AsyncLocalStorage(); + +export function withTelemetryCompany(companyId: string | null | undefined, fn: () => T): T { + return companyId ? storage.run(companyId, fn) : fn(); +} + +export function scopedTelemetryCompanyId(): string | undefined { + return storage.getStore(); +} diff --git a/tests/runner-e2e/source-sdk-public-inputs.json b/tests/runner-e2e/source-sdk-public-inputs.json index bbf3767d0fe..2235e82a0a1 100644 --- a/tests/runner-e2e/source-sdk-public-inputs.json +++ b/tests/runner-e2e/source-sdk-public-inputs.json @@ -367,7 +367,7 @@ "packages/shared/src/standing-checks.ts": "7ab71e4b6984e41e1c1dfd1bddf0116041d5abf1", "packages/shared/src/subscriptions.ts": "ef70e63395ca967bbc30776b71560e96eb354a75", "packages/shared/src/summary-slot.test.ts": "5f8e222dea425f0d5bff0adb5c683087bd7cf7fb", - "packages/shared/src/telemetry/README.md": "50ae8992b33c4141de443d42c57de24357e3d0aa", + "packages/shared/src/telemetry/README.md": "eaa75efee24352f97ddc7d72b2e99be8b39a241b", "packages/shared/src/telemetry/client-types.test.ts": "3b630edda4972b929ba4decd7457f226eb9cfc5b", "packages/shared/src/telemetry/client.test.ts": "5a95f2305a393c5b2e7750b10e0ed65da0f30cf8", "packages/shared/src/telemetry/client.ts": "d6534ce8aaa6f69fb2dd03e12487db024e250a96",