mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-19 16:29:42 +01:00
feat(telemetry): add company_id to Cloud direct events from background work (#22183)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Paperclip Cloud sends server telemetry straight to PostHog (`server/src/services/posthog-direct/`) > - Request events get `company_id` from the request context. Background events have no request. > - So `agent.task_run`, `routine.run`, `connection.invoked`, and other background events arrive with `stack_id` but without `company_id`. On staging, 66 of 66 `agent.task_run` events had a null `company_id`. > - Without `company_id`, product analytics cannot split agent and routine activity by company. > - This pull request lets each background emit site name the company of the entity that it reports on. Only the direct PostHog transport reads that company. > - The benefit is that Cloud background events carry `company_id`, and the self-hosted collector output does not change. ## Linked Issues or Issue Description Refs #22006 **Subsystem affected** Server telemetry: Cloud direct delivery to PostHog (`server/src/services/posthog-direct/`) and the server emit sites for background events. **Problem or motivation** Cloud direct events from background work have no `company_id`. Heartbeat runs, scheduled routines, connection invocations, and interaction updates do not run inside an HTTP request, so the request context gives no company. On paperclip.staging, all 66 sampled `agent.task_run` events had `company_id` null. **Proposed solution** Add a small AsyncLocalStorage scope, `withTelemetryCompany(companyId, fn)`, in `server/src/telemetry-company-scope.ts`. Each background emit site wraps its typed `track*` call with the company id of the row that it already has. The direct transport reads the scope at track time, through the existing `captureEventContext()` seam. A live request company still wins. The collector path never reads the scope. **Alternatives considered** 1. An optional context argument on the shared `track*` helpers. This changes the public helpers in `@paperclipai/shared` for a Cloud-only value, so I did not use it. 2. A scope around each whole work unit (heartbeat run, routine run, invocation). The emit calls are fire-and-forget and are spread across many modules, so it is easy to miss one or to leak a company into other work. A scope at the emit site is narrow and explicit. 3. Let the transport look up the company in the database. The transport must not query the database. **Roadmap alignment** This completes the Cloud direct PostHog work from #22006. It does not change the telemetry contract. ## What Changed - Add `server/src/telemetry-company-scope.ts`. It has `withTelemetryCompany(companyId, fn)` and `scopedTelemetryCompanyId()`. With no company id, it calls `fn` directly. - `capturePostHogEventContext()` uses the scoped company when no live request gives a company. The request company, user, and session behave as before. - Wrap these emit sites with the company of their entity: - `agent.task_run` (`emitAgentTaskRun`, `emitAgentTaskRunById`; `companyId` is now part of the run that the function reads). This covers heartbeat, native runtime, recovery, run-dispatch, and watchdog callers. - `routine.run` (`routineService` dispatch, which covers scheduler, webhook, and manual runs) - `agent.first_heartbeat` (heartbeat agent status) - `connection.created`, `connection.updated`, `connection.invoked` - `interaction.created`, `interaction.resolved` - `agent.task_completed` (issue route; the issue company is the fallback when the request gives no company) - Request-only events (`project.created`, `goal.created`, `agent.created`, `routine.created`, `skill.imported`) keep the request context. `server.started` and `error_handler.crash` are not company-scoped. Dynamic plugin and `question_response.delivery` events are not in the contract, so the direct path drops them. - Update the "Cloud Direct Delivery" section of `packages/shared/src/telemetry/README.md`. Design note on module boundaries: the scope lives beside `server/src/telemetry.ts`, not inside `posthog-direct/`. Emit sites do not import the transport, and the transport does not know which services exist. The shared telemetry package does not change. ## Verification - Red/green: with the source changes removed, these 4 new tests fail. With the changes, they pass. - `agent-task-run-telemetry.test.ts`: `emitAgentTaskRun` and `emitAgentTaskRunById` track in the run company scope. - `routine-run-telemetry.test.ts`: a routine run tracks in the routine company scope. - `posthog-direct-telemetry.test.ts`: on Cloud, background `agent.task_run` and `routine.run` (also after an `await`) carry `company_id`, and an unscoped event does not. - `posthog-direct-telemetry.test.ts`: a live request company wins over the scope, and the scope fills the gap when the request has no company. - Red/green per emit family: the tests for `connection.created`, `connection.updated`, `connection.invoked`, `interaction.created`, `interaction.resolved`, `agent.first_heartbeat`, and `agent.task_completed` assert the company scope at track time. With the emit-site wrappers removed, all 7 assertions fail. - New guard test: on self-hosted, a scoped `agent.task_run` and `routine.run` produce a byte-identical collector envelope with no `company` text. - `cd server && npx vitest run` on 16 telemetry, PostHog, routine, heartbeat status, interaction, and connector test files: 408 passed. - `pnpm --filter @paperclipai/server typecheck`: passed. - Local check with a real `posthog-node` client sent to a local capture server (never real PostHog), embedded Postgres, and a real `routineService.tickScheduledTriggers()` tick plus `emitAgentTaskRun`: - Cloud mode: the captured `routine.run` (`source: schedule`) and `agent.task_run` both had the seeded `company_id`, plus `stack_id` and `environment`. `server.started` had no `company_id`. - Self-hosted mode: the collector body from this branch and from `origin/master` are identical after I normalize random ids and timestamps. - I did not boot a full Cloud-managed server, because it needs S3 and an external Postgres. The check above uses the same server modules and the real PostHog client. ## Risks - Privacy review: this is a Telemetry change under `rules/data-paths.mdc`, so I request a privacy review. Background Cloud events now send the real Paperclip `company_id` to PostHog. #22006 already approved `company_id` as a Cloud direct property for request events. This PR only fills it for background events. It adds no new property, dimension, or event, so the generated contract does not change. `packages/shared/src/telemetry/README.md` is updated in this PR. - Low runtime risk. The self-hosted collector path does not read the scope, and a test proves its output does not change. - On Cloud, more events now have `company_id`. No event loses a field. - If a request with a company tracks an event for an entity in a different company, the request company wins. This is the same as before this change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, and test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
e2dda16b52
commit
2a43fe3e34
18 files changed
+214
-31
No files matched your search
@@ -143,6 +143,13 @@ exposes the `TelemetryTransport` seam on `TelemetryClient`:
|
||||
`source_ref_hashed` stays `true` for them.
|
||||
- The transport can capture per-event context at track time. That context is
|
||||
never added to the collector envelope.
|
||||
- Background work (heartbeat runs, routine runs, connection invocations, and
|
||||
interactions) has no request. The server emit site wraps the track call in
|
||||
`withTelemetryCompany(companyId, fn)` from
|
||||
`server/src/telemetry-company-scope.ts`, with the company of the entity it
|
||||
reports on. Only the direct transport reads this scope, and it uses it only
|
||||
when no live request supplies a company. The collector envelope and the
|
||||
dimensions do not change.
|
||||
|
||||
Cloud is Paperclip's own service under the Cloud terms, so its events carry
|
||||
real identifiers: raw agent and task ids, the stack id, and the company and
|
||||
|
||||
@@ -6,6 +6,7 @@ import {
|
||||
getEmbeddedPostgresTestSupport,
|
||||
startEmbeddedPostgresTestDatabase,
|
||||
} from "./helpers/embedded-postgres.js";
|
||||
import { scopedTelemetryCompanyId } from "../telemetry-company-scope.js";
|
||||
|
||||
const mockTelemetryClient = vi.hoisted(() => ({ track: vi.fn() }));
|
||||
const mockTrackAgentTaskRun = vi.hoisted(() => vi.fn());
|
||||
@@ -105,6 +106,20 @@ describeEmbeddedPostgres("emitAgentTaskRun", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it("tracks agent.task_run in the run's company scope", async () => {
|
||||
await seedCompanyAndAgent();
|
||||
const [run] = await db.insert(heartbeatRuns).values({ companyId, agentId, status: "succeeded" }).returning();
|
||||
const scopedCompanies: Array<string | undefined> = [];
|
||||
const recordScope = () => scopedCompanies.push(scopedTelemetryCompanyId());
|
||||
mockTrackAgentTaskRun.mockImplementationOnce(recordScope).mockImplementationOnce(recordScope);
|
||||
|
||||
await emitAgentTaskRun(db, run);
|
||||
await emitAgentTaskRunById(db, { runId: run.id, companyId });
|
||||
|
||||
expect(scopedCompanies).toEqual([companyId, companyId]);
|
||||
expect(scopedTelemetryCompanyId()).toBeUndefined();
|
||||
});
|
||||
|
||||
it("emits the same event from a small scoped run read without loading execution payloads", async () => {
|
||||
await seedCompanyAndAgent();
|
||||
const payload = randomBytes(128 * 1024).toString("base64");
|
||||
|
||||
@@ -124,6 +124,7 @@ import {
|
||||
getEmbeddedPostgresTestSupport,
|
||||
startEmbeddedPostgresTestDatabase,
|
||||
} from "./helpers/embedded-postgres.js";
|
||||
import { scopedTelemetryCompanyId } from "../telemetry-company-scope.js";
|
||||
import type { EnvironmentLease } from "@paperclipai/shared";
|
||||
import type { NativeExecutionInput } from "../vendor/paperclip-runner/index.js";
|
||||
import { runningProcesses, type AdapterExecutionContext, type AdapterExecutionResult } from "../adapters/index.ts";
|
||||
@@ -10035,11 +10036,13 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
|
||||
});
|
||||
|
||||
it("tracks the first heartbeat with the agent role instead of adapter type", async () => {
|
||||
const { agentId, runId } = await seedRunFixture({
|
||||
const { companyId, agentId, runId } = await seedRunFixture({
|
||||
agentStatus: "running",
|
||||
includeIssue: false,
|
||||
});
|
||||
const heartbeat = heartbeatService(db);
|
||||
const scopedCompanies: Array<string | undefined> = [];
|
||||
mockTrackAgentFirstHeartbeat.mockImplementationOnce(() => scopedCompanies.push(scopedTelemetryCompanyId()));
|
||||
|
||||
await heartbeat.cancelRun(runId);
|
||||
|
||||
@@ -10050,6 +10053,7 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
|
||||
agentId,
|
||||
}),
|
||||
);
|
||||
expect(scopedCompanies).toEqual([companyId]);
|
||||
});
|
||||
|
||||
it("preserves first-heartbeat telemetry after a timer interval claim", async () => {
|
||||
|
||||
@@ -185,6 +185,9 @@ describe("issue telemetry routes", () => {
|
||||
}, 60_000);
|
||||
|
||||
it("emits task-completed telemetry with the agent role, adapter type, and model", async () => {
|
||||
const { scopedTelemetryCompanyId } = await import("../telemetry-company-scope.js");
|
||||
const scopedCompanies: Array<string | undefined> = [];
|
||||
mockTrackAgentTaskCompleted.mockImplementationOnce(() => scopedCompanies.push(scopedTelemetryCompanyId()));
|
||||
mockAgentService.getById.mockResolvedValue({
|
||||
id: "agent-1",
|
||||
companyId: "company-1",
|
||||
@@ -213,6 +216,7 @@ describe("issue telemetry routes", () => {
|
||||
taskId: "11111111-1111-4111-8111-111111111111",
|
||||
});
|
||||
});
|
||||
expect(scopedCompanies).toEqual(["company-1"]);
|
||||
}, 10_000);
|
||||
|
||||
it("does not emit agent task-completed telemetry for board-driven completions", async () => {
|
||||
|
||||
@@ -20,6 +20,7 @@ import {
|
||||
startEmbeddedPostgresTestDatabase,
|
||||
} from "./helpers/embedded-postgres.js";
|
||||
import { issueThreadInteractionService } from "../services/issue-thread-interactions.js";
|
||||
import { scopedTelemetryCompanyId } from "../telemetry-company-scope.js";
|
||||
|
||||
const telemetryMocks = vi.hoisted(() => ({
|
||||
track: vi.fn(),
|
||||
@@ -47,6 +48,7 @@ describeEmbeddedPostgres("issueThreadInteractionService telemetry", () => {
|
||||
|
||||
beforeEach(() => {
|
||||
telemetryMocks.track.mockClear();
|
||||
telemetryMocks.track.mockImplementation(() => scopedTelemetryCompanyId());
|
||||
telemetryMocks.getTelemetryClient.mockReturnValue({
|
||||
track: telemetryMocks.track,
|
||||
hashPrivateRef: vi.fn((value: string) => `hashed:${value}`),
|
||||
@@ -135,6 +137,11 @@ describeEmbeddedPostgres("issueThreadInteractionService telemetry", () => {
|
||||
return calls.at(-1)?.[1] as Record<string, unknown>;
|
||||
}
|
||||
|
||||
function scopesOf(name: string) {
|
||||
return telemetryMocks.track.mock.calls.flatMap(([eventName], index) =>
|
||||
eventName === name ? [telemetryMocks.track.mock.results[index]?.value] : []);
|
||||
}
|
||||
|
||||
it("emits deprecated resolver alias use at creation without interaction content", async () => {
|
||||
const { companyId, issueId } = await seedIssue("Deprecated alias telemetry issue title");
|
||||
|
||||
@@ -161,6 +168,7 @@ describeEmbeddedPostgres("issueThreadInteractionService telemetry", () => {
|
||||
interaction_kind: "request_confirmation",
|
||||
used_deprecated_resolver_policy_alias: true,
|
||||
});
|
||||
expect(scopesOf("interaction.created")).toEqual([companyId]);
|
||||
});
|
||||
|
||||
it("emits skipped resolution telemetry for composer Skip", async () => {
|
||||
@@ -197,6 +205,7 @@ describeEmbeddedPostgres("issueThreadInteractionService telemetry", () => {
|
||||
target_type: "none",
|
||||
});
|
||||
expectNoRawInteractionIds(dimensions);
|
||||
expect(scopesOf("interaction.resolved")).toEqual([companyId]);
|
||||
});
|
||||
|
||||
it("emits accepted suggested-task telemetry with created and skipped task counts", async () => {
|
||||
|
||||
@@ -5,7 +5,7 @@ import path from "node:path";
|
||||
import express from "express";
|
||||
import request from "supertest";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { trackAgentTaskRun, trackCompanyImported, trackRoutineCreated } from "@paperclipai/shared/telemetry";
|
||||
import { trackAgentTaskRun, trackCompanyImported, trackRoutineCreated, trackRoutineRun } from "@paperclipai/shared/telemetry";
|
||||
|
||||
const hoisted = vi.hoisted(() => ({ instanceRoot: "", posthogNodeLoads: 0 }));
|
||||
|
||||
@@ -31,6 +31,7 @@ import {
|
||||
capturePostHogEventContext,
|
||||
postHogRequestContextMiddlewareForEnv,
|
||||
} from "../services/posthog-direct/request-context.js";
|
||||
import { withTelemetryCompany } from "../telemetry-company-scope.js";
|
||||
import { serverVersion } from "../version.js";
|
||||
|
||||
const COLLECTOR = "http://collector.test/ingest";
|
||||
@@ -120,6 +121,44 @@ describe("server telemetry sender selection", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it("self-hosted: a background company scope leaves the collector envelope byte-identical", async () => {
|
||||
const client = initTelemetry({ enabled: true }, { env: {} })!;
|
||||
await withTelemetryCompany("company-bg", async () => {
|
||||
await Promise.resolve();
|
||||
trackAgentTaskRun(client, { agentId: "agent-1", state: "succeeded" });
|
||||
trackRoutineRun(client, { source: "schedule", status: "issue_created" });
|
||||
});
|
||||
await shutdownTelemetry();
|
||||
|
||||
expect(hoisted.posthogNodeLoads).toBe(0);
|
||||
expect(fetchMock).toHaveBeenCalledTimes(1);
|
||||
const body = String((fetchMock.mock.calls[0]![1] as RequestInit).body);
|
||||
const parsed = JSON.parse(body);
|
||||
expect(body).toBe(
|
||||
JSON.stringify({
|
||||
app: "paperclip",
|
||||
schemaVersion: "1",
|
||||
installId: readState().installId,
|
||||
version: serverVersion,
|
||||
events: [
|
||||
{ name: "server.started", occurredAt: parsed.events[0].occurredAt, dimensions: { os_platform: process.platform } },
|
||||
{
|
||||
name: "agent.task_run",
|
||||
occurredAt: parsed.events[1].occurredAt,
|
||||
dimensions: { agent_id: "agent-1", state: "succeeded" },
|
||||
},
|
||||
{
|
||||
name: "routine.run",
|
||||
occurredAt: parsed.events[2].occurredAt,
|
||||
dimensions: { source: "schedule", status: "issue_created" },
|
||||
},
|
||||
],
|
||||
batchId: parsed.batchId,
|
||||
}),
|
||||
);
|
||||
expect(body).not.toContain("company");
|
||||
});
|
||||
|
||||
it("self-hosted: ignores posthog-js tracing headers", async () => {
|
||||
expect(postHogRequestContextMiddlewareForEnv({})).toBeNull();
|
||||
expect(postHogRequestContextMiddlewareForEnv({ PAPERCLIP_POSTHOG_PROJECT_TOKEN: "phc_test" })).toBeNull();
|
||||
@@ -345,4 +384,52 @@ describe("server telemetry sender selection", () => {
|
||||
expect(routine.distinctId).toBe("workspace:stack-123");
|
||||
expect(routine.properties).not.toHaveProperty("$session_id");
|
||||
});
|
||||
|
||||
it("Cloud: background events carry the company in scope", async () => {
|
||||
const posthog = fakePostHog();
|
||||
const client = initTelemetry({ enabled: true }, { env: CLOUD_ENV, postHogFactory: posthog.factory })!;
|
||||
await withTelemetryCompany("company-bg", async () => {
|
||||
await new Promise((resolve) => setTimeout(resolve, 1));
|
||||
trackAgentTaskRun(client, { agentId: "agent-1", state: "succeeded" });
|
||||
});
|
||||
withTelemetryCompany("company-bg", () => trackRoutineRun(client, { source: "schedule", status: "issue_created" }));
|
||||
trackRoutineCreated(client);
|
||||
await shutdownTelemetry();
|
||||
|
||||
const run = posthog.captured.find((message) => message.event === "agent.task_run")!;
|
||||
expect(run.distinctId).toBe("workspace:stack-123");
|
||||
expect(run.properties).toMatchObject({ company_id: "company-bg", stack_id: "stack-123", $process_person_profile: false });
|
||||
expect(run.properties).not.toHaveProperty("user_id");
|
||||
const routineRun = posthog.captured.find((message) => message.event === "routine.run")!;
|
||||
expect(routineRun.properties).toMatchObject({ company_id: "company-bg", stack_id: "stack-123" });
|
||||
const unscoped = posthog.captured.find((message) => message.event === "routine.created")!;
|
||||
expect(unscoped.properties).not.toHaveProperty("company_id");
|
||||
});
|
||||
|
||||
it("Cloud: a live request's company wins over a background company scope", async () => {
|
||||
const posthog = fakePostHog();
|
||||
initTelemetry({ enabled: true }, { env: CLOUD_ENV, postHogFactory: posthog.factory });
|
||||
const app = express();
|
||||
app.use(postHogRequestContextMiddlewareForEnv(CLOUD_ENV)!);
|
||||
app.use((req, _res, next) => {
|
||||
req.actor = { type: "board", userId: "user-1", source: "session" };
|
||||
next();
|
||||
});
|
||||
app.post("/companies/:companyId/routines", (_req, res) => {
|
||||
withTelemetryCompany("company-bg", () => trackRoutineCreated(getTelemetryClient()!));
|
||||
res.json({});
|
||||
});
|
||||
app.post("/runs", (_req, res) => {
|
||||
withTelemetryCompany("company-bg", () => trackRoutineCreated(getTelemetryClient()!));
|
||||
res.json({});
|
||||
});
|
||||
await request(app).post("/companies/company-9/routines").expect(200);
|
||||
await request(app).post("/runs").expect(200);
|
||||
await shutdownTelemetry();
|
||||
|
||||
const routines = posthog.captured.filter((message) => message.event === "routine.created");
|
||||
expect(routines).toHaveLength(2);
|
||||
expect(routines[0]!.properties).toMatchObject({ company_id: "company-9", user_id: "user-1" });
|
||||
expect(routines[1]!.properties).toMatchObject({ company_id: "company-bg", user_id: "user-1" });
|
||||
});
|
||||
});
|
||||
@@ -21,6 +21,7 @@ import {
|
||||
getEmbeddedPostgresTestSupport,
|
||||
startEmbeddedPostgresTestDatabase,
|
||||
} from "./helpers/embedded-postgres.js";
|
||||
import { scopedTelemetryCompanyId } from "../telemetry-company-scope.js";
|
||||
|
||||
const mockTelemetryClient = vi.hoisted(() => ({ track: vi.fn() }));
|
||||
const mockTrackRoutineRun = vi.hoisted(() => vi.fn());
|
||||
@@ -168,4 +169,14 @@ describeEmbeddedPostgres("routine run telemetry", () => {
|
||||
status: "issue_created",
|
||||
});
|
||||
});
|
||||
|
||||
it("tracks a routine run in the routine's company scope", async () => {
|
||||
const { routine, svc } = await seedFixture();
|
||||
const scopedCompanies: Array<string | undefined> = [];
|
||||
mockTrackRoutineRun.mockImplementationOnce(() => scopedCompanies.push(scopedTelemetryCompanyId()));
|
||||
|
||||
await svc.runRoutine(routine.id, { source: "manual" });
|
||||
|
||||
expect(scopedCompanies).toEqual([routine.companyId]);
|
||||
});
|
||||
});
|
||||
@@ -26,8 +26,9 @@ import {
|
||||
getEmbeddedPostgresTestSupport,
|
||||
startEmbeddedPostgresTestDatabase,
|
||||
} from "./helpers/embedded-postgres.js";
|
||||
import { scopedTelemetryCompanyId } from "../telemetry-company-scope.js";
|
||||
|
||||
const track = vi.fn();
|
||||
const track = vi.fn((..._args: unknown[]) => scopedTelemetryCompanyId());
|
||||
const isRegisteredEventName = vi.fn(() => true);
|
||||
vi.mock("../telemetry.js", () => ({
|
||||
getTelemetryClient: () => ({ track, isRegisteredEventName }),
|
||||
@@ -44,6 +45,11 @@ const describeEmbeddedPostgres = embeddedPostgresSupport.supported
|
||||
|
||||
type Db = ReturnType<typeof createDb>;
|
||||
|
||||
function scopesOf(name: string) {
|
||||
return track.mock.calls.flatMap(([eventName], index) =>
|
||||
eventName === name ? [track.mock.results[index]?.value] : []);
|
||||
}
|
||||
|
||||
function createdEvents() {
|
||||
return track.mock.calls.filter(([name]) => name === "connection.created");
|
||||
}
|
||||
@@ -235,6 +241,7 @@ describeEmbeddedPostgres("connector lifecycle telemetry (tool-access)", () => {
|
||||
status: "active",
|
||||
enabled: true,
|
||||
});
|
||||
expect(scopesOf("connection.created")).toEqual([company.id]);
|
||||
}, 30_000);
|
||||
|
||||
it("updateConnection emits committed transitions only; metadata saves stay silent", async () => {
|
||||
@@ -266,6 +273,7 @@ describeEmbeddedPostgres("connector lifecycle telemetry (tool-access)", () => {
|
||||
previous_status: "active",
|
||||
status: "active",
|
||||
});
|
||||
expect(scopesOf("connection.updated")).toEqual([company.id]);
|
||||
|
||||
track.mockClear();
|
||||
await svc.updateConnection(connection.id, { name: "Renamed fixture" });
|
||||
|
||||
@@ -29,12 +29,13 @@ import {
|
||||
getEmbeddedPostgresTestSupport,
|
||||
startEmbeddedPostgresTestDatabase,
|
||||
} from "./helpers/embedded-postgres.js";
|
||||
import { scopedTelemetryCompanyId } from "../telemetry-company-scope.js";
|
||||
|
||||
// Proposal wrappers drop unregistered names inside the real client, so the
|
||||
// integration assertions use a client double that reports the proposed event
|
||||
// as registered. `track` observes exactly what the emitter would hand the
|
||||
// real client after schema adoption.
|
||||
const track = vi.fn();
|
||||
const track = vi.fn((..._args: unknown[]) => scopedTelemetryCompanyId());
|
||||
const isRegisteredEventName = vi.fn(() => true);
|
||||
vi.mock("../telemetry.js", () => ({
|
||||
getTelemetryClient: () => ({ track, isRegisteredEventName }),
|
||||
@@ -236,6 +237,11 @@ async function createRemoteMcpTool(db: Db, companyId: string) {
|
||||
return { application: application!, connection: connection!, catalogEntry: catalogEntry! };
|
||||
}
|
||||
|
||||
function scopesOf(name: string) {
|
||||
return track.mock.calls.flatMap(([eventName], index) =>
|
||||
eventName === name ? [track.mock.results[index]?.value] : []);
|
||||
}
|
||||
|
||||
async function settledEvents(expectedCount: number) {
|
||||
await vi.waitFor(() => {
|
||||
expect(
|
||||
@@ -336,6 +342,7 @@ describeEmbeddedPostgres("gateway connector invocation telemetry", () => {
|
||||
status: "succeeded",
|
||||
origin: "agent",
|
||||
});
|
||||
expect(scopesOf("connection.invoked")).toEqual([f.company.id]);
|
||||
}, 30_000);
|
||||
|
||||
it("emits a single failed event when post-success audit work throws (no contradictory pair)", async () => {
|
||||
|
||||
@@ -208,6 +208,7 @@ import {
|
||||
} from "@paperclipai/shared";
|
||||
import { trackAgentTaskCompleted } from "@paperclipai/shared/telemetry";
|
||||
import { getTelemetryClient } from "../telemetry.js";
|
||||
import { withTelemetryCompany } from "../telemetry-company-scope.js";
|
||||
import { isUniqueViolation } from "../db-errors.js";
|
||||
import type { StorageService } from "../storage/types.js";
|
||||
import { validate, validateIssueMutationBody } from "../middleware/validate.js";
|
||||
@@ -16704,13 +16705,13 @@ export function issueRoutes(
|
||||
typeof actorAgent.adapterConfig?.model === "string"
|
||||
? actorAgent.adapterConfig.model
|
||||
: undefined;
|
||||
trackAgentTaskCompleted(tc, {
|
||||
withTelemetryCompany(issue.companyId, () => trackAgentTaskCompleted(tc, {
|
||||
agentRole: actorAgent.role,
|
||||
agentId: actorAgent.id,
|
||||
adapterType: actorAgent.adapterType,
|
||||
model,
|
||||
taskId: issue.id,
|
||||
});
|
||||
}));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -4,10 +4,11 @@ import { trackAgentTaskRun } from "@paperclipai/shared/telemetry";
|
||||
import { parseObject } from "../adapters/utils.js";
|
||||
import { logger } from "../middleware/logger.js";
|
||||
import { getTelemetryClient } from "../telemetry.js";
|
||||
import { withTelemetryCompany } from "../telemetry-company-scope.js";
|
||||
|
||||
type HeartbeatRun = typeof heartbeatRuns.$inferSelect;
|
||||
type TelemetryRun = Pick<HeartbeatRun,
|
||||
"id" | "agentId" | "status" | "startedAt" | "finishedAt" | "usageJson" | "contextSnapshot">;
|
||||
"id" | "companyId" | "agentId" | "status" | "startedAt" | "finishedAt" | "usageJson" | "contextSnapshot">;
|
||||
|
||||
function readNonEmptyString(value: unknown): string | null {
|
||||
return typeof value === "string" && value.trim().length > 0
|
||||
@@ -79,7 +80,7 @@ export async function emitAgentTaskRun(db: Db, run: TelemetryRun): Promise<void>
|
||||
readNonEmptyString(parseObject(run.contextSnapshot).issueId) ??
|
||||
undefined;
|
||||
|
||||
trackAgentTaskRun(client, {
|
||||
withTelemetryCompany(run.companyId, () => trackAgentTaskRun(client, {
|
||||
agentId: run.agentId,
|
||||
state: run.status,
|
||||
...(agent
|
||||
@@ -116,7 +117,7 @@ export async function emitAgentTaskRun(db: Db, run: TelemetryRun): Promise<void>
|
||||
}
|
||||
: {}),
|
||||
...(taskId ? { taskId } : {}),
|
||||
});
|
||||
}));
|
||||
} catch (err) {
|
||||
logger.warn(
|
||||
{ err, runId: run.id },
|
||||
@@ -134,6 +135,7 @@ export async function emitAgentTaskRunById(
|
||||
const run = await db
|
||||
.select({
|
||||
id: heartbeatRuns.id,
|
||||
companyId: heartbeatRuns.companyId,
|
||||
agentId: heartbeatRuns.agentId,
|
||||
status: heartbeatRuns.status,
|
||||
startedAt: heartbeatRuns.startedAt,
|
||||
|
||||
@@ -8,6 +8,7 @@ import {
|
||||
} from "@paperclipai/shared/telemetry";
|
||||
import { logger } from "../middleware/logger.js";
|
||||
import { getTelemetryClient } from "../telemetry.js";
|
||||
import { withTelemetryCompany } from "../telemetry-company-scope.js";
|
||||
|
||||
type ToolConnectionRow = typeof toolConnections.$inferSelect;
|
||||
type ToolInvocationRow = typeof toolInvocations.$inferSelect;
|
||||
@@ -93,14 +94,14 @@ export function emitConnectionCreated(
|
||||
const client = getTelemetryClient();
|
||||
if (!client) return;
|
||||
if (!isToolPurpose(connection)) return;
|
||||
trackConnectionCreated(client, {
|
||||
withTelemetryCompany(connection.companyId, () => trackConnectionCreated(client, {
|
||||
connector_key: connectorKeyForConnection(connection),
|
||||
transport: connection.transport,
|
||||
auth_kind: connection.authKind,
|
||||
setup_flow: setupFlow,
|
||||
status: connection.status,
|
||||
enabled: connection.enabled,
|
||||
});
|
||||
}));
|
||||
} catch (err) {
|
||||
logger.warn(
|
||||
{ err, connectionId: connection.id },
|
||||
@@ -131,7 +132,7 @@ export function emitConnectionUpdated(
|
||||
) {
|
||||
return;
|
||||
}
|
||||
trackConnectionUpdated(client, {
|
||||
withTelemetryCompany(connection.companyId, () => trackConnectionUpdated(client, {
|
||||
connector_key: connectorKeyForConnection(connection),
|
||||
transport: connection.transport,
|
||||
auth_kind: connection.authKind,
|
||||
@@ -140,7 +141,7 @@ export function emitConnectionUpdated(
|
||||
status: connection.status,
|
||||
previous_enabled: previous.enabled,
|
||||
enabled: connection.enabled,
|
||||
});
|
||||
}));
|
||||
} catch (err) {
|
||||
logger.warn(
|
||||
{ err, connectionId: connection.id },
|
||||
@@ -212,13 +213,13 @@ export async function emitConnectionInvoked(
|
||||
? Math.max(0, Math.round((completedAtMs - startedAtMs) / 1000))
|
||||
: undefined;
|
||||
|
||||
trackConnectionInvoked(client, {
|
||||
withTelemetryCompany(connection.companyId, () => trackConnectionInvoked(client, {
|
||||
connector_key: connectorKeyForConnection(connection),
|
||||
transport: connection.transport,
|
||||
status: invocation.status,
|
||||
origin: invocationOrigin(invocation),
|
||||
...(durationSeconds === undefined ? {} : { duration_seconds: durationSeconds }),
|
||||
});
|
||||
}));
|
||||
} catch (err) {
|
||||
logger.warn(
|
||||
{ err, invocationId },
|
||||
|
||||
@@ -5,6 +5,7 @@ import { agents, heartbeatRuns, type Db } from "@paperclipai/db";
|
||||
import { and, asc, desc, eq, lte, sql } from "drizzle-orm";
|
||||
import { trackAgentFirstHeartbeat } from "@paperclipai/shared/telemetry";
|
||||
import { getTelemetryClient } from "../../telemetry.js";
|
||||
import { withTelemetryCompany } from "../../telemetry-company-scope.js";
|
||||
import { logActivity, publishActivity, type ActivityPublication } from "../activity-log.js";
|
||||
import { publishLiveEvent } from "../live-events.js";
|
||||
import { finishedIdleRun, finishedIdleWake, finishedFailedIdleLease } from "../idle-work-proof.js";
|
||||
@@ -116,7 +117,7 @@ export function createHeartbeatAgentStatus(db: Db) {
|
||||
if (!result?.updated) return;
|
||||
if (result.isFirstHeartbeat) {
|
||||
const client = getTelemetryClient();
|
||||
if (client) trackAgentFirstHeartbeat(client, { agentRole: result.updated.role, agentId: result.updated.id });
|
||||
if (client) withTelemetryCompany(result.updated.companyId, () => trackAgentFirstHeartbeat(client, { agentRole: result.updated.role, agentId: result.updated.id }));
|
||||
}
|
||||
publishStatus(result.updated, outcome);
|
||||
}
|
||||
|
||||
@@ -147,6 +147,7 @@ import { z } from "zod";
|
||||
import { logger } from "../middleware/logger.js";
|
||||
import { conflict, forbidden, notFound, unprocessable } from "../errors.js";
|
||||
import { getTelemetryClient } from "../telemetry.js";
|
||||
import { withTelemetryCompany } from "../telemetry-company-scope.js";
|
||||
import { authorizationService } from "./authorization.js";
|
||||
import { isCloudManagedInstance } from "./cloud-instance.js";
|
||||
import { resolveDeploymentMode } from "../config-file.js";
|
||||
@@ -1797,7 +1798,7 @@ async function emitInteractionResolvedTelemetry(
|
||||
? (roleByAgentId.get(interaction.createdByAgentId) ?? undefined)
|
||||
: undefined;
|
||||
|
||||
trackInteractionResolved(telemetryClient, {
|
||||
withTelemetryCompany(interaction.companyId, () => trackInteractionResolved(telemetryClient, {
|
||||
interactionKind:
|
||||
interaction.kind === "connection_intent" || interaction.kind === "suggest_routine"
|
||||
|| interaction.kind === "propose_responsibility_bar" || interaction.kind === "propose_automation"
|
||||
@@ -1814,7 +1815,7 @@ async function emitInteractionResolvedTelemetry(
|
||||
}),
|
||||
legacyInheritedRestriction:
|
||||
interaction.resolverPolicyProvenance === "legacy_inherited_restriction",
|
||||
});
|
||||
}));
|
||||
} catch (error) {
|
||||
console.error(
|
||||
"[paperclip] Failed to emit interaction.resolved telemetry",
|
||||
@@ -1823,7 +1824,8 @@ async function emitInteractionResolvedTelemetry(
|
||||
}
|
||||
}
|
||||
|
||||
function emitInteractionCreatedTelemetry(args: {
|
||||
function emitInteractionCreatedTelemetry({ companyId, ...args }: {
|
||||
companyId: string;
|
||||
interactionKind: IssueThreadInteractionKind;
|
||||
usedDeprecatedResolverPolicyAlias: boolean;
|
||||
}) {
|
||||
@@ -1831,7 +1833,7 @@ function emitInteractionCreatedTelemetry(args: {
|
||||
if (!telemetryClient) return;
|
||||
|
||||
try {
|
||||
trackInteractionCreated(telemetryClient, {
|
||||
withTelemetryCompany(companyId, () => trackInteractionCreated(telemetryClient, {
|
||||
...args,
|
||||
interactionKind:
|
||||
args.interactionKind === "connection_intent" || args.interactionKind === "suggest_routine"
|
||||
@@ -1839,7 +1841,7 @@ function emitInteractionCreatedTelemetry(args: {
|
||||
|| args.interactionKind === "propose_dashboard_changes"
|
||||
? "other"
|
||||
: args.interactionKind,
|
||||
});
|
||||
}));
|
||||
} catch (error) {
|
||||
console.error(
|
||||
"[paperclip] Failed to emit interaction.created telemetry",
|
||||
@@ -3363,6 +3365,7 @@ export function issueThreadInteractionService(
|
||||
|
||||
const interaction = hydrateInteraction(created) as ConnectionIntentInteraction;
|
||||
if (inserted) emitInteractionCreatedTelemetry({
|
||||
companyId: issue.companyId,
|
||||
interactionKind: "connection_intent",
|
||||
usedDeprecatedResolverPolicyAlias: false,
|
||||
});
|
||||
@@ -3555,6 +3558,7 @@ export function issueThreadInteractionService(
|
||||
return row!;
|
||||
});
|
||||
emitInteractionCreatedTelemetry({
|
||||
companyId: issue.companyId,
|
||||
interactionKind: "suggest_routine",
|
||||
usedDeprecatedResolverPolicyAlias: false,
|
||||
});
|
||||
@@ -3881,6 +3885,7 @@ export function issueThreadInteractionService(
|
||||
return { row: row!, replacedIds: replaced.map((entry) => entry.id) };
|
||||
});
|
||||
emitInteractionCreatedTelemetry({
|
||||
companyId: issue.companyId,
|
||||
interactionKind: "propose_responsibility_bar",
|
||||
usedDeprecatedResolverPolicyAlias: false,
|
||||
});
|
||||
@@ -4056,6 +4061,7 @@ export function issueThreadInteractionService(
|
||||
return { row: row!, replacedIds: replaced.map((entry) => entry.id) };
|
||||
});
|
||||
emitInteractionCreatedTelemetry({
|
||||
companyId: issue.companyId,
|
||||
interactionKind: "propose_dashboard_changes",
|
||||
usedDeprecatedResolverPolicyAlias: false,
|
||||
});
|
||||
@@ -4231,6 +4237,7 @@ export function issueThreadInteractionService(
|
||||
return { row: row!, replacedIds: replaced.map((entry) => entry.id) };
|
||||
});
|
||||
emitInteractionCreatedTelemetry({
|
||||
companyId: issue.companyId,
|
||||
interactionKind: "propose_automation",
|
||||
usedDeprecatedResolverPolicyAlias: false,
|
||||
});
|
||||
@@ -5457,6 +5464,7 @@ export function issueThreadInteractionService(
|
||||
}
|
||||
const interaction = hydrateInteraction(created);
|
||||
emitInteractionCreatedTelemetry({
|
||||
companyId: issue.companyId,
|
||||
interactionKind: interaction.kind,
|
||||
usedDeprecatedResolverPolicyAlias,
|
||||
});
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { AsyncLocalStorage } from "node:async_hooks";
|
||||
import type { Request, RequestHandler } from "express";
|
||||
import { isPostHogDirectDeliveryEnabled, resolveCloudPostHogConfig } from "../cloud-posthog-config.js";
|
||||
import { scopedTelemetryCompanyId } from "../../telemetry-company-scope.js";
|
||||
|
||||
/**
|
||||
* Request-scoped analytics context for Paperclip Cloud direct delivery.
|
||||
@@ -98,13 +99,18 @@ function knownCompanyId(req: Request): string | undefined {
|
||||
/** Snapshot of the current request's analytics context, taken at track time. */
|
||||
export function capturePostHogEventContext(): PostHogEventContext | undefined {
|
||||
const store = storage.getStore();
|
||||
if (!store || !store.active) return undefined;
|
||||
const context: PostHogEventContext = {};
|
||||
const userId = actingUserId(store.req);
|
||||
const companyId = knownCompanyId(store.req);
|
||||
if (userId) context.userId = userId;
|
||||
if (companyId) context.companyId = companyId;
|
||||
if (store.sessionId) context.sessionId = store.sessionId;
|
||||
if (store.windowId) context.windowId = store.windowId;
|
||||
if (store?.active) {
|
||||
const userId = actingUserId(store.req);
|
||||
const companyId = knownCompanyId(store.req);
|
||||
if (userId) context.userId = userId;
|
||||
if (companyId) context.companyId = companyId;
|
||||
if (store.sessionId) context.sessionId = store.sessionId;
|
||||
if (store.windowId) context.windowId = store.windowId;
|
||||
}
|
||||
if (!context.companyId) {
|
||||
const scopedCompanyId = scopedTelemetryCompanyId();
|
||||
if (scopedCompanyId) context.companyId = scopedCompanyId;
|
||||
}
|
||||
return Object.keys(context).length > 0 ? context : undefined;
|
||||
}
|
||||
@@ -69,6 +69,7 @@ import { trackRoutineRun } from "@paperclipai/shared/telemetry";
|
||||
import { HttpError, badRequest, conflict, forbidden, notFound, unauthorized, unprocessable } from "../errors.js";
|
||||
import { logger } from "../middleware/logger.js";
|
||||
import { getTelemetryClient } from "../telemetry.js";
|
||||
import { withTelemetryCompany } from "../telemetry-company-scope.js";
|
||||
import { getConfiguredSecretProvider } from "../secrets/configured-provider.js";
|
||||
import { issueService } from "./issues.js";
|
||||
import { assertAssignableAgent } from "./agent-assignability.js";
|
||||
@@ -2083,10 +2084,10 @@ export function routineService(
|
||||
|
||||
const telemetryClient = getTelemetryClient();
|
||||
if (telemetryClient) {
|
||||
trackRoutineRun(telemetryClient, {
|
||||
withTelemetryCompany(input.routine.companyId, () => trackRoutineRun(telemetryClient, {
|
||||
source: run.source,
|
||||
status: run.status,
|
||||
});
|
||||
}));
|
||||
}
|
||||
|
||||
return run;
|
||||
|
||||
@@ -0,0 +1,11 @@
|
||||
import { AsyncLocalStorage } from "node:async_hooks";
|
||||
|
||||
const storage = new AsyncLocalStorage<string>();
|
||||
|
||||
export function withTelemetryCompany<T>(companyId: string | null | undefined, fn: () => T): T {
|
||||
return companyId ? storage.run(companyId, fn) : fn();
|
||||
}
|
||||
|
||||
export function scopedTelemetryCompanyId(): string | undefined {
|
||||
return storage.getStore();
|
||||
}
|
||||
@@ -367,7 +367,7 @@
|
||||
"packages/shared/src/standing-checks.ts": "7ab71e4b6984e41e1c1dfd1bddf0116041d5abf1",
|
||||
"packages/shared/src/subscriptions.ts": "ef70e63395ca967bbc30776b71560e96eb354a75",
|
||||
"packages/shared/src/summary-slot.test.ts": "5f8e222dea425f0d5bff0adb5c683087bd7cf7fb",
|
||||
"packages/shared/src/telemetry/README.md": "50ae8992b33c4141de443d42c57de24357e3d0aa",
|
||||
"packages/shared/src/telemetry/README.md": "eaa75efee24352f97ddc7d72b2e99be8b39a241b",
|
||||
"packages/shared/src/telemetry/client-types.test.ts": "3b630edda4972b929ba4decd7457f226eb9cfc5b",
|
||||
"packages/shared/src/telemetry/client.test.ts": "5a95f2305a393c5b2e7750b10e0ed65da0f30cf8",
|
||||
"packages/shared/src/telemetry/client.ts": "d6534ce8aaa6f69fb2dd03e12487db024e250a96",
|
||||
|
||||
Reference in new issue
Block a user