Commit Graph
4 Commits
Author SHA1 Message Date
DottaandClaude Opus 5.5 55b692d532 feat(telemetry): send Cloud server telemetry directly to PostHog (#22006)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The server sends first-party telemetry events through a collector.
The collector anonymizes the events and copies them into PostHog.
> - Paperclip Cloud is our own service. It needs real IDs (workspace,
user, agent, task) so that a person can go from a PostHog event to the
exact workspace and task.
> - The collector path hashes these IDs. It also cannot link a server
event to the browser session that caused it.
> - This pull request adds a server-owned direct sender for Cloud only.
Self-hosted installs keep the collector path with no change.
> - The benefit is real IDs and session links on Cloud, and no change to
self-hosted privacy.

## Linked Issues or Issue Description

**Subsystem affected**

Server telemetry (`packages/shared/src/telemetry`,
`server/src/telemetry.ts`).

**Problem or motivation**

All Cloud telemetry goes through the collector. The collector hashes the
install ID, agent IDs and task IDs. So, on Cloud, you cannot find the
workspace, user or task of an event. Server events also cannot show in a
browser session replay.

**Proposed solution**

On a Cloud-managed server that has `PAPERCLIP_POSTHOG_PROJECT_TOKEN`,
`PAPERCLIP_CLOUD_STACK_ID` and `PAPERCLIP_POSTHOG_ENVIRONMENT`, send
each telemetry batch straight to PostHog with `posthog-node`. Use the
same event shape that the telemetry backend loader makes today, and add
the real IDs. Keep every other install on the collector.

**Alternatives considered**

Send real IDs through the collector. We did not do this, because the
collector exists to anonymize self-hosted data.

**Roadmap alignment**

This is Cloud analytics infrastructure. It does not change a core
product feature.

## What Changed

- Add `server/src/services/cloud-posthog-config.ts`.
`resolveCloudPostHogConfig()` returns `null` unless the instance is
Cloud-managed and has a token and a stack ID.
`isPostHogDirectDeliveryEnabled()` also requires an environment and the
kill switch on (`PAPERCLIP_POSTHOG_DIRECT` is not `0`).
- Add a narrow `TelemetryTransport` seam to `TelemetryClient` in
`packages/shared`. With a transport, each flushed batch goes to the
transport and never to the collector. Without one, the client does what
it did before. A transport can capture per-event context at track time.
The context does not go into the collector envelope.
- Typed helpers hash `task_id` unless the transport sets
`hashesPrivateRefs: false`. Private `source_ref` values (for example a
private repository name) are customer content, so they stay hashed on
every path, Cloud included, with `source_ref_hashed: true`.
- Add `server/src/services/posthog-direct/`:
- `event-shape.ts` builds PostHog messages with the loader's event
names, metadata keys and key order, the deterministic event uuid
formula, and the unknown-enum fallback. It drops events that the backend
quarantines (unregistered names).
- `enum-fallbacks.generated.ts` holds the backend fallback rules. Enum
values come from the shared generated contract.
- `transport.ts` loads `posthog-node` with dynamic `import()` only on
the first Cloud send.
- `request-context.ts` is Cloud-only Express middleware. It reads only
`X-POSTHOG-SESSION-ID` and `X-POSTHOG-WINDOW-ID`. It ignores
`X-POSTHOG-DISTINCT-ID`. The acting user comes only from the
authenticated request actor. Context stops when the response finishes,
so later background work does not use that user.
- `server/src/telemetry.ts` selects exactly one sender per process. On
start, the Cloud sender sends `$groupidentify` (group `workspace`, key =
stack ID) and one `cloud.workspace_started` event with `install_hash`
(sha256 of the raw install ID). Both go straight to `posthog-node` and
are not in `PAPERCLIP_EVENTS`.
- `distinct_id` is the acting user ID, else `workspace:<stackId>`.
Events without an acting user keep `$process_person_profile: false`.
GeoIP stays off on all server events.
- Shutdown flushes the client and then shuts down `posthog-node`.
- Add `scripts/generate-posthog-direct-fixtures.mjs`. It runs the
telemetry backend's own ingest and loader on one envelope and writes the
golden fixture and the fallback table. The script header tells how to
regenerate them.
- Update `packages/shared/src/telemetry/README.md` with a "Cloud Direct
Delivery" section about the privacy boundary.
- Add `posthog-node@5.38.2` to `server/package.json`. This PR does not
change `pnpm-lock.yaml`, because the lockfile refresh workflow owns it.

Intended differences from the loader output on Cloud: `distinct_id`
(user or workspace, not the HMAC install ID), raw `agent_id`, raw
`task_id` (private `source_ref` stays hashed), and these added
properties: `stack_id`, `company_id` and `user_id` when known,
`environment`, `$groups`, `$session_id` and `$window_id`. The uuid
formula is the same. The uuid is different only when the dimensions are
different (raw `task_id`).

## Verification

- `pnpm verify` returned exit 2 (a global CI input changed: the new
script). I ran focused checks. Required CI supplies the broad coverage.
- Focused tests pass: `packages/shared` `src/telemetry` (6 files, 42
tests). Server: `posthog-direct-telemetry`,
`posthog-direct-event-shape`, `cloud-posthog-config`,
`shared-telemetry-events`, `heartbeat-status-preserving-telemetry`,
`issue-thread-interactions-telemetry`, `native-run-finalizer-telemetry`,
`agent-task-run-telemetry`, `error-handler` (9 files, 115 tests), and
`server-startup-feedback-export` (133 tests).
- `tsc --noEmit` passes for `packages/shared`, `server` and `cli`.
- New tests:
- Config matrix: not Cloud, no token, no stack ID, no environment, kill
switch, browser and replay flags.
- Self-hosted: `posthog-node` is never loaded, the collector gets the
batch, and the envelope bytes are the same as before (with `task_id`
hashed). Tracing headers are ignored.
- Cloud: a private `source_ref` stays hashed and the raw repository name
never reaches the sender.
- Cloud: one batch goes only to PostHog. Real IDs, `$groups`,
`environment` on every event. `cloud.workspace_started` reaches the
PostHog sender with `install_hash`. No other event has `install_hash`.
- Golden: our builder output equals the backend loader output, except
the intended differences above. Key order is the same.
- `distinct_id`: user, agent and anonymous requests. A spoofed
`X-POSTHOG-DISTINCT-ID` never changes `distinct_id` or `user_id`. A bad
session header is dropped. Work after the response ends does not keep
the user (red without the response-end guard, green with it).
- Kill switch and a missing environment both fall back to the collector.
- Local run with a real server and a local capture server (no real
PostHog project):
- Cloud mode (authenticated, local Postgres, `PAPERCLIP_POSTHOG_HOST`
set to the capture server): the capture server got `$groupidentify`,
`cloud.workspace_started`, `server.started` and `goal.created` on
`/batch/`. `goal.created` from a Cloud tenant user had `distinct_id` =
that user, `user_id`, `company_id`, `$session_id`, `$window_id`,
`$groups` and no `$process_person_profile`. The spoofed distinct ID
header was not used. Nothing went to `/ingest`. `SIGTERM` ran the
`telemetry_flush` step and sent the queued events.
- Self-hosted mode (same token variables but no Cloud signal): the
capture server got only `/ingest` with the normal envelope. Nothing went
to `/batch/`.
- Runtime dependency note: `posthog-node` loads with dynamic `import()`
only on Cloud with direct delivery on. The load happens after startup,
outside the boot path. The unit test mocks the package to count loads.
The local run used the real package.

## Risks

- Medium. This is a telemetry change, so a privacy review is requested.
Self-hosted output does not change. The direct path is off unless all
three Cloud variables are set, so a merge does nothing until fleet
config sets them.
- Cloud events with an acting user create PostHog person profiles keyed
by the Paperclip user ID. This is intended under the Cloud terms.
- The fallback table can drift if the backend adds a new normalization
rule. A test fails when a rule does not match the shared enum contract.
Enum values come from the shared contract, so new enum values do not
drift.
- `posthog-node` keeps network failures in its own queue and drops
events that PostHog rejects. The client does not resend these, so events
are not sent twice.
- `pnpm-lock.yaml` is not in this PR. The lockfile refresh workflow adds
`posthog-node` after merge.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edits, local server runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 08:54:32 -06:00
27a825da2f feat(telemetry): track the server OS with the registered contract (#18897)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Its first-party telemetry uses a generated event contract and typed
helpers.
> - The existing events do not identify the server's OS family.
> - A proposed event is dropped until the generated contract registers
it.
> - This PR syncs the registered contract and emits the OS family when
an enabled server telemetry client starts.
> - The benefit is a way to compare Windows, Linux, and macOS
installations while keeping machine details out of the event.

## Linked Issues or Issue Description

**Problem or motivation**

We cannot compare Paperclip installations by host OS. Counting task or
install events does not provide this dimension.

**Proposed solution**

Send one `server.started` event with Node's raw `process.platform` as
`os_platform` each time the server initializes its enabled telemetry
client. Compare distinct install identifiers per platform, rather than
restart event totals.

**Alternatives considered**

A proposal-only emitter would send no data. A separate later
contract-sync PR would leave the feature incomplete. This PR includes
the canonical generated SDK and promotes the emitter together. CPU
architecture and OS versions are outside this feature's scope.

**Roadmap alignment**

This is a narrow telemetry addition to existing server startup, with no
new user flow. The full contract copy also adopts the three existing
connector proposals described in #13578; their schemas are already
registered. The generated file stays an exact copy, so their helpers and
tests must be promoted in the same change.

## What Changed

- Export `trackServerStarted` and call it once when server telemetry
initializes. Repeated initialization returns the same client and emits
no duplicate event.
- Copy the canonical generated telemetry SDK byte for byte from schema
source commit `5369e6b`. Remove the startup proposal marker. Normal
opt-out and CI suppression apply before initialization.
- Promote the three connector events included in the canonical SDK.
Derive enum types from the generated map and preserve raw values. Update
their real-client transport and disabled-state tests.
- Assign startup to the existing `operational_enum_count` retention
class. Document OS privacy, container/WSL interpretation, and
distinct-install counting.
- Add real-client startup transport tests and server initialization
tests, including file opt-out, environment opt-out, and CI suppression.
- Keep startup running when telemetry state storage fails. Publish the
client and start periodic flushing only after the startup event
succeeds. On failure, clean up the client, disable telemetry for the
process, and log a generic warning without the storage path or
underlying error.

## Verification

- `pnpm --filter @paperclipai/shared typecheck`: passed.
- `pnpm --filter @paperclipai/shared build`: passed.
- `pnpm --filter @paperclipai/shared exec vitest run`: 98 files, 1,055
tests passed.
- `pnpm --filter @paperclipai/shared exec vitest run src/telemetry`: 6
files, 39 tests passed.
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/telemetry-startup.test.ts
src/__tests__/shared-telemetry-events.test.ts`: 2 files, 19 tests
passed.
- Exact generated artifact comparison, `git diff --check`, and local
secret/PII review passed.
- Proposal inventory contains no startup or connector proposals. The
repository extractor fails with TypeScript 7's missing compiler API;
rerunning it with the installed TypeScript 5.9 compiler API passed. No
extractor source change is included.
- Initial local server typecheck attempts found missing workspace
dependencies and a missing plugin SDK link in this fresh checkout.
Dependencies and the local SDK link were repaired; `pnpm --filter
@paperclipai/server typecheck` then passed. Full repository typecheck,
tests, and build are covered by the PR CI rather than claimed as local
passes.

Greptile found that a telemetry state-file error could abort startup.
New regression tests reproduced this on the previous head with both an
injected permission error and the real state store blocked by a file.
Both now pass, including no published client, no periodic flushing, no
send, and no retry after failure. Shared telemetry tests and server
typecheck were rerun after the fix.

Full PR CI passed on `a41540b7899bc8a0aedbd78d72f8b69ad1a4b318`,
including typecheck, build, all selected test shards, and the final
verify gate. [Successful workflow attempt
2](https://github.com/paperclipai/paperclip/actions/runs/41752039595/attempts/2).
Attempt 1 was interrupted when the server (1/12) runner received a
shutdown signal; its incomplete result was retained in the PR
discussion. The same-commit shard retry and dependent gate passed.
Greptile reviewed this exact head at 5/5 with all threads resolved.

## Risks

This change starts sending one OS family value per server startup when
telemetry is enabled. It sends no OS version, architecture, hostname,
path, prompt, or machine identifier beyond the existing envelope.
Containers and WSL report the runtime's OS. Event totals reflect restart
frequency, so use distinct install identifiers for OS adoption analysis.

The complete generated SDK also enables the previously registered
connector proposals. Those events keep their existing approved payloads;
no connection names, credentials, tool arguments, or results are added.
Privacy review is requested for both newly active paths.

Deploy the registered schema before merging this app change. Live
production delivery has not been tested. No schema migration is needed.
Revert this PR to stop the new startup emitter and restore the prior
contract.

## Model Used

- Original implementation: Anthropic Claude Opus 5.5, as recorded in the
original commit.
- Completion: OpenAI `gpt-6.1-sol`, xhigh reasoning, through Codex in T3
Code; shell, Git, and automated tests. Context window size is not
exposed by the runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have described the issue in-PR following the feature issue
template
- [x] The final change and description contain no private issue links or
proposal markers
- [x] The existing PR branch is preserved to retain its review history
- [x] I have run the targeted tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-03 21:00:47 -06:00
dottaandPaperclip f16de6026d fix: add periodic flush and graceful shutdown for server-side telemetry
The TelemetryClient only flushed at 50 events, so the server silently
lost all queued telemetry on restart. Add startPeriodicFlush/stop methods
to TelemetryClient, wire up 60s periodic flush in server initTelemetry,
and flush on SIGTERM/SIGINT before exit.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-02 10:47:29 -05:00
dottaandPaperclip 34044cdfce feat: implement app-side telemetry sender
Add the shared telemetry sender, wire the CLI/server emit points,
and cover the config and completion behavior with tests.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-02 10:47:29 -05:00