Files
PaperClipAI/packages/shared/src/telemetry/README.md
T
DottaandClaude Opus 5.5 171e70c11d refactor: drop span and run-log event catalogs from the observability docs (#21779)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - `doc/observability.md` and `doc/run-log-events.md` described every
OpenTelemetry span and every run-log event
> - `rules/data-paths.mdc` told authors to update
`doc/run-log-events.md` for each event, so both files only grew: 2,313
and 1,263 lines
> - Each span and event is already defined next to the code that writes
it, and its name is a literal string that a search finds in one step
> - This pull request removes the per-span and per-event catalogs and
keeps only what code does not say
> - The benefit is two short docs that do not drift from the code, and
no rule that makes them grow again

## Linked Issues or Issue Description

Refs #21670, #21700, and #21743 (the same cleanup for other docs).

**Issue type**

Outdated or confusing documentation.

**Where is the issue?**

`doc/observability.md`, `doc/run-log-events.md`, and
`rules/data-paths.mdc`

**What's wrong?**

The two docs repeated span names, span attributes, and event payloads
that the code defines. Operator setup started at line 138 of
`doc/observability.md`. The rule required a doc edit for every new
run-log event.

**Suggested fix**

Keep operator setup and the Sentry privacy contract. Point to the code
for span and event definitions. Remove the rule line that requires a doc
entry per event.

## What Changed

- `doc/observability.md` is now 619 lines. It keeps: how to enable
tracing, `service.version` resolution, scope, how to enable Sentry, DSN
delivery, environment attribution, privacy settings, server request
data, the core of "Run failure context", browser data, fail-open
behavior, the default capture set, and operator responsibilities. Kept
text is unchanged except the intro and two sentences noted below.
- Deleted from `doc/observability.md`: budget, Slack receipt, task
detail, and browser tracing notes; controller preparation phases; native
Runner spans; sandbox startup spans; sandbox duplex transport; shared
skill preparation; invocation attribution; and about 500 lines of
per-failure detail in "Run failure context".
- Replaced that per-failure detail with one paragraph. It names the
failures that `unexpected` mode keeps out of Sentry, and says the
remaining diagnostic contexts are defined in `server/src/sentry.ts`.
"Default capture set" now says the same.
- `doc/run-log-events.md` is now 23 lines. It says what the run log is,
where events are defined and appended, and keeps the payload storage
note.
- `rules/data-paths.mdc`: the run-log line now says to define an event
where it is written, not in a doc. The span-allowlist line now says the
allowlist is in the code that opens the span. The review levels and
globs do not change.
- Repointed links in two feature maps, two agent-access map files,
`tests/runner-e2e/SUITES.md`, `doc/runner-api-tools.md`, and the
Telemetry README. Updated the `honeycomb-tracing` skill and one code
comment in `packages/adapter-utils/src/http2-bridge-server.ts`.

## Verification

- A script built the new `doc/observability.md` from line ranges of the
old file and asserts the first line of each range.
- A link check on all changed Markdown files found 0 missing files and 0
missing anchors. The remaining inbound anchors (`#enabling-tracing`,
`#sentry-error-monitoring`) still exist.
- No test reads either doc.
- `node scripts/check-maps.mjs` passes.
- `node --test .github/scripts/tests/*.test.mjs`: 808 pass, 0 fail.
- `pnpm verify` was not run. This worktree has no installed
dependencies. The only code change is a comment.

## Risks

- The privacy contract no longer lists each run-failure diagnostic
context field. `server/src/sentry.ts` is now the list. A reviewer of a
Sentry change must read that file.
- No browsable list of span names or run-log event names remains. A
search for the quoted name finds the producer.
- `.greptile/files.json` still scopes `rules/data-paths.mdc` to both
docs. Both files still exist, so it does not change.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Claude Code desktop, with tool
use. Earlier analysis in this session used Claude Fable 5.2.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [ ] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 10:06:05 -06:00

13 KiB

Telemetry Data Contract

This document explains how contributors should use Paperclip's public telemetry contract. It does not duplicate the full list of individual events or dimensions. It documents extra semantic and privacy rules where the generated shape is not sufficient.

The canonical source for first-party event names, dimensions, optionality, allowed primitive value types, and enum descriptions is packages/shared/src/telemetry/generated/paperclip-telemetry.ts.

Shared enum constants live in packages/shared/src/constants.ts. Use those constants when code needs a reusable domain, but treat the generated telemetry types as the final authority for emitted first-party telemetry shapes.

Public Sources

Use these files when reviewing or changing telemetry code:

Contract item Public source
First-party event names PaperclipEventName in generated/paperclip-telemetry.ts
Per-event dimensions and optionality EventDimensionsMap in generated/paperclip-telemetry.ts
Enum descriptions for telemetry dimensions PAPERCLIP_ENUM_DESCRIPTIONS in generated/paperclip-telemetry.ts
Schema version and event envelope helpers SCHEMA_VERSION, makeEvent(), and makeBatch() in generated/paperclip-telemetry.ts
Runtime-safe event names and dimensions TelemetryEventName and TelemetryEventDimensions in types.ts
Allowed primitive dimension values TelemetryDimensionValue in types.ts
Shared reusable enum domains Named exports in constants.ts
First-party typed emit helpers events.ts
Generic client behavior client.ts
Retention windows and event class assignments RETENTION_DAYS and EVENT_RETENTION_CLASS in retention.ts

Do not copy generated event lists or dimension tables into this README. They will drift as the generated contract changes.

Emission Boundary

Paperclip telemetry uses named events with explicit dimension fields. Treat open-ended string dimensions as public contract values, not as a place for user content or private operational data. Do not send PII, secrets, credentials, private paths, prompts, model output, or other sensitive values through telemetry dimensions.

Telemetry emitters send raw dimension values. They must not pre-normalize enum-like values into a reporting form just to match today's known domain.

The receiving layer owns canonicalization. Keeping canonicalization in one place means emitters can stay simple and accurate: emit what the product observed, use the generated contract for required and optional fields, and let the receiving layer decide how legacy spellings, aliases, unknown names, and future values map to a stable reporting shape.

Do not add client-side lowercasing, alias mapping, or fallback mapping unless the generated telemetry contract specifically requires that emitted value.

If a dimension is privacy-protected before emission, emit only the protected value and its matching public marker as defined by the typed helper or generated contract. Do not emit private source material in telemetry dimensions.

Credential-bearing chat setup failures replace provider-controlled error names, messages, and stacks with a fresh generic error before the HTTP error handler reports the crash. The existing error.handler_crash event still uses its generated error_code: string contract; this path emits only Error, never a provider-supplied error name that may contain a credential. This is a privacy boundary, not enum canonicalization. Test the value reaching the telemetry helper as well as the separate crash-reporting and local logging sinks.

Server Startup Privacy

The startup helper emits the raw Node.js operating system family once when an enabled server telemetry client initializes. It omits OS versions, CPU architecture, hostnames, and other machine details. The usual telemetry opt-out and CI suppression apply before client initialization. If state storage fails at startup, the server disables telemetry for that process and continues to run. The warning contains no storage path or underlying error. Container and WSL servers report their runtime platform; this does not identify the underlying physical host's OS. Count distinct installation identifiers by platform to compare installations, since restart event counts also reflect restart frequency.

Contract syncs may also register existing proposals. Remove each adopted proposal marker and update its helper and transport tests in the same change. The connector helpers preserve raw enum values and send only catalog connector keys and the approved lifecycle and invocation dimensions. Connection names, credentials, tool arguments, and tool results stay out of those events.

Interaction Resolver Events

interaction.created records the interaction kind and whether the create request used a deprecated resolver-policy alias. It does not record the prompt, title, options, questions, target identifier, creator identifier, or resolver identifier.

interaction.resolved records the low-cardinality interaction outcome defined in the generated contract. Its legacy_inherited_restriction dimension is true only when stored migration provenance preserves a legacy resolver-policy restriction. It is false for canonical new writes. This dimension describes policy provenance. It does not contain user content or an identifier.

Emit interaction.resolved only after the complete decision transaction commits. Conversational answers include the card outcome, source-message reference, and activity audit in that transaction. A rollback or matching retry must not emit a resolution event. This changes emission timing only: message text, comment IDs, and user IDs remain in the instance database and are not added to telemetry.

Use trackInteractionCreated() and trackInteractionResolved() from events.ts to emit these events. The generated contract remains the authority for their exact dimensions and optionality.

Agent Task Run Events

agent.task_run records one terminal state for one agent run: succeeded, interrupted, failed, cancelled, or timed_out. Emit it once per run, at the run's terminal transition. Do not emit it for a run that is still active.

The event's task_id dimension is optional and privacy-protected. Never emit the raw task id. Pass the raw id to trackAgentTaskRun() in events.ts. The helper calls hashPrivateRef() on client.ts, which derives the emitted value with a salted hash of the per-installation secret.

Use trackAgentTaskRun() to emit this event. The generated contract remains the authority for its exact dimensions and optionality.

Other Data Paths

This document covers Paperclip Telemetry only. The generated Telemetry contract covers neither the Observability path nor the run-log path. Two other data paths document their own contract in their own file:

  • Observability — the operator-enabled OpenTelemetry trace path and Sentry error monitoring.
  • Run-Log Events — events written to the local heartbeat_run_events table.

Dimension Values

Telemetry dimension values must be primitives. Use only the value types allowed by TelemetryDimensionValue:

  • string
  • number
  • boolean

Do not emit null, undefined, arrays, or objects as dimension values. Optional dimensions should be omitted when absent.

When a dimension is enum-like, use the shared constant from constants.ts when one exists. If no shared constant exists, use the generated telemetry type as the domain. In all cases, the generated telemetry type remains the source of truth for the emitted value.

Required, Optional, And Sentinel Values

Required and optional dimensions are defined by EventDimensionsMap.

Required dimensions must be present for every event of that name. Optional dimensions should be emitted only when the value is known and useful.

Sentinel values are only for required fields that have no observed raw value at the emitting layer. Do not use a sentinel to hide a concrete value that is new, custom, or not yet represented by a shared constant. Emit the concrete raw value and let the receiving layer canonicalize it.

Adding Or Changing Telemetry

Client code is responsible for emitting approved telemetry events at the right place in the product. Stable event names, dimensions, and enum domains must come from the generated telemetry contract before normal emitters use them.

For product work that needs to propose a new first-party event before schema registration, use the proposal marker workflow in doc/TELEMETRY_WORKFLOW.md. Those proposed calls stay on client.track(), carry an @ts-expect-error marker on the event-name argument, and are swallowed at runtime until the generated schema registers the event name.

For stable event work:

  1. Start from generated/paperclip-telemetry.ts. The generated types are what reviewers use to verify event names, dimensions, optionality, value types, enum descriptions, and schema version.
  2. Choose stable event and dimension names. Do not include user content, local machine details, secrets, credentials, private paths, or values that are not part of the public event contract.
  3. Use only string, number, or boolean dimension values.
  4. Reuse a shared constant from constants.ts for enum-like dimensions when one exists. If the generated telemetry domain has values beyond a shared constant, keep the emitter aligned with the generated telemetry type.
  5. Keep emitters raw. Do not normalize, alias-map, or lowercase enum-like values in the client unless the generated contract explicitly calls for that emitted value.
  6. Add or update a typed helper in events.ts when the event is first-party and should have a stable helper API.
  7. Update tests for helper behavior, including raw pass-through for enum-like values when that is the intended boundary.
  8. Keep the telemetry transport contract assertions in client.test.ts, which owns the emitted-envelope, retry, and payload-size boundaries.
  9. Update this README only when the contributor workflow, source-of-truth pointers, or durable invariants change. Do not add an event catalog here.

Before opening a pull request, verify that the emitted code, typed helpers, and generated telemetry contract agree. If they disagree, fix the contract or code rather than documenting around the mismatch in this README.

For new first-party events that are not in the generated contract yet, follow the public proposal and promotion workflow in doc/TELEMETRY_WORKFLOW.md.

Retention

Retention windows are documented in retention.ts. Each event is assigned a retention class; the class determines the window in days. This is a housekeeping and query-cost concern managed by data-infra, not a schema concern — updating a retention window does not require a schema version bump.

Current classes:

Class Window Description
operational_enum_count 90 days Enum/boolean/count/bucket events. No token material, no PII.

When a new event carries only enums, booleans, counts, or coarse buckets and no token material or PII, assign it to operational_enum_count in EVENT_RETENTION_CLASS. If no existing class fits, define a new class in RETENTION_DAYS and document it here.

GitHub-synced skills

The legacy import endpoint retains skill.imported with its existing source_type. For source-managed GitHub skills, skill_ref is omitted, including public sources: the saved repository may be private and its key contains user-authored names. Source discovery, selection, and refresh add no first-party telemetry events. Repository URLs, paths, commits, connection IDs, names, and file contents stay out of these telemetry dimensions. Review this suppression in privacy review alongside changes to the legacy import caller; the event schema and envelope are unchanged.