mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-27 11:20:42 +01:00
Nicky LeachandPaperclip
24bbc0f57e
## Thinking Path
> - Paperclip is an open-source agentic AI management platform that
ships anonymous usage telemetry to understand product health and
adoption.
> - The telemetry system has a generated contract
(`packages/shared/src/telemetry/generated/paperclip-telemetry.ts`) that
types every first-party event the product emits.
> - When a product change needs a new first-party event that is not yet
in the generated contract, contributors had no public workflow
explaining how to propose an event or later promote it into the contract
once accepted.
> - The gap leads to confusion at call sites: contributors either skip
tracking entirely or emit untyped events that bypass the privacy and
governance safeguards built into the contract.
> - This PR fills that gap by adding `doc/TELEMETRY_WORKFLOW.md` — a
public contributor guide that covers the full propose → promote
lifecycle: the `@ts-expect-error -- proposed-telemetry(...)` marker, the
canonical multi-line `track()` shape, the TS2578 expiry signal, and the
look-up-by-event-name promotion step.
> - `packages/shared/src/telemetry/README.md` gains a cross-reference so
readers of the data-contract doc can find the workflow guide without
searching.
> - `README.md` gains a one-line pointer in the telemetry section so the
workflow is discoverable from the project entry point.
## Linked Issues or Issue Description
No pre-existing public GitHub issue covers this doc gap. Inline
description:
**Problem:** Contributors adding product telemetry for events not yet in
the generated contract have no documented workflow. The
`@ts-expect-error -- proposed-telemetry(...)` pattern exists in the
codebase but is undocumented, leading to inconsistent usage and missing
adoption signals.
**Solution:** A new public guide (`doc/TELEMETRY_WORKFLOW.md`) documents
the marker format, the recommended multi-line `track()` shape that
preserves the TS2578 expiry signal, the dimension rules, and the
promotion checklist. Cross-references are added to
`packages/shared/src/telemetry/README.md` and the top-level `README.md`.
No related open PRs found.
## What Changed
- **New file `doc/TELEMETRY_WORKFLOW.md`** — public contributor guide
for the propose/promote lifecycle: marker format, canonical multi-line
example, TS2578 single-line trap, dimension rules, and step-by-step
promotion checklist.
- **`packages/shared/src/telemetry/README.md`** — added one-line
cross-reference pointing at `doc/TELEMETRY_WORKFLOW.md` for proposed
events not yet in the generated contract.
- **`README.md`** — added one-line pointer in the telemetry section so
the new workflow guide is reachable from the top-level project entry.
## Verification
This is a docs-only change. Verification steps:
1. Open `doc/TELEMETRY_WORKFLOW.md` and confirm it contains:
- The `^[a-z0-9][a-z0-9._:-]{1,63}$` event-name grammar.
- The exact `// @ts-expect-error -- proposed-telemetry(<issue>):
<rationale>` marker.
- The multi-line `client.track()` copy-paste example (directive on line
before event-name string).
- The explanation of the TS2578 single-line trap and the
look-up-by-event-name step.
- The promotion checklist in the "Promote An Event" section.
2. Confirm `packages/shared/src/telemetry/README.md` cross-references
`doc/TELEMETRY_WORKFLOW.md`.
3. Confirm the `README.md` telemetry section links to
`doc/TELEMETRY_WORKFLOW.md`.
## Risks
Low risk — docs-only change. No runtime behavior, schema, or existing
telemetry emission is affected. The risk is that the guidance could
diverge from the actual enforcement in the codebase over time; mitigated
by linking to the generated contract and keeping the doc in the same
repo.
## Model Used
Claude — `claude-sonnet-4-6` (Anthropic Claude Sonnet 4.6). Tool use
enabled. Extended context. Produced via the Paperclip agentic workflow
with `Co-authored-by: Paperclip <noreply@paperclip.ing>`.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Co-authored-by: Paperclip <noreply@paperclip.ing>
146 lines
7.0 KiB
Markdown
146 lines
7.0 KiB
Markdown
# Telemetry Data Contract
|
|
|
|
This document explains how contributors should use Paperclip's public telemetry
|
|
contract. It intentionally does not list individual events or dimensions.
|
|
|
|
The canonical source for first-party event names, dimensions, optionality,
|
|
allowed primitive value types, and enum descriptions is
|
|
`packages/shared/src/telemetry/generated/paperclip-telemetry.ts`.
|
|
|
|
Shared enum constants live in `packages/shared/src/constants.ts`. Use those
|
|
constants when code needs a reusable domain, but treat the generated telemetry
|
|
types as the final authority for emitted first-party telemetry shapes.
|
|
|
|
## Public Sources
|
|
|
|
Use these files when reviewing or changing telemetry code:
|
|
|
|
| Contract item | Public source |
|
|
| --- | --- |
|
|
| First-party event names | `PaperclipEventName` in `generated/paperclip-telemetry.ts` |
|
|
| Per-event dimensions and optionality | `EventDimensionsMap` in `generated/paperclip-telemetry.ts` |
|
|
| Enum descriptions for telemetry dimensions | `PAPERCLIP_ENUM_DESCRIPTIONS` in `generated/paperclip-telemetry.ts` |
|
|
| Schema version and event envelope helpers | `SCHEMA_VERSION`, `makeEvent()`, and `makeBatch()` in `generated/paperclip-telemetry.ts` |
|
|
| Runtime-safe event names and dimensions | `TelemetryEventName` and `TelemetryEventDimensions` in `types.ts` |
|
|
| Allowed primitive dimension values | `TelemetryDimensionValue` in `types.ts` |
|
|
| Shared reusable enum domains | Named exports in `constants.ts` |
|
|
| First-party typed emit helpers | `events.ts` |
|
|
| Generic client behavior | `client.ts` |
|
|
| Retention windows and event class assignments | `RETENTION_DAYS` and `EVENT_RETENTION_CLASS` in `retention.ts` |
|
|
|
|
Do not copy generated event lists or dimension tables into this README. They
|
|
will drift as the generated contract changes.
|
|
|
|
## Emission Boundary
|
|
|
|
Paperclip telemetry uses named events with explicit dimension fields. Treat
|
|
open-ended string dimensions as public contract values, not as a place for user
|
|
content or private operational data. Do not send PII, secrets, credentials,
|
|
private paths, prompts, model output, or other sensitive values through
|
|
telemetry dimensions.
|
|
|
|
Telemetry emitters send raw dimension values. They must not pre-normalize
|
|
enum-like values into a reporting form just to match today's known domain.
|
|
|
|
The receiving layer owns canonicalization. Keeping canonicalization in one place
|
|
means emitters can stay simple and accurate: emit what the product observed, use
|
|
the generated contract for required and optional fields, and let the receiving
|
|
layer decide how legacy spellings, aliases, unknown names, and future values map
|
|
to a stable reporting shape.
|
|
|
|
Do not add client-side lowercasing, alias mapping, or fallback mapping unless
|
|
the generated telemetry contract specifically requires that emitted value.
|
|
|
|
If a dimension is privacy-protected before emission, emit only the protected
|
|
value and its matching public marker as defined by the typed helper or generated
|
|
contract. Do not emit private source material in telemetry dimensions.
|
|
|
|
## Dimension Values
|
|
|
|
Telemetry dimension values must be primitives. Use only the value types allowed
|
|
by `TelemetryDimensionValue`:
|
|
|
|
- `string`
|
|
- `number`
|
|
- `boolean`
|
|
|
|
Do not emit `null`, `undefined`, arrays, or objects as dimension values. Optional
|
|
dimensions should be omitted when absent.
|
|
|
|
When a dimension is enum-like, use the shared constant from `constants.ts` when
|
|
one exists. If no shared constant exists, use the generated telemetry type as the
|
|
domain. In all cases, the generated telemetry type remains the source of truth
|
|
for the emitted value.
|
|
|
|
## Required, Optional, And Sentinel Values
|
|
|
|
Required and optional dimensions are defined by `EventDimensionsMap`.
|
|
|
|
Required dimensions must be present for every event of that name. Optional
|
|
dimensions should be emitted only when the value is known and useful.
|
|
|
|
Sentinel values are only for required fields that have no observed raw value at
|
|
the emitting layer. Do not use a sentinel to hide a concrete value that is new,
|
|
custom, or not yet represented by a shared constant. Emit the concrete raw value
|
|
and let the receiving layer canonicalize it.
|
|
|
|
## Adding Or Changing Telemetry
|
|
|
|
Client code is responsible for emitting approved telemetry events at the right
|
|
place in the product. Stable event names, dimensions, and enum domains must come
|
|
from the generated telemetry contract before normal emitters use them.
|
|
|
|
For product work that needs to propose a new first-party event before schema
|
|
registration, use the proposal marker workflow in `doc/TELEMETRY_WORKFLOW.md`.
|
|
Those proposed calls stay on `client.track()`, carry an `@ts-expect-error`
|
|
marker on the event-name argument, and are swallowed at runtime until the
|
|
generated schema registers the event name.
|
|
|
|
For stable event work:
|
|
|
|
1. Start from `generated/paperclip-telemetry.ts`. The generated types are what
|
|
reviewers use to verify event names, dimensions, optionality, value types,
|
|
enum descriptions, and schema version.
|
|
2. Choose stable event and dimension names. Do not include user content, local
|
|
machine details, secrets, credentials, private paths, or values that are not
|
|
part of the public event contract.
|
|
3. Use only `string`, `number`, or `boolean` dimension values.
|
|
4. Reuse a shared constant from `constants.ts` for enum-like dimensions when one
|
|
exists. If the generated telemetry domain has values beyond a shared constant,
|
|
keep the emitter aligned with the generated telemetry type.
|
|
5. Keep emitters raw. Do not normalize, alias-map, or lowercase enum-like values
|
|
in the client unless the generated contract explicitly calls for that emitted
|
|
value.
|
|
6. Add or update a typed helper in `events.ts` when the event is first-party and
|
|
should have a stable helper API.
|
|
7. Update tests for helper behavior, including raw pass-through for enum-like
|
|
values when that is the intended boundary.
|
|
8. Update this README only when the contributor workflow, source-of-truth
|
|
pointers, or durable invariants change. Do not add an event catalog here.
|
|
|
|
Before opening a pull request, verify that the emitted code, typed helpers, and
|
|
generated telemetry contract agree. If they disagree, fix the contract or code
|
|
rather than documenting around the mismatch in this README.
|
|
|
|
For new first-party events that are not in the generated contract yet, follow
|
|
the public proposal and promotion workflow in
|
|
[`doc/TELEMETRY_WORKFLOW.md`](../../../../doc/TELEMETRY_WORKFLOW.md).
|
|
|
|
## Retention
|
|
|
|
Retention windows are documented in `retention.ts`. Each event is assigned a
|
|
retention class; the class determines the window in days. This is a
|
|
housekeeping and query-cost concern managed by data-infra, not a schema
|
|
concern — updating a retention window does not require a schema version bump.
|
|
|
|
Current classes:
|
|
|
|
| Class | Window | Description |
|
|
| --- | --- | --- |
|
|
| `operational_enum_count` | 90 days | Enum/boolean/count/bucket events. No token material, no PII. |
|
|
|
|
When a new event carries only enums, booleans, counts, or coarse buckets and
|
|
no token material or PII, assign it to `operational_enum_count` in
|
|
`EVENT_RETENTION_CLASS`. If no existing class fits, define a new class in
|
|
`RETENTION_DAYS` and document it here.
|