Commit Graph
9 Commits
Author SHA1 Message Date
Michael Nguyen 30182c704c feat(telemetry): propose connector telemetry events behind proposal markers (#13582)
Adds connection.created / connection.updated / connection.invoked as proposed-telemetry wrappers (unregistered; the TelemetryClient cannot queue or send them) plus the server-side emitter wiring and tests. No telemetry is transmitted by this change. Proposal: #13578. Coordinated release plan and review package: PAP-24 (Step A.1).
2026-09-23 22:32:40 +00:00
Nicky Leach c81a089c12 feat(telemetry): align client with wire contract — chunking, deterministic batchId, batched retry, bounded store (#9946) 2026-07-21 09:25:58 -05:00
Nicky LeachandPaperclip 90f85a7d11 Add telemetry proposal extractor (#9544)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - It emits telemetry events to understand product usage — registered
event names are gated by a generated `PAPERCLIP_EVENTS` registry so the
client only enqueues known, schema-approved events
> - When product teams want to instrument a new behaviour, they must
first register the event name — but schema registration is a
commit-and-release cycle, which creates friction in fast-moving product
iterations
> - A proposal lane is needed: let developers mark a `track()` call with
a typed `@ts-expect-error` proposal marker so the event name can be
reviewed and tracked in CI before the schema is formally registered
> - The existing client had no guard against unregistered event names,
so any call with an out-of-registry name (or a prototype-inherited key)
would silently enter the queue, state, and network flush path
> - This PR adds an `Object.hasOwn(PAPERCLIP_EVENTS, eventName)` guard
at the entry point of `track()` to swallow unregistered calls before any
side effects, adds `scripts/extract-proposed-events.mjs` to scan source
for proposal markers and emit a v2 JSON manifest with provenance and
rationale, and documents the complete proposal workflow
> - The benefit is that new instrumentation can be proposed and reviewed
in code without touching the registered schema, and tooling can surface
missing rationale before events graduate to stable

## Linked Issues or Issue Description

No existing GitHub issue covers this change. This PR introduces a new
feature.

**Feature motivation:** Paperclip's telemetry schema is intentionally
stable — registered event names are code-generated and gated. Product
engineers who want to instrument a new behaviour today must land a
schema change first, creating a two-step process that slows iteration. A
proposal lane lets developers write the instrumentation call ahead of
schema registration, protected by a compile-time `@ts-expect-error`
marker that an extractor script can surface for review. This PR
implements both the client-side safety gate and the extraction tooling.

Refs: #9518 (closed predecessor — docs-only; this PR supersedes it with
the full implementation)

## What Changed

- Added `Object.hasOwn(PAPERCLIP_EVENTS, eventName)` guard at the top of
`TelemetryClient.track()`: unregistered event names (including
prototype-inherited keys) are now swallowed before any state, queue, or
network operation
- Added `scripts/extract-proposed-events.mjs`: scans TypeScript source
for `@ts-expect-error -- proposed-telemetry(<issue>): <rationale>`
markers; emits a v2 JSON manifest per proposed event including name,
rationale, provenance (repo-relative file + line), and a
`rationale_missing` flag for CI enforcement
- Added `scripts/extract-proposed-events.test.mjs`: test suite covering
marker parsing, multi-line markers, path validation, out-of-repo
rejection, and the v2 schema output contract
- Added `doc/TELEMETRY_WORKFLOW.md`: documents the proposal workflow,
the canonical multi-line marker example, rationale requirements, and how
to graduate a proposed event to stable schema
- Updated `packages/shared/src/telemetry/README.md`: added "Proposed
Events" section to the Telemetry Data Contract per the contributing
guide requirement for telemetry changes

## Verification

Run all of the following from the repo root:

```sh
# Extractor unit tests
node --test scripts/extract-proposed-events.test.mjs

# Telemetry client + types tests
pnpm exec vitest run --config vitest.config.ts \
  src/telemetry/client.test.ts src/telemetry/client-types.test.ts \
  --reporter=verbose
# (run from packages/shared)

# Type-check
pnpm --filter @paperclipai/shared typecheck

# Smoke-run the extractor in local-test mode
node scripts/extract-proposed-events.mjs --ref local-test
```

All four commands pass locally.

## Risks

- **Silent drop on unregistered events:** The `Object.hasOwn` guard
fails closed — any event name not in `PAPERCLIP_EVENTS` is silently
dropped. If the generated registry is missing an event that was
previously tracked, those calls will be silently lost. Mitigation: the
extractor script surfaces proposed events that need registration; the
TypeScript type system already enforces `TelemetryEventName ⊆
PAPERCLIP_EVENTS` at compile time.
- **Extractor is read-only:** `extract-proposed-events.mjs` reads source
and emits JSON; it does not modify any files. No runtime or schema risk.
- Overall risk: **low**. The guard is additive and defensive; the
extractor and docs are additive only.

## Model Used

- Provider: Anthropic
- Model ID: `claude-sonnet-4-6`
- Context window: 200 K tokens
- Capabilities: tool use, extended context, code generation

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-13 23:47:27 -07:00
Nicky LeachandPaperclip 8a93a0de4c Implement generated client telemetry types (#8818)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Telemetry is part of the control plane's operational visibility and
needs stable event contracts.
> - The shared telemetry client accepted first-party event names through
a broad string surface, which weakened compile-time guarantees.
> - Plugin telemetry still needs a dynamic path because plugin-defined
events cannot be enumerated in the core generated type module.
> - This pull request vendors generated Paperclip telemetry event and
dimension types, closes the first-party event-name union, and keeps
plugin telemetry on an explicit dynamic method.
> - Review feedback clarified that backend normalization should remain
the source of truth, so telemetry helpers now preserve raw categorical
values while keeping generated per-event type hints.
> - The benefit is stricter first-party telemetry typing without hiding
backend normalization signals or changing batching, flushing, schema
versioning, sinks, or endpoints.

## Linked Issues or Issue Description

No public GitHub issue exists for this internal type-contract
maintenance change.

### Problem or motivation

The shared telemetry client should reject unregistered first-party event
names at compile time, while the plugin telemetry bridge must continue
to emit plugin-defined events through the existing batching and envelope
path. Helper wrappers should also avoid client-side enum coercion so the
backend can detect and record normalization when clients send unexpected
categorical values.

### Proposed solution

Generate and vendor the accepted Paperclip telemetry event and dimension
types, use those types for the first-party `track()` API, keep
plugin-defined telemetry on an explicit dynamic method, and let helper
wrappers pass raw categorical dimensions through to backend validation.

### Alternatives considered

Keeping `track()` open to arbitrary strings would preserve flexibility,
but it would not give first-party callers the type safety this change is
meant to provide. Enumerating plugin events in core was also ruled out
because plugin-defined events are not known to the core package.
Client-side enum normalization was removed after review because it
duplicates backend validation and can hide misbehaving-client signals.

### Roadmap alignment

This is a tightly scoped telemetry contract maintenance change and does
not overlap with a roadmap-level core feature.

## What Changed

- Vendored the generated Paperclip telemetry event and dimension type
module under shared telemetry code.
- Closed the first-party telemetry event-name union to generated
backend-accepted names plus an explicit `RegisteredPluginEventName =
never` extension point.
- Added `TelemetryClient.trackDynamic()` for plugin telemetry bridge
emission while keeping `track()` closed and typed.
- Added JSDoc explaining when to use `track()` versus `trackDynamic()`.
- Updated telemetry helper wrappers to type dimensions from each event's
generated schema entry while passing raw categorical values through for
backend normalization.
- Added `trackInteractionResolved()` and updated focused shared/server
tests for telemetry event typing, raw pass-through behavior, and plugin
telemetry bridging.

## Verification

Local verification passed before the latest push:

- `pnpm --filter @paperclipai/shared typecheck`
- `pnpm --filter @paperclipai/server typecheck`
- `pnpm exec vitest run
packages/shared/src/telemetry/client-types.test.ts
server/src/__tests__/shared-telemetry-events.test.ts
server/src/__tests__/plugin-telemetry-bridge.test.ts
server/src/__tests__/project-goal-telemetry-routes.test.ts
server/src/__tests__/routine-run-telemetry.test.ts
server/src/__tests__/issue-telemetry-routes.test.ts`
- `git diff --check`

Post-push verification completed on head
`3d973ffbea6154b19ad208dcffd1374d1b25b654`:

- GitHub PR checks passed, including `verify`, build, typecheck/release
registry, general test shards, serialized server shards, canary dry run,
e2e, and security checks.
- Greptile Review passed with 5/5 confidence.
- All PR review threads are resolved.

## Risks

Low runtime risk. The change is intended to affect TypeScript contracts
and helper typing while preserving the existing telemetry enqueue,
batching, and backend ingest path. The main intentional behavior shift
is that helper wrappers no longer coerce unexpected categorical values
on the client; those values reach the backend so backend normalization
can record the signal. Private company import source refs still use
`hashPrivateRef` when `isPrivate` is true.

## Model Used

OpenAI GPT-5 Codex, tool-enabled coding agent. Exact context window was
not exposed by the runtime; the agent used repository file access, shell
commands, and GitHub CLI operations.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-01 11:39:39 -07:00
Dotta c566a9236c fix: harden heartbeat and adapter runtime workflows 2026-04-10 22:26:21 -05:00
dottaandPaperclip 37b6ad42ea Add versioned telemetry events
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-03 09:25:00 -05:00
dottaandPaperclip 53dbcd185e fix: align telemetry client payload and dimensions with backend schema
Restructure the TelemetryClient to send the correct backend envelope
format ({app, schemaVersion, installId, events: [{name, occurredAt, dimensions}]})
instead of the old per-event format. Update all event dimension names
to match the backend registry (agent_role, adapter_type, error_code, etc.).

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-02 10:47:29 -05:00
dottaandPaperclip f16de6026d fix: add periodic flush and graceful shutdown for server-side telemetry
The TelemetryClient only flushed at 50 events, so the server silently
lost all queued telemetry on restart. Add startPeriodicFlush/stop methods
to TelemetryClient, wire up 60s periodic flush in server initTelemetry,
and flush on SIGTERM/SIGINT before exit.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-02 10:47:29 -05:00
dottaandPaperclip 34044cdfce feat: implement app-side telemetry sender
Add the shared telemetry sender, wire the CLI/server emit points,
and cover the config and completion behavior with tests.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-02 10:47:29 -05:00