mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-24 10:13:11 +02:00
0a09e4d975e1e1cd7de65fbd1ca730b9cd9f97ea
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Nicky LeachandPaperclip
|
9f7565f4ce |
feat: activate OpenTelemetry spans on the sandbox start path (#10536)
## Thinking Path > - Paperclip moves agent work through sandboxed execution and control-plane services. > - The sandbox start path now has a no-op span seam. > - This change turns that seam on when OTLP export is configured. > - It keeps the default path unchanged when export is off. > - The result is structured startup traces with low-cardinality attributes and explicit parent links. > - The benefit is better observability without changing normal behavior. ## Linked Issues or Issue Description No public GitHub issue exists for this change. ### Problem The sandbox start path has a tracer seam, but it stays a no-op unless the OTLP export path is active. ### Proposed solution Enable the server tracer on sandbox bring-up, open a root span, parent each startup boundary to that root, and keep the export path opt-in behind `OTEL_EXPORTER_OTLP_ENDPOINT`. ### Alternatives considered - Keep the start path as a no-op. I rejected that path because it leaves sandbox start opaque when OTLP export is already configured. - Add broad attributes for commands and paths. I rejected that path because the span allowlist must stay low-cardinality. ### Roadmap alignment This follows the current OTel sandbox-start work and keeps the default path unchanged. ## What Changed - Add a root sandbox startup span and child spans for each named startup boundary. - Keep concurrent bridge spans parented to the root span. - Inject the server tracer through the adapter deps without OpenTelemetry imports in the engine. - Attach host-received provider duration attributes only when the values are finite. - Keep span attributes inside the allowlist and keep command, path, id, and error text out of span data. ## Verification - The pushed branch already passed `pnpm --filter @paperclipai/adapter-utils exec tsc --noEmit`. - The pushed branch already passed `pnpm exec vitest run packages/adapter-utils/src/acpx-engine/`. - The pushed branch already passed `pnpm --filter @paperclipai/server exec vitest run src/__tests__/environment-execution-target.test.ts src/__tests__/instrumentation.test.ts`. - The pushed branch already passed `pnpm --filter @paperclipai/server exec tsc --noEmit`. ## Risks - OTel export changes trace volume when the endpoint is set. - The allowlist limits trace detail, so new fields need care. - The change stays no-op when OTLP export is off. ## Model Used - OpenAI GPT-5, tool use enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used with capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
24bbc0f57e |
docs: add telemetry marker workflow (#9543)
## Thinking Path
> - Paperclip is an open-source agentic AI management platform that
ships anonymous usage telemetry to understand product health and
adoption.
> - The telemetry system has a generated contract
(`packages/shared/src/telemetry/generated/paperclip-telemetry.ts`) that
types every first-party event the product emits.
> - When a product change needs a new first-party event that is not yet
in the generated contract, contributors had no public workflow
explaining how to propose an event or later promote it into the contract
once accepted.
> - The gap leads to confusion at call sites: contributors either skip
tracking entirely or emit untyped events that bypass the privacy and
governance safeguards built into the contract.
> - This PR fills that gap by adding `doc/TELEMETRY_WORKFLOW.md` — a
public contributor guide that covers the full propose → promote
lifecycle: the `@ts-expect-error -- proposed-telemetry(...)` marker, the
canonical multi-line `track()` shape, the TS2578 expiry signal, and the
look-up-by-event-name promotion step.
> - `packages/shared/src/telemetry/README.md` gains a cross-reference so
readers of the data-contract doc can find the workflow guide without
searching.
> - `README.md` gains a one-line pointer in the telemetry section so the
workflow is discoverable from the project entry point.
## Linked Issues or Issue Description
No pre-existing public GitHub issue covers this doc gap. Inline
description:
**Problem:** Contributors adding product telemetry for events not yet in
the generated contract have no documented workflow. The
`@ts-expect-error -- proposed-telemetry(...)` pattern exists in the
codebase but is undocumented, leading to inconsistent usage and missing
adoption signals.
**Solution:** A new public guide (`doc/TELEMETRY_WORKFLOW.md`) documents
the marker format, the recommended multi-line `track()` shape that
preserves the TS2578 expiry signal, the dimension rules, and the
promotion checklist. Cross-references are added to
`packages/shared/src/telemetry/README.md` and the top-level `README.md`.
No related open PRs found.
## What Changed
- **New file `doc/TELEMETRY_WORKFLOW.md`** — public contributor guide
for the propose/promote lifecycle: marker format, canonical multi-line
example, TS2578 single-line trap, dimension rules, and step-by-step
promotion checklist.
- **`packages/shared/src/telemetry/README.md`** — added one-line
cross-reference pointing at `doc/TELEMETRY_WORKFLOW.md` for proposed
events not yet in the generated contract.
- **`README.md`** — added one-line pointer in the telemetry section so
the new workflow guide is reachable from the top-level project entry.
## Verification
This is a docs-only change. Verification steps:
1. Open `doc/TELEMETRY_WORKFLOW.md` and confirm it contains:
- The `^[a-z0-9][a-z0-9._:-]{1,63}$` event-name grammar.
- The exact `// @ts-expect-error -- proposed-telemetry(<issue>):
<rationale>` marker.
- The multi-line `client.track()` copy-paste example (directive on line
before event-name string).
- The explanation of the TS2578 single-line trap and the
look-up-by-event-name step.
- The promotion checklist in the "Promote An Event" section.
2. Confirm `packages/shared/src/telemetry/README.md` cross-references
`doc/TELEMETRY_WORKFLOW.md`.
3. Confirm the `README.md` telemetry section links to
`doc/TELEMETRY_WORKFLOW.md`.
## Risks
Low risk — docs-only change. No runtime behavior, schema, or existing
telemetry emission is affected. The risk is that the guidance could
diverge from the actual enforcement in the codebase over time; mitigated
by linking to the generated contract and keeping the doc in the same
repo.
## Model Used
Claude — `claude-sonnet-4-6` (Anthropic Claude Sonnet 4.6). Tool use
enabled. Extended context. Produced via the Paperclip agentic workflow
with `Co-authored-by: Paperclip <noreply@paperclip.ing>`.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Co-authored-by: Paperclip <noreply@paperclip.ing>
|
||
|
Nicky LeachandPaperclip
|
90f85a7d11 |
Add telemetry proposal extractor (#9544)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - It emits telemetry events to understand product usage — registered event names are gated by a generated `PAPERCLIP_EVENTS` registry so the client only enqueues known, schema-approved events > - When product teams want to instrument a new behaviour, they must first register the event name — but schema registration is a commit-and-release cycle, which creates friction in fast-moving product iterations > - A proposal lane is needed: let developers mark a `track()` call with a typed `@ts-expect-error` proposal marker so the event name can be reviewed and tracked in CI before the schema is formally registered > - The existing client had no guard against unregistered event names, so any call with an out-of-registry name (or a prototype-inherited key) would silently enter the queue, state, and network flush path > - This PR adds an `Object.hasOwn(PAPERCLIP_EVENTS, eventName)` guard at the entry point of `track()` to swallow unregistered calls before any side effects, adds `scripts/extract-proposed-events.mjs` to scan source for proposal markers and emit a v2 JSON manifest with provenance and rationale, and documents the complete proposal workflow > - The benefit is that new instrumentation can be proposed and reviewed in code without touching the registered schema, and tooling can surface missing rationale before events graduate to stable ## Linked Issues or Issue Description No existing GitHub issue covers this change. This PR introduces a new feature. **Feature motivation:** Paperclip's telemetry schema is intentionally stable — registered event names are code-generated and gated. Product engineers who want to instrument a new behaviour today must land a schema change first, creating a two-step process that slows iteration. A proposal lane lets developers write the instrumentation call ahead of schema registration, protected by a compile-time `@ts-expect-error` marker that an extractor script can surface for review. This PR implements both the client-side safety gate and the extraction tooling. Refs: #9518 (closed predecessor — docs-only; this PR supersedes it with the full implementation) ## What Changed - Added `Object.hasOwn(PAPERCLIP_EVENTS, eventName)` guard at the top of `TelemetryClient.track()`: unregistered event names (including prototype-inherited keys) are now swallowed before any state, queue, or network operation - Added `scripts/extract-proposed-events.mjs`: scans TypeScript source for `@ts-expect-error -- proposed-telemetry(<issue>): <rationale>` markers; emits a v2 JSON manifest per proposed event including name, rationale, provenance (repo-relative file + line), and a `rationale_missing` flag for CI enforcement - Added `scripts/extract-proposed-events.test.mjs`: test suite covering marker parsing, multi-line markers, path validation, out-of-repo rejection, and the v2 schema output contract - Added `doc/TELEMETRY_WORKFLOW.md`: documents the proposal workflow, the canonical multi-line marker example, rationale requirements, and how to graduate a proposed event to stable schema - Updated `packages/shared/src/telemetry/README.md`: added "Proposed Events" section to the Telemetry Data Contract per the contributing guide requirement for telemetry changes ## Verification Run all of the following from the repo root: ```sh # Extractor unit tests node --test scripts/extract-proposed-events.test.mjs # Telemetry client + types tests pnpm exec vitest run --config vitest.config.ts \ src/telemetry/client.test.ts src/telemetry/client-types.test.ts \ --reporter=verbose # (run from packages/shared) # Type-check pnpm --filter @paperclipai/shared typecheck # Smoke-run the extractor in local-test mode node scripts/extract-proposed-events.mjs --ref local-test ``` All four commands pass locally. ## Risks - **Silent drop on unregistered events:** The `Object.hasOwn` guard fails closed — any event name not in `PAPERCLIP_EVENTS` is silently dropped. If the generated registry is missing an event that was previously tracked, those calls will be silently lost. Mitigation: the extractor script surfaces proposed events that need registration; the TypeScript type system already enforces `TelemetryEventName ⊆ PAPERCLIP_EVENTS` at compile time. - **Extractor is read-only:** `extract-proposed-events.mjs` reads source and emits JSON; it does not modify any files. No runtime or schema risk. - Overall risk: **low**. The guard is additive and defensive; the extractor and docs are additive only. ## Model Used - Provider: Anthropic - Model ID: `claude-sonnet-4-6` - Context window: 200 K tokens - Capabilities: tool use, extended context, code generation ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
1b16f9611c |
feat(telemetry): document credential health retention (#9248)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The shared telemetry package (`packages/shared/src/telemetry`) defines the public contract for first-party telemetry events — event names, dimension shapes, and now retention windows > - A new telemetry event, `codex.credential_health`, carries credential-observability fields (enums, booleans, counts, coarse buckets) — no token material and no PII > - The retention window for this event and its class was undocumented at the contract level, leaving data-infra and reviewers without a discoverable source of truth > - This pull request adds `retention.ts` as the canonical retention-contract surface, assigns `codex.credential_health` to the `operational_enum_count` class (90-day window), exports the contract from the shared package, and updates the README > - The benefit is that the retention window is discoverable from the event definition rather than being implicit pipeline knowledge, and the no-token/no-PII note is locked in a contract test rather than relying on prose ## Issue Description No public GitHub issue exists for this change. Inline description follows the [feature request template](.github/ISSUE_TEMPLATE/feature_request.yml): ### Problem or motivation `packages/shared/src/telemetry/` applies a 90-day retention window to `codex.credential_health` events at the pipeline level, but this policy is not stated anywhere in the shared telemetry contract. Without a discoverable retention declaration, reviewers and data-infra must read pipeline configuration to understand retention behaviour — there is no contract-level source of truth. The `codex.credential_health` event carries only enums, booleans, counts, and coarse buckets — no token material and no PII. ### Proposed solution Add a `retention.ts` module to the shared telemetry package that defines the `operational_enum_count` retention class (90-day window) and maps `codex.credential_health` to it. Export the contract from the package index and add a focused contract test. Update the README with a Retention section and a pointer in the Public Sources table. This is additive documentation only — no runtime paths change. ### Alternatives considered Keep retention implicit in pipeline configuration only. Rejected: this leaves no discoverable, versioned contract for reviewers or data-infra, and means every consumer must read pipeline config to understand retention semantics. A schema-level declaration is the correct long-term home. ### Roadmap alignment Aligns with the telemetry contract hardening track — making implicit operational knowledge explicit and testable at the shared-package level. ## What Changed - **`packages/shared/src/telemetry/retention.ts`** (new): defines `RETENTION_DAYS` (class → days) and `EVENT_RETENTION_CLASS` (event name → class). `operational_enum_count` is the only class: 90-day window for enum/count/bucket events with no token material or PII. `codex.credential_health` is the first entry. The `string` key type is intentional to accommodate cross-system events (e.g. the Codex CLI) not yet promoted to the first-party `PaperclipEventName` schema. - **`packages/shared/src/telemetry/retention.test.ts`** (new): three focused assertions — `operational_enum_count` is 90 days, `codex.credential_health` is assigned that class, and the resolved window is 90 days. - **`packages/shared/src/telemetry/index.ts`**: exports `RETENTION_DAYS`, `EVENT_RETENTION_CLASS`, and `RetentionClass` from the shared package. - **`packages/shared/src/telemetry/README.md`**: adds `retention.ts` to the Public Sources table and a new Retention section with a class reference table and guidance for future assignments. ## Verification - `git diff --check` passes (no whitespace errors) - `pnpm --filter @paperclipai/shared exec vitest run src/telemetry/retention.test.ts src/telemetry/readme-contract.test.ts` — runs the new contract tests and the existing readme-contract test - `pnpm --filter @paperclipai/shared typecheck` — confirms the new exports compile cleanly - CI gates green ## Risks Low risk. This is a documentation-only addition: - No new telemetry event, dimension, emitter, or schema change - No runtime code paths changed - The new file is tree-shaken away in any consumer that doesn't import from it - `satisfies Record<string, number>` on `RETENTION_DAYS` ensures the type stays correct as new classes are added ## Model Used Claude Sonnet 4.6 (`claude-sonnet-4-6`) — Anthropic, 200k context window, tool use enabled, no extended thinking mode. Used for code authoring and PR composition. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Harold Kim <harold.kim@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Harold Kim <harold-kim@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
69c55d465d |
Add telemetry data contract docs (#8886)
Adds the public telemetry data contract README, links it from contributor docs, and adds a focused README contract test for generated helper names. Verification: - git diff --check origin/master..HEAD - pnpm exec vitest run packages/shared/src/telemetry/readme-contract.test.ts - PR CI green - Greptile 5/5 Co-Authored-By: Paperclip <noreply@paperclip.ing> |