mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-12 08:19:54 +02:00
334db53750ac44910e5487dcafe84b4128beb88d
9
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Michael Nguyen
|
30182c704c |
feat(telemetry): propose connector telemetry events behind proposal markers (#13582)
Adds connection.created / connection.updated / connection.invoked as proposed-telemetry wrappers (unregistered; the TelemetryClient cannot queue or send them) plus the server-side emitter wiring and tests. No telemetry is transmitted by this change. Proposal: #13578. Coordinated release plan and review package: PAP-24 (Step A.1). |
||
|
Nicky LeachandPaperclip
|
184b014c25 |
feat(telemetry): add the agent.task_run event and emit it at every terminal run transition (#12809)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Paperclip records agent run outcomes through telemetry and run lifecycle services > - Terminal run transitions need one consistent event for outcome analysis > - The current paths do not report every terminal transition through one event > - This pull request adds the agent.task_run event and emits it at each terminal transition > - The benefit is complete run outcome data without exposing raw task identifiers ## Linked Issues or Issue Description **What existing behavior does this improve?** Paperclip telemetry reports agent activity, but it does not report every terminal task run through one event. **Subsystem affected** Cross-cutting (multiple of the above): packages/shared telemetry and server run lifecycle services. **Current behavior** Several run paths write a terminal status without a matching agent.task_run telemetry event. **Proposed behavior** Each terminal run transition emits one agent.task_run event. The event records the terminal state and uses the existing pseudonym helper for the optional task identifier. **Reason and benefit** Complete terminal-run data helps operators measure agent outcomes. The pseudonym helper prevents the raw task identifier from leaving the installation. **Breaking changes** None. The change adds an event and keeps existing event behavior compatible. ## What Changed - Add the agent.task_run telemetry contract and client helper. - Reuse the existing pseudonym helper for the task identifier. The helper hashes the identifier with a per-installation salt and returns 16 hexadecimal characters. The raw identifier never leaves the installation. Existing identifiers do not move. - Emit one event from each legacy, native, recovery, and issue terminal transition. - Keep emissions outside database transactions and make delivery best-effort. - Add regression tests for event shape, hashing, terminal transitions, and emission failures. - Document the event and its privacy rule in the telemetry data contract. ## Verification - `npx tsc --noEmit` in `server/` passes at the submitted commit. - The pull-request CI suite must pass. CI is the authority because local Vitest has a known dependency artifact. - The added regression tests cover event output shape, per-installation hash divergence, raw identifier handoff, omitted identifiers, and non-throwing emits. ## Risks - A missed terminal path could reduce event coverage. - Telemetry delivery remains best-effort and cannot change run finalization. - The pseudonym helper uses installation-specific state, so identifiers differ between installations. ## Model Used OpenAI Codex, GPT-5, tool use and code execution. Context window details were not provided. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I have addressed all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
10d0555189 |
fix(interactions): authorize resolvers consistently (#11376)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Issue interactions give agents and people a structured decision record. > - Resolver routes used different authorization rules. > - Some routes blocked valid agents, including task watchdogs with normal issue access. > - The API did not show who could resolve a pending interaction. > - This pull request gives every interaction kind one resolver policy evaluator. > - The benefit is a clear decision path with consistent governance and company isolation. ## Linked Issues or Issue Description Fixes: #8087 Refs: #7403 Related PR: #11082 proposes board-only confirmation rules. This change keeps human-only review as an explicit policy. **What happened?** Agents could create issue interactions. Some resolver routes still required board access. This left valid agent confirmations pending. Task watchdogs could see the same problem without board identity. **Expected behavior** Every interaction kind must use one resolver policy contract. The contract must support `anyone`, `not_creator`, and `human_only`. It must also apply all normal governance controls. **Steps to reproduce** 1. Create a `request_confirmation` interaction as an agent. 2. Resolve it with another authorized agent. 3. Observe the board-only denial. **Paperclip version or commit** The problem exists on `master` before this change. **Deployment mode** Local development with `pnpm dev`. ## What Changed - Add canonical policies for `anyone`, `not_creator`, and `human_only`. - Use one server evaluator for every interaction kind. - Apply named addressees, company limits, review rules, and task watchdog scope. - Charge cross-issue resolutions to the existing per-run action limit. - Return the effective resolver audience in attention and interaction data. - Show the audience, governance choices, and denial reasons in the board UI. - Add telemetry, API documents, product documents, and regression fixtures. - Add migration provenance for safe legacy behavior. - Make migration `0218` safe for complete replays and partial prior runs. ## Product Rules - An interaction records a response. It does not grant authority for the next action. - `anyone` lets any authorized issue participant respond. - `not_creator` requires a responder other than the interaction creator. - `human_only` requires an authorized person. - A named addressee, company policy, or governed action can narrow the audience. - These controls cannot widen the audience. - A task watchdog uses the same rules as an ordinary agent. - A task watchdog does not receive board authority. - An agent resolution on another issue uses the shared cross-issue action limit. - Legacy pending interactions keep their earlier restrictions. - The UI shows the effective audience and a permanent denial reason. ## Verification - `pnpm --filter @paperclipai/db check:migrations` - `pnpm --filter @paperclipai/db typecheck` - `pnpm exec vitest run packages/db/src/issue-thread-interaction-resolver-policy-migration.test.ts` - The focused PostgreSQL test applies migration `0218` twice. - The test also completes a partial prior run and preserves existing provenance. - The latest GitHub head has 29 successful checks. - The opt-in Storybook visual check skipped as expected. - Greptile reports 5/5 with no open comments. ## Risks - New interaction writes use `anyone` by default. - Callers must select `not_creator` or `human_only` when they need stricter review. - Legacy pending interactions keep the old creator and human restrictions. - Migration `0218` fills only missing provenance fields during recovery. - Cross-issue resolutions can reach the existing action limit. - The shared evaluator affects every interaction kind. - Route, service, database, shared contract, and UI tests cover these rules. > This work matches the Agent Reviews and Approvals direction in `ROADMAP.md`. It does not duplicate a planned item. ## Model Used OpenAI Codex, GPT-5. The runtime does not expose the exact deployment ID or context window. The agent used reasoning, repository tools, shell commands, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked public issues or described the issue with the required labels - [x] I have not referenced internal Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented the risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open comments - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
bf6365637a |
feat(telemetry): pin silent-drop baseline and add caps/backoff config surface (Impl-1) (#9906)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The telemetry subsystem (`packages/shared/src/telemetry`) flushes event batches to an ingest endpoint; today the client drops batches silently on 429 / 413 / 400 / network error with no re-queue and no retry > - The current behaviour is uncharacterised — there are no regression tests pinning it, so any future retry work could accidentally change the drop semantics without a test failing > - Before adding retry logic it is critical to lock the current baseline as an explicit, documented contract so regressions are immediately visible > - Additionally, the retry/cap design needs a config surface (`maxEventsPerBatch`, `maxBodyBytes`, `maxPendingRetryBatches`, `backoff`) so operators can tune behaviour without forking; that surface should be defined and defaulted before the first consumer lands, not after > - This pull request adds characterisation tests that pin today's silent-drop baseline and adds an optional, additive config surface with resolved defaults — `client.ts` is untouched so no behaviour changes > - The benefit is that the retry work in the follow-up PR can build on a verified baseline, safe defaults, and a pre-wired config contract rather than specifying everything at once ## Linked Issues or Issue Description No pre-existing public GitHub issue. Underlying problem described below (feature-request template). **Problem or motivation** The telemetry client in `packages/shared/src/telemetry/client.ts` has no regression tests covering its error-path semantics. On 429 / 413 / 400 / network failure the current code drains the whole queue via `splice(0)` and discards the batch — silent drop, no retry. That behaviour is correct for a best-effort client today, but nothing verifies it. When retry logic lands, there is no safety net to confirm it does not accidentally re-queue already-dropped batches. In parallel, the retry design requires a typed config surface (`maxEventsPerBatch`, `maxBodyBytes`, `maxPendingRetryBatches`, `backoff`) that the client and its callers need to agree on before the first consumer is wired up. **Proposed solution** Two-phase additive scaffolding: 1. Lock the silent-drop baseline as pinned tests (temporary; Impl-2 replaces them when retry lands). 2. Add the config surface and defaults now so Impl-2 can consume them immediately. **Alternatives considered** Writing the tests and config surface inside the retry PR — rejected because it makes the retry diff much harder to review and loses the regression-safety benefit of a committed baseline. **Roadmap alignment** Maintenance / client-correctness improvement; not a core roadmap feature. ## What Changed - **`client.test.ts`** — 5 new Phase-1 characterisation tests pinning silent-drop on 429, 413, 400, and network-error. A `stubFetchStatus(status)` helper DRYs the four drop cases. These are temporary regression anchors: marked `TODO(impl-2): replace when retry lands`. - **`config.ts`** — `TELEMETRY_DEFAULTS` constant (frozen) + `resolveCaps()` + `resolveTelemetryConfig()`. Provides a single exported source of truth for defaults; nothing reads these yet. - **`types.ts`** — 4 optional, additive `TelemetryConfig` fields: `maxEventsPerBatch` (default 50), `maxBodyBytes` (default 524 288), `maxPendingRetryBatches` (default 20), `backoff` (exp-with-jitter shape). All optional and backward-compatible. - **`index.ts`** — re-exports `TELEMETRY_DEFAULTS` and `resolveTelemetryConfig` from the public package surface. - **`client.ts`** — no changes (verified via `git diff --stat`). ## Verification ```bash # Unit tests — 16 passed (5 new Phase-1 pins + 2 new Phase-2 config tests + 9 existing) pnpm --filter @paperclipai/shared exec vitest run src/telemetry # Type check — clean pnpm --filter @paperclipai/shared typecheck # Confirm client.ts untouched (zero output expected) git diff HEAD~1 -- packages/shared/src/telemetry/client.ts ``` All three commands verified locally before push. ## Risks **Low risk.** `client.ts` is untouched — no behaviour change. All four modified files are additive: - New tests cannot regress production behaviour. - New config fields are `optional` and their defaults match the current implicit constants in `client.ts`, so any future reader gets identical semantics until it overrides them. - `resolveTelemetryConfig` replaces no existing function; it is new. - No envelope / wire change; no new PII / crypto / sink. The `TODO(impl-2)` markers on the Phase-1 pins make their temporary nature explicit in code. ## Model Used - **Provider:** Anthropic - **Model:** Claude Sonnet 4.6 (`claude-sonnet-4-6`) - **Context window:** 200k tokens - **Capabilities:** Tool use, code execution, agentic task completion via Paperclip agent SDK - **Mode:** Standard (no extended thinking) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Harold Kim <harold@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
1b16f9611c |
feat(telemetry): document credential health retention (#9248)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The shared telemetry package (`packages/shared/src/telemetry`) defines the public contract for first-party telemetry events — event names, dimension shapes, and now retention windows > - A new telemetry event, `codex.credential_health`, carries credential-observability fields (enums, booleans, counts, coarse buckets) — no token material and no PII > - The retention window for this event and its class was undocumented at the contract level, leaving data-infra and reviewers without a discoverable source of truth > - This pull request adds `retention.ts` as the canonical retention-contract surface, assigns `codex.credential_health` to the `operational_enum_count` class (90-day window), exports the contract from the shared package, and updates the README > - The benefit is that the retention window is discoverable from the event definition rather than being implicit pipeline knowledge, and the no-token/no-PII note is locked in a contract test rather than relying on prose ## Issue Description No public GitHub issue exists for this change. Inline description follows the [feature request template](.github/ISSUE_TEMPLATE/feature_request.yml): ### Problem or motivation `packages/shared/src/telemetry/` applies a 90-day retention window to `codex.credential_health` events at the pipeline level, but this policy is not stated anywhere in the shared telemetry contract. Without a discoverable retention declaration, reviewers and data-infra must read pipeline configuration to understand retention behaviour — there is no contract-level source of truth. The `codex.credential_health` event carries only enums, booleans, counts, and coarse buckets — no token material and no PII. ### Proposed solution Add a `retention.ts` module to the shared telemetry package that defines the `operational_enum_count` retention class (90-day window) and maps `codex.credential_health` to it. Export the contract from the package index and add a focused contract test. Update the README with a Retention section and a pointer in the Public Sources table. This is additive documentation only — no runtime paths change. ### Alternatives considered Keep retention implicit in pipeline configuration only. Rejected: this leaves no discoverable, versioned contract for reviewers or data-infra, and means every consumer must read pipeline config to understand retention semantics. A schema-level declaration is the correct long-term home. ### Roadmap alignment Aligns with the telemetry contract hardening track — making implicit operational knowledge explicit and testable at the shared-package level. ## What Changed - **`packages/shared/src/telemetry/retention.ts`** (new): defines `RETENTION_DAYS` (class → days) and `EVENT_RETENTION_CLASS` (event name → class). `operational_enum_count` is the only class: 90-day window for enum/count/bucket events with no token material or PII. `codex.credential_health` is the first entry. The `string` key type is intentional to accommodate cross-system events (e.g. the Codex CLI) not yet promoted to the first-party `PaperclipEventName` schema. - **`packages/shared/src/telemetry/retention.test.ts`** (new): three focused assertions — `operational_enum_count` is 90 days, `codex.credential_health` is assigned that class, and the resolved window is 90 days. - **`packages/shared/src/telemetry/index.ts`**: exports `RETENTION_DAYS`, `EVENT_RETENTION_CLASS`, and `RetentionClass` from the shared package. - **`packages/shared/src/telemetry/README.md`**: adds `retention.ts` to the Public Sources table and a new Retention section with a class reference table and guidance for future assignments. ## Verification - `git diff --check` passes (no whitespace errors) - `pnpm --filter @paperclipai/shared exec vitest run src/telemetry/retention.test.ts src/telemetry/readme-contract.test.ts` — runs the new contract tests and the existing readme-contract test - `pnpm --filter @paperclipai/shared typecheck` — confirms the new exports compile cleanly - CI gates green ## Risks Low risk. This is a documentation-only addition: - No new telemetry event, dimension, emitter, or schema change - No runtime code paths changed - The new file is tree-shaken away in any consumer that doesn't import from it - `satisfies Record<string, number>` on `RETENTION_DAYS` ensures the type stays correct as new classes are added ## Model Used Claude Sonnet 4.6 (`claude-sonnet-4-6`) — Anthropic, 200k context window, tool use enabled, no extended thinking mode. Used for code authoring and PR composition. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Harold Kim <harold.kim@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Harold Kim <harold-kim@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
8a93a0de4c |
Implement generated client telemetry types (#8818)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Telemetry is part of the control plane's operational visibility and needs stable event contracts. > - The shared telemetry client accepted first-party event names through a broad string surface, which weakened compile-time guarantees. > - Plugin telemetry still needs a dynamic path because plugin-defined events cannot be enumerated in the core generated type module. > - This pull request vendors generated Paperclip telemetry event and dimension types, closes the first-party event-name union, and keeps plugin telemetry on an explicit dynamic method. > - Review feedback clarified that backend normalization should remain the source of truth, so telemetry helpers now preserve raw categorical values while keeping generated per-event type hints. > - The benefit is stricter first-party telemetry typing without hiding backend normalization signals or changing batching, flushing, schema versioning, sinks, or endpoints. ## Linked Issues or Issue Description No public GitHub issue exists for this internal type-contract maintenance change. ### Problem or motivation The shared telemetry client should reject unregistered first-party event names at compile time, while the plugin telemetry bridge must continue to emit plugin-defined events through the existing batching and envelope path. Helper wrappers should also avoid client-side enum coercion so the backend can detect and record normalization when clients send unexpected categorical values. ### Proposed solution Generate and vendor the accepted Paperclip telemetry event and dimension types, use those types for the first-party `track()` API, keep plugin-defined telemetry on an explicit dynamic method, and let helper wrappers pass raw categorical dimensions through to backend validation. ### Alternatives considered Keeping `track()` open to arbitrary strings would preserve flexibility, but it would not give first-party callers the type safety this change is meant to provide. Enumerating plugin events in core was also ruled out because plugin-defined events are not known to the core package. Client-side enum normalization was removed after review because it duplicates backend validation and can hide misbehaving-client signals. ### Roadmap alignment This is a tightly scoped telemetry contract maintenance change and does not overlap with a roadmap-level core feature. ## What Changed - Vendored the generated Paperclip telemetry event and dimension type module under shared telemetry code. - Closed the first-party telemetry event-name union to generated backend-accepted names plus an explicit `RegisteredPluginEventName = never` extension point. - Added `TelemetryClient.trackDynamic()` for plugin telemetry bridge emission while keeping `track()` closed and typed. - Added JSDoc explaining when to use `track()` versus `trackDynamic()`. - Updated telemetry helper wrappers to type dimensions from each event's generated schema entry while passing raw categorical values through for backend normalization. - Added `trackInteractionResolved()` and updated focused shared/server tests for telemetry event typing, raw pass-through behavior, and plugin telemetry bridging. ## Verification Local verification passed before the latest push: - `pnpm --filter @paperclipai/shared typecheck` - `pnpm --filter @paperclipai/server typecheck` - `pnpm exec vitest run packages/shared/src/telemetry/client-types.test.ts server/src/__tests__/shared-telemetry-events.test.ts server/src/__tests__/plugin-telemetry-bridge.test.ts server/src/__tests__/project-goal-telemetry-routes.test.ts server/src/__tests__/routine-run-telemetry.test.ts server/src/__tests__/issue-telemetry-routes.test.ts` - `git diff --check` Post-push verification completed on head `3d973ffbea6154b19ad208dcffd1374d1b25b654`: - GitHub PR checks passed, including `verify`, build, typecheck/release registry, general test shards, serialized server shards, canary dry run, e2e, and security checks. - Greptile Review passed with 5/5 confidence. - All PR review threads are resolved. ## Risks Low runtime risk. The change is intended to affect TypeScript contracts and helper typing while preserving the existing telemetry enqueue, batching, and backend ingest path. The main intentional behavior shift is that helper wrappers no longer coerce unexpected categorical values on the client; those values reach the backend so backend normalization can record the signal. Private company import source refs still use `hashPrivateRef` when `isPrivate` is true. ## Model Used OpenAI GPT-5 Codex, tool-enabled coding agent. Exact context window was not exposed by the runtime; the agent used repository file access, shell commands, and GitHub CLI operations. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
dottaandPaperclip
|
37b6ad42ea |
Add versioned telemetry events
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
dottaandPaperclip
|
53dbcd185e |
fix: align telemetry client payload and dimensions with backend schema
Restructure the TelemetryClient to send the correct backend envelope
format ({app, schemaVersion, installId, events: [{name, occurredAt, dimensions}]})
instead of the old per-event format. Update all event dimension names
to match the backend registry (agent_role, adapter_type, error_code, etc.).
Co-Authored-By: Paperclip <noreply@paperclip.ing>
|
||
|
dottaandPaperclip
|
34044cdfce |
feat: implement app-side telemetry sender
Add the shared telemetry sender, wire the CLI/server emit points, and cover the config and completion behavior with tests. Co-Authored-By: Paperclip <noreply@paperclip.ing> |