Commit Graph
6 Commits
Author SHA1 Message Date
DottaandClaude Opus 5.5 067f7a0815 feat(telemetry): send product lifecycle events and agent run failure cause (#22250)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - First-party telemetry tells the team which product steps people
finish
> - Telemetry did not record when a model is connected, when a task is
created, completed, or blocked, when a plan is created or approved, or
when an artifact is created
> - Telemetry also did not tell why an agent run failed, so failures
could not be grouped by cause
> - This pull request syncs the new telemetry contract and sends these
events from the server once per real change
> - The benefit is a product funnel and a failure breakdown that use
only enums, booleans, and counts

## Linked Issues or Issue Description

**Subsystem affected**
Server telemetry (`packages/shared/src/telemetry`,
`server/src/services/product-telemetry.ts`) and the emit sites in the
issue, document, work product, interaction, and AI connection services.

**Problem or motivation**
The team cannot measure the main product path. There is no event for a
new model connection, a new task, a completed task, a blocked task, a
new plan, an approved plan, or a new artifact. `agent.task_run` tells
that a run failed, but not why. The signup funnel also must know when
the onboarding wizard created the first task.

**Proposed solution**
Register the events in the telemetry backend. Sync the generated
contract here. Add typed helpers. Send each event once, after the
transaction that makes the change commits. Use fixed enums for every
kind, reason, and cause.

**Alternatives considered**
A post-commit action queue on the issue service only. The native status
path and many other callers use an external transaction without that
queue, so most agent completions would not send an event. A generic
after-commit hook on the database client covers every caller.

**Roadmap alignment**
This is telemetry for existing features. It adds no new product feature.

**Additional context**
This change depends on paperclipai/telemetry-backend#150, which is
merged. Deploy that backend change before this change ships. Until then,
the collector rejects the new event names.

## What Changed

- Sync `generated/paperclip-telemetry.ts` from the backend branch as a
direct copy. Regenerate `posthog-direct/enum-fallbacks.generated.ts`
with `scripts/generate-posthog-direct-fixtures.mjs`. The loader golden
file did not change.
- Add `afterTransactionCommit()` to `@paperclipai/db`. It runs a
callback after the outermost transaction commits. It drops the callback
on rollback. Outside a transaction it runs the callback at once. A
callback that throws never fails the committed work.
- Add helpers `trackModelConnected`, `trackTaskCreated`,
`trackTaskCompleted`, `trackTaskBlocked`, `trackPlanCreated`,
`trackPlanApproved`, and `trackArtifactCreated`. Add `failureCause` to
`trackAgentTaskRun`. Assign the new events to `operational_enum_count`
retention.
- Add `server/src/services/product-telemetry.ts`. It sends each event
after commit inside `withTelemetryCompany()`, so Cloud direct delivery
gets `company_id` for background writes.
- Emit sites:
- `model.connected`: `aiConnectionService.save`, for a new connection
only. A reconnect sends nothing. `first_for_company` is true when the
company had no earlier model connection.
- `task.created`: `issueService.create`, after the insert. A
deduplicated create, a company import, and a chat conversation send
nothing. `created_by_kind` is `routine` for routine runs, `onboarding`
for the onboarding wizard's first task, and otherwise `user`, `agent`,
or `system`.
- `task.completed` and `task.blocked`: `issueService.update`, when the
status changes to `done` or `blocked`. The same status again sends
nothing.
- `plan.created`: `documentService.upsertIssueDocument`, for the first
revision of the `plan` document.
- `plan.approved`: the accept path of a `request_confirmation` that
targets the task's `plan` document.
- `artifact.created`: `workProductService.createForIssue` (`direct`),
the promotion route (`promotion`), and a run file deliverable
(`run_deliverable`).
- `agent.task_run.failure_cause`: `emitAgentTaskRun`, for `failed`,
`timed_out`, and `interrupted` runs only.
- `blocked_reason` mapping: unfinished blocking tasks give
`waiting_on_task`. Otherwise an unblock owner of a person or the board
gives `waiting_for_person`, and an agent owner gives
`waiting_for_agent`. A system write with neither gives `needs_recovery`.
A person or agent write with neither gives `unspecified`.
- `failure_cause` mapping (stored `errorCode`, with
`resultJson.providerFailure.category` for generic provider and adapter
codes):
- `missing_access`: auth and permission codes (`acpx_auth_required`,
`provider_auth_failed`, `user_secret_missing`, `github_access_denied`,
`github_repo_not_accessible`, `permission_denied`,
`project_access_denied`, `configuration_incomplete`,
`connection_unavailable`, `signing_secret_unconfigured`), and provider
category `authentication` or `permission`.
- `provider_limit`: `provider_quota`, `native_provider_usage_limit`,
`native_provider_overloaded`, `github_rate_limited`,
`ai_connection_busy`, and provider category `rate_limit`,
`quota_exhausted`, or `overloaded`.
- `provider_error`: `native_provider_model_rejected`,
`provider_transport_failed`, `codex_transient_upstream`,
`provider_failed`, `provider_tool_definition_invalid`, and the other
provider categories.
- `waiting_for_person`: `execution_reconciliation_required`, every code
in `EXECUTION_RECONCILIATION_CAUSES`, `issue_waiting_for_response`, and
`issue_continuation_waiting_on_review`.
- `task_changed`: the task was reassigned, paused, closed, blocked, or
stopped by a person (`operator_interrupted`, `cancelled`, `interrupted`,
`issue_*` lifecycle codes, `agent_paused`, `queued_comment_discarded`).
- `platform_error`: process, server, workspace, sandbox, runner, and
storage codes (`process_lost`, `server_shutdown_interrupted`,
`setup_failed`, `workspace_*`, `environment_provider_unavailable`,
`controller_lease_lost`, and others).
- `agent_error`: `adapter_failed` without a provider failure,
`invalid_output`, `invalid_response`, `tool_error`, `acpx_turn_failed`,
`test_run_failed`, `prompt_injection_blocked`,
`tool_execution_outcome_unknown`, `timeout`.
  - `unknown`: any other code, or no code.
- On Cloud direct delivery, every new emit site names the responsible
person with `withTelemetryActor()` and `resolveTelemetryPerson()`, as
`agent.task_run` does: the task creator, else the task's responsible
user, else the run owner. `task.created` uses the creating user, else
the agent's on-behalf user. `model.connected` uses the user who owns the
connection. A live request still wins. The self-hosted collector
envelope does not change.
- `failure_cause` builds on the run failure code sets in
`continuation-cause.ts`. A test fails when a code in those sets has no
telemetry code. The provider failure category decides only for
`adapter_failed`, `provider_failed`, or no code.
- Work product providers outside `paperclip`, `github`, `vercel`, and
`s3` are sent as `custom`.
- The dispatch workspace check and execution recovery write `blocked`
directly. They call the same after-commit helper, with the task's real
blockers and unblock owner. The blocker read runs in a savepoint in the
issue dependency layer, so a failed read never aborts the block write.
- Update the telemetry README with the emit rules, the after-commit
rule, and the `failure_cause` source.
- Refresh `tests/runner-e2e/source-sdk-public-inputs.json`, because its
check reported a change.

## Privacy Review

Privacy review requested. Every new dimension is a fixed enum, a
boolean, or a count. No dimension carries a title, a body, a plan step,
a connection or account name, an email, a URL, a repository name, a
prompt, an error message, or an endpoint address. `failure_cause` is a
code that the server picks. The run's error text is never sent. The
tests check that a provider message and a connection name do not reach
the event.

## Verification

- `pnpm vitest run packages/shared/src/telemetry/`: 7 files pass,
including the new `product-events.test.ts` (helper payloads, omitted
optional values, collector envelope with only registered dimensions,
retention class).
- `packages/db/src/after-transaction-commit.test.ts`: 4 pass (after
commit, rollback drop, savepoint, outside a transaction).
- `server/src/__tests__/product-telemetry.test.ts`: 14 pass. They check
that each event fires once with the right kind and company scope, that a
rollback sends nothing, that a repeated status sends nothing, that a
later plan revision sends nothing, and that a reconnect sends nothing.
- `server/src/__tests__/agent-task-run-telemetry.test.ts`: 35 pass,
including the `failure_cause` mapping table and the emit path.
- `server/src/__tests__/issue-thread-interactions-service.test.ts -t
Plan`: `plan.approved` fires for a plan confirmation and not for an
unrelated confirmation.
- `server/src/__tests__/posthog-direct-telemetry.test.ts`: the Cloud
direct path sends `company_id`, `user_id`, and `attributed_via` for
background lifecycle events, an event with no person stays on the
workspace, and `failure_cause` passes through. A self-hosted test checks
that a person scope leaves the collector events unchanged.
- Existing telemetry tests (connector, interaction, run finalizer,
heartbeat, AI connection): 299 pass.
- `pnpm --filter @paperclipai/shared typecheck`, `pnpm --filter
@paperclipai/db typecheck`, and `pnpm --filter @paperclipai/server
typecheck`: pass.
- `pnpm verify` returned exit 2 (global CI input changed). Required CI
supplies the broad coverage.

## Risks

- Merge order: merge and deploy paperclipai/telemetry-backend#150 first.
If this change ships first, the collector quarantines the new events.
The product behavior does not change.
- `afterTransactionCommit` uses the existing open-transaction marker in
the database client. A callback that a savepoint registers runs when the
outer transaction commits, even if that savepoint rolled back.
- `task.blocked` adds one small indexed read inside the update
transaction, only when a task enters `blocked`.
- `failure_cause` adds one small read for a failed run when the caller
did not pass the error code. It runs off the run's critical path, as
before.
- Low risk otherwise. No schema migration. No change to existing event
dimensions.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), through Claude Code,
with tool use (shell, Git, tests).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 19:32:27 -06:00
DottaandClaude Opus 5.5 92dbf39e6f feat(analytics): attribute agent work and onboarding steps to a person on Cloud (#22260)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Paperclip Cloud sends product events straight to PostHog: server
events from `server/src/services/posthog-direct/` and browser events
from `ui/src/lib/analytics/`
> - A person-level funnel goes from sign-up to the first finished task
> - Agent-driven events (`agent.task_run`, `agent.task_completed`,
`agent.first_heartbeat`) used `workspace:<stackId>` as the distinct id,
so the funnel broke at those steps
> - The in-workspace onboarding wizard sent no step events, so we could
not see where people stop before their first agent
> - This pull request attributes agent work to the responsible person on
the Cloud direct path, and adds consent-gated onboarding step events in
the browser
> - The benefit is one PostHog person per customer across sign-up,
onboarding, agent runs, and task completion. Self-hosted output does not
change.

## Linked Issues or Issue Description

**Subsystem affected**
Cloud product analytics: Cloud direct PostHog delivery (server) and
browser product analytics (UI onboarding wizard).

**Problem or motivation**
A PostHog funnel from sign-up to the first finished task cannot follow
one person. Agent-driven events run in the background, with no signed-in
request, so they use the workspace id as the distinct id. The onboarding
wizard also sends no events, so we cannot see which step people leave.

**Proposed solution**
Give background emit sites a narrow scope that names the person
responsible for the work. Use the task creator first, then the task's
responsible user, then the run owner. Keep the signed-in request user
first. Add `attributed_via` to every Cloud direct event. Add four
browser events to the onboarding wizard. Send them through the existing
analytics module, so the consent rules stay the same.

**Alternatives considered**
A lookup in the transport at send time. We did not use it: it adds
database reads to the telemetry path and loses the emit site's context.
A new contract event for onboarding steps. We did not use it: the server
does not see the wizard steps, and the UI must never send
`PAPERCLIP_EVENTS` names.

**Additional context**
Self-hosted installs keep the collector path. The collector envelope
stays byte-identical (a test checks it).

## What Changed

- `server/src/telemetry-company-scope.ts`: add `withTelemetryActor({
companyId, person }, fn)`, `scopedTelemetryPerson()`, and
`resolveTelemetryPerson()`. The resolver picks the task creator, then
the task's responsible user, then the run owner. It ignores
`local-board`. `withTelemetryCompany` now calls `withTelemetryActor`.
- `server/src/services/posthog-direct/request-context.ts`: a signed-in
request user wins (`attributed_via: request`). Without one, the scoped
person is used.
- `server/src/services/posthog-direct/event-shape.ts`: every Cloud
direct event (including `cloud.workspace_started`) has `attributed_via`
(`request`, `task_creator`, `task_assigner`, `run_owner`, or `none`). An
event with a person keeps today's person-event shape (no
`$process_person_profile: false`).
- `server/src/services/agent-task-run-telemetry.ts`: `agent.task_run`
reads the task creator and responsible user in the existing agent query
(one left join, no new query), and the run's `responsibleUserId`.
- `server/src/routes/issues.ts`: `agent.task_completed` uses the issue
row that the route already has, and the agent actor's `onBehalfOfUserId`
as the run owner.
- `server/src/services/heartbeat/agent-status.ts`:
`agent.first_heartbeat` uses the agent's `createdByUserId` (already
loaded) as the run owner.
- `ui/src/lib/analytics/product-analytics.ts`: add
`captureProductAnalyticsEvent()`. It captures only with a browser
analytics config and granted consent. Events from before posthog-js
loads wait in a queue (at most 50) and keep their time. Withdrawn
consent or a change of signed-in person drops the queue.
- `ui/src/lib/analytics/onboarding-analytics.ts` and
`ui/src/components/OnboardingWizard.tsx`: capture `onboarding_opened`,
`onboarding_step_viewed`, `onboarding_step_completed` (with `step` and,
for `connect_model` and `launch`, `outcome`), and
`onboarding_dismissed`. `step` is `create_agent`, `connect_model`,
`review`, or `launch`. No event has free text.
- Docs: `packages/shared/src/telemetry/README.md` (Cloud Direct
Delivery) and `doc/cloud-browser-analytics.md` (new "Onboarding events"
section).

## Verification

- `server/src/__tests__/posthog-direct-telemetry.test.ts`: attribution
matrix (creator, assigner, run owner, none stays on
`workspace:<stackId>` with `$process_person_profile: false`), the
request user wins over the scope, and the self-hosted collector envelope
is byte-identical with a person scope.
- `server/src/__tests__/agent-task-run-telemetry.test.ts` (embedded
Postgres): `agent.task_run` scope for creator, assigner, run owner, and
nobody; another company's task is never read.
- `server/src/__tests__/issue-telemetry-routes.test.ts`:
`agent.task_completed` scope for creator and assigner.
- `server/src/__tests__/heartbeat-process-recovery.test.ts`:
`agent.first_heartbeat` scope is the agent owner.
- `ui/src/components/OnboardingWizard.test.tsx`: a launch for the old
company that settles after a switch to another company does not end the
new analytics session.
- `ui/src/lib/analytics/onboarding-analytics.test.tsx`: each transition
fires once, held events drop on a person change, a stale opening step is
not a view, dismissal fires once, nothing is captured when analytics is
off or consent is `denied` or `ask`, and queued events keep their time.
- `ui/src/lib/analytics/product-analytics-events.test.ts` still passes
(the UI never captures a `PAPERCLIP_EVENTS` name).
- Local Cloud-mode run: an isolated instance (own Postgres, trusted
Cloud tenant headers, `PAPERCLIP_POSTHOG_HOST` set to a local capture
server, never real PostHog). A stub stood in for posthog-js and sent
through the `/_ph` proxy.
- A user created a task and assigned it to the agent. `agent.task_run`
arrived with `distinct_id: paperclip-id:local-user-1`, `attributed_via:
task_creator`, the raw `task_id`, and no `$process_person_profile`. (The
run itself failed locally because the Cloud runner is not available. The
event still shows the attribution.)
- `agent.first_heartbeat` arrived with the same person and
`attributed_via: run_owner`. `server.started` kept
`workspace:stack-local` with `attributed_via: none`.
- Headless Chromium walked the wizard: `onboarding_opened`
(create_agent), `onboarding_step_viewed` (create_agent),
`onboarding_step_completed` (create_agent), `onboarding_step_viewed`
(connect_model), `onboarding_step_completed` (connect_model, `api_key`),
`onboarding_step_viewed` (review), `onboarding_step_completed` (review),
`onboarding_step_completed` (launch, `succeeded`). Each fired once, with
the signed-in user as the distinct id. Events from before posthog-js
loaded arrived with their original time.
- A second walk left the onboarding page on the connect step and sent
one `onboarding_dismissed` (connect_model).

## Risks

- Low risk for self-hosted: the collector path never reads the new
scope, and a test checks the envelope bytes.
- Cloud: agent events move from the workspace person to a real person.
Insights that count `workspace:` distinct ids for these events will
change. `attributed_via` lets a query filter or split them.
- `task_assigner` uses the task's responsible user. Paperclip does not
store who assigned a task, so this is the person who owns or delegated
the task.
- Privacy: the new data is Cloud-only and uses ids that Cloud events
already carry. No new free text, email, or name goes on an event.
**Privacy review requested** for the Cloud attribution change (per
`rules/data-paths.mdc`). The generated contract does not change:
`attributed_via` is a Cloud-only property, like `stack_id` and
`user_id`.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edits, headless browser).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 18:44:45 -06:00
DottaandClaude Opus 5.5 2a43fe3e34 feat(telemetry): add company_id to Cloud direct events from background work (#22183)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Paperclip Cloud sends server telemetry straight to PostHog
(`server/src/services/posthog-direct/`)
> - Request events get `company_id` from the request context. Background
events have no request.
> - So `agent.task_run`, `routine.run`, `connection.invoked`, and other
background events arrive with `stack_id` but without `company_id`. On
staging, 66 of 66 `agent.task_run` events had a null `company_id`.
> - Without `company_id`, product analytics cannot split agent and
routine activity by company.
> - This pull request lets each background emit site name the company of
the entity that it reports on. Only the direct PostHog transport reads
that company.
> - The benefit is that Cloud background events carry `company_id`, and
the self-hosted collector output does not change.

## Linked Issues or Issue Description

Refs #22006

**Subsystem affected**
Server telemetry: Cloud direct delivery to PostHog
(`server/src/services/posthog-direct/`) and the server emit sites for
background events.

**Problem or motivation**
Cloud direct events from background work have no `company_id`. Heartbeat
runs, scheduled routines, connection invocations, and interaction
updates do not run inside an HTTP request, so the request context gives
no company. On paperclip.staging, all 66 sampled `agent.task_run` events
had `company_id` null.

**Proposed solution**
Add a small AsyncLocalStorage scope, `withTelemetryCompany(companyId,
fn)`, in `server/src/telemetry-company-scope.ts`. Each background emit
site wraps its typed `track*` call with the company id of the row that
it already has. The direct transport reads the scope at track time,
through the existing `captureEventContext()` seam. A live request
company still wins. The collector path never reads the scope.

**Alternatives considered**
1. An optional context argument on the shared `track*` helpers. This
changes the public helpers in `@paperclipai/shared` for a Cloud-only
value, so I did not use it.
2. A scope around each whole work unit (heartbeat run, routine run,
invocation). The emit calls are fire-and-forget and are spread across
many modules, so it is easy to miss one or to leak a company into other
work. A scope at the emit site is narrow and explicit.
3. Let the transport look up the company in the database. The transport
must not query the database.

**Roadmap alignment**
This completes the Cloud direct PostHog work from #22006. It does not
change the telemetry contract.

## What Changed

- Add `server/src/telemetry-company-scope.ts`. It has
`withTelemetryCompany(companyId, fn)` and `scopedTelemetryCompanyId()`.
With no company id, it calls `fn` directly.
- `capturePostHogEventContext()` uses the scoped company when no live
request gives a company. The request company, user, and session behave
as before.
- Wrap these emit sites with the company of their entity:
- `agent.task_run` (`emitAgentTaskRun`, `emitAgentTaskRunById`;
`companyId` is now part of the run that the function reads). This covers
heartbeat, native runtime, recovery, run-dispatch, and watchdog callers.
- `routine.run` (`routineService` dispatch, which covers scheduler,
webhook, and manual runs)
  - `agent.first_heartbeat` (heartbeat agent status)
  - `connection.created`, `connection.updated`, `connection.invoked`
  - `interaction.created`, `interaction.resolved`
- `agent.task_completed` (issue route; the issue company is the fallback
when the request gives no company)
- Request-only events (`project.created`, `goal.created`,
`agent.created`, `routine.created`, `skill.imported`) keep the request
context. `server.started` and `error_handler.crash` are not
company-scoped. Dynamic plugin and `question_response.delivery` events
are not in the contract, so the direct path drops them.
- Update the "Cloud Direct Delivery" section of
`packages/shared/src/telemetry/README.md`.

Design note on module boundaries: the scope lives beside
`server/src/telemetry.ts`, not inside `posthog-direct/`. Emit sites do
not import the transport, and the transport does not know which services
exist. The shared telemetry package does not change.

## Verification

- Red/green: with the source changes removed, these 4 new tests fail.
With the changes, they pass.
- `agent-task-run-telemetry.test.ts`: `emitAgentTaskRun` and
`emitAgentTaskRunById` track in the run company scope.
- `routine-run-telemetry.test.ts`: a routine run tracks in the routine
company scope.
- `posthog-direct-telemetry.test.ts`: on Cloud, background
`agent.task_run` and `routine.run` (also after an `await`) carry
`company_id`, and an unscoped event does not.
- `posthog-direct-telemetry.test.ts`: a live request company wins over
the scope, and the scope fills the gap when the request has no company.
- Red/green per emit family: the tests for `connection.created`,
`connection.updated`, `connection.invoked`, `interaction.created`,
`interaction.resolved`, `agent.first_heartbeat`, and
`agent.task_completed` assert the company scope at track time. With the
emit-site wrappers removed, all 7 assertions fail.
- New guard test: on self-hosted, a scoped `agent.task_run` and
`routine.run` produce a byte-identical collector envelope with no
`company` text.
- `cd server && npx vitest run` on 16 telemetry, PostHog, routine,
heartbeat status, interaction, and connector test files: 408 passed.
- `pnpm --filter @paperclipai/server typecheck`: passed.
- Local check with a real `posthog-node` client sent to a local capture
server (never real PostHog), embedded Postgres, and a real
`routineService.tickScheduledTriggers()` tick plus `emitAgentTaskRun`:
- Cloud mode: the captured `routine.run` (`source: schedule`) and
`agent.task_run` both had the seeded `company_id`, plus `stack_id` and
`environment`. `server.started` had no `company_id`.
- Self-hosted mode: the collector body from this branch and from
`origin/master` are identical after I normalize random ids and
timestamps.
- I did not boot a full Cloud-managed server, because it needs S3 and an
external Postgres. The check above uses the same server modules and the
real PostHog client.

## Risks

- Privacy review: this is a Telemetry change under
`rules/data-paths.mdc`, so I request a privacy review. Background Cloud
events now send the real Paperclip `company_id` to PostHog. #22006
already approved `company_id` as a Cloud direct property for request
events. This PR only fills it for background events. It adds no new
property, dimension, or event, so the generated contract does not
change. `packages/shared/src/telemetry/README.md` is updated in this PR.
- Low runtime risk. The self-hosted collector path does not read the
scope, and a test proves its output does not change.
- On Cloud, more events now have `company_id`. No event loses a field.
- If a request with a company tracks an event for an entity in a
different company, the request company wins. This is the same as before
this change.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, and test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 10:34:39 -06:00
Devin FoleyandPaperclip 1cfaf973cd Read release evidence and telemetry without unused output (#17485)
Read execution release evidence and terminal telemetry without unused output. Keep full stored history, in-flight guards, and accepted messages.

Validated with real PostgreSQL tests, build, typecheck, green CI, and Greptile 5/5.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-24 14:01:41 -07:00
Nicky LeachandPaperclip f65991a5f1 refactor(server): move scheduled-retry and queued-run dispatch into a run-dispatch module (#12920)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The server heartbeat service dispatches scheduled retries and queued
runs.
> - The service kept policy decisions and database writes in one large
file.
> - This layout made policy branches harder to test and transaction
boundaries harder to inspect.
> - This pull request moves the policy rules and database transactions
into a run-dispatch module.
> - The benefit is a smaller service, pure policy tests, and clear
transaction ownership.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The server heartbeat service promotes scheduled retries and cancels
stale queued runs.

**Subsystem affected**

server/ — REST API and orchestration services.

**Current behavior**

The heartbeat service contains the policy rules and the database writes
for these dispatch paths.

**Proposed behavior**

A run-dispatch module owns pure policy functions and semantic database
transactions. The public service contracts stay unchanged.

**Reason and benefit**

The new layout separates branch rules from database effects. It makes
each policy branch easier to test and keeps each operation’s row writes
in one transaction.

**Breaking changes**

None. The public service contracts stay unchanged.

## What Changed

- Move scheduled-retry promotion and queued-run staleness rules into
pure functions.
- Add table-driven unit tests for each policy branch.
- Move promotion and cancellation writes into semantic transactions.
- Keep row locking, company isolation, and post-commit effects
unchanged.

## Verification

- `node scripts/check-module-boundaries.mjs` passes.
- `tsc --noEmit` from `server/` reports no errors.
- The focused server test command passes 228 tests in six files.
- Full pull request CI passes.
- Greptile reports 5/5, and all review threads are resolved.

## Risks

The main risk concerns changed transaction boundaries in scheduled-retry
promotion and queued-run cancellation. The focused tests retain coverage
for locking, transactionality, company isolation, and transport
contracts. The public service contracts do not change.

## Model Used

Codex, GPT-5, with code execution and tool use. The context window is
not provided by the runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I have addressed all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-08 11:00:55 -07:00
Nicky LeachandPaperclip 184b014c25 feat(telemetry): add the agent.task_run event and emit it at every terminal run transition (#12809)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Paperclip records agent run outcomes through telemetry and run
lifecycle services
> - Terminal run transitions need one consistent event for outcome
analysis
> - The current paths do not report every terminal transition through
one event
> - This pull request adds the agent.task_run event and emits it at each
terminal transition
> - The benefit is complete run outcome data without exposing raw task
identifiers

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Paperclip telemetry reports agent activity, but it does not report every
terminal task run through one event.

**Subsystem affected**

Cross-cutting (multiple of the above): packages/shared telemetry and
server run lifecycle services.

**Current behavior**

Several run paths write a terminal status without a matching
agent.task_run telemetry event.

**Proposed behavior**

Each terminal run transition emits one agent.task_run event. The event
records the terminal state and uses the existing pseudonym helper for
the optional task identifier.

**Reason and benefit**

Complete terminal-run data helps operators measure agent outcomes. The
pseudonym helper prevents the raw task identifier from leaving the
installation.

**Breaking changes**

None. The change adds an event and keeps existing event behavior
compatible.

## What Changed

- Add the agent.task_run telemetry contract and client helper.
- Reuse the existing pseudonym helper for the task identifier. The
helper hashes the identifier with a per-installation salt and returns 16
hexadecimal characters. The raw identifier never leaves the
installation. Existing identifiers do not move.
- Emit one event from each legacy, native, recovery, and issue terminal
transition.
- Keep emissions outside database transactions and make delivery
best-effort.
- Add regression tests for event shape, hashing, terminal transitions,
and emission failures.
- Document the event and its privacy rule in the telemetry data
contract.

## Verification

- `npx tsc --noEmit` in `server/` passes at the submitted commit.
- The pull-request CI suite must pass. CI is the authority because local
Vitest has a known dependency artifact.
- The added regression tests cover event output shape, per-installation
hash divergence, raw identifier handoff, omitted identifiers, and
non-throwing emits.

## Risks

- A missed terminal path could reduce event coverage.
- Telemetry delivery remains best-effort and cannot change run
finalization.
- The pseudonym helper uses installation-specific state, so identifiers
differ between installations.

## Model Used

OpenAI Codex, GPT-5, tool use and code execution. Context window details
were not provided.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I have addressed all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-04 08:21:32 -07:00