Commit Graph
6 Commits
Author SHA1 Message Date
DottaandPaperclip 46de15ff4b fix(recovery): preserve native workspaces after control deadlines (#17788)
## Thinking Path

> - Paperclip lets people manage agents and their saved work.
> - Native runs can prepare a remote workspace before provider
execution.
> - A finalization deadline can stop a run outside its normal exception
path.
> - That handler marked the run failed without recording its prepared
files for recovery.
> - Retry then waited for a lease whose preservation had no durable
owner.
> - This pull request records the exact source in the deadline
transaction through the existing recovery service.
> - Existing containment and file recovery can then settle the source
before another executor starts.

## Linked Issues or Issue Description

**What happened?**

A native run failed at its control deadline with a prepared workspace
and no accepted result. Its environment lease remained active. The run
had no workspace preservation record. A saved Retry waited for remote
cleanup without a preservation operation to finish that cleanup.

**Expected behavior**

Record unfinished files when the deadline marks a native run failed. Use
the existing stop-and-retain worker. Keep accepted results, exact source
binding, and current task ownership authoritative.

**Steps to reproduce**

1. Persist a native run with a prepared workspace, an active lease, and
an expired control deadline.
2. Reconcile the deadline twice.
3. Inspect the source lease and the run's workspace recovery record.

**Paperclip version or commit**

Reproduced against `a67a18c171816b097f46eae78fc58c7e55c9c565`.

**Deployment mode**

Hosted native Runner with a remote sandbox. The regression uses real
local PostgreSQL and no provider mocks.

Related: Refs #17741 for committed Retry delivery. Refs #17073 for
remote process-exit verification. This change records file preservation
after a deadline; it does not change either policy.

## What Changed

- Call the existing native workspace preservation service in the
transaction that fails an abandoned run.
- Preserve files after reassignment without changing the current task
owner or state.
- Add real-database cases for prepared, finalized, mismatched,
reassigned, and accepted-result states. Repeated reconciliation retains
one source obligation and creates no successor.
- Document the deadline behavior and its existing access boundaries.

## Verification

- Red: on unchanged production code, the prepared-workspace case failed
because its recovery record was null. Finalized and mismatched controls
passed.
- Green: all 10 deadline reconciliation tests pass. The existing native
workspace recovery suite passes all 48 cases.
- `pnpm -r typecheck` passed.
- `pnpm build` passed.
- Map checks, module-boundary checks, and `git diff --check` passed.
- Current-head CI passes on `23bf2aa4b50d2c4b74a9ee2ee47034ae542d25c2`:
all GitHub Actions CI lanes, including grouped/serialized tests, Runner
verification, typecheck, build, E2E, and Canary Dry Run, succeeded. Two
Storybook-only checks were skipped. Greptile is 5/5 with no findings.
Superagent Security Scan also passed. Entire Gates reports passing
checks/findings and fails only because no reviewer has approved trail
#10.
- The first server shard 11/12 attempt passed all 1,750 assertions but
reported an unhandled early process-exit rejection in the unchanged
workspace-exposure fixture. Its focused local suite passed (28 passed, 3
skipped); one CI rerun passed.
- Full local `pnpm test:run` was started, then interrupted after the
complete CI lanes passed to avoid duplicate work. No full local suite
pass is claimed. Local targeted tests, typecheck, and build passed.
- On the affected staging instance, the existing deployed preservation
helper registered the exact failed source. Normal Retry then drove
actual sandbox stop, file recovery, and a persisted successful recovery
receipt. A successor reached a separate remote process-fingerprint
failure before provider execution. The automatic deadline change is not
deployed; task execution recovery remains incomplete.
- Initial local setup attempts did not reach product assertions: skipped
dependency lifecycle scripts left native PostgreSQL library links
absent, and the restricted shell blocked localhost binding. The
package's link hydration and an authorized isolated listener resolved
those setup prerequisites. One new negative-control assertion also
required null-safe access; the final red run retained only the intended
behavioral failure.

## Risks

- File preservation must not certify provider termination. The existing
stop receipt, source binding, copyback, and admission checks remain in
force.
- Preservation failure rolls back the deadline transaction, leaving its
durable deadline eligible for another bounded attempt.
- Reassignment preserves old files without changing the new owner's
task. Accepted results retain their current finalizer.
- No schema, credential, provider, or polling change. Historical failed
runs still require scoped recovery; this patch does not replay them.

## Model Used

OpenAI Codex based on GPT-6, with reasoning, repository inspection, code
execution, and browser tools. The exact serving model ID and
context-window size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (targeted suites; full
local suite interrupted after CI passed)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (CI lanes pass; external Entire
reviewer-approval gate remains pending)
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-26 14:18:53 -05:00
Devin FoleyandClaude Opus 5.5 32ee2e9a6a fix(heartbeat): let a live legacy controller finish before the finalization deadline fails its run (#16622)
The execution-control reconciler now checks hasLiveLegacyController inside its locked transaction before failing a run. While a live legacy controller lease owns the run, the reconciler moves executionControlDeadlineAt to the lease expiry instead of failing, reporting, or surfacing it. Once the lease lapses, the next sweep fails and reports the run once, as before. Native runs are unchanged.

Adds an embedded-Postgres regression test that fails on master and passes with the fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 19:07:50 -07:00
DottaandPaperclip 21ee5c9c85 feat(sentry): add opt-in run failure reporting with saved run references (#16300)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agent runs retain execution results and output in the instance.
> - Operators can report run failures to Sentry for investigation.
> - A backend Sentry DSN currently enables those reports with no
separate choice.
> - A repair bot also needs the exact instance, company, agent, and
saved run.
> - This pull request adds an explicit reporting mode and authenticated
run references.
> - The benefit is that operators can investigate every failure on
selected deployments while keeping other deployments quiet.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Optional server reporting of terminal agent run failures to Sentry.

**Current behavior**

A backend DSN enables run failure reports. Known account, quota, and
connection failures are excluded. Reports omit company and agent
references. Runs without a task are skipped. Tool-name collision errors
omit the conflicting bindings.

**Proposed behavior**

Set `PAPERCLIP_SENTRY_RUN_FAILURES=unexpected` to retain the existing
filters. Set it to `all` to report every failed or timed-out run,
including known setup and quota failures. The default is `off`. Expected
stops stay excluded. Each report supplies identifiers and API paths that
a bot can use with its own authorized Paperclip connection.

**Reason and benefit**

Operators can enable broad failure reporting on one staging deployment.
A bot can fetch the saved run, events, and logs instead of relying on a
short error label or copying complete transcripts into Sentry.

**Breaking changes**

A backend DSN alone no longer enables run failure reports. Existing
operators must set `PAPERCLIP_SENTRY_RUN_FAILURES=unexpected` to retain
the previous policy. Ordinary server exception capture remains enabled
by the DSN. Failure grouping gains the phase and original cause code
when available.

Refs #16281, #16297, #16298. These cover related tool discovery and
failure diagnostics. This change retains their existing boundaries and
does not depend on their open changes.

## What Changed

- Add `off`, `unexpected`, and `all` modes. Missing and invalid values
keep reporting off before diagnostic reads.
- Keep the Sentry backend DSN and audited optional SDK requirements.
- Include company, agent, run, and optional task identifiers. Include
versioned paths for the saved run, events, log, and task. Add absolute
run URLs only for a valid public HTTP(S) origin.
- Report company runs without a task. Prefer the first-class issue
binding when selecting the task reference.
- Select saved native provider/model settings, timestamps, and original
failure codes. Preserve gateway reason codes through redacted exception
projection.
- Group by error code, adapter, and available phase/cause. Keep run IDs
and error text out of grouping.
- Include the tool names and connection bindings in collision errors
without copying tool schemas or provider metadata.
- Document configuration, upgrade behavior, bot access, redaction,
retention, and best-effort delivery.

## Verification

- All 56 checks passed for `cccb47a721eaa1cc36e63278fc177b9101477541`.
Two unrelated Storybook jobs were skipped by their path filters. No
checks remain pending.
- Complete [Paperclip CI
run](https://github.com/paperclipai/paperclip/actions/runs/38591422284):
passed. This covers general and serialized tests, Runner checks, browser
E2E shards, typecheck, build, release-registry checks, canary dry run,
and Docker checks.
- Greptile reviewed that exact head: 5/5, with no unresolved threads or
actionable findings. Superagent, Socket, and Snyk checks passed.
- Local focused reporting, diagnostics, collision, and real SDK contract
tests: 410 passed across five suites. This includes the external-wake
filter in both reporting modes.
- The audited `@sentry/node@10.71.0` tests use an in-memory transport.
No events were sent to Sentry.
- `pnpm -r typecheck` and `pnpm build`: passed locally after the final
rebase.
- `node scripts/check-module-boundaries.mjs` and `git diff --check`:
passed.
- The full local `pnpm test:run` command was stopped to avoid
duplicating the complete CI shards. It did not finish locally. The full
suite passed in CI.

## Risks

- The new default disables existing run failure exports until the
operator selects a mode. The upgrade setting is documented.
- `all` can increase event volume and include known user-action
failures. Secret redaction and diagnostic limits still apply.
- Reporting remains asynchronous and best effort. This PR adds no
durable delivery queue and does not install a bot or Sentry alert rule.
- API references grant no authority. Existing company, private-task, and
run-output checks still apply. Complete transcripts stay in Paperclip.
- This changes optional Sentry error reporting. It changes neither
first-party Telemetry nor OpenTelemetry span exports. It adds no
database migration.

## Model Used

- OpenAI GPT-6 through Codex, with reasoning, shell, file editing, and
test execution. The exact backend deployment ID and context window are
not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-14 10:56:40 -05:00
DottaandPaperclip 84fe89906d fix: complete native agent review handoffs (#13581)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Native execution uses durable runs, issue locks, wake requests, and
typed tool authority
> - A child can finish with a native agent review request while its
original assignee stays responsible for the work
> - The reviewer then needs a bounded execution path that can inspect
the child, record one decision, and finish safely
> - Before this change, assignee-only gates rejected the reviewer or
left the parent waiting after the child review ended
> - This pull request adds typed reviewer admission, scoped reviewer
tools, durable wake and recovery handling, and parent continuation
evidence
> - The benefit is that native review handoffs complete without changing
child ownership or granting broad mutation access

## Linked Issues or Issue Description

Refs: #13314
Refs: #13574

**What happened?**

A native child run could report `needs_review` for an agent reviewer.
The reviewer wake then failed assignee and execution-lock checks. The
child remained in review and the parent remained waiting.

**Expected behavior**

The named reviewer should receive one durable wake. The reviewer should
inspect the child and resolve the exact review card. The child assignee
should stay unchanged. The parent should receive the recorded review
outcome after the child reaches its terminal state.

**Steps to reproduce**

1. Run a native task with a different named agent reviewer.
2. Keep the child assigned to its original worker.
3. Let the worker finish with a native completion review request.
4. Start the durable reviewer wake.
5. Resolve the review and finish the reviewer run.
6. Observe the child and parent state.

**Paperclip version or commit**

Base: `e926b1301`. PR head: `b31ad9ab8`. Live reviewer verification
source: `eea171aae`.

**Deployment mode**

Built from source.

**Installation method**

Built from source (pnpm build).

**Agent adapter(s) involved**

Not adapter-specific (core bug).

**Access context**

Both.

**Database mode**

Embedded PostgreSQL in the isolated live test fixtures.

## What Changed

- Add server-validated native review assignment facts.
- Admit only the exact company, issue, source run, decision, revision,
addressee, and resolver policy.
- Give reviewer runs a narrow set of Paperclip read and resolve tools.
File and shell access follow the configured agent and environment
policy, so reviewers can run tests.
- Separate server-owned reviewer instructions from untrusted persisted
review data. Escape the data boundary; retain server-enforced
authorization.
- Keep the child assignee unchanged. Atomically claim the reviewer run,
wake request, and issue execution lock. A competing lock prevents
provider startup.
- Require the exact running reviewer session and current issue lock to
resolve its assigned card. Reject missing, unrelated, or terminal
reviewer runs.
- Add durable reviewer wake, lock, stale-card, and abandoned-run
recovery handling.
- Prevent duplicate native wake dispatches during deferred admission and
recovery.
- Carry accepted or rejected child review outcomes into parent task
context and continuation evidence.
- Add focused server, runner, and native protocol coverage.
- Preserve upstream continuation rules. Add child review decisions as
separate evidence, while keeping real human answers in their own field.
- Return actionable completion validation feedback to both providers.
Permit a corrected completion after rejection. Keep strict terminal
acknowledgment validation.
- Apply exclusive shared-workspace locks to sandbox environments. Local
and SSH folders can run concurrently, including when old settings
request serialization.
- Repair test timing, native event parsing, and the review artifact
assertion. Allow a valid reject, correct, and accept review sequence.
Check the accepted card against its reviewer run and decision. Keep
polling within the existing deadline when review acceptance precedes the
parent wake projection; report a specific missing-continuation error at
timeout.
- Apply the ACPX pending-call limit to reserved finish/block calls, with
capacity-release and cancellation tests.

## Verification

- `pnpm build`: passed on `eea171aae`.
- `pnpm -r typecheck`: passed on `eea171aae`.
- `pnpm test:e2e:runner:unit`: 359 tests passed in 30 files on
`b31ad9ab8`; runner E2E typecheck also passed.
- `pnpm check:token-gates`: passed.
- Focused DB review, reviewer authority, and prompt-boundary checks: 31
tests passed. They cover invalid reviewer runs, competing locks, atomic
admission, duplicate claims, and valid resolution.
- Heartbeat, workspace, and recovery checks: 30 tests passed.
- ACPX sidecar suite: 27 tests passed. Moving the capacity guard back
below reserved handling makes both new regression cases fail.
- Four focused live continuation checks passed on their first attempt at
`f15f55e0a`: answer updates scope (6/6 each on Codex and Claude) and
question tool guidance (12/12 each). These cases do not use the reviewer
prompt path changed afterward.
- Fresh Codex and Claude review-handoff checks passed all 29 native
checks each on their first attempt at `eea171aae`. Both runs received
the expected fixed prompt and completed cleanup. Only the six selected
live flows were tested; no full paid provider catalog run.
- The final commit only extracts the existing test-harness timeout
diagnostic into a shared helper and adds positive and negative coverage.
Removing the accepted-review guard makes two regression assertions fail;
restoring it passes all six timeout tests. Production runtime code,
prompts, deadlines, and grading criteria are unchanged by this final
commit.
- Deadline regressions: a valid continuation delayed 20 seconds succeeds
within its 30-second unit-test deadline; an absent wake returns a
specific candidate-failure diagnostic at that same deadline. Both
assertions failed before the fix. Production E2E deadlines remain
unchanged.
- Historical native failures remain recorded: Docker availability
failures; a valid reject/correct/accept sequence that the first-card
grader misread; and a test that rejected the gap between accepted child
review and parent wake projection. No failed result was regraded. The
latest tests use a protected reference to the pinned Docker image and
the unchanged artifact oracle and time limits.
- Full repository verification runs in GitHub CI. Local verification
uses the focused suites above, full build, and full typecheck. An
unchanged Codex shutdown timing test failed once in CI, passed in
isolation, and its full shard passed on the final commit without changes
to that test or its causal code path. The original failure is retained
in the verification record. Greptile reviewed `b31ad9ab8` at 5/5 with no
outstanding actionable findings. All review threads are resolved. All
current-head CI gates passed, including the isolated native runner
Docker build (55 successful checks; two skipped by the workflow).

## Risks

- Reviewer admission depends on exact persisted decision and interaction
bindings. A stale or changed card is rejected.
- Paperclip control-plane tools are limited to inspection and review
resolution. This is not a filesystem permission boundary; provider file
and shell access retain the configured policy.
- Deferred wake recovery changes dispatch receipt coalescing. A
scheduler regression could delay a continuation if the receipt state is
wrong.
- Parent review outcomes are evidence for the model. They do not grant
tool authority or change issue ownership.
- This change does not address legacy lease-hold handoff behavior.

> Roadmap review: native execution, review gates, and durable recovery
are existing roadmap capabilities. This PR completes a narrow
reliability path for those capabilities.

## Model Used

OpenAI `gpt-6-astra` with reasoning, tool use, and code execution.
OpenAI `gpt-5.6-luna` assisted with bounded implementation, review, and
journal work. Context window size is not exposed by this session. Live
test subjects use `gpt-5.6-sol` and `claude-sonnet-5`; they are not the
PR authors.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-17 15:52:19 -05:00
DottaandPaperclip 9fd2e50310 feat: create company skills from runner tasks (#13538)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The Runner gives agents tools to change company resources.
> - Users need agents to save reusable skills during a task.
> - A saved skill needs a visible result that users can inspect and
edit.
> - This pull request adds `create_skill` and a task feed card linked to
Skill Studio.
> - Users can open the saved skill from the task and edit the same
resource.

## Linked Issues or Issue Description

**Subsystem affected**

Runner tools, company skill storage, task feed, and Skill Studio.

**Problem or motivation**

The Runner has no dedicated tool to create a company skill. A user
cannot follow a creation result from the task feed to the saved skill.

**Proposed solution**

Add a company-scoped `create_skill` tool. Save the skill with the
existing company policy. Add one creation card to the task. Open a named
sidebar tab from that card. Let the user open the same skill in Skill
Studio.

**Alternatives considered**

An agent can write a local file, but that file is not a company skill. A
second document copy in the task would become stale after a Studio edit.
The sidebar therefore reads the saved skill directly.

**Roadmap alignment**

This extends the shipped Skills Manager, Skill Studio, and Skills Store
milestone. The maintainer requested and approved this scope. Search
found no duplicate `create_skill` PR or issue. Related UI validation
work: #8715. This PR does not change that validation display.

## What Changed

- Add the real Runner tool, its contract, and its mock implementation.
- Validate the complete SKILL.md and derive company, task, agent, and
run identity from authentication.
- Apply the existing company skill policy. Do not assign the skill to an
agent.
- Make keyed retries return one skill and one creation event. Reject
conflicting retries.
- Make concurrent file creation safe. Never replace an existing
published skill during creation.
- Add a creation card, a named sidebar tab, and an Open in Skill Studio
action.
- Show saved Studio edits when the user returns to the task.
- Add storage, policy, mode, retry, UI, and Product E2E tests. Document
the tool.
- Fix deleted-name reuse, onboarding panel persistence, immediate feed
refresh, and mock validation parity from review.
- Serialize Studio file edits and renames with skill deletion and
recreation. Reject stale editor requests before they can change a
replacement skill.
- Generate the standalone mock parser and validator from the production
contract. Use portable UUIDs so the browser scenario bundle builds.

## Verification

- All latest-head PR checks pass on `145dd76a5`, including all server
shards, browser E2E, Runner verification, build, typecheck, and release
dry run. Greptile: 5/5 with no open findings. An interrupted CI runner
was retried successfully.
- `pnpm -r typecheck`: passed.
- `pnpm build`: passed.
- `pnpm check:token-gates`: passed.
- Review regressions: 73 storage tests, 6 real API tests, 63 UI tests,
and 61 semantic runtime tests passed. Parser synchronization passed.
- CI exposed existing fire-and-forget Sentry test races. Reproduced the
resumption race locally, then synchronized the related sweep and
finalizer assertions on the actual report; all 27 tests across the three
affected files pass.
- Runner scenario browser build and strict content-security-policy
check: passed.
- Runner suite: 2,012 tests passed; 10 skipped.
- `pnpm test:run`: the general-server batch had 12,416 passes and two
failures. The old tool-count assertion was fixed; all 16 authority tests
then passed. The chat webhook test had a socket error; it passed four
isolated reruns.
- Both workspace test groups passed. The isolated route suites
completed. Two socket failures in the initial route batches passed on
individual reruns; all remaining 61 files passed.
- Product E2E `create-skill-studio`: passed with local Codex and local
ACPX Claude.
- Manual browser test: submit a task, observe the real tool call and
creation card, open the sidebar, edit in Studio, save, and return. The
task reached Done. The saved second revision and sidebar tab survived a
server restart.
- The new companion headless Runner Eval passed. Companion coverage PR:
https://github.com/paperclipai/paperclip-evals/pull/23. Daytona was not
run because no immutable runner image was configured.

## Risks

- Database writes and local file writes cannot share one transaction.
Recovery accepts only an exact file-for-file retry after a database
rollback. Conflicting files remain untouched.
- The sidebar displays the current skill. The feed card remains the
historical creation receipt.
- No database migration, dependency, or workflow change is included.
- Remote Daytona behavior still needs a run with a configured immutable
image.

## Model Used

OpenAI GPT-6 (`gpt-6-astra`) handled design, integration, review, and
browser verification. OpenAI `gpt-5.6-luna` assisted with bounded
implementation and eval work. Both used code execution and tool access.
The host did not expose the context window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-16 13:01:58 -05:00
Nicky LeachandPaperclip c276d3fdc3 feat(observability): report terminal run failures to Sentry (#13446)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The server records the state of each agent run.
> - Terminal run failures need clear error tracking for operators.
> - The server did not report terminal `failed` or `timed_out`
transitions to Sentry.
> - This pull request reports each genuine terminal failure transition
with safe diagnostic data.
> - The benefit is faster diagnosis without changing run control flow or
exposing credentials.

## Linked Issues or Issue Description

**What happened?**

The server wrote terminal run failures but did not report them to
Sentry. Operators could not see these failures in error tracking.

**Expected behavior**

The server should report each genuine transition to `failed` or
`timed_out` to Sentry.

**Steps to reproduce**

1. Run an agent task that reaches a terminal failure state.
2. Inspect the Sentry events for the server.
3. Observe that the terminal run failure has no matching Sentry event.

**Paperclip version or commit**

The change targets the current `master` branch.

No public GitHub issue or pull request covers this change.

## What Changed

- Add `captureRunFailure()` as a fail-open Sentry entry point.
- Add `reportRunFailure()` to filter status, resolve the adapter, redact
text, and report the failure.
- Call `reportRunFailure()` beside each of the eight terminal status
writers.
- Report six diagnostic values: the instance host, task identifier, run
identifier, error message, error code, and agent adapter.
- Group events by error code and agent adapter while keeping the
redacted message in the event.
- Report only genuine transitions and avoid duplicate finalization
events.
- Keep Sentry failures outside run control flow.

## Verification

- `pnpm vitest run
server/src/services/__tests__/run-failure-report.test.ts
server/src/__tests__/run-failure-sentry.test.ts
server/src/__tests__/native-session-resumption.test.ts`
- `pnpm vitest run
server/src/services/execution-control-reconciliation.test.ts`
- `pnpm --filter @paperclipai/server typecheck`
- The full continuous-integration suite must run on this pull request.

## Risks

- The report path can add diagnostic events when Sentry is configured.
- The report path returns without action when Sentry is not configured.
- Redaction runs before length limits and before the event leaves the
process.
- The change has no migration and no schema change.

## Model Used

OpenAI Codex, GPT-5, tool use and code review support. The exact context
window and reasoning configuration are not exposed by the runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-14 23:32:15 -07:00