mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-15 03:51:22 +02:00
a96329dfa2
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Every pull request runs the `Sentry SDK contract` workflow, whose one job installs the real `@sentry/node` outside the workspace and runs `server/src/__tests__/run-failure-sentry-real-sdk.test.ts` to prove that run-failure reports never carry provider output or credentials. > - One of its assertions checks that the captured events do not contain the provider's stderr canary `<html>504 Gateway Time-out private-timeout-credential-canary</html>`, by matching the serialized events against `/private-timeout-|<html>|504|Timed out after/`. > - The bare `504` alternative also matches a Sentry event id. On pull request #15926 (a build-script change that does not touch this code) the job failed because that run's event id was `f353d184e5504ee39db611f2e60f49ff`; the re-run passed. About 0.7% of random 32-character hex ids contain `504`, and the test serializes two events, so roughly one job in seventy fails for no reason. > - This pull request narrows that alternative to `504 Gateway`, which still catches the canary with or without its HTML wrapper and cannot match a hex id. ## Linked Issues or Issue Description No GitHub issue exists for this. The description below follows the bug report template. **Pre-submission checklist** - [x] I have searched existing open and closed issues and this is not a duplicate. - [x] I am on the latest released version of Paperclip (or can reproduce on `master`). - [x] I have confirmed the error originates in Paperclip itself — not in my agent adapter, API provider, or local configuration. **What happened?** The `Sentry SDK contract` job (`Real Sentry SDK isolation`) failed on pull request #15926 at head2d57542, run 38178686828 attempt 1, in `run-failure-sentry-real-sdk.test.ts > captures an early OpenCode timeout without inventing a deadline or exporting provider output`: ``` AssertionError: expected '[{"exception":{"values":[{"type":"Err…' not to match /private-timeout-|<html>|504|Timed out after/ ``` The serialized events contained no provider output. The match was the event id `f353d184e5504ee39db611f2e60f49ff`, which contains `504`. The pull request changes build scripts only, and the same job passed on its previous head and on the re-run of this one. **Expected behavior** The assertion fails only when a captured event carries the stderr canary or a `Timed out after` message, never because of the random event id Sentry assigns. **Steps to reproduce** 1. On `master`, run `node -e 'console.log(/private-timeout-|<html>|504|Timed out after/.test(JSON.stringify([{event_id:"f353d184e5504ee39db611f2e60f49ff"}])))'`. It prints `true`. 2. Run the `Sentry SDK contract` job repeatedly; about one run in seventy draws an event id containing `504` and fails on that assertion. **Paperclip version or commit** `master` atbfe7c63. **Deployment mode** Other (CI, GitHub Actions) **Installation method** Built from source (pnpm dev / pnpm build) **Agent adapter(s) involved** - [x] Not adapter-specific (core bug) **Database mode** Not database-related **Access context** Unclear / not applicable **Node.js version** 24 (the workflow's `actions/setup-node` version) **Operating system** ubuntu-latest (GitHub-hosted runner) **Relevant logs or output** ``` ❯ src/__tests__/run-failure-sentry-real-sdk.test.ts (11 tests | 1 failed) 2571ms × captures an early OpenCode timeout without inventing a deadline or exporting provider output 510ms AssertionError: expected '[{"exception":{"values":[{"type":"Err…' not to match /private-timeout-|<html>|504|Timed out after/ "[{\"exception\":{\"values\":[{\"type\":\"Error\",\"value\":\"OpenCode execution timed out\",\"mechanism\":{\"type\":\"generic\",\"handled\":true}}]},\"event_id\":\"f353d184e5504ee39db611f2e60f49ff\", ... ``` **Relevant config (if applicable)** None. **Additional context** The assertion was added in #15789. The fixture's stderr is `<html>504 Gateway Time-out private-timeout-credential-canary</html>`, so `<html>` and `private-timeout-` already cover it; `504 Gateway` keeps a third, independent check on the status text itself. **Privacy checklist** - [x] I have reviewed all pasted output for PII (usernames, file paths, API keys, tokens, company names) and redacted where necessary. ## What Changed - `server/src/__tests__/run-failure-sentry-real-sdk.test.ts`: the forbidden-substring assertion matches `504 Gateway` instead of `504`. One line. ## Verification - With the real SDK installed outside the workspace the way the workflow does it (`npm install --prefix <tmp> @sentry/node@10.71.0`, `NODE_PATH` pointing at it, `PAPERCLIP_REQUIRE_SENTRY_TEST_SDK=1`), `pnpm --filter @paperclipai/server exec vitest run src/__tests__/run-failure-sentry-real-sdk.test.ts` passes 11/11 on three consecutive runs. - The failing run's payload against both patterns: the old regex matches it (`true`), the new one does not (`false`). The stderr canary with its HTML wrapper and the bare `504 Gateway Time-out` both still match the new regex. - 100,000 random 32-character hex ids: 0.73% contain `504`; none can contain `504 Gateway`. ## Risks - The assertion no longer catches a leaked `504` that appears without `Gateway` after it. The fixture's canary always carries `Gateway`, and the `<html>` and `private-timeout-` alternatives cover the same leak independently, so the test's purpose is unchanged. ## Model Used - Claude (Anthropic), model id `claude-fable-5-1` (Claude Fable 5.1), extended thinking, with tool use (shell, file editing, GitHub CLI) in a Paperclip agent session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>