mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-19 01:14:06 +02:00
DottaandPaperclip
7420af26fc
## Thinking Path > - Paperclip uses pull request CI to check changes before merge. > - The merged CI profiler records a selected full PR suite in Honeycomb. > - A real 48-job profile exposed an export failure after Honeycomb accepted only part of the trace. > - The same run exposed three fixture subprocesses that looked like incomplete test launches, one direct reporter without a launcher marker, and one overlong test name. > - The exporter needs to handle temporary ingest rejection without sending accepted spans again. > - This pull request fixes those cases and documents the trace display limit. > - The benefit is a complete, checkable CI waterfall with test-file timings. ## Linked Issues or Issue Description **What happened?** The first live `profile-ci` run on #16423 completed all 48 jobs, but the post-run exporter stopped when Honeycomb rejected part of a batch. The initial trace had no root span. Its coverage flag also reported four unfinished invocations and one invalid timing record. **Expected behavior** The exporter should pace large uploads, retry only temporarily rejected events, and publish the root after every child is accepted. Completed reporters should count as complete. Long test names should not erase test timing. **Steps to reproduce** 1. Add `profile-ci` to a CI CODEOWNER PR with the merged profiler. 2. Run the full PR suite, as in [run 38732093469](https://github.com/paperclipai/paperclip/actions/runs/38732093469). 3. Inspect the separate [failed exporter run](https://github.com/paperclipai/paperclip/actions/runs/38733393370) and the trace coverage fields. **Paperclip version or commit** `12612b06eb4332844ac4c8770a1fb6c6b7eccbc9` (PR run head, based on merged #16378). ## What Changed - Pace Honeycomb uploads and retry only transiently rejected events from each batch. - Preserve overlong test timings with bounded names and count complete reporter pairs without a launcher record. - Keep nested Vitest fixture runs from inheriting CI profiling and creating false coverage gaps. - Clarify ordinary CI access, large-trace display limits, and export behavior in the operator guide. ## Verification - [Full PR suite](https://github.com/paperclipai/paperclip/actions/runs/38732093469/attempts/1): 48 of 48 jobs succeeded; 10m 29s wall time. The selected jobs used the AWS public PR fleet. The protected exporter used a separate GitHub-hosted runner. - [Clean Honeycomb file waterfall](https://ui.honeycomb.io/paperclip/environments/test/datasets/paperclip-ci/trace?trace_id=01ed0c541b0442e613a030ffb6dc8d9b&trace_start_ts=1792065955&trace_end_ts=1792066584): 48 jobs, 822 steps, 2,622 file spans, and `ci.profile_complete=true`. It was reconstructed from that run's 45 timing artifacts after identifying three nested fixture records. The full trace also recorded 44,497 tests; Honeycomb's chart omits some deep test spans because it displays at most 32,000 spans per trace. - The original exporter stopped after a partially accepted batch. A local replay of that batch succeeded. I used the authorized local test-environment key to complete the original upload and create the file-level view. No key was added to this branch or PR. - `node --test .github/scripts/tests/ci-profile.test.mjs scripts/ci-profile/reporters.test.mjs`: 18 passed. - `pnpm exec vitest run server/src/__tests__/vitest-chat-shards.test.ts`: 1 passed. - `git diff --check`: passed. - The local `pnpm -r typecheck` and profiled fixture rerun could not start: the package-manager shim could not verify the `pnpm@9.15.4` registry signature because its fetches failed. `pnpm test:run` and `pnpm build` were not run locally; hosted CI is pending. - The updated exporter has not yet run from protected `master`; this PR remains draft until CI and review finish. ## Risks - A 48,000-span export now takes about three extra minutes due to pacing. Retry limits still fail a persistently rejected batch and keep the root unpublished. - Honeycomb may retain duplicates from an already partial export. The clean file-level evidence trace uses a separate trace ID. - The API key stays only in the protected `ci-profiling` environment. PR jobs write timing artifacts but never receive it. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — see `CONTRIBUTING.md`. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, code execution, GitHub API, and browser inspection. The runtime does not expose the exact serving model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>