Files
PaperClipAI/doc/plans
DottaandClaude Opus 5.5 f35eb1e5ef fix(observability): judge the TTFT budget from run creation; report the wait behind a live turn (#19941)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Invariant 8 of the durable-ownership plan says turns must start
fast. #19863 added per-run speed budgets that write
`run.performance.budget_exceeded` to the run log.
> - The Phase 2 chaos data (#19927) showed follow-up TTFT about 3x over
the claude budget.
> - The cause is measurement scope. A follow-up comment posted during a
live turn waits for that turn to end, and TTFT is anchored at the
comment. The budgets came from idle follow-ups.
> - This pull request judges the TTFT budget from run creation and
reports the wait separately.
> - The benefit is that budget breaches point at real start-up slowness,
not at time spent behind a turn the user is already watching.

## Linked Issues or Issue Description

Refs #19863 (speed budgets), #19927 (Phase 2 data), #19852 (startup
sub-spans), #19888 (warm default).

**What happened?**
In two local storm waves, C1 follow-up runs breached the claude warm
TTFT budget (p50 27.6–29.9 s against 9.5 s). Their spans show
`comment.to_run_created` at 6–62 s (waiting for the live turn), then
session resume at 1.4–3.8 s and submit-to-first-event at 2–5 s. Codex
without the wait was about 4.8 s, the same as the warm-follow-up
baseline.

**What did you expect?**
The budget should judge start-up latency, which is what its baseline
measured. The wait behind a live turn should be visible but not judged.

**Steps to reproduce**
1. Run a local chaos storm with C1 tasks.
2. Read `run.performance.budget_exceeded` rows with metric `ttft` for
`issue_commented` runs, and compare them with each run's
`comment.to_run_created` span.

## What Changed

- `native-run-trace.ts`: the TTFT budget check measures from run
creation (start of `heartbeat.queue`, or end of the ingress span). The
TTFT span itself does not change.
- `native-speed-budgets.ts`: breaches can carry `inputWaitMs` (TTFT
only).
- `tests/restart-chaos/ttft-baseline-stats.ts` and `ttft-baseline.ts`:
the wait behind a live turn is its own measured step (`inputWait`), next
to the full TTFT and the TTFT from run creation. The TTFT table gains a
"Wait behind live turn" column. The full comment-to-first-token time
stays the user-visible metric.
- `doc/run-log-events.md` and the plan's speed-budget section describe
the rule.

## Verification

- `cd server && npx vitest run
src/services/native-runtime/native-speed-budgets.test.ts
src/services/native-runtime/native-run-trace.test.ts`: 43 pass, 2 new.
`pnpm exec vitest run --config tests/restart-chaos/vitest.config.ts
tests/restart-chaos/ttft-baseline-stats.test.ts`: 12 pass, 1 new. They
cover a follow-up under budget after a 60 s wait (no breach), a
follow-up over budget (breach with `inputWaitMs`), and queueing after
run creation still judged.
- Server typecheck passes.

## Risks

- Low. This is a run-log-only change (AGENTS.md §5.7): no Telemetry and
no span attributes change.
- A slow wait before run creation is no longer a TTFT budget breach. The
TTFT span still records it, and the breach payload names it. Admission
holds after run creation are still judged.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with
tool use (shell, file edits).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 20:27:41 -06:00
..