mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-12 14:19:00 +01:00
feat(chaos): check I6 and I8 per crash point; ready the Daytona points (#19984)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The on-demand crash-point suite (#19946) kills the control plane at named durable write boundaries and checks the plan's invariants > - Its first full run measured I1–I4 and I7, but I5, I6 and I8 were reported as not measured > - #19960 (generation stamping) and #19863 (speed budgets) now record what I6 and I8 need in the run log > - The six Daytona-only points have not run yet, and three of them cannot be reached by a plain task > - This pull request checks I6 and I8 per point, names the I5 gap, and readies the Daytona points for the coming Daytona wave > - The benefit is that every invariant is either asserted or a named gap, and a Daytona run explains any point it cannot reach ## Linked Issues or Issue Description Refs #19831 (plan, Phase 5). Follows #19946 (the suite). Uses #19960 (event generations) and #19863 (speed budgets). **Problem** - The suite reported I5, I6 and I8 as "not measured". - A Daytona run would report three points as `not_reached` with no reason. - A Daytona run without its key or image failed only after a 3-minute boot. **Expected behavior** - I6 and I8 are checked at each point. - I5 is a named gap with its reason. - Daytona points say what they need, and a Daytona run checks its inputs before it boots. ## What Changed - `tests/restart-chaos/invariants.ts`: `runLogViolations` checks two invariants from the run log, scoped to the wave's company: - I6: a run that failed with an error that names a stale or earlier-generation event; - I8: each `run.performance.budget_exceeded` event, with the budget evidence (metric, measured, p95, adapter, start kind). Both are keyed by run or event id, so the per-point baseline diff charges each one to the point that produced it. I5 is reported as `notMeasured` with a reason (`I5_GAP`). - `packages/shared/src/crash-point-registry.ts`: an optional `prerequisite` on the three Daytona points a plain task cannot reach. These are the teardown claim (needs a failed release), the reusable destroy (needs a `reuse_by_environment` lease), and stop-and-retain. - `tests/restart-chaos/crash-points.ts`: - `--list` prints each point's `needs:`, and a `not_reached` result names it too; - `--env daytona` refuses to start without `DAYTONA_API_KEY` and an image. - Docs: the README and the plan describe the I5/I6/I8 coverage and the Daytona points. ## Verification - `pnpm exec vitest run --config tests/restart-chaos/vitest.config.ts tests/restart-chaos/crash-points.test.ts`: 12 passed. - `cd packages/shared && npx vitest run src/crash-points.test.ts`: 15 passed. - `node scripts/typecheck-test-projects.mjs tests/restart-chaos` and `npx tsc --noEmit -p packages/shared` both pass. - I ran `runLogViolations` against a throwaway Postgres 18 with seeded rows, then stopped and deleted it. It returned: - I8 for the budget breach, with its evidence; - I6 for the run of this company that failed on `stale_generation_event`. It excluded the other company's stale run, the `process_lost` run and a non-budget event. - `node scripts/crash-points.mjs --list --env daytona` lists 19 points, with `needs:` on the three. `--env daytona` with no key or image fails at once with the missing names. - No harness run: the harness slot is held for other work. The next suite run exercises the new checks. ## Risks - Test harness only; no product code. - I6 is a proxy: it matches the run's error text and code. A failure that names a generation for another reason would be reported, with its evidence, for a person to judge. - I8 can flag breaches on a loaded host. Each breach carries its measured and budget values, so a load-caused breach is visible as one. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
46bf913fb9
commit
68d5fb5a6e
6 files changed
+114
-19
No files matched your search
@@ -21,6 +21,8 @@ export type CrashPointDefinition = {
|
||||
environments: readonly CrashPointEnvironment[];
|
||||
kinds: readonly string[];
|
||||
invariants: readonly CrashPointInvariant[];
|
||||
/** What a run must go through first when one plain task does not reach the point. */
|
||||
prerequisite?: string;
|
||||
};
|
||||
|
||||
export const CRASH_POINT_NAME_PATTERN = /^[a-z][a-z0-9_]*\.[a-z][a-z0-9_]*\.(before|after)_[a-z0-9_]+$/;
|
||||
@@ -97,18 +99,21 @@ export const CRASH_POINTS = {
|
||||
environments: ["daytona"],
|
||||
kinds: ["environment_teardown_claim", "environment_pending_cleanup"],
|
||||
invariants: ["I2", "I7"],
|
||||
prerequisite: "A pending_cleanup lease: a provider release that failed or could not be verified. A plain task's release succeeds and never claims a teardown.",
|
||||
},
|
||||
"environment.teardown.after_destroy_before_lease": {
|
||||
description: "A reusable sandbox was destroyed at the provider; its lease row is not released.",
|
||||
environments: ["daytona"],
|
||||
kinds: ["environment_run_lease_active", "environment_retained_hold"],
|
||||
invariants: ["I2"],
|
||||
prerequisite: "A reuse_by_environment lease that expires or is replaced (destroyReusableSandboxLease). The chaos Daytona environment's leases are ephemeral by default.",
|
||||
},
|
||||
"environment.stop_retain.after_claim_before_stop": {
|
||||
description: "A stop-and-retain intent and its claim are committed; the provider stop has not been retried.",
|
||||
environments: ["daytona"],
|
||||
kinds: ["environment_stop_and_retain_intent"],
|
||||
invariants: ["I2", "I7"],
|
||||
prerequisite: "A stop-and-retain intent: a run whose sandbox must be stopped but kept, for a native workspace export resume.",
|
||||
},
|
||||
"environment.lease.after_release_before_runtime_services": {
|
||||
description: "The run's environment leases are released; its workspace runtime services are not.",
|
||||
|
||||
Reference in new issue
Block a user