docs(skills): add standalone test-audit workflow (#16211)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Repository skills guide agents through engineering work.
> - Test gardening must preserve useful regression coverage while
reducing repeated work.
> - Contributors need a local procedure for evidence, validation, and
delivery.
> - This pull request adds a standalone test-audit skill with a short
entry point and focused guides.
> - The procedure supports focused cleanups and coordinated audits
without requiring an external skill checkout.

## Linked Issues or Issue Description

**Issue type**
Missing documentation.

**Where is the issue?**
The repository has no standalone `.agents/skills/test-audit/` procedure
on master.

**What's wrong?**
Test audits need consistent rules for retaining contracts, proving
removal safety, recording cost evidence, and tracking stacked delivery.

**Suggested fix**
Add a self-contained skill with candidate evidence, validation,
campaign, and Paperclip delivery guides. Preserve upstream attribution.

Related audit work: Refs #15978, Refs #16022, Refs #16197. The duplicate
search found no standalone skill PR. This docs change does not duplicate
planned core features in ROADMAP.md.

## What Changed

- Add a short test-audit entry point and agent metadata.
- Require contract and keeper evidence before removing tests. Apply an
authoring gate to added or changed tests.
- Define targeted and exhaustive campaign modes, a second review of
redundant layers, and full keeper dependency checks.
- Document matched baselines, intended-failure controls, selection
checks, honest cost claims, and current-head delivery gates.
- Add Paperclip commands and invariants. Retain OpenClaw attribution and
its MIT notice.

## Verification

- The skill frontmatter validator passed with cached PyYAML.
- Agent metadata YAML parsed. The default prompt names `$test-audit`.
- All eight relative Markdown links resolve. All seven delivery files
matched the reviewed source.
- `git diff --check` passed. The PR contains only seven Markdown/YAML
skill files.
- An independent scenario review checked the campaign completion and
coverage rules.
- Local application typecheck, full Vitest, and build were not run for
this documentation-only change. Fresh current-head CI passed: 55
successful checks and two expected skips on
`1a9af8ffacd532882f0a01bac8ab3f5063ab9cd9`.
- E2E shard 8/8 was interrupted after its assertions passed. A single
same-head job retry passed. The interruption cause was not exposed.
- Greptile reviewed the current head at 5/5 with no actionable findings.
There are no review threads. The PR has no merge conflicts and remains
open as requested.

## Risks

- Low runtime risk. This PR changes contributor instructions only.
- Agents can misapply audit evidence. The skill requires valid keepers
and preservation proof, rejects keeper cycles, and makes incomplete
audits explicit.
- CI routing can change. The guide requires readers to inspect current
repository files.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex based on GPT-6 wrote and tightened the skill. Its exact
deployment model ID and context-window size are unavailable. Independent
scenario review and PR delivery used OpenAI `gpt-6.1-sol`, with
reasoning, repository inspection, and tool execution. The context-window
size was not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>

.agents/skills/test-audit/CAMPAIGN.md

@@ -0,0 +1,98 @@
+# Coordinating a test-gardening campaign
+
+Use this guide for a broad sweep or multiple PRs. Apply the decision and
+preservation rules in [SKILL.md](SKILL.md) to every batch.
+
+## 1. Define the audit boundary
+
+From the request, establish scope, desired benefit, time budget, target branch,
+and delivery authority. Pin the base revision and inspect CI routing and cost
+data, including setup/import overhead. Choose one mode:
+
+- **Targeted sweep:** investigate promising owners or expensive repeated work.
+  Report the examined area and exclusions; do not claim subsystem completeness.
+- **Exhaustive audit:** inventory every in-scope test file and scenario, including
+  colocated, shared-package, browser, and dedicated-lane coverage. Assign each to
+  one owner and record its baseline as passed, failed, blocked, or unrun.
+
+For an exhaustive claim, every declaration needs a decision and evidence;
+classify table rows separately when their contracts differ. Missing baseline
+results or unread tests leave the audit partial. Existing matching-revision CI
+can supply results; a targeted sweep does not require a full-repository census.
+Reserve time for preservation checks and reporting; an unfinished validation
+leaves an unvalidated local patch, not a PR-ready result.
+
+## 2. Assign owners and record decisions
+
+Split work by protected production behavior, including tests across directories.
+When delegation is authorized, assign each worker an isolated checkout, owner,
+prior findings, model if specified, and discovery/edit/publication authority.
+Tell workers to preserve others' changes. One coordinator owns shared fixtures,
+routing, and integration. Do not edit a checkout with active test runs.
+
+Keep one ledger: candidate, contract, retain/repair/consolidate/delete/defer,
+reason, keeper, baseline, validation, and branch/PR status. Record reviewed areas
+with no safe removals too. Discovery stays read-only until ownership is clear;
+qualified edits need no additional user-approval round within existing authority.
+
+## 3. Plan removal of redundant layers
+
+Before editing a broad batch, review the ledger a second time by contract and
+boundary. Look for suites that replay the same behavior through wrappers,
+mocked collaborators, or provider-specific copies of a shared implementation.
+Individual cases can look reasonable while the entire layer is redundant.
+
+Produce a compact layer plan: retired suites, retained owner per contract,
+unique assertions to transfer, affected routing/commands, validation controls,
+and any unused fixtures or production seams. Prefer a boundary that reaches
+the responsible code; preserve distinct transport, lifecycle, and permission
+risks. If no whole layer qualifies, record that and proceed with justified
+case-level changes. Large deletions are not a requirement.
+
+The coordinator reconciles keeper dependencies across all batches before
+cutover. Follow each chain to surviving proof; reject cycles and recheck keepers
+another batch modifies or excludes from routing. A batch is ready when every
+retired contract has retained proof or retirement evidence. Production-seam
+removal still needs separate caller evidence.
+
+## 4. Cut over and verify preservation
+
+Move unique assertions first, apply one coherent batch, and update selection
+and ownership references. Follow [validation.md](references/validation.md) for
+baselines, controls, restoration, and cost claims. Serialize heavy database or
+process runs when host capacity is shared; clean up only owned resources.
+
+Review the combined diff against the layer plan. For broad or consequential
+batches, use independent preservation review when delegation is available and
+authorized; otherwise disclose that limitation. Resolve each finding with
+source evidence or restored coverage and validate repaired assertions with an
+intended-failure control. No batch is complete with an unresolved coverage gap.
+
+An exhaustive audit requires a decision for every declaration, a completed and
+classified baseline for every file, and passing preservation proof for each
+removal. Diagnosed pre-existing defects may remain as explicit follow-ups;
+unresolved setup failures or unread tests leave the audit partial. If time
+expires first, report the remaining inventory and validation status.
+
+## 5. Deliver and integrate
+
+Prefer a few coherent PRs to one PR per trivial deletion. For a stack, record
+each PR's head SHA, base branch/SHA, unique diff, and intended final target.
+After a parent is squash-merged, discover its actual merge SHA and replay only
+the child's unique change if necessary. Inspect the resulting PR diff for
+accidentally reintroduced parent commits or files.
+
+Resolve integration conflicts by reading new contracts on the target branch.
+If a deleted file gained a regression, preserve that regression at an appropriate
+owner; do not blindly accept the deletion or restore the entire old suite.
+
+For authorized merges, use checks and reviews tied to the current PR head,
+inspect unresolved findings, and respect the repository's merge policy. A
+previous green head does not validate a changed head.
+
+A PR marked “merged” into an intermediate stack branch is not necessarily
+landed in the intended integration branch. Verify the full parent chain and
+final diff inclusion before reporting the campaign landed. Preserve or clean
+up branches/worktrees according to the user's instructions and retain needed
+evidence before cleanup. Report delivery separately from audit completeness;
+stop new discovery when the user asks to wrap up.

.agents/skills/test-audit/NOTICE.md

@@ -0,0 +1,34 @@
+# Attribution
+
+Paperclip's test-audit skill is inspired by the organization and audit principles
+of OpenClaw's [test-audit skill](https://github.com/openclaw/openclaw/tree/main/.agents/skills/test-audit),
+consulted on 2026-10-13. It uses the short entry point, separate campaign guide,
+and focused references pattern, with independently written Paperclip procedures
+and lessons from our test-gardening work. It has no execution or documentation
+dependency on an OpenClaw checkout or another installed skill.
+
+The upstream license is retained below for the adapted material.
+
+```text
+MIT License
+
+Copyright (c) 2026 OpenClaw Foundation
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+```

.agents/skills/test-audit/SKILL.md

@@ -0,0 +1,78 @@
+---
+name: test-audit
+description: >
+  Garden the test suite by finding and removing redundant, vacuous, or
+  implementation-coupled tests while preserving useful regression coverage.
+  Use for test audits, test pruning, duplicate coverage, or reducing CI test
+  cost. Supports focused cleanups and coordinated audit campaigns.
+---
+
+# Test Audit
+
+Reduce execution and maintenance cost while preserving meaningful failure
+detection. Investigate aggressively; delete only with evidence. Retaining an
+area after review is a valid result. This skill is self-contained; upstream
+attribution is in [NOTICE.md](NOTICE.md).
+
+## Choose the scope
+
+- Focused cleanup: follow this entry point and read the
+  [candidate evidence guide](references/audit.md) before editing.
+- Multiple owners, agents, or PRs: also read [CAMPAIGN.md](CAMPAIGN.md). Declare
+  whether the work is a targeted sweep or an exhaustive subsystem audit.
+- Validation and cost claims: read [references/validation.md](references/validation.md).
+- Paperclip commands, invariants, and delivery: read
+  [references/paperclip.md](references/paperclip.md) alongside repository instructions.
+
+## Decision gate
+
+For every proposed removal, answer:
+
+1. Which observable contract and plausible regression do the assertions cover?
+2. Which retained file/case detects that regression through the responsible
+   production code? Otherwise, why is the assertion vacuous or contract retired?
+3. Does this candidate uniquely cover a transport, permission, lifecycle,
+   storage boundary, platform, default, or other independent failure mode?
+4. What improves: executions, fixtures, processes, maintenance, or runtime?
+
+Prefer the smallest test that reaches the code responsible for the contract.
+A broader suite is not automatically stronger proof. Similarity, size, slowness,
+or suspected AI authorship alone never justify deletion.
+
+## Prevent new duplication
+
+Apply this gate to tests added or changed during gardening:
+
+- Name the regression that existing proof misses. Extend an appropriate owner
+  or table before adding another suite; another layer needs a distinct risk.
+- Exercise the responsible production behavior. Do not add a mock that supplies
+  the result being asserted or a production hook solely for convenient testing.
+- Check observable outcomes that survive behavior-preserving refactors. A
+  negative case must reach its named guard with otherwise valid inputs.
+
+## Focused workflow
+
+1. **Inspect:** pin the base; read candidates, keepers, fixtures, production
+   callers, history, and CI routing. Preserve unrelated work.
+2. **Plan:** mark retain, repair, consolidate, delete, or defer. Name remaining
+   proof and assertions to move; look for redundant suites as well as cases.
+3. **Prove:** establish the affected baseline locally or from matching-revision
+   CI. Diagnose failures before treating them as cleanup candidates.
+4. **Edit:** move unique assertions first, then retire duplicates and unused
+   support. Production-seam deletion needs its own caller/contract evidence.
+5. **Validate:** run keepers and affected siblings, verify selection, and apply
+   the validation guide's intended-failure controls where needed.
+6. **Review:** reconcile deleted contracts with retained proof. Publish or merge
+   only within the task's authorization.
+
+## Completion and reporting
+
+Report removals and keepers, retained/deferred candidates, checks and limitations,
+distinct cases versus repeated executions, and test/support versus production
+LOC. Count documentation separately. Label cost evidence as measured, estimated,
+or historical; a proven simplification does not require a measurable speedup.
+
+State the actual endpoint: findings, validated local patch, published PR,
+intermediate merge, or landed in the intended target. Stop at the requested
+scope/time limit or completed delivery, not a deletion quota. For another pass,
+refresh the base and prior findings; revisit retained tests only with new evidence.

.agents/skills/test-audit/agents/openai.yaml

@@ -0,0 +1,4 @@
+interface:
+  display_name: "Test Audit"
+  short_description: "Prune redundant tests while preserving useful coverage"
+  default_prompt: "Use $test-audit to garden this test suite and remove well-supported redundancy while preserving meaningful regression coverage."

.agents/skills/test-audit/references/audit.md

@@ -0,0 +1,72 @@
+# Candidate evidence and retention
+
+Use this reference before deleting or consolidating tests.
+
+## Search for waste, then inspect the contract
+
+| Candidate pattern | What to establish before changing it |
+| --- | --- |
+| Several tests repeat one assertion through the same production path | A retained case has equivalent inputs, failure sensitivity, and relevant setup. |
+| A test file only imports another test file | The real owner is selected in the same required runs; the alias adds no setup, configuration, or separate entry point. Check both CI and documented commands. |
+| A large table repeats equivalent rows | Preserve rows that exercise different branches, boundary values, error classes, platforms, or defaults. |
+| A bulk fixture creates many interchangeable records | Establish that the count does not cross batching, pagination, concurrency, or transaction thresholds. Retain mixed outcomes and persisted state assertions. |
+| Assertions copy lists, exports, strings, or source shapes | Determine whether the list or shape is an independent public, packaging, generated, or architecture contract. |
+| Mocks implement the behavior being asserted | Find a test that exercises the responsible production behavior; otherwise repair or relocate this proof before removing it. |
+| Fixtures fabricate receipts, ordering, or persisted state | Verify that the responsible production path creates that evidence; fixture-supplied outcomes cannot prove it does. |
+| A negative case passes for an unrelated reason | Reach the named guard with otherwise valid inputs. Fix useful coverage rather than discarding it as waste. |
+| A test-only export, reset hook, or wrapper becomes unused | Search runtime callers, package exports, generated callers, supported integrations, and history. File location alone does not make it dead production code. |
+
+Also inspect self-comparisons, assertions with expected values derived from the
+subject, probes with no checks, and names that promise more than the body tests.
+Explicitly assertion-free smoke tests can still protect module loading or
+execution; identify that purpose before deleting them.
+
+## Reasons to retain apparently similar tests
+
+Two tests can use the same payload while protecting different failure modes:
+
+- Authorization and tenant isolation at separate exposed entry points.
+- Direct API versus SDK/MCP transport, including serialization and errors.
+- Initial execution versus replay, recovery, restart, or concurrent access.
+- In-memory return values versus committed database state or rollback.
+- Default-off behavior versus explicit opt-in; missing versus invalid input.
+- Platform behavior, published package exports, migrations, generated-language
+  compatibility, or intentional prompt/protocol bytes.
+- Browser interaction, reconnect, pagination, lost responses, attachment
+  persistence, or visible ordering absent from lower-level coverage.
+
+Keep a static check when it is an independent and appropriate guard of a
+contract. Implementation-shaped checks that fail on a harmless identifier
+rename deserve scrutiny; this does not make all source inspection disposable.
+Likewise, a slow integration test can be the only meaningful proof of a
+transaction or lifecycle. Cost helps rank investigation, not determine truth.
+
+## Evidence before editing
+
+Read the candidate's full body and parameter rows, setup/teardown, production
+owner, real callers, proposed keeper, configuration, and relevant history.
+Look for why the test was introduced and regressions added since. Inspect the
+dependency's actual contract when a claim depends on library behavior.
+
+Use a compact table or PR notes:
+
+| Candidate | Decision and reason | Contract/regression | Keeper or retirement evidence | Validation and benefit |
+| --- | --- | --- | --- | --- |
+| File and case/rows | Retain / repair / consolidate / delete / defer | Observable failure detected | Exact file/case, or evidence the contract no longer exists | Commands/control, cost or maintenance reduction, limitations |
+
+For consolidation, list unique assertions to move. The keeper must actually
+exercise them with valid fixtures; adding an assertion that cannot fail is
+not coverage preservation. If setup or the selected test project differs,
+explain why it does not change the protected contract.
+
+## Two useful distinctions
+
+- An import-only alias and its directly selected owner may duplicate execution.
+  An identical-looking alias importing a package suite absent from the default
+  run may be the only entry point and must stay until routing is repaired.
+- Repeated successful records in a bulk route fixture may be reducible while
+  retaining mixed failures and persisted outcomes. The same reduction would
+  invalidate a test whose record count exercises a batch or pagination limit.
+
+If evidence is missing, defer that candidate and keep working on independent
+ones. Audit-only findings do not become permission to delete tests elsewhere.

.agents/skills/test-audit/references/paperclip.md

@@ -0,0 +1,71 @@
+# Paperclip integration
+
+This reference adapts the standalone workflow to this repository. Read root
+and scoped `AGENTS.md` and their required product/development documents. Resolve
+paths below from the repository root and inspect current contents; counts and
+shard layouts evolve.
+
+## Discovery and test selection
+
+- `package.json`, `vitest.config.ts`, and package-local Vitest configurations
+  describe the test projects and commands.
+- `scripts/run-vitest-stable.mjs` selects and runs the default suites. Inspect
+  its supported dry-run/mode/shard flags before constructing routing checks.
+- `.github/workflows/pr.yml` identifies the PR workflow; follow its reusable
+  workflow references to the actual lanes and shard matrix.
+- `scripts/general-server-shard.mjs` and the general-server, serialized, and
+  E2E duration JSON files under `scripts/` provide routing and historical cost
+  clues. Read each manifest's provenance; a file's weight is not per-case time.
+- Search tests colocated with production modules as well as `__tests__/`.
+  Some acceptance files import other suites. Verify both direct selection and
+  aliases, including package suites outside the default project list.
+
+Example focused command, replacing the example path with the actual owner:
+
+```sh
+pnpm --filter @paperclipai/server exec vitest run src/services/OWNER.test.ts
+```
+
+`pnpm test` is the normal Vitest entry point. Browser and release-smoke suites
+are opt-in; run them when affected or explicitly verifying those flows.
+Follow repository requirements for a PR-ready handoff, including the full
+typecheck/test/build checks when required; identify local versus CI execution
+and report anything unrun. Do not rerun the entire repository on every audit
+heartbeat. Test removals do not excuse required checks.
+
+## Contracts deserving special care
+
+Preserve company isolation, board/agent authorization, atomic checkout,
+single-assignee rules, approval gates, budget hard stops, and mutation activity
+logging. Persistence, idempotency, cancellation, concurrency, recovery, and
+ordering often have separate owners even when test names look similar.
+
+Distinguish first-party Telemetry, opt-in OpenTelemetry observability, and the
+local run log by their paths as defined in `AGENTS.md`. A pruning task is not
+permission to remove privacy, endpoint gating, or event-contract coverage.
+
+## Environment and delivery
+
+Database-backed tests may start real embedded Postgres even though development
+can use PGlite. Check the actual fixture. An isolated install with scripts
+disabled may need documented dependency setup before its binaries work.
+Resource exhaustion must not be interpreted as a tested product failure or a
+passing control. Coordinate heavy runs and clean up only owned resources.
+
+Follow existing branch/worktree conventions and protect unrelated working-tree
+edits. New repository audit plans/receipts belong in
+`doc/plans/YYYY-MM-DD-<scope>-test-audit.md`. Prefer one concise receipt per
+coherent batch or campaign. For Paperclip issue work, follow the issue's artifact
+and work-product requirements rather than relying only on local paths.
+
+When publishing is authorized, fill every section of
+`.github/PULL_REQUEST_TEMPLATE.md`, including exact model information that is
+actually known; do not fabricate context-window details. Use an accurate PR
+title consistent with current repository policy. A deletion-only cleanup may
+be a refactor and does not need a token added test to satisfy a title heuristic.
+
+When merging is authorized, verify current-head CI and fresh repository review
+results, inspect review threads and edited bot summaries, and resolve genuine
+findings. After a stack merge, verify inclusion in the intended final branch
+as described in [CAMPAIGN.md](../CAMPAIGN.md). No fixed check count or old
+review score substitutes for inspecting the current requirements.

.agents/skills/test-audit/references/validation.md

@@ -0,0 +1,87 @@
+# Preservation, cost, and evidence
+
+## Prove the retained behavior
+
+Pin baseline and candidate revisions and record the actual command, project,
+filters, environment differences, and pass/fail/skip counts. Prefer focused
+owner and affected sibling runs. A zero-test run, setup error, or fully skipped
+suite is not a successful baseline. Use recent CI only when its revision and
+test selection support the claim being made.
+
+For straightforward exact duplicates, compare all assertions, fixtures,
+configuration, and selection. For assertion migration, weaker-to-stronger
+coverage substitution, security/persistence boundaries, or uncertain failure
+sensitivity, use a deliberate control:
+
+1. Run the keeper normally and observe it passing.
+2. In an isolated, idle checkout, temporarily introduce the specific plausible
+   production regression the keeper is meant to detect, or revert the relevant
+   fix. Keep unrelated guards and fixtures valid.
+3. Verify the expected case fails at the intended assertion. A timeout, missing
+   dependency, unrelated rejection, or initialization failure is not proof.
+4. Restore the exact pre-control bytes in a guaranteed cleanup step. Check the
+   diff/hash and run the same selected case successfully again.
+5. Run the affected candidate owners without that narrow name filter before
+   calling the complete owner suite green.
+
+Never mutate a checkout with active tests, commit a temporary mutation, or
+overwrite someone else's source changes during restoration. Do not add a
+permanent replacement test solely to justify removing an obvious duplicate.
+
+## Verify what actually runs
+
+Inspect the real runner entry point, include/exclude rules, package projects,
+shard routing, dedicated CI lanes, and documented focused commands. A test
+existing on disk does not guarantee that it executes in the required run.
+The required invocations are the affected CI lanes and supported developer
+commands for the audit's scope. Redirecting a documented alias command to its
+owner is valid when the configuration and protected contracts remain equivalent;
+preservation does not require retaining an obsolete filename indefinitely.
+
+When removing test-import aliases, prove that the retained owners remain
+selected with the intended multiplicity in each required invocation. Update
+explicit commands and ownership inventories. Use dry-run routing to inspect
+selection and an actual focused run to confirm execution where import caching
+or collector behavior makes listing ambiguous. Intentional runs on different
+platforms/configurations are separate contracts, not duplicate registrations.
+
+## Measure the benefit honestly
+
+Keep these quantities separate:
+
+| Quantity | Interpretation |
+| --- | --- |
+| Deleted source declarations / expanded table cases | Change in test code or case inventory. |
+| Avoided registrations / suite invocations | Repeated execution eliminated; may remove no unique contract. |
+| Test/support LOC | Maintenance reduction, not a runtime estimate. Report additions and net change. |
+| Production LOC | Runtime code changed, with its own justification; do not combine it with test LOC. |
+| Historical manifest weight | Scheduling estimate at its recorded revision/date, often including setup/import cost. |
+| Case time / owner suite time / process time | Different scopes; compare like with like. |
+| Shard time / workflow critical path | Scheduling and parallelism effects; summing suite savings does not predict elapsed CI savings. |
+
+Use bounded matched baseline/candidate runs when a quantified runtime claim is
+important. Keep environment and configuration comparable, alternate order when
+practical, and disclose sample count and variation. Inspect neighboring suites
+for host-wide drift. A faster unrelated shard is not deletion evidence; a slower
+whole suite does not establish that removing a duplicate caused a regression.
+
+Logs that only print slow cases provide censored observations: an absent case
+is below the reporting threshold, not zero milliseconds. A passing retry is
+validation evidence, not a fair speed comparison with the failed attempt.
+Report uncertain speed effects as inconclusive and keep structural benefits
+separate. Do not demand significance testing for every safe maintenance edit.
+
+## Failures and durable reporting
+
+Classify failures before acting: product assertion, test defect, setup/dependency,
+host capacity, or CI infrastructure. Fix relevant defects or report the blocker;
+do not weaken assertions, increase budgets, skip suites, or erase a failing
+baseline to obtain a green audit. Retry a demonstrated transient problem within
+a stated bound; a recurring failure needs investigation.
+
+Use PR notes for small changes. For a campaign, keep a compact dated receipt
+with revisions, decisions, keepers, commands, results, controls, measurements,
+and limitations. Link accessible CI runs/artifacts when they support claims.
+Temporary paths and hashes alone do not make evidence durably inspectable.
+Do not generate large evidence dossiers for trivial deletions or store raw
+logs containing secrets. State precisely which broad checks remain unrun.