feat(dev): add incremental local verification and pass reuse (#19345)

## Thinking Path

> - Paperclip coordinates agents that make and verify repository
changes.
> - Local verification currently defaults to the complete Vitest suite.
> - Contributor instructions also require broad local checks at each PR
handoff.
> - CI already owns conservative affected-test selection and broad
final-revision checks.
> - This pull request adds local selection and pass reuse from those
existing contracts.
> - Agents can get focused feedback and use CI to close the remaining
coverage gaps.

## Linked Issues or Issue Description

**Subsystem affected**

Developer verification scripts and contributor guidance. Related
selection work: #18398 and #18464.

**Problem or motivation**

An agent that repeats an unchanged local verification step starts the
complete suite again. The handoff guidance also duplicates broad CI
coverage locally.

**Proposed solution**

Add `pnpm verify` for the actual working tree. Select affected Vitest
specifications with the CI task policy and dependency graph. Reuse
successful checks whose inputs are modeled. Run opaque checks fresh.
Report exit 2 when broader verification is required. Update the
contributor contract to accept applicable final-revision CI coverage.

**Alternatives considered**

Plain `vitest related` omits filesystem and opaque dependencies.
Commit-only cache keys miss dirty files. Running full local suites at
every handoff repeats CI work.

**Roadmap alignment**

This is repository development tooling. It does not change the product
roadmap.

## What Changed

- Add `pnpm verify`, `--explain`, `--base`, `--full`, and `--no-cache`.
- Reuse CI selection and exact runner ownership for dirty, staged,
committed, deleted, renamed, and untracked inputs.
- Keep native server suites and serialized suites in their local owners.
Keep six sequential chat shards.
- Cache selected passes within one worktree. Hash source, fixtures,
installed package contents, dependency bindings and installation
metadata, dotenv files, child environment, toolchains, platform, and
selection.
- Run opaque consumers fresh. Preserve failed, interrupted,
mutated-input, and partial-coverage outcomes.
- Update AGENTS.md, CONTRIBUTING.md, and DEVELOPING.md guidance. Record
the shared `verify` command as the sole exception to the package-script
rule. Register the new tests in the existing CI contract-test command.
- Add optional root/environment inputs to shared graph helpers. Existing
CI defaults stay the same.

## Verification

- Fourteen new integration tests pass locally across focused runs with
real Vitest selection and execution; CI colors are covered.
- 51 existing targeted selector/input, project-isolation, and
stable-runner checks passed. Four relevant runner contracts also pass
after the workspace change.
- Repeated unchanged execution skips reusable tests. Source,
environment, installed code, dependency bindings, and new untracked
tests invalidate results.
- Installed-code mutation with a restored modification time still
invalidates a cached pass. Internal skill-directory links preserve
reuse.
- Workspace B/C tests prove discovery, reusable/fresh partition
execution, pass reuse, and external-input failure. Chat selection
produces exactly six single-worker partitions.
- A deliberate unrelated failing test is omitted by narrow selection and
caught by `--full`.
- Red/green regressions prove fingerprints preserve file boundaries and
reject edits made between diff discovery and snapshotting.
- Opaque external-file changes still fail on a subsequent run. Source
mutation and process interruption cannot publish reusable passes.
- `pnpm verify --explain` works in the checkout and requests broad CI
coverage for this tooling change. No duplicate full local suite was
started.
- [Final-revision
CI](https://github.com/paperclipai/paperclip/actions/runs/42117291789)
passed on `ed85d96ddff82c05c325948680b593b0f48c473d` (attempt 2). The
scoped Canary retry preserved all passing test/build results.
- [Greptile
review](https://github.com/paperclipai/paperclip/pull/19345#issuecomment-6670050175)
is 5/5; all four review threads are resolved.
- The additional Paperclip review concluded neutral because its
execution stopped; it produced no assessment.

## Risks

- Pass reuse depends on conservative dependency classification. Opaque
consumers, external database/live settings, and non-isolated projects
retain fresh or broad coverage.
- Local passes cover selected Vitest checks. Required CI,
build/typecheck, release, browser, and platform acceptance remain
separate obligations.
- Cross-worktree caching and cross-agent scheduling are outside this
change.
- The initial installation scan reads dependency contents;
stat-validated, worktree-local digest memoization avoids rereading
unchanged contents. Subsequent scans still inspect file identity and
nanosecond change/modification times.
- No dependency lockfile or workflow changes.

## Model Used

OpenAI Codex, `gpt-6.1-sol`, max reasoning, with local shell and file
tools. Context window size is not exposed by this host.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed 2026-11-06 07:29:10 -06:00
1 parent 70d23612dd
commit 2f35eb1080
15 files changed
+907 -19

No files matched your search

+60 -2
View File
@@ -1054,7 +1054,63 @@ defines synchronization, permissions and evidence. After map edits, run
`node scripts/check-maps.mjs` for structural/reference checks; it does not start an
instance or qualify access as an agent.
Use the cheap local default unless you are specifically working on browser flows:
### Incremental local verification
Use the local verification entry point for ordinary issue work:
```sh
pnpm verify
pnpm verify --explain
pnpm verify --base origin/master
```
The command compares the actual working tree with the merge base of the supplied
local ref, including committed, staged, unstaged, deleted, renamed, and untracked
paths. The default ref is local `origin/master`, falling back to `master`; it does
not fetch. Tests read the current working-tree bytes. CI must still verify the
final committed revision. The task/input policy, conservative import and file-read
graph, exact Vitest specifications, and stable test runner are shared with CI.
Serialized server tests retain their owner. Native server tests remain in local
coverage. Selected chat tests retain six sequential isolated line shards.
Successful selected checks with complete input graphs are reusable in this
worktree. Opaque filesystem, subprocess, network, generated/untracked-module, and
unreviewed dependency consumers continue to run fresh. A selected pass never
satisfies a full run. Non-isolated projects and uncertain selection require broader
verification. The command prints the local test scope and remaining CI lanes.
An unchanged repeat can reuse both graph discovery and passing checks. Keys include
actual tracked and non-ignored file contents (including dirty and untracked files),
the shared audited-prose exclusions, dotenv files, lockfile, installation metadata,
dependency bindings and installed package contents, the complete effective child environment, Node/pnpm/Rust toolchains,
platform, checkout path, comparison, and selection. Values from the environment
are hashed, not written to results. External database/live-test settings and
out-of-checkout source symlinks disable reuse. Cache state lives under the ignored
`.turbo/local-verify/` directory and is not shared across worktrees or machines.
Installed package content digests are memoized against file identity, size, mode,
and nanosecond modification/change times; every check rescans the installation.
Generated Vite/cache directories are excluded. The first run after installation
reads package contents; unchanged repeats avoid reading those bytes again.
`last-run.json` retains the latest completed execution's status and scope. Failed/interrupted
attempts do not publish passes. Inputs changed during a run invalidate its result.
Exit 2 reports a broad input, incomplete graph, or unsupported selection. It does
not silently launch the full suite or claim affected coverage passed. Use focused
checks and required CI to close that gap, or explicitly request a broad local run:
```sh
pnpm verify --full
pnpm verify --no-cache
```
`--full` executes the complete existing Vitest path with fresh integration coverage.
`--no-cache` bypasses pass/plan reuse and does not save passes. A fresh failed or
interrupted attempt removes its old reusable pass. Required CI on the final tested
revision may supply broad typecheck/build/test coverage for a PR handoff; do not
repeat a full local suite solely because a chat resumed or a PR is ready. Keep
release checks and any real browser, provider, local-platform or acceptance proof.
The full local Vitest commands remain available:
```sh
pnpm test
@@ -1092,7 +1148,9 @@ the development server's unbundled module graph on each navigation. Browser
assertion deadlines and retries remain unchanged. Use `pnpm dev` separately when
verifying Vite/HMR behavior.
For normal issue work, start with the smallest targeted check that proves the change. Reserve repo-wide typecheck/build/test runs for PR-ready handoff or changes broad enough that narrow checks do not cover the risk.
For normal issue work, start with the smallest targeted check that proves the change.
Use the incremental command above and required final-revision CI for PR handoff.
Run broad local checks when an unresolved risk or explicit qualification needs them.
The Runner transport suite keeps attachment recovery, warm attachment, live
adoption, and suspension in separate test files. Vitest can schedule these