mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-09 07:10:34 +01:00
feat(dev): add incremental local verification and pass reuse (#19345)
## Thinking Path > - Paperclip coordinates agents that make and verify repository changes. > - Local verification currently defaults to the complete Vitest suite. > - Contributor instructions also require broad local checks at each PR handoff. > - CI already owns conservative affected-test selection and broad final-revision checks. > - This pull request adds local selection and pass reuse from those existing contracts. > - Agents can get focused feedback and use CI to close the remaining coverage gaps. ## Linked Issues or Issue Description **Subsystem affected** Developer verification scripts and contributor guidance. Related selection work: #18398 and #18464. **Problem or motivation** An agent that repeats an unchanged local verification step starts the complete suite again. The handoff guidance also duplicates broad CI coverage locally. **Proposed solution** Add `pnpm verify` for the actual working tree. Select affected Vitest specifications with the CI task policy and dependency graph. Reuse successful checks whose inputs are modeled. Run opaque checks fresh. Report exit 2 when broader verification is required. Update the contributor contract to accept applicable final-revision CI coverage. **Alternatives considered** Plain `vitest related` omits filesystem and opaque dependencies. Commit-only cache keys miss dirty files. Running full local suites at every handoff repeats CI work. **Roadmap alignment** This is repository development tooling. It does not change the product roadmap. ## What Changed - Add `pnpm verify`, `--explain`, `--base`, `--full`, and `--no-cache`. - Reuse CI selection and exact runner ownership for dirty, staged, committed, deleted, renamed, and untracked inputs. - Keep native server suites and serialized suites in their local owners. Keep six sequential chat shards. - Cache selected passes within one worktree. Hash source, fixtures, installed package contents, dependency bindings and installation metadata, dotenv files, child environment, toolchains, platform, and selection. - Run opaque consumers fresh. Preserve failed, interrupted, mutated-input, and partial-coverage outcomes. - Update AGENTS.md, CONTRIBUTING.md, and DEVELOPING.md guidance. Record the shared `verify` command as the sole exception to the package-script rule. Register the new tests in the existing CI contract-test command. - Add optional root/environment inputs to shared graph helpers. Existing CI defaults stay the same. ## Verification - Fourteen new integration tests pass locally across focused runs with real Vitest selection and execution; CI colors are covered. - 51 existing targeted selector/input, project-isolation, and stable-runner checks passed. Four relevant runner contracts also pass after the workspace change. - Repeated unchanged execution skips reusable tests. Source, environment, installed code, dependency bindings, and new untracked tests invalidate results. - Installed-code mutation with a restored modification time still invalidates a cached pass. Internal skill-directory links preserve reuse. - Workspace B/C tests prove discovery, reusable/fresh partition execution, pass reuse, and external-input failure. Chat selection produces exactly six single-worker partitions. - A deliberate unrelated failing test is omitted by narrow selection and caught by `--full`. - Red/green regressions prove fingerprints preserve file boundaries and reject edits made between diff discovery and snapshotting. - Opaque external-file changes still fail on a subsequent run. Source mutation and process interruption cannot publish reusable passes. - `pnpm verify --explain` works in the checkout and requests broad CI coverage for this tooling change. No duplicate full local suite was started. - [Final-revision CI](https://github.com/paperclipai/paperclip/actions/runs/42117291789) passed on `ed85d96ddff82c05c325948680b593b0f48c473d` (attempt 2). The scoped Canary retry preserved all passing test/build results. - [Greptile review](https://github.com/paperclipai/paperclip/pull/19345#issuecomment-6670050175) is 5/5; all four review threads are resolved. - The additional Paperclip review concluded neutral because its execution stopped; it produced no assessment. ## Risks - Pass reuse depends on conservative dependency classification. Opaque consumers, external database/live settings, and non-isolated projects retain fresh or broad coverage. - Local passes cover selected Vitest checks. Required CI, build/typecheck, release, browser, and platform acceptance remain separate obligations. - Cross-worktree caching and cross-agent scheduling are outside this change. - The initial installation scan reads dependency contents; stat-validated, worktree-local digest memoization avoids rereading unchanged contents. Subsequent scans still inspect file identity and nanosecond change/modification times. - No dependency lockfile or workflow changes. ## Model Used OpenAI Codex, `gpt-6.1-sol`, max reasoning, with local shell and file tools. Context window size is not exposed by this host. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
70d23612dd
commit
2f35eb1080
15 files changed
+907
-19
No files matched your search
+60
-2
@@ -1054,7 +1054,63 @@ defines synchronization, permissions and evidence. After map edits, run
|
||||
`node scripts/check-maps.mjs` for structural/reference checks; it does not start an
|
||||
instance or qualify access as an agent.
|
||||
|
||||
Use the cheap local default unless you are specifically working on browser flows:
|
||||
### Incremental local verification
|
||||
|
||||
Use the local verification entry point for ordinary issue work:
|
||||
|
||||
```sh
|
||||
pnpm verify
|
||||
pnpm verify --explain
|
||||
pnpm verify --base origin/master
|
||||
```
|
||||
|
||||
The command compares the actual working tree with the merge base of the supplied
|
||||
local ref, including committed, staged, unstaged, deleted, renamed, and untracked
|
||||
paths. The default ref is local `origin/master`, falling back to `master`; it does
|
||||
not fetch. Tests read the current working-tree bytes. CI must still verify the
|
||||
final committed revision. The task/input policy, conservative import and file-read
|
||||
graph, exact Vitest specifications, and stable test runner are shared with CI.
|
||||
Serialized server tests retain their owner. Native server tests remain in local
|
||||
coverage. Selected chat tests retain six sequential isolated line shards.
|
||||
|
||||
Successful selected checks with complete input graphs are reusable in this
|
||||
worktree. Opaque filesystem, subprocess, network, generated/untracked-module, and
|
||||
unreviewed dependency consumers continue to run fresh. A selected pass never
|
||||
satisfies a full run. Non-isolated projects and uncertain selection require broader
|
||||
verification. The command prints the local test scope and remaining CI lanes.
|
||||
|
||||
An unchanged repeat can reuse both graph discovery and passing checks. Keys include
|
||||
actual tracked and non-ignored file contents (including dirty and untracked files),
|
||||
the shared audited-prose exclusions, dotenv files, lockfile, installation metadata,
|
||||
dependency bindings and installed package contents, the complete effective child environment, Node/pnpm/Rust toolchains,
|
||||
platform, checkout path, comparison, and selection. Values from the environment
|
||||
are hashed, not written to results. External database/live-test settings and
|
||||
out-of-checkout source symlinks disable reuse. Cache state lives under the ignored
|
||||
`.turbo/local-verify/` directory and is not shared across worktrees or machines.
|
||||
Installed package content digests are memoized against file identity, size, mode,
|
||||
and nanosecond modification/change times; every check rescans the installation.
|
||||
Generated Vite/cache directories are excluded. The first run after installation
|
||||
reads package contents; unchanged repeats avoid reading those bytes again.
|
||||
`last-run.json` retains the latest completed execution's status and scope. Failed/interrupted
|
||||
attempts do not publish passes. Inputs changed during a run invalidate its result.
|
||||
|
||||
Exit 2 reports a broad input, incomplete graph, or unsupported selection. It does
|
||||
not silently launch the full suite or claim affected coverage passed. Use focused
|
||||
checks and required CI to close that gap, or explicitly request a broad local run:
|
||||
|
||||
```sh
|
||||
pnpm verify --full
|
||||
pnpm verify --no-cache
|
||||
```
|
||||
|
||||
`--full` executes the complete existing Vitest path with fresh integration coverage.
|
||||
`--no-cache` bypasses pass/plan reuse and does not save passes. A fresh failed or
|
||||
interrupted attempt removes its old reusable pass. Required CI on the final tested
|
||||
revision may supply broad typecheck/build/test coverage for a PR handoff; do not
|
||||
repeat a full local suite solely because a chat resumed or a PR is ready. Keep
|
||||
release checks and any real browser, provider, local-platform or acceptance proof.
|
||||
|
||||
The full local Vitest commands remain available:
|
||||
|
||||
```sh
|
||||
pnpm test
|
||||
@@ -1092,7 +1148,9 @@ the development server's unbundled module graph on each navigation. Browser
|
||||
assertion deadlines and retries remain unchanged. Use `pnpm dev` separately when
|
||||
verifying Vite/HMR behavior.
|
||||
|
||||
For normal issue work, start with the smallest targeted check that proves the change. Reserve repo-wide typecheck/build/test runs for PR-ready handoff or changes broad enough that narrow checks do not cover the risk.
|
||||
For normal issue work, start with the smallest targeted check that proves the change.
|
||||
Use the incremental command above and required final-revision CI for PR handoff.
|
||||
Run broad local checks when an unresolved risk or explicit qualification needs them.
|
||||
|
||||
The Runner transport suite keeps attachment recovery, warm attachment, live
|
||||
adoption, and suspension in separate test files. Vitest can schedule these
|
||||
|
||||
Reference in new issue
Block a user