PaperClipAI

Jannik/PaperClipAI

0 Fork 0

mirror of https://github.com/paperclipai/paperclip.git synced 2026-10-31 14:16:39 +01:00

Commit Graph

Author	SHA1	Message	Date
DottaandPaperclip	3b84c89885	fix(ci): use complete merge diffs and allow audited documentation (#18362 ) ## Thinking Path > - Paperclip manages AI agents and needs reliable checks for each contribution. > - CI selects affected suites and then affected Vitest files before allocating runners. > - Recent runs compared a tested merge against a stale event base. > - That included unrelated master changes and sometimes reached the compare API file cap. > - Ordinary documentation also forced every suite and invalidated test caches. > - This PR reads complete merge trees and exempts a small set of audited prose inputs. > - Tests retain conservative coverage when the diff or an input is uncertain. ## Linked Issues or Issue Description Refs #18348 and #18214. This is a follow-up to the merged affected-test selector. Related CI performance work: #16513 and #16500 address setup and suite execution. This PR changes diff collection and input classification. ## What Changed - Fetch the exact tested merge and its parents into a temporary bare repository. Read the complete tree diff against the actual first parent. Validate the second parent against the event PR head. Never execute PR code in the routing job. - Carry the immutable base, merge and PR-head identities into the plan. Validate the same complete diff again before file selection. - Use one prose policy across the planner, Vitest selector and task cache adapter. Exempt only `doc/DEVELOPING.md`, `doc/DATABASE.md`, and Markdown under `doc/plans/`. - Keep the uncached typecheck and CLI guidance guard for the first two documents. Keep policy checks for historical plans. Keep existing coverage for all other docs, runtime skills, fixtures and unknown inputs. - Extend candidate-workflow qualification to exercise mixed source and prose changes. ## Verification - Passed: 15 planner and Git routing tests, including a 353-file diff, stale base, rename, deletion, wrong parent and fetch-failure cases. - Passed: 83 routing, Vitest graph, exact selection, workflow and real Turbo cache checks. - Passed: `pnpm -r typecheck` and `pnpm build`. - Full local `pnpm test:run` was attempted and stopped after repeated embedded PostgreSQL startup failures in native-session and Muse tests. It did not pass or complete. The exact cause remains unverified. The changed files do not include those tests or their database setup. Linux CI results are recorded below. - Passed: real mixed Inbox/source/prose graph qualification selected 699/911 workspace test files, preserving the source-only selection. - Passed: both GitHub CI qualification jobs, including real mixed source/prose Vitest selection. Greptile reviewed commit `10ac6f5ec9` at 5/5 with no findings. - First full CI attempt: the [chat shard](https://github.com/paperclipai/paperclip/actions/runs/41111003419/job/124253342684) received a runner shutdown signal. The [Runner shard](https://github.com/paperclipai/paperclip/actions/runs/41111003419/job/124253341271) failed the unchanged Antigravity authentication success assertion. All 11 cases in that file passed when run directly. That Runner lane does not use the changed selector/cache path. The exact assertion-failure cause remains unverified. The chat shard passed on the one retry. The authentication assertion also passed on retry, but the same Runner shard failed a different unchanged test: `native-distribution-integrity.test.ts:242`, "retains spawned native bytes until the exact child exits", exceeded 5000ms. The [retry log](https://github.com/paperclipai/paperclip/actions/runs/41111003419/job/124259174201) preserves the failure. The Runner shard and required verify aggregate are still red; all other checks passed. All 33 native-distribution tests then passed in a direct local run. This shows recovery in isolation, not the cause of the Linux timeout. No further blind reruns were issued. - Replayed 15 real recent PR merge commits through a blob-filtered bare fetch. Nine needed a corrected base. The four API comparisons that returned 300 filenames became complete diffs of 3, 57, 66 and 54 files. The 3-file case (#18326) becomes eligible for graph selection. Most other samples still correctly use full CI because they include inputs outside this narrow prose policy. - Cache proof: prose edits preserve a product-test cache hit; changing runtime API documentation invalidates it. ## Risks - Prose exceptions must be removed or revised if product code starts reading those files. The allowlist is deliberately small. Runtime docs and skills remain covered. - A network or Git failure keeps the full-suite fallback. Unexpected merge topology also keeps full CI. Stack and force-full policies remain in place. - This does not claim an end-to-end speedup yet. Many mixed-content PRs still require full CI. The routing fetch reads only shallow commit/tree objects; it does not check out or execute PR files. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools and code execution. The exact served model ID and context-window size are not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 14:29:39 -05:00
DottaandPaperclip	52e846f2f3	perf(ci): select affected Vitest files before allocating test runners (#18348 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its CI must catch regressions without running unrelated work on every pull request. > - The existing task planner selects package lanes, but each selected lane still runs all its test files. > - Vitest can find related imports, but that graph misses tests that read files or launch processes. > - This pull request combines the real Vitest graph with conservative dependency checks before CI allocates test runners. > - The result runs fewer test files and can allocate fewer jobs while retaining tests with unknown dependencies. ## Linked Issues or Issue Description Refs #18214. The repository and PR search found no duplicate file-selection implementation. What existing behavior does this improve? The general and serialized Vitest lanes in pull request CI. Current behavior An affected package schedules its complete test lane and its fixed shard count. Proposed behavior For source-only diffs, select related tests plus conservative filesystem, process, network and opaque consumers. Repartition those files within their existing lane owners. Keep the complete original plan when selection cannot be trusted. Reason and benefit Avoid unrelated test execution and allocate fewer runners when the selected workload fits fewer existing job budgets. Report file counts and job counts separately. Breaking changes No product or API changes. The set of PR test files and jobs changes for eligible diffs. Full CI and the kill switch retain the original coverage. ## What Changed - Add a separate selection job on the gate-approved test fleet. The metadata gate does not load PR code. - Inspect real Vitest/Vite dependency edges. Retain opaque consumers and unchanged downstream projects. Respect only complete local mock factories. Reject unknown setup dependencies and source mutations during dependency setup. - Declare `oxc-parser` directly for AST inspection. This exact version is already a transitive dependency in the checked-in lockfile; the direct declaration avoids relying on pnpm's internal dependency layout. - Produce a merge-SHA-bound manifest with exact lane ownership and duration-aware partitions. Preserve serialized execution, chat shards and native Runner ownership. - Execute exact Vitest test specifications. Positional substring filters cannot accidentally run a similarly named file from another shard. - Use the original matrix if selection fails or times out. Fail selected jobs if their published artifact is missing or invalid. - Save the uploaded artifact ID in the executable plan. Failed-job retries reuse that artifact even when GitHub increments the attempt number. - Hash selected assignments into Turbo cache inputs. A subset cannot satisfy a full run or another subset. - Add graph, workflow, real-runner and real-Turbo qualification. The regression fixture proves a filesystem contract fails even when raw Vitest selection misses it. - Document selection limits, audit artifacts and `CI_VITEST_SELECTION=off`. ## Verification - All 138 affected routing, sharding, workflow, manifest, real Vitest and real Turbo checks passed under Node 24 on the final implementation. Additional release workflow checks passed. Cases cover opaque setup, remocking, generated inputs, Node's builtin loader, artifact retries, and exact execution of similarly named files. - Repository-wide `pnpm -r typecheck` and `pnpm build` passed before the clean rebase onto current master. The full remote typecheck and build are required on the current head. - `actionlint` passed for both changed workflows. `git diff --check` passed. - The Linux qualification run passed and selected 699 of 911 workspace test files for an Inbox change. UI graph construction took 16.1 seconds. This still uses two jobs. Synthetic partition coverage proves a 15-of-120 selection uses two of twelve jobs without dropping or duplicating files. - Both selected workspace shards passed locally. After tightening execution to exact specifications, the real colliding UI test ran as exactly one file with two passing assertions; the real server auth middleware test ran as exactly one file with 28 passing assertions. - The server probe selected 1,158 of 1,260 general files and 161 of 163 serialized files for an auth middleware change. Those conservative selections retain 12 general and nine serialized jobs. The server's temporary-home setup exception is pinned to reviewed content hashes; changed setup fails closed. - Full `pnpm test:run` has not been run locally. The full PR test matrix is required because this change modifies global CI inputs. Final-head CI is pending at runner allocation, and both qualification jobs are queued. Other recent repository workflows are also queued. Superseded runs of this PR were cancelled; this PR remains in draft until the required checks execute. - Greptile reviewed final commit `31b484fc5b412da245cffb52a8ac8977228c5651` at 5/5 with no unresolved findings. The earlier qualification run passed; it does not replace checks on the final commit. ## Risks - Dependency analysis adds setup and graph time. Fewer files do not guarantee a shorter critical path. CI summaries expose the actual file and job counts. - Static dependency graphs cannot describe all runtime inputs. Unknown inputs retain tests or use the original plan. The pure-package allowlist requires conservative review. - The trusted workflow runs from master. This PR exercises the new selector in the qualification workflow; ordinary PR selection starts after merge. Set `CI_VITEST_SELECTION=off`, use `full-ci`, or select full/shadow mode to disable narrowing. - The lockfile remains managed by the existing repository actions and is not changed here. ## Model Used OpenAI Codex, based on GPT-6, with repository tools and code execution. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 13:16:29 -05:00

Author

SHA1

Message

Date

DottaandPaperclip

3b84c89885

fix(ci): use complete merge diffs and allow audited documentation (#18362 )

## Thinking Path

> - Paperclip manages AI agents and needs reliable checks for each
contribution.
> - CI selects affected suites and then affected Vitest files before
allocating runners.
> - Recent runs compared a tested merge against a stale event base.
> - That included unrelated master changes and sometimes reached the
compare API file cap.
> - Ordinary documentation also forced every suite and invalidated test
caches.
> - This PR reads complete merge trees and exempts a small set of
audited prose inputs.
> - Tests retain conservative coverage when the diff or an input is
uncertain.

## Linked Issues or Issue Description

Refs #18348 and #18214. This is a follow-up to the merged affected-test
selector.
Related CI performance work: #16513 and #16500 address setup and suite
execution. This PR changes diff collection and input classification.

## What Changed

- Fetch the exact tested merge and its parents into a temporary bare
repository. Read the complete tree diff against the actual first parent.
Validate the second parent against the event PR head. Never execute PR
code in the routing job.
- Carry the immutable base, merge and PR-head identities into the plan.
Validate the same complete diff again before file selection.
- Use one prose policy across the planner, Vitest selector and task
cache adapter. Exempt only `doc/DEVELOPING.md`, `doc/DATABASE.md`, and
Markdown under `doc/plans/`.
- Keep the uncached typecheck and CLI guidance guard for the first two
documents. Keep policy checks for historical plans. Keep existing
coverage for all other docs, runtime skills, fixtures and unknown
inputs.
- Extend candidate-workflow qualification to exercise mixed source and
prose changes.

## Verification

- Passed: 15 planner and Git routing tests, including a 353-file diff,
stale base, rename, deletion, wrong parent and fetch-failure cases.
- Passed: 83 routing, Vitest graph, exact selection, workflow and real
Turbo cache checks.
- Passed: `pnpm -r typecheck` and `pnpm build`.
- Full local `pnpm test:run` was attempted and stopped after repeated
embedded PostgreSQL startup failures in native-session and Muse tests.
It did not pass or complete. The exact cause remains unverified. The
changed files do not include those tests or their database setup. Linux
CI results are recorded below.
- Passed: real mixed Inbox/source/prose graph qualification selected
699/911 workspace test files, preserving the source-only selection.
- Passed: both GitHub CI qualification jobs, including real mixed
source/prose Vitest selection. Greptile reviewed commit `10ac6f5ec9` at
5/5 with no findings.
- First full CI attempt: the [chat
shard](https://github.com/paperclipai/paperclip/actions/runs/41111003419/job/124253342684)
received a runner shutdown signal. The [Runner
shard](https://github.com/paperclipai/paperclip/actions/runs/41111003419/job/124253341271)
failed the unchanged Antigravity authentication success assertion. All
11 cases in that file passed when run directly. That Runner lane does
not use the changed selector/cache path. The exact assertion-failure
cause remains unverified. The chat shard passed on the one retry. The
authentication assertion also passed on retry, but the same Runner shard
failed a different unchanged test:
`native-distribution-integrity.test.ts:242`, "retains spawned native
bytes until the exact child exits", exceeded 5000ms. The [retry
log](https://github.com/paperclipai/paperclip/actions/runs/41111003419/job/124259174201)
preserves the failure. The Runner shard and required verify aggregate
are still red; all other checks passed. All 33 native-distribution tests
then passed in a direct local run. This shows recovery in isolation, not
the cause of the Linux timeout. No further blind reruns were issued.
- Replayed 15 real recent PR merge commits through a blob-filtered bare
fetch. Nine needed a corrected base. The four API comparisons that
returned 300 filenames became complete diffs of 3, 57, 66 and 54 files.
The 3-file case (#18326) becomes eligible for graph selection. Most
other samples still correctly use full CI because they include inputs
outside this narrow prose policy.
- Cache proof: prose edits preserve a product-test cache hit; changing
runtime API documentation invalidates it.

## Risks

- Prose exceptions must be removed or revised if product code starts
reading those files. The allowlist is deliberately small. Runtime docs
and skills remain covered.
- A network or Git failure keeps the full-suite fallback. Unexpected
merge topology also keeps full CI. Stack and force-full policies remain
in place.
- This does not claim an end-to-end speedup yet. Many mixed-content PRs
still require full CI. The routing fetch reads only shallow commit/tree
objects; it does not check out or execute PR files.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, repository tools and code
execution. The exact served model ID and context-window size are not
exposed to this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>

2026-10-30 14:29:39 -05:00

DottaandPaperclip

52e846f2f3

perf(ci): select affected Vitest files before allocating test runners (#18348 )

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Its CI must catch regressions without running unrelated work on
every pull request.
> - The existing task planner selects package lanes, but each selected
lane still runs all its test files.
> - Vitest can find related imports, but that graph misses tests that
read files or launch processes.
> - This pull request combines the real Vitest graph with conservative
dependency checks before CI allocates test runners.
> - The result runs fewer test files and can allocate fewer jobs while
retaining tests with unknown dependencies.

## Linked Issues or Issue Description

Refs #18214. The repository and PR search found no duplicate
file-selection implementation.

**What existing behavior does this improve?**

The general and serialized Vitest lanes in pull request CI.

**Current behavior**

An affected package schedules its complete test lane and its fixed shard
count.

**Proposed behavior**

For source-only diffs, select related tests plus conservative
filesystem, process, network and opaque consumers. Repartition those
files within their existing lane owners. Keep the complete original plan
when selection cannot be trusted.

**Reason and benefit**

Avoid unrelated test execution and allocate fewer runners when the
selected workload fits fewer existing job budgets. Report file counts
and job counts separately.

**Breaking changes**

No product or API changes. The set of PR test files and jobs changes for
eligible diffs. Full CI and the kill switch retain the original
coverage.

## What Changed

- Add a separate selection job on the gate-approved test fleet. The
metadata gate does not load PR code.
- Inspect real Vitest/Vite dependency edges. Retain opaque consumers and
unchanged downstream projects. Respect only complete local mock
factories. Reject unknown setup dependencies and source mutations during
dependency setup.
- Declare `oxc-parser` directly for AST inspection. This exact version
is already a transitive dependency in the checked-in lockfile; the
direct declaration avoids relying on pnpm's internal dependency layout.
- Produce a merge-SHA-bound manifest with exact lane ownership and
duration-aware partitions. Preserve serialized execution, chat shards
and native Runner ownership.
- Execute exact Vitest test specifications. Positional substring filters
cannot accidentally run a similarly named file from another shard.
- Use the original matrix if selection fails or times out. Fail selected
jobs if their published artifact is missing or invalid.
- Save the uploaded artifact ID in the executable plan. Failed-job
retries reuse that artifact even when GitHub increments the attempt
number.
- Hash selected assignments into Turbo cache inputs. A subset cannot
satisfy a full run or another subset.
- Add graph, workflow, real-runner and real-Turbo qualification. The
regression fixture proves a filesystem contract fails even when raw
Vitest selection misses it.
- Document selection limits, audit artifacts and
`CI_VITEST_SELECTION=off`.

## Verification

- All 138 affected routing, sharding, workflow, manifest, real Vitest
and real Turbo checks passed under Node 24 on the final implementation.
Additional release workflow checks passed. Cases cover opaque setup,
remocking, generated inputs, Node's builtin loader, artifact retries,
and exact execution of similarly named files.
- Repository-wide `pnpm -r typecheck` and `pnpm build` passed before the
clean rebase onto current master. The full remote typecheck and build
are required on the current head.
- `actionlint` passed for both changed workflows. `git diff --check`
passed.
- The Linux qualification run passed and selected 699 of 911 workspace
test files for an Inbox change. UI graph construction took 16.1 seconds.
This still uses two jobs. Synthetic partition coverage proves a
15-of-120 selection uses two of twelve jobs without dropping or
duplicating files.
- Both selected workspace shards passed locally. After tightening
execution to exact specifications, the real colliding UI test ran as
exactly one file with two passing assertions; the real server auth
middleware test ran as exactly one file with 28 passing assertions.
- The server probe selected 1,158 of 1,260 general files and 161 of 163
serialized files for an auth middleware change. Those conservative
selections retain 12 general and nine serialized jobs. The server's
temporary-home setup exception is pinned to reviewed content hashes;
changed setup fails closed.
- Full `pnpm test:run` has not been run locally. The full PR test matrix
is required because this change modifies global CI inputs. Final-head CI
is pending at runner allocation, and both qualification jobs are queued.
Other recent repository workflows are also queued. Superseded runs of
this PR were cancelled; this PR remains in draft until the required
checks execute.
- Greptile reviewed final commit
`31b484fc5b412da245cffb52a8ac8977228c5651` at 5/5 with no unresolved
findings. The earlier qualification run passed; it does not replace
checks on the final commit.

## Risks

- Dependency analysis adds setup and graph time. Fewer files do not
guarantee a shorter critical path. CI summaries expose the actual file
and job counts.
- Static dependency graphs cannot describe all runtime inputs. Unknown
inputs retain tests or use the original plan. The pure-package allowlist
requires conservative review.
- The trusted workflow runs from master. This PR exercises the new
selector in the qualification workflow; ordinary PR selection starts
after merge. Set `CI_VITEST_SELECTION=off`, use `full-ci`, or select
full/shadow mode to disable narrowing.
- The lockfile remains managed by the existing repository actions and is
not changed here.

## Model Used

OpenAI Codex, based on GPT-6, with repository tools and code execution.
The exact serving model ID and context window are not exposed in this
session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>

2026-10-30 13:16:29 -05:00