PaperClipAI

Jannik/PaperClipAI

0 Fork 0

mirror of https://github.com/paperclipai/paperclip.git synced 2026-10-31 06:04:11 +01:00

Commit Graph

Author	SHA1	Message	Date
DottaandPaperclip	52e846f2f3	perf(ci): select affected Vitest files before allocating test runners (#18348 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its CI must catch regressions without running unrelated work on every pull request. > - The existing task planner selects package lanes, but each selected lane still runs all its test files. > - Vitest can find related imports, but that graph misses tests that read files or launch processes. > - This pull request combines the real Vitest graph with conservative dependency checks before CI allocates test runners. > - The result runs fewer test files and can allocate fewer jobs while retaining tests with unknown dependencies. ## Linked Issues or Issue Description Refs #18214. The repository and PR search found no duplicate file-selection implementation. What existing behavior does this improve? The general and serialized Vitest lanes in pull request CI. Current behavior An affected package schedules its complete test lane and its fixed shard count. Proposed behavior For source-only diffs, select related tests plus conservative filesystem, process, network and opaque consumers. Repartition those files within their existing lane owners. Keep the complete original plan when selection cannot be trusted. Reason and benefit Avoid unrelated test execution and allocate fewer runners when the selected workload fits fewer existing job budgets. Report file counts and job counts separately. Breaking changes No product or API changes. The set of PR test files and jobs changes for eligible diffs. Full CI and the kill switch retain the original coverage. ## What Changed - Add a separate selection job on the gate-approved test fleet. The metadata gate does not load PR code. - Inspect real Vitest/Vite dependency edges. Retain opaque consumers and unchanged downstream projects. Respect only complete local mock factories. Reject unknown setup dependencies and source mutations during dependency setup. - Declare `oxc-parser` directly for AST inspection. This exact version is already a transitive dependency in the checked-in lockfile; the direct declaration avoids relying on pnpm's internal dependency layout. - Produce a merge-SHA-bound manifest with exact lane ownership and duration-aware partitions. Preserve serialized execution, chat shards and native Runner ownership. - Execute exact Vitest test specifications. Positional substring filters cannot accidentally run a similarly named file from another shard. - Use the original matrix if selection fails or times out. Fail selected jobs if their published artifact is missing or invalid. - Save the uploaded artifact ID in the executable plan. Failed-job retries reuse that artifact even when GitHub increments the attempt number. - Hash selected assignments into Turbo cache inputs. A subset cannot satisfy a full run or another subset. - Add graph, workflow, real-runner and real-Turbo qualification. The regression fixture proves a filesystem contract fails even when raw Vitest selection misses it. - Document selection limits, audit artifacts and `CI_VITEST_SELECTION=off`. ## Verification - All 138 affected routing, sharding, workflow, manifest, real Vitest and real Turbo checks passed under Node 24 on the final implementation. Additional release workflow checks passed. Cases cover opaque setup, remocking, generated inputs, Node's builtin loader, artifact retries, and exact execution of similarly named files. - Repository-wide `pnpm -r typecheck` and `pnpm build` passed before the clean rebase onto current master. The full remote typecheck and build are required on the current head. - `actionlint` passed for both changed workflows. `git diff --check` passed. - The Linux qualification run passed and selected 699 of 911 workspace test files for an Inbox change. UI graph construction took 16.1 seconds. This still uses two jobs. Synthetic partition coverage proves a 15-of-120 selection uses two of twelve jobs without dropping or duplicating files. - Both selected workspace shards passed locally. After tightening execution to exact specifications, the real colliding UI test ran as exactly one file with two passing assertions; the real server auth middleware test ran as exactly one file with 28 passing assertions. - The server probe selected 1,158 of 1,260 general files and 161 of 163 serialized files for an auth middleware change. Those conservative selections retain 12 general and nine serialized jobs. The server's temporary-home setup exception is pinned to reviewed content hashes; changed setup fails closed. - Full `pnpm test:run` has not been run locally. The full PR test matrix is required because this change modifies global CI inputs. Final-head CI is pending at runner allocation, and both qualification jobs are queued. Other recent repository workflows are also queued. Superseded runs of this PR were cancelled; this PR remains in draft until the required checks execute. - Greptile reviewed final commit `31b484fc5b412da245cffb52a8ac8977228c5651` at 5/5 with no unresolved findings. The earlier qualification run passed; it does not replace checks on the final commit. ## Risks - Dependency analysis adds setup and graph time. Fewer files do not guarantee a shorter critical path. CI summaries expose the actual file and job counts. - Static dependency graphs cannot describe all runtime inputs. Unknown inputs retain tests or use the original plan. The pure-package allowlist requires conservative review. - The trusted workflow runs from master. This PR exercises the new selector in the qualification workflow; ordinary PR selection starts after merge. Set `CI_VITEST_SELECTION=off`, use `full-ci`, or select full/shadow mode to disable narrowing. - The lockfile remains managed by the existing repository actions and is not changed here. ## Model Used OpenAI Codex, based on GPT-6, with repository tools and code execution. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 13:16:29 -05:00
DottaandPaperclip	99ec767a42	Fix forced CI execution and align OpenCode shell fixture (#18230 ) Use one explicit Turbo cache mode for forced CI execution and preserve cache trust boundaries. Match OpenCode’s ignored stdin in the relocated Linux shell-hook fixture while retaining strict OOM and parent-isolation assertions. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-29 18:19:11 -05:00
DottaandPaperclip	f5dcc505c0	perf(ci): select affected suites and cache task results (#18214 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Pull request CI checks the server, UI, packages, Runner, and release paths. > - An ordinary PR starts 48 jobs even when its diff affects only a small part of the repository. > - Repeated pushes also repeat successful tests with unchanged inputs. > - This PR selects affected suites before runner allocation and caches deterministic task results with Turbo. > - Required checks validate the plan. Unknown inputs and failed comparisons keep full verification. > - The benefit is fewer runner jobs and less repeated work, with full post-merge verification retained. ## Linked Issues or Issue Description What existing behavior does this improve? The GitHub Actions PR workflow repeats unrelated suites for each push. Subsystem affected Cross-cutting CI routing, test execution, and required-check aggregation. Current behavior An ordinary UI component change starts the complete 48-job PR workflow. Existing dependency and compiler caches do not reuse successful test results. Proposed behavior A trusted planner reads the tested merge diff and task inputs. A UI component change such as `ui/src/components/Sidebar.tsx` selects 18 jobs, including build and browser coverage. Successful general, serialized, and standalone Rust tasks can reuse unchanged results across pushes in the same PR. Reason and benefit Reduce unrelated jobs and repeated test execution. The 48-to-18 comparison measures planned job count. Remote wall-clock savings have not yet been measured. Breaking changes No application API changes. Required `verify` and `e2e` names remain stable. `verify` also checks serialized suites and the canary dry run. ## What Changed - Added explicit task inputs and a pinned Turbo version. An execution adapter calls existing package scripts without adding script aliases. - Added trusted diff planning and dynamic matrices with complete fallback matrices. - Added PR-scoped task result caches. External-state and revision-stamped checks remain uncached. - Added and created the `full-ci` override label, plus repository modes `enabled`, `shadow`, and `full`. Forced and shadow runs bypass result caches. - Added candidate-workflow qualification, planner and aggregate tests, and a real Turbo cache fixture. - Kept the repository-wide CLI guidance guard in the uncached typecheck lane so UI-only diffs retain this security check. - Documented rollout controls, input maintenance, and local inspection commands. ## Verification - Passed 537 focused CI policy, routing, matrix, cache, and shard tests, plus 146 existing release workflow tests. - Passed actionlint for all four changed or added workflows and `git diff --check`. - Passed `pnpm -r typecheck` and `pnpm build` on the implementation base. - Checked real repository task plans for existing general-test and native Rust commands. A real server partition passed, then restored from cache: Turbo task time was 20.022 seconds for execution and 151 ms for the cache hit, excluding wrapper/setup overhead. - The real Turbo fixture exercises the execution adapter and proves source, argument, toolchain-scope, and environment invalidation, result restoration, and retry of failed tasks. - Stopped the serial local `pnpm test:run` after over 30 minutes in favor of the complete parallel GitHub suite. A local parallel database-lane attempt had PostgreSQL startup errors and timeouts. The direct migration fixture passed, then all 273 database tests passed with one local worker. This recovery does not establish the exact cause of the earlier startup failures. The full GitHub PR suite and candidate qualification passed on commit `0bf07ae071c9fe7903edc604d44b5a175727fed0`. Greptile is 5/5 with no unresolved review threads. - All 40 CLI guidance safety tests passed with the exact uncached workflow command. - Verified that master rulesets require only `ci / verify` and `ci / e2e`; both names remain unchanged. - Trusted workflows load from master. The added GitHub-hosted qualification workflow exercises this candidate planner before merge. ## Risks - Changes to global inputs, including task definitions and execution helpers, force execution and disable cache restore/save for the full PR diff. Missing a cross-package filesystem read could omit a suite. Inputs include known server reads of UI files and shared tests of UI assets. Unknown paths, malformed or truncated diffs, and definition download failures select all suites. - Task result reuse requires complete inputs. Keys include Git source files, scripts, resolved dependencies, environment, toolchain, checkout path, and unclassified source files. Generated build artifacts are excluded from input hashes. Registry smoke, browser, and revision-stamped tasks remain uncached. - Maintainers must update task inputs when suites start reading new paths. `shadow` reports selection while running all work. `full` and `full-ci` disable selection and result reuse. - The implementation becomes active after merge. Full post-merge verification remains unchanged. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository inspection, tool use, and code execution. The session does not expose a verifiable exact runtime model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 16:56:28 -05:00

Author

SHA1

Message

Date

DottaandPaperclip

52e846f2f3

perf(ci): select affected Vitest files before allocating test runners (#18348 )

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Its CI must catch regressions without running unrelated work on
every pull request.
> - The existing task planner selects package lanes, but each selected
lane still runs all its test files.
> - Vitest can find related imports, but that graph misses tests that
read files or launch processes.
> - This pull request combines the real Vitest graph with conservative
dependency checks before CI allocates test runners.
> - The result runs fewer test files and can allocate fewer jobs while
retaining tests with unknown dependencies.

## Linked Issues or Issue Description

Refs #18214. The repository and PR search found no duplicate
file-selection implementation.

**What existing behavior does this improve?**

The general and serialized Vitest lanes in pull request CI.

**Current behavior**

An affected package schedules its complete test lane and its fixed shard
count.

**Proposed behavior**

For source-only diffs, select related tests plus conservative
filesystem, process, network and opaque consumers. Repartition those
files within their existing lane owners. Keep the complete original plan
when selection cannot be trusted.

**Reason and benefit**

Avoid unrelated test execution and allocate fewer runners when the
selected workload fits fewer existing job budgets. Report file counts
and job counts separately.

**Breaking changes**

No product or API changes. The set of PR test files and jobs changes for
eligible diffs. Full CI and the kill switch retain the original
coverage.

## What Changed

- Add a separate selection job on the gate-approved test fleet. The
metadata gate does not load PR code.
- Inspect real Vitest/Vite dependency edges. Retain opaque consumers and
unchanged downstream projects. Respect only complete local mock
factories. Reject unknown setup dependencies and source mutations during
dependency setup.
- Declare `oxc-parser` directly for AST inspection. This exact version
is already a transitive dependency in the checked-in lockfile; the
direct declaration avoids relying on pnpm's internal dependency layout.
- Produce a merge-SHA-bound manifest with exact lane ownership and
duration-aware partitions. Preserve serialized execution, chat shards
and native Runner ownership.
- Execute exact Vitest test specifications. Positional substring filters
cannot accidentally run a similarly named file from another shard.
- Use the original matrix if selection fails or times out. Fail selected
jobs if their published artifact is missing or invalid.
- Save the uploaded artifact ID in the executable plan. Failed-job
retries reuse that artifact even when GitHub increments the attempt
number.
- Hash selected assignments into Turbo cache inputs. A subset cannot
satisfy a full run or another subset.
- Add graph, workflow, real-runner and real-Turbo qualification. The
regression fixture proves a filesystem contract fails even when raw
Vitest selection misses it.
- Document selection limits, audit artifacts and
`CI_VITEST_SELECTION=off`.

## Verification

- All 138 affected routing, sharding, workflow, manifest, real Vitest
and real Turbo checks passed under Node 24 on the final implementation.
Additional release workflow checks passed. Cases cover opaque setup,
remocking, generated inputs, Node's builtin loader, artifact retries,
and exact execution of similarly named files.
- Repository-wide `pnpm -r typecheck` and `pnpm build` passed before the
clean rebase onto current master. The full remote typecheck and build
are required on the current head.
- `actionlint` passed for both changed workflows. `git diff --check`
passed.
- The Linux qualification run passed and selected 699 of 911 workspace
test files for an Inbox change. UI graph construction took 16.1 seconds.
This still uses two jobs. Synthetic partition coverage proves a
15-of-120 selection uses two of twelve jobs without dropping or
duplicating files.
- Both selected workspace shards passed locally. After tightening
execution to exact specifications, the real colliding UI test ran as
exactly one file with two passing assertions; the real server auth
middleware test ran as exactly one file with 28 passing assertions.
- The server probe selected 1,158 of 1,260 general files and 161 of 163
serialized files for an auth middleware change. Those conservative
selections retain 12 general and nine serialized jobs. The server's
temporary-home setup exception is pinned to reviewed content hashes;
changed setup fails closed.
- Full `pnpm test:run` has not been run locally. The full PR test matrix
is required because this change modifies global CI inputs. Final-head CI
is pending at runner allocation, and both qualification jobs are queued.
Other recent repository workflows are also queued. Superseded runs of
this PR were cancelled; this PR remains in draft until the required
checks execute.
- Greptile reviewed final commit
`31b484fc5b412da245cffb52a8ac8977228c5651` at 5/5 with no unresolved
findings. The earlier qualification run passed; it does not replace
checks on the final commit.

## Risks

- Dependency analysis adds setup and graph time. Fewer files do not
guarantee a shorter critical path. CI summaries expose the actual file
and job counts.
- Static dependency graphs cannot describe all runtime inputs. Unknown
inputs retain tests or use the original plan. The pure-package allowlist
requires conservative review.
- The trusted workflow runs from master. This PR exercises the new
selector in the qualification workflow; ordinary PR selection starts
after merge. Set `CI_VITEST_SELECTION=off`, use `full-ci`, or select
full/shadow mode to disable narrowing.
- The lockfile remains managed by the existing repository actions and is
not changed here.

## Model Used

OpenAI Codex, based on GPT-6, with repository tools and code execution.
The exact serving model ID and context window are not exposed in this
session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>

2026-10-30 13:16:29 -05:00

DottaandPaperclip

99ec767a42

Fix forced CI execution and align OpenCode shell fixture (#18230 )

Use one explicit Turbo cache mode for forced CI execution and preserve cache trust boundaries. Match OpenCode’s ignored stdin in the relocated Linux shell-hook fixture while retaining strict OOM and parent-isolation assertions.

Co-Authored-By: Paperclip <noreply@paperclip.ing>

2026-10-29 18:19:11 -05:00

DottaandPaperclip

f5dcc505c0

perf(ci): select affected suites and cache task results (#18214 )

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Pull request CI checks the server, UI, packages, Runner, and release
paths.
> - An ordinary PR starts 48 jobs even when its diff affects only a
small part of the repository.
> - Repeated pushes also repeat successful tests with unchanged inputs.
> - This PR selects affected suites before runner allocation and caches
deterministic task results with Turbo.
> - Required checks validate the plan. Unknown inputs and failed
comparisons keep full verification.
> - The benefit is fewer runner jobs and less repeated work, with full
post-merge verification retained.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The GitHub Actions PR workflow repeats unrelated suites for each push.

**Subsystem affected**

Cross-cutting CI routing, test execution, and required-check
aggregation.

**Current behavior**

An ordinary UI component change starts the complete 48-job PR workflow.
Existing dependency and compiler caches do not reuse successful test
results.

**Proposed behavior**

A trusted planner reads the tested merge diff and task inputs. A UI
component change such as `ui/src/components/Sidebar.tsx` selects 18
jobs, including build and browser coverage. Successful general,
serialized, and standalone Rust tasks can reuse unchanged results across
pushes in the same PR.

**Reason and benefit**

Reduce unrelated jobs and repeated test execution. The 48-to-18
comparison measures planned job count. Remote wall-clock savings have
not yet been measured.

**Breaking changes**

No application API changes. Required `verify` and `e2e` names remain
stable. `verify` also checks serialized suites and the canary dry run.

## What Changed

- Added explicit task inputs and a pinned Turbo version. An execution
adapter calls existing package scripts without adding script aliases.
- Added trusted diff planning and dynamic matrices with complete
fallback matrices.
- Added PR-scoped task result caches. External-state and
revision-stamped checks remain uncached.
- Added and created the `full-ci` override label, plus repository modes
`enabled`, `shadow`, and `full`. Forced and shadow runs bypass result
caches.
- Added candidate-workflow qualification, planner and aggregate tests,
and a real Turbo cache fixture.
- Kept the repository-wide CLI guidance guard in the uncached typecheck
lane so UI-only diffs retain this security check.
- Documented rollout controls, input maintenance, and local inspection
commands.

## Verification

- Passed 537 focused CI policy, routing, matrix, cache, and shard tests,
plus 146 existing release workflow tests.
- Passed actionlint for all four changed or added workflows and `git
diff --check`.
- Passed `pnpm -r typecheck` and `pnpm build` on the implementation
base.
- Checked real repository task plans for existing general-test and
native Rust commands. A real server partition passed, then restored from
cache: Turbo task time was 20.022 seconds for execution and 151 ms for
the cache hit, excluding wrapper/setup overhead.
- The real Turbo fixture exercises the execution adapter and proves
source, argument, toolchain-scope, and environment invalidation, result
restoration, and retry of failed tasks.
- Stopped the serial local `pnpm test:run` after over 30 minutes in
favor of the complete parallel GitHub suite. A local parallel
database-lane attempt had PostgreSQL startup errors and timeouts. The
direct migration fixture passed, then all 273 database tests passed with
one local worker. This recovery does not establish the exact cause of
the earlier startup failures. The full GitHub PR suite and candidate
qualification passed on commit
`0bf07ae071c9fe7903edc604d44b5a175727fed0`. Greptile is 5/5 with no
unresolved review threads.
- All 40 CLI guidance safety tests passed with the exact uncached
workflow command.
- Verified that master rulesets require only `ci / verify` and `ci /
e2e`; both names remain unchanged.
- Trusted workflows load from master. The added GitHub-hosted
qualification workflow exercises this candidate planner before merge.

## Risks

- Changes to global inputs, including task definitions and execution
helpers, force execution and disable cache restore/save for the full PR
diff. Missing a cross-package filesystem read could omit a suite. Inputs
include known server reads of UI files and shared tests of UI assets.
Unknown paths, malformed or truncated diffs, and definition download
failures select all suites.
- Task result reuse requires complete inputs. Keys include Git source
files, scripts, resolved dependencies, environment, toolchain, checkout
path, and unclassified source files. Generated build artifacts are
excluded from input hashes. Registry smoke, browser, and
revision-stamped tasks remain uncached.
- Maintainers must update task inputs when suites start reading new
paths. `shadow` reports selection while running all work. `full` and
`full-ci` disable selection and result reuse.
- The implementation becomes active after merge. Full post-merge
verification remains unchanged.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, repository inspection, tool
use, and code execution. The session does not expose a verifiable exact
runtime model ID or context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>

2026-10-29 16:56:28 -05:00