mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-21 08:22:20 +02:00
d9bcf4dfe8e4f6c46bb0c3cd2ff0e56bb73e0cfe
3
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
9077748894 |
Remove the Pipelines experiment and preserve its data (#16834)
## Thinking Path > - Paperclip manages AI agents and their work. > - Pipelines was an unused experiment with its own UI, API, CLI, and automation. > - Keeping it leaves unsupported product surfaces and extra task-detail queries. > - Removing the experiment must preserve data that users already saved. > - This pull request removes the feature and keeps its database schema and migration history. > - Cases and Routines remain available as separate features. ## Linked Issues or Issue Description **What existing behavior does this improve?** Remove the unused Pipelines experiment across the application. **Current behavior** The experiment includes pipeline boards, stage settings, case automation, review queues, learnings, permissions, and CLI commands. Task detail also queries pipeline links. **Proposed behavior** Remove all these product surfaces and their implementation. Preserve existing tables, records, and documents. Ignore the retired boolean in older managed configurations so upgrades still start. **Breaking changes** Pipeline URLs, API endpoints, CLI commands, and `pipelines:write` are removed. Task detail no longer includes the old `linkedCases` field. The independent Cases API remains available. No database migration or cleanup is included. Related feature work: #7907, #9110, #10905, #11375, #13004, #14671, #16133. Related issues: #9663, #10310, #10312. This removal supersedes the feature; it does not implement those pending enhancements or delete their saved data. ## What Changed - Remove Pipeline UI pages, navigation, gates, clients, query keys, helpers, and unused design tokens. - Remove Pipeline REST routes, services, automation dispatch, task-detail queries, and OpenAPI discovery. - Remove the CLI commands, shared contracts, experimental flag, catalog entry, and permission defaults. - Remove feature-specific tests, tutorials, smoke scripts, maps, and Storybook examples. - Preserve database schema, migration history, and the reserved document key used by old records. - Keep Cases routes independent and cover obsolete stored and managed settings. ## Verification - Passed: `pnpm build` and `pnpm -r typecheck`. - Passed: targeted settings, managed configuration, sidebar, settings UI, routing, and feature-catalog tests. A first sidebar run caught a shared Workspaces icon import; the import was restored and the sidebar rerun passed. - Passed: 40 tests covering Cases routes, OpenAPI discovery, retained Pipeline table constraints, and schema snapshot drift. - Passed: token gates, map validation, generated API metadata check, Storybook catalog validation, and 28 map/catalog tests. - Passed: isolated local browser verification of navigation and Experimental settings with Pipelines absent; Cases and Routines still open normally. Live GET/POST requests to retired Pipeline endpoints return 404, and the retired setting is absent. CLI help has no Pipeline commands. - Passed: 22 access-service and agent-default-grant tests after restoring unrelated permission assertions flagged in review. - Local full run: `pnpm test:run` reported one failure in the unchanged workspace-runtime port-allocation test. That test passed in an isolated rerun. The long local run was stopped after full CI passed; it is not reported as a complete local pass. - Passed: all 56 checks on `def9181f3a4491d95fca1d38e9739f5b821aae3e` (2 conditional checks skipped), including build, typecheck, test shards, and browser suites. Eleven interrupted CI jobs reported “The operation was canceled”; their one-time retry passed. [CI run](https://github.com/paperclipai/paperclip/actions/runs/39255211006). - Greptile: 5/5 on the final commit, with the only review finding fixed and resolved. - Sandbox socket restrictions affected the first build/API test attempts. The same commands ran with local sockets available. ## Risks - Existing Pipeline clients must stop using the removed endpoints and commands. - Existing data remains in the database and backups, but the removed UI no longer displays it. - Historical boolean `enablePipelines` values are ignored. They cannot restore the feature. - Cases and Routines must retain their separate behavior. Targeted route tests cover this boundary. ## Model Used OpenAI Codex, GPT-6, with repository inspection, editing, shell execution, and browser verification. The exact deployed model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
8df3ee2cf5 |
ci: rebalance serialized tests with current suite durations (#13328)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Cloud deployments wait for verified source commits. > - Verification splits serialized server tests across independent runners. > - The shard duration estimates came from August and no longer match current tests. > - Stale estimates put much more work on one runner than the others. > - This PR refreshes the estimates from a complete successful run to balance the existing runners. ## Linked Issues or Issue Description **What existing behavior does this improve?** The time spent waiting for the slowest serialized server-test shard in PR and release verification. **Current behavior** In [Cloud readiness run 34705914878](https://github.com/paperclipai/paperclip/actions/runs/34705914878), the five serialized shards spent 479, 371, 322, 390, and 365 seconds running tests. The recovery suite had a 55-second estimate but now takes about 156 seconds including process overhead. **Proposed behavior** Use fresh per-suite measurements with the existing deterministic duration balancer. Applying the same measured costs to the new assignment gives 385, 385, 386, 385, and 385 seconds. This predicts about 93 seconds less waiting for the slowest shard, before runner/setup overhead. Live CI will confirm the result. **Reason and benefit** Use the existing runners more evenly. No extra runner, test parallelism, cache, timeout, or routing change is needed. **Breaking changes** Suite-to-shard assignments change. The full suite set, assertions, and per-suite process isolation stay the same. **Additional context** Searched related CI and shard PRs. This updates the existing duration manifest, without duplicating a pending sharding implementation. ## What Changed - Refresh all 145 serialized suite weights from the same successful release verification run. - Record source job IDs and the measurement method in the manifest. Durations include process startup, imports, collection, tests, and shutdown. ## Verification - Passed all 30 shard and release-workflow tests: `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs scripts/__tests__/release-verify-workflow.test.mjs`. - Confirmed every measured suite appears exactly once across all five source logs. - Compared old and new assignments using the same measured weights. The maximum fell from 478862ms to 385551ms. - In [PR CI run 34710696242](https://github.com/paperclipai/paperclip/actions/runs/34710696242), all five serialized jobs passed in 7m05s–7m27s including setup. The measured assignment is now balanced in a live run. - The same run passed full typecheck, all grouped tests, native verification, build, release dry run, and browser checks. Local full-suite verification on this base was limited by disk exhaustion; local typecheck and targeted shard tests passed. - Latest-head Greptile is 5/5 with no open findings. Every current-head CI check must be green or intentionally skipped before merge. ## Risks - Individual durations vary with load and future test changes. These estimates affect assignment only; missing or renamed suites receive the existing median weight. - Both PR and release verification read this manifest, so both receive the new assignments. Each suite still runs in its own serialized Vitest process. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, and code execution. The exact serving model ID and context window are not exposed by this environment. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
49217aadf0 |
refactor: balance serialized server shards by recorded suite duration (#11528)
<!-- Write all pull request text in Simplified Technical English (ASD-STE100). --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The PR verify workflow gates every pull request; its wall-clock time sets the feedback loop for all contributors > - In a recent successful PR run (actions run 32012408876), the slowest check was "Verify serialized server suites (1/5)" at 337s, while its four sibling shards finished in 212-238s > - The serialized lane assigns suites to shards round-robin over an alphabetical list, so the heavy heartbeat and issues suites cluster on one runner > - The general-server lane already solves this with a duration-aware LPT partition backed by a recorded manifest > - This pull request reuses that partitioner for the serialized lane with a fresh per-suite duration manifest > - The benefit is a balanced serialized matrix: the measured 968s suite total levels to about 194s per shard, which removes about 80-100s from the run's slowest check ## Linked Issues or Issue Description **What existing behavior does this improve?** The `Verify serialized server suites` shard matrix in `.github/workflows/pr.yml` distributes route/authz test suites across five runners. **Subsystem affected** CI / test infrastructure (`scripts/run-vitest-stable.mjs`). **Current behavior** `selectSerializedSuites` assigns suites round-robin (`index % shardCount`) over the alphabetically sorted file list. The heavy suites cluster on shard 1/5. In actions run 32012408876, shard 1/5 spent 291s in its test step while the other shards spent 170-201s, which made that job (337s total) the slowest check of the whole PR run. **Proposed behavior** Partition the serialized suites with the same duration-aware LPT algorithm the general-server lane already uses (`scripts/general-server-shard.mjs`), backed by a new per-suite duration manifest. All five shards then carry about 194s of measured test time. **Reason and benefit** The slowest check bounds PR feedback time. Balancing the serialized matrix removes about 80-100s from that bound without adding runners. **Breaking changes** None. The partition remains deterministic, complete, and non-overlapping; suites missing from the manifest get the median weight. ## What Changed - Added `scripts/serialized-shard-durations.json`: per-suite wall-clock durations (ms) for all 134 serialized suites, sampled from actions run 32012408876 by diffing consecutive per-suite label timestamps in the shard logs (captures vitest spawn overhead, not just reported test time) - `scripts/run-vitest-stable.mjs`: `selectSerializedSuites` now uses the existing LPT partitioner (`selectGeneralServerShard`) with the new manifest instead of round-robin - `scripts/__tests__/run-vitest-stable-shard.test.mjs`: added a manifest-freshness test and a shard-balance test for the serialized lane, mirroring the general-server ones - `.github/workflows/pr.yml`: updated the serialized matrix comment with the new measurement and mechanism ## Verification - `node --test ./scripts/__tests__/run-vitest-stable-shard.test.mjs` passes (13 tests), including the existing test that the serialized shards form a complete, non-overlapping partition - Dry-run of all five shards shows estimated totals of 194/194/194/194/193s (round-robin was 276/175/160/172/187s): `node scripts/run-vitest-stable.mjs --mode serialized --shard-index N --shard-count 5 --dry-run` - The `Verify serialized server suites` jobs on this PR run the real partition end to end ## Risks - Low risk. Selection logic only; the vitest invocation per suite is unchanged - A stale manifest degrades gracefully: unknown suites get the median weight, and a dedicated test fails if fewer than half the current suites have recorded durations ## Model Used - Claude (Anthropic), model ID `claude-fable-5`, agentic coding session with tool use (Claude Code / Claude Agent SDK); no extended-thinking mode ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Related prior work: #10923 (split serialized tests into five shards), #10925 (general-server duration manifest), #11156 (workspaces-a native shards). Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Paperclip <noreply@paperclip.ing> |