2 Commits

Include renames

Author	SHA1	Message	Date
DottaandPaperclip	39eafad47d	test(e2e): shorten and split Smoke Lab coverage (#12506) ... ## Thinking Path > - Paperclip uses browser tests to protect critical operator flows. > - The trusted pull request workflow runs the E2E catalog on three existing runners. > - Smoke Lab was one 168-second spec, so the shard scheduler could not divide it. > - The spec also repeated service-start calls, page loads, and full-page screenshots. > - This pull request removes that repeated work and divides the scenario catalog into two independent specs. > - The benefit is a shorter Smoke Lab run and a balanced E2E lane without more AWS capacity. ## Linked Issues or Issue Description Refs: #10629 **What existing behavior does this improve?** This improves the trusted pull request E2E lane and its Smoke Lab Playwright coverage. **Current behavior** Smoke Lab is one indivisible 168-second CI spec. It starts services for every scenario, loads the same evidence page twice, and captures a full-page success screenshot for all 56 lifecycle steps. **Proposed behavior** Start Smoke Lab services once per spec. Keep the per-scenario fixture reset. Capture one representative success screenshot per scenario and keep every failure screenshot. Run P1–P4 and P5–P7 as separate specs so the existing duration-aware scheduler can put them on different runners. **Reason and benefit** The optimized lifecycle reduced local Smoke Lab wall time from 57.68 seconds to 37.04 seconds. This is a 35.8% reduction. The two halves also let the existing three runners target about 125, 124, and 124 seconds of recorded spec work instead of about 168, 125, and 124 seconds. **Breaking changes** None. The same seven scenarios and eight lifecycle steps still run. The result API still records every step. Successful non-connect steps no longer attach redundant screenshots. ## What Changed - Reused one Smoke Lab service start within each spec while retaining isolated fixture installation for every scenario. - Removed the duplicate catalog evidence navigation. - Reduced success screenshots from 56 to 7 while retaining screenshots for every failed step. - Split the shared lifecycle runner into P1–P4 and P5–P7 specs. - Mark each successful split result as partial and keep dashboard health amber until one run covers the full catalog. - Updated the duration manifest and contributor docs for the split. ## Verification - `pnpm -r typecheck` passed on Node.js 24.20.0. - `pnpm build` passed on Node.js 24.20.0. - `node --test scripts/__tests__/e2e-shard.test.mjs` passed 9 tests. - `pnpm exec vitest run ui/src/pages/tools/smoke-lab-matrix.test.ts` passed 8 tests. - Both split specs passed together on Node.js 24.20.0 after the review fixes: 2 passed in 35.7 seconds; shell wall time was 36.86 seconds. - The pre-change Smoke Lab baseline passed with a 57.68-second shell wall time. The optimized unsplit A/B run passed with a 37.04-second shell wall time. - The full local E2E catalog passed 44 tests and skipped 2 tests. One existing `pipelines-tutorial-flow.spec.ts` assertion failed again when run alone. - The broad local unit run reproduced failures in untouched workspace-runtime suites. Typecheck, build, shard tests, and all changed browser coverage pass. CI remains the authoritative full-suite result. ## Risks - The split duration weights use the measured local reduction and the previous 168-second CI weight. They should be refreshed after two real pull request runs. - Service state is shared within each half. Fixture installation still runs before every scenario to reset connection, policy, and catalog state. - Each half records passed execution with partial coverage. Dashboard health recognizes the partial flag and stays amber because no single runner covers the full catalog. A failed half still records failed/red. - Fewer success screenshots reduce redundant artifacts. Every scenario keeps its connect screenshot, and every failure still captures evidence. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5. The exact deployment ID and context window are not exposed in this session. The model used agentic reasoning, code editing, shell execution, browser testing, and GitHub tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-08-31 10:15:39 -05:00
Devin FoleyandPaperclip	1944c86153	fix(ci): preserve required e2e check for sharded runs (#9923) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The pull request workflow is the main merge gate for changes to that app. > - The Playwright e2e lane is expensive because every spec shares one isolated server and runs serially. > - Splitting that lane across runners shortens the critical path, but the public required-check contract still needs a check named exactly `e2e`. > - This pull request shards the real e2e work while preserving a fast aggregate `e2e` job for branch protection. > - The benefit is a faster PR workflow without making otherwise-good PRs unmergeable because a legacy required check disappeared. ## Linked Issues or Issue Description No public GitHub issue exists for this CI follow-up. Related prior CI work: - Refs #8360 - Refs #9168 - Refs #9516 Bug report: ### What happened? Sharding the PR e2e lane directly at the workflow job level changes the emitted check names to shard-specific names, while existing branch protection expects a check named exactly `e2e`. ### Expected behavior The PR workflow should be able to run e2e specs across multiple runners while still emitting a stable aggregate check named `e2e`. ### Steps to reproduce 1. Open a PR against `master`. 2. Run the PR workflow with the e2e lane split only as a matrix job. 3. Observe that the shard checks complete, but a required check named exactly `e2e` never appears. ### Paperclip version or commit Current `master`. ### Deployment mode GitHub Actions pull request workflow. ## What Changed - Added `scripts/e2e-shard.mjs`, which partitions default Playwright e2e specs by recorded per-spec duration. - Added `scripts/e2e-shard-durations.json` with measured e2e spec durations so the slow smoke-lab spec does not dominate one runner. - Split the PR workflow e2e lane into two `e2e_shards` matrix jobs and added a fast aggregate job named exactly `e2e`. - Added `scripts/__tests__/e2e-shard.test.mjs` to lock the shard partition, ignored-spec sync, manifest coverage, and aggregate required-check contract. ## Verification - `node --test scripts/__tests__/e2e-shard.test.mjs` - `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs` - `git diff --check upstream/master..HEAD` - Searched GitHub for duplicate or related e2e-shard / required-check PRs and issues before opening this PR; no direct duplicate was found. ## Risks Low risk. The main risk is that the duration manifest can drift as specs are added or runtimes change; missing specs fall back to the median known duration, and the focused shard test catches empty, overlapping, or badly imbalanced partitions. ## Model Used OpenAI GPT-5 via Codex CLI coding agent, with shell/tool execution and repository inspection. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-20 16:40:19 -07:00