3 Commits

Include renames

Author	SHA1	Message	Date
DottaandPaperclip	48b57e0c94	test(evals): pin held scenario repairs and await Cloud return (#19188) ... ## Thinking Path > - Paperclip helps people manage AI agents and their work. > - Protected TestOps runs admit one immutable catalog source. > - Three held scenario observers now have reviewed, locally qualified repairs. > - A separate Cloud recovery test started its heading wait before the Cloud return. > - This PR pins the qualified successor and corrects that test boundary. > - It changes no product behavior, model, execution budget, or private evidence protocol. ## Linked Issues or Issue Description Refs: #19155, #19093, #19095, #19112. The next candidate retains the reviewed skill, company-secret, and team-install observer repairs. It adds a narrow timeline v9 correction: a recorded task identifier can be the subject of its actual reassignment. The real timeline attempt remains failed; the diagnostic correction is not a regrade or a new native pass. Earlier failed attempts stay retained; a source pin and component tests do not establish a native pass. The Cloud recovery fixture failed both direct-task variants in the first CI attempt of #19155. Its five-second heading check started before the external Cloud roundtrip ended. The failure image already showed the expected task shortly afterward. One unchanged local direct-task control passed. The single authorized CI recovery also passed; this did not establish a reliable boundary. This PR waits for one handoff and the exact return URL before the original heading check. ## What Changed - Pin source `5aef4bffa2b106f2066bf2485338798f04a702bd`, tree `464be838a448c728d44e2d5593b9d7faea266663`. - Update the two manifest target literals and expected pin test literal. Bind the exact v9 descriptor snapshot and its one changed timeline oracle leaf. - Add the reviewed two-line Cloud return readiness witness. Keep the original five-second heading assertion and existing overall test limit. - Use descriptor `744a566faaa90b8048fff526e0a30b7475b86c2de88d697783f8da9e02f70e20`. Keep the launcher, all private helpers, 244 identities, requests, setup, human/native counts, clocks, profile, model, retry policy, permissions, and the 196/244 gate unchanged. ## Verification - Current infrastructure worktree: **131 pin, admission, selection, streaming, API, and reconciliation controls passed**. - The fixture owner ran the exact patched Cloud spec: **all five real auth cases passed** in 37.4s. A six-second Cloud hold passed the unchanged heading check after return. Missing handoff and missing return controls rejected. This PR's fresh master base is verified by current-head CI, not credited from the earlier run. - Fresh v9 source qualification: **268 focused timeline controls**, the 242-declaration gate, fresh server declarations, and full E2E typecheck passed under local Node **26.4.0**. The preceding held-helper qualification (239 Node controls and one offline Chromium route control) remains evidence for the byte-identical unaffected helper closures. The original browser sandbox failure is retained. Actual AWS Node and model configuration remain unchanged. - All 506 registered SDK input Git blobs and registry `efa70621e9fb42aa185f7ee616a1a726aa3eccb367e67e28817cedf26ef2ade8` equal the prior qualified source. No new SDK build or runtime refresh. - All 75 private protocol files, production trees, YAML, lockfile, launcher, identities, counts, clocks, and sourceFiles declarations equal the prior source. Only the timeline descriptor oracle advances v8 to v9. - Prior head `ce18f2bd52ed00ce750302f63aecc4167a3daff3` finished CI with 43 passed, one expected skip, four originating runner interruptions, and a failed dependent verify aggregate. Two logs show runner shutdown signals; two annotations show lost runner communication. No assertion failure was observed. Those outcomes are retained; no unchanged recovery was requested. - Required fresh current-head CI, genuine review, thread clearance, and Root's exact-diff acceptance are pending. No merge or new paid run is authorized by this PR's publication. - The earlier 3 PASS / 7 FAIL uniform attempt and all later original outcomes remain unchanged. No new native pass or whole-catalog readiness claim is made. ## Risks The source remains a frozen candidate with unmerged catalog repairs. A real measured run is still required. The readiness witness requires both a real handoff and the exact task URL; it does not extend the heading timeout, bypass auth, skip a case, or change production code. The active measurement on the previous pin is not altered. ## Model Used OpenAI GPT-6.1 (`gpt-6.1-sol`), with reasoning, code execution, and repository tools. The host did not expose a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched for and linked related public PRs above - [x] I have linked public PRs and described the issue - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change - [x] Relevant local pin tests passed; source and fixture qualification scope is stated above - [x] I have updated the expected pin test and the observed fixture boundary - [x] Owning source metadata and this description document the change - [x] I have considered and documented the risks - [ ] All current-head infrastructure CI gates are green - [ ] Greptile is 5/5 with no unresolved material findings - [x] I will address reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-04 23:14:14 -06:00
DottaandPaperclip	c99bcd89fe	fix: preserve Product TestOps evidence and CI readiness (#19000) ... ## Thinking Path > - Paperclip manages agent work through governed product workflows. > - Product E2E tests check those workflows through the app and Runner. > - A trusted reporter sends retained results to TestOps. > - Free-form metadata and a short upload observer can lose safe, valid evidence. > - UUID navigation can also discard task metadata before the canonical query returns. > - Hermetic CI files start child processes that need CPU capacity within their readiness limits. > - This change closes the reporting contract, retains authenticated core aliases, and reserves CI CPU capacity. > - Original case identities, conditions, assertions, verdicts and deadlines remain intact. ## Linked Issues or Issue Description Refs #18983. This addresses its late [metadata disclosure finding](https://github.com/paperclipai/paperclip/pull/18983#discussion_r4470789114) and [upload readiness finding](https://github.com/paperclipai/paperclip/pull/18983#discussion_r4470790366). **What happened?** The reporter accepted model IDs and matcher names through syntax filters. A secret-shaped string could pass those filters. First-task results also use their top-level model field for an observed alias. Treating it as configured metadata can reject valid retained evidence. The upload observer stopped after 22 seconds of waits. [Required CI on the earlier head](https://github.com/paperclipai/paperclip/actions/runs/41886885193) also found two real browser title failures and five native package readiness failures. A deterministic control reproduced metadata loss during UUID-to-identifier navigation. A separate fixed-resource control reproduced startup deadline failures with four concurrent hermetic fixtures, while one and two fixtures passed. **Expected behavior** Reject unknown transmitted metadata before creating upload files. Preserve every original matcher verdict and attempt receipt. Observe server processing without rerunning the product. Keep authenticated task metadata visible through canonical navigation. Reserve capacity for hermetic child processes without changing the original readiness deadlines. **Steps to reproduce** Use a secret-shaped model ID or cleanup state in retained evidence, return an observed first-task model alias, or let upload processing settle after 40 seconds of virtual time. Hold the canonical core query pending after an authenticated UUID read. For the CPU control, run the same providerless startup/recovery fixture alone, four times concurrently, and twice concurrently under a fixed 4-vCPU quota and its original 8-second readiness limit. ## What Changed - Bind configured model receipts to each exact declared case model. Four first-task cases use captured agent configuration; their observed aliases and fallback text remain digest-only. A missing capture stays null / Not recorded. The other 16 cells use their configured profile model. Reject wrong or unknown configuration before any upload file exists. - Keep the existing model distribution: 18 Codex cells on `gpt-5.6-sol`, configured-repository case 9 on its existing `gpt-6.1-sol` override, and task-workspaces case 54 on `openrouter/deepseek/deepseek-v4-flash-0731`. The repository override exists at the [original canary source](https://github.com/paperclipai/paperclip/blob/86814c6550f52e99ede150a883bdc773cbb540e3/tests/runner-e2e/catalog.ts#L1148). No model configuration changes. - Allow only existing Matcher kinds, closed failure classes, and cleanup states `passed`, `failed`, or `not_started`. Publish each matcher's kind, ordinal, original boolean verdict and SHA256 path receipt. Full conditions remain in the linked original workflow artifact; raw dynamic path bytes do not enter TestOps. - Observe the same upload for at most 55 seconds of poll waits. Product scenarios, provider retries and scenario deadlines do not change. - Retain task core data under only the authenticated response's own UUID and identifier core keys. Preserve existing canonical data. Keep arbitrary references and full-detail keys empty until their own reads. - Retain a first-failure trace for the existing Cloud recovery test. Keep its assertions, timeout, retries, mock data and artifact policy unchanged. This adds diagnosis evidence; it does not claim to fix that separate Cloud title timeout. - Reserve half the available CPUs for Runner package CI workers, with the existing upper cap of four. A 4-vCPU host uses two workers; an 8-vCPU host still uses four. Local defaults and test selection do not change. ## Verification - Reporting controls first reproduced the two original gaps. All 14 Node contract tests now pass. Unknown configuration creates neither a result directory nor an export summary. A model valid for a different selected case is also rejected. Original functional failures and earlier accounting receipts survive. - The catalog compatibility test passes for all 20 selected cells, exact model tuples, real pure grader outputs and actual `projectFirstTaskResult` shapes. It covers observed aliases, fallback text, absent evidence and wrong or secret-shaped configured metadata. These are contract controls, not live journey passes. - Virtual-clock controls accept a correct result that settles at 40 seconds and reject a persistently wrong verdict after 55 seconds. They make no product or provider calls. - Two new core-cache controls failed before repair. The query/cache suites now pass all 8 tests. They hold canonical refresh pending and verify title/company retention, newer canonical data, full-detail enrichment and unrelated keys. UI typecheck and token gates pass. - The unchanged exact UUID cross-company link/upload browser test passed once after repair: 5.8-second scenario, 32.0-second complete command. The unchanged exact Cloud task-entry diagnostic passed once: 5.4-second scenario, 23.3-second complete command. Its original canonical-only timeout remains unexplained. All five Cloud tests still collect after the trace setting. - The providerless Linux control used the retained Runner and compiled controller from source `3a2249dd`, with a fixed 4-vCPU quota and original 8-second readiness limit. Solo passed 1/1; four concurrent fixtures failed 4/4; two concurrent fixtures passed 2/2. At each failure, Rust had accepted command 2 but startup was still pending; checkpoint and published status agreed, one connection was authenticated, no replay backlog existed, and the sidecar had not launched. Passing controls preserved the expected startup failure through controller replacement. No network or credentials were available. - That control uses Linux amd64 emulation and Node 24.21; hosted CI used Node 24.22.1. It supports the worker-cap change, but does not prove that the original `m5n.xlarge` host caused every CI failure. The original red attempts remain recorded. - Actual config-load controls verify 1 CPU → 1 worker, 2 → 1, 4 → 2, 8 → 4 and 16 → 4, with no local, timeout or retry override. The isolated policy change preserves the same 297 test-file inventory; integration of current master `89703c3ff7` retains all 298 current native package files. The original PR package/native command plans remain unchanged. - Final head `877e28da32f2d2f0d12906c89aff085e2c6ea943` passes both required checks, `ci / verify` and `ci / e2e`, in [CI run 41897677652, attempt 2](https://github.com/paperclipai/paperclip/actions/runs/41897677652/attempts/2). The native package passes 4,950 tests with 10 existing skips. All eight browser shards pass; the Cloud shard passes all 41 tests, including all four unchanged recovery cases. - Attempt 1 preserved 46 successful jobs but its one-file heartbeat recovery shard hit the original 20-minute job limit without a named assertion or completion verdict. Its cause remains unidentified. One explicitly bounded debug replay retained those 46 successes and ran only that leaf and its dependent rollup. The unchanged file passed all 412 tests (346.91-second file, 371.34-second Vitest command, 7m11s job). Source, cases and deadlines did not change. This establishes intermittency, not a causal repair. The original timeout and debug pass both remain recorded. - Fresh [Greptile 5/5](https://github.com/paperclipai/paperclip/pull/19000#issuecomment-6637604918) and [Ponytail 5/5](https://github.com/paperclipai/paperclip/pull/19000#issuecomment-6637489454) reviews name exact head `877e28`. The single review thread is resolved. Live TestOps bootstrap and complete 20-case acceptance remain held until reviewed delivery. CI and focused controls do not count as product plan passes. ## Risks - Future model or Matcher kind changes require a reviewed reporting-contract update. - TestOps shows a closed Matcher kind and ordinal. The path digest binds the original condition in retained evidence. - Core alias data retains the existing authenticated query-client and server company boundaries. It never becomes full-detail evidence. - Fewer CI workers can increase one job's duration. The full suite must verify the final policy. The paid evaluation matrix and its parallelism do not change. - The original Cloud title timeout and the first-attempt heartbeat CI hang do not have proven causes. Current unchanged assertions pass, but this does not establish repeatability. The Cloud test now retains its first-failure trace. A successful upload or focused control alone cannot establish a complete product pass. ## Model Used OpenAI Codex with reasoning, tool use and code execution. The host does not expose the exact model ID or context window to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-04 14:41:28 -06:00
DottaandPaperclip	d9d2147171	fix(auth): keep Cloud tenants on the Cloud sign-in flow (#14407) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Cloud owns human identity and passes a verified identity to each tenant. > - The tenant can report no session while the Cloud session is still valid. > - The access gate and direct `/auth` route then show the instance password form. > - This pull request sends those users through the configured Cloud entry endpoint. > - Cloud can renew the tenant session or show its login page, then return to the original task. ## Linked Issues or Issue Description **What happened?** A Cloud tenant can display the self-hosted email/password form after an instance session check returns no session. This gives Cloud users the wrong login method. **Expected behavior** An active Cloud session renews tenant access automatically. A signed-out user signs in through Cloud. Staging and production use their own configured Cloud origins. Self-hosted instances keep their instance login form. **Steps to reproduce** 1. Open a Cloud tenant task or an `/auth?next=...` link. 2. Keep the Cloud session active but make the instance session check return 401. 3. Observe the instance password form instead of Cloud session recovery. **Deployment mode** Cloud-managed authenticated instances. No database or server API changes. Searched related authentication PRs. Native self-hosted OIDC support in #10411 is a separate feature; this change uses the existing Cloud entry contract. ## What Changed - Wait for deployment metadata before showing an instance login form. - Use the health response's Cloud origin and stack slug for session recovery. - Preserve the tenant path, query, and fragment. Reject external and recursive login return targets. - Limit automatic recovery per tab. Show a manual Cloud retry after failed recovery. Show service failures as errors. - Keep self-hosted login and local trusted access. Add focused tests, browser regressions, deployment documentation, and an unavailable-state design example. ## Verification - `pnpm -r typecheck` and `pnpm build` passed. - UI typecheck and `pnpm check:token-gates` passed after the final UI edits. - All UI tests passed: 639 files, 6,777 tests. - 63 focused Vitest tests passed across Auth, CloudAccessGate, Cloud links, and recovery coordination. - `pnpm exec playwright test --config tests/e2e/playwright.config.ts tests/e2e/cloud-auth.spec.ts`: 5 passed. The tests use the real tenant UI and database with a simulated Cloud HTTP endpoint. They cover both Cloud origins, direct auth/task links, no password-form flash, preserved URLs, reload, and self-hosted login. - Hands-on browser test used the real Cloud gateway and a fresh tenant build with disposable local data. Active Cloud session plus a forced missing instance session returned to the task. An expired tenant cookie also renewed automatically and returned to the task. Removing both sessions reached the real Cloud email/social login UI. Persistent failure stopped at the retry screen; retry succeeded after removing the injected fault. The fixture used a loopback transport adapter and a simulated signed-out OIDC issuer. No production session or deployment was changed. - The default local browser startup hit the host's embedded PostgreSQL resource limit. The passing run used a separate disposable database on the test PostgreSQL process. - The full local `pnpm test:run` sweep was stopped after about 31 minutes once CI completed the full suite. It had reported 33 failures in the unchanged runner API unit/integration files; both files pass in isolation (1,749 + 28 tests). The local sweep did not reach the later workspace/serialized groups. CI completed all of those groups successfully. - Greptile reviewed commit `d603fd4e39455de44da9dae81b72197096c0e1e8` at 5/5 with its only thread resolved. All CI gates are green on this commit, including all general/serialized server groups, workspace tests, Runner checks, typecheck, build, and all eight browser shards ([run](https://github.com/paperclipai/paperclip/actions/runs/36446230697)). ## Risks - Recovery depends on valid Cloud origin and stack metadata. Incomplete metadata shows an unavailable message instead of a password form. - Browsers with session storage disabled use the manual Cloud link, since automatic retries cannot be bounded across documents. - The external identity provider's email/social login was not completed in this local test. Existing Cloud authentication owns that flow. - No migration, credential format, membership rule, or production deployment changes. ## Model Used OpenAI GPT-6 through Codex. The exact served variant and context-window limit are not exposed in this session. Used reasoning, repository tools, code execution, and browser testing. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-28 13:45:27 -05:00