diff --git a/.agents/skills/paperclip-dev-workspace-run-verify-fix/SKILL.md b/.agents/skills/paperclip-dev-workspace-run-verify-fix/SKILL.md index 29af882a84b..06e6b2efe59 100644 --- a/.agents/skills/paperclip-dev-workspace-run-verify-fix/SKILL.md +++ b/.agents/skills/paperclip-dev-workspace-run-verify-fix/SKILL.md @@ -208,7 +208,7 @@ is missing, the cloned app does not have the expected companies/issues/agents, or the user explicitly asks for the normal isolated-workspace database. ```sh -pnpm paperclipai worktree reseed --from-instance default --seed-mode full --yes +npx paperclipai worktree reseed --from-instance default --seed-mode full --yes ``` After reseed, restart through the managed runtime path. A reseed can copy diff --git a/.agents/skills/paperclip-page/README.md b/.agents/skills/paperclip-page/README.md index 4ba859741b4..21bf74d3349 100644 --- a/.agents/skills/paperclip-page/README.md +++ b/.agents/skills/paperclip-page/README.md @@ -73,8 +73,8 @@ Required for live publishes: export AWS_REGION=us-east-1 export PAPERCLIP_PAGE_BUCKET=paperclip-pages-prod export PAPERCLIP_PAGE_BASE_URL=https://pages.paperclip.ing -export AWS_ACCESS_KEY_ID=... -export AWS_SECRET_ACCESS_KEY=... +export PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID=... +export PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY=... ``` Optional: @@ -82,15 +82,35 @@ Optional: ```bash export PAPERCLIP_PAGE_DEFAULT_PREFIX="" export PAPERCLIP_PAGE_AWS_PROFILE=paperclip-page-uploader +export PAPERCLIP_PAGE_AWS_SESSION_TOKEN=... # only with the namespaced key pair ``` +Credential resolution order inside `publish.sh`: + +1. `PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID` + `PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY` + (scoped to the helper's `aws` calls; the surrounding process identity is + untouched) +2. `PAPERCLIP_PAGE_AWS_PROFILE`, passed to `aws` as `--profile` (ambient + `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` / `AWS_SESSION_TOKEN` / + `AWS_PROFILE` are stripped from the helper's `aws` calls so the named + profile always wins) +3. The ambient AWS credential chain + +Setting both the namespaced key pair and `PAPERCLIP_PAGE_AWS_PROFILE` is an +error. + Recommended Paperclip secret names: - `paperclip-page-aws-access-key-id` - `paperclip-page-aws-secret-access-key` -Bind those secrets into publisher agents as `AWS_ACCESS_KEY_ID` and -`AWS_SECRET_ACCESS_KEY`. Do not reuse Paperclip's internal S3 attachment/object +Bind those secrets into publisher agents as `PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID` +and `PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY`. Never bind them as the global +`AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` names: static env keys override +`AWS_PROFILE` in the AWS CLI and every SDK, so global names silently switch the +whole agent run — and every subprocess — to the page-uploader identity and +break access to anything the uploader cannot reach (Secrets Manager, STS role +use, other buckets). Do not reuse Paperclip's internal S3 attachment/object storage credentials. ## AWS Setup @@ -473,27 +493,29 @@ Create secrets from environment variables so values do not land in shell history export PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID="$(jq -r '.AccessKey.AccessKeyId' /tmp/paperclip-page-uploader-key.json)" export PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY="$(jq -r '.AccessKey.SecretAccessKey' /tmp/paperclip-page-uploader-key.json)" -pnpm paperclipai secrets create \ +npx paperclipai secrets create \ --company-id \ --name paperclip-page-aws-access-key-id \ --value-env PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID -pnpm paperclipai secrets create \ +npx paperclipai secrets create \ --company-id \ --name paperclip-page-aws-secret-access-key \ --value-env PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY ``` -Bind runtime env to publishing agents: +Bind runtime env to publishing agents. Use the namespaced names — never the +global `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY`, which would shadow the +host `AWS_PROFILE` identity for the entire agent run: ```json { - "AWS_ACCESS_KEY_ID": { + "PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID": { "type": "secret_ref", "secretId": "", "version": "latest" }, - "AWS_SECRET_ACCESS_KEY": { + "PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY": { "type": "secret_ref", "secretId": "", "version": "latest" @@ -510,7 +532,7 @@ Bind runtime env to publishing agents: Create or update the company skill from this package: ```bash -pnpm paperclipai skills create \ +npx paperclipai skills create \ --company-id \ --name "Paperclip Page" \ --slug paperclip-page \ @@ -521,7 +543,7 @@ pnpm paperclipai skills create \ Attach it to an agent: ```bash -pnpm paperclipai skills agent sync \ +npx paperclipai skills agent sync \ --company-id \ --skill paperclip-page ``` diff --git a/.agents/skills/paperclip-page/SKILL.md b/.agents/skills/paperclip-page/SKILL.md index e9e74cfe085..df77444874d 100644 --- a/.agents/skills/paperclip-page/SKILL.md +++ b/.agents/skills/paperclip-page/SKILL.md @@ -19,10 +19,20 @@ host, for example `https://pages.paperclip.ing//`. - `PAPERCLIP_PAGE_BUCKET` - `PAPERCLIP_PAGE_BASE_URL` - `AWS_REGION` - - AWS credentials via Paperclip Secrets or an approved AWS vault + - `PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID` and `PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY` + with the page-uploader credentials from Paperclip Secrets - Optional environment variables: - `PAPERCLIP_PAGE_DEFAULT_PREFIX` - - `PAPERCLIP_PAGE_AWS_PROFILE` + - `PAPERCLIP_PAGE_AWS_PROFILE` (alternative to the namespaced key pair) + - `PAPERCLIP_PAGE_AWS_SESSION_TOKEN` (only together with the namespaced key + pair) + +Do not bind the page-uploader credentials as global `AWS_ACCESS_KEY_ID` / +`AWS_SECRET_ACCESS_KEY`: static env keys take precedence over `AWS_PROFILE` in +every AWS SDK, so global names silently replace the host identity for every +process in the agent run. The namespaced variables scope the uploader identity +to this helper only. The ambient credential chain still works as a fallback +when none of the `PAPERCLIP_PAGE_AWS_*` credential variables are set. ## Workflow diff --git a/.agents/skills/paperclip-page/scripts/publish.sh b/.agents/skills/paperclip-page/scripts/publish.sh index 2aa3a356add..086003a1e11 100755 --- a/.agents/skills/paperclip-page/scripts/publish.sh +++ b/.agents/skills/paperclip-page/scripts/publish.sh @@ -15,6 +15,15 @@ Required environment for live publish: Optional environment: PAPERCLIP_PAGE_DEFAULT_PREFIX, PAPERCLIP_PAGE_AWS_PROFILE + PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID, PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY, + PAPERCLIP_PAGE_AWS_SESSION_TOKEN + +Credential resolution for aws calls made by this helper: + 1. PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID + PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY + (used only by this helper; ambient AWS_PROFILE/AWS_* identity is untouched) + 2. PAPERCLIP_PAGE_AWS_PROFILE (passed as --profile; ambient AWS_* identity + variables are stripped from the helper's aws calls) + 3. Ambient AWS credential chain (env keys, profile, instance role) Options: --slug SLUG Lowercase URL slug. Allowed: a-z, 0-9, hyphen. @@ -125,9 +134,26 @@ join_prefix() { } aws_base_args=() +aws_env_unset=() +aws_env_overrides=() aws_cli() { - aws "${aws_base_args[@]}" "$@" + local name pair + if [[ ${#aws_env_unset[@]} -gt 0 || ${#aws_env_overrides[@]} -gt 0 ]]; then + # Scope the page-uploader identity to this helper's aws calls only, and + # drop the ambient identity variables that would otherwise mix with or + # shadow the configured credential source. Apply the overrides with shell + # builtins in a subshell — passing them to an external `env` command would + # expose the credential values in its argv (world-readable via + # /proc//cmdline) while it runs. + ( + for name in "${aws_env_unset[@]}"; do unset "$name"; done + for pair in "${aws_env_overrides[@]}"; do export "$pair"; done + exec aws "${aws_base_args[@]}" "$@" + ) + else + aws "${aws_base_args[@]}" "$@" + fi } object_exists() { @@ -297,6 +323,18 @@ default_prefix="$(normalize_default_prefix "${PAPERCLIP_PAGE_DEFAULT_PREFIX:-}") [[ -n "$base_url" ]] || die "PAPERCLIP_PAGE_BASE_URL is required" base_url="$(normalize_base_url "$base_url")" +page_access_key_id="${PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID:-}" +page_secret_access_key="${PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY:-}" +if [[ -n "$page_access_key_id" || -n "$page_secret_access_key" ]]; then + [[ -n "$page_access_key_id" && -n "$page_secret_access_key" ]] || + die "PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID and PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY must be set together" + [[ -z "${PAPERCLIP_PAGE_AWS_PROFILE:-}" ]] || + die "set PAPERCLIP_PAGE_AWS_PROFILE or the PAPERCLIP_PAGE_AWS_* key pair, not both" +fi +if [[ -n "${PAPERCLIP_PAGE_AWS_SESSION_TOKEN:-}" && -z "$page_access_key_id" ]]; then + die "PAPERCLIP_PAGE_AWS_SESSION_TOKEN requires the PAPERCLIP_PAGE_AWS_* key pair" +fi + explicit_slug=0 if [[ -n "$slug_arg" ]]; then explicit_slug=1 @@ -310,8 +348,18 @@ if [[ "$dry_run" == "0" ]]; then require_command curl [[ -n "$region" ]] || die "AWS_REGION is required for live publish" aws_base_args=(--region "$region") - if [[ -n "${PAPERCLIP_PAGE_AWS_PROFILE:-}" ]]; then + if [[ -n "$page_access_key_id" ]]; then + aws_env_unset=(AWS_PROFILE AWS_SESSION_TOKEN) + aws_env_overrides=( + AWS_ACCESS_KEY_ID="$page_access_key_id" + AWS_SECRET_ACCESS_KEY="$page_secret_access_key" + ) + if [[ -n "${PAPERCLIP_PAGE_AWS_SESSION_TOKEN:-}" ]]; then + aws_env_overrides+=(AWS_SESSION_TOKEN="$PAPERCLIP_PAGE_AWS_SESSION_TOKEN") + fi + elif [[ -n "${PAPERCLIP_PAGE_AWS_PROFILE:-}" ]]; then aws_base_args+=(--profile "$PAPERCLIP_PAGE_AWS_PROFILE") + aws_env_unset=(AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN AWS_PROFILE) fi fi diff --git a/.agents/skills/paperclip-page/scripts/publish.test.mjs b/.agents/skills/paperclip-page/scripts/publish.test.mjs index 8ef4b47bd2a..bb8384bfb0b 100644 --- a/.agents/skills/paperclip-page/scripts/publish.test.mjs +++ b/.agents/skills/paperclip-page/scripts/publish.test.mjs @@ -131,6 +131,215 @@ test("rejects hidden files in the source tree", () => { assert.match(result.output, /hidden files and dot paths are not allowed/); }); +test("namespaced page keys require both halves of the pair", () => { + const result = runPublish( + [createSite(), "--slug", "demo-page", "--dry-run"], + { PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID: "AKIAPAGEUPLOADER" }, + ); + + assert.notEqual(result.status, 0); + assert.match(result.output, /must be set together/); +}); + +test("namespaced session token requires the namespaced key pair", () => { + const result = runPublish( + [createSite(), "--slug", "demo-page", "--dry-run"], + { PAPERCLIP_PAGE_AWS_SESSION_TOKEN: "page-session-token" }, + ); + + assert.notEqual(result.status, 0); + assert.match(result.output, /requires the PAPERCLIP_PAGE_AWS_\* key pair/); +}); + +test("namespaced page keys conflict with PAPERCLIP_PAGE_AWS_PROFILE", () => { + const result = runPublish( + [createSite(), "--slug", "demo-page", "--dry-run"], + { + PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID: "AKIAPAGEUPLOADER", + PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY: "page-secret", + PAPERCLIP_PAGE_AWS_PROFILE: "paperclip-page-uploader", + }, + ); + + assert.notEqual(result.status, 0); + assert.match(result.output, /not both/); +}); + +test("namespaced page keys are scoped to the helper's aws calls", () => { + const siteDir = createSite(); + const binDir = mkdtempSync(join(tmpdir(), "paperclip-page-bin-")); + tempDirs.add(binDir); + const envDump = join(binDir, "aws-env.txt"); + + writeExecutable( + join(binDir, "aws"), + `#!/usr/bin/env bash +set -euo pipefail +{ + echo "AWS_ACCESS_KEY_ID=\${AWS_ACCESS_KEY_ID:-}" + echo "AWS_SECRET_ACCESS_KEY=\${AWS_SECRET_ACCESS_KEY:-}" + echo "AWS_SESSION_TOKEN=\${AWS_SESSION_TOKEN:-}" + echo "AWS_PROFILE=\${AWS_PROFILE:-}" +} >"${envDump}" +while [[ "$1" == "--region" || "$1" == "--profile" ]]; do + shift 2 +done +if [[ "$1" == "s3api" ]]; then + echo "None" + exit 0 +fi +if [[ "$1" == "s3" && "$2" == "sync" ]]; then + exit 0 +fi +echo "unexpected aws call: $*" >&2 +exit 1 +`, + ); + writeExecutable( + join(binDir, "curl"), + `#!/usr/bin/env bash +exit 0 +`, + ); + + const result = runPublish([siteDir, "--slug", "demo-page"], { + AWS_REGION: "us-east-1", + PATH: `${binDir}:${process.env.PATH}`, + AWS_ACCESS_KEY_ID: "AKIAAMBIENTIDENTITY", + AWS_SECRET_ACCESS_KEY: "ambient-secret", + AWS_SESSION_TOKEN: "ambient-session-token", + AWS_PROFILE: "ambient-profile", + PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID: "AKIAPAGEUPLOADER", + PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY: "page-secret", + }); + + assert.equal(result.status, 0); + + const seen = readFileSync(envDump, "utf8"); + assert.match(seen, /^AWS_ACCESS_KEY_ID=AKIAPAGEUPLOADER$/m); + assert.match(seen, /^AWS_SECRET_ACCESS_KEY=page-secret$/m); + assert.match(seen, /^AWS_SESSION_TOKEN=$/m); + assert.match(seen, /^AWS_PROFILE=$/m); +}); + +test("credential values never pass through an external env command's argv", () => { + const siteDir = createSite(); + const binDir = mkdtempSync(join(tmpdir(), "paperclip-page-bin-")); + tempDirs.add(binDir); + const envArgvDump = join(binDir, "env-argv.txt"); + + writeExecutable( + join(binDir, "aws"), + `#!/usr/bin/env bash +set -euo pipefail +while [[ "$1" == "--region" || "$1" == "--profile" ]]; do + shift 2 +done +if [[ "$1" == "s3api" ]]; then + echo "None" + exit 0 +fi +if [[ "$1" == "s3" && "$2" == "sync" ]]; then + exit 0 +fi +echo "unexpected aws call: $*" >&2 +exit 1 +`, + ); + writeExecutable( + join(binDir, "curl"), + `#!/usr/bin/env bash +exit 0 +`, + ); + // Shim env: record every argv it is invoked with, then behave normally. + // Credentials in that argv would be world-readable via /proc//cmdline. + writeExecutable( + join(binDir, "env"), + `#!/bin/bash +printf '%s\\n' "$@" >>"${envArgvDump}" +exec /usr/bin/env "$@" +`, + ); + + const result = runPublish([siteDir, "--slug", "demo-page"], { + AWS_REGION: "us-east-1", + PATH: `${binDir}:${process.env.PATH}`, + PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID: "AKIAPAGEUPLOADER", + PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY: "page-secret-argv-canary", + PAPERCLIP_PAGE_AWS_SESSION_TOKEN: "page-session-argv-canary", + }); + + assert.equal(result.status, 0); + + const argvSeen = existsSync(envArgvDump) ? readFileSync(envArgvDump, "utf8") : ""; + assert.doesNotMatch(argvSeen, /page-secret-argv-canary/); + assert.doesNotMatch(argvSeen, /page-session-argv-canary/); + assert.doesNotMatch(argvSeen, /AKIAPAGEUPLOADER/); +}); + +test("PAPERCLIP_PAGE_AWS_PROFILE strips ambient static credentials", () => { + const siteDir = createSite(); + const binDir = mkdtempSync(join(tmpdir(), "paperclip-page-bin-")); + tempDirs.add(binDir); + const envDump = join(binDir, "aws-env.txt"); + + writeExecutable( + join(binDir, "aws"), + `#!/usr/bin/env bash +set -euo pipefail +profile="" +while [[ "$1" == "--region" || "$1" == "--profile" ]]; do + if [[ "$1" == "--profile" ]]; then + profile="$2" + fi + shift 2 +done +{ + echo "PROFILE_ARG=$profile" + echo "AWS_ACCESS_KEY_ID=\${AWS_ACCESS_KEY_ID:-}" + echo "AWS_SECRET_ACCESS_KEY=\${AWS_SECRET_ACCESS_KEY:-}" + echo "AWS_SESSION_TOKEN=\${AWS_SESSION_TOKEN:-}" + echo "AWS_PROFILE=\${AWS_PROFILE:-}" +} >"${envDump}" +if [[ "$1" == "s3api" ]]; then + echo "None" + exit 0 +fi +if [[ "$1" == "s3" && "$2" == "sync" ]]; then + exit 0 +fi +echo "unexpected aws call: $*" >&2 +exit 1 +`, + ); + writeExecutable( + join(binDir, "curl"), + `#!/usr/bin/env bash +exit 0 +`, + ); + + const result = runPublish([siteDir, "--slug", "demo-page"], { + AWS_REGION: "us-east-1", + PATH: `${binDir}:${process.env.PATH}`, + AWS_ACCESS_KEY_ID: "AKIAAMBIENTIDENTITY", + AWS_SECRET_ACCESS_KEY: "ambient-secret", + AWS_SESSION_TOKEN: "ambient-session-token", + AWS_PROFILE: "ambient-profile", + PAPERCLIP_PAGE_AWS_PROFILE: "paperclip-page-uploader", + }); + + assert.equal(result.status, 0); + + const seen = readFileSync(envDump, "utf8"); + assert.match(seen, /^PROFILE_ARG=paperclip-page-uploader$/m); + assert.match(seen, /^AWS_ACCESS_KEY_ID=$/m); + assert.match(seen, /^AWS_SECRET_ACCESS_KEY=$/m); + assert.match(seen, /^AWS_SESSION_TOKEN=$/m); + assert.match(seen, /^AWS_PROFILE=$/m); +}); + test("live publish writes state before URL verification", () => { const siteDir = createSite(); const binDir = mkdtempSync(join(tmpdir(), "paperclip-page-bin-")); diff --git a/.agents/skills/release-changelog-discord-message/SKILL.md b/.agents/skills/release-changelog-discord-message/SKILL.md index 367ee8474d2..7852ad1a83c 100644 --- a/.agents/skills/release-changelog-discord-message/SKILL.md +++ b/.agents/skills/release-changelog-discord-message/SKILL.md @@ -11,7 +11,11 @@ description: > Write the Discord release announcement for the **stable** Paperclip release. This is the companion to `.agents/skills/release-changelog/SKILL.md`. That skill -generates the file at `releases/vYYYY.MDD.P.md`. This skill turns that file into +writes the changelog — during the beta soak it lives at +`releases/beta/v{beta-version}.md` on the `release-notes/v{beta-version}` +branch, and after the stable ships a canonicalization PR renames it to +`releases/vYYYY.MDD.P.md` (see that skill's Channel Process section). This +skill turns that file into a single copy-pasteable Discord block, in dotta's voice, and posts it as the `discord_announcement` document on the release issue. @@ -27,8 +31,9 @@ current Paperclip work — not invented. ## When to use -- After `release-changelog` has produced `releases/vYYYY.MDD.P.md` on the - release worktree/PR. +- After `release-changelog` has produced the changelog (beta-keyed on the + `release-notes/v{beta-version}` branch during the soak, or the + canonicalized `releases/vYYYY.MDD.P.md` after the stable ships). - When the release issue (the one assigned by the release routine) asks for a Discord announcement, or has a `discord_announcement` document that needs to be refreshed for a new date/version. @@ -123,6 +128,12 @@ Notes on the template: - The opening and closing `:paperclip: :paperclip: :paperclip:` bookends are part of the brand — keep them. +- Name the install channels somewhere in the post: `npx paperclipai@latest` + for the stable, `@beta` / `@nightly` / `@canary` for earlier access, and + Docker `:latest` moving **only** on stable releases. +- The FULL RELEASE NOTES link points at `releases/v{VERSION}.md` on + `master` — that file exists only after the post-stable canonicalization + PR merges. Merge it before the announcement is posted. - Sections may be UPPERCASE or Title Case — dotta has used both. Pick a style and stay consistent within a single post. - Use `||@everyone||` (Discord spoiler-wrapped) at the very end so it pings @@ -163,7 +174,8 @@ Mimic this register; do not invent a "professional" tone. (follow the twitter, intros, beta sign-ups). No real ask → drop it. - **Community** is the same contributors list that's in the changelog file, fenced in a triple-backtick block, comma-separated `@username, @username`. - Exclude bots and Paperclip founders, same rules as the changelog skill. + Exclude bots and the specific folks on the changelog skill's canonical + exclusion list — same rules. - **The "In Summary" mission line** evolves slowly. Use the most recent variant unless dotta tells you otherwise. Recent variants: - "PAPERCLIP IS THE AI ORCHESTRATOR FOR HUMANS TO ACCOMPLISH 100x MORE WORK" @@ -174,15 +186,19 @@ Mimic this register; do not invent a "professional" tone. ## Workflow -1. Read the matching `releases/vYYYY.MDD.P.md` produced by `release-changelog`. - Use the version and contributor list from that file — never re-derive them. +1. Read the matching changelog produced by `release-changelog` — the + beta-keyed file during the soak, `releases/vYYYY.MDD.P.md` once + canonicalized. Use the version and contributor list from that file — + never re-derive them. 2. Resolve the parent `release` case with key `paperclip-release:vYYYY.MDD.P`. If it does not exist and Cases are enabled, create it using the schema in `.agents/skills/release-changelog/SKILL.md` before creating child cases. 3. Read the **release issue thread** (the one assigned to you that ran the release routine) — comments + linked issues + recent issues in the company - are the source for `WHATS NEXT` and `What's on my mind`. Pull real themes, - not invented ones. + are the source for `WHATS NEXT` and `What's on my mind`. Commits already + on `origin/master` **after** the beta source commit are prime "what's + next" material: they are literally the next release's content. Pull real + themes, not invented ones. 4. Re-read the three verbatim examples below — they're the canonical voice. 5. Draft the announcement using the template above. 6. PUT it as the `discord_announcement` document on the release issue (see @@ -444,7 +460,8 @@ https://github.com/paperclipai/paperclip/blob/master/releases/v2026.427.0.md Before handing off: 1. Version + date match the matching `releases/vYYYY.MDD.P.md` exactly. -2. Contributor list matches the changelog (same exclusions: bots, founders). +2. Contributor list matches the changelog (same exclusions: bots and the + changelog skill's canonical excluded-folks list). 3. Highlights are a subset of the changelog Highlights — same shipped features, not invented or pre-alpha work. 4. `WHATS NEXT` and `What's on my mind` are pulled from real recent issues / diff --git a/.agents/skills/release-changelog/SKILL.md b/.agents/skills/release-changelog/SKILL.md index 8bf662ed6b3..da1d141d336 100644 --- a/.agents/skills/release-changelog/SKILL.md +++ b/.agents/skills/release-changelog/SKILL.md @@ -32,14 +32,57 @@ Important rules: - do not derive versions from semver bump types - do not create canary changelog files +## Channel Process — Source Commit and File Location + +Stables promote a **soaked beta**, so the changelog describes the beta's +source commit, not the tip of `master`: + +- The release **source** is the commit the newest `beta/v` + tag points at (`{beta-src}` below). Resolve it with: + + ```bash + git fetch origin --tags + npm view paperclipai dist-tags # the beta dist-tag names the version + git rev-parse 'beta/v{beta-version}^{commit}' + ``` + +- Commits on `master` after `{beta-src}` ship in the **next** release. + Never include them; they are input for a "what's next" section, not the + changelog. +- During the soak, the file lives at `releases/beta/v{beta-version}.md` + on the branch `release-notes/v{beta-version}` (PR to `master`). The + release workflow pushes that branch with a generated skeleton when the + beta publishes; work on it and rewrite the skeleton in place. If the + branch does not exist (a beta cut before the automation), create it + from `origin/master` and seed the skeleton: + + ```bash + ./scripts/draft-stable-notes.sh {beta-version} + ``` + +- The PR must merge to `master` **before** the stable is dispatched: the + stable preflight reads the file from `master` and fails without it. +- Never create `releases/vYYYY.MDD.P.md` yourself on this path — after + the stable ships, the workflow opens a canonicalization PR that renames + the beta-keyed file to it. +- **Fix path exception** (patch releases from a `candidate/release-*` + branch): there the notes *do* go directly on the candidate branch as + `releases/vYYYY.MDD.P.md`, committed alongside the cherry-picked fixes. + ## Step 0 — Idempotency Check -Before generating anything, check whether the file already exists: +Before generating anything, check whether the changelog already exists: ```bash -ls releases/vYYYY.MDD.P.md 2>/dev/null +ls releases/beta/v{beta-version}.md 2>/dev/null # soak-window home +ls releases/vYYYY.MDD.P.md 2>/dev/null # canonicalized / fix path +git ls-remote origin 'refs/heads/release-notes/v{beta-version}' ``` +A `release-notes/v{beta-version}` branch holding only the generated +skeleton is the normal starting state, not a conflict — rewrite it in +place. + If it exists: 1. read it first @@ -49,13 +92,17 @@ If it exists: ## Step 1 — Determine the Stable Range -Find the last stable tag: +Find the last stable tag and the beta source commit: ```bash git tag --list 'v*' --sort=-version:refname | head -1 -git log v{last}..HEAD --oneline --no-merges +beta_src="$(git rev-parse 'beta/v{beta-version}^{commit}')" +git log v{last}..${beta_src} --oneline --no-merges ``` +The changelog range is always `v{last}..{beta-src}` — never `..HEAD` and +never `..origin/master`. + The stable version comes from one of: - an explicit maintainer request @@ -76,8 +123,8 @@ Collect release data from: Useful commands: ```bash -git log v{last}..HEAD --oneline --no-merges -git log v{last}..HEAD --format="%H %s" --no-merges +git log v{last}..{beta-src} --oneline --no-merges +git log v{last}..{beta-src} --format="%H %s" --no-merges ls .changeset/*.md | grep -v README.md gh pr list --state merged --search "merged:>={last-tag-date}" --json number,title,body,labels ``` @@ -94,10 +141,10 @@ Look for: Key commands: ```bash -git diff --name-only v{last}..HEAD -- packages/db/src/migrations/ -git diff v{last}..HEAD -- packages/db/src/schema/ -git diff v{last}..HEAD -- server/src/routes/ server/src/api/ -git log v{last}..HEAD --format="%s" | rg -n 'BREAKING CHANGE|BREAKING:|^[a-z]+!:' || true +git diff --name-only v{last}..{beta-src} -- packages/db/src/migrations/ +git diff v{last}..{beta-src} -- packages/db/src/schema/ +git diff v{last}..{beta-src} -- server/src/routes/ server/src/api/ +git log v{last}..{beta-src} --format="%s" | rg -n 'BREAKING CHANGE|BREAKING:|^[a-z]+!:' || true ``` If breaking changes are detected, flag them prominently — they must appear in the @@ -143,6 +190,14 @@ Rules: ## Step 5 — Write the File +The **file path** is the beta-keyed one from the Channel Process section +(`releases/beta/v{beta-version}.md`), but the **content** is titled with +the planned stable version. Resolve it with +`./scripts/release.sh stable --date {planned-promotion-date} --print-version` +(promotion is normally the beta publish date plus the 3-day soak). If the +promotion date slips, the version re-resolves at dispatch — the beta-keyed +filename makes that harmless; refresh the title when it happens. + The opening line of the changelog must be an H1 of the format `# Paperclip {version}` (no braces), e.g. `# Paperclip v2026.618.0`. Always include the `Paperclip ` prefix and the `v` on the version. @@ -177,14 +232,18 @@ The `Contributors` section should always be included. List every person who auth commits in the release range, @-mentioning them by their **GitHub username** (not their real name or email). To find GitHub usernames: -1. Extract usernames from merge commit messages: `git log v{last}..HEAD --oneline --merges` — the branch prefix (e.g. `from username/branch`) gives the GitHub username. +1. Extract usernames from merge commit messages: `git log v{last}..{beta-src} --oneline --merges` — the branch prefix (e.g. `from username/branch`) gives the GitHub username. 2. For noreply emails like `user@users.noreply.github.com`, the username is the part before `@`. 3. For contributors whose username is ambiguous, check `gh api users/{guess}` or the PR page. **Never expose contributor email addresses.** Use `@username` only. Exclude bot accounts (e.g. `lockfile-bot`, `dependabot`) from the list. -Exclude Paperclip founders from the list (e.g. `cryppadotta`, `forgottendev`, `devinfoley`, `sockmonster`, `scotttong`) +Exclude specific folks from the list — the Contributors section credits +community contributors only. The canonical exclusion list (keep it here; +the Discord skill defers to it): +`cryppadotta`, `forgottendev`, `devinfoley`, `sockmonster`, `scotttong`, +`nguyenm7`, `nickyleach`, `tonio-alucema` List contributors in alphabetical order by GitHub username (case-insensitive). diff --git a/.claude/launch.json b/.claude/launch.json index 1af7d4ce098..7cbc05c40a6 100644 --- a/.claude/launch.json +++ b/.claude/launch.json @@ -7,13 +7,6 @@ "runtimeArgs": ["-c", "TMPDIR=/tmp pnpm dev"], "port": 3108, "autoPort": false - }, - { - "name": "ui-preview", - "runtimeExecutable": "pnpm", - "runtimeArgs": ["--filter", "@paperclipai/ui", "exec", "vite"], - "port": 5188, - "autoPort": true } ] } diff --git a/.claude/skills/design-guide/references/component-index.md b/.claude/skills/design-guide/references/component-index.md index 93c43b1754a..88e75178b31 100644 --- a/.claude/skills/design-guide/references/component-index.md +++ b/.claude/skills/design-guide/references/component-index.md @@ -188,24 +188,6 @@ Use in property rows, comment headers, assignee displays, and anywhere a user/ag **File:** `CompanySwitcher.tsx` **Usage:** Company selector dropdown in sidebar header. -### AgentCapsule - -**File:** `AgentCapsule.tsx` -**Props:** `state: "slot" | "configured" | "online"`, `gradient?: 1–10`, `size?: "sm" | "md" | "lg" | {width,height}`, `glow?: "green" | "blue"` -**Usage:** The brand "capsule is the agent" pill; evolves in place across onboarding steps. Fill uses `--agent-Na/Nb` gradient tokens; honors `prefers-reduced-motion`. - -### Onboarding primitives (OnboardingCard, OnboardingHeading, Stepper, Chip, ChoiceCard, ConnectorRow) - -**File:** `onboarding/OnboardingPrimitives.tsx` -**Usage:** Presentational pieces for the full-screen onboarding flow (`onboarding/OnboardingFlow.tsx`): the 560px card frame (`--sz-560px`), display heading + lede (text-4xl), 3-segment stepper, selectable mission chips, selectable choice cards, and connector rows. Bespoke dimensions route through verbatim `--sz-*` tokens; fields inside the flow use the shared Input/Textarea/Select/Label primitives. - -```tsx - - - - -``` - --- ## Layout Components diff --git a/.env.example b/.env.example index e747df0914d..ddd69484baa 100644 --- a/.env.example +++ b/.env.example @@ -4,5 +4,11 @@ SERVE_UI=false BETTER_AUTH_SECRET=paperclip-dev-secret PAPERCLIP_TOOL_ACTION_SIGNING_SECRET=paperclip-dev-tool-action-signing-secret-change-me +# Process-wide protection for expensive full-tree workspace Git scans. +# PAPERCLIP_WORKSPACE_GIT_SCAN_CONCURRENCY=2 +# PAPERCLIP_WORKSPACE_GIT_SCAN_QUEUE_CAPACITY=32 +# PAPERCLIP_WORKSPACE_GIT_SCAN_TIMEOUT_MS=8000 +# PAPERCLIP_WORKSPACE_GIT_SCAN_CACHE_TTL_MS=10000 + # Discord webhook for daily merge digest (scripts/discord-daily-digest.sh) # DISCORD_WEBHOOK_URL=https://discord.com/api/webhooks/... diff --git a/.github/dependabot.yml b/.github/dependabot.yml index ff0b351468a..33caa46d21c 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -9,10 +9,16 @@ updates: open-pull-requests-limit: 10 labels: - "dependencies" - ignore: - # Ignore major version bumps — review those manually - - dependency-name: "*" - update-types: ["version-update:semver-major"] + groups: + # Bundle all major version bumps into a single weekly PR so they + # stay visible without flooding the PR list. Minor/patch updates + # still get individual PRs. + major-updates: + applies-to: version-updates + update-types: + - "major" + patterns: + - "*" - package-ecosystem: github-actions directory: "/" diff --git a/.github/workflows/docker.yml b/.github/workflows/docker.yml index ba9c25e4009..2e41dd7657b 100644 --- a/.github/workflows/docker.yml +++ b/.github/workflows/docker.yml @@ -6,6 +6,13 @@ on: - "master" tags: - "v*" + - "nightly/v*" + - "beta/v*" + # Release workflows push lane tags with GITHUB_TOKEN, and GitHub suppresses + # push-triggered runs for those, so release.yml dispatches this workflow at + # the new tag ref instead. The tag mapping below keys off github.ref either + # way. + workflow_dispatch: permissions: contents: read @@ -40,7 +47,19 @@ jobs: id: build-version run: | set -euo pipefail - version="$(git describe --tags --match 'v*' --long --dirty 2>/dev/null || true)" + case "${GITHUB_REF}" in + refs/tags/nightly/v*) + # Lane tags carry the exact published version; stamp it verbatim + # instead of describing drift from the nearest stable tag. + version="${GITHUB_REF#refs/tags/nightly/v}" + ;; + refs/tags/beta/v*) + version="${GITHUB_REF#refs/tags/beta/v}" + ;; + *) + version="$(git describe --tags --match 'v*' --long --dirty 2>/dev/null || true)" + ;; + esac echo "version=${version}" >> "$GITHUB_OUTPUT" echo "Stamping build version: ${version:-}" @@ -131,15 +150,21 @@ jobs: echo "last=${last}" >> "$GITHUB_OUTPUT" echo "count=${count}" >> "$GITHUB_OUTPUT" + # Lane tag mapping: master pushes publish `:canary`, nightly/v* tags + # publish `:nightly`, and only stable v* tags move `:latest` and the + # versioned tags. `:sha-` is published on every build. - name: Docker meta id: meta uses: docker/metadata-action@v6 with: images: ghcr.io/${{ github.repository }} tags: | - type=raw,value=latest,enable={{is_default_branch}} - type=semver,pattern={{version}} - type=semver,pattern={{major}}.{{minor}} + type=raw,value=canary,enable=${{ github.ref == 'refs/heads/master' }} + type=raw,value=nightly,enable=${{ startsWith(github.ref, 'refs/tags/nightly/v') }} + type=raw,value=beta,enable=${{ startsWith(github.ref, 'refs/tags/beta/v') }} + type=raw,value=latest,enable=${{ startsWith(github.ref, 'refs/tags/v') }} + type=semver,pattern={{version}},enable=${{ startsWith(github.ref, 'refs/tags/v') }} + type=semver,pattern={{major}}.{{minor}},enable=${{ startsWith(github.ref, 'refs/tags/v') }} type=sha labels: | io.github.paperclipai.schema.last-migration=${{ steps.schema.outputs.last }} @@ -197,7 +222,19 @@ jobs: id: build-version run: | set -euo pipefail - version="$(git describe --tags --match 'v*' --long --dirty 2>/dev/null || true)" + case "${GITHUB_REF}" in + refs/tags/nightly/v*) + # Lane tags carry the exact published version; stamp it verbatim + # instead of describing drift from the nearest stable tag. + version="${GITHUB_REF#refs/tags/nightly/v}" + ;; + refs/tags/beta/v*) + version="${GITHUB_REF#refs/tags/beta/v}" + ;; + *) + version="$(git describe --tags --match 'v*' --long --dirty 2>/dev/null || true)" + ;; + esac echo "version=${version}" >> "$GITHUB_OUTPUT" echo "Stamping build version: ${version:-}" @@ -288,8 +325,9 @@ jobs: echo "last=${last}" >> "$GITHUB_OUTPUT" echo "count=${count}" >> "$GITHUB_OUTPUT" - # Published under the same tag set with a `-cloud` suffix - # (sha--cloud, latest-cloud, -cloud). + # Published under the same lane tag set as the self-hosted image, with a + # `-cloud` suffix (canary-cloud, nightly-cloud, latest-cloud, + # -cloud, sha--cloud). - name: Docker meta (cloud) id: meta-cloud uses: docker/metadata-action@v6 @@ -298,9 +336,12 @@ jobs: flavor: | suffix=-cloud,onlatest=true tags: | - type=raw,value=latest,enable={{is_default_branch}} - type=semver,pattern={{version}} - type=semver,pattern={{major}}.{{minor}} + type=raw,value=canary,enable=${{ github.ref == 'refs/heads/master' }} + type=raw,value=nightly,enable=${{ startsWith(github.ref, 'refs/tags/nightly/v') }} + type=raw,value=beta,enable=${{ startsWith(github.ref, 'refs/tags/beta/v') }} + type=raw,value=latest,enable=${{ startsWith(github.ref, 'refs/tags/v') }} + type=semver,pattern={{version}},enable=${{ startsWith(github.ref, 'refs/tags/v') }} + type=semver,pattern={{major}}.{{minor}},enable=${{ startsWith(github.ref, 'refs/tags/v') }} type=sha labels: | io.github.paperclipai.schema.last-migration=${{ steps.schema.outputs.last }} diff --git a/.github/workflows/pr.yml b/.github/workflows/pr.yml index 097692cdecb..7fe5fca58e4 100644 --- a/.github/workflows/pr.yml +++ b/.github/workflows/pr.yml @@ -174,8 +174,20 @@ jobs: group_label: server (5/5) shard_index: 4 shard_count: 5 + # workspaces-a was the slowest check in the fully-green PR run + # 31371439296 (2026-08-10) at 319s, with the ui project's single + # vitest invocation accounting for ~224s and the paperclipai CLI + # ~37s. Two shards use Vitest's native --shard on each project's + # file list (ui: 439 files, cli: 54), bringing each job to roughly + # half the suite time (~130s + setup) without a duration manifest. - group: general-workspaces-a - group_label: workspaces-a + group_label: workspaces-a (1/2) + shard_index: 0 + shard_count: 2 + - group: general-workspaces-a + group_label: workspaces-a (2/2) + shard_index: 1 + shard_count: 2 - group: general-workspaces-b group_label: workspaces-b @@ -275,9 +287,11 @@ jobs: fail-fast: false matrix: include: - # A successful PR run on 2026-08-04 (30876682788) spent 256s in - # serialized shard 2/4, making its 305s job the run's slowest check. - # Five shards reduce the measured 739s suite total to about 148s per + # A successful PR run on 2026-08-17 (32012408876) spent 291s in + # serialized shard 1/5 while its siblings ran 170-201s: round-robin + # clustered the heavy suites on one runner. Shards are now balanced + # by recorded duration (scripts/serialized-shard-durations.json), + # which levels the measured 968s suite total to about 194s per # runner before setup overhead. - shard_index: 0 shard_count: 5 diff --git a/.github/workflows/release-smoke.yml b/.github/workflows/release-smoke.yml index 923bb896a3e..5ec66da8a69 100644 --- a/.github/workflows/release-smoke.yml +++ b/.github/workflows/release-smoke.yml @@ -10,6 +10,8 @@ on: type: choice options: - canary + - nightly + - beta - latest host_port: description: Host port for the Docker smoke container @@ -69,6 +71,7 @@ jobs: HOST_PORT="${{ inputs.host_port }}" \ DATA_DIR="$RUNNER_TEMP/release-smoke-data" \ PAPERCLIPAI_VERSION="${{ inputs.paperclip_version }}" \ + SMOKE_READY_TIMEOUT_SECONDS=420 \ SMOKE_DETACH=true \ SMOKE_METADATA_FILE="$metadata_file" \ ./scripts/docker-onboard-smoke.sh diff --git a/.github/workflows/release-verify.yml b/.github/workflows/release-verify.yml index cac52d75b5d..df146e7faac 100644 --- a/.github/workflows/release-verify.yml +++ b/.github/workflows/release-verify.yml @@ -64,8 +64,16 @@ jobs: group_label: server (3/3) shard_index: 2 shard_count: 3 + # Keep parity with pr.yml: workspaces-a is split with Vitest's + # native --shard because the ui project dominates the lane. - group: general-workspaces-a - group_label: workspaces-a + group_label: workspaces-a (1/2) + shard_index: 0 + shard_count: 2 + - group: general-workspaces-a + group_label: workspaces-a (2/2) + shard_index: 1 + shard_count: 2 - group: general-workspaces-b group_label: workspaces-b diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index d46cf1fb594..3f0e1e98a66 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -4,10 +4,22 @@ on: push: branches: - master + schedule: + # Nightly cut at 09:00 UTC, after the workday's merges have settled. + - cron: "0 9 * * *" workflow_dispatch: inputs: + channel: + description: Release channel to publish + required: true + type: choice + options: + - stable + - beta + - nightly + default: stable source_ref: - description: Commit SHA, branch, or tag to publish as stable + description: (stable) Commit SHA, branch, or tag to publish as stable required: true type: string default: master @@ -15,8 +27,20 @@ on: description: Enter a UTC date in YYYY-MM-DD format, for example 2026-03-18. Do not enter a version string. The workflow will resolve that date to a stable version such as 2026.318.0, then 2026.318.1 for the next same-day stable. required: false type: string + source_version: + description: For nightly, the explicit canary version to promote (empty selects the newest canary on master). For beta, the explicit nightly version to promote (empty selects the newest nightly on master). + required: false + type: string + candidate_branch: + description: (beta) candidate/beta-* branch to build a cherry-picked beta from. Leave empty to promote a nightly. Mutually exclusive with source_version. + required: false + type: string + skip_soak_justification: + description: (stable) Written justification for publishing a stable whose source has not soaked as a beta for 3 days. Leave empty for normal releases. + required: false + type: string dry_run: - description: Preview the stable release without publishing + description: Preview the release without publishing required: true type: boolean default: false @@ -97,15 +121,761 @@ jobs: fi git push origin "refs/tags/${tag}" - verify_stable: - if: github.event_name == 'workflow_dispatch' + # ----- Nightly lane ----------------------------------------------------- + # Once a night (or on a forced nightly dispatch), promote the newest master + # commit that already shipped a green canary: smoke-test that exact + # published canary first, then republish the same commit under the nightly + # identity. The candidate commit already passed release-verify during its + # canary publish, so the nightly publish skips re-verification. + + select_nightly: + if: github.event_name == 'schedule' || (github.event_name == 'workflow_dispatch' && inputs.channel == 'nightly') + runs-on: ubuntu-latest + timeout-minutes: 10 + permissions: + contents: read + outputs: + proceed: ${{ steps.select.outputs.proceed }} + sha: ${{ steps.select.outputs.sha }} + canary_version: ${{ steps.select.outputs.canary_version }} + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + ref: master + fetch-depth: 0 + + - name: Select nightly candidate + id: select + env: + EXPLICIT_CANARY_VERSION: ${{ github.event_name == 'workflow_dispatch' && inputs.source_version || '' }} + run: | + set -euo pipefail + + git fetch origin --tags --prune --quiet + + skip() { + echo "proceed=false" >> "$GITHUB_OUTPUT" + { + echo "## Nightly skipped" + echo "" + echo "$1" + } >> "$GITHUB_STEP_SUMMARY" + echo "Nightly skipped: $1" + } + + if [ -n "${EXPLICIT_CANARY_VERSION:-}" ]; then + tag="canary/v${EXPLICIT_CANARY_VERSION}" + sha="$(git rev-list -n 1 "$tag" 2>/dev/null || true)" + if [ -z "$sha" ]; then + echo "Error: tag $tag does not exist." >&2 + exit 1 + fi + else + # Newest canary-tagged commit on master. Canary tags are pushed + # only after a successful canary publish, so tag presence is the + # green-publish signal. Walk master newest-first and stop at the + # first commit that carries a canary tag. + sha="$(grep -m1 -F \ + -f <(git for-each-ref 'refs/tags/canary/v*' --format='%(objectname)') \ + <(git rev-list origin/master -n 500) || true)" + + if [ -z "$sha" ]; then + skip "No canary/v* tag found on the last 500 commits of master." + exit 0 + fi + + tag="$(git tag --points-at "$sha" | grep '^canary/v' | sort -V | tail -1)" + fi + + canary_version="${tag#canary/v}" + + existing_nightly="$(git tag --points-at "$sha" | grep '^nightly/v' | head -1 || true)" + if [ -n "$existing_nightly" ]; then + skip "Candidate \`$sha\` (canary \`$canary_version\`) already shipped as \`$existing_nightly\`." + exit 0 + fi + + # Promotions run the release tooling of the source commit, so the + # source must already understand the nightly channel. (Literal match + # of release.sh's channel case arm; if that line is reformatted this + # fails closed and should be updated alongside it.) + if ! git show "${sha}:scripts/release.sh" | grep -qF 'canary|nightly'; then + echo "Error: source commit $sha predates nightly release tooling; promote a newer canary." >&2 + exit 1 + fi + + echo "proceed=true" >> "$GITHUB_OUTPUT" + echo "sha=$sha" >> "$GITHUB_OUTPUT" + echo "canary_version=$canary_version" >> "$GITHUB_OUTPUT" + { + echo "## Nightly candidate" + echo "" + echo "- Source SHA: \`$sha\`" + echo "- Source canary: \`$canary_version\`" + } >> "$GITHUB_STEP_SUMMARY" + + # Gate the promotion on the release smoke suite, run against the exact + # published canary artifact that would become tonight's nightly. Red smoke + # means no nightly tonight. Skipped for dry-run dispatches. + smoke_nightly: + needs: select_nightly + if: needs.select_nightly.outputs.proceed == 'true' && !(github.event_name == 'workflow_dispatch' && inputs.dry_run) + uses: ./.github/workflows/release-smoke.yml + with: + paperclip_version: ${{ needs.select_nightly.outputs.canary_version }} + artifact_name: nightly-release-smoke + + publish_nightly: + needs: [select_nightly, smoke_nightly] + # Publish when smoke passed, or when smoke was deliberately skipped by a + # dry-run dispatch (the publish itself is a dry-run in that case). + if: >- + !cancelled() && + needs.select_nightly.outputs.proceed == 'true' && + (needs.smoke_nightly.result == 'success' || + (needs.smoke_nightly.result == 'skipped' && github.event_name == 'workflow_dispatch' && inputs.dry_run)) + runs-on: ubuntu-latest + timeout-minutes: 45 + environment: npm-canary + # The workflow-level concurrency group is per event, so a forced dispatch + # nightly could otherwise overlap the scheduled one and race it to the + # same next -nightly.N version. Serialize actual nightly publishes across + # events here; release.sh additionally refuses to double-publish a commit + # that already carries a nightly tag. + concurrency: + group: release-publish-nightly + cancel-in-progress: false + permissions: + contents: write + id-token: write + actions: write + + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + ref: ${{ needs.select_nightly.outputs.sha }} + fetch-depth: 0 + + - name: Setup pnpm + uses: pnpm/action-setup@v6 + with: + version: 9.15.4 + + - name: Setup Node.js + uses: actions/setup-node@v7 + with: + node-version: 24 + cache: pnpm + + - name: Validate release package manifest + run: node ./scripts/release-package-map.mjs check + + - name: Install dependencies + run: pnpm install --no-frozen-lockfile + + - name: Restore tracked install-time changes + run: git checkout -- pnpm-lock.yaml + + - name: Configure git author + run: | + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + + - name: Publish nightly + env: + GITHUB_ACTIONS: "true" + run: | + args=(nightly --skip-verify) + if [ "${{ github.event_name == 'workflow_dispatch' && inputs.dry_run }}" = "true" ]; then + args+=(--dry-run) + fi + ./scripts/release.sh "${args[@]}" + + - name: Dump npm debug logs + if: failure() + run: | + shopt -s nullglob + for f in "$HOME"/.npm/_logs/*.log; do + echo "===== $f =====" + tail -n 300 "$f" | sed -E \ + -e 's#((authorization|_authToken|_auth|node_auth_token|npm_token)"?[[:space:]]*[:=][[:space:]]*"?)(Bearer[[:space:]]+)?[^",[:space:]]+#\1***REDACTED***#Ig' + done + + - name: Push nightly tag + if: ${{ !(github.event_name == 'workflow_dispatch' && inputs.dry_run) }} + run: | + tag="$(git tag --points-at HEAD | grep '^nightly/v' | head -1)" + if [ -z "$tag" ]; then + echo "Error: no nightly tag points at HEAD after release." >&2 + exit 1 + fi + if ! git push origin "refs/tags/${tag}"; then + sha="$(git rev-parse HEAD)" + { + echo "## Tag push rejected" + echo "" + echo "The npm publish succeeded, but pushing \`${tag}\` was rejected." + echo "This usually means the tagged commit modifies workflow files," + echo "which GITHUB_TOKEN may not reference when creating refs from" + echo "dispatch or scheduled runs. Recover with maintainer credentials:" + echo "" + echo '```' + echo "git tag ${tag} ${sha}" + echo "git push origin refs/tags/${tag}" + echo "gh workflow run docker.yml --ref refs/tags/${tag}" + echo '```' + } >> "$GITHUB_STEP_SUMMARY" + echo "::error::Tag push rejected; see the job summary for recovery commands." >&2 + exit 1 + fi + + # Tag pushes made with GITHUB_TOKEN do not fire docker.yml's tag + # trigger (GitHub suppresses workflow runs caused by GITHUB_TOKEN + # pushes), so dispatch the image build at the new tag explicitly. + - name: Build Docker images for the nightly tag + if: ${{ !(github.event_name == 'workflow_dispatch' && inputs.dry_run) }} + env: + GH_TOKEN: ${{ github.token }} + run: | + tag="$(git tag --points-at HEAD | grep '^nightly/v' | head -1)" + { + echo "## Nightly published" + echo "" + echo "- Source SHA: \`${{ needs.select_nightly.outputs.sha }}\`" + echo "- Source canary: \`${{ needs.select_nightly.outputs.canary_version }}\`" + echo "- Published nightly: \`${tag#nightly/v}\`" + echo "- Docker build dispatched at \`${tag}\`" + } >> "$GITHUB_STEP_SUMMARY" + gh workflow run docker.yml --ref "refs/tags/${tag}" --repo "$GITHUB_REPOSITORY" + + # ----- Beta lane -------------------------------------------------------- + # Beta is a manual, human-approved promotion of a nightly. Unlike the + # scheduled nightly lane, a beta dispatch is explicit operator intent, so + # selection problems fail the run loudly instead of skipping quietly. The + # publish runs behind the npm-beta environment, whose required reviewers + # are the approval gate. + + select_beta: + if: github.event_name == 'workflow_dispatch' && inputs.channel == 'beta' + runs-on: ubuntu-latest + timeout-minutes: 10 + permissions: + contents: read + outputs: + sha: ${{ steps.select.outputs.sha }} + nightly_version: ${{ steps.select.outputs.nightly_version }} + mode: ${{ steps.select.outputs.mode }} + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + ref: master + fetch-depth: 0 + + - name: Select beta candidate + id: select + env: + EXPLICIT_NIGHTLY_VERSION: ${{ inputs.source_version }} + CANDIDATE_BRANCH: ${{ inputs.candidate_branch }} + run: | + set -euo pipefail + + git fetch origin --tags --prune --quiet + + # Candidate mode: build a cherry-picked beta from a short-lived + # candidate branch instead of promoting a nightly. + if [ -n "${CANDIDATE_BRANCH:-}" ]; then + if [ -n "${EXPLICIT_NIGHTLY_VERSION:-}" ]; then + echo "Error: candidate_branch and source_version are mutually exclusive." >&2 + exit 1 + fi + case "$CANDIDATE_BRANCH" in + candidate/beta-*) ;; + *) + echo "Error: candidate branches must be named candidate/beta- (got: $CANDIDATE_BRANCH)." >&2 + exit 1 + ;; + esac + git fetch origin "$CANDIDATE_BRANCH" --quiet + sha="$(git rev-parse --verify "origin/${CANDIDATE_BRANCH}^{commit}" 2>/dev/null || true)" + if [ -z "$sha" ]; then + echo "Error: candidate branch $CANDIDATE_BRANCH does not exist on origin." >&2 + exit 1 + fi + + existing_beta="$(git tag --points-at "$sha" | grep '^beta/v' | head -1 || true)" + if [ -n "$existing_beta" ]; then + echo "Error: candidate head $sha already shipped as $existing_beta." >&2 + exit 1 + fi + + if ! git show "${sha}:scripts/release.sh" | grep -qF -- '--from-candidate'; then + echo "Error: candidate head $sha predates candidate-build release tooling; rebase the candidate onto a newer base." >&2 + exit 1 + fi + + merge_base="$(git merge-base origin/master "$sha")" + echo "mode=candidate" >> "$GITHUB_OUTPUT" + echo "sha=$sha" >> "$GITHUB_OUTPUT" + echo "nightly_version=" >> "$GITHUB_OUTPUT" + { + echo "## Beta candidate branch" + echo "" + echo "- Branch: \`$CANDIDATE_BRANCH\`" + echo "- Head: \`$sha\`" + echo "- Base (merge-base with master): \`$merge_base\`" + echo "- Cherry-picked commits:" + echo "" + echo '\`\`\`' + git log --oneline "${merge_base}..${sha}" + echo '\`\`\`' + } >> "$GITHUB_STEP_SUMMARY" + exit 0 + fi + + echo "mode=promote" >> "$GITHUB_OUTPUT" + + if [ -n "${EXPLICIT_NIGHTLY_VERSION:-}" ]; then + tag="nightly/v${EXPLICIT_NIGHTLY_VERSION}" + sha="$(git rev-list -n 1 "$tag" 2>/dev/null || true)" + if [ -z "$sha" ]; then + echo "Error: tag $tag does not exist." >&2 + exit 1 + fi + else + # Newest nightly-tagged commit on master. + sha="$(grep -m1 -F \ + -f <(git for-each-ref 'refs/tags/nightly/v*' --format='%(objectname)') \ + <(git rev-list origin/master -n 2000) || true)" + + if [ -z "$sha" ]; then + echo "Error: no nightly/v* tag found on the last 2000 commits of master. Publish a nightly first, or pass source_version." >&2 + exit 1 + fi + + tag="$(git tag --points-at "$sha" | grep '^nightly/v' | sort -V | tail -1)" + fi + + nightly_version="${tag#nightly/v}" + + existing_beta="$(git tag --points-at "$sha" | grep '^beta/v' | head -1 || true)" + if [ -n "$existing_beta" ]; then + echo "Error: candidate $sha (nightly $nightly_version) already shipped as $existing_beta." >&2 + exit 1 + fi + + # Promotions run the release tooling of the source commit, so the + # source must already understand the beta channel. (Literal match of + # release.sh's channel case arm; if that line is reformatted this + # fails closed and should be updated alongside it.) + if ! git show "${sha}:scripts/release.sh" | grep -qF 'canary|nightly|beta|stable)'; then + echo "Error: source commit $sha predates beta release tooling; promote a newer nightly whose source contains the beta channel." >&2 + exit 1 + fi + + echo "sha=$sha" >> "$GITHUB_OUTPUT" + echo "nightly_version=$nightly_version" >> "$GITHUB_OUTPUT" + { + echo "## Beta candidate" + echo "" + echo "- Source SHA: \`$sha\`" + echo "- Source nightly: \`$nightly_version\`" + } >> "$GITHUB_STEP_SUMMARY" + + # Candidate-branch heads are new commits that never went through a canary + # or nightly, so they must pass full verification before publishing. + # Promoted nightlies were already verified by their canary run and skip it. + verify_beta_candidate: + needs: select_beta + if: needs.select_beta.outputs.mode == 'candidate' uses: ./.github/workflows/release-verify.yml with: - ref: ${{ inputs.source_ref }} + ref: ${{ needs.select_beta.outputs.sha }} + + publish_beta: + needs: [select_beta, verify_beta_candidate] + if: >- + !cancelled() && + needs.select_beta.result == 'success' && + (needs.verify_beta_candidate.result == 'success' || + (needs.verify_beta_candidate.result == 'skipped' && needs.select_beta.outputs.mode == 'promote')) + runs-on: ubuntu-latest + timeout-minutes: 45 + environment: npm-beta + # Serialize beta publishes so two dispatches cannot race to the same next + # -beta.N version; release.sh additionally refuses to double-publish a + # commit that already carries a beta tag. + concurrency: + group: release-publish-beta + cancel-in-progress: false + permissions: + contents: write + id-token: write + actions: write + outputs: + beta_version: ${{ steps.result.outputs.beta_version }} + + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + ref: ${{ needs.select_beta.outputs.sha }} + fetch-depth: 0 + + - name: Setup pnpm + uses: pnpm/action-setup@v6 + with: + version: 9.15.4 + + - name: Setup Node.js + uses: actions/setup-node@v7 + with: + node-version: 24 + cache: pnpm + + - name: Validate release package manifest + run: node ./scripts/release-package-map.mjs check + + - name: Install dependencies + run: pnpm install --no-frozen-lockfile + + - name: Restore tracked install-time changes + run: git checkout -- pnpm-lock.yaml + + - name: Configure git author + run: | + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + + - name: Publish beta + env: + GITHUB_ACTIONS: "true" + run: | + args=(beta --skip-verify) + if [ "${{ needs.select_beta.outputs.mode }}" = "candidate" ]; then + args+=(--from-candidate) + fi + if [ "${{ inputs.dry_run }}" = "true" ]; then + args+=(--dry-run) + fi + ./scripts/release.sh "${args[@]}" + + - name: Dump npm debug logs + if: failure() + run: | + shopt -s nullglob + for f in "$HOME"/.npm/_logs/*.log; do + echo "===== $f =====" + tail -n 300 "$f" | sed -E \ + -e 's#((authorization|_authToken|_auth|node_auth_token|npm_token)"?[[:space:]]*[:=][[:space:]]*"?)(Bearer[[:space:]]+)?[^",[:space:]]+#\1***REDACTED***#Ig' + done + + - name: Push beta tag + if: ${{ !inputs.dry_run }} + run: | + tag="$(git tag --points-at HEAD | grep '^beta/v' | head -1)" + if [ -z "$tag" ]; then + echo "Error: no beta tag points at HEAD after release." >&2 + exit 1 + fi + if ! git push origin "refs/tags/${tag}"; then + sha="$(git rev-parse HEAD)" + { + echo "## Tag push rejected" + echo "" + echo "The npm publish succeeded, but pushing \`${tag}\` was rejected." + echo "This usually means the tagged commit modifies workflow files," + echo "which GITHUB_TOKEN may not reference when creating refs from" + echo "dispatch or scheduled runs. Recover with maintainer credentials:" + echo "" + echo '```' + echo "git tag ${tag} ${sha}" + echo "git push origin refs/tags/${tag}" + echo "gh workflow run docker.yml --ref refs/tags/${tag}" + echo '```' + } >> "$GITHUB_STEP_SUMMARY" + echo "::error::Tag push rejected; see the job summary for recovery commands." >&2 + exit 1 + fi + + # Tag pushes made with GITHUB_TOKEN do not fire docker.yml's tag + # trigger (GitHub suppresses workflow runs caused by GITHUB_TOKEN + # pushes), so dispatch the image build at the new tag explicitly. + - name: Build Docker images for the beta tag + id: result + if: ${{ !inputs.dry_run }} + env: + GH_TOKEN: ${{ github.token }} + run: | + tag="$(git tag --points-at HEAD | grep '^beta/v' | head -1)" + echo "beta_version=${tag#beta/v}" >> "$GITHUB_OUTPUT" + { + echo "## Beta published" + echo "" + echo "- Source SHA: \`${{ needs.select_beta.outputs.sha }}\`" + echo "- Source nightly: \`${{ needs.select_beta.outputs.nightly_version }}\`" + echo "- Published beta: \`${tag#beta/v}\`" + echo "- Docker build dispatched at \`${tag}\`" + } >> "$GITHUB_STEP_SUMMARY" + gh workflow run docker.yml --ref "refs/tags/${tag}" --repo "$GITHUB_REPOSITORY" + + # Draft the eventual stable's release notes the moment the beta exists: + # the promoted bits are frozen now, and the 3-day soak is the natural + # review window. The draft lands on a machine-owned branch; a human opens + # and merges the PR, because a PR created with GITHUB_TOKEN would not + # trigger the pr.yml checks a merge requires. + draft_stable_notes: + needs: [select_beta, publish_beta] + if: ${{ !cancelled() && !inputs.dry_run && needs.publish_beta.result == 'success' }} + runs-on: ubuntu-latest + timeout-minutes: 10 + permissions: + contents: write + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + ref: master + fetch-depth: 0 + + - name: Configure git author + run: | + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + + - name: Draft stable notes from the published beta + env: + BETA_VERSION: ${{ needs.publish_beta.outputs.beta_version }} + SOURCE_SHA: ${{ needs.select_beta.outputs.sha }} + run: | + set -euo pipefail + git fetch origin --tags --quiet + # A rejected beta tag push (the workflows-permission case) leaves + # the tag absent from origin while npm already has the beta; + # recreate it locally so drafting does not block on the manual + # tag recovery. + if ! git rev-parse --verify "refs/tags/beta/v${BETA_VERSION}" >/dev/null 2>&1; then + git tag "beta/v${BETA_VERSION}" "${SOURCE_SHA}" + fi + ./scripts/draft-stable-notes.sh "${BETA_VERSION}" + + - name: Push the draft branch + env: + BETA_VERSION: ${{ needs.publish_beta.outputs.beta_version }} + run: | + set -euo pipefail + branch="release-notes/v${BETA_VERSION}" + git checkout -B "$branch" + git add "releases/beta/v${BETA_VERSION}.md" + git commit -m "docs(release): draft stable notes for beta ${BETA_VERSION}" + # Machine-owned branch: force push so a re-run regenerates cleanly. + git push -f origin "$branch" + { + echo "## Stable notes draft pushed" + echo "" + echo "- Branch: \`${branch}\`" + echo "- Open the PR (a human opens it so CI runs):" + echo " https://github.com/${GITHUB_REPOSITORY}/compare/master...${branch}?expand=1" + echo "- Edit it during the soak. The stable promotion reads" + echo " \`releases/beta/v${BETA_VERSION}.md\` from master." + } >> "$GITHUB_STEP_SUMMARY" + + # Post-publish verification: run the release smoke suite against the exact + # beta version that was just published. + # + # The condition must carry an explicit status-check function: without one, + # GitHub attaches an implicit success(), which evaluates the needs chain + # transitively — and publish_beta's chain contains verify_beta_candidate, + # which is skipped on every promote-mode beta. The implicit form silently + # skipped this job on the first promote-mode beta after the candidate + # lane landed. + smoke_beta: + needs: publish_beta + if: ${{ !cancelled() && needs.publish_beta.result == 'success' && !inputs.dry_run }} + uses: ./.github/workflows/release-smoke.yml + with: + paperclip_version: ${{ needs.publish_beta.outputs.beta_version }} + artifact_name: beta-release-smoke + + # ----- Stable lane ------------------------------------------------------ + + # Stable releases promote a soaked beta. The preflight enforces that the + # source commit shipped as a beta at least 3 days ago (measured from the + # npm publish time of that beta version), unless a written justification + # is provided. Dry runs report soak state without blocking. + # Resolves source_ref to an immutable commit exactly once; every downstream + # stable job consumes that SHA. Otherwise a branch or movable tag that + # advances mid-run could be soak-checked at one commit and verified or + # published at another. + preflight_stable: + if: github.event_name == 'workflow_dispatch' && inputs.channel == 'stable' + runs-on: ubuntu-latest + timeout-minutes: 10 + permissions: + contents: read + outputs: + sha: ${{ steps.soak.outputs.sha }} + beta_version: ${{ steps.soak.outputs.beta_version }} + stable_version: ${{ steps.notes.outputs.stable_version }} + notes_mode: ${{ steps.notes.outputs.notes_mode }} + notes_path: ${{ steps.notes.outputs.notes_path }} + notes_ref: ${{ steps.notes.outputs.notes_ref }} + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + ref: master + fetch-depth: 0 + + - name: Check beta soak + id: soak + env: + SOURCE_REF: ${{ inputs.source_ref }} + JUSTIFICATION: ${{ inputs.skip_soak_justification }} + DRY_RUN: ${{ inputs.dry_run }} + run: | + set -euo pipefail + + git fetch origin --tags --prune --quiet + + sha="$(git rev-parse --verify "${SOURCE_REF}^{commit}" 2>/dev/null || true)" + if [ -z "$sha" ]; then + git fetch origin "$SOURCE_REF" --quiet || true + sha="$(git rev-parse --verify "FETCH_HEAD^{commit}" 2>/dev/null || true)" + fi + if [ -z "$sha" ]; then + echo "Error: could not resolve source_ref '$SOURCE_REF' to a commit." >&2 + exit 1 + fi + + echo "sha=$sha" >> "$GITHUB_OUTPUT" + { + echo "## Stable source pinned" + echo "" + echo "- source_ref: \`${SOURCE_REF}\` -> \`$sha\`" + } >> "$GITHUB_STEP_SUMMARY" + + fail_or_justify() { + if [ -n "${JUSTIFICATION:-}" ]; then + { + echo "## Stable soak gate bypassed" + echo "" + echo "$1" + echo "" + echo "Justification: ${JUSTIFICATION}" + } >> "$GITHUB_STEP_SUMMARY" + echo "::warning::Soak gate bypassed: $1" + return 0 + fi + if [ "${DRY_RUN}" = "true" ]; then + echo "::warning::Soak gate would block a real release: $1" + { + echo "## Stable soak gate (dry run)" + echo "" + echo "A real release would be blocked: $1" + } >> "$GITHUB_STEP_SUMMARY" + return 0 + fi + echo "Error: $1" >&2 + echo "Pass skip_soak_justification with a written reason to release anyway." >&2 + exit 1 + } + + beta_tag="$(git tag --points-at "$sha" | grep '^beta/v' | sort -V | tail -1 || true)" + if [ -z "$beta_tag" ]; then + fail_or_justify "source commit $sha never shipped as a beta (no beta/v* tag)." + exit 0 + fi + + beta_version="${beta_tag#beta/v}" + echo "beta_version=${beta_version}" >> "$GITHUB_OUTPUT" + publish_time="$(npm view "paperclipai@${beta_version}" time --json 2>/dev/null \ + | node -e 'let d="";process.stdin.on("data",c=>d+=c).on("end",()=>{const t=JSON.parse(d);process.stdout.write(typeof t === "string" ? t : (t[process.argv[1]] ?? ""))})' "$beta_version" || true)" + if [ -z "$publish_time" ]; then + fail_or_justify "could not determine the npm publish time of beta ${beta_version}." + exit 0 + fi + + age_seconds="$(node -e 'process.stdout.write(String(Math.floor((Date.now() - Date.parse(process.argv[1])) / 1000)))' "$publish_time")" + min_seconds=$((3 * 24 * 60 * 60)) + age_days="$(node -e 'process.stdout.write((Number(process.argv[1]) / 86400).toFixed(1))' "$age_seconds")" + + if [ "$age_seconds" -lt "$min_seconds" ]; then + fail_or_justify "beta ${beta_version} has only soaked ${age_days} days (minimum is 3)." + exit 0 + fi + + { + echo "## Stable soak gate passed" + echo "" + echo "- Source beta: \`${beta_version}\`" + echo "- Soak time: ${age_days} days" + } >> "$GITHUB_STEP_SUMMARY" + + # The stable notes need not exist inside the promoted source commit: + # a promoted beta's notes are drafted on master (releases/beta/v*.md) + # at beta-publish time and edited during the soak. Resolve which copy + # publish_stable should read, and fail early — before the npm-stable + # approval gate — when none exists. Notes inside the source tree + # (the candidate fix path) take precedence. + - name: Resolve stable release notes + id: notes + env: + SHA: ${{ steps.soak.outputs.sha }} + BETA_VERSION: ${{ steps.soak.outputs.beta_version }} + STABLE_DATE: ${{ inputs.stable_date }} + DRY_RUN: ${{ inputs.dry_run }} + run: | + set -euo pipefail + + # Pin the master revision the notes were resolved at, so the + # publish and canonicalization steps read the same content even + # when master advances during the approval delay. + echo "notes_ref=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT" + + args=(stable --print-version) + if [ -n "${STABLE_DATE}" ]; then + args+=(--date "${STABLE_DATE}") + fi + version="$(./scripts/release.sh "${args[@]}")" + echo "stable_version=${version}" >> "$GITHUB_OUTPUT" + + if git cat-file -e "${SHA}:releases/v${version}.md" 2>/dev/null; then + echo "notes_mode=source_tree" >> "$GITHUB_OUTPUT" + echo "notes_path=releases/v${version}.md" >> "$GITHUB_OUTPUT" + echo "- Stable notes: \`releases/v${version}.md\` at the source commit" >> "$GITHUB_STEP_SUMMARY" + exit 0 + fi + + if [ -n "${BETA_VERSION}" ] && [ -f "releases/beta/v${BETA_VERSION}.md" ]; then + echo "notes_mode=master_beta" >> "$GITHUB_OUTPUT" + echo "notes_path=releases/beta/v${BETA_VERSION}.md" >> "$GITHUB_OUTPUT" + echo "- Stable notes: \`releases/beta/v${BETA_VERSION}.md\` on master" >> "$GITHUB_STEP_SUMMARY" + exit 0 + fi + + msg="no release notes found for stable ${version}: neither releases/v${version}.md at the source commit nor releases/beta/v${BETA_VERSION:-}.md on master. Merge the notes PR from the beta's draft branch (release-notes/v), or add the file to the source ref." + if [ "${DRY_RUN}" = "true" ]; then + echo "::warning::${msg}" + echo "- Stable notes: MISSING (a real release would be blocked)" >> "$GITHUB_STEP_SUMMARY" + echo "notes_mode=missing" >> "$GITHUB_OUTPUT" + exit 0 + fi + echo "::error::${msg}" + exit 1 + + verify_stable: + if: github.event_name == 'workflow_dispatch' && inputs.channel == 'stable' + needs: preflight_stable + uses: ./.github/workflows/release-verify.yml + with: + ref: ${{ needs.preflight_stable.outputs.sha }} preview_stable: - if: github.event_name == 'workflow_dispatch' && inputs.dry_run - needs: verify_stable + if: github.event_name == 'workflow_dispatch' && inputs.channel == 'stable' && inputs.dry_run + needs: [preflight_stable, verify_stable] runs-on: ubuntu-latest timeout-minutes: 45 permissions: @@ -116,7 +886,7 @@ jobs: uses: actions/checkout@v7 with: fetch-depth: 0 - ref: ${{ inputs.source_ref }} + ref: ${{ needs.preflight_stable.outputs.sha }} - name: Setup pnpm uses: pnpm/action-setup@v6 @@ -146,21 +916,24 @@ jobs: ./scripts/release.sh "${args[@]}" publish_stable: - if: github.event_name == 'workflow_dispatch' && !inputs.dry_run - needs: verify_stable + if: github.event_name == 'workflow_dispatch' && inputs.channel == 'stable' && !inputs.dry_run + needs: [preflight_stable, verify_stable] runs-on: ubuntu-latest timeout-minutes: 45 environment: npm-stable permissions: contents: write id-token: write + actions: write + outputs: + stable_version: ${{ steps.tag.outputs.version }} steps: - name: Checkout repository uses: actions/checkout@v7 with: fetch-depth: 0 - ref: ${{ inputs.source_ref }} + ref: ${{ needs.preflight_stable.outputs.sha }} - name: Setup pnpm uses: pnpm/action-setup@v6 @@ -184,6 +957,22 @@ jobs: git config user.name "github-actions[bot]" git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + # A promoted beta's notes live on master (drafted at beta publish, + # edited during the soak), not inside the promoted source commit — a + # commit cannot carry a file named for a promotion date that was + # unknown when it was created. Materialize them outside the worktree + # so release.sh still runs against a clean source checkout. + - name: Materialize stable release notes from master + if: needs.preflight_stable.outputs.notes_mode == 'master_beta' + env: + NOTES_REF: ${{ needs.preflight_stable.outputs.notes_ref }} + NOTES_PATH: ${{ needs.preflight_stable.outputs.notes_path }} + run: | + set -euo pipefail + git fetch origin master --quiet + git show "${NOTES_REF}:${NOTES_PATH}" > "$RUNNER_TEMP/stable-notes.md" + echo "STABLE_NOTES_FILE=$RUNNER_TEMP/stable-notes.md" >> "$GITHUB_ENV" + - name: Publish stable env: GITHUB_ACTIONS: "true" @@ -192,6 +981,9 @@ jobs: if [ -n "${{ inputs.stable_date }}" ]; then args+=(--date "${{ inputs.stable_date }}") fi + if [ -n "${STABLE_NOTES_FILE:-}" ]; then + args+=(--notes-file "$STABLE_NOTES_FILE") + fi ./scripts/release.sh "${args[@]}" - name: Dump npm debug logs @@ -205,13 +997,53 @@ jobs: done - name: Push stable tag + id: tag run: | tag="$(git tag --points-at HEAD | grep '^v' | head -1)" if [ -z "$tag" ]; then echo "Error: no stable tag points at HEAD after release." >&2 exit 1 fi - git push origin "refs/tags/${tag}" + echo "version=${tag#v}" >> "$GITHUB_OUTPUT" + if ! git push origin "refs/tags/${tag}"; then + sha="$(git rev-parse HEAD)" + { + echo "## Tag push rejected" + echo "" + echo "The npm publish succeeded, but pushing \`${tag}\` was rejected." + echo "This usually means the tagged commit modifies workflow files," + echo "which GITHUB_TOKEN may not reference when creating refs from" + echo "dispatch or scheduled runs. Recover with maintainer credentials:" + echo "" + echo '```' + echo "git tag ${tag} ${sha}" + echo "git push origin refs/tags/${tag}" + echo "gh workflow run docker.yml --ref refs/tags/${tag}" + echo "./scripts/create-github-release.sh ${tag#v}" + echo '```' + } >> "$GITHUB_STEP_SUMMARY" + echo "::error::Tag push rejected; see the job summary for recovery commands." >&2 + exit 1 + fi + + # Tag pushes made with GITHUB_TOKEN do not fire docker.yml's tag + # trigger (GitHub suppresses workflow runs caused by GITHUB_TOKEN + # pushes), so dispatch the image build at the new tag explicitly. This + # is what moves Docker `:latest` and publishes the versioned stable + # image tags. + - name: Build Docker images for the stable tag + env: + GH_TOKEN: ${{ github.token }} + run: | + tag="$(git tag --points-at HEAD | grep '^v' | head -1)" + if gh workflow run docker.yml --ref "refs/tags/${tag}" --repo "$GITHUB_REPOSITORY"; then + echo "Dispatched docker.yml at ${tag}." + else + # Older source commits may predate docker.yml's workflow_dispatch + # trigger; the dispatch then fails while the npm release is + # already complete and correct. + echo "::warning::Could not dispatch docker.yml at ${tag}. Run docker.yml manually at that tag to publish the stable images." + fi - name: Create GitHub Release env: @@ -223,4 +1055,59 @@ jobs: echo "Error: no v* tag points at HEAD after stable release." >&2 exit 1 fi - ./scripts/create-github-release.sh "$version" + args=("$version") + if [ -n "${STABLE_NOTES_FILE:-}" ]; then + args+=(--notes-file "$STABLE_NOTES_FILE") + fi + ./scripts/create-github-release.sh "${args[@]}" + + # After a stable ships from master-side beta notes, move the file to its + # canonical home (releases/vYYYY.MDD.P.md) so the stable-notes invariant + # holds durably. Machine-owned branch + human-opened PR, as with the + # draft job above. + canonicalize_stable_notes: + needs: [preflight_stable, publish_stable] + if: ${{ needs.preflight_stable.outputs.notes_mode == 'master_beta' && needs.publish_stable.result == 'success' }} + runs-on: ubuntu-latest + timeout-minutes: 10 + permissions: + contents: write + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + ref: master + fetch-depth: 0 + + - name: Configure git author + run: | + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + + - name: Push the canonicalization branch + env: + STABLE_VERSION: ${{ needs.publish_stable.outputs.stable_version }} + NOTES_PATH: ${{ needs.preflight_stable.outputs.notes_path }} + NOTES_REF: ${{ needs.preflight_stable.outputs.notes_ref }} + run: | + set -euo pipefail + branch="release-notes/v${STABLE_VERSION}-canonicalize" + git checkout -B "$branch" + # Canonicalize exactly what shipped: take the notes at the pinned + # revision the release read them from, so a master edit made + # during the run surfaces as a reviewable diff in this PR instead + # of silently diverging from the published GitHub Release. + git show "${NOTES_REF}:${NOTES_PATH}" > "releases/v${STABLE_VERSION}.md" + git add "releases/v${STABLE_VERSION}.md" + if git ls-files --error-unmatch "${NOTES_PATH}" >/dev/null 2>&1; then + git rm -q "${NOTES_PATH}" + fi + git commit -m "docs(release): canonicalize stable notes for v${STABLE_VERSION}" + git push -f origin "$branch" + { + echo "## Stable notes canonicalization pushed" + echo "" + echo "- Branch: \`${branch}\`" + echo "- Open and merge the PR (a human opens it so CI runs):" + echo " https://github.com/${GITHUB_REPOSITORY}/compare/master...${branch}?expand=1" + } >> "$GITHUB_STEP_SUMMARY" diff --git a/.gitignore b/.gitignore index b09d5225c24..b47b07530b9 100644 --- a/.gitignore +++ b/.gitignore @@ -61,6 +61,7 @@ tests/e2e/test-results/ tests/e2e/playwright-report/ tests/release-smoke/test-results/ tests/release-smoke/playwright-report/ +test-results/issue-detail-perf/ tests/storybook-visual/.cache/ tests/storybook-visual/.snapshots/ tests/storybook-visual/baseline-review/ diff --git a/Dockerfile b/Dockerfile index 2769078b668..a449c77942b 100644 --- a/Dockerfile +++ b/Dockerfile @@ -25,6 +25,7 @@ COPY packages/google-sheets-mcp-server/package.json packages/google-sheets-mcp-s COPY packages/kv-demo-mcp-server/package.json packages/kv-demo-mcp-server/ COPY packages/mcp-server/package.json packages/mcp-server/ COPY packages/skills-catalog/package.json packages/skills-catalog/ +COPY packages/tailscale-https-broker/package.json packages/tailscale-https-broker/ COPY packages/teams-catalog/package.json packages/teams-catalog/ COPY packages/adapters/claude-local/package.json packages/adapters/claude-local/ COPY packages/adapters/codex-local/package.json packages/adapters/codex-local/ diff --git a/cli/src/__tests__/channels.test.ts b/cli/src/__tests__/channels.test.ts new file mode 100644 index 00000000000..97c0ad62220 --- /dev/null +++ b/cli/src/__tests__/channels.test.ts @@ -0,0 +1,81 @@ +import { describe, expect, it } from "vitest"; +import { + channelForVersion, + collectChannelState, + RELEASE_CHANNELS, +} from "../commands/channels.js"; +import type { CommandRunner } from "../commands/install.js"; + +describe("channelForVersion", () => { + it("maps published versions to their lane", () => { + expect(channelForVersion("2026.811.0")).toBe("stable"); + expect(channelForVersion("2026.811.0-beta.0")).toBe("beta"); + expect(channelForVersion("2026.811.0-nightly.0")).toBe("nightly"); + expect(channelForVersion("2026.811.0-canary.3")).toBe("canary"); + }); + + it("treats non-CalVer versions as unknown instead of guessing", () => { + expect(channelForVersion("0.3.1")).toBe("unknown"); + expect(channelForVersion("2026.811.0-rc.1")).toBe("unknown"); + expect(channelForVersion("garbage")).toBe("unknown"); + }); +}); + +describe("collectChannelState", () => { + const versionsByTag: Record = { + latest: "2026.722.0", + beta: "2026.811.0-beta.0", + nightly: "2026.811.0-nightly.0", + canary: "2026.811.0-canary.3", + }; + + const fakeRunner: CommandRunner = async (_command, args) => { + const spec = (args ?? []).find((arg) => arg.startsWith("paperclipai@")); + const tag = spec?.slice("paperclipai@".length) ?? ""; + const version = versionsByTag[tag]; + if (!version) throw new Error(`unexpected dist-tag: ${tag}`); + return { stdout: JSON.stringify(version), stderr: "" }; + }; + + it("resolves every channel's dist-tag from the registry", async () => { + const state = await collectChannelState(fakeRunner); + + expect(state.map((entry) => entry.channel)).toEqual([ + "stable", + "beta", + "nightly", + "canary", + ]); + expect(state.map((entry) => entry.version)).toEqual([ + "2026.722.0", + "2026.811.0-beta.0", + "2026.811.0-nightly.0", + "2026.811.0-canary.3", + ]); + }); + + it("degrades a single unavailable channel to null without failing the rest", async () => { + const flakyRunner: CommandRunner = async (command, args, options) => { + const spec = (args ?? []).find((arg) => arg.startsWith("paperclipai@")); + if (spec === "paperclipai@nightly") throw new Error("registry timeout"); + return fakeRunner(command, args, options); + }; + + const state = await collectChannelState(flakyRunner); + const byChannel = Object.fromEntries(state.map((entry) => [entry.channel, entry.version])); + + expect(byChannel.nightly).toBeNull(); + expect(byChannel.stable).toBe("2026.722.0"); + expect(byChannel.beta).toBe("2026.811.0-beta.0"); + expect(byChannel.canary).toBe("2026.811.0-canary.3"); + }); + + it("keeps the channel table and dist-tags in sync", () => { + expect(RELEASE_CHANNELS.map((entry) => entry.distTag)).toEqual([ + "latest", + "beta", + "nightly", + "canary", + ]); + }); +}); diff --git a/cli/src/__tests__/company-import-transfer.test.ts b/cli/src/__tests__/company-import-transfer.test.ts new file mode 100644 index 00000000000..c8b560849d5 --- /dev/null +++ b/cli/src/__tests__/company-import-transfer.test.ts @@ -0,0 +1,494 @@ +import { createHash } from "node:crypto"; +import { deflateRawSync } from "node:zlib"; +import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { Command } from "commander"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { readZipArchive } from "@paperclipai/shared/portability-zip"; +import { + CHUNKED_IMPORT_THRESHOLD_BYTES, + IMPORT_TRANSFER_PART_SIZE_BYTES, + buildImportTransferManifest, + registerCompanyCommands, + EXISTING_COMPANY_CHUNKED_IMPORT_THRESHOLD_BYTES, + resolveChunkedImportZip, + uploadCompanyImportTransfer, +} from "../commands/client/company.js"; +import { createStoredZipArchive } from "./helpers/zip.js"; + +const ORIGINAL_ENV = { ...process.env }; + +const tempDirs: string[] = []; + +async function makeTempDir(): Promise { + const dir = await mkdtemp(path.join(os.tmpdir(), "paperclip-company-import-transfer-")); + tempDirs.push(dir); + return dir; +} + +function sha256Hex(bytes: Uint8Array): string { + return createHash("sha256").update(bytes).digest("hex"); +} + +/** Two-part fixture: one full 32 MB part plus a short tail. */ +function buildTwoPartZipBytes(): Uint8Array { + const bytes = new Uint8Array(IMPORT_TRANSFER_PART_SIZE_BYTES + 3); + bytes.set([1, 2, 3], IMPORT_TRANSFER_PART_SIZE_BYTES); + return bytes; +} + +afterEach(async () => { + for (const dir of tempDirs.splice(0)) { + await rm(dir, { recursive: true, force: true }); + } +}); + +describe("buildImportTransferManifest", () => { + it("declares the whole zip and its 32 MB byte-range parts with content hashes", () => { + const zipBytes = buildTwoPartZipBytes(); + const manifest = buildImportTransferManifest(zipBytes); + + expect(manifest.totalBytes).toBe(zipBytes.length); + expect(manifest.partSizeBytes).toBe(IMPORT_TRANSFER_PART_SIZE_BYTES); + expect(manifest.zipSha256).toBe(sha256Hex(zipBytes)); + expect(manifest.parts).toEqual([ + { + index: 0, + byteSize: IMPORT_TRANSFER_PART_SIZE_BYTES, + sha256: sha256Hex(zipBytes.subarray(0, IMPORT_TRANSFER_PART_SIZE_BYTES)), + }, + { + index: 1, + byteSize: 3, + sha256: sha256Hex(zipBytes.subarray(IMPORT_TRANSFER_PART_SIZE_BYTES)), + }, + ]); + }); +}); + +// Minimal single-entry DEFLATE zip, byte-compatible with the shared reader — +// the stored-zip helper cannot model a small-compressed/large-inflated entry. +function buildDeflateZip(entryPath: string, text: string): Uint8Array { + const raw = Buffer.from(text, "utf8"); + const body = deflateRawSync(raw); + const name = Buffer.from(entryPath, "utf8"); + let crc = 0xffffffff; + for (const byte of raw) { + crc ^= byte; + for (let bit = 0; bit < 8; bit += 1) crc = (crc & 1) === 1 ? (crc >>> 1) ^ 0xedb88320 : crc >>> 1; + } + crc = (crc ^ 0xffffffff) >>> 0; + const local = Buffer.alloc(30 + name.length); + local.writeUInt32LE(0x04034b50, 0); + local.writeUInt16LE(20, 4); + local.writeUInt16LE(0x0800, 6); + local.writeUInt16LE(8, 8); + local.writeUInt32LE(crc, 14); + local.writeUInt32LE(body.length, 18); + local.writeUInt32LE(raw.length, 22); + local.writeUInt16LE(name.length, 26); + name.copy(local, 30); + const central = Buffer.alloc(46 + name.length); + central.writeUInt32LE(0x02014b50, 0); + central.writeUInt16LE(20, 4); + central.writeUInt16LE(20, 6); + central.writeUInt16LE(0x0800, 8); + central.writeUInt16LE(8, 10); + central.writeUInt32LE(crc, 16); + central.writeUInt32LE(body.length, 20); + central.writeUInt32LE(raw.length, 24); + central.writeUInt16LE(name.length, 28); + central.writeUInt32LE(0, 42); + name.copy(central, 46); + const eocd = Buffer.alloc(22); + eocd.writeUInt32LE(0x06054b50, 0); + eocd.writeUInt16LE(1, 8); + eocd.writeUInt16LE(1, 10); + eocd.writeUInt32LE(central.length, 12); + eocd.writeUInt32LE(local.length + body.length, 16); + return new Uint8Array(Buffer.concat([local, body, central, eocd])); +} + +describe("resolveChunkedImportZip", () => { + + it("returns null for a zip at or under the threshold", async () => { + const dir = await makeTempDir(); + const zipPath = path.join(dir, "small.zip"); + await writeFile(zipPath, Buffer.alloc(1024)); + + expect(await resolveChunkedImportZip(zipPath)).toBeNull(); + }); + + it("takes the chunked path for a small zip whose entries inflate past the threshold", async () => { + const dir = await makeTempDir(); + const zipPath = path.join(dir, "dense-package.zip"); + // ~64 MB of repetitive text DEFLATEs to a tiny file: far under the raw + // 48 MB threshold, but the inline body would carry the inflated entries, + // so the estimated request size sends the zip down the chunked path. + const zipBytes = buildDeflateZip("dense-package/NOTES.md", "paperclip agent docs\n".repeat(3_200_000)); + await writeFile(zipPath, zipBytes); + + const resolved = await resolveChunkedImportZip(zipPath); + expect(resolved).not.toBeNull(); + expect(resolved!.rootPath).toBe("dense-package"); + expect(sha256Hex(resolved!.zipBytes)).toBe(sha256Hex(zipBytes)); + }); + + it("uses the lower existing-company threshold for the chunk decision", async () => { + const dir = await makeTempDir(); + const packageDir = path.join(dir, "midsize-package"); + await mkdir(path.join(packageDir, "blobs"), { recursive: true }); + await writeFile(path.join(packageDir, "COMPANY.md"), "# Company\n"); + // ~9 MB of blob bytes: estimated inline ~12 MB — fine for the generic + // import path (64 MB parser) but over the existing-company path's + // default 10 MB parser, so only the existing-target threshold chunks it. + await writeFile(path.join(packageDir, "blobs", "1a2b3c4d"), Buffer.alloc(9 * 1024 * 1024, 5)); + + expect(await resolveChunkedImportZip(packageDir)).toBeNull(); + const chunked = await resolveChunkedImportZip( + packageDir, + EXISTING_COMPANY_CHUNKED_IMPORT_THRESHOLD_BYTES, + ); + expect(chunked).not.toBeNull(); + expect(chunked!.rootPath).toBe("midsize-package"); + }); + + it("keeps a small zip inline when its entries stay under the estimated threshold", async () => { + const dir = await makeTempDir(); + const zipPath = path.join(dir, "modest-package.zip"); + await writeFile(zipPath, buildDeflateZip("modest-package/COMPANY.md", "# Company\n")); + + expect(await resolveChunkedImportZip(zipPath)).toBeNull(); + }); + + it("reads an oversized zip file as-is so its declared hashes match the file on disk", async () => { + const dir = await makeTempDir(); + const zipPath = path.join(dir, "big-package.zip"); + const zipBytes = Buffer.alloc(CHUNKED_IMPORT_THRESHOLD_BYTES + 1024, 7); + await writeFile(zipPath, zipBytes); + + const resolved = await resolveChunkedImportZip(zipPath); + expect(resolved).not.toBeNull(); + expect(resolved!.rootPath).toBe("big-package"); + expect(resolved!.zipBytes.length).toBe(zipBytes.length); + expect(sha256Hex(resolved!.zipBytes)).toBe(sha256Hex(zipBytes)); + }); + + it("returns null for a folder whose portable content is under the threshold", async () => { + const dir = await makeTempDir(); + const packageDir = path.join(dir, "small-package"); + await mkdir(packageDir, { recursive: true }); + await writeFile(path.join(packageDir, "COMPANY.md"), "# Company\n"); + + expect(await resolveChunkedImportZip(packageDir)).toBeNull(); + }); + + it("takes the chunked path for a binary-heavy folder whose inline body outgrows the threshold", async () => { + const dir = await makeTempDir(); + const packageDir = path.join(dir, "binary-package"); + await mkdir(path.join(packageDir, "blobs"), { recursive: true }); + await writeFile(path.join(packageDir, "COMPANY.md"), "# Company\n"); + // 40 MB of raw blob bytes: under the 48 MB raw threshold, but the inline + // JSON body would carry them base64-inflated (~53 MB) — past the + // threshold on the estimated request size, so the zip travels chunked. + await writeFile( + path.join(packageDir, "blobs", "9a1b2c3d"), + Buffer.alloc(40 * 1024 * 1024, 3), + ); + + const resolved = await resolveChunkedImportZip(packageDir); + expect(resolved).not.toBeNull(); + expect(resolved!.rootPath).toBe("binary-package"); + const archive = await readZipArchive(resolved!.zipBytes); + expect(Object.keys(archive.files).sort()).toEqual(["COMPANY.md", "blobs/9a1b2c3d"]); + }); + + it("keeps a text folder under both the raw and estimated measures inline", async () => { + const dir = await makeTempDir(); + const packageDir = path.join(dir, "text-package"); + await mkdir(packageDir, { recursive: true }); + await writeFile(path.join(packageDir, "COMPANY.md"), "# Company\n"); + // Sizable but nowhere near the threshold on either measure: text entries + // travel JSON-escaped, close to their raw size, so no base64 inflation + // pushes this folder onto the chunked path. + await writeFile(path.join(packageDir, "NOTES.md"), "agent docs line\n".repeat(200_000)); + + expect(await resolveChunkedImportZip(packageDir)).toBeNull(); + }); + + it("zips an oversized folder in memory with the same walk filters as the inline path", async () => { + const dir = await makeTempDir(); + const packageDir = path.join(dir, "big-package"); + await mkdir(path.join(packageDir, "blobs"), { recursive: true }); + await mkdir(path.join(packageDir, ".git"), { recursive: true }); + await writeFile(path.join(packageDir, "COMPANY.md"), "# Company\n"); + await writeFile(path.join(packageDir, "notes.txt"), "not portable\n"); + await writeFile(path.join(packageDir, ".git", "HEAD"), "ref: refs/heads/main\n"); + await writeFile( + path.join(packageDir, "blobs", "4f2d1c9a"), + Buffer.alloc(CHUNKED_IMPORT_THRESHOLD_BYTES + 1024, 9), + ); + + const resolved = await resolveChunkedImportZip(packageDir); + expect(resolved).not.toBeNull(); + expect(resolved!.rootPath).toBe("big-package"); + + // The archive unzips back into the same bundle the inline source carries. + const archive = await readZipArchive(resolved!.zipBytes); + expect(archive.rootPath).toBe("big-package"); + expect(Object.keys(archive.files).sort()).toEqual(["COMPANY.md", "blobs/4f2d1c9a"]); + expect(archive.files["COMPANY.md"]).toBe("# Company\n"); + }); +}); + +describe("uploadCompanyImportTransfer", () => { + const zipBytes = buildTwoPartZipBytes(); + type TransferApi = Parameters[0]; + + function fakeApi(overrides: { post?: ReturnType; putRaw?: ReturnType } = {}) { + const post = overrides.post + ?? vi.fn().mockResolvedValue({ + transferId: "transfer-1", + status: "running", + alreadyCompleted: false, + totalParts: 2, + missingParts: [0, 1], + }); + const putRaw = overrides.putRaw ?? vi.fn().mockResolvedValue({ ok: true }); + return { api: { post, putRaw } as unknown as TransferApi, post, putRaw }; + } + + it("uploads only the parts the server reports missing", async () => { + const { api, post, putRaw } = fakeApi({ + post: vi.fn().mockResolvedValue({ + transferId: "transfer-1", + status: "running", + alreadyCompleted: false, + totalParts: 2, + missingParts: [1], + }), + }); + const progress: number[] = []; + + const transferId = await uploadCompanyImportTransfer(api, zipBytes, { + onProgress: (update) => progress.push(update.uploadedParts), + }); + + expect(transferId).toBe("transfer-1"); + expect(post).toHaveBeenCalledWith( + "/api/companies/import/transfers", + expect.objectContaining({ totalBytes: zipBytes.length }), + ); + expect(putRaw).toHaveBeenCalledTimes(1); + expect(putRaw.mock.calls[0]![0]).toBe("/api/companies/import/transfers/transfer-1/parts/1"); + expect(putRaw.mock.calls[0]![1]).toHaveLength(3); + expect(progress).toEqual([2]); + }); + + it("retries a failed part before succeeding", async () => { + const putRaw = vi.fn() + .mockRejectedValueOnce(new Error("socket hang up")) + .mockRejectedValueOnce(new Error("socket hang up")) + .mockResolvedValue({ ok: true }); + const { api } = fakeApi({ putRaw }); + + await expect(uploadCompanyImportTransfer(api, zipBytes)).resolves.toBe("transfer-1"); + // Part 0 took three attempts; part 1 succeeded first try. + expect(putRaw).toHaveBeenCalledTimes(4); + }); + + it("surfaces the upload error after exhausting the per-part attempts", async () => { + const putRaw = vi.fn().mockRejectedValue(new Error("socket hang up")); + const { api } = fakeApi({ putRaw }); + + await expect(uploadCompanyImportTransfer(api, zipBytes)).rejects.toThrow("socket hang up"); + expect(putRaw).toHaveBeenCalledTimes(3); + }); + + it("refuses a transfer whose content was already applied", async () => { + const { api, putRaw } = fakeApi({ + post: vi.fn().mockResolvedValue({ + transferId: "transfer-1", + status: "completed", + alreadyCompleted: true, + totalParts: 2, + missingParts: [], + }), + }); + + await expect(uploadCompanyImportTransfer(api, zipBytes)).rejects.toThrow(/already imported/); + expect(putRaw).not.toHaveBeenCalled(); + }); +}); + +describe("company import command over the chunked transfer path", () => { + let fetchMock: ReturnType; + let logSpy: ReturnType; + + beforeEach(() => { + process.env = { ...ORIGINAL_ENV }; + delete process.env.PAPERCLIP_API_URL; + delete process.env.PAPERCLIP_API_KEY; + delete process.env.PAPERCLIP_COMPANY_ID; + fetchMock = vi.fn(); + vi.stubGlobal("fetch", fetchMock); + logSpy = vi.spyOn(console, "log").mockImplementation(() => undefined); + vi.spyOn(console, "error").mockImplementation(() => undefined); + }); + + afterEach(() => { + process.env = { ...ORIGINAL_ENV }; + vi.unstubAllGlobals(); + vi.restoreAllMocks(); + }); + + async function runCommand(args: string[]): Promise { + const program = new Command(); + program.exitOverride(); + program.configureOutput({ + writeOut: () => undefined, + writeErr: () => undefined, + }); + registerCompanyCommands(program); + await program.parseAsync(args, { from: "user" }); + } + + function jsonResponse(body: unknown, status = 200): Response { + return new Response(JSON.stringify(body), { + status, + headers: { "content-type": "application/json" }, + }); + } + + function minimalPreview() { + return { + include: { company: true, agents: true, projects: true, issues: true }, + targetCompanyId: null, + targetCompanyName: null, + collisionStrategy: "rename", + selectedAgentSlugs: [], + plan: { companyAction: "create", agentPlans: [], projectPlans: [], issuePlans: [] }, + manifest: { agents: [], projects: [], issues: [], skills: [], company: null }, + files: {}, + envInputs: [], + warnings: [], + errors: [], + }; + } + + it("slices an oversized local zip into a transfer and applies it against the spool", async () => { + const dir = await makeTempDir(); + const zipPath = path.join(dir, "big-package.zip"); + await writeFile(zipPath, Buffer.alloc(CHUNKED_IMPORT_THRESHOLD_BYTES + 1024, 5)); + + fetchMock + .mockResolvedValueOnce(jsonResponse({ + transferId: "transfer-1", + status: "running", + alreadyCompleted: false, + totalParts: 2, + missingParts: [0, 1], + })) + .mockResolvedValueOnce(jsonResponse({ ok: true, index: 0, alreadyCompleted: false })) + .mockResolvedValueOnce(jsonResponse({ ok: true, index: 1, alreadyCompleted: false })) + .mockResolvedValueOnce(jsonResponse(minimalPreview())) + .mockResolvedValueOnce(jsonResponse({ + company: { id: "company-9", name: "Imported", action: "created" }, + agents: [], + skills: [], + projects: [], + routines: [], + envInputs: [], + warnings: [], + })); + + await runCommand([ + "company", + "import", + zipPath, + "--target", + "new", + "--yes", + "--json", + "--api-base", + "http://paperclip.test", + "--api-key", + "board-token", + ]); + + expect(fetchMock).toHaveBeenNthCalledWith( + 1, + "http://paperclip.test/api/companies/import/transfers", + expect.objectContaining({ method: "POST" }), + ); + const declared = JSON.parse(String(fetchMock.mock.calls[0]![1].body)); + expect(declared.totalBytes).toBe(CHUNKED_IMPORT_THRESHOLD_BYTES + 1024); + expect(declared.parts).toHaveLength(2); + + expect(fetchMock).toHaveBeenNthCalledWith( + 2, + "http://paperclip.test/api/companies/import/transfers/transfer-1/parts/0", + expect.objectContaining({ + method: "PUT", + headers: expect.objectContaining({ "content-type": "application/octet-stream" }), + }), + ); + expect(fetchMock).toHaveBeenNthCalledWith( + 3, + "http://paperclip.test/api/companies/import/transfers/transfer-1/parts/1", + expect.objectContaining({ method: "PUT" }), + ); + expect(fetchMock).toHaveBeenNthCalledWith( + 4, + "http://paperclip.test/api/companies/import/transfers/transfer-1/preview", + expect.objectContaining({ method: "POST" }), + ); + // Preview and apply carry the meta fields, never an inline source. + const previewBody = JSON.parse(String(fetchMock.mock.calls[3]![1].body)); + expect(previewBody.target).toEqual({ mode: "new_company", newCompanyName: null }); + expect(previewBody).not.toHaveProperty("source"); + expect(fetchMock).toHaveBeenNthCalledWith( + 5, + "http://paperclip.test/api/companies/import/transfers/transfer-1/apply", + expect.objectContaining({ method: "POST" }), + ); + const applyBody = JSON.parse(String(fetchMock.mock.calls[4]![1].body)); + expect(applyBody).not.toHaveProperty("source"); + + expect(JSON.parse(String(logSpy.mock.calls.at(-1)?.[0]))).toMatchObject({ + company: { id: "company-9" }, + }); + }); + + it("keeps small local zips on the inline single-shot path", async () => { + const dir = await makeTempDir(); + const zipPath = path.join(dir, "small-package.zip"); + await writeFile(zipPath, createStoredZipArchive({ "COMPANY.md": "# Company\n" }, "small-package")); + + fetchMock.mockResolvedValueOnce(jsonResponse(minimalPreview())); + + await runCommand([ + "company", + "import", + zipPath, + "--target", + "new", + "--dry-run", + "--json", + "--api-base", + "http://paperclip.test", + "--api-key", + "board-token", + ]); + + expect(fetchMock).toHaveBeenCalledTimes(1); + expect(fetchMock).toHaveBeenCalledWith( + "http://paperclip.test/api/companies/import/preview", + expect.objectContaining({ method: "POST" }), + ); + const body = JSON.parse(String(fetchMock.mock.calls[0]![1].body)); + expect(body.source.type).toBe("inline"); + expect(body.source.files["COMPANY.md"]).toBe("# Company\n"); + }); +}); diff --git a/cli/src/__tests__/database-check.test.ts b/cli/src/__tests__/database-check.test.ts new file mode 100644 index 00000000000..51c42cf7717 --- /dev/null +++ b/cli/src/__tests__/database-check.test.ts @@ -0,0 +1,79 @@ +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import { databaseCheck } from "../checks/database-check.js"; +import type { PaperclipConfig } from "../config/schema.js"; + +const created: string[] = []; +const ORIGINAL_IN_WORKTREE = process.env.PAPERCLIP_IN_WORKTREE; + +function makeBase(): string { + const base = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-dbcheck-")); + created.push(base); + return base; +} + +function embeddedConfig(dataDir: string): PaperclipConfig { + return { + database: { + mode: "embedded-postgres", + embeddedPostgresDataDir: dataDir, + embeddedPostgresPort: 54321, + }, + } as unknown as PaperclipConfig; +} + +afterEach(() => { + vi.restoreAllMocks(); + if (ORIGINAL_IN_WORKTREE === undefined) delete process.env.PAPERCLIP_IN_WORKTREE; + else process.env.PAPERCLIP_IN_WORKTREE = ORIGINAL_IN_WORKTREE; + while (created.length > 0) { + const dir = created.pop(); + if (dir) fs.rmSync(dir, { recursive: true, force: true }); + } +}); + +describe("databaseCheck — embedded postgres temp-dir guard", () => { + it("passes when the data dir is on persistent (non-temp) storage", async () => { + const base = makeBase(); + // Treat a sibling dir as the OS temp root so the persistent dir is outside it. + vi.spyOn(os, "tmpdir").mockReturnValue(path.join(base, "fake-tmp")); + process.env.PAPERCLIP_IN_WORKTREE = "true"; + const persistentDataDir = path.join(base, "persistent", "instances", "default", "db"); + + const result = await databaseCheck(embeddedConfig(persistentDataDir), path.join(base, "config.json")); + + expect(result.status).toBe("pass"); + expect(result.message).toContain("Embedded PostgreSQL configured at"); + }); + + it("warns when a worktree-mode data dir lives inside the OS temp directory", async () => { + const base = makeBase(); + const fakeTmp = path.join(base, "fake-tmp"); + vi.spyOn(os, "tmpdir").mockReturnValue(fakeTmp); + process.env.PAPERCLIP_IN_WORKTREE = "true"; + const tmpDataDir = path.join(fakeTmp, "instances", "default", "db"); + + const result = await databaseCheck(embeddedConfig(tmpDataDir), path.join(base, "config.json")); + + expect(result.status).toBe("warn"); + expect(result.message).toMatch(/temp directory/i); + expect(result.message).toMatch(/ephemeral/i); + expect(result.repairHint).toContain("PAPERCLIP_HOME"); + // Must warn BEFORE creating anything — don't bootstrap the throwaway temp dir. + expect(fs.existsSync(tmpDataDir)).toBe(false); + }); + + it("does NOT warn for a temp data dir when not in worktree mode (intentional ephemeral/CI use)", async () => { + const base = makeBase(); + const fakeTmp = path.join(base, "fake-tmp"); + vi.spyOn(os, "tmpdir").mockReturnValue(fakeTmp); + delete process.env.PAPERCLIP_IN_WORKTREE; + const tmpDataDir = path.join(fakeTmp, "instances", "default", "db"); + + const result = await databaseCheck(embeddedConfig(tmpDataDir), path.join(base, "config.json")); + + expect(result.status).toBe("pass"); + }); +}); diff --git a/cli/src/__tests__/env-lab.test.ts b/cli/src/__tests__/env-lab.test.ts index 02d6d7daf18..6afa22660b0 100644 --- a/cli/src/__tests__/env-lab.test.ts +++ b/cli/src/__tests__/env-lab.test.ts @@ -1,6 +1,13 @@ import path from "node:path"; -import { describe, expect, it } from "vitest"; -import { collectEnvLabDoctorStatus, resolveEnvLabSshStatePath } from "../commands/env-lab.js"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import * as p from "@clack/prompts"; +import { + buildEnvLabCleanupCommand, + collectEnvLabDoctorStatus, + envLabDoctorCommand, + resolveEnvLabCliInvocation, + resolveEnvLabSshStatePath, +} from "../commands/env-lab.js"; describe("env-lab command", () => { it("resolves the default SSH fixture state path under the instance root", () => { @@ -22,3 +29,259 @@ describe("env-lab command", () => { expect(status.ssh.environment).toBeNull(); }); }); + +describe("env-lab cleanup command hint", () => { + const originalCwd = process.cwd(); + + afterEach(() => { + process.chdir(originalCwd); + }); + + // Resolve a source-checkout invocation for a fabricated checkout root. A source + // checkout runs this module from `/src/commands/env-lab.ts`, so the + // resolver reads that layout and returns the tsx runner and source entry. + function sourceInvocation(root: string) { + return resolveEnvLabCliInvocation(path.join(root, "src", "commands", "env-lab.ts")); + } + + // Resolve a bundled-build invocation for a fabricated package root. The bundled + // build runs this module from `/dist/index.js`, so the resolver returns + // that file as the entry with no tsx runner. + function bundledInvocation(root: string) { + return resolveEnvLabCliInvocation(path.join(root, "dist", "index.js")); + } + + // Split a command that uses POSIX single-quoting into its argument tokens. The + // parser reads a single-quoted span verbatim and reads `\'` outside a span as a + // literal single quote. This is the same rule a POSIX shell obeys, so a token + // list proves the shell reads the exact paths and runs no embedded command. + function tokenizePosix(command: string): string[] { + const tokens: string[] = []; + let current = ""; + let started = false; + let inQuotes = false; + for (let index = 0; index < command.length; index += 1) { + const character = command[index]; + if (inQuotes) { + if (character === "'") { + inQuotes = false; + } else { + current += character; + } + } else if (character === "'") { + inQuotes = true; + started = true; + } else if (character === "\\") { + index += 1; + current += command[index]; + started = true; + } else if (character === " ") { + if (started) { + tokens.push(current); + current = ""; + started = false; + } + } else { + current += character; + started = true; + } + } + if (started) { + tokens.push(current); + } + return tokens; + } + + // Return the two path arguments from the `node` command. + function extractPaths(command: string): string[] { + const tokens = tokenizePosix(command); + return tokens.slice(1, 3); + } + + it("resolves both CLI paths to absolute paths", () => { + const command = buildEnvLabCleanupCommand(); + const paths = extractPaths(command); + + expect(paths).toHaveLength(2); + for (const resolved of paths) { + expect(path.isAbsolute(resolved)).toBe(true); + } + expect(command.endsWith("env-lab down")).toBe(true); + }); + + it("points at the checked-out tsx runner and cli source entry", () => { + const [tsxBin, entry] = extractPaths(buildEnvLabCleanupCommand()); + + expect(tsxBin).toContain( + path.join("cli", "node_modules", "tsx", "dist", "cli.mjs"), + ); + expect(entry).toContain(path.join("cli", "src", "index.ts")); + }); + + it("returns the same command from a checkout subdirectory", () => { + const fromRoot = buildEnvLabCleanupCommand(); + + // Simulate a contributor who runs `env-lab doctor` from a subdirectory of + // the checkout. A relative path would change with the working directory, so + // this asserts the command stays constant. + process.chdir(path.dirname(originalCwd)); + const fromParent = buildEnvLabCleanupCommand(); + process.chdir(originalCwd); + + expect(fromParent).toBe(fromRoot); + }); + + it("never restores the unsafe pnpm invocation forms", () => { + const command = buildEnvLabCleanupCommand(); + + // The bare `pnpm paperclipai` script form is unsafe. The `pnpm exec` form + // does not resolve the CLI binary. Keep both out of the hint. + expect(command).not.toContain("pnpm paperclipai"); + expect(command).not.toContain("pnpm exec paperclipai"); + }); + + // A checkout path can hold shell metacharacters. A contributor copies the hint + // and pastes it into a shell. The hint must neutralize each metacharacter, so + // the shell reads the exact path and runs no embedded command. Each case below + // is a checkout root with one dangerous construct. + const dangerousRoots = [ + { label: "a dollar sign", root: "/tmp/env$lab/checkout" }, + { label: "command substitution", root: "/tmp/$(touch pwned)/checkout" }, + { label: "backticks", root: "/tmp/`touch pwned`/checkout" }, + { label: "a double quote", root: '/tmp/env"lab/checkout' }, + { label: "a single quote", root: "/tmp/env'lab/checkout" }, + ]; + + for (const { label, root } of dangerousRoots) { + it(`keeps a checkout path with ${label} inert in the cleanup hint`, () => { + const command = buildEnvLabCleanupCommand({ invocation: sourceInvocation(root) }); + const tokens = tokenizePosix(command); + const tsxBin = path.join(root, "node_modules", "tsx", "dist", "cli.mjs"); + const entry = path.join(root, "src", "index.ts"); + + // The shell reads the exact paths as single argument tokens. It does not + // split the paths or run the embedded construct. + expect(tokens).toEqual(["node", tsxBin, entry, "env-lab", "down"]); + + // The old double-quoted form left `$(...)`, a backtick pair, and `$NAME` + // live. Do not restore it. + expect(command).not.toContain(`"${tsxBin}"`); + expect(command).not.toContain(`"${entry}"`); + }); + } + + it("forwards the inspected instance to the cleanup hint", () => { + const command = buildEnvLabCleanupCommand({ + instance: "fixture-test", + invocation: sourceInvocation("/tmp/checkout"), + }); + const tokens = tokenizePosix(command); + + // The hint ends with `--instance `, so it stops the fixture the doctor + // command diagnosed, not the default instance. + expect(tokens.slice(-2)).toEqual(["--instance", "fixture-test"]); + }); + + it("omits the instance flag when the doctor command uses the default instance", () => { + const command = buildEnvLabCleanupCommand({ invocation: sourceInvocation("/tmp/checkout") }); + + // Without a selected instance, `env-lab down` resolves the same default + // instance the doctor command inspected. Do not add an empty flag. + expect(command).not.toContain("--instance"); + expect(command.endsWith("env-lab down")).toBe(true); + }); + + it("keeps an instance id with shell metacharacters inert", () => { + const command = buildEnvLabCleanupCommand({ + instance: "$(touch pwned)", + invocation: sourceInvocation("/tmp/checkout"), + }); + const tokens = tokenizePosix(command); + + // The shell reads the instance id as one literal token and runs no embedded + // command. + expect(tokens.slice(-2)).toEqual(["--instance", "$(touch pwned)"]); + expect(command).not.toContain('"$(touch pwned)"'); + }); + + it("runs the bundled dist entry directly, without the tsx runner", () => { + const command = buildEnvLabCleanupCommand({ + invocation: bundledInvocation("/opt/pkg"), + }); + const tokens = tokenizePosix(command); + + // The published package ships one `dist/index.js` file and no tsx runner, so + // node runs that file directly. + expect(tokens).toEqual(["node", path.join("/opt/pkg", "dist", "index.js"), "env-lab", "down"]); + expect(command).not.toContain("tsx"); + expect(command).not.toContain(path.join("src", "index.ts")); + }); + + it("keeps a bundled package path with shell metacharacters inert", () => { + const root = "/opt/$(touch pwned)/pkg"; + const command = buildEnvLabCleanupCommand({ invocation: bundledInvocation(root) }); + const tokens = tokenizePosix(command); + const entry = path.join(root, "dist", "index.js"); + + // The shell reads the exact bundled path as one token and runs no embedded + // command. + expect(tokens).toEqual(["node", entry, "env-lab", "down"]); + expect(command).not.toContain(`"${entry}"`); + }); +}); + +describe("env-lab doctor cleanup hint instance", () => { + const originalInstanceId = process.env.PAPERCLIP_INSTANCE_ID; + + afterEach(() => { + if (originalInstanceId === undefined) { + delete process.env.PAPERCLIP_INSTANCE_ID; + } else { + process.env.PAPERCLIP_INSTANCE_ID = originalInstanceId; + } + vi.restoreAllMocks(); + }); + + // Capture the cleanup hint the doctor prints. The doctor reports through + // `p.log`, so the test replaces each channel and reads the captured lines. + function captureDoctorMessages(): string[] { + const messages: string[] = []; + vi.spyOn(p.log, "message").mockImplementation((message?: string) => { + messages.push(message ?? ""); + }); + vi.spyOn(p.log, "success").mockImplementation(() => {}); + vi.spyOn(p.log, "warn").mockImplementation(() => {}); + vi.spyOn(p.log, "info").mockImplementation(() => {}); + return messages; + } + + it("pins the PAPERCLIP_INSTANCE_ID instance when opts.instance is absent", async () => { + // The doctor diagnoses the instance that `PAPERCLIP_INSTANCE_ID` selects. + // The cleanup hint must target that instance, not the default instance. + process.env.PAPERCLIP_INSTANCE_ID = "env-selected-instance"; + const messages = captureDoctorMessages(); + + await envLabDoctorCommand({ instance: undefined }); + + const cleanup = messages.find((message) => message.startsWith("Cleanup:")); + expect(cleanup).toBeDefined(); + expect(cleanup).toContain("env-lab down"); + expect(cleanup).toContain("--instance"); + expect(cleanup).toContain("env-selected-instance"); + }); + + it("pins the explicit instance over PAPERCLIP_INSTANCE_ID", async () => { + // An explicit `--instance` flag overrides the environment variable, so the + // hint targets the explicit instance the doctor inspected. + process.env.PAPERCLIP_INSTANCE_ID = "env-selected-instance"; + const messages = captureDoctorMessages(); + + await envLabDoctorCommand({ instance: "explicit-instance" }); + + const cleanup = messages.find((message) => message.startsWith("Cleanup:")); + expect(cleanup).toBeDefined(); + expect(cleanup).toContain("--instance"); + expect(cleanup).toContain("explicit-instance"); + expect(cleanup).not.toContain("env-selected-instance"); + }); +}); diff --git a/cli/src/__tests__/http.test.ts b/cli/src/__tests__/http.test.ts index 0829f06baa8..e0e1e61b747 100644 --- a/cli/src/__tests__/http.test.ts +++ b/cli/src/__tests__/http.test.ts @@ -79,7 +79,7 @@ describe("PaperclipApiClient", () => { /curl http:\/\/localhost:3100\/api\/health/, ); await expect(client.post("/api/companies/import/preview", {})).rejects.toThrow( - /pnpm dev|pnpm paperclipai run/, + /pnpm dev|npx paperclipai run/, ); }); diff --git a/cli/src/__tests__/worktree.test.ts b/cli/src/__tests__/worktree.test.ts index ce063abfd79..dab7f78bfd9 100644 --- a/cli/src/__tests__/worktree.test.ts +++ b/cli/src/__tests__/worktree.test.ts @@ -31,6 +31,7 @@ import { resolveWorktreeReseedTargetPaths, resolveGitWorktreeAddArgs, resolvePnpmInstallInvocation, + resolveCurrentWorktreeEndpoint, resolveWorktreeSeedBackupEngine, resolveWorktreeMakeTargetPath, worktreeRepairCommand, @@ -167,6 +168,49 @@ function buildSourceConfig(): PaperclipConfig { } describe("worktree helpers", () => { + it("uses the repo-local config for the current worktree", () => { + const targetRoot = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-current-worktree-")); + try { + const localConfig = path.join(targetRoot, ".paperclip", "config.json"); + fs.mkdirSync(path.dirname(localConfig), { recursive: true }); + fs.writeFileSync(localConfig, "{}\n"); + process.env.PAPERCLIP_CONFIG = "/tmp/ambient-paperclip/config.json"; + process.chdir(targetRoot); + + expect(resolveCurrentWorktreeEndpoint()).toMatchObject({ + rootPath: targetRoot, + configPath: localConfig, + isCurrent: true, + }); + } finally { + process.chdir(ORIGINAL_CWD); + fs.rmSync(targetRoot, { recursive: true, force: true }); + } + }); + + it("uses the repository config from a nested working directory", () => { + const targetRoot = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-current-worktree-nested-")); + try { + execFileSync("git", ["init", "-q"], { cwd: targetRoot }); + const nestedDirectory = path.join(targetRoot, "packages", "example", "src"); + const localConfig = path.join(targetRoot, ".paperclip", "config.json"); + fs.mkdirSync(nestedDirectory, { recursive: true }); + fs.mkdirSync(path.dirname(localConfig), { recursive: true }); + fs.writeFileSync(localConfig, "{}\n"); + process.env.PAPERCLIP_CONFIG = "/tmp/ambient-paperclip/config.json"; + process.chdir(nestedDirectory); + + expect(resolveCurrentWorktreeEndpoint()).toMatchObject({ + rootPath: targetRoot, + configPath: localConfig, + isCurrent: true, + }); + } finally { + process.chdir(ORIGINAL_CWD); + fs.rmSync(targetRoot, { recursive: true, force: true }); + } + }); + it("sanitizes instance ids", () => { expect(sanitizeWorktreeInstanceId("feature/worktree-support")).toBe("feature-worktree-support"); expect(sanitizeWorktreeInstanceId(" ")).toBe("worktree"); diff --git a/cli/src/checks/database-check.ts b/cli/src/checks/database-check.ts index 4d4ef811e11..8e15cad14a3 100644 --- a/cli/src/checks/database-check.ts +++ b/cli/src/checks/database-check.ts @@ -1,8 +1,16 @@ import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; import type { PaperclipConfig } from "../config/schema.js"; import type { CheckResult } from "./index.js"; import { resolveRuntimeLikePath } from "./path-resolver.js"; +function isInsideOsTmpDir(targetPath: string): boolean { + const tmpRoot = path.resolve(os.tmpdir()); + const resolved = path.resolve(targetPath); + return resolved === tmpRoot || resolved.startsWith(`${tmpRoot}${path.sep}`); +} + export async function databaseCheck(config: PaperclipConfig, configPath?: string): Promise { if (config.database.mode === "postgres") { if (!config.database.connectionString) { @@ -37,9 +45,33 @@ export async function databaseCheck(config: PaperclipConfig, configPath?: string if (config.database.mode === "embedded-postgres") { const dataDir = resolveRuntimeLikePath(config.database.embeddedPostgresDataDir, configPath); - const reportedPath = dataDir; + + // A worktree-mode instance whose data dir lives under the OS temp dir is a red + // flag: this is what happens when PAPERCLIP_HOME / PAPERCLIP_IN_WORKTREE leak + // into a PRIMARY instance's environment and silently relocate it to a throwaway + // temp home, so it boots an empty DB and locks everyone out. (Intentional + // ephemeral/CI instances that don't set PAPERCLIP_IN_WORKTREE are not flagged.) + // Check BEFORE creating the dir so we don't bootstrap the very temp location + // we're warning about. + if (isInsideOsTmpDir(dataDir) && process.env.PAPERCLIP_IN_WORKTREE === "true") { + return { + name: "Database", + status: "warn", + message: + `Embedded PostgreSQL data dir is inside the OS temp directory (${dataDir}) ` + + "while running in worktree mode (PAPERCLIP_IN_WORKTREE=true). Data stored here is " + + "ephemeral and will be lost on reboot or a temp cleanup. If this is your primary " + + "instance, PAPERCLIP_HOME / PAPERCLIP_IN_WORKTREE likely leaked into its environment, " + + "pointing it at a throwaway worktree home instead of your real data.", + canRepair: false, + repairHint: + "If this is the primary instance, unset PAPERCLIP_HOME and PAPERCLIP_IN_WORKTREE " + + "(or pass --data-dir ) and restart so it uses the persistent instance.", + }; + } + if (!fs.existsSync(dataDir)) { - fs.mkdirSync(reportedPath, { recursive: true }); + fs.mkdirSync(dataDir, { recursive: true }); } return { diff --git a/cli/src/client/http.ts b/cli/src/client/http.ts index fe7abb748c0..dd4f784776d 100644 --- a/cli/src/client/http.ts +++ b/cli/src/client/http.ts @@ -88,6 +88,15 @@ export class PaperclipApiClient { }, opts); } + /** Raw binary upload (e.g. one chunked import-transfer part); the body travels as-is. */ + putRaw(path: string, body: Uint8Array, opts?: RequestOptions): Promise { + return this.request(path, { + method: "PUT", + body: body as unknown as BodyInit, + headers: { "content-type": "application/octet-stream" }, + }, opts); + } + delete(path: string, opts?: RequestOptions): Promise { return this.request(path, { method: "DELETE" }, opts); } @@ -224,7 +233,7 @@ function buildConnectionErrorMessage(input: { "This usually means the Paperclip server is not running, the configured URL is wrong, or the request is being blocked before it reaches Paperclip.", "", "Try:", - "- Start Paperclip with `pnpm dev` or `pnpm paperclipai run`.", + "- Start Paperclip with `pnpm dev` (from a source checkout) or `npx paperclipai run`.", `- Verify the server is reachable with \`curl ${healthUrl}\`.`, `- If Paperclip is running elsewhere, pass \`--api-base ${input.apiBase.replace(/\/+$/, "")}\` or set \`PAPERCLIP_API_URL\`.`, ); diff --git a/cli/src/commands/channels.ts b/cli/src/commands/channels.ts new file mode 100644 index 00000000000..3fe211b4957 --- /dev/null +++ b/cli/src/commands/channels.ts @@ -0,0 +1,119 @@ +import { execFile } from "node:child_process"; +import { promisify } from "node:util"; +import pc from "picocolors"; +import { resolvePublishedVersion, type CommandRunner } from "./install.js"; +import { packageVersion } from "../version.js"; + +const execFileAsync = promisify(execFile); + +const defaultRunCommand: CommandRunner = (command, args, options) => + execFileAsync(command, args, { ...options, encoding: "utf8" }); + +export type ChannelName = "stable" | "beta" | "nightly" | "canary"; + +export type ChannelDescriptor = { + channel: ChannelName; + distTag: string; + cadence: string; + audience: string; +}; + +// Ordered from most to least stable — the order users should consider them. +export const RELEASE_CHANNELS: readonly ChannelDescriptor[] = [ + { + channel: "stable", + distTag: "latest", + cadence: "manual, soaked in beta for 3+ days", + audience: "the recommended release for almost everyone", + }, + { + channel: "beta", + distTag: "beta", + cadence: "manual promotion behind an approval gate", + audience: "release candidates: what stable becomes a few days later", + }, + { + channel: "nightly", + distTag: "nightly", + cadence: "once a night, smoke-gated", + audience: "yesterday's merges, tested as a unit", + }, + { + channel: "canary", + distTag: "canary", + cadence: "every merge to master", + audience: "the bleeding edge", + }, +]; + +const CALVER_RE = /^\d{4}\.\d{1,4}\.\d+$/; +const PRERELEASE_RE = /^\d{4}\.\d{1,4}\.\d+-(canary|nightly|beta)\.\d+$/; + +// Published versions carry their lane in the version string; the source +// checkout's package.json holds a placeholder that matches neither form. +export function channelForVersion(version: string): ChannelName | "unknown" { + const prerelease = version.match(PRERELEASE_RE); + if (prerelease) return prerelease[1] as ChannelName; + if (CALVER_RE.test(version)) return "stable"; + return "unknown"; +} + +export type ChannelState = ChannelDescriptor & { version: string | null }; + +export async function collectChannelState( + runCommand: CommandRunner = defaultRunCommand, +): Promise { + const resolved = await Promise.allSettled( + RELEASE_CHANNELS.map((entry) => resolvePublishedVersion(entry.distTag, runCommand)), + ); + return RELEASE_CHANNELS.map((entry, index) => { + const outcome = resolved[index]; + return { + ...entry, + version: outcome.status === "fulfilled" ? outcome.value : null, + }; + }); +} + +export type ChannelsOptions = { json?: boolean }; + +export async function channelsCommand( + options: ChannelsOptions = {}, + runCommand: CommandRunner = defaultRunCommand, +): Promise { + const state = await collectChannelState(runCommand); + const currentChannel = channelForVersion(packageVersion); + + if (options.json) { + console.log( + JSON.stringify( + { + current: { version: packageVersion, channel: currentChannel }, + channels: state, + }, + null, + 2, + ), + ); + return; + } + + console.log(pc.bold("Paperclip release channels")); + console.log(""); + for (const entry of state) { + const version = entry.version ?? pc.yellow("unavailable"); + console.log(` ${pc.bold(entry.channel.padEnd(8))} ${version}`); + console.log(` ${" ".repeat(8)} ${pc.dim(`${entry.cadence} — ${entry.audience}`)}`); + console.log(` ${" ".repeat(8)} ${pc.dim(`npx paperclipai@${entry.distTag} onboard`)}`); + console.log(""); + } + + if (currentChannel === "unknown") { + console.log( + `This install reports version ${pc.bold(packageVersion)}, which does not map to a published channel (source checkouts report the repository placeholder).`, + ); + } else { + console.log(`This install is version ${pc.bold(packageVersion)} on the ${pc.bold(currentChannel)} channel.`); + } + console.log(`Docker images use the same names: ghcr.io/paperclipai/paperclip:{latest,beta,nightly,canary}`); +} diff --git a/cli/src/commands/client/company.ts b/cli/src/commands/client/company.ts index 72864f6482c..e1e7bea44b1 100644 --- a/cli/src/commands/client/company.ts +++ b/cli/src/commands/client/company.ts @@ -1,4 +1,5 @@ import { Command } from "commander"; +import { createHash } from "node:crypto"; import { mkdir, readdir, readFile, stat, writeFile } from "node:fs/promises"; import path from "node:path"; import * as p from "@clack/prompts"; @@ -12,10 +13,24 @@ import type { CompanyPortabilityPreviewResult, CompanyPortabilityImportResult, } from "@paperclipai/shared"; +import { + companyImportTransferApplyPath, + companyImportTransferPartPath, + companyImportTransferPreviewPath, + COMPANY_IMPORT_TRANSFERS_ROUTE_PATH, + type CompanyImportTransferCreated, + type CompanyImportTransferDeclaration, +} from "@paperclipai/shared/company-import-transfer"; import { getTelemetryClient, trackCompanyImported } from "../../telemetry.js"; -import { ApiRequestError } from "../../client/http.js"; +import { ApiRequestError, type PaperclipApiClient } from "../../client/http.js"; import { openUrl } from "../../client/board-auth.js"; -import { binaryContentTypeByExtension, bytesToPortableFileEntry, isBlobStorePath, readZipArchive } from "./zip.js"; +import { + binaryContentTypeByExtension, + bytesToPortableFileEntry, + createStoredZipArchive, + isBlobStorePath, + readZipArchive, +} from "./zip.js"; import { addCommonClientOptions, apiPath, @@ -927,23 +942,23 @@ async function pathExists(inputPath: string): Promise { } } -async function collectPackageFiles( +async function collectPackageFileBytes( root: string, current: string, - files: Record, + files: Record, ): Promise { const entries = await readdir(current, { withFileTypes: true }); for (const entry of entries) { if (entry.name.startsWith(".git")) continue; const absolutePath = path.join(current, entry.name); if (entry.isDirectory()) { - await collectPackageFiles(root, absolutePath, files); + await collectPackageFileBytes(root, absolutePath, files); continue; } if (!entry.isFile()) continue; const relativePath = path.relative(root, absolutePath).replace(/\\/g, "/"); if (!shouldIncludePortableFile(relativePath)) continue; - files[relativePath] = bytesToPortableFileEntry(relativePath, await readFile(absolutePath)); + files[relativePath] = await readFile(absolutePath); } } @@ -965,14 +980,232 @@ export async function resolveInlineSourceFromPath(inputPath: string): Promise<{ } const rootDir = resolvedStat.isDirectory() ? resolved : path.dirname(resolved); - const files: Record = {}; - await collectPackageFiles(rootDir, rootDir, files); + const fileBytes: Record = {}; + await collectPackageFileBytes(rootDir, rootDir, fileBytes); return { rootPath: path.basename(rootDir), - files, + files: Object.fromEntries( + Object.entries(fileBytes).map(([relativePath, bytes]) => [ + relativePath, + bytesToPortableFileEntry(relativePath, bytes), + ]), + ), }; } +// ── Chunked transfer flow for large local packages ─────────────────── +// +// A local package over the threshold is not posted as one inline JSON body: +// its zip is declared as a chunked transfer (whole-file and per-part sha256), +// the parts are uploaded individually with per-part retries, and preview and +// apply run server-side against the assembled spool. Re-declaring the same +// content — after a failure or an interrupted run — resumes the prior +// transfer, so only the parts the server is missing are ever re-uploaded. + +export const CHUNKED_IMPORT_THRESHOLD_BYTES = 48 * 1024 * 1024; +// Imports into an EXISTING company post to /api/companies/:id/imports/*, +// which sits behind the server's default 10 MB JSON parser — only the +// generic /api/companies/import path carries the 64 MB portable limit. The +// chunk decision for existing targets therefore uses this lower threshold +// (margin under 10 MB for the envelope), or the inline body would 413. +export const EXISTING_COMPANY_CHUNKED_IMPORT_THRESHOLD_BYTES = 8 * 1024 * 1024; +export const IMPORT_TRANSFER_PART_SIZE_BYTES = 32 * 1024 * 1024; +const IMPORT_TRANSFER_PART_ATTEMPTS = 3; + +// ── Inline request size estimation ─────────────────────────────────── +// +// Mirrors `estimateInlineImportBytes` in ui/src/lib/import-preflight.ts (the +// CLI cannot import from ui/) — keep the math on both sides in sync. The +// server enforces its body limit on raw request bytes, so each entry is +// measured the way it actually travels: JSON-escaped UTF-8 for text +// (multi-byte characters and escape sequences both inflate past +// `String.length`), and the base64 payload plus its object structure for +// binary entries (base64 and MIME types are ASCII, one byte per character). + +const inlineEstimateUtf8 = new TextEncoder(); + +// Fixed serialization overhead of a base64 entry object around its data and +// contentType values: {"encoding":"base64","data":"…","contentType":"…"}. +const BASE64_ENTRY_STRUCTURE_BYTES = '{"encoding":"base64","data":"","contentType":""}'.length; + +// Allowance for everything in the request body besides the files map itself +// (rootPath, include flags, target, collision strategy, adapter overrides, +// braces and commas). Deliberately generous so the estimate never undercounts. +const REQUEST_ENVELOPE_ALLOWANCE_BYTES = 256 * 1024; + +function fileEntryInlineBytes(entry: CompanyPortabilityFileEntry): number { + if (typeof entry === "string") return inlineEstimateUtf8.encode(JSON.stringify(entry)).length; + return BASE64_ENTRY_STRUCTURE_BYTES + entry.data.length + (entry.contentType?.length ?? 0); +} + +/** + * Approximate JSON request size of an inline import: JSON-escaped UTF-8 text + * bytes, base64 payloads with their entry structure, the serialized file-path + * keys (thousands of paths are real bytes), and an envelope allowance for the + * rest of the request body. + */ +function estimateInlineImportBytes(files: Record): number { + let total = REQUEST_ENVELOPE_ALLOWANCE_BYTES; + for (const [filePath, entry] of Object.entries(files)) { + // "path": entry, → key bytes + colon + comma. + total += inlineEstimateUtf8.encode(JSON.stringify(filePath)).length + 2 + fileEntryInlineBytes(entry); + } + return total; +} + +export interface ImportTransferUploadProgress { + uploadedParts: number; + totalParts: number; + uploadedBytes: number; + totalBytes: number; +} + +export function buildImportTransferManifest(zipBytes: Uint8Array): CompanyImportTransferDeclaration { + const sha256 = (bytes: Uint8Array) => createHash("sha256").update(bytes).digest("hex"); + const parts: CompanyImportTransferDeclaration["parts"] = []; + for (let offset = 0; offset < zipBytes.length; offset += IMPORT_TRANSFER_PART_SIZE_BYTES) { + const byteSize = Math.min(IMPORT_TRANSFER_PART_SIZE_BYTES, zipBytes.length - offset); + parts.push({ + index: parts.length, + byteSize, + sha256: sha256(zipBytes.subarray(offset, offset + byteSize)), + }); + } + return { + totalBytes: zipBytes.length, + zipSha256: sha256(zipBytes), + partSizeBytes: IMPORT_TRANSFER_PART_SIZE_BYTES, + parts, + }; +} + +/** + * Resolve a local import source into raw zip bytes when its package is too + * large to travel as one inline JSON body: a .zip file is read as-is (so its + * declared hashes match the file on disk), a folder is packaged as a stored + * zip in memory with the same walk filters the inline path uses. Both source + * kinds are measured twice — raw bytes as a fast path, then the estimated + * inline request size, because base64 inflates binary entries ~4/3 and a + * compressed zip can expand far past its file size. Returns null for sources + * under the threshold on both measures — those keep the inline JSON path. + */ +export async function resolveChunkedImportZip( + inputPath: string, + thresholdBytes: number = CHUNKED_IMPORT_THRESHOLD_BYTES, +): Promise<{ + zipBytes: Uint8Array; + rootPath: string; +} | null> { + const resolved = path.resolve(inputPath); + const resolvedStat = await stat(resolved); + if (resolvedStat.isFile() && path.extname(resolved).toLowerCase() === ".zip") { + const zipBytes = new Uint8Array(await readFile(resolved)); + const rootPath = path.basename(resolved, ".zip"); + if (resolvedStat.size > thresholdBytes) return { zipBytes, rootPath }; + // A small compressed zip can still expand past server caps as inline + // JSON (text compresses well and binary re-inflates ~4/3 as base64), so + // the stay-inline decision uses the estimated request size of the same + // entries the inline path would send. An unreadable zip stays inline so + // that path surfaces its canonical parse error. + let archive: Awaited>; + try { + archive = await readZipArchive(zipBytes); + } catch { + return null; + } + if (estimateInlineImportBytes(archive.files) <= thresholdBytes) return null; + return { zipBytes, rootPath }; + } + if (!resolvedStat.isDirectory()) return null; + const fileBytes: Record = {}; + await collectPackageFileBytes(resolved, resolved, fileBytes); + const rootPath = path.basename(resolved); + // Content bytes alone already past the threshold means the stored zip + // (content plus headers) is too. + const contentBytes = Object.values(fileBytes).reduce((sum, bytes) => sum + bytes.length, 0); + if (contentBytes <= thresholdBytes) { + // Raw bytes under the threshold can still blow past server caps once the + // inline body is built (binary entries travel base64-inflated), so the + // stay-inline decision is made on the estimated request size — the same + // entries the inline path would send. + const inlineEntries = Object.fromEntries( + Object.entries(fileBytes).map(([relativePath, bytes]) => [ + relativePath, + bytesToPortableFileEntry(relativePath, bytes), + ]), + ); + if (estimateInlineImportBytes(inlineEntries) <= thresholdBytes) return null; + } + return { zipBytes: createStoredZipArchive(fileBytes, rootPath), rootPath }; +} + +/** + * Declare (or resume) the transfer for these zip bytes and upload every part + * the server reports missing, sequentially with per-part retries. Resolves + * with the transfer id once the server holds every part. + */ +export async function uploadCompanyImportTransfer( + api: Pick, + zipBytes: Uint8Array, + opts: { onProgress?: (progress: ImportTransferUploadProgress) => void } = {}, +): Promise { + const manifest = buildImportTransferManifest(zipBytes); + const created = await api.post( + `/api/companies${COMPANY_IMPORT_TRANSFERS_ROUTE_PATH}`, + manifest, + ); + if (!created) { + throw new Error("Import transfer declaration returned no data."); + } + if (created.alreadyCompleted) { + // The server keys transfers by content, and this exact zip already + // finished an apply — its spooled parts are gone, so it cannot re-run. + throw new Error( + "This exact package was already imported by a completed transfer. Re-export the package to import it again.", + ); + } + const missing = new Set(created.missingParts); + let uploadedParts = manifest.parts.length - missing.size; + let uploadedBytes = manifest.parts.reduce( + (sum, part) => (missing.has(part.index) ? sum : sum + part.byteSize), + 0, + ); + for (const part of manifest.parts) { + if (!missing.has(part.index)) continue; + const offset = part.index * manifest.partSizeBytes; + const bytes = zipBytes.subarray(offset, offset + part.byteSize); + let lastError: unknown = null; + let uploaded = false; + for (let attempt = 0; attempt < IMPORT_TRANSFER_PART_ATTEMPTS && !uploaded; attempt += 1) { + try { + await api.putRaw( + `/api/companies${companyImportTransferPartPath(created.transferId, part.index)}`, + bytes, + ); + uploaded = true; + } catch (err) { + lastError = err; + } + } + if (!uploaded) { + // Parts already uploaded stay spooled server-side; re-running the + // import resumes from them instead of starting over. + throw lastError instanceof Error + ? lastError + : new Error(`Import transfer part ${part.index} failed to upload.`); + } + uploadedParts += 1; + uploadedBytes += part.byteSize; + opts.onProgress?.({ + uploadedParts, + totalParts: manifest.parts.length, + uploadedBytes, + totalBytes: manifest.totalBytes, + }); + } + return created.transferId; +} + export async function writeExportToFolder(outDir: string, exported: CompanyPortabilityExportResult): Promise { const root = path.resolve(outDir); await mkdir(root, { recursive: true }); @@ -1464,6 +1697,7 @@ export function registerCompanyCommands(program: Command): void { let sourcePayload: | { type: "inline"; rootPath?: string | null; files: Record } | { type: "github"; url: string }; + let chunkedZip: { zipBytes: Uint8Array; rootPath: string } | null = null; const treatAsLocalPath = !isHttpUrl(from) && await pathExists(from); const isGithubSource = looksLikeRepoUrl(from) || (isGithubShorthand(from) && !treatAsLocalPath); @@ -1480,12 +1714,24 @@ export function registerCompanyCommands(program: Command): void { if (opts.ref?.trim()) { throw new Error("--ref is only supported for GitHub import sources."); } - const inline = await resolveInlineSourceFromPath(from); - sourcePayload = { - type: "inline", - rootPath: inline.rootPath, - files: inline.files, - }; + chunkedZip = await resolveChunkedImportZip( + from, + target === "existing" + ? EXISTING_COMPANY_CHUNKED_IMPORT_THRESHOLD_BYTES + : CHUNKED_IMPORT_THRESHOLD_BYTES, + ); + if (chunkedZip) { + // Too large for one request: the zip travels as a chunked + // transfer, so the inline files map is never built or sent. + sourcePayload = { type: "inline", rootPath: chunkedZip.rootPath, files: {} }; + } else { + const inline = await resolveInlineSourceFromPath(from); + sourcePayload = { + type: "inline", + rootPath: inline.rootPath, + files: inline.files, + }; + } } const sourceLabel = formatSourceLabel(sourcePayload); @@ -1496,15 +1742,40 @@ export function registerCompanyCommands(program: Command): void { companyId: targetPayload.mode === "existing_company" ? targetPayload.companyId : null, }); + // The transfer meta mirrors the inline preview payload minus its + // `source` — the source is the assembled zip, spooled server-side. + const transferMeta = { + include, + target: targetPayload, + agents, + collisionStrategy: collision, + }; + let transferId: string | null = null; + if (chunkedZip) { + transferId = await uploadCompanyImportTransfer(ctx.api, chunkedZip.zipBytes, { + onProgress: ctx.json + ? undefined + : ({ uploadedParts, totalParts, uploadedBytes, totalBytes }) => { + console.log( + pc.dim( + `Uploaded part ${uploadedParts}/${totalParts} (${Math.round(uploadedBytes / (1024 * 1024))} of ${Math.round(totalBytes / (1024 * 1024))} MB)`, + ), + ); + }, + }); + } + const transferPreviewPath = transferId + ? `/api/companies${companyImportTransferPreviewPath(transferId)}` + : null; + let selectedFiles: string[] | undefined; if (interactiveView && !opts.yes && !opts.include?.trim()) { - const initialPreview = await ctx.api.post(previewApiPath, { - source: sourcePayload, - include, - target: targetPayload, - agents, - collisionStrategy: collision, - }); + const initialPreview = transferPreviewPath + ? await ctx.api.post(transferPreviewPath, transferMeta) + : await ctx.api.post(previewApiPath, { + source: sourcePayload, + ...transferMeta, + }); if (!initialPreview) { throw new Error("Import preview returned no data."); } @@ -1513,13 +1784,15 @@ export function registerCompanyCommands(program: Command): void { const previewPayload = { source: sourcePayload, - include, - target: targetPayload, - agents, - collisionStrategy: collision, + ...transferMeta, selectedFiles, }; - const preview = await ctx.api.post(previewApiPath, previewPayload); + const preview = transferPreviewPath + ? await ctx.api.post(transferPreviewPath, { + ...transferMeta, + selectedFiles, + }) + : await ctx.api.post(previewApiPath, previewPayload); if (!preview) { throw new Error("Import preview returned no data."); } @@ -1576,10 +1849,15 @@ export function registerCompanyCommands(program: Command): void { targetMode: targetPayload.mode, companyId: targetPayload.mode === "existing_company" ? targetPayload.companyId : null, }); - const imported = await ctx.api.post(importApiPath, { - ...previewPayload, - adapterOverrides, - }); + const imported = transferId + ? await ctx.api.post( + `/api/companies${companyImportTransferApplyPath(transferId)}`, + { ...transferMeta, selectedFiles, adapterOverrides }, + ) + : await ctx.api.post(importApiPath, { + ...previewPayload, + adapterOverrides, + }); if (!imported) { throw new Error("Import request returned no data."); } diff --git a/cli/src/commands/client/zip.ts b/cli/src/commands/client/zip.ts index 8f7b4db4aba..a927d239c18 100644 --- a/cli/src/commands/client/zip.ts +++ b/cli/src/commands/client/zip.ts @@ -8,3 +8,110 @@ export { isBlobStorePath, readZipArchive, } from "@paperclipai/shared/portability-zip"; + +// STORE-only zip writer used to package a local folder in memory for the +// chunked import transfer path. STORE keeps the writer dependency-free; the +// transfer slices the raw archive bytes, so compression only trades CPU for +// part count. Classic zip only: entry counts and offsets past the 16/32-bit +// header fields would need zip64, which the reader side does not require and +// this writer refuses to emit. + +const ZIP_MAX_ENTRIES = 0xffff; +const ZIP_MAX_OFFSET_BYTES = 0xffffffff; + +function writeUint16(target: Uint8Array, offset: number, value: number) { + target[offset] = value & 0xff; + target[offset + 1] = (value >>> 8) & 0xff; +} + +function writeUint32(target: Uint8Array, offset: number, value: number) { + target[offset] = value & 0xff; + target[offset + 1] = (value >>> 8) & 0xff; + target[offset + 2] = (value >>> 16) & 0xff; + target[offset + 3] = (value >>> 24) & 0xff; +} + +function crc32(bytes: Uint8Array) { + let crc = 0xffffffff; + for (const byte of bytes) { + crc ^= byte; + for (let bit = 0; bit < 8; bit += 1) { + crc = (crc & 1) === 1 ? (crc >>> 1) ^ 0xedb88320 : crc >>> 1; + } + } + return (crc ^ 0xffffffff) >>> 0; +} + +/** + * Build a stored (uncompressed) zip of `files` under a single `rootPath/` + * top-level directory — the layout `readZipArchive` folds back into the + * inline `{ rootPath, files }` bundle. Entries are written in sorted path + * order and carry no timestamps, so the same content always produces the + * same bytes and a re-run resumes its content-addressed transfer. + */ +export function createStoredZipArchive(files: Record, rootPath: string): Uint8Array { + const entries = Object.entries(files).sort(([left], [right]) => left.localeCompare(right)); + if (entries.length > ZIP_MAX_ENTRIES) { + throw new Error(`Package has too many files to zip (${entries.length}; the zip format caps at ${ZIP_MAX_ENTRIES}).`); + } + const encoder = new TextEncoder(); + const localChunks: Uint8Array[] = []; + const centralChunks: Uint8Array[] = []; + let localOffset = 0; + + for (const [relativePath, body] of entries) { + const fileName = encoder.encode(`${rootPath}/${relativePath}`); + const checksum = crc32(body); + + const localHeader = new Uint8Array(30 + fileName.length); + writeUint32(localHeader, 0, 0x04034b50); + writeUint16(localHeader, 4, 20); + writeUint16(localHeader, 6, 0x0800); + writeUint16(localHeader, 8, 0); + writeUint32(localHeader, 14, checksum); + writeUint32(localHeader, 18, body.length); + writeUint32(localHeader, 22, body.length); + writeUint16(localHeader, 26, fileName.length); + localHeader.set(fileName, 30); + + const centralHeader = new Uint8Array(46 + fileName.length); + writeUint32(centralHeader, 0, 0x02014b50); + writeUint16(centralHeader, 4, 20); + writeUint16(centralHeader, 6, 20); + writeUint16(centralHeader, 8, 0x0800); + writeUint16(centralHeader, 10, 0); + writeUint32(centralHeader, 16, checksum); + writeUint32(centralHeader, 20, body.length); + writeUint32(centralHeader, 24, body.length); + writeUint16(centralHeader, 28, fileName.length); + writeUint32(centralHeader, 42, localOffset); + centralHeader.set(fileName, 46); + + localChunks.push(localHeader, body); + centralChunks.push(centralHeader); + localOffset += localHeader.length + body.length; + if (body.length > ZIP_MAX_OFFSET_BYTES || localOffset > ZIP_MAX_OFFSET_BYTES) { + throw new Error("Package is too large to zip in memory (zip64 archives are not supported)."); + } + } + + const centralDirectoryLength = centralChunks.reduce((sum, chunk) => sum + chunk.length, 0); + const archive = new Uint8Array(localOffset + centralDirectoryLength + 22); + let offset = 0; + for (const chunk of localChunks) { + archive.set(chunk, offset); + offset += chunk.length; + } + const centralDirectoryOffset = offset; + for (const chunk of centralChunks) { + archive.set(chunk, offset); + offset += chunk.length; + } + writeUint32(archive, offset, 0x06054b50); + writeUint16(archive, offset + 8, entries.length); + writeUint16(archive, offset + 10, entries.length); + writeUint32(archive, offset + 12, centralDirectoryLength); + writeUint32(archive, offset + 16, centralDirectoryOffset); + + return archive; +} diff --git a/cli/src/commands/env-lab.ts b/cli/src/commands/env-lab.ts index 55227c9f631..2e65bd36bf1 100644 --- a/cli/src/commands/env-lab.ts +++ b/cli/src/commands/env-lab.ts @@ -1,4 +1,5 @@ import path from "node:path"; +import { fileURLToPath } from "node:url"; import type { Command } from "commander"; import * as p from "@clack/prompts"; import pc from "picocolors"; @@ -111,6 +112,76 @@ export async function envLabDownCommand(opts: { instance?: string; json?: boolea p.log.message(`State: ${pc.dim(statePath)}`); } +// Quote one argument for a POSIX shell. The env-lab cleanup hint is copyable, so +// a contributor can paste it into a shell. A checkout path can hold shell +// metacharacters, such as `$`, a backtick, or a double quote. Inside double +// quotes a POSIX shell still expands `$(...)`, a backtick pair, and `$NAME`, and +// a double quote in the path ends the quoted span. So double quotes do not make +// the path safe. Single quotes stop every expansion. This function wraps the +// value in single quotes and rewrites each embedded single quote as the `'\''` +// sequence. The shell then reads the exact path and runs no embedded command. +function shellQuoteArgument(value: string): string { + return "'" + value.replace(/'/g, "'\\''") + "'"; +} + +// Describe how to re-run the env-lab CLI to stop the fixture. The bundled build +// emits one `dist/index.js` file, so node runs that file directly and `tsxBin` +// is `null`. A source checkout runs `src/index.ts` through the checked-out tsx +// runner, because the entry is TypeScript. +interface EnvLabCliInvocation { + entry: string; + tsxBin: string | null; +} + +// Resolve how to re-run the CLI from the running module location. The cleanup +// hint must run the same CLI that prints it, so it stops the correct version. +// `import.meta.url` gives the running module. The bundled build runs this module +// from `/dist/index.js`, so the hint runs that exact file with node. The +// published package ships no `src` directory and no tsx runner. A source +// checkout runs this module from `/src/commands/env-lab.ts`, so the hint +// runs `/src/index.ts` through the checked-out tsx runner. This resolver +// reads an absolute path from the module location, so the hint works from any +// working directory. The `modulePath` parameter is a test seam; production +// callers use the running module path. +export function resolveEnvLabCliInvocation( + modulePath: string = fileURLToPath(import.meta.url), +): EnvLabCliInvocation { + const moduleDir = path.dirname(modulePath); + const isSourceCheckout = + path.basename(moduleDir) === "commands" && path.basename(path.dirname(moduleDir)) === "src"; + if (isSourceCheckout) { + const cliRoot = path.resolve(moduleDir, "..", ".."); + return { + entry: path.join(cliRoot, "src", "index.ts"), + tsxBin: path.join(cliRoot, "node_modules", "tsx", "dist", "cli.mjs"), + }; + } + return { entry: modulePath, tsxBin: null }; +} + +// Build the env-lab cleanup hint as a copyable shell command. The hint stops the +// fixture that `env-lab doctor` inspected. It runs the same CLI that prints it, +// so it stops the correct version, and it forwards the inspected instance, so it +// stops the correct instance. It passes an inert `argv` value, so no shell reads +// the argument. Each path and the instance id pass through `shellQuoteArgument`, +// so a shell metacharacter stays inert when a contributor pastes the command. +// The `invocation` parameter is a test seam; production callers use the resolved +// running-module invocation. +export function buildEnvLabCleanupCommand( + opts: { instance?: string; invocation?: EnvLabCliInvocation } = {}, +): string { + const invocation = opts.invocation ?? resolveEnvLabCliInvocation(); + const parts = ["node"]; + if (invocation.tsxBin !== null) { + parts.push(shellQuoteArgument(invocation.tsxBin)); + } + parts.push(shellQuoteArgument(invocation.entry), "env-lab down"); + if (opts.instance !== undefined) { + parts.push("--instance", shellQuoteArgument(opts.instance)); + } + return parts.join(" "); +} + export async function envLabDoctorCommand(opts: { instance?: string; json?: boolean }) { const status = await collectEnvLabDoctorStatus(opts); @@ -138,7 +209,19 @@ export async function envLabDoctorCommand(opts: { instance?: string; json?: bool p.log.message(`State: ${pc.dim(status.statePath)}`); } - p.log.message(`Cleanup: ${pc.dim("pnpm paperclipai env-lab down")}`); + // The cleanup hint runs the same CLI that prints it, so it stops the correct + // version. The bundled build runs `dist/index.js`; a source checkout runs + // `src/index.ts` through the checked-out tsx runner. The hint uses absolute + // paths, so it works from any working directory. It passes an inert `argv` + // value, so no shell reads the argument. See `doc/CLI.md`, "safe invocation". + // + // The doctor diagnoses the instance that `resolvePaperclipInstanceId` selects + // from `opts.instance` or the `PAPERCLIP_INSTANCE_ID` environment variable. + // The hint pins that resolved instance, so a contributor who pastes the hint + // in a shell without `PAPERCLIP_INSTANCE_ID` stops the diagnosed fixture, not + // the default instance. + const cleanupInstance = resolvePaperclipInstanceId(opts.instance); + p.log.message(`Cleanup: ${pc.dim(buildEnvLabCleanupCommand({ instance: cleanupInstance }))}`); } export function registerEnvLabCommands(program: Command) { diff --git a/cli/src/commands/worktree.ts b/cli/src/commands/worktree.ts index 676ee7d52ff..fe07f3ce618 100644 --- a/cli/src/commands/worktree.ts +++ b/cli/src/commands/worktree.ts @@ -2218,10 +2218,13 @@ async function closeDb(db: ClosableDb): Promise { await db.$client?.end?.({ timeout: 5 }).catch(() => undefined); } -function resolveCurrentEndpoint(): ResolvedWorktreeEndpoint { +export function resolveCurrentWorktreeEndpoint(): ResolvedWorktreeEndpoint { + const cwd = path.resolve(process.cwd()); + const rootPath = detectGitWorkspaceInfo(cwd)?.root ?? cwd; + const localConfigPath = path.join(rootPath, ".paperclip", "config.json"); return { - rootPath: path.resolve(process.cwd()), - configPath: resolveConfigPath(), + rootPath, + configPath: existsSync(localConfigPath) ? localConfigPath : resolveConfigPath(), label: "current", isCurrent: true, }; @@ -2233,7 +2236,7 @@ function resolveAttachmentLookupStorages(input: { }): ConfiguredStorage[] { const orderedConfigPaths = [ input.sourceEndpoint.configPath, - resolveCurrentEndpoint().configPath, + resolveCurrentWorktreeEndpoint().configPath, input.targetEndpoint.configPath, ...toMergeSourceChoices(process.cwd()) .filter((choice) => choice.hasPaperclipConfig) @@ -2804,7 +2807,7 @@ export async function worktreeListCommand(opts: WorktreeListOptions): Promise { const excluded = excludeWorktreePath ? path.resolve(excludeWorktreePath) : null; - const currentEndpoint = resolveCurrentEndpoint(); + const currentEndpoint = resolveCurrentWorktreeEndpoint(); const choices = toMergeSourceChoices(process.cwd()) .filter((choice) => choice.hasPaperclipConfig || choice.isCurrent) .filter((choice) => path.resolve(choice.worktree) !== excluded) @@ -3295,7 +3298,7 @@ export async function worktreeMergeHistoryCommand(sourceArg: string | undefined, const targetEndpoint = opts.to ? resolveWorktreeEndpointFromSelector(opts.to, { allowCurrent: true }) - : resolveCurrentEndpoint(); + : resolveCurrentWorktreeEndpoint(); const sourceEndpoint = opts.from ? resolveWorktreeEndpointFromSelector(opts.from, { allowCurrent: true }) : sourceArg @@ -3400,7 +3403,7 @@ async function runWorktreeReseed(opts: WorktreeReseedOptions): Promise { const targetEndpoint = opts.to ? resolveWorktreeEndpointFromSelector(opts.to, { allowCurrent: true }) - : resolveCurrentEndpoint(); + : resolveCurrentWorktreeEndpoint(); const source = resolveWorktreeReseedSource(opts); if (path.resolve(source.configPath) === path.resolve(targetEndpoint.configPath)) { diff --git a/cli/src/index.ts b/cli/src/index.ts index fd8d1ee4f77..b26b5cbe187 100644 --- a/cli/src/index.ts +++ b/cli/src/index.ts @@ -2,6 +2,7 @@ import { Command } from "commander"; import { onboard } from "./commands/onboard.js"; import { doctor } from "./commands/doctor.js"; import { envCommand } from "./commands/env.js"; +import { channelsCommand } from "./commands/channels.js"; import { configure } from "./commands/configure.js"; import { addAllowedHostname } from "./commands/allowed-hostname.js"; import { heartbeatRun } from "./commands/heartbeat-run.js"; @@ -130,6 +131,14 @@ program .option("-d, --data-dir ", DATA_DIR_OPTION_HELP) .action(envCommand); +program + .command("channels") + .description("Show the release channels and which one this install follows") + .option("--json", "Machine-readable output") + .action(async (opts) => { + await channelsCommand(opts); + }); + program .command("configure") .description("Update configuration sections") diff --git a/doc/CHANNELS.md b/doc/CHANNELS.md new file mode 100644 index 00000000000..f809a386fc4 --- /dev/null +++ b/doc/CHANNELS.md @@ -0,0 +1,99 @@ +# Release Channels + +Paperclip ships on four channels. Pick the one that matches your appetite for +freshness versus stability — switching is just a matter of which version you +install. + +| Channel | What it is | Updates | npm | Docker | +| --- | --- | --- | --- | --- | +| `stable` | The recommended release | every week or two | `paperclipai@latest` | `ghcr.io/paperclipai/paperclip:latest` | +| `beta` | Release candidates soaking before stable | when promoted | `paperclipai@beta` | `ghcr.io/paperclipai/paperclip:beta` | +| `nightly` | Yesterday's merges, smoke-tested as a unit | once a night | `paperclipai@nightly` | `ghcr.io/paperclipai/paperclip:nightly` | +| `canary` | Every merge to `master`, as it happens | many times a day | `paperclipai@canary` | `ghcr.io/paperclipai/paperclip:canary` | + +## Choosing a channel + +**stable** is the right choice for almost everyone. It only moves when a +release has been explicitly vetted and promoted by a maintainer, and every +stable must first soak as a beta for at least 3 days. + +**beta** is for people who want the next stable early. A beta is a nightly +that a maintainer hand-picked and explicitly promoted behind an approval +gate, and it is re-smoked after publishing. Betas are the release candidates: +what you run on beta today is what stable becomes a few days later. + +**nightly** is for people who want new features quickly but not the churn of +tracking every merge. Once a night, the newest master build that published +green is run through the full release smoke suite (real Docker container, real +onboarding flow, browser-driven). Only if that passes does it ship as the +nightly. If smoke fails, there is no nightly that night — the channel never +ships a build that failed its checks. + +**canary** is the bleeding edge: it publishes on every merge to `master`. +It is primarily the lane that continuously exercises our release automation, +but it's available to anyone who wants the newest bits and accepts the risk. + +## Installing from a channel + +npm / npx: + +```bash +npx paperclipai@latest onboard # stable +npx paperclipai@beta onboard +npx paperclipai@nightly onboard +npx paperclipai@canary onboard +``` + +Docker: + +```bash +docker pull ghcr.io/paperclipai/paperclip:latest # stable +docker pull ghcr.io/paperclipai/paperclip:beta +docker pull ghcr.io/paperclipai/paperclip:nightly +docker pull ghcr.io/paperclipai/paperclip:canary +``` + +Every image is also published as `:sha-` for exact pinning, and +stable images additionally get `:YYYY.MDD.P` version tags. + +## Seeing where you are + +```bash +npx paperclipai channels +``` + +prints every channel with the version it currently resolves to, the install +command for each, and which channel your install follows (with `--json` for +scripting). + +## Switching channels + +Channel choice is per-install: install from a different tag and you're on that +channel. Moving forward (stable → nightly) is always safe. Moving backward +(nightly → stable) can mean running an older schema than your data was created +with — treat a downgrade like a restore and keep a backup of your data +directory before switching down. + +## Reading version strings + +The version tells you which channel a build came from: + +- `2026.807.0` — stable, published Aug 7 2026 +- `2026.807.0-beta.0` — beta promoted on Aug 7 2026 +- `2026.807.0-nightly.0` — nightly cut on Aug 7 2026 +- `2026.807.0-canary.4` — the fifth canary for the Aug 7 line + +Each promotion republishes the exact source commit of the previous lane's +build: a nightly shares its source SHA with a canary, and a beta with a +nightly. The version dates the promotion, and the shared SHA is visible in +the release job summaries and as git tags on the commit. + +One quirk to be aware of: npm's semver ordering compares prerelease names +alphabetically, so `-beta.N` sorts below `-canary.N`, which sorts below +`-nightly.N` for the same base version. This never matters when installing by +dist-tag (the recommended way), only if you write version ranges by hand. + +## For maintainers + +The publishing mechanics, promotion flow, and release checklist live in +[`RELEASING.md`](RELEASING.md). diff --git a/doc/CLI.md b/doc/CLI.md index 5c75aeb4e85..f01291b2994 100644 --- a/doc/CLI.md +++ b/doc/CLI.md @@ -6,6 +6,115 @@ Paperclip CLI now supports both: - instance setup/diagnostics (`onboard`, `doctor`, `configure`, `env`, `allowed-hostname`, `env-lab`) - control-plane client operations (issues, approvals, agents, activity, dashboard) +## Security: safe invocation for content-bearing arguments + +Use `npx paperclipai` for any command whose argument can hold untrusted or +semi-trusted content. Untrusted content includes issue text, comment bodies, +Markdown, pasted snippets, and model output. `npx` runs the CLI binary directly. +It passes the argument as an inert `argv` value. It does not run a shell over the +value. `npx paperclipai` works on any machine with Node: it runs a local install +of the `paperclipai` package, and it fetches the published package when no local +install is present. + +Do not use `pnpm paperclipai` for a content-bearing argument. `pnpm paperclipai` +is a `package.json` script. `pnpm` builds a `/bin/sh` command string and appends +the argument to it, so the shell reads the argument first. The shell interprets +these spans before the CLI starts: + +- command substitution: a backtick pair or `$( )` +- variable expansion: `$NAME` or `${NAME}` (this can leak a secret value into the persisted argument) + +A crafted value can run an arbitrary command as the invoking user. A crafted +value can also expand an environment variable into the stored argument. No +CLI-side check stops this, because the shell runs before `cli/src` starts. This +is true even when the argument comes from a quoted shell variable, because `pnpm` +re-evaluates the value in its own shell. + +Safe forms: + +- `npx paperclipai ` — the documented default. It passes an inert + `argv` value and runs on any machine. +- `node cli/node_modules/tsx/dist/cli.mjs cli/src/index.ts ` — + the safe form to run the local source from a monorepo checkout. It is the exact + command that the `pnpm paperclipai` script wraps, but it runs directly, so no + shell reads the argument. Use it when you must test your local `cli/src` + changes with a content-bearing argument. + +Unsafe or broken forms: + +- `pnpm paperclipai ` — unsafe. `pnpm` runs the argument through a + shell first. +- `pnpm run - - -
- - - diff --git a/ui/package.json b/ui/package.json index 6788beded3f..1e81e21011c 100644 --- a/ui/package.json +++ b/ui/package.json @@ -33,7 +33,7 @@ "@dnd-kit/core": "^6.3.1", "@dnd-kit/sortable": "^10.0.0", "@dnd-kit/utilities": "^3.2.2", - "@lexical/link": "0.48.0", + "@lexical/link": "0.49.0", "@mdxeditor/editor": "^3.55.0", "@paperclipai/adapter-claude-local": "workspace:*", "@paperclipai/adapter-codex-local": "workspace:*", @@ -56,10 +56,9 @@ "clsx": "^2.1.1", "cmdk": "^1.1.1", "i18next": "^26.3.6", - "lexical": "0.48.0", + "lexical": "0.49.0", "lucide-react": "^0.577.0", - "mermaid": "^11.16.0", - "motion": "^12.42.2", + "mermaid": "^11.16.1", "radix-ui": "^1.6.4", "react": "^19.2.7", "react-dom": "^19.2.7", @@ -68,8 +67,7 @@ "react-resizable-panels": "^4.12.2", "react-router-dom": "^7.18.1", "remark-gfm": "^4.0.1", - "tailwind-merge": "^3.6.0", - "three": "^0.185.1" + "tailwind-merge": "^3.6.0" }, "devDependencies": { "@storybook/addon-a11y": "10.5.4", @@ -79,12 +77,11 @@ "@types/node": "^22.20.1", "@types/react": "^19.2.17", "@types/react-dom": "^19.2.3", - "@types/three": "^0.185.1", "@vitejs/plugin-react": "^4.3.4", "storybook": "10.5.5", "tailwindcss": "^4.3.2", "typescript": "^5.7.3", - "vite": "^6.1.0", + "vite": "^6.4.3", "vitest": "^4.1.10" } } diff --git a/ui/public/sw.js b/ui/public/sw.js index f90d12156f8..e5997304dc8 100644 --- a/ui/public/sw.js +++ b/ui/public/sw.js @@ -32,11 +32,16 @@ self.addEventListener("fetch", (event) => { } return response; }) - .catch(() => { + .catch(async () => { + // caches.match() resolves undefined on a miss (and the promise itself + // is always truthy, so `||` can never supply a fallback). respondWith + // must always receive a real Response — resolving undefined breaks + // the navigation with "Failed to convert value to 'Response'" instead + // of showing anything. if (request.mode === "navigate") { - return caches.match("/") || new Response("Offline", { status: 503 }); + return (await caches.match("/")) ?? new Response("Offline", { status: 503 }); } - return caches.match(request); + return (await caches.match(request)) ?? Response.error(); }) ); }); diff --git a/ui/src/App.activity-routing.test.tsx b/ui/src/App.activity-routing.test.tsx index 534bf3ecb11..5a7e5ce7de4 100644 --- a/ui/src/App.activity-routing.test.tsx +++ b/ui/src/App.activity-routing.test.tsx @@ -111,12 +111,16 @@ function renderAppAt(container: HTMLElement, path: string) { return root; } +/** + * Waits on the condition, not on a fixed number of turns. The previous version + * yielded at most five macrotasks before asserting, which is ample on an idle + * machine and not when the suite is running many workers in parallel — the + * container was still empty and the assertion failed on a route that resolves + * perfectly well. `vi.waitFor` retries against a time budget instead, so a + * loaded worker gets more turns rather than a failure. + */ async function waitForRoute(container: HTMLElement, text: string) { - for (let attempt = 0; attempt < 5; attempt += 1) { - if (container.textContent?.includes(text)) return; - await new Promise((resolve) => window.setTimeout(resolve, 0)); - } - expect(container.textContent).toContain(text); + await vi.waitFor(() => expect(container.textContent).toContain(text)); } describe("App Activity routing (PAP-16302)", () => { diff --git a/ui/src/App.cases-routing.test.tsx b/ui/src/App.cases-routing.test.tsx index a46b31b371f..7a85e819814 100644 --- a/ui/src/App.cases-routing.test.tsx +++ b/ui/src/App.cases-routing.test.tsx @@ -100,12 +100,19 @@ async function renderAppAt(container: HTMLElement, path: string) { return root; } +/** + * Waits on the condition, not on a fixed number of turns. The previous version + * yielded at most three macrotasks before asserting, which is ample on an idle + * machine and not when the suite is running many workers in parallel — the + * container was still empty and the assertion failed on a route that resolves + * perfectly well. `vi.waitFor` retries against a time budget instead, so a + * loaded worker gets more turns rather than a failure. + * + * The same fix #11499 applied to the sibling `App.activity-routing.test.tsx`, + * which had the identical loop with five turns instead of three. + */ async function waitForRoute(container: HTMLElement, text: string) { - for (let attempt = 0; attempt < 3; attempt += 1) { - if (container.textContent?.includes(text)) return; - await new Promise((resolve) => window.setTimeout(resolve, 0)); - } - expect(container.textContent).toContain(text); + await vi.waitFor(() => expect(container.textContent).toContain(text)); } describe("App Cases routing (PAP-13002)", () => { diff --git a/ui/src/App.onboarding-launcher.test.tsx b/ui/src/App.onboarding-launcher.test.tsx new file mode 100644 index 00000000000..84e14e530c9 --- /dev/null +++ b/ui/src/App.onboarding-launcher.test.tsx @@ -0,0 +1,218 @@ +// @vitest-environment jsdom + +import { act } from "react"; +import { createRoot, type Root } from "react-dom/client"; +import { QueryClient, QueryClientProvider } from "@tanstack/react-query"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { + ONBOARDING_AGENT_STEP, + ONBOARDING_MISSION_STEP, +} from "./lib/onboarding-route"; + +/** + * The third way into onboarding for a company that already exists: the "Add + * Agent" card on `/{prefix}/onboarding`, shown once the wizard is dismissed. + * + * The route resolver and the dashboard both choose the step from whether the + * company already has its mission. This button hardcoded the mission step, so + * the one entry point whose own copy says "add another agent" was the one that + * stopped to ask for the mission again. + */ + +// eslint-disable-next-line @typescript-eslint/no-explicit-any +(globalThis as any).IS_REACT_ACT_ENVIRONMENT = true; + +// jsdom's CSS parser rejects the custom-property marker rule stitches inserts +// (`--sxs{--sxs:N}`), pulled into 's eager import graph transitively via +// @codesandbox/sandpack-react. Substitute a benign, valid rule on parse failure +// so stitches' index bookkeeping stays intact and the module graph evaluates. +vi.hoisted(() => { + const sheetProto = window.CSSStyleSheet.prototype as unknown as { + insertRule: (rule: string, index?: number) => number; + __papLauncherPatched?: boolean; + }; + if (!sheetProto.__papLauncherPatched) { + const original = sheetProto.insertRule; + sheetProto.insertRule = function patched(this: CSSStyleSheet, rule: string, index?: number) { + try { + return original.call(this, rule, index); + } catch { + try { + return original.call(this, ".pap-launcher-noop{}", index); + } catch { + return this.cssRules?.length ?? 0; + } + } + }; + sheetProto.__papLauncherPatched = true; + } +}); + +const routeState = vi.hoisted(() => ({ companyPrefix: "PC1" as string | undefined })); +const dialogState = vi.hoisted(() => ({ + onboardingOpen: false, + onboardingRouteDismissed: true, + openOnboarding: vi.fn(), +})); +const mockGoalsApi = vi.hoisted(() => ({ list: vi.fn() })); + +vi.mock("@/lib/router", async () => { + const actual = await vi.importActual("react-router-dom"); + return { + ...actual, + useParams: () => ({ companyPrefix: routeState.companyPrefix }), + useActiveCompanyPrefix: () => routeState.companyPrefix ?? null, + }; +}); + +vi.mock("./context/CompanyContext", () => ({ + useCompany: () => ({ + companies: [{ id: "company-1", name: "Acme", issuePrefix: "PC1" }], + selectedCompanyId: "company-1", + loading: false, + error: null, + }), +})); + +vi.mock("./context/DialogContext", () => ({ + useDialogActions: () => ({ openOnboarding: dialogState.openOnboarding }), + useDialogState: () => ({ + onboardingOpen: dialogState.onboardingOpen, + onboardingRouteDismissed: dialogState.onboardingRouteDismissed, + }), +})); + +vi.mock("./api/goals", () => ({ goalsApi: mockGoalsApi })); + +const { OnboardingRoutePage } = await import("./App"); + +const COMPANY_GOAL = { + id: "goal-1", + companyId: "company-1", + title: "Scale the marketplace", + description: null, + level: "company", + status: "active", + parentId: null, + ownerAgentId: null, + createdAt: new Date("2026-03-02T00:00:00Z"), + updatedAt: new Date("2026-03-02T00:00:00Z"), +}; + +describe("the onboarding launcher's Add Agent button", () => { + let container: HTMLDivElement; + let queryClient: QueryClient; + let root: Root | null = null; + + async function render() { + root = createRoot(container); + await act(async () => { + root!.render( + + + , + ); + }); + } + + async function settle(ticks = 12) { + for (let i = 0; i < ticks; i++) { + await act(async () => { + await new Promise((resolve) => setTimeout(resolve, 0)); + }); + } + } + + function addAgentButton(): HTMLButtonElement { + const button = [...container.querySelectorAll("button")].find((b) => + b.textContent?.includes("Add Agent"), + ); + // Asserted rather than optional-chained: a lookup that silently matches + // nothing turns the click below into a no-op and the test into decoration. + expect(button).toBeDefined(); + return button!; + } + + beforeEach(() => { + container = document.createElement("div"); + document.body.appendChild(container); + queryClient = new QueryClient({ defaultOptions: { queries: { retry: false } } }); + routeState.companyPrefix = "PC1"; + dialogState.onboardingOpen = false; + dialogState.onboardingRouteDismissed = true; + mockGoalsApi.list.mockResolvedValue([]); + }); + + afterEach(async () => { + await act(async () => root?.unmount()); + root = null; + queryClient.clear(); + container.remove(); + document.body.innerHTML = ""; + vi.clearAllMocks(); + }); + + it("opens on the agent step for a company that already has its mission", async () => { + mockGoalsApi.list.mockResolvedValue([COMPANY_GOAL]); + await render(); + await settle(); + + await act(async () => { + addAgentButton().dispatchEvent(new MouseEvent("click", { bubbles: true })); + }); + + expect(dialogState.openOnboarding).toHaveBeenCalledWith({ + initialStep: ONBOARDING_AGENT_STEP, + companyId: "company-1", + }); + }); + + it("still asks for the mission when the company has none", async () => { + mockGoalsApi.list.mockResolvedValue([]); + await render(); + await settle(); + + await act(async () => { + addAgentButton().dispatchEvent(new MouseEvent("click", { bubbles: true })); + }); + + expect(dialogState.openOnboarding).toHaveBeenCalledWith({ + initialStep: ONBOARDING_MISSION_STEP, + companyId: "company-1", + }); + }); + + it("asks for the mission when the lookup fails, rather than skipping it", async () => { + // Same fail-open rule the other two entry points follow: an unknown + // mission costs the step, which the customer can answer. Confirming it now + // updates the company's existing goal rather than adding a second one. + mockGoalsApi.list.mockRejectedValue(new Error("goals unavailable")); + await render(); + await settle(); + + await act(async () => { + addAgentButton().dispatchEvent(new MouseEvent("click", { bubbles: true })); + }); + + expect(dialogState.openOnboarding).toHaveBeenCalledWith({ + initialStep: ONBOARDING_MISSION_STEP, + companyId: "company-1", + }); + }); + + it("opens with no company when the prefix matches none", async () => { + routeState.companyPrefix = "NOPE"; + await render(); + await settle(); + + const start = [...container.querySelectorAll("button")].find((b) => + b.textContent?.includes("Start Onboarding"), + ); + expect(start).toBeDefined(); + await act(async () => { + start!.dispatchEvent(new MouseEvent("click", { bubbles: true })); + }); + + expect(dialogState.openOnboarding).toHaveBeenCalledWith(); + }); +}); diff --git a/ui/src/App.test.tsx b/ui/src/App.test.tsx index 9f7b3cd32d6..920f192baf8 100644 --- a/ui/src/App.test.tsx +++ b/ui/src/App.test.tsx @@ -43,17 +43,18 @@ vi.mock("@/lib/router", () => ({ useParams: () => ({}), })); -async function flushReact() { - await Promise.resolve(); - await new Promise((resolve) => window.setTimeout(resolve, 0)); -} - +/** + * Waits on the condition, not on a fixed number of turns. A hand-rolled retry + * loop is ample on an idle machine and not when the suite runs many workers in + * parallel: it gives up after N turns and reports a failure on behaviour that + * works. `vi.waitFor` retries against a time budget, so a loaded worker gets + * more turns instead. + * + * Same replacement as #11499 and #11521, which fixed the shorter-budget + * instances of this in the routing tests. + */ async function waitForText(container: HTMLElement, text: string) { - for (let attempt = 0; attempt < 20; attempt += 1) { - if (container.textContent?.includes(text)) return; - await flushReact(); - } - expect(container.textContent).toContain(text); + await vi.waitFor(() => expect(container.textContent).toContain(text)); } function renderGate(container: HTMLElement) { @@ -160,7 +161,7 @@ describe("CloudAccessGate", () => { expect(container.textContent).toContain("Finish setting up this Paperclip"); expect(container.textContent).toContain("Sign in / Create account"); - expect(container.textContent).toContain("pnpm paperclipai auth bootstrap-ceo"); + expect(container.textContent).toContain("npx paperclipai auth bootstrap-ceo"); expect(mockAccessApi.getCurrentBoardAccess).not.toHaveBeenCalled(); unmountRoot(root); @@ -242,3 +243,22 @@ describe("Skill Studio routes", () => { expect(createIndexes[1]).toBeLessThan(detailIndexes[1]!); }); }); + +describe("Apps routes", () => { + it("uses browse as the Apps landing page and gives connections a canonical URL", () => { + expect(appSource).toContain('} />'); + expect(appSource).toContain('} />'); + expect(appSource).toContain('} />'); + expect(appSource).toContain('} />'); + expect(appSource).toContain('} />'); + }); +}); + +describe("Decisions routes", () => { + it("does not register decision-training views", () => { + expect(appSource).not.toContain('path="decisions/training"'); + expect(appSource).not.toContain('path="decisions/training/:id"'); + expect(appSource).not.toContain('path="training"'); + expect(appSource).not.toContain('path="training/:id"'); + }); +}); diff --git a/ui/src/App.tsx b/ui/src/App.tsx index 0e309f4fa20..29609322666 100644 --- a/ui/src/App.tsx +++ b/ui/src/App.tsx @@ -3,7 +3,6 @@ import { Button } from "@/components/ui/button"; import { useTranslation } from "@/i18n"; import { Layout } from "./components/Layout"; import { ConferenceRoomChatGate } from "./components/ConferenceRoomChatGate"; -import { TaskChatRedesignGate } from "./components/TaskChatRedesignGate"; import { TaskChatLab } from "./pages/TaskChatLab"; import { PipelinesExperimentalGate } from "./components/PipelinesExperimentalGate"; import { CasesExperimentalGate } from "./components/CasesExperimentalGate"; @@ -45,7 +44,6 @@ import { CompanyActivity } from "./pages/audit/CompanyActivity"; import { Inbox } from "./pages/Inbox"; import { WhatNeedsMe } from "./pages/WhatNeedsMe"; import { DecisionQueuePage } from "./pages/DecisionQueuePage"; -import { TrainingInspector, TrainingLibrary } from "./pages/Training"; import { BoardChat } from "./pages/BoardChat"; import { CompanySettings } from "./pages/CompanySettings"; import { CompanyEnvironments } from "./pages/CompanyEnvironments"; @@ -95,8 +93,10 @@ import { useDialogActions, useDialogState } from "./context/DialogContext"; import { loadLastInboxTab } from "./lib/inbox"; import { isOnboardingWizardActive, + onboardingStepForCompany, shouldRedirectCompanylessRouteToOnboarding, } from "./lib/onboarding-route"; +import { useCompanyMission } from "./hooks/useCompanyMission"; import { normalizeRememberedInstanceSettingsPath } from "./lib/instance-settings"; function boardRoutes() { @@ -122,14 +122,15 @@ function boardRoutes() { } /> } /> }> - } /> - } /> + } /> + } /> + } /> } /> - } /> - } /> + } /> + } /> } /> {/* Needs attention folded into Connections (PAP-13254); keep legacy links working. */} - } /> + } /> } /> } /> } /> @@ -275,20 +276,12 @@ function boardRoutes() { } /> } /> - {/* Task Chat Redesign dev harness — dev builds only, and additionally - gated by enableTaskChatRedesign (redirects to /dashboard when the - flag is off). */} + {/* Task chat dev harness — dev builds only. */} {import.meta.env.DEV ? ( - }> - } /> - + } /> ) : null} } /> } /> - } /> - } /> - } /> - } /> } /> } /> } /> @@ -309,18 +302,13 @@ function boardRoutes() { function AppsConnectEntryRoute() { const location = useLocation(); const searchParams = new URLSearchParams(location.search); - return canEnterAppsConnect(searchParams) ? : ; + return canEnterAppsConnect(searchParams) ? : ; } function InboxRootRedirect() { return ; } -function LegacyTrainingRedirect() { - const { id } = useParams<{ id: string }>(); - return ; -} - function LegacySkillStudioRedirect() { const location = useLocation(); const { companies, selectedCompany, loading } = useCompany(); @@ -403,15 +391,22 @@ function LegacyToolsRedirect() { function legacyToolsRedirectTarget(tab?: string) { if (!tab) return "/apps/advanced/profiles"; - if (tab === "applications" || tab === "connections" || tab === "overview" || tab === "examples") return "/apps"; + if (tab === "applications" || tab === "connections" || tab === "overview" || tab === "examples") return "/apps/connections"; return `/apps/advanced/${tab}`; } -function OnboardingRoutePage() { +export function OnboardingRoutePage() { const { companies } = useCompany(); const { openOnboarding } = useDialogActions(); const { onboardingOpen, onboardingRouteDismissed } = useDialogState(); const { companyPrefix } = useParams<{ companyPrefix?: string }>(); + const matchedCompany = companyPrefix + ? companies.find((company) => company.issuePrefix.toUpperCase() === companyPrefix.toUpperCase()) ?? null + : null; + // Which step this company belongs on, by the same rule the route resolver + // and the dashboard already use. Resolved above the early return below, + // because a hook cannot be called after it. + const { hasMission } = useCompanyMission(matchedCompany?.id ?? null); // The OnboardingWizard auto-opens on this route (and can also be opened // explicitly). While it is showing it covers the whole screen, so the @@ -421,9 +416,6 @@ function OnboardingRoutePage() { if (isOnboardingWizardActive({ onboardingOpen, routeDismissed: onboardingRouteDismissed })) { return null; } - const matchedCompany = companyPrefix - ? companies.find((company) => company.issuePrefix.toUpperCase() === companyPrefix.toUpperCase()) ?? null - : null; const title = matchedCompany ? `Add another agent to ${matchedCompany.name}` @@ -445,7 +437,16 @@ function OnboardingRoutePage() { agentAction.mutate("invoke")} + onClick={() => { + if (navigateToRunOnInvoke && !confirmNavigationStart(agentActionStartedDirtyRef)) return; + agentAction.mutate("invoke"); + }} disabled={assignAndRunDisabled} label={runLabel} size={size} @@ -423,8 +454,9 @@ export function AgentActionButtons({ + + + )} + +
+ {cards + ?

Secret access

+ :
Secret access
+ } +
+

{help.secretAccess}

+ createSecret.mutateAsync({ name, value })} + proposals={agentBindingProposals} + onApproveProposal={proposalReview.requestApprove} + onRejectProposal={proposalReview.requestReject} + /> + {proposalReview.dialogs} +
+
+ + ); + } + return (
{/* ---- Floating Save button (edit mode, when dirty) ---- */} @@ -1228,6 +1498,19 @@ export function AgentConfigForm(props: AgentConfigFormProps) { )} + {showInlineAdapterTestEnvironmentFeedback && showAdapterLogin && ( + + )} + {/* Working directory */} {showLegacyWorkingDirectoryField && ( @@ -1441,8 +1724,7 @@ export function AgentConfigForm(props: AgentConfigFormProps) { value={ isCreate ? ((val!.envBindings ?? EMPTY_ENV) as Record) - : ((eff("adapterConfig", "env", (config.env ?? EMPTY_ENV) as Record)) - ) + : (eff("adapterConfig", "env", (config.env ?? EMPTY_ENV) as Record)) } secrets={availableSecrets} userSecretDefinitions={userSecretDefinitions} @@ -1458,20 +1740,6 @@ export function AgentConfigForm(props: AgentConfigFormProps) { /> - {!isCreate && ( - - - {proposalReview.dialogs} - - )} - {/* Edit-only: timeout + grace period */} {!isCreate && ( <> @@ -1633,6 +1901,825 @@ export function AgentConfigForm(props: AgentConfigFormProps) { ); } +// The public session states that end a login. The panel stops the status poll +// and shows a terminal message when the session reaches one of these. +const ADAPTER_LOGIN_TERMINAL_STATUSES = new Set([ + "authenticated", + "failed", + "timed_out", + "cancelled", +]); + +// The status route poll interval while a session is active (Decision A). The +// poll stops at a terminal state. +const ADAPTER_LOGIN_POLL_INTERVAL_MS = 2000; + +// A copy-to-clipboard button. It mirrors the workspace service control bar: a +// short "copied" flash, then it returns to the copy icon. +function AdapterLoginCopyButton({ value, label }: { value: string; label: string }) { + const [copied, setCopied] = useState(false); + const timeoutRef = useRef | null>(null); + useEffect( + () => () => { + if (timeoutRef.current) clearTimeout(timeoutRef.current); + }, + [], + ); + return ( + + ); +} + +// The terminal message for a finished login. It never shows a secret. It shows +// only the fixed, non-secret failure message the server returns. +function AdapterLoginTerminalState({ + status, + message, +}: { + status: AdapterAuthSessionStatus; + message: string | null; +}) { + if (status === "authenticated") { + return ( +
+ + Authenticated. The sandbox has credentials now. +
+ ); + } + const label = + status === "timed_out" + ? "Login timed out" + : status === "cancelled" + ? "Login cancelled" + : "Login failed"; + return ( +
+ + + {label} + {message ? `: ${message}` : "."} + +
+ ); +} + +// The login panel for one adapter in one sandbox environment. It starts a login +// session, polls the status route, and shows the one-time code and the +// authentication URL with copy and open actions. It shows the terminal states. +// It never writes the code, the URL, or any credential byte to a log line. +// +// The panel holds its own session state. The parent gives it a stable `key` from +// the adapter type and the environment id, so a change to either remounts the +// panel with a fresh session state. +// The props that identify one login panel: one adapter in one sandbox +// environment for one company. A parent that lifts the test feedback renders +// the panel from this descriptor. +export type AdapterLoginDescriptor = { + companyId: string; + adapterType: string; + environmentId: string; +}; + +// The panel props. `onStored` reports the non-secret `storedSessionId` claim from +// a Claude login that reaches the server `stored` state. The create-mode form +// holds that claim for the fixed binding; the callback never carries a token. +// `onApplyStored` binds the fixed reference to an existing stored login with no +// new login round trip. The panel shows the apply-existing affordance only when +// the status route reports a stored value. +export type AdapterLoginPanelProps = AdapterLoginDescriptor & { + onStored?: (storedSessionId: string) => void; + onApplyStored?: () => void; +}; + +// The login panel dispatcher. It picks the panel mode from the adapter. The +// Claude adapter shows the submitted-browser-code panel. Every other adapter +// shows the displayed-code panel. +export function AdapterLoginPanel(props: AdapterLoginPanelProps) { + if (props.adapterType === "claude_local") { + return ; + } + return ; +} + +function DisplayedCodeLoginPanel({ + companyId, + adapterType, + environmentId, +}: AdapterLoginPanelProps) { + const [sessionId, setSessionId] = useState(null); + const [startError, setStartError] = useState(null); + // The server delivers the one-time prompt on the first owner read only. Latch + // it so a later poll that returns a null prompt does not hide the code and the + // URL. + const [latchedPrompt, setLatchedPrompt] = useState(null); + + const startLogin = useMutation({ + mutationFn: () => agentsApi.startAdapterAuthLogin(companyId, adapterType, { environmentId }), + onSuccess: (session) => { + setStartError(null); + setLatchedPrompt(null); + setSessionId(session.sessionId); + }, + onError: (error) => { + setStartError(error instanceof Error ? error.message : "Could not start the login."); + }, + }); + + const cancelLogin = useMutation({ + mutationFn: () => agentsApi.cancelAdapterAuthLogin(companyId, adapterType, sessionId!), + onSuccess: () => { + // Reset local state, so the panel returns to its idle start state and the + // Log in button is available again. + setSessionId(null); + setLatchedPrompt(null); + setStartError(null); + }, + onError: (error) => { + setStartError(error instanceof Error ? error.message : "Could not cancel the login."); + }, + }); + + const statusQuery = useQuery({ + queryKey: ["adapter-login-status", companyId, adapterType, sessionId], + queryFn: () => agentsApi.getAdapterAuthLoginStatus(companyId, adapterType, sessionId!), + enabled: Boolean(sessionId), + refetchInterval: (query) => { + const status = query.state.data?.status; + return status && ADAPTER_LOGIN_TERMINAL_STATUSES.has(status) + ? false + : ADAPTER_LOGIN_POLL_INTERVAL_MS; + }, + }); + + // Latch the first non-null prompt for the current session. A later poll + // returns a null prompt after the one-time delivery, so keep the latched value. + useEffect(() => { + const next = statusQuery.data?.prompt ?? null; + if (next) setLatchedPrompt(next); + }, [statusQuery.data]); + + const session = statusQuery.data ?? startLogin.data ?? null; + const status = session?.status ?? null; + const prompt = latchedPrompt; + const isTerminal = status ? ADAPTER_LOGIN_TERMINAL_STATUSES.has(status) : false; + const isActive = Boolean(sessionId) && !isTerminal; + const startDisabled = startLogin.isPending || isActive; + + return ( +
+
+ Sign in to the sandbox +
+ {isActive && ( + + )} + +
+
+ + {startError && ( +
+ {startError} +
+ )} + + {/* One live region announces the loading, prompt, and terminal states, so a + screen reader reports each transition without a re-navigation. */} +
+ {isActive && !prompt && ( +
+ + Preparing the login… +
+ )} + + {isActive && prompt && ( +
+
+ Open the authentication page and enter the code. +
+
+
+
+ Code +
+ {prompt.code} +
+ +
+
+
+
+ Authentication URL +
+ {prompt.url} +
+
+ + +
+
+
+ )} + + {isTerminal && status && ( + + )} +
+
+ ); +} + +// The public session states that fail a Claude login. The panel returns to its +// start state and shows a fixed message at one of these. +const CLAUDE_LOGIN_FAILURE_STATUSES = new Set([ + "failed", + "timed_out", + "cancelled", +]); + +// The fixed, non-secret message for a failed Claude login. The panel shows this +// text and returns to its start state. It never shows a provider message that +// could carry a secret. +const CLAUDE_LOGIN_FAILED_MESSAGE = "The login did not finish. Start the login again."; + +// The client wall-clock cap for one active login. The panel polls the status +// route and the prompt route every two seconds. The server can leave a session +// short of a terminal status, and the prompt route returns 404 until the URL is +// ready. Without a cap, the panel polls forever. When this cap passes, the panel +// stops both polls and shows the timed-out state, so the user can start again. +// A generous client failsafe for a live login whose server deadline has not +// arrived yet. The server `expiresAt` is the real cutoff; the panel drives the +// timer from it. This failsafe only bounds a stuck poll while the status route +// has not yet returned `expiresAt`, so a login can never poll forever. The +// value is far longer than the server budget, so it never cuts a normal login +// short (the earlier fixed 60-second cutoff did, which drove this fix). +const CLAUDE_LOGIN_FAILSAFE_TIMEOUT_MS = 15 * 60_000; + +// The fixed, non-secret message for a timed-out Claude login. The panel shows +// this text, stops both polls, and returns to its start state. +const CLAUDE_LOGIN_TIMED_OUT_MESSAGE = "The login timed out. Start the login again."; + +// The submitted-browser-code login panel for the Claude adapter. It starts a +// setup-token login, polls the status route, and reads the authorization URL +// from the guarded prompt route. The user opens the URL, reads the browser code, +// and submits it. The panel clears the code right after submit. The panel treats +// only the server `stored` state as success, and it never shows the OAuth token. +function SubmittedBrowserCodeLoginPanel({ + companyId, + environmentId, + onStored, + onApplyStored, +}: AdapterLoginPanelProps) { + const [sessionId, setSessionId] = useState(null); + const [startError, setStartError] = useState(null); + // The server delivers the authorization URL on the guarded prompt read only. + // Latch it so a later poll does not hide the URL. + const [authorizationUrl, setAuthorizationUrl] = useState(null); + // True when the guarded prompt read reports a non-confidential transport. The + // server does not block a plain-HTTP login. The panel shows a non-blocking + // disclaimer and the login still proceeds. Latch it beside the URL. + const [transportInsecure, setTransportInsecure] = useState(false); + // The browser code the user submits. The panel clears it right after submit, so + // the secret never lingers in the input. + const [browserCode, setBrowserCode] = useState(""); + // The non-secret stored-session claim from a completed login. It marks the + // server `stored` state, the only success state. + const [storedSessionId, setStoredSessionId] = useState(null); + // True after a completion read fails. The panel returns to its start state. + const [completionFailed, setCompletionFailed] = useState(false); + // True after the client wall-clock cap passes for the active login. The panel + // stops both polls and shows the timed-out state. + const [timedOut, setTimedOut] = useState(false); + // True after the status poll returns 404. The server removes the row and the + // in-memory session at once on any non-stored terminal state, so a status 404 + // means the login failed and the server cleaned up. The panel stops both + // polls and shows a terminal failure state. It shows no credential material. + const [statusGone, setStatusGone] = useState(false); + // Guards the one completion read per session. + const completionStartedRef = useRef(false); + // True after the owner applies an existing stored login in this panel. The + // apply-existing path binds the fixed reference with no new login round trip, + // so the panel shows the applied confirmation and hides the apply affordance. + const [appliedStored, setAppliedStored] = useState(false); + + const resetLocalState = () => { + setStartError(null); + setAuthorizationUrl(null); + setTransportInsecure(false); + setBrowserCode(""); + setStoredSessionId(null); + setCompletionFailed(false); + setTimedOut(false); + setStatusGone(false); + completionStartedRef.current = false; + }; + + // Read the stored Claude OAuth token status when the panel mounts. A 200 body + // carries the secret id and the version of the owner value; a 404 means the + // owner has no stored value. The panel applies the stored token first: when a + // value exists, a replacement login rotates it under the captured version + // instead of a blind first write. The server rejects a stale capture, so a + // change by another process after the capture fails the overwrite. + const storedTokenQuery = useQuery({ + queryKey: ["claude-oauth-token-status", companyId], + queryFn: async () => { + try { + return await agentsApi.getClaudeOAuthTokenStatus(companyId); + } catch (error) { + // A 404 means the owner has no stored value (indistinguishable from a + // foreign value). The panel treats it as "no stored token". + if (error instanceof ApiError && error.status === 404) return null; + throw error; + } + }, + retry: (failureCount, error) => { + if (error instanceof ApiError && error.status === 404) return false; + return failureCount < 2; + }, + }); + const storedToken = storedTokenQuery.data ?? null; + + const startLogin = useMutation({ + mutationFn: () => + agentsApi.startClaudeSetupTokenLogin(companyId, { + environmentId, + // When the owner already has a stored token, the login rotates it under + // the captured version, so a replacement login never conflicts with an + // existing value. Without a stored token the login is a first write. + ...(storedToken + ? { + overwrite: { + expectedSecretId: storedToken.secretId, + expectedLatestVersion: storedToken.latestVersion, + }, + } + : {}), + }), + onSuccess: (session) => { + resetLocalState(); + setSessionId(session.sessionId); + }, + onError: (error) => { + setStartError(error instanceof Error ? error.message : "Could not start the login."); + }, + }); + + const clearActiveSession = () => { + // Return the panel to its idle start state. The Log in button is available + // again, and both polls stop because the session id is null. + setSessionId(null); + resetLocalState(); + }; + + const cancelLogin = useMutation({ + mutationFn: () => agentsApi.cancelClaudeSetupTokenLogin(companyId, sessionId!), + onSuccess: () => { + clearActiveSession(); + }, + onError: (error) => { + // The cancel is resilient. The server removes a terminal session, so a + // cancel of an already-terminal or unknown session can return a 404. Treat + // that 404 the same as a successful cancel: clear the session and stop the + // polls. The panel never keeps polling a session the server no longer + // holds. Any other error surfaces and keeps the active login. + if (error instanceof ApiError && error.status === 404) { + clearActiveSession(); + return; + } + setStartError(error instanceof Error ? error.message : "Could not cancel the login."); + }, + }); + + // Release the server session at once, without a change to the panel state. The + // client-cutoff timer and the unmount path both use this. The server holds a + // per-owner reservation until the session reaches a terminal state, so an + // abandoned session locks the owner out until the server deadline. A best- + // effort cancel frees that reservation now, so the same owner can start a new + // login and does not hit the "too many active sessions" cap. The server + // removes a terminal session, so a 404 means the session is already gone. Treat + // that 404 the same as a successful cancel. This is a fire-and-forget cleanup, + // so it drops every error. The manual Cancel button uses the `cancelLogin` + // mutation instead, because that path also returns the panel to its idle start + // state. + const releaseServerSession = useCallback( + (id: string) => { + void agentsApi.cancelClaudeSetupTokenLogin(companyId, id).catch(() => { + // Drop the error. A 404 means the server already removed the session. A + // cleanup path cannot surface any other error, so it stays silent. + }); + }, + [companyId], + ); + + // Both polls run only while a session is active and the client cap has not + // passed. The timeout stops the polls, so the panel never polls forever. A + // status 404 also stops the polls: the server cleaned up the session, so the + // panel enters a terminal failure state instead. + const pollingEnabled = Boolean(sessionId) && !timedOut && !statusGone; + + const statusQuery = useQuery({ + queryKey: ["claude-setup-token-status", companyId, sessionId], + queryFn: () => agentsApi.getClaudeSetupTokenLoginStatus(companyId, sessionId!), + enabled: pollingEnabled, + // A status 404 is terminal. The server removes a cleaned-up session at once, + // so a retry cannot recover it. Stop at once and fail loudly. + retry: (failureCount, error) => { + if (error instanceof ApiError && error.status === 404) return false; + return failureCount < 3; + }, + refetchInterval: (query) => { + if (timedOut || statusGone) return false; + const status = query.state.data?.status; + return status && ADAPTER_LOGIN_TERMINAL_STATUSES.has(status) + ? false + : ADAPTER_LOGIN_POLL_INTERVAL_MS; + }, + }); + + // Fail loudly on a status 404. The server cleans up the reservation, the row, + // and the in-memory session at once on any non-stored terminal state, so the + // status route returns 404 when a login fails and the server cleanup wins the + // race against the next poll. React Query keeps the last successful data on + // error, so without this branch the panel would hold stale data and show + // nothing. Enter the terminal failure state, which stops both polls. + useEffect(() => { + const error = statusQuery.error; + if (error instanceof ApiError && error.status === 404) { + setStatusGone(true); + } + }, [statusQuery.error]); + + // Poll the guarded prompt route until it returns the authorization URL. The + // route returns 404 until the URL is ready, so the panel treats a 404 as + // not-ready and keeps polling. + const promptQuery = useQuery({ + queryKey: ["claude-setup-token-prompt", companyId, sessionId], + enabled: pollingEnabled && !authorizationUrl, + queryFn: async () => { + try { + return await agentsApi.getClaudeSetupTokenLoginPrompt(companyId, sessionId!); + } catch (error) { + if (error instanceof ApiError && error.status === 404) return null; + throw error; + } + }, + refetchInterval: (query) => + timedOut || query.state.data?.authorizationUrl ? false : ADAPTER_LOGIN_POLL_INTERVAL_MS, + }); + + useEffect(() => { + const url = promptQuery.data?.authorizationUrl ?? null; + if (url) setAuthorizationUrl(url); + // The advisory rides the same guarded prompt read as the URL. Latch it, so a + // later poll does not clear the disclaimer. + if (promptQuery.data?.transportAdvisory) setTransportInsecure(true); + }, [promptQuery.data]); + + const submitCode = useMutation({ + mutationFn: (code: string) => + agentsApi.submitClaudeSetupTokenBrowserCode(companyId, sessionId!, code), + onError: (error) => { + setStartError(error instanceof Error ? error.message : "Could not submit the browser code."); + }, + }); + + const completeLogin = useMutation({ + mutationFn: () => agentsApi.completeClaudeSetupTokenLogin(companyId, sessionId!), + onSuccess: (result) => { + // The server reached the `stored` state. Hold the non-secret claim and + // report it to the parent. The response carries no token. + setStoredSessionId(result.storedSessionId); + onStored?.(result.storedSessionId); + }, + onError: () => { + setCompletionFailed(true); + }, + }); + + const status = statusQuery.data?.status ?? startLogin.data?.status ?? null; + // The server deadline for the active session (ISO string). The status route + // and the start response both carry it. The panel drives the client cutoff + // from this value, so the client and the server share one deadline. + const expiresAt = statusQuery.data?.expiresAt ?? startLogin.data?.expiresAt ?? null; + + // Complete the login once the server authenticates the session. The completion + // read returns the non-secret stored-session claim. Fire it once per session. + const completeLoginRef = useRef(completeLogin.mutate); + completeLoginRef.current = completeLogin.mutate; + useEffect(() => { + if (status === "authenticated" && !completionStartedRef.current) { + completionStartedRef.current = true; + completeLoginRef.current(); + } + }, [status]); + + const isStored = storedSessionId !== null; + const isFailure = + completionFailed || + statusGone || + Boolean(status && CLAUDE_LOGIN_FAILURE_STATUSES.has(status)); + const isCompleting = status === "authenticated" && !isStored && !completionFailed; + const isActive = Boolean(sessionId) && !isStored && !isFailure && !timedOut; + const startDisabled = startLogin.isPending || isActive; + + // Hold the active session id for the unmount cleanup. The panel updates it on + // every render. When the panel unmounts, or the parent removes it as the login + // closes, with an active, non-terminal session, the cleanup releases that + // session on the server. The ref is null once the session leaves the active + // state, so the cleanup never cancels a session the server already removed. + const activeSessionRef = useRef(null); + activeSessionRef.current = isActive ? sessionId : null; + + useEffect(() => { + return () => { + const id = activeSessionRef.current; + if (id) releaseServerSession(id); + }; + }, [releaseServerSession]); + + // Cap the active login at the server deadline. The timer arms when the login + // becomes active and clears when the login leaves the active state (a terminal + // status, a stored success, or a new login). It re-arms when `expiresAt` + // arrives or changes. When it fires, the panel enters the timed-out state. The + // `isActive` guard already excludes `timedOut`, so the timer does not re-arm + // after it fires. + // + // The delay comes from the server `expiresAt`, so the client cutoff matches + // the server session budget instead of a fixed short window. While `expiresAt` + // has not arrived for the live session, the panel uses a generous failsafe, so + // a stuck poll still stops. + useEffect(() => { + if (!isActive) return; + const deadlineMs = expiresAt !== null ? new Date(expiresAt).getTime() : Number.NaN; + const remainingMs = Number.isFinite(deadlineMs) + ? Math.max(0, deadlineMs - Date.now()) + : CLAUDE_LOGIN_FAILSAFE_TIMEOUT_MS; + const timer = setTimeout(() => { + // Release the server session first, then show the timed-out state. The + // cancel frees the server reservation now, so an immediate retry by the + // same owner starts a new session instead of a lockout on the per-owner + // cap. The panel keeps the timed-out state, so the user sees why the login + // stopped. + if (sessionId) releaseServerSession(sessionId); + setTimedOut(true); + }, remainingMs); + return () => clearTimeout(timer); + }, [isActive, expiresAt, sessionId, releaseServerSession]); + + const trimmedCode = browserCode.trim(); + const canSubmit = + isActive && + Boolean(authorizationUrl) && + !isCompleting && + isValidBrowserCode(trimmedCode) && + !submitCode.isPending; + + const handleSubmit = () => { + if (!canSubmit) return; + submitCode.mutate(trimmedCode); + // Clear the browser code right after submit, so the secret never lingers in + // the input. + setBrowserCode(""); + }; + + return ( +
+
+ Sign in to the sandbox +
+ {isActive && ( + + )} + {/* Apply the existing stored login with no new login round trip. The + affordance shows only when the status route reports a stored value + and no active or completed login runs in this panel. */} + {onApplyStored && storedToken && !isActive && !isStored && !appliedStored && ( + + )} + +
+
+ + {/* When the owner already has a stored Claude login, the panel applies it + first and does not force a fresh login. Use saved login binds the stored + token; a replacement login rotates it under the captured version. */} + {storedToken && !isActive && !isStored && !appliedStored && ( +
+ You have a saved Claude login. Use it to bind this agent, or log in again to replace the + stored token. +
+ )} + + {appliedStored && ( +
+ + The saved Claude login is bound to this agent now. +
+ )} + + {startError && ( +
+ {startError} +
+ )} + + {/* One live region announces the loading, prompt, completing, and terminal + states, so a screen reader reports each transition. */} +
+ {isActive && !authorizationUrl && !isCompleting && ( +
+ + Preparing the login… +
+ )} + + {isActive && authorizationUrl && !isCompleting && ( +
+ {transportInsecure && ( +
+ + + This connection is not encrypted. The login code travels in clear text on this + network. Continue only on a network you trust. + +
+ )} +
+ Open the authorization page, then enter the browser code it shows. +
+
+
+
+ Authorization URL +
+ {authorizationUrl} +
+
+ + +
+
+
+
+ Browser code +
+
+ setBrowserCode(event.target.value)} + onKeyDown={(event) => { + if (event.key === "Enter") { + event.preventDefault(); + handleSubmit(); + } + }} + className="flex-1 min-w-0 rounded-md border border-border bg-background px-2 py-1 font-mono text-xs text-foreground" + /> + +
+
+
+ )} + + {isCompleting && ( +
+ + Completing the login… +
+ )} + + {isStored && ( +
+ + Authenticated. The sandbox has credentials now. +
+ )} + + {isFailure && ( +
+ + {CLAUDE_LOGIN_FAILED_MESSAGE} +
+ )} + + {timedOut && !isFailure && !isStored && ( +
+ + {CLAUDE_LOGIN_TIMED_OUT_MESSAGE} +
+ )} +
+
+ ); +} + export function AdapterEnvironmentResult({ result }: { result: AdapterEnvironmentTestResult }) { const statusLabel = result.status === "pass" ? "Passed" : result.status === "warn" ? "Warnings" : "Failed"; diff --git a/ui/src/components/AgentSecretAccessEditor.test.tsx b/ui/src/components/AgentSecretAccessEditor.test.tsx index bc1df77a460..9078713752b 100644 --- a/ui/src/components/AgentSecretAccessEditor.test.tsx +++ b/ui/src/components/AgentSecretAccessEditor.test.tsx @@ -5,18 +5,29 @@ import { createRoot, type Root } from "react-dom/client"; import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import type { CompanySecret, EnvSecretRefBinding } from "@paperclipai/shared"; -// Stub SecretBindingPicker so the editor renders without CompanyContext / -// react-query. The stub exposes a button that binds a fixed secret. -vi.mock("./SecretBindingPicker", () => ({ - SecretBindingPicker: ({ - onChange, - }: { - onChange: (next: { secretId: string; version?: number | "latest" } | null) => void; - }) => ( - - ), +const mockSecretPickerRender = vi.hoisted(() => vi.fn()); + +// Keep this component test focused on access-row behavior while asserting that +// the editor routes selection through the shared, folder-aware env picker. +vi.mock("./environment-variables-editor/SecretPicker", () => ({ + SecretPicker: (props: { + secretId: string; + secrets: readonly CompanySecret[]; + onSelect: (secretId: string) => void; + onCreateNew?: (query: string) => void; + }) => { + mockSecretPickerRender(props); + return <> + + {props.onCreateNew ? ( + + ) : null} + ; + }, })); import { @@ -155,12 +166,43 @@ describe("AgentSecretAccessEditor component", () => { // Bind a secret via the stubbed picker. const pick = container.querySelector('[data-testid="pick-secret"]')!; + expect(mockSecretPickerRender).toHaveBeenLastCalledWith( + expect.objectContaining({ secretId: "", secrets }), + ); flushSync(() => pick.click()); const last = emitted.at(-1)!; expect(last).toEqual({ STRIPE: { type: "secret_ref", secretId: "s1", version: "latest" } }); }); + it("keeps the create form open when focus returns to the shared picker anchor", async () => { + vi.useFakeTimers(); + try { + render( + {}} + onCreateSecret={async () => secrets[0]!} + />, + ); + + const createButton = container.querySelector('[data-testid="create-secret"]')!; + flushSync(() => createButton.click()); + flushSync(() => vi.runAllTimers()); + expect(document.body.textContent).toContain("Create secret"); + + const pickerButton = container.querySelector('[data-testid="pick-secret"]')!; + pickerButton.focus(); + flushSync(() => {}); + + expect(document.body.textContent).toContain("Create secret"); + expect(document.querySelector('input[aria-label="Secret name"]')).toBeTruthy(); + } finally { + vi.useRealTimers(); + } + }); + it("renders pending binding proposals as Proposed rows with approve/reject", () => { const approved: string[] = []; const rejected: string[] = []; diff --git a/ui/src/components/AgentSecretAccessEditor.tsx b/ui/src/components/AgentSecretAccessEditor.tsx index 1152964836d..272e02568fb 100644 --- a/ui/src/components/AgentSecretAccessEditor.tsx +++ b/ui/src/components/AgentSecretAccessEditor.tsx @@ -9,14 +9,16 @@ import type { import { cn } from "../lib/utils"; import { Input } from "@/components/ui/input"; import { Badge } from "@/components/ui/badge"; -import { SecretBindingPicker, type SecretBindingValue } from "./SecretBindingPicker"; +import { SecretPicker } from "./environment-variables-editor/SecretPicker"; +import { CreateSecretPopover } from "./environment-variables-editor/CreateSecretPopover"; +import { Popover, PopoverAnchor, PopoverContent } from "@/components/ui/popover"; import { AGENT_ACCESS_CONFIG_PATH_PREFIX, ENV_CONFIG_PATH_PREFIX, SECRET_ALIAS_RE, deliveryModeDescription, } from "../lib/secret-delivery"; -import { envKeyFromSecretName } from "./environment-variables-editor/model"; +import { envKeyFromSecretName, secretNameFromKey } from "./environment-variables-editor/model"; import { DeliveryBadge as ProposalDeliveryBadge, ProposalActions, @@ -157,6 +159,8 @@ export interface AgentSecretAccessEditorProps { * parent diffs this against the current `access.*` keys to add/remove them. */ onChange: (next: Record) => void; + /** Create a company secret from the shared picker's pinned create action. */ + onCreateSecret?: (name: string, value: string) => Promise; disabled?: boolean; /** Pending binding proposals targeting this agent (PAP-14731). */ proposals?: readonly SecretProposalView[]; @@ -191,6 +195,7 @@ export function AgentSecretAccessEditor({ config, secrets, onChange, + onCreateSecret, disabled, proposals, onApproveProposal, @@ -208,6 +213,8 @@ export function AgentSecretAccessEditor({ const incomingKey = useMemo(() => normalizeAccessMapKey(incomingMap), [incomingMap]); const [rows, setRows] = useState(() => entriesToRows(apiBindings)); + const [createRequest, setCreateRequest] = useState<{ rowId: string; name: string } | null>(null); + const pickerAnchorRefs = useRef(new Map()); const lastEmittedKeyRef = useRef(incomingKey); const lastIncomingKeyRef = useRef(incomingKey); @@ -334,9 +341,7 @@ export function AgentSecretAccessEditor({ const trimmedAlias = row.alias.trim(); const aliasInvalid = Boolean(trimmedAlias) && !SECRET_ALIAS_RE.test(trimmedAlias); const aliasDuplicate = Boolean(trimmedAlias) && (aliasCounts.get(trimmedAlias) ?? 0) > 1; - const bindingValue: SecretBindingValue | null = row.secretId - ? { secretId: row.secretId, version: row.version } - : null; + const selectedSecret = secrets.find((secret) => secret.id === row.secretId) ?? null; return (
@@ -360,23 +365,106 @@ export function AgentSecretAccessEditor({ )} />
-
- +
+ { + if (!open && createRequest?.rowId === row.id) setCreateRequest(null); + }} + > + +
{ + if (node) pickerAnchorRefs.current.set(row.id, node); + else pickerAnchorRefs.current.delete(row.id); + }} + className="min-w-0 flex-1" + > + + patchRow(row.id, { + secretId, + version: "latest", + alias: + !row.alias.trim() && secretId + ? envKeyFromSecretName(secretName(secretId)) + : row.alias, + }) + } + onCreateNew={onCreateSecret + ? (query) => { + window.setTimeout(() => { + setCreateRequest({ + rowId: row.id, + name: secretNameFromKey(query) || query.trim(), + }); + }, 0); + } + : undefined} + disabled={disabled} + triggerClassName="h-9 min-h-9" + /> +
+
+ { + // Closing the picker returns focus to its trigger inside + // this popover's anchor. Keep that focus restoration from + // dismissing the create form that just opened. + const target = event.detail.originalEvent.target as Node | null; + if (target && pickerAnchorRefs.current.get(row.id)?.contains(target)) { + event.preventDefault(); + } + }} + > + {createRequest?.rowId === row.id && onCreateSecret ? ( + secret.name)} + onCancel={() => setCreateRequest(null)} + onSubmit={async (name, value) => { + const created = await onCreateSecret(name, value); + patchRow(row.id, { + secretId: created.id, + version: "latest", + alias: row.alias.trim() || envKeyFromSecretName(created.name), + }); + setCreateRequest(null); + }} + /> + ) : null} + +
+
+
+
+ ); + } +} diff --git a/ui/src/components/ApprovalPayload.test.tsx b/ui/src/components/ApprovalPayload.test.tsx index c11405e92da..95339474730 100644 --- a/ui/src/components/ApprovalPayload.test.tsx +++ b/ui/src/components/ApprovalPayload.test.tsx @@ -4,6 +4,7 @@ import { act } from "react"; import { createRoot } from "react-dom/client"; import { afterEach, beforeEach, describe, expect, it } from "vitest"; import { ApprovalPayloadRenderer, approvalLabel } from "./ApprovalPayload"; +import { ThemeProvider } from "../context/ThemeContext"; // eslint-disable-next-line @typescript-eslint/no-explicit-any (globalThis as any).IS_REACT_ACT_ENVIRONMENT = true; @@ -35,17 +36,19 @@ describe("ApprovalPayloadRenderer", () => { act(() => { root.render( - , + + + , ); }); @@ -62,19 +65,140 @@ describe("ApprovalPayloadRenderer", () => { }); }); + it("renders markdown in board approval prose fields", () => { + const root = createRoot(container); + + act(() => { + root.render( + + + , + ); + }); + + const bodies = container.querySelectorAll(".paperclip-markdown"); + expect(bodies.length).toBe(4); + + const summary = bodies[0]; + expect(summary.querySelector("strong")?.textContent).toBe("Bold"); + expect(summary.querySelector("code")?.textContent).toBe("code"); + const link = summary.querySelector("a"); + expect(link?.getAttribute("href")).toBe("https://example.com"); + expect(link?.textContent).toBe("a link"); + + // The raw markdown characters must not survive into the rendered text. + expect(container.textContent).not.toContain("**Bold**"); + expect(container.textContent).not.toContain("[a link](https://example.com)"); + + expect(bodies[1].querySelector("strong")?.textContent).toBe("frog"); + expect(bodies[2].querySelector("code")?.textContent).toBe("frog"); + expect(bodies[3].querySelector("strong")?.textContent).toBe("frog"); + + act(() => { + root.unmount(); + }); + }); + + it("does not nest a second bullet when a risk is authored as a markdown list item", () => { + const root = createRoot(container); + + act(() => { + root.render( + + + , + ); + }); + + const bodies = container.querySelectorAll(".paperclip-markdown"); + expect(bodies.length).toBe(5); + for (const body of bodies) { + expect(body.querySelector("ul")).toBeNull(); + expect(body.querySelector("ol")).toBeNull(); + expect(body.querySelector("li")).toBeNull(); + } + + expect(bodies[0].querySelector("strong")?.textContent).toBe("Leading dash"); + expect(container.textContent).toContain("Leading star risk."); + expect(container.textContent).toContain("Leading dot risk."); + expect(container.textContent).toContain("Leading number risk."); + expect(container.textContent).toContain("Leading paren risk."); + expect(container.textContent).not.toContain("- **Leading dash**"); + + act(() => { + root.unmount(); + }); + }); + + it("renders every risk when two entries collapse to the same text after marker stripping", () => { + const root = createRoot(container); + const errors: unknown[] = []; + const originalError = console.error; + console.error = (...args: unknown[]) => { + errors.push(args); + }; + + try { + act(() => { + root.render( + + + , + ); + }); + + expect(container.querySelectorAll(".paperclip-markdown").length).toBe(2); + expect(errors).toEqual([]); + } finally { + console.error = originalError; + act(() => { + root.unmount(); + }); + } + }); + it("can hide the repeated title when the card header already shows it", () => { const root = createRoot(container); act(() => { root.render( - , + + + , ); }); diff --git a/ui/src/components/ApprovalPayload.tsx b/ui/src/components/ApprovalPayload.tsx index 165ee8a2235..6f8f9752d84 100644 --- a/ui/src/components/ApprovalPayload.tsx +++ b/ui/src/components/ApprovalPayload.tsx @@ -1,4 +1,5 @@ import { UserPlus, Lightbulb, ShieldAlert, ShieldCheck } from "lucide-react"; +import { MarkdownBody } from "./MarkdownBody"; import { formatCents } from "../lib/utils"; export const typeLabel: Record = { @@ -161,11 +162,19 @@ export function BoardApprovalPayload({ ); } +/** + * Risks render inside a custom bullet row, so a leading markdown list marker + * would nest a second bullet inside the first. Strip one leading marker. + */ +function stripLeadingListMarker(value: string): string { + return value.replace(/^(?:[-*•]|\d+[.)])\s+/, ""); +} + function BoardApprovalPayloadContent({ payload }: { payload: Record }) { const risks = Array.isArray(payload.risks) ? payload.risks .filter((value): value is string => typeof value === "string") - .map((value) => value.trim()) + .map((value) => stripLeadingListMarker(value.trim())) .filter(Boolean) : []; const title = firstNonEmptyString(payload.title); @@ -185,7 +194,7 @@ function BoardApprovalPayloadContent({ payload }: { payload: Record

Summary

-

{summary}

+ {summary}
)} {recommendedAction && ( @@ -193,23 +202,23 @@ function BoardApprovalPayloadContent({ payload }: { payload: Record Recommended action

-

{recommendedAction}

+ {recommendedAction} )} {nextActionOnApproval && (

On approval

-

{nextActionOnApproval}

+ {nextActionOnApproval}
)} {risks.length > 0 && (

Risks

    - {risks.map((risk) => ( -
  • - - {risk} + {risks.map((risk, index) => ( +
  • + + {risk}
  • ))}
diff --git a/ui/src/components/AppsSidebar.test.tsx b/ui/src/components/AppsSidebar.test.tsx index 4e15eebf978..ff0c9e403b4 100644 --- a/ui/src/components/AppsSidebar.test.tsx +++ b/ui/src/components/AppsSidebar.test.tsx @@ -126,16 +126,18 @@ describe("AppsSidebar", () => { // "Run your own" / "Paste a config" moved to the Connect-an-app page (PAP-10922); // assert their absence at the item level below. - // Three peer consumer doors: Browse (store) · Connections · Review (PAP-13254). + // Consumer doors stay above the Developer boundary. expect(sidebarNavItemMock).toHaveBeenCalledWith( - expect.objectContaining({ to: "/apps/browse", label: "Browse" }), + expect.objectContaining({ to: "/apps", label: "Browse", end: true }), ); expect(sidebarNavItemMock).toHaveBeenCalledWith( - expect.objectContaining({ to: "/apps", label: "Connections", end: true }), + expect.objectContaining({ to: "/apps/connections", label: "Connections", end: true }), ); expect(sidebarNavItemMock).toHaveBeenCalledWith( expect.objectContaining({ to: "/apps/review", label: "Review" }), ); + const sidebarText = container.textContent ?? ""; + expect(sidebarText.indexOf("Connections")).toBeGreaterThan(sidebarText.indexOf("Developer")); // "Needs attention" is no longer a top-level door — it folds into Connections. expect(sidebarNavItemMock).not.toHaveBeenCalledWith( expect.objectContaining({ label: "Needs attention" }), diff --git a/ui/src/components/AppsSidebar.tsx b/ui/src/components/AppsSidebar.tsx index 09d26c78f56..f339f8e9bc6 100644 --- a/ui/src/components/AppsSidebar.tsx +++ b/ui/src/components/AppsSidebar.tsx @@ -14,12 +14,11 @@ import { SidebarNavItem } from "./SidebarNavItem"; * Secondary sidebar for the prosumer Apps area (PAP-10856; three-door IA * PAP-13254 / U3). * - * ← Back · APPS: Browse / Connections / Review (n) - * DEVELOPER: Gateways / Profiles / Rules / Health / Activity + * ← Back · APPS: Browse / Review (n) + * DEVELOPER: Connections / Gateways / Profiles / Rules / Health / Activity * - * The three consumer doors are peers: "Browse" (the store — discover + add), - * "Connections" (your connected tools + health), and "Review" (PAP-12371, - * Finding B — decisions waiting on your OK, with a live pending count). + * "Browse" is the store and "Review" holds decisions waiting on the user's + * OK. Connection management lives with the Developer tools. * "Needs attention" is no longer a door: health/error triage folds into * Connections as a status filter + banner, so approvals are never buried * behind an error label. The Developer section was folded in from the retired @@ -71,8 +70,7 @@ export function AppsSidebar() { Apps
- - +
+ {developerTabs.map((tab) => ( { + it("immediately replaces a pending secret proposal with its stitched failure receipt", () => { + const resolved = { + ...failedSecretProposalInteraction, + id: pendingSecretProposalInteraction.id, + }; + + const next = replaceResolvedInteraction([pendingSecretProposalInteraction], resolved); + + expect(next).toHaveLength(1); + expect(next[0]?.status).toBe("accepted"); + expect(next[0]?.kind).toBe("request_confirmation"); + if (next[0]?.kind === "request_confirmation") { + expect(next[0].result?.secretProposal).toMatchObject({ + status: "failed", + errorCode: "binding_snapshot_stale", + }); + } + }); + + it("retains the stitched receipt when the interaction cache was empty", () => { + expect(replaceResolvedInteraction(undefined, failedSecretProposalInteraction)).toEqual([ + failedSecretProposalInteraction, + ]); + }); +}); diff --git a/ui/src/components/AttentionInteractionResolver.tsx b/ui/src/components/AttentionInteractionResolver.tsx index 8176f534d8b..05acfd5b912 100644 --- a/ui/src/components/AttentionInteractionResolver.tsx +++ b/ui/src/components/AttentionInteractionResolver.tsx @@ -28,6 +28,18 @@ interface AttentionInteractionResolverProps { onResolved?: () => void; } +export function replaceResolvedInteraction( + current: IssueThreadInteraction[] | undefined, + resolvedInteraction: IssueThreadInteraction, +): IssueThreadInteraction[] { + const existing = current ?? []; + const index = existing.findIndex((entry) => entry.id === resolvedInteraction.id); + if (index < 0) return [...existing, resolvedInteraction]; + return existing.map((entry, entryIndex) => + entryIndex === index ? resolvedInteraction : entry, + ); +} + /** * Lazily fetches the full issue-thread interaction referenced by an attention * row and renders the existing {@link IssueThreadInteractionCard} inline, so @@ -56,7 +68,17 @@ export function AttentionInteractionResolver({ return match && isIssueThreadInteraction(match) ? match : null; }, [interactions, interactionId]); - const invalidate = () => { + const invalidate = (resolvedInteraction?: IssueThreadInteraction) => { + // The accept route returns the fully stitched interaction (including + // secretProposal.executed/failed). Put that receipt in the cache before + // invalidating so the attention surface never flashes a generic Accepted + // state while the refetch catches up. + if (resolvedInteraction) { + queryClient.setQueryData( + queryKeys.issues.interactions(issueId), + (current) => replaceResolvedInteraction(current, resolvedInteraction), + ); + } queryClient.invalidateQueries({ queryKey: queryKeys.issues.interactions(issueId) }); queryClient.invalidateQueries({ queryKey: queryKeys.attention(companyId) }); onResolved?.(); diff --git a/ui/src/components/AttentionQueueRow.test.tsx b/ui/src/components/AttentionQueueRow.test.tsx index 56b7bae0eaa..ee0ae3fb187 100644 --- a/ui/src/components/AttentionQueueRow.test.tsx +++ b/ui/src/components/AttentionQueueRow.test.tsx @@ -7,6 +7,7 @@ import { QueryClient, QueryClientProvider } from "@tanstack/react-query"; import { afterEach, describe, expect, it, vi } from "vitest"; import type { AttentionItem, AttentionSourceKind } from "@paperclipai/shared"; import { approvalsApi } from "../api/approvals"; +import { ApiError } from "../api/client"; import { issuesApi } from "../api/issues"; import { ToastViewport } from "./ToastViewport"; import { ToastProvider } from "../context/ToastContext"; @@ -180,6 +181,90 @@ describe("AttentionQueueRow", () => { expect(el.textContent).toContain("Send back to work"); }); + // PAP-16506 P4: an opt-in policy warns before the card offers Approve, because + // the server refuses the verdict. The default earns no pixels. + const reviewItemWithPolicy = (reviewPolicy?: unknown) => + buildItem({ + sourceKind: "review" as AttentionSourceKind, + inlineResolvable: true, + subject: { + kind: "issue", + id: "issue-1", + companyId: "c1", + title: "PR ready for review", + identifier: null, + status: "in_review", + href: "/PAP/issues/PAP-1", + metadata: { + reviewAttentionState: "stalled", + ...(reviewPolicy === undefined ? {} : { reviewPolicy }), + }, + }, + }); + + it("says nothing about who can approve on a review with no policy set", () => { + const el = render( + , + ); + expect(el.querySelector('[data-testid="review-policy-badge"]')).toBeNull(); + expect(el.textContent).not.toContain("Anyone else"); + expect(el.textContent).not.toContain("Human only"); + // The verbs still render — suppressing the badge must not suppress the card. + expect(el.textContent).toContain("Send back to work"); + }); + + it("badges the opt-in constraint when the issue carries one", () => { + const el = render( + , + ); + const badge = el.querySelector('[data-testid="review-policy-badge"]'); + expect(badge?.getAttribute("data-review-policy")).toBe("human_only"); + expect(badge?.textContent).toContain("Human only"); + expect(badge?.getAttribute("title")).toContain("Agents cannot"); + }); + + it("badges a not_creator policy as 'Anyone else'", () => { + const el = render( + , + ); + const badge = el.querySelector('[data-testid="review-policy-badge"]'); + expect(badge?.getAttribute("data-review-policy")).toBe("not_creator"); + expect(badge?.textContent).toContain("Anyone else"); + }); + + it("shows no badge for an explicit default or an unrecognised policy", () => { + for (const policy of ["anyone", "board_only"]) { + const el = render( + , + ); + expect(el.querySelector('[data-testid="review-policy-badge"]')).toBeNull(); + } + }); + it("deep-links a covered review instead of inlining", () => { const el = render( { expect(gallery?.querySelectorAll("a")).toHaveLength(0); }); - // Decision training (PAP-14299): a trainable row shows the train affordance; - // the trained/untrained state renders purely from `trainingExampleId`. - function trainableItem(overrides: Partial = {}): AttentionItem { + // PAP-17287: the collapsed row must say who may resolve an interaction before + // its compact verbs are used, and a denial must keep the server's reason. + function interactionItem( + audience: AttentionItem["resolverAudience"], + verbs: AttentionItem["decisionVerbs"] = [ + { id: "accept", label: "Accept", description: null }, + { id: "reject", label: "Reject", description: null }, + ], + ) { return buildItem({ sourceKind: "issue_thread_interaction", subject: { kind: "interaction", id: "interaction-1", companyId: "c1", - title: "Approve the migration plan?", + title: "Close this confirmation?", identifier: null, status: "pending", - href: "/PAP/issues/PAP-1", - metadata: { issueId: "issue-1", kind: "request_confirmation" }, + href: "/PAP/issues/issue-1#interaction-interaction-1", + metadata: { kind: "request_confirmation", issueId: "issue-1" }, }, - ...overrides, + decisionVerbs: verbs, + resolverAudience: audience, }); } - // Training lives in the row's overflow menu AND on a visible inline "Train" - // pill for untrained rows. Menu items live in a portal that - // only mounts once opened — environment-flaky in jsdom (see the dismiss test - // above) — so the untrained path asserts the menu exists, no trained badge is - // shown, and the onTrain contract is exercised through the visible pill. - it("shows a visible Train pill (no trained badge) and fires onTrain when clicked", () => { - const onTrain = vi.fn(); + const openAudience: AttentionItem["resolverAudience"] = { + requestedResolverPolicy: "anyone", + effectiveResolverPolicy: "anyone", + effectiveResolverPolicySource: "requested", + resolverPolicyProvenance: "inherited", + addresseeAgentId: null, + addresseeName: null, + createdByAgentId: "agent-watchdog", + createdByAgentName: "Watchdog", + }; + + it("states the server-derived audience before the compact decision verbs", () => { render( , ); - expect(container?.querySelector('[aria-label="Row actions"]')).toBeTruthy(); + + const audience = container?.querySelector('[data-testid="interaction-audience"]'); + expect(audience?.getAttribute("data-audience-policy")).toBe("anyone"); + expect(audience?.getAttribute("data-audience-open")).toBe("true"); + expect(audience?.textContent).toContain("Anyone"); + expect(audience?.textContent).toContain("Anyone can respond"); + + // Reading order is the point: the audience qualifies the buttons, so it has + // to precede them in the DOM as well as on screen. + const actions = container?.querySelector('[aria-label="Decision actions"]'); + expect(audience && actions && audience.compareDocumentPosition(actions)) + .toBe(Node.DOCUMENT_POSITION_FOLLOWING); + }); + + it("names the addressed responder on a restricted collapsed row", () => { + render( + , + ); + + const audience = container?.querySelector('[data-testid="interaction-audience"]'); + expect(audience?.getAttribute("data-audience-open")).toBe("false"); + expect(audience?.textContent).toContain("Only CodexCoder or the board can respond"); + // A collapsed row spends its line on the responder, not on a badge that + // repeats the clause beside it. + expect(audience?.textContent).not.toContain("Addressed —"); + }); + + // PAP-17289: the row's shell is `overflow-hidden`, so a clause that cannot + // wrap is cut mid-word with no ellipsis and names a responder that does not + // exist. jsdom does no layout, so this asserts the wrapping rule itself. + it("lets a long addressee name wrap instead of being cut mid-word", () => { + const addresseeName = "ReleaseEngineeringPlatformCoordinationServiceBot"; + render( + , + ); + + const audience = container?.querySelector('[data-testid="interaction-audience"]'); + const summary = audience?.querySelector('[data-testid="interaction-audience-summary"]'); + expect(summary?.textContent).toBe(`Only ${addresseeName} or the board can respond`); + expect(summary?.parentElement?.className).toContain("break-words"); + }); + + it("renders no audience line when the feed carries no resolver metadata", () => { + render( + , + ); + + // Never guess a policy client-side: silence beats a wrong audience. + expect(container?.querySelector('[data-testid="interaction-audience"]')).toBeNull(); + }); + + it("keeps the server denial reason and names the responder when a compact accept is refused", async () => { + vi.mocked(issuesApi.acceptInteraction).mockRejectedValue( + new ApiError("This issue-thread interaction is human-only", 403, { + error: "This issue-thread interaction is human-only", + code: "interaction_human_only", + }), + ); + render( + , + ); + + const accept = Array.from(container?.querySelectorAll("button") ?? []).find( + (button) => button.textContent === "Accept", + ); + act(() => accept?.dispatchEvent(new MouseEvent("click", { bubbles: true }))); + await act(async () => { + await new Promise((resolve) => setTimeout(resolve, 0)); + }); + + const feedback = document.body.textContent ?? ""; + expect(feedback).toContain("This issue-thread interaction is human-only."); + expect(feedback).toContain("Only the board can respond."); + expect(feedback).not.toMatch(/try again/i); + }); + + it("does not surface training state or actions for decisions", () => { + render( + , + ); + expect(container?.textContent).not.toContain("Train"); expect(container?.querySelector('[data-testid="attention-trained-badge"]')).toBeNull(); - const trainPill = container?.querySelector('[data-testid="attention-train-inline"]'); - expect(trainPill?.textContent).toContain("Train"); - act(() => trainPill?.dispatchEvent(new MouseEvent("click", { bubbles: true }))); - expect(onTrain).toHaveBeenCalledWith(expect.objectContaining({ id: "a1" })); - }); - - it("hides the visible Train pill once trained (the badge stands in for it)", () => { - render( - , - ); expect(container?.querySelector('[data-testid="attention-train-inline"]')).toBeNull(); - expect(container?.querySelector('[data-testid="attention-trained-badge"]')).toBeTruthy(); - }); - - it("renders a Trained ✓ badge once trained and fires onTrain when it is clicked", () => { - const onTrain = vi.fn(); - render( - , - ); - const badge = container?.querySelector('[data-testid="attention-trained-badge"]'); - expect(badge?.textContent).toContain("Trained"); - act(() => badge?.dispatchEvent(new MouseEvent("click", { bubbles: true }))); - expect(onTrain).toHaveBeenCalledWith(expect.objectContaining({ id: "a1" })); - }); - - it("does not offer training on a decision that isn't anchored to an issue", () => { - render( - , - ); expect(container?.querySelector('[data-testid="attention-train-button"]')).toBeNull(); }); }); diff --git a/ui/src/components/AttentionQueueRow.tsx b/ui/src/components/AttentionQueueRow.tsx index 91037c27bef..9387d5b585c 100644 --- a/ui/src/components/AttentionQueueRow.tsx +++ b/ui/src/components/AttentionQueueRow.tsx @@ -6,7 +6,6 @@ import { ChevronDown, ChevronUp, ExternalLink, - GraduationCap, Loader2, MoreHorizontal, RotateCcw, @@ -19,6 +18,8 @@ import { approvalsApi } from "../api/approvals"; import { issuesApi } from "../api/issues"; import { useToastActions } from "../context/ToastContext"; import { queryKeys } from "../lib/queryKeys"; +import { describeAttentionResolverAudience, type InteractionAudienceDescription } from "../lib/interaction-audience"; +import { interactionResolutionErrorMessage } from "../lib/interaction-resolution-error"; import { attentionDetailImages, attentionDetailLine, @@ -29,9 +30,9 @@ import { isInlineResolvable, sourceMeta, } from "../lib/attention"; -import { isTrainable } from "../lib/decisionTraining"; import { cn, relativeTime } from "../lib/utils"; import { DecisionTriageStrip } from "./DecisionTriageStrip"; +import { InteractionAudienceLine } from "./InteractionAudienceLine"; import { StatusGlyph } from "./StatusGlyph"; import { Button } from "./ui/button"; import { Collapsible, CollapsibleContent } from "./ui/collapsible"; @@ -49,6 +50,7 @@ import { import { AttentionInteractionResolver } from "./AttentionInteractionResolver"; import { DecisionResolver } from "./DecisionResolver"; import { StalledReviewActions } from "./StalledReviewActions"; +import { readIssueReviewPolicyMetadata } from "../lib/review-policy"; const HOUR_MS = 60 * 60 * 1000; const DAY_MS = 24 * HOUR_MS; @@ -83,8 +85,6 @@ interface AttentionQueueRowProps { onToggleExpand: (item: AttentionItem) => void; onDismiss: (item: AttentionItem) => void; onSnooze?: (item: AttentionItem, snoozedUntil: string) => void; - /** Open the decision-training drawer for this row (create or view). */ - onTrain?: (item: AttentionItem) => void; /** Restore a snoozed/dismissed row (curtain variant only). */ onRestore?: (item: AttentionItem) => void; /** "active" renders the live queue row; "hidden" renders a curtain row. */ @@ -113,7 +113,6 @@ export const AttentionQueueRow = memo(function AttentionQueueRow({ onToggleExpand, onDismiss, onSnooze, - onTrain, onRestore, variant = "active", agentMap, @@ -146,12 +145,6 @@ export const AttentionQueueRow = memo(function AttentionQueueRow({ // with no triage keep the explicit Open button and never toggle on a stray click. const triageEnabled = showTriage && !isHidden; const expandable = inline || (!isHidden && hasImages) || triageEnabled; - // Any issue-anchored approval or interaction is - // trainable at any time (pending or resolved). Trained/untrained renders - // purely from the feed's `trainingExampleId` — no per-row fetch. - const trainable = !isHidden && !!onTrain && isTrainable(item); - const trained = item.trainingExampleId != null; - const activate = () => { if (expandable) onToggleExpand(item); }; @@ -166,6 +159,11 @@ export const AttentionQueueRow = memo(function AttentionQueueRow({ // Which rows contribute an action bar. Inline rows carry compact decision // verbs; deep-link rows carry an Open button; curtain rows carry Restore. const compactActions = !isHidden ? collectCompactActions(item) : []; + // Who the server will let resolve this interaction. A collapsed row offers + // Accept/Reject before anything fetches the interaction, so the audience + // travels with the feed item; null for every non-interaction source and for a + // feed built before the metadata existed (PAP-17287). + const audience = describeAttentionResolverAudience(item); const showOpen = !inline && !!href; const showRestore = isHidden && !!onRestore; // An expanded inline row hands its footer to the resolver, which owns the @@ -206,7 +204,12 @@ export const AttentionQueueRow = memo(function AttentionQueueRow({
{showCompact && ( - onToggleExpand(item)} /> + onToggleExpand(item)} + /> )} {showOpen && ( @@ -284,38 +287,6 @@ export const AttentionQueueRow = memo(function AttentionQueueRow({ )} - {trainable && trained && ( - - )} - {/* Visible train affordance for untrained rows. Trained - rows already carry the "Trained ✓" badge above; both surfaces also - keep the overflow "Train this decision" entry. Sits in the same slot - as the badge so a row's training state reads from one place. */} - {trainable && !trained && ( - - )}
@@ -342,19 +313,6 @@ export const AttentionQueueRow = memo(function AttentionQueueRow({ - {/* Training moved off the header strip (which now carries only - recency + overflow) but keeps its testids so the affordance - is still addressable. */} - {trainable && ( - onTrain?.(item)} - > - - {trained ? "View training example" : "Train this decision"} - - )} {onSnooze && onSnooze(item, iso)} />} onDismiss(item)}> @@ -408,6 +366,10 @@ export const AttentionQueueRow = memo(function AttentionQueueRow({
{hasImages && } + {/* The audience reads *before* the verbs it qualifies: a compact + Accept sitting alone asks for a decision without saying whose + it is (PAP-17287). */} + {audience && } {inline && renderFooter({ compact: true })}
@@ -513,10 +475,13 @@ function collectCompactActions(item: AttentionItem): CompactAction[] { function CompactDecisionActions({ item, companyId, + audience, onOpen, }: { item: AttentionItem; companyId: string; + /** Effective resolver audience, so a denial can name who *can* respond. */ + audience: InteractionAudienceDescription | null; onOpen: () => void; }) { const queryClient = useQueryClient(); @@ -556,9 +521,11 @@ function CompactDecisionActions({ }); }, onError: (error, action) => { + // A policy denial is permanent, so it keeps the server's reason and names + // the real responder instead of asking for a retry that will fail again. pushToast({ title: `Could not ${decisionLabel(action)}`, - body: error instanceof Error ? error.message : "Please try again.", + body: interactionResolutionErrorMessage(error, audience), tone: "error", }); }, @@ -843,6 +810,7 @@ function InlineResolver({ issueId={item.subject.id} companyId={companyId} footerSlot={toggle} + reviewPolicy={readIssueReviewPolicyMetadata(item.subject.metadata)} /> ); } diff --git a/ui/src/components/BudgetPolicyCard.tsx b/ui/src/components/BudgetPolicyCard.tsx index d392ff0c0e2..2b0624ac190 100644 --- a/ui/src/components/BudgetPolicyCard.tsx +++ b/ui/src/components/BudgetPolicyCard.tsx @@ -101,6 +101,11 @@ export function BudgetPolicyCard({
vi.fn()); const resumeMock = vi.hoisted(() => vi.fn()); +const getExperimentalMock = vi.hoisted(() => vi.fn()); vi.mock("@/api/builtInAgents", async (importOriginal) => { const actual = await importOriginal(); @@ -23,6 +24,10 @@ vi.mock("@/api/agents", () => ({ agentsApi: { resume: resumeMock }, })); +vi.mock("@/api/instanceSettings", () => ({ + instanceSettingsApi: { getExperimental: getExperimentalMock }, +})); + // The configure modal pulls in the full AgentConfigForm; stub it so the gate // test stays focused on state selection. vi.mock("@/components/ConfigureBuiltInAgentModal", () => ({ @@ -90,6 +95,8 @@ describe("BuiltInAgentGate (PAP-12978)", () => { document.body.appendChild(container); listMock.mockReset(); resumeMock.mockReset(); + getExperimentalMock.mockReset(); + getExperimentalMock.mockResolvedValue({ enableBuiltInAgents: true }); }); afterEach(() => { @@ -153,6 +160,16 @@ describe("BuiltInAgentGate (PAP-12978)", () => { expect(container.textContent).not.toContain("Set up the Briefs Agent"); }); + it("does not query built-in agents when the feature flag is off", async () => { + getExperimentalMock.mockResolvedValue({ enableBuiltInAgents: false }); + listMock.mockResolvedValue([makeState("ready")]); + + await renderGate(); + + expect(listMock).not.toHaveBeenCalled(); + expect(container.querySelector('[data-testid="feature"]')).not.toBeNull(); + }); + it("fails open to the feature when the key is unknown", async () => { listMock.mockResolvedValue([makeState("ready", { definition: { ...makeState("ready").definition, key: "learning" }, diff --git a/ui/src/components/BuiltInAgentGate.tsx b/ui/src/components/BuiltInAgentGate.tsx index 4d7cb8e4a0a..93a75a090f8 100644 --- a/ui/src/components/BuiltInAgentGate.tsx +++ b/ui/src/components/BuiltInAgentGate.tsx @@ -10,6 +10,7 @@ import { Button } from "@/components/ui/button"; import { ConfigureBuiltInAgentModal } from "@/components/ConfigureBuiltInAgentModal"; import { builtInAgentsApi, type BuiltInAgentState } from "@/api/builtInAgents"; import { agentsApi } from "@/api/agents"; +import { instanceSettingsApi } from "@/api/instanceSettings"; import { queryKeys } from "@/lib/queryKeys"; import { agentUrl } from "@/lib/utils"; import { relativeTime } from "@/lib/utils"; @@ -36,10 +37,15 @@ export function BuiltInAgentGate({ agentKey, companyId, featureLabel, children } const queryClient = useQueryClient(); const [configureOpen, setConfigureOpen] = useState(false); - const { data: states, isLoading } = useQuery({ + const experimentalQuery = useQuery({ + queryKey: queryKeys.instance.experimentalSettings, + queryFn: () => instanceSettingsApi.getExperimental(), + }); + const builtInAgentsEnabled = experimentalQuery.data?.enableBuiltInAgents === true; + const { data: states, isLoading: statesLoading } = useQuery({ queryKey: queryKeys.builtInAgents.list(companyId ?? "__none__"), queryFn: () => builtInAgentsApi.list(companyId!), - enabled: Boolean(companyId), + enabled: Boolean(companyId && builtInAgentsEnabled), }); const state: BuiltInAgentState | undefined = states?.find((entry) => entry.definition.key === agentKey); @@ -54,7 +60,7 @@ export function BuiltInAgentGate({ agentKey, companyId, featureLabel, children } // Unknown key or still resolving the company — fail open to the feature. if (!companyId) return <>{children}; - if (isLoading && !states) return ; + if ((experimentalQuery.isLoading || statesLoading) && !states) return ; if (!state) return <>{children}; const label = featureLabel ?? state.definition.displayName; diff --git a/ui/src/components/CompanySwitcher.tsx b/ui/src/components/CompanySwitcher.tsx index 4c38e66d5ac..46067c0149a 100644 --- a/ui/src/components/CompanySwitcher.tsx +++ b/ui/src/components/CompanySwitcher.tsx @@ -1,4 +1,4 @@ -import { ChevronsUpDown, Plus, Settings } from "lucide-react"; +import { ChevronsUpDown, Plus, RefreshCw, Settings } from "lucide-react"; import { Link } from "@/lib/router"; import { useCompany } from "../context/CompanyContext"; import { @@ -32,7 +32,8 @@ interface CompanySwitcherProps { export function CompanySwitcher({ open: controlledOpen, onOpenChange }: CompanySwitcherProps = {}) { const [internalOpen, setInternalOpen] = useState(false); - const { companies, selectedCompany, setSelectedCompanyId } = useCompany(); + const { companies, selectedCompany, setSelectedCompanyId, companyListUnavailable, retryCompanies } = + useCompany(); const sidebarCompanies = companies.filter((company) => company.status !== "archived"); const open = controlledOpen ?? internalOpen; const setOpen = onOpenChange ?? setInternalOpen; @@ -69,7 +70,26 @@ export function CompanySwitcher({ open: controlledOpen, onOpenChange }: CompanyS ))} {sidebarCompanies.length === 0 && ( - No companies + // "No companies" is a claim about the account, and after a failed list + // request it is one we cannot make — say what actually happened and + // give the customer the way out, since nothing else in the app does. + companyListUnavailable ? ( + <> + Couldn't load companies + { + // Keep the menu open so the result of the retry is visible. + event.preventDefault(); + void retryCompanies?.(); + }} + > + + Try again + + + ) : ( + No companies + ) )} diff --git a/ui/src/components/DecisionShelf.tsx b/ui/src/components/DecisionShelf.tsx index a5a05807dd3..addfb35e812 100644 --- a/ui/src/components/DecisionShelf.tsx +++ b/ui/src/components/DecisionShelf.tsx @@ -58,7 +58,6 @@ export function AgingItemRow({ onToggleExpand, onDismiss, onSnooze, - onTrain, }: { item: AttentionItem; companyId: string; @@ -70,7 +69,6 @@ export function AgingItemRow({ onToggleExpand: (item: AttentionItem) => void; onDismiss: (item: AttentionItem) => void; onSnooze: (item: AttentionItem, snoozedUntil: string) => void; - onTrain: (item: AttentionItem) => void; }) { const queryClient = useQueryClient(); const { pushToast } = useToastActions(); @@ -115,7 +113,6 @@ export function AgingItemRow({ onToggleExpand={onToggleExpand} onDismiss={onDismiss} onSnooze={onSnooze} - onTrain={onTrain} agentMap={agentMap} agents={agents} showTriage diff --git a/ui/src/components/DecisionTrainingDrawer.test.tsx b/ui/src/components/DecisionTrainingDrawer.test.tsx deleted file mode 100644 index b34cf79502b..00000000000 --- a/ui/src/components/DecisionTrainingDrawer.test.tsx +++ /dev/null @@ -1,300 +0,0 @@ -// @vitest-environment jsdom - -import { flushSync } from "react-dom"; -import { createRoot } from "react-dom/client"; -import { QueryClient, QueryClientProvider } from "@tanstack/react-query"; -import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; -import type { - AttentionItem, - DecisionTrainingExample, - DecisionTrainingPreview, -} from "@paperclipai/shared"; - -const mockApi = vi.hoisted(() => ({ - preview: vi.fn(), - create: vi.fn(), - get: vi.fn(), - updateNotes: vi.fn(), - delete: vi.fn(), -})); - -vi.mock("../api/decisionTraining", async (importOriginal) => { - const original = await importOriginal(); - return { ...original, decisionTrainingApi: mockApi }; -}); - -vi.mock("@/lib/router", () => ({ - Link: ({ children, to, ...props }: React.ComponentProps<"a"> & { to: string }) => ( - {children} - ), -})); - -(globalThis as unknown as { IS_REACT_ACT_ENVIRONMENT: boolean }).IS_REACT_ACT_ENVIRONMENT = true; - -import { DecisionTrainingDrawer } from "./DecisionTrainingDrawer"; -import { ToastProvider } from "../context/ToastContext"; - -function act(callback: () => void | Promise) { - let result: void | Promise | undefined; - flushSync(() => { - result = callback(); - }); - return result; -} - -async function waitFor(predicate: () => boolean, attempts = 40): Promise { - for (let i = 0; i < attempts; i += 1) { - if (predicate()) return; - await act(async () => { - await new Promise((resolve) => setTimeout(resolve, 5)); - }); - } - throw new Error("waitFor predicate did not become true"); -} - -function setTextareaValue(el: HTMLTextAreaElement, value: string) { - const setter = Object.getOwnPropertyDescriptor(window.HTMLTextAreaElement.prototype, "value")!.set!; - setter.call(el, value); - el.dispatchEvent(new Event("input", { bubbles: true })); -} - -/** Text query across the whole document (Radix portals content into body). */ -function bodyText(): string { - return document.body.textContent ?? ""; -} - -function findButton(label: string): HTMLButtonElement | null { - return [...document.body.querySelectorAll("button")].find( - (b) => (b.textContent ?? "").trim().includes(label), - ) as HTMLButtonElement | null; -} - -/** Buttons or anchors (asChild renders the trigger as its child element). */ -function findClickable(label: string): HTMLElement | null { - return [...document.body.querySelectorAll("button, a")].find( - (el) => (el.textContent ?? "").trim().includes(label), - ) as HTMLElement | null; -} - -function buildItem(overrides: Partial = {}): AttentionItem { - return { - id: "row-1", - companyId: "c1", - sourceKind: "issue_thread_interaction", - subject: { - kind: "interaction", - id: "interaction-1", - companyId: "c1", - title: "Approve the migration plan?", - identifier: null, - status: "pending", - href: "/tasks/task-1", - metadata: { issueId: "issue-1", kind: "request_confirmation" }, - }, - whyNow: "", - decisionVerbs: [], - inlineResolvable: true, - entryRule: "", - exitRule: "", - dedupKey: "interaction:interaction-1", - dismissalKey: "attention:interaction:interaction-1", - severity: "medium", - rank: 0, - activityAt: "2026-07-09T12:00:00Z", - createdAt: "2026-07-09T12:00:00Z", - updatedAt: "2026-07-09T12:00:00Z", - relatedIssue: null, - project: null, - workspace: null, - detail: null, - dismissal: null, - ...overrides, - expiresAt: overrides.expiresAt ?? null, - ruleKey: overrides.ruleKey ?? null, - originAgentName: overrides.originAgentName ?? null, - queues: overrides.queues ?? [], - shelf: overrides.shelf ?? false, - retentionDays: overrides.retentionDays ?? 30, - keep: overrides.keep ?? false, - archivedAt: overrides.archivedAt ?? null, - retentionVersion: overrides.retentionVersion ?? 1, - decideBy: overrides.decideBy ?? null, - decideByAttribution: overrides.decideByAttribution ?? null, - snoozedUntil: overrides.snoozedUntil ?? null, - trainingExampleId: overrides.trainingExampleId ?? null, - }; -} - -function buildSnapshot(): DecisionTrainingExample["snapshot"] { - return { - version: 1, - capturedAt: "2026-07-10T00:00:00Z", - cutoff: { at: "2026-07-10T00:00:00Z", lastCommentId: "comment-abcdef12", commentCount: 3 }, - issue: {}, - comments: [{}, {}, {}], - runs: [{}, {}], - decision: { kind: "interaction", payload: {}, actor: null, outcome: "accepted" }, - code: { repoUrl: "r", ref: "main", commitSha: "0123456789abcdef", resolution: "exact" }, - }; -} - -function buildPreview(): DecisionTrainingPreview { - return { cutoffAt: "2026-07-10T00:00:00Z", decisionOutcome: "accepted", snapshot: buildSnapshot() }; -} - -function buildExample(overrides: Partial = {}): DecisionTrainingExample { - return { - id: "example-1", - companyId: "c1", - sourceKind: "interaction", - sourceId: "interaction-1", - issueId: "issue-1", - cutoffAt: "2026-07-10T00:00:00Z", - notes: "I accepted because the plan covered rollback.", - notesHistory: [], - decisionOutcome: "accepted", - snapshot: buildSnapshot(), - createdByUserId: "user-1", - createdAt: "2026-07-10T00:00:00Z", - updatedAt: "2026-07-10T00:00:00Z", - ...overrides, - retentionPolicy: overrides.retentionPolicy ?? "scrub_deleted_comments_v1", - }; -} - -let container: HTMLDivElement; -let root: ReturnType; - -function render(node: React.ReactNode) { - const queryClient = new QueryClient({ - defaultOptions: { queries: { retry: false, staleTime: 0, gcTime: 0 } }, - }); - root = createRoot(container); - act(() => { - root.render( - - {node} - , - ); - }); -} - -beforeEach(() => { - container = document.createElement("div"); - document.body.appendChild(container); - Object.values(mockApi).forEach((fn) => fn.mockReset()); -}); - -afterEach(() => { - act(() => root?.unmount()); - container.remove(); - document.body.innerHTML = ""; -}); - -describe("DecisionTrainingDrawer — create state", () => { - it("previews the frozen snapshot and saves an example", async () => { - mockApi.preview.mockResolvedValue(buildPreview()); - mockApi.create.mockResolvedValue(buildExample()); - - render( - {}} companyId="c1" item={buildItem()} />, - ); - - await waitFor(() => bodyText().includes("before cutoff")); - - // Snapshot preview surfaces cutoff, comment/run counts and the commit. - expect(bodyText()).toContain("3 · last comment-"); - expect(bodyText()).toContain("2 before cutoff"); - expect(bodyText()).toContain("0123456789"); - expect(bodyText()).toContain("Resolved · accepted"); - - const textarea = document.body.querySelector("textarea") as HTMLTextAreaElement; - expect(textarea).toBeTruthy(); - act(() => setTextareaValue(textarea, "Trusting the rollback plan.")); - - await act(() => findButton("Save example")?.click()); - await waitFor(() => mockApi.create.mock.calls.length > 0); - - expect(mockApi.create).toHaveBeenCalledWith("c1", { - sourceKind: "interaction", - sourceId: "interaction-1", - issueId: "issue-1", - notes: "Trusting the rollback plan.", - }); - }); - - it("refuses to train a decision with no issue anchor", () => { - const item = buildItem({ subject: { ...buildItem().subject, metadata: {} }, relatedIssue: null }); - render( {}} companyId="c1" item={item} />); - expect(bodyText()).toContain("isn't anchored to an issue"); - expect(mockApi.preview).not.toHaveBeenCalled(); - }); -}); - -describe("DecisionTrainingDrawer — saved state", () => { - it("shows provenance and a read-only snapshot, and round-trips notes edits", async () => { - mockApi.get.mockResolvedValue(buildExample()); - mockApi.updateNotes.mockResolvedValue(buildExample({ notes: "Revised reasoning.", updatedAt: "2026-07-11T00:00:00Z" })); - - render( - {}} - companyId="c1" - item={buildItem({ trainingExampleId: "example-1" })} - currentUserId="user-1" - />, - ); - - expect(bodyText()).toContain("Training example"); - expect(bodyText()).not.toContain("Train this decision"); - expect(mockApi.preview).not.toHaveBeenCalled(); - await waitFor(() => bodyText().includes("Frozen state")); - expect(bodyText()).toContain("You"); // provenance author - expect(bodyText()).toContain("Read-only"); // snapshot is visibly read-only - expect(bodyText()).toContain("I accepted because the plan covered rollback."); - expect(findClickable("Open full record")).toBeTruthy(); - - await act(() => findButton("Edit")?.click()); - const textarea = document.body.querySelector("textarea") as HTMLTextAreaElement; - expect(textarea).toBeTruthy(); - act(() => setTextareaValue(textarea, "Revised reasoning.")); - await act(() => findButton("Save notes")?.click()); - await waitFor(() => mockApi.updateNotes.mock.calls.length > 0); - - expect(mockApi.updateNotes).toHaveBeenCalledWith("example-1", "Revised reasoning."); - }); - - it("deletes after confirmation", async () => { - mockApi.get.mockResolvedValue(buildExample()); - mockApi.delete.mockResolvedValue(undefined); - const onOpenChange = vi.fn(); - - render( - , - ); - - await waitFor(() => bodyText().includes("Frozen state")); - await act(() => { - findButton("Delete")?.dispatchEvent(new MouseEvent("click", { bubbles: true, cancelable: true })); - }); - // Confirm dialog action. - await waitFor(() => bodyText().includes("Delete this training example?")); - const confirm = document.body.querySelector( - '[data-slot="alert-dialog-action"]', - ); - expect(confirm).toBeTruthy(); - await act(() => { - confirm?.dispatchEvent(new MouseEvent("click", { bubbles: true, cancelable: true })); - }); - await waitFor(() => mockApi.delete.mock.calls.length > 0); - - expect(mockApi.delete).toHaveBeenCalledWith("example-1"); - }); -}); diff --git a/ui/src/components/DecisionTrainingDrawer.tsx b/ui/src/components/DecisionTrainingDrawer.tsx deleted file mode 100644 index 7e31f83b1e0..00000000000 --- a/ui/src/components/DecisionTrainingDrawer.tsx +++ /dev/null @@ -1,521 +0,0 @@ -import { useState } from "react"; -import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query"; -import { - Clock, - ExternalLink, - FileText, - GitCommitHorizontal, - GraduationCap, - Loader2, - Lock, - MessageSquare, - Pencil, - Play, - Trash2, -} from "lucide-react"; -import type { AttentionItem, DecisionTrainingExample } from "@paperclipai/shared"; -import { Link } from "@/lib/router"; -import { decisionTrainingApi, type DecisionTrainingTarget } from "../api/decisionTraining"; -import { useToastActions } from "../context/ToastContext"; -import { queryKeys } from "../lib/queryKeys"; -import { codeResolutionLabel, decisionTrainingHref, trainingTargetForItem } from "../lib/decisionTraining"; -import { relativeTime } from "../lib/utils"; -import { Button } from "./ui/button"; -import { Textarea } from "./ui/textarea"; -import { - Sheet, - SheetContent, - SheetDescription, - SheetHeader, - SheetTitle, -} from "./ui/sheet"; -import { - AlertDialog, - AlertDialogAction, - AlertDialogCancel, - AlertDialogContent, - AlertDialogDescription, - AlertDialogFooter, - AlertDialogHeader, - AlertDialogTitle, - AlertDialogTrigger, -} from "./ui/alert-dialog"; - -const NOTES_PLACEHOLDER = - "How you thought about it, what signals mattered, what you decided and why…"; - -interface DecisionTrainingDrawerProps { - open: boolean; - onOpenChange: (open: boolean) => void; - companyId: string; - /** The Decisions row being trained; null when the drawer is closed. */ - item: AttentionItem | null; - currentUserId?: string | null; -} - -/** - * Right-hand drawer for capturing a decision-training example from a Decisions - * row. Renders a create state (immutable snapshot preview + notes) - * for untrained decisions and a saved state (provenance, editable notes, - * read-only snapshot, delete) once an example exists. Write affordances are - * only ever mounted for human users — agents never see this drawer. - */ -export function DecisionTrainingDrawer({ - open, - onOpenChange, - companyId, - item, - currentUserId, -}: DecisionTrainingDrawerProps) { - const [createdExample, setCreatedExample] = useState<{ - itemId: string; - exampleId: string; - } | null>(null); - const locallyCreatedExampleId = createdExample && createdExample.itemId === item?.id - ? createdExample.exampleId - : null; - const savedExampleId = item?.trainingExampleId ?? locallyCreatedExampleId; - - const target = item ? trainingTargetForItem(item) : null; - - return ( - - - - - - {savedExampleId ? "Training example" : "Train this decision"} - - - {savedExampleId - ? "The frozen state is read-only; your notes stay editable." - : "Freeze this decision's state and record how you'd want it decided."} - - - - {!item || !target ? ( -
- This decision can't be trained — it isn't anchored to an issue. -
- ) : savedExampleId ? ( - { - setCreatedExample(null); - onOpenChange(false); - }} - /> - ) : ( - setCreatedExample({ itemId: item.id, exampleId: example.id })} - onCancel={() => onOpenChange(false)} - /> - )} -
-
- ); -} - -function CreateState({ - companyId, - item, - target, - onCreated, - onCancel, -}: { - companyId: string; - item: AttentionItem; - target: DecisionTrainingTarget; - onCreated: (example: DecisionTrainingExample) => void; - onCancel: () => void; -}) { - const queryClient = useQueryClient(); - const { pushToast } = useToastActions(); - const [notes, setNotes] = useState(""); - - const preview = useQuery({ - queryKey: [...queryKeys.decisionTraining.list(companyId), "preview", target.sourceKind, target.sourceId, target.issueId], - queryFn: () => decisionTrainingApi.preview(companyId, target), - enabled: true, - }); - - const create = useMutation({ - mutationFn: () => decisionTrainingApi.create(companyId, { ...target, notes: notes.trim() }), - onSuccess: (example) => { - queryClient.invalidateQueries({ queryKey: queryKeys.attention(companyId) }); - queryClient.invalidateQueries({ queryKey: queryKeys.decisionTraining.list(companyId) }); - pushToast({ title: "Decision trained", tone: "success" }); - onCreated(example); - }, - onError: (error) => { - pushToast({ - title: "Could not train this decision", - body: error instanceof Error ? error.message : "Please try again.", - tone: "error", - }); - }, - }); - - return ( -
-
- - -
- -