Jannik/PaperClipAI
1
0 Fork 0
mirror of https://github.com/paperclipai/paperclip.git synced 2026-10-14 17:40:37 +02:00
Code Issues Packages Projects Releases Wiki Activity

docs(skills): distinguish Paperclip verification targets (#16145)

Browse Source 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Verification must show which runtime and user journey passed.
> - The existing skill centers on an empty instance launched from
source.
> - Seeded data, reused databases, published packages, and hosted stacks
prove different outcomes.
> - This pull request makes the target and starting conditions explicit.
> - It routes deployment to private Cloud skills and keeps feature
checks in Verify Paperclip.

## Linked Issues or Issue Description

**Issue type**
Missing documentation.

**Where is the issue?**
`.agents/skills/verify-paperclip/SKILL.md`.

**What's wrong?**
The skill does not distinguish source development, npm installation,
staging, and production. It also needs to distinguish fresh onboarding
from seeded or reused data and make platform scope explicit.

**Suggested fix**
Add target selection and focused references. Preserve the existing
empty-instance helper. Name the private Cloud deployment skills as a
separate prerequisite when deployment is requested. No duplicate
documentation PR was found in the open-PR search.

## What Changed

- Record artifact, environment, starting data, platform, and journey
before testing.
- Document dev servers, fresh instances, seeded/reused test drives, and
cloned worktrees.
- Separate exact-version canary and stable npm verification from source
and Cloud evidence.
- Document staging and production checks and route deployment to
`deploy-paperclip-stack` or `deploy-cloud-harness` in the private Cloud
repo.
- Limit local ownership helpers to instances created by their launcher.

## Verification

- Current-head GitHub CI: 55 passed, 2 skipped, no failures. Typecheck,
tests, build, and the aggregate gate passed. Greptile: 5/5, all threads
resolved.

- Skill validator: passed.
- Relative Markdown file links: passed across the changed skills and
references (29 links in final validation).
- `python3 -m unittest discover -s .agents/skills/verify-paperclip/tests
-v`: 2 passed.
- `node scripts/check-maps.mjs`: passed (structural coverage only).
- `git diff --check`: passed.
- `pnpm -r typecheck`, `pnpm test:run`, and `pnpm build`: attempted;
blocked by missing workspace dependencies (`node_modules`, Node type
definitions, and the checkout tsx entry point). These are not passing
full-suite results.
- Existing published-launcher test attempt was blocked by sandbox
loopback permissions; it does not provide package-install evidence.
- Reviewed commands against development docs, release workflow, and
current Cloud runbooks. No live deployment, publication, or production
mutation was performed for these documentation changes.

## Risks

- Low runtime risk: only skill documentation changes.
- Private deployment skills require access to the Cloud repo and must be
available before that handoff works. The verifier reports the
prerequisite when unavailable.
- Workflow changes can make command guidance stale. The skills direct
readers to maintained workflows and runbooks before use.

## Model Used

OpenAI GPT-6 via Codex, with reasoning, file editing, shell execution,
and GitHub CLI access. The runtime does not expose a more specific model
ID or context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (the focused ownership
suite; full-suite limits are listed above)
- [x] I have added or updated tests where applicable (no runtime change;
existing tests retained)
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed 2026-10-13 07:02:37 -05:00
1 parent 9bd820e687
commit fd61dc1dd5
3 files changed
+240 -7
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.agents/skills/verify-paperclip/SKILL.md
+75 -7
View File
@@ -1,6 +1,6 @@
---
name: verify-paperclip
description: Launch an isolated Paperclip instance, drive its real web UI or CLI using the existing feature-map recipes, capture evidence, and clean up owned processes. Use to verify user-facing Paperclip behavior.
description: Verify Paperclip from a source checkout, published npm install, or existing staging or production stack. Select the environment, data state, version, and platform; drive real UI or CLI journeys and capture evidence. Route deployment to the private Cloud skills.
---
# Verify Paperclip
@@ -15,7 +15,66 @@ recipe or access drift with evidence; map updates belong to
[maintain-verification-skill](../maintain-verification-skill/SKILL.md).
Read [development instructions](../../../doc/DEVELOPING.md) for current commands.
## Launch
## Select the verification target
State what is being tested before starting. Use the requested target; for ordinary
feature development, default to the current checkout on the current host with
isolated data. Expand into cross-platform or release qualification only when
requested or needed to reproduce the problem.
Record these independent dimensions together:
- **Artifact:** source checkout (path, commit, dirty state), published npm package
(requested channel and resolved exact version), or deployed image/release.
- **Environment:** local, staging, or production; target URL and intended company
or workspace; board/user/agent identity and authentication mode.
- **Starting data:** fresh and unonboarded, freshly seeded, cloned, or reused;
embedded versus external PostgreSQL; configuration/data location or a redacted
database identifier. Never record credentials or connection strings.
- **Platform:** server OS/architecture, Node/npm versions, and browser/client OS
when relevant. A Mac browser against a Linux server is Linux server evidence;
WSL is Linux evidence, not native Windows evidence.
- **Journey:** first installation/onboarding, an existing-user feature, upgrade,
or a live agent run; expected outcome and real versus simulated dependencies.
| Target | Use it to prove | Procedure |
| --- | --- | --- |
| Checkout dev server, existing or cloned data | Source behavior against existing state | [Checkout modes](references/verification-targets.md#checkout-and-data-modes) |
| Checkout with fresh empty data | First-run onboarding and subsequent features | [Launch](#launch-an-empty-checkout-instance) below |
| Checkout test drive, fresh seed or reused data | Features with a configured company/CEO | [Checkout modes](references/verification-targets.md#checkout-and-data-modes) |
| Published npm canary | End-user installation of a specific prerelease | [npm verification](references/npm-install.md) |
| Published npm stable (`latest`) | End-user installation of the released package | [npm verification](references/npm-install.md) |
| Existing staging stack | Deployed build, hosted auth, flags, integrations | [Hosted verification](references/verification-targets.md#staging-and-production) |
| Existing production stack | Current production journey within the authorized scope | [Hosted verification](references/verification-targets.md#staging-and-production) |
Fresh data does not imply a fresh install. A seeded test drive does not prove
onboarding. A source build does not prove npm packaging. A passing staging check
does not prove production, and a production Cloud stack does not prove the stable
npm install. Report only the combinations actually exercised.
## Deployment handoff
When a requested revision must first be built or deployed, use the skills in the
private `paperclip-cloud` repo:
- `deploy-paperclip-stack` at `.agents/skills/deploy-paperclip-stack/SKILL.md`
for a tenant workspace or an explicitly scoped fleet rollout.
- `deploy-cloud-harness` at `.agents/skills/deploy-cloud-harness/SKILL.md`
for the Cloud control plane itself.
Locate the authorized Cloud checkout or installed skill by name; do not assume a
sibling directory exists. Those skills own build preparation, deployment,
migrations, and rollout recovery. Return here with environment, target, resolved
source SHA, release/image identity, deployment result, and evidence link. If
private access is unavailable, report that prerequisite as blocked and continue
checks possible on the supplied target without inventing deploy commands.
A verification request alone does not authorize deployment, npm publication,
fleet-default changes, or a production rollout. Preserve explicit deployment
instructions already given. Publishing npm packages belongs to the existing
[release skill](../release/SKILL.md); installing one does not require publishing it.
## Launch an empty checkout instance
Run from the repository root with Node 24.11+, Python 3, `lsof`, and dependencies
installed with `pnpm install --frozen-lockfile`. Browser driving uses the existing
@@ -53,7 +112,11 @@ For Paperclip-managed reusable services, follow
instead of launching a second server. Its cloned-data/login requirements differ
from this empty local instance.
## Doctor
## Doctor (owned checkout helper only)
The doctor/stop commands require the real `launch.json` created by `launch.py`.
Do not fabricate one for dev servers, npm installs, test drives, or hosted stacks.
Use their own readiness and lifecycle procedures instead.
After startup, and again after any surprising behavior, run:
@@ -98,14 +161,16 @@ guessing resource URLs. The unprefixed `/onboarding` route starts naming; use th
company-prefixed onboarding route to resume its agent setup. Check relevant
experimental gates and actor permissions.
For CLI checks, discover exact flags with
For source-checkout CLI checks, discover exact flags with
`node cli/node_modules/tsx/dist/cli.mjs cli/src/index.ts <command> --help`, then
select the disposable data directory/API/company explicitly. API reads can
select the disposable data directory/API/company explicitly. For published npm
checks, use the installed version’s CLI and help instead. API reads can
corroborate UI writes; API-created state does not prove a creation UI.
## Evidence
Record feature file, sub-feature and entry-point ID, commit/dirty state, owned URL,
Include the target dimensions above, then record feature file, sub-feature and
entry-point ID, commit/dirty state, target URL,
company/user, flags, runtime/harness, live versus simulated dependencies, command,
exit code, expected and observed behavior, and evidence links. Report each entry
as passed, failed, blocked with its prerequisite, or not run.
@@ -126,7 +191,10 @@ documentation coverage rather than live test health.
## Cleanup
Close the browser context you created (the smoke does this in `finally`). Stop the owned runtime with:
Close the browser context you created (the smoke does this in `finally`). Leave
shared dev, staging, and production services running. The remaining cleanup steps
in this section apply only to `launch.py` instances; use the selected target’s
procedure for other launchers. Stop the helper-owned runtime with:
```sh
python3 .agents/skills/verify-paperclip/scripts/stop.py \
.agents/skills/verify-paperclip/references/npm-install.md
+78
View File
@@ -0,0 +1,78 @@
# Verify the published npm experience
Use this route for “can a user install and run this package?” Source startup,
workspace links, `pnpm build`, and local tarballs do not answer that question.
## Resolve the exact artifact
Read [.github/workflows/release.yml](../../../../.github/workflows/release.yml)
for publication behavior. Canary publication follows verified pushes to `master`;
an arbitrary feature-branch SHA need not have a published canary. Publication can
fail or still be running, so prove registry availability before claiming coverage.
Resolve the requested channel once against the public registry:
```sh
npm view paperclipai@canary version --registry=https://registry.npmjs.org
npm view paperclipai@latest version --registry=https://registry.npmjs.org
```
Use the one requested by the task: `canary` for prerelease verification, `latest`
for stable production npm verification, or an explicit version. Record resolved
version, registry, dist integrity, and source provenance from release metadata,
workflow, and tags. Pin that version for all later commands. If testing a SHA,
confirm the package maps to it; never substitute the current canary silently.
Missing publication is a blocked check, not a reason to publish during verification.
## Install like a new user
Use a disposable directory outside the checkout and fresh npm cache,
instance/config, and user-config paths. Use an isolated OS user/container when
proving absence of all pre-existing user state. Inspect inherited routing and
registry overrides without exposing secrets. Avoid workspace dependencies, local
`file:` packages, preinstalled Paperclip binaries, and cached `npx` installs.
Run the documented end-user entry point with the pinned package, for example
`npx --yes --registry=https://registry.npmjs.org paperclipai@<exact-version>`.
Read that version's `--help` before selecting `onboard`, `install`, or service
flags. For a foreground disposable check use `onboard --yes --bind loopback
--no-install-service --run --data-dir <fresh-directory>` when supported by that
version; keep ordinary interactive onboarding when that is the journey requested.
A managed CLI/service installation is separate from an ephemeral `npx` launch;
test it explicitly if requested.
Prove each relevant stage separately: registry download/install, CLI execution,
server readiness, visible onboarding, saved company/setup state, restart/reload,
and a first agent task when authorized. Successful `--version` alone does not
prove a usable install. For upgrades, first create state with the previous exact
version, then install the target against that same disposable data and verify
state survives; report this separately from a fresh install.
## Existing automation and its limits
The repo has a published-package onboarding suite:
```sh
PAPERCLIPAI_VERSION=<exact-published-canary-version> pnpm test:canary-onboarding-smoke
```
Read [its configuration](../../../../tests/canary-onboarding/playwright.config.ts)
and [launcher](../../../../tests/canary-onboarding/start-published-canary.mjs)
before running. It installs the published package into isolated temporary state,
uses a fresh npm cache and loopback server, and retains failure diagnostics.
It uses checkout Playwright tooling to drive a published runtime; record both
revisions. Its name and automated release wiring target canaries. For stable
verification, inspect compatibility with the selected version or use the manual
pinned-package path above; do not relabel a canary result as stable proof.
The Docker release-smoke suite is separate image evidence. Likewise,
`scripts/clean-install-npm.sh` defaults to source, and its published mode uses a
moving version and source-driven uninstall; it does not by itself prove the
whole lifecycle of a specific published version.
Capture OS/architecture, Node/npm versions, exact commands and exit codes,
installed version, serving revision when available, initial data state, screenshots,
logs, and observed outcomes. Redact credentials. Stop owned foreground processes
or uninstall an owned test service with the tested version's supported command;
retain evidence outside disposable data. Never run checkout ownership helpers
with invented metadata to stop a package process.
.agents/skills/verify-paperclip/references/verification-targets.md
+87
View File
@@ -0,0 +1,87 @@
# Checkout, data, and hosted targets
## Checkout and data modes
Read [DEVELOPING.md](../../../../doc/DEVELOPING.md) and
[DATABASE.md](../../../../doc/DATABASE.md) for the current startup contract.
- **Existing dev server:** use `pnpm dev` (watch) or `pnpm dev:once` from the
intended checkout. Inspect its instance routing and data configuration first;
do not assume the default port belongs to this checkout. Record the actual
served commit and dirty state. Reuse a matching running server when appropriate.
- **Fresh empty instance:** use the skill's `launch.py` helper with a new data
directory. Complete the UI or CLI onboarding journey under test. This is the
credential-free route; a live model run needs separate provider setup.
- **Seeded test drive:** run the documented CLI `test-drive --no-browser` with
an authorized environment-backed provider credential. It creates a temporary
embedded instance, completes setup, and seeds Test Company and a CEO. It does
not execute the first task. Verify that separately when required.
- **Reused test drive:** pass `--data-dir <existing-test-directory>`. If it already
contains a company, bootstrap flags do not replace companies, agents, or
secrets. Record existing state; do not label the result fresh onboarding.
Test drive rejects external PostgreSQL configuration and database URL overrides.
- **Managed or cloned worktree:** use
[dev-workspace run/verify/fix](../../paperclip-dev-workspace-run-verify-fix/SKILL.md).
Verify seed completion, instance routing, and login handoff before driving it.
Cloned data is existing-user evidence even when the checkout is new.
`--data-dir` alone is not proof of a fresh database: check effective config and
whether `DATABASE_URL` routes to an external database. Do not print its value.
For ordinary dev, inspect both environment overrides and the active database
config: with `DATABASE_URL` unset, `database.mode: postgres` and a saved
`connectionString` still select external PostgreSQL. Embedded PostgreSQL is the
default only when neither an environment override nor the active config selects
an external database; its instance directory may already hold data. Do not reset
shared data to make a test pass. Choose separate disposable data for first-run
checks and verify the effective backend before calling it isolated.
Record runtime PID, URL, port owner, checkout, and data location for a process you
start. Check `/api/health`, expected served revision, bootstrap/auth readiness,
and the actual page. Capture starting state and verify persistence after reload.
Use that launcher's documented stop procedure for an owned process; leave a
server you borrowed running.
## Platform scope
Ordinary development covers the current host unless the request or failure calls
for more. For a platform issue or release matrix, list required combinations:
macOS, Linux, native Windows, architecture, runtime, and install path. Mark each
combination passed, failed, blocked, or not run separately.
The local helper needs POSIX tools such as `lsof`; do not assume it qualifies
native Windows. Use the supported CLI and native shell for that platform, with
isolated instance/config/cache paths and platform-appropriate process cleanup.
A container proves its guest OS and selected architecture, not the host's native
installer, service manager, file permissions, or path behavior. Keep browser and
server platforms distinct when testing a remote Linux box from a Mac.
## Staging and production
Use the supplied tenant URL and intended identity. Determine whether the target
is the Paperclip tenant application or the Cloud control plane. Confirm the
environment from the operator's deployment record and authenticated routing;
a page title or local Git checkout is not evidence of the serving revision.
If a build/deploy is needed, follow the private skill handoff in `SKILL.md` first.
Record the exact serving SHA and release/image identity after deployment, not
just the requested branch, CI result, or fleet default. Recheck after a long run:
a staging auto-roll or another deployment can change the serving version. Mark
unknown or changing provenance as inconclusive for a claim about a specific SHA.
For staging, use a designated test workspace and its real hosted sign-in/entry
flow. Record fresh versus existing workspace state, experimental flags, relevant
provider connections, and whether dependencies are live. Check tenant readiness
and then the feature journey, including persistence or continuation. A control
plane `/healthz` response or tenant HTML 200 is not feature acceptance.
For production, keep verification within the requested stack, identity, and
journey. Prefer observation of existing state unless writes are authorized; use
an agreed test workspace for mutating checks. Do not run the empty-instance
onboarding smoke, reset data, change flags, or trigger paid agent work merely to
make a production check pass. If a necessary write is outside the request,
report that missing authorization and complete the read-only checks.
Never use local ownership helpers against a remote server. Close only browser
sessions you created and clean up only test records whose removal is authorized.
Keep shared services running. Report production Cloud verification separately
from stable npm verification; the shipped artifacts and configuration differ.