fix(skills-catalog): make the harness non-destructive and ship its evidence

Address the three review findings on the connector skills. - offline-verification.md: the documented harness took the output directory as a required argument and opened with `rm -rf "$OUT"`. It now allocates a fresh directory with `mktemp -d` when none is given, and refuses every existing destination, the filesystem root, relative paths, symlinks, and paths inside the checkout or the corpus. It deletes nothing. The two helper scripts drop a marker file in their own root and refuse to run without it. All seven refusals and both helper guards were executed against real paths with canary files; the recorded output is in the document. - connect-agent-tools: add references/verification-log.md, so the evidence behind every `verified` label ships with the package instead of pointing at a file outside the repository. The two deferred rows named a private issue tracker; they now describe what the follow-up needs and what it starts from. Regenerated the catalog manifest. Both packages stay markdown_only. Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-28 12:04:41 +01:00 · 2026-09-20 19:53:06 +00:00
1 parent 0fe055cc8f
commit 72d065a535
@@ -25,9 +25,42 @@ if you prefer to run it as a script.
 Save as `make-harness.sh` outside the App checkout, then:

 ```sh
-./make-harness.sh <app-repo> <commit-ish> <out-dir> <corpus-dir>
+./make-harness.sh <app-repo> <commit-ish> <corpus-dir> [out-dir]
 ```

+Omit `[out-dir]` and the script allocates a fresh directory for you. That is the
+recommended form.
+
+### The script deletes nothing
+
+An earlier revision of this recipe took the output directory as a required
+argument and opened with `rm -rf "$OUT"`. That is a foot-gun: a contributor who
+typed `/`, their checkout path, or any directory they cared about lost it before
+verification started. The recipe below has no `rm -rf` on a caller-supplied
+path, and no `rm` of any kind outside a directory it created itself.
+
+The rules it follows:
+
+| Input | What happens |
+| --- | --- |
+| No `out-dir` given | `mktemp -d` allocates a fresh unique directory. |
+| A relative path | Refused. It resolves against the current directory, which is too easy to get wrong. |
+| `/` | Refused. |
+| A path that already exists — file or directory, empty or not | Refused. Pass a new path or omit the argument. |
+| A symlink | Refused before anything is read or written through it. |
+| A path whose parent does not exist | Refused. The script creates one level, never a tree. |
+| A path inside the App checkout or the corpus | Refused after resolving the parent with `pwd -P`, so `..` and symlinked parents cannot slip past. |
+| Anything else | `mkdir` creates it — not `mkdir -p`, so a race that creates it first is an error, not a silent reuse. |
+
+Refusing every existing destination is what makes this safe, and it is why the
+list above is short: `/`, the repository, and "some directory I cared about" are
+all the same case. Nothing needs to decide which existing directories are
+precious, because none of them are accepted.
+
+The two helper scripts the harness writes each drop a marker file in their own
+root and refuse to run without it, so neither can operate on your checkout if
+you run it from the wrong directory.
+
 <details>
 <summary><code>make-harness.sh</code></summary>

@@ -36,25 +69,67 @@ Save as `make-harness.sh` outside the App checkout, then:
 # Build an isolated harness that runs the real catalog generator, the real brand
 # validators, and the real catalog tests against a copy of a pinned commit.
 #
-# Nothing in the App checkout is modified: files are exported with `git archive`
-# and `git show`, and node_modules is symlinked read-only.
+# This script deletes nothing. Nothing in the App checkout is modified: files
+# are exported with `git archive` and `git show`, and node_modules is symlinked
+# read-only.
 #
 # Usage:
-#   make-harness.sh <app-repo> <commit-ish> <out-dir> <corpus-dir>
+#   make-harness.sh <app-repo> <commit-ish> <corpus-dir> [out-dir]
+#
+# With no [out-dir] a fresh directory is allocated with `mktemp -d`. With one,
+# the path must be absolute and must not exist: the script creates it, and
+# refuses every other case rather than clearing anything.

 set -euo pipefail

 APP_REPO=${1:?app repo path}
 COMMIT=${2:?commit-ish}
-OUT=${3:?output dir}
-CORPUS=${4:?ingestion corpus dir}
+CORPUS=${3:?ingestion corpus dir}
+OUT=${4:-}

-[ -d "$CORPUS" ] || { echo "corpus not found: $CORPUS" >&2; exit 1; }
-[ -d "$APP_REPO/node_modules" ] || { echo "install dependencies in $APP_REPO first" >&2; exit 1; }
+die() { printf 'make-harness: %s\n' "$*" >&2; exit 2; }
+
+[ -d "$APP_REPO" ] || die "app repo not found: $APP_REPO"
+[ -d "$CORPUS" ] || die "corpus not found: $CORPUS"
+[ -d "$APP_REPO/node_modules" ] || die "install dependencies in $APP_REPO first"
+
+APP_REPO_REAL=$(cd "$APP_REPO" && pwd -P)
+CORPUS_REAL=$(cd "$CORPUS" && pwd -P)
+
+# --- allocate the output directory -------------------------------------------
+# Every branch below either creates a new directory or refuses. There is no
+# path through this block that removes anything.
+if [ -z "$OUT" ]; then
+  OUT=$(mktemp -d "${TMPDIR:-/tmp}/paperclip-harness.XXXXXXXX")
+  echo "allocated a fresh harness directory: $OUT"
+else
+  case $OUT in
+    /*) ;;
+    *) die "out-dir must be an absolute path, got '$OUT'" ;;
+  esac
+  OUT=${OUT%/}
+  [ -n "$OUT" ] || die "refusing the filesystem root as out-dir"
+  # -L first: a dangling symlink is invisible to -e.
+  if [ -L "$OUT" ]; then die "out-dir is a symlink, refusing to write through it: $OUT"; fi
+  if [ -e "$OUT" ]; then die "out-dir already exists: $OUT (pass a new path, or omit it to allocate a fresh one)"; fi
+  PARENT=$(dirname "$OUT")
+  [ -d "$PARENT" ] || die "parent directory does not exist: $PARENT"
+  # Resolve the parent so `..` and symlinked parents cannot escape the checks.
+  OUT="$(cd "$PARENT" && pwd -P)/$(basename "$OUT")"
+  case $OUT/ in
+    "$APP_REPO_REAL"/*) die "out-dir is inside the App checkout: $OUT" ;;
+    "$CORPUS_REAL"/*) die "out-dir is inside the corpus: $OUT" ;;
+  esac
+  # mkdir, not `mkdir -p`: if something created the path in the meantime, stop.
+  mkdir "$OUT" || die "could not create $OUT"
+fi

 SHA=$(git -C "$APP_REPO" rev-parse "$COMMIT")
-rm -rf "$OUT"; mkdir -p "$OUT/gen/scripts" "$OUT/vt/packages/shared" "$OUT/vt/ui/public/brands"
-echo "$SHA" > "$OUT/PINNED_COMMIT"
+mkdir -p "$OUT/gen/scripts" "$OUT/vt/packages/shared" "$OUT/vt/ui/public/brands"
+printf '%s\n' "$SHA" > "$OUT/PINNED_COMMIT"
+# Markers. The helper scripts below refuse to run without the one for their root.
+: > "$OUT/gen/.paperclip-harness-gen"
+: > "$OUT/vt/.paperclip-harness-vt"

 # --- generator harness -------------------------------------------------------
 for f in scripts/ingest-app-definitions.mjs \
@@ -81,6 +156,10 @@ cat > "$OUT/gen/verify-fidelity.sh" <<'FIDELITY'
 # Regenerate from the pristine source and prove the harness reproduces the
 # checked-in output exactly. Run this BEFORE applying your own change.
 set -euo pipefail
+# The generator writes into its working directory. Refuse to run anywhere but
+# the harness root, so a wrong `cd` cannot regenerate over a real checkout.
+[ -f .paperclip-harness-gen ] || {
+  echo "run this from the harness gen root (no .paperclip-harness-gen here)" >&2; exit 2; }
 : "${PAPERCLIP_CONTENT_TEMPLATES:?point this at the ingestion corpus}"
 node scripts/ingest-app-definitions.mjs
 diff -rq .baseline-definitions packages/shared/src/app-definitions
@@ -103,7 +182,14 @@ cat > "$OUT/vt/sync-from-gen.sh" <<'SYNC'
 # tests run against exactly what you generated.
 set -euo pipefail
 GEN=${1:?path to the gen harness}
-rm -rf ui/public/brands/apps && mkdir -p ui/public/brands
+# Both ends must be harness roots. The only `rm` below is a fixed relative path
+# under a directory these two markers prove is a harness, never a checkout.
+[ -f .paperclip-harness-vt ] || {
+  echo "run this from the harness vt root (no .paperclip-harness-vt here)" >&2; exit 2; }
+[ -f "$GEN/.paperclip-harness-gen" ] || {
+  echo "not a harness gen root: $GEN" >&2; exit 2; }
+mkdir -p ui/public/brands
+rm -rf ./ui/public/brands/apps
 cp -r "$GEN/ui/public/brands/apps" ui/public/brands/apps
 cp "$GEN/packages/shared/src/app-definitions.ts" packages/shared/src/app-definitions.ts
 cp "$GEN/packages/shared/src/app-definitions.generated.ts" packages/shared/src/app-definitions.generated.ts
@@ -120,14 +206,66 @@ echo "  vitest:    $OUT/vt/packages/shared"
 </details>

 ```text
-harness ready at /tmp/harness (pinned e558f25e99020b8ab762b91a8eabb0fe383f993f)
-  generator: /tmp/harness/gen
-  vitest:    /tmp/harness/vt/packages/shared
+$ ./make-harness.sh <app-repo> HEAD <corpus-dir>
+allocated a fresh harness directory: /tmp/paperclip-harness.NWeZ7RkG
+harness ready at /tmp/paperclip-harness.NWeZ7RkG (pinned 2a99de80ec52db01eead901f28323926ceaf3c1d)
+  generator: /tmp/paperclip-harness.NWeZ7RkG/gen
+  vitest:    /tmp/paperclip-harness.NWeZ7RkG/vt/packages/shared
+
+$ ./make-harness.sh <app-repo> HEAD <corpus-dir> /tmp/named-harness
+harness ready at /tmp/named-harness (pinned 2a99de80ec52db01eead901f28323926ceaf3c1d)
+  generator: /tmp/named-harness/gen
+  vitest:    /tmp/named-harness/vt/packages/shared
 ```

-The output above is from a run at Paperclip App commit `e558f25e` on
-15 September 2026, Node v24.20.0. The harness was previously exercised at
-`728f7185`; the recipe did not need changing between the two.
+Both forms above are from runs on 20 September 2026 at Paperclip App commit
+`2a99de80ec`, Node v24.20.0, bash 5. The recipe was previously exercised at
+`728f7185` and `e558f25e` under its older signature; the only change since is
+the output-directory handling.
+
+### The refusals, executed
+
+Every unsafe input below was run against a **real** path, each with a canary
+file where a directory was involved. Nothing was deleted: the canaries, the App
+checkout, and `/` were all intact afterwards.
+
+```text
+$ make-harness.sh <app-repo> HEAD <corpus> /
+make-harness: refusing the filesystem root as out-dir                              exit 2
+
+$ make-harness.sh <app-repo> HEAD <corpus> <app-repo>
+make-harness: out-dir already exists: <app-repo>
+              (pass a new path, or omit it to allocate a fresh one)                exit 2
+
+$ make-harness.sh <app-repo> HEAD <corpus> /tmp/canary/existing
+make-harness: out-dir already exists: /tmp/canary/existing
+              (pass a new path, or omit it to allocate a fresh one)                exit 2
+
+$ make-harness.sh <app-repo> HEAD <corpus> relative-harness
+make-harness: out-dir must be an absolute path, got 'relative-harness'             exit 2
+
+$ make-harness.sh <app-repo> HEAD <corpus> /tmp/canary/link       # → symlink-target
+make-harness: out-dir is a symlink, refusing to write through it: /tmp/canary/link exit 2
+
+$ make-harness.sh <app-repo> HEAD <corpus> <app-repo>/tmp-harness
+make-harness: out-dir is inside the App checkout: <app-repo>/tmp-harness           exit 2
+
+$ make-harness.sh <app-repo> HEAD <corpus> /tmp/no/such/parent/out
+make-harness: parent directory does not exist: /tmp/no/such/parent                 exit 2
+```
+
+The two helper guards were exercised the same way, by running each from the App
+checkout instead of its own harness root:
+
+```text
+$ cd <app-repo> && <out-dir>/gen/verify-fidelity.sh
+run this from the harness gen root (no .paperclip-harness-gen here)                exit 2
+
+$ cd <app-repo> && <out-dir>/vt/sync-from-gen.sh <out-dir>/gen
+run this from the harness vt root (no .paperclip-harness-vt here)                  exit 2
+```
+
+`git status` in the App checkout was empty after all nine runs.

 Two roots, because they have different working directories:

@@ -148,10 +286,13 @@ PAPERCLIP_CONTENT_TEMPLATES=<corpus-dir> ./verify-fidelity.sh
 ```

 ```text
-Parsed 99 captures and 179 states; emitted 70 Wave 1 definitions and flagged 63 states for review.
+Parsed 99 captures and 179 states; emitted 72 Wave 1 definitions and flagged 63 states for review.
 fidelity OK: harness reproduces the checked-in output byte-for-byte
 ```

+That is today's run at `2a99de80ec`. The definition count tracks the commit —
+it was 70 at `e558f25e` — so compare the `fidelity OK` line, not the number.
+
 The corpus is in the non-public `paperclip-content` repository. Without it the
 generator refuses to run and this harness cannot be built — report that as a
 blocked prerequisite rather than editing the guard.
@@ -191,6 +332,20 @@ Test Files  2 passed (2)
     Tests  27 passed (27)
 ```

+The same two suites run green through a clean harness before you change
+anything. On 20 September 2026 at `2a99de80ec`, straight after
+`sync-from-gen.sh`:
+
+```text
+ RUN  v4.1.11 /tmp/named-harness/vt/packages/shared
+
+ Test Files  2 passed (2)
+      Tests  28 passed (28)
+   Duration  632ms
+```
+
+Establish that baseline first. A red suite you did not cause is not your result.
+
 Also diff the generated registry and confirm the only changes are positional.
 At this commit one inserted provider moved 125 lines of `a<N>` imports in
 `app-definitions.generated.ts` and changed nothing semantic in another
@@ -33,7 +33,10 @@ Paperclip Cloud account, no managed Cloud OAuth client, and no Paperclip-operate
 infrastructure. Cloud is covered as a third deployment shape, not as the
 reference implementation. Where the two diverge, the divergence is written down
 in [`references/deployment-support-matrix.md`](references/deployment-support-matrix.md)
-with the guard or document behind it.
+with the guard or document behind it. Where that file says `verified`, the run
+behind it — commands, observed output, and the correlated agent run — is in
+[`references/verification-log.md`](references/verification-log.md), so you can
+redo it rather than take the label on trust.

 ## Use This When

@@ -102,8 +102,10 @@ third-party public server with `auth: "none"`.
 `unsupported`, and none of it is inferred from a self-hosted result. The same
 honesty runs in reverse: a shape-A pass is not shape-B evidence.

-Full reproduction, run IDs, and provider-side artifacts:
-[`PAP-18465-SELF-HOSTED-VERIFICATION.md`](../../PAP-18465-SELF-HOSTED-VERIFICATION.md).
+Full reproduction — instance configuration, the disposable provider, all nine
+runbook scenarios with their commands and observed output, the correlated run
+ids, the provider-side artifact, and every finding below in detail — ships in
+this package: [`verification-log.md`](./verification-log.md).

 ## Matrix

@@ -195,11 +197,13 @@ public HTTPS origin. Do not tell them Cloud is the only answer, and do not embed
 a server tutorial in the wizard — link maintained setup documentation
 (`CHAT-CONNECTOR-UX.md:81-84`).

-## Two findings that change what you should tell an operator
+## Three findings that change what you should tell an operator

 These are reproducible product behaviours at commit `9335b7db10`, not advice.
-Both are filed for product engineering; until they change, the skill's guidance
-has to account for them.
+Until they change, the skill's guidance has to account for them. Each one below
+has its executed reproduction and a suggested fix in
+[`verification-log.md`](./verification-log.md) §4, along with six further
+findings that do not change the advice.

 ### F1 — a pasted MCP server can widen its own permissions after approval

@@ -263,8 +267,8 @@ on the path the skill recommends.
 | --- | --- | --- |
 | Cloud parity for every row above | No Cloud instance was reachable, and the self-hosted-first priority defers it | Operator with a Cloud instance, after self-hosted |
 | OAuth (DCR / CIMD / own client) on any shape | Needs an authorized provider account | Operator with a provider account |
-| `local_stdio` execution | The live `supportMatrix` says shape A supports it, which is not the same as running it | Follow-up under [PAP-18464](/PAP/issues/PAP-18464) |
-| Chat and email inbound delivery | Out of scope for the first pass | Follow-up under [PAP-18464](/PAP/issues/PAP-18464) |
+| `local_stdio` execution | The live `supportMatrix` says shape A supports it, which is not the same as running it | Unclaimed. Needs a shape-A operator to create one `local_stdio` connection from an approved template and run a gateway call through it. Everything needed is in [`verification-log.md`](./verification-log.md) §1 and §3; swap the transport. |
+| Chat and email inbound delivery | Out of scope for the first pass | Unclaimed. Needs an operator with a chat provider and a decision on inbound mechanism — see [`chat-and-email.md`](./chat-and-email.md) for the questions to settle first. |
 | Shape-B write execution and provider-side readback | The shape-B provider is read-only | Operator with a writable third-party provider |
 | Link-local denial | Source-cited only; no live call | Low priority — the guard is unconditional |

@@ -0,0 +1,493 @@
+# Verification log — how the support matrix was filled
+
+This is the evidence behind
+[`deployment-support-matrix.md`](./deployment-support-matrix.md). Every
+`verified` label in that file traces to a run recorded here.
+
+Executed 19 September 2026 against Paperclip App commit
+`9335b7db10425277bcb84ba8137d08c498324dba`, on **two disposable self-hosted
+instances started for this pass and torn down afterwards**. No Cloud instance
+was in reach. No third-party account was used beyond one public read-only
+server. Nothing was installed, released, or written to anyone else's data.
+
+The point of shipping this inside the package is that you can redo it. The
+instance shapes, the disposable provider, and every command below are
+reproducible on a laptop in well under an hour, with no Paperclip account and no
+credential you do not already hold.
+
+Run ids and connection ids appear truncated. They are here so you can see that
+correlation between an agent run and a gateway invocation was actually
+demonstrated, not to be looked up — they belong to instances that no longer
+exist. No credential value, secret, private log, or account identifier appears
+anywhere in this file.
+
+## 1. The two instances
+
+Read from each instance's `config.json` and confirmed against its live
+`/api/health`. Neither instance had `PAPERCLIP_DEPLOYMENT_MODE`,
+`PAPERCLIP_DEPLOYMENT_EXPOSURE`, or `PAPERCLIP_PUBLIC_URL` set in the
+environment; the equivalent `config.json` keys are the source, which is what
+`server/src/config.ts:170-190` reads after the environment.
+
+| | **Instance A — shape A** | **Instance B — shape B** |
+| --- | --- | --- |
+| Deployment mode | `config.server.deploymentMode = local_trusted` | `config.server.deploymentMode = authenticated` |
+| Exposure | **forced** `private` by `config.ts:187-190` | run as `private` **and** `public` |
+| Public base URL | not set, not needed | `auth.publicBaseUrl` set and mandatory — see F7 |
+| Bind | loopback `127.0.0.1:3810` | loopback `127.0.0.1:3820` |
+| Database | embedded PostgreSQL | **external** PostgreSQL — embedded is refused, see F7 |
+| Health `commit` | `9335b7db10…` | same |
+
+`GET /api/companies/{companyId}/tools/runtime-health` → `supportMatrix`, read
+with board access on each instance:
+
+**Instance A, `local_trusted` (⇒ `private`):**
+
+```json
+{
+  "remoteHttp": {
+    "supported": true,
+    "note": "mcp_remote MCP connections are supported in hosted cloud and local deployments."
+  },
+  "localStdio": {
+    "supported": true,
+    "note": "local_stdio is available for local trusted mode or through the configured trusted MCP runtime host."
+  }
+}
+```
+
+**Instance B, `authenticated` + `private`:**
+
+```json
+{
+  "remoteHttp": {
+    "supported": true,
+    "note": "mcp_remote MCP connections are supported in hosted cloud and local deployments."
+  },
+  "localStdio": {
+    "supported": false,
+    "note": "local_stdio should stay disabled for authenticated/private; use mcp_remote or configure a trusted runtime worker."
+  }
+}
+```
+
+**Instance B, `authenticated` + `public`:** identical to the private block, with
+the `authenticated/public` wording. The `supportMatrix` does **not** narrow for
+public exposure; the private-endpoint refusal happens at connect time, not here.
+
+## 2. The providers, and why these two
+
+1. **A disposable loopback MCP server with `auth: "none"`** — a small "notes"
+   server written for this pass, streamable HTTP on `http://127.0.0.1:38111/mcp`,
+   storing notes in a scratch directory.
+
+   Two reasons. `auth: "none"` means a real agent run needs no OAuth consent and
+   no credential the tester is not authorized to hold. And a loopback endpoint
+   is the one class a `local_trusted` instance may reach and Cloud may not, so
+   it is the case that proves the most about the self-hosted column. It also
+   makes an authorized write safe: the write target is a scratch directory, with
+   no third-party account anywhere near it.
+
+2. **DeepWiki (`https://mcp.deepwiki.com/mcp`), a real third-party public
+   provider with `auth: "none"`.** Used on instance B so the public-endpoint row
+   is proved against something outside the machine rather than by analogy.
+   Read-only; nothing was written to it.
+
+3. **Not used: an OAuth provider.** Completing consent needs an authorized
+   account on that provider. The unauthenticated metadata probe in the skill's
+   quickstart is current; the consent half stays `untested`.
+
+No catalog definition was authored. `CONNECTOR-PLAYBOOK.md:23-26` says the
+pasted-URL path needs no code change, and it did not.
+
+### Provider evidence record
+
+| Item | Disposable notes | DeepWiki |
+| --- | --- | --- |
+| Endpoint | `http://127.0.0.1:38111/mcp` | `https://mcp.deepwiki.com/mcp` |
+| Unauthenticated `initialize` | `HTTP 200`, `protocolVersion 2025-06-18`, `serverInfo disposable-notes/1.0.0` | `HTTP/2 200`, `serverInfo DeepWiki 2.14.3` |
+| Auth | none | none |
+| Registration / PKCE | n/a — no OAuth challenge | n/a — no OAuth challenge |
+| Prerequisites | the server process running on the same host | public egress |
+| Revocation | stop the process, delete the connection | delete the connection |
+| Date read | 19 September 2026 | 19 September 2026 |
+
+## 3. The nine production-validation scenarios
+
+These are the runbook's scenarios (`doc/connections/CONNECTOR-PLAYBOOK.md:1445-1455`).
+Environment for every row unless stated otherwise: **instance A**,
+`local_trusted` + `private`, commit `9335b7db10`, method key
+`connect-your-own-mcp-server` (pasted URL, `authMode: none`).
+
+### 1. Setup and consent — pass
+
+- **Reproduce.** `POST /api/companies/{c}/tools/mcp/import-json` with a standard
+  `mcpServers` block → one draft, `transport: mcp_remote`, no warnings. Then
+  `POST /api/companies/{c}/tools/apps/connect` with `{link, name, authMode: "none"}`
+  → draft connection, `enabled: false`, `config.unverifiedServer: true`. Then
+  `POST …/tools/apps/{id}/finish` with explicit `enabledCatalogEntryIds`,
+  `askFirstCatalogEntryIds` and `access.agentIds`.
+- **Expected.** The no-code path completes with no code change, and access is not
+  granted until the operator finishes the wizard.
+- **Actual.** Exactly that. `finish` is where consent is expressed: it refuses an
+  empty body, and the draft grants nothing until it is called.
+- **Evidence.** Connection `f79c1799…` ended `status: active, enabled: true,
+  healthStatus: ok`; activity rows `tool_app.connected` then `tool_app.finished`
+  carrying `access: {agentIds: [...]}`.
+
+### 2. Authentication — pass, for the two shapes exercised
+
+- **Reproduce.** (a) an `authKind: "none"` connection, health-checked.
+  (b) an `authKind: "api_key"` connection with
+  `credentialRefs: [{placement: "header", key: "x-api-key", secretId}]`, where the
+  secret is a synthetic string created through `POST /api/companies/{c}/secrets`
+  and never printed.
+- **Expected.** The credential reaches the provider as a header and is never
+  stored or echoed in clear.
+- **Actual.** The provider's own request log records the inbound request with
+  `credentialHeaderKeys: ["x-api-key"]`, and the health check returned `ok`.
+- **Not covered.** No OAuth path was exercised. That is `untested`, not passed.
+- **Evidence.** Connection `9736bf6d…`; the redaction scan is scenario 9.
+
+### 3. Catalog and configuration — pass, with an adverse finding
+
+- **Reproduce.** `POST /api/tool-connections/{id}/catalog/refresh`, then read
+  `…/catalog` and the effective profile.
+- **Expected.** Discovery returns the reviewed actions with correct risk
+  annotations, and selected access persists.
+- **Actual, good half.** All 7 tools discovered. Risk classification is correct
+  and automatic: `list_notes`, `read_note`, `count_notes`, `note_stats` and
+  `export_notes` = `read`; `create_note` = `write`; `delete_note` =
+  `destructive, isDestructive: true`. Selections persisted across refreshes: a
+  tool whose profile entry was removed stayed off.
+- **Actual, adverse half.** See **F1**: a tool the provider adds *after* approval
+  is auto-allowed on the next refresh.
+- **Evidence.** Activity rows
+  `tool_connection.catalog_refresh {discoveredCount: 5|6|7, quarantinedCount: 0}`.
+
+### 4. Allowed execution — pass
+
+- **Reproduce.** Narrow read: an agent run calling `list-notes` and `read-note`.
+  Authorized write: `create_note` against the disposable store, promoted into the
+  profile deliberately after the denial case was recorded.
+- **Expected.** The read succeeds through the gateway; the write follows the
+  effective policy.
+- **Actual.** Both succeeded — `policyDecision: allow`, `reasonCode: allow_profile`,
+  `status: succeeded`, with argument and result hashes recorded.
+- **Evidence.** Invocation `38fa28e0…` (`read-note`, run `af9afa19…`) and the
+  three invocations of run `135b2654…`.
+
+### 5. Denied execution — pass on the agent path, fail on the board Test path
+
+- **Reproduce.** The policy simulator
+  (`POST /api/companies/{c}/tools/policy/test`) for four actor/tool pairs, plus
+  live agent execution, plus the board test-call endpoint.
+- **Expected.** Ungranted actors and unselected tools cannot execute, and a tool
+  the policy hides does not appear in the listing.
+- **Actual, agent path — correct.**
+
+  | Actor | Tool | Decision |
+  | --- | --- | --- |
+  | granted agent | `list_notes` | `allow / allow_profile` |
+  | granted agent | `create_note`, before promotion | `deny / deny_default` |
+  | ungranted agent | `list_notes` | `deny / deny_default` |
+  | granted agent, disabled connection | `list_notes` | `deny / deny_disabled_connection` |
+
+  Hiding works too. In run `af9afa19…` the agent searched its session for both
+  `create-note` and `create_note`, found neither, attempted the call anyway, and
+  got `No such tool available` — a local dispatch failure, with **no request
+  reaching the provider**, confirmed in the provider's request log.
+
+- **Actual, board Test path — fails the scenario.** See **F2**.
+
+### 6. Runtime delivery — pass on both instances
+
+- **Reproduce.** Create an issue, assign it to the agent that has the connection
+  installed, let the scheduler start a heartbeat run, then correlate
+  `GET /api/companies/{c}/tools/runs/{runId}/decisions`.
+- **Expected.** A real agent run's gateway call succeeds and correlates to the run.
+- **Actual.** Two real runs on instance A, both `succeeded`:
+
+  | Run | Calls | Result |
+  | --- | --- | --- |
+  | `af9afa19…` | `list-notes`, `read-note` | allowed; `create-note` absent from the session |
+  | `135b2654…` | `create-note`, `read-note`, `list-notes` | all allowed |
+
+  The tool namespace the agent used was
+  `mcp__paperclip-assigned__mcp_disposable-notes-…_*` — the managed connection,
+  not a side channel.
+
+- **Provider-side readback.** Read from the provider's own store, not through
+  Paperclip. The note file written by the agent contains:
+
+  ```text
+  Written by Notes Runner during connector verification.
+  Heartbeat run id: 135b2654-…
+  ```
+
+  The artifact names the run that created it, and that run id matches the gateway
+  invocation for `create-note`. That correlation is the whole proof: the agent,
+  not the tester, did the work, and it went through the managed connection.
+
+- **Instance B, `authenticated` + `public`.** Run `f027aa16…`, `succeeded`, one
+  correlated invocation:
+  `mcp.app-gallery-link-…:read-wiki-structure | allow | succeeded | args {"repoName": "paperclipai/paperclip"}`.
+  The agent reported DeepWiki's 13 top-level sections verbatim, and separately
+  attempted `read_wiki_contents` — the tool it was *not* granted at `finish` —
+  which failed at local dispatch with no request reaching DeepWiki. So shape B
+  has an allowed agent execution and a hidden-tool denial in the same run,
+  against a real third-party provider. No provider-side readback there: DeepWiki
+  is read-only.
+- **It took three shape-B runs to get there, and both failures were findings
+  rather than flakes.** Run 1 failed on `ENOTFOUND` because the mandatory
+  `auth.publicBaseUrl` is also what the runtime hands spawned agents as their
+  control-plane URL (**F8b**). Run 2 failed because the wizard had granted the
+  policy but not the install (**F10**). Both agents refused to fabricate a result
+  and reported the blocker accurately, which is why the failures were
+  diagnosable.
+
+### 7. Refresh and recovery — pass on recovery, fail on refresh
+
+- **Reproduce.** Stop the provider → health-check → gateway call. Restart →
+  health-check → refresh.
+- **Expected.** Failures are recoverable, identity and policy survive, no
+  duplicate connection.
+- **Actual.** Down: `healthStatus: error`, `healthMessage: "fetch failed"`,
+  `HTTP 502 {"code": "runtime_error"}`. Up: `healthStatus: ok` on the next check,
+  same connection id, grants and profile intact, connection count unchanged.
+  Refresh itself carries **F1**; the down-state gateway error carries **F3**.
+
+### 8. Revoke and reconnect — pass, with a reporting gap
+
+- **Reproduce.** `DELETE /api/tool-connections/{id}/grants/{grantId}` → policy
+  simulator and a live gateway call. Then
+  `POST …/grants/installations {"isDefault": true}` → a live gateway call.
+- **Expected.** Revocation blocks later execution; reconnect reuses the intended
+  identity.
+- **Actual.** After revocation the gateway refuses:
+  `{"error": {"message": "Organization authorization is required", "reasonCode": "organization_authorization_required"}}`,
+  logged as `call_failed`. Disabling the connection refuses earlier, at policy:
+  `deny / deny_disabled_connection`. After re-adding the installation the same
+  read succeeds again, on the same connection id, with a new grant id and no
+  duplicate connection. The gap is **F4**.
+
+### 9. Activity and secret handling — pass
+
+- **Reproduce.** Read `…/tool-connections/{id}/activity` and
+  `…/companies/{c}/activity`. Then scan for the synthetic header value across the
+  database, the instance file tree, and the API.
+- **Actual, activity.** 20 connection events and 49 company `tool_*` rows, each
+  carrying actor type, decision, reason code, tool, risk, and run/issue
+  correlation — for example `policy_decision | allow | allow_profile |
+  mcp.disposable-notes-…:read-note | agent` paired with
+  `call_completed | tool_completed`, and
+  `call_failed | deny | organization_authorization_required` for the
+  revoked-grant attempt.
+- **Actual, redaction.**
+
+  | Surface scanned | Occurrences of the secret value |
+  | --- | --- |
+  | Instance database, all 1358 `text` / `varchar` / `json` / `jsonb` columns | **0** |
+  | Whole instance tree — config, `.env`, logs, run logs, storage, backups | **0** |
+  | `GET …/tools/connections`, `GET /tool-connections/{id}`, `GET …/secrets`, `GET …/activity?limit=50`, `GET …/tool-connections/{id}/activity` | **0** |
+  | Provider-side request log | **0** — the header key is recorded, the value never is |
+
+  The value was nonetheless delivered: the provider recorded the inbound
+  `x-api-key` header on the health check. Stored encrypted, projected at call
+  time, never echoed.
+
+## 4. Findings
+
+Reproducible product behaviour at commit `9335b7db10`, not advice. F1, F2 and
+F10 are the three that change what this skill is allowed to tell you; they are
+restated in
+[`deployment-support-matrix.md`](./deployment-support-matrix.md) so you meet
+them at the step where they bite. The rest are recorded here only.
+
+### F1 — a pasted MCP server can widen its own permissions after approval
+
+On every catalog refresh, newly discovered entries join the connection's active
+managed profile unless `quarantineNewEntries` is set — and for a **user-pasted
+URL** it is explicitly set to `false`, in the same expression that flags the
+connection `unverifiedServer: true`
+(`server/src/services/tool-access.ts:12721`). Curated gallery apps and
+Paperclip-managed cloud connectors *do* quarantine. The least-trusted class does
+not.
+
+**Executed, on the recommended wizard path.** A connection was finished with
+exactly two tools enabled (`list_notes` allowed, `create_note` ask-first) and
+access limited to one agent. The provider then advertised `export_notes`, which
+returns every note body in one response. After one `catalog/refresh`, with no
+operator action:
+
+```text
+policy test: granted agent → export_notes  ⇒  allow / allow_profile
+new-tools review queue: {"pendingCount": 0, "tools": []}
+```
+
+**Executed proof that risk tier does not gate it.** On a second connection, a
+deliberately partial profile — 3 of 4 tools allowed, the write removed — was
+refreshed after the provider advertised `delete_note`. Paperclip classified it
+correctly as `riskLevel: destructive, isDestructive: true`, and then allowed it:
+`allow / allow_profile`, `newToolsPendingCount: 0`.
+
+**Suggested fix.** Default `quarantineNewEntries: true` for
+`unverifiedServer: true` connections, so new entries land in the existing
+new-tools review queue instead of the allow list. The queue, the
+`newToolsPendingCount` field and the review endpoint all exist already; only the
+default is wrong. A narrower version: quarantine any new entry classified
+`write` or `destructive`, whatever the connection class.
+
+### F2 — the board "Test as agent" call runs with the board's authority
+
+`executeTestCall` builds a session with `actorType: "user"` and the board user's
+id, then asks the policy service to decide. Agent tool profiles bind to agents,
+so the board user is not subject to them.
+
+| Call | Expected from the agent's policy | Actual |
+| --- | --- | --- |
+| Test as the **ungranted** agent, `list_notes` | denied (`allowedToolNames: []`) | `HTTP 200 {"decision": "allowed"}`, read returned |
+| Test as the **granted** agent, `create_note`, before promotion | denied (`deny / deny_default`) | `HTTP 200 {"decision": "allowed"}` — **and the file was created on the provider** |
+
+The policy simulator, asked the same questions with `actorType: "agent"`, answers
+correctly.
+
+**Why it matters.** On the surface an operator reaches for first — a Test button
+labelled with an agent — an ungranted agent does execute, and a write the
+agent's own policy forbids lands on the provider. Validating a connector through
+Test alone records a false pass, in both directions.
+
+**Suggested fix.** Either evaluate the test call under the selected agent's
+policy and surface the board override explicitly, or relabel the surface "Test
+as board" and show the agent's own decision beside the result.
+
+### F10 — the connect wizard grants the policy but not the install
+
+`POST …/tools/apps/{id}/finish` with `access: {agentIds: [...]}` creates the
+tool-profile binding — the agent's `allowedToolNames` correctly lists the chosen
+tool — but creates **no `tool_connection_install` row**. Agent readiness
+(`usableConnectionForAgent`, `server/src/services/connection-intents.ts:241-248`)
+requires the connection to be both installed for the agent and permitted:
+
+```ts
+const usable = (connection) => connection
+  && installedIds.has(connection.id) && permittedIds.has(connection.id)
+  && connection.status === "active" && connection.enabled && …
+```
+
+With `installs: []` the first clause fails, so the tool never enters the agent's
+session. Both instances, both wizard-created connections:
+
+```text
+GET /api/tool-connections/{id}/installs        ⇒ {"installs": []}
+GET …/tools/profiles/effective/agents/{agent}  ⇒ allowedToolNames: ["read_wiki_structure"]
+                                                 installedConnections: []
+```
+
+On instance B this produced a genuinely confusing run: the agent found the
+connection, got `state: "needs_user_action", reason: "Review identity and access
+for this agent"`, raised a connection request, and correctly refused to fake the
+answer — while the board's own access view showed the tool allowed
+(`allowedCount: 1`) and the connection `active / enabled / ok`. Adding the
+install with `PUT …/installs`, the only change, made the same agent's next run
+succeed.
+
+**Suggested fix.** Have `finish` write the installs implied by its own `access`
+argument, or have the connection detail view show "allowed but not installed"
+instead of a green badge.
+
+### F3 — a provider that is down reports as a missing tool
+
+With the provider stopped, a gateway call returned
+`HTTP 404 {"error": "Tool \"list_notes\" not found", "reasonCode": "tool_not_found"}`.
+The same code comes back when the connection is merely disabled. Meanwhile
+`health-check` correctly reported `runtime_error / fetch failed`. Distinguish
+"upstream unreachable" from "tool absent from the catalog" in the gateway's
+reason code.
+
+### F4 — revocation is enforced, but the policy view still says allow
+
+After the organization grant was revoked, the policy simulator kept answering
+`allow / allow_profile` while the live call failed with
+`organization_authorization_required`. Enforcement is correct; the operator's
+"who can do what" view is not. Fold grant state into the simulated decision, or
+annotate the result with the credential-resolution outcome.
+
+### F5 — `PAPERCLIP_HOME` loses to an ancestor `.paperclip/config.json`
+
+`resolvePaperclipConfigPath` (`server/src/paths.ts`) walks up from
+`process.cwd()` looking for `.paperclip/config.json` and only then falls back to
+the home-derived path. Starting the server from inside a checkout with
+`PAPERCLIP_HOME=<isolated>` therefore loads the **checkout's** config and binds
+a different instance's database, while the banner reports the isolated paths.
+`PAPERCLIP_CONFIG=<path>` is the override that actually wins.
+
+This matters to anyone following `doc/DEVELOPING.md`, which recommends isolating
+with `PAPERCLIP_HOME` / `--data-dir`. Use `PAPERCLIP_CONFIG`, and check the bound
+database before you trust an "isolated" instance — a start against the wrong
+database can apply pending migrations to it.
+
+### F6 — a server started inside an agent's run hands its children that run's control-plane URL
+
+Starting an instance from inside an agent's shell carries that run's
+`PAPERCLIP_API_URL`, `PAPERCLIP_API_KEY`, `PAPERCLIP_RUN_ID` and friends. The
+agent the instance spawns inherits them, points at the *other* instance, and
+gets `401` on every call including its own MCP servers — a failure that reads
+exactly like a broken connector. Restarting under `env -i` with only the
+variables the instance needs fixes it. Troubleshooting rule: **if a self-hosted
+agent run 401s on everything, check which `PAPERCLIP_*` variables the server
+process inherited.**
+
+### F7 — `authenticated` + `public` has two hard prerequisites
+
+Both were found by hitting them:
+
+1. `StartupRefusalError: authenticated public deployments require DATABASE_URL or
+   config.database.connectionString; refusing embedded PostgreSQL fallback`. A
+   public self-hosted instance cannot use the embedded database at all.
+2. `auth.baseUrlMode must be explicit when deploymentMode=authenticated and
+   exposure=public`, plus `auth.publicBaseUrl` required
+   (`packages/shared/src/config-schema.ts:167-187`).
+
+Neither is a defect. Both are the first two walls a self-hoster hits, which is
+why they are in the shape-B row of the matrix.
+
+### F8 — first-admin claim, and what the public base URL silently controls
+
+**F8a.** `POST /api/bootstrap/claim` returns
+`404 Browser first-admin claim is not available` unless exposure is `private`,
+and a merely signed-up user is not an instance admin (`Instance admin required`
+on the first board mutation). The way through: claim the admin while the
+instance is `private`, then restart it as `public`.
+
+**F8b.** `auth.publicBaseUrl` is not only browser-facing — it is what the agent
+runtime gives spawned agents as their control-plane URL. With a placeholder
+hostname, a shape-B agent run fails with `getaddrinfo ENOTFOUND` on every MCP
+server and every API call. **On shape B it must resolve from the machine the
+runtime runs on, not just from the operator's browser.** Changing it also
+invalidates existing board sessions.
+
+### F9 — the gateway's MCP client sends `tools/list` with no `initialize`
+
+The provider's request log shows Paperclip posting `tools/list` directly, with no
+preceding `initialize` handshake and no `MCP-Protocol-Version` header. The
+disposable server is lenient and answered. A strict MCP server may reject an
+uninitialised session, which would present as a connector that health-checks
+green in testing and fails against a compliant provider. Not confirmed against a
+strict implementation — an observation, not a defect.
+
+## 5. What was not covered
+
+- **Cloud: nothing.** No Cloud instance was reachable. Every Cloud cell in the
+  matrix is `untested` or `unsupported`, and none of it is inferred from a
+  self-hosted result.
+- **OAuth on every shape.** Needs an authorized provider account.
+- **`local_stdio` execution.** The live `supportMatrix` differs between shapes A
+  and B, which is itself useful, but no stdio connection was created or run.
+- **Chat and email.** Not touched.
+- **Shape-B write execution and provider-side readback.** The shape-B provider is
+  read-only.
+- **Link-local denial.** Source-cited only; no live call.
+
+F1 and F2 need product decisions rather than more testing. Both reproduce from
+this document in under ten minutes on a fresh instance.
@@ -2,7 +2,7 @@
  "schemaVersion": 1,
  "packageName": "@paperclipai/skills-catalog",
  "packageVersion": "0.3.1",
-  "generatedAt": "2026-09-19T16:58:28.123Z",
+  "generatedAt": "2026-09-20T19:50:57.718Z",
  "skills": [
    {
      "id": "paperclipai:bundled:docs:doc-maintenance",
@@ -1165,11 +1165,11 @@
        {
          "path": "references/offline-verification.md",
          "kind": "reference",
-          "sizeBytes": 9519,
-          "sha256": "ca38bcddf376aa1bc2ea5f392032c37cda204af21a412ec4b6e984343087c2e2"
+          "sizeBytes": 16768,
+          "sha256": "296f08e6249687255162eb3afee40e51992ae1f5a86677594e63e95f91ebdaf2"
        }
      ],
-      "contentHash": "sha256:c7a267d390dee25ac65750b46f9aaad69993e08eb42c8133bf52a05701a6953a"
+      "contentHash": "sha256:26c08b6917c5bf80dc6b99b62f177746bfafed458905182782aa14652e0c3b84"
    },
    {
      "id": "paperclipai:optional:software-development:connect-agent-tools",
@@ -1202,8 +1202,8 @@
        {
          "path": "SKILL.md",
          "kind": "skill",
-          "sizeBytes": 26730,
-          "sha256": "7fa19dc00395a5b8214f1a725c259dd1aa97ce580a43d0f8b8a13d2586c83bbb"
+          "sizeBytes": 26975,
+          "sha256": "13ea45dccab92a5f7dfdd67baa5ffe2952987cbad7fa085cff8de5d0c5fc53ed"
        },
        {
          "path": "references/chat-and-email.md",
@@ -1214,11 +1214,17 @@
        {
          "path": "references/deployment-support-matrix.md",
          "kind": "reference",
-          "sizeBytes": 20738,
-          "sha256": "fa7b868c2c3997e5433ad4845738d7092dc2a7f621e420d7a28dbf1f29e7c2b5"
+          "sizeBytes": 21348,
+          "sha256": "7afb36d0e2baa4e3920d83332f539b332267ae63e41194dfcbfd145f48addb67"
+        },
+        {
+          "path": "references/verification-log.md",
+          "kind": "reference",
+          "sizeBytes": 25233,
+          "sha256": "56cd4452bd75ad646113c1f62926aa6f2b2f965db2d1e9adea350afe60c55b14"
        }
      ],
-      "contentHash": "sha256:ab06d5fcaa0ce39865e53d9ee45345cc07d72f4be51ce201d13b195e4f8872ba"
+      "contentHash": "sha256:15e41bb565bafee8b284fa21e32948a191e828f5956ee641ffe641d11804bd58"
    },
    {
      "id": "paperclipai:optional:software-development:prepare-mcp-integration",