fix(skills-catalog): the corpus is not a gate, and three smaller findings

Builds on c46092f202, which landed the discovery-URL guard and the citation and visibility work in parallel. These are the Enterpret acceptance findings that commit did not cover, plus one claim of its own that turned out wrong. - **The ingestion corpus is not a gate on authoring, and the skill said it was.** Both files told an external contributor that without the non-public `paperclip-content` corpus "this whole path is closed to you". Executed at `066a4e8019`: `node scripts/ingest-app-definitions.mjs --definitions-only`, with no corpus present and `PAPERCLIP_CONTENT_TEMPLATES` unset, reproduced all 72 checked-in `app-definitions/<slug>.json` files and `app-definitions.generated.ts` byte-for-byte — `git status` clean. Adding a throwaway provider tuple and re-running emitted 73 and changed exactly the new `<slug>.json` and the positional registry. The flag costs only `app-definitions.ingestion-report.json`, which is built from captures and so is correctly unchanged by a provider that has none. This is the single largest barrier the acceptance test found for a contributor outside Paperclip, and it was not real. Also records that a missing default corpus path raises a raw `ENOENT` from `fs.readdirSync` before the documented `Expected 99 captures` guard can run. - **Executing the unlisted path.** c46092f202 documents the three visibility states, which is the better framing, but not the mechanics of taking the second one — the finding was that the reference does not enumerate them. Now it does: both slug sets, the sorted literal in the test, and `catalogVisible: false` on the manifest row, plus the instruction *not* to bump the store count, which is the opposite of what the store-visible path says. Checked against `18dac1e1`: all 20 hidden slugs already have a manifest row, all carry `catalogVisible: false`, and the 56 rows marked visible are exactly `APP_STORE_DEFINITIONS`. - **Asserting absence, with a canary that works.** The obvious credential canary fires on all 72 shipped definitions, because `keyPlacement` legitimately carries `"prefix": "Bearer "`. A value-shaped pattern fires on 0 of 72 and still matches `sk-…`, `ghp_…` and `xoxb-…`. Both measured; the worked example is in the skill. - **`method()`'s `ownershipModes` default is a silent capability claim.** Using the helper asserts both `customer` and `dcr`. Say which one you observed advertised, or pass the set explicitly. - The offline harness now says when it is not worth building: it protects a checkout other people depend on, and an author in their own disposable worktree does not have one. Regenerated the catalog manifest: 19 skills, validate clean, 19/20 package tests pass. The one failure is `packaged-artifacts.test.ts`, which shells out to `pnpm --filter … build`; it fails identically at unmodified HEAD in this worktree because the worktree has no pnpm install of its own. Seventeen packaged relative links, none broken, no private references. Both packages stay markdown_only. Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-06 21:05:21 +02:00 · 2026-09-24 21:14:17 +00:00
1 parent c46092f202
commit eb32c6cc91
@@ -105,11 +105,13 @@ Collect these before step 1. Ask only for what you cannot determine safely.
   without a live provider proof belongs in the third state, not the second.
   See the visibility chain in `references/catalog-contract.md`.
 5. **Location of the ingestion corpus** (`paperclip-content`), or the value for
-   `PAPERCLIP_CONTENT_TEMPLATES`. This corpus is in a non-public repository; the
-   generator refuses to run without it. If you do not have it, this skill's
-   path is closed to you — say so plainly rather than editing the guard, and
-   note that connecting the server directly (`connect-agent-tools` path 2) is a
-   different thing, not a lesser substitute.
+   `PAPERCLIP_CONTENT_TEMPLATES`, *or* a decision to run the generator with
+   `--definitions-only`. The corpus is in a non-public repository, but it is
+   not a gate on authoring: `--definitions-only` skips it and still emits every
+   definition and the positional registry, verified byte-for-byte. What it does
+   not write is `app-definitions.ingestion-report.json`, which a provider with
+   no capture of its own does not change anyway. See **Generator preconditions**
+   in `references/catalog-contract.md`. Never edit the guard.
 6. **The deployment you will validate on.** An isolated self-hosted instance is
   the expected answer and is sufficient.

@@ -136,7 +138,8 @@ reviewable claim and must come from the provider's own metadata or documentation
  script, a definition, connection config, application metadata, a committed
  fixture, a comment, a screenshot, a trace, a HAR file, or a report. If you are
  handed a credential, propose it as a Paperclip secret immediately and never
-  echo it. Assert absence in a test rather than merely not printing it.
+  echo it. Assert absence in a test rather than merely not printing it —
+  see **Asserting absence** below for the canary that does not false-positive.
 - **No external writes during research.** Unauthenticated metadata reads are
  allowed. Dynamic client registration, consent, account mutation, and any write
  tool call are not research — they need explicit authorization.
@@ -147,6 +150,47 @@ reviewable claim and must come from the provider's own metadata or documentation
  skill ends at a verified local change.
 - **No roadmap claims.** Do not write "coming soon" or promise a future method.

+### Asserting absence
+
+"Assert absence in a test" is easy to get wrong in one specific way, so here is
+the shape that works.
+
+The obvious canary — grep the definition for `/bearer|token|secret|key/i` —
+fails on the checked-in catalog before you have added anything. `keyPlacement`
+legitimately carries `"prefix": "Bearer "`, and at `18dac1e1` the shipped
+definitions carry three such values (`"Bearer "`, `"Basic "`, `"Token token="`).
+A canary that fires on a clean tree gets deleted within the week, which leaves
+you with no canary at all.
+
+Assert on **values and shapes**, not on the words that describe them:
+
+```ts
+it("ships no credential value for acme", () => {
+  const definition = APP_DEFINITIONS.find((app) => app.slug === "acme")!;
+  const fields = definition.methods.flatMap((m) => m.credentialFields ?? []);
+
+  // A credential field declares how to collect a secret. It never carries one.
+  for (const field of fields) {
+    expect(field).not.toHaveProperty("value");
+    expect(field).not.toHaveProperty("default");
+  }
+
+  // Value-shaped canary over the whole serialized definition. `"Bearer "` is a
+  // scheme name and does not match; a real token does.
+  expect(JSON.stringify(definition)).not.toMatch(
+    /\b(?:sk|pk|rk)-[A-Za-z0-9]{16,}|\bgh[pousr]_[A-Za-z0-9]{20,}|\bxox[baprs]-[A-Za-z0-9-]{10,}/,
+  );
+});
+```
+
+Executed over the 72 checked-in definitions: the naive pattern fires on
+**72 of 72**, the value-shaped one on **0 of 72**, and it still matches
+`sk-…`, `ghp_…` and `xoxb-…` test strings.
+
+Run the same value-shaped canary over every artifact you produce, not only the
+definition: your report, any fixture, and any captured output. A credential that
+never reached the definition but did reach the report is the same incident.
+
 ## Step 1 — Discover The Current Contract

 Read from the recorded commit, not from memory and not from a stale worktree:
@@ -349,7 +393,9 @@ Stop and report rather than working around any of these:

 - An applicable research gate has not been taken, or does not cover this
  provider.
- The ingestion corpus is unavailable, so the generator cannot run.
+- The change needs `app-definitions.ingestion-report.json` refreshed and the
+  ingestion corpus is unavailable. Authoring a definition does not — use
+  `--definitions-only`.
 - The provider has no compatible MCP server, or needs a transport with no execution
  path.
 - The only viable method is Cloud-gated (`platform_shared` Paperclip-managed
@@ -53,14 +53,34 @@ change the next run reverts.

 ## Generator preconditions

- **Corpus.** `scripts/ingest-app-definitions.mjs` resolves
-  `PAPERCLIP_CONTENT_TEMPLATES`, defaulting to
+- **Corpus, and how to author without it.** `scripts/ingest-app-definitions.mjs`
+  resolves `PAPERCLIP_CONTENT_TEMPLATES`, defaulting to
  `../../paperclip-content/research/connections/vercel/templates`. It throws
  `Expected 99 captures, found N` unless exactly 99 `.md` files (excluding
-  `INDEX.md`) are present. A new provider needs no capture of its own; the
-  corpus still has to be complete. The corpus lives in the non-public
-  `paperclip-content` repository: if you do not have it, this whole path is
-  closed to you and you report that rather than stubbing the guard out.
+  `INDEX.md`) are present. A new provider needs no capture of its own. That
+  corpus is in the non-public `paperclip-content` repository.
+
+  Two corrections to what that implies, both executed at `066a4e8019`:
+
+  - **When the default path does not exist you get a raw `ENOENT` out of
+    `fs.readdirSync`, not the `Expected 99 captures` guard.** The guard only
+    runs once the directory has been read. Do not read that `ENOENT` as a
+    broken checkout; it is the missing corpus.
+  - **`--definitions-only` skips the corpus entirely, and it is enough to
+    author a definition.** `node scripts/ingest-app-definitions.mjs
+    --definitions-only`, with no corpus present and `PAPERCLIP_CONTENT_TEMPLATES`
+    unset, reproduced all 72 checked-in `app-definitions/<slug>.json` files and
+    `app-definitions.generated.ts` byte-for-byte — `git status` was clean
+    afterwards. Adding one throwaway provider tuple and re-running emitted 73
+    definitions and changed exactly the new `<slug>.json` plus the positional
+    registry.
+
+  What the flag costs you is `app-definitions.ingestion-report.json`, which it
+  does not write. The report is built from corpus captures, so a provider with
+  no capture of its own contributes nothing to it and it is correctly left
+  unchanged — in the run above it stayed clean. If your change *does* need the
+  report refreshed, you need the corpus, and you say so rather than stubbing
+  the guard out.
 - **Branding.** `brandingFor` throws
  `<slug>: missing local branding provenance` unless the slug has a manifest row
  (only `oauth-generic` and `api-key-generic` are exempt). Branding precedes
@@ -82,6 +102,15 @@ change the next run reverts.
 - `method(key, transport, auth, defaults, riskTier, guidanceMd, extra)` fills
  `ownershipModes` as `["customer", "dcr"]` for `oauth` and `["customer"]`
  otherwise, plus a default `whenToUse`.
+
+  **That default is a capability claim, and it is silent.** Using the helper
+  asserts both that the provider accepts an operator-registered client
+  (`customer`) and that Paperclip may auto-register one (`dcr`). Neither is
+  free: omit `dcr` for a provider Paperclip must not auto-register, and omit
+  `customer` for a provider with no way for an operator to register a client of
+  their own. Pass `ownershipModes` explicitly in `extra` whenever your probe did
+  not establish both, and say in your report which of the two you observed
+  advertised rather than inherited from the helper.
 - `featured` is a hard-coded six-slug list in the mapper. Do not add to it as
  part of authoring a new provider.

@@ -129,6 +158,33 @@ connection intents (`server/src/services/connection-intents.ts`), and rendered
 as a disabled card with the reason in `ConnectionSetupFlow.tsx` and
 `Browse.tsx`.

+### Executing the unlisted path
+
+The second state is the *default* outcome of offline authoring, not an
+exception, so here are its mechanics in full. Three edits, all required, and
+one thing not to do:
+
+1. **`packages/shared/src/app-definitions.ts`** — add the slug to
+   `CONNECTABLE_APP_SLUGS` *and* to `APP_STORE_HIDDEN_SLUGS`. Missing the first
+   makes the definition unreachable; missing the second makes it listed.
+2. **`packages/shared/src/app-definitions.test.ts`** — add the slug to the exact
+   sorted list in the `APP_STORE_HIDDEN_SLUGS` assertion. It is `toEqual`
+   against a literal array and it is sorted, so insert it in position rather
+   than appending.
+3. **`ui/public/brands/apps/manifest.json`** — the branding row still has to
+   exist, because `brandingFor` throws without it, but set
+   `"catalogVisible": false`. The manifest-parity assertion compares the
+   `catalogVisible` set against the *store-visible* set, so an unlisted slug
+   left `catalogVisible: true` fails it.
+4. **Do not touch the store count.** `APP_STORE_DEFINITIONS` excludes hidden
+   slugs, so its length does not change. Bumping it — the instruction the
+   store-visible path gives — fails the test by one in the other direction.
+
+Checked against `18dac1e1`: all 20 slugs in `APP_STORE_HIDDEN_SLUGS` have a
+manifest row, every one carries `catalogVisible: false`, and the 56 rows with
+`catalogVisible: true` are exactly `APP_STORE_DEFINITIONS`. The four steps above
+are what the existing unlisted providers already do.
+
 ## Assertions with exact counts or sets

 These fail on any addition. Update them deliberately.
@@ -6,6 +6,13 @@ you have permission to write to the repository, when another task owns the
 working tree, or when you want to prove a failure is attributable to your change
 and not to the environment.

+**Check whether you need it first.** If you are already working in a disposable
+worktree you created and own — the normal shape — the harness buys you little:
+what it protects is a checkout other people depend on, and you do not have one.
+Run the ladder in `references/catalog-contract.md` directly and `git status` at
+the end. Reach for the harness when the checkout is shared, is someone else's,
+or has to be provably untouched.
+
 The recipe below builds it. It exports the needed files with `git archive` and
 `git show` at a pinned commit and symlinks `node_modules` read-only, so the
 checkout is untouched.
@@ -2,7 +2,7 @@
  "schemaVersion": 1,
  "packageName": "@paperclipai/skills-catalog",
  "packageVersion": "0.3.1",
-  "generatedAt": "2026-09-24T21:08:13.357Z",
+  "generatedAt": "2026-09-24T21:13:47.041Z",
  "skills": [
    {
      "id": "paperclipai:bundled:docs:doc-maintenance",
@@ -1147,14 +1147,14 @@
        {
          "path": "SKILL.md",
          "kind": "skill",
-          "sizeBytes": 23654,
-          "sha256": "2b8ed818bc969425f6ebb7de0b70893b69ba6cbeae55d4a75660b4ae82046302"
+          "sizeBytes": 25787,
+          "sha256": "47ddc79f4dc46b60aac3b399c2e486e994b2587dffdb6d160a58f8a0bc8cc5aa"
        },
        {
          "path": "references/catalog-contract.md",
          "kind": "reference",
-          "sizeBytes": 15776,
-          "sha256": "e4a7f973ac2278bc23f86bf74e0199574d3341e91fe092a0fa06494a736ce930"
+          "sizeBytes": 19071,
+          "sha256": "f649eb72a31f843d566e1dbafa21b4930a6ce1bc4fee167403d51f2cfb310882"
        },
        {
          "path": "references/live-acceptance.md",
@@ -1165,8 +1165,8 @@
        {
          "path": "references/offline-verification.md",
          "kind": "reference",
-          "sizeBytes": 18480,
-          "sha256": "f887188302dab5c3ea947a29fb21ebfe4281e57cf83c02bcfbfb42c4f2711500"
+          "sizeBytes": 18916,
+          "sha256": "a0843a9df5d1869f6e059c06c38fc63aded9e6d54f43aad83b5114b498581be6"
        },
        {
          "path": "references/safe-discovery.md",
@@ -1175,7 +1175,7 @@
          "sha256": "d65e3dbfee183e46923af1ac3b088dc589277551749d45cd8e4867db50c0d687"
        }
      ],
-      "contentHash": "sha256:853e1f442eb14f95e72d312f305c657cf235c4cd501455ca0846fb58a043a889"
+      "contentHash": "sha256:eda0526e6cf158cd5a66d9bf678b7318cc4cee42bab4d62dc5758d27c120ba97"
    },
    {
      "id": "paperclipai:optional:software-development:connect-agent-tools",