Files
PaperClipAI/scripts/check-maps.test.mjs
T
DottaandPaperclip 7d9b93aee2 docs: establish paired feature and agent access maps (#16025)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Contributors use the feature map to understand and verify product
capabilities.
> - Agents need APIs and usable entry points within their permission
boundaries.
> - Product recipes and agent access need separate records with shared
capability IDs.
> - This pull request establishes that contract and gives maintenance
ownership of both maps.
> - The benefit is repeatable access review without changing product
behavior or permissions.

## Linked Issues or Issue Description

Refs: #15983, which added the portable verification skills and updated
the feature map. This is its contract-only follow-up. Related: #15288
added the original feature map.

**Issue type**

Missing documentation and maintenance contract.

**Where is the issue?**

The product map, its missing agent access companions, and the
verification skills.

**What's wrong?**

There is no shared convention for API presence, agent entry points,
permission boundaries, access gaps, or evidence. An API declaration does
not prove that a permitted native or remote agent can use it.

**Suggested fix**

Keep the product and agent access maps as siblings under `maps/`. Reuse
capability IDs. Record representative source inventory separately from
operation review and live proof. Keep both maps aligned through the
maintenance skill.

## What Changed

- Move the existing product map to `maps/feature-map/`. Preserve its
recipes and Poteto format. Update relative links and add companion
links.
- Add `maps/agent-access-map/` with 35 area files and 171 matching
capability IDs. Seed representative API operations, agent paths,
policies and known gaps from the reviewed source inventory. Mark entries
as `baseline`, `source-only`, and live verification `not-run`.
- Define the shared contract in `maps/README.md`. Track API presence,
agent availability, policy, evidence and review depth separately.
Require operation enumeration before marking a broad capability
operation-reviewed.
- Extend `maintain-verification-skill` to reconcile both maps and
inspect actual route policy and production tool admission. Update
creation, verification, notices and contributor references to use the
shared contract.
- Add `scripts/check-maps.mjs` and ten regression tests. Check matching
areas/IDs, indexes, references, contract fields, API method/path
examples, justified exceptions and evidence-state consistency.
- Keep the coverage inventory unchanged. Preserve skill invocation
policies and both MIT license blocks. No product API, permission, UI,
runtime helper, migration, dependency or workflow changes.

## Verification

- Passed: `node scripts/check-maps.mjs` reports 35 paired areas and 171
matching capability IDs. All local map references resolve.
- Passed: `node --test scripts/check-maps.test.mjs` runs ten tests.
Fixtures cover missing companions, renamed/orphan/duplicate IDs, broken
references, missing operation fields, empty exceptions, missing API
examples, empty policy fields, and verified evidence without passed
verification.
- Passed: metadata validation for all three verification skills. Skill
and contributor links resolve. Invocation policies and both license
blocks are unchanged.
- Compared relocated recipes with current master. Contents differ only
in relative links and companion links. `coverage.json` is byte-for-byte
unchanged. Master's updated read-only interaction test reference is
retained.
- Passed: `git diff --check`. The branch starts from current master and
contains only this contract change.
- Repository-wide typecheck, tests and build were not run locally for
this documentation and Node structural-check scope. Live agent access
was not exercised. Those gaps are explicit in the baseline entries; the
structural check does not certify behavior. CI and automated review are
pending.

## Risks

- The companions are representative source baselines. They are not
complete operation inventories or passed agent journeys.
- The checker cannot establish actual permissions, native tool
admission, operation completeness or truthful live evidence. Maintenance
must inspect source and verify with the actual permitted agent context.
- External links to the old product-map directory may need updates.
Repository references have been updated.
- Existing access gaps and board/user restrictions remain recorded. This
contract does not change authority or implement missing paths.

## Model Used

OpenAI GPT-6 through Codex assisted with this change. The session does
not expose the exact serving model ID, configured context-window size,
or reasoning setting. Capabilities used: source analysis, file edits,
shell commands, and structural regression tests.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-12 14:52:28 -05:00

90 lines
4.7 KiB
JavaScript

import assert from "node:assert/strict";
import { mkdtempSync, mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs";
import os from "node:os";
import path from "node:path";
import test from "node:test";
import { checkMaps } from "./check-maps.mjs";
const record = `API: \`present\` — POST /api/example.
Agent access: \`partial\` — agent REST; native binding missing.
Policy: same company and calling owner.
Evidence: \`source-only\` — implementation inspected.
Verification: \`not-run\` — assert permitted owner and denied other company.
Gaps: native mutation path.
`;
function fixture(t) {
const root = mkdtempSync(path.join(os.tmpdir(), "paperclip-map-contract-"));
t.after(() => rmSync(root, { recursive: true, force: true }));
for (const dir of ["maps/feature-map", "maps/agent-access-map"]) mkdirSync(path.join(root, dir), { recursive: true });
const product = path.join(root, "maps/feature-map/example.md");
const agent = path.join(root, "maps/agent-access-map/example.md");
writeFileSync(path.join(root, "maps/README.md"), "# Contract\n");
for (const dir of ["feature-map", "agent-access-map"]) writeFileSync(path.join(root, "maps", dir, "README.md"), "[Example](example.md)\n");
writeFileSync(product, "# Example\n\n[Agent access](../agent-access-map/example.md)\n\n## Sub-features\n\n- `create`: create a record.\n\n## How to get to it (user POV)\n");
writeFileSync(agent, `# Agent access\n\nProduct map: [Example](../feature-map/example.md).\n\nReview date: 2026-10-12. Source commit: \`${"a".repeat(40)}\`.\n\n## \`create\`\n\nReview: \`baseline\`.\n\n${record}`);
return { root, product, agent };
}
test("paired source-only records pass without being promoted to live proof", (t) => {
const { root } = fixture(t);
assert.deepEqual(checkMaps(root), { errors: [], areas: 1, capabilities: 1 });
});
test("a product area cannot lose its companion", (t) => {
const { root, agent } = fixture(t);
rmSync(agent);
assert.ok(checkMaps(root).errors.some((error) => error.includes("missing file")));
});
test("renaming a capability on one side reports missing and orphan IDs", (t) => {
const { root, agent } = fixture(t);
writeFileSync(agent, readFileSync(agent, "utf8").replace("## `create`", "## `update`"));
const result = checkMaps(root);
assert.ok(result.errors.some((error) => error.includes("missing capability: create")));
assert.ok(result.errors.some((error) => error.includes("orphan capability: update")));
});
test("duplicate IDs cannot disguise the missing capability", (t) => {
const { root, agent } = fixture(t);
writeFileSync(agent, readFileSync(agent, "utf8") + `\n## \`create\`\n\nReview: \`baseline\`.\n\n${record}`);
assert.ok(checkMaps(root).errors.some((error) => error.includes("duplicate capability: create")));
});
test("a moved or deleted source reference is reported", (t) => {
const { root, agent } = fixture(t);
writeFileSync(agent, readFileSync(agent, "utf8") + "\n[Source](../../server/deleted.ts#L12)\n");
assert.ok(checkMaps(root).errors.some((error) => error.includes("broken local reference")));
});
test("every reviewed operation needs its own access fields", (t) => {
const { root, agent } = fixture(t);
const text = readFileSync(agent, "utf8").replace("Review: `baseline`.", "Review: `operation-reviewed`.\n\n### `create`");
writeFileSync(agent, text + `\n### \`update\`\n\n${record.replace(/^API:.*\n/m, "")}`);
assert.ok(checkMaps(root).errors.some((error) => error.includes("missing API field")));
});
test("verified evidence cannot coexist with verification not run", (t) => {
const { root, agent } = fixture(t);
writeFileSync(agent, readFileSync(agent, "utf8").replace("Evidence: `source-only`", "Evidence: `verified`"));
assert.ok(checkMaps(root).errors.some((error) => error.includes("verified evidence requires a passed verification record")));
});
test("an exception needs its reason rather than an empty exemption label", (t) => {
const { root, agent } = fixture(t);
writeFileSync(agent, readFileSync(agent, "utf8").replace(/^API:.*$/m, "API: `not-applicable`."));
assert.ok(checkMaps(root).errors.some((error) => error.includes("not-applicable needs a reason")));
});
test("API presence needs a concrete method and path", (t) => {
const { root, agent } = fixture(t);
writeFileSync(agent, readFileSync(agent, "utf8").replace("POST /api/example", "see the backend"));
assert.ok(checkMaps(root).errors.some((error) => error.includes("present API needs an exact method/path")));
});
test("an empty policy cannot borrow the next field's content", (t) => {
const { root, agent } = fixture(t);
writeFileSync(agent, readFileSync(agent, "utf8").replace(/^Policy:.*$/m, "Policy:\n"));
assert.ok(checkMaps(root).errors.some((error) => error.includes("missing Policy field")));
});