Files
PaperClipAI/scripts/check-maps.mjs
T
DottaandPaperclip 7d9b93aee2 docs: establish paired feature and agent access maps (#16025)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Contributors use the feature map to understand and verify product
capabilities.
> - Agents need APIs and usable entry points within their permission
boundaries.
> - Product recipes and agent access need separate records with shared
capability IDs.
> - This pull request establishes that contract and gives maintenance
ownership of both maps.
> - The benefit is repeatable access review without changing product
behavior or permissions.

## Linked Issues or Issue Description

Refs: #15983, which added the portable verification skills and updated
the feature map. This is its contract-only follow-up. Related: #15288
added the original feature map.

**Issue type**

Missing documentation and maintenance contract.

**Where is the issue?**

The product map, its missing agent access companions, and the
verification skills.

**What's wrong?**

There is no shared convention for API presence, agent entry points,
permission boundaries, access gaps, or evidence. An API declaration does
not prove that a permitted native or remote agent can use it.

**Suggested fix**

Keep the product and agent access maps as siblings under `maps/`. Reuse
capability IDs. Record representative source inventory separately from
operation review and live proof. Keep both maps aligned through the
maintenance skill.

## What Changed

- Move the existing product map to `maps/feature-map/`. Preserve its
recipes and Poteto format. Update relative links and add companion
links.
- Add `maps/agent-access-map/` with 35 area files and 171 matching
capability IDs. Seed representative API operations, agent paths,
policies and known gaps from the reviewed source inventory. Mark entries
as `baseline`, `source-only`, and live verification `not-run`.
- Define the shared contract in `maps/README.md`. Track API presence,
agent availability, policy, evidence and review depth separately.
Require operation enumeration before marking a broad capability
operation-reviewed.
- Extend `maintain-verification-skill` to reconcile both maps and
inspect actual route policy and production tool admission. Update
creation, verification, notices and contributor references to use the
shared contract.
- Add `scripts/check-maps.mjs` and ten regression tests. Check matching
areas/IDs, indexes, references, contract fields, API method/path
examples, justified exceptions and evidence-state consistency.
- Keep the coverage inventory unchanged. Preserve skill invocation
policies and both MIT license blocks. No product API, permission, UI,
runtime helper, migration, dependency or workflow changes.

## Verification

- Passed: `node scripts/check-maps.mjs` reports 35 paired areas and 171
matching capability IDs. All local map references resolve.
- Passed: `node --test scripts/check-maps.test.mjs` runs ten tests.
Fixtures cover missing companions, renamed/orphan/duplicate IDs, broken
references, missing operation fields, empty exceptions, missing API
examples, empty policy fields, and verified evidence without passed
verification.
- Passed: metadata validation for all three verification skills. Skill
and contributor links resolve. Invocation policies and both license
blocks are unchanged.
- Compared relocated recipes with current master. Contents differ only
in relative links and companion links. `coverage.json` is byte-for-byte
unchanged. Master's updated read-only interaction test reference is
retained.
- Passed: `git diff --check`. The branch starts from current master and
contains only this contract change.
- Repository-wide typecheck, tests and build were not run locally for
this documentation and Node structural-check scope. Live agent access
was not exercised. Those gaps are explicit in the baseline entries; the
structural check does not certify behavior. CI and automated review are
pending.

## Risks

- The companions are representative source baselines. They are not
complete operation inventories or passed agent journeys.
- The checker cannot establish actual permissions, native tool
admission, operation completeness or truthful live evidence. Maintenance
must inspect source and verify with the actual permitted agent context.
- External links to the old product-map directory may need updates.
Repository references have been updated.
- Existing access gaps and board/user restrictions remain recorded. This
contract does not change authority or implement missing paths.

## Model Used

OpenAI GPT-6 through Codex assisted with this change. The session does
not expose the exact serving model ID, configured context-window size,
or reasoning setting. Capabilities used: source analysis, file edits,
shell commands, and structural regression tests.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-12 14:52:28 -05:00

132 lines
6.7 KiB
JavaScript

#!/usr/bin/env node
import { existsSync, readFileSync, readdirSync } from "node:fs";
import path from "node:path";
import { fileURLToPath } from "node:url";
const scriptPath = fileURLToPath(import.meta.url);
const defaultRoot = path.resolve(path.dirname(scriptPath), "..");
const fields = ["API", "Agent access", "Policy", "Evidence", "Verification", "Gaps"];
const values = {
Review: ["baseline", "operation-reviewed"],
API: ["present", "missing", "not-applicable", "unreviewed"],
"Agent access": ["available", "partial", "missing", "restricted", "operator-only", "not-applicable", "unreviewed"],
Evidence: ["source-only", "verified", "failed", "blocked", "not-run"],
Verification: ["passed", "failed", "blocked", "not-run"],
};
function withoutExamples(text) {
return text.replace(/^(`{3,}|~{3,})[^\n]*\n[\s\S]*?^\1\s*$/gm, "");
}
function sections(text, level) {
const pattern = new RegExp("^" + "#".repeat(level) + " `([^`]+)`[^\\n]*$", "gm");
const matches = [...text.matchAll(pattern)];
return matches.map((match, index) => ({
id: match[1],
body: text.slice(match.index + match[0].length, matches[index + 1]?.index ?? text.length),
}));
}
export function checkMaps(repoRoot = defaultRoot) {
const errors = [];
const featureRoot = path.join(repoRoot, "maps/feature-map");
const agentRoot = path.join(repoRoot, "maps/agent-access-map");
const fail = (file, message) => errors.push(`${path.relative(repoRoot, file)}: ${message}`);
function read(file) {
if (!existsSync(file)) { fail(file, "missing file"); return ""; }
return withoutExamples(readFileSync(file, "utf8"));
}
function files(dir) {
if (!existsSync(dir)) { fail(dir, "missing map directory"); return []; }
return readdirSync(dir).filter((name) => name.endsWith(".md") && name !== "README.md").sort();
}
function unique(ids, file, label) {
const seen = new Set();
for (const id of ids) {
if (seen.has(id)) fail(file, `duplicate ${label}: ${id}`);
seen.add(id);
}
}
function links(file, text) {
for (const match of text.matchAll(/\]\(([^)\n]+)\)/g)) {
const target = match[1].replace(/^<|>$/g, "");
if (/^(?:[a-z][a-z\d+.-]*:|#)/i.test(target)) continue;
if (target.startsWith("/")) { fail(file, `use a relative map reference: ${target}`); continue; }
let local;
try { local = decodeURIComponent(target.split(/[?#]/, 1)[0]); }
catch { fail(file, `invalid local reference: ${target}`); continue; }
if (!existsSync(path.resolve(path.dirname(file), local))) fail(file, `broken local reference: ${target}`);
}
}
function field(file, body, name) {
const escaped = name.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
const match = body.match(new RegExp(`^${escaped}:[ \\t]*(\\S[^\\n]*)$`, "m"));
if (!match) { fail(file, `missing ${name} field`); return null; }
if (!values[name]) return null;
const state = match[1].match(/^`([^`]+)`/);
if (!state || !values[name].includes(state[1])) { fail(file, `invalid ${name} state`); return null; }
if (["not-applicable", "restricted", "operator-only"].includes(state[1]) && !match[1].slice(state[0].length).replace(/[\s—–:.-]/g, "")) {
fail(file, `${name} ${state[1]} needs a reason or path`);
}
return state[1];
}
function accessRecord(file, body) {
const states = Object.fromEntries(fields.map((name) => [name, field(file, body, name)]));
if (states.Evidence === "verified" && states.Verification !== "passed") {
fail(file, "verified evidence requires a passed verification record");
}
const api = body.split(/^API:/m)[1]?.split(/^(?:Agent access|Policy|Evidence|Verification|Gaps):/m)[0] ?? "";
if (states.API === "present" && !/\b(?:GET|POST|PUT|PATCH|DELETE|HEAD|OPTIONS) \/[^\s`]+/.test(api)) {
fail(file, "present API needs an exact method/path example");
}
}
const productFiles = files(featureRoot);
const agentFiles = files(agentRoot);
const indexes = [read(path.join(featureRoot, "README.md")), read(path.join(agentRoot, "README.md"))];
let capabilities = 0;
for (const name of new Set([...productFiles, ...agentFiles])) {
const productFile = path.join(featureRoot, name);
const agentFile = path.join(agentRoot, name);
const product = read(productFile);
const agent = read(agentFile);
const subfeatures = product.split(/^## Sub-features\s*$/m)[1]?.split(/^## /m)[0] ?? "";
const ids = [...subfeatures.matchAll(/^- `([^`]+)`:/gm)].map((match) => match[1]);
if (!ids.length) fail(productFile, "missing stable sub-feature IDs");
unique(ids, productFile, "capability");
const entries = sections(agent, 2);
unique(entries.map((entry) => entry.id), agentFile, "capability");
for (const id of ids) if (!entries.some((entry) => entry.id === id)) fail(agentFile, `missing capability: ${id}`);
for (const entry of entries) {
if (!ids.includes(entry.id)) fail(agentFile, `orphan capability: ${entry.id}`);
const review = field(agentFile, entry.body, "Review");
if (review === "operation-reviewed") {
const operations = sections(entry.body, 3);
if (!operations.length) fail(agentFile, `operation-reviewed capability ${entry.id} has no operations`);
unique(operations.map((operation) => operation.id), agentFile, "operation");
for (const operation of operations) accessRecord(agentFile, operation.body);
} else accessRecord(agentFile, entry.body);
}
if (!agent.includes(`](../feature-map/${name})`)) fail(agentFile, "missing matching product link");
if (!product.includes(`](../agent-access-map/${name})`)) fail(productFile, "missing matching companion link");
for (const [index, text] of indexes.entries()) {
if (!new RegExp(`\\]\\((?:\\./)?${name.replaceAll(".", "\\.")}\\)`).test(text)) {
fail(path.join(index === 0 ? featureRoot : agentRoot, "README.md"), `missing area: ${name}`);
}
}
if (!/^Review date: \d{4}-\d{2}-\d{2}\. Source commit: `[a-f\d]{40}`\./m.test(agent)) fail(agentFile, "missing review date/source commit");
links(productFile, product);
links(agentFile, agent);
capabilities += ids.length;
}
for (const file of [path.join(repoRoot, "maps/README.md"), path.join(featureRoot, "README.md"), path.join(agentRoot, "README.md")]) links(file, read(file));
return { errors, areas: productFiles.length, capabilities };
}
if (process.argv[1] && path.resolve(process.argv[1]) === scriptPath) {
const result = checkMaps();
for (const error of result.errors) console.error(error);
if (result.errors.length) process.exitCode = 1;
else console.log(`Maps aligned: ${result.areas} areas, ${result.capabilities} capability IDs; references and contract fields valid. Structural checks only.`);
}