mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-04 11:01:16 +01:00
docs: establish paired feature and agent access maps (#16025)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Contributors use the feature map to understand and verify product capabilities. > - Agents need APIs and usable entry points within their permission boundaries. > - Product recipes and agent access need separate records with shared capability IDs. > - This pull request establishes that contract and gives maintenance ownership of both maps. > - The benefit is repeatable access review without changing product behavior or permissions. ## Linked Issues or Issue Description Refs: #15983, which added the portable verification skills and updated the feature map. This is its contract-only follow-up. Related: #15288 added the original feature map. **Issue type** Missing documentation and maintenance contract. **Where is the issue?** The product map, its missing agent access companions, and the verification skills. **What's wrong?** There is no shared convention for API presence, agent entry points, permission boundaries, access gaps, or evidence. An API declaration does not prove that a permitted native or remote agent can use it. **Suggested fix** Keep the product and agent access maps as siblings under `maps/`. Reuse capability IDs. Record representative source inventory separately from operation review and live proof. Keep both maps aligned through the maintenance skill. ## What Changed - Move the existing product map to `maps/feature-map/`. Preserve its recipes and Poteto format. Update relative links and add companion links. - Add `maps/agent-access-map/` with 35 area files and 171 matching capability IDs. Seed representative API operations, agent paths, policies and known gaps from the reviewed source inventory. Mark entries as `baseline`, `source-only`, and live verification `not-run`. - Define the shared contract in `maps/README.md`. Track API presence, agent availability, policy, evidence and review depth separately. Require operation enumeration before marking a broad capability operation-reviewed. - Extend `maintain-verification-skill` to reconcile both maps and inspect actual route policy and production tool admission. Update creation, verification, notices and contributor references to use the shared contract. - Add `scripts/check-maps.mjs` and ten regression tests. Check matching areas/IDs, indexes, references, contract fields, API method/path examples, justified exceptions and evidence-state consistency. - Keep the coverage inventory unchanged. Preserve skill invocation policies and both MIT license blocks. No product API, permission, UI, runtime helper, migration, dependency or workflow changes. ## Verification - Passed: `node scripts/check-maps.mjs` reports 35 paired areas and 171 matching capability IDs. All local map references resolve. - Passed: `node --test scripts/check-maps.test.mjs` runs ten tests. Fixtures cover missing companions, renamed/orphan/duplicate IDs, broken references, missing operation fields, empty exceptions, missing API examples, empty policy fields, and verified evidence without passed verification. - Passed: metadata validation for all three verification skills. Skill and contributor links resolve. Invocation policies and both license blocks are unchanged. - Compared relocated recipes with current master. Contents differ only in relative links and companion links. `coverage.json` is byte-for-byte unchanged. Master's updated read-only interaction test reference is retained. - Passed: `git diff --check`. The branch starts from current master and contains only this contract change. - Repository-wide typecheck, tests and build were not run locally for this documentation and Node structural-check scope. Live agent access was not exercised. Those gaps are explicit in the baseline entries; the structural check does not certify behavior. CI and automated review are pending. ## Risks - The companions are representative source baselines. They are not complete operation inventories or passed agent journeys. - The checker cannot establish actual permissions, native tool admission, operation completeness or truthful live evidence. Maintenance must inspect source and verify with the actual permitted agent context. - External links to the old product-map directory may need updates. Repository references have been updated. - Existing access gaps and board/user restrictions remain recorded. This contract does not change authority or implement missing paths. ## Model Used OpenAI GPT-6 through Codex assisted with this change. The session does not expose the exact serving model ID, configured context-window size, or reasoning setting. Capabilities used: source analysis, file edits, shell commands, and structural regression tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed
1 parent
4b766b8729
commit
7d9b93aee2
83 files changed
+5425
-186
No files matched your search
@@ -0,0 +1,131 @@
|
||||
#!/usr/bin/env node
|
||||
import { existsSync, readFileSync, readdirSync } from "node:fs";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const scriptPath = fileURLToPath(import.meta.url);
|
||||
const defaultRoot = path.resolve(path.dirname(scriptPath), "..");
|
||||
const fields = ["API", "Agent access", "Policy", "Evidence", "Verification", "Gaps"];
|
||||
const values = {
|
||||
Review: ["baseline", "operation-reviewed"],
|
||||
API: ["present", "missing", "not-applicable", "unreviewed"],
|
||||
"Agent access": ["available", "partial", "missing", "restricted", "operator-only", "not-applicable", "unreviewed"],
|
||||
Evidence: ["source-only", "verified", "failed", "blocked", "not-run"],
|
||||
Verification: ["passed", "failed", "blocked", "not-run"],
|
||||
};
|
||||
|
||||
function withoutExamples(text) {
|
||||
return text.replace(/^(`{3,}|~{3,})[^\n]*\n[\s\S]*?^\1\s*$/gm, "");
|
||||
}
|
||||
|
||||
function sections(text, level) {
|
||||
const pattern = new RegExp("^" + "#".repeat(level) + " `([^`]+)`[^\\n]*$", "gm");
|
||||
const matches = [...text.matchAll(pattern)];
|
||||
return matches.map((match, index) => ({
|
||||
id: match[1],
|
||||
body: text.slice(match.index + match[0].length, matches[index + 1]?.index ?? text.length),
|
||||
}));
|
||||
}
|
||||
|
||||
export function checkMaps(repoRoot = defaultRoot) {
|
||||
const errors = [];
|
||||
const featureRoot = path.join(repoRoot, "maps/feature-map");
|
||||
const agentRoot = path.join(repoRoot, "maps/agent-access-map");
|
||||
const fail = (file, message) => errors.push(`${path.relative(repoRoot, file)}: ${message}`);
|
||||
function read(file) {
|
||||
if (!existsSync(file)) { fail(file, "missing file"); return ""; }
|
||||
return withoutExamples(readFileSync(file, "utf8"));
|
||||
}
|
||||
function files(dir) {
|
||||
if (!existsSync(dir)) { fail(dir, "missing map directory"); return []; }
|
||||
return readdirSync(dir).filter((name) => name.endsWith(".md") && name !== "README.md").sort();
|
||||
}
|
||||
function unique(ids, file, label) {
|
||||
const seen = new Set();
|
||||
for (const id of ids) {
|
||||
if (seen.has(id)) fail(file, `duplicate ${label}: ${id}`);
|
||||
seen.add(id);
|
||||
}
|
||||
}
|
||||
function links(file, text) {
|
||||
for (const match of text.matchAll(/\]\(([^)\n]+)\)/g)) {
|
||||
const target = match[1].replace(/^<|>$/g, "");
|
||||
if (/^(?:[a-z][a-z\d+.-]*:|#)/i.test(target)) continue;
|
||||
if (target.startsWith("/")) { fail(file, `use a relative map reference: ${target}`); continue; }
|
||||
let local;
|
||||
try { local = decodeURIComponent(target.split(/[?#]/, 1)[0]); }
|
||||
catch { fail(file, `invalid local reference: ${target}`); continue; }
|
||||
if (!existsSync(path.resolve(path.dirname(file), local))) fail(file, `broken local reference: ${target}`);
|
||||
}
|
||||
}
|
||||
function field(file, body, name) {
|
||||
const escaped = name.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
|
||||
const match = body.match(new RegExp(`^${escaped}:[ \\t]*(\\S[^\\n]*)$`, "m"));
|
||||
if (!match) { fail(file, `missing ${name} field`); return null; }
|
||||
if (!values[name]) return null;
|
||||
const state = match[1].match(/^`([^`]+)`/);
|
||||
if (!state || !values[name].includes(state[1])) { fail(file, `invalid ${name} state`); return null; }
|
||||
if (["not-applicable", "restricted", "operator-only"].includes(state[1]) && !match[1].slice(state[0].length).replace(/[\s—–:.-]/g, "")) {
|
||||
fail(file, `${name} ${state[1]} needs a reason or path`);
|
||||
}
|
||||
return state[1];
|
||||
}
|
||||
function accessRecord(file, body) {
|
||||
const states = Object.fromEntries(fields.map((name) => [name, field(file, body, name)]));
|
||||
if (states.Evidence === "verified" && states.Verification !== "passed") {
|
||||
fail(file, "verified evidence requires a passed verification record");
|
||||
}
|
||||
const api = body.split(/^API:/m)[1]?.split(/^(?:Agent access|Policy|Evidence|Verification|Gaps):/m)[0] ?? "";
|
||||
if (states.API === "present" && !/\b(?:GET|POST|PUT|PATCH|DELETE|HEAD|OPTIONS) \/[^\s`]+/.test(api)) {
|
||||
fail(file, "present API needs an exact method/path example");
|
||||
}
|
||||
}
|
||||
|
||||
const productFiles = files(featureRoot);
|
||||
const agentFiles = files(agentRoot);
|
||||
const indexes = [read(path.join(featureRoot, "README.md")), read(path.join(agentRoot, "README.md"))];
|
||||
let capabilities = 0;
|
||||
for (const name of new Set([...productFiles, ...agentFiles])) {
|
||||
const productFile = path.join(featureRoot, name);
|
||||
const agentFile = path.join(agentRoot, name);
|
||||
const product = read(productFile);
|
||||
const agent = read(agentFile);
|
||||
const subfeatures = product.split(/^## Sub-features\s*$/m)[1]?.split(/^## /m)[0] ?? "";
|
||||
const ids = [...subfeatures.matchAll(/^- `([^`]+)`:/gm)].map((match) => match[1]);
|
||||
if (!ids.length) fail(productFile, "missing stable sub-feature IDs");
|
||||
unique(ids, productFile, "capability");
|
||||
const entries = sections(agent, 2);
|
||||
unique(entries.map((entry) => entry.id), agentFile, "capability");
|
||||
for (const id of ids) if (!entries.some((entry) => entry.id === id)) fail(agentFile, `missing capability: ${id}`);
|
||||
for (const entry of entries) {
|
||||
if (!ids.includes(entry.id)) fail(agentFile, `orphan capability: ${entry.id}`);
|
||||
const review = field(agentFile, entry.body, "Review");
|
||||
if (review === "operation-reviewed") {
|
||||
const operations = sections(entry.body, 3);
|
||||
if (!operations.length) fail(agentFile, `operation-reviewed capability ${entry.id} has no operations`);
|
||||
unique(operations.map((operation) => operation.id), agentFile, "operation");
|
||||
for (const operation of operations) accessRecord(agentFile, operation.body);
|
||||
} else accessRecord(agentFile, entry.body);
|
||||
}
|
||||
if (!agent.includes(`](../feature-map/${name})`)) fail(agentFile, "missing matching product link");
|
||||
if (!product.includes(`](../agent-access-map/${name})`)) fail(productFile, "missing matching companion link");
|
||||
for (const [index, text] of indexes.entries()) {
|
||||
if (!new RegExp(`\\]\\((?:\\./)?${name.replaceAll(".", "\\.")}\\)`).test(text)) {
|
||||
fail(path.join(index === 0 ? featureRoot : agentRoot, "README.md"), `missing area: ${name}`);
|
||||
}
|
||||
}
|
||||
if (!/^Review date: \d{4}-\d{2}-\d{2}\. Source commit: `[a-f\d]{40}`\./m.test(agent)) fail(agentFile, "missing review date/source commit");
|
||||
links(productFile, product);
|
||||
links(agentFile, agent);
|
||||
capabilities += ids.length;
|
||||
}
|
||||
for (const file of [path.join(repoRoot, "maps/README.md"), path.join(featureRoot, "README.md"), path.join(agentRoot, "README.md")]) links(file, read(file));
|
||||
return { errors, areas: productFiles.length, capabilities };
|
||||
}
|
||||
|
||||
if (process.argv[1] && path.resolve(process.argv[1]) === scriptPath) {
|
||||
const result = checkMaps();
|
||||
for (const error of result.errors) console.error(error);
|
||||
if (result.errors.length) process.exitCode = 1;
|
||||
else console.log(`Maps aligned: ${result.areas} areas, ${result.capabilities} capability IDs; references and contract fields valid. Structural checks only.`);
|
||||
}
|
||||
@@ -0,0 +1,89 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdtempSync, mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import test from "node:test";
|
||||
import { checkMaps } from "./check-maps.mjs";
|
||||
|
||||
const record = `API: \`present\` — POST /api/example.
|
||||
Agent access: \`partial\` — agent REST; native binding missing.
|
||||
Policy: same company and calling owner.
|
||||
Evidence: \`source-only\` — implementation inspected.
|
||||
Verification: \`not-run\` — assert permitted owner and denied other company.
|
||||
Gaps: native mutation path.
|
||||
`;
|
||||
|
||||
function fixture(t) {
|
||||
const root = mkdtempSync(path.join(os.tmpdir(), "paperclip-map-contract-"));
|
||||
t.after(() => rmSync(root, { recursive: true, force: true }));
|
||||
for (const dir of ["maps/feature-map", "maps/agent-access-map"]) mkdirSync(path.join(root, dir), { recursive: true });
|
||||
const product = path.join(root, "maps/feature-map/example.md");
|
||||
const agent = path.join(root, "maps/agent-access-map/example.md");
|
||||
writeFileSync(path.join(root, "maps/README.md"), "# Contract\n");
|
||||
for (const dir of ["feature-map", "agent-access-map"]) writeFileSync(path.join(root, "maps", dir, "README.md"), "[Example](example.md)\n");
|
||||
writeFileSync(product, "# Example\n\n[Agent access](../agent-access-map/example.md)\n\n## Sub-features\n\n- `create`: create a record.\n\n## How to get to it (user POV)\n");
|
||||
writeFileSync(agent, `# Agent access\n\nProduct map: [Example](../feature-map/example.md).\n\nReview date: 2026-10-12. Source commit: \`${"a".repeat(40)}\`.\n\n## \`create\`\n\nReview: \`baseline\`.\n\n${record}`);
|
||||
return { root, product, agent };
|
||||
}
|
||||
|
||||
test("paired source-only records pass without being promoted to live proof", (t) => {
|
||||
const { root } = fixture(t);
|
||||
assert.deepEqual(checkMaps(root), { errors: [], areas: 1, capabilities: 1 });
|
||||
});
|
||||
|
||||
test("a product area cannot lose its companion", (t) => {
|
||||
const { root, agent } = fixture(t);
|
||||
rmSync(agent);
|
||||
assert.ok(checkMaps(root).errors.some((error) => error.includes("missing file")));
|
||||
});
|
||||
|
||||
test("renaming a capability on one side reports missing and orphan IDs", (t) => {
|
||||
const { root, agent } = fixture(t);
|
||||
writeFileSync(agent, readFileSync(agent, "utf8").replace("## `create`", "## `update`"));
|
||||
const result = checkMaps(root);
|
||||
assert.ok(result.errors.some((error) => error.includes("missing capability: create")));
|
||||
assert.ok(result.errors.some((error) => error.includes("orphan capability: update")));
|
||||
});
|
||||
|
||||
test("duplicate IDs cannot disguise the missing capability", (t) => {
|
||||
const { root, agent } = fixture(t);
|
||||
writeFileSync(agent, readFileSync(agent, "utf8") + `\n## \`create\`\n\nReview: \`baseline\`.\n\n${record}`);
|
||||
assert.ok(checkMaps(root).errors.some((error) => error.includes("duplicate capability: create")));
|
||||
});
|
||||
|
||||
test("a moved or deleted source reference is reported", (t) => {
|
||||
const { root, agent } = fixture(t);
|
||||
writeFileSync(agent, readFileSync(agent, "utf8") + "\n[Source](../../server/deleted.ts#L12)\n");
|
||||
assert.ok(checkMaps(root).errors.some((error) => error.includes("broken local reference")));
|
||||
});
|
||||
|
||||
test("every reviewed operation needs its own access fields", (t) => {
|
||||
const { root, agent } = fixture(t);
|
||||
const text = readFileSync(agent, "utf8").replace("Review: `baseline`.", "Review: `operation-reviewed`.\n\n### `create`");
|
||||
writeFileSync(agent, text + `\n### \`update\`\n\n${record.replace(/^API:.*\n/m, "")}`);
|
||||
assert.ok(checkMaps(root).errors.some((error) => error.includes("missing API field")));
|
||||
});
|
||||
|
||||
test("verified evidence cannot coexist with verification not run", (t) => {
|
||||
const { root, agent } = fixture(t);
|
||||
writeFileSync(agent, readFileSync(agent, "utf8").replace("Evidence: `source-only`", "Evidence: `verified`"));
|
||||
assert.ok(checkMaps(root).errors.some((error) => error.includes("verified evidence requires a passed verification record")));
|
||||
});
|
||||
|
||||
test("an exception needs its reason rather than an empty exemption label", (t) => {
|
||||
const { root, agent } = fixture(t);
|
||||
writeFileSync(agent, readFileSync(agent, "utf8").replace(/^API:.*$/m, "API: `not-applicable`."));
|
||||
assert.ok(checkMaps(root).errors.some((error) => error.includes("not-applicable needs a reason")));
|
||||
});
|
||||
|
||||
test("API presence needs a concrete method and path", (t) => {
|
||||
const { root, agent } = fixture(t);
|
||||
writeFileSync(agent, readFileSync(agent, "utf8").replace("POST /api/example", "see the backend"));
|
||||
assert.ok(checkMaps(root).errors.some((error) => error.includes("present API needs an exact method/path")));
|
||||
});
|
||||
|
||||
test("an empty policy cannot borrow the next field's content", (t) => {
|
||||
const { root, agent } = fixture(t);
|
||||
writeFileSync(agent, readFileSync(agent, "utf8").replace(/^Policy:.*$/m, "Policy:\n"));
|
||||
assert.ok(checkMaps(root).errors.some((error) => error.includes("missing Policy field")));
|
||||
});
|
||||
Reference in new issue
Block a user