feat(runner): add lazy capability discovery

This commit is contained in:
Dotta committed 2026-08-20 18:44:05 -05:00
1 parent 1f4499460e
commit a18aab01ad
8 files changed
+415 -9

No files matched your search

+178
View File
@@ -0,0 +1,178 @@
# Paperclip Runner lazy tool discovery
Date: 2026-08-20
Status: fallback implemented and evaluated; native adapter pending transport support
Reference: [OpenAI Tool Search](https://developers.openai.com/api/docs/guides/tools-tool-search)
## Outcome
Keep the model's initial protocol surface small while allowing it to discover optional
Paperclip operations at runtime. Discovery never grants authority: the runner filters
what may be discovered, and the dispatcher independently authorizes every invocation.
OpenAI Tool Search is the preferred provider capability where the selected model and
driver support it. The runner also owns a provider-neutral fallback so Paperclip's
protocol and security semantics do not depend on one transport.
## Product contract
The initial model context contains:
- a deliberately small always-present task core;
- high-level descriptions of searchable namespaces, not every operation schema;
- either native `tool_search` or runner-managed `discover_capabilities`;
- explicit guidance that discovery should happen only when the task requires it.
Candidate namespace projection, kept below ten operations per namespace:
| Namespace | Purpose | Initial operations |
| --- | --- | --- |
| `discovery` | Find company tasks, agents, projects, and goals | `search_tasks`, `list_agents`, `get_agent`, `list_projects`, `list_goals` |
| `delegation` | Create bounded child work and dependencies | `create_task`, `set_dependencies` |
| `governance` | Read, request, discuss, and decide approvals | approval operations |
| `workspace` | Inspect and control task workspace services | workspace operations |
| `cases` | Read and maintain case records | case operations |
| `routines` | Read and maintain routines | routine operations |
| `company_skills` | Inspect and synchronize company skills | skill operations |
| `secrets` | Discover metadata and perform tightly governed secret reads | secret operations |
| `company_admin` | Portability and company administration | admin operations |
The exact always-present core is a measured product decision. Start with active-task
context/history, progress/status reporting, and terminal/disposition operations; use
eval results to decide whether document reads and writes remain eager or become
deferred.
## Architecture
### Canonical registry projection
Extend the canonical operation registry with discovery metadata: namespace, concise
search description, searchable aliases, sensitivity classification, and whether an
operation's name may be disclosed when unauthorized. Continue deriving provider
schemas and scenario descriptors from the same canonical operation identity.
### Discovery policy
Given a query and session identity, compute candidates from the canonical registry,
then filter by company, actor, task, mode, role, claims, enabled capabilities, scenario
policy, disabled-by-default status, and driver support. Rank only the permitted set.
Sensitive undiscoverable operations must not leak through names, counts, descriptions,
errors, or timing where practical.
Return trusted registry definitions only. Never accept a model-authored schema or an
untrusted remote definition as executable protocol authority.
### Invocation policy
Loaded tools receive no durable authority grant. Every call goes through the existing
semantic dispatcher and repeats the current policy decision. Revoked claims or changed
task state therefore take effect even if the schema remains in conversation context.
### Provider adapters
1. **Native adapter:** project namespaces/deferred functions and consume provider
tool-search call/output events.
2. **Runner-managed adapter:** expose `discover_capabilities`; resolve the query
locally; add selected trusted definitions on the next model continuation.
3. Normalize both paths into PRP events so Runner Lab and Evalbook do not depend on
provider-specific event shapes.
### Protocol evidence
Add evidence events for discovery requested, candidates considered (redacted), policy
decision, definitions loaded, namespace activated, and discovery failure. Preserve the
query and returned operation IDs when safe, plus model/driver/configuration, token and
cost measurements. Link the eventual invocation back to the discovery event.
## Delivery phases
### D0 — transport spike
Prove whether the Codex app-server driver can send deferred definitions and surface
native tool-search events for the configured GPT-5.4-class model. Produce one recorded
attempt. Do not change default exposure.
Result: the current Codex app-server driver accepts dynamic tools at thread creation but
does not expose Responses API native Tool Search events or a supported mid-turn schema
append operation. Native support is therefore not claimed. The provider-neutral gateway
below is the implemented path.
### D1 — provider-neutral domain layer
Add discovery metadata and namespace validation to the canonical registry. Implement
the policy-filtered search service and normalized discovery event contract. Unit-test
ranking, nondisclosure, mode/role/claim filtering, and registry drift.
### D2 — runner-managed fallback
Implement `discover_capabilities`, continuation-time schema loading, policy recheck on
invocation, limits on results/searches/loaded tools, and deterministic errors. Ship
behind `toolExposure: "eager" | "lazy"`, defaulting to `eager`.
Implemented: lazy threads receive the always-present core plus
`discover_capabilities` and `invoke_discovered_capability`. Search is deterministic,
namespace-aware, limited to eight results, filtered by live exposure policy, and returns
only trusted catalog schemas. Invocation requires prior discovery and is reauthorized by
the semantic dispatcher. Loaded operation identities survive runner snapshots.
### D3 — native Tool Search adapter
If D0 succeeds, implement native namespace/deferred-tool projection and normalize its
events to the same domain contract. If D0 fails, retain the evidence and defer this
adapter without blocking the fallback.
### D4 — Runner Lab and reports
Render discovery inline in the issue thread, expose policy detail in Devtools, connect
loaded definitions to subsequent calls, and report eager/lazy configuration plus token,
latency, and cost deltas.
Implemented: discovery is retained as redacted `tool_discovery` evidence, appears on the
semantic-tool boundary, and Devtools reports exposure mode and loaded operation IDs.
### D5 — eval and rollout
Implement the lazy-discovery eval track in the sibling `paperclip-evals` repository.
Run deterministic checks first and live comparisons on GPT-5.4 mini only. Switch the
default to lazy after semantic parity, authorization/nondisclosure, observability, and
efficiency gates pass. Keep eager mode as a temporary diagnostic control until one
release cycle completes.
Initial result: all 35 GPT-5.4-mini live roster cases passed on
`20260820-full-protocol-lazy-01`, including three lazy-discovery cases and four new
multi-operation workflows. Eager remains the default pending a broader repetition and
efficiency comparison; this single green run is functional evidence, not a reliability
estimate.
## Guardrails and limits
- Maximum returned definitions per search and maximum loaded definitions per session.
- Maximum discovery calls per turn/session, with loop detection.
- No `generic_api_request` outside explicit skill-test scenarios.
- No disclosure of secret/admin tools to unauthorized actors.
- Namespace and operation descriptions are trusted code-reviewed content.
- Search results are company-scoped and deterministic for identical policy state.
- Tool invocation always rechecks live policy and terminal task state.
- Cache behavior is measured but never allowed to weaken authorization freshness.
## Acceptance criteria
- Lazy sessions begin without optional operation schemas.
- A permitted optional operation can be found and invoked end to end.
- An unauthorized operation is neither disclosed nor invokable.
- Revocation after loading prevents invocation.
- Native and fallback routes produce equivalent semantic PRP evidence.
- Runner Lab makes discovery and its causal invocation inspectable.
- Evalbook compares eager and lazy correctness, tokens, latency, and cost.
- No regression in the existing deterministic capability suite or protocol eval roster.
## Open decisions to resolve with evidence
1. Exact always-present core, especially document operations.
2. Lexical versus embedding-assisted local ranking; begin lexical and deterministic.
3. Whether sensitive namespaces are hidden entirely or return a generic unavailable
result; prefer complete nondisclosure.
4. Session persistence rules for loaded schemas across compaction and continuation.
5. Whether native Tool Search is available through Codex app-server with the required
event fidelity; D0 decides this rather than assuming it.
@@ -96,7 +96,7 @@ function publicCompanyState(serialized: string): CapabilityJsonValue {
}
function boundary(kind: string): string {
if (kind === "tool_call" || kind === "tool_result") return "semantic tool";
if (kind === "tool_call" || kind === "tool_result" || kind === "tool_discovery") return "semantic tool";
if (kind === "provider_event") return "Codex app-server";
if (kind === "process") return "runnerd process";
if (kind === "interaction") return "mock control plane";
@@ -157,6 +157,8 @@ export function projectCapabilityDevtools(snapshot: CapabilityLiveSessionSnapsho
currentAttemptId: snapshot.currentAttemptId ?? null,
terminalTurns: snapshot.terminalTurns ?? [],
usageLedger: snapshot.usageLedger ?? [],
toolExposure: snapshot.config.toolExposure ?? "eager",
loadedOperationIds: snapshot.loadedOperationIds ?? [],
createdAt: snapshot.createdAt,
updatedAt: snapshot.updatedAt,
}),
@@ -166,6 +168,7 @@ export function projectCapabilityDevtools(snapshot: CapabilityLiveSessionSnapsho
effectiveCapabilities: snapshot.config.capabilities,
explicitClaims: snapshot.config.explicitClaims,
turnTimeoutMs: snapshot.config.turnTimeoutMs,
toolExposure: snapshot.config.toolExposure ?? "eager",
workingDirectory: "[withheld]",
}),
};
@@ -32,6 +32,7 @@ export type CapabilityEvidenceKind =
| "session"
| "provider_event"
| "tool_exposure"
| "tool_discovery"
| "tool_call"
| "tool_result"
| "interaction"
@@ -313,6 +314,15 @@ export function redactCapabilityEvidenceData(
.sort(),
scenarioId: clamp(asString(data.scenarioId), 120),
};
case "tool_discovery":
return {
action: clamp(asString(data.action), 80),
query: clamp(asString(data.query), 500),
namespace: clamp(asString(data.namespace), 120),
operationIds: (Array.isArray(data.operationIds) ? data.operationIds : [])
.filter((id): id is string => typeof id === "string" && CATALOG_OPERATION_IDS.has(id))
.sort(),
};
case "tool_call": {
const operationId = asString(data.operationId);
return {
@@ -374,6 +384,10 @@ export function capabilityEvidenceDetail(
const ids = Array.isArray(data.operationIds) ? data.operationIds.length : 0;
return `tool exposure · ${ids} operation${ids === 1 ? "" : "s"}`;
}
case "tool_discovery": {
const ids = Array.isArray(data.operationIds) ? data.operationIds.length : 0;
return `tool discovery · ${value("action") || "searched"} · ${ids} loaded`;
}
case "tool_call": {
const fields = asRecord(data.input).fields;
const count = Array.isArray(fields) ? fields.length : 0;
@@ -450,6 +464,12 @@ export function capabilityEvidenceDetails(
add("Scenario", data.scenarioId);
addList("Operations", data.operationIds);
break;
case "tool_discovery":
add("Action", data.action);
add("Query", data.query);
add("Namespace", data.namespace);
addList("Operations", data.operationIds);
break;
case "tool_call": {
const input = asRecord(data.input);
add("Call", data.callId);
@@ -273,6 +273,24 @@ function providerState(): FakeProviderState {
}
describe("Capability live runnerd and Codex session", () => {
it("opens lazy sessions with core and discovery gateways instead of optional schemas", async () => {
const state = providerState();
const claim = "discovery:agents:read";
const service = new CapabilityLiveSessionService({ transportFactory: fakeTransportFactory(state) });
const session = await service.create({
runId: "run-live-lazy", sessionId: "session-live-lazy", toolExposure: "lazy",
capabilities: [claim], explicitClaims: [claim], scenario: { id: "lazy", claims: [claim] },
});
const start = state.transports[0]!.requests.find((entry) => entry.method === "thread/start")!;
const names = (start.params.dynamicTools as Array<{ name: string }>).map((tool) => tool.name);
expect(names).toContain("get_task_context");
expect(names).toContain("discover_capabilities");
expect(names).toContain("invoke_discovered_capability");
expect(names).not.toContain("list_agents");
expect(session.snapshot().config.toolExposure).toBe("lazy");
await service.shutdown(session.id);
});
it("returns typed mock results to the same multi-turn Codex thread without Paperclip network calls", async () => {
const state = providerState();
const fetchSpy = vi.spyOn(globalThis, "fetch");
@@ -17,6 +17,7 @@ import type {
} from "../mock-core/capability-control-plane-types.js";
import { CapabilityMockControlPlaneAdapter } from "../mock-core/capability-mock-control-plane-adapter.js";
import { CapabilitySemanticDispatcher } from "../semantic-tools/dispatcher.js";
import { CAPABILITY_DISCOVERY_NAMESPACES, type CapabilityToolExposureMode } from "../semantic-tools/discovery.js";
import type {
CapabilitySemanticAuthorizationRecord,
CapabilitySemanticScenarioPolicy,
@@ -44,6 +45,8 @@ const LIVE_BASE_INSTRUCTIONS = [
"Do not discover skills, credentials, endpoints, or hidden control-plane capabilities.",
].join(" ");
const CODEX_PERMISSION_PROFILE = "paperclip-runner-workspace-only";
const DISCOVER_TOOL = "discover_capabilities";
const INVOKE_DISCOVERED_TOOL = "invoke_discovered_capability";
export type CapabilityLiveSessionStatus =
| "starting"
@@ -95,6 +98,7 @@ export interface CapabilityLiveSessionConfigSnapshot {
explicitClaims: string[];
seedState: string;
turnTimeoutMs: number;
toolExposure?: CapabilityToolExposureMode;
}
export type CapabilityLiveAttemptStatus = "running" | "succeeded" | "failed" | "terminated";
@@ -183,6 +187,7 @@ export interface CapabilityLiveSessionSnapshot {
usageLedger?: CapabilityLiveUsageReceipt[];
/** Bounded mock-control-plane snapshots for DevTools time travel and diffs. */
stateHistory?: CapabilityLiveStateRevision[];
loadedOperationIds?: string[];
}
export interface CreateCapabilityLiveSessionInput {
@@ -199,6 +204,7 @@ export interface CreateCapabilityLiveSessionInput {
sessionId?: string;
attemptId?: string;
turnTimeoutMs?: number;
toolExposure?: CapabilityToolExposureMode;
}
export interface ResumeCapabilityLiveSessionInput {
@@ -534,6 +540,18 @@ function dynamicToolSpec(definition: CapabilitySemanticToolDefinition): Record<s
};
}
function discoveryToolSpecs(): Record<string, unknown>[] {
return [{
name: DISCOVER_TOOL,
description: `Search authorized optional Paperclip capabilities. Namespaces: ${CAPABILITY_DISCOVERY_NAMESPACES.map((item) => `${item.name} (${item.description})`).join("; ")}`,
inputSchema: { type: "object", properties: { query: { type: "string", minLength: 1, maxLength: 500 }, namespace: { type: "string" }, limit: { type: "integer", minimum: 1, maximum: 8 } }, required: ["query"], additionalProperties: false },
}, {
name: INVOKE_DISCOVERED_TOOL,
description: "Invoke one optional Paperclip operation returned by discover_capabilities. Authority is rechecked at invocation.",
inputSchema: { type: "object", properties: { operationId: { type: "string" }, input: { type: "object", additionalProperties: true } }, required: ["operationId", "input"], additionalProperties: false },
}];
}
function userInput(message: string): Record<string, unknown> {
return { type: "text", text: message, text_elements: [] };
}
@@ -608,6 +626,7 @@ export class CapabilityLiveSessionService {
explicitClaims: authority.explicitClaims,
seedState,
turnTimeoutMs: input.turnTimeoutMs ?? 120_000,
toolExposure: input.toolExposure ?? "eager",
},
store: this.#store,
transportFactory: this.#transportFactory,
@@ -742,6 +761,7 @@ export class CapabilityLiveSessionService {
capabilities: config.capabilities,
explicitClaims: config.explicitClaims,
turnTimeoutMs: config.turnTimeoutMs,
toolExposure: config.toolExposure ?? "eager",
});
}
@@ -810,6 +830,7 @@ export class CapabilityLiveSession {
#eventSeq = 0;
#turnEventCount = 0;
readonly #rawUsageByTurn = new Map<string, { providerRequests: number; inputTokens: number; outputTokens: number; cachedInputTokens: number; reasoningTokens: number }>();
readonly #loadedOperationIds = new Set<string>();
private constructor(options: OpenSessionOptions & { snapshot?: CapabilityLiveSessionSnapshot }) {
this.#port = options.port;
@@ -847,6 +868,7 @@ export class CapabilityLiveSession {
operationId: "session.open",
state: this.#port.serialize(),
}]);
for (const operationId of options.snapshot?.loadedOperationIds ?? []) this.#loadedOperationIds.add(operationId);
this.#providerThreadId = options.snapshot?.providerThreadId ?? "";
this.#providerSessionId = options.snapshot?.providerSessionId ?? null;
this.#providerModel = options.snapshot?.providerModel === undefined
@@ -1015,6 +1037,7 @@ export class CapabilityLiveSession {
terminalTurns: structuredClone(this.#terminalTurns),
usageLedger: structuredClone(this.#usageLedger),
stateHistory: structuredClone(this.#stateHistory),
loadedOperationIds: [...this.#loadedOperationIds].sort(),
};
}
@@ -1360,10 +1383,14 @@ export class CapabilityLiveSession {
capabilities: { experimentalApi: true, requestAttestation: false },
});
this.#transport.notify("initialized");
const tools = this.#dispatcher.listTools(this.#authority.runId);
const authorizedTools = this.#dispatcher.listTools(this.#authority.runId);
const tools = (this.#config.toolExposure ?? "eager") === "lazy"
? authorizedTools.filter((tool) => tool.annotations.exposure === "always")
: authorizedTools;
this.#appendEvidence("tool_exposure", null, {
operationIds: tools.map((tool) => tool.name),
scenarioId: this.#config.scenario.id,
toolExposure: this.#config.toolExposure ?? "eager",
});
if (resume) {
const read = await this.#transport.request("thread/read", {
@@ -1427,7 +1454,10 @@ export class CapabilityLiveSession {
runtimeWorkspaceRoots: [this.#config.workingDirectory],
approvalPolicy: "never",
baseInstructions: LIVE_BASE_INSTRUCTIONS,
dynamicTools: tools.map(dynamicToolSpec),
dynamicTools: [
...tools.map(dynamicToolSpec),
...((this.#config.toolExposure ?? "eager") === "lazy" ? discoveryToolSpecs() : []),
],
experimentalRawEvents: true,
persistExtendedHistory: true,
});
@@ -1515,25 +1545,55 @@ export class CapabilityLiveSession {
contentItems: [{ type: "inputText", text: "Tool call was outside the active Capability thread and turn." }],
};
}
if (operationId === DISCOVER_TOOL) {
const args = record(request.params.arguments);
try {
const discovered = this.#dispatcher.discoverTools(
this.#authority.runId,
text(args.query),
{ ...(text(args.namespace) ? { namespace: text(args.namespace) } : {}), ...(typeof args.limit === "number" ? { limit: args.limit } : {}) },
);
for (const operation of discovered.operations) this.#loadedOperationIds.add(operation.name);
this.#appendEvidence("tool_discovery", turnId, {
action: "loaded", query: discovered.query, namespace: discovered.namespace ?? "",
operationIds: discovered.operations.map((operation) => operation.name),
});
await this.#persist();
return { success: true, contentItems: [{ type: "inputText", text: JSON.stringify(discovered) }] };
} catch (error) {
this.#appendEvidence("tool_discovery", turnId, { action: "rejected", query: text(args.query), namespace: text(args.namespace), operationIds: [] });
return { success: false, contentItems: [{ type: "inputText", text: error instanceof Error ? error.message : "Capability discovery failed." }] };
}
}
let semanticOperationId = operationId;
let semanticInput = request.params.arguments;
if (operationId === INVOKE_DISCOVERED_TOOL) {
const args = record(request.params.arguments);
semanticOperationId = text(args.operationId);
semanticInput = args.input;
if (!this.#loadedOperationIds.has(semanticOperationId)) {
return { success: false, contentItems: [{ type: "inputText", text: "Operation was not loaded by discover_capabilities." }] };
}
}
const beforeRevision = this.#port.snapshot().revision;
this.#appendEvidence("tool_call", turnId, {
callId,
operationId,
input: jsonValue(request.params.arguments),
operationId: semanticOperationId,
input: jsonValue(semanticInput),
beforeRevision,
});
this.#emit({ turnId, kind: "activity", reason: "tool_call" });
const result = await this.#dispatcher.dispatch({
runId: this.#authority.runId,
callId,
operationId,
input: request.params.arguments,
operationId: semanticOperationId,
input: semanticInput,
});
this.#recordStateRevision(turnId, operationId);
this.#recordStateRevision(turnId, semanticOperationId);
const replayDisposition = result.ok ? text(record(result.result).disposition) : "";
this.#appendEvidence("tool_result", turnId, {
callId,
operationId,
operationId: semanticOperationId,
result: jsonValue(result),
beforeRevision,
afterRevision: result.stateRevision,
@@ -0,0 +1,98 @@
import { CAPABILITY_SEMANTIC_TOOL_CATALOG } from "./catalog.js";
import { decideCapabilitySemanticAuthorization } from "./policy.js";
import type {
CapabilitySemanticOperationId,
CapabilitySemanticPolicyContext,
CapabilitySemanticToolDefinition,
CapabilitySemanticToolDescriptor,
} from "./types.js";
export type CapabilityToolExposureMode = "eager" | "lazy";
export interface CapabilityDiscoveryResult {
readonly schema: "paperclip.capability-discovery.v1";
readonly query: string;
readonly namespace: string | null;
readonly operations: readonly CapabilitySemanticToolDefinition[];
readonly truncated: boolean;
}
const NAMESPACE: Readonly<Record<CapabilitySemanticOperationId, string>> = Object.freeze({
get_task_context: "active_task", get_task_history: "active_task",
list_documents: "documents", read_document: "documents", list_document_revisions: "documents",
report_progress: "active_task", answer_status_question: "active_task", write_document: "documents",
request_human_input: "documents", register_deliverable: "documents", finish_task: "active_task",
block_task: "active_task", request_review: "active_task", search_tasks: "discovery",
list_agents: "discovery", get_agent: "discovery", create_task: "delegation",
set_dependencies: "delegation", list_approvals: "governance", get_approval: "governance",
get_approval_context: "governance", request_approval: "governance",
decide_approval: "governance", comment_on_approval: "governance",
get_workspace_runtime: "workspace", control_workspace_service: "workspace",
schedule_wake: "continuation", generic_api_request: "test_infrastructure",
});
export const CAPABILITY_DISCOVERY_NAMESPACES = Object.freeze([
{ name: "discovery", description: "Find company tasks and agents." },
{ name: "delegation", description: "Create bounded child work and task dependencies." },
{ name: "governance", description: "Read, request, discuss, and decide approvals." },
{ name: "workspace", description: "Inspect and control active-task workspace services." },
{ name: "continuation", description: "Schedule a bounded continuation wake." },
]);
export function capabilityToolNamespace(operationId: CapabilitySemanticOperationId): string {
return NAMESPACE[operationId];
}
export function alwaysCapabilityDefinitions(): readonly CapabilitySemanticToolDefinition[] {
return CAPABILITY_SEMANTIC_TOOL_CATALOG.filter((item) => item.exposure === "always").map(toDefinition);
}
export function discoverCapabilityDefinitions(
query: string,
context: CapabilitySemanticPolicyContext,
options: { namespace?: string; limit?: number } = {},
): CapabilityDiscoveryResult {
const normalized = query.trim().toLowerCase();
if (normalized.length === 0) throw new Error("discovery_query_empty");
const limit = Math.max(1, Math.min(options.limit ?? 5, 8));
const tokens = normalized.split(/[^a-z0-9]+/).filter((token) => token.length > 1);
const permitted = CAPABILITY_SEMANTIC_TOOL_CATALOG
.filter((descriptor) => descriptor.exposure === "optional")
.filter((descriptor) => options.namespace === undefined || capabilityToolNamespace(descriptor.operationId) === options.namespace)
.filter((descriptor) => decideCapabilitySemanticAuthorization(descriptor, context, "exposure").allowed)
.map((descriptor) => ({ descriptor, score: score(descriptor, tokens, normalized) }))
.filter((candidate) => candidate.score > 0)
.sort((left, right) => right.score - left.score || left.descriptor.operationId.localeCompare(right.descriptor.operationId));
return Object.freeze({
schema: "paperclip.capability-discovery.v1",
query,
namespace: options.namespace ?? null,
operations: Object.freeze(permitted.slice(0, limit).map(({ descriptor }) => toDefinition(descriptor))),
truncated: permitted.length > limit,
});
}
function score(descriptor: CapabilitySemanticToolDescriptor, tokens: readonly string[], query: string): number {
const namespace = capabilityToolNamespace(descriptor.operationId);
const id = descriptor.operationId.toLowerCase();
const haystack = `${id} ${descriptor.title} ${descriptor.description} ${namespace}`.toLowerCase();
let value = id === query ? 100 : namespace === query ? 50 : 0;
for (const token of tokens) {
if (id.includes(token)) value += 12;
if (namespace.includes(token)) value += 8;
if (haystack.includes(token)) value += 3;
}
return value;
}
function toDefinition(descriptor: CapabilitySemanticToolDescriptor): CapabilitySemanticToolDefinition {
return {
name: descriptor.operationId, description: descriptor.description,
inputSchema: descriptor.inputSchema, outputSchema: descriptor.outputSchema,
annotations: {
semanticContract: descriptor.schema, operationId: descriptor.operationId,
version: descriptor.version, exposure: descriptor.exposure,
requiredClaims: descriptor.requiredClaims,
},
};
}
@@ -19,6 +19,7 @@ import {
containsProtectedSemanticData,
redactSemanticValue,
} from "./redaction.js";
import { discoverCapabilityDefinitions } from "./discovery.js";
import type {
CapabilitySemanticAuthorizationDecision,
CapabilitySemanticAuthorizationRecord,
@@ -89,6 +90,14 @@ export class CapabilitySemanticDispatcher {
return deepFreeze(definitions);
}
discoverTools(
runId: string,
query: string,
options: { namespace?: string; limit?: number } = {},
) {
return discoverCapabilityDefinitions(query, this.#policyContext(runId), options);
}
authorizationRecords(): readonly CapabilitySemanticAuthorizationRecord[] {
return deepFreeze(structuredClone(this.#records));
}
@@ -84,6 +84,26 @@ describe("Capability semantic catalog and authorization", () => {
expect(adapter.snapshot().revision).toBe(before);
});
it("discovers only authorized optional tools and returns trusted schemas", async () => {
const claims = ["discovery:agents:read", "governance:approvals:read"];
const adapter = await running(claims);
const dispatcher = new CapabilitySemanticDispatcher(adapter, {
scenario: { id: "lazy-search", claims }, explicitClaims: claims,
});
const found = dispatcher.discoverTools(OPEN.identity.runId, "find company agents", { namespace: "discovery" });
expect(found.operations.map((tool) => tool.name)).toEqual(["list_agents", "get_agent"]);
expect(found.operations.every((tool) => tool.annotations.semanticContract === "paperclip.semantic-tool.v1")).toBe(true);
expect(JSON.stringify(found)).not.toContain("list_approvals");
});
it("does not disclose optional tools that current authority cannot invoke", async () => {
const adapter = await running();
const dispatcher = new CapabilitySemanticDispatcher(adapter);
const found = dispatcher.discoverTools(OPEN.identity.runId, "create child task approval secret admin");
expect(found.operations).toEqual([]);
expect(JSON.stringify(found.operations)).not.toMatch(/create_task|approval|secret|administer_company/);
});
it("executes a granted optional operation through the mock port", async () => {
const claim = "dependencies:write";
const adapter = await running([claim], {