Files
PaperClipAI/packages/paperclip-runner/protocol/fixtures/evals/native-execution-seeded.json
T
DottaandPaperclip eb049aebf2 feat(skills): let agents update company skills safely (#15049)
## Thinking Path

> - Paperclip is an open source control plane for AI-agent companies.
> - Company skills give agents reusable work instructions.
> - Skill Studio can edit skill files and save version history.
> - Agents can create a skill, but they do not have a first-class update
tool.
> - An agent update needs a version check and safe retry behavior to
prevent lost edits.
> - This pull request adds `update_skill` through the existing company
skill file API.
> - The change keeps company policy, version history, and audit records
in one path.

## Linked Issues or Issue Description

**Subsystem affected**

Cross-cutting: the server API, shared validation, and runner tool
catalog.

**Problem or motivation**

An agent can create a company skill but cannot update its `SKILL.md`
through a first-class tool. An unguarded retry can also create duplicate
versions or overwrite a newer edit.

**Proposed solution**

Add `update_skill` with a required current version ID and a retry key.
Route it through the existing skill file API. Reject stale versions and
changed-input retries. Save the version and audit event together.

**Alternatives considered**

A separate write endpoint would duplicate the Skill Studio mutation path
and policy checks. This PR reuses that path instead.

**Roadmap alignment**

This work extends Skills Manager and Skill Studio, which are listed in
`ROADMAP.md`.

**Additional context**

The tool accepts a complete `SKILL.md`, not a partial patch. Callers
must read the current version before they edit it.

## What Changed

- Add optional version and retry fields to the existing skill file
update contract.
- Add a guarded API update with a stable retry receipt and attributed
audit event.
- Add `update_skill` to native and semantic runner tool catalogs, with
mode and policy gates.
- Add unit, integration, protocol, and semantic-tool regression
coverage.
- Document agent use and extend the OpenAPI request contract.

## Verification

- Focused tests and direct server and runner TypeScript checks passed
before this PR.
- `git diff --check` passed after the rebase onto `master`.
- CI passed on the latest PR head, including the full test matrix,
typecheck, and build. Local full typecheck and build stopped because
`cargo` is not installed. The local full test run ended without a
verdict.
- No dedicated end-to-end eval scenario was added or run. The protocol
coverage and semantic-tool test cover the new action deterministically.
- Reviewers can read a skill version, call `update_skill`, repeat the
same key, then try a stale version and a changed-input key. Only the
first edit must create a new version.

## Risks

- File writes and database transactions must stay in sync when a write
fails. The integration tests cover failed writes and retry behavior, but
CI must verify them on the PR head.
- Existing Skill Studio callers do not send the new optional
coordination fields. Their request shape remains valid.

## Model Used

- OpenAI Codex CLI assisted with this change. The runner did not expose
the exact model ID or context window. The agent used code execution and
repository tools.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details; exact model ID and context window were not exposed)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (focused tests; full suite
is pending CI)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green (53 pass, 4 skip on the latest
head)
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-03 12:37:55 -05:00

263 lines
8.1 KiB
JSON

{
"schema": "paperclip-runner/native-execution/v1",
"provenance": "seeded",
"identity": {
"runId": "run_evals_seeded",
"caseId": "case_semantic_denial",
"configId": "config_deterministic_v1",
"attemptId": "attempt_semantic_denial_0001"
},
"input": {
"caseSha256": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"configSha256": "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
"deterministicSeed": "paperclip-evals-v1-seed"
},
"runner": {
"package": {
"name": "@paperclipai/paperclip-runner",
"version": "0.0.0"
},
"binary": {
"name": "paperclip-runnerd",
"sha256": "sha256:cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc"
},
"prpVersion": 1,
"nativeExecutionVersion": 1,
"catalogVersion": 1,
"catalogSha256": "sha256:1dd4904ba9ae5c2b6bb0cbd4cfef65734d855bb5ed10ed05c70b603b8589bec3",
"driverContractVersion": 1,
"driverKind": "paperclip-deterministic",
"driverVersion": "1.0.0"
},
"events": [
{
"schema": "paperclip.prp.event.v1",
"sourceEventId": "evals_seeded_01",
"sourceSeq": 1,
"sourceInstanceId": "runner_evals_seeded",
"sourceKind": "runner",
"runId": "run_evals_seeded",
"normalizedSessionId": "session_evals_seeded",
"turnId": "turn_evals_seeded",
"itemId": "item_evals_seeded",
"eventType": "mcp_app.tool_input",
"schemaVersion": 1,
"priority": 1,
"emittedAt": "2026-08-11T12:00:00.000Z",
"payload": {
"semantic_tool": {
"schema": "paperclip.prp.semantic_tool.v1",
"schemaVersion": 1,
"phase": "input",
"operationId": "decide_approval",
"callId": "call_evals_seeded",
"correlation": {
"runId": "run_evals_seeded",
"normalizedSessionId": "session_evals_seeded",
"turnId": "turn_evals_seeded",
"itemId": "item_evals_seeded"
},
"idempotencyKey": "decide_evals_seeded",
"content": {
"digest": "sha256:dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd",
"redactionDisposition": "redacted",
"references": []
}
}
}
},
{
"schema": "paperclip.prp.event.v1",
"sourceEventId": "evals_seeded_02",
"sourceSeq": 2,
"sourceInstanceId": "runner_evals_seeded",
"sourceKind": "runner",
"runId": "run_evals_seeded",
"normalizedSessionId": "session_evals_seeded",
"turnId": "turn_evals_seeded",
"itemId": "item_evals_seeded",
"eventType": "mcp_app.tool_result",
"schemaVersion": 1,
"priority": 1,
"emittedAt": "2026-08-11T12:00:00.010Z",
"payload": {
"semantic_tool": {
"schema": "paperclip.prp.semantic_tool.v1",
"schemaVersion": 1,
"phase": "result",
"operationId": "decide_approval",
"callId": "call_evals_seeded",
"correlation": {
"runId": "run_evals_seeded",
"normalizedSessionId": "session_evals_seeded",
"turnId": "turn_evals_seeded",
"itemId": "item_evals_seeded"
},
"idempotencyKey": "decide_evals_seeded",
"content": {
"digest": "sha256:eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee",
"redactionDisposition": "redacted",
"references": []
},
"outcome": "denied",
"code": "required_claim_missing",
"retryable": false,
"authorizationBoundary": "grant",
"operationReceiptId": "receipt_evals_seeded",
"auditReceiptId": "audit_evals_seeded"
}
}
},
{
"schema": "paperclip.prp.event.v1",
"sourceEventId": "evals_seeded_03",
"sourceSeq": 3,
"sourceInstanceId": "runner_evals_seeded",
"sourceKind": "runner",
"runId": "run_evals_seeded",
"normalizedSessionId": "session_evals_seeded",
"turnId": "turn_evals_seeded",
"eventType": "run.result.proposed",
"schemaVersion": 1,
"priority": 0,
"emittedAt": "2026-08-11T12:00:00.020Z",
"payload": {
"schema": "paperclip.run_result.v1",
"reportedWorkDisposition": "needs_review",
"summary": "The governed tool effect was rejected and preserved as a denial.",
"completionClaim": {
"contractRevision": "evals-native-execution-v1",
"objectiveSatisfied": false,
"criteria": [],
"remainingWork": [
{
"description": "An authorized approver must decide.",
"blocksCompletion": true
}
]
},
"evidence": [
{
"receiptId": "receipt_evals_seeded"
}
],
"verification": [
{
"commandOrCheck": "rejected tool effect stays denied",
"status": "passed"
}
],
"attentionRequests": [],
"artifacts": []
}
},
{
"schema": "paperclip.prp.event.v1",
"sourceEventId": "evals_seeded_04",
"sourceSeq": 4,
"sourceInstanceId": "runner_evals_seeded",
"sourceKind": "runner",
"runId": "run_evals_seeded",
"normalizedSessionId": "session_evals_seeded",
"turnId": "turn_evals_seeded",
"eventType": "run.terminal",
"schemaVersion": 1,
"priority": 0,
"emittedAt": "2026-08-11T12:00:00.030Z",
"payload": {
"schema": "paperclip.prp.terminal.v1",
"turnTerminalState": "completed",
"runTerminalState": "succeeded",
"reportedWorkDisposition": "needs_review",
"workAssessmentId": "assessment_evals_seeded",
"statusDecisionId": "decision_evals_seeded"
}
}
],
"semanticTools": {
"definitions": [
{
"operationId": "decide_approval",
"schemaVersion": 1,
"availability": "denied"
}
],
"calls": [
{
"callId": "call_evals_seeded",
"operationId": "decide_approval",
"eventId": "evals_seeded_01"
}
],
"results": [
{
"callId": "call_evals_seeded",
"operationId": "decide_approval",
"eventId": "evals_seeded_02",
"outcome": "denied"
}
],
"denials": [
{
"callId": "call_evals_seeded",
"code": "required_claim_missing",
"authorizationBoundary": "grant",
"retryable": false
}
]
},
"terminal": {
"schema": "paperclip.prp.terminal.v1",
"turnTerminalState": "completed",
"runTerminalState": "succeeded",
"reportedWorkDisposition": "needs_review",
"workAssessmentId": "assessment_evals_seeded",
"statusDecisionId": "decision_evals_seeded"
},
"observations": {
"checks": [
{
"id": "tool-effect-denied",
"passed": true,
"receiptId": "receipt_evals_seeded"
}
]
},
"usage": {
"inputTokens": 12,
"outputTokens": 8,
"totalTokens": 20,
"requestCount": 1,
"durationMs": 30,
"cost": {
"currency": "USD",
"amountMicros": 0
}
},
"transcript": {
"uri": "artifact://native/attempt_semantic_denial_0001/transcript.jsonl",
"sha256": "sha256:ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff",
"byteSize": 2048,
"mediaType": "application/x-ndjson",
"complete": true,
"eventCount": 4,
"omissionReason": null
},
"artifactRoot": {
"uri": "artifact://native/attempt_semantic_denial_0001/",
"sha256": "sha256:1212121212121212121212121212121212121212121212121212121212121212",
"byteSize": 4096,
"mediaType": "application/vnd.paperclip.runner-artifact-root+json"
},
"artifacts": [
{
"uri": "artifact://native/attempt_semantic_denial_0001/denial-receipt.json",
"sha256": "sha256:1313131313131313131313131313131313131313131313131313131313131313",
"byteSize": 512,
"mediaType": "application/json"
}
],
"failure": null,
"x_fixturePurpose": "Proves seeded provenance, denied effects, and additive field preservation."
}