Files
PaperClipAI/tests
DottaandPaperclip b4c69ded9f test(e2e): attribute native OpenCode skill loads (#18929)
## Thinking Path

> - Paperclip lets people manage agents and assign saved company skills.
> - Daily QA must prove that an agent loads the saved procedure it uses.
> - The saved-skill journey accepts several native source-read paths.
> - OpenCode's native proxy changes its built-in skill loader's
normalized operation to unknown.
> - The grader expected execute and rejected a real completed skill
load.
> - This PR accepts that one observed native shape with strict source
and identity checks.
> - Genuine loads can qualify while missing, foreign or incomplete
evidence still fails.

## Linked Issues or Issue Description

Refs: #18783.

**What happened?**

A combined QA run on `0884e73b46ee064b3d242e467035dde6c32e5cb5` failed
`saved-skill-native-read-attributed`. The agent's successful native
skill load exposed the complete saved procedure. The exact immutable
assignment and source checks passed. Core recorded the built-in loader
as `operation: unknown` and `readOnly: false`. The grader accepted this
loader only with `operation: execute`.

**Expected behavior**

A complete native OpenCode skill load can qualify when its actual run,
session, immutable source and paired receipt all match. A generic
unknown operation cannot qualify.

**Steps to reproduce**

Run the saved-skill journey through Core
`09162fe893b464dd2ccdafd14bd3ef1611202a86` with OpenCode. Let the agent
use its native `skill` loader. The direct TypeScript converter emits
execute, but the real proxy forwards a `builtinToolCall` without that
operation. Core's native converter records unknown. The former
TypeScript-only fixture missed this boundary.

## What Changed

- Add one closed native OpenCode skill-loader branch. Require the actual
run, runner, native session, item, turn and paired call.
- Bind the API assignment, admitted native skill, immutable version map
and materialized runtime directory to the same canonical skill.
- Require the exact wrapper name, heading and complete saved procedure.
Require an absolute local base directory and an empty file inventory.
Reject traversal, URI encodings, backslashes, query or fragment markers
and all ASCII controls. Keep valid spaces.
- Require a successful, complete and untruncated native receipt with the
exact output byte count. Reject missing, duplicate, reordered, foreign,
failed, dynamic or MCP carriers.
- Add 65 loader and boundary controls. Drive the real
TypeScript-to-proxy input and use the observed Core native carrier
shape. The JavaScript test does not claim to run the Rust converter.
- Document the native loader path. Preserve every existing accepted
execute, filesystem, first-party and Codex parser path byte-for-byte.
Keep all seven checks, one-run limit, 500-cent guard, task prompt, model
and ordinary Beta-off Latest assignment unchanged.

## Verification

- The sealed original Core receipt reproduces RED: the former grader
returns zero reads. Changing only its normalized operation to execute
reaches the old accepted branch. The narrow correction returns one
attributed read in separate local analysis. The original QA grade
remains failed.
- The actual TypeScript, proxy and native normalization source regions
match between Core091 and the tested baseline0884. The proxy emits
`builtinToolCall` without an operation. The native converter supplies
unknown.
- 334 focused tests pass across nine files, including 65 new loader
controls and all existing saved-skill evidence, flow, contract and
catalog controls. Run `node node_modules/vitest/vitest.mjs run --config
tests/runner-e2e/vitest.config.ts
tests/runner-e2e/product-journeys/saved-skill-use-native-loader.test.ts`
for the narrow regression.
- TAB, ESC and DEL controls fail before the review hardening and pass
after it. The valid-space positive remains green.
- Canonical Runner TypeScript prerequisites, own server declarations and
E2E types pass. The final E2E-only check reuses those same-commit
declarations after only the two test-scope files changed. The checked
candidate blobs match this committed head.
- Credential-free discovery lists 25 catalog journeys, including
saved-skill use. Registration and selection do not change.
- `git diff --check` passes. No Rust build or product-model call ran
locally. Full CI
[41773295359](https://github.com/paperclipai/paperclip/actions/runs/41773295359)
passed on `fb537b598da40176fa1752b49a4f676f468de1f7`: 48 successful jobs
and one skip. Fresh Greptile is 5/5 with zero unresolved threads. The
earlier same-head canceled run remains retained; no retry was requested.
The optional native review ended neutral with a 5/5 source assessment
and explicit execution/qualification coverage limits. Fresh staging
qualification
[41777787338](https://pages.paperclip.ing/evals-qa-smoke-41777787338-1/)
passed on this exact head: all seven checks and all five phases, exactly
one native OpenCode run, independent saved-document reopen/source
preservation, and normal cleanup complete. TestOps recorded a new passed
result and the automatic Slack report succeeded.
- The failed original stopped before independent document, reload and
source-after checks. Those checks remain unqualified for that run. The
original failed grade remains unchanged. A separate genuine fresh QA
pass now verifies the complete journey before daily re-admission.

## Risks

This path depends on OpenCode's observed native wrapper and materialized
directory contract. Missing or changed evidence fails closed. The fix
adds no runtime service, schema or API behavior. The native-loader
branch is covered by strict local regression controls and the original
sealed Core receipt; a separate fresh staging run also passed the
complete saved-skill journey.

## Model Used

OpenAI GPT-6 through Codex with reasoning, tool use and code execution.
The runtime does not expose a more specific deployment ID or
context-window size. Local checks made no product-model calls.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-03 23:21:17 -06:00
..