docs: add native Runner integration playbook (#16846)

## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native Runner connects those agents to harnesses through provider drivers. > - Adding a harness crosses protocol, setup, lifecycle, packaging, and product boundaries. > - Existing guidance does not provide one complete procedure for this work. > - This pull request adds a native Runner skill, detailed playbook, and working templates. > - Contributors can use them to define a release, implement the integration, and verify the delivered result. ## Linked Issues or Issue Description **Issue type** Missing documentation. **Where is the issue?** Native Runner contribution guidance in `doc/DEVELOPING.md` and `packages/paperclip-runner/README.md`. **What's wrong?** Contributors must reconstruct the requirements, protocol mapping, normal setup, lifecycle rules, packaging, and acceptance process when they add a harness. The legacy direct-adapter recipe covers a different execution path. **Suggested fix** Add a self-contained native Runner procedure. Include capability research, a finite release contract, implementation boundaries, reproducible tests, normal installation, and release evidence. State each lesson directly so contributors need no private chats, tasks, or evidence archives. The duplicate search found no existing playbook PR. ## What Changed - Add `create-runner-adapter` for new native Runner harness integrations. - Add a detailed playbook based on the Pi, Cursor, and Copilot integration lessons. - Cover model and instruction admission, connection setup, rich events, durable interactions, recovery, cancellation, packaging, and Product E2E acceptance. - Add templates for scope, capabilities, attempts, failures, jobs, and release handoff. - Link the procedure from the development guide and Runner README. - Keep legacy adapters as a compatibility boundary. ## Verification - `git diff origin/master --check` passed. - `node scripts/validate-doc-links.mjs` passed from `packages/paperclip-runner` and checked 53 documentation files. - All 71 Markdown links in the new reusable files resolve to repository paths or public provider documentation. - Skill YAML, naming, description, and scaffold checks passed. - A private-reference audit passed for the reusable skill, playbook, and templates. - The legacy direct-adapter skill matches the base version. - Full local runtime checks were attempted with pnpm 9.15.4. `pnpm -r typecheck` stopped because Node type definitions are missing. `pnpm test:run` and `pnpm build` stopped because `cli/node_modules/tsx` is missing. This checkout has no installed dependencies. These commands did not establish runtime test or build results. - Full remote CI passed on commit `88b4fc4832d95807da1f5e48029a183b3dc62b6c`: build, typecheck, general and serialized server tests, Runner checks, browser tests, and the canary dry run. [CI run](https://github.com/paperclipai/paperclip/actions/runs/39262297681). - All 55 applicable checks passed. The two Storybook checks were skipped for this change. - Greptile gave the current commit 5/5. There are no review comments or unresolved threads. ## Risks Low risk. The procedure can drift as the Runner contracts and test entry points change. It records its source baseline and tells contributors to inspect current declarations. Historical lessons describe failure patterns; they do not certify current binaries. ## Model Used OpenAI Codex with `gpt-6-astra` and `gpt-6.1-sol`, both using xhigh reasoning. Context-window sizes were not exposed. The models used repository search, terminal tools, and file edits. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used, including available version and capability details - [x] I have checked ROADMAP.md and confirmed this documentation does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs - [x] I have described the issue in this PR using the documentation issue template fields - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket or instance details - [x] I have run the relevant local documentation checks; they pass. Blocked runtime checks are recorded above - [x] I have considered tests for this documentation-only change and validated its links and skill metadata - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented the risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before merge Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-21 08:22:20 +02:00 · 2026-10-19 08:50:44 -05:00
1 parent 98ca6ff7c5
commit 56e09cd19f
@@ -0,0 +1,86 @@
+---
+name: create-runner-adapter
+description: >
+  Add a new harness/provider integration to native Paperclip Runner, from
+  requirements and protocol research through implementation, packaging, and
+  real end-to-end acceptance. Use for new Runner adapters; legacy direct
+  adapter authoring and migration are outside this skill.
+---
+
+# Add a native Runner adapter
+
+Use this skill to add a harness to the **new, native Paperclip Runner** execution
+system. Here, adapter means the provider translation and launch integration
+behind a Runner harness driver. Preserve existing direct adapters and historical
+records as compatibility constraints; do not route this work through their
+package-authoring recipe or migrate agents as a side effect.
+
+The complete procedure is the [Runner integration playbook](../../../doc/RUNNER-INTEGRATION-PLAYBOOK.md).
+Use its [working templates](../../../doc/runner-integration-templates.md) for the
+release contract, capability inventory, acceptance ledger, and failure record.
+These repository files contain the decision rules and test criteria themselves.
+No private chat, task, prior campaign, or separate private repository is required.
+Source links identify code to inspect, not evidence that a new integration passed.
+
+If this skill was installed separately, locate the target checkout from the
+current workspace or `git rev-parse --show-toplevel`. Verify it contains
+`packages/paperclip-runner` and `tests/runner-e2e`; resolve `doc/` paths there.
+Follow that checkout's `AGENTS.md` and scoped rules before editing.
+
+## Work sequence
+
+1. **Define a finite release.** Read playbook sections 1–3. Record the user
+   outcome, first environment, supported platforms, setup/auth, required
+   capabilities, selected acceptance model, spending limits, and delivery target.
+   Keep the complete capability inventory separate from the release requirements.
+2. **Investigate the actual harness.** Use section 4. Inspect versioned official
+   docs, native source or shipped code, real protocol traffic, and account/model
+   behavior. Track every capability through native harness, wrapper, Runner,
+   controller, and UI. Distinguish documented, observed, and qualified support.
+3. **Prove one complete slice.** Use sections 5–7. Establish exact model and
+   instruction admission, scoped tools and credentials, then complete one real
+   Paperclip task with independently verified output and physical cleanup.
+   Exercise ordinary connect/save/reopen/Test/reconnect in the selected environment;
+   check database constraints and account/model routing as well as driver execution.
+4. **Complete the promised behavior.** Use sections 8–10. Preserve rich events,
+   durable human decisions, callback identity, recovery, cancellation, and terminal
+   settlement. Keep provider-specific semantics in provider code.
+5. **Prove delivery.** Use section 11. Exercise clean installed packages and the
+   agreed execution environment; verify the complete runtime closure and immutable
+   image identity. A source checkout or startup probe is insufficient delivery proof.
+6. **Qualify the release.** Use sections 12–15. Begin with deterministic and real
+   dependency checks, then run exact Product E2E cells through browser/server/DB/
+   Runner/provider. Section 12 includes a repository-contained test-authoring path.
+   Freeze the candidate and retain original failures, cost coverage, and cleanup.
+7. **Hand off an operable result.** Use section 16 and the templates. Account for
+   repository checks, exact passing evidence, remaining limits, installation,
+   operation, and rollback. Keep merge, publish, deploy, and canary claims distinct.
+
+## Rules that must survive implementation pressure
+
+- A qualification model is a repeatable test choice, not a product allowlist.
+  Verify the caller's exact model at the native boundary before the first prompt.
+- Instruction transport success is insufficient: bind exact content on create
+  and load, reject missing acknowledgments, and verify resulting behavior.
+- A native question, semantic question, plan, and permission request are distinct
+  contracts. Preserve identity and prove delivery; queued does not mean answered.
+- Same conversation does not establish same process or recoverable pending input.
+  Stop must reach the owned native work and prevent late callbacks from reviving it.
+- A screenshot, model-authored success claim, or green unit suite cannot replace
+  independent file/artifact/state/cleanup checks through the promised user path.
+- Diagnose the first divergence across arguments, transport, files, persistence,
+  and grader before another paid attempt. Preserve the original grade.
+- Changes after a pass require an evidence-impact decision. Old results retain
+  their original runtime, model, environment, and fixture identity.
+- Prove spontaneous native-child loss and bounded shutdown, not only cleanup
+  invoked by a test. Preserve durable expiry for each supported request kind.
+- Establish the concurrency being graded: attached work must overlap finish;
+  a Stop case must reach real pending work before its observation window closes.
+- Keep campaign evidence, budget history, and candidate/publication state durable
+  across interruptions. Reconcile shipping dependencies before final qualification.
+- Preserve legacy behavior through focused regression checks. Legacy authoring,
+  runtime conversion, and migration are not part of adding a native integration.
+
+Keep new guidance shareable: put the complete lesson and reproducible acceptance
+condition in tracked documentation. Do not substitute personal session links,
+private task URLs, inaccessible evidence archives, or workstation-specific paths.
@@ -1770,6 +1770,16 @@ Legacy CLI paths that put prompts in command-line arguments (Gemini, Grok, Kimi,
 Pi, and Hermes) still have argument-size limits. ACP turns, SDK requests, and
 CLI paths that use stdin avoid that separate limit for the wake prompt.

+## Adding a native Runner harness
+
+To add a harness/provider to the new native Paperclip Runner, use
+[create-runner-adapter](../.agents/skills/create-runner-adapter/SKILL.md), the
+[Runner integration playbook](RUNNER-INTEGRATION-PLAYBOOK.md) and its
+[working templates](runner-integration-templates.md). They cover requirements,
+protocol research, implementation, packaging, and real end-to-end acceptance.
+Existing direct adapters remain a compatibility boundary; their authoring and
+agent conversion are outside this playbook.
+
 ## Paperclip Runner Adapter Conversion

 The experimental Paperclip Runner offers native Codex, OpenCode, and **ACPX
@@ -0,0 +1,223 @@
+# Native Runner integration working templates
+
+Use these with the [Runner integration playbook](RUNNER-INTEGRATION-PLAYBOOK.md).
+They cover adding a harness/provider to the **new, native Paperclip Runner**.
+Legacy direct-adapter authoring and migration are outside scope; existing behavior
+is preserved through compatibility checks. The templates require no historical
+chat, private task, evidence archive, or separate private repository.
+Copy only the sections needed for the current integration. Replace placeholders;
+these are documentation templates, not executable schemas or proof of completion.
+Repository plans belong in dated `doc/plans/YYYY-MM-DD-<harness>-integration.md`
+files. A Paperclip issue's requested plan belongs in its plan document.
+
+## 1. Release contract
+
+```markdown
+# <Harness> integration — <date>
+
+Owner:
+Source baseline / branch / prerequisite:
+Requested finish line: <reviewable PR / merged experimental / staging / production>
+Authorized actions and spending:
+
+## User outcome
+
+<The ordinary task, actual execution location, and inspectable result.>
+
+## Scope
+
+- Native Runner harness driver, provider translation, and transport:
+- First acceptance environment:
+- Supported OS/architectures and required proof level for each:
+- Authentication/setup path:
+- Runtime-auth versus repository/MCP connection method, ownership, and grants:
+- Save/reopen/Test/reconnect/revoke proof and selected probe environment:
+- Caller-selected model/mode policy:
+- Acceptance model/reasoning/mode, separately from product policy:
+- Required capabilities and applicable acceptance cell IDs:
+- Deferred/unsupported capabilities and product treatment:
+- Accounting requirements and unknowns:
+- Campaign/account/key/infrastructure/review limits:
+- Retry policy and cleanup obligations:
+- Publication/deployment/merge authority:
+
+## Architecture
+
+- Provider-owned translation and launch code:
+- Shared contracts that need a new provider-neutral capability:
+- Instruction/tool/credential admission:
+- Recovery identity and allowed refreshes:
+- Cancellation and terminal settlement:
+- Packaging and normal installation:
+- Database constraints, migrations, and generated declaration/catalog sources:
+- Clean PR-layer builds and normal local/remote artifact resolution:
+- Existing native-provider and legacy-direct-adapter behavior preserved:
+
+## Acceptance and delivery
+
+<Link capability matrix, acceptance ledger, immutable evidence, operator recipe.>
+
+## Scope changes
+
+| Date | Decision and authority | New scope | Preserved exclusions/failures |
+| --- | --- | --- | --- |
+```
+
+## 2. Capability inventory
+
+Use one row per capability or meaningful field. “Implemented” and “qualified”
+are separate columns. An absence claim needs evidence at the relevant layer.
+
+| Capability/field | Exact harness/version/mode | Native evidence | Transport/wrapper | Runner mapping | Durable product/UI | Qualification | Loss/reason/next action |
+| --- | --- | --- | --- | --- | --- | --- | --- |
+| Example: active steering | `<version>` | `<method or observed absence>` | `<preserved/dropped>` | `<control mapping>` | `<visible action>` | `<attempt ID + environment>` | `<why unavailable or what remains>` |
+
+Allowed evidence labels: documented; source-observed; credential-free observed;
+authenticated observed; product-qualified; unverified; confirmed absent in the
+specified implementation. Do not collapse them into a single supported flag.
+
+## 3. Acceptance and attempt ledger
+
+Keep one row per required cell, with every attempt linked. A pass on another
+profile/environment is historical evidence, not this row's pass.
+
+| Gate | Exact cell/environment | Required observation | Candidate identity | Attempts | Current verdict | Remaining action/owner |
+| --- | --- | --- | --- | --- | --- | --- |
+| Normal task delivery | `<catalog ID>` | `<native completion + task Done + verified download + cleanup>` | `<candidate record>` | `<failure 1, pass 2>` | `<pass/fail/unattempted/blocked>` | `<specific action>` |
+
+Keep deferred capabilities in the release contract, outside the denominator.
+Keep cleanup failures visible even when task assertions passed. Never calculate
+the denominator by counting only attempted or successful cells.
+
+Report totals explicitly: `<passed> passed, <failed> failed, <unattempted>
+unattempted, <blocked> setup-blocked of <required>`. Record whether each failed
+attempt reached its intended boundary; a startup failure did not exercise Stop.
+
+Example candidate record:
+
+```yaml
+candidate: <stable campaign-local name>
+application_commit: <40-character SHA>
+runner_source_commit: <SHA if distinct>
+shipping_source_dirty: false
+profile_version: <value>
+profile_digest: <digest>
+provider_package_version: <name@version>
+wrapper_and_patch_digests: <manifest reference>
+dependency_lock_digest: <digest>
+daemon_and_provider_pack_digests: <manifest reference>
+public_package_digests: <manifest reference>
+image_digest: <immutable digest or not-applicable>
+environment: <local/SSH/Daytona + actual architecture>
+platform_proof: <native-live/native-startup/emulated-startup/build-only>
+model: <explicit exact ID>
+mode_and_reasoning: <exact settings>
+product_fixture_commit: <SHA>
+protocol_case_revision: <repository-relative case files + commit/digest>
+catalog_or_definition_digest: <digest>
+grader_revision: <revision>
+credentials: <binding identifiers only; never values>
+spending_bound: <authorization + verified cap + remaining reservation>
+shareable_evidence: <reviewer-accessible sanitized inputs, results, and reproduction steps>
+sensitive_evidence_store: <restricted location if needed; not a prerequisite for understanding the claim>
+durable_resume_record: <manifest, budget window/holds, active jobs, gates, cleanup; outside temporary storage>
+published_review_head: <exact commit or unpublished>
+shipping_base_and_dependencies: <base commit, PR ordering, required fixtures/installers>
+```
+
+## 4. Failure and retry decision
+
+```markdown
+### <Attempt ID> — <timestamp>
+
+- User outcome and target boundary:
+- Exact candidate / cell / environment:
+- Expected observation:
+- Actual observation and first divergence:
+- Was the target boundary reached?
+- Was required concurrency observed (pending input, attached child at finish)?
+- Did the observer capture native order, or only delayed projection/persistence?
+- Relevant native request/tool/turn identity (redacted where necessary):
+- Original grade and retained evidence:
+- Primary class: product / model-provider / grading-evidence / infrastructure
+- Supported cause, confidence, and alternative explanations:
+- Cleanup result and remaining resource obligations:
+- Measured cost / estimate / unknown / budget reservation:
+
+Correction:
+<Owning boundary and smallest general change; no fixture-specific product hack.>
+
+Cheap discriminating proof:
+<Real dependency or controlled replay; positive and negative cases.>
+
+Retry decision:
+<Changed input, existing authority, selected cell, new attempt ID, budget.>
+
+Evidence impact:
+<Which previous results remain relevant; which require fresh measurement.>
+```
+
+## 5. Active work ledger
+
+| Handle | Kind | Owner | Candidate/cell | Started | Next useful observation/deadline | Budget reserved | Cleanup |
+| --- | --- | --- | --- | --- | --- | --- | --- |
+| `<actual job/process ID>` | `<build/cell/review>` | `<owner>` | `<identity>` | `<time>` | `<expected state>` | `<amount/unknown>` | `<required action>` |
+
+After compaction, inspect actual handles and persisted state before restarting.
+Record superseded jobs and their disposition. A stale lock file or previous
+commentary is not evidence that a job is live.
+
+Before a long pause, persist this ledger, the candidate manifest, retained results,
+budget window, reservations, and cleanup obligations in a durable destination.
+Record secret references only. Recover them before spending or duplicating work;
+missing temporary files do not reset the campaign's budget or prove old outcomes.
+
+For a real publication approval gate, record the action, exact artifact identity,
+destination, existing authority, and whether it is current or superseded. Keep
+source publication, build, image publication, and credential transfer distinct.
+Do not introduce new approval requirements beyond the actual execution policy.
+
+## 6. Release handoff
+
+```markdown
+<Harness> is ready for <exact target> with <explicit limitations>.
+
+User-visible outcome:
+<What works and where it runs.>
+
+How to try it:
+<Normal install/setup, configuration, task action, result access, Stop/restart.>
+
+Evidence:
+- Shipping commit/package/image/profile:
+- Required Product cells: <passed/required>, environments, attempts:
+- Runner protocol results, reported separately:
+- Clean installation and actual platform proof:
+- Real authenticated canary, if required:
+- Cleanup and resource disposition:
+- Required CI/local checks, incomplete runs and retained failures:
+- Budget: measured / reserved / unknown:
+
+Limits:
+<Unverified models/platforms/capabilities, accounting and recovery constraints.>
+
+Rollout/rollback:
+<Fresh admission, active sessions, historical reads, artifacts, recovery.>
+
+Next action:
+<Exact remaining review/merge/publish/deploy action and its authority.>
+```
+
+Do not write “production ready” when only the source patch or protocol probe is
+ready. Do not write “all tests pass” when required cells were unattempted or the
+full command was interrupted. Link the concrete evidence instead of a local
+scratch directory. For Paperclip issue work, register the artifact/work product
+and link it before setting the final issue status.
+
+Before sharing, make every principle and conclusion understandable from the
+record itself. Include the failure mechanism, correction, reproducible test,
+expected result, and actual sanitized observation. Do not use private chat/task
+links, personal filesystem paths, or inaccessible campaign archives in place of
+that explanation. Code references should be repository-relative; external
+research references should be public official sources. Secrets and raw sensitive
+traces remain restricted and never enter the shareable skill or playbook.
@@ -1,5 +1,12 @@
 # Paperclip Native Runner

+To add a harness/provider to native Runner, start with
+[create-runner-adapter](../../.agents/skills/create-runner-adapter/SKILL.md), the
+[Runner integration playbook](../../doc/RUNNER-INTEGRATION-PLAYBOOK.md), and
+[acceptance/evidence templates](../../doc/runner-integration-templates.md).
+They cover the new Runner integration path; legacy adapters are preserved as a
+compatibility boundary.
+
 This package is the standalone development boundary for Paperclip's native
 runner protocol, process supervision, durable transport, provider drivers, and
 normalized session backends. Rust owns the production runner under `runner/`;