Files
PaperClipAI/maps/feature-map/runs-adapters.md
T
DottaandPaperclip 063a498b54 Bound native OpenCode assigned-tool catalog projection (#17740)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Native agents use assigned connection tools through a governed
gateway.
> - OpenCode received the gateway catalog directly, outside the bounded
server tool projection.
> - A large catalog can exceed a provider grammar compiler's capacity
even when it fits the transport limits.
> - Synthetic requests reproduced the Fireworks grammar timeout on the
configured DeepSeek model.
> - This pull request gives OpenCode two assigned-tool discovery and
call schemas through the existing server relay.
> - The gateway keeps the same grants, validation, approvals,
revocation, and audit rules.

## Linked Issues or Issue Description

**What happened?**

A native OpenCode run could fail during provider grammar compilation
before its first assigned-tool call. OpenCode advertised the full
assigned gateway catalog. A synthetic catalog of 741 tool schemas
reproduced `Grammar processing took too much time` on Fireworks through
OpenRouter with `deepseek/deepseek-v4.1-flash-1017`. The same model and
backend accepted a compact two-tool discovery/call catalog. This
reproduces the mechanism; it does not replay a historical private
request.

**Expected behavior**

Assigned-tool discovery must keep provider grammar size bounded. Every
call must retain the run's original gateway binding and permissions.

**Steps to reproduce**

1. Give a native OpenCode run a large assigned connection catalog.
2. Use a provider that compiles tool schemas into a constrained output
grammar.
3. Observe the full catalog in the provider tool surface before this
change.
4. After this change, observe only `paperclip_search_assigned_tools` and
`paperclip_call_assigned_tool` for assigned tools. Discover and call a
permitted tool through its pinned alias.

**Paperclip version or commit**

Source checkout based on current `master`.

**Deployment mode**

Native local or sandbox execution with OpenCode 1.18.34.

Related work: Refs #14258 (discovery paging), #16678 (local adapter MCP
launch), and #17688 (gateway capability advertisement). This change
covers native OpenCode catalog projection.

## What Changed

- Route native OpenCode assigned tools through the existing server-owned
authenticated relay.
- Use bounded discovery/call schemas for OpenCode, including small
catalogs. Preserve transport capacity checks.
- Keep assigned gateway credentials on the controller. Ignore gateway
and credential values in tool arguments.
- Preserve exact declared MCP operation names through admission and
driver dispatch. Resolve transport prefixes only against the closed
catalog. Reject unknown or shortened names.
- Refresh former OpenCode direct-catalog and shortened-name checkpoints.
Keep the existing fingerprints for other providers.
- Add local and remote wiring, catalog capacity, alias, scope,
work-mode, approval, and revocation tests.
- Allow the native test files to use a validated dedicated external test
database when embedded PostgreSQL is unavailable.
- Document the assigned-tool execution contract.

- Bound OpenCode discovery pages to 32 KiB and large-schema chunks to
16K characters. Keep other harness budgets unchanged.
- Test medium pages and escaped large schemas through the actual HTTP
MCP bridge on the explicit Runner testing surface.

## Verification

- Red before the catalog fix: OpenCode did not create the server relay,
and forced compact delivery advertised 1 or 741 direct schemas.
- Red before the alias repair: HTTP MCP changed declared operation
names, and the driver sent `search_assigned_tools` instead of the
authorized `paperclip_search_assigned_tools`.
- Current repair tests: 112 OpenCode/proxy tests pass, plus one actual
Rust/proxy/HTTP MCP regression. Two checkpoint regressions fail with the
former scoped revision and pass with the new revision. Other providers
keep their prior fingerprints.
- The combined QA candidate with this exact PR head passes all 916
native tests and all 113 transport tests on supported Node 24.22. An
earlier Node 26.4/macOS run passed the 113 assertions but had an
uncaught networking error. That non-clean run remains recorded; its
stack matches [the upstream Undici
defect](https://github.com/nodejs/undici/issues/5544). No production
workaround or error suppression was added.
- `pnpm -r typecheck`, `pnpm build`, and `git diff --check` pass on
`bc41ba72be2195eabeb0d7727227ee0cd02e8532`. The maps structural check
passes.
- The local `pnpm test:run` attempt stopped after host embedded
PostgreSQL initialization failures. The relevant database suites pass
with isolated PostgreSQL 18. All repository CI gates, including
aggregate `ci / verify`, pass on the current head.
- Synthetic live provider comparison: 741 invented schemas produced the
exact Fireworks grammar timeout in 20.9 seconds. Two compact invented
schemas produced a discovery call in 585 ms on the same model and
backend.
- The first hosted QA attempt exposed the alias failure described above.
On the patched source, a real local Rust runner → proxy → OpenCode
1.18.34 → exact DeepSeek smoke completed discovery, one synthetic
gateway read, and `paperclip_finish`, then emitted `turn/completed`. It
used the production compact projection and unchanged Rust allowlist.
- Two earlier local smoke attempts exposed missing terminal authority
and completion-contract fields in the smoke harness. Those failures
remain recorded. The corrected ordinary bindings produced the clean
result.
- Fresh Greptile: 5/5 on `bc41ba72be2195eabeb0d7727227ee0cd02e8532`,
with no new actionable findings or unresolved threads.
- Hosted QA publication and lifecycle acceptance remain pending. The
separate deployed-agent `Paperclip Review` check is also pending; it
follows a task that failed on the earlier deployed runtime.

- Local OpenCode/OpenRouter acceptance at `bc41ba72`: 530 synthetic
tools were authorized and installed. A simple model-ID reply completed
in 17.3 seconds with no app calls. A discovery task completed in 37.6
seconds, used the search/call relay, made one upstream call, and
returned the exact synthetic value. This is local source acceptance, not
a staging sandbox timing comparison.
- The large-result review finding has a failing-before/passing-after
HTTP MCP regression. Five medium schemas now page without invalid JSON.
A large schema with escaped text reconstructs exactly through bounded
chunks. All 35 assigned-tool tests and the server typecheck pass after
this follow-up.
- Final head `d84ce4856a6e65a35ad5fb8b561492598b525f4e`: all CI gates
pass. Greptile and Paperclip Review both rate the change 5/5. No review
threads remain open. The final full build, server typecheck, and 35
assigned-tool tests pass locally.

## Risks

- OpenCode uses discovery before it calls an assigned tool, including
for small catalogs. This can add a model turn.
- An existing OpenCode provider checkpoint rotates once because it
retains the former tool declarations. Paperclip task history remains
available.
- The historical filtered catalog was not captured. The synthetic
comparison proves the failure mechanism, not the exact cardinality of an
earlier private request.
- No model, provider route, permission, or automatic retry policy
changes.

## Model Used

The bounded-result follow-up also used OpenAI Codex (GPT-6 family). The
exact backend model ID and context-window size were not exposed for that
session. It used reasoning, code execution, and tool use.

OpenAI `gpt-6.1-sol` in Codex, with reasoning, tool use, code execution,
and source inspection. The runtime does not expose the context-window
size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-26 10:49:23 -05:00

248 lines
16 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
navigation: {"section":"Agents and reusable capabilities","sectionOrder":30,"order":20}
---
# Runs, harnesses, and model accounts
Operators choose an agent harness and model account, inspect execution history and transcripts, and distinguish a working configuration from a successful task result.
Feature ID: `runs-adapters`.
Implementation: [adapter manager](../../ui/src/pages/AdapterManager.tsx), [run host](../../ui/src/pages/AgentDetail.tsx), [configuration](../../ui/src/components/AgentConfigForm.tsx).
Agent access: [companion](../agent-access-map/runs-adapters.md).
## Sub-features
- `availability`: `runs-adapters/availability` — discover installed adapters and available models/accounts.
- `validation`: `runs-adapters/validation` — test an adapter in the selected execution environment and surface authentication failures.
- `runs`: `runs-adapters/runs` — inspect run status, events, output, costs, and the source task.
- `session`: `runs-adapters/session` — preserve or reset execution context through supported lifecycle actions.
## How to get to it (user POV)
### `adapter-setup`
Open an agent’s model/adapter configuration or the instance adapter settings page when permitted.
### `run-history`
Open an agent run at `/agents/:agentId/runs/:runId` from the agent or task activity.
The agent header has one Run now action. Administrators can enable
Capture raw provider traces under the agent configuration’s Debugging section
for future runs. The header shows Raw tracing on while that setting is enabled.
For latency diagnosis with an operator OTLP endpoint, search the hashed run ID
without restricting the root span name. Compare workspace preparation with its
`workspace_<step>` intervals. Measure controller readiness separately through
`controller.startup.<phase>`. A recovery pause or deployment drain is not model
execution time. See the [tracing contract](../../doc/observability.md#controller-preparation-phases).
These source-level diagnostics do not establish a live response-time budget.
`task.time_to_first_token` measures accepted ingress through the first committed
text that the task transcript can show, including thinking and completed
responses. It includes preparation and startup, and excludes browser latency.
Inspect `native.controller.<phase>` for backend construction, assigned-tool
discovery and post-turn cleanup. Compare those intervals with the actual model
turn; do not count a nested discovery interval twice.
## Driving it with Playwright and the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Record adapter, model, account, native/legacy runtime, and environment. Capabilities vary; one harness result is not a matrix-wide qualification.
### `session`
For `runs-adapters/session`, complete a task in a disposable warm sandbox and
retain its saved workspace file. Let the warm session close normally and verify
that the retained lease has a complete, lease-bound harness backup before
injecting sandbox loss. Removing a still-warm sandbox before that checkpoint
exercises the separate acquisition safety gate, which must remain blocked.
Use a board-authorized session reset, then remove only that fixture's inactive
provider sandbox. Send a follow-up that
reads the retained file and appends one new line. Verify a replacement sandbox,
a new provider session, the exact file contents, successful finalization and no
stranded task lock. Send another follow-up and verify ordinary session reuse.
Repeat without resetting the session: its replacement must restore verified
history or remain blocked. Missing backup state must not silently start an
existing session again. The [executor controls](../../server/src/services/native-runtime/native-session-executor.test.ts)
cover the bootstrap identity, activity and atomic directory claim; those
controls do not establish live provider acceptance.
### `adapter-setup`
Automated: [adapter validation routes](../../server/src/__tests__/agent-adapter-validation-routes.test.ts) and [model refresh](../../server/src/__tests__/adapter-model-refresh-routes.test.ts) cover server behavior.
Claude native setup uses the existing Anthropic API-key or subscription connection
flow. Record the selected account separately from successful execution; unavailable
models/authentication and unsupported Legacy-only options must fail visibly.
Claude CLI tests and runs also prepare a compatible private executable when a
managed sandbox image's default CLI is too old for the selected model. Verify
the command used for the hello test and task, reuse of a compatible private
installation, and a blocking failure when installation cannot establish a valid
version. Local, SSH and explicit custom commands retain operator management.
Cloud onboarding must not offer shell commands the customer cannot run.
Manual: Select the intended adapter and account, refresh model choices, and run its environment test. Save, reload, then execute a small task. Repeat with a deliberately unavailable account and verify the failure remains visible.
For Legacy Codex, assign a service secret such as `SLACK_BOT_TOKEN` through agent
configuration and run a task that checks its presence without printing its value.
Check both ACP and CLI execution. Remove the binding and repeat on the next run.
The assigned secret must reach shell tools; an unassigned host token and Codex's
own login keys must remain unavailable. Verify the same delivery without an
agent identity. Shell-policy metadata must contain names rather than values.
Choose Codex as the harness; the UI displays its name/icon for both runners.
Advanced identifies the automatic default and offers Paperclip Runner or Legacy.
New Codex agents default to the native runner on supported macOS or GNU/Linux
targets; the [shared selection contract](../../packages/shared/src/agent-runner.ts)
and [runtime prerequisites](../../packages/paperclip-runner/README.md) define the
platform/libc boundary. Windows and Alpine/musl remain Legacy. Claude follows
its qualified rollout described above. Disabled adapters, external overrides, and selected-target
qualification also affect availability.
Choose OpenCode with the native-runner experimental flag off on a qualified
macOS or GNU/Linux x64 target. Linux ARM64 must retain Legacy because its pinned
OpenCode runtime is not qualified. Advanced should show Paperclip Runner as the
automatic default on qualified targets. Save an inactive draft without a model,
resume it, select an explicit `provider/model`,
test, finish setup, and reload. Verify the saved provider is `opencode`. Repeat
with explicit Legacy and an existing Legacy agent; neither should migrate on an
unrelated edit. Check that saved native permission/lifecycle choices survive
resuming setup. Custom commands, arguments, variants, and inherited permissions
must give Legacy remediation when native is selected. A live task with the
selected account is separate proof from these setup and persistence controls.
Repeat with Pi on macOS or GNU/Linux x64 after explicit operator setup of the
pinned runtime (`paperclipai runtime setup pi`). Linux ARM64 retains Legacy.
Resume an inactive model-free draft, connect an account, choose a `provider/model`,
and finish setup with the generic native flag off. Verify Pi branding and native
ACPX Pi execution by completing a simple assigned task. Preserve saved native
thinking/permission/lifecycle settings on resume and unrelated edits. Native
thinking supports `off`, `low`, `high`, `max`; legacy aliases must give remediation,
and returning to Legacy must reject settings it cannot honor.
On a fresh forced-Kubernetes instance without an environment row, inspect discovery
and create with automatic selection. Both must retain Legacy without provisioning;
an explicit Paperclip request must give target-qualification guidance. Begin the
ordinary task to exercise the existing lazy environment path. This live journey
remains separate from the resolver's source controls.
For sandbox defaults, follow the [environment probe](./execution-environments.md#environment-settings)
recipe first. Discovery/creation read company-bound immutable-image evidence
without allocating a target. A mutable image or snapshot name does not establish
native compatibility. Then test the actual selected runtime with its selected
account/model: a legacy CLI or authentication probe alone cannot establish native
readiness. For an image-provided Codex harness, verify both final executables
after concurrent artifact preparation on an eligible provider. Repeat with a
failed runner probe and confirm the harness work has settled before failure
returns. Providers without concurrent sync and explicit harness installs remain
serial. The [real-shell artifact controls](../../server/src/services/native-runtime/native-artifact-preparation.test.ts)
cover these source contracts; they do not establish live reply latency.
For remote native Codex with managed GitHub access, inspect the controller
preparation spans for overlapping GitHub setup and assigned MCP discovery.
Provider startup must wait for both. Force either preparation to fail while the
other is pending; confirm that the pending operation settles and run-owned access
is released before failure returns. Repeat a warm follow-up and retain its actual
reply time. The [executor controls](../../server/src/services/native-runtime/native-session-executor.test.ts)
cover ordering, failure drain and credential projection; they do not qualify a
live response-time target.
For native OpenCode with assigned connections, inspect the provider tool surface
on both local and sandbox execution. It should contain bounded assigned-tool
search/call schemas. Discover an allowed tool, call its returned alias, revoke
its grant, and confirm discovery and execution enforce that revocation. Repeat
in planning/ask mode with a write tool and confirm denial. The controller must
retain the assigned gateway URL and token. An older direct-catalog OpenCode
checkpoint must refresh without changing another provider's fingerprint. The
[assigned-tool](../../server/src/services/native-runtime/assigned-mcp-tools.test.ts)
and executor controls cover source behavior; live sandbox discovery and a real
call remain separate acceptance evidence.
Missing dependencies, failed authentication, invalid models and
incompatible custom settings must remain actionable errors without silent runner
switching. Saved unchanged execution can still be tested without applying new
selection policy, but runtime/account validation must still run.
Verify an automatic Codex hire and an explicit Legacy control through a useful
file artifact, follow-up and reload. Preserve Codex reasoning effort and account
isolation; retain explicit Legacy permission choices, including false. Reopen an
existing agent, make an unrelated edit, and confirm both its persisted runner and
next run stay unchanged. Use [hiring/import preservation](./agents.md#hire-agent)
for approval and portability controls.
Test native Codex with a saved subscription whose setup auth file is a managed
link, as well as a regular file and an API-key account. Setup must seed the
isolated probe from the checked account without exposing credentials or changing
its saved runner. With subscription authentication, an unbound configured
`CODEX_HOME` with linked credentials must fail setup with managed-account or explicit Legacy guidance; it must not claim
readiness when task isolation would omit the file. Dangling or incorrectly owned
credentials must also fail clearly. A selected `CODEX_API_KEY` or
`OPENAI_API_KEY` takes precedence over an unrelated subscription file. Test valid,
dangling and invalid subscription links with that selected key; setup must test
the key without reading or refreshing the unrelated account. The private probe copy must be removed after confirmed retirement following success,
failure or cancellation. Unconfirmed shutdown remains a cleanup failure.
Exercise an unavailable local daemon and a preparation failure before probe
handoff. No owned temporary credential copy may remain, and original account
files must be unchanged. A probe whose shutdown is unconfirmed must retain its
recovery state; do not treat it as a preparation-only failure.
Existing setup controls use fake credentials; real account execution remains
a separate acceptance requirement.
### `run-history`
Automated: [live run routes](../../server/src/__tests__/agent-live-run-routes.test.ts) and [run event sequencing](../../server/src/__tests__/heartbeat-run-event-sequencing.test.ts) cover run data contracts.
Manual: Start a disposable task, open its running transcript, and follow it to a terminal outcome. Reload and compare task/run IDs, status, output, and cost. Verify a failed run remains distinguishable from a completed run with useful output.
For Legacy OpenCode on a disposable Linux sandbox, run a Bash tool that reads
`/proc/self/oom_score_adj` and its parent's score. The tool should report `1000`;
OpenCode should retain its original score. Repeat with an operator `BASH_ENV`
and confirm that it takes precedence. Repeat with a linked operator plugins
directory; it must remain unchanged, and the uploaded runtime must contain a
real plugins directory with the guard and existing plugins. Under controlled memory pressure, record
the cgroup OOM counters and verify that a killed tool can return an error while
the agent continues. Local and SSH execution must keep their original shell
environment. See the [adapter contract](../../docs/adapters/overview.md#opencode-sandbox-memory-pressure).
These controls do not guarantee that every memory exhaustion preserves the agent.
An explicit runner change starts a configuration revision and invalidates the
eligible session. Active runs, restart recovery and historical transcript parsing
retain their recorded execution configuration; do not infer their runner from a
later edit to the agent. Account/harness branding and execution identity are
different observations, both required in the evidence.
For an onboarding child-completion callback, verify actual session/turn events
and the source-correlated reply. A `succeeded` row alone is insufficient: the
literal `ea6c8eb5e16d61111968f633a66748b397b64a3e` managed attempt was marked
successful during startup without a provider turn. The recovery correction
preserves the existing validated Done-parent handoff; process-death and ownership
checks remain active. Its PostgreSQL controls are separate from the required
revised-source live onboarding and follow-up proof.
## Gotchas
- Model accounts are managed in Apps and inline agent account controls.
`/company/settings/connections` configures decision/fast-response models,
not the general model-account library.
- The run log is instance database data; it is not the opt-out first-party Telemetry system or operator-configured OpenTelemetry tracing.
- Authentication/model discovery does not establish tool, image, steering, or resumption capability parity.
- Use [recovery](./recovery.md) for interrupted runs and [execution environments](./execution-environments.md) for local/SSH/sandbox differences.
Native setup retirement joins owned startup before cleanup can be confirmed.
[Transport ownership controls](../../packages/paperclip-runner/src/live/runnerd-codex-transport.test.ts)
hold preparation or registration without launching a provider, then check bounded
close, rejected duplicate requests, no later launch and retention of unconfirmed
state. Readiness observes shutdown; resumed startup cannot reactivate ingress
after close. These are lifecycle controls, separate from installed provider proof.
The real daemon readiness and recovery controls require their existing native
fixtures in hosted CI; passing credential-free controls does not qualify them.
A primary setup timeout remains a failure even when teardown also fails.
For remote setup shutdown, verify that each durable-state read respects the
remaining suspension deadline. A late result cannot establish suspension or
queue a replacement command after that deadline. An intentional ingress close
must observe physical socket closure before retiring its connection owner and
releasing the endpoint; an unresponsive peer must not extend the setup budget.
The existing transport and ingress suites exercise these conditions without a
provider. Repeat the actual managed setup and useful task journey separately;
the literal `651a3de64fc943216cc414b64b1443809f9499da` Test attempt failed
cleanup before hire and retains its unconfirmed recovery state.