feat(runner): add remote execution substrate (#12638)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Runner gives native runs a durable and governed execution
path.
> - The current native path runs on the control-plane host.
> - Remote environments need an authenticated execution-target contract.
> - The contract must not change direct adapters or enable new runtimes
by default.
> - This pull request adds the remote execution substrate and Daytona
ingress.
> - The benefit is a bounded base for later remote runner transport
work.

## Linked Issues or Issue Description

Refs #12616.
Refs #12352.

**Subsystem affected**

Cross-cutting. This change touches runner transport, server
orchestration, plugin contracts, and shared settings.

**Problem or motivation**

Native execution cannot resolve an authenticated runner ingress through
a remote environment. The server also lacks one provider-neutral
contract for remote execution targets.

**Proposed solution**

Add a default-off runner preview ingress capability. Add
transport-neutral runner connectivity. Add remote execution target and
lifecycle handling. Add a Daytona ingress implementation with redacted
credentials.

**Alternatives considered**

A provider-specific server path would duplicate orchestration and
authorization. A public endpoint without an environment contract would
weaken the trust boundary.

**Roadmap alignment**

This work supports the Cloud and Sandbox agents milestone. It also
supports self-healing runs and governed tool access.

## What Changed

- Added execution-target traits for local, SSH, and sandbox
environments.
- Added plugin RPC contracts for runner ingress endpoints.
- Added authenticated Daytona preview ingress.
- Added transport-neutral PRP outbound connections.
- Added remote runner artifact verification and fail-closed provider
selection.
- Added bounded native session resume, cancellation, and lifecycle
recovery.
- Preserved Codex-only selection for fresh experimental runner starts.
- Preserved all direct adapter execution and finalization paths.
- Removed stale Pi provider-pack requirements that security review
rejected.
- Kept the rollout controls off by default.
- Did not change pnpm-lock.yaml, Cargo, database migrations, or GitHub
workflows.

## Verification

- GitHub Actions will run the repository test, typecheck, build,
security, and policy gates.
- Focused tests cover ingress validation, redaction, execution targets,
remote lifecycle, cancellation, resume, and legacy adapter selection.
- Local tests were not run. The requested verification policy uses
GitHub Actions for this series.
- `git diff --check origin/master...HEAD` passes.
- The diff contains 52 files.

## Risks

- Remote execution crosses a trust boundary.
- The implementation validates target capabilities, artifact digests,
provider-pack pins, and connection metadata.
- The feature remains default-off.
- Fresh native selection remains Codex-only.
- Existing direct adapters remain on the legacy path.
- This PR does not yet make remote Codex runnable. The next PR adds the
Rust WSS and TLS transport.

## Model Used

OpenAI Codex with GPT-5.6. The work used high-reasoning agent mode,
repository tools, GitHub tools, and parallel code-audit agents.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with Fixes: / Closes /
Refs OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
This commit is contained in:
Dotta authored and GitHub committed 2026-09-01 01:29:06 -05:00
1 parent 209680409d
commit 0a422fda52
52 files changed
+6920 -223

No files matched your search

+32
View File
@@ -19,9 +19,41 @@ All environment variables that Paperclip uses for server configuration.
| `PAPERCLIP_DEPLOYMENT_MODE` | `local_trusted` | Runtime mode override |
| `PAPERCLIP_DEPLOYMENT_EXPOSURE` | `private` | Exposure policy when deployment mode is `authenticated` |
| `PAPERCLIP_API_URL` | (auto-derived) | Paperclip API base URL. When set externally (e.g., via Kubernetes ConfigMap, load balancer, or reverse proxy), the server preserves the value instead of deriving it from the listen host and port. Useful for deployments where the public-facing URL differs from the local bind address. |
| `PAPERCLIP_RUNNER_PUBLIC_URL` | (unset) | Explicit `wss://` base URL used only when a remote `paperclip_runner` target dials Paperclip directly. Paperclip appends `/api/runner/v1/connect/<runId>`; the reverse proxy must forward WebSocket upgrades for that route. This value is never inferred from request headers. Daytona ignores it and uses provider ingress. |
| `PAPERCLIP_RUNNER_CA_BUNDLE_PATH` | (unset) | Optional PEM CA bundle for direct runner WSS. Platform roots remain enabled. There is no insecure TLS bypass. |
| `PAPERCLIP_RUNNER_REMOTE_BINARY_PATH` | (host build) | Host-local path to a `paperclip-runnerd` artifact built for the remote target OS and architecture. Required when Paperclip and the remote sandbox do not share a compatible platform; build metadata and the required transport mode are verified before launch. |
| `PAPERCLIP_RUNNER_REMOTE_CODEX_PATH` | (unset) | Optional host-local path to a Codex executable built for the remote target OS and architecture. For remote Codex-backed runners, Paperclip stages and verifies this executable beside `paperclip-runnerd`. |
| `PAPERCLIP_RUNNER_REMOTE_CODEX_NPM_SPEC` | (unset) | Optional pinned npm package spec (for example, `@openai/codex@0.148.0`) installed inside each fresh remote lease when its Codex harness is not baked into the sandbox image. Mutually exclusive with `PAPERCLIP_RUNNER_REMOTE_CODEX_PATH`; Paperclip verifies the installed executable before starting `runnerd`. |
| `PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH` | (unset) | Host-local path to the immutable provider pack built by `pnpm --filter @paperclipai/paperclip-runner build:provider-pack`. The pack includes its target-built Node 24.11 runtime, locked production dependencies, OpenCode proxy/executable, and ACPX sidecar. Remote OpenCode and ACPX fail closed without it. A preinstalled pack is accepted only when its complete digested manifest matches this build-owned pack; otherwise Paperclip stages this pack into the sandbox. |
| `PAPERCLIP_HIDDEN_SETTINGS` | (unset) | Comma-separated settings surfaces to hide from the UI and floor at the API, for operators hosting Paperclip for others (managed cloud, internal shared server). See [Hiding settings surfaces](#hiding-settings-surfaces). |
| `PAPERCLIP_SETTING_DEFAULTS` | (unset) | JSON object replacing the schema default of selected instance settings, for hosting operators. See [Operator setting defaults](#operator-setting-defaults). |
Daytona connectivity for `paperclip_runner` uses authenticated preview WSS and
is gated by the instance experimental setting `enableRunnerPreviewIngress`
(default `false`). The setting has no effect on legacy adapters or callback
bridges.
### Preinstalled remote runner images
Remote sandbox images may preinstall `paperclip-runnerd`, `codex`, and the
provider pack at `/opt/paperclip-runner/provider-pack` instead of
paying the upload and npm-install cost on every fresh lease. Put both executable
names on the sandbox user's `PATH`; `$HOME/.local/bin` is checked explicitly
before `PATH`. Paperclip verifies runner build metadata, the selected PRP
transport capability, Codex startup, the provider-pack digest, exact harness
pins, Node compatibility, and packaged bridge digests before linking artifacts
into the run-specific runtime directory. A missing or incompatible executable falls back
to `PAPERCLIP_RUNNER_REMOTE_BINARY_PATH` and
`PAPERCLIP_RUNNER_REMOTE_CODEX_NPM_SPEC` (or
`PAPERCLIP_RUNNER_REMOTE_CODEX_PATH`) without changing the selected transport.
OpenCode and ACPX instead fall back only to
`PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH`; they never start a provider
process on the Paperclip host for a remote target.
The Daytona environment editor's **Configure image** action can create this
image without a separate container registry: install the executables in its
setup sandbox, finish setup, and Paperclip captures and promotes the resulting
Daytona snapshot for future leases.
### Hiding settings surfaces
`PAPERCLIP_HIDDEN_SETTINGS` takes keys from the registry in