Commit Graph
12 Commits
Author SHA1 Message Date
Nicky LeachandPaperclip b4f302d040 feat(grok-local): copy a refreshed sandbox credential back to the host (#12696)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Paperclip runs agents through provider-specific adapters in local
and remote environments
> - A remote Grok run can refresh its credential inside its sandbox
> - The host copy can become stale when teardown discards that refreshed
credential
> - This pull request copies the refreshed credential back through a
locked, fail-closed teardown path
> - The benefit is that later Grok runs can use the refreshed host
credential without another login

## Linked Issues or Issue Description

Refs: #12618

**Agent or provider**

Grok local adapter.

**Why this adapter is useful**

A remote Grok run can refresh its access token during a run. Copying the
refreshed credential back to the host keeps later runs ready to use.

**How the agent is invoked**

Paperclip invokes the Grok local adapter through its remote subscription
run path. The adapter stages the company Grok home as a sandbox asset.

The change adds a copy-out step on the teardown path.

## What Changed

- `grok-auth-merge-decision.cjs` adds a host predicate in its own
process. It compares the whole `<issuer>::<uuid>` identity key of the
two files. It reads `expires_at` as an ISO-8601 string, an epoch-seconds
number, or an epoch-milliseconds number. It exits 10 to use the source,
20 to keep the destination, 21 when the expiry shape is unreadable, and
22 when the source expiry sits more than 400 days after the host clock.
It fails closed in every unclear case: an unusable side, a different
identity, an absent expiry, a tie, an unreadable expiry, and an
implausible expiry all keep the destination.
- `grok-auth-merge-decision.ts` adds a wrapper that runs the predicate
and maps the exit code to a typed result.
- `grok-auth-copyback.ts` adds `copyBackGrokAuth({ hostHomeDir,
readSandboxAuth, log, env })`. It locks on `hostHomeDir` with
`withDirectoryMergeLock`, stages the sandbox bytes into a private `0600`
temporary file, runs the predicate, and installs the file with an atomic
rename in the same directory. It keeps no backup of the displaced
credential. It leaves no temporary file on the success path, the keep
path, or an error path. On an error it logs the `errno` code only, then
re-throws.
- `execute.ts` adds a `restore` callback to the Grok `home` asset. The
callback takes the destination from
`resolveManagedGrokHomeDir(process.env, agent.companyId)`, never from
`env.GROK_HOME`. A copy-out failure does not fail the run.
- `package.json` updates the `build` script to copy
`grok-auth-merge-decision.cjs` into `dist/server/`, because `tsc` does
not copy a `.cjs` file.

**The credential shape this predicate reads**

A redacted sample of a real vendor credential answered four structural
questions. The answers hold no credential bytes, no account identifier,
no file path, and no timestamp value.

1. `expires_at` is present.
2. `expires_at` sits inside the value object, under the
`<issuer>::<uuid>` key. It is not a top-level field.
3. `expires_at` is an ISO-8601 string. It carries UTC time with a
trailing `Z` and six fractional-second digits.
4. A normal run rewrites `auth.json`. The value object carries a
`refresh_token` next to `expires_at`, so the client refreshes the access
token and rewrites the file.

## Verification

- [x] `pnpm vitest run packages/adapters/grok-local` — 114 tests in 12
files pass.
- [x] `pnpm --filter @paperclipai/adapter-grok-local typecheck` — clean.
- [x] `pnpm --filter @paperclipai/adapter-grok-local build` — succeeds,
and `dist/server/grok-auth-merge-decision.cjs` exists after the build.
- [x] Continuous integration is green on every check.

## Risks

The predicate keeps the host credential when identity, expiry, file
access, or freshness data is unclear. The copy-out path can log an error
and leave the run successful when it cannot install the refreshed
credential. The atomic rename and directory lock protect the host file
from partial writes and concurrent copy-out actions.

## Model Used

OpenAI GPT-5, current deployment. The exact runtime version and context
window are not exposed to this agent. The model used tool calls and code
inspection.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-01 18:27:00 -07:00
DottaandPaperclip 86ebdf842e fix(runner): keep agents running when app connections expire (#12670)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents can receive governed access to connected apps through the
runtime MCP gateway.
> - A connected app can become unavailable when its sign-in expires or
its health state needs attention.
> - The native runner treated that optional app state as a fatal runtime
setup error.
> - One unavailable app could therefore stop all unrelated agent work.
> - This pull request removes the fatal dependency and keeps the
available app assignment immutable.
> - The benefit is that an agent can continue its work while the stream
tells the user which app needs reconnection.

## Linked Issues or Issue Description

**What happened?**

An agent could not start a native run when one assigned app connection
was disabled, degraded, failed, or missing its secret. Runtime context
creation or MCP delivery threw an error before the agent could do
unrelated work.

**Expected behavior**

The run must continue without the unavailable app. Healthy assigned apps
must remain available. The stream must explain which app needs
reconnection. A changed assignment must not give a native run new access
after its immutable context is captured.

**Steps to reproduce**

1. Assign an MCP app connection to a Paperclip Runner agent.
2. Set the connection to a state that needs attention, such as
`degraded`.
3. Start a task run for that agent.
4. Observe that native runtime setup fails before the agent starts.

**Paperclip version or commit**

Reproduced from `ee2a19062`. The branch is rebased on `dda4dff64`.

**Deployment mode**

Local development from source with embedded Postgres.

No matching public issue or open pull request was found in the GitHub
search.

## What Changed

- Filter unavailable assigned app connections from the immutable native
runtime MCP snapshot.
- Keep healthy assigned connections and their tools in the snapshot.
- Replace the fatal native MCP availability check with an optional
stream warning callback.
- Withhold MCP delivery when the current assignment digest does not
match the captured native context.
- Prevent a warning delivery failure from stopping the agent run.
- Add regression tests for disabled, degraded, mixed healthy and
unavailable, and assignment-drift cases.

## Verification

- `pnpm exec vitest run
server/src/services/native-runtime/runtime-context.test.ts
server/src/__tests__/heartbeat-runtime-mcp-servers.test.ts` passes with
8 tests.
- `pnpm -r typecheck` passes.
- `pnpm check:token-gates` passes.
- `pnpm build` passes.
- `pnpm test:run` was attempted. Unrelated workspace runtime and
port-exposure tests failed on this macOS host. The same files also
failed when run without the changed MCP tests. The changed MCP tests
remained green. Clean GitHub CI is the final full-suite check.

## Risks

- Low migration risk. This change has no schema or API contract
migration.
- An unavailable app is absent from the run MCP surface until it is
reconnected and a later run captures it again.
- Assignment drift fails closed. The agent keeps running, but the
changed gateway is not delivered.
- This pull request does not auto-block the issue before the agent
decides that the app is required. It emits reconnect guidance in the
stream. The existing connection-request interaction remains the path for
a required app.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, `gpt-5.6-sol`, with high reasoning, repository tools,
code execution, and browser automation. The runtime did not expose the
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-01 10:48:18 -05:00
Dotta ee2a190626 Unify Paperclip Runner experimental controls (#12666)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Runner is an experimental execution adapter.
> - The adapter and its required sandbox ingress had separate settings.
> - A user could enable one setting and still have an unusable runner
configuration.
> - The runtime already makes one durable native or legacy decision for
each run.
> - This pull request uses that runtime decision for ingress
authorization.
> - The benefit is one clear opt-in with safe recovery for existing
native runs.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

This improves the experimental settings and transport authorization for
Paperclip Runner.

**Subsystem affected**

Cross-cutting. This change affects the React settings UI, shared
settings contracts, adapter utilities, and server runtime selection.

**Current behavior**

Settings shows separate Paperclip Runner and Runner Preview Ingress
controls. A user can enable the runner but leave required sandbox
ingress disabled.

**Proposed behavior**

Settings shows only Paperclip Runner. Its native runtime decision also
authorizes provider WebSocket ingress when the execution target requires
it. A persisted native run keeps its recovery transport after the
setting is disabled.

**Reason and benefit**

Paperclip Runner is one experimental capability. One opt-in removes an
invalid partial configuration and makes the rollout boundary easier to
understand.

**Breaking changes**

The Runner Preview Ingress card is removed. The old
`enableRunnerPreviewIngress` key remains accepted in stored settings and
managed configuration, but it has no server runtime effect. The public
adapter-utils input remains compatible through a deprecated alias.

**Additional context**

Refs: #12638, #12641, #12656.

## What Changed

- Removed the separate Runner Preview Ingress card from Experimental
Settings.
- Made resolved native runtime selection authorize required provider
ingress.
- Preserved ingress recovery for persisted native runs after the rollout
flag is disabled.
- Kept the old settings key and adapter-utils input as deprecated
compatibility contracts.
- Added focused UI, runtime policy, transport, stored-settings, and
managed-config regression tests.
- Updated deployment documentation and feature descriptions.

## Verification

- GitHub Actions will run typecheck, tests, build, policy, and browser
shards.
- Focused tests cover the single settings control, runtime
authorization, fail-closed transport selection, the deprecated public
input, and old managed configuration.
- No local tests were run, per the maintainer request to use GitHub
Actions for verification.
- `git diff --check` passes.

## Risks

Low to moderate risk. The effective ingress gate changes from a separate
stored flag to the resolved native run decision. Fresh runs still
require `enableNativeRunner`. Persisted native runs remain recoverable.
Legacy adapters never receive ingress authorization.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex, GPT-5, with reasoning, tool use, and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-01 09:21:23 -05:00
Dotta 1ed29abaa6 fix(runner): harden dormant provider boundaries (#12654)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Runner currently enables only the Codex production path.
> - The package also contains dormant OpenCode and ACPX provider
boundaries.
> - Dormant boundaries must still fail safe before later activation
work.
> - Provider children must not inherit unrelated server secrets or host
homes.
> - Permission defaults must require interaction instead of broad
automatic approval.
> - This pull request hardens those boundaries without activating them.
> - The benefit is a safer base for later provider-specific runnerd
work.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

This improves the inactive OpenCode and ACPX provider boundary in
Paperclip Runner.

**Subsystem affected**

The adapter permission contract, Runner provider environment, and native
execution input builder.

**Current behavior**

Dormant OpenCode code can inherit the full server environment. Its
default permission mode allows operations. ACPX also defaults to broad
approval. The provider guard can accept inherited object property names.

**Proposed behavior**

Use exact provider identifiers. Use interactive defaults. Allow only
required OpenCode environment keys. Reject invalid proxy permission
modes.

**Reason and benefit**

This reduces accidental authority and secret exposure before future
provider activation.

**Breaking changes**

No production provider is activated. Codex runtime selection and Codex
credential-home discovery do not change. Dormant OpenCode and ACPX
callers that omit permission modes now receive safer defaults.

## What Changed

- Change dormant OpenCode and ACPX permission defaults to interactive
modes.
- Reject prototype property names as provider identifiers.
- Default dormant ACPX input to the qualified Codex agent profile.
- Add an explicit OpenCode runner environment allowlist.
- Exclude host homes, server credentials, database values, and Node
injection options.
- Add a fail-closed OpenCode proxy permission parser.
- Add focused tests for defaults, filtering, and invalid values.

## Verification

GitHub Actions must run:

- Adapter utility tests.
- Paperclip Runner tests, type checks, and build.
- Server native runtime tests.
- Repository test, type-check, build, policy, and security gates.

No local test command was run. The repository owner requested
GitHub-only verification.

## Risks

Future OpenCode credential providers must add required variables to the
allowlist through review. The safer defaults can pause dormant internal
scenarios that relied on implicit broad approval. Production Codex
behavior is unchanged.

## Model Used

OpenAI Codex with the GPT-5 agent model. The work used high reasoning,
repository inspection, tool use, and parallel security review.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-01 04:46:36 -05:00
Dotta 131f5c4065 feat(runner): add administration and observability (#12641)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Administrators need bounded controls for experimental native
execution.
> - The lower stack adds remote Codex execution and the task workspace.
> - Operators need to configure Codex safely and inspect provider
traces.
> - Unsupported providers must not appear as runnable choices.
> - This pull request adds Codex-only administration and observability.
> - The benefit is a default-off operational surface for production
diagnosis.

## Linked Issues or Issue Description

Refs #12640.
Refs #12616.
Refs #12352.

**Subsystem affected**

Agent configuration, instance experimental settings, run ledger,
provider trace inspector, and administrator actions.

**Problem or motivation**

The native runner lacks one safe operator surface for Codex permissions,
lifecycle, raw trace capture, and run inspection. The integration branch
also contains provider choices that the production backend cannot
execute yet.

**Proposed solution**

Expose only the qualified Codex controls. Keep Paperclip Developer Mode
and runner preview ingress off by default. Gate raw trace actions by
administrator access and existing trace authorization.

**Alternatives considered**

Exposing unfinished providers would create configurations that fail at
runtime. Always-on tracing would increase sensitive data and storage
risk.

**Roadmap alignment**

This work supports governed Cloud and Sandbox agents and production
diagnostics.

## Stack

- Base PR: #12640.
- Lower PRs: #12639 and #12638.
- This PR contains only its 54-file administration and observability
delta.
- This is the final feature PR in the Codex production stack.

## What Changed

- Added Codex-only Paperclip Runner permission and lifecycle controls.
- Added bounded warm idle configuration.
- Kept the provider field fixed to Codex.
- Added administrator-only one-run raw trace requests.
- Added a persistent future-run raw trace toggle.
- Added trace status, metadata, ledger, and canonical runner inspection.
- Added JSON-RPC request-origin grouping and finalization lineage.
- Restored the stateful PRP transcript parser and focused projection
tests required by trace inspection.
- Added default-off Paperclip Developer Mode.
- Added Honeycomb run links for authorized developer mode.
- Disabled the legacy operational skill for `paperclip_runner`.
- Did not expose OpenCode, ACPX, Pi, Claude Managed, or AWS runner
choices.
- Did not change migrations, workflows, dependencies, or
`pnpm-lock.yaml`.

## Verification

- GitHub Actions will run UI tests, server tests, repository typecheck,
build, browser tests, security, and policy gates.
- Tests cover Codex configuration defaults and bounds, administrator
trace actions, persistent settings, ledger inspection, trace lineage,
and Honeycomb links.
- Existing server trace authorization and retention tests remain the
backend authority.
- Local tests were not run. The requested verification policy uses
GitHub Actions for this series.
- `git diff --check runner/task-workspace-experience...HEAD` passes.
- The delta contains 54 files.

## Risks

- Raw provider traces can contain sensitive provider data.
- Existing server authorization controls access, reveal, download,
retention, and deletion.
- The UI gates trace actions by administrator access and developer mode.
- All new instance settings remain off by default.
- Fresh Paperclip Runner configuration remains Codex-only.
- Direct adapters and legacy task behavior do not change in this PR.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex with GPT-5.6. The work used high-reasoning agent mode,
repository tools, GitHub tools, and parallel code-audit agents.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with Fixes: / Closes /
Refs OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-01 03:41:23 -05:00
Dotta 0a422fda52 feat(runner): add remote execution substrate (#12638)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Runner gives native runs a durable and governed execution
path.
> - The current native path runs on the control-plane host.
> - Remote environments need an authenticated execution-target contract.
> - The contract must not change direct adapters or enable new runtimes
by default.
> - This pull request adds the remote execution substrate and Daytona
ingress.
> - The benefit is a bounded base for later remote runner transport
work.

## Linked Issues or Issue Description

Refs #12616.
Refs #12352.

**Subsystem affected**

Cross-cutting. This change touches runner transport, server
orchestration, plugin contracts, and shared settings.

**Problem or motivation**

Native execution cannot resolve an authenticated runner ingress through
a remote environment. The server also lacks one provider-neutral
contract for remote execution targets.

**Proposed solution**

Add a default-off runner preview ingress capability. Add
transport-neutral runner connectivity. Add remote execution target and
lifecycle handling. Add a Daytona ingress implementation with redacted
credentials.

**Alternatives considered**

A provider-specific server path would duplicate orchestration and
authorization. A public endpoint without an environment contract would
weaken the trust boundary.

**Roadmap alignment**

This work supports the Cloud and Sandbox agents milestone. It also
supports self-healing runs and governed tool access.

## What Changed

- Added execution-target traits for local, SSH, and sandbox
environments.
- Added plugin RPC contracts for runner ingress endpoints.
- Added authenticated Daytona preview ingress.
- Added transport-neutral PRP outbound connections.
- Added remote runner artifact verification and fail-closed provider
selection.
- Added bounded native session resume, cancellation, and lifecycle
recovery.
- Preserved Codex-only selection for fresh experimental runner starts.
- Preserved all direct adapter execution and finalization paths.
- Removed stale Pi provider-pack requirements that security review
rejected.
- Kept the rollout controls off by default.
- Did not change pnpm-lock.yaml, Cargo, database migrations, or GitHub
workflows.

## Verification

- GitHub Actions will run the repository test, typecheck, build,
security, and policy gates.
- Focused tests cover ingress validation, redaction, execution targets,
remote lifecycle, cancellation, resume, and legacy adapter selection.
- Local tests were not run. The requested verification policy uses
GitHub Actions for this series.
- `git diff --check origin/master...HEAD` passes.
- The diff contains 52 files.

## Risks

- Remote execution crosses a trust boundary.
- The implementation validates target capabilities, artifact digests,
provider-pack pins, and connection metadata.
- The feature remains default-off.
- Fresh native selection remains Codex-only.
- Existing direct adapters remain on the legacy path.
- This PR does not yet make remote Codex runnable. The next PR adds the
Rust WSS and TLS transport.

## Model Used

OpenAI Codex with GPT-5.6. The work used high-reasoning agent mode,
repository tools, GitHub tools, and parallel code-audit agents.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with Fixes: / Closes /
Refs OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-01 01:29:06 -05:00
Dotta 51ad751e0b feat(runner): integrate Codex native execution (#12616)
## Thinking Path

> - Paperclip is the open source control plane for teams of AI agents.
> - Agent runs currently use direct adapters and their established
finalization paths.
> - The new runner package needs one production integration before it
can execute a real provider through the server.
> - That integration must not change direct adapters or expose
unsupported providers.
> - The rollout must also preserve native runs that were already
recorded when the feature flag changes.
> - This pull request adds a default-off, Codex-only native execution
path and its authority boundary.
> - The benefit is a recoverable production vertical slice with explicit
compatibility guards.

## Linked Issues or Issue Description

**Subsystem affected**

Cross-cutting server orchestration and adapter selection.

**Problem or motivation**

The runner package exists, but the server cannot yet start and recover a
governed Codex run through it. A careless integration could also route
existing direct adapters into the native runtime or lose cancellation
and finalization state.

**Proposed solution**

Add a hidden `paperclip_runner` adapter for Codex. Keep it behind the
default-off instance flag. Bind native execution, resume, cancellation,
semantic tool authority, and finalization to the recorded company,
issue, run, and coordinator identities. Leave every direct adapter on
its existing path.

**Alternatives considered**

A multi-provider launch was rejected because only Codex has the complete
production bridge in this series. Replacing direct adapter execution was
rejected because the runner remains experimental.

**Roadmap alignment**

This work supports governed tool access, action attribution, and
self-healing runs. It keeps the integration narrow and default-off.

## What Changed

- Add the Codex-only native session executor and persisted resumption
path.
- Add run-scoped semantic tool projection, authorization, receipts, and
idempotency.
- Add audited native cancellation with durable issue and coordinator
binding.
- Add result fencing so a recorded result cannot reacquire the provider
and run twice.
- Reject fresh runner starts when the rollout flag is off while
preserving recorded native recovery.
- Keep direct adapters outside native status, cancellation, record
creation, and finalization.
- Add focused conformance, recovery, cancellation, status, portability,
and compatibility coverage.

## Verification

- GitHub Actions is the authoritative test environment for this large
stack.
- The PR policy and lightweight stack checks run while this is a middle
PR.
- The full required suite runs when this PR becomes the lowest unmerged
or top PR.
- Greptile will review this exact delta after the branch is pushed.

## Risks

- The main risk is routing a legacy adapter into native execution.
Runtime selection and heartbeat tests cover that boundary.
- The next risk is stale or cross-company cancellation. Durable binding
checks and transactional audit persistence cover it.
- The adapter remains hidden and default-off. Only Codex is admitted.
- There are no database migration, lockfile, or GitHub workflow changes
in this PR.

## Stack

1. [Runner package, SDK, and developer
tools](https://github.com/paperclipai/paperclip/pull/12608)
2. This PR: Codex production server integration
3. [Provider-neutral task-thread
UI](https://github.com/paperclipai/paperclip/pull/12617)

## Model Used

OpenAI Codex with GPT-5, extended reasoning, repository tools, and
parallel review agents.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-31 22:51:17 -05:00
DottaandDev Agent 25cf079ec5 feat(runner): add Codex-native application integration (#12591)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The runner package is useful only when the application can start,
observe, and recover a native Codex run safely.
> - Existing direct adapters must keep their current execution and
finalization paths.
> - The application boundary therefore needs additive persistence,
authorization, coordination, and recovery behind an explicit
experimental adapter.
> - This pull request adds that Codex-only boundary without activating
generalized providers, remote environments, or the later task/SDK
surfaces.

## Linked Issues or Issue Description

**Subsystem affected**

Shared contracts, database persistence, adapter utilities, server
native-runtime services, and the experimental Paperclip Runner adapter.

**Problem or motivation**

The already-landed runner package has a qualified Codex path, but the
application needs durable native-run state, guarded runtime selection,
authenticated coordination, tool security, finalization, and recovery
before the experimental adapter can be exercised safely.

**Proposed solution**

Add a Codex-only `paperclip_runner` application path behind the existing
default-off native-runner setting. Bind native state and coordination to
company/run identity, preserve persisted-run recovery, and leave every
direct adapter on its existing legacy execution path.

**Alternatives considered**

The earlier stack boundary introduced a generalized executor and
remote-environment lifecycle here. That made this PR depend on
implementations in higher PRs and changed reusable sandbox behavior
globally. Those pieces are now deferred together to #12592.

**Roadmap alignment**

ROADMAP.md does not list a conflicting native-runner integration
project. This change adds the application boundary for the existing
Runner architecture.

## What Changed

- Added native run/result/finalization/provider-trace persistence,
shared validators, and idempotent migration/replay coverage.
- Added guarded Codex-only runtime selection, authenticated PRP
coordination, recovery, finalization, and interaction services.
- Added run/company-bound tool-gateway authorization, credential
redaction, SSRF protections, and replay-safe behavior.
- Added the explicit `paperclip_runner` adapter behind the default-off
rollout setting.
- Preserved legacy answered-question wake projection and direct-adapter
execution/finalization paths.
- Hardened cancellation so only owned in-memory child processes are
signaled; persisted recycled PIDs/process groups are never trusted.
- Retained the narrow Claude ACPX isolated-context security follow-up
discovered after #12590.
- Deferred the generalized executor, provider ingress, remote lifecycle,
SDK/lab/eval work, release-process changes, and lockfile.

## Verification

- Changed-file delta against `master`: 133 files.
- GitHub Actions is the authoritative verification environment for this
PR.
- Full CI, security, and Greptile review will run on this lowest
unmerged stack PR.
- Local tests/build/typecheck were not run because this checkout is
resource constrained.
- Static diff/reference checks pass, and `pnpm-lock.yaml` is unchanged.

## Risks

- This touches central heartbeat and agent-route code, so legacy
compatibility is the primary risk.
- Runtime selection remains Codex-only and explicit; direct Codex,
Claude, OpenCode, process, HTTP, and plugin adapters remain on their
existing paths.
- Fresh native starts fail closed while the rollout flag is off;
persisted native records remain readable and recoverable.
- Cancellation, company/run binding, tool calls, status decisions, and
completion writes are guarded or replay-safe.

> For core feature work, check [ROADMAP.md](ROADMAP.md) first and
discuss it in #dev before opening the PR. Feature PRs that overlap with
planned core work may need to be redirected.

## Model Used

OpenAI Codex, GPT-5.6, with repository tools, code execution, and
parallel agent review.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either linked existing issues or described the issue in-PR
following the relevant issue template
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id
- [ ] I have run tests locally and they pass — GitHub Actions is
authoritative for this resource-constrained checkout
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented risks above
- [ ] All Paperclip CI and security gates are green
- [ ] Greptile is 5/5 with no open actionable findings
- [x] I will address all Greptile and reviewer comments before merge

## Stack

- Position: 3 of 5 overall; lowest of 3 currently unmerged
- Base: `master`
- Previous:
[#12590](https://github.com/paperclipai/paperclip/pull/12590), qualified
Claude ACPX runtime — merged
- Next: [#12592](https://github.com/paperclipai/paperclip/pull/12592),
generalized Codex executor, task experience, and developer SDKs

---------

Co-authored-by: Dev Agent <dev@paperclip.ing>
2026-08-31 14:38:38 -05:00
Dotta 7a3abb88a0 feat(runner): authorize server Codex tools (#12385)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The hidden native coordinator already computes a run-scoped semantic
tool projection.
> - The durable Codex backend now accepts and enforces that projection.
> - The server did not include the projection in its `run.prepare`
command.
> - Codex therefore received no production semantic tools even when the
server authorized them.
> - This pull request adds the deterministic wire projection and sends
it to runnerd.
> - The benefit is one fail-closed authorization catalog from the server
through Codex.

## Linked Issues or Issue Description

Refs #12384

**What existing behavior does this improve?**

This improves the existing flagged Paperclip Runner Codex path.

**Current behavior**

The server creates a run-scoped list of authorized read tools. It does
not pass that list to runnerd, so the production Codex session starts
with no tools.

**Proposed behavior**

The server maps the authorized definitions to the versioned runner
contract. It computes a cross-language catalog digest. It includes that
immutable contract in `run.prepare`.

**Reason and benefit**

Runnerd and the server now enforce the same catalog identity. Unknown,
duplicate, changed, or malformed tool contracts fail before Codex can
use them.

**Breaking changes**

None. Direct adapters are unchanged. A native run with an empty server
projection still starts with no dynamic tools.

## What Changed

- Add a deterministic semantic-definition to runner-authorization
projection.
- Match the Rust canonical digest with a shared test vector.
- Include the server coordinator projection in the native Codex
`run.prepare` command.
- Extend the native Codex vertical slice to require and execute a
semantic tool.
- Verify the production prepare payload in a host-independent server
test.

## Verification

- `pnpm --filter @paperclipai/paperclip-runner test:typescript` (354
tests pass)
- `pnpm --filter @paperclipai/server exec vitest run
src/services/native-runtime/native-codex-runner.test.ts`
- `pnpm --filter @paperclipai/server typecheck`
- `pnpm -r typecheck`
- `pnpm build`
- The embedded-Postgres vertical slice is present for CI. This local
host reports that embedded Postgres is unavailable, so Vitest skips that
host-dependent test locally.
- Confirmed that the PR changes 7 files against
`runner-codex-durable-tools`.
- Confirmed that `pnpm-lock.yaml` is unchanged.

## Risks

The main risk is a catalog digest mismatch between TypeScript and Rust.
Both implementations use canonical JSON. They share the same fixed
digest vector. Runnerd also recomputes the digest and rejects a
mismatch.

The rollout flag and the existing native runtime selection rules remain
unchanged.

## Model Used

OpenAI Codex with GPT-5 and repository tool use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-30 11:43:34 -05:00
Dotta bc9ba7cd26 feat(runner): project native runs into task threads (#12321)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The experimental Paperclip Runner can execute a guarded Codex run
and persist provider-neutral events.
> - The task page still reads direct-adapter transcripts and cannot
present those native events.
> - Structured runner questions must also use the existing task
interaction experience.
> - Runtime selection must use the persisted run mode, not an adapter
name or a current feature flag.
> - This pull request projects native events and questions into the
existing task thread.
> - Direct adapters keep their existing transcript, composer,
interaction, and finalization paths.
> - The benefit is a complete native Codex task thread without a
behavior change for existing adapters.

## Linked Issues or Issue Description

Refs #12202. This pull request replaces that stale implementation on
current `master`.

**What happened?**

The server persists native runner events and structured input requests.
The task page only consumes direct-adapter transcripts. A native run
therefore cannot present a complete transcript, usage, or question flow
through the normal task experience.

**Expected behavior**

Native runs project persisted provider-neutral events into the existing
task thread. Native structured questions use the existing interaction
card. Direct adapters retain their current behavior.

**Steps to reproduce**

1. Enable the experimental runner.
2. Start a native Codex run that emits progress, usage, a structured
question, and a final reply.
3. Open the task page.
4. Observe that the direct-adapter transcript path cannot project the
native event records.

**Paperclip version or commit**

`master` at `67f9867bc`.

## What Changed

- Add the canonical structured-question validator and shared contract
exports.
- Materialize native input requests as existing task interactions.
- Validate native answers and deliver them through the durable
question-response receipt.
- Resume the original PRP request with an idempotent `request.resolve`
command.
- Project native messages, tool activity, cumulative usage, and final
replies into the existing transcript model.
- Propagate persisted `runtimeMode` to the task page and select native
handling only for `runtimeMode: "native"`.
- Expire pending interactions through the shared issue service on every
terminal transition, including decisions, stalled reviews, tree control,
and pipeline retry cleanup.
- Queue native run cancellation while a transaction is open and execute
it only after the owning transaction commits.
- Keep nonterminal and non-runner issue paths on their existing service
call shapes and behavior.

## Verification

- `pnpm --filter @paperclipai/server typecheck` — passed, including the
Rust runner release build and protocol/catalog drift gates.
- Focused native-thread and lifecycle suites — 18 files and 481 tests
passed during review.
- `issue-execution-policy-routes.test.ts` — 19/19 passed after the final
transactional-queue expectation update.
- `issue-agent-mutation-ownership-routes.test.ts` — 87/87 passed in the
final isolated compatibility rerun.
- GitHub Actions — policy, build, canary, typecheck/release registry, 5
serialized server shards, 8 general-test shards, 3 browser shards, and
both aggregate gates passed on `7793f3193`.
- Security — Snyk, Socket Project Report, Socket PR Alerts, and
Superagent passed.
- Greptile — 5/5 on `7793f3193`; all actionable review threads resolved.
- `git diff --check` — passed.
- Diff against `master`: 44 files.

## Compatibility Boundary

- Native transcript polling only runs when the persisted run reports
`runtimeMode: "native"`.
- Missing or legacy runtime modes continue through
`useLiveRunTranscripts`.
- Legacy questions keep the existing optional free-text choice.
- Native closed select sets can suppress that legacy fallback.
- Terminal cleanup uses the same issue service for native and legacy
interactions; only a bound native question schedules a native run
cancellation.
- Native cancellation happens after transaction commit, so failed or
rolled-back writes do not cancel a still-valid run.
- The durable delivery service checks the original native request before
it considers a continuation run.
- This pull request adds no migration, dependency, workflow, manifest,
or lockfile change.

## Risks

The main risk is routing a direct-adapter task through native handling
or changing terminal issue behavior. The implementation selects the
native path only from persisted runtime facts, retains the existing
nonterminal call shape, and schedules native cancellation only for a
validated bound native question after commit. Focused and
repository-wide tests cover both paths. Native requests remain bound to
the company, issue, run, and agent; answers are validated, durable, and
idempotent across reconnects.

## Model Used

OpenAI Codex, GPT-5 family. The client does not expose the exact
deployment ID or context window. Agentic reasoning, tool use, and code
execution were enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run the affected local tests and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated the compatibility notes for this change
- [x] I have considered and documented risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I addressed all Greptile and reviewer comments before requesting
merge
2026-08-29 19:26:20 -05:00
Dotta 397de98193 feat(runner): add flagged Codex execution adapter (#12188)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The Paperclip Runner now has protocol, provider, tool, package,
persistence, and hidden server boundaries.
> - The server still cannot select that path for a real agent heartbeat.
> - A new runtime must not change any existing direct adapter.
> - An experimental runtime must fail closed when its rollout flag is
off.
> - This pull request adds one guarded Codex vertical slice through
runnerd.
> - The benefit is a production-built runner path that users cannot
start by default.

## Linked Issues or Issue Description

Refs #11962

Refs #12111

Refs #12169

Refs #12176

**Subsystem affected**

Cross-cutting. The change affects the runner package, server
orchestration, shared settings, and adapter configuration UI.

**Problem or motivation**

The hidden PRP coordinator cannot execute a real heartbeat. The
application also needs an explicit rollout boundary before it can expose
the experimental runner. Existing direct adapters must keep their
current execution and finalization behavior.

**Proposed solution**

Add `paperclip_runner` as a Codex-only adapter behind the default-off
`enableNativeRunner` instance flag. Select the native runtime only for
that adapter. Persist the run binding before runnerd starts. Wait for
the durable PRP result and terminal event. Resume the real Codex
provider thread on later heartbeats. Keep persisted native runs readable
and recoverable after the flag changes.

**Alternatives considered**

The server could route `codex_local` through runnerd. That option would
change an existing adapter and weaken rollback safety. The server could
expose all providers now. That option would add unreviewed provider
behavior. The build could depend on a prebuilt runner binary. That
option would make source builds architecture-dependent and difficult to
verify.

**Roadmap alignment**

This work supports the shipped enforced-outcomes, governed-tool, and
self-healing-run milestones. It does not add a new roadmap surface. It
is the guarded execution step after the merged hidden runner boundaries.

**Additional context**

This is the next replacement for the closed large runner pull request.
Task-thread presentation remains a separate follow-up so this change can
preserve the current direct-adapter UI.

## What Changed

- Add `paperclip_runner` as an explicit Codex-only adapter.
- Add the default-off `enableNativeRunner` instance flag.
- Reject fresh create, hire, import, switch, and execution requests
while the flag is off.
- Allow edits to persisted runner agents while the flag is off.
- Recover an already persisted native run even after the flag is
disabled.
- Keep every built-in direct adapter on its existing runtime path.
- Persist an immutable native run binding and revisioned completion
contract before runnerd starts.
- Execute server to PRP to runnerd to Codex to server through the hidden
coordinator.
- Validate the durable result against the terminal event and exact
completion criteria before finalization.
- Preserve the Codex provider thread ID and use `thread/resume` on the
next heartbeat.
- Strip unsupported Codex configuration fields from the experimental
adapter.
- Build a target-native release runner binary from source and vendor it
into the server distribution.
- Install Rust only in the Docker build stage. Do not add a workflow or
lockfile change.
- Stop the runner process group on completion, cancellation, and forced
shutdown.

## Verification

- Run `pnpm --filter @paperclipai/paperclip-runner check:all`. All 69
TypeScript tests and 58 Rust tests pass. Protocol, conformance, replay,
formatting, and generated-file checks pass.
- Run the 12 focused adapter, settings, runtime-selection, coordinator,
direct-isolation, and real Codex integration test files. All 186 tests
pass.
- The real integration test uses PostgreSQL, HTTP, WebSocket, runnerd,
and a fake Codex app server. It proves one `thread/start` followed by
one `thread/resume`.
- Run `pnpm -r typecheck`.
- Run `pnpm build`.
- Run `pnpm check:token-gates`.
- Build the Docker `build` target from a clean context. Confirm that the
server distribution contains an executable `paperclip-runnerd` built
with Debian Rust 1.85.
- Start the server through the source-mode tsx entry point with the
package `dist` directory absent. Confirm the vendor shim resolves source
exports and the server boots.
- Run `pnpm test:run` twice. On this macOS host, 405 files pass and 1
file skips. Eight untouched workspace and loopback tests fail because
macOS resolves `/tmp` and `/var` through `/private` and because
PID-derived test ports exceed 65535. Linux CI must pass the full suite.
- Confirm that the diff contains 52 files. Confirm that it contains no
`.github` or `pnpm-lock.yaml` change.

## Risks

- The feature flag is off by default. A fresh native start fails with a
stable error while the flag is off.
- A persisted native run remains recoverable after the flag changes.
This prevents rollout changes from corrupting recorded work.
- Only local Codex execution is accepted. Other providers and remote
work modes fail closed.
- Existing direct adapters do not start runnerd, create native rows, use
native status arbitration, or enter native finalization.
- The runner receives its one-use bootstrap ticket through the child
environment. The server does not put the ticket in command arguments or
logs.
- The server validates the company, task, agent, run, runner, session,
completion contract, result, and terminal binding before it accepts
completion.
- The build compiles a target-native Rust binary. Cross-platform release
packaging remains a later concern. Source builds and Docker builds
compile for their current target.
- Docker needs enough build memory for the existing server TypeScript
compile. The Docker build stage sets a 4 GB V8 heap limit.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex with GPT-5. The exact deployment ID and context-window
size are not exposed. The model used agentic reasoning, repository
tools, code execution, and test execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and applicable tests pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-25 16:03:41 -05:00
Dotta 9964b034bb feat(runner): add hidden server PRP coordinator (#12176)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The Paperclip Runner needs a narrow server trust boundary before an
adapter can start it.
> - The package has durable runner transport, but the server does not
host or authorize that transport.
> - Native persistence exists, but no writer connects PRP events to
those records.
> - A direct adapter must not enter this path by accident.
> - This pull request adds a hidden, run-bound PRP server coordinator.
> - The benefit is a recoverable server boundary that remains
unavailable to normal execution.

## Linked Issues or Issue Description

Refs #11962

Refs #12129

Refs #12169

**Subsystem affected**

Cross-cutting. The change affects the runner package and server
orchestration.

**Problem or motivation**

The server cannot authenticate runnerd, commit PRP events before ACK,
authorize semantic tools, or enter native finalization from a durable
runner result. The application must have this hidden boundary before a
guarded adapter can use the runner.

**Proposed solution**

Add an authenticated PRP WebSocket authority and register it only for
one exact persisted native Codex run. Bind each connection and event to
the company, issue, agent, run, runner, session, turn, item, and
verified runner identity. Commit each event before its cumulative ACK.
Project only authorized same-task read tools. Rebuild the accepted
result and finalization record from durable result and terminal events.

**Alternatives considered**

The server could expose a broad runner API key or route semantic calls
through existing adapter endpoints. Those options grant too much
authority and weaken replay recovery. The server could also add the
user-facing adapter in this pull request. That option would mix rollout
selection with the transport trust boundary and make legacy
compatibility harder to review.

**Roadmap alignment**

This work supports the shipped enforced-outcomes, governed-tool, and
self-healing-run milestones. It does not add a new roadmap surface.

## What Changed

- Add the durable PRP server authority with one-use bootstrap tickets,
reconnect leases, encrypted frames, bounded state, cumulative ACKs, and
idempotent commands.
- Add `/api/runner/v1/connect/:runId`. Derive its `ws://` or `wss://`
URL from the configured Paperclip API URL.
- Register one authority only after the coordinator verifies the
complete native Codex run binding.
- Commit validated PRP events to `heartbeat_run_events` before ACK.
Reject source gaps and conflicting replays.
- Rebuild accepted results and finalization records from durable result
and terminal events. Enforce finalization owner leases and retry times.
- Project five same-task read operations. Recheck run, agent, task, and
company authority for each call.
- Keep the route hidden. No adapter selects this coordinator, and no
code starts runnerd.
- Vendor the compiled runner TypeScript runtime into the server package
while keeping the workspace package development-only for the server.
- Document the package, database writer, run-log payload, and credential
exclusions.

## Verification

- Run `pnpm --filter @paperclipai/paperclip-runner check:all`. All
TypeScript protocol checks and 69 Vitest tests pass, including
commit-before-ACK crash recovery. All 43 Rust unit tests and 13 Rust
integration tests pass. Conformance and replay parity pass.
- Run the focused server WebSocket, coordinator, package-build, and
startup-wiring suites. All 26 tests pass, including a clean-checkout
reproduction with the runner `dist` directory absent.
- Run `pnpm -r typecheck`.
- Run `pnpm test:run`.
- Run `pnpm build`.
- Confirm that the diff contains 19 files. Confirm that it contains no
workflow or `pnpm-lock.yaml` change.

## Risks

- The server installs the WebSocket route at startup. An unregistered or
malformed run path fails closed and creates no native record.
- Bootstrap tickets are one use. The private state directory uses mode
`0700`, and the state file uses mode `0600`. The file stores derived
authentication verifiers and never stores raw tickets or lease tokens.
- The journal has explicit frame, command, event-window, and file-size
bounds. A bound violation closes the runner connection or rejects the
command.
- A runner event reaches the database before its ACK. A crash between
event commit and ACK causes a byte-equivalent replay, not a second
logical effect.
- The coordinator accepts only an existing queued or running native
Codex row with exact company, task, agent, runner, session, and
completion-contract ownership.
- Existing direct adapters do not call this service. They keep their
current execution, transcript, result, and finalization paths.
- The server has no production dependency on the private runner package.
Its build copies the compiled runtime into `server/dist`; the workspace
link is development-only. This adds no external package and does not
change the lockfile.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex with GPT-5. The exact deployment ID and context-window
size are not exposed. The model used agentic reasoning, repository
tools, code execution, and test execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-25 14:17:14 -05:00