0 Fork 0

mirror of https://github.com/paperclipai/paperclip.git synced 2026-11-08 01:57:31 +01:00

Files

T

DottaandPaperclip b193077582 fix: report Slack callback health correctly behind Cloud proxies (#13694 )

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Slack connections turn messages into governed agent runs and return
replies to Slack.
> - Remote runs must replace an incompatible sandbox runner with the
controller's packaged binary.
> - The fallback used a package-relative path that does not match the
vendored server layout.
> - Slack callback health also compared the internal proxy address with
the public callback address.
> - Master now contains the runner fallback fix; this pull request fixes
callback health and extends missing-runner regression coverage.

## Linked Issues or Issue Description

Refs #13677, #13680, #13691, and #13686.

**What happened?**

A packaged controller stopped a Slack-triggered remote run with
`runner_remote_artifact_unavailable` when the sandbox runner needed
replacement. Working Slack callbacks also showed a stale URL warning
behind the Cloud gateway.

**Expected behavior**

The controller stages its packaged runner when needed. Callback health
uses the observed public address and still detects real address changes.

**Steps to reproduce**

1. Run a packaged server with a sandbox that has an older runner or no
runner.
2. Send a Slack mention to an agent that uses that sandbox.
3. Route signed Slack callbacks through a claimed Cloud gateway that
rewrites the upstream host.
4. Check the run and the Slack callback health panel.

**Paperclip version or commit**

Reproduced on master at `aeef493f4a7603b7b1254421b80fb00212982390`. The
fix branch also includes #13691.

**Deployment mode**

Packaged server with a Cloud gateway and a Daytona sandbox.

## What Changed

- Extend the controller-owned runner fallback tests with a missing
sandbox binary case; retain the fix now merged in #13686.
- Prefer dedicated gateway diagnostic headers that survive provider
rewrites of standard forwarded headers. Use validated host hints only
for callback-health evidence on claimed Cloud instances after provider
acceptance. Preserve request bodies, routing, authentication, and
configured callback URLs.
- Cover current, stale, and missing sandbox runners, all Slack callback
surfaces, rejected callbacks, malformed proxy hints, real host and port
changes, and the existing self-hosted behavior.
- Document the artifact lookup and callback-health boundaries.

## Verification

- Native session executor and binary resolver suites: 379 tests passed
after merging current master (`45c99a0d0`).
- Targeted callback integration suite with disposable PostgreSQL: 4
tests passed before rebase.
- Full `pnpm -r typecheck` and `pnpm build` passed after merging current
master. The full local suite passed 12,668 tests; one suite failed to
start its disposable PostgreSQL. Rerunning that suite alone passed all
31 tests.
- The built server resolver selected the executable under
`server/dist/vendor/paperclip-runner/bin/`.
- Final callback regression: all 4 targeted integration tests pass,
covering provider header rewrites, default ports, uppercase/trailing-dot
hosts, and ignored self-hosted hints.
- Live staging proof of the runner fix: a previously failed Slack thread
recovered, a new mention received its requested response, and the
account-connect command succeeded. Unsigned callbacks returned 401.
- Live browser and Slack acceptance passed: generated callback URLs,
account linking, a new mention, an interactive question and answer, and
all three callback-health indicators. The connector was activated
through the onboarding UI.
- All 54 latest-head checks pass, two optional checks are skipped, and
Greptile is 5/5 with no unresolved threads.

## Risks

- Runner fallback must select a binary for the remote platform. This
preserves explicit remote artifact overrides and the existing capability
checks.
- Proxy headers are not identity proof. They are used only for
diagnostics on claimed Cloud instances after the provider accepts the
request. Self-hosted instances ignore them. Wrong public hosts and ports
still warn.
- No database migration or new public API contract.

## Model Used

OpenAI GPT-6 via Codex. Used reasoning, repository tools, code
execution, and browser/native-app testing. Exact model variant and
context window were not exposed by the session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>

2026-09-19 17:32:51 -05:00

5.0 KiB

Raw Blame History

Paperclip Runner in Daytona sandboxes

Intended topology

Paperclip creates the run and a short-lived, one-use runner bootstrap ticket. The Daytona sandbox starts only paperclip-runnerd; runnerd then starts the configured provider (Codex initially). Runnerd listens on a fixed sandbox-local port and Paperclip connects through Daytona's authenticated preview WebSocket ingress. The provider never receives a Paperclip API credential or Daytona preview credential.

Paperclip control plane
       |
       | outbound WSS + X-Daytona-Preview-Token
       v
Daytona preview proxy -> paperclip-runnerd:43127 -> Codex app-server

The PRP identity binds company, issue, agent, run, environment lease, runner instance, normalized session, artifact version, artifact digest, and catalog digest. After the one-use ticket challenge succeeds, runnerd receives a renewable connection lease. Revocation, expiry, replay cursors, and the durable outbox continue to work across transient network loss and sandbox/provider restarts.

Network policy

Allow Paperclip to make outbound TCP 443 connections to the configured Daytona preview origin. The Paperclip host does not need public ingress.
Allow the sandbox provider's private preview proxy to reach runnerd on TCP 43127. Do not expose the port with a public sandbox or signed URL.
Allow the sandbox outbound access only to destinations explicitly required by the provider runtime.
Use Daytona's wss:// preview URL with normal certificate and hostname validation. The preview token is sent only as the X-Daytona-Preview-Token header by Paperclip.
Runnerd rejects public plaintext ws:// destinations. Public dial targets require wss:// with hostname validation and either the platform trust store or an explicitly staged private CA bundle.
Listener mode binds only 0.0.0.0:43127, accepts only the run-specific /api/runner/v1/connect/:runId path, and rejects WebSocket extension negotiation. PRP authentication and application-layer secure frames remain mandatory above the preview-proxy hop.
Do not put the bootstrap ticket in argv, files, provider environment, logs, or model context. Inject it into runnerd's initial environment/secret channel; runnerd already removes it from its environment immediately.
Runnerd is the only process allowed to reach PRP. The provider communicates with runnerd over inherited pipes.

Lifecycle

Paperclip allocates the Daytona environment and persists its environment lease.
Paperclip creates a runner ticket with a very short expiry and binds it to the run, environment lease, runner digest/version, and allowed catalog digest.
The sandbox startup command launches the pinned runnerd artifact in listener mode on 0.0.0.0:43127 and the run-specific PRP path. The ticket is supplied separately as secret environment material.
Paperclip acquires an authenticated Daytona preview endpoint and dials runnerd. After PRP authentication, Paperclip sends run.prepare; only then may runnerd start Codex and advertise the run-authorized tools.
Suspend/drain/revoke commands stop new turns and durably flush terminal events. Paperclip revokes the lease before destroying or recycling the sandbox.
A recovered sandbox reuses its durable runner state and an unexpired connection lease; it must not mint a second provider session when a resumable one exists.

Deployment-mode boundary

Daytona always selects provider ingress for paperclip_runner; it does not fall back to direct outbound WSS or a legacy callback bridge. Same-host native runs keep loopback ws://, and other remote providers may use direct outbound wss:// only when an operator explicitly configures a reachable Paperclip runner URL. Legacy adapters retain their existing transport paths.

Runner image compatibility

A sandbox image can outlive its Paperclip controller version. Before reusing its preinstalled runner, Paperclip checks the binary contract, selected transport, and durableSessionCapabilities for unlimited_runtime and connection_lease_renewal. Some older contract-v2 runners reject the controller's zero lifetime despite reporting a compatible binary contract. Paperclip stages and verifies its bundled runner instead; it can still reuse the image's compatible provider CLI. Explicit remote runner artifacts undergo the same checks. This metadata gate does not change the PRP version or invalidate existing durable session manifests.

The replacement artifact uses the controller's server-owned binary resolver, including the vendored binary in packaged deployments. It must not depend on a source checkout or a local Rust build directory being present on the server.

For a controller deployed before this capability check, operators can set PAPERCLIP_RUNNER_REMOTE_BINARY_PATH in the server process environment to a current runner binary built for the sandbox's OS and architecture. This bypasses image discovery and stages that binary for each run. The agent's environment variables do not configure this controller setting. Remove the override after deploying the capability check so automatic artifact selection resumes.

5.0 KiB Raw Blame History