mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-18 23:08:26 +01:00
## Thinking Path > - Paperclip coordinates agents that run on local and remote computers. > - SSH environments use the harness installed on a computer the operator controls. > - Cloud SSH tasks now use durable remote workspaces. > - Claude and Codex default to ACP, but ACP rejects those in-place SSH workspaces. > - The shared ACP engine already has SSH process bridges and managed account delivery. > - This pull request lets that engine use the remote task directory and stage runtime assets separately. > - The benefit is ACP execution with the selected Paperclip account and the existing SSH environment. ## Linked Issues or Issue Description **What happened?** A Claude or Codex agent using the default ACP engine cannot run in a volume-free Cloud SSH environment. The adapter rejects its in-place workspace before execution. Setting the engine to CLI works around that restriction. **Expected behavior** ACP runs directly in the authoritative remote task directory. Paperclip keeps using the selected account, permission settings, process bridges, and durable workspace owner. **Steps to reproduce** 1. Create an SSH environment with agent-managed authentication on a volume-free server. 2. Select a Paperclip AI account for a Claude or Codex agent and leave its engine at the default. 3. Assign a task to the agent. 4. Observe the ACP in-place workspace rejection. Related: #21957 adds durable Cloud SSH workspaces. #21248 adds agent-selected SSH accounts. A search for SSH ACP work found #21248 and no open duplicate. ROADMAP.md has no overlapping item. ## What Changed - Admit managed SSH ACP with an in-place workspace in both adapters. - Keep ACP's session directory and fingerprint bound to the remote task directory. - Stage private runtime assets with workspace copying disabled. The remote workspace owner still saves task files. - Retain the copy-mode SSH path and existing account, confinement, and command availability checks. No engine fallback or permission change is added. - Extend the existing lifecycle tests for in-place staging, session reuse, current credentials, account changes, and cleanup after bridge or turn failures. - Extend the real OpenSSH/Postgres heartbeat test to the default ACP engine. Its protocol fixture creates a file and verifies conversation history on a follow-up. - Update the SSH setup documentation. Workspace placement belongs to the shared ACP engine. The adapter changes remove their explicit SSH admission refusals. The server test exercises the existing heartbeat path; no server execution logic changes. Existing large test files retain their shared harnesses instead of adding a second lifecycle mock graph. ## Verification - 418 focused tests pass across 7 suites: 411 adapter/ACP lifecycle tests, all 4 real SSH workspace tests, and 3 real ACP host-spawn smoke tests. - Typechecks pass for adapter-utils, Claude, Codex, and the server. - `pnpm verify` returns 2 because this touches shared runtime code and deployment documentation. Required CI supplies the broader build, browser, and release coverage. - Tested head: `33b17a9a21c0a5929523702ebed9c8faa3b22955`. - `git diff --check` and the local secret/PII scan pass. Greptile reviewed the tested head at 5/5 with no actionable findings. Final-head [CI attempt 2](https://github.com/paperclipai/paperclip/actions/runs/44167483231/attempts/2) passes on `33b17a9a21c0a5929523702ebed9c8faa3b22955`: 58 checks pass and 6 are skipped, with no failures or pending checks. Both required gates pass. Four first-attempt jobs were cancelled after their runners received shutdown signals, confirmed in each job log; automatic recovery reran those jobs successfully. No source change was needed for that recovery. Greptile is 5/5 with no unresolved findings, and the branch has no merge conflicts. GitHub still requires code-owner review before merge. ```sh pnpm exec vitest run --maxWorkers=2 \ packages/adapter-utils/src/acpx-engine/execute.test.ts \ packages/adapters/claude-local/src/server/execute.remote.test.ts \ packages/adapters/codex-local/src/server/execute.remote.test.ts \ packages/adapters/claude-local/src/server/acp.test.ts \ packages/adapters/codex-local/src/server/acp.test.ts PAPERCLIP_ENABLE_DARWIN_SSH_ENV_LAB=1 pnpm exec vitest run --maxWorkers=2 \ server/src/services/ssh-remote-workspace.integration.test.ts pnpm exec vitest run packages/adapter-utils/src/acpx-engine/remote-spawn-smoke.test.ts pnpm --filter @paperclipai/adapter-utils typecheck pnpm --filter @paperclipai/adapter-claude-local typecheck pnpm --filter @paperclipai/adapter-codex-local typecheck pnpm --filter @paperclipai/server typecheck ``` The new admission and staging cases failed before the source fix. The first whole-heartbeat follow-up test also changed an agent setting between turns. That correctly reset the session. The fixture now keeps those settings unchanged and verifies the resumed conversation. Running both engine cases together also exposed duplicate company prefixes and environment names in the fixture; each case now uses its own names. The SSH qualification uses a real isolated OpenSSH server, real Postgres, real ACPX runtime and bridges, and a synthetic ACP provider with a selected test account. It does not call a live provider or run an agent on a user's computer. ## Risks The change affects the shared ACP staging path, restricted to SSH targets that declare in-place realization. Sandbox staging retains its prior behavior, including the existing Codex in-place refusal. Copy-mode SSH sessions retain their prior directory layout. Previously saved sessions keep their format; this adds no migration or session reset. ACP still requires its server command on the remote login PATH. Host authentication and local confinement remain CLI-only. Cold controller/provider reconnection and live Claude/Codex provider acceptance are not qualified by this test. No staging deployment or existing agent configuration is changed by this pull request. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) through the Codex harness, with xhigh reasoning effort and code tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>
253 lines
15 KiB
Markdown
253 lines
15 KiB
Markdown
# Use Tailscale machines in SSH environments
|
||
|
||
The Tailscale connector gives Paperclip access to a tailnet. Paperclip joins a
|
||
separate device to that network, then uses the existing SSH environment driver
|
||
to run tasks on selected computers. The connector is managed from **Connectors**
|
||
and can serve multiple SSH environments in the same company. The computer is
|
||
not replaced or provisioned by Paperclip.
|
||
|
||
In **Connectors**, find **Tailscale**, open it and click **Connect Tailscale Network**.
|
||
After approving consent, click **Create SSH environment**, or create an SSH
|
||
environment and select **Tailscale** under **Network**. Choose the connection
|
||
and machine, then configure the SSH user, port, absolute workspace path, private
|
||
key, and trusted host keys. The Tailscale network option appears once a tailnet
|
||
is connected. On a Mac
|
||
with the normal Tailscale application, enable macOS Remote Login. Tailscale SSH
|
||
server support on macOS requires the open-source tailscaled variant; ordinary
|
||
OpenSSH over Tailscale works with the standard Mac application.
|
||
|
||
Device sign-in opens Tailscale in a separate tab and keeps the connector page
|
||
open. A normal sign-in link remains available if the browser blocks the tab.
|
||
Approve the Paperclip device in your own tailnet; no OAuth app, API key, Cloud
|
||
broker credential, or per-instance pilot admission is needed. If your tailnet
|
||
requires device approval, the connector shows that separate pending state.
|
||
Sign-in expires after ten minutes and can be cancelled from the connector.
|
||
|
||
SSH setup hides the Network selector when Direct connection is the only
|
||
available option. A sole connected tailnet is selected automatically and its
|
||
connection selector is hidden; multiple connected tailnets remain selectable.
|
||
Switching an existing environment to **Direct connection** clears its saved
|
||
Tailscale machine and credential references and saves the direct host you enter.
|
||
|
||
After selecting a machine, **Prepare machine pairing** creates an SSH key in
|
||
Paperclip’s vault and displays a command containing only the public key. Each
|
||
preparation generates a fresh key pair. The command uses the public key produced
|
||
during that request, never mutable metadata from a saved secret. Run it
|
||
as the target SSH user. The command preserves existing authorized keys, including
|
||
files without a final newline. Before installing the key, it checks for readable
|
||
Ed25519, ECDSA, or RSA host public keys in `/etc/ssh` and stops without changing
|
||
`authorized_keys` if none are available. It prints that user's name, home
|
||
workspace path, and known-hosts entries for every available host key for the
|
||
form. You can also select an existing SSH private key secret.
|
||
|
||
Pairing output is tied to the selected company, tailnet, machine, and SSH port.
|
||
After editing the port, prepare pairing again for matching known-hosts output.
|
||
A late response for an earlier port cannot replace the selected private key.
|
||
Switching to Direct connection or closing the form also prevents pending pairing
|
||
results from changing the selected key.
|
||
|
||
Save the environment and test it. Select it as an agent’s environment and choose
|
||
any harness supported by SSH environments. Select the agent's AI account in
|
||
Paperclip. Runs use that account's subscription or API key, with private
|
||
credential directories on the target computer, just as sandbox runs do. The
|
||
computer supplies the installed harness and workspace; its SSH user's provider
|
||
login is not used for an agent with a configured AI account. Test the chosen
|
||
harness separately: SSH access does not prove provider authentication.
|
||
|
||
SSH's **Advanced** section includes **Use authentication configured on the remote
|
||
machine**. New environments leave it off, requiring the agent's Paperclip AI
|
||
account. Turn it on to use the remote user's provider login instead. Existing
|
||
SSH environments display it as on; opening or upgrading an installation does
|
||
not rewrite their saved configuration. The same option applies to direct SSH
|
||
and Tailscale connections.
|
||
|
||
## SSH on a volume-free Cloud stack
|
||
|
||
Cloud SSH execution uses the same durable task workspace storage as Cloud
|
||
sandboxes. Claude Code and Codex use their existing SSH adapters. With an
|
||
agent-selected Paperclip AI account, the default ACP engine runs directly in the
|
||
remote task directory and stages only private runtime assets. The CLI engine
|
||
also supports this directory. ACP requires its server command, such as
|
||
`claude-agent-acp`, on the SSH login PATH. Selecting SSH does not change the
|
||
agent's engine or permission settings.
|
||
|
||
The SSH account supplies the installed harness; the agent's Paperclip AI account
|
||
supplies authentication unless the remote authentication option is enabled.
|
||
Host authentication remains supported by the CLI engine.
|
||
|
||
Each task receives a private directory below the configured SSH workspace path,
|
||
inside `.paperclip-workspaces`. The task runs in that remote directory. Core
|
||
keeps the workspace binding and saved file versions in its database and object
|
||
storage. A Cloud restart reconnects to the same remote files. Restoring a saved
|
||
workspace to another allocation uses the existing snapshot verification and
|
||
single-writer rules. It does not restore private provider credentials from a
|
||
task snapshot.
|
||
|
||
The SSH machine needs Node.js and Git on its login PATH in addition to its agent
|
||
harness. The workspace owner uses Node.js for preparation and verified file
|
||
archives. SSH access alone does not qualify these dependencies or the selected
|
||
AI account.
|
||
|
||
An existing server-local workspace is not automatically imported. Cloud still
|
||
rejects local filesystem authority and unsupported adapters. An uncertain SSH
|
||
exit or failed save retains the remote files and the writer hold; it does not
|
||
authorize another run to overwrite them. Paperclip does not stop or delete the
|
||
SSH computer. This change does not add automatic reconciliation of an unknown
|
||
remote provider process after an SSH connection is lost.
|
||
|
||
The environment keeps the `ssh` driver. Machine selection stores the connector
|
||
ID and Tailscale’s stable device ID, and runtime
|
||
resolution looks up its current address through that connection’s private
|
||
socket. Offline, removed, disconnected, and wrong-company machines fail closed.
|
||
The host’s existing Tailscale connection is never a fallback.
|
||
|
||
## Device sign-in and credential custody
|
||
|
||
It is a network connector on the Connectors surface; it does not publish agent
|
||
tools or enter MCP discovery. Management requires an instance administrator with
|
||
access to the company. Only the person who started a pending sign-in can retrieve
|
||
its sign-in link. Existing SSH environment access governs which agents execute
|
||
there. Cloud and self-hosted instances use the same Tailscale device sign-in.
|
||
For Cloud owners, the board access summary reports the effective instance-admin
|
||
rights computed by Cloud authentication while `enableOwnerInstanceAdmin` is
|
||
enabled. Disabling that setting removes those rights on the next request.
|
||
|
||
The instance starts a private userspace Tailscale device and returns its generated
|
||
`https://login.tailscale.com/a/...` sign-in address. Pending device state is stored
|
||
in the configured company vault before the address is returned. The active
|
||
`tailscale up` command watches Tailscale state until consent and any required
|
||
device approval complete, or the ten-minute deadline expires. It saves the
|
||
registered device state before marking the connection connected. Pending sign-ins
|
||
can resume from the saved state after a server restart without extending their
|
||
deadline. There is no background database scan: browser progress requests exist
|
||
only during the bounded sign-in, and the CLI waits for Tailscale state events.
|
||
|
||
The previous OAuth callback remains available for already-started pilot flows;
|
||
existing enrolled devices continue to use their saved state. New connections
|
||
use device sign-in and do not call the Cloud OAuth broker.
|
||
|
||
The connection records
|
||
the exact trusted vault version and a SHA-256 digest of the server-written value.
|
||
Runtime reads that version and verifies its digest. A generic company secret
|
||
rotation cannot change the authorized network. Trusted node-state saves advance
|
||
both fields together, with a compare-and-set guard so an older save cannot
|
||
replace a newer state. Reads keep using the previous trusted version until the
|
||
new version is committed. Private node state, digests and enrollment credentials
|
||
never appear in connection or environment responses.
|
||
|
||
Enrollment tokens and node state are server-owned secrets with a durable
|
||
`system_purpose` classification. Generic company catalogs omit them. Generic
|
||
bindings, environment injection, and agent value reads reject them, including
|
||
bindings saved before the restriction. Renaming or rotating a secret cannot
|
||
remove this classification. Disconnecting or replacing a connection leaves its
|
||
old credential protected. Only the connector's server consumer can resolve its
|
||
trusted value; an SSH environment binds its private key, never the node state.
|
||
Unclassified legacy node-state values are also rejected by generic reads,
|
||
including state left behind by an earlier connection replacement.
|
||
Migration `0482_aberrant_rumiko_fujikawa` classifies existing connector-referenced secrets.
|
||
Apply it before running the updated server. If node state was copied before
|
||
this restriction, revoke that enrolled device in Tailscale; a local disconnect
|
||
cannot stop a copy outside Paperclip.
|
||
|
||
Connections created before migration `0481_round_pet_avengers` have no trusted version
|
||
and digest. They fail closed and require a fresh **Connect Tailscale Network**
|
||
consent. The migration leaves these fields empty; it does not trust or backfill a
|
||
generic secret's latest value.
|
||
|
||
Each connection has a separate userspace daemon and private socket. A stale
|
||
credential read reloads the trusted value before replacing a ready daemon and
|
||
rechecks which daemon is active after the reload. It preserves a healthy newer
|
||
socket and cannot cross a manager shutdown. Failed connection requests leave
|
||
cleanup to the network manager's exact client; they cannot disconnect a newer
|
||
retry using the same connector identity. The daemon accepts no subnet routes or
|
||
DNS changes and blocks incoming connections. Normal shutdown saves
|
||
node state and removes temporary files. A restart restores the enrolled device;
|
||
it does not reuse the OAuth token or require consent for every task. Disconnect
|
||
invalidates pending consent and stops the connection’s local daemon. A callback
|
||
already exchanging or saving credentials cannot reconnect after disconnect or
|
||
replace a newer authorization. Completion retains the consumed state's digest
|
||
as a private attempt marker, with no expiry; disconnect replaces that marker.
|
||
|
||
Database writes and credential reloads verify ownership without holding a database
|
||
lock across broker or daemon startup. Cancelled attempts close their daemon and
|
||
remove their newly created secret without touching a newer connection's state.
|
||
Disconnect does not remove the target computer from Tailscale.
|
||
|
||
Company deletion removes connector records before vault secrets in the company
|
||
transaction. After commit, it stops every daemon owned by those records. A
|
||
rolled-back deletion leaves the connector and its daemon intact. Deleting a
|
||
connector's vault secret invalidates pending consent, disconnects its network,
|
||
and clears the trusted credential reference before removing the secret. If
|
||
shutdown fails, the disconnected reference remains available for a deletion
|
||
retry. Late daemon state writes cannot restore a retired connector.
|
||
|
||
Task runtime resolution obtains its socket from this same guarded connection
|
||
service after loading the SSH key. It requires the connection to remain connected
|
||
with the environment's saved credential identity during credential reload,
|
||
daemon startup, and device discovery. A disconnect or replacement rejects the
|
||
run before returning a socket; runtime resolution cannot restart the connection
|
||
from a stale saved secret.
|
||
Saved network credentials without a connector and machine selection are rejected;
|
||
all Tailscale environments use the connector's lifecycle.
|
||
|
||
This implementation currently supports one serving process per instance.
|
||
Concurrent replicas need a durable connection ownership lease before rollout.
|
||
|
||
Tailscale identity belongs to the persisted machine key in the saved node state,
|
||
not the container ID or hostname. A redeploy must retain both the database's
|
||
encrypted node state and the same vault master key. Cloud already materializes
|
||
the stable per-stack `PAPERCLIP_SECRETS_MASTER_KEY` across release rolls. Do not
|
||
make the connector ephemeral or replace its node state at each start. Losing
|
||
that state, device removal, or a tailnet reauthentication policy can require
|
||
fresh consent. The app-owned device has the consenting user's tailnet rights.
|
||
|
||
## Local development
|
||
|
||
Run the Paperclip instance at `http://localhost:3100`. No local Cloud broker or
|
||
Tailscale OAuth app is required. The instance needs `tailscale` and `tailscaled`.
|
||
For a separate local userspace client, install the pinned programs outside the
|
||
checkout:
|
||
|
||
```sh
|
||
GOBIN="$HOME/.local/share/paperclip-tailscale/bin" go install tailscale.com/cmd/tailscale@v1.104.5
|
||
GOBIN="$HOME/.local/share/paperclip-tailscale/bin" go install tailscale.com/cmd/tailscaled@v1.104.5
|
||
```
|
||
|
||
Set `PAPERCLIP_TAILSCALE_CLI` and `PAPERCLIP_TAILSCALED` to their absolute paths.
|
||
These settings can be set in the supported `.env` files. Each new client reads
|
||
them at startup, after environment loading.
|
||
|
||
## Verification and rollout gates
|
||
|
||
Offline tests cover device sign-in without an auth key, saved state and restart,
|
||
separate company identities, initiator binding, cancellation and expiry, stale
|
||
completion, trusted credential versions, device approval, popup fallback, and
|
||
route authorization/CSRF. Existing tests retain legacy broker callback, SSH,
|
||
pairing, and network lifecycle coverage. These do not prove live consent or
|
||
remote execution.
|
||
|
||
Before rollout, qualify two unrelated customer tailnets through ordinary device
|
||
sign-in, machine discovery, and an agent task using its selected AI account that
|
||
writes a file on the selected computer, result streaming, cancellation, server
|
||
restart, machine sleep/reboot, and revocation. A local instance whose public URL
|
||
is localhost also needs an API path reachable by the remote agent for tasks that
|
||
call back to Paperclip unless the adapter uses the existing remote callback
|
||
bridge. Production Cloud uses its public HTTPS stack URL.
|
||
|
||
The connector's bundled artwork comes from the vendor's
|
||
`https://tailscale.com/favicon.png`, copied without modification (SHA-256
|
||
`680570c9090b604450792f68ceed10beeb33a69c9bccc2a92e398564c7075a5c`).
|
||
|
||
The old managed OAuth profile remains disabled unless explicitly configured for
|
||
its existing pilot. Its same-tailnet registration restriction does not apply to
|
||
the new device sign-in path. Do not configure a customer-specific OAuth app or
|
||
admit a pilot instance to make device sign-in work.
|
||
|
||
Provider references:
|
||
|
||
- [Device authentication](https://tailscale.com/docs/features/tsnet#device-creation-and-authentication)
|
||
- [Tailscale CLI](https://tailscale.com/docs/reference/tailscale-cli/up)
|
||
- [OAuth apps](https://tailscale.com/docs/features/oauth-apps)
|
||
- [Device provisioning](https://tailscale.com/docs/features/oauth-apps/device-provisioning)
|
||
- [Tailscale SSH](https://tailscale.com/docs/features/tailscale-ssh)
|