mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-17 13:01:05 +01:00
fix(runtime): keep managed GitHub access across a controller restart (#21928)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Native runs give agents managed GitHub access through `gh` and `git` launchers on the provider's `PATH` > - A provider process can keep running while the server restarts, and the next controller adopts it > - The detaching controller deleted the launcher directory, and the adopting controller staged launchers in a new random directory > - The running provider still names the old directory, so every later `gh` or `git` call ran without credentials > - This pull request keeps the launcher directory stable for one provider session and task, and re-arms it after the restart > - The benefit is that a long run keeps its GitHub identity across a server deploy or restart ## Linked Issues or Issue Description No public issue exists. This describes the bug in-PR, with the bug template labels. **What happened?** An agent run on a Daytona sandbox used `gh` with managed GitHub access. The server restarted during the run (a rolling deploy), and the controller re-attached to the live provider process. After that, every `gh` call failed with `You are not logged into any GitHub hosts`. `git credential fill` returned no token. The agent reported that its token had "expired" and used the unauthenticated public API for the rest of the run. **Steps to reproduce** 1. Start a native run with managed GitHub access on a remote sandbox. 2. Run a `gh` command in the run. It succeeds. 3. Restart the server while the provider process is still running. The new controller re-attaches to the process. 4. Run a `gh` command again in the same run. It fails with "not logged in", and no `Paperclip: GitHub ...` diagnostic appears. **Expected behavior** After the new controller adopts the provider, `gh` and `git` get credentials from the new controller, as they did before the restart. **Root cause** - `createNativeGitHubAccess` staged the launchers in `native-session-<random UUID>` and put that path in the provider's `PATH`, `BASH_ENV`, and `GH_CONFIG_DIR`. - On a restart detach, `githubAccess.stop()` ran `rm -rf` on that directory (`detachReleasedWarmNativeSession`, and the run's final `github_stop` while `nativeRunsDetachingForRestart` is set). - The adopting controller made a new random directory. The provider environment cannot change after spawn, so the provider never saw it. - The shims were gone. `gh` resolved to the real binary with an empty `GH_CONFIG_DIR`, so no Paperclip diagnostic appeared. Refs #15354. That PR keeps ACPX sessions across launcher rotation in the legacy adapter path. This PR covers the native runtime path. ## What Changed - `createNativeGitHubAccess` takes an optional `launcherKey`. The launcher directory is `native-github-<sha256(key)>`, so the same provider session and task always get the same directory. - The native session executor passes a key built from `nativeSessionScopeKey` and the issue id. - `prepareGitHubOperationLaunchers` takes an optional `broker`. It then stages broker-file launchers and an owner-only `broker.json`. The launchers already read `broker.json` on each operation in this mode (`githubLauncherSource({ brokerFile: true })`). A restaged directory therefore gives the running provider the new endpoint. - Native access passes `broker` only for remote sandbox targets. A sandbox provider is the one that outlives a controller restart. Local launchers live in the shared host temp directory, so they keep the capability in the provider environment only, as before. - `stop({ preserveLaunchers: true })` keeps the launcher directory. A restart detach of an active run and of a released warm session use this option. A normal stop still removes the directory. - A final stop does not remove a directory that a newer access in the same process has already restaged. - The callback bridge runtime stays in its own per-access directory. A new bridge never shares a queue directory with an old bridge. - `syncRemoteTextFiles` takes a `private` flag. It sets `umask 077` for the whole staging script, so the upload copy, the partial file, and the final file are never readable by group or other users. Local launcher files are written to a temporary file and renamed, so a running launcher never reads a partly written file. ## Verification - New `native-github-access` test on a sandbox target: a live parent process gets credentials, the access stops with `preserveLaunchers`, and a new access with the same key re-arms the directory. The same parent PID then gets the new controller's credentials, with empty stderr. `broker.json` has mode `600`. The final stop removes the directory. - New previous-release cutover test: a provider launched with the old random-directory layout keeps working without managed GitHub after its controller removes the directory. The new controller does not resolve credentials for it. A provider that the new controller launches gets credentials. - New local regression: local launcher files do not contain the broker capability, and there is no `broker.json`. - New local restart test: a local provider adopted after a restart gets a Paperclip diagnostic and no credentials. The new controller resolves nothing for the old endpoint. - New `sandbox-callback-bridge` test: staging a private file runs with umask `077` and leaves the file at mode `600`. - New `native-github-access` test: when a superseded access stops, its successor's launchers stay usable. A different key gives a different directory. - New `native-session-executor` test: a run detached for restart while active stops its GitHub access with `{ preserveLaunchers: true }`, and creates it with a `launcherKey`. The existing overlapping-failure test now asserts `{ preserveLaunchers: false }`. - The new tests fail on the old behavior. I checked this by forcing `brokerFile: false`, forcing `preserveLaunchers = false`, removing the `umask 077` line, and writing `broker.json` for local targets. - Passed: `native-github-access` (26), `native-session-executor` + `github-operation-credentials` + `heartbeat-github-launchers` + `native-stop-accounting` + `heartbeat-project-env` (1640 passed, 7 skipped), `native-runner-restart-recovery.integration` and `heartbeat-native-restart-suspension`. - Passed: adapter-utils `execution-target`, `github-launcher`, `github-launcher-environment`, `sandbox-callback-bridge`; runner `acpx/github-launcher`, `acpx/environment`, `codex/codex-security-config`. - `pnpm --filter @paperclipai/adapter-utils typecheck` and the server `pnpm typecheck` pass. - Not run locally: `native-warm-checkpoint-ownership.integration` needs the compiled `paperclip-runnerd` binary. CI builds it. ## Previous-release story - No persisted run, session, checkpoint, or database state changes. Only the in-sandbox launcher layout changes. - A provider launched by the previous release has a random `native-session-*` launcher directory in its environment. The previous release's controller removes that directory at the cutover restart. The new controller cannot find it, so it does not re-arm it. - That provider carries forward without managed GitHub until its next spawn, which is the current behavior. The run is not refused and is not retried. `gh` and `git` run without managed credentials, as they do today after a restart. The cutover test covers this. - A provider that the new release launches gets the stable directory and survives later restarts. ## Risks - On sandbox targets, the provider can now read the bridge capability in `broker.json`, as well as in its environment. The file holds the same per-access transport token that the environment already holds, not a run token or a GitHub credential. The file and its staging copies are mode `600`, and the broker still checks the active run and grants on each request. Local targets write no `broker.json`. - Local providers that outlive a restart still lose managed GitHub. This is not a regression: master deletes the launcher directory at the detach, so the old local provider also runs without credentials there. With this PR, the local launchers stay in place and print a `Paperclip: GitHub ...` diagnostic instead of a silent "not logged in". Re-arming local providers needs a broker channel that other local runs under the same user cannot read. That is out of scope here, because `broker.json` in the shared temp directory was a security finding on this PR. - On sandbox targets, `git` and `gh` children no longer inherit `PAPERCLIP_API_URL`, `PAPERCLIP_API_KEY`, or `PAPERCLIP_GITHUB_BRIDGE_TOKEN`. Broker-file mode removes them, as it already did in its existing tests. - After a restart detach, a launcher directory stays in the workspace runtime directory. If no controller adopts the provider, that directory stays until a later access with the same key stops, or until the workspace is removed. It contains no GitHub credential. - Between the detach and the adoption, `gh` gets a transport error from the old endpoint. It then prints `broker_transport_unavailable` and continues without credentials, as before. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, test runs). Standard context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
0308108531
commit
cc9ff75cb8
7 files changed
+222
-22
No files matched your search
@@ -1832,6 +1832,7 @@ printf '\0PAPERCLIP_GIT_CONTEXT_END\0'
|
||||
/** Stage token-free launchers next to the execution, not in shared global Git config. */
|
||||
export async function prepareGitHubOperationLaunchers(input: {
|
||||
runId: string; target: AdapterExecutionTarget | null | undefined; cwd: string; env: Record<string, string>;
|
||||
broker?: { url: string; token: string };
|
||||
}): Promise<Record<string, string>> {
|
||||
const remote = input.target?.kind === "remote" ? input.target : null;
|
||||
const directory = githubOperationLauncherDirectory(input);
|
||||
@@ -1851,8 +1852,9 @@ export async function prepareGitHubOperationLaunchers(input: {
|
||||
// Remote launchers live beneath the checkout. Pin their own package scope
|
||||
// so an enclosing project's "type": "module" cannot reinterpret require().
|
||||
["package.json", '{"type":"commonjs"}\n'],
|
||||
...["git", "gh"].map((name) => [name, githubLauncherSource()] as const),
|
||||
...["git", "gh"].map((name) => [name, githubLauncherSource({ brokerFile: input.broker !== undefined })] as const),
|
||||
...[".zshenv", ".zprofile", ".zshrc", ".bash_profile", ".bashrc", ".profile"].map((name) => [name, profile] as const),
|
||||
...(input.broker ? [["broker.json", `${JSON.stringify({ url: input.broker.url, token: input.broker.token })}\n`] as const] : []),
|
||||
]);
|
||||
if (remote) {
|
||||
const runner = adapterExecutionTargetCommandRunner(remote);
|
||||
@@ -1863,6 +1865,7 @@ export async function prepareGitHubOperationLaunchers(input: {
|
||||
label: "GitHub operation launcher", action: "stage GitHub operation launcher",
|
||||
lockDir: path.posix.join(directory, `.${program}.lock`),
|
||||
executable: program === "git" || program === "gh",
|
||||
private: program === "broker.json",
|
||||
})),
|
||||
ensureDirectories: [configDirectory], action: "stage GitHub operation launchers",
|
||||
timeoutMs: 15_000, shellCommand: adapterExecutionTargetShellCommand(remote),
|
||||
@@ -1870,7 +1873,11 @@ export async function prepareGitHubOperationLaunchers(input: {
|
||||
} else {
|
||||
await fs.mkdir(directory, { recursive: true, mode: 0o700 });
|
||||
await fs.mkdir(configDirectory, { recursive: true, mode: 0o700 });
|
||||
for (const [program, body] of Object.entries(files)) await fs.writeFile(path.join(directory, program), body, { mode: 0o700 });
|
||||
for (const [program, body] of Object.entries(files)) {
|
||||
const partial = path.join(directory, `.${program}.${randomUUID()}.partial`);
|
||||
await fs.writeFile(partial, body, { mode: program === "broker.json" ? 0o600 : 0o700 });
|
||||
await fs.rename(partial, path.join(directory, program));
|
||||
}
|
||||
}
|
||||
return { ...input.env, PATH: managedPath, ZDOTDIR: directory, BASH_ENV: `${directory}/.bashrc`,
|
||||
GH_CONFIG_DIR: configDirectory, PAPERCLIP_GITHUB_LAUNCHER_DIR: directory };
|
||||
|
||||
@@ -12,6 +12,7 @@ export interface RemoteTextFileSyncFile {
|
||||
action: string;
|
||||
lockDir: string;
|
||||
executable?: boolean;
|
||||
private?: boolean;
|
||||
}
|
||||
|
||||
export interface RemoteTextFileSyncInput extends RemoteTextFileSyncFile {
|
||||
@@ -52,6 +53,7 @@ function program(input: RemoteTextFileSyncFile) {
|
||||
const sha256 = createHash("sha256").update(input.body, "utf8").digest("hex");
|
||||
const script = [
|
||||
"set -eu",
|
||||
...(input.private ? ["umask 077"] : []),
|
||||
`remote_dir=${shellQuote(input.remoteDir)}`,
|
||||
`remote_path=${shellQuote(input.remotePath)}`,
|
||||
`remote_partial=${shellQuote(remotePartial)}`,
|
||||
@@ -79,7 +81,7 @@ function program(input: RemoteTextFileSyncFile) {
|
||||
" current_sha=\"$(hash_file \"$remote_path\" 2>/dev/null)\" || current_sha=\"\"",
|
||||
"fi",
|
||||
"if [ -n \"$current_sha\" ] && [ \"$current_sha\" = \"$expected_sha\" ]; then",
|
||||
...(input.executable ? [' chmod 700 "$remote_path"'] : []),
|
||||
...(input.executable ? [' chmod 700 "$remote_path"'] : input.private ? [' chmod 600 "$remote_path"'] : []),
|
||||
" printf '{\"uploaded\":false}\\n'",
|
||||
" exit 0",
|
||||
"fi",
|
||||
@@ -94,7 +96,7 @@ function program(input: RemoteTextFileSyncFile) {
|
||||
"else",
|
||||
` echo ${shellQuote(`${input.label} sha verify skipped: no sha256sum/shasum on remote.`)} >&2`,
|
||||
"fi",
|
||||
...(input.executable ? ['chmod 700 "$remote_partial"'] : []),
|
||||
...(input.executable ? ['chmod 700 "$remote_partial"'] : input.private ? ['chmod 600 "$remote_partial"'] : []),
|
||||
"mv \"$remote_partial\" \"$remote_path\"",
|
||||
"printf '{\"uploaded\":true}\\n'",
|
||||
].join("\n");
|
||||
|
||||
@@ -1446,6 +1446,31 @@ describe("sandbox callback bridge", () => {
|
||||
).resolves.toEqual([]);
|
||||
});
|
||||
|
||||
it("creates every staged copy of a private file without group or other access", async () => {
|
||||
const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-private-sync-"));
|
||||
cleanupDirs.push(rootDir);
|
||||
const shimDir = path.join(rootDir, "shim");
|
||||
const umaskLog = path.join(rootDir, "umask.log");
|
||||
await mkdir(shimDir);
|
||||
await writeFile(path.join(shimDir, "base64"),
|
||||
`#!/bin/sh\numask > ${JSON.stringify(umaskLog)}\nPATH=${JSON.stringify(process.env.PATH ?? "")} exec base64 "$@"\n`, { mode: 0o700 });
|
||||
const inner = createExecRunner();
|
||||
const runner = {
|
||||
execute: (input: Parameters<typeof inner.execute>[0]) =>
|
||||
inner.execute({ ...input, env: { ...input.env, PATH: `${shimDir}:${process.env.PATH ?? ""}` } }),
|
||||
};
|
||||
const remoteDir = path.join(rootDir, "launchers");
|
||||
const remotePath = path.posix.join(remoteDir, "broker.json");
|
||||
const result = await syncRemoteTextFileWithHashSkip({
|
||||
runner, remoteCwd: rootDir, remoteDir, remotePath, body: "{\"token\":\"fixture\"}\n",
|
||||
label: "Private fixture", action: "sync private fixture",
|
||||
lockDir: path.posix.join(remoteDir, ".broker.json.lock"), private: true, timeoutMs: 30_000,
|
||||
});
|
||||
expect(result.uploaded).toBe(true);
|
||||
expect((await readFile(umaskLog, "utf8")).trim()).toMatch(/^0?077$/);
|
||||
expect((await fs.stat(remotePath)).mode & 0o777).toBe(0o600);
|
||||
});
|
||||
|
||||
it("test_process_session_script_rewritten_on_hash_mismatch: a mismatched remote hash still rewrites the script", async () => {
|
||||
const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-hashskip-cold-"));
|
||||
cleanupDirs.push(rootDir);
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { afterEach, expect, it, vi } from "vitest";
|
||||
import { spawn, execFile } from "node:child_process";
|
||||
import { mkdtemp, mkdir, readFile, rm, writeFile, access } from "node:fs/promises";
|
||||
import { mkdtemp, mkdir, readdir, readFile, rm, stat, writeFile, access } from "node:fs/promises";
|
||||
import { promisify } from "node:util";
|
||||
import type { CommandManagedDuplexChannel, CommandManagedRuntimeRunner } from "@paperclipai/adapter-utils/command-managed-runtime";
|
||||
import type { DuplexObservabilityRecorder } from "@paperclipai/adapter-utils/duplex-observability";
|
||||
@@ -40,11 +40,12 @@ function localDuplexChannel(command: readonly string[], cwd: string): CommandMan
|
||||
async function broker(resolveCredentials = vi.fn(async (binding: ReturnType<typeof run>) => ({
|
||||
status: "available", env: { GH_TOKEN: `fixture-${binding.runId}` },
|
||||
})), remote = false, bridgeUnavailable = false, onLog?: (stream: "stdout" | "stderr", chunk: string) => Promise<void>,
|
||||
duplex?: { enabled?: boolean; capability: boolean; unavailable?: boolean; startupFailure?: "lost" | "unreachable"; fallbackUnavailable?: boolean; fileProbeFails?: boolean; recorder?: DuplexObservabilityRecorder }) {
|
||||
const root = await mkdtemp(path.join(tmpdir(), "native-github-reuse-"));
|
||||
cleanups.push(() => rm(root, { recursive: true, force: true }));
|
||||
duplex?: { enabled?: boolean; capability: boolean; unavailable?: boolean; startupFailure?: "lost" | "unreachable"; fallbackUnavailable?: boolean; fileProbeFails?: boolean; recorder?: DuplexObservabilityRecorder },
|
||||
placement: { root?: string; launcherKey?: string } = {}) {
|
||||
const root = placement.root ?? await mkdtemp(path.join(tmpdir(), "native-github-reuse-"));
|
||||
if (!placement.root) cleanups.push(() => rm(root, { recursive: true, force: true }));
|
||||
const bin = path.join(root, "real-bin");
|
||||
await mkdir(bin);
|
||||
await mkdir(bin, { recursive: true });
|
||||
await writeFile(path.join(bin, "gh"), `#!${process.execPath}\nprocess.stdout.write(process.env.GH_TOKEN || 'anonymous');`, { mode: 0o700 });
|
||||
const execute = vi.fn<CommandManagedRuntimeRunner["execute"]>(async (input) => {
|
||||
const startedAt = new Date().toISOString();
|
||||
@@ -70,7 +71,8 @@ duplex?: { enabled?: boolean; capability: boolean; unavailable?: boolean; startu
|
||||
const bridgeModes: (string | undefined)[] = [];
|
||||
const bridges: NonNullable<Awaited<ReturnType<typeof startAdapterExecutionTargetPaperclipBridge>>>[] = [];
|
||||
const result = await createNativeGitHubAccess({ scope, target, cwd: root,
|
||||
env: { PATH: `${bin}:${path.dirname(process.execPath)}:/usr/bin:/bin`, PAPERCLIP_API_KEY: "old-run-api-key" }, resolveCredentials, onLog },
|
||||
env: { PATH: `${bin}:${path.dirname(process.execPath)}:/usr/bin:/bin`, PAPERCLIP_API_KEY: "old-run-api-key" }, resolveCredentials, onLog,
|
||||
launcherKey: placement.launcherKey },
|
||||
bridgeUnavailable ? async () => { throw new Error("fixture bridge unavailable"); } : async input => {
|
||||
if (duplex?.fallbackUnavailable && input.enableSandboxDuplexBridge === false) throw new Error("fixture fallback unavailable");
|
||||
const bridge = await startAdapterExecutionTargetPaperclipBridge(input);
|
||||
@@ -273,6 +275,103 @@ it.each([false, true])("keeps one live parent and its original launcher environm
|
||||
releaseB(); await b.stop();
|
||||
await expect(access(b.env.PAPERCLIP_GITHUB_LAUNCHER_DIR)).rejects.toThrow();
|
||||
}, 45_000);
|
||||
function liveParent(env: Record<string, string>) {
|
||||
const parent = spawn(process.execPath, ["-e", `
|
||||
const {execFile}=require('node:child_process');
|
||||
require('node:readline').createInterface({input:process.stdin}).on('line', () => {
|
||||
execFile('gh', [], (error, stdout, stderr) => console.log(JSON.stringify({pid:process.pid, value:stdout, error:error?.message, stderr})));
|
||||
});
|
||||
`], { env: { ...process.env, ...env }, stdio: ["pipe", "pipe", "pipe"] });
|
||||
cleanups.push(async () => { const exited = new Promise<void>(resolve => parent.once("exit", () => resolve())); parent.kill(); await exited; });
|
||||
const lines = createInterface({ input: parent.stdout });
|
||||
return () => new Promise<{ pid: number; value: string; stderr: string }>(resolve => {
|
||||
lines.once("line", line => resolve(JSON.parse(line))); parent.stdin.write("run\n");
|
||||
});
|
||||
}
|
||||
it("re-arms a live sandbox provider's launchers after its controller restarts", async () => {
|
||||
const remote = true;
|
||||
const root = await mkdtemp(path.join(tmpdir(), "native-github-restart-"));
|
||||
cleanups.push(() => rm(root, { recursive: true, force: true }));
|
||||
const before = await broker(undefined, remote, false, undefined, undefined, { root, launcherKey: "session-a" });
|
||||
const operation = liveParent(before.env);
|
||||
const releaseBefore = before.activate(run("run-a"));
|
||||
const first = await operation();
|
||||
expect(first.value).toBe("fixture-run-a");
|
||||
releaseBefore();
|
||||
await before.stop({ preserveLaunchers: true });
|
||||
const launchers = before.env.PAPERCLIP_GITHUB_LAUNCHER_DIR;
|
||||
await access(path.join(launchers, "gh"));
|
||||
const after = await broker(vi.fn(async (binding: ReturnType<typeof run>) => ({
|
||||
status: "available", env: { GH_TOKEN: `adopted-${binding.runId}` },
|
||||
})), remote, false, undefined, undefined, { root, launcherKey: "session-a" });
|
||||
expect(after.env.PAPERCLIP_GITHUB_LAUNCHER_DIR).toBe(launchers);
|
||||
expect(after.env.PAPERCLIP_GITHUB_BROKER_TOKEN).not.toBe(before.env.PAPERCLIP_GITHUB_BROKER_TOKEN);
|
||||
const releaseAfter = after.activate(run("run-a"));
|
||||
const adopted = await operation();
|
||||
expect(adopted).toMatchObject({ pid: first.pid, value: "adopted-run-a", stderr: "" });
|
||||
expect(await readFile(path.join(launchers, "gh"), "utf8")).not.toContain(after.env.PAPERCLIP_GITHUB_BROKER_TOKEN);
|
||||
expect((await stat(path.join(launchers, "broker.json"))).mode & 0o777).toBe(0o600);
|
||||
releaseAfter();
|
||||
await after.stop();
|
||||
await expect(access(launchers)).rejects.toThrow();
|
||||
}, 45_000);
|
||||
it("keeps launchers a successor re-armed when a superseded access stops", async () => {
|
||||
const root = await mkdtemp(path.join(tmpdir(), "native-github-successor-"));
|
||||
cleanups.push(() => rm(root, { recursive: true, force: true }));
|
||||
const superseded = await broker(undefined, true, false, undefined, undefined, { root, launcherKey: "session-a" });
|
||||
const successor = await broker(vi.fn(async () => ({ status: "available", env: { GH_TOKEN: "successor" } })),
|
||||
true, false, undefined, undefined, { root, launcherKey: "session-a" });
|
||||
const operation = liveParent(superseded.env);
|
||||
await superseded.stop();
|
||||
successor.activate(run("run-b"));
|
||||
expect(await operation()).toMatchObject({ value: "successor", stderr: "" });
|
||||
const other = await broker(undefined, true, false, undefined, undefined, { root, launcherKey: "session-b" });
|
||||
expect(other.env.PAPERCLIP_GITHUB_LAUNCHER_DIR).not.toBe(successor.env.PAPERCLIP_GITHUB_LAUNCHER_DIR);
|
||||
});
|
||||
it("keeps a previous-release provider running without managed GitHub after the cutover restart", async () => {
|
||||
const root = await mkdtemp(path.join(tmpdir(), "native-github-cutover-"));
|
||||
cleanups.push(() => rm(root, { recursive: true, force: true }));
|
||||
const previous = await broker(undefined, true, false, undefined, undefined, { root });
|
||||
const operation = liveParent(previous.env);
|
||||
const releasePrevious = previous.activate(run("run-a"));
|
||||
expect((await operation()).value).toBe("fixture-run-a");
|
||||
releasePrevious();
|
||||
await previous.stop();
|
||||
await expect(access(previous.env.PAPERCLIP_GITHUB_LAUNCHER_DIR)).rejects.toThrow();
|
||||
const current = await broker(undefined, true, false, undefined, undefined, { root, launcherKey: "session-a" });
|
||||
expect(current.env.PAPERCLIP_GITHUB_LAUNCHER_DIR).not.toBe(previous.env.PAPERCLIP_GITHUB_LAUNCHER_DIR);
|
||||
current.activate(run("run-a"));
|
||||
expect(await operation()).toMatchObject({ value: "anonymous", stderr: "" });
|
||||
expect(current.resolveCredentials).not.toHaveBeenCalled();
|
||||
await expect(liveParent(current.env)()).resolves.toMatchObject({ value: "fixture-run-a" });
|
||||
}, 45_000);
|
||||
it("tells a local provider adopted after a restart that managed GitHub is unavailable", async () => {
|
||||
const root = await mkdtemp(path.join(tmpdir(), "native-github-local-restart-"));
|
||||
cleanups.push(() => rm(root, { recursive: true, force: true }));
|
||||
const before = await broker(undefined, false, false, undefined, undefined, { root, launcherKey: "session-a" });
|
||||
const operation = liveParent(before.env);
|
||||
const releaseBefore = before.activate(run("run-a"));
|
||||
expect((await operation()).value).toBe("fixture-run-a");
|
||||
releaseBefore();
|
||||
await before.stop({ preserveLaunchers: true });
|
||||
const after = await broker(undefined, false, false, undefined, undefined, { root, launcherKey: "session-a" });
|
||||
after.activate(run("run-a"));
|
||||
const adopted = await operation();
|
||||
expect(adopted.value).toBe("anonymous");
|
||||
expect(adopted.stderr).toMatch(/^Paperclip: GitHub (broker_transport_unavailable|capability_rejected|broker_response_unavailable); continuing without managed credentials\.\n$/);
|
||||
expect(after.resolveCredentials).not.toHaveBeenCalled();
|
||||
}, 45_000);
|
||||
it("keeps the broker capability out of local launcher files", async () => {
|
||||
const b = await broker(undefined, false, false, undefined, undefined, { launcherKey: "session-a" });
|
||||
const files = await readdir(b.env.PAPERCLIP_GITHUB_LAUNCHER_DIR, { recursive: true });
|
||||
expect(files).not.toContain("broker.json");
|
||||
for (const file of files) {
|
||||
const target = path.join(b.env.PAPERCLIP_GITHUB_LAUNCHER_DIR, file);
|
||||
if ((await stat(target)).isFile()) expect(await readFile(target, "utf8")).not.toContain(b.env.PAPERCLIP_GITHUB_BROKER_TOKEN);
|
||||
}
|
||||
b.activate(run("local"));
|
||||
expect(await liveParent(b.env)()).toMatchObject({ value: "fixture-local", stderr: "" });
|
||||
});
|
||||
it("rejects other scopes, concurrent bindings, browser requests and forged authority", async () => {
|
||||
const b = await broker();
|
||||
for (const field of ["companyId", "agentId", "issueId"] as const) {
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { randomBytes, randomUUID, timingSafeEqual } from "node:crypto";
|
||||
import { createHash, randomBytes, randomUUID, timingSafeEqual } from "node:crypto";
|
||||
import { createServer } from "node:http";
|
||||
import {
|
||||
cleanupGitHubOperationLaunchers,
|
||||
@@ -15,6 +15,8 @@ type Binding = { companyId: string; agentId: string; issueId: string; runId: str
|
||||
type LauncherInput = Parameters<typeof prepareGitHubOperationLaunchers>[0];
|
||||
export type NativeGitHubAccess = Awaited<ReturnType<typeof createNativeGitHubAccess>>;
|
||||
|
||||
const launcherOwners = new Map<string, symbol>();
|
||||
|
||||
/** A process-lifetime transport, with authority only while its controller owns a run.
|
||||
* No run token or GitHub credential is stored in the provider's environment/files.
|
||||
* The resolver still checks the active run and current identity/grants per operation.
|
||||
@@ -26,10 +28,17 @@ export async function createNativeGitHubAccess(input: {
|
||||
env: NodeJS.ProcessEnv;
|
||||
resolveCredentials: (binding: Binding) => Promise<unknown>;
|
||||
onLog?: (stream: "stdout" | "stderr", chunk: string) => Promise<void>;
|
||||
launcherKey?: string;
|
||||
}, startBridge = startAdapterExecutionTargetPaperclipBridge) {
|
||||
const duplexRequested = adapterExecutionTargetEnablesSandboxDuplexBridge(input.target);
|
||||
const token = randomBytes(32).toString("hex");
|
||||
const location = { runId: `native-session-${randomUUID()}`, target: input.target };
|
||||
const launcherLocation = input.launcherKey === undefined ? location : {
|
||||
runId: `native-github-${createHash("sha256").update(input.launcherKey).digest("hex").slice(0, 32)}`,
|
||||
target: input.target,
|
||||
};
|
||||
const launcherDirectory = githubOperationLauncherDirectory(launcherLocation);
|
||||
const launcherOwner = Symbol("GitHub launcher owner");
|
||||
let active: Binding | null = null;
|
||||
let stopped = false;
|
||||
let ready = true;
|
||||
@@ -82,14 +91,17 @@ export async function createNativeGitHubAccess(input: {
|
||||
});
|
||||
server.requestTimeout = 15_000;
|
||||
server.headersTimeout = 10_000;
|
||||
const stop = () => stopping ??= (async () => {
|
||||
const stop = (options: { preserveLaunchers?: boolean } = {}) => stopping ??= (async () => {
|
||||
stopped = true;
|
||||
active = null;
|
||||
server.closeAllConnections();
|
||||
const removeLaunchers = !options.preserveLaunchers && launcherOwners.get(launcherDirectory) === launcherOwner;
|
||||
if (removeLaunchers) launcherOwners.delete(launcherDirectory);
|
||||
const results = await Promise.allSettled([
|
||||
stopBridge(),
|
||||
new Promise<void>(resolve => server.close(() => resolve())),
|
||||
cleanupGitHubOperationLaunchers(location),
|
||||
...(launcherLocation !== location ? [cleanupGitHubOperationLaunchers(location)] : []),
|
||||
...(removeLaunchers ? [cleanupGitHubOperationLaunchers(launcherLocation)] : []),
|
||||
]);
|
||||
if (results.some(result => result.status === "rejected")) {
|
||||
await input.onLog?.("stderr", "[paperclip] GitHub session cleanup incomplete.\n").catch(() => undefined);
|
||||
@@ -161,15 +173,17 @@ export async function createNativeGitHubAccess(input: {
|
||||
await new Promise<void>(resolve => server.close(() => resolve()));
|
||||
await input.onLog?.("stderr", "[paperclip] GitHub runtime transport unavailable; continuing without managed GitHub access.\n").catch(() => undefined);
|
||||
}
|
||||
const broker = {
|
||||
url: ready ? bridge?.env.PAPERCLIP_API_URL ?? url : "",
|
||||
token: ready ? bridge?.env.PAPERCLIP_API_KEY ?? token : "",
|
||||
};
|
||||
launcherOwners.set(launcherDirectory, launcherOwner);
|
||||
const env = await prepareGitHubOperationLaunchers({
|
||||
...location, cwd: input.cwd,
|
||||
...launcherLocation, cwd: input.cwd, ...(input.target?.kind === "remote" ? { broker } : {}),
|
||||
env: {
|
||||
...githubBrokerEnvironment({ PATH: input.env.PATH }, {
|
||||
url: ready ? bridge?.env.PAPERCLIP_API_URL ?? url : "",
|
||||
token: ready ? bridge?.env.PAPERCLIP_API_KEY ?? token : "",
|
||||
}),
|
||||
...githubBrokerEnvironment({ PATH: input.env.PATH }, broker),
|
||||
// Never retain an old run's bridge authentication override.
|
||||
PAPERCLIP_GITHUB_BRIDGE_TOKEN: ready ? bridge?.env.PAPERCLIP_API_KEY ?? token : "",
|
||||
PAPERCLIP_GITHUB_BRIDGE_TOKEN: broker.token,
|
||||
},
|
||||
});
|
||||
return {
|
||||
|
||||
@@ -8692,6 +8692,49 @@ describe("native startup restart detachment", () => {
|
||||
} finally { await newerDetach; issueService.mockRestore(); }
|
||||
});
|
||||
|
||||
it("keeps managed GitHub launchers for a provider that outlives its restart detach", async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), "native-github-detach-"));
|
||||
const previous = process.env.PAPERCLIP_RUNNER_STATE_DIR;
|
||||
process.env.PAPERCLIP_RUNNER_STATE_DIR = root;
|
||||
const restarting = structuredClone(execution);
|
||||
restarting.binding.runId = "restart-managed-github";
|
||||
restarting.session.normalizedSessionId = "restart-managed-github-session";
|
||||
let open!: () => void, started!: () => void;
|
||||
const opening = new Promise<void>(resolve => { open = resolve; });
|
||||
const admitted = new Promise<void>(resolve => { started = resolve; });
|
||||
const stop = vi.fn(async (_options?: { preserveLaunchers?: boolean }) => undefined);
|
||||
githubAccess.create.mockImplementationOnce(async () => ({ env: {}, ready: true, duplexRequested: false,
|
||||
activate: vi.fn(() => vi.fn()), stop }));
|
||||
state.execute.mockReset().mockImplementationOnce(async (options) => {
|
||||
started();
|
||||
await opening;
|
||||
await options.onSession({ detachControllerForRestart: async () => undefined });
|
||||
await options.onSession(null);
|
||||
throw new Error("detachment closed the old event stream");
|
||||
});
|
||||
const outcome = executePaperclipNativeSession({ db: leaseDb(restarting), execution: restarting,
|
||||
runnerInstanceId: "runner", useRunnerd: true, managedGitHub: true }).catch(error => error);
|
||||
let detaching: Promise<unknown> | undefined;
|
||||
try {
|
||||
await admitted;
|
||||
detaching = detachNativeSessionsForRestart([restarting.binding.runId]);
|
||||
open();
|
||||
expect(await detaching).toMatchObject({ detachedRunIds: [restarting.binding.runId] });
|
||||
expect(await outcome).toBeInstanceOf(NativeControllerDetachedForRestartError);
|
||||
expect(githubAccess.create).toHaveBeenLastCalledWith(expect.objectContaining({
|
||||
launcherKey: expect.stringContaining(restarting.binding.issueId),
|
||||
}));
|
||||
expect(stop).toHaveBeenCalledExactlyOnceWith({ preserveLaunchers: true });
|
||||
} finally {
|
||||
open();
|
||||
await outcome;
|
||||
await detaching;
|
||||
if (previous === undefined) delete process.env.PAPERCLIP_RUNNER_STATE_DIR;
|
||||
else process.env.PAPERCLIP_RUNNER_STATE_DIR = previous;
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("uses one admission deadline for the whole restart sweep", async () => {
|
||||
const inputs = ["first", "second"].map(label => {
|
||||
const input = structuredClone(execution);
|
||||
@@ -18483,7 +18526,7 @@ describe("runnerd provider runtime wiring", () => {
|
||||
if (failure === "assigned") {
|
||||
expect(activate).toHaveBeenCalledWith(current.binding);
|
||||
expect(deactivate).toHaveBeenCalledOnce();
|
||||
expect(stop).toHaveBeenCalledOnce();
|
||||
expect(stop).toHaveBeenCalledExactlyOnceWith({ preserveLaunchers: false });
|
||||
}
|
||||
} finally {
|
||||
releaseGitHub();
|
||||
|
||||
@@ -922,7 +922,7 @@ async function detachReleasedWarmNativeSession(entry: WarmNativeSession) {
|
||||
entry.controllerDetachedForRestart = true;
|
||||
if (!entry.session.detachControllerForRestart) throw new Error("native_session_released_detach_unavailable");
|
||||
await entry.session.detachControllerForRestart();
|
||||
await entry.githubAccess?.stop();
|
||||
await entry.githubAccess?.stop({ preserveLaunchers: true });
|
||||
}
|
||||
|
||||
const warmNativeSessions = new Map<string, WarmNativeSession>();
|
||||
@@ -2559,6 +2559,14 @@ function nativeSessionScopeKey(execution: NativeExecutionInput): string {
|
||||
});
|
||||
}
|
||||
|
||||
function nativeGitHubLauncherKey(execution: NativeExecutionInput): string {
|
||||
return canonicalJson({
|
||||
schema: "paperclip.native-github-launcher.v1",
|
||||
session: nativeSessionScopeKey(execution),
|
||||
issueId: execution.binding.issueId,
|
||||
});
|
||||
}
|
||||
|
||||
// A plan acceptance or explicit reset can rotate the provider/session identity
|
||||
// while retaining the same task sandbox and its fixed ingress port.
|
||||
function nativeSessionOwnerScope(
|
||||
@@ -12456,6 +12464,7 @@ async function executePaperclipNativeSessionAttempt(
|
||||
env: resolveNativeProviderEnvironment(input.execution.provider, input.runnerEnvironment),
|
||||
resolveCredentials: (binding) => resolveGitHubOperationCredentials(input.db, binding),
|
||||
onLog: input.onLog,
|
||||
launcherKey: nativeGitHubLauncherKey(input.execution),
|
||||
}));
|
||||
executionContext?.assertOpen();
|
||||
releaseGitHubRun = githubAccess.activate(input.execution.binding);
|
||||
@@ -13878,7 +13887,8 @@ async function executePaperclipNativeSessionAttempt(
|
||||
settlingNativeSessionScopes.set(nativeSessionScopeKey(input.execution), input.execution.binding.runId);
|
||||
releaseGitHubRun?.();
|
||||
if (githubAccess && (!warmSessionId || warmNativeSessions.get(warmSessionId)?.githubAccess !== githubAccess)) {
|
||||
await timeController("github_stop", () => githubAccess!.stop());
|
||||
const preserveLaunchers = nativeRunsDetachingForRestart.has(input.execution.binding.runId);
|
||||
await timeController("github_stop", () => githubAccess!.stop({ preserveLaunchers }));
|
||||
}
|
||||
}
|
||||
let postExecutionFailed = false;
|
||||
|
||||
Reference in new issue
Block a user