Files
PaperClipAI/packages/adapter-utils/src/github-launcher.ts
T
DottaandClaude Opus 5.5 57e85d0d10 fix(adapter-utils): wait for a replacement GitHub broker after a controller restart (#22209)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for work
> - Agents in sandbox tasks use managed `git` and `gh` launchers. Each launcher gets a GitHub credential from a broker for each command.
> - A controller restart stops the broker of a task. The provider in the sandbox continues to run. The restored controller then writes a replacement broker location to `broker.json`.
> - The launcher read `broker.json` one time only. It retried a transport failure for about 1.5 seconds. Then it ran the command without credentials.
> - Thus, commands in the restart window failed with "You are not logged into any GitHub hosts" or "could not read Username". Agents lost pull-request checks and pushes, but the GitHub connection was healthy.
> - This pull request makes broker-file launchers wait for a bounded time and read `broker.json` again before each retry.
> - The benefit is that GitHub commands continue with managed credentials through a controller restart.

## Linked Issues or Issue Description

No public issue exists. This is the bug description.

**What happened?**

A sandbox task ran `gh` or `git` while the controller restarted. The launcher wrote `Paperclip: GitHub broker_transport_unavailable; continuing without managed credentials.` Then `gh` wrote "You are not logged into any GitHub hosts", and `git` wrote "could not read Username". Private reads, pushes and pull-request commands failed. Failures occurred in many tasks at the same time, for some minutes.

**Expected behavior**

The launcher waits for the replacement broker for a bounded time. Then it uses the managed credential. If the broker does not return, the launcher continues without credentials and shows the same diagnostic as before.

**Steps to reproduce**

1. Write a broker-file launcher. Set `broker.json` to a closed local port.
2. Run `gh` through the launcher.
3. After 3 seconds, write a reachable broker location to `broker.json`.
4. The command ran without credentials before the replacement was written.

**Relevant logs or output**

`Paperclip: GitHub broker_transport_unavailable; continuing without managed credentials.` followed by `You are not logged into any GitHub hosts. To log in, run: gh auth login`.

## What Changed

- `packages/adapter-utils/src/github-launcher.ts`: Broker-file launchers retry for up to 60 seconds. Before each retry, they read `broker.json` again. A replacement broker is used when the restored controller publishes it.
- The launcher retries two failure types in this wait: a refused or failed connection, and an HTTP 503 answer. With HTTP/2, the sandbox gateway can continue to listen after its controller connection closes. It then answers `503 {"error":"bridge_unavailable"}`.
- HTTP 401 and 403 answers are real denials. The launcher does not wait for them.
- If `broker.json` cannot be read or parsed (for example during a rewrite), the launcher keeps the last broker location and continues the wait. If `broker.json` is missing, has no broker, or has an empty URL or token, the launcher stops the wait at once, because the access of the task ended. (When a replacement bridge cannot start, the controller writes empty values.)
- After the third failure, the launcher writes one line to stderr: it waits for the credential service to return.
- Launchers with an inherited broker URL keep the old short retry budget and the old 503 behavior. Their URL cannot change, so a longer wait cannot help.
- `packages/adapter-utils/src/github-launcher.test.ts`: Regression tests for three outage types (a closed port, a 503 `bridge_unavailable` gateway, and a partly written `broker.json`). Other tests remove `broker.json` or write empty values to it during the wait, and check that 401 and 403 answers do not wait.
- `doc/execution-github-identity.md`: One sentence describes the wait.

## Verification

- Failing before the first fix: `waits for a replacement task broker instead of running without credentials (closed port)` failed with `expected 'anonymous' to be 'reattached-run-credential'`.
- Failing before the second fix: the `(bridge unavailable)` and `(partly written broker file)` cases failed with `expected 'anonymous' to be 'reattached-run-credential'`. The `(closed port)` case passed.
- Failing before the third fix: `stops waiting for an unreachable task broker once its broker file is emptied` timed out after 20 seconds, because the launcher continued to wait. The `removed` case passed.
- Passing after the fixes: `npx vitest run packages/adapter-utils/src/github-launcher.test.ts packages/adapter-utils/src/execution-target.test.ts packages/adapter-utils/src/github-launcher-environment.test.ts server/src/services/heartbeat-github-launchers.test.ts server/src/services/native-runtime/native-github-access.test.ts` passed 138 of 138 tests in 5 files. `native-github-access.test.ts` includes the controller restart and launcher re-arm tests.
- `npx tsc --noEmit -p packages/adapter-utils` passed.
- I did not run the full `pnpm verify`. The local disk was too small for a full install.
- Note: Run these tests outside a managed sandbox, or remove the sandbox GitHub launcher from `PATH` and unset the sandbox broker variables. The sandbox values otherwise leak into the tests on `master` too.

## Risks

- If the broker does not return, a GitHub command that needs credentials now waits up to about 60 seconds (plus one 10-second request timeout) before it continues without credentials. Before, it waited about 1.5 seconds. Local Git commands do not ask the broker, so they do not wait.
- A broker that answers 503 for a long time (for example, when its credential lookup fails) also causes this 60-second wait. Then the launcher continues without credentials, as before.
- The launcher sends no credential in the new stderr line. The tests check that broker tokens do not appear in stderr.
- Low risk otherwise. The change is in the generated launcher source only.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context to this change
- [x] I have specified the model used (with version and capability details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)





Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 15:24:11 -06:00

226 lines
13 KiB
TypeScript

import { isAbsolute } from "node:path";
/** Standalone source for local, SSH, and sandbox runtimes. No secrets in files. */
export function githubLauncherSource(options: { brokerFile?: boolean; interpreterPath?: string } = {}): string {
if (options.interpreterPath !== undefined && (typeof options.interpreterPath !== "string"
|| !isAbsolute(options.interpreterPath) || /[\s\0]/u.test(options.interpreterPath))) {
throw new Error("GitHub launcher interpreter must be an absolute path without whitespace");
}
// A controller restart stops a task's broker while its provider keeps running,
// then publishes a replacement to broker.json. Only that file can change, so
// only broker-file launchers wait for it; an inherited URL never recovers.
const brokerRecoveryMs = options.brokerFile ? 60_000 : 0;
return String.raw`#!${options.interpreterPath ?? "/usr/bin/env node"}
const fs = require('node:fs');
const path = require('node:path');
const os = require('node:os');
const { spawn } = require('node:child_process');
// A PATH shim can capture this launcher's absolute path and call it again.
// Reject that cycle before resolving another executable or capturing authority.
if (process.env.PAPERCLIP_GITHUB_OPERATION_ACTIVE) {
process.stderr.write('Paperclip: GitHub launcher_recursion_rejected.\n');
process.exit(126);
}
const directory = path.dirname(fs.realpathSync(process.argv[1]));
const program = path.basename(process.argv[1]);
const originalPath = (process.env.PATH || '').split(path.delimiter).filter(p => {
try { return fs.realpathSync(p) !== directory; } catch { return true; }
});
const executable = originalPath.map(p => path.join(p, program)).find(p => {
try { fs.accessSync(p, fs.constants.X_OK); return fs.statSync(p).isFile(); } catch { return false; }
});
if (!['git', 'gh'].includes(program) || !executable) {
process.stderr.write('Paperclip: requested GitHub command is not installed.\n');
process.exit(127);
}
const localGitCommands = new Set(['init', 'status', 'add', 'config', 'rev-parse', 'log', 'diff', 'diff-files', 'write-tree', 'show', 'branch', 'checkout', 'switch', 'restore', 'reset', 'rm', 'mv', 'worktree', 'grep', 'blame', 'describe', 'ls-files', 'ls-tree', 'cat-file', 'rev-list', 'merge-base', 'show-ref', 'for-each-ref', 'symbolic-ref', 'check-ignore', 'version', 'help']);
const gitValueOptions = ['-C', '-c', '--git-dir', '--work-tree', '--namespace', '--config-env', '--super-prefix', '--attr-source'];
const gitFlagOptions = ['-p', '-P', '--paginate', '--no-pager', '--bare', '--no-replace-objects', '--no-lazy-fetch', '--no-optional-locks', '--no-advice', '--literal-pathspecs', '--glob-pathspecs', '--noglob-pathspecs', '--icase-pathspecs'];
function isLocalGit(args) {
if (program !== 'git') return false;
for (let index = 0; index < args.length; index++) {
const arg = args[index];
if (gitValueOptions.includes(arg)) { index += 1; continue; }
if (gitValueOptions.some(option => arg.startsWith(option + '='))) continue;
if (gitFlagOptions.includes(arg)) continue;
return arg === '--version' || arg === '--help' || localGitCommands.has(arg);
}
return false;
}
// With a Paperclip GitHub connection, repositories are created and forked
// through its tools so access rules, approvals, owner defaults and audit apply.
const connectorRepositoryTools = new Map([['create', 'create_repository'], ['new', 'create_repository'], ['fork', 'fork_repository']]);
function connectorRepositoryTool(args) {
if (program !== 'gh' || args[0] !== 'repo' || args.includes('--help') || args.includes('-h')) return null;
return connectorRepositoryTools.get(args[1]) || null;
}
async function main() {
let env = { ...process.env };
${options.brokerFile ? String.raw`
delete env.PAPERCLIP_API_URL;
delete env.PAPERCLIP_API_KEY;
delete env.PAPERCLIP_GITHUB_BRIDGE_TOKEN;
env.PAPERCLIP_GITHUB_BROKER_URL = '';
env.PAPERCLIP_GITHUB_BROKER_TOKEN = '';
// An unreadable file (for example during a rewrite) keeps the last location.
// Only a missing file, or one without a usable broker, means the task's access ended.
const readBroker = () => {
let broker;
try { broker = JSON.parse(fs.readFileSync(path.join(directory, 'broker.json'), 'utf8')); }
catch (error) { return error && error.code === 'ENOENT' ? 'absent' : 'unreadable'; }
// A failed replacement bridge publishes empty values, which also means no broker.
if (!broker || typeof broker.url !== 'string' || typeof broker.token !== 'string' || !broker.url || !broker.token) return 'absent';
env.PAPERCLIP_GITHUB_BROKER_URL = broker.url;
env.PAPERCLIP_GITHUB_BROKER_TOKEN = broker.token;
return 'present';
};
readBroker();
` : String.raw`
const readBroker = () => 'present';
`}
const diagnostic = (code) => process.stderr.write('Paperclip: GitHub ' + code + '; continuing without managed credentials.\n');
let connected = false;
const configRoot = env.GH_CONFIG_DIR || os.tmpdir();
// A missing/unwritable scratch directory must not break local Git. The
// fallback deliberately cannot load the host's gh authentication files.
let configDirectory = path.join(directory, 'unavailable-gh-config');
let configReady = false;
let authorReady = false;
try {
fs.mkdirSync(configRoot, { recursive: true, mode: 0o700 });
configDirectory = fs.mkdtempSync(path.join(configRoot, 'paperclip-github-operation-'));
fs.chmodSync(configDirectory, 0o700);
configReady = true;
process.once('exit', () => { try { fs.rmSync(configDirectory, { recursive: true, force: true }); } catch {} });
fs.writeFileSync(path.join(configDirectory, 'gitconfig'), '[user]\n\tname = Paperclip\n\temail = noreply@paperclip.ing\n', { mode: 0o600 });
authorReady = true;
} catch { diagnostic('configuration_directory_unavailable'); }
{
for (const key of Object.keys(env)) {
if (/^(GH_TOKEN|GITHUB_TOKEN|GH_ENTERPRISE_TOKEN|GITHUB_ENTERPRISE_TOKEN|PAPERCLIP_GIT_TOKEN|GIT_AUTHOR_.*|GIT_COMMITTER_.*|GIT_CONFIG_.*|GIT_ASKPASS|SSH_ASKPASS|SSH_AUTH_SOCK|GIT_SSH.*)$/.test(key)) delete env[key];
}
Object.assign(env, {
GH_CONFIG_DIR: configDirectory, SSH_AUTH_SOCK: '',
GIT_CONFIG_GLOBAL: authorReady ? path.join(configDirectory, 'gitconfig') : '/dev/null', GIT_CONFIG_SYSTEM: '/dev/null',
GIT_TERMINAL_PROMPT: '0',
GIT_CONFIG_COUNT: '5', GIT_CONFIG_KEY_0: 'credential.helper', GIT_CONFIG_VALUE_0: '',
GIT_CONFIG_KEY_1: 'url.https://github.com/.insteadOf', GIT_CONFIG_VALUE_1: 'git@github.com:',
GIT_CONFIG_KEY_2: 'url.https://github.com/.insteadOf', GIT_CONFIG_VALUE_2: 'ssh://git@github.com/',
GIT_CONFIG_KEY_3: 'core.askPass', GIT_CONFIG_VALUE_3: '',
GIT_CONFIG_KEY_4: 'user.useConfigOnly', GIT_CONFIG_VALUE_4: 'true',
});
const base = env.PAPERCLIP_GITHUB_BROKER_URL || env.PAPERCLIP_API_URL;
const askBroker = !isLocalGit(process.argv.slice(2));
try {
let response;
if (askBroker && base && env.PAPERCLIP_GITHUB_BROKER_TOKEN) {
// A slow or restarting control plane must not cost the operation its
// managed identity, so a failed request is retried before giving up.
// Busy (409) responses and transport failures keep separate budgets, and
// the body is read inside the retry so a failed read is retried too.
// Transport failures and 503 responses retry until the recovery deadline,
// reading the broker location again so a replacement is used once published.
// Authentication denials (401/403) still end the request at once.
const recoveryDeadline = Date.now() + ${brokerRecoveryMs};
let transportFailures = 0, conflicts = 0, result;
for (;;) {
try {
const url = (env.PAPERCLIP_GITHUB_BROKER_URL || env.PAPERCLIP_API_URL).replace(/\/+$/, '').replace(/\/api$/, '') + '/runtime-tools/github/credentials';
response = await fetch(url, {
method: 'POST', redirect: 'error', signal: AbortSignal.timeout(10000),
headers: { authorization: 'Bearer ' + (env.PAPERCLIP_GITHUB_BRIDGE_TOKEN || env.PAPERCLIP_API_KEY || env.PAPERCLIP_GITHUB_BROKER_TOKEN),
'x-paperclip-github-capability': env.PAPERCLIP_GITHUB_BROKER_TOKEN, 'content-type': 'application/json' },
body: '{}',
});
// A sandbox gateway that outlived its controller answers 503 until
// the replacement broker is published, so it waits like a refusal.
if (response.status === 503 && Date.now() < recoveryDeadline) {
await response.arrayBuffer().catch(() => {});
throw new Error('broker unavailable');
}
if (response.status === 409 && conflicts < 29) {
conflicts += 1;
await response.arrayBuffer();
await new Promise(resolve => setTimeout(resolve, 1000));
continue;
}
result = response.ok ? await response.json() : null;
break;
} catch (error) {
transportFailures += 1;
if (transportFailures >= 3 && Date.now() >= recoveryDeadline) throw error;
if (transportFailures === 3) {
process.stderr.write('Paperclip: GitHub credential service is not reachable; waiting up to ' +
Math.ceil((recoveryDeadline - Date.now()) / 1000) + ' seconds for it to return.\n');
}
await new Promise(resolve => setTimeout(resolve, Math.min(500 * transportFailures, 2000)));
// A removed broker file means the task's access ended; nothing will return.
if (readBroker() === 'absent') throw error;
}
}
if (!response.ok) {
diagnostic(response.status === 401 || response.status === 403 ? 'capability_rejected' : 'broker_response_unavailable');
} else {
if (result.status === 'unavailable') {
const reason = typeof result.reason === 'string'
? result.reason.replace(/[\x00-\x1f\x7f]/g, ' ').slice(0, 500)
: 'Check the GitHub connection in Paperclip';
process.stderr.write('Paperclip: GitHub access unavailable: ' + reason + '. Continuing without GitHub credentials.\n');
}
connected = result.status === 'available';
if (result.status === 'available' && configReady) {
for (const [key, value] of Object.entries(result.env || {})) {
if (/^(GH_TOKEN|GITHUB_TOKEN|PAPERCLIP_GIT_TOKEN|GIT_TERMINAL_PROMPT|GIT_AUTHOR_(NAME|EMAIL)|GIT_COMMITTER_(NAME|EMAIL)|GIT_CONFIG_COUNT|GIT_CONFIG_(KEY|VALUE)_\d+)$/.test(key) && typeof value === 'string') env[key] = value;
}
}
}
} else if (askBroker) { diagnostic('capability_missing'); }
} catch { diagnostic('broker_transport_unavailable'); }
}
const connectorTool = connected ? connectorRepositoryTool(process.argv.slice(2)) : null;
if (connectorTool) {
process.stderr.write("Paperclip: use the GitHub connection's " + connectorTool + " tool instead of gh repo " + process.argv[3] +
", so the connection's access rules apply. " + (connectorTool === 'fork_repository'
? 'Then clone the fork or add it as a git remote.' : 'Then add the repository as a git remote and push.') + '\n');
process.exitCode = 1;
return;
}
// Only this invocation and its children inherit the captured credential.
env.PAPERCLIP_GITHUB_OPERATION_ACTIVE = '1';
// Its Git children use the real binary, so steering cannot split a gh operation.
env.PATH = originalPath.join(path.delimiter);
// Nested shell aliases must not reload the parent launcher profile and
// recapture a newer identity. All ordinary descendants stay in this operation.
env.ZDOTDIR = configDirectory;
env.BASH_ENV = '/dev/null';
env.GIT_SSH_COMMAND = 'ssh -F /dev/null -o IdentityAgent=none -o IdentitiesOnly=yes -o IdentityFile=none -o BatchMode=yes';
const child = spawn(executable, process.argv.slice(2), { env, stdio: 'inherit' });
for (const signal of ['SIGINT', 'SIGTERM', 'SIGHUP']) process.on(signal, () => child.kill(signal));
child.once('error', () => { process.stderr.write('Paperclip: GitHub command could not start.\n'); process.exitCode = 1; });
child.once('exit', (code, signal) => { process.exitCode = code === null ? 128 : code; });
}
main().catch(() => { process.stderr.write('Paperclip: GitHub launcher_setup_failed.\n'); process.exitCode = 1; });
`;
}
/** Override inherited credentials even when adapters merge the host environment later. */
export function githubBrokerEnvironment(input: Record<string, unknown>, broker: { url: string; token: string }): Record<string, string> {
const env: Record<string, string> = {};
for (const [key, value] of Object.entries(input)) if (typeof value === "string") env[key] = value;
for (const key of ["GH_TOKEN", "GITHUB_TOKEN", "GH_ENTERPRISE_TOKEN", "GITHUB_ENTERPRISE_TOKEN", "PAPERCLIP_GIT_TOKEN", "GIT_AUTHOR_NAME", "GIT_AUTHOR_EMAIL", "GIT_COMMITTER_NAME", "GIT_COMMITTER_EMAIL", "GIT_CONFIG_COUNT", "PAPERCLIP_GITHUB_OPERATION_ACTIVE"]) env[key] = "";
for (const key of Object.keys(env)) {
if (/^GIT_CONFIG_(KEY|VALUE)_\d+$/.test(key)) env[key] = "";
}
env.GIT_CONFIG_GLOBAL = "/dev/null";
env.GIT_CONFIG_SYSTEM = "/dev/null";
env.GIT_CONFIG_NOSYSTEM = "1";
env.GIT_TERMINAL_PROMPT = "0";
env.GIT_ASKPASS = "";
env.SSH_ASKPASS = "";
env.GIT_SSH_COMMAND = "ssh -F /dev/null -o IdentityAgent=none -o IdentitiesOnly=yes -o IdentityFile=none -o BatchMode=yes";
env.SSH_AUTH_SOCK = "";
env.PAPERCLIP_GITHUB_BROKER_URL = broker.url;
env.PAPERCLIP_GITHUB_BROKER_TOKEN = broker.token;
return env;
}