Files
PaperClipAI/doc/execution-github-identity.md
T
Nicky LeachandPaperclip aab87ce707 Prove workspace delivery before inspecting cleanup candidates (#18493)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Completed tasks leave workspaces that need safe cleanup.
> - Providers without ancestry inspection currently need a recorded
merged PR before cleanup can start.
> - A squash merge changes commit identity, so direct ancestry cannot
prove that delivery.
> - This change proves delivery from the finalized HEAD and captured
destination before storage inspection.
> - Fresh storage checks and a second delivery check still protect
removal.
> - This reuses the existing worker and receipts. It adds no table or
timer.

## Linked Issues or Issue Description

Refs #18410. Related: #13400 addresses workspace identity during
inspection; this change adds destination-aware delivery discovery.

## What Changed

- Resolve direct ancestry or discover an associated merged GitHub PR
without a recorded work product. Require exact HEAD, destination
repository and branch, and a reachable integration commit.
- Use the latest settled finalization receipt as a candidate. Reject
missing, failed, pending, conflicting, or changed evidence.
- Inspect storage only after positive remote proof for providers that
disable ancestry inspection. Recheck HEAD, branch, source identity and
delivery before removal intent.
- Add an optional provider capability for a specific PR recovery hint
and an optional prepared destination receipt. Older providers keep their
existing protocol.
- Wake cleanup when finalization writes settle or gain a workspace
binding. Keep remote-only retries for concrete unresolved delivery. Park
missing or unsupported candidate identity.
- Reuse the source project's explicit GitHub consent for delivery checks
through the existing authorization service. Revalidate the bound
repository and consent before removal; missing or revoked consent
retains storage without switching identities. Public sources remain
anonymous.
- Update the contracts, documentation, access map, and targeted
regressions.

## Verification

- Initial-head GitHub CI passed at
`5acd4c818be6f1764f8319f42891f9956c357c26`, including full build and
typecheck. Standard Greptile is 5/5 with no open threads. The first Apex
review scored 3/5 and found startup latency from remote proof lookups
and removal retry identity changes across provider capability changes.
Both corrections are pushed in `9d1a38df3d`, with the qualified
destination-ref length fix. Red regressions reproduced each issue.
Latest targeted checks passed: cleanup/worker 26, provider integration
41, SDK contract 8. Head `9d1a38df3d` passed standard and Apex reviews
at 5/5 with no open threads, plus all CI including full build/typecheck.
Canary Dry Run initially failed on an npm 404 fetching
`rollup-4.67.3.tgz`; the exact tarball subsequently returned HTTP 200,
and one targeted retry passed without source changes. Aggregate
verification also passed. After #18403 merged, master introduced
conflicts in shared workspace cleanup and metadata handling. Resolved
them in `7700832536`, preserving remote snapshot cleanup and Git
delivery-proof cleanup. Targeted merge verification: 93 tests passed in
the first run (59 opt-in skips), then all 16 remote lifecycle tests
passed with explicit database opt-in; map checks passed. The isolated
database is stopped. The merge head received standard review 5/5. Its CI
found two failures in unchanged test cleanup paths: native runtime
temporary-directory ENOTEMPTY and agent-chat cancellation lock timeout.
Those failures remain recorded and require successful final-head
verification. The subsequent project-consent change has targeted
coverage: authorization36, resolver33 and cleanup19 tests passed,
including revoked consent, anonymous public access and source-binding
races. The positive consent test failed before the fix. Current head
`c772679e05` has green GitHub CI, including full build/typecheck and
both previously failing test jobs, plus standard Greptile 5/5 with no
open threads. Neither cleanup failure required changes to its test path.
Apex also completed at 5/5 on this exact head (check `124651548532`,
summary updated 2026-10-31 20:45 UTC), with no open review threads. The
PR remains draft. Earlier failures remain recorded above.

- Final focused run on the rebased branch: **182 tests passed across six
files**, covering the resolver, delivery cleanup, provider bridge,
finalization notifications, existing workspace service and SDK
contracts. The disposable test database was stopped afterward.
- Targeted resolver tests: 33 passed. These cover ancestry, discovered
squash PRs, wrong HEAD/base/repository, removed integration commits,
bounded responses, cancellation and credential isolation.
- Targeted cleanup integration tests: 17 passed. Provider bridge
integration: 36 passed. SDK contract tests: 7 passed. Finalization
notification tests: 9 passed.
- A negative control that removed the final delivery check failed the
race regression as expected. The check was restored.
- Read-only live GitHub probe: the new resolver discovered PR #18410
from HEAD `3f5ade4cd42b707040e9a82fdaed343e3d736a09`, without PR hints.
It proved integration commit `6484f96c28bfb11a793ce4be6cc5ac15643a5a02`
reachable from observed master
`5bc029bf6c1db6647e96acea323f3723fd1d3eba`. GitHub also advertised the
exact original HEAD at `refs/pull/18410/head`.
- `node scripts/check-maps.mjs` passed. Structural checks do not prove
deployment behavior.
- Local tests are targeted by user request. Full build, typecheck and
test suite run in GitHub CI. A standalone server typecheck could not
complete because local Runner build declarations were absent; no
changed-file diagnostics were reported.
- Initial local execution hit a stalled pnpm wrapper, sandbox socket
denial and stale generated SDK declarations. Direct installed binaries,
the authorized isolated test database and rebuilt SDK resolved those
setup failures. No deployment was performed.

## Risks

- Destructive cleanup still requires fresh storage safety and lifecycle
checks. Remote proof alone never authorizes removal. A persisted removal
intent preserves existing recovery semantics.
- Legacy workspaces without an unambiguous destination or completion
HEAD remain retained. This does not guess the original default branch.
- Private repositories use the explicit standing GitHub consent on the
bound source project, with current membership, grant, repository access
and trust checks. No separate company token is required when that
consent is valid. Missing or revoked consent and unavailable or
rate-limited access retain storage; the worker does not borrow a task or
agent identity.
- A known undelivered commit keeps the existing five-minute retry
because external GitHub changes have no guaranteed ingress to this lane.
This can keep cleanup scheduled. Missing evidence parks instead.
- Providers need the new recovery capability to clean a squash merge
whose original HEAD is reachable only through its PR ref. Providers
without it remain conservative.
- The resolver currently supports GitHub.com. Existing local and
other-provider ancestry paths stay in place.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, repository tools and
parallel code agents. The exact runtime model ID and context-window size
were not exposed to this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-02 07:59:17 -08:00

239 lines
21 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# GitHub identity during agent execution
Shared agents use the GitHub connection of the person whose accepted instructions they are executing. Task ownership remains unchanged. GitHub is optional: ordinary work can start without a connection. Managed launchers provide a default author for local commits; private checkouts and authenticated API calls still require eligible credentials.
## Accepted instructions and continuations
`run_identity_contexts` records ordered revisions, stored message authors, originating causes, parent contexts, acceptance state, and redacted GitHub outcomes. `heartbeat_runs.active_identity_context_id` selects the current revision. Existing historical runs are not backfilled with inferred authorship.
Human messages use their stored authenticated author. Queued messages retain their delivery order. Accepting steering reserves a pending revision before delivery, then activates it after the provider acknowledgement. Rejected delivery leaves the prior revision active. An uncertain acknowledgement holds new credential acquisition; a later acknowledgement or its authenticated native event receipt reconciles the reservation. Replays cannot reactivate an older revision. Activation locks the task before the run, matching task and queue mutations so concurrent status changes cannot deadlock identity initialization.
Delegated work and interactions persist their originating context. Retries retain the originating run's active context. Background continuations carry their source run; dependency wakes use the task's continuation context, independently of its owner. Scheduled and webhook routines use the routine's responsible person; manual invocations use the caller, and edits preserve the routine's responsible person.
## Managed GitHub operations
Executions with managed GitHub configured receive token-free `git` and `gh` launchers. Each launcher invocation resolves one eligible credential from the current run's accepted identity at operation start. A `gh` command's child Git processes inherit that command's captured identity; later steering does not change already-started operations.
A `git` command that never uses the network does not ask the broker. The launcher reads past the global options, such as `-C` and `-c`, and checks the subcommand against a short list (for example `status`, `diff`, `diff-files`, `write-tree`, `log`, `rev-parse`, `add`, `config`, `branch`, `checkout`). This includes the index comparison and tree writing used by workspace snapshots. An unknown subcommand or option still asks. `commit` and `var` still ask, because the managed author comes from the broker answer. The launcher keeps no broker answer between invocations and never writes a credential to a file.
GitHub credential capture reads the current task, project, and retained trust
policy bindings without transferring saved prompts, runner profiles, results,
or log excerpts. It keeps the existing task/run lock order and accepted-identity
reconciliation. Other identity callers retain their complete execution input,
including the native input required by project creation authorization.
Native runners retain a session-owned broker and launcher path across warm turns. The provider keeps an opaque transport token, not a GitHub credential or the previous run's signed capability. After acquiring exclusive session ownership, the controller binds the broker to the current company, agent, task, and run. Requests cannot choose another run or responsible person. The broker rejects requests while idle and discards credential responses if their run binding changed during acquisition. Each operation still rechecks the live run, accepted identity, grants, and trust policy through the shared credential resolver. Changing run IDs alone no longer replaces the provider process; changes to authentication mode, provider credentials, permissions, or other session configuration retain their existing retirement rules.
The session broker listens only on controller loopback and accepts only its authenticated GitHub credential operation. Remote executions reach it through the existing authenticated callback bridge. Its transport and launchers are retired with the provider session. If remote bridge startup fails, anonymous launchers keep ordinary work available; the next run retries setup with a fresh session. Controller restart/cold recovery still uses the existing checkpoint and process-recovery rules; the broker's in-memory authority is not persisted for adoption.
The native GitHub callback bridge uses HTTP/2 only when the server-selected
`enableSandboxDuplexBridge` flag and the sandbox's verified duplex capability
permit it. Missing settings, unsupported providers and failed channel startup
keep the authenticated file bridge. Changing the server flag retires a retained
warm bridge at the next normal run boundary, before a retained instruction copy
is claimed. A lost HTTP/2 channel also makes the broker unavailable and retires
that warm session at its next admission. An unchanged flag and a healthy broker
permit reuse. Normal session cleanup closes the channel as an orderly completion.
This selection changes transport only; operation-time credential checks remain
the same. Local and SSH execution retain their existing transport.
Other adapters continue using the run-scoped signed capability and public broker endpoint. That endpoint rejects browser origins and session cookies, validates a distinct signed runtime scope, and rechecks the company, agent, and live run. GitHub credentials are returned only to the managed command process, never persisted in identity history or injected into the long-lived provider process.
Low-trust executions cannot receive raw GitHub credentials, including dedicated
agent tokens. The broker rechecks current agent, project, task, and retained run
policies before credential resolution. An external guest's internal sponsor is
accountable for the task, but does not authorize using the sponsor's account.
Read-only access must use separately authorized tools that enforce that boundary.
Server-side Git operations and GitHub gateway calls follow the same selection rules. Approved gateway operations retain their signed originating identity. Connection audience and tool policies continue to apply to the selected person's connection. Native catalogs remain stable across identity changes, but each invocation resolves the selected grant again. Personal OAuth secret declarations survive connection pauses and metadata edits.
Managed commands disable ambient Git credential helpers, Git global/system configuration, host GitHub CLI configuration, and host SSH identity access. Per-operation GitHub CLI configuration is isolated in a writable configuration directory beneath the managed launcher directory. Missing credentials clear previous author and token values; no teammate, standing delegation, host token, or company-default user's account is substituted. Anonymous/local operations remain available where supported.
OpenCode retains the controller-projected GitHub broker environment through
runner startup and provider startup. Shell operations acquire credentials for
the active run through that broker. They do not require a persistent GitHub CLI
login or inherit an ambient host broker capability.
Without a managed identity, local commits default to
`Paperclip <noreply@paperclip.ing>`. The launcher writes this token-free default
to a private per-operation global Git configuration file, removed on exit.
Repository identity and `git -c user.name=... -c user.email=...` override it.
The launcher leaves author/committer environment variables unset instead of
guessing the host user's details. `git config --global` changes only this
temporary file; use repository configuration for persistent overrides.
If the private configuration directory or default author file cannot be prepared,
Git ignores the incomplete file. Local commits still require an explicit
repository or command identity when no managed identity is available. A failed
default author write does not remove valid managed credentials. Managed shell profiles
remove empty identity overrides after environment merging, so agents do not
need to unset them per command. A captured managed identity still takes
precedence over repository configuration.
Remote launchers prepend their directory to the execution target's effective
`PATH`. An explicit remote `PATH` override is preserved; otherwise Paperclip
reads the provider's environment before staging the launcher shell files.
This keeps legacy NVM and user-local agent installations available alongside
newer images with system-wide CLIs. The generated shell files retain that
combined path with managed `git` and `gh` first. The launcher directory has
its own CommonJS package scope, so the extensionless Node launchers work
inside repositories that declare `"type": "module"` without changing the
project's package configuration. Sandbox command checks use
the same sanitized environment as execution, so a CLI visible only in the
provider's default environment cannot pass the launch check. Failed path
discovery stops startup instead of silently falling back to a minimal path.
Providers that declare concurrent sync support stage at most three independent
launcher files at once. Other providers keep serial staging. Each file retains
its content hash, upload lock, integrity check and atomic rename. A failed batch
settles all started uploads before startup rejects; later files and the final
permission step do not start. This changes startup scheduling, not credentials
or callback transport selection.
Scripts that previously read a persistent `GH_TOKEN` must use managed `git`, `gh`, or GitHub gateway tools. Managed execution skips legacy GitHub token bindings in agent, environment, project, and routine configuration before secret preflight. Configure personal or dedicated access through the GitHub connection instead. Directly invoking an unmanaged executable or retaining a token obtained during an earlier invocation is outside the managed invocation contract.
## Legacy hosts and networking
When no managed GitHub connection is installed for an agent, standard-trust
local and SSH executions retain that execution host's existing Git and GitHub
CLI credentials, configuration, credential helpers, and SSH agent. Paperclip
does not import controller credentials into an SSH target. Sandbox, plugin,
and low-trust executions do not receive this compatibility fallback. Once a
managed connection is configured, unavailable or revoked access never falls
back to host authentication. Switching modes replaces the provider process
while preserving the settled conversation.
Runner network access is independent of GitHub credentials. The controller
enables networking for standard-trust execution. Low-trust runs and runners
without a controller network decision retain a restricted default. An operator
can set `PAPERCLIP_RUNNER_NETWORK_ACCESS=disabled` to restrict normal execution;
user environment bindings cannot override that decision. Outer execution-
environment network restrictions still apply. The controller projects the assigned worktree's Git metadata paths so
Git can operate without exposing unrelated workspace or provider state. The
sandbox also receives read access to validated provider executable resources
and the target host's DNS and CA files, including resolver symlink targets
outside `/etc`. Provider credential directories remain isolated.
A managed broker outage does not prevent local Git operations. Launchers clear
credentials and run the command without authentication, with a redacted error
category identifying configuration setup, transport, or capability rejection.
They do not retain a previous operation's token or replay a GitHub operation.
Healthy eligible grants for the same stable GitHub account ID take precedence
over duplicates with failed health checks. Credential acquisition can retry
once against another grant for that same principal and account, before any
GitHub operation begins. Run identity diagnostics include the selected
connection and grant IDs, without credential values. Access-refresh conflicts
retry once against current state and never turn a concurrency conflict into
a reconnect requirement.
## Dedicated accounts and diagnostics
An explicit dedicated-agent grant overrides personal selection. Revoked, disabled, unavailable, or ambiguous dedicated grants do not fall back to a person's account. Removing the dedicated configuration restores personal selection.
Connection setup and permissions display: “This agent uses this GitHub account for everyone's work, instead of the person giving instructions.”
The GitHub permissions page shows repositories across all connected accounts in one scrollable list. It has no account filter or repository search. Repository icons, private-repository indicators, refresh, and GitHub configuration links remain available. The “Add More Repos on GitHub” button opens GitHub’s app installation and repository-access setup.
Multiple eligible connections for the same GitHub account are treated as one
identity, using GitHub's stable account ID rather than its login. The resolver
selects an available grant, preferring the newest authorization with a stable
ID tie-breaker. Duplicate eligibility includes an active credential record with
the correct owner, the OAuth access-token reference, and repository access
metadata. It keeps that grant's credential and connection policy together;
it does not combine repository access or bypass connection audiences. Distinct
accounts or unidentifiable duplicate grants remain ambiguous. Managed commands
print the redacted reason when GitHub access is unavailable, while unrelated
local operations can still proceed without credentials.
Run details show identity revisions and redacted GitHub results: responsible person, selected login when available, personal/dedicated source, and an unavailable reason. Tasks do not receive an additional identity indicator or takeover action.
## Deployment and verification
Deploy the schema, server broker, launcher staging, and runtime environment contract together. Already-running processes retain their original environment; only newly dispatched processes receive the broker contract. Run-scoped capabilities remain valid only while their bound run is active.
Focused coverage lives in `run-identity.test.ts`, `github-operation-credentials.test.ts`, and `github-launcher.test.ts`, alongside the native steering, gateway, routine, and callback-bridge suites. Live acceptance additionally requires two authenticated Paperclip users, two authorized GitHub accounts, and a designated disposable repository for push verification. Local commit metadata and mocked API results do not replace that live push test.
### Release procedure
1. Back up the instance database using the normal deployment procedure.
2. Build and deploy one revision containing migrations 0240–0245, the server broker, managed launchers, and the runner artifacts. Run the standard pending-migration check before admitting new runs. These additive migrations are safe to replay and do not infer authorship for historical runs.
3. Let pre-rollout executions finish with their original runtime contract. New executions must have an active identity context and the managed launcher capability before provider startup.
4. Check one ordinary run without a GitHub connection, then an authenticated GitHub operation. Inspect the run details for the responsible person and credential outcome. Verify a queued continuation on the same conversation.
5. If rollback is needed, finish or explicitly stop executions using the new broker before removing its endpoint. Keep the additive schema and identity history. Do not drop identity columns or tables to roll back application code.
Remote acceptance uses the existing paid runner workflow with a narrow selection. Run it against the same immutable revision as the release; a successful local test does not qualify a different remote runner artifact.
Identity history survives deletion of the originating agent or run, so surviving
subtasks and approvals retain their responsible person. The company foreign key and company-deletion service remove
these company-scoped records when their company is deleted. Run-scoped adapters remove their managed launcher files at the terminal boundary.
Native warm sessions retain their token-free launchers and inactive broker until
session retirement; environment deletion and orderly controller shutdown close
idle owners first. Cleanup failures are logged and do not change the run result.
## Project authorization for background repository work
Background provisioning and workspace maintenance have no responsible task user.
They must not retain a task's identity. A signed-in project workspace writer can
select their own active, company-installed GitHub Connector grant in project
settings and explicitly authorize the project. The binding stores the exact grant
ID and consenting user, never a credential. Revocation records a project lifecycle
event so subscribed providers can reconcile the change.
`GET /api/projects/:id/git-authorization` returns the current binding and only the
viewer's eligible personal grants. `PUT` accepts `{ grantId }` or `{ grantId: null }`.
Agents cannot create this standing consent. Project workspace writers can remove it.
Core rechecks membership, Connector installation/enablement, grant status, and
repository metadata on each background use. Public repositories never require
a connection or credential. Private repositories wait for authorization before
provisioning starts, so missing consent cannot leave a failed anonymous clone.
Repository selections retain the same visibility metadata used by the UI lock.
Legacy workspaces use existing Connector catalog metadata, with anonymous
GitHub discovery only when visibility is unknown. Unknown visibility defers
provisioning; it is never assumed public. It never substitutes another grant
when the selected one fails. Replacing a grant requires explicit reauthorization.
Project consent follows the selected connection's current GitHub repository access,
including repositories added to the project later. It does not create a second
repository approval list. Project repository editors may add sources within that
connection's access; manage permitted repositories in the GitHub connection.
Background export also rechecks the consenting user's project read access and the
project's existing trust policy. Low-trust or unverified projects cannot export a
raw credential through standing consent.
Cloud owner access is rechecked through current Cloud authority and the managed
owner-elevation setting on every private background export; saved local admin
roles cannot replace that authority. An unavailable authority check cannot confer
owner access, though explicit project membership still applies.
The trusted plugin capability `projects.git-credentials` enables
`ctx.projects.resolveGitCredential(projectId, workspaceId, companyId)`. The host
binds requests to the active project lifecycle or workspace operation; data, UI,
tool, and proactive calls do not carry this authority. Repository URLs come from
stored project workspaces. Foreground preparation uses the live run's accepted
identity and standard trust checks, even when background project consent exists.
Maintenance ignores originating run/agent context and uses project consent only.
Core delivery-proof cleanup uses the same project consent before contacting GitHub,
bound to the provider's stored source project and repository. It repeats authorization
for the final proof before removal; a changed source or revoked grant retains storage.
After archival, existing consent remains usable only for host-issued workspace
inspection and removal. New provisioning and new consent remain blocked.
The provider receives a transient GitHub token with its existing provider
permissions. This is not a repository-scoped or read-only token minting API.
Only grant-listed repositories can trigger background export. Plugins must never
save tokens in entities, state, configuration, operation receipts, or logs, and
must send them only to trusted execution infrastructure over HTTPS. Revocation
prevents subsequent credential acquisition; it cannot recall a token already
sent to an in-flight operation. Existing tasks and Git credential fallback rules
outside this plugin API are unchanged.
### Plugin-owned task command environments
Task-only environment plugins negotiate `githubCredentialBrokerVersion: 1` and
receive the current run's broker capability through structured preparation.
Core does not prescribe executable names, files, scripts, or a staging layout.
The plugin owns its command implementation and isolates ambient host identity.
See [the plugin broker contract](plugins/PLUGIN_RUNNER_EXECUTION.md#operation-time-github-credentials).
These task providers use the signed, run-scoped public broker endpoint; they do
not require the controller's remote shell or session callback bridge. Warm task
preparation replaces the capability before the next run's commands. Credentials
are still captured separately for each operation through the same live resolver.