Commit Graph
6 Commits
Author SHA1 Message Date
DottaandPaperclip 82f662656a fix(runner): restore legacy Git access and independent networking (#13094)
## Thinking Path

> - Paperclip runs agents for people with different GitHub accounts.
> - Managed operations must use the intended person's eligible
connection.
> - A failed duplicate connection must not hide a healthy grant for the
same account.
> - Legacy hosts also need their existing Git configuration when managed
access is not configured.
> - Runner networking and local Git operations must not depend on GitHub
broker availability.
> - This pull request separates those policies and improves failure
diagnostics.

## Linked Issues or Issue Description

**What happened?** New runs always cleared host Git credentials and
installed managed launchers. Network permission depended on GitHub
environment variables. A launcher failure could stop even local `git
status`. A newer unhealthy duplicate could take precedence over a
healthy connection, and generic health errors were shown as reconnect
requirements.

**Expected behavior:** Use a healthy eligible managed connection for the
intended account. Preserve host authentication only for unconfigured
standard-trust local or SSH execution. Permit local Git during broker
failures and keep network permission independent of GitHub credentials.

**Steps to reproduce:** Configure healthy and unhealthy grants for one
GitHub account, dispatch an agent, and execute Git commands. Separately
run an unconfigured legacy host with existing GitHub CLI authentication.
Stop the broker and run local `git status`.

**Paperclip version or commit:** Master at 668110469. **Deployment
mode:** Self-hosted. **Installation method:** Git checkout. **Agent
adapters involved:** Native Codex runner and shared adapter execution
setup. **Database mode:** Existing instance database; no migration.
**Access context:** Responsible person's managed grant, or explicitly
unconfigured legacy host. **Node.js version:** 26.4.0 locally.
**Operating system:** macOS development and Linux execution hosts.

**Relevant logs or output:** Previously `GitHub credential context
unavailable` hid configuration, transport, and capability errors. New
diagnostics identify these categories without credential values.

**Additional context:** Refs #13005 and #13022. Dependency provisioning
is addressed separately in #13093.

## What Changed

- Prefer healthy eligible grants and retry credential acquisition once
for the same principal and account before starting an operation.
- Preserve host Git configuration only when managed access is
unconfigured on a standard-trust local or SSH target.
- Project authentication mode and validated Git metadata into native
runner boundaries; refresh resumed provider settings when modes change.
- Enable network access through an explicit standard-trust controller
decision, independently of GitHub. Omitted or restricted decisions stay
disabled; replace warm providers when that decision changes.
- Run local Git with cleared credentials when the managed broker fails,
with specific redacted diagnostics.
- Retry access-refresh conflicts once without treating concurrency as
expired authorization.
- Show retry instead of reconnect for transient GitHub health failures.
Add optional authorization and run-diagnostic fields without a database
migration.

## Verification

- All latest-head CI gates are green, including typecheck, general and
serialized suites, browser tests, canary, native runner verification,
and build. Greptile is 5/5 with no remaining findings; the security scan
passed.
- Full recursive typecheck and build passed. UI token gates passed.
- Full general server run: 7,110 passed, one transient socket hangup;
that file passed on retry. All remaining workspace groups passed,
including 5,552 UI and 478 CLI tests. The complete serialized rerun
passed all 144 suites / 2,179 tests after the initial isolated timeout
passed on retry.
- 195 final launcher and native session tests passed, including
host/managed transitions, local/remote warm network-policy changes,
broker rotation, and attempts to override validated controller
filesystem roots.
- GitHub gateway fallback, duplicate connection selection, refresh
conflicts, per-user reauthorization, and native transport/security
suites passed.
- Additional live native fixtures passed SSH public-key authentication
and a Git credential helper in fresh and resumed host-mode sessions. An
unwritable managed configuration directory preserved local Git (exit 0)
while GitHub CLI failed with `configuration_directory_unavailable` (exit
4). The temporary SSH listener and keys were removed.
- Applicable Rust suites passed except two timing failures under load;
each failed case passed in isolation. The final environment contract
test passed.
- Linux native runner acceptance passed in both managed and legacy host
modes: DNS, HTTPS, npm package download, fresh-worktree Git status,
authenticated GitHub user lookup, repository read, and a new run
continuing the same provider conversation. Managed broker outage
preserved local Git and rejected authenticated access without host
fallback.
- Matching Linux server/runner artifacts and the separate provisioning
repair are deployed to the development instance. A fresh UI-dispatched
task and a new run after a server restart both passed all six shell
checks through the live controller and credential broker. Both runs
selected the expected healthy connection/grant and retained the same
provider conversation. Neither connection was repaired or reconnected.

The sandbox roots are assigned from the validated execution-target probe
**after** ordinary bindings are merged. Regression coverage supplies
forged roots and verifies they cannot override the controller paths.
Networking is enabled only for an explicit
`PAPERCLIP_RUNNER_NETWORK_ACCESS=enabled` controller decision; omitted
values remain disabled.

## Risks

Unconfigured standard-trust local and SSH runs regain access to host Git
authentication resources. Managed, sandbox, plugin, and low-trust runs
do not gain this fallback. Revoked managed access never falls back to
another account. Deploy server and runner artifacts together;
already-started operations retain their captured identity. An
unauthenticated command can still fail when it requires GitHub access.

## Model Used

OpenAI GPT-6 through Codex, with code editing, shell execution, tests,
and browser inspection. The exact model variant and context-window size
are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-09 10:15:10 -05:00
DottaandPaperclip 297d8741f5 fix: resolve duplicate connections to the same GitHub account (#13022)
## Thinking Path

> - Paperclip lets people share agents while keeping GitHub access
personal.
> - Each managed Git or GitHub operation selects an eligible connection
grant.
> - Connecting the same GitHub account twice creates two grants.
> - The old resolver counted grants and rejected them as competing
identities.
> - Managed commands then ran anonymously and reported a misleading
login failure.
> - This change compares GitHub account IDs and selects one eligible
grant for the same account.
> - The benefit is reliable access after reconnecting, with clear
diagnostics for real failures.

## Linked Issues or Issue Description

Refs #13005.

**What happened?**

Two active connections owned by one Paperclip user pointed to the same
GitHub account. Managed Git refused both as ambiguous. The agent could
not push, although the account was connected and had repository access.

**Expected behavior**

Multiple grants for the same GitHub account resolve to one eligible
authorization. Different accounts remain ambiguous. Unavailable access
explains its cause without blocking unrelated work.

**Steps to reproduce**

1. Connect the same GitHub account twice for one Paperclip user and
allow the shared agent through both connection audiences.
2. Start an instruction as that user.
3. Run managed gh or git push. Before this fix, no credential is
provided.

## What Changed

- Compare stable GitHub account IDs when more than one eligible grant
exists. Never deduplicate by login alone.
- Prefer an available grant, then the newest authorization with a stable
ID tie-breaker. Refresh and webhook timestamps do not change the
selection.
- Keep the selected credential and connection policy together. Do not
combine permissions or fall back from a dedicated account to a personal
account.
- Print the redacted unavailable reason in managed command output.
Unrelated local operations still work anonymously.
- Add database and executable launcher regressions, and document
selection behavior.

## Verification

- Final `pnpm -r typecheck` and `pnpm build` passed.
- Fourteen operation credential integration tests passed, covering
duplicate personal/dedicated grants, incomplete credentials, distinct
accounts with the same login, missing identity metadata, revocation,
membership, connection audiences, and A → B → A steering. Existing Git
credential and gateway suites and both executable launcher tests also
passed.
- The local broad test run encountered three embedded-Postgres lifecycle
timeouts and stale modules from edits made during that run. A fresh
process rerun of all four affected suites passed all 35 tests. The full
Node 24 CI test matrix passed on the final commit.
- CI passed all 31 checks on `797973b30beb16ba5fa69ed281835e1ab812b449`
(Storybook visual regression was correctly skipped). An unrelated
Company Settings UI test failed once; the focused local reproduction and
rerun of its CI shard both passed without code changes.
- Fresh Greptile review of the final commit: 5/5, with no open findings.
Security checks passed.
- Live acceptance passed with both duplicate connections enabled:
managed `gh api user` returned the expected account, managed `git push`
succeeded, and the agent created #13023 and pushed its review fixes. No
host login or credential changes were used.
- Applied the final source/compiled patch to the affected instance with
backups, after confirming no runs were active. Restarted service health
and the final resolver selection were verified. The patch is an overlay
on the existing deployment; this PR supplies the upstream fix.

## Risks

The resolver selects one authorization for an already permitted GitHub
account. It does not combine repository permissions across connections.
If the selected authorization has narrower access, that operation can
still be denied by GitHub. Different provider account IDs and unknown
duplicate identities continue to fail closed. No schema, host
credential, or connection permission changes are included.

## Model Used

OpenAI GPT-6 through Codex assisted implementation and verification with
shell, database, and browser tools. The exact model variant and
context-window size are not exposed in this session.

## Checklist


- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-07 22:43:37 -05:00
DottaandPaperclip 1cc45086d3 feat: use the responsible person's GitHub for shared agent operations (#13005)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Several people can send instructions to the same agent and task.
> - A fixed GitHub token in the provider process can keep the first
person's access after another person's message is accepted.
> - Task ownership cannot select credentials for each accepted
instruction or preserve the identity of an operation already in
progress.
> - This pull request records ordered execution identity contexts and
resolves credentials when managed Git, gh, or GitHub tools start.
> - The benefit is automatic personal GitHub access for shared agents,
with durable continuation rules and no teammate credential fallback.

## Linked Issues or Issue Description

**Subsystem affected**

Cross-cutting: orchestration, connection grants, database, runtime
adapters, native runners, and run details.

**Problem or motivation**

A shared agent must use the person whose instructions it has accepted. A
queued message must retain its author. A retry or approval without new
instructions must retain the originating identity. GitHub must remain
optional for ordinary work.

**Proposed solution**

Persist execution identity separately from task ownership. Give new
processes a run-scoped broker capability and token-free managed
launchers. Capture identity at operation start. Keep an explicit
dedicated-agent grant as an override. Show redacted diagnostics in run
details.

**Alternatives considered**

Per-task ownership, fixed provider tokens, and mutable repository author
configuration do not handle accepted steering or concurrent operations.
A manual account-selection action would add unnecessary setup to each
turn.

**Roadmap alignment**

This completes the existing Multiple Human Users, MCP Tool Gateway &
Apps, Secrets Manager, and Self-healing Runs capabilities. The
implementation follows the maintainer-approved plan.

Related work: Refs #12843, Refs #12907. Existing proposals #4618 and
#8945 cover per-agent or per-worktree author configuration. This change
instead follows the accepted human instruction across runtime types.
Refs #11831 for governed personal connection delegation; this change
preserves connection audience checks and does not use standing
delegation as a personal credential fallback.

## What Changed

- Add durable, ordered identity contexts and active run references.
Preserve message authors through consolidation, steering, retries,
delegation, approvals, routines, and restart.
- Add an authenticated operation-time GitHub credential broker and
local/remote managed git and gh launchers. Keep personal tokens out of
the long-lived provider process.
- Resolve GitHub gateway and server-side Git operations through the same
responsible-person or dedicated-grant selection rules.
- Make absent and unavailable GitHub credentials non-blocking at generic
startup. Clear host and prior-person credentials. Keep anonymous Git
access where supported.
- Add run-detail identity history and the dedicated-account warning.
Keep task ownership and queue-versus-steer decisions unchanged.
- Preserve personal OAuth declarations through connection edits. Retain
exact selected grants in the gateway.
- Fix continuation races found during real acceptance: verify a warm
owner before credential rotation, and wait for bounded durable runner
suspension before the next run starts.
- Make migrations replay-safe. Retain identity through agent/run
deletion, remove it with its company, and clean terminal launcher
directories before releasing execution environments. Document
coordinated release and rollback.

## Verification

- Full workspace typecheck, build, and token gates passed. The complete
local suite passed in its normal test groups: 17,120 passing tests,
including all 143 serialized server suites. After integrating the newly
merged runner API work, full local typecheck and build passed again,
along with 890 focused integration tests. All 31 checks on the
integrated revision passed, including build, browser E2E, release
registry, canary dry run, typecheck, security and all test suites.
Greptile is 5/5 with all review threads resolved.
- Current focused checks passed: 142 native executor tests, 67 runtime
lifecycle tests, 9 durable identity tests, 75 credential/routine tests,
19 low-trust/resumption tests, and the executable migration replay test.
- Authenticated browser acceptance with two Paperclip users and two
GitHub accounts on one shared native agent passed. Real commits and
pushes followed A → B accepted steering → queued A continuation in the
same saved conversation. GitHub commit author and committer identities
matched all three operations. Both runs succeeded and task ownership
stayed unchanged.
- Real GitHub MCP calls switched from A to B after accepted steering. A
delegated subtask retained its originating identity across a server
restart.
- Disabling B's GitHub connection left ordinary work successful. Managed
gh was unauthenticated and the provider had no inherited GH_TOKEN or
GITHUB_TOKEN.
- The browser displayed run-detail diagnostics and the exact
dedicated-account warning. A final controller-restart check followed by
another-person continuation retained the conversation, selected the
correct GitHub login and Git author, and removed each terminal launcher
directory.
- Company-lifetime migration and all five previously failing CI suites
passed locally (167 tests). Same-token gateway A → B → A and six
broker/launcher boundary tests passed.
- Remote callback, launcher, sandbox, and runtime contract tests passed.
Both native and legacy Codex completed actual Daytona executions on the
integrated revision ([campaign
results](https://github.com/paperclipai/paperclip/actions/runs/34155056509)).
The remote package-manager shim staging regression also passed locally.

## Risks

- Deploy the migrations, server broker, launchers, and runner artifacts
together. Existing processes finish with their original contract. New
managed processes need the broker endpoint for GitHub operations.
- Finish or stop new managed executions before rolling application code
back. Keep the additive schema and identity history during rollback.
- Scripts that require a persistent raw GH_TOKEN must use managed git,
gh, or GitHub gateway tools. Run capabilities authorize code executing
within that run to acquire its current identity; this is not
hostile-code isolation within one execution principal. Managed commands
prevent automatic credential carryover; arbitrary code deliberately
copying a credential is outside that boundary.
- Uncertain steering acknowledgement deliberately holds new credential
acquisition until reconciliation. Already-started operations retain
their captured identity.
- GitHub private access and provider outages can still fail the specific
operation that needs them. Dedicated grant failure does not fall back to
personal access.

## Model Used

OpenAI GPT-6 through Codex assisted implementation, review, shell
execution, and browser acceptance. The exact model variant and
context-window size are not exposed in this session. Tool use included
TypeScript and Rust tests, database integration tests, GitHub CLI, and
authenticated browser control.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-07 14:32:20 -05:00
DottaandPaperclip 0ffc091473 feat(connections): add durable GitHub identities and webhooks (#12843)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents need source control access for repository work
> - A shared token cannot preserve the responsible person's identity or
an agent's dedicated identity
> - GitHub App tokens also need durable refresh, repository access
checks, and webhook delivery
> - Paperclip already has managed connections, encrypted grants, run
secret leases, and merge-confirmation behavior
> - This pull request extends those systems with GitHub identities
instead of adding a parallel credential system
> - The benefit is durable GitHub access with explicit identity,
repository, runtime, and webhook boundaries

## Linked Issues or Issue Description

No public GitHub issue describes this connection change. This
description follows the feature request template.

**Subsystem affected**

Connected Apps, connection grants, secret resolution, native Git runtime
setup, webhook processing, and the Apps UI.

**Problem or motivation**

Users need to connect GitHub once and let agents use the correct GitHub
identity. A run should use a dedicated agent account when one exists.
Otherwise, it should use the responsible person's account. The
connection must survive token expiry, repository access changes, and
temporary instance downtime.

**Proposed solution**

Add user-owned and agent-owned GitHub grants to the existing connection
model. Resolve one identity for MCP, Git, `gh`, health checks, and
webhook bindings. Store provider tokens in the existing encrypted secret
system. Refresh expiring token pairs under the existing lease and
compare-and-swap path. Register signed Cloud webhook bindings and
process normalized pull request and installation events through a
durable local inbox.

**Alternatives considered**

An organization-wide GitHub token would lose person and agent
attribution. Environment variables alone would bypass the managed
connection and grant model. A new GitHub-only credential store would
duplicate the existing secret and access systems. GitHub App
installation tokens and private-key custody remain outside this first
version.

**Roadmap alignment**

This change implements the Connected Apps direction. It also extends the
shipped MCP Tool Gateway, per-agent secret access, and
action-attribution systems. It does not add a repository catalog. The
open repository catalog work in
[#11234](https://github.com/paperclipai/paperclip/pull/11234) is related
and complementary.

## What Changed

- Added agent-owned connection grants and a per-agent credential policy
with company and subject constraints.
- Added a managed GitHub App method while keeping the personal access
token method as an advanced fallback.
- Added durable access-token and refresh-token handling with proactive
rotation and one automatic recovery after a provider `401`.
- Added GitHub identity and installation summaries without storing
repository-name lists.
- Added signed Cloud webhook binding, event lease, acknowledgement,
local idempotency, pull request merge processing, and installation
access handling.
- Added one identity resolver for MCP, native Git, `gh`, checkout,
health checks, and webhook bindings.
- Added a class-3 run projection for `GH_TOKEN`, `GITHUB_TOKEN`, a
`github.com`-only credential helper, SSH-to-HTTPS rewrite, and GitHub
noreply commit attribution.
- Added personal and dedicated-agent setup choices plus identity,
repository, continuity, and webhook status in the Apps UI.
- Added schema migrations, tests, and connection documentation.

## Verification

- The current head is fully green in GitHub CI, including build,
typecheck, all serialized/general server shards, all browser shards,
policy, canary dry run, review, and security checks.
- Live staging proof completed with a non-expiring GitHub App user
token, selected-repository installation, repository add/remove refresh,
managed MCP, native `gh`, HTTPS clone/push/delete, GitHub noreply commit
attribution, signed merged-PR webhook acceptance, durable
Cloud-to-instance delivery, and installation-access event processing.
Temporary branches and temporary repository access were removed
afterward.
- `pnpm check:token-gates` passed.
- `pnpm -r typecheck` passed before and after the rebase onto
`origin/master`.
- `pnpm build` passed.
- The focused connector suite passed 285 tests after the rebase.
- The full stable suite passed 5,790 tests and failed 22 tests across 8
general server files. The failures reproduced as shared-runner
environment issues. They included `/tmp` versus `/private/tmp`, closed
database connections, and invalid high ephemeral ports. The focused
connection tests pass in isolation.

## Risks

- Migrations add agent grant subjects and a durable connection-event
inbox. Migration numbering and safety checks pass.
- A raw GitHub user token enters the agent process for Git and `gh`.
Per-tool Ask-first controls cannot limit those shell operations. The UI
warns users about this boundary.
- GitHub App user tokens can be non-expiring. Paperclip performs a
continuity check every 30 days, but provider revocation still requires a
reconnect.
- The webhook path accepts only signed and bounded payloads. It stores a
minimal normalized record and no raw provider payload.
- GitHub repository permissions remain authoritative. Removed access can
make a cached repository count temporarily stale, but runtime access
fails immediately.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex, `gpt-5.6-sol`, extended reasoning, tool use, code
execution, browser control, and multi-file repository editing. The
context window size was not provided.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-04 18:02:52 -05:00
Devin Foley 2c90cf0f2c fix(server): serialize managed-checkout materialization and stop misattributing clone failures (#10723)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Repo-only project workspaces are materialized by a server-side
managed `git clone` into a per-project directory (#10720 added
credentials for private repos)
> - Two issues on the same project routinely wake seconds apart, and
both runs race the same clone target
> - The loser fails with "destination path already exists", and its
failure cleanup removes the directory out from under the winner's
in-progress clone — both runs then fail every round
> - This pull request serializes materialization per target directory
and makes the clone land atomically via a temp sibling + rename, so the
shared target is never partial and never removed
> - The benefit is that concurrent runs on the same project converge:
one clone happens, everyone adopts it, and unrelated failures no longer
blame the GitHub credential

## Linked Issues or Issue Description

**What happened?**

With isolated workspaces enabled on a project whose only workspace is
repo-only, unblocking two issues at once produced lockstep mutual
destruction (observed live, two consecutive rounds): both runs called
the managed-checkout materialization concurrently; one clone created the
target directory, the other's `git clone` failed with `fatal:
destination path '…' already exists and is not an empty directory`, and
that run's failure cleanup deleted the directory while the first clone
was still writing into it (`fatal: could not set 'core…'`). Both runs
failed `workspace_validation_failed`; their staggered retries could race
again. The failure message also wrongly claimed the GitHub credential
"was rejected or lacks access" — the collision had nothing to do with
auth.

**Expected behavior**

Concurrent materializations of the same project checkout share one
clone; a completed checkout is never removed by a failing sibling; the
credential is only blamed for auth-shaped failures.

**Steps to reproduce**

1. Project with a repo-only workspace (private repo, isolated workspaces
on).
2. Move two issues on that project to `todo` at the same time so both
runs start within seconds.
3. Both runs fail workspace validation with "destination path already
exists" / "could not set 'core…'" instead of one clone succeeding.

**Paperclip version or commit**

`master` (75f6256b76).

## What Changed

- `ensureManagedProjectWorkspace` serializes in-flight materializations
per target cwd (a module-level promise map): concurrent callers share
one attempt.
- The clone lands in a `<target>.clone-XXXXXX` temp sibling created with
`mkdtemp`, then moves into place with an atomic `rename`. Clone failure
removes only the temp directory; the shared target is never created
partially and never deleted. If the target appears between the emptiness
check and the rename (another process won), the completed checkout is
adopted instead of failing the run.
- `describeGitAuthFailure` attributes the GitHub credential only when
the error matches the auth-failure pattern; unrelated failures (path
collisions, network errors) no longer claim the token was rejected.

## Verification

- `cd server && npx vitest run
src/__tests__/heartbeat-managed-clone-credentials.test.ts
src/__tests__/git-credentials.test.ts
src/__tests__/heartbeat-workspace-session.test.ts` — 169 tests pass,
including new cases: concurrent materializations of the same checkout
succeed with one shared result and no temp litter; failed clones leave
neither target nor temp directories; an authenticated clone failing for
non-auth reasons does not blame the credential.
- `pnpm --filter @paperclipai/server typecheck` — clean.

## Risks

- Low risk. The serialization is in-process and keyed by exact target
path; the temp+rename pattern stays on the same filesystem (sibling
path) so the rename is atomic. Single-run behavior is byte-identical
apart from the temp-dir intermediate.
- The rename-conflict adoption path accepts a checkout another
materialization completed; the pre-existing `gitDirExists` adoption
semantics are unchanged.

## Model Used

Claude Fable 5 (`claude-fable-5`, extended thinking, agentic tool use
via Claude Code CLI).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-02 22:14:58 -07:00
Devin Foley e0c2448267 feat(server): authenticate server-side git clone and fetch with a company-secret GitHub token (#10720)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Repo-only project workspaces are materialized by a server-side `git
clone`, and isolated `git_worktree` runs refresh their base ref with
server-side `git fetch`
> - Both operations run outside the agent process with no credentials,
so private GitHub repositories can never be cloned or refreshed —
agent-scoped credential env bindings do not reach them
> - The company secret store already has a well-known GitHub token
convention (`GITHUB_TOKEN` / `GH_TOKEN` / `PAPERCLIP_GITHUB_TOKEN`,
consumed by the external-object provider for API reads), but nothing
server-side consults it for git
> - This pull request resolves that token per run and authenticates the
managed clone and every base-ref refresh with it through an ephemeral
credential helper
> - The benefit is that isolated workspaces work on private repositories
with one company secret, while public repositories and self-hosted
ambient git configuration keep working unchanged

## Linked Issues or Issue Description

**Subsystem affected**

Server workspace materialization (`server/src/services/heartbeat.ts`)
and execution-workspace realization
(`server/src/services/workspace-runtime.ts`).

**Problem or motivation**

A project workspace configured with only a private GitHub `repoUrl`
cannot be used for isolated `git_worktree` runs: the managed `git clone`
runs with a sanitized, credential-less environment, and plain git cannot
consume a bare token env variable without a credential helper. There is
no way to give the server a git credential — storing a `GH_TOKEN`
company secret has no effect on server-side git, and a credential-less
private clone hangs on a terminal prompt until the ten-minute clone
timeout. Base-ref refreshes (`git fetch`) during worktree realization
have the same gap.

**Proposed solution**

A `git-credentials` module resolves a token per run — company secret by
well-known name (`GITHUB_TOKEN`, `GH_TOKEN`, `PAPERCLIP_GITHUB_TOKEN`),
then `GITHUB_TOKEN`/`GH_TOKEN` in the server process environment for
self-hosted deployments, then none — and builds a git invocation that
authenticates via an inline credential helper. The token travels in an
env variable; it never appears in argv, URLs, or on disk. Only
`https://github.com` remotes are authenticated; everything else keeps
ambient behavior. The provider is a single factory seam so a future
brokered credential source can replace it without touching call sites.

**Alternatives considered**

- A GitHub OAuth "connect your account" flow: heavier product surface,
needs app registration and callback custody; out of scope for a server
credential and better served by a dedicated connector later. The
provider seam keeps that path open.
- `gh auth setup-git`: writes helper configuration to disk and requires
a global token env; rejected in favor of per-invocation config with no
persistent state.
- Embedding the token in the clone URL: leaks into argv, error messages,
and `.git/config`; rejected.

## What Changed

- New `server/src/services/git-credentials.ts`:
`createGitRemoteAuthProvider` (memoized per run, one secret resolution
and one audit event), `buildGitAuthInvocation` (helper-reset + inline
helper, `x-access-token` username, `GIT_TERMINAL_PROMPT=0`),
`isGitHubHttpsRemoteUrl` host gating (rejects ssh/GHES/http/other
hosts/userinfo URLs), `describeGitAuthFailure`, and the canonical
`scrubGitCredentialText`. Secret resolutions pass a `system` consumer
access context so they are recorded as secret access events.
- `ensureManagedProjectWorkspace` (now exported) accepts an optional
auth provider; the clone env spreads the token after
`sanitizeRuntimeServiceBaseEnv` (which strips `PAPERCLIP_*`), always
sets `GIT_TERMINAL_PROMPT=0`, distinguishes "credential rejected" from
"no credential configured — add a GITHUB_TOKEN or GH_TOKEN company
secret" in the error, and removes the partially created directory on
clone failure so a timeout-killed clone cannot be adopted as a broken
checkout by the next run.
- `refreshRemoteTrackingBaseRef` (now exported) captures the remote URL
it already looked up, asks the provider for an invocation, and
attributes failed authenticated fetches to the credential in a scrubbed
warning. The optional provider threads through `detectDefaultBranch`,
`resolveAuthoritativeBaseRef`, `inspectExecutionWorkspaceBaseDrift`,
`realizeExecutionWorkspace`, and
`ensurePersistedExecutionWorkspaceAvailable`; heartbeat builds one
provider per run for both the anchor-resolution clone path and workspace
realization/restore.
- `github-external-object-provider.ts` imports the shared secret-name
list; `isGitHubDotCom` is exported from `github-fetch.ts`.
- Docs: "Private repositories and repo-only project workspaces" section
in the execution-workspaces guide, cross-linked from the secrets deploy
doc.

## Verification

- `cd server && npx vitest run src/__tests__/git-credentials.test.ts` —
resolution chain order and precedence, env fallback, memoization,
audited access context, host-gating matrix, invocation shape (token
absent from argv), scrubber, failure descriptions, and a real-git `git
credential fill` round trip that proves the helper executes and answers
with the env-carried token (no network).
- `cd server && npx vitest run
src/__tests__/heartbeat-managed-clone-credentials.test.ts` — clones
behave byte-identically with no provider or a null-returning provider
(local repos, no network), authenticated-failure errors name the
credential, non-auth failures do not mention credentials, partial clone
directories are removed, pre-existing non-git directories keep the
"Using it as-is" path, and the sanitizer spread order keeps the token
env alive.
- `cd server && npx vitest run src/__tests__/workspace-runtime.test.ts`
— new `refreshRemoteTrackingBaseRef` cases: provider offered the remote
URL and null keeps behavior identical; failed authenticated fetch
warning names the credential; unauthenticated failure warning stays
credential-free.
- `pnpm --filter @paperclipai/server typecheck` is clean.
- Manual (optional, networked): store a `GH_TOKEN` company secret,
configure a repo-only project workspace pointing at a private GitHub
repository, run an isolated-workspace issue — the managed clone succeeds
and the worktree run proceeds.

## Risks

- Every new parameter is optional; with no provider the git invocations
are byte-identical to before. Public repos and ambient credential
helpers keep working whenever no token resolves.
- Precedence change when a token exists: a stored company secret now
wins over ambient helpers for `https://github.com` remotes (the helper
list is reset for that invocation). The rejected-credential error names
the secret so an operator can fix or remove it.
- `GIT_TERMINAL_PROMPT=0` on the managed clone is the one always-on
change: a credential-less private clone now fails fast with a clear
message instead of hanging until the ten-minute timeout (it could only
ever "succeed" interactively on a TTY dev server).
- The token is scoped to the git process env for one invocation; it is
never written to agent env, run context, disk, or logs, and error text
is scrubbed of URL userinfo.
- No migrations, no image changes (git ships in the image).

## Model Used

Claude Fable 5 (`claude-fable-5`, extended thinking, agentic tool use
via Claude Code CLI).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-02 20:27:09 -07:00