Commit Graph
3 Commits
Author SHA1 Message Date
DottaandPaperclip b54b2dc35c fix: preserve warm Codex turns with incremental managed file checkpoints (#14735)
## Thinking Path

> - Paperclip manages AI agents and keeps their instructions and files
durable.
> - Native Codex runners can keep a process alive between compatible
turns.
> - Managed file collection stopped that process after each turn, which
defeated warm reuse.
> - Agent folders can contain large images and other files, so full
copies on every turn are expensive.
> - This change keeps one managed directory for the live session and
saves only file changes after each turn.
> - Ownership, authorization, instruction changes, and process
retirement still control when reuse is safe.

## Linked Issues or Issue Description

Related: #13710 introduced native warm session reuse. This fixes managed
file collection that still forced those sessions to stop. No duplicate
open PR or issue was found.

**What happened?**

With managed instructions and warm native Codex enabled, consecutive
turns reused a Daytona sandbox but started a new runner process each
time. The managed directory collector required process termination
before saving files.

**Expected behavior**

Compatible turns keep the same process and managed `AGENT_HOME`. Each
completed turn saves added, changed, and deleted files before the next
turn starts. Unchanged large files do not transfer again.

**Steps to reproduce**

1. Use a native Codex agent with managed instructions and a reusable
Daytona environment.
2. Enable warm session reuse and run three turns on the same task.
3. Write a large binary on the first turn, edit a small note on each
turn, and delete a file on the second turn.
4. Compare process identity across turns and read the canonical files
through the public agent-files API.

**Paperclip version or commit**

Reproduced on `d30b03bd8c17604cdab1533eeeeb087aba30e8b1`.

**Deployment mode**

Local server with remote Daytona execution; cloud native runner uses the
same path.

## What Changed

- Retain the managed directory only for the verified owner of a live
native Codex session.
- Checkpoint each completed turn before releasing the session for reuse.
Retry unstable captures, then stop and collect when a warm checkpoint
cannot be validated.
- Compare metadata and cached hashes, stream only changed file payloads,
record deletions, and validate path, content, quota, and authorization
before saving.
- Rotate sessions when canonical files, loaded instructions,
credentials, or launch policy change. Fence stale collection and cleanup
callbacks from later owners.
- Keep cleanup and recovery aware of the current session owner. Recheck
canonical files under the writer lock at handoff, attach the successor
collector before fallible bookkeeping, and emit one final save receipt
on checkpoint fallback. Preserve storage warnings across unchanged
checkpoints.
- Add regression coverage and a three-turn Daytona test with independent
public API file checks, an unchanged 8 MiB binary, deletion checks, and
strict process identity checks.
- Document checkpoint consistency, lifecycle behavior, and local run-log
counters.
- Replace a timing assumption in the Daytona teardown test with explicit
transfer-arrival gates after CI exposed an unset release callback.

## Verification

- Full local `pnpm -r typecheck` and `pnpm build` passed. Server checks
were repeated after the final storage-warning fix.
- Runner E2E typecheck and 749 runner E2E unit tests passed.
- Focused file checkpoint, directory ownership, instruction collection,
native session, and merge tests passed. After review fixes, the
managed-directory and native-session suites passed 550 tests, including
intervening canonical edits, same-run fresh restore, failed handoff
collection, and one-call fallback collection. Server typecheck passed
again. The Daytona plugin suite passed 218 tests. The quota-warning
regression failed before the fix and passed afterward.
- Three real Daytona campaigns passed before the final handoff review
fixes. The latest kept PID 547 across all three turns. The first
checkpoint copied 8,388,635 bytes; the next two copied 36 and 54 bytes.
Public API reads verified the binary, note contents, and deletion after
every turn. Test cleanup deleted the sandbox.
- The final head was also deployed to an isolated cloud staging instance
and passed three UI-triggered native Codex turns with managed
instructions. All three retained the same process ID/start time, native
session, provider session, runner instance, and Daytona sandbox.
Checkpoints copied 8,388,643 bytes on turn 1, then only 52 and 78 bytes
on turns 2 and 3; those warm captures also hashed only 52 and 78 bytes.
Independent canonical API reads verified every byte of the unchanged 8
MiB binary and the exact note contents after every turn; the deleted
file returned 404 after turns 2 and 3. After restoring the original
lifecycle and agent-auth configuration, removing the temporary secret,
pausing the test agent, and deleting both test sandboxes, independent
canonical API reads still verified the entire binary, the final 78-byte
three-line note, and the deletion. The native runner flag remained
enabled and the final serving revision remained the PR head.
- Two earlier staging attempts are preserved as failures and are
excluded from the acceptance result: a saved ChatGPT login failed with a
provider routing 401, and its subsequent stopped-sandbox retry failed
before provider startup with a closed-lease admission error. The
successful campaign used a fresh sandbox and a temporary encrypted
API-key binding. The stopped-lease retry remains unexplained; this
campaign does not establish recovery of that failed sandbox.
- All [Paperclip CI
gates](https://github.com/paperclipai/paperclip/actions/runs/36750397355)
pass on `26ef2ef56a389259246809805c0b34a4747eb86b`, including full test
partitions, build, typecheck, runner verification, E2E shards, and the
Canary clean public-npm install. Greptile reviewed that exact head at
5/5 with no unresolved review threads or outstanding findings.
- Full local repository coverage used the existing CI partitions, but
the 40,000-file Git streaming stress test timed out and its local retry
was interrupted by macOS thermal emergency sleep; this is not a green
full local suite claim. The exact stress test passed on the final head
in [CI server shard
2/12](https://github.com/paperclipai/paperclip/actions/runs/36750397355/job/110008294290),
in 111.9 seconds.
- Repeat the live test with configured credentials and a Linux runner
artifact: `pnpm test:e2e:runner -- --id
daytona-warm-continuity.runner-codex.daytona.warm-three-turn`.

## Risks

- This is a file-level checkpoint, not an atomic snapshot of the whole
folder. Background writes after a capture are saved by the next
checkpoint or final stopped collection.
- Metadata scans still visit all paths. Modified files transfer in full;
unchanged files do not rehash or transfer.
- Incorrect ownership or reuse could collect the wrong directory. Run
ownership fences, current authorization, stable capture validation, and
stopped collection fallbacks are covered by tests.
- Warm reuse remains opt-in. No database migration or fleet default
changes.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, code editing, tool use, and
test execution. The exact serving model ID and context-window size are
not exposed by this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-30 13:33:37 -05:00
Nicky LeachandPaperclip a20a4944ec feat: add Grok device login to the sandbox login panel (#12469)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Paperclip uses adapters to connect agents and model providers to its
control plane
> - The sandbox login panel supports displayed-code login for selected
adapters
> - Grok users need the same login path and a private credential home
for later runs
> - This pull request adds Grok support to the shared device-login path
and preserves the existing Codex path
> - The benefit is one secure login flow for both adapters with
company-scoped credential storage

## Linked Issues or Issue Description

**Agent or provider**

Grok Local needs displayed-code login support in the sandbox login
panel.

**Why this adapter is useful**

This change lets users sign in to Grok from the sandbox login panel. It
also gives later Grok runs access to the stored credential.

**How the agent is invoked**

The Grok local adapter uses its login command through the shared
displayed-code login flow. Later runs receive the managed home through
`GROK_HOME`.

**Additional context**

The change uses adapter-scoped login lifecycle handling. It stores the
credential in a company-scoped directory with mode `0700`, and it stores
the credential file with mode `0600`.

## What Changed

- Rename the shared device-login modules to adapter-neutral names.
- Scope the shared login lifecycle to a closed adapter set.
- Return the device-login URL that the provider prints.
- Add the Grok prompt parser, login command, capability, and login panel
entry.
- Store the Grok credential in a private, company-scoped home directory.
- Pass `GROK_HOME` to later Grok runs.
- Add tests for the Grok adapter, the Daytona sandbox provider, the
server login path, and the user interface.

## Verification

- Run `pnpm vitest run
packages/adapters/grok-local/src/server/adapter-auth-promotion.test.ts`.
- Run the Grok adapter package suite.
- Run the Daytona sandbox provider suite.
- Run the server device-login suites.
- Run the user interface suite.
- Confirm the full CI suite passes.

## Risks

The change extends shared login lifecycle code to another adapter. A
regression could affect Codex login. The credential path uses explicit
`chmod` calls to keep the directory at mode `0700` and the file at mode
`0600`.

## Model Used

OpenAI Codex, GPT-5. The runtime used tool calls and code review
support. The runtime did not provide a context-window value.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-28 21:48:34 -07:00
Nicky LeachandPaperclip 63df7ad2b3 feat(login): use the login pseudo-terminal for Codex device login and de-Claude the shared channel (#12020)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agent adapters use provider-specific login flows
> - Codex device login needs a live pseudo-terminal (PTY), while the
shared channel still uses Claude-specific names
> - The old streamed-exec path does not provide the prompt transport
that Codex needs
> - This pull request moves Codex device login to the shared login PTY
and removes the dead streamed-exec path
> - The benefit is one controlled login transport with fail-closed
capability checks and safer credential reads

## Linked Issues or Issue Description

**Problem or motivation**

Codex device login used a streamed-exec path that did not provide the
required prompt transport. The shared login channel also exposed
Claude-specific names outside Claude code.

**Expected behavior**

The host selects a fixed login command from trusted adapter data. Codex
login uses the provider login PTY. Providers without that capability
fail closed.

**Proposed solution**

Use a server-controlled session home, create and validate it as a fresh
0700 directory, read credentials from one validated descriptor, and
rename shared channel names to the neutral login PTY family.

**Alternatives considered**

Keep the shared login PTY as the single transport. Do not keep the
removed streamed-exec path because it cannot provide the required prompt
transport.

**Roadmap alignment**

This change supports the planned login transport work. It does not add a
separate roadmap item.

## What Changed

- Route Codex device login through the shared login PTY transport.
- Select the login command from a closed internal command key.
- Carry a server-controlled session home through the launch contract.
- Create and validate the session home as a fresh 0700 directory owned
by the login user.
- Read the credential file with descriptor-relative, no-follow path
walking and final descriptor checks.
- Gate the login route and run lease on the provider login PTY
capability.
- Rename shared channel names to the neutral login PTY family.
- Remove the streamed-exec transport value, selector field, driver
branch, and related tests.
- Hide Codex login in the user interface when the provider lacks the
login PTY capability.

## Verification

- Server unit suites pass: 89/89.
- Adapter-utils suites pass: 262/262.
- Codex-local suites pass: 326/326.
- Credential-read reader suite passes: 20/20.
- Daytona login PTY suite passes: 30/30.
- Device-login suites pass: 56/56.
- TypeScript checks pass for server, adapter-utils, and UI.
- GitHub Actions must pass after pull request creation.
- Greptile review must reach 5/5 with no open P2 findings,
recommendations, or follow-ups.

## Risks

- Providers without a login PTY capability lose Codex login support by
design.
- The credential read rejects invalid ownership, mode, type, path, and
size.
- The launch-time sandbox directory race remains outside the threat
model because the login runs inside the sandbox and a hostile sandbox
already controls its credential.

## Model Used

OpenAI Codex, GPT-5, tool use and code review assistance. The exact
context window and reasoning mode are not exposed by the runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-23 09:44:59 -07:00