Commit Graph
5 Commits
Author SHA1 Message Date
DottaandPaperclip 042051904d fix(codex): deliver configured service secrets to shell tools (#17216)
## Thinking Path

> - Paperclip lets operators assign tools and credentials to agents.
> - Legacy Codex starts provider processes and builds a separate shell
environment policy.
> - Both ACP and CLI removed every variable whose name contained KEY,
SECRET, or TOKEN.
> - This also removed service credentials that the operator explicitly
assigned, such as SLACK_BOT_TOKEN.
> - This pull request uses the resolved configuration to admit assigned
service secrets by name.
> - Every Codex launch keeps a bounded shell allowlist, including
launches without an agent identity.
> - Assigned scripts can use their credentials without adding unassigned
host secrets to shell tools.

## Linked Issues or Issue Description

Related: #16255. That PR fixes project-level exclude patterns. This PR
fixes the removal of explicitly assigned service secrets and the
fallback to an inherited policy when no agent identity is present.

**What happened?**

A Codex task could not run a Slack posting script. SLACK_BOT_TOKEN was
present in its resolved launch environment, but Paperclip removed its
name from shell_environment_policy.include_only. A live removal check
also showed that an identity-free launch without an explicit policy
could inherit an unassigned host token.

**Expected behavior**

An explicitly assigned service credential reaches shell commands.
Removing that binding removes its shell access on the next launch.
Unassigned host credentials stay outside the allowlist. Codex login
credentials and observability headers keep their separate provider
paths.

**Steps to reproduce**

1. Configure a Legacy Codex agent with a synthetic SLACK_BOT_TOKEN
value.
2. Run a shell command that checks whether the variable exists without
printing its value.
3. Before this fix, the variable is absent. Repeat through ACP and CLI.
4. Remove the binding and run a second turn. Check that neither the
removed token nor an unassigned host token reaches the command.

**Paperclip version or commit**

Reproduced in the executor regressions before the fix on master at
4dfcb4c9b8. The final change is rebased on 25f2d05282.

**Deployment mode**

Source checkout. Local macOS arm64 provider acceptance with Codex CLI
0.160.0 and Codex ACP 1.6.2, gpt-5.6-sol, and an existing ChatGPT login.

## What Changed

- Added one shared Codex shell-name policy for Legacy ACP and CLI
execution.
- Admit explicitly resolved service-secret bindings. Keep Codex login
keys, observability credentials, and private runtime transport tokens
outside that opt-in.
- Reject glob-shaped environment names so one configured name cannot
match other credentials.
- Always emit the bounded policy. Do not fall back to host inheritance
after a binding is removed.
- Updated executor regressions, the identity contract, and the paired
verification maps.

## Verification

- Red before the fix: executor regressions failed because
SLACK_BOT_TOKEN was absent in ACP and CLI, with and without agent
identities.
- Green: 267 tests passed across the two executor suites and the shared
shell-policy suite. Coverage includes host/provider isolation,
unresolved and removed bindings, glob names, identity delivery, and
names-only metadata.
- The complete Codex adapter source suite passed after the CI follow-up:
571 tests passed, 1 skipped across 31 files. Its typecheck also passed.
- The first CI run caught four stale expectations in ACP config and SSH
launch tests. Those now assert the bounded policy and the existing
command/resume behavior. No production code changed in that follow-up.
- Live CLI and ACP acceptance: run a Python presence check through each
real adapter, remove the synthetic Slack binding, and run again. The
assigned token was present only on the first turn. The unassigned host
token and OpenAI API key were absent on both turns. CLI retained its
session; ACP correctly invalidated its session after the configuration
changed. Workspace proof files persisted.
- An earlier live CLI check failed after binding removal because the
no-policy fallback exposed the synthetic host token. The final change
removes that fallback. Both live checks then passed.
- Full pnpm -r typecheck and pnpm build passed. The changed packages
were compiled again after the safety correction.
- node scripts/check-maps.mjs and git diff --check passed.
- The local pnpm test:run attempt reported failures in native
warm-session tests and was interrupted. That full-suite result is not a
pass; PR CI must establish the final repository-wide gate. One named
native failure passed alone, and the complete native executor file then
passed separately (774 tests). The original broad-run failure cause
remains unconfirmed.
- After rebase, one Gemini skill-root case timed out at five seconds
while the broad run was active. The case passed alone, and the full
focused set then passed again at the PR head (267 tests). The earlier
timeout remains part of the verification record.
- An initial broad adapter run failed because sandbox restrictions
denied loopback listeners. Running the same suites with loopback access
passed.

## Risks

- Explicitly assigned service secrets now reach shell commands. This is
the intended behavior change: assigning a credential authorizes the run
to use it, and a model-directed command can read it. This is not a
broker that hides assigned credentials from the model. Operators must
assign only credentials and service permissions authorized for the run.
Configuration and secret-access authorization remain with their existing
owners.
- Superagent initially flagged this intended assigned-credential
exposure as a P2. The inline reply records the trust-model rationale and
the documentation states it explicitly. The security scan on the final
documented revision passed without any check override.
- The launch policy contains names, not values. Log redaction remains in
place. Provider login credentials and observability headers remain
excluded from the service-secret opt-in.
- Identity-free Codex runs now also receive a bounded policy. This
removes reliance on the operator's inherited shell policy.
- Existing explicit project exclude patterns retain their behavior;
#16255 addresses that separate case.
- The change has not been deployed to a shared instance.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository tools, and code
execution. The exact served model ID and context-window size are not
exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with Fixes / Closes /
Refs OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run the focused tests locally and they pass; the full-suite
limitation is recorded above
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-22 18:17:34 -05:00
DottaandPaperclip a590ab769d Give managed agents persistent cryptographic identities (#15352)
Give agents persistent Ed25519 identities encrypted with the existing instance master key. Create keys transactionally for new agents and lazily before supported managed runs, expose public identities in the API and agent UI, and protect private material during runtime delivery and output persistence.

Preserve identities in recovery backups while giving imported and development-cloned agents fresh keys.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-06 13:57:03 -05:00
DottaandPaperclip cbc5132e6c fix(adapters): expose a verified provider stop before workspace restoration (#14311)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agent adapters own local or remote provider processes.
> - Run cleanup must know when the final provider process has stopped.
> - A remote timeout or a lost transport does not prove that the
provider stopped.
> - Retried provider invocations also make an earlier stop signal stale.
> - This pull request adds a verified final-invocation stop callback
before workspace restoration.
> - The dependent instruction revision change uses that boundary to
preserve private instruction edits safely.

## Linked Issues or Issue Description

**What happened?**

Adapter completion did not expose a reliable point between provider
shutdown and workspace restoration. Cleanup could lose provider-written
files, or treat a remote timeout as proof that a process stopped.

**Expected behavior**

Cleanup runs once after the final provider invocation has a verified
stop receipt and before workspace restoration. An incomplete remote
command keeps collection pending.

**Steps to reproduce**

1. Run a remote provider that returns a timeout without a numeric exit
code.
2. Let the adapter return or retry the provider.
3. Attempt to collect provider-written files during cleanup. The adapter
has no verified final-process boundary to use.

This is the prerequisite for the stacked canonical instruction revision
pull request. It has no database or UI dependency. Related #13291
verifies remote termination for later recovery; this change exposes the
earlier adapter-owned stop boundary before workspace restoration. The
scopes do not duplicate each other.

## What Changed

- Add a stop callback to adapter execution context and a
final-invocation fence.
- Confirm local child closure and complete remote exit receipts. Reject
remote timeouts, missing exits, transport failures, and SSH exit 255 as
stop proof.
- Invoke collection once before workspace restoration in eight CLI
adapters and at the confirmed ACP stop boundary.
- Preserve the stop observation when a later log flush fails.
- Run bridge and workspace cleanup in `finally` even when collection
rejects. ACP records a safe error without exposing a raw filesystem
path. Grok keeps collection errors separate from workspace restore
failures and preserves completed provider results when both cleanup
steps fail.
- Correct the existing Cursor test shell fixture so bounded remote file
reads run against real fixture files.

## Verification

- Three stop-boundary regressions failed before the callback
implementation and passed after it.
- Independent prerequisite branch: 377 tests passed across 24 adapter,
process-target, and ACP suites. Two added collector-rejection tests
failed before the cleanup fix and passed after it.
- A third regression reproduced Grok misclassifying a collection failure
as failed workspace restoration. Two additional cases covered completed
and failed provider turns when collection and restore both fail. The
Grok and restore-classifier suites passed 48 tests.
- All nine affected package typechecks and affected package builds
passed; Grok checks passed again after its classification fix.
- Integrated instruction branch: native local, legacy local, and native
Daytona each passed three browser tasks with exact persisted bytes,
fresh-task readback, history/restore, and explicit conflict resolution.
Unchanged warm Daytona passed three turns. Legacy Codex passed all five
checks again after the exception-safe cleanup fix.
- Alternate staging passed the same three-task native Daytona flow:
exact stopped-run save, independent downloaded readback, browser
history/restore, and explicit resolution of a real concurrent edit. The
deployed source was `14c3d810c9e05625121b3d27767aea9317b03125`, which
covers the initial adapter callback. Later Grok cleanup failures are
qualified by the adapter tests above.

- The final combined native Daytona flow passed again on deployed
`2bedd0f23bf4698b1f8b818f6796900647030427`: three fresh tasks proved
ordinary instruction edits, independent readback, History/Restore,
concurrent board conflict, and explicit candidate resolution. This
native staging flow does not claim to exercise the Grok adapter.

## Risks

- An unverified remote stop intentionally does not trigger collection. A
later controller with verified stop evidence must recover it or report
the copy unavailable.
- The callback is optional. Callers that do not register it retain their
existing behavior.
- The callback runs before workspace restoration and can delay cleanup
if its caller does not bound its own work. The dependent instruction
collector uses bounded reads and retries. A rejected callback still
permits bridge and workspace cleanup; it cannot claim an instruction
save.

## Model Used

OpenAI Codex, GPT-6, with tool use and code execution. The runtime does
not expose the exact deployment variant or context-window size. Multiple
Codex agents implemented and verified the change.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-28 08:45:53 -05:00
Dotta 1dceee9a4e fix(runner): persist warm Daytona workspaces (#12901)
## Thinking Path

> - Paperclip manages AI agent work and the execution state for each
task.
> - Remote agents run in sandbox environments such as Daytona.
> - Daytona keeps files while a sandbox is stopped, but deletion removes
those files.
> - Runner Codex did not copy successful remote workspace changes back
to the host workspace.
> - A warm sandbox could therefore hide data loss until Daytona replaced
or deleted the sandbox.
> - This pull request makes the host workspace durable after every
successful turn and keeps verified reusable sandboxes warm.
> - The benefit is reliable multi-turn work across warm reuse, restart,
stop, and sandbox replacement.

## Linked Issues or Issue Description

**What happened?**

A successful native Codex turn in Daytona could leave workspace changes
only in the remote sandbox. A later warm turn appeared to work because
it reused that filesystem. A replacement sandbox could start from stale
host data and lose the successful changes.

**Expected behavior**

Paperclip must merge each successful remote turn into the authoritative
host workspace before it completes the run. A verified warm lease may
reuse its remote files. A replacement lease must reconstruct the exact
durable workspace seed.

**Steps to reproduce**

1. Run Codex in a reusable Daytona environment.
2. Write a file during one successful turn.
3. Replace the Daytona sandbox before the next turn.
4. Observe that the next turn can start without the prior file on the
unpatched code.

Related remote workspace foundation: #10070.

## What Changed

- Added explicit `host_current`, `durable_seed`, and `adopt_remote`
workspace preparation modes.
- Added atomic, versioned native workspace descriptors and seed archives
under `PAPERCLIP_HOME`.
- Added real native sandbox export and three-way host merge before
terminal result completion.
- Added workspace-only recovery after a proposed result. Recovery does
not submit another provider turn or consume the provider retry budget.
- Added fail-closed handling when a sandbox with unexported changes is
gone.
- Kept healthy reusable Daytona sandboxes started for legacy Codex and
Runner Codex.
- Kept the Runner Codex process and provider session across verified
warm turns.
- Added the paid `daytona-warm-continuity` browser suite. It contains
exactly the legacy Codex and Runner Codex cells. Each cell performs
three measured turns.
- Documented `pnpm test:e2e:runner -- --suite daytona-warm-continuity`.
No package script was added.
- Added no database migration. The metadata format is backward
compatible and idempotent.

## Verification

- `pnpm typecheck`
- `pnpm test:e2e:runner:unit` — 114 passed
- Native workspace, finalizer, session, and environment tests — 232
passed
- Daytona provider tests — 150 passed
- Workspace staging and merge tests — 98 passed
- Runner transport tests — 63 passed
- Legacy Codex restore tests — 5 passed
- Rust format and compile checks pass through root typecheck
- The paid Daytona suite was not run locally because the required
Daytona, OpenAI, and immutable image credentials are not present.

## Risks

- The main risk is an incorrect workspace identity or merge after a
crash. Durable descriptors bind the run, workspace, lease, provider
lease, local root, remote root, and baseline digest. Ambiguous evidence
fails closed.
- The host merge may conflict with concurrent host edits. The existing
three-way merge and exclusion rules handle this case and surface
failures.
- A deleted sandbox cannot recover unexported bytes. Paperclip now
blocks with `workspace_sync_out_unrecoverable` instead of reporting
success or rerunning the provider.
- There is no database migration. Descriptor writes and recovery are
atomic and idempotent.

## Model Used

OpenAI Codex with GPT-5. The run used agentic reasoning, repository
inspection, code execution, test execution, Git, and GitHub CLI tools.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-05 13:00:57 -05:00
Nicky Leach 14da75dfc7 feat(codex-local): outbound auth copy-back as home-asset restore contribution (#9788) 2026-07-18 09:13:45 -05:00