Jannik/PaperClipAI
1
0 Fork 0
mirror of https://github.com/paperclipai/paperclip.git synced 2026-10-22 00:30:19 +02:00
Code Issues Packages Projects Releases Wiki Activity
Files
23b2ceb0d19d4dff0e34f0d957aa41efec739816
PaperClipAI/maps/feature-map/execution-environments.md
T
DottaandPaperclip 1a79c8bdc7 fix(runtime): honor provider startup budgets on sandbox resume (#16802) 
## Thinking Path

> - Paperclip manages AI agents and their execution environments.
> - Follow-up runs reuse saved sandbox computers to keep their
workspace.
> - Starting a stopped or archived computer can take more than one
minute.
> - Acquisition already uses the provider's startup budget, but reuse
imposed a separate one-minute deadline.
> - This pull request gives resume the configured or declared provider
startup budget.
> - The benefit is that a healthy slow start can continue on the same
computer.

## Linked Issues or Issue Description

**What happened?**

A follow-up run failed during setup with `ReusableSandboxResumeError`.
The saved computer became ready just after the host stopped waiting. The
environment had no explicit timeout. Its provider declared a five-minute
startup default, which the resume path ignored.

**Expected behavior**

Resume must honor the configured startup budget or the provider's
declared default. It must retain the same lease. Failed or uncertain
resumes must keep the saved computer.

**Steps to reproduce**

1. Use a reusable sandbox provider with a startup budget longer than one
minute.
2. Retain its lease and start a follow-up run.
3. Make provider resume finish after 61 seconds, within that budget.
4. Observe that the old host rejects at 60 seconds.

**Paperclip version or commit**

Reproduced on `aaeff418cf72341c674ff3393cd94f40dc4855f6`.

**Deployment mode**

Hosted sandbox execution. The regression uses a local isolated
PostgreSQL database and controlled worker RPC timers.

Searched open and closed PRs and open issues for sandbox resume and
timeout fixes. No duplicate fix was found. Related acquisition timeout
handling already exists on the base branch.

## What Changed

- Use the provider's declared acquisition timeout when no explicit
startup timeout exists.
- Apply the resolved RPC budget to the shared resume/retry deadline
instead of 60 seconds.
- Add two deterministic 61-second resume regressions for declared and
explicit startup budgets. Assert exact RPC budgets, including the
30-second margin, and check that the same lease remains active without
replacement.
- Document the timeout behavior, provider-author contract, and
verification recipe.

## Verification

- Red: both new cases failed before the production change with
`ReusableSandboxResumeError`, caused by `Plugin RPC timed out`.
- Green: `pnpm exec vitest run --fsModuleCache
server/src/__tests__/environment-runtime.test.ts` passed all 163 cases.
The final review follow-up also passed both slow-resume cases with exact
RPC-budget assertions (330 seconds by declaration; 150 seconds by
explicit config).
- An earlier full run passed the 161 existing cases but timed out in the
two new tests because their fake clock also captured database
scheduling. The corrected test restores real timers before lease
persistence. The complete rerun then passed.
- `node scripts/check-maps.mjs` passed.
- `pnpm -r typecheck` and `pnpm build` passed locally.
- Started `pnpm test:run`, then stopped the duplicate serial run in
favor of complete parallel PR CI. The repository timing data totals
about 126 minutes for server suites alone. Full parallel CI passed on
the final head.
- CI attempt 1 had one unrelated failure:
`FileViewerSheet.copy.test.tsx` expected the asynchronous `Copied link`
confirmation before React rendered it. That test and component are
unchanged from the base branch. All six cases passed unchanged locally.
A single rerun of the failed shard passed; attempt 2 of the full
workflow succeeded. The timing-race diagnosis is an inference from the
test helper, not a claim that this PR repairs that UI test.
- Greptile scored the final head 5/5. Both review findings are fixed and
both threads are resolved.
- The consequential assumption is that provider startup may exceed one
minute. The test preserves the RPC timeout race with controlled timers
and uses real lease persistence. Read-only provider evidence also showed
the retained computer ready just after the failed run ended. The patch
has not been deployed to that instance.

## Risks

- A slow or unavailable provider may hold setup for its configured
startup budget instead of one minute. Retries still share one finite
deadline. Providers with no explicit or declared timeout keep the
one-minute default.
- Lease identity, company scope, backup validation, and replacement
decisions use the existing paths. No schema or dependency changes are
required.

## Model Used

- OpenAI Codex, GPT-6. This session supports reasoning, tool use, code
editing, and test execution. The exact runtime variant and
context-window size are not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-18 22:04:23 -05:00

125 lines
8.6 KiB
Markdown
Raw Blame History

---
navigation: {"section":"Projects and execution","sectionOrder":40,"order":40}
---
# Execution environments
Operators configure where agents execute, probe readiness, select an environment for work, and inspect remote provisioning or custom-image state when the installed provider supports it.
Implementation: [environment settings](../../ui/src/pages/CompanyEnvironments.tsx), [environment runtime](../../server/src/services/environment-runtime.ts).
Agent access: [companion](../agent-access-map/execution-environments.md).
## Sub-features
- `configuration`: inspect local environments and configure available SSH/sandbox targets.
- `capabilities`: select only drivers/providers that advertise the required execution capability.
- `probe`: test reachability and runtime prerequisites in the actual target.
- `assignment`: bind the intended agent/project/workspace to its execution target.
- `images`: inspect supported sandbox custom-image preparation and terminal workflows.
## How to get to it (user POV)
### `environment-settings`
Open Settings → Environments, then create/edit a supported target.
The canonical URL is `/company/settings/instance/environments`. Enable
`enableEnvironments` first; keep `enableManagedSandboxOnly` off to inspect local
targets. The catalog is instance-scoped and mutation requires a local operator
or instance administrator.
### `remote-execution`
Choose the environment in agent/project/workspace configuration, then inspect the resulting task/run and workspace.
## Driving it with Playwright and the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use an owned disposable target and a supported provider plugin. Record provider, target identity, account, and spending constraints before a live remote run.
### `environment-settings`
Automated: [environment UI](../../ui/src/pages/CompanyEnvironments.test.tsx), [driver configuration UI](../../ui/src/pages/CompanySettings.test.tsx), and [environment service](../../server/src/__tests__/environment-service.test.ts) cover configuration behavior.
Manual: Save a disposable target, reload it, and run its available probe. Verify both success and missing-credential/reachability errors identify the target. Confirm unavailable providers are not offered as working run targets.
For an immutable sandbox image, pin the OCI image with `@sha256` and explicitly
probe the saved environment in the intended company. The existing connection
probe allocates a fresh lease and can measure its OS, architecture and libc.
Qualification is recorded only when the selected image/configuration matches
the live lease and cleanup is confirmed. Inspect `platformQualificationRecorded`
and its diagnostic in the probe result: connection success alone does not mean
the image has native runtime qualification.
The measured record is server-owned, company-bound and tied to the effective
configuration and active custom-image identity. Mutable tags, provider defaults
and name-only snapshots do not qualify. Changing the target or image invalidates
the record; normal metadata edits cannot forge or overwrite it, and API reads
omit the private metadata. Reading this evidence for discovery or creation does
not allocate a sandbox. The draft `probe-config` checks are not a saved-image
qualification operation. See the [runtime contract](../../packages/paperclip-runner/README.md)
and [adapter validation recipe](./runs-adapters.md#adapter-setup) for native
binary/provider readiness after platform qualification.
Verification recipe: probe a pinned owned image, record its company/config/image
identity and cleanup result, then change the image or target and verify its old
qualification is not reused. A mutable-image connection can succeed while
qualification remains absent. Assert ordinary agent and non-admin board probe
requests are denied, and environment create/PATCH payloads cannot supply
qualification metadata.
This recipe and its [probe controls](../../server/src/__tests__/environment-probe.test.ts)
are not claims that a live provider or agent-access journey ran for this map update.
The local row has no UI Test button. Its API/CLI probe reports host/cwd
availability only; it does not validate an adapter binary, workspace, or task
execution. Remote-driver probes and a real bounded run need their own evidence.
### `remote-execution`
Automated: [execution target](../../server/src/__tests__/environment-execution-target.test.ts) and [custom images](../../server/src/__tests__/environment-custom-images-service.test.ts) cover contracts; [SSH live test](../../server/src/__tests__/environment-live-ssh.test.ts) requires its own external setup.
The [environment runtime regression](../../server/src/__tests__/environment-runtime.test.ts)
also covers sandbox reuse after a start longer than one minute. Acquisition
uses the configured or provider-declared startup budget for resume and shares
one deadline across retries. Failure preserves the recorded sandbox. This
controlled timing check does not establish live provider acceptance.
Manual: Run a bounded command in the intended target, inspect its output and workspace, and verify execution did not silently fall back to the server host. For custom-image changes, build/probe the disposable image before assigning it. Inspect cleanup and shutdown after the run.
## Gotchas
- A probe is narrower evidence than a complete task with tools, files, and resumption.
- Platform qualification does not authenticate a model account or qualify every
harness; the selected native runtime still needs its own setup and useful-task proof.
- The local option may appear only when editing an existing local environment; availability follows current UI/provider capabilities.
- Workspace service reachability and environment reachability are separate checks.
### Storybook verification
#### `configuration`
Scope: ui
Storybook: Local/SSH/sandbox editors; unavailable providers, missing credentials, save pending/error and restricted operator.
Production: Persist target configuration and enforce instance-admin/local-operator mutation boundary; target catalog is instance-scoped.
Sources: Source: [CompanyEnvironments.tsx](../../ui/src/pages/CompanyEnvironments.tsx), [environments.ts](../../server/src/routes/environments.ts). Tests: [CompanyEnvironments.test.tsx](../../ui/src/pages/CompanyEnvironments.test.tsx), [environment-service.test.ts](../../server/src/__tests__/environment-service.test.ts)
#### `capabilities`
Scope: ui
Storybook: Supported versus unavailable driver/provider controls and interactive-setup/template-capture limitations.
Production: Verify advertised driver/plugin capabilities match operations; unavailable provider cannot become working target; capability discovery must not allocate sandbox.
Sources: Source: [CompanyEnvironments.tsx](../../ui/src/pages/CompanyEnvironments.tsx), [environments.ts](../../ui/src/api/environments.ts). Tests: [environment-capability-contract.test.ts](../../server/src/__tests__/environment-capability-contract.test.ts), [environment-execution-target-capabilities.test.ts](../../server/src/__tests__/environment-execution-target-capabilities.test.ts), [plugin-environment-driver-sandbox-capabilities.test.ts](../../server/src/__tests__/plugin-environment-driver-sandbox-capabilities.test.ts)
#### `probe`
Scope: ui
Storybook: Remote Test pending, success, reachability/auth failure and platform qualification absent/invalidated notice.
Production: Probe actual owned target; verify selected saved immutable config/image/company qualification plus lease cleanup; local API probe proves host/cwd only and has no UI Test button.
Sources: Source: [CompanyEnvironments.tsx](../../ui/src/pages/CompanyEnvironments.tsx), [environment-probe.ts](../../server/src/services/environment-probe.ts). Tests: [CompanyEnvironments.test.tsx](../../ui/src/pages/CompanyEnvironments.test.tsx), [environment-probe.test.ts](../../server/src/__tests__/environment-probe.test.ts), [environment-live-ssh.test.ts](../../server/src/__tests__/environment-live-ssh.test.ts)
#### `images`
Scope: ui
Storybook: Custom-image setup/capture/progress, terminal connect/disconnect/failure, unsupported capability and active-image drift.
Production: Execute owned provider setup/capture/probe/terminal transport with correct saved image identity, authority, cleanup and shutdown; terminal screenshot cannot prove operation.
Sources: Source: [CompanyEnvironments.tsx](../../ui/src/pages/CompanyEnvironments.tsx), [environment-custom-images.ts](../../server/src/services/environment-custom-images.ts). Tests: [CompanyEnvironments.test.tsx](../../ui/src/pages/CompanyEnvironments.test.tsx), [environment-custom-images-service.test.ts](../../server/src/__tests__/environment-custom-images-service.test.ts)
Reference in new issue View Git Blame Copy Permalink