DottaandPaperclip
60b8089a44
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Cloud selects and updates the sandbox image used by its agents. > - Runner admission also required saved qualification for the exact snapshot and configuration. > - An image update left the old record stale and stopped work before runtime startup. > - This pull request trusts the image selected by Cloud and uses the real runtime startup check. > - Agents can continue after an image update without an operator running a separate qualification test. ## Linked Issues or Issue Description **What happened?** A managed Cloud image update could cause `sandbox_snapshot_qualification_required` before provider work began. Runner discovery also fell back to Legacy when saved platform qualification was absent. The agent Test action could require a separate qualification request even when discovery already offered Paperclip Runner. **Expected behavior** A managed Cloud image can start without a saved qualification record. Runtime startup and the handshake report actual compatibility failures. Custom images keep their existing qualification rules. **Steps to reproduce** 1. Use Paperclip Runner on a Paperclip-managed Cloud sandbox. 2. Save qualification for its current snapshot. 3. Update the platform-selected snapshot and leave the old record in place. 4. Start a task. Before this fix, admission rejects the stale record before starting the provider. Related work: Refs #17445, Refs #17724, Refs #17448. These changes cover saved-computer qualification and setup tests. This fix removes the qualification prerequisite only for managed Cloud images. ## What Changed - Use one managed-image policy in runner selection and lease admission. Require the active Cloud configuration to provision the provider. Active captured custom images are excluded. - Use the current Cloud image and ignore missing, stale, revoked or valid saved qualification records for that image. - Reuse the existing provisioned-runner selection path. Keep adapter availability checks and the Legacy choice for managed sandboxes. - Read custom-image metadata directly during discovery without resolving provider credentials. - Use the existing template-matching rule for captured images. Ignore outdated captures, preserve qualification for matching custom captures, and fall back to Legacy discovery for malformed saved capture config. - Skip the separate qualification request when discovery already offers Paperclip Runner as the default. Test checks the actual runtime. - Add regressions for image updates, provider failures, custom images, self-hosted environments and company access. Update the development and verification guides. ## Verification - Before the fix, managed-image discovery regressions returned Legacy, and the rendered Test action called the separate qualification endpoint. - The original runtime source fails four of the five managed-image regression cases. The fixed runtime passes all five, including provider startup error propagation. The baseline check uses the original module source through a temporary Vitest transform and the same database-backed tests. - All 603 focused tests pass: 231 runner selection, 185 environment runtime, 10 snapshot qualification, 168 rendered agent form and nine form source controls. Discovery controls cover unavailable credentials, stale managed markers without an active provisioner, outdated captures, matching custom captures and malformed saved config. The outdated-capture regression fails before the fix and passes afterward. - `node scripts/check-maps.mjs` and `pnpm check:token-gates` pass. - Full `pnpm -r typecheck` and `pnpm build` pass. All five focused files passed together on `8275bbe72bb4d91fb3a02fb0658f0846b2abc966` (`codex/simplify-managed-runner-readiness`, Node 24.19.0, macOS ARM64). - The full local `pnpm test:run` suite is still in progress. Hosted CI is waiting at `ci / gate / Select trusted runner`; the separate `review` workflow is queued. Greptile is 5/5 on the current head, and both review threads are resolved. This PR is not yet verified by full CI. - Live fixed-source Daytona acceptance is not verified. This workspace has no Daytona API key or image configuration. The earlier deployed recovery is not evidence for this source change. ## Risks - Managed Cloud now trusts its selected image. A bad image can fail at real runtime startup instead of a saved qualification gate. - Company access, credentials, runtime integrity and handshake checks still apply. Self-hosted sandboxes, custom environments and active captured custom images keep their qualification and immutable snapshot binding. - No schema change or data migration is required. Existing qualification records can remain in place. ## Model Used - OpenAI Codex, GPT-6 family, with reasoning, tool use and code execution. The exact model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
13 KiB
navigation
| navigation | ||||||
|---|---|---|---|---|---|---|
|
Execution environments
Operators configure where agents execute, probe readiness, select an environment for work, and inspect remote provisioning or custom-image state when the installed provider supports it.
Feature ID: execution-environments.
Implementation: environment settings, environment runtime.
Agent access: companion.
Sub-features
configuration:execution-environments/configuration— inspect local environments and configure available SSH/sandbox targets.capabilities:execution-environments/capabilities— select only drivers/providers that advertise the required execution capability.probe:execution-environments/probe— test reachability and runtime prerequisites in the actual target.assignment:execution-environments/assignment— bind the intended agent/project/workspace to its execution target.images:execution-environments/images— inspect supported sandbox custom-image preparation and terminal workflows.
How to get to it (user POV)
environment-settings
Open Settings → Environments, then create/edit a supported target.
The canonical URL is /company/settings/instance/environments. Enable
enableEnvironments first; keep enableManagedSandboxOnly off to inspect local
targets. The catalog is instance-scoped and mutation requires a local operator
or instance administrator.
remote-execution
Choose the environment in agent/project/workspace configuration, then inspect the resulting task/run and workspace.
Driving it with Playwright and the Paperclip CLI
Preconditions: follow the baseline. Use an owned disposable target and a supported provider plugin. Record provider, target identity, account, and spending constraints before a live remote run.
environment-settings
Automated: environment UI, driver configuration UI, and environment service cover configuration behavior.
Manual: Save a disposable target, reload it, and run its available probe. Verify both success and missing-credential/reachability errors identify the target. Confirm unavailable providers are not offered as working run targets.
For an immutable sandbox image, pin the OCI image with @sha256, or select
a Daytona snapshot whose provider ID can be verified, and explicitly probe the saved environment in the intended company. The existing connection
probe allocates a fresh lease and can measure its OS, architecture and libc.
Qualification is recorded only when the selected image/configuration matches
the live lease and cleanup is confirmed. Inspect platformQualificationRecorded
and its diagnostic in the probe result: connection success alone does not mean
the image has native runtime qualification.
Paperclip-managed Cloud images use the current platform-selected image without a saved qualification prerequisite. A missing, stale or revoked qualification does not block execution or pin an older snapshot. Runtime startup and the handshake detect real incompatibility. This exception excludes custom environments and active captured custom images. The active Cloud configuration must provision the sandbox provider. A stale managed marker alone keeps the normal qualification rules. An outdated captured image that execution ignores must not block discovery of the current managed image. A matching custom capture keeps qualification.
For other images, the measured record is server-owned, company-bound and tied to the effective
configuration and active custom-image identity. Mutable tags, provider defaults
and snapshots without verified provider identities do not qualify. Daytona
qualification creates by provider ID and binds native execution to that same ID;
a deleted ID fails instead of resolving a replacement name. Changing the target,
account, image or active captured template invalidates
the record; normal metadata edits cannot forge or overwrite it, and API reads
omit the private metadata. Reading this evidence for discovery or creation does
not allocate a sandbox. The draft probe-config checks are not a saved-image
qualification operation. See the runtime contract
and adapter validation recipe for native
binary/provider readiness after platform qualification.
Verification recipe: probe a pinned owned image, record its company/config/image identity and cleanup result, then change the image or target and verify its old qualification is not reused. A mutable-image connection can succeed while qualification remains absent. Assert ordinary agent and non-admin board probe requests are denied, and environment create/PATCH payloads cannot supply qualification metadata. For a managed Cloud image, replace the platform-selected snapshot while an old qualification remains. Discovery must still offer Paperclip Runner, and Test must check the runtime directly. Verify execution selects the replacement image and still reports a real provider startup failure. Keep a custom-image control that requires its existing qualification. This recipe and its probe controls are not claims that a live provider or agent-access journey ran for this map update.
The local row has no UI Test button. Its API/CLI probe reports host/cwd availability only; it does not validate an adapter binary, workspace, or task execution. Remote-driver probes and a real bounded run need their own evidence.
remote-execution
For paperclip_runner, a saved plugin environment can provide per-turn Runner
execution when its manifest and worker expose the Runner task API. The task
contract starts a provider-owned Runner without shell or workspace hooks. Core
resolves the saved lease, connects through PRP, and verifies stop before lease
cleanup. The provider owns the installed agent home and retained session files;
data transfer is outside task execution. See the
plugin contract.
The fake-provider integration
checks acquisition, SDK dispatch, scope denial, stop proof and deletion recovery.
Its WebSocket is mocked; it does not qualify a live provider or a real Runner handshake.
Controller replacement reuses a surviving plugin task's active lease and exact
Runner identity. The integration checks reconnect without prepare, submit, or
stop. The continuity recipe
also covers real-process recovery after controller files are removed; deployed
plugin and Daytona acceptance require their own evidence.
For a same-run retry on a replacement plugin task, verify that the retained checkpoint loads before provider startup. The prior execution must have a stop receipt, and the checkpoint must match the original session identity. Include a turn that ends without a completion result; a successful initial turn alone does not exercise this recovery path. Reject missing stop proof and changed identities.
Automated: execution target and custom images cover contracts; SSH live test requires its own external setup.
The environment runtime regression also covers sandbox reuse after a start longer than one minute. Acquisition uses the configured or provider-declared startup budget for resume and shares one deadline across retries. Failure preserves the recorded sandbox. This controlled timing check does not establish live provider acceptance.
Manual: Run a bounded command in the intended target, inspect its output and workspace, and verify execution did not silently fall back to the server host. For custom-image changes, build/probe the disposable image before assigning it. Inspect cleanup and shutdown after the run.
Gotchas
- A probe is narrower evidence than a complete task with tools, files, and resumption.
- Platform qualification does not authenticate a model account or qualify every harness; the selected native runtime still needs its own setup and useful-task proof.
- The local option may appear only when editing an existing local environment; availability follows current UI/provider capabilities.
- Workspace service reachability and environment reachability are separate checks.
The task-only setup path uses a Core-issued expiring probe identity and a real, read-only Runner hello. The plugin checks project authentication before provider startup. Setup probe coverage uses the real Runner and a synthetic Codex provider through loopback PRP; TLS termination is replaced by the fixture. It does not qualify a live plugin provider.
The plugin prepares execution-host resources and the provider descriptor. Core validates the result, then supplies task identity and PRP launch limits.
Storybook verification
configuration
Capability ID: execution-environments/configuration.
Scope: ui Storybook: Local/SSH/sandbox editors; unavailable providers, missing credentials, save pending/error and restricted operator. Production: Persist target configuration and enforce instance-admin/local-operator mutation boundary; target catalog is instance-scoped. Sources: Source: CompanyEnvironments.tsx, environments.ts. Tests: CompanyEnvironments.test.tsx, environment-service.test.ts
capabilities
Capability ID: execution-environments/capabilities.
Scope: ui Storybook: Supported versus unavailable driver/provider controls and interactive-setup/template-capture limitations. Production: Verify advertised driver/plugin capabilities match operations; unavailable provider cannot become working target; capability discovery must not allocate sandbox. Sources: Source: CompanyEnvironments.tsx, environments.ts. Tests: environment-capability-contract.test.ts, environment-execution-target-capabilities.test.ts, plugin-environment-driver-sandbox-capabilities.test.ts
probe
Capability ID: execution-environments/probe.
Scope: ui Storybook: Remote Test pending, success, reachability/auth failure and platform qualification absent/invalidated notice. Production: Probe actual owned target; verify selected saved immutable config/image/company qualification plus lease cleanup; local API probe proves host/cwd only and has no UI Test button. Sources: Source: CompanyEnvironments.tsx, environment-probe.ts. Tests: CompanyEnvironments.test.tsx, environment-probe.test.ts, environment-live-ssh.test.ts
images
Capability ID: execution-environments/images.
Scope: ui Storybook: Custom-image setup/capture/progress, terminal connect/disconnect/failure, unsupported capability and active-image drift. Production: Execute owned provider setup/capture/probe/terminal transport with correct saved image identity, authority, cleanup and shutdown; terminal screenshot cannot prove operation. Sources: Source: CompanyEnvironments.tsx, environment-custom-images.ts. Tests: CompanyEnvironments.test.tsx, environment-custom-images-service.test.ts
Managed account transfer requires projectAuthCheckVersion: 1 and a successful
check_project_auth result for the exact prepared configuration, provider, and
project set. New preparation removes old validation. Core checks the saved
validation again in the dispatch update.