Files
PaperClipAI/doc/project-environment-compatibility.md
T
Nicky LeachandPaperclip f30ac226d6 Keep project environment mappings independent of shared defaults (#20651)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - A project supplies work context and repository storage to task runs.
> - Creation selected a storage owner but did not retain the selected
environment on the project.
> - Configuration stored an environment inside checkout policy, but
policy reads and run selection dropped it.
> - Projects therefore continued to follow shared agent and instance
defaults.
> - This pull request saves placement on each project and uses it for
new runs.
> - The benefit is an independent Project to Environment association
with the existing storage, trust, and recovery safeguards.

## Linked Issues or Issue Description

Refs #20459 and #11644. Follow-up to #20587.

The reported journey is Project > Configuration: changing one project's
environment must not change another project's mapping or shared
defaults.

Related open PR #18176 repairs placement through the legacy
workspace-policy field. This follow-up introduces a separate project
association, saves creation defaults, and migrates saved legacy
references. It overlaps that PR's placement, permission, and recovery
safeguards. The changes should be reviewed together. #9588 concerns
environment-variable response privacy and does not address placement.
ROADMAP.md has no separate project-association work planned.

## What Changed

- Add nullable `projects.environment_id` and expose
`Project.environmentId`. Save the effective creation default once on the
project. Keep environment placement separate from checkout/isolation
policy.
- Read and write Environment directly in Project Configuration. Keep the
control available with isolated workspaces disabled. Each form reads its
project's value, even when environment choices share a query cache.
- Offer environments compatible with the current storage owner. Change
the target and storage together when the existing storage selector
attaches a core project to a plugin. Preserve retained-data restrictions
and validate the observed storage owner again on save.
- Resolve new task runs from their organizational project's target
before agent and instance defaults. File sources from another project do
not change placement. Explicit low-trust sandbox choices and enforced
sandbox policies retain precedence. External provider placement remains
separate.
- Restore admitted native runs from their owned environment lease. Later
project edits apply to new admissions. Require workspace-write
permission for environment changes and clears. Check company bindings
before runtime admission and during lease creation.
- Migrate valid saved policy references into the project column.
Normalize legacy API writes. Count and clear matching project references
when an environment is deleted. Update API and behavior documentation.
- Refresh the maintained SDK source registry for the two changed shared
files. Runtime pins are unchanged.

## Verification

- [Hosted
CI](https://github.com/paperclipai/paperclip/actions/runs/42908045146)
is green on `8b3b7d5b5aed51fc4c8e9c2e3506508bbaf9e00c`: **60 successful
checks**, three skipped gates, no failures, and `ci / verify` passed.
GitHub reports no merge conflicts. Standard Greptile scored the same
commit **5/5** with both review threads resolved.
- Source checkout on macOS arm64, Node 26.12.0, with fresh isolated
PostgreSQL. No production data or live provider calls.
- The two-project component test changes the first form with isolation
disabled and a shared query cache. The second form keeps its target.
Storage changes update only the selected project. A saved target remains
selected while the list loads, and an explicit clear remains possible.
- API integration creates two projects from the same default, patches
one, reads both, and checks that agent and instance defaults remain
unchanged. A later instance-default change does not rewrite either saved
project target. Tests cover unavailable/incompatible choices, anonymous
and cross-company denials, and storage preconditions.
- The migration test runs the actual SQL against an existing project row
and verifies its environment and disabled checkout policy survive.
Deletion accounting and clearing affect only matching project
references.
- The final guard set passes **53 tests across four files**:
`project-environments.test.ts`,
`environment-selection-route-guards.test.ts`,
`heartbeat/environment-selection.test.ts`, and
`ProjectProperties.environment.test.tsx`. All **64
environment-run-orchestrator tests** pass, including direct lease
acquisition without an extra caller argument. The broader affected set
passed **109 tests across eight files** before the additional admission
guards. These sets overlap.
- UI typecheck, migration numbering/safety, token gates, module
boundaries, query-error rendering, and diff checks pass. `pnpm verify
--explain` returns exit 2 and requires broad CI because shared
schema/runtime paths and behavior documentation changed. Broad coverage
is delegated to current-head CI.
- Early iterations failed due to a legacy parser that drops the
environment field, a fixture without real company membership, and
incorrect expectations for privacy-preserving 404 denials. The service
now reads legacy references before normalization, the API fixture has
real membership, and denial assertions match the existing privacy
contract. The orchestrator mock also needed the reused company-binding
guard in its service contract. The focused suites above pass after those
corrections.
- Local server typecheck remains blocked by installed ACPX, Slack, AI
SDK, and Runner type mismatches in unchanged dependency-facing code. It
reports no errors in the changed production paths. This is not a full
local typecheck or suite pass. Hosted CI supplies full current-head
coverage.
- SDK source-proof and privacy qualification passes **56 tests across
three files**, including the real built SDK. The maintained source
registry matches the committed revision; runtime pins are unchanged.
- Review reproduced two failures: an incoming policy without a storage
owner could be validated before the existing owner was restored, and
uppercase legacy UUIDs missed migration matching. Preserve the owner
before validation and compare normalized UUID text. Both regressions now
pass. The post-fix project/storage/orchestrator batch passes **107
tests**: `node node_modules/vitest/vitest.mjs run
server/src/__tests__/project-environments.test.ts
server/src/__tests__/project-workspace-provider.integration.test.ts
server/src/__tests__/environment-run-orchestrator.test.ts --silent`.
- The first hosted run failed build, typecheck, and canary at the same
`run-completion.test.ts` call because lease acquisition required a new
argument. The lease method now enforces binding internally. Its existing
caller compiles without a new field, and the orchestrator tests prove
the binding flag reaches the runtime. The full server check no longer
reports that error; the local dependency errors above remain.

## Risks

- Apply the normal database migration before serving the new revision.
The migration adds a nullable foreign key and copies valid legacy
project selections. It does not move repository data or resize provider
storage. The five newest schema snapshots are retained.
- New projects retain their creation target when shared defaults change.
Existing projects without a saved selection continue to inherit
defaults. A null target explicitly opts into inheritance.
- Existing saved project selections now affect placement. Missing,
archived, incompatible, and foreign-company targets fail validation;
they do not silently choose another storage owner. A running admitted
process keeps its original owned lease.
- Local dependency limitations are listed above. No live-provider,
deployed-browser, or production rollout qualification is claimed.

## Model Used

OpenAI Codex, based on GPT-6, with reasoning, tool use, and code
execution. The exact serving snapshot and context-window size are not
exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-10 14:31:48 -08:00

5.9 KiB

Project environments and repository storage

A repository project belongs to one storage owner. Creation shows Available in when the configured environments require more than one owner. Each choice names all environments that use that owner. An environment choice saves both the storage owner and the project's environmentId. This is a project-owned execution target, not a company or agent setting. Compatible environments can share its repository storage.

Compatibility rule

Use the existing workspace execution contract, not environment names, driver names alone, images, operating systems, endpoints, or complete config equality:

  • Local, SSH, and sandbox environments use core repository staging. They can use the same repository project even across driver types and sandbox providers.
  • Plugin-managed storage belongs to an installed plugin ID. It can be consumed only by that plugin's registered environment_driver with supportsTasks: true. Its manifest must declare environment.drivers.register, workspaces.manage, a workspace provider, and the project resource lifecycle.
  • Two task environments from that installed plugin share its project storage contract, even when their other settings differ. Storage operations are scoped by company, project, and plugin; they do not take an environment configuration. Separate installed storage plugins remain incompatible even when both use the plugin driver. Core cannot transfer their managed bindings between them.
  • Unknown plugin drivers and the probe-only fake sandbox do not establish a compatible project storage contract. They are excluded from creation choices.

These are project repository compatibility rules. They do not assert that arbitrary local paths exist remotely, that a retained task workspace can switch hosts, or that an adapter, credential, image, or running worker is ready. Existing run-time validation still checks those conditions. Cross-plugin storage sharing would require a new execution/storage contract; matching config fields cannot establish it.

Defaults and validation

For an agent API request, use that agent's default environment. For a person, use their primary agent's default environment when present. Fall back to the instance default, then local execution. There is no separate user or company defaultEnvironmentId setting today. Managed-only and forced Kubernetes policies retain their execution restrictions and default redirection.

GET /api/companies/:companyId/project-environments returns environment IDs and names grouped by storage owner, plus the caller's effective default. It excludes archived environments, disabled/uninstalled storage plugins, and managed environments bound only to another company. It does not contact workers or return provider configs, credentials, endpoints, or host paths. A cold worker does not change a durable storage choice.

POST /api/companies/:companyId/projects accepts an optional environmentId. The server validates the choice again and stores its storage owner and environment on that project. Creation saves the effective default once; changing the instance or primary agent later does not rewrite existing projects. Compatible environments share repository storage; the saved environment is the run target. An explicit executionWorkspacePolicy.storageProviderPluginId continues to work. The UI sends both the environment and its observed storage owner in expectedStorageProviderPluginId, so a changed environment cannot silently select different storage. This guard does not create a workspace-mode policy; instance task-isolation defaults remain inherited. An explicit policy and environment must agree when both are supplied. Without either, storage follows the caller's effective environment. A plugin's defaultForProjects claim alone no longer attaches a project to that plugin.

The hidden repository-free “My private tasks” project is independent of environment/storage availability, so creating a private note never requires repairing an environment.

An unavailable configured default produces an error and permits an explicit available choice. It does not silently select another storage owner. Existing projects keep their stored owner. This change does not migrate managed storage or detach projects that contain existing work. A managed project run in an incompatible environment fails with project_environment_incompatible before a workspace provider operation, with a message naming the storage and environment.

Project Configuration

PATCH /api/projects/:id accepts environmentId and the optional expectedStorageProviderPluginId precondition. The Environment selector reads and writes this project field, independently of checkout/isolation settings. Changing one project never changes another project, the agent, or instance settings. Only environments compatible with the project's current storage are offered. Changing storage ownership is subject to the existing retained-data safeguards.

Ordinary run selection uses the project's saved target before the agent default, instance default, and local fallback. A null project target explicitly inherits those defaults. An operator-selected low-trust task sandbox and enforced sandbox policies retain precedence. Provider API execution keeps its existing placement.

The migration preserves existing valid project-policy environment selections in the new project field. Older clients may still write the policy field; the service normalizes it into the project association. Existing projects with no selection remain unset. Environment deletion counts and clears only matching associations.

Changing or clearing placement requires project workspace-write authority. A saved selection is checked against company bindings before runtime admission and again during lease creation. Recovery uses the admitted run's owned environment lease; a later project edit affects new admissions. The task's organizational project owns placement even when its files come from another project.