## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its web server also owns controller admission and user commands. > - A rolling deployment starts a replacement before the old server exits. > - Shared database state alone does not preserve local agent files, browser chunks, or in-process commands. > - This pull request adds an opt-in ephemeral server contract and a local overlap test. > - The benefit is a web server that can relinquish traffic and local files without losing acknowledged work. ## Linked Issues or Issue Description Refs #18403, #18327, #18335, #18341. **Current behavior** The control-plane web server still has local agent-file authority and local command targets. A new container can miss saved bytes or send a command to the wrong process. An existing browser tab also needs chunks from its original build. **Proposed behavior** Eligible servers share only PostgreSQL, object storage and stable secrets. A replacement waits for readiness. The old server drains admitted requests, keeps owner commands available through handoff, and releases controller leases. Explicit maintenance adopts existing files before the volume is removed. **Reason and benefit** This is the application prerequisite for rolling app-only deployments. Schema migrations and permanent multi-replica operation are outside this change. ## What Changed - Store agent-file manifests in PostgreSQL and immutable bytes in S3. Treat local roots as disposable projections. Preserve existing permission checks and edit conflicts. - Add an explicit migration, eligibility and object-retention command to the production runtime bundle. - Add `/api/ready`, admitted-request drain, browser restart signaling and retained immutable UI chunks. - Carry transaction-aware work hints across controllers during handoff and active execution. Keep listeners closed when idle. - Deliver durable Stop and runtime responses to the execution owner. Recheck request lifecycle and access before response dispatch. - Reject local-only configuration in ephemeral mode. Keep the feature disabled for existing installations by default. - Add an isolated PostgreSQL/S3/proxy/browser deployment harness with independent server homes and retained failure evidence. ## Verification - All CI checks pass on `ce17b95019d3cf1cabd357d19b6d37f96782cc5c`: workspace typecheck, build, Runner checks, server/general tests and browser shards. Greptile reviewed the runtime changes at 5/5 with all findings resolved; the final commit updates maps and the plan. - Local targeted checks passed: coordinator 27; Stop/recovery 33; PostgreSQL command delivery 6; route/workspace regressions 8; bootstrap Stop 1; restart recovery 75. PostgreSQL command tests simulate the provider edge. - Local browser proof passed 91 reads and 19 saves; reconnect took 1,013ms. The tab opened Audit from S3 after deletion of A's whole home/build. This uses compiled workspace code and local-trusted auth, not released containers or live agents. - Local workspace typecheck/build passed; current server and runtime bundle passed. The earlier broad local suite was stopped after three hours and is not a passing receipt. Its retained failures and isolated reruns are documented in the plan. - Stacked #18526 adds stricter graceful-exit proof, an unready replacement control, live Runner qualification and a Stop handoff correction. Hosted enablement remains gated on those results and Cloud #1096. No hosted stack has been enabled. ## Risks - Apply migration 0422 before enabling the feature. Existing data must pass each owner's migration and eligibility checks. Do not discard a volume based on a single migration page. - Durable manifests remain authoritative after the opt-in is disabled. Rolling back to a binary that ignores them is unsupported. - Cold directory restoration and complete shutdown timing still need qualification before enabling a hosted stack. - Retained UI assets must cover the supported browser and rollback window. Cleanup is explicit; no new idle maintenance polling is introduced. - This change does not claim production zero downtime or authorize a fleet-wide enablement. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model ID and context-window limit are not exposed in this session. Used reasoning, code editing, shell execution and browser automation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
13 KiB
Installation, configuration, updates, and backups — agent access
Product map: Installation, configuration, updates, and backups.
Feature ID: instance-operations.
Review date: 2026-10-12. Source commit: f981fa3cc10cb8663a87dec60345fbff46d54d0a.
This is a source-reviewed baseline of representative operations and known gaps. Each capability needs operation-level review before its entire scope can be qualified. No live agent execution is claimed. Follow the map contract for format, permission checks, evidence and maintenance.
install-update
Capability ID: instance-operations/install-update.
Review: baseline.
API: not-applicable — No uniform company REST updater API. Treat pre-start
installation as an operator exception or design an explicit delegated operator API.
Agent access: operator-only — Existing CLI install/update/rollback commands.
Policy: Local machine operator and instance-administrator authority; distinct from company-scoped agent keys.
Evidence: source-only — implementation at the reviewed
commit above.
Verification: not-run — exercise the listed local workflow in an authorized, owned
operator environment. Verify its isolation and result; do not count local machine
authority as company-agent API access.
Gaps: No uniform company REST updater API. Treat pre-start installation as an operator exception or design an explicit delegated operator API.
server-service
Capability ID: instance-operations/server-service.
Review: baseline.
API: present — representative operation IDs:
GET /api/health.
Agent access: operator-only — Existing CLI service/dev controls; health REST; owned
workspace services have separate agent APIs.
Policy: Local machine operator and instance-administrator authority; distinct from company-scoped agent keys.
Evidence: source-only — implementation at the reviewed
commit above.
Verification: not-run — exercise the listed local workflow in an authorized, owned
operator environment. Verify its isolation and result; do not count local machine
authority as company-agent API access.
Gaps: Instance start/stop and task workspace start/stop are different capabilities. Local service control is not agent company authority.
Historical Cloud sandbox cleanup remains server-owned. Cleanup-only absence reads authenticate the tenant host with its server capability; they expose no new board or agent endpoint. Their signed scope cannot authorize task recovery. Runtime and idle-gate regressions cover reconciliation, while receipt checks cover signature, identity, freshness and separation from recovery authority. These checks do not establish a deployed sleep/wake result.
configuration
Capability ID: instance-operations/configuration.
Review: baseline.
API: present — representative operation IDs:
GET /api/instance/settings.PATCH /api/instance/settings.GET /api/instance/settings/experimental.PATCH /api/instance/settings/experimental.GET /api/adapters.GET /api/adapters/{type}.PATCH /api/adapters/{type}.
Agent access: restricted — Board/admin instance settings; CLI configure/env; local
configuration files.
Policy: Local machine operator and instance-administrator authority; distinct from company-scoped agent keys.
Evidence: source-only — implementation at the reviewed
commit above.
Verification: not-run — assert the documented agent denial with an owned test fixture;
verify any listed agent request/result path separately. A delegated operation needs its
own policy and access review.
Cloud runtime configuration uses a separate private PostgreSQL singleton and adds no public read or mutation endpoint. Existing instance settings APIs do not expose it. Bootstrap database credentials and trust keys remain in the supplied environment; the strict saved allowlist excludes API keys and connection URLs. Internal configuration updates require a matching revision and transaction lock. Database controls cover seed/restart, credential exclusion, conflicts and rollback. These are local controls, not agent authorization or deployed Cloud qualification.
Gaps: Settings APIs cover parts of configuration; pre-start database/binding/storage configuration is local operator tooling.
Adapter toggles require instance-admin authority and remain blocked when
instance.adapters is hidden. Ordinary board members can read the catalog;
agent keys cannot toggle adapters. Managed Cloud preferences are stored in a
private database singleton. The operator adapter declaration sets the effective
list and rejects conflicting toggles. PostgreSQL regression coverage
exercises persistence, concurrent writers, admission freshness, activity audit,
and failed writes; authorization tests
cover admin and hidden-setting gates. These are local tests, not deployed proof.
Cloud configuration migration and package qualification are operator commands, not agent-accessible REST capabilities. They require the managed environment and database operator access. The command composes transaction-safe owner seeds and writes a private receipt. Every qualification rereads current revisions and image package evidence; a completed historical receipt grants no deployment or execution authority. PostgreSQL coverage uses independent clients, lost-home reconstruction, a real replacement fixture worker, rollback/retry, and durable blockers. Reader coverage checks bounded sources and pnpm-style link integrity. These are local fixtures; see the scope and operator procedure.
Cloud stack Admins may update experimental settings through the trusted Cloud
actor without receiving other instance-admin powers. This uses the current
stack role on each request; company membership alone does not grant it.
Members, Support users, agents, and ordinary self-hosted sessions remain denied.
Cloud staging exposes experimental controls to instance admins and stack Admins.
It permits saved overrides of Cloud-disabled experimental defaults. Cloud-enabled
features remain true on every managed stack, including staging, and the settings
service rejects attempts to disable them with 403 before saving.
This exception requires a managed instance
and a provisioner-supplied *.staging.paperclip.app primary hostname; request
headers do not establish staging. Ordinary board members and agents still cannot
mutate instance settings. Only an explicit feature allowlist is editable on staging. Cloud
owner-admin elevation and managed-sandbox execution policy remain managed, and
production policy and other Cloud floors remain.
Local route tests
cover stack Admin experiment access and auditing, denial for other instance
controls and actor sources, hidden-setting restrictions, and a spoofed request hostname;
the Cloud actor tests
cover role changes and preserved instance-admin isolation;
no live Cloud or agent-credential verification is claimed.
diagnostics
Capability ID: instance-operations/diagnostics.
Review: baseline.
API: present — representative operation IDs:
GET /api/health.GET /api/instance/task-drain?upgradeReadiness=1— instance-admin or signed Cloud control only. Returns aggregate restart eligibility, never checkpoint contents, task identities or credentials. Company members and agent keys are denied. The explicit request performs bounded read-only SQL; ordinary drain status remains an in-memory read. Route coverage and PostgreSQL coverage establish this local boundary; hosted active-run continuity requires separate proof.GET /api/readyandHEAD /api/ready— public deployment readiness only; 200 after recovery, 503 while starting, recovering, or draining. No SQL, session resolution, company identity, or control authority.GET /api/livenessandHEAD /api/liveness— public process startup status and build commit only; no database readiness or workspace identity.
Liveness is anonymous and runs before actor/session resolution. Hostname and signed-bootstrap endpoint validation still apply. It grants no instance control authority. Liveness tests cover idle drain, invalid ambient credentials, unavailable SQL, and startup recovery with local Express fixtures; no live provider or agent access is claimed.
Agent access: partial — REST health plus local CLI doctor.
Policy: Local machine operator and instance-administrator authority; distinct from company-scoped agent keys.
Evidence: source-only — implementation at the reviewed
commit above.
Verification: not-run — exercise a listed permitted operation with actual agent
credentials/run context; inspect its result and applicable persistence/attribution.
Assert the relevant denied actor, owner, company or mode from Policy. Review the
remaining operations before qualifying the whole capability.
Gaps: Health is an API; full local doctor/repair inspects host configuration and is not equivalent to that response.
backup
Capability ID: instance-operations/backup.
Review: baseline.
API: present — representative operation IDs:
POST /api/instance/database-backups.
Agent access: restricted — Board/admin backup API; local CLI db:backup; restore into
an authorized separate target.
Policy: Local machine operator and instance-administrator authority; distinct from company-scoped agent keys.
Evidence: source-only — implementation at the reviewed
commit above.
Verification: not-run — assert the documented agent denial with an owned test fixture;
verify any listed agent request/result path separately. A delegated operation needs its
own policy and access review.
Gaps: API creation exists. Restore/lifecycle are operator workflows rather than ordinary company agent capabilities.
runtime-setup
Capability ID: instance-operations/runtime-setup.
Review: baseline.
API: not-applicable — these commands provision an operator-owned published package and local filesystem cache. They have no company-domain REST equivalent and deliberately bypass instance config, environment, and credential initialization.
Agent access: operator-only — explicit host/package provisioning through the runtime CLI, with the installed release’s verified assets. A company agent grant does not authorize another host’s installation or private operator cache.
Policy: Host setup verifies public pinned assets without model calls and requires the published server package layout/identity. Import requires a trusted manifest digest, exact installed source/profile identity, Linux x64 companion inventory/pack/ELF integrity, and a private operator-owned cache. Import performs no downloads or executable probes. A Pi companion does not enable remote Cursor or qualify credentials.
Evidence: source-only — reviewed at 9ef434fe79c57d976b8b03b14c5242db270b8320. Runtime CLI, instance-config bypass, Pi provisioning, Cursor provisioning, and companion importer.
Verification: not-run — Inspect setup identity/path and repeat reuse on an owned disposable published installation. Verify a matching trusted companion import and reject altered/cross-release/linked inventory while preserving unrelated files. Check no instance credentials are loaded and no model is called. Qualify actual adapter/account task execution separately.
Gaps: Source-checkout setup, different platforms, remote Cursor, provider execution, and published-package acceptance remain separate scopes. Local fixture materialization does not establish remote model readiness.
Antigravity addition (2026-10-28): paperclipai runtime setup antigravity stages
the verified official and patched two-executable closures on macOS ARM64. The
operator-only authority above is unchanged. The native profile remains pending
at API, driver, sidecar and Rust admission; a company grant cannot enable it.
Google AI API-key binding is distinct from Workspace OAuth. Evidence remains
source-only for agent access: setup,
provisioner,
and gate regressions.
Credential-free public-layout provisioning and controlled native tests do not
qualify a company agent task or authenticated connection/revocation.