Merge recorded master baseline into Pi 1 production candidate

Integrate 8ec4b84e1c before runtime qualification, preserving the new workspace restore lock, continuation and Docker packaging fixes. Pi profile 11 source and distribution inputs are unchanged by this merge.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip committed 2026-10-01 15:17:17 -05:00
commit 5eba61ece9
61 files changed
+2894 -2014

No files matched your search

+20 -2
View File
@@ -6,6 +6,9 @@ This project can run fully in local dev without setting up PostgreSQL manually.
For mode definitions and intended CLI behavior, see `doc/DEPLOYMENT-MODES.md`.
For sandbox file synchronization, lock ownership, and the required upgrade
procedure from directory locks, see [Workspace restore locks](workspace-restore-locks.md).
Current implementation status:
- canonical model: `local_trusted` and `authenticated` (with `private/public` exposure)
@@ -100,6 +103,20 @@ pnpm build-storybook
These run the `@paperclipai/ui` Storybook on port `6006` and build the static output to `ui/storybook-static/`.
Use **Components → Agent setup prompt** to review the shared setup handoff:
hover/focus logo motion, one-click copying with a prompt preview, animated
confirmation, and manual-copy recovery. Opening the preview copies immediately;
clicking its trigger again copies without closing. Close and Escape only dismiss;
the inner copy button remains available for retries and later copies. Stories include Slack and API copy, a compact quick-access placement,
light and mobile views, and a clipboard paste check. `AgentSetupPrompt` accepts
the complete `prompt`, `title`, `description`, trigger `label`, and popover
placement. Use it for prompts handed to an external agent: Slack and GitHub
connections, MCP configuration help, routine webhook setup, external-agent
invitations, and task continuation. The **App placements** story collects the
production controls; **Inside MCP help** exercises the popover inside a dialog.
Prompt generation can pass `initialCopyStatus` to preserve its automatic copy
result, and `onCopied` to clear an earlier generation-time clipboard error.
Use **Design explorations → Agent chat sidebar** to review the production secondary
agent navigation. **Components** covers selection, search, loading, empty results,
and larger teams. **Pages** puts it beside the production agent chat inside the
@@ -823,8 +840,9 @@ The default `worktree init` still seeds eagerly. A lean worktree (created withou
- `pnpm paperclipai worktree ensure-seeded` performs the deferred seed **exactly once**. It is lock-guarded and idempotent: only a complete `verified` manifest short-circuits it, so it is safe to call repeatedly and from concurrent processes. Managed workspaces derive the source from the control-plane-provided base project workspace when it carries its own `.paperclip/config.json`, and otherwise from the control plane's own registered instance config; either way the workspace's manifest never selects it. Manual worktrees must pass `--from-config`.
- `paperclipai run` calls `ensureWorktreeSeeded` automatically before doctor/boot. Managed runs transparently seed a lean worktree from their registered base workspace; an unmanaged lean worktree must first run `worktree ensure-seeded --from-config <source-config>`.
- Managed Paperclip git worktrees default to the repository's `scripts/provision-worktree.sh` when the strategy omits `provisionCommand`, so the isolated config and pending manifest cannot be silently skipped. Runtime startup also runs `scripts/provision-worktree-runtime.sh` automatically when no explicit runtime provision command is configured and the manifest is not verified. Explicitly configured provision commands still take precedence.
- If the built-in provisioner reports an unavailable seed source config, decide whether the task needs a seeded development instance or only an isolated checkout. A seeded instance needs a canonical config from the registered base workspace or control-plane instance; environment-only server configuration does not provide that file. For a checkout-only worktree, explicitly set the `git_worktree` strategy's `provisionCommand` to `"true"`. This skips setup and does not establish runtime or seed readiness. Repair rejected symlinks or non-regular source files instead of treating those validation failures as a missing prerequisite.
- Managed Paperclip git worktrees default to the repository's `scripts/provision-worktree.sh` when the strategy omits `provisionCommand`. When a registered source config exists, setup creates the isolated config and pending manifest. Runtime startup also runs `scripts/provision-worktree-runtime.sh` automatically when no explicit runtime provision command is configured and the manifest is not verified. Explicitly configured provision commands still take precedence.
- An environment-configured server may have no local seed config. If neither the base checkout nor the default control-plane instance has one, a fresh worktree prepares its dependencies without creating a development instance. This also applies when `PAPERCLIP_CONFIG` names the default `$PAPERCLIP_HOME/instances/$PAPERCLIP_INSTANCE_ID/config.json` path, as the Docker image does. Setup creates no config, environment file, or seed manifest and does not claim runtime or seed readiness. A later request for a seeded development runtime still needs a canonical registered source config. Once that source exists, provisioning the checkout again creates the development instance normally.
- A missing custom `PAPERCLIP_CONFIG`, rejected symlink, or non-regular source file still fails setup. An existing worktree with a config, environment file, or seed state also fails if its source disappears; setup never downgrades that instance to a plain checkout. Repair the source before retrying.
- The built-in deferred seed is recorded as its own terminal `workspace_seed` operation. A zero exit code is not enough for success: the operation succeeds only when `.paperclip/seed-manifest.json` contains complete verified evidence; failed, missing, or malformed manifests produce a failed operation with the seed phase in metadata.
- Worktrees created before lazy seeding shipped may have neither marker. Paperclip adopts them only after their configured database proves a compatible migration journal and the core Paperclip schema; otherwise managed startup creates a pending manifest and performs the normal verified seed. Manual markerless worktrees must provide `--from-config` so the source remains explicit.
+13
View File
@@ -40,6 +40,19 @@ and ARM64. Canonical master pushes also publish
for its immutable multi-platform digest. Downstream services can compose their
own images from this public base without rebuilding Core.
Stamped standard images include the build-owned remote provider pack at
`/opt/paperclip-runner/provider-pack` and set
`PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH` to that directory. Downstream
compositions and the explicit `cloud` target inherit both. The pack is required
on the controller for remote OpenCode and ACPX runs, even when the sandbox has
preinstalled providers; setting the environment variable alone cannot repair an
image that omits the pack. Rebuild and redeploy from a standard image containing
the pack if runs fail with `runner_remote_provider_artifact_incompatible`.
Local Docker builds must supply a full `PAPERCLIP_BUILD_COMMIT` to include a
qualified pack. Unstamped builds remain usable for local adapters and skip pack
generation. The pack stays root-owned and readable by remapped runtime UIDs.
The legacy recurring public `-cloud` publisher is retired. Master pushes,
release tags, and manual `Docker` dispatches no longer build that variant.
Existing `-cloud` tags and digests remain in the registry for rollback; their
+10
View File
@@ -1197,6 +1197,14 @@ Dashboard payload must include:
- month-to-date spend and budget utilization
- pending approvals count
The dashboard agent cards show each linked task at most once. The server selects
distinct task cards from bounded active and recent run samples before it applies
the dashboard card limit. When multiple runs belong to one task, an active run
takes precedence over completed runs.
Runs without a linked task remain separate cards. The dashboard keeps the
count of additional distinct cards in its link to the live runs page, which
can show every run.
## 10.10 Error Semantics
- `400` validation error
@@ -1661,6 +1669,8 @@ for persistence, migration, rendering, and integration contracts.
`GET /api/companies/:companyId/chats/:agentRef` reads an existing conversation or null. `POST` atomically resolves its issue when adding a chat or on first send/upload. `GET /api/companies/:companyId/chats` lists only the current board user’s conversations in that company, subject to ordinary issue read access. The Chat navigation opens a searchable secondary sidebar with agent avatars and a picker for starting or reopening the same per-agent conversation. Existing issue comment, attachment, document, interaction, and run APIs apply thereafter. User chat comments require an idempotent UUID `clientRequestId`. Conversation delivery preserves comment order through the existing issue execution queue; the durable comment outbox repairs the commit-to-enqueue crash window.
The Chat navigation entry reopens the last agent conversation visited by the current user in the current company. The browser keeps this recent order and any existing conversation ID locally; unavailable agents and removed conversations are skipped. An agent chat that has no issue yet can still reopen from the agent roster. The agent chooser remains the landing view when no saved chat is available.
The server owns conversation state: `waiting` plus `in_review` denotes a healthy idle conversation, and `active` denotes an unanswered or executing turn. Successful replies settle a turn; they do not finish the issue. Idle containers are excluded from execution-work counts, ordinary task lists, timer work, and recovery invocations. Failed/unanswered turns retain normal handling. Child completion never wakes or completes the conversation. Search and direct task access preserve history.
Standalone `/new` is an ordered queue command with no model response. It advances a durable session generation and boundary comment, resets only this issue's provider context, and preserves the issue ID and history. Generation checks reject stale context writes and replies. Fresh replay excludes earlier messages and summaries. The shared transcript renders a session divider.
+3
View File
@@ -513,6 +513,9 @@ connection work or enforce a real tenant boundary. Follow these rules:
controls and per-action Test dialog. Do not build a parallel permissions list or
provider-specific testing page. Discovery errors stay inline on Connect; an empty
returned catalog belongs to the ordinary saved-connection state.
- The per-action Test dialog shows structured MCP results as readable fields,
tables, or cards when possible. It keeps the full raw response available for
diagnosis and opens it by default when the result cannot be rendered safely.
- Enable every discovered tool automatically. Put later Allowed / Ask first / Off
controls in management, separate from connection access. Reconnect and refresh
retain existing restrictions; new tools are Allowed under the existing access rules.
+2
View File
@@ -482,6 +482,8 @@ The handshake failure code is distinct from a session-identity mismatch. A timeo
An explicit recovery action is a typed liveness repair path for a source issue. It is the recovery primitive; the action can be rendered directly on the source issue or backed by a separate recovery issue when the repair needs its own work item.
A terminal native failure can retain a result accepted before checkpoint or cleanup failed. That result is historical evidence, not a live controller. New user input may start a fresh turn after the controller and execution environment have stopped and ordinary admission checks pass. Preserve the failed run, its result, and its recovery budget. Do not commit the old result, infer action outcomes, or replay the failed turn. A message saved while cleanup is pending must be reconsidered after verified cleanup and delivered once. Admission must consume its deferred receipt in the same transaction that creates the new run, including in agent chat, where later messages keep their separate turns. Completing that run must not promote the consumed receipt again. Workspace-export repair retains its separate saved-result recovery path.
A new user message can continue a terminal native run whose process fields were cleared before local stop receipts existed. Admission must verify the exact run, runner, workspace, and provider session in the retained suspended state, with no active provider turn, pending tool call, or undelivered output. Missing or mismatched state keeps the hold. A later recorded process launch also keeps the hold until its stop is verified. Normal assignment, decision, controller, environment cleanup, and active-run gates still apply. The message starts one fresh conversation turn; it does not replay the failed run, reset its recovery budget, or certify unknown action outcomes.
The task thread exposes the existing guarded Retry action for failed or timed-out legacy conversation runs. Where the server supports an explicit new attempt after a stopped legacy conversation, the thread must not hide that action solely because the old run still has a recovery-needed projection. Native and process recovery holds, pending decisions, active execution, and other retry gates remain in force. When a gate hides Retry, the thread says the message is preserved instead of promising an unavailable action. This presentation change does not rewrite historical outcomes or certify prior actions.
+56
View File
@@ -0,0 +1,56 @@
# Workspace restore locks
Workspace restore and agent-file collection serialize writes to each canonical
target directory. Their lock files live in
`<instance root>/locks/directory-merge`, outside the writable target. All writers
must use the same instance root and a filesystem with reliable SQLite file
locking. Network filesystems that do not provide that locking are not supported.
Each target has a permanent `<hash>.lock.sqlite` file. An open SQLite
`BEGIN IMMEDIATE` transaction holds its reserved file lock for the entire write
operation. Contenders retry without blocking the Node event loop, up to the
existing 30-second limit. Closing the connection releases the lock; the operating
system also releases it when the process exits or crashes. Independent targets
use different files and can proceed concurrently.
This uses the same built-in `node:sqlite` dependency as workspace manifests.
See [SQLite file locking](https://www.sqlite.org/lockingv3.html) for the reserved
lock contract. No application database or schema migration is involved.
The adjacent `<hash>.lock.owner.json` records a PID and creation time only for
bounded timeout diagnostics. Missing, stale, or incorrect metadata cannot grant
or retain ownership. PID reuse, PID namespaces, and wall-clock changes do not
decide whether a writer owns the lock.
**Never delete, replace, or move a `.lock.sqlite` file while an instance can
write to it.** Its stable inode is part of the locking contract. Replacing it
could let two writers lock different files for the same target. Files remain
after release, including for targets that no longer exist. Their diagnostic
owner sidecars are normally removed after release.
## Upgrade from directory locks
Older versions created `<hash>.lock/owner.json` and checked only whether the
recorded PID existed. A server restart could reuse that PID and leave an orphaned
lock permanently protected. Those records cannot establish a process lifetime or
PID namespace, so the new implementation never guesses that a legacy holder is
dead. An existing legacy directory continues to block admission.
Do not run old and new lock protocols concurrently against the same instance
root. An old process does not participate in the SQLite lock protocol.
1. Drain and stop **all** old server and worker processes that can write to the
instance root, including processes on other hosts or in other containers.
2. Preserve any unfinished-run evidence needed for recovery. After all writers
are stopped, move leftover legacy `<hash>.lock/` directories to an operator
scratch directory outside the lock root. Do not remove `.lock.sqlite` files.
3. Start all writers on the new version. Verify that a run completes both the
provider turn and file collection/restore. A successful model response alone
does not prove that its local file changes were saved.
The same drain requirement applies to rollback. Existing permanent SQLite files
can remain on disk; older versions ignore them. Do not infer that a legacy lock
is safe to remove from its age, an absent PID, or a successful task response.
This change prevents new orphaned ownership. It cannot recover file changes
that an earlier failed collection discarded.