mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-27 19:35:13 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Tasks can use isolated repository workspaces. > - Core currently prepares those repositories on its own filesystem. > - A storage provider may only expose repositories inside remote agent computers. > - Project storage and task workspace preparation must have one provider owner. > - This pull request adds that contract and keeps execution as a separate plugin interface. > - Core keeps authorization, durable task bindings, and lifecycle fences. ## Linked Issues or Issue Description **Subsystem affected** Plugin SDK, project workspace policy, heartbeat preparation, execution leases, and workspace lifecycle. **Problem or motivation** Core cannot prepare a task workspace when its repository storage is unavailable on the Core host. An execution lease must not be required just to prepare persistent storage. **Proposed solution** Let one plugin provide project resource lifecycle and workspace prepare, inspect, restore, and remove behavior. Core reserves a task workspace before it calls the plugin. The plugin returns an opaque durable reference. The matching execution plugin receives that reference on its task RPC. **Alternatives considered** Preparing repositories on Core would require a provider mount on Core. Preparing them only during task launch would tie persistent workspace lifecycle to execution leases and agent availability. **Roadmap alignment** This extends the remote provider model in ROADMAP.md. Related execution work is in #16978. This PR does not replace that task bridge. Native execution of managed storage requires the bridge; this change fails closed when it is absent. ## What Changed - Add one storage provider declaration with supported modes/strategies, a capability, a validated RPC contract, and a test harness. - Persist provider ownership, repository revision, and execution-workspace identity before external effects. Reconcile retries with a stable operation ID. - Prepare provider storage before environment acquisition. Keep Core filesystem preparation out of this path. - Pin the durable reference to the matching execution plugin and validate the lease binding before task dispatch. - Route inspection, reopen, and removal through the existing lifecycle process. Keep local data when remote recovery is uncertain. - Keep the application database pool free during lifecycle locks. Preserve pending workspace selections, validate source edits before retirement, and retry unfinished closed storage cleanup. - Protect provider metadata and live repository sources from caller edits. Storage edits retire only closed records with confirmed removal; retired records cannot reopen. ## Verification - Current head `8214b3ca5`: all 25 provider lifecycle tests pass with `maxConnections=1`. They cover preparation, inspection, reopening, removal, a paused provider with responsive ordinary SQL and a competing source edit, pending selection preservation, rejected source edits, cleanup retry scheduling, and providers that cache operation results, and committed cleanup notifications for storage intent and recovered state. - Final frozen-tree lifecycle run on `8214b3ca5`: all 318 tests passed in 10 files, covering workspace services, reopen, compaction, task binding, project sources, heartbeat preparation, display caching, and source locks. - Direct server `tsc --noEmit` passes. The SDK build and all 128 SDK tests passed during initial verification; later SDK changes are documentation only. - Full CI on the previous head `ec5c7209a` passed all 55 applicable checks; 2 conditional checks skipped. Standard Greptile reviewed current head `342e6d9c3` at 5/5 with no unresolved threads. All 55 applicable CI checks passed on `342e6d9c3`; 2 conditional checks skipped. The Slack browser and CLI startup jobs passed on their single unchanged-head rerun. Its Apex review returned 4/5 with one pending-restoration issue. Commit `1b58943b1` reconciles that durable intent before cleanup, advances the generation without reopening the Core row, and blocks direct removal or retirement while intent remains unresolved. Standard Greptile reviewed `1b58943b1` at 5/5 with no new findings. All 54 applicable CI checks passed on `1b58943b1`; 3 conditional checks skipped. The chat job passed on its single unchanged-head rerun. Apex then returned 4/5 for a missing cleanup wake after recovery. Commit `8214b3ca5` atomically registers cleanup intent before storage intent and recovery-result writes. Its provider test observes real committed notification hints and the visible advanced generation. Standard Greptile reviewed `8214b3ca5` at 5/5 with no new findings. All 55 applicable current-head CI checks passed on the first run; 2 conditional checks skipped. Final Apex verification completed at 5/5 on this exact head: 38 files reviewed, no new comments, and no unresolved threads. The PR is ready for review and mergeable. - The first Apex review found five issues after a standard 5/5: application-pool starvation, stale pending selections, retirement during rejected source edits, lost cleanup retries, and reused inspection identities. Commits `6d72e7fd4` and `342e6d9c3` fix each with regression coverage. Archive lock waits use dedicated connections, and source edits fail promptly if the lifecycle is busy. Retirement compares normalized values. The normalized-source regression initially failed with the raw-value comparison; after moving the check after normalization, a clean provider-suite rerun passed all 24 tests at that stage. Those threads are resolved; the earlier Apex 0/5 is not a passing acceptance result. - Earlier local integration attempts could not bind localhost inside the sandbox. The same suites passed with permission to start local PostgreSQL. - The earlier broad local `pnpm test:run` overlapped edits and reported stale-revision and checkout-path/Git-fixture failures. It was not a final-tree result. Full CI subsequently passed; the original local failure remains part of verification history. - Local repository-wide typecheck/build cannot complete because `cargo` is unavailable. Direct server TypeScript checks and affected tests pass; full Rust and build validation comes from CI. - An earlier browser run on `ff918c5a8` failed in the unchanged case-filter journey: expected “Done,” received “All statuses,” with an announcement handler recorded during the assertion. The cause is not confirmed. [The failed job](https://github.com/paperclipai/paperclip/actions/runs/39500611857/job/118896954508) remains verification history. The unchanged shard passed in the next full CI run on `ec5c7209a`; no UI or fixture change was made. - Current-head browser shard 2 failed waiting five seconds for the mocked Slack setup Success heading. This PR changes neither that UI nor its fixture. [The failed job](https://github.com/paperclipai/paperclip/actions/runs/39506070513/job/118916021232) remains verification evidence. The cause is not confirmed; the single unchanged-shard rerun passed; no UI or fixture change was made, and the original cause remains unconfirmed. - The workspace test shard passed its 3,978 general tests, then its CLI company-import/export fixture timed out after 30 seconds waiting for startup health before creating any company or workspace. [The failed job](https://github.com/paperclipai/paperclip/actions/runs/39506070513/job/118916021855) remains verification history. The child reached external-PostgreSQL startup without exiting or reporting an exception. Its cause is not confirmed; the single unchanged-job rerun passed, including the CLI fixture. No startup or fixture change was made, and the original cause remains unconfirmed. - The new cached-removal regression initially failed because its fake worker advertised availability for unrelated fixture plugins, so another workspace changed its fake checkout state. Worker availability now matches the fixture plugin; the complete 25-test provider suite passes. - On `1b58943b1`, the chat shard timed out polling for one conversation after an authorized GitHub description edit. [The failed job](https://github.com/paperclipai/paperclip/actions/runs/39510521657/job/118930193418) remains evidence. The exact case passes locally against PostgreSQL (1 passed, 1,380 unrelated cases skipped), and this PR changes neither that test nor its handler. The original cause is unconfirmed; the single unchanged-job CI rerun passed without a code or fixture change. The original cause remains unconfirmed. - The diff was checked for credentials, private references, and generated artifacts. ## Risks This is an additive provider contract with no database migration. Existing projects keep their local behavior unless their policy selects a storage plugin. A selected provider must be ready and enabled. Unknown or unsafe cleanup fails closed. Managed native execution depends on #16978. Core refuses a host Runner fallback. Providers must fence lifecycle generations, preserve ambiguous retries, and report remote recovery honestly. Provider-specific live storage and guest mount qualification remain separate from these Core contract tests. ## Model Used OpenAI Codex (GPT-6), with repository inspection, code execution, and tests. The exact model variant was not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run tests locally and they pass (affected suites; broad-run limits are documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
183 lines
6.8 KiB
TypeScript
183 lines
6.8 KiB
TypeScript
import { z } from "zod";
|
|
import {
|
|
WORKSPACE_OVERVIEW_DEFAULT_LIMIT,
|
|
WORKSPACE_OVERVIEW_MAX_LIMIT,
|
|
} from "../constants.js";
|
|
|
|
export const executionWorkspaceStatusSchema = z.enum([
|
|
"active",
|
|
"idle",
|
|
"in_review",
|
|
"archived",
|
|
"cleanup_failed",
|
|
]);
|
|
|
|
export const executionWorkspaceDeliveryStateSchema = z.enum([
|
|
"merged_via_pr",
|
|
"merged_by_ancestry",
|
|
"unmerged",
|
|
"unknown",
|
|
]);
|
|
|
|
const workspaceOverviewStatusFilterSchema = z.preprocess((value) => {
|
|
if (value === undefined || value === null) return undefined;
|
|
const rawValues = Array.isArray(value) ? value : [value];
|
|
const statuses = rawValues.flatMap((entry) => {
|
|
if (typeof entry !== "string") return [];
|
|
return entry.split(",").map((part) => part.trim()).filter(Boolean);
|
|
});
|
|
return statuses.length > 0 ? statuses : undefined;
|
|
}, z.array(executionWorkspaceStatusSchema).optional());
|
|
|
|
export const workspaceOverviewQuerySchema = z.object({
|
|
projectId: z.string().guid().optional(),
|
|
status: workspaceOverviewStatusFilterSchema,
|
|
limit: z.coerce.number().int().min(1).max(WORKSPACE_OVERVIEW_MAX_LIMIT).optional().default(WORKSPACE_OVERVIEW_DEFAULT_LIMIT),
|
|
offset: z.coerce.number().int().min(0).optional().default(0),
|
|
}).strict();
|
|
|
|
export const executionWorkspaceConfigSchema = z.object({
|
|
environmentId: z.string().guid().optional().nullable(),
|
|
provisionCommand: z.string().optional().nullable(),
|
|
runtimeProvisionCommand: z.string().optional().nullable(),
|
|
teardownCommand: z.string().optional().nullable(),
|
|
cleanupCommand: z.string().optional().nullable(),
|
|
workspaceRuntime: z.record(z.string(), z.unknown()).optional().nullable(),
|
|
desiredState: z.enum(["running", "stopped", "manual"]).optional().nullable(),
|
|
serviceStates: z.record(z.string(), z.enum(["running", "stopped", "manual"])).optional().nullable(),
|
|
}).strict();
|
|
|
|
export const workspaceRuntimeControlTargetSchema = z.object({
|
|
workspaceCommandId: z.string().min(1).optional().nullable(),
|
|
runtimeServiceId: z.string().guid().optional().nullable(),
|
|
serviceIndex: z.number().int().nonnegative().optional().nullable(),
|
|
}).strict();
|
|
|
|
export const executionWorkspaceCloseReadinessStateSchema = z.enum([
|
|
"ready",
|
|
"ready_with_warnings",
|
|
"blocked",
|
|
]);
|
|
|
|
export const executionWorkspaceCloseActionKindSchema = z.enum([
|
|
"archive_record",
|
|
"stop_runtime_services",
|
|
"cleanup_command",
|
|
"teardown_command",
|
|
"remove_provider_workspace",
|
|
"git_worktree_remove",
|
|
"git_branch_delete",
|
|
"remove_local_directory",
|
|
]);
|
|
|
|
export const executionWorkspaceCloseActionSchema = z.object({
|
|
kind: executionWorkspaceCloseActionKindSchema,
|
|
label: z.string(),
|
|
description: z.string(),
|
|
command: z.string().nullable(),
|
|
}).strict();
|
|
|
|
export const executionWorkspaceCloseLinkedIssueSchema = z.object({
|
|
id: z.string().guid(),
|
|
identifier: z.string().nullable(),
|
|
title: z.string(),
|
|
status: z.string(),
|
|
isTerminal: z.boolean(),
|
|
}).strict();
|
|
|
|
export const executionWorkspaceCloseGitReadinessSchema = z.object({
|
|
repoRoot: z.string().nullable(),
|
|
workspacePath: z.string().nullable(),
|
|
branchName: z.string().nullable(),
|
|
baseRef: z.string().nullable(),
|
|
hasDirtyTrackedFiles: z.boolean(),
|
|
hasUntrackedFiles: z.boolean(),
|
|
dirtyEntryCount: z.number().int().nonnegative(),
|
|
untrackedEntryCount: z.number().int().nonnegative(),
|
|
aheadCount: z.number().int().nonnegative().nullable(),
|
|
behindCount: z.number().int().nonnegative().nullable(),
|
|
isMergedIntoBase: z.boolean().nullable(),
|
|
createdByRuntime: z.boolean(),
|
|
headSha: z.string().nullable().optional(),
|
|
}).strict();
|
|
|
|
export const workspaceRuntimeServiceSchema = z.object({
|
|
id: z.string(),
|
|
companyId: z.string().guid(),
|
|
projectId: z.string().guid().nullable(),
|
|
projectWorkspaceId: z.string().guid().nullable(),
|
|
executionWorkspaceId: z.string().guid().nullable(),
|
|
issueId: z.string().guid().nullable(),
|
|
scopeType: z.enum(["project_workspace", "execution_workspace", "run", "agent"]),
|
|
scopeId: z.string().nullable(),
|
|
serviceName: z.string(),
|
|
status: z.enum(["provisioning", "starting", "running", "stopped", "failed"]),
|
|
lifecycle: z.enum(["shared", "ephemeral"]),
|
|
reuseKey: z.string().nullable(),
|
|
command: z.string().nullable(),
|
|
cwd: z.string().nullable(),
|
|
port: z.number().int().nullable(),
|
|
url: z.string().nullable(),
|
|
provider: z.enum(["local_process", "adapter_managed"]),
|
|
providerRef: z.string().nullable(),
|
|
ownerAgentId: z.string().guid().nullable(),
|
|
startedByRunId: z.string().guid().nullable(),
|
|
lastUsedAt: z.coerce.date(),
|
|
startedAt: z.coerce.date(),
|
|
stoppedAt: z.coerce.date().nullable(),
|
|
stopPolicy: z.record(z.string(), z.unknown()).nullable(),
|
|
healthStatus: z.enum(["unknown", "healthy", "unhealthy"]),
|
|
configIndex: z.number().int().nonnegative().nullable().optional(),
|
|
createdAt: z.coerce.date(),
|
|
updatedAt: z.coerce.date(),
|
|
}).strict();
|
|
export const executionWorkspaceCloseReadinessSchema = z.object({
|
|
workspaceId: z.string().guid(),
|
|
deliveryState: executionWorkspaceDeliveryStateSchema,
|
|
state: executionWorkspaceCloseReadinessStateSchema,
|
|
blockingReasons: z.array(z.string()),
|
|
warnings: z.array(z.string()),
|
|
linkedIssues: z.array(executionWorkspaceCloseLinkedIssueSchema),
|
|
plannedActions: z.array(executionWorkspaceCloseActionSchema),
|
|
isDestructiveCloseAllowed: z.boolean(),
|
|
isSharedWorkspace: z.boolean(),
|
|
isProjectPrimaryWorkspace: z.boolean(),
|
|
git: executionWorkspaceCloseGitReadinessSchema.nullable(),
|
|
runtimeServices: z.array(workspaceRuntimeServiceSchema),
|
|
}).strict();
|
|
|
|
export const updateExecutionWorkspaceSchema = z.object({
|
|
name: z.string().min(1).optional(),
|
|
cwd: z.string().optional().nullable(),
|
|
repoUrl: z.string().optional().nullable(),
|
|
baseRef: z.string().optional().nullable(),
|
|
branchName: z.string().optional().nullable(),
|
|
providerRef: z.string().optional().nullable(),
|
|
status: executionWorkspaceStatusSchema.optional(),
|
|
cleanupEligibleAt: z.string().datetime().optional().nullable(),
|
|
cleanupReason: z.string().optional().nullable(),
|
|
config: executionWorkspaceConfigSchema.optional().nullable(),
|
|
metadata: z.record(z.string(), z.unknown()).optional().nullable(),
|
|
}).strict();
|
|
|
|
const branchReconcileReasonSchema = z.string().trim().min(1);
|
|
|
|
export const reconcileExecutionWorkspaceBranchSchema = z.discriminatedUnion("mode", [
|
|
z.object({
|
|
mode: z.literal("forward"),
|
|
reason: branchReconcileReasonSchema.optional().nullable(),
|
|
}).strict(),
|
|
z.object({
|
|
mode: z.literal("override"),
|
|
reason: branchReconcileReasonSchema,
|
|
}).strict(),
|
|
z.object({
|
|
mode: z.literal("quarantine_restore"),
|
|
reason: branchReconcileReasonSchema.optional().nullable(),
|
|
}).strict(),
|
|
]);
|
|
|
|
export type UpdateExecutionWorkspace = z.infer<typeof updateExecutionWorkspaceSchema>;
|
|
export type ReconcileExecutionWorkspaceBranch = z.infer<typeof reconcileExecutionWorkspaceBranchSchema>;
|
|
export type WorkspaceOverviewQuery = z.infer<typeof workspaceOverviewQuerySchema>;
|