diff --git a/doc/DATABASE.md b/doc/DATABASE.md index 151581873d..3a348cbe52 100644 --- a/doc/DATABASE.md +++ b/doc/DATABASE.md @@ -442,6 +442,17 @@ revokes the previous boot identity with a conditional update. Its own claim also expires so another sweep can finish cleanup after a restart. Historical rows keep null ownership fields and follow the previous recovery path. +## Agent file persistence and legacy revisions + +Managed agent files are current filesystem contents, using the same persistent +instance storage as other workspaces. `agent_instruction_revisions` and +`agent_instruction_heads` are retained as read-only upgrade input. Their heads +are adopted once into the managed directory; new saves never append revisions. +`agent_instruction_working_copies` holds per-run baseline hashes, state, and +capture receipts. New receipts identify `paperclip.agent-files.v1`; historical +rows retain the instruction-only format. Completed directory runs discard their +baseline and private copies. See [Persistent agent files](agent-files.md). + ## Large API response snapshots `assets.byte_size` uses PostgreSQL `bigint` so saved responses and byte ranges can diff --git a/doc/SPEC-implementation.md b/doc/SPEC-implementation.md index 927e7f1a7e..12e000fea5 100644 --- a/doc/SPEC-implementation.md +++ b/doc/SPEC-implementation.md @@ -1725,6 +1725,23 @@ read or change these preferences. The legacy instance general setting is retaine for API compatibility but no longer controls shortcut behavior in the app; users opt in individually after the upgrade. +### Persistent managed agent files (2026-09-28) + +The Instructions Editor and agent execution share one current agent-owned +directory, scoped by company and agent. The configured instruction entry is one +file in this directory. Registered private copies synchronize supported files +across tasks and sessions, separately from task workspace persistence. Saves +require verified provider stop and current authorization, then synchronize changed +files using per-file last-sync-wins; +new content is not stored as revision history. Existing deployed revisions and +saved execution formats remain compatible during adoption. See +[Persistent agent files](agent-files.md). + +Persistent-file storage limits are advisory for execution: a full folder cannot +pause the agent, fail its run, or prevent later runs. Show a warning on each run +while storage remains full, restore existing files so the agent can remove them, +and enforce the limits on saves. Cleanup clears the warning for future runs. + ### Unsafe native workspace exports An unsafe workspace link does not fail an accepted native task result. Retry diff --git a/doc/SPEC.md b/doc/SPEC.md index 3e5fa410fe..e69a6f1b8b 100644 --- a/doc/SPEC.md +++ b/doc/SPEC.md @@ -620,6 +620,18 @@ read or change these preferences. The legacy instance general setting is retaine for API compatibility but no longer controls shortcut behavior in the app; users opt in individually after the upgrade. +Managed agents own a persistent file directory across tasks and sessions. The +Instructions Editor and stopped agent execution synchronize the same current +files, including AGENTS.md and its supporting files. Task working directories and +provider home directories remain separate concepts. Concurrent runs synchronize only +the files they change, with the last sync winning for the same file. Temporary +copies are cleaned up; this storage does not add a revision-history system. See +[agent-files.md](agent-files.md) for lifecycle and upgrade compatibility. + +Full agent storage produces a run warning without stopping current or future +work. Storage limits constrain saved file changes, not the agent's ability to run +and remove files to recover space. + ### Unsafe native workspace exports An unsafe workspace link does not fail an accepted native task result. Retry diff --git a/doc/agent-files.md b/doc/agent-files.md new file mode 100644 index 0000000000..956db02c02 --- /dev/null +++ b/doc/agent-files.md @@ -0,0 +1,177 @@ +# Persistent agent files + +Each managed agent has one current directory, scoped by company and agent. The +Instructions Editor reads and writes this directory. `AGENTS.md` (or the +configured entry) is one file in it. Agents may create ordinary files and nested +folders for notes, memory, and other personal working material. Files in the +task working directory remain task files. + +Agents can edit their own managed files under the responsible user’s current +target permissions. Access to another agent’s files additionally requires the +caller’s own target-scoped configuration permission; shared company membership +or a responsible user alone does not grant peer access. + +## Layout + +The canonical host directory keeps its existing physical location: + +``` +/companies//agents// + instructions/ current agent files (editor) + AGENTS.md + notes/ + any-supported-file + file-sync/ controller-only operational state + adopted.json + runs//live/ isolated writable copy while a run executes +``` + +The process starts in its existing task workspace. `AGENT_HOME` points to the +registered writable agent copy. Adapter `HOME` and `CODEX_HOME` keep their +existing meanings and are not personal-file storage. Local copies are outside +the task workspace. Remote providers currently confine file sync to their +workspace: their independent agent copy therefore lives under the excluded +`.paperclip-runtime/agent-files///` area. It is not included in task +workspace sync, Git staging, or task deliverables. + +Regular files (including binary bytes) and directories are supported, up to +100,000 entries (files and folders), 256 MiB per file and 2 GiB total. Symlinks, +hardlinks, and special +files are rejected, rather than followed or silently skipped. The instruction +entry remains valid UTF-8, at most 1 MiB, and cannot be deleted. The editor edits +text up to 1 MiB and offers downloads for binary or larger files. The reserved +`.paperclip-runtime` directory and the compatibility-only virtual file +`promptTemplate.legacy.md` are not user storage. Task cache and Git ignore +exclusions do not apply to this directory. + +These storage limits are separate from the 1 MiB instruction/editor limit. Large +files are hashed and downloaded as streams; listings bound concurrent reads, +and only editor-sized text is buffered. Storage counts uncompressed file bytes, +not allocated disk +blocks. These are sync validation limits, not live filesystem quotas: an agent +can write beyond them while running. Storage limits never pause an agent, fail +a provider run, or block future task admission. A folder at or above a limit +produces a warning on each run until enough files have been removed or shrunk. +Existing saved files are restored even when already over quota, so the agent +can continue working and clean them up with ordinary filesystem tools. Unsafe +paths and links still fail validation; bypassing a storage quota does not +bypass those checks. + +An API save above a storage limit returns 422 without changing the saved files. +If a stopped run exceeds a storage limit, none of its agent-folder changes are +saved. The run shows a nonblocking storage warning and its save receipt reports +`AGENT_FILES_LIMIT_EXCEEDED` with the specific limit +and, for an oversized file, its path. The previous saved folder is used on the +next run. The temporary run copy is discarded, including on a limit failure; +there is no retained recovery archive or partial-save option. Transient sync +failures get up to three attempts at the stop boundary before cleanup and an +explicit failure receipt. Individual file writes are atomic, but an I/O failure +partway through a sync can leave some files updated; a failed receipt does not +claim whole-folder success. + +Sync failures are diagnostics for the affected run, not errors on the current +files in the Instructions Editor. Historical failures remain in the run log; +the run detail also shows warnings from its save receipt. The editor only shows +preserved instruction-only candidates that may need review, alongside errors +from the current browser edit. Later successful saves do not erase run history. + +Larger folders take longer to hash, copy, and transfer on each run. There is one +canonical folder plus temporary working copies for currently active runs (and +remote staging when the transport needs it). No additional captured tree is +created. Terminal runs remove their private trees and baseline metadata, keeping +only a small receipt. Restart recovery retries interrupted cleanup without +removing a running provider's files. These are not aggregate disk quotas; the +operator still provisions storage for agents and the configured run concurrency. + +## Run lifecycle + +1. Under the agent lock, restore current files into a private run copy and save + a baseline of paths, kinds, modes, and hashes. This is sync metadata, not a + revision history. +2. Stage the copy through the existing workspace transport. Point `AGENT_HOME` + and instruction guidance at that registered root. +3. At the provider's verified checkpoint-and-stop boundary, retrieve the entire + directory into the existing working copy before releasing its environment. +4. Recheck the responsible user's current authorization. Under the same agent + lock used by editor writes, apply only files changed or deleted relative to + the starting baseline. For a competing edit or deletion of the same file, + the last synchronization to acquire the lock wins. Unchanged files do not + overwrite another run's changes; newly added unrelated files survive. +5. Record the outcome and remove temporary copies for successful and failed + runs. No per-run file versions, conflict copies, or review queue accumulate. + The next run starts with the current directory. + +The whole-directory contract closes the provider process to establish a safe +collection boundary, including child processes. It preserves the provider's +resumable conversation. Only the loaded instruction entry participates in the +new runtime instruction digest; adding or editing another file does not change +that digest. Relative supporting files are read from `AGENT_HOME`, not from the +read-only prompt snapshot. + +The editor supplies the hash of the file it read. A stale browser save returns +409 and retains the user's unsaved draft. Run synchronization itself uses +per-file last-sync-wins: a later run can overwrite a saved browser edit to the +same file. There is no text merge or historical copy to recover the overwritten +version. Ordinary task files continue using their existing workspace contract. + +## Upgrade and recovery + +Migration 0287 creates the preview tables idempotently after master’s 0285/0286. +Existing preview receipts, rows, constraints, and pending captures are retained. +On first use, while holding +the agent row lock, import any deployed revision heads into the existing managed +directory once. A controller-owned marker outside agent files prevents any +later replay of those heads. Existing revision rows remain readable for recovery; +new saves never append to them. Old UUID-based clients receive content tokens +and can still submit their previously recorded revision IDs, which are checked +against the corresponding bytes before a write. + +Working-copy receipts and native runtime inputs record the new file contract. +A restored native session with no contract field keeps the old instruction-only +copy shape, prompt digest, paths, and collector. Its writes use the compatibility +bridge into current files, with the original baseline fence. Existing pending +legacy candidates remain resolvable. Neither old task workspaces nor arbitrary +external instruction roots are imported as agent directories. + +Stock-agent and plugin resets update their declared files while retaining unrelated +personal files and formerly configured entries. Automatic stock upgrades first +record baseline hashes in the existing resource binding, then apply and finalize +under the agent lock. A failed file write or database commit retries against +those hashes and already-applied bytes. Removed, unchanged stock files are +removed; intervening personal edits stop the retry. This pending operation +metadata is cleared on success and does not retain file revisions. + +External bundles retain their existing behavior. Their migration to managed +storage is an explicit configuration action. Historical task cwd, provider-home, +checkpoint, and workspace restoration formats are not rewritten. + +Backups must include the persistent instance filesystem as well as the database. +New current-file bytes are not database revision rows. Old instruction-only +candidates are retained solely for upgrade compatibility. + +Crash recovery can collect a stopped working copy without starting a model. +Missing stop proof or lost remote bytes produce a visible diagnostic, never a +save receipt. An interrupted apply can replay its changed files with the same +last-sync-wins rule. Cleanup resumes for terminal runs; no copy is retained as +an archive after cleanup succeeds. + +## Verification + +`agent-directory-working-copies.test.ts` exercises nested/binary files, directory +isolation, last-sync-wins edits and deletions, terminal cleanup, link rejection, old-head +adoption, and stable prompt digests. The legacy working-copy and native-tool +suites exercise compatibility. Workspace merge tests exercise preflight and +interrupted replay. + +The explicit Product E2E `instruction-persistence` suite creates a file through +the browser editor, runs an agent that changes instructions and supporting files, +checks exact binary bytes via the public download route, restarts the server, +and asks a fresh task to prove restored contents using an independent nonce. A +third task edits its entry while the browser saves that same file; the later +run sync wins while a separate browser-created file survives, with no conflict +candidate or manual resolution. Three more tasks save a sparse file at its +256 MiB boundary, exceed that boundary with a nonfatal save rejection, then +remove it and save a new small file. All tasks must succeed, with warnings +visible in run details while full and cleared after cleanup. +Run results, including unavailable credentials, must be reported separately from +unit or matcher results; a passing matcher does not prove a live run. diff --git a/doc/agent-instruction-revisions.md b/doc/agent-instruction-revisions.md new file mode 100644 index 0000000000..d4512d6867 --- /dev/null +++ b/doc/agent-instruction-revisions.md @@ -0,0 +1,15 @@ +# Legacy instruction revision compatibility + +The revision-based design in the first version of PR #14325 has been superseded +by [persistent agent files](agent-files.md). Current managed files are authoritative; +new edits do not create revision or head rows. + +Published migrations 0285 and 0286 and existing revision records are retained so +preview installations and saved sessions can upgrade safely. Old heads are +adopted once. Legacy collectors and UUID-based APIs compare their baseline bytes +against current files before writing. Read-only history and restore endpoints +remain for pre-upgrade records; they are not advertised as new agent capabilities +or shown as a history feature in the current file editor. + +See the current document for directory layout, synchronization, conflicts, +authorization, recovery, and verification. diff --git a/doc/evals.md b/doc/evals.md index cbfd78e78b..40d298b5db 100644 --- a/doc/evals.md +++ b/doc/evals.md @@ -276,6 +276,11 @@ results and follow-up coverage are recorded in that suite's guide. Continuation accounting has an explicit-only eight-cell Product E2E [baseline suite](../tests/runner-e2e/CONTINUATION-ACCOUNTING.md), complementing the deterministic lifecycle inventory. +The explicit Product E2E `instruction-persistence` suite verifies private file +edits, nested and binary agent files, stopped-provider directory saves, server restart, and a fresh task's +downloaded proof on local native/legacy Codex and native Daytona. See the +[Product E2E runbook](../tests/runner-e2e/README.md). + The explicit-only Product E2E `api-response-reading` suite verifies retrieval of large saved API responses on local and Daytona native Codex runs. See the [Runner E2E guide](../tests/runner-e2e/README.md#bounded-api-response-reading). diff --git a/packages/adapter-utils/src/acpx-engine/execute.ts b/packages/adapter-utils/src/acpx-engine/execute.ts index 2c4b029e12..7aaef803b9 100644 --- a/packages/adapter-utils/src/acpx-engine/execute.ts +++ b/packages/adapter-utils/src/acpx-engine/execute.ts @@ -2058,7 +2058,11 @@ async function buildRuntime(input: { paperclipClaudeSettings = await writePaperclipClaudeSettings({ cwd, stateDir, - agentHome, + // A run's writable AGENT_HOME moves, but the existing permission root + // must not invalidate its resumable conversation. Run copies are covered + // by the company root already granted in these settings; the process env + // still receives this run's actual AGENT_HOME. + agentHome: asString(workspaceContext.agentHomeForPermissions, agentHome), companyId: agent.companyId, }); skillCommandNotes.push( diff --git a/packages/db/src/agent-instruction-migration.test.ts b/packages/db/src/agent-instruction-migration.test.ts new file mode 100644 index 0000000000..6def7b7e88 --- /dev/null +++ b/packages/db/src/agent-instruction-migration.test.ts @@ -0,0 +1,141 @@ +import { createHash, randomUUID } from "node:crypto"; +import { readFile } from "node:fs/promises"; +import postgres from "postgres"; +import { describe, expect, it } from "vitest"; +import { applyPendingMigrations, inspectMigrations } from "./client.js"; +import { EMBEDDED_POSTGRES_TEST_TIMEOUT_MS, getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./test-embedded-postgres.js"; + +// Exact SQL from PR #14325 before master assigned 0285/0286 to API response storage. +// Exercise real preview tables/receipts, not a second application of the new schema. +const previewMigrations = [ + { when: 1790474441205, sql: `CREATE TABLE "agent_instruction_heads" ( + "company_id" uuid NOT NULL, + "agent_id" uuid NOT NULL, + "entry_file" text NOT NULL, + "revision_id" uuid NOT NULL, + "updated_at" timestamp with time zone DEFAULT now() NOT NULL, + CONSTRAINT "agent_instruction_heads_identity_uq" UNIQUE("company_id","agent_id","entry_file") +); +--> statement-breakpoint +CREATE TABLE "agent_instruction_revisions" ( + "id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL, + "company_id" uuid NOT NULL, + "agent_id" uuid NOT NULL, + "entry_file" text NOT NULL, + "content_base64" text NOT NULL, + "content_hash" text NOT NULL, + "byte_length" integer NOT NULL, + "parent_revision_id" uuid, + "base_revision_id" uuid, + "restored_from_revision_id" uuid, + "actor_agent_id" uuid, + "actor_user_id" text, + "responsible_user_id" text, + "source_run_id" uuid, + "source" text NOT NULL, + "created_at" timestamp with time zone DEFAULT now() NOT NULL, + CONSTRAINT "agent_instruction_revisions_identity_uq" UNIQUE("company_id","agent_id","entry_file","id") +); +--> statement-breakpoint +ALTER TABLE "agent_instruction_heads" ADD CONSTRAINT "agent_instruction_heads_company_id_agent_id_entry_file_revision_id_agent_instruction_revisions_company_id_agent_id_entry_file_id_fk" FOREIGN KEY ("company_id","agent_id","entry_file","revision_id") REFERENCES "public"."agent_instruction_revisions"("company_id","agent_id","entry_file","id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint +ALTER TABLE "agent_instruction_revisions" ADD CONSTRAINT "agent_instruction_revisions_company_id_agent_id_agents_company_id_id_fk" FOREIGN KEY ("company_id","agent_id") REFERENCES "public"."agents"("company_id","id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint +CREATE INDEX "agent_instruction_revisions_history_idx" ON "agent_instruction_revisions" USING btree ("company_id","agent_id","entry_file","created_at","id");` }, + { when: 1790543348340, sql: `CREATE TABLE "agent_instruction_working_copies" ( + "run_id" uuid PRIMARY KEY NOT NULL, + "company_id" uuid NOT NULL, + "agent_id" uuid NOT NULL, + "responsible_user_id" text NOT NULL, + "entry_file" text NOT NULL, + "base_revision_id" uuid, + "base_hash" text NOT NULL, + "local_root" text NOT NULL, + "execution_root" text NOT NULL, + "location" text NOT NULL, + "state" text DEFAULT 'prepared' NOT NULL, + "candidate_base64" text, + "candidate_hash" text, + "error_code" text, + "error_message" text, + "receipt" jsonb, + "process_stopped_at" timestamp with time zone, + "attempts" integer DEFAULT 0 NOT NULL, + "next_attempt_at" timestamp with time zone, + "created_at" timestamp with time zone DEFAULT now() NOT NULL, + "updated_at" timestamp with time zone DEFAULT now() NOT NULL +); +--> statement-breakpoint +ALTER TABLE "agent_instruction_working_copies" ADD CONSTRAINT "agent_instruction_working_copies_run_id_heartbeat_runs_id_fk" FOREIGN KEY ("run_id") REFERENCES "public"."heartbeat_runs"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint +ALTER TABLE "agent_instruction_working_copies" ADD CONSTRAINT "agent_instruction_working_copies_company_id_agent_id_agents_company_id_id_fk" FOREIGN KEY ("company_id","agent_id") REFERENCES "public"."agents"("company_id","id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint +CREATE INDEX "agent_instruction_copies_pending_idx" ON "agent_instruction_working_copies" USING btree ("state","next_attempt_at");--> statement-breakpoint +CREATE INDEX "agent_instruction_copies_agent_idx" ON "agent_instruction_working_copies" USING btree ("company_id","agent_id","created_at");` }, +]; +const migration = await readFile(new URL("./migrations/0287_serious_tinkerer.sql", import.meta.url), "utf8"); +const migrationHash = createHash("sha256").update(migration).digest("hex"); + +const support = await getEmbeddedPostgresTestSupport(); +const describePostgres = support.supported ? describe : describe.skip; + +describePostgres("instruction revision migrations", () => { + it("keeps applied migration receipts, revisions, and pending copies intact on repeated upgrades", async () => { + const database = await startEmbeddedPostgresTestDatabase("instruction-migration-upgrade-"); + const sql = postgres(database.connectionString, { max: 1, onnotice: () => {} }); + try { + await sql`DROP TABLE agent_instruction_heads, agent_instruction_working_copies, agent_instruction_revisions`; + await sql`DELETE FROM drizzle.__drizzle_migrations WHERE hash = ${migrationHash}`; + for (const preview of previewMigrations) { + for (const statement of preview.sql.split("--> statement-breakpoint")) await sql.unsafe(statement); + await sql`INSERT INTO drizzle.__drizzle_migrations (hash, created_at) + VALUES (${createHash("sha256").update(preview.sql).digest("hex")}, ${preview.when})`; + } + const companyId = randomUUID(), otherCompanyId = randomUUID(), agentId = randomUUID(); + const runId = randomUUID(), revisionId = randomUUID(); + const bytes = Buffer.from("\uFEFF# Saved\r\nExact bytes ☃\n"); + const contentBase64 = bytes.toString("base64"); + const contentHash = createHash("sha256").update(bytes).digest("hex"); + await sql`INSERT INTO companies (id, name, issue_prefix) VALUES + (${companyId}, 'Instructions', 'INS'), (${otherCompanyId}, 'Other', 'OTH')`; + await sql`INSERT INTO agents (id, company_id, name) VALUES (${agentId}, ${companyId}, 'Writer')`; + await sql`INSERT INTO heartbeat_runs (id, company_id, agent_id) VALUES (${runId}, ${companyId}, ${agentId})`; + await sql`INSERT INTO agent_instruction_revisions + (id, company_id, agent_id, entry_file, content_base64, content_hash, byte_length, source, source_run_id) + VALUES (${revisionId}, ${companyId}, ${agentId}, 'AGENTS.md', ${contentBase64}, ${contentHash}, ${bytes.length}, 'cleanup', ${runId})`; + await sql`INSERT INTO agent_instruction_heads (company_id, agent_id, entry_file, revision_id) + VALUES (${companyId}, ${agentId}, 'AGENTS.md', ${revisionId})`; + await sql`INSERT INTO agent_instruction_working_copies + (run_id, company_id, agent_id, responsible_user_id, entry_file, base_revision_id, base_hash, local_root, execution_root, location, state, candidate_base64, candidate_hash) + VALUES (${runId}, ${companyId}, ${agentId}, 'editor', 'AGENTS.md', ${revisionId}, ${contentHash}, '/private/copy', '/private/copy', 'local', 'pending_commit', ${contentBase64}, ${contentHash})`; + const snapshot = async () => ({ + journal: await sql`SELECT * FROM drizzle.__drizzle_migrations ORDER BY id`, + relations: await sql`SELECT oid::text, relname FROM pg_class WHERE relname IN + ('agent_instruction_revisions', 'agent_instruction_heads', 'agent_instruction_working_copies') ORDER BY relname`, + constraints: await sql`SELECT oid::text, conname, pg_get_constraintdef(oid) AS definition FROM pg_constraint + WHERE conrelid IN ('agent_instruction_revisions'::regclass, 'agent_instruction_heads'::regclass, + 'agent_instruction_working_copies'::regclass) ORDER BY conname`, + revisions: await sql`SELECT * FROM agent_instruction_revisions`, + heads: await sql`SELECT * FROM agent_instruction_heads`, + copies: await sql`SELECT * FROM agent_instruction_working_copies`, + }); + const before = await snapshot(); + expect(before.relations).toHaveLength(3); + expect(await inspectMigrations(database.connectionString)).toMatchObject({ + status: "needsMigrations", pendingMigrations: ["0287_serious_tinkerer.sql"], + }); + await applyPendingMigrations(database.connectionString); + const upgraded = await snapshot(); + expect(upgraded).toEqual({ ...before, journal: expect.arrayContaining(before.journal) }); + expect(upgraded.journal).toHaveLength(before.journal.length + 1); + for (let attempt = 0; attempt < 2; attempt += 1) { + expect(await inspectMigrations(database.connectionString)).toMatchObject({ status: "upToDate" }); + await applyPendingMigrations(database.connectionString); + expect(await snapshot()).toEqual(upgraded); + } + await expect(sql`UPDATE agent_instruction_heads SET company_id = ${otherCompanyId} WHERE revision_id = ${revisionId}`) + .rejects.toMatchObject({ code: "23503" }); + await expect(sql`UPDATE agent_instruction_working_copies SET company_id = ${otherCompanyId} WHERE run_id = ${runId}`) + .rejects.toMatchObject({ code: "23503" }); + } finally { + await sql.end(); + await database.cleanup(); + } + }, EMBEDDED_POSTGRES_TEST_TIMEOUT_MS); +}); diff --git a/packages/db/src/migrations/0287_serious_tinkerer.sql b/packages/db/src/migrations/0287_serious_tinkerer.sql new file mode 100644 index 0000000000..2810534d4c --- /dev/null +++ b/packages/db/src/migrations/0287_serious_tinkerer.sql @@ -0,0 +1,61 @@ +-- Retain preview instruction rows and pending captures when upgrading PR #14325. +CREATE TABLE IF NOT EXISTS "agent_instruction_heads" ( + "company_id" uuid NOT NULL, + "agent_id" uuid NOT NULL, + "entry_file" text NOT NULL, + "revision_id" uuid NOT NULL, + "updated_at" timestamp with time zone DEFAULT now() NOT NULL, + CONSTRAINT "agent_instruction_heads_identity_uq" UNIQUE("company_id","agent_id","entry_file") +); +--> statement-breakpoint +CREATE TABLE IF NOT EXISTS "agent_instruction_revisions" ( + "id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL, + "company_id" uuid NOT NULL, + "agent_id" uuid NOT NULL, + "entry_file" text NOT NULL, + "content_base64" text NOT NULL, + "content_hash" text NOT NULL, + "byte_length" integer NOT NULL, + "parent_revision_id" uuid, + "base_revision_id" uuid, + "restored_from_revision_id" uuid, + "actor_agent_id" uuid, + "actor_user_id" text, + "responsible_user_id" text, + "source_run_id" uuid, + "source" text NOT NULL, + "created_at" timestamp with time zone DEFAULT now() NOT NULL, + CONSTRAINT "agent_instruction_revisions_identity_uq" UNIQUE("company_id","agent_id","entry_file","id") +); +--> statement-breakpoint +CREATE TABLE IF NOT EXISTS "agent_instruction_working_copies" ( + "run_id" uuid PRIMARY KEY NOT NULL, + "company_id" uuid NOT NULL, + "agent_id" uuid NOT NULL, + "responsible_user_id" text NOT NULL, + "entry_file" text NOT NULL, + "base_revision_id" uuid, + "base_hash" text NOT NULL, + "local_root" text NOT NULL, + "execution_root" text NOT NULL, + "location" text NOT NULL, + "state" text DEFAULT 'prepared' NOT NULL, + "candidate_base64" text, + "candidate_hash" text, + "error_code" text, + "error_message" text, + "receipt" jsonb, + "process_stopped_at" timestamp with time zone, + "attempts" integer DEFAULT 0 NOT NULL, + "next_attempt_at" timestamp with time zone, + "created_at" timestamp with time zone DEFAULT now() NOT NULL, + "updated_at" timestamp with time zone DEFAULT now() NOT NULL +); +--> statement-breakpoint +DO $$ BEGIN ALTER TABLE "agent_instruction_heads" ADD CONSTRAINT "agent_instruction_heads_company_id_agent_id_entry_file_revision_id_agent_instruction_revisions_company_id_agent_id_entry_file_id_fk" FOREIGN KEY ("company_id","agent_id","entry_file","revision_id") REFERENCES "public"."agent_instruction_revisions"("company_id","agent_id","entry_file","id") ON DELETE cascade ON UPDATE no action; EXCEPTION WHEN duplicate_object THEN NULL; END $$;--> statement-breakpoint +DO $$ BEGIN ALTER TABLE "agent_instruction_revisions" ADD CONSTRAINT "agent_instruction_revisions_company_id_agent_id_agents_company_id_id_fk" FOREIGN KEY ("company_id","agent_id") REFERENCES "public"."agents"("company_id","id") ON DELETE cascade ON UPDATE no action; EXCEPTION WHEN duplicate_object THEN NULL; END $$;--> statement-breakpoint +DO $$ BEGIN ALTER TABLE "agent_instruction_working_copies" ADD CONSTRAINT "agent_instruction_working_copies_run_id_heartbeat_runs_id_fk" FOREIGN KEY ("run_id") REFERENCES "public"."heartbeat_runs"("id") ON DELETE cascade ON UPDATE no action; EXCEPTION WHEN duplicate_object THEN NULL; END $$;--> statement-breakpoint +DO $$ BEGIN ALTER TABLE "agent_instruction_working_copies" ADD CONSTRAINT "agent_instruction_working_copies_company_id_agent_id_agents_company_id_id_fk" FOREIGN KEY ("company_id","agent_id") REFERENCES "public"."agents"("company_id","id") ON DELETE cascade ON UPDATE no action; EXCEPTION WHEN duplicate_object THEN NULL; END $$;--> statement-breakpoint +CREATE INDEX IF NOT EXISTS "agent_instruction_revisions_history_idx" ON "agent_instruction_revisions" USING btree ("company_id","agent_id","entry_file","created_at","id");--> statement-breakpoint +CREATE INDEX IF NOT EXISTS "agent_instruction_copies_pending_idx" ON "agent_instruction_working_copies" USING btree ("state","next_attempt_at");--> statement-breakpoint +CREATE INDEX IF NOT EXISTS "agent_instruction_copies_agent_idx" ON "agent_instruction_working_copies" USING btree ("company_id","agent_id","created_at"); diff --git a/packages/db/src/migrations/meta/0282_snapshot.json b/packages/db/src/migrations/meta/0287_snapshot.json similarity index 97% rename from packages/db/src/migrations/meta/0282_snapshot.json rename to packages/db/src/migrations/meta/0287_snapshot.json index 5c16a75ee3..258c8e4a6a 100644 --- a/packages/db/src/migrations/meta/0282_snapshot.json +++ b/packages/db/src/migrations/meta/0287_snapshot.json @@ -1,6 +1,6 @@ { - "id": "23f8fed6-33cf-43ff-b711-8adb151a4bfc", - "prevId": "8d202a5a-5d6b-44d9-9336-3bb39a8b99bc", + "id": "26beb23c-50c3-42f9-8a1c-814d0c8acfbe", + "prevId": "33120bf3-9cac-4909-ae94-d514ddc1e9e2", "version": "7", "dialect": "postgresql", "tables": { @@ -829,6 +829,479 @@ "checkConstraints": {}, "isRLSEnabled": false }, + "public.agent_instruction_heads": { + "name": "agent_instruction_heads", + "schema": "", + "columns": { + "company_id": { + "name": "company_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "agent_id": { + "name": "agent_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "entry_file": { + "name": "entry_file", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "revision_id": { + "name": "revision_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": { + "agent_instruction_heads_company_id_agent_id_entry_file_revision_id_agent_instruction_revisions_company_id_agent_id_entry_file_id_fk": { + "name": "agent_instruction_heads_company_id_agent_id_entry_file_revision_id_agent_instruction_revisions_company_id_agent_id_entry_file_id_fk", + "tableFrom": "agent_instruction_heads", + "tableTo": "agent_instruction_revisions", + "columnsFrom": [ + "company_id", + "agent_id", + "entry_file", + "revision_id" + ], + "columnsTo": [ + "company_id", + "agent_id", + "entry_file", + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "agent_instruction_heads_identity_uq": { + "name": "agent_instruction_heads_identity_uq", + "nullsNotDistinct": false, + "columns": [ + "company_id", + "agent_id", + "entry_file" + ] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.agent_instruction_revisions": { + "name": "agent_instruction_revisions", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "company_id": { + "name": "company_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "agent_id": { + "name": "agent_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "entry_file": { + "name": "entry_file", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "content_base64": { + "name": "content_base64", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "content_hash": { + "name": "content_hash", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "byte_length": { + "name": "byte_length", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "parent_revision_id": { + "name": "parent_revision_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "base_revision_id": { + "name": "base_revision_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "restored_from_revision_id": { + "name": "restored_from_revision_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "actor_agent_id": { + "name": "actor_agent_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "actor_user_id": { + "name": "actor_user_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "responsible_user_id": { + "name": "responsible_user_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "source_run_id": { + "name": "source_run_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "source": { + "name": "source", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "agent_instruction_revisions_history_idx": { + "name": "agent_instruction_revisions_history_idx", + "columns": [ + { + "expression": "company_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "agent_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "entry_file", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "agent_instruction_revisions_company_id_agent_id_agents_company_id_id_fk": { + "name": "agent_instruction_revisions_company_id_agent_id_agents_company_id_id_fk", + "tableFrom": "agent_instruction_revisions", + "tableTo": "agents", + "columnsFrom": [ + "company_id", + "agent_id" + ], + "columnsTo": [ + "company_id", + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "agent_instruction_revisions_identity_uq": { + "name": "agent_instruction_revisions_identity_uq", + "nullsNotDistinct": false, + "columns": [ + "company_id", + "agent_id", + "entry_file", + "id" + ] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.agent_instruction_working_copies": { + "name": "agent_instruction_working_copies", + "schema": "", + "columns": { + "run_id": { + "name": "run_id", + "type": "uuid", + "primaryKey": true, + "notNull": true + }, + "company_id": { + "name": "company_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "agent_id": { + "name": "agent_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "responsible_user_id": { + "name": "responsible_user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "entry_file": { + "name": "entry_file", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "base_revision_id": { + "name": "base_revision_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "base_hash": { + "name": "base_hash", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "local_root": { + "name": "local_root", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "execution_root": { + "name": "execution_root", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "location": { + "name": "location", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "state": { + "name": "state", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'prepared'" + }, + "candidate_base64": { + "name": "candidate_base64", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "candidate_hash": { + "name": "candidate_hash", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "error_code": { + "name": "error_code", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "error_message": { + "name": "error_message", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "receipt": { + "name": "receipt", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "process_stopped_at": { + "name": "process_stopped_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "attempts": { + "name": "attempts", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "next_attempt_at": { + "name": "next_attempt_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "agent_instruction_copies_pending_idx": { + "name": "agent_instruction_copies_pending_idx", + "columns": [ + { + "expression": "state", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "next_attempt_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "agent_instruction_copies_agent_idx": { + "name": "agent_instruction_copies_agent_idx", + "columns": [ + { + "expression": "company_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "agent_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "agent_instruction_working_copies_run_id_heartbeat_runs_id_fk": { + "name": "agent_instruction_working_copies_run_id_heartbeat_runs_id_fk", + "tableFrom": "agent_instruction_working_copies", + "tableTo": "heartbeat_runs", + "columnsFrom": [ + "run_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "agent_instruction_working_copies_company_id_agent_id_agents_company_id_id_fk": { + "name": "agent_instruction_working_copies_company_id_agent_id_agents_company_id_id_fk", + "tableFrom": "agent_instruction_working_copies", + "tableTo": "agents", + "columnsFrom": [ + "company_id", + "agent_id" + ], + "columnsTo": [ + "company_id", + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, "public.agent_memberships": { "name": "agent_memberships", "schema": "", @@ -2917,7 +3390,7 @@ }, "byte_size": { "name": "byte_size", - "type": "integer", + "type": "bigint", "primaryKey": false, "notNull": true }, @@ -3302,6 +3775,13 @@ "primaryKey": false, "notNull": false }, + "keyboard_shortcuts": { + "name": "keyboard_shortcuts", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, "created_at": { "name": "created_at", "type": "timestamp with time zone", @@ -8109,6 +8589,519 @@ }, "isRLSEnabled": false }, + "public.chat_github_configurations": { + "name": "chat_github_configurations", + "schema": "", + "columns": { + "endpoint_id": { + "name": "endpoint_id", + "type": "uuid", + "primaryKey": true, + "notNull": true + }, + "company_id": { + "name": "company_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "revision": { + "name": "revision", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 1 + }, + "configuration": { + "name": "configuration", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "updated_by_user_id": { + "name": "updated_by_user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": { + "chat_github_configurations_company_id_endpoint_id_chat_endpoints_company_id_id_fk": { + "name": "chat_github_configurations_company_id_endpoint_id_chat_endpoints_company_id_id_fk", + "tableFrom": "chat_github_configurations", + "tableTo": "chat_endpoints", + "columnsFrom": [ + "company_id", + "endpoint_id" + ], + "columnsTo": [ + "company_id", + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "chat_github_config_revision_check": { + "name": "chat_github_config_revision_check", + "value": "\"chat_github_configurations\".\"revision\" > 0" + } + }, + "isRLSEnabled": false + }, + "public.chat_github_registrations": { + "name": "chat_github_registrations", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "company_id": { + "name": "company_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "endpoint_id": { + "name": "endpoint_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "state_hash": { + "name": "state_hash", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "trusted_origin": { + "name": "trusted_origin", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "expires_at": { + "name": "expires_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true + }, + "consumed_at": { + "name": "consumed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "chat_github_registration_state_uq": { + "name": "chat_github_registration_state_uq", + "columns": [ + { + "expression": "state_hash", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "chat_github_registration_endpoint_idx": { + "name": "chat_github_registration_endpoint_idx", + "columns": [ + { + "expression": "endpoint_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "expires_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "chat_github_registrations_company_id_endpoint_id_chat_endpoints_company_id_id_fk": { + "name": "chat_github_registrations_company_id_endpoint_id_chat_endpoints_company_id_id_fk", + "tableFrom": "chat_github_registrations", + "tableTo": "chat_endpoints", + "columnsFrom": [ + "company_id", + "endpoint_id" + ], + "columnsTo": [ + "company_id", + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "chat_github_registration_status_check": { + "name": "chat_github_registration_status_check", + "value": "\"chat_github_registrations\".\"status\" in ('pending', 'exchanging', 'completed', 'failed')" + } + }, + "isRLSEnabled": false + }, + "public.chat_github_reviews": { + "name": "chat_github_reviews", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "company_id": { + "name": "company_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "endpoint_id": { + "name": "endpoint_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "issue_id": { + "name": "issue_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "run_id": { + "name": "run_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "repository_id": { + "name": "repository_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "repository": { + "name": "repository", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "pull_number": { + "name": "pull_number", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "head_sha": { + "name": "head_sha", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "delivery_id": { + "name": "delivery_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "configuration_revision": { + "name": "configuration_revision", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "policy_snapshot": { + "name": "policy_snapshot", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "event": { + "name": "event", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "state": { + "name": "state", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'queued'" + }, + "assessment": { + "name": "assessment", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "conclusion": { + "name": "conclusion", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "check_id": { + "name": "check_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "check_url": { + "name": "check_url", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "summary_id": { + "name": "summary_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "summary_url": { + "name": "summary_url", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "publication_receipts": { + "name": "publication_receipts", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "chat_github_reviews_delivery_uq": { + "name": "chat_github_reviews_delivery_uq", + "columns": [ + { + "expression": "endpoint_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "delivery_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "chat_github_reviews_pull_idx": { + "name": "chat_github_reviews_pull_idx", + "columns": [ + { + "expression": "endpoint_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "repository_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "pull_number", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "chat_github_reviews_task_idx": { + "name": "chat_github_reviews_task_idx", + "columns": [ + { + "expression": "company_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "issue_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "run_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "chat_github_reviews_run_id_heartbeat_runs_id_fk": { + "name": "chat_github_reviews_run_id_heartbeat_runs_id_fk", + "tableFrom": "chat_github_reviews", + "tableTo": "heartbeat_runs", + "columnsFrom": [ + "run_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + }, + "chat_github_reviews_company_id_endpoint_id_chat_endpoints_company_id_id_fk": { + "name": "chat_github_reviews_company_id_endpoint_id_chat_endpoints_company_id_id_fk", + "tableFrom": "chat_github_reviews", + "tableTo": "chat_endpoints", + "columnsFrom": [ + "company_id", + "endpoint_id" + ], + "columnsTo": [ + "company_id", + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "chat_github_reviews_company_id_issue_id_issues_company_id_id_fk": { + "name": "chat_github_reviews_company_id_issue_id_issues_company_id_id_fk", + "tableFrom": "chat_github_reviews", + "tableTo": "issues", + "columnsFrom": [ + "company_id", + "issue_id" + ], + "columnsTo": [ + "company_id", + "id" + ], + "onDelete": "restrict", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "chat_github_reviews_company_id_uq": { + "name": "chat_github_reviews_company_id_uq", + "nullsNotDistinct": false, + "columns": [ + "company_id", + "id" + ] + } + }, + "policies": {}, + "checkConstraints": { + "chat_github_review_pull_number_check": { + "name": "chat_github_review_pull_number_check", + "value": "\"chat_github_reviews\".\"pull_number\" > 0" + }, + "chat_github_review_state_check": { + "name": "chat_github_review_state_check", + "value": "\"chat_github_reviews\".\"state\" in ('queued', 'running', 'completed', 'incomplete', 'error', 'superseded', 'manual_required')" + } + }, + "isRLSEnabled": false + }, "public.chat_teams_file_transfers": { "name": "chat_teams_file_transfers", "schema": "", @@ -38419,6 +39412,113 @@ "checkConstraints": {}, "isRLSEnabled": false }, + "public.runner_api_response_reservations": { + "name": "runner_api_response_reservations", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "company_id": { + "name": "company_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "run_id": { + "name": "run_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "asset_id": { + "name": "asset_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "reserved_bytes": { + "name": "reserved_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "runner_api_response_reservations_company_idx": { + "name": "runner_api_response_reservations_company_idx", + "columns": [ + { + "expression": "company_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "runner_api_response_reservations_company_id_companies_id_fk": { + "name": "runner_api_response_reservations_company_id_companies_id_fk", + "tableFrom": "runner_api_response_reservations", + "tableTo": "companies", + "columnsFrom": [ + "company_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "runner_api_response_reservations_run_id_heartbeat_runs_id_fk": { + "name": "runner_api_response_reservations_run_id_heartbeat_runs_id_fk", + "tableFrom": "runner_api_response_reservations", + "tableTo": "heartbeat_runs", + "columnsFrom": [ + "run_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + }, + "runner_api_response_reservations_asset_id_assets_id_fk": { + "name": "runner_api_response_reservations_asset_id_assets_id_fk", + "tableFrom": "runner_api_response_reservations", + "tableTo": "assets", + "columnsFrom": [ + "asset_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, "public.secret_access_events": { "name": "secret_access_events", "schema": "", diff --git a/packages/db/src/migrations/meta/_journal.json b/packages/db/src/migrations/meta/_journal.json index 264cefe446..f193765264 100644 --- a/packages/db/src/migrations/meta/_journal.json +++ b/packages/db/src/migrations/meta/_journal.json @@ -1996,6 +1996,13 @@ "when": 1790607525758, "tag": "0286_complete_alice", "breakpoints": true + }, + { + "idx": 287, + "version": "7", + "when": 1790611116286, + "tag": "0287_serious_tinkerer", + "breakpoints": true } ] } \ No newline at end of file diff --git a/packages/db/src/schema/agent_instruction_revisions.ts b/packages/db/src/schema/agent_instruction_revisions.ts new file mode 100644 index 0000000000..39db9af723 --- /dev/null +++ b/packages/db/src/schema/agent_instruction_revisions.ts @@ -0,0 +1,119 @@ +import { + pgTable, + uuid, + text, + integer, + timestamp, + index, + unique, + foreignKey, + jsonb, +} from "drizzle-orm/pg-core"; +import { agents } from "./agents.js"; +import { heartbeatRuns } from "./heartbeat_runs.js"; + +/** Append-only content. Base64 preserves every UTF-8 byte, including NUL and BOM. */ +export const agentInstructionRevisions = pgTable( + "agent_instruction_revisions", + { + id: uuid("id").primaryKey().defaultRandom(), + companyId: uuid("company_id").notNull(), + agentId: uuid("agent_id").notNull(), + entryFile: text("entry_file").notNull(), + contentBase64: text("content_base64").notNull(), + contentHash: text("content_hash").notNull(), + byteLength: integer("byte_length").notNull(), + parentRevisionId: uuid("parent_revision_id"), + baseRevisionId: uuid("base_revision_id"), + restoredFromRevisionId: uuid("restored_from_revision_id"), + actorAgentId: uuid("actor_agent_id"), + actorUserId: text("actor_user_id"), + responsibleUserId: text("responsible_user_id"), + sourceRunId: uuid("source_run_id"), + source: text("source").notNull(), + createdAt: timestamp("created_at", { withTimezone: true }) + .notNull() + .defaultNow(), + }, + (t) => ({ + owner: foreignKey({ + columns: [t.companyId, t.agentId], + foreignColumns: [agents.companyId, agents.id], + }).onDelete("cascade"), + identity: unique("agent_instruction_revisions_identity_uq").on( + t.companyId, + t.agentId, + t.entryFile, + t.id, + ), + history: index("agent_instruction_revisions_history_idx").on( + t.companyId, + t.agentId, + t.entryFile, + t.createdAt, + t.id, + ), + }), +); + +export const agentInstructionHeads = pgTable( + "agent_instruction_heads", + { + companyId: uuid("company_id").notNull(), + agentId: uuid("agent_id").notNull(), + entryFile: text("entry_file").notNull(), + revisionId: uuid("revision_id").notNull(), + updatedAt: timestamp("updated_at", { withTimezone: true }) + .notNull() + .defaultNow(), + }, + (t) => ({ + identity: unique("agent_instruction_heads_identity_uq").on( + t.companyId, + t.agentId, + t.entryFile, + ), + revision: foreignKey({ + columns: [t.companyId, t.agentId, t.entryFile, t.revisionId], + foreignColumns: [ + agentInstructionRevisions.companyId, + agentInstructionRevisions.agentId, + agentInstructionRevisions.entryFile, + agentInstructionRevisions.id, + ], + }).onDelete("cascade"), + }), +); + +/** A server-owned collection receipt, retained even when canonical CAS rejects an edit. */ +export const agentInstructionWorkingCopies = pgTable( + "agent_instruction_working_copies", + { + runId: uuid("run_id").primaryKey().references(() => heartbeatRuns.id, { onDelete: "cascade" }), + companyId: uuid("company_id").notNull(), + agentId: uuid("agent_id").notNull(), + responsibleUserId: text("responsible_user_id").notNull(), + entryFile: text("entry_file").notNull(), + baseRevisionId: uuid("base_revision_id"), + baseHash: text("base_hash").notNull(), + localRoot: text("local_root").notNull(), + executionRoot: text("execution_root").notNull(), + location: text("location").notNull(), + state: text("state").notNull().default("prepared"), + candidateBase64: text("candidate_base64"), + candidateHash: text("candidate_hash"), + errorCode: text("error_code"), + errorMessage: text("error_message"), + receipt: jsonb("receipt").$type>(), + processStoppedAt: timestamp("process_stopped_at", { withTimezone: true }), + attempts: integer("attempts").notNull().default(0), + nextAttemptAt: timestamp("next_attempt_at", { withTimezone: true }), + createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(), + updatedAt: timestamp("updated_at", { withTimezone: true }).notNull().defaultNow(), + }, + (t) => ({ + owner: foreignKey({ columns: [t.companyId, t.agentId], foreignColumns: [agents.companyId, agents.id] }).onDelete("cascade"), + pending: index("agent_instruction_copies_pending_idx").on(t.state, t.nextAttemptAt), + history: index("agent_instruction_copies_agent_idx").on(t.companyId, t.agentId, t.createdAt), + }), +); diff --git a/packages/db/src/schema/index.ts b/packages/db/src/schema/index.ts index db17f2aa69..bd007401db 100644 --- a/packages/db/src/schema/index.ts +++ b/packages/db/src/schema/index.ts @@ -211,3 +211,5 @@ export { aiConnectionDefaults } from "./ai_connection_defaults.js"; export { aiProviderDefaults } from "./ai_provider_defaults.js"; export * from "./email.js"; export { announcementDismissals, announcementPublications } from "./announcement_dismissals.js"; + +export { agentInstructionRevisions, agentInstructionHeads, agentInstructionWorkingCopies } from "./agent_instruction_revisions.js"; diff --git a/packages/paperclip-runner/generated/capability/semantic-tool-contracts.json b/packages/paperclip-runner/generated/capability/semantic-tool-contracts.json index 9ca2c4ba8d..a42ac9b66b 100644 --- a/packages/paperclip-runner/generated/capability/semantic-tool-contracts.json +++ b/packages/paperclip-runner/generated/capability/semantic-tool-contracts.json @@ -1 +1 @@ -[{"annotations":{"exposure":"always","operationId":"get_task_context","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read the active task and actor, including the exact approved Markdown revision when this issue has an accepted plan.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"get_task_context","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"get_task_history","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read bounded comments on the active task.","inputSchema":{"additionalProperties":false,"properties":{"limit":{"default":50,"maximum":200,"minimum":1,"type":"integer"}},"required":[],"type":"object"},"name":"get_task_history","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"list_documents","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"List revisioned documents on the active task.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"list_documents","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"read_document","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read the current revision of one active-task document.","inputSchema":{"additionalProperties":false,"properties":{"key":{"description":"Stable issue-document key.","maxLength":120,"minLength":1,"type":"string"}},"required":["key"],"type":"object"},"name":"read_document","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"list_document_revisions","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read bounded revision history for one active-task document.","inputSchema":{"additionalProperties":false,"properties":{"key":{"description":"Stable issue-document key.","maxLength":120,"minLength":1,"type":"string"},"limit":{"default":50,"maximum":200,"minimum":1,"type":"integer"}},"required":["key"],"type":"object"},"name":"list_document_revisions","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"report_progress","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Append a durable progress comment to the active task.","inputSchema":{"additionalProperties":false,"properties":{"body":{"description":"Multiline progress update.","maxLength":20000,"minLength":1,"type":"string"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"}},"required":["idempotencyKey","body"],"type":"object"},"name":"report_progress","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"answer_status_question","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Append the answer to a status-only wake without changing task disposition.","inputSchema":{"additionalProperties":false,"properties":{"body":{"description":"Concise status answer.","maxLength":20000,"minLength":1,"type":"string"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"}},"required":["idempotencyKey","body"],"type":"object"},"name":"answer_status_question","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"write_document","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Create or update an active-task document with optimistic revision safety.","inputSchema":{"additionalProperties":false,"properties":{"baseRevisionId":{"description":"Current revision id, or null when creating.","maxLength":20000,"type":["string","null"]},"body":{"description":"Markdown document body.","maxLength":200000,"minLength":1,"type":"string"},"changeSummary":{"description":"Optional revision summary.","maxLength":20000,"type":["string","null"]},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"},"key":{"description":"Stable issue-document key.","maxLength":120,"minLength":1,"type":"string"},"title":{"description":"Document title.","maxLength":300,"minLength":1,"type":"string"}},"required":["idempotencyKey","key","title","body","baseRevisionId"],"type":"object"},"name":"write_document","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"request_human_input","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Create a durable human question or approval card on the current Paperclip task bound to this run; Paperclip renders it and authenticates the response. Use questions with continuationPolicy='wake_assignee' when an answer is needed, including otherwise tool-free chat turns. Supply a stable idempotencyKey and reuse it on retries. For one question at a time, ask only the next unanswered question and wait for its real answer. Never infer answers, answer your own card, or treat clarification as approval. Preserve existing review gates. Call this tool before claiming a question was asked; if creation fails, report the failure. Do not fabricate answer links or Markdown buttons, post duplicate cards, or substitute call_api. Use payload.questions for choices and payload.questionSet for text fields; see the payload schema for formats.","inputSchema":{"additionalProperties":false,"properties":{"continuationPolicy":{"enum":["none","wake_assignee","wake_assignee_on_accept"]},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"},"interactionKind":{"enum":["confirmation","checkbox","questions","suggest_tasks","item_verdicts"]},"payload":{"additionalProperties":true,"description":"Kind-specific interaction data. For interactionKind='questions', use version:1 and questions:[{id,prompt,selectionMode:'single'|'multi',required?,options:[{id,label,description?,freeText?}]}]. Choice questions need at least two distinct meaningful options. For an open-ended text answer, ALSO include questionSet:{schema:'paperclip.question_set.v1',questions:[{id,prompt,answerMode:'text',required?}]} with no options or customAnswer in its text questions. Keep matching IDs/prompts in both arrays; the required compatibility questions entry uses selectionMode:'single' and options:[{id:'describe',label:'Your answer',freeText:true}]. Without questionSet this incorrectly renders as a one-option choice. Never use a lone Other or describe option as the presentation. Option keys are id/label, not value. For confirmation, payload may be {}. Keep IDs stable across retries.","type":"object"},"prompt":{"description":"Question or decision prompt.","maxLength":10000,"minLength":1,"type":"string"},"targetRevisionId":{"description":"Optional bound document revision.","maxLength":20000,"type":["string","null"]},"title":{"description":"Interaction card title.","maxLength":300,"minLength":1,"type":"string"}},"required":["idempotencyKey","interactionKind","title","prompt","continuationPolicy"],"type":"object"},"name":"request_human_input","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"register_deliverable","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Register attachment metadata and its artifact work product without credentials or bytes in the tool result.","inputSchema":{"additionalProperties":false,"properties":{"byteSize":{"maximum":100000000,"minimum":0,"type":"integer"},"contentRef":{"description":"Opaque package-local content reference.","maxLength":2000,"minLength":1,"type":"string"},"contentType":{"description":"Media type.","maxLength":200,"minLength":1,"type":"string"},"filename":{"description":"Display filename.","maxLength":500,"minLength":1,"type":"string"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"},"sha256":{"pattern":"^[a-fA-F0-9]{64}$","type":"string"},"title":{"description":"Work-product title.","maxLength":500,"minLength":1,"type":"string"}},"required":["idempotencyKey","filename","contentType","byteSize","sha256","contentRef","title"],"type":"object"},"name":"register_deliverable","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"finish_task","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Finish the active task with a durable summary.","inputSchema":{"additionalProperties":false,"properties":{"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"},"summary":{"description":"Completion summary.","maxLength":20000,"minLength":1,"type":"string"}},"required":["idempotencyKey","summary"],"type":"object"},"name":"finish_task","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"block_task","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Block the active task with a durable reason and optional first-class dependencies.","inputSchema":{"additionalProperties":false,"properties":{"blockedByTaskIds":{"description":"Internal task ids that block this task.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"},"reason":{"description":"Block reason.","maxLength":20000,"minLength":1,"type":"string"}},"required":["idempotencyKey","reason"],"type":"object"},"name":"block_task","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"request_review","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Move the active task to review with a durable summary.","inputSchema":{"additionalProperties":false,"properties":{"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"},"summary":{"description":"Review handoff summary.","maxLength":20000,"minLength":1,"type":"string"}},"required":["idempotencyKey","summary"],"type":"object"},"name":"request_review","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"optional","operationId":"list_agents","requiredClaims":["discovery:agents:read"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"List redacted actor profiles.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"list_agents","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"get_agent","requiredClaims":["discovery:agents:read"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read one redacted actor profile.","inputSchema":{"additionalProperties":false,"properties":{"actorId":{"description":"Actor id.","maxLength":200,"minLength":1,"type":"string"}},"required":["actorId"],"type":"object"},"name":"get_agent","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"hire_agent","requiredClaims":["delegation:agents:create"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Create a persistent native Runner teammate with an identity and persona. The teammate reports to the caller and inherits the caller's native runtime; provider, adapter, environment, and credential settings are selected by Paperclip and are never caller-supplied here. Reuse a suitable teammate from list_agents when possible.","inputSchema":{"additionalProperties":false,"properties":{"capabilities":{"maxLength":2000,"type":["string","null"]},"instructions":{"maxLength":20000,"type":["string","null"]},"name":{"maxLength":200,"minLength":1,"type":"string"},"role":{"enum":["ceo","cto","cmo","cfo","security","engineer","designer","pm","qa","devops","researcher","general"],"type":"string"},"title":{"maxLength":300,"type":["string","null"]}},"required":["name"],"type":"object"},"name":"hire_agent","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"search_tasks","requiredClaims":["discovery:tasks:read"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Search tasks by text and status within the run company.","inputSchema":{"additionalProperties":false,"properties":{"limit":{"default":50,"maximum":200,"minimum":1,"type":"integer"},"query":{"maxLength":500,"type":"string"},"statuses":{"items":{"enum":["backlog","todo","in_progress","in_review","done","blocked","cancelled"]},"maxItems":7,"type":"array","uniqueItems":true}},"required":[],"type":"object"},"name":"search_tasks","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"list_approvals","requiredClaims":["governance:approvals:read"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"List approvals in the run company.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"list_approvals","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"get_approval","requiredClaims":["governance:approvals:read"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read one approval without protected data.","inputSchema":{"additionalProperties":false,"properties":{"approvalId":{"description":"Approval id.","maxLength":200,"minLength":1,"type":"string"}},"required":["approvalId"],"type":"object"},"name":"get_approval","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"get_approval_context","requiredClaims":["governance:approvals:read"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read one approval, its comments, and linked tasks.","inputSchema":{"additionalProperties":false,"properties":{"approvalId":{"description":"Approval id.","maxLength":200,"minLength":1,"type":"string"}},"required":["approvalId"],"type":"object"},"name":"get_approval_context","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"get_workspace_runtime","requiredClaims":["workspace:read"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read active-task workspace services.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"get_workspace_runtime","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"control_workspace_service","requiredClaims":["workspace:control"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Start, stop, or fault one active-task workspace service.","inputSchema":{"additionalProperties":false,"properties":{"action":{"enum":["start","stop","fail"]},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"},"serviceId":{"description":"Workspace service id.","maxLength":200,"minLength":1,"type":"string"},"url":{"description":"Optional service URL.","maxLength":20000,"type":["string","null"]}},"required":["idempotencyKey","serviceId","action"],"type":"object"},"name":"control_workspace_service","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"optional","operationId":"set_dependencies","requiredClaims":["dependencies:write"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Replace the active task's first-class blocker set.","inputSchema":{"additionalProperties":false,"properties":{"blockedByTaskIds":{"description":"Replacement blocker task ids.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"}},"required":["idempotencyKey","blockedByTaskIds"],"type":"object"},"name":"set_dependencies","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"optional","operationId":"create_task","requiredClaims":["delegation:tasks:create"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Create a project task from a conversation, or a child from an ordinary task. Persist initialPlan before execution. Set status to backlog when the user wants to save or plan work without starting it; backlog tasks never wake an agent. Omitted status means todo, subject to blockers.","inputSchema":{"additionalProperties":false,"properties":{"assigneeActorId":{"description":"Optional agent assignee. Omit to assign the current agent.","maxLength":20000,"type":["string","null"]},"blockedByTaskIds":{"description":"Initial blocker task ids.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"description":{"description":"Child task description.","maxLength":20000,"type":["string","null"]},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"},"initialPlan":{"description":"Relevant markdown plan saved on the new task before execution starts.","maxLength":200000,"type":["string","null"]},"priority":{"enum":["critical","high","medium","low"]},"projectId":{"description":"Project ID for the task.","type":["string","null"]},"status":{"description":"Initial status. Use backlog to save work without executing it. Defaults to todo (blocked when dependencies are unresolved).","enum":["backlog","todo"]},"title":{"description":"Child task title.","maxLength":500,"minLength":1,"type":"string"}},"required":["idempotencyKey","title"],"type":"object"},"name":"create_task","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"},"task":{"additionalProperties":false,"properties":{"assigneeActorId":{"type":["string","null"]},"id":{"minLength":1,"type":"string"},"identifier":{"type":["string","null"]},"parentId":{"minLength":1,"type":["string","null"]},"projectId":{"minLength":1,"type":["string","null"]},"status":{"minLength":1,"type":"string"}},"required":["id","identifier","parentId","status","assigneeActorId"],"type":"object"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds","task"],"type":"object"}},{"annotations":{"exposure":"optional","operationId":"request_approval","requiredClaims":["governance:approvals:request"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Create a governed approval and waiting posture.","inputSchema":{"additionalProperties":false,"properties":{"approvalType":{"description":"Stable approval type.","maxLength":200,"minLength":1,"type":"string"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"},"payload":{"additionalProperties":true,"type":"object"}},"required":["idempotencyKey","approvalType","payload"],"type":"object"},"name":"request_approval","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"optional","operationId":"decide_approval","requiredClaims":["governance:approvals:decide"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Decide an approval as an explicitly authorized approver.","inputSchema":{"additionalProperties":false,"properties":{"approvalId":{"description":"Approval id.","maxLength":200,"minLength":1,"type":"string"},"decision":{"enum":["approved","rejected","cancelled"]},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"},"note":{"description":"Decision note.","maxLength":20000,"minLength":1,"type":"string"}},"required":["idempotencyKey","approvalId","decision","note"],"type":"object"},"name":"decide_approval","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"optional","operationId":"comment_on_approval","requiredClaims":["governance:approvals:comment"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Add a durable comment to an approval.","inputSchema":{"additionalProperties":false,"properties":{"approvalId":{"description":"Approval id.","maxLength":200,"minLength":1,"type":"string"},"body":{"description":"Approval comment.","maxLength":20000,"minLength":1,"type":"string"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"}},"required":["idempotencyKey","approvalId","body"],"type":"object"},"name":"comment_on_approval","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"optional","operationId":"schedule_wake","requiredClaims":["control_plane:wakes"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Schedule a deterministic continuation wake.","inputSchema":{"additionalProperties":false,"properties":{"delayTicks":{"maximum":10000,"minimum":1,"type":"integer"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"},"payload":{"additionalProperties":true,"type":"object"},"reason":{"enum":["manual","issue_commented","interaction_resolved","approval_resolved","blockers_resolved","scheduled_retry","resume"]}},"required":["idempotencyKey","reason","delayTicks"],"type":"object"},"name":"schedule_wake","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"optional","operationId":"generic_api_request","requiredClaims":["test:generic_api_request"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Test-only escape hatch. Disabled unless the scenario and explicit claim both enable it.","inputSchema":{"additionalProperties":false,"properties":{"body":{"additionalProperties":true,"type":"object"},"method":{"enum":["GET","POST","PATCH"]},"path":{"maxLength":500,"pattern":"^/mock/","type":"string"}},"required":["method","path"],"type":"object"},"name":"generic_api_request","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"search_api","requiredClaims":["api:discover"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Fallback only: discover Paperclip API operations when the available dedicated tools cannot express the task. Prefer dedicated tools for common operations; do not search before using them. For a persistent hire, first list_agents to reuse a suitable teammate. Search agent-hires for the hiring schema and agent-configurations for compatible adapter/runtime settings, then use call_api with the returned operationId. Supply role-specific instructionsBundle.files as a filename-to-content record. Use the returned agent.id for delegation and obey any pending approval. Provider helper threads are temporary workers, not Paperclip hires. If a hire response is uncertain, reconcile with list_agents before retrying.","inputSchema":{"additionalProperties":false,"properties":{"cursor":{"maxLength":200,"type":"string"},"limit":{"default":5,"maximum":10,"minimum":1,"type":"integer"},"query":{"maxLength":500,"minLength":1,"type":"string"}},"required":["query"],"type":"object"},"name":"search_api","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"call_api","requiredClaims":["api:call"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Fallback only: call a discovered Paperclip API operation when dedicated tools lack the required operation or parameters. Uses your existing permissions. Prefer dedicated tools; never bypass a denial or runner lifecycle tool. For large text responses, read the returned artifact with GET /api/assets/{assetId}/content and responseText; follow nextOffsetBytes until null.","inputSchema":{"additionalProperties":false,"properties":{"body":{"anyOf":[{"additionalProperties":true,"type":"object"},{"items":{},"type":"array"},{"type":"string"},{"type":"number"},{"type":"boolean"},{"type":"null"}],"description":"Request value matching the discovered schema. For JSON object or array requests, pass the object or array directly, never a JSON-encoded string. Strings are for text bodies or endpoints whose schema explicitly accepts a string."},"contentType":{"maxLength":120,"type":"string"},"files":{"items":{"additionalProperties":false,"oneOf":[{"properties":{"artifactId":{}},"required":["artifactId"]},{"properties":{"path":{}},"required":["path"]}],"properties":{"artifactId":{"type":"string"},"field":{"type":"string"},"path":{"description":"File relative to the active issue workspace. Remote files must first be uploaded as an artifact.","type":"string"}},"type":"object"},"maxItems":10,"type":"array"},"operationId":{"description":"Exact operationId returned by search_api, for example GET /api/projects/{id}. Do not guess identifiers.","maxLength":500,"minLength":1,"type":"string"},"pathParams":{"additionalProperties":{"type":"string"},"type":"object"},"query":{"additionalProperties":true,"type":"object"},"responseText":{"additionalProperties":false,"description":"GET only: return a bounded UTF-8 text window inline, including JSON as text, without saving another artifact. Offsets and limits are bytes. Use the returned nextOffsetBytes to continue; null means complete. Prefer reading a saved artifact for a stable snapshot. New live responses have a 1 GiB capture limit and a 4 GiB per-run capture budget. Existing larger assets remain readable in bounded pages. If a live response returns an artifact, continue on its content operation for a stable snapshot.","properties":{"limitBytes":{"default":24576,"maximum":24576,"minimum":4,"type":"integer"},"offsetBytes":{"default":0,"maximum":9007199254740991,"minimum":0,"type":"integer"}},"type":"object"}},"required":["operationId"],"type":"object"},"name":"call_api","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"create_project","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Create a project after considering existing projects and available repositories. repositoryIds and repositoryUrls accept multiple existing repositories. Use HTTPS GitHub repositoryUrls when an accessible repo is not in the catalog; this registers project repositories, not remote GitHub repositories. Non-code projects may omit repositories. Cannot combine repositoryIds/repositoryUrls with workspace. Reuse the idempotency key on retries.","inputSchema":{"additionalProperties":false,"properties":{"archivedAt":{"description":"Archive timestamp.","maxLength":20000,"type":["string","null"]},"color":{"description":"Project color.","maxLength":20000,"type":["string","null"]},"description":{"description":"Project outcome and context.","maxLength":20000,"type":["string","null"]},"env":{"additionalProperties":true,"type":"object"},"executionWorkspacePolicy":{"additionalProperties":true,"type":"object"},"goalId":{"description":"Goal ID.","maxLength":20000,"type":["string","null"]},"goalIds":{"description":"Goal IDs.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"icon":{"description":"Project icon.","enum":["folder","rocket","code","terminal","database","globe","package","boxes","box","layers","briefcase","compass","target","flame","zap","star","bug","wrench","hammer","lightbulb","sparkles","shield","lock","search","cog","brain","cpu","git-branch","file-code","puzzle","gem","atom","heart","mail","message-square","crown","radar","telescope","hexagon",null],"type":["string","null"]},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"},"leadAgentId":{"description":"Lead agent ID.","maxLength":20000,"type":["string","null"]},"name":{"description":"Project name.","maxLength":500,"minLength":1,"type":"string"},"repositoryIds":{"description":"Authorized repository IDs from list_project_repositories; may contain multiple repositories.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"repositoryUrls":{"description":"Existing HTTPS GitHub repository URLs, including repos absent from the catalog.","items":{"maxLength":2000,"pattern":"^https://github\\.com/(?!\\.{1,2}/)[A-Za-z0-9_.-]+/(?!\\.{1,2}/?$)[A-Za-z0-9_.-]+/?$","type":"string"},"maxItems":100,"type":"array"},"status":{"enum":["backlog","planned","in_progress","completed","cancelled"]},"targetDate":{"description":"Target date.","maxLength":20000,"type":["string","null"]},"workspace":{"additionalProperties":true,"type":"object"}},"required":["idempotencyKey","name"],"type":"object"},"name":"create_project","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"list_project_repositories","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"List authorized repositories with stable IDs and names. Consider appropriate repositories before creating a project; never invent IDs.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"list_project_repositories","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"list_projects","requiredClaims":["discovery:projects:read"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Inspect available company projects before selecting a project for new work.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"list_projects","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"create_skill","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Create a reusable single-file skill in the company library. Supply a complete SKILL.md whose name and description match the inputs. This saves the skill and shows a card; it does not assign the skill to any agent. Reuse idempotencyKey on retries.","inputSchema":{"additionalProperties":false,"properties":{"description":{"maxLength":2000,"minLength":1,"type":"string"},"idempotencyKey":{"maxLength":240,"minLength":1,"type":"string"},"markdown":{"description":"Complete SKILL.md including name and description frontmatter and substantive instructions.","maxLength":200000,"minLength":1,"type":"string"},"name":{"description":"Lowercase skill name, matching SKILL.md frontmatter.","maxLength":120,"minLength":1,"pattern":"^[a-z0-9]+(?:-[a-z0-9]+)*$","type":"string"},"slug":{"description":"Optional; must equal name.","maxLength":120,"minLength":1,"pattern":"^[a-z0-9]+(?:-[a-z0-9]+)*$","type":"string"}},"required":["idempotencyKey","name","description","markdown"],"type":"object"},"name":"create_skill","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"reassign_task","requiredClaims":["delegation:tasks:assign"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Reassign an existing task to another company agent. Read search_tasks first and supply its current assignee and statusVersion to prevent overwriting a concurrent change. Preserve the task, documents, dependencies, and blocked/backlog status. Active work is stopped before handoff. Use a stable idempotency key for retries. Cannot reassign this run’s own task, conversations, completed tasks, or pending reviews; use create_task for delegation from the current task.","inputSchema":{"additionalProperties":false,"properties":{"assigneeActorId":{"description":"New company agent ID from list_agents.","minLength":1,"type":"string"},"expectedAssigneeActorId":{"description":"Current assigneeAgentId from search_tasks; null means unassigned.","type":["string","null"]},"expectedStatusVersion":{"description":"Current statusVersion from search_tasks.","minimum":0,"type":"integer"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"},"reason":{"description":"Why the task should move and context for the new owner.","maxLength":20000,"minLength":1,"type":"string"},"taskId":{"description":"Existing task ID from search_tasks.","minLength":1,"type":"string"}},"required":["idempotencyKey","taskId","assigneeActorId","expectedAssigneeActorId","expectedStatusVersion","reason"],"type":"object"},"name":"reassign_task","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}}] +[{"annotations":{"exposure":"always","operationId":"get_task_context","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read the active task and actor, including the exact approved Markdown revision when this issue has an accepted plan.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"get_task_context","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"get_task_history","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read bounded comments on the active task.","inputSchema":{"additionalProperties":false,"properties":{"limit":{"default":50,"maximum":200,"minimum":1,"type":"integer"}},"required":[],"type":"object"},"name":"get_task_history","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"list_documents","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"List revisioned documents on the active task.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"list_documents","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"read_document","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read the current revision of one active-task document.","inputSchema":{"additionalProperties":false,"properties":{"key":{"description":"Stable issue-document key.","maxLength":120,"minLength":1,"type":"string"}},"required":["key"],"type":"object"},"name":"read_document","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"list_document_revisions","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read bounded revision history for one active-task document.","inputSchema":{"additionalProperties":false,"properties":{"key":{"description":"Stable issue-document key.","maxLength":120,"minLength":1,"type":"string"},"limit":{"default":50,"maximum":200,"minimum":1,"type":"integer"}},"required":["key"],"type":"object"},"name":"list_document_revisions","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"report_progress","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Append a durable progress comment to the active task.","inputSchema":{"additionalProperties":false,"properties":{"body":{"description":"Multiline progress update.","maxLength":20000,"minLength":1,"type":"string"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"}},"required":["idempotencyKey","body"],"type":"object"},"name":"report_progress","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"answer_status_question","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Append the answer to a status-only wake without changing task disposition.","inputSchema":{"additionalProperties":false,"properties":{"body":{"description":"Concise status answer.","maxLength":20000,"minLength":1,"type":"string"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"}},"required":["idempotencyKey","body"],"type":"object"},"name":"answer_status_question","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"write_document","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Create or update an active-task document with optimistic revision safety.","inputSchema":{"additionalProperties":false,"properties":{"baseRevisionId":{"description":"Current revision id, or null when creating.","maxLength":20000,"type":["string","null"]},"body":{"description":"Markdown document body.","maxLength":200000,"minLength":1,"type":"string"},"changeSummary":{"description":"Optional revision summary.","maxLength":20000,"type":["string","null"]},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"},"key":{"description":"Stable issue-document key.","maxLength":120,"minLength":1,"type":"string"},"title":{"description":"Document title.","maxLength":300,"minLength":1,"type":"string"}},"required":["idempotencyKey","key","title","body","baseRevisionId"],"type":"object"},"name":"write_document","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"request_human_input","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Create a durable human question or approval card on the current Paperclip task bound to this run; Paperclip renders it and authenticates the response. Use questions with continuationPolicy='wake_assignee' when an answer is needed, including otherwise tool-free chat turns. Supply a stable idempotencyKey and reuse it on retries. For one question at a time, ask only the next unanswered question and wait for its real answer. Never infer answers, answer your own card, or treat clarification as approval. Preserve existing review gates. Call this tool before claiming a question was asked; if creation fails, report the failure. Do not fabricate answer links or Markdown buttons, post duplicate cards, or substitute call_api. Use payload.questions for choices and payload.questionSet for text fields; see the payload schema for formats.","inputSchema":{"additionalProperties":false,"properties":{"continuationPolicy":{"enum":["none","wake_assignee","wake_assignee_on_accept"]},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"},"interactionKind":{"enum":["confirmation","checkbox","questions","suggest_tasks","item_verdicts"]},"payload":{"additionalProperties":true,"description":"Kind-specific interaction data. For interactionKind='questions', use version:1 and questions:[{id,prompt,selectionMode:'single'|'multi',required?,options:[{id,label,description?,freeText?}]}]. Choice questions need at least two distinct meaningful options. For an open-ended text answer, ALSO include questionSet:{schema:'paperclip.question_set.v1',questions:[{id,prompt,answerMode:'text',required?}]} with no options or customAnswer in its text questions. Keep matching IDs/prompts in both arrays; the required compatibility questions entry uses selectionMode:'single' and options:[{id:'describe',label:'Your answer',freeText:true}]. Without questionSet this incorrectly renders as a one-option choice. Never use a lone Other or describe option as the presentation. Option keys are id/label, not value. For confirmation, payload may be {}. Keep IDs stable across retries.","type":"object"},"prompt":{"description":"Question or decision prompt.","maxLength":10000,"minLength":1,"type":"string"},"targetRevisionId":{"description":"Optional bound document revision.","maxLength":20000,"type":["string","null"]},"title":{"description":"Interaction card title.","maxLength":300,"minLength":1,"type":"string"}},"required":["idempotencyKey","interactionKind","title","prompt","continuationPolicy"],"type":"object"},"name":"request_human_input","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"register_deliverable","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Register attachment metadata and its artifact work product without credentials or bytes in the tool result.","inputSchema":{"additionalProperties":false,"properties":{"byteSize":{"maximum":100000000,"minimum":0,"type":"integer"},"contentRef":{"description":"Opaque package-local content reference.","maxLength":2000,"minLength":1,"type":"string"},"contentType":{"description":"Media type.","maxLength":200,"minLength":1,"type":"string"},"filename":{"description":"Display filename.","maxLength":500,"minLength":1,"type":"string"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"},"sha256":{"pattern":"^[a-fA-F0-9]{64}$","type":"string"},"title":{"description":"Work-product title.","maxLength":500,"minLength":1,"type":"string"}},"required":["idempotencyKey","filename","contentType","byteSize","sha256","contentRef","title"],"type":"object"},"name":"register_deliverable","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"finish_task","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Finish the active task with a durable summary.","inputSchema":{"additionalProperties":false,"properties":{"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"},"summary":{"description":"Completion summary.","maxLength":20000,"minLength":1,"type":"string"}},"required":["idempotencyKey","summary"],"type":"object"},"name":"finish_task","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"block_task","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Block the active task with a durable reason and optional first-class dependencies.","inputSchema":{"additionalProperties":false,"properties":{"blockedByTaskIds":{"description":"Internal task ids that block this task.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"},"reason":{"description":"Block reason.","maxLength":20000,"minLength":1,"type":"string"}},"required":["idempotencyKey","reason"],"type":"object"},"name":"block_task","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"request_review","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Move the active task to review with a durable summary.","inputSchema":{"additionalProperties":false,"properties":{"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"},"summary":{"description":"Review handoff summary.","maxLength":20000,"minLength":1,"type":"string"}},"required":["idempotencyKey","summary"],"type":"object"},"name":"request_review","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"optional","operationId":"list_agents","requiredClaims":["discovery:agents:read"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"List redacted actor profiles.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"list_agents","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"get_agent","requiredClaims":["discovery:agents:read"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read one redacted actor profile.","inputSchema":{"additionalProperties":false,"properties":{"actorId":{"description":"Actor id.","maxLength":200,"minLength":1,"type":"string"}},"required":["actorId"],"type":"object"},"name":"get_agent","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"hire_agent","requiredClaims":["delegation:agents:create"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Create a persistent native Runner teammate with an identity and persona. The teammate reports to the caller and inherits the caller's native runtime; provider, adapter, environment, and credential settings are selected by Paperclip and are never caller-supplied here. Reuse a suitable teammate from list_agents when possible.","inputSchema":{"additionalProperties":false,"properties":{"capabilities":{"maxLength":2000,"type":["string","null"]},"instructions":{"maxLength":20000,"type":["string","null"]},"name":{"maxLength":200,"minLength":1,"type":"string"},"role":{"enum":["ceo","cto","cmo","cfo","security","engineer","designer","pm","qa","devops","researcher","general"],"type":"string"},"title":{"maxLength":300,"type":["string","null"]}},"required":["name"],"type":"object"},"name":"hire_agent","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"search_tasks","requiredClaims":["discovery:tasks:read"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Search tasks by text and status within the run company.","inputSchema":{"additionalProperties":false,"properties":{"limit":{"default":50,"maximum":200,"minimum":1,"type":"integer"},"query":{"maxLength":500,"type":"string"},"statuses":{"items":{"enum":["backlog","todo","in_progress","in_review","done","blocked","cancelled"]},"maxItems":7,"type":"array","uniqueItems":true}},"required":[],"type":"object"},"name":"search_tasks","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"list_approvals","requiredClaims":["governance:approvals:read"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"List approvals in the run company.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"list_approvals","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"get_approval","requiredClaims":["governance:approvals:read"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read one approval without protected data.","inputSchema":{"additionalProperties":false,"properties":{"approvalId":{"description":"Approval id.","maxLength":200,"minLength":1,"type":"string"}},"required":["approvalId"],"type":"object"},"name":"get_approval","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"get_approval_context","requiredClaims":["governance:approvals:read"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read one approval, its comments, and linked tasks.","inputSchema":{"additionalProperties":false,"properties":{"approvalId":{"description":"Approval id.","maxLength":200,"minLength":1,"type":"string"}},"required":["approvalId"],"type":"object"},"name":"get_approval_context","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"get_workspace_runtime","requiredClaims":["workspace:read"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read active-task workspace services.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"get_workspace_runtime","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"control_workspace_service","requiredClaims":["workspace:control"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Start, stop, or fault one active-task workspace service.","inputSchema":{"additionalProperties":false,"properties":{"action":{"enum":["start","stop","fail"]},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"},"serviceId":{"description":"Workspace service id.","maxLength":200,"minLength":1,"type":"string"},"url":{"description":"Optional service URL.","maxLength":20000,"type":["string","null"]}},"required":["idempotencyKey","serviceId","action"],"type":"object"},"name":"control_workspace_service","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"optional","operationId":"set_dependencies","requiredClaims":["dependencies:write"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Replace the active task's first-class blocker set.","inputSchema":{"additionalProperties":false,"properties":{"blockedByTaskIds":{"description":"Replacement blocker task ids.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"}},"required":["idempotencyKey","blockedByTaskIds"],"type":"object"},"name":"set_dependencies","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"optional","operationId":"create_task","requiredClaims":["delegation:tasks:create"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Create a project task from a conversation, or a child from an ordinary task. Persist initialPlan before execution. Set status to backlog when the user wants to save or plan work without starting it; backlog tasks never wake an agent. Omitted status means todo, subject to blockers.","inputSchema":{"additionalProperties":false,"properties":{"assigneeActorId":{"description":"Optional agent assignee. Omit to assign the current agent.","maxLength":20000,"type":["string","null"]},"blockedByTaskIds":{"description":"Initial blocker task ids.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"description":{"description":"Child task description.","maxLength":20000,"type":["string","null"]},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"},"initialPlan":{"description":"Relevant markdown plan saved on the new task before execution starts.","maxLength":200000,"type":["string","null"]},"priority":{"enum":["critical","high","medium","low"]},"projectId":{"description":"Project ID for the task.","type":["string","null"]},"status":{"description":"Initial status. Use backlog to save work without executing it. Defaults to todo (blocked when dependencies are unresolved).","enum":["backlog","todo"]},"title":{"description":"Child task title.","maxLength":500,"minLength":1,"type":"string"}},"required":["idempotencyKey","title"],"type":"object"},"name":"create_task","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"},"task":{"additionalProperties":false,"properties":{"assigneeActorId":{"type":["string","null"]},"id":{"minLength":1,"type":"string"},"identifier":{"type":["string","null"]},"parentId":{"minLength":1,"type":["string","null"]},"projectId":{"minLength":1,"type":["string","null"]},"status":{"minLength":1,"type":"string"}},"required":["id","identifier","parentId","status","assigneeActorId"],"type":"object"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds","task"],"type":"object"}},{"annotations":{"exposure":"optional","operationId":"request_approval","requiredClaims":["governance:approvals:request"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Create a governed approval and waiting posture.","inputSchema":{"additionalProperties":false,"properties":{"approvalType":{"description":"Stable approval type.","maxLength":200,"minLength":1,"type":"string"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"},"payload":{"additionalProperties":true,"type":"object"}},"required":["idempotencyKey","approvalType","payload"],"type":"object"},"name":"request_approval","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"optional","operationId":"decide_approval","requiredClaims":["governance:approvals:decide"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Decide an approval as an explicitly authorized approver.","inputSchema":{"additionalProperties":false,"properties":{"approvalId":{"description":"Approval id.","maxLength":200,"minLength":1,"type":"string"},"decision":{"enum":["approved","rejected","cancelled"]},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"},"note":{"description":"Decision note.","maxLength":20000,"minLength":1,"type":"string"}},"required":["idempotencyKey","approvalId","decision","note"],"type":"object"},"name":"decide_approval","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"optional","operationId":"comment_on_approval","requiredClaims":["governance:approvals:comment"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Add a durable comment to an approval.","inputSchema":{"additionalProperties":false,"properties":{"approvalId":{"description":"Approval id.","maxLength":200,"minLength":1,"type":"string"},"body":{"description":"Approval comment.","maxLength":20000,"minLength":1,"type":"string"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"}},"required":["idempotencyKey","approvalId","body"],"type":"object"},"name":"comment_on_approval","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"optional","operationId":"schedule_wake","requiredClaims":["control_plane:wakes"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Schedule a deterministic continuation wake.","inputSchema":{"additionalProperties":false,"properties":{"delayTicks":{"maximum":10000,"minimum":1,"type":"integer"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"},"payload":{"additionalProperties":true,"type":"object"},"reason":{"enum":["manual","issue_commented","interaction_resolved","approval_resolved","blockers_resolved","scheduled_retry","resume"]}},"required":["idempotencyKey","reason","delayTicks"],"type":"object"},"name":"schedule_wake","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"optional","operationId":"generic_api_request","requiredClaims":["test:generic_api_request"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Test-only escape hatch. Disabled unless the scenario and explicit claim both enable it.","inputSchema":{"additionalProperties":false,"properties":{"body":{"additionalProperties":true,"type":"object"},"method":{"enum":["GET","POST","PATCH"]},"path":{"maxLength":500,"pattern":"^/mock/","type":"string"}},"required":["method","path"],"type":"object"},"name":"generic_api_request","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"read_agent_instructions","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read the current canonical instruction entry and its revision, or inspect a historical revision by supplying both entryFile and revisionId. Read before updating; do not edit shared instruction caches.","inputSchema":{"additionalProperties":false,"properties":{"entryFile":{"description":"Configured relative entry filename returned by read_agent_instructions; retain it with the revision.","maxLength":4096,"minLength":1,"type":"string"},"revisionId":{"description":"Historical revision to inspect; also provide its entryFile.","pattern":"^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$","type":"string"},"targetAgentId":{"description":"Same-company target agent. Omit to use the calling agent.","pattern":"^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$","type":"string"}},"required":[],"type":"object"},"name":"read_agent_instructions","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"search_api","requiredClaims":["api:discover"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Fallback only: discover Paperclip API operations when the available dedicated tools cannot express the task. Prefer dedicated tools for common operations; do not search before using them. For a persistent hire, first list_agents to reuse a suitable teammate. Search agent-hires for the hiring schema and agent-configurations for compatible adapter/runtime settings, then use call_api with the returned operationId. Supply role-specific instructionsBundle.files as a filename-to-content record. Use the returned agent.id for delegation and obey any pending approval. Provider helper threads are temporary workers, not Paperclip hires. If a hire response is uncertain, reconcile with list_agents before retrying.","inputSchema":{"additionalProperties":false,"properties":{"cursor":{"maxLength":200,"type":"string"},"limit":{"default":5,"maximum":10,"minimum":1,"type":"integer"},"query":{"maxLength":500,"minLength":1,"type":"string"}},"required":["query"],"type":"object"},"name":"search_api","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"update_agent_instructions","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Commit instruction content with the exact entryFile and baseRevisionId from a prior read. Requires the responsible user’s current target edit permission. On conflict, preserve your candidate and explicitly resolve it; never overwrite the newer head.","inputSchema":{"additionalProperties":false,"properties":{"baseRevisionId":{"description":"Revision read before editing; null only when no canonical entry exists. Never replace a stale base silently.","pattern":"^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$","type":["string","null"]},"content":{"description":"Complete UTF-8 instruction content, at most 1 MiB; empty content is valid.","maxLength":1048576,"type":"string"},"entryFile":{"description":"Configured relative entry filename returned by read_agent_instructions; retain it with the revision.","maxLength":4096,"minLength":1,"type":"string"},"targetAgentId":{"description":"Same-company target agent. Omit to use the calling agent.","pattern":"^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$","type":"string"}},"required":["entryFile","content","baseRevisionId"],"type":"object"},"name":"update_agent_instructions","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"call_api","requiredClaims":["api:call"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Fallback only: call a discovered Paperclip API operation when dedicated tools lack the required operation or parameters. Uses your existing permissions. Prefer dedicated tools; never bypass a denial or runner lifecycle tool. For large text responses, read the returned artifact with GET /api/assets/{assetId}/content and responseText; follow nextOffsetBytes until null.","inputSchema":{"additionalProperties":false,"properties":{"body":{"anyOf":[{"additionalProperties":true,"type":"object"},{"items":{},"type":"array"},{"type":"string"},{"type":"number"},{"type":"boolean"},{"type":"null"}],"description":"Request value matching the discovered schema. For JSON object or array requests, pass the object or array directly, never a JSON-encoded string. Strings are for text bodies or endpoints whose schema explicitly accepts a string."},"contentType":{"maxLength":120,"type":"string"},"files":{"items":{"additionalProperties":false,"oneOf":[{"properties":{"artifactId":{}},"required":["artifactId"]},{"properties":{"path":{}},"required":["path"]}],"properties":{"artifactId":{"type":"string"},"field":{"type":"string"},"path":{"description":"File relative to the active issue workspace. Remote files must first be uploaded as an artifact.","type":"string"}},"type":"object"},"maxItems":10,"type":"array"},"operationId":{"description":"Exact operationId returned by search_api, for example GET /api/projects/{id}. Do not guess identifiers.","maxLength":500,"minLength":1,"type":"string"},"pathParams":{"additionalProperties":{"type":"string"},"type":"object"},"query":{"additionalProperties":true,"type":"object"},"responseText":{"additionalProperties":false,"description":"GET only: return a bounded UTF-8 text window inline, including JSON as text, without saving another artifact. Offsets and limits are bytes. Use the returned nextOffsetBytes to continue; null means complete. Prefer reading a saved artifact for a stable snapshot. New live responses have a 1 GiB capture limit and a 4 GiB per-run capture budget. Existing larger assets remain readable in bounded pages. If a live response returns an artifact, continue on its content operation for a stable snapshot.","properties":{"limitBytes":{"default":24576,"maximum":24576,"minimum":4,"type":"integer"},"offsetBytes":{"default":0,"maximum":9007199254740991,"minimum":0,"type":"integer"}},"type":"object"}},"required":["operationId"],"type":"object"},"name":"call_api","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"get_agent_instruction_history","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"List bounded canonical instruction revision metadata, including actor, source run, and restore origin. Use read_agent_instructions with entryFile and revisionId to inspect exact historical content.","inputSchema":{"additionalProperties":false,"properties":{"cursor":{"maxLength":2048,"minLength":1,"type":"string"},"entryFile":{"description":"Configured relative entry filename returned by read_agent_instructions; retain it with the revision.","maxLength":4096,"minLength":1,"type":"string"},"limit":{"maximum":100,"minimum":1,"type":"integer"},"targetAgentId":{"description":"Same-company target agent. Omit to use the calling agent.","pattern":"^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$","type":"string"}},"required":["entryFile"],"type":"object"},"name":"get_agent_instruction_history","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"restore_agent_instructions","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Append a historical instruction revision as the current content using the current baseRevisionId. Requires the responsible user’s current target edit permission. History remains intact; conflicts require an explicit resolution.","inputSchema":{"additionalProperties":false,"properties":{"baseRevisionId":{"description":"Current revision read before restoring. Never replace a stale base silently.","pattern":"^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$","type":"string"},"entryFile":{"description":"Configured relative entry filename returned by read_agent_instructions; retain it with the revision.","maxLength":4096,"minLength":1,"type":"string"},"revisionId":{"description":"Historical revision whose exact content should be restored.","pattern":"^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$","type":"string"},"targetAgentId":{"description":"Same-company target agent. Omit to use the calling agent.","pattern":"^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$","type":"string"}},"required":["entryFile","revisionId","baseRevisionId"],"type":"object"},"name":"restore_agent_instructions","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"create_project","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Create a project after considering existing projects and available repositories. repositoryIds and repositoryUrls accept multiple existing repositories. Use HTTPS GitHub repositoryUrls when an accessible repo is not in the catalog; this registers project repositories, not remote GitHub repositories. Non-code projects may omit repositories. Cannot combine repositoryIds/repositoryUrls with workspace. Reuse the idempotency key on retries.","inputSchema":{"additionalProperties":false,"properties":{"archivedAt":{"description":"Archive timestamp.","maxLength":20000,"type":["string","null"]},"color":{"description":"Project color.","maxLength":20000,"type":["string","null"]},"description":{"description":"Project outcome and context.","maxLength":20000,"type":["string","null"]},"env":{"additionalProperties":true,"type":"object"},"executionWorkspacePolicy":{"additionalProperties":true,"type":"object"},"goalId":{"description":"Goal ID.","maxLength":20000,"type":["string","null"]},"goalIds":{"description":"Goal IDs.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"icon":{"description":"Project icon.","enum":["folder","rocket","code","terminal","database","globe","package","boxes","box","layers","briefcase","compass","target","flame","zap","star","bug","wrench","hammer","lightbulb","sparkles","shield","lock","search","cog","brain","cpu","git-branch","file-code","puzzle","gem","atom","heart","mail","message-square","crown","radar","telescope","hexagon",null],"type":["string","null"]},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"},"leadAgentId":{"description":"Lead agent ID.","maxLength":20000,"type":["string","null"]},"name":{"description":"Project name.","maxLength":500,"minLength":1,"type":"string"},"repositoryIds":{"description":"Authorized repository IDs from list_project_repositories; may contain multiple repositories.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"repositoryUrls":{"description":"Existing HTTPS GitHub repository URLs, including repos absent from the catalog.","items":{"maxLength":2000,"pattern":"^https://github\\.com/(?!\\.{1,2}/)[A-Za-z0-9_.-]+/(?!\\.{1,2}/?$)[A-Za-z0-9_.-]+/?$","type":"string"},"maxItems":100,"type":"array"},"status":{"enum":["backlog","planned","in_progress","completed","cancelled"]},"targetDate":{"description":"Target date.","maxLength":20000,"type":["string","null"]},"workspace":{"additionalProperties":true,"type":"object"}},"required":["idempotencyKey","name"],"type":"object"},"name":"create_project","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"list_project_repositories","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"List authorized repositories with stable IDs and names. Consider appropriate repositories before creating a project; never invent IDs.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"list_project_repositories","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"list_projects","requiredClaims":["discovery:projects:read"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Inspect available company projects before selecting a project for new work.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"list_projects","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"create_skill","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Create a reusable single-file skill in the company library. Supply a complete SKILL.md whose name and description match the inputs. This saves the skill and shows a card; it does not assign the skill to any agent. Reuse idempotencyKey on retries.","inputSchema":{"additionalProperties":false,"properties":{"description":{"maxLength":2000,"minLength":1,"type":"string"},"idempotencyKey":{"maxLength":240,"minLength":1,"type":"string"},"markdown":{"description":"Complete SKILL.md including name and description frontmatter and substantive instructions.","maxLength":200000,"minLength":1,"type":"string"},"name":{"description":"Lowercase skill name, matching SKILL.md frontmatter.","maxLength":120,"minLength":1,"pattern":"^[a-z0-9]+(?:-[a-z0-9]+)*$","type":"string"},"slug":{"description":"Optional; must equal name.","maxLength":120,"minLength":1,"pattern":"^[a-z0-9]+(?:-[a-z0-9]+)*$","type":"string"}},"required":["idempotencyKey","name","description","markdown"],"type":"object"},"name":"create_skill","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"optional","operationId":"reassign_task","requiredClaims":["delegation:tasks:assign"],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Reassign an existing task to another company agent. Read search_tasks first and supply its current assignee and statusVersion to prevent overwriting a concurrent change. Preserve the task, documents, dependencies, and blocked/backlog status. Active work is stopped before handoff. Use a stable idempotency key for retries. Cannot reassign this run’s own task, conversations, completed tasks, or pending reviews; use create_task for delegation from the current task.","inputSchema":{"additionalProperties":false,"properties":{"assigneeActorId":{"description":"New company agent ID from list_agents.","minLength":1,"type":"string"},"expectedAssigneeActorId":{"description":"Current assigneeAgentId from search_tasks; null means unassigned.","type":["string","null"]},"expectedStatusVersion":{"description":"Current statusVersion from search_tasks.","minimum":0,"type":"integer"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"},"reason":{"description":"Why the task should move and context for the new owner.","maxLength":20000,"minLength":1,"type":"string"},"taskId":{"description":"Existing task ID from search_tasks.","minLength":1,"type":"string"}},"required":["idempotencyKey","taskId","assigneeActorId","expectedAssigneeActorId","expectedStatusVersion","reason"],"type":"object"},"name":"reassign_task","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}}] diff --git a/packages/paperclip-runner/protocol/fixtures/evals/native-execution-seeded.json b/packages/paperclip-runner/protocol/fixtures/evals/native-execution-seeded.json index 74e4b4be81..e535bd9f8f 100644 --- a/packages/paperclip-runner/protocol/fixtures/evals/native-execution-seeded.json +++ b/packages/paperclip-runner/protocol/fixtures/evals/native-execution-seeded.json @@ -24,7 +24,7 @@ "prpVersion": 1, "nativeExecutionVersion": 1, "catalogVersion": 1, - "catalogSha256": "sha256:a19534304480dfd3d357cc902e785af05158ce1496f4d324184112732ff94c9c", + "catalogSha256": "sha256:61c6ba99926c14285ca6a89a05fe316848b9e5bc38840d43dd19b0990dc482d0", "driverContractVersion": 1, "driverKind": "paperclip-deterministic", "driverVersion": "1.0.0" diff --git a/packages/paperclip-runner/protocol/manifest.json b/packages/paperclip-runner/protocol/manifest.json index bd8c300f1d..ba8d17e452 100644 --- a/packages/paperclip-runner/protocol/manifest.json +++ b/packages/paperclip-runner/protocol/manifest.json @@ -160,7 +160,7 @@ }, { "path": "fixtures/evals/native-execution-seeded.json", - "sha256": "c6367c7c7e3dd01fed341b7a83b0008c9d5d1efdfa85bc70e1013bc304a008c7", + "sha256": "c9c56cc11b420302be2d0044aeee30302e43da74c076e13a31addfbcaacd0458", "expectation": "accept", "compatibilityCase": "canonical" }, diff --git a/packages/paperclip-runner/spec/capability/protocol-coverage.json b/packages/paperclip-runner/spec/capability/protocol-coverage.json index 8c72856beb..2371a89f6d 100644 --- a/packages/paperclip-runner/spec/capability/protocol-coverage.json +++ b/packages/paperclip-runner/spec/capability/protocol-coverage.json @@ -7,10 +7,86 @@ "src/scenarios/scenario-plan.ts" ], "counts": { - "actions": 48, + "actions": 52, "legacyRequirements": 106 }, "actions": [ + { + "id": "read_agent_instructions", + "ownership": "always_agent_tool", + "surfaces": [ + "live" + ], + "legacyAliases": [], + "contractCase": "protocol-action:read_agent_instructions", + "contractOwner": "src/catalog/protocol-action-contracts.test.ts::read_agent_instructions has a schema-valid canonical example and every declared projection", + "legacyBehavioralCases": [], + "deterministicCases": [ + "protocol-action:read_agent_instructions" + ], + "legacyRequirementCases": [], + "deterministicOwners": [ + "src/catalog/protocol-action-contracts.test.ts::read_agent_instructions has a schema-valid canonical example and every declared projection", + "src/scenarios/scenario-explorer.test.ts::renders every scenario with exposure, control plane, authorization, diff, and parity" + ] + }, + { + "id": "update_agent_instructions", + "ownership": "always_agent_tool", + "surfaces": [ + "live" + ], + "legacyAliases": [], + "contractCase": "protocol-action:update_agent_instructions", + "contractOwner": "src/catalog/protocol-action-contracts.test.ts::update_agent_instructions has a schema-valid canonical example and every declared projection", + "legacyBehavioralCases": [], + "deterministicCases": [ + "protocol-action:update_agent_instructions" + ], + "legacyRequirementCases": [], + "deterministicOwners": [ + "src/catalog/protocol-action-contracts.test.ts::update_agent_instructions has a schema-valid canonical example and every declared projection", + "src/scenarios/scenario-explorer.test.ts::renders every scenario with exposure, control plane, authorization, diff, and parity" + ] + }, + { + "id": "get_agent_instruction_history", + "ownership": "always_agent_tool", + "surfaces": [ + "live" + ], + "legacyAliases": [], + "contractCase": "protocol-action:get_agent_instruction_history", + "contractOwner": "src/catalog/protocol-action-contracts.test.ts::get_agent_instruction_history has a schema-valid canonical example and every declared projection", + "legacyBehavioralCases": [], + "deterministicCases": [ + "protocol-action:get_agent_instruction_history" + ], + "legacyRequirementCases": [], + "deterministicOwners": [ + "src/catalog/protocol-action-contracts.test.ts::get_agent_instruction_history has a schema-valid canonical example and every declared projection", + "src/scenarios/scenario-explorer.test.ts::renders every scenario with exposure, control plane, authorization, diff, and parity" + ] + }, + { + "id": "restore_agent_instructions", + "ownership": "always_agent_tool", + "surfaces": [ + "live" + ], + "legacyAliases": [], + "contractCase": "protocol-action:restore_agent_instructions", + "contractOwner": "src/catalog/protocol-action-contracts.test.ts::restore_agent_instructions has a schema-valid canonical example and every declared projection", + "legacyBehavioralCases": [], + "deterministicCases": [ + "protocol-action:restore_agent_instructions" + ], + "legacyRequirementCases": [], + "deterministicOwners": [ + "src/catalog/protocol-action-contracts.test.ts::restore_agent_instructions has a schema-valid canonical example and every declared projection", + "src/scenarios/scenario-explorer.test.ts::renders every scenario with exposure, control plane, authorization, diff, and parity" + ] + }, { "id": "create_skill", "ownership": "optional_agent_tool", diff --git a/packages/paperclip-runner/spec/operation-groups/source.json b/packages/paperclip-runner/spec/operation-groups/source.json index 2d1cecf0f8..6b24551b0e 100644 --- a/packages/paperclip-runner/spec/operation-groups/source.json +++ b/packages/paperclip-runner/spec/operation-groups/source.json @@ -263,13 +263,13 @@ "id": "rf", "legacyGroup": 13, "name": "Reference files", - "owner": "optional domain tools + test-only escape hatch", - "operationIds": ["list_cases", "upsert_case", "list_routines", "manage_routine", "create_skill", "list_company_skills", "sync_company_skills", "list_secret_metadata", "read_secret_value", "export_company", "administer_company", "generic_api_request", "search_api", "call_api", "create_project"], + "owner": "always instruction tools + optional domain tools + test-only escape hatch", + "operationIds": ["list_cases", "upsert_case", "list_routines", "manage_routine", "create_skill", "list_company_skills", "sync_company_skills", "list_secret_metadata", "read_secret_value", "export_company", "administer_company", "generic_api_request", "search_api", "call_api", "create_project", "read_agent_instructions", "update_agent_instructions", "get_agent_instruction_history", "restore_agent_instructions"], "controlPlaneOperationIds": ["append_audit_record"], - "realSurface": "project, case, routine, company-skill, secret, portability, and administration services", + "realSurface": "agent instruction revisions, project, case, routine, company-skill, secret, portability, and administration services", "mockStateDomains": ["company", "cases", "routines", "skills", "secrets", "audit", "fault"], "prpEvidence": "bounded domain projections, redacted broker receipts, company diffs, and audit references", - "gap": "Project and skill creation and API tools require the live server authority; generic_api_request is test-only and the remaining domain operations are scenario-only. Broad administer_company is deferred and cannot claim product coverage. Production API and paired dedicated-tool regressions are recorded separately in paperclip-evals/evals/runner-api-tools; the legacy scenario count is not evidence of that coverage." + "gap": "Instruction read/update/history/restore require the live canonical revision service, current responsible-user permissions, and pinned base revisions for writes. Their service, authority, and product E2E tests are separate from the legacy mock scenarios. Project and skill creation and API tools require the live server authority; generic_api_request is test-only and the remaining domain operations are scenario-only. Broad administer_company is deferred and cannot claim product coverage. Production API and paired dedicated-tool regressions are recorded separately in paperclip-evals/evals/runner-api-tools; the legacy scenario count is not evidence of that coverage." }, { "id": "mh", diff --git a/packages/paperclip-runner/spec/paperclip-agent-operation-groups.md b/packages/paperclip-runner/spec/paperclip-agent-operation-groups.md index b6bfa69167..afadc264d7 100644 --- a/packages/paperclip-runner/spec/paperclip-agent-operation-groups.md +++ b/packages/paperclip-runner/spec/paperclip-agent-operation-groups.md @@ -6,7 +6,7 @@ Status: canonical explanatory contract for the Paperclip runner V1 surface. This document keeps three independent meanings of **group** separate. PRP families describe wire evidence and controller commands; capability placement decides who owns an operation; behavioral eval groups organize the 106 scenario corpus. None of the three axes can be used as a substitute for another. -The generated totals are **105 PRP events in 31 event families**, **18 controller commands in 7 command families**, **10 control-plane operations**, **48 reconciled semantic operations** (14 always, 34 optional), and **106 scenarios in 16 behavior groups**. +The generated totals are **105 PRP events in 31 event families**, **18 controller commands in 7 command families**, **10 control-plane operations**, **52 reconciled semantic operations** (18 always, 34 optional), and **106 scenarios in 16 behavior groups**. ## Axis 1: PRP v1 event and command families @@ -83,9 +83,9 @@ Placement has exactly three outcomes: | `schedule_blocker_wake` | Dependency-resolution wake scheduling. | | `reconcile_run` | Work assessment, terminal status arbitration, and recovery reconciliation. | -### Always-agent operations (14) +### Always-agent operations (18) -`answer_status_question`, `block_task`, `finish_task`, `get_task_context`, `get_task_history`, `inspect_operation_result`, `list_document_revisions`, `list_documents`, `read_document`, `register_deliverable`, `report_progress`, `request_human_input`, `request_review`, `write_document`. +`answer_status_question`, `block_task`, `finish_task`, `get_agent_instruction_history`, `get_task_context`, `get_task_history`, `inspect_operation_result`, `list_document_revisions`, `list_documents`, `read_agent_instructions`, `read_document`, `register_deliverable`, `report_progress`, `request_human_input`, `request_review`, `restore_agent_instructions`, `update_agent_instructions`, `write_document`. ### Optional operations (34) and grant groups (13) @@ -127,6 +127,7 @@ Grant groups are documentation/exposure bundles, not additional authority. The o | `finish_task` | `always_agent_tool` | none | `standard`
`skill_test` | `task_write` | `required` | no | `semantic_command:finish_task` | `scenario` + `live`
`live_codex` | `unbound`
semantic-operation item event plus active-task state diff, work-assessment, and issue-status-decision events
catalog PRP status: `audit_pending` | | `generic_api_request` | `optional_agent_tool` | `test:generic_api_request` | `skill_test` | `test_escape_hatch` | `required` | yes | `mock_extension:test.generic_api` | `scenario` + `live`
`test_only` | `unbound`
test-only; excluded from product PRP evidence
catalog PRP status: `audit_pending` | | `get_agent` | `optional_agent_tool` | `discovery:agents:read` | `standard`
`ask`
`planning`
`skill_test` | `read` | `none` | no | inline/no mapping | `live`
`live_codex` | `unbound`
read projection surfaced via a tool-result item event; no control-plane state diff
catalog PRP status: `audit_pending` | +| `get_agent_instruction_history` | `always_agent_tool` | none | `standard`
`ask`
`planning`
`skill_test` | `read` | `none` | no | inline/no mapping | `live`
`live_codex` | `agentInstructionRevisionService`
tool-result item event plus canonical revision receipt for writes; no scenario mock coverage
catalog PRP status: `bound` | | `get_approval` | `optional_agent_tool` | `governance:approvals:read` | `standard`
`ask`
`planning`
`skill_test` | `read` | `none` | no | inline/no mapping | `live`
`live_codex` | `unbound`
read projection surfaced via a tool-result item event; no control-plane state diff
catalog PRP status: `audit_pending` | | `get_approval_context` | `optional_agent_tool` | `governance:approvals:read` | `standard`
`ask`
`planning`
`skill_test` | `read` | `none` | no | inline/no mapping | `live`
`live_codex` | `unbound`
read projection surfaced via a tool-result item event; no control-plane state diff
catalog PRP status: `audit_pending` | | `get_task_context` | `always_agent_tool` | none | `standard`
`ask`
`planning`
`skill_test` | `read` | `none` | no | `context_read:active_task` | `scenario` + `live`
`live_codex` | `PaperclipRunnerToolAuthority active issue/run + accepted plan revision`
bound company/assignment query plus exact accepted document revision projection
catalog PRP status: `bound` | @@ -146,6 +147,7 @@ Grant groups are documentation/exposure bundles, not additional authority. The o | `list_routines` | `optional_agent_tool` | `routines:read` | `standard`
`skill_test` | `read` | `none` | no | `mock_extension:routines.list` | `scenario`
`scenario_mock` | `unbound`
read projection surfaced via a tool-result item event; no control-plane state diff
catalog PRP status: `audit_pending` | | `list_secret_metadata` | `optional_agent_tool` | `secrets:metadata:read` | `standard`
`skill_test` | `read` | `none` | no | `mock_extension:secrets.metadata` | `scenario`
`scenario_mock` | `unbound`
read projection surfaced via a tool-result item event; no control-plane state diff
catalog PRP status: `audit_pending` | | `manage_routine` | `optional_agent_tool` | `routines:write` | `standard`
`skill_test` | `admin` | `required` | no | `mock_extension:routines.manage` | `scenario`
`scenario_mock` | `unbound`
company admin/portability item event plus audit record
catalog PRP status: `audit_pending` | +| `read_agent_instructions` | `always_agent_tool` | none | `standard`
`ask`
`planning`
`skill_test` | `read` | `none` | no | inline/no mapping | `live`
`live_codex` | `agentInstructionRevisionService`
tool-result item event plus canonical revision receipt for writes; no scenario mock coverage
catalog PRP status: `bound` | | `read_document` | `always_agent_tool` | none | `standard`
`ask`
`planning`
`skill_test` | `read` | `none` | no | `snapshot_read:active_task_document` | `scenario` + `live`
`live_codex` | `unbound`
read projection surfaced via a tool-result item event; no control-plane state diff
catalog PRP status: `audit_pending` | | `read_secret_value` | `optional_agent_tool` | `secrets:values:read` | `standard`
`skill_test` | `secret_read` | `none` | yes | `mock_extension:secrets.value` | `scenario`
`scenario_mock` | `unbound`
redacted tool-result item event; secret value never reaches the wire
catalog PRP status: `audit_pending` | | `reassign_task` | `optional_agent_tool` | `delegation:tasks:assign` | `standard` | `company_write` | `required` | no | `semantic_command:reassign_task` | `scenario` + `live`
`live_codex` | `PaperclipRunnerToolAuthority.reassign_task`
semantic-operation item event plus company-entity state diff and audit record
catalog PRP status: `bound` | @@ -154,11 +156,13 @@ Grant groups are documentation/exposure bundles, not additional authority. The o | `request_approval` | `optional_agent_tool` | `governance:approvals:request` | `standard`
`skill_test` | `governance` | `required` | no | `semantic_command:request_approval` | `scenario` + `live`
`live_codex` | `unbound`
approval lifecycle plus governed-wait continuation and audit events
catalog PRP status: `audit_pending` | | `request_human_input` | `always_agent_tool` | none | `standard`
`planning`
`ask`
`skill_test` | `task_write` | `required` | no | `semantic_command:request_human_input` | `scenario` + `live`
`live_codex` | `unbound`
semantic-operation item event plus active-task state diff, work-assessment, and issue-status-decision events
catalog PRP status: `audit_pending` | | `request_review` | `always_agent_tool` | none | `standard`
`skill_test` | `task_write` | `required` | no | `semantic_command:request_review` | `scenario` + `live`
`live_codex` | `unbound`
semantic-operation item event plus active-task state diff, work-assessment, and issue-status-decision events
catalog PRP status: `audit_pending` | +| `restore_agent_instructions` | `always_agent_tool` | none | `standard`
`planning` | `company_write` | `recommended` | no | inline/no mapping | `live`
`live_codex` | `agentInstructionRevisionService`
tool-result item event plus canonical revision receipt for writes; no scenario mock coverage
catalog PRP status: `bound` | | `schedule_wake` | `optional_agent_tool` | `control_plane:wakes` | `standard`
`skill_test` | `task_write` | `required` | no | inline/no mapping | `live`
`live_codex` | `unbound`
semantic-operation item event plus active-task state diff, work-assessment, and issue-status-decision events
catalog PRP status: `audit_pending` | | `search_api` | `optional_agent_tool` | `api:discover` | `standard`
`ask`
`planning`
`skill_test` | `read` | `none` | no | inline/no mapping | `live`
`live_codex` | `PaperclipRunnerToolAuthority`
Authenticated PRP tool input/result and existing HTTP route authorization/activity records.
catalog PRP status: `bound` | | `search_tasks` | `optional_agent_tool` | `discovery:tasks:read` | `standard`
`ask`
`planning`
`skill_test` | `read` | `none` | no | `snapshot_read:company_tasks` | `scenario` + `live`
`live_codex` | `unbound`
read projection surfaced via a tool-result item event; no control-plane state diff
catalog PRP status: `audit_pending` | | `set_dependencies` | `optional_agent_tool` | `dependencies:write` | `standard`
`skill_test` | `company_write` | `required` | no | `semantic_command:set_dependencies` | `scenario` + `live`
`live_codex` | `issues.update.blockedByIssueIds`
semantic-operation item event plus company-entity state diff and audit record
catalog PRP status: `bound` | | `sync_company_skills` | `optional_agent_tool` | `company_skills:write` | `standard`
`skill_test` | `admin` | `required` | no | `mock_extension:company_skills.sync` | `scenario`
`scenario_mock` | `unbound`
company admin/portability item event plus audit record
catalog PRP status: `audit_pending` | +| `update_agent_instructions` | `always_agent_tool` | none | `standard`
`planning` | `company_write` | `recommended` | no | inline/no mapping | `live`
`live_codex` | `agentInstructionRevisionService`
tool-result item event plus canonical revision receipt for writes; no scenario mock coverage
catalog PRP status: `bound` | | `upsert_case` | `optional_agent_tool` | `cases:write` | `standard`
`skill_test` | `company_write` | `required` | no | `mock_extension:cases.upsert` | `scenario`
`scenario_mock` | `unbound`
semantic-operation item event plus company-entity state diff and audit record
catalog PRP status: `audit_pending` | | `write_document` | `always_agent_tool` | none | `standard`
`planning`
`skill_test` | `task_write` | `required` | no | `semantic_command:write_document` | `scenario` + `live`
`live_codex` | `unbound`
semantic-operation item event plus active-task state diff, work-assessment, and issue-status-decision events
catalog PRP status: `audit_pending` | @@ -182,7 +186,7 @@ Behavior groups describe expected outcomes and trajectories. They do not grant t | [`ap` — Approvals](#behavior-group-ap-approvals) | optional governance tools + governed approver | `list_approvals`
`get_approval`
`get_approval_context`
`request_approval`
`decide_approval`
`comment_on_approval` | `route_wake`
`append_audit_record` | company approval, decision, issue-link, comment, and governed-action services | `company`
`task`
`approvals`
`actor`
`wake`
`audit`
`idempotency` | 6 | governed semantic receipts, audit references, and attention/continuation linkage | Production binding and additive governed-action receipts are unbound; board-only authority stays outside grants. | | [`ar` — Artifacts](#behavior-group-ar-artifacts) | always tools + artifact/work-product services | `register_deliverable` | `append_audit_record` | attachment upload and issue work-product routes | `task`
`artifacts`
`workProducts`
`workspace`
`audit`
`idempotency` | 4 | artifact/work-product reference and durable inspectability receipt; never binary bytes | Production upload/register composite and additive durable-reference receipt are unbound. | | [`er` — Errors and critical rules](#behavior-group-er-errors-and-critical-rules) | runner/control plane + optional workspace/wake tools | `get_workspace_runtime`
`control_workspace_service`
`schedule_wake`
`inspect_operation_result` | `release_task`
`enforce_budget`
`persist_run`
`replay_run`
`reconcile_run` | workspace runtime, monitor/recovery, budget, run persistence/replay, release, and terminal services | `workspace`
`budget`
`run`
`wake`
`audit`
`idempotency`
`fault` | 9 | runtime/workspace/attention/run lifecycle, typed denials, replay facts, and terminal causality | Budget stop reasons and semantic denial/conflict receipts require additive v1 envelopes; inspect_operation_result remains scenario-only. | -| [`rf` — Reference files](#behavior-group-rf-reference-files) | optional domain tools + test-only escape hatch | `list_cases`
`upsert_case`
`list_routines`
`manage_routine`
`create_skill`
`list_company_skills`
`sync_company_skills`
`list_secret_metadata`
`read_secret_value`
`export_company`
`administer_company`
`generic_api_request`
`search_api`
`call_api`
`create_project` | `append_audit_record` | project, case, routine, company-skill, secret, portability, and administration services | `company`
`cases`
`routines`
`skills`
`secrets`
`audit`
`fault` | 22 | bounded domain projections, redacted broker receipts, company diffs, and audit references | Project and skill creation and API tools require the live server authority; generic_api_request is test-only and the remaining domain operations are scenario-only. Broad administer_company is deferred and cannot claim product coverage. Production API and paired dedicated-tool regressions are recorded separately in paperclip-evals/evals/runner-api-tools; the legacy scenario count is not evidence of that coverage. | +| [`rf` — Reference files](#behavior-group-rf-reference-files) | always instruction tools + optional domain tools + test-only escape hatch | `list_cases`
`upsert_case`
`list_routines`
`manage_routine`
`create_skill`
`list_company_skills`
`sync_company_skills`
`list_secret_metadata`
`read_secret_value`
`export_company`
`administer_company`
`generic_api_request`
`search_api`
`call_api`
`create_project`
`read_agent_instructions`
`update_agent_instructions`
`get_agent_instruction_history`
`restore_agent_instructions` | `append_audit_record` | agent instruction revisions, project, case, routine, company-skill, secret, portability, and administration services | `company`
`cases`
`routines`
`skills`
`secrets`
`audit`
`fault` | 22 | bounded domain projections, redacted broker receipts, company diffs, and audit references | Instruction read/update/history/restore require the live canonical revision service, current responsible-user permissions, and pinned base revisions for writes. Their service, authority, and product E2E tests are separate from the legacy mock scenarios. Project and skill creation and API tools require the live server authority; generic_api_request is test-only and the remaining domain operations are scenario-only. Broad administer_company is deferred and cannot claim product coverage. Production API and paired dedicated-tool regressions are recorded separately in paperclip-evals/evals/runner-api-tools; the legacy scenario count is not evidence of that coverage. | | [`mh` — Multi-hop](#behavior-group-mh-multi-hop) | composed semantic operations + control-plane continuation | `create_task`
`set_dependencies`
`request_human_input`
`request_approval`
`register_deliverable` | `route_wake`
`reconcile_run` | delegation, dependency, interaction, approval, artifact, and terminal orchestration services | `task`
`blockers`
`interactions`
`approvals`
`artifacts`
`wake`
`run`
`audit` | 4 | correlated operation receipts, state diffs, attention hops, work assessment, status decision, and terminal outcome | No generic transaction tool is allowed; shared mock/real conformance must prove each composed effect. | | [`rs` — Restraint and no-call](#behavior-group-rs-restraint-and-no-call) | policy/exposure layer | `answer_status_question`
`read_secret_value`
`generic_api_request` | `enforce_budget` | task-mode, secret-broker, test-scope, pause, and budget policy checks | `actor`
`task`
`budget`
`secrets`
`audit`
`fault` | 3 | absence of forbidden effects plus typed policy denial/redaction receipts when a call is attempted | Typed redaction/authorization receipts need additive v1 evidence; generic_api_request is never a product fallback. | | [`wk` — Wake situations](#behavior-group-wk-wake-situations) | control plane + always context/history tools | `get_task_context`
`get_task_history`
`schedule_wake` | `select_work`
`route_wake` | wakeup requests, heartbeat context, comment/interaction/approval/blocker wake routing, and scheduled wake services | `wake`
`task`
`comments`
`interactions`
`approvals`
`blockers`
`run` | 8 | attention request routing/resolution plus resumed session/run causality | Production scheduling binding is unbound; control-plane routing remains non-callable. | @@ -459,10 +463,10 @@ Current responsibility-based paths are normative. Numbered `phase-*` or mileston ### Catalog split and deliberate replacement - Scenario/eval catalog: **39** operations. -- Live dispatcher catalog: **36** operations. -- Shared: **27**; union/canonical authority: **48**. +- Live dispatcher catalog: **40** operations. +- Shared: **27**; union/canonical authority: **52**. - Scenario-only: `administer_company`, `export_company`, `inspect_operation_result`, `list_cases`, `list_company_skills`, `list_goals`, `list_routines`, `list_secret_metadata`, `manage_routine`, `read_secret_value`, `sync_company_skills`, `upsert_case`. -- Live-only: `call_api`, `create_project`, `get_agent`, `get_approval`, `get_approval_context`, `hire_agent`, `list_project_repositories`, `schedule_wake`, `search_api`. +- Live-only: `call_api`, `create_project`, `get_agent`, `get_agent_instruction_history`, `get_approval`, `get_approval_context`, `hire_agent`, `list_project_repositories`, `read_agent_instructions`, `restore_agent_instructions`, `schedule_wake`, `search_api`, `update_agent_instructions`. - The generated provider contract contains exactly the live catalog; the canonical union remains the migration authority until all scenario-only operations are either implemented, deferred, or removed by an explicit reconciliation decision. - `generic_api_request` stays exported only for controlled tests and cannot be cited as real-surface, mock-parity, or PRP product coverage. diff --git a/packages/paperclip-runner/src/backends/codex-native-backend.ts b/packages/paperclip-runner/src/backends/codex-native-backend.ts index d334079b78..1cdd0bb6af 100644 --- a/packages/paperclip-runner/src/backends/codex-native-backend.ts +++ b/packages/paperclip-runner/src/backends/codex-native-backend.ts @@ -156,6 +156,7 @@ function createTransportBackedNativeSessionBackend( ? (input.provider.approvalPolicy ?? "never") : "never", baseInstructions: nativeSystemInstructions(input), + instructionWorkingCopyRoot: "runtimeContext" in input ? input.runtimeContext.instructions.workingCopy?.rootPath : undefined, includeSkillInstructions: isCodex && "runtimeContext" in input, skillInputs: isCodex ? nativeTaskSkillInputs( diff --git a/packages/paperclip-runner/src/backends/runtime-context.ts b/packages/paperclip-runner/src/backends/runtime-context.ts index 203e42552a..05f7d177c3 100644 --- a/packages/paperclip-runner/src/backends/runtime-context.ts +++ b/packages/paperclip-runner/src/backends/runtime-context.ts @@ -104,6 +104,11 @@ export function nativeTaskConstraints(input: NativeExecutionInput): string[] { } return [ "Use only the assigned skills and provider-native tools.", + ...(input.runtimeContext.instructions.workingCopy ? [ + input.runtimeContext.instructions.workingCopy.kind === "agent_files" + ? `For this turn, AGENT_HOME is ${input.runtimeContext.instructions.workingCopy.rootPath}. This replaces any prior turn's agent directory path. It contains your instructions and persistent personal files, separate from the task working directory. Changes save after the provider stops; check the save receipt.` + : `For this turn, the editable agent instruction file is ${input.runtimeContext.instructions.workingCopy.rootPath}/${input.runtimeContext.instructions.workingCopy.entryPath}. This replaces any private working-copy path from a previous turn. Ordinary edits save after the provider stops and only with a durable revision receipt. Use the agent instruction tools for immediate saves. Shared instruction assets and repository instructions are not collected.`, + ] : []), "Use Paperclip semantic tools for coordination and finalization.", "Save requested plans and Paperclip documents directly with write_document. A saved Paperclip document is already a durable deliverable. Do not create a local file, compute file hashes, or call register_deliverable for it unless the user also requests a downloadable file. Cite the saved document in your completion evidence and final response.", "When the requested result is a file, use register_deliverable before paperclip_finish. Compute its exact byte size and SHA-256, register the workspace-relative file, cite deliverable: from the receipt as completion evidence, and include /api/attachments//content as the download link in your answer. A bare workspace filename is not a delivered result. For repository edits, cite an accessible PR or registered work product. Preserve existing work; do not upload unrelated files. If file publication fails, fix it or report the concrete blocker instead of claiming the file is delivered.", diff --git a/packages/paperclip-runner/src/catalog/reconciliation.test.ts b/packages/paperclip-runner/src/catalog/reconciliation.test.ts index 9ab2cdf7c8..0f69be379c 100644 --- a/packages/paperclip-runner/src/catalog/reconciliation.test.ts +++ b/packages/paperclip-runner/src/catalog/reconciliation.test.ts @@ -20,21 +20,25 @@ describe("canonical semantic-catalog reconciliation authority", () => { it("pins the reconciled op-set relationship between the two catalogs", () => { const summary = capabilityCatalogReconciliation(); expect(summary.scenarioCount).toBe(39); - expect(summary.liveCount).toBe(36); + expect(summary.liveCount).toBe(40); expect(summary.sharedCount).toBe(27); - expect(summary.unionCount).toBe(48); + expect(summary.unionCount).toBe(52); // Any operation added to or removed from either catalog without a // reconciliation decision changes these exact sets and fails the gate. expect(summary.liveOnly).toEqual([ "call_api", "create_project", "get_agent", + "get_agent_instruction_history", "get_approval", "get_approval_context", "hire_agent", "list_project_repositories", + "read_agent_instructions", + "restore_agent_instructions", "schedule_wake", "search_api", + "update_agent_instructions", ]); expect(summary.scenarioOnly).toEqual([ "administer_company", @@ -53,7 +57,7 @@ describe("canonical semantic-catalog reconciliation authority", () => { }); it("is the single source both catalogs derive their operation set from", () => { - expect(CAPABILITY_CANONICAL_OPERATIONS).toHaveLength(48); + expect(CAPABILITY_CANONICAL_OPERATIONS).toHaveLength(52); const canonicalIds = new Set(CAPABILITY_CANONICAL_OPERATIONS.map((operation) => operation.operationId)); // Neither catalog may contain an operation absent from the canonical source. for (const tool of SCENARIO_CATALOG) expect(canonicalIds.has(tool.operationId)).toBe(true); @@ -87,7 +91,7 @@ describe("canonical semantic-catalog reconciliation authority", () => { }); it("names placement, claims, task modes, side-effect class, idempotency, redaction, mock mapping, real binding status, and PRP evidence for every operation", () => { - expect(CAPABILITY_CANONICAL_CATALOG).toHaveLength(48); + expect(CAPABILITY_CANONICAL_CATALOG).toHaveLength(52); for (const operation of CAPABILITY_CANONICAL_CATALOG) { expect(operation.placement).toMatch(/^(always|optional)_agent_tool$/); expect(Array.isArray(operation.requiredClaims)).toBe(true); @@ -113,7 +117,7 @@ describe("canonical semantic-catalog reconciliation authority", () => { it("classifies real binding status so generic_api_request is never product coverage", () => { const summary = capabilityCatalogReconciliation(); expect(summary.byRealBindingStatus).toEqual({ - live_codex: 35, + live_codex: 39, scenario_mock: 12, test_only: 1, }); diff --git a/packages/paperclip-runner/src/contracts/runtime-context.ts b/packages/paperclip-runner/src/contracts/runtime-context.ts index 948cdbc62f..bbf7e89ab5 100644 --- a/packages/paperclip-runner/src/contracts/runtime-context.ts +++ b/packages/paperclip-runner/src/contracts/runtime-context.ts @@ -15,7 +15,12 @@ export interface NativeRuntimeAssetReference { export interface NativeRuntimeContextSnapshot { prompt: { revision: typeof PAPERCLIP_EXECUTION_PROMPT_REVISION; text: typeof PAPERCLIP_EXECUTION_PROMPT; digest: string }; - instructions: { entryPath: string; bundle: NativeRuntimeAssetReference }; + instructions: { + entryPath: string; + bundle: NativeRuntimeAssetReference; + /** Server-registered writable copy; excluded from the pinned prompt digest. */ + workingCopy?: { rootPath: string; entryPath: string; kind?: "agent_files" }; + }; skills: Array<{ key: string; runtimeName: string; versionId: string | null; bundle: NativeRuntimeAssetReference }>; mcp: { assignmentSetId: string; digest: string; bindingId: string | null }; aggregateDigest: string; @@ -103,7 +108,10 @@ export function parseNativeRuntimeContext(value: unknown): NativeRuntimeContextS throw new NativeRuntimeContextError("input.runtimeContext.prompt.digest does not match prompt text"); } const instructions = object(context.instructions, "input.runtimeContext.instructions"); - exact(instructions, ["entryPath", "bundle"], "input.runtimeContext.instructions"); + exact(instructions, ["entryPath", "bundle", "workingCopy"], "input.runtimeContext.instructions"); + const workingCopy = instructions.workingCopy === undefined ? undefined : object(instructions.workingCopy, "input.runtimeContext.instructions.workingCopy"); + if (workingCopy) exact(workingCopy, ["rootPath", "entryPath", "kind"], "input.runtimeContext.instructions.workingCopy"); + if (workingCopy?.kind !== undefined && workingCopy.kind !== "agent_files") throw new NativeRuntimeContextError("Unknown agent file contract"); if (!Array.isArray(context.skills)) throw new NativeRuntimeContextError("input.runtimeContext.skills must be an array"); const skills = context.skills.map((value, index) => { const skill = object(value, `input.runtimeContext.skills[${index}]`); @@ -122,7 +130,15 @@ export function parseNativeRuntimeContext(value: unknown): NativeRuntimeContextS exact(mcp, ["assignmentSetId", "digest", "bindingId"], "input.runtimeContext.mcp"); const parsed = { prompt: { revision: PAPERCLIP_EXECUTION_PROMPT_REVISION, text: PAPERCLIP_EXECUTION_PROMPT, digest: nativeRuntimePromptDigest() }, - instructions: { entryPath: safeRelativePath(instructions.entryPath, "input.runtimeContext.instructions.entryPath"), bundle: parseAsset(instructions.bundle, "input.runtimeContext.instructions.bundle") }, + instructions: { + entryPath: safeRelativePath(instructions.entryPath, "input.runtimeContext.instructions.entryPath"), + bundle: parseAsset(instructions.bundle, "input.runtimeContext.instructions.bundle"), + ...(workingCopy ? { workingCopy: { + ...(workingCopy.kind === "agent_files" ? { kind: "agent_files" as const } : {}), + rootPath: text(workingCopy.rootPath, "input.runtimeContext.instructions.workingCopy.rootPath"), + entryPath: safeRelativePath(workingCopy.entryPath, "input.runtimeContext.instructions.workingCopy.entryPath"), + } } : {}), + }, skills, mcp: { assignmentSetId: text(mcp.assignmentSetId, "input.runtimeContext.mcp.assignmentSetId"), @@ -141,6 +157,12 @@ export function composeNativeSystemInstructions(context: NativeRuntimeContextSna return [ context.prompt.text, entryContent.trim(), + context.instructions.workingCopy?.kind === "agent_files" + ? `Your persistent agent directory (AGENT_HOME) is ${context.instructions.workingCopy.rootPath}. Your instruction entry is ${context.instructions.workingCopy.entryPath}, relative to that directory. All supported files and subfolders there are restored across tasks and sessions, and collected after this provider stops. Write task deliverables in the task working directory. Only changed or deleted files synchronize; the last sync wins for the same file. Temporary copies are cleaned up without retaining file history. Check the save receipt before claiming persistence.` + : context.instructions.workingCopy + ? `Your editable agent instruction file is ${context.instructions.workingCopy.rootPath}/${context.instructions.workingCopy.entryPath}. Edit this registered private copy normally. After this run stops, Paperclip saves changed content as a persistent revision if your responsible user still has permission and the baseline has not changed. Check the run's instruction-save receipt before claiming persistence. Conflicts are preserved for explicit resolution. Repository instruction files, skills, and this run's loaded prompt are separate and are not collected.` + : null, + // Keep this canonical suffix intact for provider-specific asset remapping. `Read-only instruction sibling root: ${context.instructions.bundle.rootPath}`, ].filter(Boolean).join("\n\n"); } diff --git a/packages/paperclip-runner/src/drivers/codex/app-server-transport.ts b/packages/paperclip-runner/src/drivers/codex/app-server-transport.ts index 172aec5657..4dedcb6726 100644 --- a/packages/paperclip-runner/src/drivers/codex/app-server-transport.ts +++ b/packages/paperclip-runner/src/drivers/codex/app-server-transport.ts @@ -194,6 +194,7 @@ class BoundedLineDecoder { } const SAFE_ENVIRONMENT_KEYS = [ + "AGENT_HOME", "ALL_PROXY", "CODEX_HOME", "HOME", diff --git a/packages/paperclip-runner/src/drivers/codex/codex-app-server-driver-impl.ts b/packages/paperclip-runner/src/drivers/codex/codex-app-server-driver-impl.ts index 40ef0df026..0fba00ebad 100644 --- a/packages/paperclip-runner/src/drivers/codex/codex-app-server-driver-impl.ts +++ b/packages/paperclip-runner/src/drivers/codex/codex-app-server-driver-impl.ts @@ -660,7 +660,7 @@ export class CodexAppServerDriver implements HarnessDriver { this.#options.transportFactory?.(context) ?? new ProcessCodexAppServerTransport({ workingDirectory, - args: createIsolatedCodexAppServerArgs(this.#options.environment, codexExecutableReadOnlyRoots(this.#options.environment ?? process.env)), + args: createIsolatedCodexAppServerArgs(this.#options.environment, codexExecutableReadOnlyRoots(this.#options.environment ?? process.env), this.#options.instructionWorkingCopyRoot), environment: createSanitizedCodexEnvironment(this.#options.environment), onDiagnostic: this.#options.onDiagnostic, processGroup: true, diff --git a/packages/paperclip-runner/src/drivers/codex/codex-driver-types.ts b/packages/paperclip-runner/src/drivers/codex/codex-driver-types.ts index 0414b2ccf7..12e6265b6e 100644 --- a/packages/paperclip-runner/src/drivers/codex/codex-driver-types.ts +++ b/packages/paperclip-runner/src/drivers/codex/codex-driver-types.ts @@ -20,6 +20,8 @@ export interface CodexAppServerDriverOptions { approvalPolicy?: "never" | "on-request" | "untrusted"; baseInstructions?: string; includeSkillInstructions?: boolean; + /** Private instruction directory registered by the control plane for this run. */ + instructionWorkingCopyRoot?: string; /** Explicit selected skills, resolved from this task's assigned runtime assets. */ skillInputs?: readonly import("../../contracts/runtime-context.js").NativeSkillInput[]; conversationMode?: "task" | "direct" | "prepared"; diff --git a/packages/paperclip-runner/src/drivers/codex/codex-security-config.test.ts b/packages/paperclip-runner/src/drivers/codex/codex-security-config.test.ts index f22d328ffb..acecaf2c83 100644 --- a/packages/paperclip-runner/src/drivers/codex/codex-security-config.test.ts +++ b/packages/paperclip-runner/src/drivers/codex/codex-security-config.test.ts @@ -12,6 +12,25 @@ import { } from "./codex-security-config.js"; describe("Codex security configuration", () => { + it("allows only the registered private instruction directory while keeping shared context read-only", () => { + const args = createIsolatedCodexAppServerArgs({ HOME: "/host/home" }, ["/runtime/immutable-context"], "/runtime/instruction-edits/run-1").join("\n"); + expect(args).toContain('"/runtime/instruction-edits/run-1"="write"'); + expect(args).toContain('"/runtime/immutable-context"="read"'); + expect(args).not.toContain('"/runtime"="write"'); + expect(args).toContain('"/host/home"="none"'); + expect(createIsolatedCodexAppServerArgs({ PAPERCLIP_INSTRUCTION_ROOT: "/host/home" }).join("\n")) + .not.toContain('"/host/home"="write"'); + }); + + it("exposes AGENT_HOME only when it matches the controller-registered writable directory", () => { + const root = "/agent-files/run-1"; + const registered = createIsolatedCodexAppServerArgs({ AGENT_HOME: root, HOME: "/provider" }, [], root).join("\n"); + expect(registered).toContain('AGENT_HOME="/agent-files/run-1"'); + expect(registered).toContain('"/provider"="none"'); + expect(createIsolatedCodexAppServerArgs({ AGENT_HOME: "/arbitrary" }, [], root).join("\n")).not.toContain("AGENT_HOME"); + expect(createIsolatedCodexAppServerArgs({ AGENT_HOME: root }).join("\n")).not.toContain("AGENT_HOME"); + }); + it("makes the installed npm Codex native sandbox executable readable without exposing its parent workspace", () => { const command = evalProviderTransportOptions("codex").codexCommand!; const manifest = createRequire(command).resolve(`@openai/codex-${process.platform}-${process.arch}/package.json`); diff --git a/packages/paperclip-runner/src/drivers/codex/codex-security-config.ts b/packages/paperclip-runner/src/drivers/codex/codex-security-config.ts index b041ff4363..f6480c12c0 100644 --- a/packages/paperclip-runner/src/drivers/codex/codex-security-config.ts +++ b/packages/paperclip-runner/src/drivers/codex/codex-security-config.ts @@ -169,6 +169,8 @@ function tomlString(value: string): string { export function createIsolatedCodexAppServerArgs( source: NodeJS.ProcessEnv = process.env, readOnlyRoots: string[] = [], + /** Server-registered run copy, never an environment/config-supplied root. */ + instructionWorkingCopyRoot?: string, ): string[] { const gitRoots = gitFilesystemRoots(source); readOnlyRoots = [...new Set([...readOnlyRoots, ...codexNetworkReadOnlyRoots(source)])]; @@ -179,6 +181,7 @@ export function createIsolatedCodexAppServerArgs( // Codex filters the configured `set` values through include_only as well. // Retain the explicit command PATH/HOME/locale settings, not ambient secrets. const commandEnvironment = codexCommandEnvironment(source); + if (instructionWorkingCopyRoot && source.AGENT_HOME === instructionWorkingCopyRoot) commandEnvironment.AGENT_HOME = instructionWorkingCopyRoot; const shellEnvironmentKeys = [...new Set([...inheritedGitHubKeys, ...Object.keys(commandEnvironment)])].sort(); if (source.PAPERCLIP_GITHUB_LAUNCHER_DIR) readOnlyRoots = [...readOnlyRoots, source.PAPERCLIP_GITHUB_LAUNCHER_DIR]; const deniedHostRoots = [ @@ -199,6 +202,7 @@ export function createIsolatedCodexAppServerArgs( ...readOnlyRoots.map((path) => `${tomlString(resolve(path))}="read"`), ...gitRoots.read.map((path) => `${tomlString(path)}="read"`), ...gitRoots.write.map((path) => `${tomlString(path)}="write"`), + ...(instructionWorkingCopyRoot ? [`${tomlString(resolve(instructionWorkingCopyRoot))}="write"`] : []), ...(source.PAPERCLIP_GITHUB_BROKER_TOKEN && source.GH_CONFIG_DIR ? [`${tomlString(resolve(source.GH_CONFIG_DIR))}="write"`] : []), `":workspace_roots"={"."="write"}`, @@ -210,6 +214,7 @@ export function createIsolatedCodexAppServerArgs( ...deniedHostRoots.map((path) => `${tomlString(path)}="none"`), ...readOnlyRoots.map((path) => `${tomlString(resolve(path))}="read"`), ...[...gitRoots.read, ...gitRoots.write].map((path) => `${tomlString(path)}="read"`), + ...(instructionWorkingCopyRoot ? [`${tomlString(resolve(instructionWorkingCopyRoot))}="read"`] : []), ...(source.PAPERCLIP_GITHUB_BROKER_TOKEN && source.GH_CONFIG_DIR ? [`${tomlString(resolve(source.GH_CONFIG_DIR))}="write"`] : []), `":workspace_roots"={"."="read"}`, diff --git a/packages/paperclip-runner/src/drivers/runtime-context-materializer.test.ts b/packages/paperclip-runner/src/drivers/runtime-context-materializer.test.ts index 0898771d66..6f59edee75 100644 --- a/packages/paperclip-runner/src/drivers/runtime-context-materializer.test.ts +++ b/packages/paperclip-runner/src/drivers/runtime-context-materializer.test.ts @@ -20,6 +20,7 @@ import { PAPERCLIP_EXECUTION_PROMPT_REVISION, canonicalNativeRuntimeContextDigest, nativeRuntimePromptDigest, + parseNativeRuntimeContext, type NativeRuntimeContextSnapshot, } from "../contracts/runtime-context.js"; import { nativeMcpLaunchBinding } from "./native-mcp.js"; @@ -92,6 +93,17 @@ function context( } describe("runtime context materialization", () => { + it("restores the legacy working-copy contract without changing its digest or adding new fields", () => { + const old = context("/skills", "/instructions"); + old.instructions.workingCopy = { rootPath: "/old-run/copy", entryPath: "AGENTS.md" }; + old.aggregateDigest = canonicalNativeRuntimeContextDigest(old); + expect(parseNativeRuntimeContext(JSON.parse(JSON.stringify(old)))).toEqual(old); + expect(parseNativeRuntimeContext(old).instructions.workingCopy).not.toHaveProperty("kind"); + const next = { ...old, instructions: { ...old.instructions, workingCopy: { ...old.instructions.workingCopy, kind: "agent_files" as const } } }; + next.aggregateDigest = canonicalNativeRuntimeContextDigest(next); + expect(parseNativeRuntimeContext(next).instructions.workingCopy?.kind).toBe("agent_files"); + }); + it("validates native MCP launch bindings before they reach Codex", () => { expect(nativeMcpLaunchBinding({})).toBeNull(); expect(() => nativeMcpLaunchBinding({ diff --git a/packages/paperclip-runner/src/eval/workflow-evals.test.ts b/packages/paperclip-runner/src/eval/workflow-evals.test.ts index 8deb3154b1..14cbe7199c 100644 --- a/packages/paperclip-runner/src/eval/workflow-evals.test.ts +++ b/packages/paperclip-runner/src/eval/workflow-evals.test.ts @@ -466,13 +466,13 @@ describe("workflow reports and stress traceability", () => { candidateFailures: 36, }); expect(report.coverage).toMatchObject({ - canonicalOperations: 48, + canonicalOperations: 52, capabilityCases: 106, workflows: 12, stressFindings: 44, stressExclusions: 1, }); - expect(report.coverage.operations).toHaveLength(48); + expect(report.coverage.operations).toHaveLength(52); expect(report.coverage.composedWorkflows).toHaveLength(12); expect( report.coverage.operations.find( @@ -480,7 +480,7 @@ describe("workflow reports and stress traceability", () => { )?.workflowIds.length, ).toBeGreaterThan(0); expect(renderRunnerWorkflowMarkdown(report)).toContain( - "48 operations · 106 capability cases · 12 workflows", + "52 operations · 106 capability cases · 12 workflows", ); expect(renderRunnerWorkflowJUnit(report)).toContain( 'tests="36" failures="36" skipped="0"', diff --git a/packages/paperclip-runner/src/live/runnerd-codex-transport.ts b/packages/paperclip-runner/src/live/runnerd-codex-transport.ts index 1b35308400..5de9203e64 100644 --- a/packages/paperclip-runner/src/live/runnerd-codex-transport.ts +++ b/packages/paperclip-runner/src/live/runnerd-codex-transport.ts @@ -3277,6 +3277,7 @@ export function createRunnerdCodexAppServerArgs(input: { codexHome: string; codexCommand?: string; readOnlyRoots?: string[]; + instructionWorkingCopyRoot?: string; }): string[] { // The filesystem policy denies HOME and CODEX_HOME to keep credentials and // runner state outside provider reach. Always bind those names to the actual @@ -3289,6 +3290,7 @@ export function createRunnerdCodexAppServerArgs(input: { CODEX_HOME: input.codexHome, }, [...(input.readOnlyRoots ?? []), ...codexExecutableReadOnlyRoots(input.environment ?? {}, input.codexCommand)], + input.instructionWorkingCopyRoot, ); } @@ -4628,6 +4630,7 @@ class DurablePrpCodexTransport implements CodexAppServerTransport { environment: this.options.environment, codexHome, codexCommand: this.options.codexCommand, + instructionWorkingCopyRoot: runtimeContext?.instructions.workingCopy?.rootPath, readOnlyRoots: [ ...trustedRuntimeReadOnlyRoots( this.options.environment, @@ -5119,6 +5122,7 @@ class DurablePrpCodexTransport implements CodexAppServerTransport { environment: this.options.environment, codexHome, codexCommand: this.options.codexCommand, + instructionWorkingCopyRoot: runtimeContext?.instructions.workingCopy?.rootPath, readOnlyRoots: [ ...trustedRuntimeReadOnlyRoots(this.options.environment), ...(runtimeContext diff --git a/packages/paperclip-runner/src/native-session-runtime.test.ts b/packages/paperclip-runner/src/native-session-runtime.test.ts index ab82510cf9..1c02ca05c5 100644 --- a/packages/paperclip-runner/src/native-session-runtime.test.ts +++ b/packages/paperclip-runner/src/native-session-runtime.test.ts @@ -4014,6 +4014,45 @@ describe("executeNativeSession recovery", () => { }, ); + it("collects a failed run's private instructions only after its owned provider close joins", async () => { + const scopedIdentity = { ...identity, companyId: "instruction-close-company", runId: "instruction-close-run" }; + let finishClose!: () => void; + const closed = new Promise((resolve) => { finishClose = resolve; }); + const order: string[] = []; + const capabilities = { resume: true, typedEvents: true, steering: false, interruption: false, structuredResult: true }; + const session: NativeSession = { + identity: () => scopedIdentity, + capabilities: async () => capabilities, + async *events() { throw new Error("provider failed after editing instructions"); }, + startTurn: async () => ({ turnId: "instruction-turn" }), + result: async () => null, + snapshot: async () => ({ backendKind: "local", sessionId: "instruction-session", identity: scopedIdentity, + providerSessionId: "instruction-provider", cursor: null, activeTurnId: null, pendingRuntimeRequests: [], lineage: [] }), + close: async () => { order.push("closing"); await closed; order.push("stopped"); }, + }; + const backend: NativeSessionBackend = { + descriptor: async () => ({ kind: "local", name: "instruction-close-test", version: "1", capabilities }), + openSession: async () => session, + }; + const controlPlane: ControlPlanePort = { + openRun: async () => {}, checkpointSession: async () => {}, + appendEvent: async () => ({ cursor: 0, highestContiguousSourceSeq: 0, disposition: "committed" }), + replayEvents: async () => ({ events: [], highestContiguousSourceSeq: 0 }), completeRun: async () => {}, + }; + const options = { + input: { ...input, binding: { ...input.binding, companyId: scopedIdentity.companyId, runId: scopedIdentity.runId } }, + backend, controlPlane, runnerInstanceId: "instruction-runner", controlPlaneInstanceId: "control", + requireSessionCloseBeforeReturn: true, + onSessionClosed: async () => { order.push("collected"); }, + }; + const execution = executeNativeSession(options); + const failed = expect(execution).rejects.toThrow("provider failed after editing instructions"); + await vi.waitFor(() => expect(order).toEqual(["closing"])); + finishClose(); + await failed; + expect(order).toEqual(["closing", "stopped", "collected"]); + }); + it("retires only the terminated remote resource, including two sandboxes for one run", async () => { const scopedIdentity = { ...identity, companyId: "remote-stop-company", runId: "remote-stop-run" }; const binding = { ...scopedIdentity, remoteCleanupScope: "first-sandbox" }; @@ -4107,6 +4146,7 @@ describe("executeNativeSession recovery", () => { vi.useFakeTimers(); try { const closeFailure = new Error("required remote checkpoint close failed"); + const onSessionClosed = vi.fn(async () => {}); const close = vi.fn(({ reason }: { reason: string }) => reason === "native session quarantined cleanup recovery" ? Promise.resolve() @@ -4191,11 +4231,13 @@ describe("executeNativeSession recovery", () => { runnerInstanceId: "runner-recovery", controlPlaneInstanceId: "control-recovery", requireSessionCloseBeforeReturn: true, + onSessionClosed, }), ).rejects.toThrow(closeFailure); await vi.advanceTimersByTimeAsync(3_000); await execution; expect(close).toHaveBeenCalledTimes(5); + expect(onSessionClosed).not.toHaveBeenCalled(); } finally { vi.useRealTimers(); } diff --git a/packages/paperclip-runner/src/native-session-runtime.ts b/packages/paperclip-runner/src/native-session-runtime.ts index 408f3adc9e..27b5c204c3 100644 --- a/packages/paperclip-runner/src/native-session-runtime.ts +++ b/packages/paperclip-runner/src/native-session-runtime.ts @@ -220,6 +220,9 @@ export interface ExecuteNativeSessionOptions { * carries required persistence (for example, a remote runner checkpoint). */ requireSessionCloseBeforeReturn?: boolean; + /** Trusted cleanup observer, called only after the owned close completes. + * Requires requireSessionCloseBeforeReturn; never called for a live warm session. */ + onSessionClosed?: () => Promise; onCheckpoint?: ( snapshot: PersistedNativeSession, options?: CheckpointControlPlaneSessionOptions, @@ -2756,6 +2759,7 @@ export async function executeNativeSession( // generic retryable transport failure at the control-plane boundary. throw protocolIntegrityFailure ?? closeError; } + await options.onSessionClosed?.(); } } else if (shouldClose && !failedCleanupDeferred) { // A provider that ignores close must not keep execution pending forever. diff --git a/packages/paperclip-runner/src/protocol-actions/core.ts b/packages/paperclip-runner/src/protocol-actions/core.ts index ee19581527..8e0a5695da 100644 --- a/packages/paperclip-runner/src/protocol-actions/core.ts +++ b/packages/paperclip-runner/src/protocol-actions/core.ts @@ -1,3 +1,7 @@ +import { readAgentInstructionsAction } from "./read-agent-instructions.js"; +import { updateAgentInstructionsAction } from "./update-agent-instructions.js"; +import { getAgentInstructionHistoryAction } from "./get-agent-instruction-history.js"; +import { restoreAgentInstructionsAction } from "./restore-agent-instructions.js"; import { answerStatusQuestionAction } from "./answer-status-question.js"; import { blockTaskAction } from "./block-task.js"; import { finishTaskAction } from "./finish-task.js"; @@ -16,6 +20,11 @@ import { deepFreezeProtocolAction } from "./freeze.js"; /** Core actions that are always present in an authorized runner projection. */ export const PAPERCLIP_CORE_PROTOCOL_ACTIONS = deepFreezeProtocolAction([ + readAgentInstructionsAction, + updateAgentInstructionsAction, + getAgentInstructionHistoryAction, + restoreAgentInstructionsAction, + answerStatusQuestionAction, blockTaskAction, finishTaskAction, diff --git a/packages/paperclip-runner/src/protocol-actions/get-agent-instruction-history.ts b/packages/paperclip-runner/src/protocol-actions/get-agent-instruction-history.ts new file mode 100644 index 0000000000..11046457e0 --- /dev/null +++ b/packages/paperclip-runner/src/protocol-actions/get-agent-instruction-history.ts @@ -0,0 +1,99 @@ +/** Canonical definition for `get_agent_instruction_history`. */ +export const getAgentInstructionHistoryAction = { + "id": "get_agent_instruction_history", + "canonical": { + "operationId": "get_agent_instruction_history", + "surfaces": [ + "live" + ], + "placement": "always_agent_tool", + "optionalGroup": null, + "requiredClaims": [], + "taskModes": [ + "standard", + "ask", + "planning", + "skill_test" + ], + "sideEffectClass": "read", + "idempotency": "none", + "disabledByDefault": false, + "realBindingStatus": "live_codex", + "realServiceBinding": "agentInstructionRevisionService", + "prpEvidence": "tool-result item event plus canonical revision receipt for writes; no scenario mock coverage", + "prpBindingStatus": "bound", + "legacyAliases": [], + "note": "Live canonical instruction service; company scope and current responsible-user authorization are checked at invocation." + }, + "documentation": { + "title": "Get agent instruction history", + "description": "List bounded canonical instruction revision metadata, including actor, source run, and restore origin. Use read_agent_instructions with entryFile and revisionId to inspect exact historical content.", + "note": null + }, + "examples": { + "call": { + "operationId": "get_agent_instruction_history", + "input": { + "entryFile": "AGENTS.md" + } + }, + "success": { + "ok": true, + "operationId": "get_agent_instruction_history", + "result": {} + } + }, + "live": { + "order": 42, + "descriptor": { + "schema": "paperclip.semantic-tool.v1", + "operationId": "get_agent_instruction_history", + "version": 1, + "title": "Get agent instruction history", + "description": "List bounded canonical instruction revision metadata, including actor, source run, and restore origin. Use read_agent_instructions with entryFile and revisionId to inspect exact historical content.", + "exposure": "always", + "requiredClaims": [], + "allowedModes": [ + "standard", + "ask", + "planning", + "skill_test" + ], + "inputSchema": { + "type": "object", + "properties": { + "targetAgentId": { + "type": "string", + "pattern": "^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$", + "description": "Same-company target agent. Omit to use the calling agent." + }, + "entryFile": { + "type": "string", + "minLength": 1, + "maxLength": 4096, + "description": "Configured relative entry filename returned by read_agent_instructions; retain it with the revision." + }, + "cursor": { + "type": "string", + "minLength": 1, + "maxLength": 2048 + }, + "limit": { + "type": "integer", + "minimum": 1, + "maximum": 100 + } + }, + "required": [ + "entryFile" + ], + "additionalProperties": false + }, + "outputSchema": { + "type": "object", + "additionalProperties": true + } + } + }, + "scenario": null +} as const; diff --git a/packages/paperclip-runner/src/protocol-actions/index.ts b/packages/paperclip-runner/src/protocol-actions/index.ts index 8f07df6695..4dcfcc66b0 100644 --- a/packages/paperclip-runner/src/protocol-actions/index.ts +++ b/packages/paperclip-runner/src/protocol-actions/index.ts @@ -1,3 +1,7 @@ +import { readAgentInstructionsAction } from "./read-agent-instructions.js"; +import { updateAgentInstructionsAction } from "./update-agent-instructions.js"; +import { getAgentInstructionHistoryAction } from "./get-agent-instruction-history.js"; +import { restoreAgentInstructionsAction } from "./restore-agent-instructions.js"; import { reassignTaskAction } from "./reassign-task.js"; import { createSkillAction } from "./create-skill.js"; import { createProjectAction } from "./create-project.js"; @@ -49,6 +53,11 @@ import { writeDocumentAction } from "./write-document.js"; import { deepFreezeProtocolAction } from "./freeze.js"; export const PAPERCLIP_PROTOCOL_ACTIONS = deepFreezeProtocolAction([ + readAgentInstructionsAction, + updateAgentInstructionsAction, + getAgentInstructionHistoryAction, + restoreAgentInstructionsAction, + createSkillAction, createProjectAction, listProjectRepositoriesAction, diff --git a/packages/paperclip-runner/src/protocol-actions/read-agent-instructions.ts b/packages/paperclip-runner/src/protocol-actions/read-agent-instructions.ts new file mode 100644 index 0000000000..9a947d1a2c --- /dev/null +++ b/packages/paperclip-runner/src/protocol-actions/read-agent-instructions.ts @@ -0,0 +1,90 @@ +/** Canonical definition for `read_agent_instructions`. */ +export const readAgentInstructionsAction = { + "id": "read_agent_instructions", + "canonical": { + "operationId": "read_agent_instructions", + "surfaces": [ + "live" + ], + "placement": "always_agent_tool", + "optionalGroup": null, + "requiredClaims": [], + "taskModes": [ + "standard", + "ask", + "planning", + "skill_test" + ], + "sideEffectClass": "read", + "idempotency": "none", + "disabledByDefault": false, + "realBindingStatus": "live_codex", + "realServiceBinding": "agentInstructionRevisionService", + "prpEvidence": "tool-result item event plus canonical revision receipt for writes; no scenario mock coverage", + "prpBindingStatus": "bound", + "legacyAliases": [], + "note": "Live canonical instruction service; company scope and current responsible-user authorization are checked at invocation." + }, + "documentation": { + "title": "Read canonical agent instructions", + "description": "Read the current canonical instruction entry and its revision, or inspect a historical revision by supplying both entryFile and revisionId. Read before updating; do not edit shared instruction caches.", + "note": null + }, + "examples": { + "call": { + "operationId": "read_agent_instructions", + "input": {} + }, + "success": { + "ok": true, + "operationId": "read_agent_instructions", + "result": {} + } + }, + "live": { + "order": 40, + "descriptor": { + "schema": "paperclip.semantic-tool.v1", + "operationId": "read_agent_instructions", + "version": 1, + "title": "Read canonical agent instructions", + "description": "Read the current canonical instruction entry and its revision, or inspect a historical revision by supplying both entryFile and revisionId. Read before updating; do not edit shared instruction caches.", + "exposure": "always", + "requiredClaims": [], + "allowedModes": [ + "standard", + "ask", + "planning", + "skill_test" + ], + "inputSchema": { + "type": "object", + "properties": { + "targetAgentId": { + "type": "string", + "pattern": "^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$", + "description": "Same-company target agent. Omit to use the calling agent." + }, + "entryFile": { + "type": "string", + "minLength": 1, + "maxLength": 4096, + "description": "Configured relative entry filename returned by read_agent_instructions; retain it with the revision." + }, + "revisionId": { + "type": "string", + "pattern": "^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$", + "description": "Historical revision to inspect; also provide its entryFile." + } + }, + "required": [], + "additionalProperties": false + }, + "outputSchema": { + "type": "object", + "additionalProperties": true + } + } + }, + "scenario": null +} as const; diff --git a/packages/paperclip-runner/src/protocol-actions/restore-agent-instructions.ts b/packages/paperclip-runner/src/protocol-actions/restore-agent-instructions.ts new file mode 100644 index 0000000000..464a556f41 --- /dev/null +++ b/packages/paperclip-runner/src/protocol-actions/restore-agent-instructions.ts @@ -0,0 +1,99 @@ +/** Canonical definition for `restore_agent_instructions`. */ +export const restoreAgentInstructionsAction = { + "id": "restore_agent_instructions", + "canonical": { + "operationId": "restore_agent_instructions", + "surfaces": [ + "live" + ], + "placement": "always_agent_tool", + "optionalGroup": null, + "requiredClaims": [], + "taskModes": [ + "standard", + "planning" + ], + "sideEffectClass": "company_write", + "idempotency": "recommended", + "disabledByDefault": false, + "realBindingStatus": "live_codex", + "realServiceBinding": "agentInstructionRevisionService", + "prpEvidence": "tool-result item event plus canonical revision receipt for writes; no scenario mock coverage", + "prpBindingStatus": "bound", + "legacyAliases": [], + "note": "Live canonical instruction service; company scope and current responsible-user authorization are checked at invocation." + }, + "documentation": { + "title": "Restore canonical agent instructions", + "description": "Append a historical instruction revision as the current content using the current baseRevisionId. Requires the responsible user\u2019s current target edit permission. History remains intact; conflicts require an explicit resolution.", + "note": null + }, + "examples": { + "call": { + "operationId": "restore_agent_instructions", + "input": { + "entryFile": "AGENTS.md", + "revisionId": "00000000-0000-4000-8000-000000000001", + "baseRevisionId": "00000000-0000-4000-8000-000000000002" + } + }, + "success": { + "ok": true, + "operationId": "restore_agent_instructions", + "result": {} + } + }, + "live": { + "order": 43, + "descriptor": { + "schema": "paperclip.semantic-tool.v1", + "operationId": "restore_agent_instructions", + "version": 1, + "title": "Restore canonical agent instructions", + "description": "Append a historical instruction revision as the current content using the current baseRevisionId. Requires the responsible user\u2019s current target edit permission. History remains intact; conflicts require an explicit resolution.", + "exposure": "always", + "requiredClaims": [], + "allowedModes": [ + "standard", + "planning" + ], + "inputSchema": { + "type": "object", + "properties": { + "targetAgentId": { + "type": "string", + "pattern": "^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$", + "description": "Same-company target agent. Omit to use the calling agent." + }, + "entryFile": { + "type": "string", + "minLength": 1, + "maxLength": 4096, + "description": "Configured relative entry filename returned by read_agent_instructions; retain it with the revision." + }, + "revisionId": { + "type": "string", + "pattern": "^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$", + "description": "Historical revision whose exact content should be restored." + }, + "baseRevisionId": { + "type": "string", + "description": "Current revision read before restoring. Never replace a stale base silently.", + "pattern": "^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$" + } + }, + "required": [ + "entryFile", + "revisionId", + "baseRevisionId" + ], + "additionalProperties": false + }, + "outputSchema": { + "type": "object", + "additionalProperties": true + } + } + }, + "scenario": null +} as const; diff --git a/packages/paperclip-runner/src/protocol-actions/update-agent-instructions.ts b/packages/paperclip-runner/src/protocol-actions/update-agent-instructions.ts new file mode 100644 index 0000000000..a42356b6ca --- /dev/null +++ b/packages/paperclip-runner/src/protocol-actions/update-agent-instructions.ts @@ -0,0 +1,102 @@ +/** Canonical definition for `update_agent_instructions`. */ +export const updateAgentInstructionsAction = { + "id": "update_agent_instructions", + "canonical": { + "operationId": "update_agent_instructions", + "surfaces": [ + "live" + ], + "placement": "always_agent_tool", + "optionalGroup": null, + "requiredClaims": [], + "taskModes": [ + "standard", + "planning" + ], + "sideEffectClass": "company_write", + "idempotency": "recommended", + "disabledByDefault": false, + "realBindingStatus": "live_codex", + "realServiceBinding": "agentInstructionRevisionService", + "prpEvidence": "tool-result item event plus canonical revision receipt for writes; no scenario mock coverage", + "prpBindingStatus": "bound", + "legacyAliases": [], + "note": "Live canonical instruction service; company scope and current responsible-user authorization are checked at invocation." + }, + "documentation": { + "title": "Update canonical agent instructions", + "description": "Commit instruction content with the exact entryFile and baseRevisionId from a prior read. Requires the responsible user\u2019s current target edit permission. On conflict, preserve your candidate and explicitly resolve it; never overwrite the newer head.", + "note": null + }, + "examples": { + "call": { + "operationId": "update_agent_instructions", + "input": { + "entryFile": "AGENTS.md", + "content": "# Instructions\n", + "baseRevisionId": null + } + }, + "success": { + "ok": true, + "operationId": "update_agent_instructions", + "result": {} + } + }, + "live": { + "order": 41, + "descriptor": { + "schema": "paperclip.semantic-tool.v1", + "operationId": "update_agent_instructions", + "version": 1, + "title": "Update canonical agent instructions", + "description": "Commit instruction content with the exact entryFile and baseRevisionId from a prior read. Requires the responsible user\u2019s current target edit permission. On conflict, preserve your candidate and explicitly resolve it; never overwrite the newer head.", + "exposure": "always", + "requiredClaims": [], + "allowedModes": [ + "standard", + "planning" + ], + "inputSchema": { + "type": "object", + "properties": { + "targetAgentId": { + "type": "string", + "pattern": "^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$", + "description": "Same-company target agent. Omit to use the calling agent." + }, + "entryFile": { + "type": "string", + "minLength": 1, + "maxLength": 4096, + "description": "Configured relative entry filename returned by read_agent_instructions; retain it with the revision." + }, + "content": { + "type": "string", + "maxLength": 1048576, + "description": "Complete UTF-8 instruction content, at most 1 MiB; empty content is valid." + }, + "baseRevisionId": { + "type": [ + "string", + "null" + ], + "description": "Revision read before editing; null only when no canonical entry exists. Never replace a stale base silently.", + "pattern": "^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$" + } + }, + "required": [ + "entryFile", + "content", + "baseRevisionId" + ], + "additionalProperties": false + }, + "outputSchema": { + "type": "object", + "additionalProperties": true + } + } + }, + "scenario": null +} as const; diff --git a/packages/paperclip-runner/src/semantic-tools/discovery.ts b/packages/paperclip-runner/src/semantic-tools/discovery.ts index d1bf2bd433..7d6c28a0d2 100644 --- a/packages/paperclip-runner/src/semantic-tools/discovery.ts +++ b/packages/paperclip-runner/src/semantic-tools/discovery.ts @@ -27,7 +27,12 @@ const NAMESPACE: Readonly> = Objec report_progress: "active_task", answer_status_question: "active_task", write_document: "documents", request_human_input: "documents", register_deliverable: "documents", finish_task: "active_task", block_task: "active_task", request_review: "active_task", search_tasks: "discovery", - list_agents: "discovery", get_agent: "discovery", create_skill: "skills", create_task: "delegation", create_project: "projects", list_project_repositories: "projects", list_projects: "projects", + list_agents: "discovery", get_agent: "discovery", + read_agent_instructions: "agent_instructions", + update_agent_instructions: "agent_instructions", + get_agent_instruction_history: "agent_instructions", + restore_agent_instructions: "agent_instructions", + create_skill: "skills", create_task: "delegation", create_project: "projects", list_project_repositories: "projects", list_projects: "projects", set_dependencies: "delegation", reassign_task: "delegation", list_approvals: "governance", get_approval: "governance", get_approval_context: "governance", request_approval: "governance", decide_approval: "governance", comment_on_approval: "governance", @@ -102,6 +107,9 @@ export function discoverCapabilityDefinitions( const tokens = normalized.split(/[^a-z0-9]+/).filter((token) => token.length > 1); const permitted = CAPABILITY_SEMANTIC_TOOL_CATALOG .filter((descriptor) => descriptor.exposure === "optional") + // Retain dispatch support for pre-upgrade sessions, but do not advertise + // revision history/restore as a capability of current agent directories. + .filter((descriptor) => descriptor.operationId !== "get_agent_instruction_history" && descriptor.operationId !== "restore_agent_instructions") .filter((descriptor) => options.namespace === undefined || capabilityToolNamespace(descriptor.operationId) === options.namespace) .filter((descriptor) => decideCapabilitySemanticAuthorization(descriptor, context, "exposure").allowed) .map((descriptor) => ({ descriptor, score: score(descriptor, tokens, normalized) })) diff --git a/packages/paperclip-runner/src/semantic-tools/semantic-tools.test.ts b/packages/paperclip-runner/src/semantic-tools/semantic-tools.test.ts index af1344a298..1e6ebc1bb5 100644 --- a/packages/paperclip-runner/src/semantic-tools/semantic-tools.test.ts +++ b/packages/paperclip-runner/src/semantic-tools/semantic-tools.test.ts @@ -94,7 +94,7 @@ describe("Capability semantic catalog and authorization", () => { it("publishes a stable narrow catalog without credentials or control-plane-owned tools", () => { const names = CAPABILITY_SEMANTIC_TOOL_CATALOG.map((tool) => tool.operationId); expect(new Set(names).size).toBe(names.length); - expect(names).toHaveLength(36); + expect(names).toHaveLength(40); expect(names).toContain("get_task_context"); expect(names).toContain("finish_task"); expect(names).not.toContain("checkout_task"); diff --git a/packages/paperclip-runner/src/semantic-tools/types.ts b/packages/paperclip-runner/src/semantic-tools/types.ts index ea9339edf8..31178d2690 100644 --- a/packages/paperclip-runner/src/semantic-tools/types.ts +++ b/packages/paperclip-runner/src/semantic-tools/types.ts @@ -33,6 +33,10 @@ export type CapabilitySemanticOperationId = | "request_review" | "list_agents" | "get_agent" + | "read_agent_instructions" + | "update_agent_instructions" + | "get_agent_instruction_history" + | "restore_agent_instructions" | "search_tasks" | "list_approvals" | "get_approval" diff --git a/packages/shared/src/index.ts b/packages/shared/src/index.ts index 04b1216786..9591a8ce37 100644 --- a/packages/shared/src/index.ts +++ b/packages/shared/src/index.ts @@ -2789,3 +2789,10 @@ export { MEMORY_CONNECTOR_IDS, isMemoryConnectorId, type MemoryConnectorId } fro export * from "./connection-routing.js"; export { WORKSPACE_RESTORE_FAILURE_CODES, hasWorkspaceRestoreFailure, safeWorkspaceRestorePath, isNativeWorkspaceExportRepairCause } from "./workspace-restore.js"; + + +export type { AgentInstructionErrorCode, AgentInstructionErrorDetails, AgentInstructionSource, AgentInstructionRevision, AgentInstructionSnapshot, AgentInstructionCommitReceipt, AgentInstructionHistory, AgentInstructionDiff } from "./types/agent.js"; +export { restoreAgentInstructionSchema } from "./validators/agent.js"; + +export type { AgentInstructionCandidate } from "./types/agent.js"; +export { resolveAgentInstructionCandidateSchema, type ResolveAgentInstructionCandidate } from "./validators/agent.js"; diff --git a/packages/shared/src/types/agent.ts b/packages/shared/src/types/agent.ts index e775c5b994..443e61f217 100644 --- a/packages/shared/src/types/agent.ts +++ b/packages/shared/src/types/agent.ts @@ -32,6 +32,8 @@ export type AgentInstructionsBundleMode = "managed" | "external"; export interface AgentInstructionsFileSummary { path: string; + contentHash?: string; + binary?: boolean; size: number; language: string; markdown: boolean; @@ -43,11 +45,14 @@ export interface AgentInstructionsFileSummary { export interface AgentInstructionsFileDetail extends AgentInstructionsFileSummary { content: string; + revision?: AgentInstructionRevision; + receipt?: AgentInstructionCommitReceipt; } export interface AgentInstructionsBundle { agentId: string; companyId: string; + persistence?: "agent_files"; mode: AgentInstructionsBundleMode | null; rootPath: string | null; managedRootPath: string; @@ -336,3 +341,77 @@ export type AdapterAuthSignal = "present" | "absent" | "unknown"; export interface AdapterAuthSignalResponse { status: AdapterAuthSignal; } + +export type AgentInstructionSource = "seed" | "board" | "api" | "tool" | "cleanup" | "restore"; +export interface AgentInstructionRevision { + id: string; + companyId: string; + agentId: string; + entryFile: string; + contentHash: string; + byteLength: number; + parentRevisionId: string | null; + baseRevisionId: string | null; + restoredFromRevisionId: string | null; + actorAgentId: string | null; + actorUserId: string | null; + responsibleUserId: string | null; + sourceRunId: string | null; + source: AgentInstructionSource; + createdAt: string; +} +export interface AgentInstructionSnapshot { + revision: AgentInstructionRevision; + content: string; +} +export interface AgentInstructionCommitReceipt extends AgentInstructionSnapshot { + changed: boolean; + materialization: "current" | "pending"; +} +export interface AgentInstructionHistory { + revisions: AgentInstructionRevision[]; + nextCursor: string | null; +} +export interface AgentInstructionDiff { + from: AgentInstructionSnapshot; + to: AgentInstructionSnapshot; + prefix: string; + removed: string; + added: string; + suffix: string; +} + +export type AgentInstructionErrorCode = + | "INSTRUCTION_BASE_REQUIRED" + | "INSTRUCTION_REVISION_REQUIRED" + | "INSTRUCTION_REVISION_CONFLICT" + | "INSTRUCTION_ENTRY_CHANGED" + | "INSTRUCTION_MANAGED_BUNDLE_REQUIRED" + | "INSTRUCTION_IDENTITY_INVALID" + | "INSTRUCTION_CONTENT_INVALID" + | "INSTRUCTION_PATH_INVALID" + | "RESPONSIBLE_USER_UNAVAILABLE" + | "RESPONSIBLE_USER_UNAUTHORIZED"; +export interface AgentInstructionErrorDetails { + code: AgentInstructionErrorCode; + baseRevisionId?: string | null; + currentRevisionId?: string | null; + entryFile?: string; + reason?: string; +} + +/** Preserved instruction edits; filesystem locations and responsible identity stay server-side. */ +export interface AgentInstructionCandidate { + contract?: "agent_files" | "legacy"; + runId: string; + entryFile: string; + baseRevisionId: string | null; + baseHash: string; + state: "conflict" | "pending_collection" | "pending_commit" | "unavailable"; + candidateHash: string | null; + content: string | null; + errorCode: string | null; + errorMessage: string | null; + createdAt: string; + updatedAt: string; +} diff --git a/packages/shared/src/types/index.ts b/packages/shared/src/types/index.ts index 179832a3de..87ace9cd47 100644 --- a/packages/shared/src/types/index.ts +++ b/packages/shared/src/types/index.ts @@ -1069,3 +1069,7 @@ export * from "./chat-channels.js"; export * from "./chat-github.js"; export * from "./email.js"; + +export type { AgentInstructionErrorCode, AgentInstructionErrorDetails, AgentInstructionSource, AgentInstructionRevision, AgentInstructionSnapshot, AgentInstructionCommitReceipt, AgentInstructionHistory, AgentInstructionDiff } from "./agent.js"; + +export type { AgentInstructionCandidate } from "./agent.js"; diff --git a/packages/shared/src/validators/agent.ts b/packages/shared/src/validators/agent.ts index e40a7cf6ce..9c2ec1d061 100644 --- a/packages/shared/src/validators/agent.ts +++ b/packages/shared/src/validators/agent.ts @@ -34,10 +34,18 @@ export const updateAgentInstructionsBundleSchema = z.object({ export type UpdateAgentInstructionsBundle = z.infer; export const upsertAgentInstructionsFileSchema = z.object({ - path: z.string().trim().min(1), - content: z.string(), + path: z.string().min(1).max(512), + content: z.string().max(1024 * 1024), + baseRevisionId: z.string().uuid().nullable().optional(), + baseHash: z.string().regex(/^[a-f0-9]{64}$/).nullable().optional(), clearLegacyPromptTemplate: z.boolean().optional().default(false), -}); +}).strict(); + +export const restoreAgentInstructionSchema = z.object({ + path: z.string().min(1).max(512), + revisionId: z.string().uuid(), + baseRevisionId: z.string().uuid(), +}).strict(); export type UpsertAgentInstructionsFile = z.infer; @@ -289,3 +297,9 @@ export const updateAgentPermissionsSchema = z.object({ }); export type UpdateAgentPermissions = z.infer; + +export const resolveAgentInstructionCandidateSchema = z.object({ + baseRevisionId: z.string().uuid().nullable(), + content: z.string().max(1024 * 1024), +}).strict(); +export type ResolveAgentInstructionCandidate = z.infer; diff --git a/packages/shared/src/validators/index.ts b/packages/shared/src/validators/index.ts index a39388a080..8489488d1c 100644 --- a/packages/shared/src/validators/index.ts +++ b/packages/shared/src/validators/index.ts @@ -985,3 +985,5 @@ export * from "./chat-channels.js"; export * from "./chat-github.js"; export * from "./email.js"; + +export { restoreAgentInstructionSchema } from "./agent.js"; diff --git a/server/src/__tests__/agent-directory-working-copies.test.ts b/server/src/__tests__/agent-directory-working-copies.test.ts new file mode 100644 index 0000000000..17d7acdb88 --- /dev/null +++ b/server/src/__tests__/agent-directory-working-copies.test.ts @@ -0,0 +1,539 @@ +import fs from "node:fs/promises"; +import { execFile as execFileCallback } from "node:child_process"; +import { promisify } from "node:util"; +import * as executionTargetTools from "@paperclipai/adapter-utils/execution-target"; +import * as ssh from "@paperclipai/adapter-utils/ssh"; +const execFile = promisify(execFileCallback); +import os from "node:os"; +import path from "node:path"; +import { createHash, randomUUID } from "node:crypto"; +import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from "vitest"; +import { eq } from "drizzle-orm"; +import { agentFileStore, fileHash, inspectAgentFile, snapshotAgentFiles, MAX_AGENT_FILE_BYTES, MAX_AGENT_DIRECTORY_BYTES, MAX_AGENT_DIRECTORY_ENTRIES } from "../services/agent-file-store.js"; +import { agents, companies, authUsers, companyMemberships, principalPermissionGrants, heartbeatRuns, environmentLeases, environments, agentInstructionWorkingCopies, agentInstructionRevisions, agentInstructionHeads, createDb } from "@paperclipai/db"; +import { startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js"; +import { agentInstructionRevisionService } from "../services/agent-instruction-revisions.js"; +import { agentInstructionWorkingCopyService, instructionWorkingCopyGuidance } from "../services/agent-instruction-working-copies.js"; +import { resolveManagedInstructionsRoot } from "../services/agent-instructions.js"; +import { buildNativeRuntimeContext } from "../services/native-runtime/runtime-context.js"; +import type { EnvironmentRuntimeService } from "../services/environment-runtime.js"; +import { remoteTerminationReceipt } from "../services/remote-execution-termination.js"; + +describe("persistent agent directories", () => { + let database: Awaited>; + let db: ReturnType; + let copies: ReturnType; + let revisions: ReturnType; + const previousHome = process.env.PAPERCLIP_HOME; + let home: string; + let companyId: string, agentId: string, userId: string, root: string; + const entryFile = "policy/INSTRUCTIONS.txt"; + const initial = "\uFEFF# Original\r\n☃\n"; + const target = () => ({ companyId, agentId }); + const board = () => ({ type: "board" as const, userId, source: "session" as const }); + async function run() { + const runId = randomUUID(); + await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, invocationSource: "on_demand", responsibleUserId: userId }); + return (await copies.prepare({ ...target(), runId, cwd: home }))!; + } + beforeAll(async () => { + home = await fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), "instruction-working-copies-"))); + process.env.PAPERCLIP_HOME = home; + database = await startEmbeddedPostgresTestDatabase("instruction-copies-db-"); + db = createDb(database.connectionString); + copies = agentInstructionWorkingCopyService(db); + revisions = agentInstructionRevisionService(db); + }, 90_000); + afterAll(async () => { + if (previousHome === undefined) delete process.env.PAPERCLIP_HOME; else process.env.PAPERCLIP_HOME = previousHome; + await database?.cleanup(); + if (home) { + const writable = async (dir: string) => { + await fs.chmod(dir, 0o700); + for (const entry of await fs.readdir(dir, { withFileTypes: true })) if (entry.isDirectory()) await writable(path.join(dir, entry.name)); + }; + await writable(home); + await fs.rm(home, { recursive: true, force: true }); + } + }); + beforeEach(async () => { + companyId = randomUUID(); agentId = randomUUID(); userId = randomUUID(); + await db.insert(companies).values({ id: companyId, name: "Instruction tests", issuePrefix: randomUUID().slice(0, 8) }); + await db.insert(authUsers).values({ id: userId, name: "Editor", email: `${userId}@example.test`, createdAt: new Date(), updatedAt: new Date() }); + root = resolveManagedInstructionsRoot({ ...target(), id: agentId, name: "Target", adapterConfig: {} }); + await db.insert(agents).values({ id: agentId, companyId, name: "Target", adapterConfig: { instructionsBundleMode: "managed", instructionsRootPath: root, instructionsEntryFile: entryFile } }); + await db.insert(companyMemberships).values([ + { companyId, principalType: "user", principalId: userId, membershipRole: "operator" }, + { companyId, principalType: "agent", principalId: agentId, membershipRole: "member" }, + ]); + await db.insert(principalPermissionGrants).values({ companyId, principalType: "user", principalId: userId, permissionKey: "agents:configure", scope: { agentIds: [agentId] } }); + await fs.mkdir(path.dirname(path.join(root, entryFile)), { recursive: true }); + await fs.writeFile(path.join(root, entryFile), initial); + }); + + it.each([".paperclip-runtime/state", "notes/.paperclip-runtime/state", "promptTemplate.legacy.md"])("rejects reserved board path %s before mutation", async (reserved) => { + await expect(agentFileStore(db).write({ ...target(), path: reserved, bytes: Buffer.from("reserved"), baseHash: null }, board())).rejects.toMatchObject({ status: 422 }); + await expect(fs.stat(path.join(root, reserved))).rejects.toMatchObject({ code: "ENOENT" }); + expect(await run()).toBeTruthy(); + }); + + it("round trips nested text, empty directories and binary bytes independently of task files", async () => { + const first = await run(); + expect(first.localRoot.startsWith(path.join(home, ".paperclip-runtime"))).toBe(false); + await fs.mkdir(path.join(first.localRoot, "notes", "empty"), { recursive: true }); + await fs.writeFile(path.join(first.localRoot, "notes", "fact.txt"), "remember me"); + const bytes = Buffer.from([0, 255, 17, 128, 9]); + await fs.writeFile(path.join(first.localRoot, "image.bin"), bytes); + await fs.writeFile(path.join(home, "task-only.txt"), "not personal"); + expect((await copies.collectStopped({ companyId, runId: first.runId }))?.state).toBe("saved"); + copies = agentInstructionWorkingCopyService(db); + const next = await run(); + expect(await fs.readFile(path.join(next.localRoot, "notes", "fact.txt"), "utf8")).toBe("remember me"); + expect(await fs.readFile(path.join(next.localRoot, "image.bin"))).toEqual(bytes); + expect((await fs.stat(path.join(next.localRoot, "notes", "empty"))).isDirectory()).toBe(true); + await expect(fs.stat(path.join(next.localRoot, "task-only.txt"))).rejects.toMatchObject({ code: "ENOENT" }); + expect(await db.select().from(agentInstructionRevisions).where(eq(agentInstructionRevisions.agentId, agentId))).toHaveLength(0); + }); + + async function sparseFile(filename: string, size: number) { + const handle = await fs.open(filename, "w"); + try { await handle.truncate(size); } finally { await handle.close(); } + } + + it("saves and restores files beyond the former file and folder limits, with streaming inspection and download", async () => { + const first = await run(); + const size = 66 * 1024 * 1024; + await sparseFile(path.join(first.localRoot, "large.bin"), size); + expect((await copies.collectStopped({ companyId, runId: first.runId }))?.state).toBe("saved"); + await copies.release(companyId, first.runId); + const next = await run(); + const restored = await inspectAgentFile(next.localRoot, "large.bin"); + expect(restored).toMatchObject({ size, bytes: null }); + const store = agentFileStore(db); + const download = (await store.download(companyId, agentId, "large.bin", board()))!; + expect(download.size).toBe(size); + // The lock is released before consuming the stream. An editor replacement + // must not change the already opened download's bytes. + await store.write({ ...target(), path: "large.bin", bytes: Buffer.from("replacement"), baseHash: restored!.hash }, board()); + const hash = createHash("sha256"); + let downloaded = 0; + for await (const chunk of download.stream) { downloaded += chunk.length; hash.update(chunk); } + expect(downloaded).toBe(size); + expect(hash.digest("hex")).toBe(restored!.hash); + expect(await fs.readFile(path.join(root, "large.bin"), "utf8")).toBe("replacement"); + await store.write({ ...target(), path: "empty.bin", bytes: Buffer.alloc(0), baseHash: null }, board()); + const empty = (await store.download(companyId, agentId, "empty.bin", board()))!; + expect(empty.size).toBe(0); + for await (const _chunk of empty.stream) throw new Error("Empty download must have no chunks"); + }, 30_000); + + it("accepts the exact 256 MiB file boundary without retaining a text buffer", async () => { + expect(MAX_AGENT_FILE_BYTES).toBe(256 * 1024 * 1024); + await sparseFile(path.join(root, "boundary.bin"), MAX_AGENT_FILE_BYTES); + expect(await inspectAgentFile(root, "boundary.bin")).toMatchObject({ size: MAX_AGENT_FILE_BYTES, bytes: null }); + }); + + it("checks quota for a small editor save without reading unrelated file bytes", async () => { + const asset = path.join(root, "large.bin"); + await sparseFile(asset, MAX_AGENT_FILE_BYTES); + const open = vi.spyOn(fs, "open"); + try { + await agentFileStore(db).write({ ...target(), path: "note.txt", bytes: Buffer.from("small edit"), baseHash: null }, board()); + expect(open.mock.calls.some(([filename]) => filename === asset)).toBe(false); + expect(await fs.readFile(path.join(root, "note.txt"), "utf8")).toBe("small edit"); + } finally { open.mockRestore(); } + }); + + it("reports an oversized run file without a partial save and removes its temporary copy", async () => { + const copy = await run(); + await sparseFile(path.join(copy.localRoot, "too-large.bin"), MAX_AGENT_FILE_BYTES + 1); + await fs.writeFile(path.join(copy.localRoot, entryFile), "changed instructions"); + const failed = await copies.collectStopped({ companyId, runId: copy.runId }); + expect(failed).toMatchObject({ state: "unavailable", errorCode: "AGENT_FILES_LIMIT_EXCEEDED", candidateHash: null, nextAttemptAt: null, attempts: 1 }); + expect(failed?.receipt?.storageWarning).toContain("Runs can continue"); + expect(failed?.errorMessage).toContain('"too-large.bin" exceeds the 256 MiB'); + expect(await fs.readFile(path.join(root, entryFile), "utf8")).toBe(initial); + await expect(fs.stat(path.join(root, "too-large.bin"))).rejects.toMatchObject({ code: "ENOENT" }); + expect((await copies.reportUnavailable(companyId, copy.runId))?.errorCode).toBe("AGENT_FILES_LIMIT_EXCEEDED"); + await copies.release(companyId, copy.runId); + await expect(fs.stat(copy.localRoot)).rejects.toMatchObject({ code: "ENOENT" }); + expect((await copies.get(companyId, copy.runId))?.receipt?.baseline).toBeUndefined(); + const next = await run(); + expect(await fs.readFile(path.join(next.localRoot, entryFile), "utf8")).toBe(initial); + await fs.writeFile(path.join(next.localRoot, "next-task.txt"), "still working"); + expect((await copies.collectStopped({ companyId, runId: next.runId }))?.state).toBe("saved"); + expect(await fs.readFile(path.join(root, "next-task.txt"), "utf8")).toBe("still working"); + }); + + it("warns on each run at the file limit and clears the warning after ordinary agent cleanup", async () => { + await sparseFile(path.join(root, "full.bin"), MAX_AGENT_FILE_BYTES); + const first = await run(); + expect(first.receipt?.storageWarning).toContain("256 MiB"); + expect(instructionWorkingCopyGuidance(first)).toContain("Runs can continue"); + const unchanged = await copies.collectStopped({ companyId, runId: first.runId }); + expect(unchanged).toMatchObject({ state: "unchanged", errorCode: null }); + expect(unchanged?.receipt?.storageWarning).toContain("Agent storage is full"); + copies = agentInstructionWorkingCopyService(db); + const second = await run(); + expect(second.receipt?.storageWarning).toContain("Agent storage is full"); + await fs.unlink(path.join(second.localRoot, "full.bin")); + await fs.writeFile(path.join(second.localRoot, "task-output.txt"), "work continues"); + expect(await copies.collectStopped({ companyId, runId: second.runId })).toMatchObject({ state: "saved", receipt: { storageWarning: null } }); + const next = await run(); + expect(next.receipt?.storageWarning).toBeNull(); + expect(await fs.readFile(path.join(next.localRoot, "task-output.txt"), "utf8")).toBe("work continues"); + }, 30_000); + + it("starts successive runs with an already oversized saved file and lets the agent remove it", async () => { + await sparseFile(path.join(root, "old-large.bin"), MAX_AGENT_FILE_BYTES + 1); + const first = await run(); + expect(first.state).toBe("prepared"); + expect(first.receipt?.storageWarning).toContain("256 MiB"); + expect(await fs.readFile(path.join(first.localRoot, entryFile), "utf8")).toBe(initial); + const stopped = await copies.collectStopped({ companyId, runId: first.runId }); + expect(stopped).toMatchObject({ state: "unavailable", errorCode: "AGENT_FILES_LIMIT_EXCEEDED" }); + await expect(fs.stat(first.localRoot)).rejects.toMatchObject({ code: "ENOENT" }); + copies = agentInstructionWorkingCopyService(db); + const cleanup = await run(); + expect(cleanup.state).toBe("prepared"); + await fs.unlink(path.join(cleanup.localRoot, "old-large.bin")); + expect(await copies.collectStopped({ companyId, runId: cleanup.runId })).toMatchObject({ state: "saved", receipt: { storageWarning: null } }); + expect((await run()).receipt?.storageWarning).toBeNull(); + }, 30_000); + + it("starts when the existing folder exceeds its total quota and accepts agent cleanup", async () => { + for (let index = 0; index < 8; index++) await fs.writeFile(path.join(root, `quota-part-${index}.bin`), "fixture"); + const lstat = fs.lstat.bind(fs); + // Keep this regression small on disk while exercising the real filesystem, + // snapshots, database receipts, restore, and merge with 2 GiB of metadata. + const sizes = vi.spyOn(fs, "lstat").mockImplementation(async (...args: Parameters) => { + const stat = await lstat(...args); + if (path.basename(String(args[0])).startsWith("quota-part-")) Object.assign(stat, { size: MAX_AGENT_FILE_BYTES }); + return stat; + }); + try { + const first = await run(); + expect(first.receipt?.storageWarning).toContain("2 GiB"); + expect((await copies.collectStopped({ companyId, runId: first.runId }))?.errorCode).toBe("AGENT_FILES_LIMIT_EXCEEDED"); + const cleanup = await run(); + for (let index = 0; index < 8; index++) await fs.unlink(path.join(cleanup.localRoot, `quota-part-${index}.bin`)); + await fs.writeFile(path.join(cleanup.localRoot, "small.txt"), "recovered"); + expect(await copies.collectStopped({ companyId, runId: cleanup.runId })).toMatchObject({ state: "saved", receipt: { storageWarning: null } }); + expect((await run()).receipt?.storageWarning).toBeNull(); + } finally { sizes.mockRestore(); } + }); + + it("rejects more than 2 GiB in aggregate before hashing or saving any file", async () => { + expect(MAX_AGENT_DIRECTORY_BYTES).toBe(2 * 1024 * 1024 * 1024); + const copy = await run(); + for (let index = 0; index < 8; index++) await sparseFile(path.join(copy.localRoot, `part-${index}.bin`), MAX_AGENT_FILE_BYTES); + // The eight allowed files exactly fill the quota; the entry is additional. + const result = await copies.collectStopped({ companyId, runId: copy.runId }); + expect(result).toMatchObject({ state: "unavailable", errorCode: "AGENT_FILES_LIMIT_EXCEEDED" }); + expect(result?.errorMessage).toContain("2 GiB total storage limit"); + expect(await fs.readdir(root)).toEqual(["policy"]); + }); + + it("rejects excessive entry count before snapshotting", async () => { + const directory = path.join(home, "many-entries"); + await fs.mkdir(directory); + await fs.writeFile(path.join(directory, "empty"), ""); + const entries = await fs.readdir(directory, { withFileTypes: true }); + // Exercise the walker bound without creating 100,001 physical files. + const readdir = vi.spyOn(fs, "readdir").mockResolvedValue(Array(MAX_AGENT_DIRECTORY_ENTRIES + 1).fill(entries[0])); + try { + await expect(snapshotAgentFiles(directory)).rejects.toMatchObject({ status: 422, message: expect.stringContaining("100,000-entry limit") }); + } finally { readdir.mockRestore(); } + }, 30_000); + + it("keeps the entry's 1 MiB limit and cleans up failed synchronization", async () => { + const copy = await run(); + await fs.writeFile(path.join(copy.localRoot, entryFile), "a".repeat(1024 * 1024 + 1)); + const result = await copies.collectStopped({ companyId, runId: copy.runId }); + expect(result?.state).toBe("unavailable"); + await expect(fs.stat(copy.localRoot)).rejects.toMatchObject({ code: "ENOENT" }); + expect(result?.errorMessage).toContain("at most 1 MiB"); + expect(await fs.readFile(path.join(root, entryFile), "utf8")).toBe(initial); + }); + + it("merges independent changes and uses the last synchronization for same-file edits", async () => { + const a = await run(), b = await run(); + await fs.writeFile(path.join(a.localRoot, "a.txt"), "A"); + await fs.writeFile(path.join(b.localRoot, "b.txt"), "B"); + await Promise.all([a, b].map(copy => copies.collectStopped({ companyId, runId: copy.runId }))); + expect(await fs.readFile(path.join(root, "a.txt"), "utf8")).toBe("A"); + expect(await fs.readFile(path.join(root, "b.txt"), "utf8")).toBe("B"); + const c = await run(), d = await run(); + await fs.writeFile(path.join(c.localRoot, "a.txt"), "C"); + await fs.writeFile(path.join(d.localRoot, "a.txt"), "D"); + await fs.writeFile(path.join(c.localRoot, "b.txt"), "new B"); + expect((await copies.collectStopped({ companyId, runId: c.runId }))?.state).toBe("saved"); + expect((await copies.collectStopped({ companyId, runId: d.runId }))?.state).toBe("saved"); + expect(await fs.readFile(path.join(root, "a.txt"), "utf8")).toBe("D"); + expect(await fs.readFile(path.join(root, "b.txt"), "utf8")).toBe("new B"); + expect(await copies.list(companyId, agentId, board())).toEqual([]); + for (const copy of [a, b, c, d]) await expect(fs.stat(path.dirname(copy.localRoot))).rejects.toMatchObject({ code: "ENOENT" }); + expect(await fs.readdir(path.join(path.dirname(root), "file-sync", "runs"))).toEqual([]); + }); + + it("makes concurrent edit/delete races last-sync-wins and preserves unrelated new files", async () => { + await fs.mkdir(path.join(root, "notes")); + await fs.writeFile(path.join(root, "notes", "old.txt"), "old"); + const deleting = await run(), editing = await run(); + await fs.rm(path.join(deleting.localRoot, "notes"), { recursive: true }); + await fs.writeFile(path.join(editing.localRoot, "notes", "old.txt"), "edited"); + await fs.writeFile(path.join(editing.localRoot, "notes", "new.txt"), "unrelated"); + await copies.collectStopped({ companyId, runId: editing.runId }); + expect((await copies.collectStopped({ companyId, runId: deleting.runId }))?.state).toBe("saved"); + await expect(fs.stat(path.join(root, "notes", "old.txt"))).rejects.toMatchObject({ code: "ENOENT" }); + expect(await fs.readFile(path.join(root, "notes", "new.txt"), "utf8")).toBe("unrelated"); + const remove = await run(), change = await run(); + await fs.unlink(path.join(remove.localRoot, "notes", "new.txt")); + await fs.writeFile(path.join(change.localRoot, "notes", "new.txt"), "restored by later edit"); + await copies.collectStopped({ companyId, runId: remove.runId }); + await copies.collectStopped({ companyId, runId: change.runId }); + expect(await fs.readFile(path.join(root, "notes", "new.txt"), "utf8")).toBe("restored by later edit"); + }); + + it("handles a concurrently replaced parent and keeps the later file change", async () => { + await fs.mkdir(path.join(root, "notes")); + await fs.writeFile(path.join(root, "notes", "fact.txt"), "old"); + const replacing = await run(), editing = await run(); + await fs.rm(path.join(replacing.localRoot, "notes"), { recursive: true }); + await fs.writeFile(path.join(replacing.localRoot, "notes"), "now a file"); + await fs.writeFile(path.join(editing.localRoot, "notes", "fact.txt"), "new fact"); + await copies.collectStopped({ companyId, runId: replacing.runId }); + expect(await fs.readFile(path.join(root, "notes"), "utf8")).toBe("now a file"); + expect((await copies.collectStopped({ companyId, runId: editing.runId }))?.state).toBe("saved"); + expect(await fs.readFile(path.join(root, "notes", "fact.txt"), "utf8")).toBe("new fact"); + }); + + it("cleans stopped copies after a crash during cleanup without touching active copies", async () => { + const finished = await run(), active = await run(); + await db.update(agentInstructionWorkingCopies).set({ state: "saved", processStoppedAt: new Date() }).where(eq(agentInstructionWorkingCopies.runId, finished.runId)); + copies = agentInstructionWorkingCopyService(db); + await copies.recoverCaptured(); + await expect(fs.stat(finished.localRoot)).rejects.toMatchObject({ code: "ENOENT" }); + expect((await fs.stat(active.localRoot)).isDirectory()).toBe(true); + }); + + it("cleans up a staging failure before any provider starts", async () => { + const runId = randomUUID(); + await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, invocationSource: "on_demand", responsibleUserId: userId }); + const shell = vi.spyOn(executionTargetTools, "runAdapterExecutionTargetShellCommand").mockResolvedValue({ exitCode: 1, signal: null, timedOut: false, stdout: "", stderr: "fixture failure" }); + try { + await expect(copies.prepare({ ...target(), runId, cwd: home, target: { kind: "remote", transport: "ssh", environmentId: randomUUID(), remoteCwd: "/fixture/task", + spec: { host: "unused.invalid", port: 22, username: "test", remoteCwd: "/fixture/task" } } })).rejects.toThrow("Could not exclude"); + const row = (await copies.get(companyId, runId))!; + expect(row.state).toBe("unavailable"); + expect(row.receipt?.baseline).toBeUndefined(); + await expect(fs.stat(path.dirname(row.localRoot))).rejects.toMatchObject({ code: "ENOENT" }); + } finally { shell.mockRestore(); } + }); + + it.each(["succeeded", "failed", "cancelled", "timed_out", "interrupted"])("registers staging ownership before copying and cleans preparation for a %s run after restart", async (status) => { + const original = fs.cp.bind(fs); + const copy = vi.spyOn(fs, "cp").mockImplementationOnce(async (source, destination, options) => { + const rows = await db.select().from(agentInstructionWorkingCopies).where(eq(agentInstructionWorkingCopies.agentId, agentId)); + expect(rows).toHaveLength(1); + expect(rows[0]).toMatchObject({ state: "preparing", localRoot: destination }); + return original(source, destination, options); + }); + const interrupted = await run(); + copy.mockRestore(); + const active = await run(); + for (const row of [interrupted, active]) { + await db.update(agentInstructionWorkingCopies).set({ state: "preparing", baseHash: "preparing", receipt: { schema: "paperclip.agent-files.v1" } }).where(eq(agentInstructionWorkingCopies.runId, row.runId)); + } + await db.update(heartbeatRuns).set({ status }).where(eq(heartbeatRuns.id, interrupted.runId)); + copies = agentInstructionWorkingCopyService(db); + await copies.recoverStopped(); + expect((await copies.get(companyId, interrupted.runId))?.state).toBe("unavailable"); + await expect(fs.stat(path.dirname(interrupted.localRoot))).rejects.toMatchObject({ code: "ENOENT" }); + expect((await fs.stat(active.localRoot)).isDirectory()).toBe(true); + }); + + it("retains remote cleanup across restart and failed retries until the original lease is cleaned", async () => { + const copy = await run(); + const environmentId = randomUUID(), leaseId = randomUUID(), remoteCwd = "/fixture/task"; + const executionRoot = path.posix.join(remoteCwd, ".paperclip-runtime", "agent-files", agentId, copy.runId); + await db.insert(environments).values({ id: environmentId, name: environmentId, driver: "sandbox" }); + await db.insert(environmentLeases).values({ id: leaseId, companyId, environmentId, heartbeatRunId: copy.runId, provider: "daytona", providerLeaseId: "original-sandbox" }); + await db.update(agentInstructionWorkingCopies).set({ state: "saved", processStoppedAt: new Date(), location: `remote:${environmentId}`, executionRoot, + receipt: { ...copy.receipt, cleanup: { leaseId, remoteCwd } } }).where(eq(agentInstructionWorkingCopies.runId, copy.runId)); + copies = agentInstructionWorkingCopyService(db); + await copies.recoverCaptured(); + const pending = (await copies.get(companyId, copy.runId))!; + expect(pending.receipt).toMatchObject({ cleanupPending: true, cleanup: { leaseId, remoteCwd } }); + expect(pending.receipt?.baseline).toBeUndefined(); + await expect(fs.stat(copy.localRoot)).rejects.toMatchObject({ code: "ENOENT" }); + const execute = vi.fn().mockRejectedValueOnce(new Error("provider temporarily unavailable")).mockResolvedValue({ exitCode: 0, stdout: "", stderr: "" }); + copies = agentInstructionWorkingCopyService(db, { environmentRuntime: { execute } as unknown as EnvironmentRuntimeService }); + for (const expectedPending of [true, false]) { + await db.update(agentInstructionWorkingCopies).set({ nextAttemptAt: null }).where(eq(agentInstructionWorkingCopies.runId, copy.runId)); + await copies.recoverCaptured(); + expect((await copies.get(companyId, copy.runId))?.receipt?.cleanupPending).toBe(expectedPending); + } + expect(execute).toHaveBeenCalledTimes(2); + expect(execute).toHaveBeenLastCalledWith(expect.objectContaining({ lease: expect.objectContaining({ id: leaseId, providerLeaseId: "original-sandbox" }), + command: "rm", args: ["-rf", "--", executionRoot], bypassSession: true })); + await copies.recoverCaptured(); + expect(execute).toHaveBeenCalledTimes(2); + await copies.release(companyId, copy.runId); + expect((await copies.get(companyId, copy.runId))?.receipt?.cleanupPending).toBe(false); + expect(execute).toHaveBeenCalledTimes(2); + }); + + it("finishes remote cleanup from a destruction receipt after the environment is deleted", async () => { + const copy = await run(); + const environmentId = randomUUID(), leaseId = randomUUID(), remoteCwd = "/fixture/task"; + const lease = { id: leaseId, companyId, environmentId, heartbeatRunId: copy.runId, provider: "daytona", providerLeaseId: "destroyed-sandbox" }; + await db.insert(environments).values({ id: environmentId, name: environmentId, driver: "sandbox" }); + await db.insert(environmentLeases).values({ ...lease, status: "released", releasedAt: new Date(), cleanupStatus: "success", + metadata: { remoteExecutionTermination: remoteTerminationReceipt(lease, { providerLeaseId: lease.providerLeaseId, state: "destroyed" }) } }); + await db.update(agentInstructionWorkingCopies).set({ state: "saved", processStoppedAt: new Date(), location: `remote:${environmentId}`, + executionRoot: path.posix.join(remoteCwd, ".paperclip-runtime", "agent-files", agentId, copy.runId), + receipt: { ...copy.receipt, cleanupPending: true, cleanup: { leaseId, remoteCwd } } }).where(eq(agentInstructionWorkingCopies.runId, copy.runId)); + await db.delete(environments).where(eq(environments.id, environmentId)); + copies = agentInstructionWorkingCopyService(db); + await copies.recoverCaptured(); + expect((await copies.get(companyId, copy.runId))?.receipt?.cleanupPending).toBe(false); + await expect(fs.stat(copy.localRoot)).rejects.toMatchObject({ code: "ENOENT" }); + }); + + it.each([false, true])("rejects symlinks without saving any part of the tree, preserving an existing full-storage warning: %s", async (full) => { + if (full) await sparseFile(path.join(root, "full.bin"), MAX_AGENT_FILE_BYTES); + const copy = await run(); + await fs.writeFile(path.join(copy.localRoot, "innocent.txt"), "changed"); + await fs.symlink(path.join(home, "outside"), path.join(copy.localRoot, "escape")); + const result = await copies.collectStopped({ companyId, runId: copy.runId }); + expect(result?.state).toBe("unavailable"); + expect(result?.errorCode).toBe("AGENT_FILES_SAVE_FAILED"); + expect(result?.receipt?.storageWarning).toBe(copy.receipt?.storageWarning); + if (full) expect(result?.receipt?.storageWarning).toContain("Agent storage is full"); + await expect(fs.stat(copy.localRoot)).rejects.toMatchObject({ code: "ENOENT" }); + await expect(fs.stat(path.join(root, "innocent.txt"))).rejects.toMatchObject({ code: "ENOENT" }); + }, 30_000); + + it("adopts the last deployed head once and bridges an old collector without appending history", async () => { + const id = randomUUID(); + const content = "last deployed revision"; + await db.insert(agentInstructionRevisions).values({ id, ...target(), entryFile, contentBase64: Buffer.from(content).toString("base64"), contentHash: fileHash(Buffer.from(content)), byteLength: content.length, source: "board" }); + await db.insert(agentInstructionHeads).values({ ...target(), entryFile, revisionId: id }); + const current = await revisions.readCurrent(target(), board()); + expect(current?.content).toBe(content); + await revisions.commit({ ...target(), entryFile, content: "new directory contents", baseRevisionId: id, source: "cleanup" }, board()); + await revisions.materializeCurrent(target()); + expect(await fs.readFile(path.join(root, entryFile), "utf8")).toBe("new directory contents"); + await expect(revisions.commit({ ...target(), entryFile, content: "stale legacy run", baseRevisionId: id, source: "cleanup" }, board())).rejects.toMatchObject({ status: 409 }); + expect(await db.select().from(agentInstructionRevisions).where(eq(agentInstructionRevisions.agentId, agentId))).toHaveLength(1); + }); + + it("ordinary file edits leave the loaded instruction digest unchanged", async () => { + const make = async () => { + const copy = await run(); + const context = await buildNativeRuntimeContext({ db, agent: { id: agentId, companyId, name: "Target", adapterConfig: { instructionsBundleMode: "managed", instructionsRootPath: root, instructionsEntryFile: entryFile } }, + runId: copy.runId, runtimeConfig: {}, runtimeSkillEntries: [], instructionWorkingCopy: { rootPath: copy.executionRoot, entryPath: entryFile, kind: "agent_files" } }); + return { copy, context }; + }; + const before = await make(); + await fs.writeFile(path.join(before.copy.localRoot, "notes.txt"), "new personal knowledge"); + await copies.collectStopped({ companyId, runId: before.copy.runId }); + const after = await make(); + expect(after.context.aggregateDigest).toBe(before.context.aggregateDigest); + expect(after.context.instructions.bundle.fileCount).toBe(1); + }); + it("uses the workspace transport to restore after destruction of the remote filesystem", async () => { + const remoteCwd = path.join(home, "remote-task"); + await fs.mkdir(remoteCwd, { recursive: true }); + await execFile("git", ["init", remoteCwd]); + const runner: import("@paperclipai/adapter-utils/command-managed-runtime").CommandManagedRuntimeRunner = { + execute: async input => { + const startedAt = new Date().toISOString(); + const env = { ...process.env, ...input.env }; + const args = [...(input.args ?? [])]; + if (input.stdin != null && (args[0] === "-c" || args[0] === "-lc")) { + env.PAPERCLIP_TEST_STDIN = input.stdin; + args[1] = `printf '%s' "$PAPERCLIP_TEST_STDIN" | (${args[1]})`; + } + try { + const result = await execFile(input.command, args, { cwd: input.cwd, env, timeout: input.timeoutMs, maxBuffer: 32 * 1024 * 1024 }); + return { exitCode: 0, signal: null, timedOut: false, stdout: result.stdout, stderr: result.stderr, pid: null, startedAt }; + } catch (error) { + const e = error as { code?: number; signal?: NodeJS.Signals; stdout?: string; stderr?: string }; + return { exitCode: typeof e.code === "number" ? e.code : 1, signal: e.signal ?? null, timedOut: false, stdout: e.stdout ?? "", stderr: e.stderr ?? "", pid: null, startedAt }; + } + }, + }; + const executionTarget = { kind: "remote" as const, transport: "sandbox" as const, environmentId: randomUUID(), remoteCwd, runner }; + const prepare = async () => { + const runId = randomUUID(); + await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, invocationSource: "on_demand", responsibleUserId: userId }); + return (await copies.prepare({ ...target(), runId, cwd: home, target: executionTarget }))!; + }; + await fs.mkdir(path.join(root, "build")); + await fs.writeFile(path.join(root, "build", "personal.txt"), "cache-like names are still agent files"); + const first = await prepare(); + expect(await fs.readFile(path.join(first.executionRoot, "build", "personal.txt"), "utf8")).toBe("cache-like names are still agent files"); + await fs.mkdir(path.join(first.executionRoot, "notes")); + await fs.writeFile(path.join(first.executionRoot, "notes", "bytes.bin"), Buffer.from([0, 128, 255])); + await fs.mkdir(path.join(first.executionRoot, "node_modules")); + await fs.writeFile(path.join(first.executionRoot, "node_modules", "personal.txt"), "retain this too"); + await fs.writeFile(path.join(remoteCwd, "task-only.txt"), "task"); + expect((await execFile("git", ["-C", remoteCwd, "status", "--porcelain", "--untracked-files=all"])).stdout).toBe("?? task-only.txt\n"); + expect((await copies.collectStopped({ companyId, runId: first.runId, target: executionTarget }))?.state).toBe("saved"); + await copies.release(companyId, first.runId); + expect((await copies.get(companyId, first.runId))?.receipt?.baseline).toBeUndefined(); + await expect(fs.stat(first.localRoot)).rejects.toMatchObject({ code: "ENOENT" }); + await fs.rm(remoteCwd, { recursive: true }); + await fs.mkdir(remoteCwd); + copies = agentInstructionWorkingCopyService(db); + const second = await prepare(); + expect(await fs.readFile(path.join(second.executionRoot, "notes", "bytes.bin"))).toEqual(Buffer.from([0,128,255])); + expect(await fs.readFile(path.join(second.executionRoot, "node_modules", "personal.txt"), "utf8")).toBe("retain this too"); + await expect(fs.stat(path.join(second.executionRoot, "task-only.txt"))).rejects.toMatchObject({ code: "ENOENT" }); + await copies.collectStopped({ companyId, runId: second.runId, target: executionTarget }); + await copies.release(companyId, second.runId); + }); + + it("stages SSH agent files at the registered root without a nested task workspace", async () => { + const remoteCwd = path.join(home, "ssh-task"); + const exclude = vi.spyOn(executionTargetTools, "runAdapterExecutionTargetShellCommand").mockResolvedValue({ exitCode: 0, signal: null, timedOut: false, stdout: "", stderr: "" }); + const stage = vi.spyOn(ssh, "syncDirectoryToSsh").mockImplementation(async input => { + await fs.mkdir(path.dirname(input.remoteDir), { recursive: true }); + await fs.cp(input.localDir, input.remoteDir, { recursive: true }); + }); + const restore = vi.spyOn(ssh, "restoreWorkspaceFromSshExecution").mockImplementation(async input => { + await fs.cp(input.remoteDir!, input.localDir, { recursive: true }); + }); + try { + const runId = randomUUID(); + await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, invocationSource: "on_demand", responsibleUserId: userId }); + const target = { kind: "remote" as const, transport: "ssh" as const, environmentId: randomUUID(), remoteCwd, + spec: { host: "unused.invalid", port: 22, username: "test", remoteCwd } }; + const copy = (await copies.prepare({ companyId, agentId, runId, cwd: home, target }))!; + expect(stage).toHaveBeenCalledWith(expect.objectContaining({ remoteDir: copy.executionRoot })); + expect(await fs.readFile(path.join(copy.executionRoot, entryFile), "utf8")).toBe(initial); + await fs.writeFile(path.join(copy.executionRoot, "ssh-note.txt"), "persistent SSH file"); + expect((await copies.collectStopped({ companyId, runId, target }))?.state).toBe("saved"); + expect(restore).toHaveBeenCalledWith(expect.objectContaining({ remoteDir: copy.executionRoot, restoreGitHistory: false })); + expect(await fs.readFile(path.join(root, "ssh-note.txt"), "utf8")).toBe("persistent SSH file"); + } finally { stage.mockRestore(); restore.mockRestore(); exclude.mockRestore(); } + }); + + it("deduplicates concurrent stopped callbacks and discards completed operational snapshots", async () => { + const copy = await run(); + await fs.writeFile(path.join(copy.localRoot, "note.txt"), "one write"); + const results = await Promise.all([copies.collectStopped({ companyId, runId: copy.runId }), copies.collectStopped({ companyId, runId: copy.runId })]); + expect(results.every(result => result?.state === "saved")).toBe(true); + // Simulate a crash after save but before release. The restart sweeper must + // reclaim operational baselines without touching canonical files. + copies = agentInstructionWorkingCopyService(db); + await copies.recoverCaptured(); + const row = await copies.get(companyId, copy.runId); + expect(row?.receipt?.schema).toBe("paperclip.agent-files.v1"); + expect(row?.receipt?.baseline).toBeUndefined(); + expect(await fs.readFile(path.join(root, "note.txt"), "utf8")).toBe("one write"); + }); + +}); diff --git a/server/src/__tests__/agent-instruction-git-exclusion.test.ts b/server/src/__tests__/agent-instruction-git-exclusion.test.ts new file mode 100644 index 0000000000..73f81d4c92 --- /dev/null +++ b/server/src/__tests__/agent-instruction-git-exclusion.test.ts @@ -0,0 +1,87 @@ +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { execFile as execFileCallback } from "node:child_process"; +import { promisify } from "node:util"; +import { afterEach, beforeEach, expect, it } from "vitest"; +import { instructionGitExcludeProgram } from "../services/agent-instruction-files.js"; + +const execFile = promisify(execFileCallback); +let root: string; +beforeEach(async () => { root = await fs.mkdtemp(path.join(os.tmpdir(), "agent-git-exclusion-")); }); +afterEach(async () => { await fs.rm(root, { recursive: true, force: true }); }); +const git = (cwd: string, ...args: string[]) => execFile("git", ["-C", cwd, ...args]); +const exclude = (cwd: string) => execFile(process.execPath, ["-e", instructionGitExcludeProgram, cwd]); + +async function assertPrivateFilesIgnored(cwd: string) { + const runtime = path.join(cwd, ".paperclip-runtime"); + await fs.writeFile(path.join(runtime, "private.md"), "agent instructions"); + await git(cwd, "add", "-A"); + expect((await git(cwd, "diff", "--cached", "--name-only")).stdout).not.toContain(".paperclip-runtime"); + expect((await git(cwd, "check-ignore", "--", ".paperclip-runtime/private.md", ".paperclip-runtime/.gitignore")).stdout.trim().split("\n")).toHaveLength(2); +} + +it("excludes runtime files without changing tracked ignore rules or Git metadata", async () => { + await git(root, "init"); + await fs.writeFile(path.join(root, ".gitignore"), "keep-me\n"); + const metadata = path.join(root, ".git", "info", "exclude"); + const before = await fs.readFile(metadata, "utf8"); + await exclude(root); + await exclude(root); + await assertPrivateFilesIgnored(root); + expect(await fs.readFile(path.join(root, ".gitignore"), "utf8")).toBe("keep-me\n"); + expect(await fs.readFile(metadata, "utf8")).toBe(before); +}); + +it("never writes to an external gitdir selected by task-controlled metadata", async () => { + const workspace = path.join(root, "workspace"), outside = path.join(root, "outside.git"); + await fs.mkdir(workspace); + await git(workspace, "init", `--separate-git-dir=${outside}`); + const metadata = path.join(outside, "info", "exclude"); + const before = await fs.readFile(metadata, "utf8"); + await exclude(workspace); + await assertPrivateFilesIgnored(workspace); + expect(await fs.readFile(metadata, "utf8")).toBe(before); +}); + +it("supports linked worktrees and nested task directories without changing the common gitdir", async () => { + const repository = path.join(root, "repository"), worktree = path.join(root, "worktree"); + await fs.mkdir(repository); + await git(repository, "init"); + await git(repository, "-c", "user.name=Test", "-c", "user.email=test@example.test", "commit", "--allow-empty", "-m", "initial"); + await git(repository, "worktree", "add", "--detach", worktree); + const nested = path.join(worktree, "nested"); + await fs.mkdir(nested); + const metadata = path.join(repository, ".git", "info", "exclude"); + const before = await fs.readFile(metadata, "utf8"); + await exclude(nested); + await assertPrivateFilesIgnored(nested); + expect(await fs.readFile(metadata, "utf8")).toBe(before); +}); + +it.each(["directory", "file"])("rejects a symlink at the runtime exclusion %s", async (kind) => { + const workspace = path.join(root, "workspace"), outside = path.join(root, "outside"); + await fs.mkdir(workspace); + await fs.mkdir(outside); + const externalFile = path.join(outside, ".gitignore"); + await fs.writeFile(externalFile, "untouched\n"); + const runtime = path.join(workspace, ".paperclip-runtime"); + if (kind === "directory") await fs.symlink(outside, runtime); + else { + await fs.mkdir(runtime); + await fs.symlink(externalFile, path.join(runtime, ".gitignore")); + } + await expect(exclude(workspace)).rejects.toThrow("Unsafe runtime exclusion"); + expect(await fs.readFile(externalFile, "utf8")).toBe("untouched\n"); +}); + +it("replaces a linked ignore file without writing through its external inode", async () => { + const externalFile = path.join(root, "outside-ignore"), workspace = path.join(root, "workspace"); + const runtime = path.join(workspace, ".paperclip-runtime"); + await fs.mkdir(runtime, { recursive: true }); + await fs.writeFile(externalFile, "untouched\n"); + await fs.link(externalFile, path.join(runtime, ".gitignore")); + await exclude(workspace); + expect(await fs.readFile(externalFile, "utf8")).toBe("untouched\n"); + expect(await fs.readFile(path.join(runtime, ".gitignore"), "utf8")).toBe("*\n"); +}); diff --git a/server/src/__tests__/agent-instruction-revisions.test.ts b/server/src/__tests__/agent-instruction-revisions.test.ts new file mode 100644 index 0000000000..6d2a516cb2 --- /dev/null +++ b/server/src/__tests__/agent-instruction-revisions.test.ts @@ -0,0 +1,247 @@ +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { randomUUID } from "node:crypto"; +import { beforeAll, afterAll, beforeEach, describe, expect, it, vi } from "vitest"; +import { and, eq, sql } from "drizzle-orm"; +import { agents, agentApiKeys, companies, authUsers, companyMemberships, principalPermissionGrants, heartbeatRuns, + agentInstructionRevisions, agentInstructionHeads, issueThreadInteractions, issues, createDb } from "@paperclipai/db"; +import { startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js"; +import { agentInstructionRevisionService } from "../services/agent-instruction-revisions.js"; +import { agentInstructionWorkingCopyService } from "../services/agent-instruction-working-copies.js"; +import { instructionBytes, instructionPath, materializeInstructionBytes, readInstructionBytes } from "../services/agent-instruction-files.js"; +import { agentInstructionsService, resolveManagedInstructionsRoot } from "../services/agent-instructions.js"; +import type { AuthorizationActor } from "../services/authorization.js"; +import { upsertAgentInstructionsFileSchema } from "@paperclipai/shared"; + +describe("canonical instruction revisions", () => { + let database: Awaited>; + let db: ReturnType; + let service: ReturnType; + const previousHome = process.env.PAPERCLIP_HOME; + let home: string; + let companyId: string, agentId: string, userId: string, actorId: string, runId: string, root: string; + let actor: AuthorizationActor; + const entryFile = "policy/INSTRUCTIONS.txt"; + const initial = "\uFEFF# Original\r\n☃\0\n"; + beforeAll(async () => { + home = await fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), "instruction-revisions-"))); + process.env.PAPERCLIP_HOME = home; + database = await startEmbeddedPostgresTestDatabase("instruction-revisions-db-"); + db = createDb(database.connectionString); + service = agentInstructionRevisionService(db); + }, 90_000); + afterAll(async () => { + vi.restoreAllMocks(); + if (previousHome === undefined) delete process.env.PAPERCLIP_HOME; else process.env.PAPERCLIP_HOME = previousHome; + await database?.cleanup(); + if (home) await fs.rm(home, { recursive: true, force: true }); + }); + beforeEach(async () => { + companyId = randomUUID(); agentId = randomUUID(); actorId = randomUUID(); userId = randomUUID(); runId = randomUUID(); + await db.insert(companies).values({ id: companyId, name: "Instruction tests", issuePrefix: randomUUID().slice(0, 8) }); + await db.insert(authUsers).values({ id: userId, name: "Editor", email: `${userId}@example.test`, createdAt: new Date(), updatedAt: new Date() }); + root = resolveManagedInstructionsRoot({ id: agentId, companyId, name: "Target", adapterConfig: {} }); + await db.insert(agents).values([ + { id: agentId, companyId, name: "Target", adapterConfig: { instructionsBundleMode: "managed", instructionsRootPath: root, instructionsEntryFile: entryFile } }, + { id: actorId, companyId, name: "Writer" }, + ]); + await db.insert(companyMemberships).values([ + { companyId, principalType: "user", principalId: userId, membershipRole: "operator" }, + { companyId, principalType: "agent", principalId: actorId, membershipRole: "member" }, + ]); + await db.insert(principalPermissionGrants).values([ + { companyId, principalType: "user", principalId: userId, permissionKey: "agents:configure", scope: { agentIds: [agentId] } }, + { companyId, principalType: "agent", principalId: actorId, permissionKey: "agents:configure", scope: { agentIds: [agentId] } }, + ]); + await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId: actorId, invocationSource: "on_demand", responsibleUserId: userId }); + actor = { type: "agent", companyId, agentId: actorId, runId, source: "agent_jwt" }; + await fs.mkdir(path.dirname(path.join(root, entryFile)), { recursive: true }); + await fs.writeFile(path.join(root, entryFile), initial); + }); + const target = () => ({ companyId, agentId }); + async function base() { return (await service.readCurrent(target(), actor))!; } + async function save(content: string, baseRevisionId: string | null) { + return service.commit({ ...target(), entryFile, content, baseRevisionId, source: "cleanup" }, actor); + } + it.each(["current", "history", "revision", "diff", "candidates"])("denies peer %s reads without a configuration grant or responsible-user target access", async (operation) => { + const first = await base(); + await db.delete(principalPermissionGrants).where(eq(principalPermissionGrants.principalId, userId)); + const read = () => { + if (operation === "current") return service.readCurrent(target(), actor); + if (operation === "history") return service.history({ ...target(), entryFile }, actor); + if (operation === "revision") return service.readRevision({ ...target(), entryFile, revisionId: first.revision.id }, actor); + if (operation === "diff") return service.diff({ ...target(), entryFile, fromRevisionId: first.revision.id, toRevisionId: first.revision.id }, actor); + return agentInstructionWorkingCopyService(db).list(companyId, agentId, actor); + }; + await expect(read()).rejects.toMatchObject({ status: 403 }); + await db.insert(principalPermissionGrants).values({ companyId, principalType: "user", principalId: userId, permissionKey: "agents:configure", scope: { agentIds: [actorId] } }); + await expect(read()).rejects.toMatchObject({ status: 403 }); + }); + it.each(["current", "history", "revision", "diff", "candidates", "write"])("requires the caller's own target-scoped grant for peer %s access", async (operation) => { + const first = await base(); + await db.delete(principalPermissionGrants).where(eq(principalPermissionGrants.principalId, actorId)); + const access = () => { + if (operation === "current") return service.readCurrent(target(), actor); + if (operation === "history") return service.history({ ...target(), entryFile }, actor); + if (operation === "revision") return service.readRevision({ ...target(), entryFile, revisionId: first.revision.id }, actor); + if (operation === "diff") return service.diff({ ...target(), entryFile, fromRevisionId: first.revision.id, toRevisionId: first.revision.id }, actor); + if (operation === "write") return save("authorized peer edit", first.revision.id); + return agentInstructionWorkingCopyService(db).list(companyId, agentId, actor); + }; + // The responsible user retains permission on the target throughout. + await expect(access()).rejects.toMatchObject({ status: 403 }); + await db.insert(principalPermissionGrants).values({ companyId, principalType: "agent", principalId: actorId, permissionKey: "agents:configure", scope: { agentIds: [actorId] } }); + await expect(access()).rejects.toMatchObject({ status: 403 }); + await db.update(principalPermissionGrants).set({ scope: { agentIds: [agentId] } }).where(eq(principalPermissionGrants.principalId, actorId)); + // New saves have no historical revision row; authorized history lookups + // reach the normal not-found response rather than an authorization denial. + if (operation === "revision" || operation === "diff") await expect(access()).rejects.toMatchObject({ status: 404 }); + else await expect(access()).resolves.toBeDefined(); + }); + it("allows self and board reads without requiring instruction edit permission", async () => { + await db.delete(principalPermissionGrants).where(eq(principalPermissionGrants.principalId, userId)); + const selfRoot = resolveManagedInstructionsRoot({ id: actorId, companyId, name: "Writer", adapterConfig: {} }); + await fs.mkdir(selfRoot, { recursive: true }); + await fs.writeFile(path.join(selfRoot, "AGENTS.md"), "Self instructions"); + expect((await service.readCurrent({ companyId, agentId: actorId }, actor))?.content).toBe("Self instructions"); + expect((await service.readCurrent(target(), { type: "board", userId, source: "session" }))?.content).toBe(initial); + }); + it("reads and replaces current bytes using content tokens without storing revisions", async () => { + const first = await base(); + expect(first.content).toBe(initial); + const saved = await save("# Changed\r\n", first.revision.id); + expect(saved).toMatchObject({ changed: true, materialization: "current" }); + expect(await fs.readFile(path.join(root, entryFile), "utf8")).toBe(saved.content); + expect((await save(saved.content, first.revision.id)).changed).toBe(false); + expect((await service.history({ ...target(), entryFile }, actor)).revisions).toHaveLength(0); + expect(await db.select().from(agentInstructionRevisions).where(eq(agentInstructionRevisions.agentId, agentId))).toHaveLength(0); + }); + it("requires a matching initial content token and serializes competing saves", async () => { + await expect(save("without reading", null)).rejects.toMatchObject({ status: 409 }); + const first = await base(); + const results = await Promise.allSettled([save("one", first.revision.id), save("two", first.revision.id)]); + expect(results.filter(result => result.status === "fulfilled")).toHaveLength(1); + expect(results.find(result => result.status === "rejected")).toMatchObject({ reason: { status: 409 } }); + }); + it("does not claim a save when atomic file replacement fails", async () => { + const first = await base(); + const rename = vi.spyOn(fs, "rename").mockRejectedValueOnce(new Error("interrupted replacement")); + await expect(save("new content", first.revision.id)).rejects.toThrow("interrupted replacement"); + rename.mockRestore(); + expect(await fs.readFile(path.join(root, entryFile), "utf8")).toBe(initial); + service = agentInstructionRevisionService(db); + await service.materializeCurrent(target()); + expect((await base()).content).toBe(initial); + expect((await save("new content", first.revision.id)).content).toBe("new content"); + }); + it("allows authorized peer edits but rejects missing, forged and revoked responsible identity", async () => { + const first = await base(); + await save("peer save", first.revision.id); + await expect(service.commit({ ...target(), entryFile, content: "forged", baseRevisionId: first.revision.id, source: "tool" }, { ...actor, onBehalfOfUserId: "forged" })).rejects.toMatchObject({ status: 403 }); + await db.update(heartbeatRuns).set({ responsibleUserId: null }).where(eq(heartbeatRuns.id, runId)); + await expect(save("missing", first.revision.id)).rejects.toMatchObject({ status: 403 }); + await db.update(heartbeatRuns).set({ responsibleUserId: userId }).where(eq(heartbeatRuns.id, runId)); + await db.update(companyMemberships).set({ status: "suspended" }).where(and(eq(companyMemberships.principalId, userId), eq(companyMemberships.companyId, companyId))); + await expect(save("revoked", first.revision.id)).rejects.toMatchObject({ status: 403 }); + }); + it("rechecks user target scope and agent containment, even with shadow authorization enabled", async () => { + const first = await base(); + const previousShadow = process.env.PAPERCLIP_RESPONSIBLE_USER_AUTHZ_SHADOW; + process.env.PAPERCLIP_RESPONSIBLE_USER_AUTHZ_SHADOW = "true"; + await db.update(principalPermissionGrants).set({ scope: { agentIds: [actorId] } }).where(eq(principalPermissionGrants.principalId, userId)); + await expect(save("denied", first.revision.id)).rejects.toMatchObject({ status: 403 }); + await db.update(principalPermissionGrants).set({ scope: null }).where(eq(principalPermissionGrants.principalId, userId)); + await db.update(agents).set({ permissions: { trustPreset: "low_trust_review" } }).where(eq(agents.id, actorId)); + await expect(save("restricted", first.revision.id)).rejects.toMatchObject({ status: 403 }); + if (previousShadow === undefined) delete process.env.PAPERCLIP_RESPONSIBLE_USER_AUTHZ_SHADOW; else process.env.PAPERCLIP_RESPONSIBLE_USER_AUTHZ_SHADOW = previousShadow; + }); + it("keeps suggest-only protected-change consent and explicit configure scope restrictions", async () => { + const first = await base(); + await db.delete(principalPermissionGrants).where(eq(principalPermissionGrants.principalId, actorId)); + await db.insert(principalPermissionGrants).values({ companyId, principalType: "agent", principalId: actorId, permissionKey: "agents:suggest-changes" }); + await expect(save("needs approval", first.revision.id)).rejects.toMatchObject({ status: 403 }); + await db.delete(principalPermissionGrants).where(eq(principalPermissionGrants.principalId, actorId)); + await db.insert(principalPermissionGrants).values({ companyId, principalType: "agent", principalId: actorId, permissionKey: "agents:configure", scope: { agentIds: [actorId] } }); + await expect(save("out of scope", first.revision.id)).rejects.toMatchObject({ status: 403 }); + }); + it("consumes accepted protected-change approval only with a successful CAS commit", async () => { + const first = await base(); + const sourceRunId = randomUUID(), issueId = randomUUID(), interactionId = randomUUID(); + await db.insert(heartbeatRuns).values({ id: sourceRunId, companyId, agentId: actorId, status: "succeeded" }); + await db.insert(issues).values({ id: issueId, companyId, title: "Review instructions" }); + await db.delete(principalPermissionGrants).where(eq(principalPermissionGrants.principalId, actorId)); + await db.insert(principalPermissionGrants).values({ companyId, principalType: "agent", principalId: actorId, permissionKey: "agents:suggest-changes" }); + await db.insert(issueThreadInteractions).values({ id: interactionId, companyId, issueId, kind: "request_confirmation", status: "accepted", sourceRunId, createdByAgentId: actorId, + payload: { version: 1, prompt: "Apply the instruction change?", detailsMarkdown: "```diff\n+approved\n```", target: { type: "custom", key: `agent:${agentId}:instructions`, revisionId: "proposal" } }, + result: { version: 1, outcome: "accepted" }, resolvedByUserId: userId, resolvedAt: new Date(), + }); + expect((await service.readCurrent(target(), actor))?.revision.id).toBe(first.revision.id); + expect((await db.select().from(issueThreadInteractions).where(eq(issueThreadInteractions.id, interactionId)))[0].result).not.toHaveProperty("consumedAt"); + await expect(save("approved", randomUUID())).rejects.toMatchObject({ status: 409 }); + expect((await db.select().from(issueThreadInteractions).where(eq(issueThreadInteractions.id, interactionId)))[0].result).not.toHaveProperty("consumedAt"); + const receipt = await save("approved", first.revision.id); + expect(receipt.changed).toBe(true); + expect((await db.select().from(issueThreadInteractions).where(eq(issueThreadInteractions.id, interactionId)))[0].result).toMatchObject({ consumedByRunId: runId }); + await expect(save("another unapproved change", receipt.revision.id)).rejects.toMatchObject({ status: 403 }); + }); + it("shares head/CAS and attribution between board and agent writers", async () => { + const first = await base(); + const board = { type: "board", userId, source: "session" } as const; + const saved = await service.commit({ ...target(), entryFile, content: "board edit", baseRevisionId: first.revision.id, source: "board" }, board); + expect(saved.revision).toMatchObject({ actorUserId: userId, responsibleUserId: userId, actorAgentId: null, sourceRunId: null, source: "api" }); + await expect(save("stale agent edit", first.revision.id)).rejects.toMatchObject({ status: 409 }); + }); + it("seeds a nested legacy managed entry and never imports a different AGENTS.md", async () => { + await fs.writeFile(path.join(root, "AGENTS.md"), "unrelated"); + await db.update(agents).set({ adapterConfig: { instructionsFilePath: path.join(root, entryFile) } }).where(eq(agents.id, agentId)); + const first = await base(); + expect(first.content).toBe(initial); + expect(first.revision.entryFile).toBe(entryFile); + await db.update(agents).set({ adapterConfig: { instructionsBundleMode: "managed", instructionsRootPath: root, instructionsEntryFile: "missing.md" } }).where(eq(agents.id, agentId)); + expect(await service.readCurrent(target(), actor)).toBeNull(); + await expect(save("wrong entry", first.revision.id)).rejects.toMatchObject({ status: 409, details: { code: "INSTRUCTION_ENTRY_CHANGED" } }); + }); + it("rejects cross-company targets, foreign revisions, and revoked API keys", async () => { + const first = await base(); + await expect(service.readRevision({ ...target(), entryFile, revisionId: randomUUID() }, actor)).rejects.toMatchObject({ status: 404 }); + await expect(service.readCurrent({ companyId: randomUUID(), agentId }, actor)).rejects.toMatchObject({ status: 404 }); + const keyId = randomUUID(); + await db.insert(agentApiKeys).values({ id: keyId, companyId, agentId: actorId, name: "test", keyHash: randomUUID(), responsibleUserId: userId, revokedAt: new Date() }); + await expect(service.commit({ ...target(), entryFile, content: "key", baseRevisionId: first.revision.id, source: "api" }, { ...actor, keyId })).rejects.toMatchObject({ status: 403 }); + await expect(db.insert(agentInstructionHeads).values({ companyId: randomUUID(), agentId, entryFile, revisionId: first.revision.id })).rejects.toThrow(); + }); + it("rejects invalid content, symlinks, traversal and external bundles without changing canonical bytes", async () => { + const first = await base(); + expect(() => instructionBytes("\ud800")).toThrow("UTF-8"); + expect(() => instructionBytes(new Uint8Array([0xff]))).toThrow("UTF-8"); + expect(() => instructionBytes("a".repeat(1024 * 1024 + 1))).toThrow("1 MiB"); + for (const value of ["../AGENTS.md", "/etc/passwd", "a/../AGENTS.md", "a\\b", "a//b"]) expect(() => instructionPath(value)).toThrow(); + await fs.unlink(path.join(root, entryFile)); + await fs.symlink(path.join(home, "outside"), path.join(root, entryFile)); + await expect(save("symlink", first.revision.id)).rejects.toMatchObject({ status: 422 }); + await fs.unlink(path.join(root, entryFile)); + await fs.rm(path.dirname(path.join(root, entryFile)), { recursive: true }); + await fs.symlink(home, path.dirname(path.join(root, entryFile))); + await expect(readInstructionBytes(root, entryFile)).rejects.toMatchObject({ status: 422 }); + await db.update(agents).set({ adapterConfig: { instructionsBundleMode: "external", instructionsRootPath: home } }).where(eq(agents.id, agentId)); + await expect(save("external", first.revision.id)).rejects.toMatchObject({ details: { code: "INSTRUCTION_MANAGED_BUNDLE_REQUIRED" } }); + expect((await db.select().from(agentInstructionRevisions).where(eq(agentInstructionRevisions.agentId, agentId)))).toHaveLength(0); + }); + it.skipIf(process.platform !== "darwin")("accepts macOS temporary directory aliases while rejecting links within storage", async () => { + const aliasRoot = home.replace(/^\/private\/(var|tmp)\//, "/$1/"); + expect(aliasRoot).not.toBe(home); + await materializeInstructionBytes(aliasRoot, "alias/AGENTS.md", Buffer.from("persisted")); + expect(await readInstructionBytes(aliasRoot, "alias/AGENTS.md")).toEqual(Buffer.from("persisted")); + await fs.symlink(path.join(home, "alias"), path.join(home, "linked-agent")); + await expect(readInstructionBytes(path.join(aliasRoot, "linked-agent"), "AGENTS.md")) + .rejects.toMatchObject({ status: 422 }); + }); + it("requires the content API for the configured entry and rejects forged fields", async () => { + await base(); + const [agent] = await db.select().from(agents).where(eq(agents.id, agentId)); + await expect(agentInstructionsService(db).writeFile(agent, entryFile, "bypass")).rejects.toMatchObject({ details: { code: "INSTRUCTION_REVISION_REQUIRED" } }); + await expect(agentInstructionsService(db).deleteFile(agent, entryFile)).rejects.toMatchObject({ status: 422 }); + expect(upsertAgentInstructionsFileSchema.safeParse({ path: entryFile, content: "bad", responsibleUserId: userId }).success).toBe(false); + }); +}); diff --git a/server/src/__tests__/agent-instruction-working-copies.test.ts b/server/src/__tests__/agent-instruction-working-copies.test.ts new file mode 100644 index 0000000000..1b6c9b2eb3 --- /dev/null +++ b/server/src/__tests__/agent-instruction-working-copies.test.ts @@ -0,0 +1,345 @@ +import fs from "node:fs/promises"; +import { execFile as execFileCallback } from "node:child_process"; +import { promisify } from "node:util"; +import * as gitWorkspaceSync from "@paperclipai/adapter-utils/git-workspace-sync"; +import { DatabaseSync } from "node:sqlite"; +import { captureDirectorySnapshot } from "@paperclipai/adapter-utils/workspace-restore-merge"; +const execFile = promisify(execFileCallback); +import os from "node:os"; +import path from "node:path"; +import { randomUUID } from "node:crypto"; +import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from "vitest"; +import { eq } from "drizzle-orm"; +import { agents, companies, authUsers, companyMemberships, principalPermissionGrants, heartbeatRuns, agentInstructionWorkingCopies, createDb } from "@paperclipai/db"; +import { startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js"; +import { agentInstructionRevisionService } from "../services/agent-instruction-revisions.js"; +import { agentInstructionWorkingCopyService } from "../services/agent-instruction-working-copies.js"; +import { appendHeartbeatRunEvent } from "../services/heartbeat-run-events.js"; +import { resolveManagedInstructionsRoot } from "../services/agent-instructions.js"; +import { buildNativeRuntimeContext } from "../services/native-runtime/runtime-context.js"; + +describe("registered run instruction copies", () => { + let database: Awaited>; + let db: ReturnType; + let copies: ReturnType; + let revisions: ReturnType; + const previousHome = process.env.PAPERCLIP_HOME; + let home: string; + let companyId: string, agentId: string, userId: string, root: string; + const entryFile = "policy/INSTRUCTIONS.txt"; + const initial = "\uFEFF# Original\r\n☃\n"; + const target = () => ({ companyId, agentId }); + const board = () => ({ type: "board" as const, userId, source: "session" as const }); + async function run() { + const runId = randomUUID(); + await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, invocationSource: "on_demand", responsibleUserId: userId }); + return (await copies.prepare({ legacy: true, ...target(), runId, cwd: home }))!; + } + beforeAll(async () => { + home = await fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), "instruction-working-copies-"))); + process.env.PAPERCLIP_HOME = home; + database = await startEmbeddedPostgresTestDatabase("instruction-copies-db-"); + db = createDb(database.connectionString); + copies = agentInstructionWorkingCopyService(db); + revisions = agentInstructionRevisionService(db); + }, 90_000); + afterAll(async () => { + if (previousHome === undefined) delete process.env.PAPERCLIP_HOME; else process.env.PAPERCLIP_HOME = previousHome; + await database?.cleanup(); + if (home) { + const writable = async (dir: string) => { + await fs.chmod(dir, 0o700); + for (const entry of await fs.readdir(dir, { withFileTypes: true })) if (entry.isDirectory()) await writable(path.join(dir, entry.name)); + }; + await writable(home); + await fs.rm(home, { recursive: true, force: true }); + } + }); + beforeEach(async () => { + companyId = randomUUID(); agentId = randomUUID(); userId = randomUUID(); + await db.insert(companies).values({ id: companyId, name: "Instruction tests", issuePrefix: randomUUID().slice(0, 8) }); + await db.insert(authUsers).values({ id: userId, name: "Editor", email: `${userId}@example.test`, createdAt: new Date(), updatedAt: new Date() }); + root = resolveManagedInstructionsRoot({ ...target(), id: agentId, name: "Target", adapterConfig: {} }); + await db.insert(agents).values({ id: agentId, companyId, name: "Target", adapterConfig: { instructionsBundleMode: "managed", instructionsRootPath: root, instructionsEntryFile: entryFile } }); + await db.insert(companyMemberships).values([ + { companyId, principalType: "user", principalId: userId, membershipRole: "operator" }, + { companyId, principalType: "agent", principalId: agentId, membershipRole: "member" }, + ]); + await db.insert(principalPermissionGrants).values({ companyId, principalType: "user", principalId: userId, permissionKey: "agents:configure", scope: { agentIds: [agentId] } }); + await fs.mkdir(path.dirname(path.join(root, entryFile)), { recursive: true }); + await fs.writeFile(path.join(root, entryFile), initial); + }); + + it("persists an atomic editor replacement, ignores repository instructions, and gives the next run exact bytes", async () => { + const copy = await run(); + await fs.writeFile(path.join(home, "AGENTS.md"), "Repository text must not become agent instructions"); + const edited = "\uFEFF# Updated\r\nKeep this persistent. ☃\n"; + const replacement = path.join(copy.localRoot, "edited.tmp"); + await fs.writeFile(replacement, edited); + await fs.rename(replacement, path.join(copy.localRoot, entryFile)); + expect(await fs.readFile(path.join(root, entryFile), "utf8")).toBe(initial); + const saved = await copies.collectStopped({ companyId, runId: copy.runId }); + expect(saved).toMatchObject({ state: "saved", candidateHash: expect.any(String), processStoppedAt: expect.any(Date) }); + expect(await fs.readFile(path.join(root, entryFile), "utf8")).toBe(edited); + copies = agentInstructionWorkingCopyService(db); + expect((await copies.collectStopped({ companyId, runId: copy.runId }))?.receipt).toEqual(saved?.receipt); + expect((await revisions.history({ ...target(), entryFile }, board())).revisions).toHaveLength(0); + const next = await run(); + expect(await fs.readFile(path.join(next.localRoot, entryFile), "utf8")).toBe(edited); + }); + + it.each(["failed", "cancelled", "timed_out"])("retains edits from stopped %s runs without restarting execution", async (status) => { + const copy = await run(); + await fs.writeFile(path.join(copy.localRoot, entryFile), status); + await db.update(heartbeatRuns).set({ status, finishedAt: new Date() }).where(eq(heartbeatRuns.id, copy.runId)); + expect((await copies.collectStopped({ companyId, runId: copy.runId }))?.state).toBe("saved"); + expect((await revisions.readCurrent(target(), board()))?.content).toBe(status); + const [unchangedRun] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, copy.runId)); + expect(unchangedRun.status).toBe(status); + }); + + it.each(["saved", "unchanged"])("refreshes a stopped %s copy from the current canonical revision before retry", async (state) => { + const copy = await run(); + if (state === "saved") await fs.writeFile(path.join(copy.localRoot, entryFile), "previous saved turn"); + expect((await copies.collectStopped({ companyId, runId: copy.runId }))?.state).toBe(state); + const before = (await revisions.readCurrent(target(), board()))!; + const current = await revisions.commit({ ...target(), entryFile, baseRevisionId: before.revision.id, content: "new board instructions", source: "api" }, board()); + const retried = (await copies.prepare({ legacy: true, ...target(), runId: copy.runId, cwd: home }))!; + expect(retried).toMatchObject({ state: "prepared", baseRevisionId: current.revision.id, baseHash: current.revision.contentHash, processStoppedAt: null }); + expect(await fs.readFile(path.join(retried.localRoot, entryFile), "utf8")).toBe("new board instructions"); + await fs.writeFile(path.join(retried.localRoot, entryFile), "retry adds a change"); + expect((await copies.collectStopped({ companyId, runId: copy.runId }))?.state).toBe("saved"); + }); + + it("never rebases uncollected private bytes or a live warm copy onto a newer head", async () => { + const stopped = await run(); + await copies.collectStopped({ companyId, runId: stopped.runId }); + await fs.writeFile(path.join(stopped.localRoot, entryFile), "uncollected private edit"); + const warm = await run(); + expect(await copies.hasChanges({ companyId, runId: warm.runId })).toBe(false); + const prior = (await revisions.readCurrent(target(), board()))!; + await revisions.commit({ ...target(), entryFile, baseRevisionId: prior.revision.id, content: "board change", source: "api" }, board()); + for (const copy of [stopped, warm]) { + const retried = (await copies.prepare({ legacy: true, ...target(), runId: copy.runId, cwd: home }))!; + expect(retried.baseRevisionId).toBe(copy.baseRevisionId); + } + expect(await fs.readFile(path.join(stopped.localRoot, entryFile), "utf8")).toBe("uncollected private edit"); + expect(await fs.readFile(path.join(warm.localRoot, entryFile), "utf8")).toBe(initial); + expect((await copies.collectStopped({ companyId, runId: stopped.runId }))?.state).toBe("conflict"); + expect((await revisions.readCurrent(target(), board()))?.content).toBe("board change"); + }); + + it.each(["before replacement", "after replacement"])("preserves the completed receipt when retry staging fails %s", async (point) => { + const copy = await run(); + await fs.writeFile(path.join(copy.localRoot, entryFile), "completed turn"); + const saved = (await copies.collectStopped({ companyId, runId: copy.runId }))!; + expect(saved.state).toBe("saved"); + const prior = (await revisions.readCurrent(target(), board()))!; + const current = await revisions.commit({ ...target(), entryFile, baseRevisionId: prior.revision.id, content: "new board instructions", source: "api" }, board()); + const originalMkdir = fs.mkdir.bind(fs); + const originalChmod = fs.chmod.bind(fs); + const mkdir = vi.spyOn(fs, "mkdir").mockImplementation(async (...args: Parameters) => { + if (point === "before replacement" && args[0] === copy.localRoot) throw new Error("retry staging failed"); + return originalMkdir(...args); + }); + const chmod = vi.spyOn(fs, "chmod").mockImplementation(async (...args) => { + if (point === "after replacement" && args[0] === path.join(copy.localRoot, entryFile)) throw new Error("retry staging failed"); + return originalChmod(...args); + }); + try { + await expect(copies.prepare({ legacy: true, ...target(), runId: copy.runId, cwd: home })).rejects.toThrow("retry staging failed"); + } finally { mkdir.mockRestore(); chmod.mockRestore(); } + expect(await copies.get(companyId, copy.runId)).toEqual(saved); + copies = agentInstructionWorkingCopyService(db); + await copies.recoverStopped(); + expect(await copies.get(companyId, copy.runId)).toEqual(saved); + expect((await revisions.readCurrent(target(), board()))?.revision.id).toBe(current.revision.id); + const retried = (await copies.prepare({ legacy: true, ...target(), runId: copy.runId, cwd: home }))!; + expect(retried).toMatchObject({ state: "prepared", baseRevisionId: current.revision.id, processStoppedAt: null }); + expect(await fs.readFile(path.join(retried.localRoot, entryFile), "utf8")).toBe(current.content); + }); + + it("reads an existing canonical runtime snapshot without seeding or changing instruction bytes", async () => { + expect((await revisions.readCommittedForRuntime(target()))?.content).toBe(initial); + const copy = await run(); + await fs.writeFile(path.join(copy.localRoot, entryFile), "saved runtime content"); + await copies.collectStopped({ companyId, runId: copy.runId }); + const current = (await revisions.readCurrent(target(), board()))!; + await fs.writeFile(path.join(root, entryFile), "current filesystem contents"); + expect((await revisions.readCommittedForRuntime(target()))?.content).toBe("current filesystem contents"); + const [agent] = await db.select().from(agents).where(eq(agents.id, agentId)); + const context = await buildNativeRuntimeContext({ db, agent, runId: copy.runId, runtimeConfig: {}, runtimeSkillEntries: [] }); + try { + expect(context.instructions.entryPath).toBe(entryFile); + expect(await fs.readFile(path.join(context.instructions.bundle.rootPath, entryFile), "utf8")) + .toBe("current filesystem contents"); + } finally { + // Runtime assets are immutable, so make only this fixture's directories + // removable before the ordinary temporary-home cleanup. + const makeRemovable = async (directory: string): Promise => { + await fs.chmod(directory, 0o700); + for (const child of await fs.readdir(directory, { withFileTypes: true })) { + if (child.isDirectory()) await makeRemovable(path.join(directory, child.name)); + } + }; + await makeRemovable(context.instructions.bundle.rootPath); + } + expect(await fs.readFile(path.join(root, entryFile), "utf8")).toBe("current filesystem contents"); + expect((await revisions.history({ ...target(), entryFile }, board())).revisions).toHaveLength(0); + await expect(revisions.readCommittedForRuntime({ companyId: randomUUID(), agentId })).rejects.toThrow("Agent not found"); + }); + + it("preserves a competing stale candidate and requires explicit resolution against the new head", async () => { + const first = await run(); + const second = await run(); + await fs.writeFile(path.join(first.localRoot, entryFile), "first run"); + await fs.writeFile(path.join(second.localRoot, entryFile), "second run"); + await copies.collectStopped({ companyId, runId: first.runId }); + const conflict = await copies.collectStopped({ companyId, runId: second.runId }); + expect(conflict).toMatchObject({ state: "conflict", errorCode: "INSTRUCTION_REVISION_CONFLICT" }); + expect(Buffer.from(conflict!.candidateBase64!, "base64").toString("utf8")).toBe("second run"); + expect((await revisions.readCurrent(target(), board()))?.content).toBe("first run"); + copies = agentInstructionWorkingCopyService(db); + expect(await copies.list(companyId, agentId, board())).toHaveLength(1); + const current = (await revisions.readCurrent(target(), board()))!; + await copies.resolve({ ...target(), runId: second.runId, baseRevisionId: current.revision.id, content: "explicitly combined" }, board()); + expect((await revisions.readCurrent(target(), board()))?.content).toBe("explicitly combined"); + expect((await copies.collectStopped({ companyId, runId: second.runId }))?.state).toBe("resolved"); + const resumed = (await copies.prepare({ legacy: true, ...target(), runId: second.runId, cwd: home }))!; + const resolved = (await revisions.readCurrent(target(), board()))!; + expect(resumed.baseRevisionId).toBe(resolved.revision.id); + expect(await fs.readFile(path.join(resumed.localRoot, entryFile), "utf8")).toBe("explicitly combined"); + }); + + it("rechecks responsible-user permission at collection and preserves denied edits", async () => { + const copy = await run(); + await fs.writeFile(path.join(copy.localRoot, entryFile), "candidate after revocation"); + await db.update(companyMemberships).set({ status: "suspended" }).where(eq(companyMemberships.principalId, userId)); + const denied = await copies.collectStopped({ companyId, runId: copy.runId }); + expect(denied).toMatchObject({ state: "conflict", errorCode: "RESPONSIBLE_USER_UNAVAILABLE" }); + expect(denied?.candidateBase64).not.toBeNull(); + expect(await fs.readFile(path.join(root, entryFile), "utf8")).toBe(initial); + }); + + it("does not import a symlink or confuse a removed entry with an empty file", async () => { + const linked = await run(); + await fs.unlink(path.join(linked.localRoot, entryFile)); + await fs.symlink(path.join(home, "AGENTS.md"), path.join(linked.localRoot, entryFile)); + expect((await copies.collectStopped({ companyId, runId: linked.runId }))?.state).toBe("pending_collection"); + const missing = await run(); + await fs.unlink(path.join(missing.localRoot, entryFile)); + expect((await copies.collectStopped({ companyId, runId: missing.runId }))?.state).toBe("unavailable"); + const empty = await run(); + await fs.writeFile(path.join(empty.localRoot, entryFile), ""); + expect((await copies.collectStopped({ companyId, runId: empty.runId }))?.state).toBe("saved"); + expect((await revisions.readCurrent(target(), board()))?.content).toBe(""); + }); + + it("keeps unchanged warm copies reusable without a content revision", async () => { + const copy = await run(); + expect(await copies.hasChanges({ companyId, runId: copy.runId })).toBe(false); + expect((await copies.get(companyId, copy.runId))?.processStoppedAt).toBeNull(); + expect((await revisions.history({ ...target(), entryFile }, board())).revisions).toHaveLength(0); + await fs.writeFile(path.join(copy.localRoot, entryFile), "changed after turn"); + expect(await copies.hasChanges({ companyId, runId: copy.runId })).toBe(true); + expect((await revisions.readCurrent(target(), board()))?.content).toBe(initial); + }); + + it("advances an explicit save baseline only when the private copy matches that revision", async () => { + const copy = await run(); + const local = path.join(copy.localRoot, entryFile); + await fs.writeFile(local, "explicit self-save"); + const first = await revisions.commit({ ...target(), entryFile, baseRevisionId: copy.baseRevisionId, content: "explicit self-save", source: "tool" }, board()); + await copies.acknowledgeExplicitSave({ ...target(), runId: copy.runId, entryFile, revisionId: first.revision.id, contentHash: first.revision.contentHash }); + await fs.writeFile(local, "local edit after explicit save"); + expect((await copies.collectStopped({ companyId, runId: copy.runId }))?.state).toBe("saved"); + expect((await revisions.readCurrent(target(), board()))?.content).toBe("local edit after explicit save"); + + const competing = await run(); + await fs.writeFile(path.join(competing.localRoot, entryFile), "older local draft"); + const separate = await revisions.commit({ ...target(), entryFile, baseRevisionId: competing.baseRevisionId, content: "different explicit content", source: "tool" }, board()); + await copies.acknowledgeExplicitSave({ ...target(), runId: competing.runId, entryFile, revisionId: separate.revision.id, contentHash: separate.revision.contentHash }); + expect((await copies.collectStopped({ companyId, runId: competing.runId }))?.state).toBe("conflict"); + expect((await revisions.readCurrent(target(), board()))?.content).toBe("different explicit content"); + expect((await copies.list(companyId, agentId, board()))[0]).not.toHaveProperty("localRoot"); + }); + + it("recovers captured bytes after restart and deduplicates concurrent cleanup callbacks", async () => { + const copy = await run(); + await db.update(agentInstructionWorkingCopies).set({ state: "pending_commit", candidateBase64: Buffer.from("captured before restart").toString("base64"), processStoppedAt: new Date() }) + .where(eq(agentInstructionWorkingCopies.runId, copy.runId)); + copies = agentInstructionWorkingCopyService(db); + await copies.recoverCaptured(); + expect((await revisions.readCurrent(target(), board()))?.content).toBe("captured before restart"); + const duplicate = await run(); + await fs.writeFile(path.join(duplicate.localRoot, entryFile), "one final edit"); + await Promise.all([copies.collectStopped({ companyId, runId: duplicate.runId }), copies.collectStopped({ companyId, runId: duplicate.runId })]); + expect((await copies.get(companyId, duplicate.runId))?.state).toBe("saved"); + expect((await revisions.history({ ...target(), entryFile }, board())).revisions).toHaveLength(0); + }); + it("requires durable stop evidence before recovering an uncaptured copy", async () => { + const copy = await run(); + await fs.writeFile(path.join(copy.localRoot, entryFile), "edit before controller restart"); + await db.update(heartbeatRuns).set({ runtimeMode: "native", status: "failed", finishedAt: new Date() }).where(eq(heartbeatRuns.id, copy.runId)); + copies = agentInstructionWorkingCopyService(db); + await copies.recoverStopped(); + expect((await revisions.readCurrent(target(), board()))?.content).toBe(initial); + expect((await copies.list(companyId, agentId, board()))[0]).toMatchObject({ state: "pending_collection", content: null }); + await appendHeartbeatRunEvent(db, { ...target(), runId: copy.runId, eventType: "native.local_process_stopped", stream: "system" }); + await copies.recoverStopped(); + expect((await revisions.readCurrent(target(), board()))?.content).toBe("edit before controller restart"); + }); + + it("never treats a superseded stop receipt or unavailable environment as a save", async () => { + const copy = await run(); + await fs.writeFile(path.join(copy.localRoot, entryFile), "unverified live edit"); + await db.update(heartbeatRuns).set({ runtimeMode: "native", status: "failed", finishedAt: new Date() }).where(eq(heartbeatRuns.id, copy.runId)); + await appendHeartbeatRunEvent(db, { ...target(), runId: copy.runId, eventType: "native.local_process_stopped", stream: "system" }); + await appendHeartbeatRunEvent(db, { ...target(), runId: copy.runId, eventType: "native.process_start_requested", stream: "system" }); + await copies.recoverStopped(); + expect((await revisions.readCurrent(target(), board()))?.content).toBe(initial); + const lost = await copies.reportUnavailable(companyId, copy.runId); + expect(lost).toMatchObject({ state: "unavailable", candidateBase64: null, processStoppedAt: null }); + expect(lost?.errorMessage).not.toContain(copy.localRoot); + }); + + it("keeps private run copies out of normal Git staging and workspace snapshots", async () => { + const workspace = path.join(home, `workspace-${randomUUID()}`); + await fs.mkdir(workspace); + await execFile("git", ["init", workspace]); + await execFile("git", ["-C", workspace, "-c", "user.name=Fixture", "-c", "user.email=fixture@example.test", "commit", "--allow-empty", "-m", "fixture"]); + const runId = randomUUID(); + await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, invocationSource: "on_demand", responsibleUserId: userId }); + const copy = (await copies.prepare({ legacy: true, ...target(), runId, cwd: workspace }))!; + expect(copy.localRoot).toBe(path.join(workspace, ".paperclip-runtime", `instruction-edits-${runId}`, "instructions")); + await fs.writeFile(path.join(workspace, "deliverable.txt"), "public work"); + await execFile("git", ["-C", workspace, "add", "."]); + const staged = await execFile("git", ["-C", workspace, "diff", "--cached", "--name-only"]); + expect(staged.stdout).toBe("deliverable.txt\n"); + const snapshot = await gitWorkspaceSync.readGitWorkspaceSnapshot(workspace); + try { + const paths: unknown = snapshot?.overlayPaths; + if (Array.isArray(paths)) { + expect(paths).toEqual(["deliverable.txt"]); + } else { + // The streaming Git snapshot stores the same path set in a private + // manifest. Inspect its actual records, not only the reported count. + expect(paths).toMatchObject({ kind: "path_manifest", version: 1, category: "overlay", count: 1 }); + const manifest = paths as { filePath: string; category: string }; + const manifestDb = new DatabaseSync(manifest.filePath, { readOnly: true, allowExtension: false }); + try { + const rows = manifestDb.prepare("SELECT path FROM records WHERE category = ? ORDER BY path").all(manifest.category); + expect(rows.map((row) => row.path)).toEqual(["deliverable.txt"]); + } finally { manifestDb.close(); } + } + } finally { + if ("disposeGitWorkspaceSnapshot" in gitWorkspaceSync && typeof gitWorkspaceSync.disposeGitWorkspaceSnapshot === "function") { + await gitWorkspaceSync.disposeGitWorkspaceSnapshot(snapshot); + } + } + const files = await captureDirectorySnapshot(workspace, { exclude: [".git", ".paperclip-runtime"] }); + expect([...files.entries].map(([relative]) => relative)).toEqual(["deliverable.txt"]); + await expect(copies.prepare({ legacy: true, ...target(), runId, cwd: home })).rejects.toThrow("different run workspace"); + }); + +}); diff --git a/server/src/__tests__/agent-instructions-routes.test.ts b/server/src/__tests__/agent-instructions-routes.test.ts index 0a110b7802..494f89ffb0 100644 --- a/server/src/__tests__/agent-instructions-routes.test.ts +++ b/server/src/__tests__/agent-instructions-routes.test.ts @@ -1,5 +1,6 @@ import express from "express"; import request from "supertest"; +import { Readable } from "node:stream"; import { beforeEach, describe, expect, it, vi } from "vitest"; const mockAgentService = vi.hoisted(() => ({ @@ -24,6 +25,21 @@ const mockAgentInstructionsService = vi.hoisted(() => ({ materializeManagedBundle: vi.fn(), })); +const mockInstructionWorkingCopies = vi.hoisted(() => ({ list: vi.fn(), resolve: vi.fn(), acknowledgeExplicitSave: vi.fn() })); +vi.mock("../services/agent-instruction-working-copies.js", () => ({ agentInstructionWorkingCopyService: () => mockInstructionWorkingCopies })); + +const mockDownloadAgentFile = vi.hoisted(() => vi.fn()); +vi.mock("../services/agent-file-store.js", async (importOriginal) => { + const actual = await importOriginal(); + return { ...actual, agentFileStore: (...args: Parameters) => ({ ...actual.agentFileStore(...args), download: mockDownloadAgentFile }) }; +}); + +const mockInstructionRevisions = vi.hoisted(() => ({ readCurrent: vi.fn(), commit: vi.fn(), restore: vi.fn(), history: vi.fn(), readRevision: vi.fn(), diff: vi.fn(), materializeCurrent: vi.fn() })); +vi.mock("../services/agent-instruction-revisions.js", () => ({ agentInstructionRevisionService: () => mockInstructionRevisions })); + +const mockAuthorizeInstructionRead = vi.hoisted(() => vi.fn()); +vi.mock("../services/agent-instruction-authorization.js", () => ({ authorizeInstructionRead: mockAuthorizeInstructionRead })); + const mockAccessService = vi.hoisted(() => ({ canUser: vi.fn(), decide: vi.fn(), @@ -202,6 +218,9 @@ describe("agent instructions bundle routes", () => { vi.doUnmock("../middleware/index.js"); registerModuleMocks(); vi.clearAllMocks(); + mockAuthorizeInstructionRead.mockImplementation(async (_db, actor) => actor); + mockInstructionRevisions.readCurrent.mockResolvedValue(null); + mockInstructionRevisions.commit.mockResolvedValue({ revision: { id: "33333333-3333-4333-8333-333333333333", entryFile: "AGENTS.md", byteLength: 18 }, content: "# Updated Agent\n", changed: true, materialization: "current" }); mockBuiltInAgentService.ensureCompanyDefaultAgentGrants.mockResolvedValue(0); mockSyncInstructionsBundleConfigFromFilePath.mockImplementation((_agent, config) => config); mockFindServerAdapter.mockImplementation((_type: string) => ({ type: _type })); @@ -286,6 +305,8 @@ describe("agent instructions bundle routes", () => { entryFile: "AGENTS.md", }); expect(mockAgentInstructionsService.getBundle).toHaveBeenCalled(); + expect(mockAuthorizeInstructionRead).toHaveBeenCalledWith(expect.anything(), expect.objectContaining({ type: "board" }), + expect.objectContaining({ companyId: "company-1", id: "11111111-1111-4111-8111-111111111111" })); }); it("requires instance-admin access for every external instruction entry point", async () => { @@ -331,6 +352,7 @@ describe("agent instructions bundle routes", () => { expect(mockAgentInstructionsService.updateBundle).not.toHaveBeenCalled(); expect(mockAgentInstructionsService.writeFile).not.toHaveBeenCalled(); expect(mockAgentInstructionsService.deleteFile).not.toHaveBeenCalled(); + expect(mockAuthorizeInstructionRead).not.toHaveBeenCalled(); }); it("treats a host root mislabeled as managed as external", async () => { @@ -383,6 +405,7 @@ describe("agent instructions bundle routes", () => { expect(res.status, JSON.stringify(res.body)).toBe(200); expect(mockAgentInstructionsService.getBundle).toHaveBeenCalled(); + expect(mockAuthorizeInstructionRead).not.toHaveBeenCalled(); }); it("rejects a company admin that requests a new external instruction root", async () => { @@ -484,7 +507,8 @@ describe("agent instructions bundle routes", () => { expect(mockAgentService.create).not.toHaveBeenCalled(); }); - it("denies non-privileged agents from reading peer instructions bundles", async () => { + it.each(["", "/file?path=AGENTS.md", "/file?path=private-support.md"])( + "denies peer instruction reads before any data dispatch: %s", async (suffix) => { mockAgentService.getById.mockImplementation(async (id: string) => { if (id === "agent-reader") { return { @@ -496,11 +520,10 @@ describe("agent instructions bundle routes", () => { } return makeAgent(); }); - mockAccessService.decide.mockResolvedValue({ - allowed: false, - reason: "deny_no_grant", - explanation: "Missing permission: agents:configure or agents:suggest-changes.", - }); + // General agent visibility is allowed. It must not authorize instruction + // metadata, canonical entry bytes, or supporting files from a peer. + const { forbidden } = await vi.importActual("../errors.js"); + mockAuthorizeInstructionRead.mockRejectedValue(forbidden("Missing permission to read peer instructions")); const res = await requestApp( await createApp({ @@ -510,20 +533,18 @@ describe("agent instructions bundle routes", () => { source: "agent_key", }), (baseUrl) => request(baseUrl) - .get("/api/agents/11111111-1111-4111-8111-111111111111/instructions-bundle"), + .get(`/api/agents/11111111-1111-4111-8111-111111111111/instructions-bundle${suffix}`), ); expect(res.status, JSON.stringify(res.body)).toBe(403); expect(res.body.error).toContain("Missing permission"); - expect(mockAccessService.decide).toHaveBeenCalledWith(expect.objectContaining({ - action: "agent_config:read", - resource: { - type: "agent", - companyId: "company-1", - agentId: "11111111-1111-4111-8111-111111111111", - }, - })); + expect(mockAuthorizeInstructionRead).toHaveBeenCalledWith(expect.anything(), expect.objectContaining({ + type: "agent", agentId: "agent-reader", + }), { companyId: "company-1", id: "11111111-1111-4111-8111-111111111111" }); expect(mockAgentInstructionsService.getBundle).not.toHaveBeenCalled(); + expect(mockAgentInstructionsService.readFile).not.toHaveBeenCalled(); + expect(mockInstructionRevisions.readCurrent).not.toHaveBeenCalled(); + expect(mockInstructionRevisions.materializeCurrent).not.toHaveBeenCalled(); }); it("allows agents to read their own instructions bundles", async () => { @@ -540,9 +561,12 @@ describe("agent instructions bundle routes", () => { expect(res.status, JSON.stringify(res.body)).toBe(200); expect(mockAgentInstructionsService.getBundle).toHaveBeenCalled(); + expect(mockAuthorizeInstructionRead).toHaveBeenCalledWith(expect.anything(), expect.objectContaining({ + type: "agent", agentId: "11111111-1111-4111-8111-111111111111", + }), { companyId: "company-1", id: "11111111-1111-4111-8111-111111111111" }); }); - it("allows agents with suggest grants to read peer instructions bundles", async () => { + it.each(["AGENTS.md", "private-support.md"])("allows an authorized peer to read instruction file %s", async (filePath) => { mockAccessService.decide.mockResolvedValue({ allowed: true, reason: "allow_explicit_grant", @@ -570,52 +594,100 @@ describe("agent instructions bundle routes", () => { }), (baseUrl) => request(baseUrl) .get("/api/agents/11111111-1111-4111-8111-111111111111/instructions-bundle/file") - .query({ path: "AGENTS.md" }), + .query({ path: filePath }), ); expect(res.status, JSON.stringify(res.body)).toBe(200); - expect(mockAccessService.decide).toHaveBeenCalledWith(expect.objectContaining({ - action: "agent_config:read", - resource: { - type: "agent", - companyId: "company-1", - agentId: "11111111-1111-4111-8111-111111111111", - }, - })); + expect(mockAuthorizeInstructionRead).toHaveBeenCalledWith(expect.anything(), expect.objectContaining({ + type: "agent", agentId: "coach-agent", + }), { companyId: "company-1", id: "11111111-1111-4111-8111-111111111111" }); expect(mockAgentInstructionsService.readFile).toHaveBeenCalledWith( expect.objectContaining({ id: "11111111-1111-4111-8111-111111111111" }), - "AGENTS.md", + filePath, ); }); - it("writes a bundle file and persists compatibility config", async () => { + it("commits entry bytes through the canonical service with the authenticated actor", async () => { const res = await requestApp(await createApp(), (baseUrl) => request(baseUrl) .put("/api/agents/11111111-1111-4111-8111-111111111111/instructions-bundle/file?companyId=company-1") - .send({ - path: "AGENTS.md", - content: "# Updated Agent\n", - clearLegacyPromptTemplate: true, - })); - + .send({ path: "AGENTS.md", content: "# Updated Agent\n", baseRevisionId: null })); expect(res.status, JSON.stringify(res.body)).toBe(200); - expect(mockAgentInstructionsService.writeFile).toHaveBeenCalledWith( - expect.objectContaining({ id: "11111111-1111-4111-8111-111111111111" }), - "AGENTS.md", - "# Updated Agent\n", - { clearLegacyPromptTemplate: true }, - ); - expect(mockAgentService.update).toHaveBeenCalledWith( - "11111111-1111-4111-8111-111111111111", - expect.objectContaining({ - adapterConfig: expect.objectContaining({ - instructionsBundleMode: "managed", - instructionsRootPath: "/tmp/agent-1", - instructionsEntryFile: "AGENTS.md", - instructionsFilePath: "/tmp/agent-1/AGENTS.md", - }), - }), - expect.any(Object), - ); + expect(mockInstructionRevisions.commit).toHaveBeenCalledWith(expect.objectContaining({ + agentId: "11111111-1111-4111-8111-111111111111", entryFile: "AGENTS.md", content: "# Updated Agent\n", baseRevisionId: null, source: "board", + }), expect.objectContaining({ type: "board" })); + expect(mockAgentInstructionsService.writeFile).not.toHaveBeenCalled(); + expect(res.body.receipt.revision.id).toBe("33333333-3333-4333-8333-333333333333"); + }); + + it("returns committed entry content independently of its disk copy", async () => { + mockInstructionRevisions.readCurrent.mockResolvedValue({ revision: { id: "33333333-3333-4333-8333-333333333333", entryFile: "AGENTS.md", byteLength: 9 }, content: "committed" }); + const res = await requestApp(await createApp(), (baseUrl) => request(baseUrl) + .get("/api/agents/11111111-1111-4111-8111-111111111111/instructions-bundle/file?path=AGENTS.md")); + expect(res.status).toBe(200); + expect(res.body.content).toBe("committed"); + expect(res.body.revision.id).toBe("33333333-3333-4333-8333-333333333333"); + expect(mockAgentInstructionsService.readFile).not.toHaveBeenCalled(); + }); + + it.each([Buffer.from([0, 255, 128, 17]), Buffer.alloc(0)])("streams an authorized binary download without text conversion (%j)", async bytes => { + mockAgentService.getById.mockResolvedValue({ ...makeAgent(), adapterConfig: { instructionsBundleMode: "managed" } }); + mockDownloadAgentFile.mockResolvedValue({ size: bytes.length, stream: Readable.from(bytes.length ? [bytes] : []) }); + const res = await requestApp(await createApp(), url => request(url) + .get("/api/agents/11111111-1111-4111-8111-111111111111/instructions-bundle/file") + .query({ path: "notes/data.bin", download: "true" })); + expect(res.status, JSON.stringify(res.body)).toBe(200); + expect(res.body).toEqual(bytes); + expect(res.headers["content-length"]).toBe(String(bytes.length)); + expect(res.headers["content-disposition"]).toContain('filename="data.bin"'); + expect(mockDownloadAgentFile).toHaveBeenCalledWith("company-1", "11111111-1111-4111-8111-111111111111", "notes/data.bin", expect.objectContaining({ type: "board" })); + expect(mockAgentInstructionsService.readFile).not.toHaveBeenCalled(); + }); + + it("lists and resolves preserved edits with server scope, actor, and explicit revision", async () => { + const app = await createApp(); + const runId = "55555555-5555-4555-8555-555555555555"; + const baseRevisionId = "44444444-4444-4444-8444-444444444444"; + const prefix = "/api/agents/11111111-1111-4111-8111-111111111111/instructions-bundle/candidates"; + mockInstructionWorkingCopies.list.mockResolvedValue([{ runId, entryFile: "AGENTS.md", state: "conflict", content: "preserved" }]); + mockInstructionWorkingCopies.resolve.mockResolvedValue({ revision: { id: baseRevisionId, entryFile: "AGENTS.md", byteLength: 8 }, content: "resolved", changed: true, materialization: "current" }); + const listed = await requestApp(app, (url) => request(url).get(prefix)); + expect(listed.status).toBe(200); + expect(listed.body).toEqual([{ runId, entryFile: "AGENTS.md", state: "conflict", content: "preserved" }]); + expect(mockInstructionWorkingCopies.list).toHaveBeenCalledWith("company-1", "11111111-1111-4111-8111-111111111111", expect.objectContaining({ type: "board", userId: "local-board" })); + const resolved = await requestApp(app, (url) => request(url).post(`${prefix}/${runId}/resolve`).send({ baseRevisionId, content: "resolved" })); + expect(resolved.status).toBe(200); + expect(resolved.body).toMatchObject({ content: "resolved", receipt: { changed: true } }); + expect(mockInstructionWorkingCopies.resolve).toHaveBeenCalledWith({ companyId: "company-1", agentId: "11111111-1111-4111-8111-111111111111", runId, baseRevisionId, content: "resolved" }, expect.objectContaining({ type: "board", userId: "local-board" })); + for (const body of [{ content: "missing base" }, { baseRevisionId, content: "forged", responsibleUserId: "other" }, { baseRevisionId, content: "forged", entryFile: "other.md" }]) { + expect((await requestApp(app, (url) => request(url).post(`${prefix}/${runId}/resolve`).send(body))).status).toBe(400); + } + expect(mockInstructionWorkingCopies.resolve).toHaveBeenCalledOnce(); + }); + + it("exposes scoped history, diff and restore with the server actor", async () => { + const app = await createApp(); + const revisionId = "33333333-3333-4333-8333-333333333333"; + const baseRevisionId = "44444444-4444-4444-8444-444444444444"; + const prefix = "/api/agents/11111111-1111-4111-8111-111111111111/instructions-bundle"; + mockInstructionRevisions.history.mockResolvedValue({ revisions: [], nextCursor: null }); + mockInstructionRevisions.diff.mockResolvedValue({ removed: "old", added: "new" }); + mockInstructionRevisions.restore.mockResolvedValue({ revision: { id: revisionId, entryFile: "AGENTS.md", byteLength: 3 }, content: "old", changed: true, materialization: "current" }); + expect((await requestApp(app, (url) => request(url).get(`${prefix}/history?path=AGENTS.md`))).status).toBe(200); + expect((await requestApp(app, (url) => request(url).get(`${prefix}/diff?path=AGENTS.md&from=${revisionId}&to=${baseRevisionId}`))).status).toBe(200); + const restored = await requestApp(app, (url) => request(url).post(`${prefix}/restore`).send({ path: "AGENTS.md", revisionId, baseRevisionId })); + expect(restored.status).toBe(200); + expect(restored.body.receipt.changed).toBe(true); + expect(mockInstructionRevisions.restore).toHaveBeenCalledWith({ companyId: "company-1", agentId: "11111111-1111-4111-8111-111111111111", entryFile: "AGENTS.md", revisionId, baseRevisionId }, expect.objectContaining({ type: "board", userId: "local-board" })); + }); + + it("requires a base revision and rejects client-supplied responsible identity", async () => { + const app = await createApp(); + for (const body of [ { path: "AGENTS.md", content: "stale" }, { path: "AGENTS.md", content: "forged", baseRevisionId: null, responsibleUserId: "forged" } ]) { + const res = await requestApp(app, (baseUrl) => request(baseUrl) + .put("/api/agents/11111111-1111-4111-8111-111111111111/instructions-bundle/file").send(body)); + expect([400, 422]).toContain(res.status); + } + expect(mockInstructionRevisions.commit).not.toHaveBeenCalled(); }); it("preserves managed instructions config when switching adapters", async () => { diff --git a/server/src/__tests__/agent-instructions-service.test.ts b/server/src/__tests__/agent-instructions-service.test.ts index 2dc4719d1a..7641dcc677 100644 --- a/server/src/__tests__/agent-instructions-service.test.ts +++ b/server/src/__tests__/agent-instructions-service.test.ts @@ -1,7 +1,7 @@ import fs from "node:fs/promises"; import os from "node:os"; import path from "node:path"; -import { afterEach, describe, expect, it } from "vitest"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; import { agentInstructionsService } from "../services/agent-instructions.js"; type TestAgent = { @@ -12,7 +12,7 @@ type TestAgent = { }; async function makeTempDir(prefix: string) { - return fs.mkdtemp(path.join(os.tmpdir(), prefix)); + return fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), prefix))); } function makeAgent(adapterConfig: Record): TestAgent { @@ -29,6 +29,13 @@ describe("agent instructions service", () => { const originalPaperclipInstanceId = process.env.PAPERCLIP_INSTANCE_ID; const cleanupDirs = new Set(); + beforeEach(async () => { + const home = await makeTempDir("agent-instructions-test-home-"); + cleanupDirs.add(home); + process.env.PAPERCLIP_HOME = home; + process.env.PAPERCLIP_INSTANCE_ID = "instructions-service-test"; + }); + afterEach(async () => { if (originalPaperclipHome === undefined) delete process.env.PAPERCLIP_HOME; else process.env.PAPERCLIP_HOME = originalPaperclipHome; @@ -41,6 +48,37 @@ describe("agent instructions service", () => { })); }); + it.each(["./AGENTS.md", "notes/../AGENTS.md", "notes\\..\\AGENTS.md", "../invalid"])("reads legacy entry configuration %s without breaking the bundle", async (entry) => { + const root = await makeTempDir("legacy-entry-"); cleanupDirs.add(root); + await fs.writeFile(path.join(root, "AGENTS.md"), "legacy instructions"); + const agent = makeAgent({ instructionsBundleMode: "external", instructionsRootPath: root, instructionsEntryFile: entry }); + const svc = agentInstructionsService(); + const bundle = await svc.getBundle(agent); + expect(bundle.entryFile).toBe("AGENTS.md"); + expect((await svc.readFile(agent, bundle.entryFile)).content).toBe("legacy instructions"); + if (entry === "../invalid") expect(bundle.warnings.length).toBeGreaterThan(0); + }); + + it("lists external bundles containing large binary assets", async () => { + const root = await makeTempDir("external-large-assets-"); cleanupDirs.add(root); + await fs.writeFile(path.join(root, "AGENTS.md"), "external instructions"); + const asset = await fs.open(path.join(root, "large.bin"), "w"); + try { await asset.truncate(17 * 1024 * 1024); } finally { await asset.close(); } + const agent = makeAgent({ instructionsBundleMode: "external", instructionsRootPath: root }); + const svc = agentInstructionsService(); + const bundle = await svc.getBundle(agent); + expect(bundle.files).toContainEqual(expect.objectContaining({ path: "large.bin", binary: true, editable: false })); + expect((await svc.readFile(agent, "AGENTS.md")).content).toBe("external instructions"); + }); + + it("rejects reserved paths while initializing an unconfigured managed bundle", async () => { + const root = await makeTempDir("unconfigured-reserved-path-"); cleanupDirs.add(root); + process.env.PAPERCLIP_HOME = root; + const svc = agentInstructionsService(); + await expect(svc.writeFile(makeAgent({}), ".paperclip-runtime/state", "invalid")).rejects.toMatchObject({ status: 422 }); + expect(await fs.readdir(root)).toEqual([]); + }); + it("copies the existing bundle into the managed root when switching to managed mode", async () => { const paperclipHome = await makeTempDir("paperclip-agent-instructions-home-"); const externalRoot = await makeTempDir("paperclip-agent-instructions-external-"); @@ -53,7 +91,7 @@ describe("agent instructions service", () => { await fs.mkdir(path.join(externalRoot, "docs"), { recursive: true }); await fs.writeFile(path.join(externalRoot, "docs", "TOOLS.md"), "## Tools\n", "utf8"); - const svc = agentInstructionsService(); + const svc = agentInstructionsService({ select: () => ({ from: () => ({ where: async () => [] }) }) } as never); const agent = makeAgent({ instructionsBundleMode: "external", instructionsRootPath: externalRoot, @@ -250,11 +288,10 @@ describe("agent instructions service", () => { expect(bundle.mode).toBe("managed"); expect(bundle.rootPath).toBe(managedRoot); - expect(bundle.entryFile).toBe("AGENTS.md"); + expect(bundle.entryFile).toBe("docs/MISSING.md"); expect(bundle.files.map((file) => file.path)).toEqual(["AGENTS.md"]); expect(bundle.warnings).toEqual([ `Recovered managed instructions from disk at ${managedRoot}; ignoring stale configured root ${staleRoot}.`, - "Recovered managed instructions entry file from disk as AGENTS.md; previous entry docs/MISSING.md was missing.", ]); expect(exported.files).toEqual({ "AGENTS.md": "# Managed Agent\n" }); }); @@ -293,8 +330,8 @@ describe("agent instructions service", () => { expect(result.adapterConfig).toMatchObject({ instructionsBundleMode: "managed", instructionsRootPath: managedRoot, - instructionsEntryFile: "AGENTS.md", - instructionsFilePath: path.join(managedRoot, "AGENTS.md"), + instructionsEntryFile: "docs/MISSING.md", + instructionsFilePath: path.join(managedRoot, "docs/MISSING.md"), }); await expect(fs.readFile(path.join(managedRoot, "docs", "TOOLS.md"), "utf8")).resolves.toBe("## Tools\n"); }); @@ -334,8 +371,8 @@ describe("agent instructions service", () => { expect(result.adapterConfig).toMatchObject({ instructionsBundleMode: "managed", instructionsRootPath: managedRoot, - instructionsEntryFile: "AGENTS.md", - instructionsFilePath: path.join(managedRoot, "AGENTS.md"), + instructionsEntryFile: "docs/MISSING.md", + instructionsFilePath: path.join(managedRoot, "docs/MISSING.md"), }); await expect(fs.stat(path.join(managedRoot, "docs", "TOOLS.md"))).rejects.toThrow(); expect(result.bundle.files.map((file) => file.path)).toEqual(["AGENTS.md"]); @@ -373,11 +410,10 @@ describe("agent instructions service", () => { expect(bundle.mode).toBe("managed"); expect(bundle.rootPath).toBe(managedRoot); - expect(bundle.entryFile).toBe("AGENTS.md"); + expect(bundle.entryFile).toBe("docs/MISSING.md"); expect(bundle.files.map((file) => file.path)).toEqual(["AGENTS.md"]); expect(bundle.warnings).toEqual([ `Recovered managed instructions from disk at ${managedRoot}; ignoring stale configured root ${staleRoot}.`, - "Recovered managed instructions entry file from disk as AGENTS.md; previous entry docs/MISSING.md was missing.", ]); expect(exported.files).toEqual({ "AGENTS.md": "# Managed Agent\n" }); }); diff --git a/server/src/__tests__/built-in-agent-routes.test.ts b/server/src/__tests__/built-in-agent-routes.test.ts index acae6704dc..e51166f2ba 100644 --- a/server/src/__tests__/built-in-agent-routes.test.ts +++ b/server/src/__tests__/built-in-agent-routes.test.ts @@ -412,7 +412,7 @@ describe("built-in agent routes", () => { .send({}); expect(res.status, JSON.stringify(res.body)).toBe(200); - expect(mockBuiltInAgentService.reset).toHaveBeenCalledWith(companyId, "briefs", {}); + expect(mockBuiltInAgentService.reset).toHaveBeenCalledWith(companyId, "briefs", {}, expect.objectContaining({ type: "agent", agentId: "manager-agent" })); expect(mockLogActivity).toHaveBeenCalledWith(expect.anything(), expect.objectContaining({ companyId, actorType: "agent", diff --git a/server/src/__tests__/built-in-agents.test.ts b/server/src/__tests__/built-in-agents.test.ts index 8f3abf3972..69de71c730 100644 --- a/server/src/__tests__/built-in-agents.test.ts +++ b/server/src/__tests__/built-in-agents.test.ts @@ -1,3 +1,6 @@ +import fs from "node:fs/promises"; +import { fileHash } from "../services/agent-file-store.js"; +import { stockHash } from "../services/managed-resource-drift.js"; import { randomUUID } from "node:crypto"; import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; @@ -27,6 +30,7 @@ import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase, } from "./helpers/embedded-postgres.js"; +import { agentInstructionRevisionService } from "../services/agent-instruction-revisions.js"; import { agentInstructionsService } from "../services/agent-instructions.ts"; import { agentService } from "../services/agents.ts"; import { approvalService } from "../services/approvals.ts"; @@ -116,6 +120,16 @@ describe("built-in agent asset loading", () => { describeEmbeddedPostgres("built-in agents", () => { let db!: ReturnType; let tempDb: Awaited> | null = null; + const instructionOperator = { type: "board", userId: "local-board", source: "local_implicit" } as const; + async function writeInstructionEntry(agent: { id: string; companyId: string }, entryFile: string, content: string) { + const service = agentInstructionRevisionService(db); + const target = { companyId: agent.companyId, agentId: agent.id }; + const baseline = await service.readCurrent(target, instructionOperator); + await service.commit({ ...target, entryFile, content, baseRevisionId: baseline?.revision.id ?? null, source: "board" }, instructionOperator); + const refreshed = await agentService(db).getById(agent.id); + return { adapterConfig: refreshed!.adapterConfig }; + } + beforeAll(async () => { tempDb = await startEmbeddedPostgresTestDatabase("paperclip-built-in-agents-"); @@ -553,7 +567,7 @@ describeEmbeddedPostgres("built-in agents", () => { capabilities: "Custom purpose", }); - const reset = await builtIns.reset(companyId, "briefs"); + const reset = await builtIns.reset(companyId, "briefs", {}, instructionOperator); expect(reset).toMatchObject({ status: "ready", @@ -795,13 +809,111 @@ describeEmbeddedPostgres("built-in agents", () => { expect(approvalRows).toHaveLength(1); }); + it("automatically upgrades untouched stock instructions while preserving personal files", async () => { + const companyId = await seedCompany({ requireApproval: false }); + const svc = builtInAgentService(db); + const created = await svc.ensure(companyId, "reflection-coach"); + const agent = created.agent!; + const oldFiles = { "AGENTS.md": "# Previous stock instructions\n", "obsolete.txt": "old stock support" }; + await writeInstructionEntry(agent, "AGENTS.md", oldFiles["AGENTS.md"]); + await agentInstructionsService(db).writeFile(agent, "obsolete.txt", oldFiles["obsolete.txt"]); + await agentInstructionsService(db).writeFile(agent, "personal.txt", "keep my notes"); + await db.update(builtInManagedResources).set({ stockHash: stockHash(oldFiles), stockVersion: "previous", + defaultsJson: { entryFile: "AGENTS.md", files: Object.keys(oldFiles) }, + }).where(and(eq(builtInManagedResources.companyId, companyId), eq(builtInManagedResources.resourceKind, "instructions"))); + const updated = await svc.ensure(companyId, "reflection-coach"); + expect(updated.resources.find(resource => resource.resourceKind === "instructions")).toMatchObject({ stockStatus: "stock_current" }); + expect((await agentInstructionsService().readFile(updated.agent!, "AGENTS.md")).content).toContain("You are Reflection Coach"); + expect((await agentInstructionsService().readFile(updated.agent!, "personal.txt")).content).toBe("keep my notes"); + await expect(agentInstructionsService().readFile(updated.agent!, "obsolete.txt")).rejects.toMatchObject({ status: 404 }); + const [event] = await db.select().from(activityLog).where(and(eq(activityLog.companyId, companyId), eq(activityLog.actorId, "built-in-reconcile"))); + expect(event).toMatchObject({ actorType: "system", action: "agent.files_updated" }); + }); + + it.each(["file", "database"])("retries a stock update after a %s failure without overwriting intervening edits", async (failure) => { + const companyId = await seedCompany({ requireApproval: false }); + const svc = builtInAgentService(db); + const created = await svc.ensure(companyId, "reflection-coach"); + const agent = created.agent!; + const oldFiles = { "AGENTS.md": "# Old stock\n", "obsolete.txt": "old supporting file" }; + await writeInstructionEntry(agent, "AGENTS.md", oldFiles["AGENTS.md"]); + await agentInstructionsService(db).writeFile(agent, "obsolete.txt", oldFiles["obsolete.txt"]); + await db.update(builtInManagedResources).set({ stockHash: stockHash(oldFiles), stockVersion: "old", + defaultsJson: { entryFile: "AGENTS.md", files: Object.keys(oldFiles) }, + }).where(and(eq(builtInManagedResources.companyId, companyId), eq(builtInManagedResources.resourceKind, "instructions"))); + const trigger = `stock_failure_${companyId.replaceAll("-", "")}`; + const remove = fs.rm.bind(fs); + const fault = failure === "file" ? vi.spyOn(fs, "rm").mockImplementation(async (file, options) => { + if (String(file).endsWith("/obsolete.txt")) throw new Error("injected stock file failure"); + return remove(file, options); + }) : null; + if (failure === "database") { + await db.execute(sql.raw(`CREATE FUNCTION ${trigger}() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN + IF NEW.company_id = '${companyId}' AND NEW.resource_kind = 'instructions' AND NEW.stock_hash <> OLD.stock_hash THEN + RAISE EXCEPTION 'injected stock binding failure'; END IF; RETURN NEW; END $$`)); + await db.execute(sql.raw(`CREATE TRIGGER ${trigger} BEFORE UPDATE ON built_in_managed_resources FOR EACH ROW EXECUTE FUNCTION ${trigger}()`)); + } + try { await expect(svc.ensure(companyId, "reflection-coach")).rejects.toThrow(); } + finally { + fault?.mockRestore(); + if (failure === "database") { + await db.execute(sql.raw(`DROP TRIGGER ${trigger} ON built_in_managed_resources`)); + await db.execute(sql.raw(`DROP FUNCTION ${trigger}()`)); + } + } + const [pending] = await db.select().from(builtInManagedResources).where(and(eq(builtInManagedResources.companyId, companyId), eq(builtInManagedResources.resourceKind, "instructions"))); + expect(pending.defaultsJson.pendingInstructionsUpdate).toBeTruthy(); + expect(pending.stockHash).toBe(stockHash(oldFiles)); + const partial = (await agentInstructionsService().readFile(agent, "AGENTS.md")).content; + expect(partial).toContain("You are Reflection Coach"); + // A real operator edit after the failed attempt must stop the retry. + await writeInstructionEntry(agent, "AGENTS.md", "operator edit after failure"); + await svc.ensure(companyId, "reflection-coach"); + expect((await agentInstructionsService().readFile(agent, "AGENTS.md")).content).toBe("operator edit after failure"); + await writeInstructionEntry(agent, "AGENTS.md", partial); + const retried = await svc.ensure(companyId, "reflection-coach"); + expect(retried.resources.find(resource => resource.resourceKind === "instructions")).toMatchObject({ stockStatus: "stock_current" }); + await expect(agentInstructionsService().readFile(agent, "obsolete.txt")).rejects.toMatchObject({ status: 404 }); + const [complete] = await db.select().from(builtInManagedResources).where(eq(builtInManagedResources.id, pending.id)); + expect(complete.defaultsJson.pendingInstructionsUpdate).toBeUndefined(); + }); + + it.each([false, true])("supersedes an interrupted stock version, including rollback=%s", async (rollback) => { + const companyId = await seedCompany({ requireApproval: false }); + const svc = builtInAgentService(db); + const created = await svc.ensure(companyId, "reflection-coach"); + const agent = created.agent!; + const currentStock = (await agentInstructionsService().readFile(agent, "AGENTS.md")).content; + const oldFiles = { "AGENTS.md": rollback ? currentStock : "# Original stock" }; + const interruptedFiles = { "AGENTS.md": "# Intermediate stock", "intermediate-only.txt": "intermediate support" }; + await writeInstructionEntry(agent, "AGENTS.md", interruptedFiles["AGENTS.md"]); + await agentInstructionsService(db).writeFile(agent, "intermediate-only.txt", interruptedFiles["intermediate-only.txt"]); + await agentInstructionsService(db).writeFile(agent, "personal.txt", "personal notes"); + await db.update(builtInManagedResources).set({ stockHash: stockHash(oldFiles), stockVersion: "old", + defaultsJson: { entryFile: "AGENTS.md", files: Object.keys(oldFiles), pendingInstructionsUpdate: { + stockHash: stockHash(interruptedFiles), + baseHashes: { "AGENTS.md": fileHash(Buffer.from(oldFiles["AGENTS.md"])), "intermediate-only.txt": null }, + nextHashes: Object.fromEntries(Object.entries(interruptedFiles).map(([file, text]) => [file, fileHash(Buffer.from(text))])), + } }, + }).where(and(eq(builtInManagedResources.companyId, companyId), eq(builtInManagedResources.resourceKind, "instructions"))); + await writeInstructionEntry(agent, "AGENTS.md", "intervening operator edit"); + await svc.ensure(companyId, "reflection-coach"); + expect((await agentInstructionsService().readFile(agent, "AGENTS.md")).content).toBe("intervening operator edit"); + await writeInstructionEntry(agent, "AGENTS.md", interruptedFiles["AGENTS.md"]); + const updated = await svc.ensure(companyId, "reflection-coach"); + expect(updated.resources.find(resource => resource.resourceKind === "instructions")).toMatchObject({ stockStatus: "stock_current" }); + expect((await agentInstructionsService().readFile(agent, "AGENTS.md")).content).toBe(currentStock); + await expect(agentInstructionsService().readFile(agent, "intermediate-only.txt")).rejects.toMatchObject({ status: 404 }); + expect((await agentInstructionsService().readFile(agent, "personal.txt")).content).toBe("personal notes"); + }); + it("preserves Reflection Coach instruction drift on reconcile and restores it on reset", async () => { const companyId = await seedCompany(); const builtIns = builtInAgentService(db); const created = await builtIns.ensure(companyId, "reflection-coach"); const instructions = agentInstructionsService(); - await instructions.writeFile(created.agent!, "AGENTS.md", "# Custom Reflection Coach\n\nOperator edit.\n"); + await writeInstructionEntry(created.agent!, "AGENTS.md", "# Custom Reflection Coach\n\nOperator edit.\n"); const reconciled = await builtIns.ensure(companyId, "reflection-coach"); const drift = reconciled.resources.find((resource) => resource.resourceKind === "instructions"); @@ -815,7 +927,7 @@ describeEmbeddedPostgres("built-in agents", () => { content: "# Custom Reflection Coach\n\nOperator edit.\n", }); - const reset = await builtIns.reset(companyId, "reflection-coach"); + const reset = await builtIns.reset(companyId, "reflection-coach", {}, instructionOperator); expect(reset.resources.find((resource) => resource.resourceKind === "instructions")).toMatchObject({ stockStatus: "stock_current", resetAvailable: false, @@ -823,6 +935,12 @@ describeEmbeddedPostgres("built-in agents", () => { const resetFile = await instructions.readFile(reset.agent!, "AGENTS.md"); expect(resetFile.content).toContain("Reflection Coach"); expect(resetFile.content).not.toContain("Operator edit."); + const revisions = agentInstructionRevisionService(db); + const target = { companyId, agentId: created.agent!.id, entryFile: "AGENTS.md" }; + const history = await revisions.history(target, instructionOperator); + expect(history.revisions).toHaveLength(0); + const current = await revisions.readCurrent(target, instructionOperator); + expect(current?.content).toBe(resetFile.content); }); it("blocks deleting a built-in agent", async () => { @@ -1270,7 +1388,7 @@ describeEmbeddedPostgres("built-in agents", () => { const created = await builtIns.ensure(companyId, "summarizer"); const instructions = agentInstructionsService(); - await instructions.writeFile(created.agent!, "AGENTS.md", "# Custom Summarizer\n\nOperator edit.\n"); + await writeInstructionEntry(created.agent!, "AGENTS.md", "# Custom Summarizer\n\nOperator edit.\n"); const reconciled = await builtIns.ensure(companyId, "summarizer"); expect(reconciled.resources.find((resource) => resource.resourceKind === "instructions")).toMatchObject({ @@ -1279,7 +1397,7 @@ describeEmbeddedPostgres("built-in agents", () => { changedFiles: ["AGENTS.md"], }); - const reset = await builtIns.reset(companyId, "summarizer"); + const reset = await builtIns.reset(companyId, "summarizer", {}, instructionOperator); expect(reset.resources.find((resource) => resource.resourceKind === "instructions")).toMatchObject({ stockStatus: "stock_current", resetAvailable: false, @@ -1419,7 +1537,7 @@ describeEmbeddedPostgres("built-in agents", () => { const coach = created.agent!; const instructionsSvc = agentInstructionsService(); const originalInstructions = "# Target Coder\n\nWork from the assigned issue.\n"; - const prepared = await instructionsSvc.writeFile(target, "AGENTS.md", originalInstructions); + const prepared = await writeInstructionEntry(target, "AGENTS.md", originalInstructions); let persistedTarget = (await agentsSvc.update(target.id, { adapterConfig: prepared.adapterConfig }))!; const interactionsSvc = issueThreadInteractionService(db); @@ -1431,7 +1549,7 @@ describeEmbeddedPostgres("built-in agents", () => { if (interaction?.kind !== "request_confirmation" || interaction.status !== "accepted") { return false; } - const written = await instructionsSvc.writeFile(persistedTarget, "AGENTS.md", input.nextInstructions); + const written = await writeInstructionEntry(persistedTarget, "AGENTS.md", input.nextInstructions); persistedTarget = (await agentsSvc.update(persistedTarget.id, { adapterConfig: written.adapterConfig }))!; return true; }; @@ -1568,7 +1686,7 @@ describeEmbeddedPostgres("built-in agents", () => { const agent = created.agent!; const instructionsSvc = agentInstructionsService(); - await instructionsSvc.writeFile(agent, "AGENTS.md", "# Custom Reflection Coach\n\nDo not overwrite me.\n"); + await writeInstructionEntry(agent, "AGENTS.md", "# Custom Reflection Coach\n\nDo not overwrite me.\n"); await db .update(companySkills) .set({ markdown: "---\nname: reflection-coach\n---\n\n# Custom skill\n" }) @@ -1597,7 +1715,7 @@ describeEmbeddedPostgres("built-in agents", () => { const reset = await builtInAgentService(db).reset(companyId, "reflection-coach", { resources: ["instructions", "routine"], - }); + }, instructionOperator); expect(reset.resources.find((resource) => resource.resourceKind === "instructions")).toMatchObject({ stockStatus: "stock_current", resetAvailable: false, diff --git a/server/src/__tests__/heartbeat-list.test.ts b/server/src/__tests__/heartbeat-list.test.ts index 465514840b..b07c8e4368 100644 --- a/server/src/__tests__/heartbeat-list.test.ts +++ b/server/src/__tests__/heartbeat-list.test.ts @@ -264,6 +264,12 @@ describeEmbeddedPostgres("heartbeat list", () => { summary: "completed", stdout: oversizedStdout, nestedHuge: { payload: oversizedNestedPayload }, + instructionSave: { + state: "unavailable", contract: "agent_files", entryFile: "AGENTS.md", + errorCode: "AGENT_FILES_LIMIT_EXCEEDED", + storageWarning: "Agent storage is full. Runs can continue.".repeat(50), + privateSyncMetadata: oversizedNestedPayload, + }, workspaceRestoreFailure: "restore_unsafe_archive", finalResponseRecorded: true, executionBeforeRestore: { errorCode: "model_error", exitCode: 2, timedOut: false }, @@ -278,6 +284,11 @@ describeEmbeddedPostgres("heartbeat list", () => { truncated: true, truncationReason: "oversized_result_json", stdoutTruncated: true, + instructionSave: { + state: "unavailable", contract: "agent_files", entryFile: "AGENTS.md", + errorCode: "AGENT_FILES_LIMIT_EXCEEDED", + storageWarning: "Agent storage is full. Runs can continue.".repeat(50).slice(0, 1024), + }, workspaceRestoreFailure: "restore_unsafe_archive", finalResponseRecorded: true, executionBeforeRestore: { errorCode: "model_error", exitCode: 2, timedOut: false }, @@ -285,6 +296,7 @@ describeEmbeddedPostgres("heartbeat list", () => { expect(typeof result?.stdout).toBe("string"); expect((result?.stdout as string).length).toBeLessThan(oversizedStdout.length); expect(result).not.toHaveProperty("nestedHuge"); + expect(result?.instructionSave).not.toHaveProperty("privateSyncMetadata"); }); }); diff --git a/server/src/__tests__/plugin-managed-agents.test.ts b/server/src/__tests__/plugin-managed-agents.test.ts index 34ba238c55..0b06a55126 100644 --- a/server/src/__tests__/plugin-managed-agents.test.ts +++ b/server/src/__tests__/plugin-managed-agents.test.ts @@ -7,6 +7,8 @@ import { afterAll, afterEach, beforeAll, describe, expect, it } from "vitest"; import { activityLog, agentConfigRevisions, + agentInstructionRevisions, + agentInstructionHeads, agents, approvals, companies, @@ -23,6 +25,9 @@ import { } from "./helpers/embedded-postgres.js"; import { buildHostServices } from "../services/plugin-host-services.js"; import { agentService } from "../services/agents.js"; +import { agentInstructionRevisionService } from "../services/agent-instruction-revisions.js"; +import { agentInstructionsService } from "../services/agent-instructions.js"; +import { pluginRegistryService } from "../services/plugin-registry.js"; const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport(); const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip; @@ -312,7 +317,7 @@ describeEmbeddedPostgres("plugin-managed agents", () => { it("materializes declared managed agent instructions with local folder paths", async () => { const previousHome = process.env.PAPERCLIP_HOME; const previousInstance = process.env.PAPERCLIP_INSTANCE_ID; - const tempHome = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-managed-agent-home-")); + const tempHome = await fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-managed-agent-home-"))); const wikiRoot = await fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-managed-agent-wiki-"))); process.env.PAPERCLIP_HOME = tempHome; process.env.PAPERCLIP_INSTANCE_ID = "test"; @@ -369,6 +374,39 @@ describeEmbeddedPostgres("plugin-managed agents", () => { expect(content).toContain("You are the LLM Wiki Maintainer."); expect(content).toContain(`Wiki root: \`${wikiRoot}\``); expect(content).toContain(`Wiki schema: \`${path.join(wikiRoot, "AGENTS.md")}\``); + + // A managed plugin reset must preserve the previous entry in canonical + // history instead of deleting a shared bundle or bypassing its write guard. + const edited = "# User-customized wiki instructions\n"; + await fs.writeFile(instructionsFilePath as string, edited); + const reset = await services.agents.managedReset({ companyId, agentKey: "wiki-maintainer" }); + expect(reset.status).toBe("reset"); + expect(await fs.readFile(instructionsFilePath as string, "utf8")).toBe(content); + const history = await db.select().from(agentInstructionRevisions) + .where(eq(agentInstructionRevisions.agentId, created.agentId!)); + expect(history).toHaveLength(0); + expect(await db.select().from(agentInstructionHeads) + .where(eq(agentInstructionHeads.agentId, created.agentId!))).toHaveLength(0); + const repeated = await services.agents.managedReset({ companyId, agentKey: "wiki-maintainer" }); + expect(repeated.status).toBe("reset"); + expect(await db.select().from(agentInstructionRevisions) + .where(eq(agentInstructionRevisions.agentId, created.agentId!))).toHaveLength(0); + const audit = await db.select().from(activityLog).where(eq(activityLog.action, "agent.files_updated")); + expect(audit).toHaveLength(1); + expect(audit.every((row) => row.actorType === "plugin" && row.actorId === pluginId)).toBe(true); + const bundles = agentInstructionsService(db); + await bundles.writeFile(repeated.agent!, "CUSTOM.md", "# Alternate configured entry\n"); + const switched = await bundles.updateBundle(repeated.agent!, { entryFile: "CUSTOM.md" }); + await agentService(db).update(created.agentId!, { adapterConfig: switched.adapterConfig }); + const resetEntry = await services.agents.managedReset({ companyId, agentKey: "wiki-maintainer" }); + expect(resetEntry.agent?.adapterConfig.instructionsEntryFile).toBe("AGENTS.md"); + expect(resetEntry.defaultDrift).toBeNull(); + expect(await fs.readFile(instructionsFilePath as string, "utf8")).toBe(content); + const preserved = await db.select().from(agentInstructionRevisions).where(eq(agentInstructionRevisions.agentId, created.agentId!)); + expect(preserved).toHaveLength(0); + expect(await fs.readFile(path.join(path.dirname(instructionsFilePath as string), "CUSTOM.md"), "utf8")).toBe("# Alternate configured entry\n"); + + } finally { if (previousHome === undefined) delete process.env.PAPERCLIP_HOME; else process.env.PAPERCLIP_HOME = previousHome; @@ -379,6 +417,60 @@ describeEmbeddedPostgres("plugin-managed agents", () => { } }); + it("fences plugin instruction reset by exact ownership, current capability and canonical CAS", async () => { + const previousHome = process.env.PAPERCLIP_HOME; + const tempHome = await fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), "plugin-reset-cas-"))); + process.env.PAPERCLIP_HOME = tempHome; + try { + const pluginManifest = manifest(); + pluginManifest.agents![0]!.instructions = { content: "# Default\n" }; + const { companyId, pluginId, services } = await seedCompanyAndPlugin({ manifest: pluginManifest }); + const created = await services.agents.managedReconcile({ companyId, agentKey: "wiki-maintainer" }); + const target = { companyId, agentId: created.agentId! }; + const actor = { type: "plugin" as const, pluginId, pluginKey: pluginManifest.id, agentKey: "wiki-maintainer" }; + const revisions = agentInstructionRevisionService(db); + const baselineRead = await revisions.readForPluginReset(target, actor); + const baseline = baselineRead.snapshot!; + const input = { ...target, entryFile: "AGENTS.md", baseRevisionId: baseline.revision.id, + configuredEntryFile: baselineRead.configuredEntryFile, content: "# Reset\n" }; + await expect(revisions.commitPluginReset(input, { ...actor, pluginId: randomUUID() })).rejects.toMatchObject({ status: 403 }); + await expect(revisions.commitPluginReset({ ...input, companyId: randomUUID() }, actor)).rejects.toMatchObject({ status: 404 }); + await expect(revisions.commitPluginReset(input, { ...actor, agentKey: "foreign-agent" })).rejects.toMatchObject({ status: 403 }); + await db.update(plugins).set({ manifestJson: { ...pluginManifest, capabilities: [] } }).where(eq(plugins.id, pluginId)); + await expect(revisions.commitPluginReset(input, actor)).rejects.toMatchObject({ status: 403 }); + await db.update(plugins).set({ manifestJson: pluginManifest }).where(eq(plugins.id, pluginId)); + const originalMetadata = created.agent!.metadata; + await db.update(agents).set({ metadata: {} }).where(eq(agents.id, target.agentId)); + await expect(revisions.commitPluginReset(input, actor)).rejects.toMatchObject({ status: 403 }); + await db.update(agents).set({ metadata: originalMetadata }).where(eq(agents.id, target.agentId)); + const [binding] = await db.select().from(pluginManagedResources).where(eq(pluginManagedResources.resourceId, target.agentId)); + await db.update(pluginManagedResources).set({ resourceId: randomUUID() }).where(eq(pluginManagedResources.id, binding.id)); + await expect(revisions.commitPluginReset(input, actor)).rejects.toMatchObject({ status: 403 }); + await db.update(pluginManagedResources).set({ resourceId: target.agentId }).where(eq(pluginManagedResources.id, binding.id)); + await expect(revisions.commitPluginReset({ ...input, entryFile: "OTHER.md" }, actor)).rejects.toMatchObject({ status: 403 }); + const bundles = agentInstructionsService(db); + await bundles.writeFile(created.agent!, "OTHER.md", "# Other entry\n"); + const switched = await bundles.updateBundle(created.agent!, { entryFile: "OTHER.md" }); + await agentService(db).update(target.agentId, { adapterConfig: switched.adapterConfig }); + await expect(revisions.commitPluginReset(input, actor)).rejects.toMatchObject({ status: 409, details: { code: "INSTRUCTION_ENTRY_CHANGED" } }); + await agentService(db).update(target.agentId, { adapterConfig: created.agent!.adapterConfig }); + const results = await Promise.allSettled([ + revisions.commitPluginReset(input, actor), + revisions.commitPluginReset({ ...input, content: "# Concurrent reset\n" }, actor), + ]); + expect(results.filter((result) => result.status === "fulfilled")).toHaveLength(1); + const loser = results.find((result) => result.status === "rejected") as PromiseRejectedResult; + expect(loser.reason).toMatchObject({ status: 409, details: { code: "INSTRUCTION_REVISION_CONFLICT" } }); + const winner = results.find((result) => result.status === "fulfilled") as PromiseFulfilledResult>>; + expect(await fs.readFile(created.agent!.adapterConfig.instructionsFilePath as string, "utf8")).toBe(winner.value.content); + const history = await db.select().from(agentInstructionRevisions).where(eq(agentInstructionRevisions.agentId, target.agentId)); + expect(history).toHaveLength(0); + } finally { + if (previousHome === undefined) delete process.env.PAPERCLIP_HOME; else process.env.PAPERCLIP_HOME = previousHome; + await fs.rm(tempHome, { recursive: true, force: true }); + } + }); + it("repairs a missing binding by relinking a same-company managed agent marker", async () => { const { companyId, pluginId, pluginManifest, services } = await seedCompanyAndPlugin(); const agentId = randomUUID(); @@ -410,6 +502,76 @@ describeEmbeddedPostgres("plugin-managed agents", () => { expect(binding?.data).toMatchObject({ agentId }); }); + it("preserves current-file authority when relinking after a hard uninstall and reinstall", async () => { + const previousHome = process.env.PAPERCLIP_HOME; + const tempHome = await fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), "plugin-reinstall-reset-"))); + process.env.PAPERCLIP_HOME = tempHome; + try { + const pluginManifest = manifest(); + pluginManifest.agents![0]!.instructions = { content: "# Original stock\n" }; + const { companyId, pluginId, services } = await seedCompanyAndPlugin({ manifest: pluginManifest }); + const created = await services.agents.managedReconcile({ companyId, agentKey: "wiki-maintainer" }); + const target = { companyId, agentId: created.agentId! }; + const revisions = agentInstructionRevisionService(db); + const original = await revisions.readForPluginReset(target, { + type: "plugin", pluginId, pluginKey: pluginManifest.id, agentKey: "wiki-maintainer", + }); + const registry = pluginRegistryService(db); + await registry.uninstall(pluginId, true); + expect(await db.select().from(pluginManagedResources)).toHaveLength(0); + expect(await db.select().from(pluginEntities)).toHaveLength(0); + const nextManifest = structuredClone(pluginManifest); + nextManifest.agents![0]!.instructions = { content: "# Reinstalled stock\n" }; + const installed = await registry.install({ packageName: "@paperclipai/plugin-managed-agents-test" }, nextManifest); + expect(installed!.id).not.toBe(pluginId); + await registry.updateStatus(installed!.id, { status: "ready" }); + const reinstalledServices = buildHostServices(db, installed!.id, nextManifest.id, createEventBusStub(), undefined, { + manifest: nextManifest, + }); + const relinked = await reinstalledServices.agents.managedReconcile({ companyId, agentKey: "wiki-maintainer" }); + expect(relinked.status).toBe("relinked"); + expect(relinked.agentId).toBe(created.agentId); + const reset = await reinstalledServices.agents.managedReset({ companyId, agentKey: "wiki-maintainer" }); + expect(reset.agentId).toBe(created.agentId); + expect(reset.agent!.metadata).toMatchObject({ paperclipManagedResource: { pluginId: installed!.id }, pluginManagedAgent: { pluginId: installed!.id } }); + expect(await fs.readFile(reset.agent!.adapterConfig.instructionsFilePath as string, "utf8")).toBe("# Reinstalled stock\n"); + const history = await db.select().from(agentInstructionRevisions).where(eq(agentInstructionRevisions.agentId, created.agentId!)); + expect(history).toHaveLength(0); + } finally { + if (previousHome === undefined) delete process.env.PAPERCLIP_HOME; else process.env.PAPERCLIP_HOME = previousHome; + await fs.rm(tempHome, { recursive: true, force: true }); + } + }); + + it.each(["not-ready", "capability-removed", "declaration-removed", "other-plugin-owner"])( + "does not relink a managed agent when its authority is %s", async (reason) => { + const { companyId, pluginId, pluginManifest, services } = await seedCompanyAndPlugin(); + const created = await services.agents.managedReconcile({ companyId, agentKey: "wiki-maintainer" }); + await db.delete(pluginEntities); + await db.delete(pluginManagedResources); + if (reason === "not-ready") { + await db.update(plugins).set({ status: "disabled" }).where(eq(plugins.id, pluginId)); + } else if (reason === "capability-removed") { + await db.update(plugins).set({ manifestJson: { ...pluginManifest, capabilities: [] } }).where(eq(plugins.id, pluginId)); + } else if (reason === "declaration-removed") { + await db.update(plugins).set({ manifestJson: { ...pluginManifest, agents: [] } }).where(eq(plugins.id, pluginId)); + } else { + const otherId = randomUUID(); + await db.insert(plugins).values({ id: otherId, pluginKey: "paperclip.other-owner", packageName: "other", + version: "0.1.0", apiVersion: 1, categories: [], manifestJson: { ...pluginManifest, id: "paperclip.other-owner" }, status: "ready", installOrder: 2 }); + await db.update(agents).set({ metadata: { ...created.agent!.metadata, + paperclipManagedResource: { pluginId: otherId, pluginKey: pluginManifest.id, resourceKind: "agent", resourceKey: "wiki-maintainer" }, + } }).where(eq(agents.id, created.agentId!)); + } + const [before] = await db.select().from(agents).where(eq(agents.id, created.agentId!)); + await expect(services.agents.managedReconcile({ companyId, agentKey: "wiki-maintainer" })).rejects.toMatchObject({ status: 403 }); + expect(await db.select().from(pluginEntities)).toHaveLength(0); + expect(await db.select().from(pluginManagedResources)).toHaveLength(0); + const [after] = await db.select().from(agents).where(eq(agents.id, created.agentId!)); + expect(after.metadata).toEqual(before.metadata); + }, + ); + it("respects board approval policy for new managed agents", async () => { const { companyId, services } = await seedCompanyAndPlugin({ requireApproval: true }); diff --git a/server/src/__tests__/rerun-survival.integration.test.ts b/server/src/__tests__/rerun-survival.integration.test.ts index 1708febead..aa3d5cb48c 100644 --- a/server/src/__tests__/rerun-survival.integration.test.ts +++ b/server/src/__tests__/rerun-survival.integration.test.ts @@ -9,7 +9,7 @@ import { writeEnvFileAtomicallyIfChanged, } from "@paperclipai/shared/env-file"; import { resourceStatus, stockHash } from "../services/managed-resource-drift.js"; -import { agentInstructionsService } from "../services/agent-instructions.js"; +import { agentInstructionsService, resolveManagedInstructionsRoot } from "../services/agent-instructions.js"; /** * Cross-cutting setup/sync rerun survival test. @@ -23,7 +23,7 @@ import { agentInstructionsService } from "../services/agent-instructions.js"; * - .env -> updateEnvFileContents + writeEnvFileAtomicallyIfChanged * - managed sandbox env -> resourceStatus/stockHash drift gate (the `shouldWrite` * classifier the boot reconciler and built-in sync share) - * - managed instructions -> agentInstructionsService materialize + the drift gate that + * - managed instructions -> agentInstructionsService export + the drift gate that * makes built-in/plugin/portability rebuilds skip operator edits * * The DB-backed end-to-end variants of the sandbox-row, skills-assignment, and @@ -31,7 +31,7 @@ import { agentInstructionsService } from "../services/agent-instructions.js"; * server/src/__tests__/company-skills-service.test.ts (skills add/remove/replace) * server/src/services/managed-environments.test.ts (sandbox row boot reconcile) * server/src/__tests__/environment-service.test.ts (sandbox skip + stock update) - * server/src/__tests__/agent-instructions-service.test.ts (managed instruction drift) + * server/src/__tests__/agent-instruction-revisions.test.ts (guarded bundle materialization) */ const cleanupDirs = new Set(); @@ -147,7 +147,7 @@ describe("setup/sync rerun survival — cross-cutting", () => { ).toBe("stock_current"); }); - it("keeps operator edits and additions in a managed instructions tree across a re-materialize", async () => { + it("keeps operator edits and additions when managed instructions reconciliation detects drift", async () => { const home = await tmp("pap16587-instr-home-"); process.env.PAPERCLIP_HOME = home; process.env.PAPERCLIP_INSTANCE_ID = "test-instance"; @@ -156,18 +156,25 @@ describe("setup/sync rerun survival — cross-cutting", () => { const agent = { id: "agent-1", companyId: "company-1", name: "Agent 1", adapterConfig: {} }; const stockFiles = { "AGENTS.md": "# stock\n", "docs/TOOLS.md": "## stock tools\n" }; - const first = await svc.materializeManagedBundle(agent, stockFiles, { entryFile: "AGENTS.md" }); - const root = first.bundle.managedRootPath!; + // Initial files are fixture setup. Bundle materialization now requires a + // real database so that it cannot bypass a canonical instruction revision. + const root = resolveManagedInstructionsRoot(agent); + for (const [name, content] of Object.entries(stockFiles)) { + await fs.mkdir(path.dirname(path.join(root, name)), { recursive: true }); + await fs.writeFile(path.join(root, name), content, "utf8"); + } + const managedAgent = { ...agent, adapterConfig: { + instructionsBundleMode: "managed", instructionsRootPath: root, + instructionsEntryFile: "AGENTS.md", instructionsFilePath: path.join(root, "AGENTS.md"), + } }; // Operator hand-edits a managed file and adds a brand-new operator-only file. await fs.writeFile(path.join(root, "AGENTS.md"), "# stock\n\noperator note\n", "utf8"); await fs.writeFile(path.join(root, "OPERATOR.md"), "operator-added\n", "utf8"); // The drift gate the built-in/plugin/portability rebuilds share: classify the tree. - const currentFiles = { - "AGENTS.md": await fs.readFile(path.join(root, "AGENTS.md"), "utf8"), - "docs/TOOLS.md": await fs.readFile(path.join(root, "docs", "TOOLS.md"), "utf8"), - }; + const currentFiles = (await svc.exportFiles(managedAgent)).files; + expect(currentFiles["OPERATOR.md"]).toBe("operator-added\n"); const status = resourceStatus({ resourceId: agent.id, currentHash: stockHash(currentFiles), diff --git a/server/src/routes/agents.ts b/server/src/routes/agents.ts index 2efcc76457..756d0c7c37 100644 --- a/server/src/routes/agents.ts +++ b/server/src/routes/agents.ts @@ -1,3 +1,5 @@ +import { agentFileStore, agentFileTokenFromHash } from "../services/agent-file-store.js"; +import { pipeline } from "node:stream/promises"; import { resolveAgentAppearance, agentAvatarUrl } from "@paperclipai/shared"; import { listOpenRouterModels } from "../services/openrouter-models.js"; import { prepareManagedAiRuntime, assertManagedAiProjectAuth, stripAiAuthBindings } from "../services/ai-connection-runtime.js"; @@ -38,6 +40,8 @@ import { type AgentSkillSnapshot, type InstanceSchedulerHeartbeatAgent, upsertAgentInstructionsFileSchema, + restoreAgentInstructionSchema, + resolveAgentInstructionCandidateSchema, updateAgentInstructionsBundleSchema, updateAgentPermissionsSchema, updateAgentInstructionsPathSchema, @@ -63,7 +67,11 @@ import { import { trackAgentCreated } from "@paperclipai/shared/telemetry"; import { validate } from "../middleware/validate.js"; import { inheritNativeRunnerAdapterConfig } from "../services/native-runtime/native-agent-runtime-inheritance.js"; -import { agentInstructionsBundleMode } from "../services/agent-instructions.js"; +import { agentInstructionRevisionService } from "../services/agent-instruction-revisions.js"; +import { agentInstructionWorkingCopyService } from "../services/agent-instruction-working-copies.js"; +import { authorizeInstructionRead } from "../services/agent-instruction-authorization.js"; +import { instructionPath } from "../services/agent-instruction-files.js"; +import { agentInstructionsBundleMode, deriveBundleState } from "../services/agent-instructions.js"; import { agentService, agentInstructionsService, @@ -724,7 +732,17 @@ export function agentRoutes( const recovery = recoveryService(db, { enqueueWakeup: heartbeat.wakeup }); const issueApprovalsSvc = issueApprovalService(db); const secretsSvc = secretService(db); - const instructions = agentInstructionsService(); + const instructions = agentInstructionsService(db); + const agentFiles = agentFileStore(db); + const instructionRevisions = agentInstructionRevisionService(db); + const instructionWorkingCopies = agentInstructionWorkingCopyService(db); + function instructionFileDetail(snapshot: import("@paperclipai/shared").AgentInstructionSnapshot, + receipt?: import("@paperclipai/shared").AgentInstructionCommitReceipt) { + const path = snapshot.revision.entryFile; + return { path, content: snapshot.content, contentHash: snapshot.revision.contentHash, size: snapshot.revision.byteLength, revision: snapshot.revision, receipt, + language: path.toLowerCase().endsWith(".md") ? "markdown" : "text", markdown: path.toLowerCase().endsWith(".md"), + isEntryFile: true, editable: true, deprecated: false, virtual: false }; + } const companySkills = companySkillService(db); const workspaceOperations = workspaceOperationService(db); const instanceSettings = instanceSettingsService(db); @@ -5067,8 +5085,24 @@ export function agentRoutes( const id = req.params.id as string; const existing = await getAccessibleResource(req, res, svc.getById(id), "Agent not found"); if (!existing) return; - await assertCanReadAgent(req, existing); assertExternalInstructionsAdmin(req, existing); + if (agentInstructionsBundleMode(existing) === "external") { + await assertCanReadAgent(req, existing); + } else { + await authorizeInstructionRead(db, req.actor, { companyId: existing.companyId, id: existing.id }); + } + if (agentInstructionsBundleMode(existing) !== "external") { + const target = { companyId: existing.companyId, agentId: existing.id }; + const current = await instructionRevisions.readCurrent(target, req.actor); + if (current) { + try { await instructionRevisions.materializeCurrent(target); } + catch (error) { + const bundle = await instructions.getBundle(existing); + bundle.warnings.push(`Saved instruction revision needs materialization: ${error instanceof Error ? error.message : String(error)}`); + res.json(bundle); return; + } + } + } res.json(await instructions.getBundle(existing)); }); @@ -5124,8 +5158,12 @@ export function agentRoutes( const id = req.params.id as string; const existing = await getAccessibleResource(req, res, svc.getById(id), "Agent not found"); if (!existing) return; - await assertCanReadAgent(req, existing); assertExternalInstructionsAdmin(req, existing); + if (agentInstructionsBundleMode(existing) === "external") { + await assertCanReadAgent(req, existing); + } else { + await authorizeInstructionRead(db, req.actor, { companyId: existing.companyId, id: existing.id }); + } const relativePath = typeof req.query.path === "string" ? req.query.path : ""; if (!relativePath.trim()) { @@ -5133,6 +5171,19 @@ export function agentRoutes( return; } + if (req.query.download === "true" && agentInstructionsBundleMode(existing) === "managed") { + const download = await agentFiles.download(existing.companyId, existing.id, relativePath, req.actor); + if (download === null) throw notFound("Agent file not found"); + res.setHeader("Content-Type", "application/octet-stream"); + res.setHeader("X-Content-Type-Options", "nosniff"); + res.attachment(relativePath.split("/").at(-1)!); + res.setHeader("Content-Length", download.size); + await pipeline(download.stream, res); return; + } + if (agentInstructionsBundleMode(existing) !== "external" && instructionPath(relativePath) === deriveBundleState(existing).entryFile) { + const snapshot = await instructionRevisions.readCurrent({ companyId: existing.companyId, agentId: existing.id }, req.actor); + if (snapshot) { res.json(instructionFileDetail(snapshot)); return; } + } res.json(await instructions.readFile(existing, relativePath)); }); @@ -5140,9 +5191,42 @@ export function agentRoutes( const id = req.params.id as string; const existing = await getAccessibleResource(req, res, svc.getById(id), "Agent not found"); if (!existing) return; + const entryFile = deriveBundleState(existing).entryFile; + if (instructionPath(req.body.path) === entryFile) { + assertExternalInstructionsAdmin(req, existing); + if (req.body.baseHash !== undefined) req.body.baseRevisionId = req.body.baseHash === null ? null : agentFileTokenFromHash(req.body.baseHash); + if (req.body.baseRevisionId === undefined) throw unprocessable("Read the entry and supply baseRevisionId (null for a new entry)", { code: "INSTRUCTION_BASE_REQUIRED" }); + // Clearing legacy prompt configuration remains a protected config change. + if (req.body.clearLegacyPromptTemplate) await assertCanManageInstructionsPath(req, existing); + const receipt = await instructionRevisions.commit({ companyId: existing.companyId, agentId: existing.id, + entryFile, content: req.body.content, baseRevisionId: req.body.baseRevisionId, + source: req.actor.type === "board" ? "board" : "api" }, req.actor); + if (req.actor.type === "agent" && req.actor.runId) { + await instructionWorkingCopies.acknowledgeExplicitSave({ companyId: existing.companyId, agentId: existing.id, + runId: req.actor.runId, entryFile: receipt.revision.entryFile, revisionId: receipt.revision.id, + contentHash: receipt.revision.contentHash }).catch(() => undefined); + } + if (req.body.clearLegacyPromptTemplate) { + const fresh = await svc.getById(existing.id); + if (fresh) { + const adapterConfig = { ...asRecord(fresh.adapterConfig) }; + delete adapterConfig.promptTemplate; + delete adapterConfig.bootstrapPromptTemplate; + await svc.update(existing.id, { adapterConfig }); + } + } + res.json(instructionFileDetail(receipt, receipt)); + return; + } await assertCanManageInstructionsPath(req, existing); assertExternalInstructionsAdmin(req, existing); + if (agentInstructionsBundleMode(existing) === "managed" && req.body.path !== "promptTemplate.legacy.md") { + if (req.body.baseHash === undefined) throw unprocessable("Read the file and supply baseHash (null for a new file)"); + await agentFiles.write({ companyId: existing.companyId, agentId: existing.id, path: req.body.path, + bytes: Buffer.from(req.body.content, "utf8"), baseHash: req.body.baseHash }, req.actor); + res.json(await instructions.readFile(existing, req.body.path)); return; + } const actor = getActorInfo(req); const result = await instructions.writeFile(existing, req.body.path, req.body.content, { clearLegacyPromptTemplate: req.body.clearLegacyPromptTemplate, @@ -5184,6 +5268,66 @@ export function agentRoutes( res.json(result.file); }); + router.get("/agents/:id/instructions-bundle/candidates", async (req, res) => { + const existing = await getAccessibleResource(req, res, svc.getById(req.params.id as string), "Agent not found"); + if (!existing) return; + assertExternalInstructionsAdmin(req, existing); + res.json(await instructionWorkingCopies.list(existing.companyId, existing.id, req.actor)); + }); + + router.post("/agents/:id/instructions-bundle/candidates/:runId/resolve", validate(resolveAgentInstructionCandidateSchema), async (req, res) => { + const existing = await getAccessibleResource(req, res, svc.getById(req.params.id as string), "Agent not found"); + if (!existing) return; + assertExternalInstructionsAdmin(req, existing); + const runId = req.params.runId as string; + if (!isUuidLike(runId)) throw unprocessable("Invalid instruction candidate run id"); + const receipt = await instructionWorkingCopies.resolve({ companyId: existing.companyId, agentId: existing.id, + runId, baseRevisionId: req.body.baseRevisionId, content: req.body.content }, req.actor); + res.json(instructionFileDetail(receipt, receipt)); + }); + + router.get("/agents/:id/instructions-bundle/history", async (req, res) => { + const existing = await getAccessibleResource(req, res, svc.getById(req.params.id as string), "Agent not found"); + if (!existing) return; + assertExternalInstructionsAdmin(req, existing); + const entryFile = typeof req.query.path === "string" ? req.query.path : deriveBundleState(existing).entryFile; + const cursor = typeof req.query.cursor === "string" ? req.query.cursor : undefined; + if (cursor && !isUuidLike(cursor)) throw unprocessable("Invalid history cursor"); + res.json(await instructionRevisions.history({ companyId: existing.companyId, agentId: existing.id, entryFile, cursor }, req.actor)); + }); + router.get("/agents/:id/instructions-bundle/revision/:revisionId", async (req, res) => { + const existing = await getAccessibleResource(req, res, svc.getById(req.params.id as string), "Agent not found"); + if (!existing) return; + assertExternalInstructionsAdmin(req, existing); + const revisionId = req.params.revisionId as string; + if (!isUuidLike(revisionId)) throw unprocessable("Invalid instruction revision id"); + const entryFile = typeof req.query.path === "string" ? req.query.path : deriveBundleState(existing).entryFile; + res.json(await instructionRevisions.readRevision({ companyId: existing.companyId, agentId: existing.id, entryFile, revisionId }, req.actor)); + }); + router.get("/agents/:id/instructions-bundle/diff", async (req, res) => { + const existing = await getAccessibleResource(req, res, svc.getById(req.params.id as string), "Agent not found"); + if (!existing) return; + assertExternalInstructionsAdmin(req, existing); + const fromRevisionId = typeof req.query.from === "string" ? req.query.from : ""; + const toRevisionId = typeof req.query.to === "string" ? req.query.to : ""; + if (!isUuidLike(fromRevisionId) || !isUuidLike(toRevisionId)) throw unprocessable("Provide valid from and to revision ids"); + const entryFile = typeof req.query.path === "string" ? req.query.path : deriveBundleState(existing).entryFile; + res.json(await instructionRevisions.diff({ companyId: existing.companyId, agentId: existing.id, entryFile, fromRevisionId, toRevisionId }, req.actor)); + }); + router.post("/agents/:id/instructions-bundle/restore", validate(restoreAgentInstructionSchema), async (req, res) => { + const existing = await getAccessibleResource(req, res, svc.getById(req.params.id as string), "Agent not found"); + if (!existing) return; + assertExternalInstructionsAdmin(req, existing); + const receipt = await instructionRevisions.restore({ companyId: existing.companyId, agentId: existing.id, + entryFile: req.body.path, baseRevisionId: req.body.baseRevisionId, revisionId: req.body.revisionId }, req.actor); + if (req.actor.type === "agent" && req.actor.runId) { + await instructionWorkingCopies.acknowledgeExplicitSave({ companyId: existing.companyId, agentId: existing.id, + runId: req.actor.runId, entryFile: receipt.revision.entryFile, revisionId: receipt.revision.id, + contentHash: receipt.revision.contentHash }).catch(() => undefined); + } + res.json(instructionFileDetail(receipt, receipt)); + }); + router.delete("/agents/:id/instructions-bundle/file", async (req, res) => { const id = req.params.id as string; const existing = await getAccessibleResource(req, res, svc.getById(id), "Agent not found"); @@ -5197,6 +5341,12 @@ export function agentRoutes( return; } + if (agentInstructionsBundleMode(existing) === "managed") { + const baseHash = typeof req.query.baseHash === "string" && /^[a-f0-9]{64}$/.test(req.query.baseHash) ? req.query.baseHash : null; + if (baseHash === null) throw unprocessable("Read the file and supply baseHash before deleting it"); + await agentFiles.write({ companyId: existing.companyId, agentId: existing.id, path: relativePath, bytes: null, baseHash }, req.actor); + res.json(await instructions.getBundle(existing)); return; + } const actor = getActorInfo(req); const result = await instructions.deleteFile(existing, relativePath); await logActivity(db, { diff --git a/server/src/routes/built-in-agents.ts b/server/src/routes/built-in-agents.ts index 9c16ba6f53..c881ae9c93 100644 --- a/server/src/routes/built-in-agents.ts +++ b/server/src/routes/built-in-agents.ts @@ -217,7 +217,7 @@ export function builtInAgentRoutes(db: Db) { const key = req.params.key as string; await assertBuiltInAgentsEnabled(); await assertCanProvisionBuiltInAgents(req, companyId); - const state = await svc.reset(companyId, key, req.body); + const state = await svc.reset(companyId, key, req.body, req.actor); await logBuiltInAgentMutation(req, { companyId, action: "built_in_agent.reset", diff --git a/server/src/routes/openapi.ts b/server/src/routes/openapi.ts index 9cd65e6f6d..38b876197d 100644 --- a/server/src/routes/openapi.ts +++ b/server/src/routes/openapi.ts @@ -27,6 +27,8 @@ import { updateAgentInstructionsPathSchema, updateAgentInstructionsBundleSchema, upsertAgentInstructionsFileSchema, + restoreAgentInstructionSchema, + resolveAgentInstructionCandidateSchema, createAgentKeySchema, builtInAgentEmptyMutationSchema, builtInAgentProvisionSchema, @@ -3455,8 +3457,8 @@ registry.registerPath({ method: "get", path: "/api/agents/{id}/instructions-bundle/file", tags: ["agents"], - summary: "Get agent instructions file", - request: { params: z.object({ id: z.string() }) }, + summary: "Get agent file content or download its original bytes", + request: { params: z.object({ id: z.string() }), query: z.object({ path: z.string(), download: z.enum(["true", "false"]).optional() }) }, responses: { 200: r.ok(), 401: r.unauthorized, 404: r.notFound }, }); @@ -3472,12 +3474,46 @@ registry.registerPath({ responses: { 200: r.ok(), 400: r.badRequest, 401: r.unauthorized }, }); +for (const operation of [ + { suffix: "history", summary: "List immutable instruction revisions", query: z.object({ path: z.string().optional(), cursor: z.string().uuid().optional() }) }, + { suffix: "revision/{revisionId}", summary: "Read exact instruction content at a revision", query: z.object({ path: z.string().optional() }) }, + { suffix: "diff", summary: "Compare instruction revisions (exact common prefix, removed, added, suffix)", query: z.object({ path: z.string().optional(), from: z.string().uuid(), to: z.string().uuid() }) }, +]) { + registry.registerPath({ method: "get", path: `/api/agents/{id}/instructions-bundle/${operation.suffix}`, tags: ["agents"], summary: operation.summary, + request: { params: operation.suffix.includes("revisionId") ? z.object({ id: z.string(), revisionId: z.string().uuid() }) : z.object({ id: z.string() }), query: operation.query }, + responses: { 200: r.ok(), 401: r.unauthorized, 403: r.forbidden, 404: r.notFound } }); +} +registry.registerPath({ method: "post", path: "/api/agents/{id}/instructions-bundle/restore", tags: ["agents"], summary: "Restore a pre-upgrade instruction snapshot into current files with compare-and-swap", + request: { params: z.object({ id: z.string() }), body: jsonBody(restoreAgentInstructionSchema) }, + responses: { 200: r.ok(), 401: r.unauthorized, 403: r.forbidden, 404: r.notFound, 409: r.conflict } }); + +registry.registerPath({ + method: "get", + path: "/api/agents/{id}/instructions-bundle/candidates", + tags: ["agents"], + summary: "List preserved instruction edits and collection diagnostics", + request: { params: z.object({ id: z.string() }) }, + responses: { 200: r.ok(), 401: r.unauthorized, 403: r.forbidden, 404: r.notFound }, +}); + +registry.registerPath({ + method: "post", + path: "/api/agents/{id}/instructions-bundle/candidates/{runId}/resolve", + tags: ["agents"], + summary: "Explicitly save a preserved instruction edit with compare-and-swap", + request: { + params: z.object({ id: z.string(), runId: z.string().uuid() }), + body: jsonBody(resolveAgentInstructionCandidateSchema), + }, + responses: { 200: r.ok(), 401: r.unauthorized, 403: r.forbidden, 404: r.notFound, 409: r.conflict, 422: r.unprocessable }, +}); + registry.registerPath({ method: "delete", path: "/api/agents/{id}/instructions-bundle/file", tags: ["agents"], - summary: "Delete agent instructions file", - request: { params: z.object({ id: z.string() }) }, + summary: "Delete agent file with compare-and-swap for managed storage", + request: { params: z.object({ id: z.string() }), query: z.object({ path: z.string(), baseHash: z.string().regex(/^[a-f0-9]{64}$/).optional() }) }, responses: { 200: r.ok(), 401: r.unauthorized }, }); diff --git a/server/src/services/agent-directory-working-copies.ts b/server/src/services/agent-directory-working-copies.ts new file mode 100644 index 0000000000..ab3dde2e11 --- /dev/null +++ b/server/src/services/agent-directory-working-copies.ts @@ -0,0 +1,258 @@ +import fs from "node:fs/promises"; +import path from "node:path"; +import { and, eq } from "drizzle-orm"; +import { agents, environmentLeases, environments, agentInstructionWorkingCopies as copies, type Db } from "@paperclipai/db"; +import { syncDirectoryToSsh, restoreWorkspaceFromSshExecution } from "@paperclipai/adapter-utils/ssh"; +import { prepareAdapterExecutionTargetRuntime, runAdapterExecutionTargetShellCommand, type AdapterExecutionTarget, type PreparedAdapterExecutionTargetRuntime } from "@paperclipai/adapter-utils/execution-target"; +import { withDirectoryMergeLock, directorySnapshotSha256, parseDirectorySnapshot, serializeDirectorySnapshot } from "@paperclipai/adapter-utils/workspace-restore-merge"; +import { AGENT_FILES_CONTRACT, AgentFileLimitError, agentFileStore, agentStorageWarning, inspectAgentDirectory } from "./agent-file-store.js"; +import { agentInstructionsBundleMode, deriveBundleState, resolveManagedInstructionsRoot } from "./agent-instructions.js"; +import { instructionGitExcludeProgram } from "./agent-instruction-files.js"; +import { resolveInstructionActor } from "./agent-instruction-authorization.js"; +import { HttpError, conflict, notFound } from "../errors.js"; +import type { AuthorizationActor } from "./authorization.js"; +import type { EnvironmentRuntimeService } from "./environment-runtime.js"; +import type { Environment, EnvironmentLease } from "@paperclipai/shared"; +import { hasRemoteTerminationReceipt } from "./remote-execution-termination.js"; + +type Copy = typeof copies.$inferSelect; +const completed = new Set(["saved", "unchanged", "resolved", "unavailable"]); +const transports = new Map(); +const key = (row: Pick) => `${row.companyId}:${row.runId}`; +export function isAgentDirectoryCopy(row: Pick | null): boolean { + return row?.receipt?.schema === AGENT_FILES_CONTRACT; +} +function baseline(row: Copy) { + const snapshot = parseDirectorySnapshot(row.receipt?.baseline); + if (!snapshot || directorySnapshotSha256(snapshot) !== row.baseHash) throw new Error("Agent directory baseline is invalid"); + return snapshot; +} +function actor(row: Copy): AuthorizationActor { + return { type: "agent", companyId: row.companyId, agentId: row.agentId, runId: row.runId, onBehalfOfUserId: row.responsibleUserId }; +} +export function agentDirectoryWorkingCopyService(db: Db, get: (companyId: string, runId: string) => Promise, patch: (row: Copy, values: Partial) => Promise, environmentRuntime?: EnvironmentRuntimeService) { + const store = agentFileStore(db); + async function transport(row: Copy, target: AdapterExecutionTarget, recovering: boolean) { + if (target.kind !== "remote" || row.location !== `remote:${target.environmentId ?? ""}`) throw new Error("Agent directory environment changed"); + if (recovering && target.transport === "ssh") throw new Error("The original SSH collector is unavailable"); + if (target.transport === "ssh") { + // Reuse SSH's plain directory transfer without its task-workspace suffix + // or Git-history discovery. The registered root is the exact writable root. + await syncDirectoryToSsh({ spec: target.spec, localDir: row.localRoot, remoteDir: row.executionRoot, exclude: [".paperclip-runtime"] }); + return { target, workspaceRemoteDir: row.executionRoot, runtimeRootDir: null, + assetDirs: {}, additionalSourceDirs: {}, additionalSourceFailures: [], workspaceSyncSnapshot: null, + restoreWorkspace: () => restoreWorkspaceFromSshExecution({ spec: target.spec, localDir: row.localRoot, + remoteDir: row.executionRoot, baselineSnapshot: { ...baseline(row), exclude: [".paperclip-runtime"] }, restoreGitHistory: false }) }; + } + return prepareAdapterExecutionTargetRuntime({ target, runId: row.runId, adapterKey: "agent-files", + workspaceLocalDir: row.localRoot, workspaceRemoteDir: row.executionRoot, + syncWorkspace: true, workspaceInboundMode: recovering ? "adopt_remote" : undefined, + workspaceBaseline: baseline(row), workspaceGitSnapshot: null, workspaceFileMode: "all", + workspaceExclude: [".paperclip-runtime", ".paperclip-runtime/**"] }); + } + async function prepare(input: { companyId: string; agentId: string; runId: string; target?: AdapterExecutionTarget | null; cwd: string }) { + const [agent] = await db.select().from(agents).where(and(eq(agents.id, input.agentId), eq(agents.companyId, input.companyId))); + if (!agent) throw notFound("Agent not found"); + if (agentInstructionsBundleMode(agent) !== "managed") return null; + const bound = await resolveInstructionActor(db, { type: "agent", companyId: input.companyId, agentId: input.agentId, runId: input.runId }); + const root = resolveManagedInstructionsRoot(agent); + const localRoot = path.join(path.dirname(root), "file-sync", "runs", input.runId, "live"); + // Local copies live outside the task cwd. Remote copies use the reserved, + // excluded runtime area inside the provider's confined workspace. They are + // synchronized independently; provider HOME is never reinterpreted. + const executionRoot = input.target?.kind === "remote" + ? path.posix.join(input.target.remoteCwd, ".paperclip-runtime", "agent-files", input.agentId, input.runId) + : localRoot; + const location = input.target?.kind === "remote" ? `remote:${input.target.environmentId ?? ""}` : "local"; + let row = await get(input.companyId, input.runId); + if (row && (row.localRoot !== localRoot || row.executionRoot !== executionRoot || row.agentId !== input.agentId || row.location !== location)) throw conflict("Agent directory belongs to a different execution environment"); + if (row && !completed.has(row.state) && row.state !== "preparing") { + if (input.target?.kind === "remote" && !transports.has(key(row))) transports.set(key(row), await transport(row, input.target, true)); + return row; + } + // Register ownership before creating any bytes, so a crash during the copy + // leaves a recoverable preparing receipt instead of an orphan directory. + const preparing = { entryFile: deriveBundleState(agent).entryFile, baseRevisionId: null, baseHash: "preparing", + localRoot, executionRoot, location, state: "preparing", candidateBase64: null, candidateHash: null, + receipt: { schema: AGENT_FILES_CONTRACT, ...(input.target?.kind === "remote" + ? { cleanup: { leaseId: input.target.leaseId ?? null, remoteCwd: input.target.remoteCwd } } : {}) }, processStoppedAt: null, attempts: 0, + nextAttemptAt: null, errorCode: null, errorMessage: null }; + if (row) row = await patch(row, preparing); + else { + await db.insert(copies).values({ runId: input.runId, companyId: input.companyId, agentId: input.agentId, responsibleUserId: bound.onBehalfOfUserId!, ...preparing }); + row = (await get(input.companyId, input.runId))!; + } + const { snapshot, storageWarning } = await store.locked(input.companyId, input.agentId, bound, false, async (_tx, _agent, canonical) => { + // Storage quotas govern saves, never admission to a future run. Restore + // existing bytes so the agent can work normally and remove excess files. + // Path/link validation remains mandatory even for an over-quota folder. + const inspected = await inspectAgentDirectory(canonical, false); + // A stopped lifecycle has already accounted for all its bytes. No live + // or pending candidate ever enters this replacement path. + await fs.rm(localRoot, { recursive: true, force: true }); + await fs.mkdir(path.dirname(localRoot), { recursive: true, mode: 0o700 }); + try { + await fs.cp(canonical, localRoot, { recursive: true, preserveTimestamps: true }); + return inspected; + } catch (error) { + await fs.rm(path.dirname(localRoot), { recursive: true, force: true }); + throw error; + } + }).catch(async error => { + await fs.rm(path.dirname(localRoot), { recursive: true, force: true }); + throw error; + }); + const values = { entryFile: deriveBundleState(agent).entryFile, baseRevisionId: null, baseHash: directorySnapshotSha256(snapshot), + localRoot, executionRoot, location, state: "preparing", candidateBase64: null, candidateHash: null, + receipt: { ...preparing.receipt, baseline: serializeDirectorySnapshot(snapshot), storageWarning }, + processStoppedAt: null, attempts: 0, nextAttemptAt: null, errorCode: null, errorMessage: null }; + try { + row = await patch(row, values); + } catch (error) { + await fs.rm(path.dirname(localRoot), { recursive: true, force: true }); + throw error; + } + try { + if (input.target?.kind === "remote") { + const quote = (value: string) => `'${value.replaceAll("'", `'"'"'`)}'`; + const excluded = await runAdapterExecutionTargetShellCommand(input.runId, input.target, + `node -e ${quote(instructionGitExcludeProgram)} ${quote(input.target.remoteCwd)}`, + { cwd: input.target.remoteCwd, env: {}, timeoutSec: 15 }); + if (excluded.exitCode !== 0 || excluded.timedOut) throw new Error("Could not exclude agent files from task Git staging"); + transports.set(key(row), await transport(row, input.target, false)); + } + return await patch(row, { state: "prepared" }); + } catch (error) { + // Preparation failed before a provider could start using this copy. + row = await patch(row, { state: "unavailable", processStoppedAt: new Date(), errorCode: "AGENT_FILES_PREPARE_FAILED", + errorMessage: "Agent files could not be staged. The temporary copy was discarded.", nextAttemptAt: null }); + await release(row, input.target); + throw error; + } + } + + async function retrieve(row: Copy, target?: AdapterExecutionTarget | null) { + if (row.location !== "local") { + let runtime = transports.get(key(row)); + if (!runtime) { + if (!target) throw new Error("Agent directory transport is unavailable"); + runtime = await transport(row, target, true); + transports.set(key(row), runtime); + } + await runtime.restoreWorkspace(); + } + return inspectAgentDirectory(row.localRoot); + } + async function hasChanges(_row: Copy, _target?: AdapterExecutionTarget | null) { + // A whole-directory collector needs a quiescent provider, including its + // child processes. Checkpoint and close at the turn boundary before reading + // either local or remote files. The provider conversation remains resumable; + // ordinary file edits do not change the session configuration digest. + return true; + } + async function collectStopped(row: Copy, target?: AdapterExecutionTarget | null) { + if (completed.has(row.state)) { await release(row); return (await get(row.companyId, row.runId))!; } + row = await patch(row, { processStoppedAt: row.processStoppedAt ?? new Date() }); + // The stopped working copy is the only temporary tree. Retry transient I/O + // at this boundary, then clean it up; there are no preserved run snapshots. + let inspected: Awaited> | undefined; + for (;;) { + row = await patch(row, { attempts: row.attempts + 1 }); + try { + inspected ??= await retrieve(row, target); + const { snapshot } = inspected; + const candidateHash = directorySnapshotSha256(snapshot); + let storageWarning = inspected.storageWarning; + if (candidateHash === row.baseHash) { + row = await patch(row, { state: "unchanged", errorCode: null, errorMessage: null, nextAttemptAt: null }); + } else { + ({ storageWarning } = await store.apply({ companyId: row.companyId, agentId: row.agentId, sourceDir: row.localRoot, baseline: baseline(row) }, actor(row))); + row = await patch(row, { state: "saved", candidateHash, errorCode: null, errorMessage: null, nextAttemptAt: null }); + } + row = await patch(row, { receipt: { ...row.receipt, storageWarning } }); + break; + } catch (error) { + const retryable = !(error instanceof HttpError) || error.status >= 500; + if (retryable && row.attempts < 3) continue; + row = await patch(row, { state: "unavailable", nextAttemptAt: null, + receipt: { ...row.receipt, storageWarning: error instanceof AgentFileLimitError ? agentStorageWarning(error.message) : row.receipt?.storageWarning ?? null }, + errorCode: error instanceof AgentFileLimitError ? "AGENT_FILES_LIMIT_EXCEEDED" : "AGENT_FILES_SAVE_FAILED", + errorMessage: error instanceof HttpError && error.status === 422 + ? `${error.message}. This run's agent-folder changes were not saved; the temporary copy is discarded.` + : "Agent-file synchronization failed. No successful save is claimed; the temporary copy is discarded.", + }); + break; + } + } + await release(row); + return (await get(row.companyId, row.runId))!; + } + async function release(row: Copy, target?: AdapterExecutionTarget | null) { + // Collection and environment teardown can both release the same copy. + // A compact successful cleanup receipt is final, even after restart. + if (completed.has(row.state) && row.processStoppedAt && row.receipt?.cleanupPending === false) return; + const runtime = transports.get(key(row)); + transports.delete(key(row)); + let cleanupPending = false; + await runtime?.cleanupWorkspaceSnapshot?.().catch(() => { cleanupPending = true; }); + const cleanupTarget = runtime?.target ?? target; + if (row.processStoppedAt && completed.has(row.state) && cleanupTarget?.kind === "remote") { + const expected = path.posix.join(cleanupTarget.remoteCwd, ".paperclip-runtime", "agent-files", row.agentId, row.runId); + if (row.executionRoot !== expected) throw new Error("Agent directory cleanup path changed"); + const quoted = `'${expected.replaceAll("'", `'"'"'`)}'`; + const remoteCleanupFailed = await runAdapterExecutionTargetShellCommand(row.runId, cleanupTarget, `rm -rf -- ${quoted}`, + { cwd: cleanupTarget.remoteCwd, env: {}, timeoutSec: 15 }).then(result => result.exitCode !== 0 || result.timedOut, () => true); + cleanupPending ||= remoteCleanupFailed; + } else if (row.processStoppedAt && completed.has(row.state) && row.location !== "local") { + // Rebind only the original host-owned lease. Never acquire a replacement + // sandbox or persist transport credentials in a working-copy receipt. + cleanupPending ||= !await cleanupRemoteAfterRestart(row).catch(() => false); + } + if (completed.has(row.state) && row.processStoppedAt) { + try { await fs.rm(path.dirname(row.localRoot), { recursive: true, force: true }); } + catch { + // Leave the baseline marker so restart recovery retries failed cleanup. + await patch(row, { receipt: { ...row.receipt, cleanupPending: true } }); + return; + } + await patch(row, { receipt: { schema: AGENT_FILES_CONTRACT, state: row.state, appliedCandidateHash: row.candidateHash, storageWarning: row.receipt?.storageWarning ?? null, cleanupPending, + ...(cleanupPending ? { cleanup: row.receipt?.cleanup } : {}) }, + nextAttemptAt: cleanupPending ? new Date(Date.now() + 30_000) : null }); + } + } + async function cleanupRemoteAfterRestart(row: Copy): Promise { + const cleanup = row.receipt?.cleanup as { leaseId?: string; remoteCwd?: string } | undefined; + // Older preview receipts did not record the lease ID. Accept only a unique + // lease still bound to this run and environment in that compatibility case. + const leases = await db.select().from(environmentLeases).where(and( + eq(environmentLeases.companyId, row.companyId), + cleanup?.leaseId ? eq(environmentLeases.id, cleanup.leaseId) : and( + eq(environmentLeases.environmentId, row.location.slice("remote:".length)), eq(environmentLeases.heartbeatRunId, row.runId)), + )); + if (leases.length !== 1) return false; + const lease = leases[0]!; + const termination = lease.metadata?.remoteExecutionTermination as { state?: string } | undefined; + if (hasRemoteTerminationReceipt(lease) && termination?.state === "destroyed") return true; + if (!environmentRuntime || !lease.environmentId || row.location !== `remote:${lease.environmentId}`) return false; + const [environment] = await db.select().from(environments).where(eq(environments.id, lease.environmentId)); + if (!environment) return false; + const remoteCwd = cleanup?.remoteCwd ?? lease.metadata?.remoteCwd; + if (typeof remoteCwd !== "string" || row.executionRoot !== path.posix.join(remoteCwd, ".paperclip-runtime", "agent-files", row.agentId, row.runId)) return false; + const result = await environmentRuntime.execute({ environment: environment as Environment, lease: lease as EnvironmentLease, command: "rm", args: ["-rf", "--", row.executionRoot], + cwd: remoteCwd, env: {}, timeoutMs: 15_000, bypassSession: true }); + return result.exitCode === 0 && !result.timedOut; + } + async function serial(row: Copy, fn: (current: Copy) => Promise): Promise { + // Duplicate stop callbacks and restart recovery must not race while moving + // the stopped working copy. This lock is outside the writable tree. + return withDirectoryMergeLock(path.resolve(row.localRoot, "../../.."), async () => { + const current = await get(row.companyId, row.runId); + if (!current) throw notFound("Agent directory copy not found"); + return fn(current); + }); + } + return { prepare, hasChanges, + collectStopped: (row: Copy, target?: AdapterExecutionTarget | null) => serial(row, current => collectStopped(current, target)), + release: (row: Copy) => serial(row, release), + }; +} diff --git a/server/src/services/agent-file-store.ts b/server/src/services/agent-file-store.ts new file mode 100644 index 0000000000..868a356055 --- /dev/null +++ b/server/src/services/agent-file-store.ts @@ -0,0 +1,282 @@ +import { createHash } from "node:crypto"; +import fs from "node:fs/promises"; +import { constants } from "node:fs"; +import { Readable } from "node:stream"; +import path from "node:path"; +import { and, eq } from "drizzle-orm"; +import { agents, activityLog, agentInstructionHeads, agentInstructionRevisions, type Db } from "@paperclipai/db"; +import { captureDirectorySnapshot, mergeDirectoryWithBaseline, type DirectorySnapshot } from "@paperclipai/adapter-utils/workspace-restore-merge"; +import { HttpError, conflict, notFound, unprocessable } from "../errors.js"; +import { authorizeInstructionCommit, authorizeInstructionRead } from "./agent-instruction-authorization.js"; +import { assertInstructionPathSafe, instructionPath, instructionBytes, materializeInstructionBytes, readInstructionBytes, MAX_INSTRUCTION_BYTES } from "./agent-instruction-files.js"; +import { agentInstructionsBundleMode, deriveBundleState, resolveManagedInstructionsRoot } from "./agent-instructions.js"; +import type { AuthorizationActor } from "./authorization.js"; + +type Tx = Parameters[0]>[0]; +type Agent = typeof agents.$inferSelect; +export const AGENT_FILES_CONTRACT = "paperclip.agent-files.v1"; +export const MAX_AGENT_FILE_BYTES = 256 * 1024 * 1024; +export const MAX_AGENT_DIRECTORY_BYTES = 2 * 1024 * 1024 * 1024; +export const MAX_AGENT_DIRECTORY_ENTRIES = 100_000; +export const fileHash = (bytes: Uint8Array) => createHash("sha256").update(bytes).digest("hex"); + +export class AgentFileLimitError extends HttpError { + constructor(message: string) { + super(422, message, { code: "AGENT_FILES_LIMIT_EXCEEDED" }); + } +} +function assertFileSize(size: number, relative: string) { + if (size > MAX_AGENT_FILE_BYTES) throw new AgentFileLimitError(`Agent file "${relative}" exceeds the 256 MiB per-file limit`); +} +function assertDirectorySize(size: number, count: number) { + if (size > MAX_AGENT_DIRECTORY_BYTES) throw new AgentFileLimitError("Agent directory exceeds the 2 GiB total storage limit"); + if (count > MAX_AGENT_DIRECTORY_ENTRIES) throw new AgentFileLimitError("Agent directory exceeds the 100,000-entry limit (files and folders)"); +} +export function agentStorageWarning(detail: string) { + return `Agent storage is full. ${detail}. Runs can continue; remove or shrink files in AGENT_HOME to free space. Changes exceeding the storage limits will not be saved.`; +} +function storageWarning(size: number, count: number, fullFile?: string) { + if (size >= MAX_AGENT_DIRECTORY_BYTES) return agentStorageWarning("The agent folder has reached its 2 GiB limit"); + if (count >= MAX_AGENT_DIRECTORY_ENTRIES) return agentStorageWarning("The agent folder has reached its 100,000-entry limit"); + if (fullFile) return agentStorageWarning(`Agent file "${fullFile}" has reached its 256 MiB limit`); + return null; +} + +export function agentFilePath(value: string): string { + const relative = instructionPath(value); + if (relative.split("/").includes(".paperclip-runtime") || relative === "promptTemplate.legacy.md") { + throw unprocessable(`${relative} is reserved and cannot be used for agent files`); + } + return relative; +} + +/** Compatibility ETag for clients whose old schema requires a UUID. This is a + * content token, not a revision ID: no snapshot or history row is created. */ +export function agentFileToken(bytes: Uint8Array): string { + return agentFileTokenFromHash(fileHash(bytes)); +} +export function agentFileTokenFromHash(h: string): string { + return `${h.slice(0, 8)}-${h.slice(8, 12)}-8${h.slice(13, 16)}-a${h.slice(17, 20)}-${h.slice(20, 32)}`; +} + +/** Call under the agent row lock. Import deployed revision heads once, then the + * directory is authoritative. The adoption marker lives OUTSIDE agent files. */ +export async function adoptAgentFiles(tx: Tx, agent: Agent): Promise { + if (agentInstructionsBundleMode(agent) === "external") throw unprocessable("External instructions must be migrated to managed storage first"); + const root = resolveManagedInstructionsRoot(agent); + const markerRoot = path.join(path.dirname(root), "file-sync"); + const marker = "adopted.json"; + await assertInstructionPathSafe(root, deriveBundleState(agent).entryFile); + await fs.mkdir(root, { recursive: true, mode: 0o700 }); + const filename = await assertInstructionPathSafe(markerRoot, marker); + const adopted = await fs.readFile(filename, "utf8").catch((error: NodeJS.ErrnoException) => { + if (error.code === "ENOENT") return null; + throw error; + }); + if (adopted !== null) { + if (JSON.parse(adopted).schema !== AGENT_FILES_CONTRACT) throw new Error("Unsupported agent file contract"); + return root; + } + const legacy = await tx.select({ entryFile: agentInstructionHeads.entryFile, contentBase64: agentInstructionRevisions.contentBase64 }) + .from(agentInstructionHeads).innerJoin(agentInstructionRevisions, eq(agentInstructionHeads.revisionId, agentInstructionRevisions.id)) + .where(and(eq(agentInstructionHeads.companyId, agent.companyId), eq(agentInstructionHeads.agentId, agent.id))); + for (const entry of legacy) await materializeInstructionBytes(root, entry.entryFile, Buffer.from(entry.contentBase64, "base64")); + await materializeInstructionBytes(markerRoot, marker, Buffer.from(JSON.stringify({ schema: AGENT_FILES_CONTRACT }))); + return root; +} + +async function openAgentFile(root: string, relative: string) { + const filename = await assertInstructionPathSafe(root, relative); + const handle = await fs.open(filename, constants.O_RDONLY | constants.O_NOFOLLOW).catch((error: NodeJS.ErrnoException) => { + if (error.code === "ENOENT") return null; + throw error; + }); + if (!handle) return null; + try { + const stat = await handle.stat(); + if (!stat.isFile() || stat.nlink !== 1) throw unprocessable("Agent files must be regular files without links"); + assertFileSize(stat.size, relative); + return { handle, size: stat.size }; + } catch (error) { await handle.close(); throw error; } +} + +/** Hash large files incrementally. Only editor-sized content is retained in RAM. */ +export async function inspectAgentFile(root: string, relative: string, bufferLimit = MAX_INSTRUCTION_BYTES) { + const opened = await openAgentFile(root, relative); + if (!opened) return null; + const hash = createHash("sha256"); + let chunks: Buffer[] | null = opened.size <= bufferLimit ? [] : null; + let size = 0; + try { + for await (const chunk of opened.handle.createReadStream()) { + size += chunk.length; + assertFileSize(size, relative); + hash.update(chunk); + if (size > bufferLimit) chunks = null; + chunks?.push(chunk); + } + return { size, hash: hash.digest("hex"), bytes: chunks ? Buffer.concat(chunks) : null }; + } finally { await opened.handle.close(); } +} + +export async function readAgentFile(root: string, relative: string): Promise { + return (await inspectAgentFile(root, relative, MAX_AGENT_FILE_BYTES))?.bytes ?? null; +} + +/** Validate before staging and again after provider stop; never follow links or + * silently skip an unsupported file. Bounds apply to bytes, including binaries. */ +async function scanAgentFiles(root: string, enforceLimits = true) { + await assertInstructionPathSafe(root, ".path-check"); + let size = 0, count = 0; + const entries = new Set(); + let fullFile: string | undefined; + async function walk(dir: string) { + for (const item of await fs.readdir(path.join(root, dir), { withFileTypes: true })) { + const relative = agentFilePath(dir ? `${dir}/${item.name}` : item.name); + const stat = await fs.lstat(path.join(root, relative)); + count++; + if (enforceLimits) assertDirectorySize(size, count); + entries.add(relative); + if (stat.isDirectory()) await walk(relative); + else if (stat.isFile() && stat.nlink === 1) { + size += stat.size; + if (stat.size >= MAX_AGENT_FILE_BYTES) fullFile ??= relative; + if (enforceLimits) { + assertFileSize(stat.size, relative); + assertDirectorySize(size, count); + } + } else throw unprocessable("Agent directories support regular files and directories, without links or special files"); + } + } + await walk(""); + return { size, entries, storageWarning: storageWarning(size, count, fullFile) }; +} + +export async function inspectAgentDirectory(root: string, enforceLimits = true) { + const usage = await scanAgentFiles(root, enforceLimits); + return { snapshot: await captureDirectorySnapshot(root), storageWarning: usage.storageWarning }; +} +export async function snapshotAgentFiles(root: string, enforceLimits = true): Promise { + return (await inspectAgentDirectory(root, enforceLimits)).snapshot; +} + +export function agentFileStore(db: Db) { + async function locked(companyId: string, agentId: string, actor: AuthorizationActor, write: boolean, + fn: (tx: Tx, agent: Agent, root: string, bound: AuthorizationActor) => Promise) { + return db.transaction(async tx => { + const [agent] = await tx.select().from(agents).where(and(eq(agents.companyId, companyId), eq(agents.id, agentId))).for("update"); + if (!agent) throw notFound("Agent not found"); + const bound = write ? await authorizeInstructionCommit(tx, actor, agent) : await authorizeInstructionRead(tx, actor, agent); + const root = await adoptAgentFiles(tx, agent); + return fn(tx, agent, root, bound); + }); + } + async function audit(tx: Tx, agent: Agent, actor: AuthorizationActor, details: Record) { + await tx.insert(activityLog).values({ companyId: agent.companyId, actorType: actor.type === "board" ? "user" : "agent", + actorId: (actor.type === "board" ? actor.userId : actor.agentId)!, agentId: actor.type === "agent" ? actor.agentId : null, + runId: actor.runId, responsibleUserId: actor.type === "board" ? actor.userId : actor.onBehalfOfUserId, + action: "agent.files_updated", entityType: "agent", entityId: agent.id, details }); + } + return { + locked, + download: async (companyId: string, agentId: string, relative: string, actor: AuthorizationActor) => { + const opened: { file: Awaited> } = { file: null }; + try { + const file = await locked(companyId, agentId, actor, false, async (_tx, _agent, root) => { + opened.file = await openAgentFile(root, instructionPath(relative)); + return opened.file; + }); + // Transfer outside the database lock. The open descriptor pins the file + // across concurrent atomic replacements; end bounds concurrent growth. + if (!file) return null; + if (file.size === 0) { await file.handle.close(); return { size: 0, stream: Readable.from([]) }; } + return { size: file.size, stream: file.handle.createReadStream({ end: file.size - 1 }) }; + } catch (error) { await opened.file?.handle.close(); throw error; } + }, + read: (companyId: string, agentId: string, relative: string, actor: AuthorizationActor) => + locked(companyId, agentId, actor, false, (_tx, _agent, root) => readAgentFile(root, instructionPath(relative))), + write: (input: { companyId: string; agentId: string; path: string; bytes: Buffer | null; baseHash: string | null }, actor: AuthorizationActor) => + locked(input.companyId, input.agentId, actor, true, async (tx, agent, root, bound) => { + const relative = agentFilePath(input.path); + if (input.bytes) assertFileSize(input.bytes.length, relative); + const previous = await inspectAgentFile(root, relative); + const currentHash = previous?.hash ?? null; + const incomingHash = input.bytes === null ? null : fileHash(input.bytes); + if (currentHash === incomingHash) return { contentHash: currentHash, changed: false }; + if (currentHash !== input.baseHash) throw conflict("This file changed since it was read. Reload before saving.", { code: "AGENT_FILE_CONFLICT", path: relative, currentHash }); + if (input.bytes === null && relative === deriveBundleState(agent).entryFile) throw unprocessable("The configured instruction entry cannot be deleted"); + if (input.bytes !== null) { + if (relative === deriveBundleState(agent).entryFile) instructionBytes(input.bytes); + // A quota check needs metadata only. Do not hash unrelated large + // files while holding the editor's row lock for a one-file save. + const snapshot = await scanAgentFiles(root); + const total = snapshot.size + input.bytes.length - (previous?.size ?? 0); + const newEntries = relative.split("/").map((_part, i, parts) => parts.slice(0, i + 1).join("/")).filter(name => !snapshot.entries.has(name)).length; + assertDirectorySize(total, snapshot.entries.size + newEntries); + } + if (input.bytes === null) await fs.unlink(await assertInstructionPathSafe(root, relative)); + else await materializeInstructionBytes(root, relative, input.bytes); + await audit(tx, agent, bound, { path: relative, contentHash: incomingHash }); + return { contentHash: incomingHash, changed: true }; + }), + apply: (input: { companyId: string; agentId: string; sourceDir: string; baseline: DirectorySnapshot }, actor: AuthorizationActor) => + locked(input.companyId, input.agentId, actor, true, async (tx, agent, root, bound) => { + const incoming = await snapshotAgentFiles(input.sourceDir); + const entry = await readInstructionBytes(input.sourceDir, deriveBundleState(agent).entryFile); + if (entry === null) throw unprocessable("The configured instruction entry cannot be deleted"); + instructionBytes(entry); + // Previously saved/imported bytes must remain readable, including when + // over quota. Enforce limits on the incoming and resulting tree so a + // run can delete files to recover instead of being locked out forever. + const current = await snapshotAgentFiles(root, false); + // Rebase only the run's changed paths onto the current tree. This makes + // same-file edits/deletions last-sync-wins while untouched files retain + // changes from other runs. Ancestors may need recreating after a writer + // replaced a directory with a file. + const entries = new Map(input.baseline.entries); + const changed = [...new Set([...input.baseline.entries, ...incoming.entries].map(([name]) => name))] + .filter(name => JSON.stringify(input.baseline.entries.get(name)) !== JSON.stringify(incoming.entries.get(name))); + for (const name of changed) { + const present = current.entries.get(name); + if (present) entries.set(name, present); else entries.delete(name); + if (incoming.entries.has(name)) { + for (let parent = path.posix.dirname(name); parent !== "."; parent = path.posix.dirname(parent)) { + if (current.entries.get(parent)?.kind === "dir") continue; + const before = current.entries.get(parent); + if (before) entries.set(parent, before); else entries.delete(parent); + } + } + } + const applyBaseline = { ...input.baseline, entries }; + const finalEntries = new Map([...current.entries].map(([name, value]) => [name, { value, root }])); + for (const name of changed) if (!incoming.entries.has(name)) finalEntries.delete(name); + for (const [name, value] of incoming.entries) { + if (JSON.stringify(input.baseline.entries.get(name)) === JSON.stringify(value)) continue; + if (value.kind !== "dir") for (const child of finalEntries.keys()) if (child.startsWith(`${name}/`)) finalEntries.delete(child); + finalEntries.set(name, { value, root: input.sourceDir }); + for (let parent = path.posix.dirname(name); parent !== "."; parent = path.posix.dirname(parent)) { + if (finalEntries.get(parent)?.value.kind !== "dir") finalEntries.set(parent, { value: { kind: "dir" }, root: input.sourceDir }); + } + } + // An unchanged file created by another run can keep a removed folder + // alive. Count those surviving parent directories in quota preflight. + for (const name of finalEntries.keys()) { + for (let parent = path.posix.dirname(name); parent !== "."; parent = path.posix.dirname(parent)) { + if (!finalEntries.has(parent)) finalEntries.set(parent, { value: { kind: "dir" }, root }); + } + } + let total = 0; + let fullFile: string | undefined; + for (const [name, item] of finalEntries) if (item.value.kind === "file") { + const size = (await fs.stat(path.join(item.root, name))).size; + assertFileSize(size, name); + if (size >= MAX_AGENT_FILE_BYTES) fullFile ??= name; + total += size; + } + assertDirectorySize(total, finalEntries.size); + await mergeDirectoryWithBaseline({ ...input, baseline: applyBaseline, targetDir: root }); + await audit(tx, agent, bound, { sourceRunId: actor.runId, contract: AGENT_FILES_CONTRACT }); + return { storageWarning: storageWarning(total, finalEntries.size, fullFile) }; + }), + }; +} diff --git a/server/src/services/agent-instruction-authorization.ts b/server/src/services/agent-instruction-authorization.ts new file mode 100644 index 0000000000..fe98d26995 --- /dev/null +++ b/server/src/services/agent-instruction-authorization.ts @@ -0,0 +1,195 @@ +import { and, eq, isNull } from "drizzle-orm"; +import { + agentApiKeys, + authUsers, + companyMemberships, + heartbeatRuns, + runIdentityContexts, + type Db, +} from "@paperclipai/db"; +import { forbidden } from "../errors.js"; +import { + authorizationService, + authorizationDeniedDetails, + type AuthorizationActor, +} from "./authorization.js"; +import { + agentInstructionsChangeTargetKey, + changeConsentGateService, +} from "./change-consent-gate.js"; + +type Connection = Db | Parameters[0]>[0]; + +/** Accept ONLY a server-authenticated actor, never request/tool JSON. Rebind run/key provenance on every call. */ +export async function resolveInstructionActor( + db: Connection, + actor: AuthorizationActor, +): Promise { + if (actor.type === "board") { + if (!actor.userId) + throw forbidden("Instruction edits require an authenticated user"); + // Discard caller membership caches so revoked access is observed. + return { + type: "board", + userId: actor.userId, + source: actor.source, + ignoreInstanceAdmin: actor.ignoreInstanceAdmin, + }; + } + if (actor.type !== "agent" || !actor.agentId || !actor.companyId) + throw forbidden("Instruction identity is missing"); + let responsibleUserId: string | null = null; + let keyScope = actor.keyScope; + if (actor.keyId) { + const [key] = await db + .select() + .from(agentApiKeys) + .where( + and( + eq(agentApiKeys.id, actor.keyId), + eq(agentApiKeys.companyId, actor.companyId), + eq(agentApiKeys.agentId, actor.agentId), + isNull(agentApiKeys.revokedAt), + ), + ); + if (!key) + throw forbidden("Instruction API key is missing or revoked", { + code: "INSTRUCTION_IDENTITY_INVALID", + }); + responsibleUserId = key.responsibleUserId; + keyScope = key.scopeConfig ?? undefined; + } + if (actor.runId) { + const [run] = await db + .select() + .from(heartbeatRuns) + .where( + and( + eq(heartbeatRuns.id, actor.runId), + eq(heartbeatRuns.companyId, actor.companyId), + eq(heartbeatRuns.agentId, actor.agentId), + ), + ); + if (!run) + throw forbidden("Instruction run identity does not match", { + code: "INSTRUCTION_IDENTITY_INVALID", + }); + if (actor.keyId && responsibleUserId !== run.responsibleUserId) + throw forbidden("Key and run responsible identity do not match", { + code: "INSTRUCTION_IDENTITY_INVALID", + }); + responsibleUserId = run.responsibleUserId; + if (run.activeIdentityContextId) { + const [context] = await db + .select() + .from(runIdentityContexts) + .where( + and( + eq(runIdentityContexts.id, run.activeIdentityContextId), + eq(runIdentityContexts.runId, run.id), + eq(runIdentityContexts.companyId, run.companyId), + eq(runIdentityContexts.status, "accepted"), + ), + ); + if (!context) + throw forbidden("Instruction run identity is unavailable", { + code: "INSTRUCTION_IDENTITY_INVALID", + }); + responsibleUserId = context.responsibleUserId; + } + } + if ( + !responsibleUserId || + (actor.onBehalfOfUserId && actor.onBehalfOfUserId !== responsibleUserId) + ) { + throw forbidden("A current server-bound responsible user is required", { + code: "INSTRUCTION_IDENTITY_INVALID", + }); + } + const [membership] = await db + .select({ userId: authUsers.id }) + .from(companyMemberships) + .innerJoin(authUsers, eq(authUsers.id, companyMemberships.principalId)) + .where( + and( + eq(companyMemberships.companyId, actor.companyId), + eq(companyMemberships.principalType, "user"), + eq(companyMemberships.principalId, responsibleUserId), + eq(companyMemberships.status, "active"), + ), + ); + if (!membership) + throw forbidden("The responsible user is unavailable in this company", { + code: "RESPONSIBLE_USER_UNAVAILABLE", + }); + return { + type: "agent", + agentId: actor.agentId, + companyId: actor.companyId, + runId: actor.runId, + keyId: actor.keyId, + keyScope, + source: actor.source, + onBehalfOfUserId: responsibleUserId, + }; +} + +/** Instruction content is narrower than general same-company agent visibility. */ +export async function authorizeInstructionRead( + db: Connection, + actor: AuthorizationActor, + target: { companyId: string; id: string }, +) { + const bound = await resolveInstructionActor(db, actor); + const access = authorizationService(db); + const resource = { type: "agent" as const, companyId: target.companyId, agentId: target.id }; + const peer = bound.type === "agent" && bound.agentId !== target.id; + const decision = await access.decide({ + actor: bound, + action: peer ? "agent_config:read" : "agent:read", + resource, + scope: { targetAgentId: target.id }, + }); + if (!decision.allowed) throw forbidden(decision.explanation, authorizationDeniedDetails(decision)); + return bound; +} + +/** Current target edit access + agent containment + protected-change consent, shared by all writers. */ +export async function authorizeInstructionCommit( + db: Connection, + actor: AuthorizationActor, + target: { companyId: string; id: string }, +) { + const bound = await resolveInstructionActor(db, actor); + const access = authorizationService(db); + const input = { + actor: bound, + action: "agent_instructions:update" as const, + resource: { + type: "agent" as const, + companyId: target.companyId, + agentId: target.id, + }, + scope: { targetAgentId: target.id, requiresChangeGrant: true }, + }; + let decision = await access.decide(input); + if ( + !decision.allowed && + decision.reason === "deny_missing_consent" && + bound.type === "agent" + ) { + await changeConsentGateService(db).assertConsented({ + companyId: target.companyId, + actorAgentId: bound.agentId, + actorRunId: bound.runId, + targetKeys: [agentInstructionsChangeTargetKey(target.id)], + }); + decision = await access.decide({ + ...input, + scope: { ...input.scope, consentedChange: true }, + }); + } + if (!decision.allowed) + throw forbidden(decision.explanation, authorizationDeniedDetails(decision)); + return bound; +} diff --git a/server/src/services/agent-instruction-files.ts b/server/src/services/agent-instruction-files.ts new file mode 100644 index 0000000000..6f3db874ac --- /dev/null +++ b/server/src/services/agent-instruction-files.ts @@ -0,0 +1,178 @@ +import fs from "node:fs/promises"; +import { constants } from "node:fs"; +import path from "node:path"; +import { randomUUID } from "node:crypto"; +import { unprocessable } from "../errors.js"; + +export const MAX_INSTRUCTION_BYTES = 1024 * 1024; +export function instructionPath(value: string): string { + if ( + !value || + value.length > 512 || + value.includes("\0") || + value.includes("\\") || + path.posix.isAbsolute(value) || + /^[a-z]:/i.test(value) || + value.split("/").some((part) => !part || part === "." || part === "..") + ) { + throw unprocessable("Use a relative instructions path without traversal", { + code: "INSTRUCTION_PATH_INVALID", + }); + } + return value; +} + +export function instructionBytes(content: string | Uint8Array): Buffer { + const byteLength = + typeof content === "string" + ? Buffer.byteLength(content, "utf8") + : content.byteLength; + if (byteLength > MAX_INSTRUCTION_BYTES) { + throw unprocessable("Instructions exceed the 1 MiB limit", { + code: "INSTRUCTION_CONTENT_INVALID", + }); + } + const bytes = + typeof content === "string" + ? Buffer.from(content, "utf8") + : Buffer.from(content); + try { + // Preserve BOM; reject invalid UTF-8 and unpaired JS surrogates instead of replacing bytes. + const decoded = new TextDecoder("utf-8", { + fatal: true, + ignoreBOM: true, + }).decode(bytes); + if (typeof content === "string" && decoded !== content) + throw new Error("invalid Unicode"); + } catch { + throw unprocessable("Instructions must be valid UTF-8", { + code: "INSTRUCTION_CONTENT_INVALID", + }); + } + return bytes; +} + +export async function assertInstructionPathSafe( + root: string, + entryFile: string, +) { + instructionPath(entryFile); + let absolute = path.resolve(root, entryFile); + // macOS exposes its system temporary directories through root-owned aliases. + // Resolve only those fixed OS aliases, never a link in operator/agent storage. + if (process.platform === "darwin") { + const alias = absolute.split(path.sep)[1]; + if (alias === "var" || alias === "tmp" || alias === "etc") { + const systemPath = `/${alias}`; + const stat = await fs.lstat(systemPath); + if (stat.isSymbolicLink() && stat.uid === 0 && await fs.realpath(systemPath) === `/private/${alias}`) { + absolute = `/private${absolute}`; + } + } + } + let current = path.parse(absolute).root; + const parts = absolute.slice(current.length).split(path.sep); + for (const [i, part] of parts.entries()) { + current = path.join(current, part); + const stat = await fs + .lstat(current) + .catch((error: NodeJS.ErrnoException) => { + if (error.code === "ENOENT") return null; + throw error; + }); + if (!stat) continue; + if ( + stat.isSymbolicLink() || + (i < parts.length - 1 ? !stat.isDirectory() : !stat.isFile()) + ) { + throw unprocessable( + "Instructions require regular files and directories without symlinks", + { code: "INSTRUCTION_PATH_INVALID" }, + ); + } + } + return absolute; +} + +export async function readInstructionBytes( + root: string, + entryFile: string, +): Promise { + const absolute = await assertInstructionPathSafe(root, entryFile); + const file = await fs + .open(absolute, constants.O_RDONLY | constants.O_NOFOLLOW) + .catch((error: NodeJS.ErrnoException) => { + if (error.code === "ENOENT") return null; + throw error; + }); + if (!file) return null; + try { + const stat = await file.stat(); + if (!stat.isFile() || stat.size > MAX_INSTRUCTION_BYTES) { + throw unprocessable( + "Instructions must be a regular UTF-8 file of at most 1 MiB", + { code: "INSTRUCTION_CONTENT_INVALID" }, + ); + } + const buffer = Buffer.alloc(MAX_INSTRUCTION_BYTES + 1); + let length = 0; + while (length < buffer.length) { + const result = await file.read( + buffer, + length, + buffer.length - length, + length, + ); + if (result.bytesRead === 0) break; + length += result.bytesRead; + } + return instructionBytes(buffer.subarray(0, length)); + } finally { + await file.close(); + } +} + +/** Call while holding the canonical agent lock. Never writes through a symlink. */ +export async function materializeInstructionBytes( + root: string, + entryFile: string, + bytes: Buffer, +) { + const absolute = await assertInstructionPathSafe(root, entryFile); + await fs.mkdir(path.dirname(absolute), { recursive: true }); + await assertInstructionPathSafe(root, entryFile); + const temporary = path.join( + path.dirname(absolute), + `.instruction-${randomUUID()}.tmp`, + ); + try { + const file = await fs.open(temporary, "wx", 0o600); + try { + await file.writeFile(bytes); + await file.sync(); + } finally { + await file.close(); + } + await assertInstructionPathSafe(root, entryFile); + await fs.rename(temporary, absolute); + } finally { + await fs.rm(temporary, { force: true }); + } +} + +// Keep the reserved runtime directory out of Git using a self-ignoring file +// inside that directory. Never write to a gitdir or exclude path derived from +// repository-controlled metadata; legitimate worktrees can keep external gitdirs. +export const instructionGitExcludeProgram = String.raw` +const fs=require('node:fs'),path=require('node:path'),crypto=require('node:crypto'); +const cwd=fs.realpathSync(process.argv[1]); +const root=path.join(cwd,'.paperclip-runtime'),filename=path.join(root,'.gitignore'); +const stat=(name)=>{try{return fs.lstatSync(name)}catch(e){if(e.code==='ENOENT')return null;throw e}}; +const directory=stat(root); +if(directory && (directory.isSymbolicLink() || !directory.isDirectory()))throw Error('Unsafe runtime exclusion directory'); +if(!directory)fs.mkdirSync(root,{mode:0o700}); +const existing=stat(filename); +if(existing && (existing.isSymbolicLink() || !existing.isFile()))throw Error('Unsafe runtime exclusion file'); +const temporary=path.join(root,'.ignore-'+crypto.randomUUID()+'.tmp'); +try{fs.writeFileSync(temporary,'*\n',{flag:'wx',mode:0o600});fs.renameSync(temporary,filename)}finally{fs.rmSync(temporary,{force:true})} +`; diff --git a/server/src/services/agent-instruction-revisions.ts b/server/src/services/agent-instruction-revisions.ts new file mode 100644 index 0000000000..6c07b0d08f --- /dev/null +++ b/server/src/services/agent-instruction-revisions.ts @@ -0,0 +1,429 @@ +import { createHash } from "node:crypto"; +import { adoptAgentFiles, agentFileToken } from "./agent-file-store.js"; +import { and, desc, eq, getTableColumns, lt, or, sql } from "drizzle-orm"; +import { + activityLog, + agents, + plugins, + pluginManagedResources, + agentInstructionHeads as heads, + agentInstructionRevisions as revisions, + type Db, +} from "@paperclipai/db"; +import type { + AgentInstructionCommitReceipt, + AgentInstructionDiff, + AgentInstructionHistory, + AgentInstructionSnapshot, + AgentInstructionSource, +} from "@paperclipai/shared"; +import { conflict, forbidden, notFound, unprocessable } from "../errors.js"; +import { + agentInstructionsBundleMode, + deriveBundleState, + resolveManagedInstructionsRoot, +} from "./agent-instructions.js"; +import { + instructionBytes, + instructionPath, + assertInstructionPathSafe, + readInstructionBytes, + materializeInstructionBytes, +} from "./agent-instruction-files.js"; +import { + authorizeInstructionCommit, + authorizeInstructionRead, +} from "./agent-instruction-authorization.js"; +import type { AuthorizationActor } from "./authorization.js"; + +type Tx = Parameters[0]>[0]; +type Revision = typeof revisions.$inferSelect; +type PluginResetActor = { type: "plugin"; pluginId: string; pluginKey: string; agentKey: string }; +type RevisionActor = AuthorizationActor | PluginResetActor; +const { contentBase64: _contentColumn, ...revisionMetadataColumns } = + getTableColumns(revisions); +export type InstructionTarget = { companyId: string; agentId: string }; +export type InstructionCommitInput = InstructionTarget & { + entryFile: string; + content: string | Uint8Array; + baseRevisionId: string | null; + source: Exclude; +}; +function metadata(row: Omit) { + const { createdAt, source, ...fields } = row; + return { + ...fields, + source: source as AgentInstructionSource, + createdAt: createdAt.toISOString(), + }; +} +function snapshot(row: Revision): AgentInstructionSnapshot { + const { contentBase64, ...fields } = row; + return { + revision: metadata(fields), + content: Buffer.from(contentBase64, "base64").toString("utf8"), + }; +} +function owner(target: InstructionTarget, entryFile: string) { + return and( + eq(revisions.companyId, target.companyId), + eq(revisions.agentId, target.agentId), + eq(revisions.entryFile, entryFile), + ); +} +function headOwner(target: InstructionTarget, entryFile: string) { + return and( + eq(heads.companyId, target.companyId), + eq(heads.agentId, target.agentId), + eq(heads.entryFile, entryFile), + ); +} + +export function agentInstructionRevisionService(db: Db) { + async function lockTarget( + tx: Tx, + target: InstructionTarget, + actor?: AuthorizationActor, + ) { + if (actor?.type === "agent" && actor.companyId !== target.companyId) + throw notFound("Agent not found"); + const [agent] = await tx + .select() + .from(agents) + .where( + and( + eq(agents.id, target.agentId), + eq(agents.companyId, target.companyId), + ), + ) + .for("update"); + if (!agent) throw notFound("Agent not found"); + if (agentInstructionsBundleMode(agent) === "external") { + throw unprocessable( + "External host instructions do not support revision write-back. Ask an instance administrator to migrate this bundle to managed storage.", + { code: "INSTRUCTION_MANAGED_BUNDLE_REQUIRED" }, + ); + } + const state = deriveBundleState(agent); + const entryFile = instructionPath(state.entryFile); + return { agent, entryFile, root: resolveManagedInstructionsRoot(agent) }; + } + async function authorizeRead( + tx: Tx, + actor: AuthorizationActor, + target: InstructionTarget, + ) { + return authorizeInstructionRead(tx, actor, { companyId: target.companyId, id: target.agentId }); + } + // Trusted host-only reset authority. No HTTP or agent tool accepts this actor. + async function authorizePluginReset(tx: Tx, actor: PluginResetActor, target: InstructionTarget, agent: typeof agents.$inferSelect) { + const [plugin] = await tx.select().from(plugins).where(and(eq(plugins.id, actor.pluginId), eq(plugins.pluginKey, actor.pluginKey))); + const marker = agent.metadata?.paperclipManagedResource as Record | undefined; + const [binding] = await tx.select().from(pluginManagedResources).where(and( + eq(pluginManagedResources.pluginId, actor.pluginId), eq(pluginManagedResources.pluginKey, actor.pluginKey), + eq(pluginManagedResources.companyId, target.companyId), eq(pluginManagedResources.resourceKind, "agent"), + eq(pluginManagedResources.resourceKey, actor.agentKey), eq(pluginManagedResources.resourceId, target.agentId), + )); + if (!plugin || plugin.status !== "ready" || !plugin.manifestJson.capabilities.includes("agents.managed") + || !plugin.manifestJson.agents?.some((declaration) => declaration.agentKey === actor.agentKey) + || !binding || marker?.pluginId !== actor.pluginId || marker.pluginKey !== actor.pluginKey + || marker.resourceKind !== "agent" || marker.resourceKey !== actor.agentKey) { + throw forbidden("Plugin reset is limited to its declared, bound managed agent"); + } + const declared = plugin.manifestJson.agents!.find((declaration) => declaration.agentKey === actor.agentKey)!; + return { ...actor, declaredEntryFile: instructionPath(declared.instructions?.entryFile ?? "AGENTS.md") }; + } + async function readForPluginReset(target: InstructionTarget, actor: PluginResetActor) { + return db.transaction(async (tx) => { + const state = await lockTarget(tx, target); + const bound = await authorizePluginReset(tx, actor, target, state.agent); + // Preserve the formerly configured entry before an explicit stock reset. + await currentFile(tx, target, state, bound); + const row = await currentFile(tx, target, { ...state, entryFile: bound.declaredEntryFile }, bound); + return { snapshot: row ? snapshot(row) : null, configuredEntryFile: state.entryFile }; + }); + } + async function head(tx: Tx, target: InstructionTarget, entryFile: string) { + const [result] = await tx + .select({ revision: revisions }) + .from(heads) + .innerJoin(revisions, eq(revisions.id, heads.revisionId)) + .where(headOwner(target, entryFile)); + return result?.revision ?? null; + } + // The deployed revision tables are read-only upgrade input. Current bytes + // live in the agent directory; old UUID clients receive a content ETag. + async function currentFile(tx: Tx, target: InstructionTarget, state: Awaited>, actor?: RevisionActor) { + await adoptAgentFiles(tx, state.agent); + const bytes = await readInstructionBytes(state.root, state.entryFile); + if (bytes === null) return null; + return currentRow(target, state.entryFile, bytes, actor); + } + function currentRow(target: InstructionTarget, entryFile: string, bytes: Buffer, actor?: RevisionActor): Revision { + return { ...target, id: agentFileToken(bytes), entryFile, contentBase64: bytes.toString("base64"), + contentHash: createHash("sha256").update(bytes).digest("hex"), byteLength: bytes.length, + parentRevisionId: null, baseRevisionId: null, restoredFromRevisionId: null, source: "api", + actorAgentId: actor?.type === "agent" ? actor.agentId ?? null : null, + actorUserId: actor?.type === "board" ? actor.userId ?? null : null, + responsibleUserId: actor?.type === "board" ? actor.userId ?? null : actor?.type === "agent" ? actor.onBehalfOfUserId ?? null : null, + sourceRunId: actor && actor.type !== "plugin" ? actor.runId ?? null : null, createdAt: new Date(0) }; + } + async function getRevision( + tx: Tx, + target: InstructionTarget, + entryFile: string, + id: string, + ) { + const [row] = await tx + .select() + .from(revisions) + .where(and(owner(target, entryFile), eq(revisions.id, id))); + if (!row) throw notFound("Instruction revision not found"); + return row; + } + async function readCurrent( + target: InstructionTarget, + actor: AuthorizationActor, + ): Promise { + return db.transaction(async (tx) => { + const state = await lockTarget(tx, target, actor); + const bound = await authorizeRead(tx, actor, target); + const row = await currentFile(tx, target, state, bound); + return row ? snapshot(row) : null; + }); + } + /** Trusted orchestration read for the already authorized run target. This + * never seeds or writes, and is not exposed through HTTP or model tools. */ + async function readCommittedForRuntime(target: InstructionTarget): Promise { + return db.transaction(async (tx) => { + const state = await lockTarget(tx, target); + const current = await currentFile(tx, target, state); + return current ? snapshot(current) : null; + }); + } + /** Rebuild disk from the current committed head under the same lock as commits. Safe after restart. */ + async function materializeCurrent(target: InstructionTarget): Promise { + await db.transaction(async (tx) => { + const state = await lockTarget(tx, target); + await adoptAgentFiles(tx, state.agent); + }); + } + async function commitInternal( + input: + | InstructionCommitInput + | (Omit & { + restoreRevisionId: string; + }), + actor: RevisionActor, + configuredEntryFile?: string, + ): Promise { + instructionPath(input.entryFile); + const bytes = "content" in input ? instructionBytes(input.content) : null; + const result = await db.transaction(async (tx) => { + const state = await lockTarget(tx, input, actor.type === "plugin" ? undefined : actor); + const bound = actor.type === "plugin" + ? await authorizePluginReset(tx, actor, input, state.agent) + : await authorizeInstructionCommit(tx, actor, state.agent); + if (bound.type === "plugin") { + if (input.entryFile !== bound.declaredEntryFile) throw forbidden("Plugin reset must restore its declared entry file"); + if (configuredEntryFile !== state.entryFile) throw conflict("Configured instruction entry changed; read the current entry and retry", { code: "INSTRUCTION_ENTRY_CHANGED", entryFile: state.entryFile }); + state.entryFile = input.entryFile; + } + if (state.entryFile !== input.entryFile) + throw conflict( + "Configured instruction entry changed; read the current entry and retry", + { code: "INSTRUCTION_ENTRY_CHANGED", entryFile: state.entryFile }, + ); + await assertInstructionPathSafe(state.root, state.entryFile); + const current = await currentFile(tx, input, state, bound); + const restored = + "restoreRevisionId" in input + ? await getRevision( + tx, + input, + input.entryFile, + input.restoreRevisionId, + ) + : null; + const candidate = bytes ?? Buffer.from(restored!.contentBase64, "base64"); + const bindEntry = async () => { + if ( + !state.agent.adapterConfig.instructionsRootPath || + !state.agent.adapterConfig.instructionsBundleMode || + (bound.type === "plugin" && configuredEntryFile !== input.entryFile) + ) { + await tx + .update(agents) + .set({ + adapterConfig: { + ...state.agent.adapterConfig, + instructionsBundleMode: "managed", + instructionsRootPath: state.root, + instructionsEntryFile: state.entryFile, + instructionsFilePath: `${state.root}/${state.entryFile}`, + }, + updatedAt: new Date(), + }) + .where(eq(agents.id, state.agent.id)); + } + }; + // An exact replay can safely return the durable receipt even after its base + // advanced. Restore also deduplicates identical content, preserving history. + if (current && current.contentBase64 === candidate.toString("base64")) { + await bindEntry(); + return { row: current, changed: false }; + } + // A pre-upgrade run can still pin a historical UUID. Compare its bytes, + // never let the old head project over a newer directory. + const legacyBase = input.baseRevisionId && current?.id !== input.baseRevisionId + ? await tx.select().from(revisions).where(and(owner(input, input.entryFile), eq(revisions.id, input.baseRevisionId))).then(rows => rows[0]) + : null; + if ((current?.id ?? null) !== input.baseRevisionId && !(legacyBase && legacyBase.contentHash === current?.contentHash)) { + throw conflict( + "Instructions changed since the base revision. Read the current entry before saving.", + { + code: "INSTRUCTION_REVISION_CONFLICT", + baseRevisionId: input.baseRevisionId, + currentRevisionId: current?.id ?? null, + }, + ); + } + await materializeInstructionBytes(state.root, state.entryFile, candidate); + const row = currentRow(input, input.entryFile, candidate, bound); + await tx.insert(activityLog).values({ companyId: input.companyId, + actorType: bound.type === "plugin" ? "plugin" : bound.type === "board" ? "user" : "agent", + actorId: (bound.type === "plugin" ? bound.pluginId : bound.type === "board" ? bound.userId : bound.agentId)!, + agentId: bound.type === "agent" ? bound.agentId : null, runId: row.sourceRunId, + responsibleUserId: row.responsibleUserId, action: "agent.files_updated", entityType: "agent", entityId: input.agentId, + details: { path: input.entryFile, contentHash: row.contentHash, source: restored ? "legacy_restore" : (input as InstructionCommitInput).source } }); + await bindEntry(); + return { row, changed: true }; + }); + return { ...snapshot(result.row), changed: result.changed, materialization: "current" }; + } + + async function history( + target: InstructionTarget & { + entryFile: string; + cursor?: string; + limit?: number; + }, + actor: AuthorizationActor, + ): Promise { + const limit = Math.max(1, Math.min(100, target.limit ?? 50)); + return db.transaction(async (tx) => { + const state = await lockTarget(tx, target, actor); + const bound = await authorizeRead(tx, actor, target); + await currentFile(tx, target, state, bound); + instructionPath(target.entryFile); + const cursor = target.cursor + ? await getRevision(tx, target, target.entryFile, target.cursor) + : null; + // Keep PostgreSQL timestamp precision; JS Date truncates microseconds and + // would skip revisions in the cursor's fractional millisecond. + const cursorTime = cursor + ? sql`(select created_at from agent_instruction_revisions where id = ${cursor.id})` + : null; + const rows = await tx + .select(revisionMetadataColumns) + .from(revisions) + .where( + and( + owner(target, target.entryFile), + cursor && cursorTime + ? or( + lt(revisions.createdAt, cursorTime), + and( + eq(revisions.createdAt, cursorTime), + lt(revisions.id, cursor.id), + ), + ) + : undefined, + ), + ) + .orderBy(desc(revisions.createdAt), desc(revisions.id)) + .limit(limit + 1); + return { + revisions: rows.slice(0, limit).map(metadata), + nextCursor: rows.length > limit ? rows[limit - 1]!.id : null, + }; + }); + } + async function readRevision( + target: InstructionTarget & { entryFile: string; revisionId: string }, + actor: AuthorizationActor, + ) { + return db.transaction(async (tx) => { + await lockTarget(tx, target, actor); + await authorizeRead(tx, actor, target); + return snapshot( + await getRevision( + tx, + target, + instructionPath(target.entryFile), + target.revisionId, + ), + ); + }); + } + async function diff( + target: InstructionTarget & { + entryFile: string; + fromRevisionId: string; + toRevisionId: string; + }, + actor: AuthorizationActor, + ): Promise { + const from = await readRevision( + { ...target, revisionId: target.fromRevisionId }, + actor, + ); + const to = await readRevision( + { ...target, revisionId: target.toRevisionId }, + actor, + ); + // Linear, bounded exact replacement diff. Code points keep surrogate pairs intact. + const a = Array.from(from.content), + b = Array.from(to.content); + let start = 0, + end = 0; + while (start < a.length && start < b.length && a[start] === b[start]) + start++; + while ( + end < a.length - start && + end < b.length - start && + a[a.length - end - 1] === b[b.length - end - 1] + ) + end++; + return { + from, + to, + prefix: a.slice(0, start).join(""), + removed: a.slice(start, a.length - end).join(""), + added: b.slice(start, b.length - end).join(""), + suffix: end ? a.slice(-end).join("") : "", + }; + } + return { + readCommittedForRuntime, + readCurrent, + readForPluginReset, + commitPluginReset: (input: Omit & { configuredEntryFile: string }, actor: PluginResetActor) => + commitInternal({ ...input, source: "api" }, actor, input.configuredEntryFile), + readRevision, + history, + diff, + materializeCurrent, + commit: (input: InstructionCommitInput, actor: AuthorizationActor) => + commitInternal(input, actor), + restore: ( + input: InstructionTarget & { + entryFile: string; + baseRevisionId: string; + revisionId: string; + }, + actor: AuthorizationActor, + ) => + commitInternal({ ...input, restoreRevisionId: input.revisionId }, actor), + }; +} diff --git a/server/src/services/agent-instruction-working-copies.ts b/server/src/services/agent-instruction-working-copies.ts new file mode 100644 index 0000000000..7611dcb614 --- /dev/null +++ b/server/src/services/agent-instruction-working-copies.ts @@ -0,0 +1,356 @@ +import { agentDirectoryWorkingCopyService, isAgentDirectoryCopy } from "./agent-directory-working-copies.js"; +import { createHash } from "node:crypto"; +import { execFile as execFileCallback } from "node:child_process"; +import { promisify } from "node:util"; +import fs from "node:fs/promises"; +import path from "node:path"; +import { and, asc, desc, eq, inArray, lte, or, isNull, isNotNull, sql } from "drizzle-orm"; +import { agents, heartbeatRuns, agentInstructionWorkingCopies as copies, type Db } from "@paperclipai/db"; +import { + prepareAdapterExecutionTargetRuntime, + runAdapterExecutionTargetShellCommand, + type AdapterExecutionTarget, +} from "@paperclipai/adapter-utils/execution-target"; +import { conflict, notFound } from "../errors.js"; +import { agentInstructionsService, agentInstructionsBundleMode } from "./agent-instructions.js"; +import { agentInstructionRevisionService } from "./agent-instruction-revisions.js"; +import { resolveInstructionActor } from "./agent-instruction-authorization.js"; +import { assertInstructionPathSafe, instructionBytes, instructionPath, materializeInstructionBytes, MAX_INSTRUCTION_BYTES, readInstructionBytes, instructionGitExcludeProgram } from "./agent-instruction-files.js"; +import { hasNativeLocalProcessStop } from "./native-local-process-stop.js"; +import { remoteExecutionHasStopped } from "./remote-execution-termination.js"; +import type { AuthorizationActor } from "./authorization.js"; +import type { EnvironmentRuntimeService } from "./environment-runtime.js"; + +const execFile = promisify(execFileCallback); +type Copy = typeof copies.$inferSelect; +const hash = (bytes: Uint8Array) => createHash("sha256").update(bytes).digest("hex"); +const quote = (value: string) => `'${value.replaceAll("'", `'"'"'`)}'`; +const completed = new Set(["saved", "unchanged", "resolved"]); +const MAX_COLLECTION_ATTEMPTS = 3; +// Only orchestration can register a live target; tools and HTTP callers cannot +// supply one. These handles are optional: durable captured bytes survive restart. +const liveTargets = new Map(); +const targetKey = (companyId: string, runId: string) => `${companyId}:${runId}`; + +export function instructionWorkingCopyGuidance(copy: Pick) { + if (isAgentDirectoryCopy(copy)) return `Your persistent agent directory is ${copy.executionRoot} (AGENT_HOME). Your instruction entry is ${copy.executionRoot}/${copy.entryFile}. Read and write your own files and subfolders there. This directory belongs to this agent across tasks and sessions; task files belong in the task working directory. Paperclip restores this directory before execution and saves changes after the provider stops. Regular files, including binary files, persist; symlinks and special files are unsupported. Check the agent-files save receipt before claiming persistence; Only files you change or delete are synchronized. If another run changes the same file, the last completed synchronization wins. Temporary copies are removed after synchronization; there is no per-run file history. Storage allows 256 MiB per file, 2 GiB total, and 100,000 entries; the instruction entry must remain UTF-8 and at most 1 MiB. Reaching a storage limit never prevents this or future tasks from running. Remove or shrink files to free space; changes that exceed the limits will not be saved.${typeof copy.receipt?.storageWarning === "string" ? `\n\n${copy.receipt.storageWarning}` : ""}`; + return `Your editable agent instruction file is ${copy.executionRoot}/${copy.entryFile}. Edit this registered private copy normally. After this run stops, Paperclip saves changed content as a persistent revision if your responsible user still has permission and the baseline has not changed. Check the run's instruction-save receipt before claiming persistence. Use read_agent_instructions, update_agent_instructions, get_agent_instruction_history, and restore_agent_instructions for immediate saves and history. Read first and pin the returned revision. Preserve conflicts; never silently retry against a newer head. Repository instructions, skills, and the loaded prompt are separate and are not collected.`; +} + +/** Remote reads use the registered entry only; never scan for files called AGENTS.md. */ +export function instructionCollectionScript(root: string, entry: string) { + instructionPath(entry); + return `node -e ${quote([ + "const fs=require('node:fs'),path=require('node:path')", + "const root=process.argv[1],entry=process.argv[2],max=Number(process.argv[3])", + "const absolute=path.resolve(root,entry),parts=absolute.split('/').filter(Boolean)", + "let current='/'", + "for(let i=0;imax)throw Error('INSTRUCTION_CONTENT_INVALID');const bytes=Buffer.alloc(max+1);let n=0;while(nmax)throw Error('INSTRUCTION_CONTENT_INVALID');process.stdout.write(JSON.stringify({contentBase64:bytes.subarray(0,n).toString('base64')}))}finally{fs.closeSync(fd)}", + ].join(";"))} ${quote(root)} ${quote(entry)} ${MAX_INSTRUCTION_BYTES}`; +} + + +export function agentInstructionWorkingCopyService(db: Db, options: { environmentRuntime?: EnvironmentRuntimeService } = {}) { + const revisions = agentInstructionRevisionService(db); + const scope = (companyId: string, runId: string) => and(eq(copies.companyId, companyId), eq(copies.runId, runId)); + async function get(companyId: string, runId: string) { + const [row] = await db.select().from(copies).where(scope(companyId, runId)); + return row ?? null; + } + async function patch(row: Copy, values: Partial) { + const [updated] = await db.update(copies).set({ ...values, updatedAt: new Date() }).where(and( + scope(row.companyId, row.runId), eq(copies.state, row.state), eq(copies.attempts, row.attempts), eq(copies.baseHash, row.baseHash), + )).returning(); + // A late cleanup must not overwrite an explicit resolution or a newer + // baseline. Canonical content has its own independent head CAS. + return updated ?? (await get(row.companyId, row.runId))!; + } + function actorFor(row: Copy): AuthorizationActor { + return { type: "agent", companyId: row.companyId, agentId: row.agentId, runId: row.runId, onBehalfOfUserId: row.responsibleUserId }; + } + const directories = agentDirectoryWorkingCopyService(db, get, patch, options.environmentRuntime); + async function prepare(input: { companyId: string; agentId: string; runId: string; target?: AdapterExecutionTarget | null; cwd: string; legacy?: boolean }) { + const existing = await get(input.companyId, input.runId); + if (isAgentDirectoryCopy(existing) || (!existing && !input.legacy)) return directories.prepare(input); + let refreshStoppedCopy = false; + const workspace = await fs.realpath(input.cwd); + const expectedLocalRoot = path.join(workspace, ".paperclip-runtime", `instruction-edits-${input.runId}`, "instructions"); + if (existing && existing.localRoot !== expectedLocalRoot) throw conflict("The registered instruction copy belongs to a different run workspace"); + const location = input.target?.kind === "remote" ? `remote:${input.target.environmentId ?? ""}` : "local"; + if (existing && (existing.agentId !== input.agentId || existing.location !== location)) { + throw conflict("The registered instruction copy belongs to a different run environment"); + } + // Preserve in-flight bytes and their CAS baseline. A completed, stopped copy + // can start a fresh lifecycle only after its recorded bytes are accounted for. + if (existing && existing.state !== "preparing") { + if (["conflict", "pending_commit", "unavailable"].includes(existing.state)) { + throw conflict("Resolve this run's preserved instruction candidate before editing its working copy again"); + } + if (input.target) liveTargets.set(targetKey(input.companyId, input.runId), input.target); + if (completed.has(existing.state) && existing.processStoppedAt) { + const privateBytes = await readCandidate(existing, input.target); + // Refresh only bytes whose previous lifecycle is complete. A changed + // private file must keep its old CAS fence; never silently rebase edits. + refreshStoppedCopy = privateBytes === null || hash(privateBytes) === (existing.candidateHash ?? existing.baseHash); + if (!refreshStoppedCopy && privateBytes !== null) { + // A failed staging attempt can already have replaced the file before + // the completed row advances. Exact current canonical bytes are also + // accounted for; any other private edit still keeps its old fence. + const current = await revisions.readCommittedForRuntime({ companyId: input.companyId, agentId: input.agentId }); + refreshStoppedCopy = current?.revision.entryFile === existing.entryFile + && current.revision.contentHash === hash(privateBytes); + } + } + if (!refreshStoppedCopy) { + if (completed.has(existing.state) || existing.state === "unchanged_turn") { + // unchanged_turn is a live warm-owner observation, not stop proof. + return patch(existing, { state: "prepared", candidateBase64: null, candidateHash: null, + receipt: null, processStoppedAt: null, attempts: 0, nextAttemptAt: null }); + } + return existing; + } + } + const [agent] = await db.select().from(agents).where(and(eq(agents.id, input.agentId), eq(agents.companyId, input.companyId))); + if (!agent) throw notFound("Agent not found"); + if (agentInstructionsBundleMode(agent) !== "managed") return null; + const bound = await resolveInstructionActor(db, { type: "agent", companyId: input.companyId, agentId: input.agentId, runId: input.runId }); + const baseline = existing?.baseRevisionId && !refreshStoppedCopy + ? await revisions.readRevision({ companyId: input.companyId, agentId: input.agentId, entryFile: existing.entryFile, revisionId: existing.baseRevisionId }, bound).catch(async (error) => { + if ((error as { status?: number }).status !== 404) throw error; + const current = await revisions.readCurrent({ companyId: input.companyId, agentId: input.agentId }, bound); + if (current?.revision.id !== existing.baseRevisionId) throw conflict("The instruction baseline changed during preparation"); + return current; + }) + : await revisions.readCurrent({ companyId: input.companyId, agentId: input.agentId }, bound); + if (!baseline) return null; + const exported = await agentInstructionsService().exportFiles(agent, { rejectSymlinks: true }); + const entryFile = instructionPath(baseline?.revision.entryFile ?? exported.entryFile); + const content = instructionBytes(baseline?.content ?? exported.files[entryFile] ?? ""); + const localRoot = existing?.localRoot ?? expectedLocalRoot; + const target = input.target?.kind === "remote" ? input.target : null; + const executionRoot = target + ? path.posix.join(target.remoteCwd, ".paperclip-runtime", `instruction-edits-${input.runId}`, "instructions") + : localRoot; + if (!existing) { + await db.insert(copies).values({ + runId: input.runId, companyId: input.companyId, agentId: input.agentId, + responsibleUserId: bound.onBehalfOfUserId!, entryFile, + baseRevisionId: baseline?.revision.id ?? null, baseHash: hash(content), + localRoot, executionRoot, location, state: "preparing", + }); + } + const row = (await get(input.companyId, input.runId))!; + await assertInstructionPathSafe(localRoot, entryFile); + await execFile(process.execPath, ["-e", instructionGitExcludeProgram, workspace], { timeout: 15_000 }); + await fs.mkdir(localRoot, { recursive: true, mode: 0o700 }); + for (const [name, text] of Object.entries({ ...exported.files, [entryFile]: content.toString("utf8") })) { + const relative = instructionPath(name); + const filename = await assertInstructionPathSafe(localRoot, relative); + await fs.mkdir(path.dirname(filename), { recursive: true, mode: 0o700 }); + // Atomic replacement also allows interrupted preparation to restage an + // already-read-only sibling without writing through its existing inode. + await materializeInstructionBytes(localRoot, relative, instructionBytes(text)); + await fs.chmod(filename, name === entryFile ? 0o600 : 0o400); + } + if (target) { + const staged = await prepareAdapterExecutionTargetRuntime({ + target, runId: input.runId, adapterKey: `instruction-edits-${input.runId}`, + workspaceLocalDir: input.cwd, syncWorkspace: false, + assets: [{ key: "instructions", localDir: localRoot, followSymlinks: false }], + }); + if (staged.assetDirs.instructions !== executionRoot) throw new Error("Instruction copy staging path changed"); + const excluded = await runAdapterExecutionTargetShellCommand(input.runId, target, + `node -e ${quote(instructionGitExcludeProgram)} ${quote(target.remoteCwd)}`, { cwd: target.remoteCwd, env: {}, timeoutSec: 15 }); + if (excluded.exitCode !== 0 || excluded.timedOut) throw new Error("Could not exclude private instructions from Git staging"); + } + // A refresh does not supersede the completed turn until every local and + // remote staging step succeeds. Keep its receipt and stop evidence intact + // if preparation throws or the controller stops midway through staging. + const prepared = await patch(row, { state: "prepared", ...(refreshStoppedCopy ? { + entryFile, baseRevisionId: baseline.revision.id, baseHash: hash(content), + candidateBase64: null, candidateHash: null, receipt: null, processStoppedAt: null, + attempts: 0, nextAttemptAt: null, errorCode: null, errorMessage: null, + } : {}) }); + if (input.target) liveTargets.set(targetKey(input.companyId, input.runId), input.target); + return prepared; + } + + async function readCandidate(row: Copy, target?: AdapterExecutionTarget | null) { + if (row.location === "local") return readInstructionBytes(row.localRoot, row.entryFile); + if (target?.kind !== "remote") throw new Error("The original execution environment is needed to retrieve this instruction copy"); + if (row.location !== `remote:${target.environmentId ?? ""}`) throw new Error("Instruction copy execution environment changed"); + const expected = path.posix.join(target.remoteCwd, ".paperclip-runtime", `instruction-edits-${row.runId}`, "instructions"); + if (expected !== row.executionRoot) throw new Error("Instruction copy execution environment changed"); + const result = await runAdapterExecutionTargetShellCommand(row.runId, target, instructionCollectionScript(row.executionRoot, row.entryFile), { cwd: target.remoteCwd, env: {}, timeoutSec: 30 }); + if (result.exitCode !== 0 || result.timedOut) throw new Error("Could not safely retrieve the stopped run's instruction file"); + const payload = JSON.parse(result.stdout) as { missing?: boolean; contentBase64?: string }; + if (payload.missing) return null; + if (typeof payload.contentBase64 !== "string" || payload.contentBase64.length > Math.ceil(MAX_INSTRUCTION_BYTES / 3) * 4) throw new Error("Instruction response exceeds the collection bound"); + return instructionBytes(Buffer.from(payload.contentBase64, "base64")); + } + + /** A terminal-turn probe is not a save: changed/invalid copies require a + * stopped provider before capture. Unchanged warm providers remain reusable. */ + async function hasChanges(input: { companyId: string; runId: string; target?: AdapterExecutionTarget | null }) { + const row = await get(input.companyId, input.runId); + if (!row || completed.has(row.state)) return false; + if (isAgentDirectoryCopy(row)) return directories.hasChanges(row, input.target); + try { + const content = await readCandidate(row, input.target); + if (content !== null && hash(content) === row.baseHash) { + // This is an unchanged turn receipt, not stopped-process collection. + // No candidate bytes or new revision are persisted from a live owner. + await patch(row, { state: "unchanged_turn", nextAttemptAt: null }); + return false; + } + return true; + } catch { return true; } + } + + async function acknowledgeExplicitSave(input: { companyId: string; agentId: string; runId: string; entryFile: string; revisionId: string; contentHash: string }) { + const row = await get(input.companyId, input.runId); + if (!row || isAgentDirectoryCopy(row) || row.agentId !== input.agentId || row.entryFile !== input.entryFile || row.state !== "prepared") return; + const target = liveTargets.get(targetKey(row.companyId, row.runId)); + const content = await readCandidate(row, target).catch(() => null); + // Never advance a stale local edit past an unrelated explicit update. + if (content === null || hash(content) !== input.contentHash) return; + await db.update(copies).set({ baseRevisionId: input.revisionId, baseHash: input.contentHash, updatedAt: new Date() }) + .where(and(scope(row.companyId, row.runId), eq(copies.state, "prepared"), eq(copies.baseHash, row.baseHash))); + } + + async function commitCandidate(row: Copy) { + if (isAgentDirectoryCopy(row)) return directories.collectStopped(row); + if (row.candidateBase64 === null || completed.has(row.state) || row.state === "conflict") return row; + try { + const receipt = await revisions.commit({ + companyId: row.companyId, agentId: row.agentId, entryFile: row.entryFile, + baseRevisionId: row.baseRevisionId, content: Buffer.from(row.candidateBase64, "base64"), source: "cleanup", + }, actorFor(row)); + return patch(row, { state: "saved", receipt: { ...receipt }, errorCode: null, errorMessage: null, nextAttemptAt: null }); + } catch (error) { + const detail = error as { status?: number; details?: { code?: string }; message?: string }; + const code = detail.details?.code ?? "INSTRUCTION_SAVE_FAILED"; + const retryable = !detail.status || detail.status >= 500; + const message = detail.status === 403 + ? "Current permissions do not allow this instruction save. The candidate was preserved." + : detail.status === 409 + ? "Instructions changed after this run started. Review the preserved candidate against the current revision." + : "Instruction edits were preserved but not saved. Review the candidate before retrying."; + return patch(row, { state: retryable ? "pending_commit" : "conflict", errorCode: code, errorMessage: message, nextAttemptAt: retryable && row.attempts < MAX_COLLECTION_ATTEMPTS ? new Date(Date.now() + 30_000) : null }); + } + } + + /** Caller must have joined the owned provider process before granting this proof. */ + async function collectStopped(input: { companyId: string; runId: string; target?: AdapterExecutionTarget | null }) { + let row = await get(input.companyId, input.runId); + if (!row) return row; + if (isAgentDirectoryCopy(row)) return directories.collectStopped(row, input.target); + if (completed.has(row.state) || row.state === "conflict") return row; + row = await patch(row, { processStoppedAt: row.processStoppedAt ?? new Date(), attempts: row.attempts + 1 }); + if (completed.has(row.state) || row.state === "conflict") return row; + if (row.candidateBase64 !== null) return commitCandidate(row); + try { + const candidate = await readCandidate(row, input.target); + if (candidate === null) { + return patch(row, { state: "unavailable", errorCode: "INSTRUCTION_FILE_MISSING", errorMessage: "The registered instruction file was removed. Canonical instructions were kept; no edits were saved.", nextAttemptAt: null }); + } + if (hash(candidate) === row.baseHash) return patch(row, { state: "unchanged", nextAttemptAt: null }); + // Save the candidate durably before attempting authorization/CAS or releasing the environment. + row = await patch(row, { state: "pending_commit", candidateBase64: candidate.toString("base64"), candidateHash: hash(candidate), nextAttemptAt: new Date() }); + return commitCandidate(row); + } catch (error) { + return patch(row, { state: row.attempts < MAX_COLLECTION_ATTEMPTS ? "pending_collection" : "unavailable", errorCode: "INSTRUCTION_COLLECTION_FAILED", errorMessage: "The registered instruction copy could not be read safely. Canonical instructions were kept; no save receipt exists.", nextAttemptAt: row.attempts < MAX_COLLECTION_ATTEMPTS ? new Date(Date.now() + 30_000) : null }); + } + } + + /** Restarts can retry persisted bytes without touching a live filesystem or launching a model. */ + async function recoverCaptured() { + const pending = await db.select().from(copies).where(and( + eq(copies.state, "pending_commit"), + or(isNull(copies.nextAttemptAt), lte(copies.nextAttemptAt, new Date())), + lte(copies.attempts, MAX_COLLECTION_ATTEMPTS - 1), + )).limit(20); + for (const row of pending) { + const result = await commitCandidate(await patch(row, { attempts: row.attempts + 1 })); + if (isAgentDirectoryCopy(result) && completed.has(result.state)) await directories.release(result); + } + // A crash between a durable save and release must not retain run snapshots + // forever. Only discard copies with recorded stop proof and a terminal receipt. + const cleanup = await db.select().from(copies).where(and( + inArray(copies.state, [...completed, "unavailable"]), isNotNull(copies.processStoppedAt), + sql`${copies.receipt}->>'schema' = 'paperclip.agent-files.v1'`, + or(sql`${copies.receipt} ? 'baseline'`, sql`${copies.receipt}->>'cleanupPending' = 'true'`), + or(isNull(copies.nextAttemptAt), lte(copies.nextAttemptAt, new Date())), + )).orderBy(asc(copies.updatedAt)).limit(20); + for (const row of cleanup) await directories.release(row); + return pending.length; + } + + /** Recover only the registered copy after independently recorded process-stop + * proof. Terminal run status is never a substitute for that proof. Remote + * receipts mean the environment is stopped; do not restart it just to read. */ + async function recoverStopped() { + const pending = await db.select({ copy: copies, runtimeMode: heartbeatRuns.runtimeMode }).from(copies) + .innerJoin(heartbeatRuns, and(eq(heartbeatRuns.companyId, copies.companyId), eq(heartbeatRuns.id, copies.runId))) + .where(and(or(inArray(copies.state, ["prepared", "pending_collection"]), + and(eq(copies.state, "preparing"), sql`${copies.receipt}->>'schema' = 'paperclip.agent-files.v1'`)), + inArray(heartbeatRuns.status, ["succeeded", "failed", "cancelled", "timed_out", "interrupted"]), + lte(copies.attempts, MAX_COLLECTION_ATTEMPTS - 1))).limit(20); + for (const { copy: row, runtimeMode } of pending) { + if (row.state === "preparing" && isAgentDirectoryCopy(row)) { + // A provider cannot launch until preparation records "prepared". With + // the owning run terminal, this is an interrupted staging copy only. + await directories.release(await patch(row, { state: "unavailable", processStoppedAt: new Date(), + errorCode: "AGENT_FILES_PREPARE_FAILED", errorMessage: "Agent-file preparation was interrupted. The temporary copy was discarded.", nextAttemptAt: null })); + } else if (row.location === "local" && (row.processStoppedAt || runtimeMode === "native" && + await hasNativeLocalProcessStop(db, row.companyId, row.runId))) { + const result = await collectStopped({ companyId: row.companyId, runId: row.runId }); + if (result && isAgentDirectoryCopy(result) && completed.has(result.state)) await directories.release(result); + } else if (row.location !== "local" && await remoteExecutionHasStopped(db, row.companyId, row.runId)) { + const unavailable = await reportUnavailable(row.companyId, row.runId); + if (unavailable && isAgentDirectoryCopy(unavailable)) { + await directories.release(await patch(unavailable, { processStoppedAt: unavailable.processStoppedAt ?? new Date() })); + } + } else if (row.state === "prepared") { + await patch(row, { state: "pending_collection", errorCode: "INSTRUCTION_STOP_UNCONFIRMED", + errorMessage: "The provider's stop has not been confirmed. Instruction collection is pending; no save is claimed.", nextAttemptAt: null }); + } + } + return pending.length; + } + + async function list(companyId: string, agentId: string, actor: AuthorizationActor) { + await revisions.readCurrent({ companyId, agentId }, actor); + const rows = await db.select().from(copies).where(and(eq(copies.companyId, companyId), eq(copies.agentId, agentId), inArray(copies.state, ["conflict", "pending_collection", "pending_commit", "unavailable"]))).orderBy(desc(copies.createdAt)).limit(50); + return rows.map(row => ({ runId: row.runId, entryFile: row.entryFile, baseRevisionId: row.baseRevisionId, + baseHash: row.baseHash, state: row.state as "conflict" | "pending_collection" | "pending_commit" | "unavailable", + candidateHash: row.candidateHash, contract: isAgentDirectoryCopy(row) ? "agent_files" : "legacy", + content: row.candidateBase64 === null ? null : Buffer.from(row.candidateBase64, "base64").toString("utf8"), + errorCode: row.errorCode, errorMessage: row.errorMessage, createdAt: row.createdAt.toISOString(), updatedAt: row.updatedAt.toISOString() })); + } + + async function resolve(input: { companyId: string; agentId: string; runId: string; baseRevisionId: string | null; content: string }, actor: AuthorizationActor) { + const row = await get(input.companyId, input.runId); + if (!row || row.agentId !== input.agentId) throw notFound("Instruction candidate not found"); + if (completed.has(row.state)) throw conflict("This instruction candidate has already been resolved"); + if (isAgentDirectoryCopy(row)) throw conflict("Agent-file synchronization failures have no preserved candidate"); + const receipt = await revisions.commit({ companyId: input.companyId, agentId: input.agentId, entryFile: row.entryFile, content: input.content, baseRevisionId: input.baseRevisionId, source: "api" }, actor); + await patch(row, { state: "resolved", receipt: { ...receipt }, nextAttemptAt: null }); + return receipt; + } + + /** A controller may report loss before disposal, never fabricate a save or + * read a possibly live provider. Already captured candidates remain intact. */ + async function reportUnavailable(companyId: string, runId: string) { + const row = await get(companyId, runId); + if (!row || completed.has(row.state) || row.state === "unchanged_turn" || row.candidateBase64 !== null || (isAgentDirectoryCopy(row) && row.candidateHash !== null) || row.state === "conflict") return row; + if (isAgentDirectoryCopy(row) && row.state === "unavailable") return row; + return patch(row, { state: "unavailable", errorCode: "INSTRUCTION_COLLECTION_UNAVAILABLE", + errorMessage: "The registered instruction copy could not be retrieved safely before environment release. No instruction save is claimed.", nextAttemptAt: null }); + } + + async function release(companyId: string, runId: string) { liveTargets.delete(targetKey(companyId, runId)); const row = await get(companyId, runId); if (row && isAgentDirectoryCopy(row)) await directories.release(row); } + return { prepare, get, hasChanges, acknowledgeExplicitSave, collectStopped, recoverCaptured, recoverStopped, list, resolve, reportUnavailable, release }; +} diff --git a/server/src/services/agent-instructions.ts b/server/src/services/agent-instructions.ts index 5045143eb7..4f66a27545 100644 --- a/server/src/services/agent-instructions.ts +++ b/server/src/services/agent-instructions.ts @@ -1,5 +1,9 @@ +import { inspectAgentFile, fileHash, agentFilePath, MAX_AGENT_FILE_BYTES } from "./agent-file-store.js"; import fs from "node:fs/promises"; import path from "node:path"; +import { and, eq } from "drizzle-orm"; +import { agentInstructionHeads, agents, type Db } from "@paperclipai/db"; +import { instructionPath, assertInstructionPathSafe, instructionBytes, readInstructionBytes } from "./agent-instruction-files.js"; import { notFound, unprocessable } from "../errors.js"; import { resolveHomeAwarePath, resolvePaperclipInstanceRoot } from "../home-paths.js"; @@ -36,6 +40,8 @@ type AgentLike = { type AgentInstructionsFileSummary = { path: string; + contentHash?: string; + binary?: boolean; size: number; language: string; markdown: boolean; @@ -53,6 +59,7 @@ type AgentInstructionsFileDetail = AgentInstructionsFileSummary & { type AgentInstructionsBundle = { agentId: string; companyId: string; + persistence?: "agent_files"; mode: BundleMode | null; rootPath: string | null; managedRootPath: string; @@ -112,11 +119,7 @@ function isMarkdown(relativePath: string) { } function normalizeRelativeFilePath(candidatePath: string): string { - const normalized = path.posix.normalize(candidatePath.replaceAll("\\", "/")).replace(/^\/+/, ""); - if (!normalized || normalized === "." || normalized === ".." || normalized.startsWith("../")) { - throw unprocessable("Instructions file path must stay within the bundle root"); - } - return normalized; + return instructionPath(candidatePath); } function resolvePathWithinRoot(rootPath: string, relativePath: string): string { @@ -130,7 +133,7 @@ function resolvePathWithinRoot(rootPath: string, relativePath: string): string { return absolutePath; } -function resolveManagedInstructionsRoot(agent: AgentLike): string { +export function resolveManagedInstructionsRoot(agent: AgentLike): string { return path.resolve( resolvePaperclipInstanceRoot(), "companies", @@ -153,7 +156,7 @@ function resolveLegacyInstructionsPath(candidatePath: string, config: Record null); + return fs.lstat(targetPath).catch(() => null); } function shouldIgnoreInstructionsEntry(entry: { name: string; isDirectory(): boolean; isFile(): boolean }) { @@ -172,14 +175,14 @@ function shouldIgnoreInstructionsEntry(entry: { name: string; isDirectory(): boo async function listFilesRecursive( rootPath: string, - options?: { rejectSymlinks?: boolean }, + options?: { rejectSymlinks?: boolean; legacyExcludes?: boolean }, ): Promise { const output: string[] = []; async function walk(currentPath: string, relativeDir: string) { const entries = await fs.readdir(currentPath, { withFileTypes: true }).catch(() => []); for (const entry of entries) { - if (shouldIgnoreInstructionsEntry(entry)) continue; + if (options?.legacyExcludes && shouldIgnoreInstructionsEntry(entry)) continue; const absolutePath = path.join(currentPath, entry.name); const relativePath = normalizeRelativeFilePath( relativeDir ? path.posix.join(relativeDir, entry.name) : entry.name, @@ -206,13 +209,21 @@ async function listFilesRecursive( async function readFileSummary(rootPath: string, relativePath: string, entryFile: string): Promise { const absolutePath = resolvePathWithinRoot(rootPath, relativePath); const stat = await fs.stat(absolutePath); + // External bundles may contain large assets; listing must not read them. + const file = stat.size > MAX_AGENT_FILE_BYTES ? null : await inspectAgentFile(rootPath, relativePath); + return summarizeFile(relativePath, entryFile, file?.size ?? stat.size, file?.bytes ?? null, file?.hash); +} + +function summarizeFile(relativePath: string, entryFile: string, size: number, bytes: Buffer | null, hash?: string): AgentInstructionsFileSummary { + let binary = bytes === null; + try { if (bytes?.includes(0)) binary = true; new TextDecoder("utf-8", { fatal: true }).decode(bytes ?? undefined); } catch { binary = true; } return { path: relativePath, - size: stat.size, + size, language: inferLanguage(relativePath), markdown: isMarkdown(relativePath), isEntryFile: relativePath === entryFile, - editable: true, + editable: !binary, binary, contentHash: hash, deprecated: false, virtual: false, }; @@ -231,7 +242,7 @@ async function readLegacyInstructions(agent: AgentLike, config: Record[0]>[0] | undefined = db) { + if (!connection) throw unprocessable("Bundle initialization requires the database-backed instructions service"); + const [head] = await connection.select().from(agentInstructionHeads).where(and(eq(agentInstructionHeads.companyId, agent.companyId), + eq(agentInstructionHeads.agentId, agent.id), eq(agentInstructionHeads.entryFile, entryFile))); + if (head) throw unprocessable("This entry has revision history. Use the instruction content API with its baseRevisionId to update it.", { code: "INSTRUCTION_REVISION_REQUIRED" }); + } async function getBundle(agent: AgentLike): Promise { const state = await recoverManagedBundleState(agent, deriveBundleState(agent)); if (!state.rootPath) return toBundle(agent, state, []); + await assertInstructionPathSafe(state.rootPath, state.entryFile); const stat = await statIfExists(state.rootPath); if (!stat?.isDirectory()) { return toBundle(agent, { @@ -484,8 +506,12 @@ export function agentInstructionsService() { warnings: [...state.warnings, `Instructions root does not exist: ${state.rootPath}`], }, []); } - const files = await listFilesRecursive(state.rootPath); - const summaries = await Promise.all(files.map((relativePath) => readFileSummary(state.rootPath!, relativePath, state.entryFile))); + const files = await listFilesRecursive(state.rootPath, { legacyExcludes: state.mode === "external" }); + const summaries: AgentInstructionsFileSummary[] = []; + // Bound open descriptors and text buffers even for a large personal folder. + for (let index = 0; index < files.length; index += 8) { + summaries.push(...await Promise.all(files.slice(index, index + 8).map(relativePath => readFileSummary(state.rootPath!, relativePath, state.entryFile)))); + } return toBundle(agent, state, summaries); } @@ -507,24 +533,11 @@ export function agentInstructionsService() { }; } if (!state.rootPath) throw notFound("Agent instructions bundle is not configured"); - const absolutePath = resolvePathWithinRoot(state.rootPath, relativePath); - const [content, stat] = await Promise.all([ - fs.readFile(absolutePath, "utf8").catch(() => null), - fs.stat(absolutePath).catch(() => null), - ]); - if (content === null || !stat?.isFile()) throw notFound("Instructions file not found"); - const normalizedPath = normalizeRelativeFilePath(relativePath); - return { - path: normalizedPath, - size: stat.size, - language: inferLanguage(normalizedPath), - markdown: isMarkdown(normalizedPath), - isEntryFile: normalizedPath === state.entryFile, - editable: true, - deprecated: false, - virtual: false, - content, - }; + await assertInstructionPathSafe(state.rootPath, relativePath); + const file = await inspectAgentFile(state.rootPath, relativePath); + if (file === null) throw notFound("Instructions file not found"); + const summary = summarizeFile(relativePath, state.entryFile, file.size, file.bytes, file.hash); + return { ...summary, content: summary.binary ? "" : file.bytes!.toString("utf8") }; } async function ensureWritableBundle( @@ -551,11 +564,12 @@ export function agentInstructionsService() { }); await fs.mkdir(managedRoot, { recursive: true }); - const entryPath = resolvePathWithinRoot(managedRoot, entryFile); + const entryPath = await assertInstructionPathSafe(managedRoot, entryFile); const entryStat = await statIfExists(entryPath); if (!entryStat?.isFile()) { const legacyInstructions = await readLegacyInstructions(agent, current.config); if (legacyInstructions.trim().length > 0) { + await assertUnversionedEntry(agent, entryFile); await fs.mkdir(path.dirname(entryPath), { recursive: true }); await fs.writeFile(entryPath, legacyInstructions, "utf8"); } @@ -595,9 +609,13 @@ export function agentInstructionsService() { nextRootPath = resolvedRoot; } + await assertInstructionPathSafe(nextRootPath, nextEntryFile); await fs.mkdir(nextRootPath, { recursive: true }); const existingFiles = await listFilesRecursive(nextRootPath); + if (nextMode === "managed" && !existingFiles.includes(nextEntryFile)) { + await assertUnversionedEntry(agent, nextEntryFile); + } const exported = await exportFiles(agent); if (existingFiles.length === 0) { await writeBundleFiles(nextRootPath, exported.files); @@ -618,7 +636,7 @@ export function agentInstructionsService() { return { bundle: nextBundle, adapterConfig: nextConfig }; } - async function writeFile( + async function writeFileUnversioned( agent: AgentLike, relativePath: string, content: string, @@ -642,7 +660,14 @@ export function agentInstructionsService() { return { bundle, file, adapterConfig }; } + const configured = await recoverManagedBundleState(agent, current); + if (normalizeRelativeFilePath(relativePath) === configured.entryFile) { + throw unprocessable("Entry edits require the canonical instruction commit service and baseRevisionId", { code: "INSTRUCTION_REVISION_REQUIRED" }); + } + if (configured.mode !== "external") agentFilePath(relativePath); const prepared = await ensureWritableBundle(agent, options); + instructionBytes(content); + await assertInstructionPathSafe(prepared.state.rootPath!, relativePath); const absolutePath = resolvePathWithinRoot(prepared.state.rootPath!, relativePath); await fs.mkdir(path.dirname(absolutePath), { recursive: true }); await fs.writeFile(absolutePath, content, "utf8"); @@ -654,7 +679,7 @@ export function agentInstructionsService() { return { bundle, file, adapterConfig: prepared.adapterConfig }; } - async function deleteFile(agent: AgentLike, relativePath: string): Promise<{ + async function deleteFileUnversioned(agent: AgentLike, relativePath: string): Promise<{ bundle: AgentInstructionsBundle; adapterConfig: Record; }> { @@ -668,13 +693,35 @@ export function agentInstructionsService() { if (normalizedPath === state.entryFile) { throw unprocessable("Cannot delete the bundle entry file"); } - const absolutePath = resolvePathWithinRoot(state.rootPath, normalizedPath); + const absolutePath = await assertInstructionPathSafe(state.rootPath, normalizedPath); await fs.rm(absolutePath, { force: true }); const adapterConfig = buildPersistedBundleConfig(derived, state); const bundle = await getBundle({ ...agent, adapterConfig }); return { bundle, adapterConfig }; } + // Reload configuration under the commit lock: a previously supporting file + // may have become the configured entry since the route loaded its agent. + async function withCurrentAgent(agent: AgentLike, relativePath: string, operation: (current: AgentLike) => Promise) { + if (!db) return operation(agent); + return db.transaction(async (tx) => { + const [current] = await tx.select().from(agents) + .where(and(eq(agents.id, agent.id), eq(agents.companyId, agent.companyId))).for("update"); + if (!current) throw notFound("Agent not found"); + if (relativePath !== LEGACY_PROMPT_TEMPLATE_PATH) { + // Historical entries also remain projections if the configured entry changes. + await assertUnversionedEntry(current, normalizeRelativeFilePath(relativePath), tx); + } + return operation(current); + }); + } + async function writeFile(...args: Parameters) { + return withCurrentAgent(args[0], args[1], (current) => writeFileUnversioned(current, args[1], args[2], args[3])); + } + async function deleteFile(...args: Parameters) { + return withCurrentAgent(args[0], args[1], (current) => deleteFileUnversioned(current, args[1])); + } + async function exportFiles(agent: AgentLike, options?: { rejectSymlinks?: boolean }): Promise<{ files: Record; entryFile: string; @@ -684,7 +731,7 @@ export function agentInstructionsService() { if (state.rootPath) { const stat = await statIfExists(state.rootPath); if (stat?.isDirectory()) { - const relativePaths = await listFilesRecursive(state.rootPath, options); + const relativePaths = await listFilesRecursive(state.rootPath, { ...options, legacyExcludes: true }); const files = Object.fromEntries(await Promise.all(relativePaths.map(async (relativePath) => { const absolutePath = resolvePathWithinRoot(state.rootPath!, relativePath); const content = await fs.readFile(absolutePath, "utf8"); @@ -704,7 +751,7 @@ export function agentInstructionsService() { }; } - async function materializeManagedBundle( + async function materializeManagedBundleUnversioned( agent: AgentLike, files: Record, options?: { @@ -716,6 +763,14 @@ export function agentInstructionsService() { const rootPath = resolveManagedInstructionsRoot(agent); const entryFile = options?.entryFile ? normalizeRelativeFilePath(options.entryFile) : ENTRY_FILE_DEFAULT; + for (const [relativePath, content] of Object.entries(files)) { + instructionBytes(content); + await assertInstructionPathSafe(rootPath, agentFilePath(relativePath)); + } + const previous = await readInstructionBytes(rootPath, entryFile); + if (previous && !previous.equals(instructionBytes(files[entryFile] ?? ""))) { + throw unprocessable("Existing entry content must be saved through the canonical revision API before replacing a bundle", { code: "INSTRUCTION_REVISION_REQUIRED" }); + } if (options?.replaceExisting) { await fs.rm(rootPath, { recursive: true, force: true }); } @@ -744,6 +799,24 @@ export function agentInstructionsService() { return { bundle, adapterConfig }; } + async function materializeManagedBundle( + ...args: Parameters + ) { + if (!db) throw unprocessable("Bundle initialization requires the database-backed instructions service"); + return db.transaction(async (tx) => { + const agent = args[0]; + const [owner] = await tx.select({ id: agents.id }).from(agents).where(and(eq(agents.id, agent.id), eq(agents.companyId, agent.companyId))).for("update"); + if (!owner) throw notFound("Agent not found"); + await assertUnversionedEntry(agent, args[2]?.entryFile ? normalizeRelativeFilePath(args[2].entryFile) : ENTRY_FILE_DEFAULT, tx); + if (args[2]?.replaceExisting) { + const [existingHead] = await tx.select({ id: agentInstructionHeads.revisionId }).from(agentInstructionHeads) + .where(and(eq(agentInstructionHeads.companyId, agent.companyId), eq(agentInstructionHeads.agentId, agent.id))).limit(1); + if (existingHead) throw unprocessable("Replacing this bundle would remove a versioned entry. Use canonical content commits.", { code: "INSTRUCTION_REVISION_REQUIRED" }); + } + return materializeManagedBundleUnversioned(...args); + }); + } + return { getBundle, readFile, diff --git a/server/src/services/authorization.ts b/server/src/services/authorization.ts index 70b1d2032c..ed1719e028 100644 --- a/server/src/services/authorization.ts +++ b/server/src/services/authorization.ts @@ -62,6 +62,7 @@ export type AuthorizationAction = | PermissionKey | "agent_config:read" | "agent_config:update" + | "agent_instructions:update" | "skill_config:update" | "agent:read" | "agent:wake" @@ -148,7 +149,7 @@ function companyIdForResource(resource: AuthorizationResource) { } function permissionForAction(action: AuthorizationAction): PermissionKey | null { - if (action === "agent_config:read" || action === "agent_config:update" || action === "skill_config:update") { + if (action === "agent_config:read" || action === "agent_config:update" || action === "agent_instructions:update" || action === "skill_config:update") { return null; } if ( @@ -998,6 +999,7 @@ export function authorizationService(db: Db | DbTransaction) { input.action === "decision_triage:manage" || input.action === "agent_config:read" || input.action === "agent_config:update" || + input.action === "agent_instructions:update" || input.action === "skill_config:update" || input.action === "inbox:manage" || input.action === "runtime:manage" || @@ -1734,7 +1736,7 @@ export function authorizationService(db: Db | DbTransaction) { if (input.action === "agent_config:read") { return decideWithAgentConfigReadGrant("user", input.actor.userId); } - if (input.action === "agent_config:update") { + if (input.action === "agent_config:update" || input.action === "agent_instructions:update") { return decideWithProtectedChangeGrants("user", input.actor.userId, { direct: "agents:configure", suggest: "agents:suggest-changes", @@ -2227,6 +2229,19 @@ export function authorizationService(db: Db | DbTransaction) { return decideWithAgentConfigReadGrant("agent", actorAgentId); } + if (input.action === "agent_instructions:update") { + if (!isSimpleAssignableAgentStatus(actorAgent.status) || !input.actor.onBehalfOfUserId) { + return deny({ action: input.action, reason: "deny_missing_membership", explanation: "Instruction edits require an active agent and a responsible user." }); + } + // Explicit configure/suggest restrictions still govern content changes. + // Only self edits may fall back to the responsible user's target access. + const restricted = await decideWithProtectedChangeGrants("agent", actorAgentId, { + direct: "agents:configure", suggest: "agents:suggest-changes", + }); + if (restricted.reason !== "deny_no_grant" || input.resource.type !== "agent" || input.resource.agentId !== actorAgentId) return restricted; + return allow({ action: input.action, reason: "allow_self", explanation: "Own instruction content edit, subject to the responsible user's target edit access." }); + } + if (input.action === "agent_config:update") { return decideWithProtectedChangeGrants("agent", actorAgentId, { direct: "agents:configure", @@ -2391,7 +2406,7 @@ export function authorizationService(db: Db | DbTransaction) { responsibleUserId, }, "responsible-user authorization intersection denied"); - return responsibleUserAuthzShadowMode() ? agentDecision : denied; + return input.action !== "agent_instructions:update" && responsibleUserAuthzShadowMode() ? agentDecision : denied; } async function decide(input: { diff --git a/server/src/services/built-in-agents.ts b/server/src/services/built-in-agents.ts index 718263668e..1b5f70308d 100644 --- a/server/src/services/built-in-agents.ts +++ b/server/src/services/built-in-agents.ts @@ -1,3 +1,4 @@ +import fs from "node:fs/promises"; import { readFileSync } from "node:fs"; import { createRequire } from "node:module"; import path from "node:path"; @@ -5,12 +6,16 @@ import { fileURLToPath } from "node:url"; import { readPaperclipSkillSyncPreference, writePaperclipSkillSyncPreference } from "@paperclipai/adapter-utils/server-utils"; import { and, desc, eq, ne } from "drizzle-orm"; import type { Db } from "@paperclipai/db"; -import { agents, builtInManagedResources, companies, issueThreadInteractions, issues, routines, routineTriggers } from "@paperclipai/db"; +import { activityLog, agents, builtInManagedResources, companies, issueThreadInteractions, issues, routines, routineTriggers } from "@paperclipai/db"; import { syncRoutineVariablesWithTemplate } from "@paperclipai/shared"; import type { Agent, Approval, CompanySkill, PermissionKey, Routine, RoutineTrigger, RoutineVariable } from "@paperclipai/shared"; import { conflict, HttpError, notFound, unprocessable } from "../errors.js"; import { logActivity } from "./activity-log.js"; -import { agentInstructionsService } from "./agent-instructions.js"; +import { adoptAgentFiles, agentFilePath, fileHash, readAgentFile, snapshotAgentFiles } from "./agent-file-store.js"; +import { assertInstructionPathSafe, instructionBytes, materializeInstructionBytes } from "./agent-instruction-files.js"; +import { agentInstructionRevisionService } from "./agent-instruction-revisions.js"; +import type { AuthorizationActor } from "./authorization.js"; +import { agentInstructionsBundleMode, agentInstructionsService } from "./agent-instructions.js"; import { agentService } from "./agents.js"; import { approvalService } from "./approvals.js"; import { @@ -802,7 +807,7 @@ export function builtInAgentService(db: Db) { const agentSvc = agentService(db); const accessSvc = accessService(db); const approvalSvc = approvalService(db); - const instructionsSvc = agentInstructionsService(); + const instructionsSvc = agentInstructionsService(db); const skillSvc = companySkillService(db); const routineSvc = routineService(db); @@ -916,9 +921,9 @@ export function builtInAgentService(db: Db) { stockVersion: string; stockHash: string; defaultsJson: Record; - }) { + }, connection: Db | Parameters[0]>[0] = db) { const now = new Date(); - return db + return connection .insert(builtInManagedResources) .values(input) .onConflictDoUpdate({ @@ -952,10 +957,15 @@ export function builtInAgentService(db: Db) { return currentFiles; } - async function materializeInstructions(agent: Agent, definition: BuiltInAgentDefinition, mode: "reconcile" | "reset") { + async function materializeInstructions(agent: Agent, definition: BuiltInAgentDefinition, mode: "reconcile" | "reset", actor?: AuthorizationActor) { const bundle = definition.bundle!; const stock = stockHash(bundle.instructions.files); const binding = await getManagedResourceBinding(agent.companyId, definition.key, "instructions", "AGENTS.md"); + // Stock operations also read the committed projection after interrupted writes. + if (agentInstructionsBundleMode(agent) === "managed") { + try { await agentInstructionRevisionService(db).materializeCurrent({ companyId: agent.companyId, agentId: agent.id }); } + catch (error) { if (!(error instanceof HttpError && error.status === 404)) throw error; } + } const currentFiles = await currentInstructionFiles(agent, bundle); const currentHash = Object.values(currentFiles).some((value) => value === null) ? null : stockHash(currentFiles); const currentState = stockState({ @@ -969,6 +979,100 @@ export function builtInAgentService(db: Db) { changedFiles: changedFileList(currentFiles, bundle.instructions.files), }); + if (!actor && mode === "reconcile" && agentInstructionsBundleMode(agent) === "managed" + && binding && (binding.stockHash !== stock || binding.defaultsJson.pendingInstructionsUpdate)) { + const bindingWhere = and(eq(builtInManagedResources.companyId, agent.companyId), + eq(builtInManagedResources.bundleKey, definition.key), eq(builtInManagedResources.resourceKind, "instructions"), + eq(builtInManagedResources.resourceKey, "AGENTS.md")); + const incoming = Object.fromEntries(Object.entries(bundle.instructions.files) + .map(([file, content]) => [agentFilePath(file), instructionBytes(content)])); + // Persist only the pending operation's baseline hashes before touching + // files. If a write or DB commit fails, the next reconciliation can accept + // already-applied bytes and retry. This is not instruction revision history. + const prepared = await db.transaction(async (tx) => { + const [current] = await tx.select().from(agents).where(and(eq(agents.companyId, agent.companyId), eq(agents.id, agent.id))).for("update"); + if (!current) throw notFound("Built-in agent not found"); + if (agentInstructionsBundleMode(current) !== "managed") return false; + const root = await adoptAgentFiles(tx, current); + const [latest] = await tx.select().from(builtInManagedResources).where(bindingWhere); + if (!latest) return false; + const pending = latest.defaultsJson.pendingInstructionsUpdate as { + stockHash?: string; baseHashes?: Record; nextHashes?: Record; + } | undefined; + if (pending?.stockHash === stock) return true; + if (!pending && latest.stockHash === stock) return false; + const previousPaths = latest.defaultsJson.files; + if (!Array.isArray(previousPaths) || !previousPaths.every((file): file is string => typeof file === "string")) return false; + const ownedPaths = new Set(previousPaths); + if (pending) { + // A newer release (or rollback) can supersede an interrupted update. + // Recognize only the previous operation's baseline or intended bytes; + // never mistake an intervening operator edit for partially applied stock. + if (!pending.baseHashes || !pending.nextHashes) return false; + for (const [file, baseHash] of Object.entries(pending.baseHashes)) { + if (!(file in pending.nextHashes)) return false; + const bytes = await readAgentFile(root, agentFilePath(file)); + const hash = bytes === null ? null : fileHash(bytes); + if (hash !== baseHash && hash !== pending.nextHashes[file]) return false; + ownedPaths.add(file); + } + } else { + const previousFiles: Record = {}; + for (const file of previousPaths) { + try { previousFiles[file] = (await instructionsSvc.readFile(current, file)).content; } + catch { previousFiles[file] = null; } + } + if (stockHash(previousFiles) !== latest.stockHash) return false; + } + await snapshotAgentFiles(root); + const baseHashes: Record = {}; + const nextHashes: Record = {}; + for (const file of new Set([...ownedPaths, ...Object.keys(incoming)])) { + const bytes = await readAgentFile(root, agentFilePath(file)); + // A newly declared stock path must not overwrite a personal file. + if (!ownedPaths.has(file) && bytes !== null && !bytes.equals(incoming[file]!)) return false; + baseHashes[file] = bytes === null ? null : fileHash(bytes); + nextHashes[file] = incoming[file] ? fileHash(incoming[file]) : null; + } + await tx.update(builtInManagedResources).set({ defaultsJson: { ...latest.defaultsJson, + pendingInstructionsUpdate: { stockHash: stock, baseHashes, nextHashes } }, updatedAt: new Date() }).where(bindingWhere); + return true; + }); + if (!prepared) return currentState; + return db.transaction(async (tx) => { + const [current] = await tx.select().from(agents).where(and(eq(agents.companyId, agent.companyId), eq(agents.id, agent.id))).for("update"); + if (!current) throw notFound("Built-in agent not found"); + if (agentInstructionsBundleMode(current) !== "managed") return currentState; + const root = await adoptAgentFiles(tx, current); + const [latest] = await tx.select().from(builtInManagedResources).where(bindingWhere); + const pending = latest?.defaultsJson.pendingInstructionsUpdate as { stockHash?: string; baseHashes?: Record } | undefined; + if (pending?.stockHash !== stock || !pending.baseHashes) return currentState; + await snapshotAgentFiles(root); + // Preflight every changed or removed path before mutation; preserve any + // operator edit made between preparation, failure, and this retry. + for (const [file, baseHash] of Object.entries(pending.baseHashes)) { + const bytes = await readAgentFile(root, agentFilePath(file)); + const hash = bytes === null ? null : fileHash(bytes); + const nextHash = incoming[file] ? fileHash(incoming[file]) : null; + if (hash !== baseHash && hash !== nextHash) return currentState; + } + for (const [file, bytes] of Object.entries(incoming)) await materializeInstructionBytes(root, file, bytes); + for (const file of Object.keys(pending.baseHashes)) { + if (!(file in incoming)) await fs.rm(await assertInstructionPathSafe(root, file), { force: true }); + } + await upsertManagedResourceBinding({ companyId: agent.companyId, bundleKey: definition.key, + resourceKind: "instructions", resourceKey: "AGENTS.md", resourceId: agent.id, + stockVersion: bundle.stockVersion, stockHash: stock, + defaultsJson: { entryFile: bundle.instructions.entryFile, files: Object.keys(bundle.instructions.files) }, + }, tx); + await tx.insert(activityLog).values({ companyId: agent.companyId, actorType: "system", actorId: "built-in-reconcile", + action: "agent.files_updated", entityType: "agent", entityId: agent.id, + details: { source: "built-in-stock-update", bundleKey: definition.key, stockHash: stock } }); + return stockState({ resourceKind: "instructions", resourceKey: "AGENTS.md", resourceId: agent.id, + stockVersion: bundle.stockVersion, latestStockHash: stock, currentHash: stock, bindingStockHash: stock }); + }); + } + const shouldWrite = mode === "reset" || currentState.stockStatus === "missing" @@ -992,13 +1096,36 @@ export function builtInAgentService(db: Db) { return currentState; } - const materialized = await instructionsSvc.materializeManagedBundle(agent, bundle.instructions.files, { - entryFile: bundle.instructions.entryFile, - replaceExisting: true, - clearLegacyPromptTemplate: true, - }); + let adapterConfig: Record; + if (currentFiles[bundle.instructions.entryFile] !== null && currentFiles[bundle.instructions.entryFile] !== undefined) { + if (!actor && mode === "reconcile") return currentState; + if (!actor) throw unprocessable("Resetting existing instructions requires an authenticated operator", { code: "INSTRUCTION_IDENTITY_INVALID" }); + const revisions = agentInstructionRevisionService(db); + const target = { companyId: agent.companyId, agentId: agent.id }; + const baseline = await revisions.readCurrent(target, actor); + const receipt = await revisions.commit({ ...target, entryFile: bundle.instructions.entryFile, + baseRevisionId: baseline?.revision.id ?? null, content: bundle.instructions.files[bundle.instructions.entryFile] ?? "", + source: actor.type === "board" ? "board" : "api" }, actor); + if (receipt.materialization === "pending") throw conflict("Instruction revision saved; retry reset to repair its disk copy", { revisionId: receipt.revision.id }); + const refreshed = await agentSvc.getById(agent.id); + if (!refreshed) throw notFound("Built-in agent not found"); + for (const [file, content] of Object.entries(bundle.instructions.files)) { + if (file !== bundle.instructions.entryFile) await instructionsSvc.writeFile(refreshed, file, content); + } + // Preserve personal files outside this built-in bundle's declared paths. + adapterConfig = { ...refreshed.adapterConfig }; + delete adapterConfig.promptTemplate; + delete adapterConfig.bootstrapPromptTemplate; + } else { + const materialized = await instructionsSvc.materializeManagedBundle(agent, bundle.instructions.files, { + entryFile: bundle.instructions.entryFile, + replaceExisting: false, + clearLegacyPromptTemplate: true, + }); + adapterConfig = materialized.adapterConfig; + } const updated = await agentSvc.update(agent.id, { - adapterConfig: materialized.adapterConfig, + adapterConfig, }, { allowBuiltInAgentMetadata: true, recordRevision: { source: `built-in-bundle:${mode}:instructions` }, @@ -1480,13 +1607,14 @@ export function builtInAgentService(db: Db) { definition: BuiltInAgentDefinition, mode: "reconcile" | "reset", resources?: Array<"instructions" | "skill" | "routine">, + actor?: AuthorizationActor, ) { if (!definition.bundle) return []; const selected = new Set(resources ?? ["instructions", "skill", "routine"]); const existingStates = await bundleResourceStates(agent.companyId, definition, agent); const byKind = new Map(existingStates.map((state) => [state.resourceKind, state])); const instruction = selected.has("instructions") - ? await materializeInstructions(agent, definition, mode) + ? await materializeInstructions(agent, definition, mode, actor) : byKind.get("instructions")!; const refreshedAgent = await agentSvc.getById(agent.id) as Agent | null; if (!refreshedAgent) throw notFound("Built-in agent not found"); @@ -1847,7 +1975,7 @@ export function builtInAgentService(db: Db) { return state(definition, updated as Agent); } - async function reset(companyId: string, key: string, input: { resources?: Array<"agent" | "instructions" | "skill" | "routine"> } = {}) { + async function reset(companyId: string, key: string, input: { resources?: Array<"agent" | "instructions" | "skill" | "routine"> } = {}, actor?: AuthorizationActor) { const definition = requireBuiltInAgentDefinition(key); const resetAgentDefaults = !input.resources || input.resources.includes("agent"); const current = resetAgentDefaults @@ -1862,6 +1990,7 @@ export function builtInAgentService(db: Db) { definition, "reset", input.resources ? selectedBundleResources ?? [] : undefined, + actor, ); return state(definition, await agentSvc.getById(current.agent.id) as Agent, resources); } diff --git a/server/src/services/change-consent-gate.ts b/server/src/services/change-consent-gate.ts index 8523f04bca..4569a0f8b3 100644 --- a/server/src/services/change-consent-gate.ts +++ b/server/src/services/change-consent-gate.ts @@ -112,7 +112,7 @@ function expandTargetKeysForLegacyCompatibility(targetKeys: string[]) { return [...expanded]; } -export function changeConsentGateService(db: Db) { +export function changeConsentGateService(db: Db | Parameters[0]>[0]) { return { assertConsented: async (input: { companyId: string; diff --git a/server/src/services/company-portability.ts b/server/src/services/company-portability.ts index 7c7501f631..701894a761 100644 --- a/server/src/services/company-portability.ts +++ b/server/src/services/company-portability.ts @@ -3549,7 +3549,7 @@ export function companyPortabilityService(db: Db, storage?: StorageService) { const companies = companyService(db); const agents = agentService(db); const assetRecords = assetService(db); - const instructions = agentInstructionsService(); + const instructions = agentInstructionsService(db); const access = accessService(db); const projects = projectService(db); const issues = issueService(db); diff --git a/server/src/services/heartbeat.ts b/server/src/services/heartbeat.ts index e923d178d0..60372fa015 100644 --- a/server/src/services/heartbeat.ts +++ b/server/src/services/heartbeat.ts @@ -1,3 +1,5 @@ +import { isAgentDirectoryCopy } from "./agent-directory-working-copies.js"; + import type { PaperclipTurnContext } from "@paperclipai/adapter-utils/server-utils"; import { restoreNativeWorkspaceBestEffort } from "./native-runtime/native-workspace-best-effort.js"; import { @@ -67,6 +69,7 @@ import { startAdapterExecutionTargetPaperclipBridge, } from "@paperclipai/adapter-utils/execution-target"; import { agentService } from "./agents.js"; +import { agentInstructionWorkingCopyService, instructionWorkingCopyGuidance } from "./agent-instruction-working-copies.js"; import { normalizeLegacyRunnerProvider } from "@paperclipai/adapter-utils"; import fs from "node:fs/promises"; import path from "node:path"; @@ -3442,6 +3445,15 @@ const heartbeatRunSafeResultJsonColumn = sql | null>` 'error', left(${heartbeatRuns.resultJson} ->> 'error', ${HEARTBEAT_RUN_RESULT_SUMMARY_MAX_CHARS}), 'stdout', left(${heartbeatRuns.resultJson} ->> 'stdout', ${HEARTBEAT_RUN_RESULT_OUTPUT_MAX_CHARS}), 'stderr', left(${heartbeatRuns.resultJson} ->> 'stderr', ${HEARTBEAT_RUN_RESULT_OUTPUT_MAX_CHARS}), + 'instructionSave', case when jsonb_typeof(${heartbeatRuns.resultJson} -> 'instructionSave') = 'object' + then jsonb_strip_nulls(jsonb_build_object( + 'state', left(${heartbeatRuns.resultJson} #>> '{instructionSave,state}', 32), + 'contract', left(${heartbeatRuns.resultJson} #>> '{instructionSave,contract}', 32), + 'entryFile', left(${heartbeatRuns.resultJson} #>> '{instructionSave,entryFile}', 512), + 'errorCode', left(${heartbeatRuns.resultJson} #>> '{instructionSave,errorCode}', 128), + 'errorMessage', left(${heartbeatRuns.resultJson} #>> '{instructionSave,errorMessage}', 1024), + 'storageWarning', left(${heartbeatRuns.resultJson} #>> '{instructionSave,storageWarning}', 1024) + )) end, 'workspaceRestoreFailure', case when ${heartbeatRuns.resultJson} ->> 'workspaceRestoreFailure' in ('restore_permission_denied', 'restore_lock_timeout', 'restore_unsafe_archive', 'restore_failed') then ${heartbeatRuns.resultJson} -> 'workspaceRestoreFailure' end, @@ -9500,6 +9512,7 @@ export function heartbeatService( environmentRuntimeService(db, { pluginWorkerManager: options.pluginWorkerManager, }); + const instructionCopies = agentInstructionWorkingCopyService(db, { environmentRuntime }); const envOrchestrator = environmentRunOrchestrator(db, { pluginWorkerManager: options.pluginWorkerManager, environmentRuntime, @@ -18866,6 +18879,14 @@ export function heartbeatService( async function reapOrphanedRuns(opts?: { staleThresholdMs?: number }) { const staleThresholdMs = opts?.staleThresholdMs ?? 0; const now = new Date(); + // Recovery never launches a provider or infers stopped ownership from + // terminal status. Uncaptured local copies require durable stop evidence. + await instructionCopies.recoverStopped().catch(error => { + logger.warn({ err: error }, "failed to recover stopped instruction copies"); + }); + await instructionCopies.recoverCaptured().catch(error => { + logger.warn({ err: error }, "failed to retry captured instruction revisions"); + }); // Complete persisted native results before generic orphan recovery. The // reconciler reads the durable workspace barrier and persisted runtime @@ -22348,6 +22369,29 @@ export function heartbeatService( await bindIssueToPersistedExecutionWorkspace(persistedExecutionWorkspace); const workspaceRealization = realizationResult.workspaceRealization; const executionTarget = realizationResult.executionTarget; + let instructionCopy: Awaited> = null; + let instructionSave: Record | null = null; + const collectStoppedInstructions = async () => { + if (!instructionCopy) return; + let saved = await instructionCopies.collectStopped({ companyId: agent.companyId, runId: run.id, target: executionTarget }); + // Capture before disposal. Exhausted bounded collection leaves a durable + // explicit loss report, never a claim that missing bytes were saved. + while (saved?.state === "pending_collection" && saved.attempts < 3) { + saved = await instructionCopies.collectStopped({ companyId: agent.companyId, runId: run.id, target: executionTarget }); + } + if (!saved) return; + const receipt = parseObject(saved.receipt); + const storageWarning = readNonEmptyString(receipt.storageWarning); + instructionSave = { state: saved.state, entryFile: saved.entryFile, + ...(isAgentDirectoryCopy(saved) ? { contract: "agent_files", appliedCandidateHash: saved.candidateHash } + : { revisionId: parseObject(receipt.revision).id ?? null }), storageWarning, errorCode: saved.errorCode, errorMessage: saved.errorMessage }; + await appendRunEvent(run, { eventType: "instruction_save", stream: "system", + level: !storageWarning && ["saved", "unchanged", "resolved"].includes(saved.state) ? "info" : "warn", + message: storageWarning ?? (saved.state === "saved" ? "Agent files saved." + : saved.state === "unchanged" ? "Instruction working copy is unchanged." + : saved.errorMessage ?? "Instruction edits were not saved. Review the preserved candidate in the agent instruction editor."), + payload: instructionSave }); + }; if (managedAiRuntime && aiBinding) { try { await assertManagedAiProjectAuth({ ...resolvedConfig, cwd: executionWorkspace.cwd }, aiBinding.provider, executionTarget); } catch { throw new ConfigurationIncompleteFailure("Project authentication conflicts with this agent’s managed AI connection", { configurationIncomplete: { reason: "ai_connection_incompatible", actionUrl: `/agents/${agent.id}/runtime` } }); } @@ -23157,6 +23201,62 @@ export function heartbeatService( target: executionTarget, workspaceId: persistedExecutionWorkspace?.id ?? null, }); + const hasInstructionFilesystem = nativeRuntimeResolution.kind !== "native" + ? adapter.supportsInstructionsBundle === true + : !["claude_managed_agents_api", "aws_agentcore_harness_api"].includes(nativeRuntimeResolution.profile.backend); + if (hasInstructionFilesystem) { + try { + // Missing contract fields on a restored session mean the deployed + // legacy format. New sessions opt into whole-directory persistence. + const priorFileRun = taskSessionForRun?.lastRunId + ? await db.select({ profile: heartbeatRuns.runnerProfileJson }).from(heartbeatRuns).where(and( + eq(heartbeatRuns.id, taskSessionForRun.lastRunId), eq(heartbeatRuns.companyId, agent.companyId), eq(heartbeatRuns.agentId, agent.id))).then(rows => rows[0]) + : null; + const savedFileInput = parseObject(parseObject(run.runnerProfileJson).nativeExecutionInput); + const priorFileInput = Object.keys(savedFileInput).length ? savedFileInput : parseObject(parseObject(priorFileRun?.profile).nativeExecutionInput); + const priorWorkingCopy = parseObject(parseObject(parseObject(priorFileInput.runtimeContext).instructions).workingCopy); + instructionCopy = await instructionCopies.prepare({ + companyId: agent.companyId, agentId: agent.id, runId: run.id, + target: executionTarget, cwd: executionWorkspace.cwd, + legacy: Object.keys(priorFileInput).length > 0 && priorWorkingCopy.kind !== "agent_files", + }); + } catch (error) { + if ((error as { status?: number }).status !== 403) throw error; + // Missing write identity must not break a background run's read-only + // prompt. It must also never imply that ordinary file edits will save. + await appendRunEvent(run, { eventType: "instruction_save", stream: "system", level: "warn", + message: "Persistent instruction editing is unavailable. Use an authenticated user with instruction edit access and a managed instruction bundle.", + payload: { state: "unavailable", code: "INSTRUCTION_COPY_UNAVAILABLE" } }); + const guidance = "No editable agent instruction working copy is registered for this turn. Use authenticated agent file tools for persistent edits; do not edit a private copy named in an earlier turn or claim its changes will persist."; + for (const key of ["paperclipTaskMarkdown", "paperclipTaskMarkdownCompact"]) { + context[key] = [readNonEmptyString(context[key]), guidance].filter(Boolean).join("\n\n"); + } + } + if (instructionCopy) { + const storageWarning = readNonEmptyString(instructionCopy.receipt?.storageWarning); + if (storageWarning) { + instructionSave = { state: "prepared", contract: "agent_files", storageWarning }; + // This is an advisory on the run, never an agent pause, execution + // failure, or scheduling gate. Keep it visible while work runs. + await db.update(heartbeatRuns).set({ resultJson: sql`coalesce(${heartbeatRuns.resultJson}, '{}'::jsonb) || ${JSON.stringify({ instructionSave })}::jsonb` }).where(eq(heartbeatRuns.id, run.id)); + await appendRunEvent(run, { eventType: "instruction_save", stream: "system", level: "warn", + message: storageWarning, payload: instructionSave }); + } + runtimeConfig = { ...runtimeConfig, instructionsFilePath: path.join(instructionCopy.localRoot, instructionCopy.entryFile) }; + if (isAgentDirectoryCopy(instructionCopy)) { + const workspace = parseObject(context.paperclipWorkspace); + context.paperclipWorkspace = { ...workspace, agentHome: instructionCopy.executionRoot, + // Keep the pre-existing permission root stable for ACP session + // identity. The per-run copy is already under the company root. + agentHomeForPermissions: workspace.agentHome, + }; + } + const guidance = instructionWorkingCopyGuidance(instructionCopy); + for (const key of ["paperclipTaskMarkdown", "paperclipTaskMarkdownCompact"]) { + context[key] = [readNonEmptyString(context[key]), guidance].filter(Boolean).join("\n\n"); + } + } + } let nativeExecution: NativeExecutionInput | null = null; let nativeRunnerInstanceId: string | null = null; if (nativeRuntimeResolution.kind === "native") { @@ -23586,6 +23686,7 @@ export function heartbeatService( runId: run.id, runtimeConfig, runtimeSkillEntries, + instructionWorkingCopy: instructionCopy ? { rootPath: instructionCopy.executionRoot, entryPath: instructionCopy.entryFile, ...(isAgentDirectoryCopy(instructionCopy) ? { kind: "agent_files" as const } : {}) } : undefined, }); const nativeExecutionWithCheckpoint = buildNativeExecutionWithCheckpoint({ @@ -24293,6 +24394,10 @@ export function heartbeatService( }, onLog, onEvent: onAdapterEvent, + instructionWorkingCopy: instructionCopy ? { + hasChanges: () => instructionCopies.hasChanges({ companyId: agent.companyId, runId: run.id, target: executionTarget }), + collectStopped: collectStoppedInstructions, + } : undefined, preparationSpans: nativeRunnerPreparationSpans, // Bootstrap with executable/home discovery while keeping // configured provider values and the server-selected @@ -24306,6 +24411,7 @@ export function heartbeatService( process.env, executionWorkspace.cwd, ), + ...(instructionCopy && isAgentDirectoryCopy(instructionCopy) ? { AGENT_HOME: instructionCopy.executionRoot } : {}), ...(nativeMcpServer ? { PAPERCLIP_NATIVE_MCP_NAME: nativeMcpServer.name, @@ -24486,6 +24592,7 @@ export function heartbeatService( issueId, ); }, + onProviderStopped: collectStoppedInstructions, onDispatch: markDispatchStarted, signal: executionControl.controller.signal, ...(executionTarget?.kind === "remote" && executionTarget.transport === "sandbox" ? { @@ -24534,6 +24641,7 @@ export function heartbeatService( if (!guardedDispatch.dispatched) return; adapterResult = await guardedDispatch.resultPromise; } + if (instructionSave) adapterResult.resultJson = { ...adapterResult.resultJson, instructionSave }; adapterResult = applyWorkspaceRestoreFailure(adapterResult); // A returned result can include a failed restore. Keep the workspace // barrier closed until required files have been restored. @@ -24769,6 +24877,7 @@ export function heartbeatService( "failed to revoke heartbeat-run MCP gateway tokens", ); } + await instructionCopies.release(agent.companyId, run.id); } // Reconcile the referenced-project set against the real remote staging outcome. A referenced // project can pass authorization and clone locally at run prep, then fail to stage into the @@ -26160,6 +26269,15 @@ export function heartbeatService( }, ); } + // A retained or unverified process stays above this release boundary. + // If no stopped-copy capture occurred, preserve an explicit loss report. + const uncapturedInstructions = await instructionCopies.reportUnavailable(run.companyId, run.id); + if (uncapturedInstructions?.state === "unavailable") { + await appendRunEvent(run, { eventType: "instruction_save", stream: "system", level: "warn", + message: "Instruction edits could not be recovered before environment release. No instruction save is claimed.", + payload: { state: "unavailable", code: uncapturedInstructions.errorCode } }); + } + await instructionCopies.release(run.companyId, run.id); await releaseEnvironmentLeasesForRun({ runId: run.id, companyId: run.companyId, diff --git a/server/src/services/native-runtime/agent-instruction-tools.integration.test.ts b/server/src/services/native-runtime/agent-instruction-tools.integration.test.ts new file mode 100644 index 0000000000..fe79d00aa8 --- /dev/null +++ b/server/src/services/native-runtime/agent-instruction-tools.integration.test.ts @@ -0,0 +1,89 @@ +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { randomUUID } from "node:crypto"; +import { eq } from "drizzle-orm"; +import { afterAll, beforeAll, beforeEach, describe, expect, it } from "vitest"; +import { agents, authUsers, companies, companyMemberships, principalPermissionGrants, heartbeatRuns, issues, agentInstructionRevisions, createDb } from "@paperclipai/db"; +import { startEmbeddedPostgresTestDatabase } from "../../__tests__/helpers/embedded-postgres.js"; +import { resolveManagedInstructionsRoot } from "../agent-instructions.js"; +import { PaperclipRunnerToolAuthority } from "./paperclip-runner-tool-authority.js"; + +describe("canonical instruction tools through native authority", () => { + let database: Awaited>; + let db: ReturnType; + let home: string; + const previousHome = process.env.PAPERCLIP_HOME; + let companyId: string, agentId: string, targetAgentId: string, userId: string, runId: string, issueId: string, root: string; + let authority: PaperclipRunnerToolAuthority; + const entryFile = "policy/ENTRY.md"; + const original = "\uFEFF# Original\r\n\0☃\n"; + beforeAll(async () => { + home = await fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), "instruction-tool-home-"))); + process.env.PAPERCLIP_HOME = home; + database = await startEmbeddedPostgresTestDatabase("instruction-tool-db-"); + db = createDb(database.connectionString); + }, 90_000); + afterAll(async () => { + if (previousHome === undefined) delete process.env.PAPERCLIP_HOME; else process.env.PAPERCLIP_HOME = previousHome; + await database?.cleanup(); + if (home) await fs.rm(home, { recursive: true, force: true }); + }); + beforeEach(async () => { + [companyId, agentId, targetAgentId, userId, runId, issueId] = Array.from({ length: 6 }, () => randomUUID()); + await db.insert(companies).values({ id: companyId, name: "Tool scope", issuePrefix: randomUUID().slice(0, 8) }); + await db.insert(authUsers).values({ id: userId, name: "Editor", email: `${userId}@example.test`, createdAt: new Date(), updatedAt: new Date() }); + root = resolveManagedInstructionsRoot({ id: targetAgentId, companyId, name: "Target", adapterConfig: {} }); + await db.insert(agents).values([ + { id: agentId, companyId, name: "Caller", status: "active" }, + { id: targetAgentId, companyId, name: "Target", adapterConfig: { instructionsBundleMode: "managed", instructionsRootPath: root, instructionsEntryFile: entryFile } }, + ]); + await db.insert(companyMemberships).values([ + { companyId, principalType: "user", principalId: userId, membershipRole: "operator" }, + { companyId, principalType: "agent", principalId: agentId, membershipRole: "member" }, + ]); + await db.insert(principalPermissionGrants).values([ + { companyId, principalType: "user", principalId: userId, permissionKey: "agents:configure", scope: { agentIds: [targetAgentId] } }, + { companyId, principalType: "agent", principalId: agentId, permissionKey: "agents:configure", scope: { agentIds: [targetAgentId] } }, + ]); + await db.insert(issues).values({ id: issueId, companyId, title: "Maintain instructions", status: "in_progress", workMode: "standard", assigneeAgentId: agentId }); + await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, status: "running", runtimeMode: "native", nativeIssueId: issueId, invocationSource: "on_demand", responsibleUserId: userId }); + await db.update(issues).set({ executionRunId: runId }).where(eq(issues.id, issueId)); + authority = new PaperclipRunnerToolAuthority(db, { companyId, agentId, issueId, runId }); + await fs.mkdir(path.dirname(path.join(root, entryFile)), { recursive: true }); + await fs.writeFile(path.join(root, entryFile), original); + }); + const call = (tool: string, args: Record = {}) => authority.execute({ tool, callId: randomUUID(), arguments: { targetAgentId, ...args } }) as Promise; + it("bridges old native instruction tools to current files without writing history", async () => { + const first = await call("read_agent_instructions"); + expect(first).toMatchObject({ entryFile, content: original }); + const changed = await call("update_agent_instructions", { entryFile, content: "changed\r\n", baseRevisionId: first.revision.id }); + expect(changed.revision).toMatchObject({ responsibleUserId: userId, actorAgentId: agentId, sourceRunId: runId }); + expect((await call("get_agent_instruction_history", { entryFile })).revisions).toHaveLength(0); + expect(await fs.readFile(path.join(root, entryFile), "utf8")).toBe("changed\r\n"); + await expect(call("update_agent_instructions", { entryFile, content: "stale", baseRevisionId: first.revision.id })).rejects.toMatchObject({ status: 409 }); + expect((await call("read_agent_instructions")).content).toBe("changed\r\n"); + }); + it("rejects supplied identity and missing CAS bases before changing content", async () => { + const first = await call("read_agent_instructions"); + for (const identity of ["companyId", "agentId", "runId", "responsibleUserId", "onBehalfOfUserId", "actor"]) { + await expect(call("update_agent_instructions", { entryFile, content: "forged", baseRevisionId: first.revision.id, [identity]: randomUUID() })).rejects.toMatchObject({ status: 400 }); + } + await expect(call("update_agent_instructions", { entryFile, content: "missing base" })).rejects.toMatchObject({ status: 400 }); + expect((await call("read_agent_instructions")).content).toBe(original); + expect(await db.select().from(agentInstructionRevisions).where(eq(agentInstructionRevisions.agentId, targetAgentId))).toHaveLength(0); + }); + it("rechecks responsible-user access and rejects a cross-company target", async () => { + const first = await call("read_agent_instructions"); + await db.delete(principalPermissionGrants).where(eq(principalPermissionGrants.principalId, userId)); + await expect(call("update_agent_instructions", { entryFile, content: "revoked", baseRevisionId: first.revision.id })).rejects.toMatchObject({ status: 403 }); + const foreignCompanyId = randomUUID(), foreignAgentId = randomUUID(); + await db.insert(companies).values({ id: foreignCompanyId, name: "Foreign", issuePrefix: randomUUID().slice(0, 8) }); + await db.insert(agents).values({ id: foreignAgentId, companyId: foreignCompanyId, name: "Foreign" }); + await expect(call("read_agent_instructions", { targetAgentId: foreignAgentId })).rejects.toMatchObject({ status: 404 }); + await expect(call("update_agent_instructions", { targetAgentId: foreignAgentId, entryFile, content: "foreign", baseRevisionId: null })).rejects.toMatchObject({ status: 404 }); + await db.update(heartbeatRuns).set({ status: "cancelled" }).where(eq(heartbeatRuns.id, runId)); + await expect(call("read_agent_instructions")).rejects.toThrow("paperclip_runner_tool_binding_not_authorized"); + expect(await fs.readFile(path.join(root, entryFile), "utf8")).toBe(original); + }); +}); diff --git a/server/src/services/native-runtime/agent-instruction-tools.test.ts b/server/src/services/native-runtime/agent-instruction-tools.test.ts new file mode 100644 index 0000000000..619d05a9f6 --- /dev/null +++ b/server/src/services/native-runtime/agent-instruction-tools.test.ts @@ -0,0 +1,20 @@ +import { describe, expect, it } from "vitest"; +import type { Db } from "@paperclipai/db"; +import { PaperclipRunnerToolAuthority } from "./paperclip-runner-tool-authority.js"; + +const binding = { companyId: "00000000-0000-4000-8000-000000000001", agentId: "00000000-0000-4000-8000-000000000002", issueId: "00000000-0000-4000-8000-000000000003", runId: "00000000-0000-4000-8000-000000000004" }; + +describe("dedicated canonical instruction tools", () => { + it("advertises read, CAS update, history, and restore on ordinary native runs", () => { + const authority = new PaperclipRunnerToolAuthority({} as Db, binding); + const tools = authority.definitions(); + expect(tools.map((tool) => tool.name)).toEqual(expect.arrayContaining([ + "read_agent_instructions", "update_agent_instructions", "get_agent_instruction_history", "restore_agent_instructions", + ])); + for (const name of ["update_agent_instructions", "restore_agent_instructions"]) { + const tool = tools.find((candidate) => candidate.name === name)!; + expect(tool.inputSchema).toMatchObject({ required: expect.arrayContaining(["baseRevisionId"]), additionalProperties: false }); + expect(tool.inputSchema).not.toHaveProperty("properties.responsibleUserId"); + } + }); +}); diff --git a/server/src/services/native-runtime/agent-instruction-tools.ts b/server/src/services/native-runtime/agent-instruction-tools.ts new file mode 100644 index 0000000000..032e66699f --- /dev/null +++ b/server/src/services/native-runtime/agent-instruction-tools.ts @@ -0,0 +1,76 @@ +import type { AgentInstructionCommitReceipt } from "@paperclipai/shared"; +import { z } from "zod"; +import type { Db } from "@paperclipai/db"; +import { badRequest, notFound } from "../../errors.js"; +import { agentInstructionRevisionService } from "../agent-instruction-revisions.js"; +import { agentInstructionWorkingCopyService } from "../agent-instruction-working-copies.js"; +import { deriveBundleState } from "../agent-instructions.js"; +import { agentService } from "../agents.js"; +import type { AuthorizationActor } from "../authorization.js"; + +const target = { targetAgentId: z.string().uuid().optional() }; +const entryFile = z.string().min(1).max(4096); +const baseRevisionId = z.string().uuid().nullable(); +const schemas = { + read_agent_instructions: z.object({ ...target, entryFile: entryFile.optional(), revisionId: z.string().uuid().optional() }).strict() + .refine((input) => Boolean(input.entryFile) === Boolean(input.revisionId), "Historical reads require both entryFile and revisionId"), + update_agent_instructions: z.object({ ...target, entryFile, content: z.string().max(1024 * 1024), baseRevisionId }).strict(), + get_agent_instruction_history: z.object({ ...target, entryFile, cursor: z.string().min(1).max(2048).optional(), limit: z.number().int().min(1).max(100).optional() }).strict(), + restore_agent_instructions: z.object({ ...target, entryFile, revisionId: z.string().uuid(), baseRevisionId: z.string().uuid() }).strict(), +}; + +export type AgentInstructionToolName = keyof typeof schemas; + +/** Called only after the native authority validates its server-bound active run. */ +export async function executeAgentInstructionTool(input: { + db: Db; + binding: { companyId: string; agentId: string; runId: string }; + tool: AgentInstructionToolName; + arguments: unknown; +}) { + // Never accept an actor, responsible user, company, or source run from tool JSON. + const actor: AuthorizationActor = { type: "agent", source: "agent_jwt", companyId: input.binding.companyId, agentId: input.binding.agentId, runId: input.binding.runId }; + const service = agentInstructionRevisionService(input.db); + const parse = (schema: z.ZodType): T => { + const result = schema.safeParse(input.arguments); + if (!result.success) throw badRequest("Invalid instruction tool arguments", result.error.flatten()); + return result.data; + }; + const scope = (targetAgentId?: string) => ({ companyId: input.binding.companyId, agentId: targetAgentId ?? input.binding.agentId }); + const acknowledge = async (receipt: AgentInstructionCommitReceipt, targetAgentId?: string) => { + // The commit is already durable. A failed baseline refresh keeps the old CAS + // fence, so later cleanup preserves a conflict instead of overwriting it. + await agentInstructionWorkingCopyService(input.db).acknowledgeExplicitSave({ + ...scope(targetAgentId), runId: input.binding.runId, entryFile: receipt.revision.entryFile, + revisionId: receipt.revision.id, contentHash: receipt.revision.contentHash, + }).catch(() => undefined); + return receipt; + }; + switch (input.tool) { + case "read_agent_instructions": { + const args = parse(schemas.read_agent_instructions); + const target = scope(args.targetAgentId); + if (args.entryFile && args.revisionId) return service.readRevision({ ...target, entryFile: args.entryFile, revisionId: args.revisionId }, actor); + const snapshot = await service.readCurrent(target, actor); + if (snapshot) return { ...snapshot, entryFile: snapshot.revision.entryFile }; + // An unseeded empty entry still needs its configured filename for the first CAS. + const agent = await agentService(input.db).getById(target.agentId); + if (!agent || agent.companyId !== target.companyId) throw notFound("Agent not found"); + return { entryFile: deriveBundleState(agent).entryFile, revision: null, content: null }; + } + case "update_agent_instructions": { + const args = parse(schemas.update_agent_instructions); + const receipt = await service.commit({ ...scope(args.targetAgentId), entryFile: args.entryFile, content: args.content, baseRevisionId: args.baseRevisionId, source: "tool" }, actor); + return acknowledge(receipt, args.targetAgentId); + } + case "get_agent_instruction_history": { + const args = parse(schemas.get_agent_instruction_history); + return service.history({ ...scope(args.targetAgentId), entryFile: args.entryFile, cursor: args.cursor, limit: args.limit }, actor); + } + case "restore_agent_instructions": { + const args = parse(schemas.restore_agent_instructions); + const receipt = await service.restore({ ...scope(args.targetAgentId), entryFile: args.entryFile, revisionId: args.revisionId, baseRevisionId: args.baseRevisionId }, actor); + return acknowledge(receipt, args.targetAgentId); + } + } +} diff --git a/server/src/services/native-runtime/native-session-executor.test.ts b/server/src/services/native-runtime/native-session-executor.test.ts index b6d7316911..30ae4895d8 100644 --- a/server/src/services/native-runtime/native-session-executor.test.ts +++ b/server/src/services/native-runtime/native-session-executor.test.ts @@ -5870,6 +5870,57 @@ describe("native session same-turn steering", () => { }); describe("native warm session supervision", () => { + it.each([ + { changed: false, closeFails: false }, + { changed: true, closeFails: false }, + { changed: true, closeFails: true }, + ])("collects changed instructions only after the owned warm provider stops (changed=$changed, close fails=$closeFails)", async ({ changed, closeFails }) => { + const identity = `instruction-close-${changed}-${closeFails}`; + let releaseClose!: () => void; + const closed = new Promise((resolve) => { releaseClose = resolve; }); + const close = vi.fn(async () => { await closed; if (closeFails) throw new Error("instruction provider close failed"); }); + const collectStopped = vi.fn(async () => {}); + const hasChanges = vi.fn(async () => changed); + const warmExecution = { ...execution, + binding: { ...execution.binding, runId: identity, executionWorkspaceId: identity }, + session: { ...execution.session, normalizedSessionId: identity, lifecyclePolicy: { mode: "warm" as const, idleTimeoutMs: 60_000 } }, + } as NativeExecutionInputV1; + state.execute.mockReset().mockImplementationOnce(async (options) => { + expect(options.requireSessionCloseBeforeReturn).toBe(true); + expect(options.onSessionClosed).toBe(collectStopped); + await options.onSession?.({ close }); + return { result: { summary: "completed" }, terminal: { runTerminalState: "succeeded" }, + turnId: identity, normalizedSessionId: identity, providerSessionId: identity, + driverKind: "test", driverVersion: "1", nativeEventCount: 1, highestContiguousSourceSeq: 1, usage: null }; + }); + const running = executePaperclipNativeSession({ db: leaseDb(warmExecution), execution: warmExecution, + runnerInstanceId: identity, runnerExecutionTarget: { kind: "remote", transport: "sandbox", environmentId: identity, remoteCwd: `/tmp/${identity}` }, + instructionWorkingCopy: { hasChanges, collectStopped } }); + const observed = running.then(() => null, error => error); + try { + if (changed) { + await vi.waitFor(() => expect(close).toHaveBeenCalledOnce()); + expect(collectStopped).not.toHaveBeenCalled(); + releaseClose(); + const error = await observed; + if (closeFails) { + expect(error?.message).toBe("instruction provider close failed"); + expect(collectStopped).not.toHaveBeenCalled(); + } else { + expect(error).toBeNull(); + expect(collectStopped).toHaveBeenCalledOnce(); + } + } else { + expect(await observed).toBeNull(); + expect(close).not.toHaveBeenCalled(); + expect(collectStopped).not.toHaveBeenCalled(); + } + } finally { + releaseClose(); + await closeWarmNativeSessionsForEnvironment({ environmentId: identity, reason: "test cleanup" }); + } + }); + describe("warm session identity transitions", () => { let previousHome: string | undefined; let isolatedHome: string; diff --git a/server/src/services/native-runtime/native-session-executor.ts b/server/src/services/native-runtime/native-session-executor.ts index 1e75231a0b..2da691b5e1 100644 --- a/server/src/services/native-runtime/native-session-executor.ts +++ b/server/src/services/native-runtime/native-session-executor.ts @@ -7167,6 +7167,12 @@ export async function executePaperclipNativeSession(input: { chatAttachmentReadScope?: NativeChatAttachmentReadScope; onLog?: (stream: "stdout" | "stderr", chunk: string) => Promise; onEvent?: (event: AdapterRuntimeEvent) => Promise; + /** Only this run's registered private instruction entry. + * Probe at terminal; persist only after owned shutdown. */ + instructionWorkingCopy?: { + hasChanges: () => Promise; + collectStopped: () => Promise; + }; /** Persist task-level continuity before a durable goal can outlive this run. */ onGoalCheckpoint?: (snapshot: PersistedNativeSession) => Promise; sessionGoalControl?: NativeSessionGoalControl | null; @@ -8305,7 +8311,8 @@ async function executePaperclipNativeSessionWithinScope( resumeSessionGoalHeartbeat: input.resumeSessionGoalHeartbeat, // Every durable runner must finish its bounded suspension before // the next run verifies and rotates the saved authority. - requireSessionCloseBeforeReturn: runnerdBackend !== null, + requireSessionCloseBeforeReturn: runnerdBackend !== null || input.instructionWorkingCopy !== undefined, + onSessionClosed: input.instructionWorkingCopy?.collectStopped, onCheckpoint: async (snapshot) => { if (warmSessionId !== null && warmConfigDigest !== null) { await persistWarmNativeCheckpoint( @@ -9065,12 +9072,22 @@ async function executePaperclipNativeSessionWithinScope( // A following run cannot attach until the prior run's durable finalization // is committed. Provider completion alone is not an authority boundary. if (warmSessionId !== null && lifecyclePolicy.mode === "warm") { + const instructionCopy = input.instructionWorkingCopy; + const ownedSession = warmNativeSessions.get(warmSessionId); + const collectInstructions = Boolean(instructionCopy && ownedSession?.ownerToken === warmSessionOwnerToken && + await instructionCopy.hasChanges()); + if (collectInstructions && ownedSession) { + // Keep the unchanged warm path intact. A changed private instruction copy + // requires the existing checkpoint-and-close boundary before collection. + ownedSession.closeOnReleaseReason = "registered instruction edits require stopped-provider collection"; + } await releaseWarmNativeSession( warmSessionId, warmSessionOwnerToken, lifecyclePolicy.idleTimeoutMs, false, ); + if (collectInstructions) await instructionCopy!.collectStopped(); } const adapterResult: AdapterExecutionResult = { exitCode: native.terminal.runTerminalState === "succeeded" ? 0 : 1, diff --git a/server/src/services/native-runtime/paperclip-runner-tool-authority.test.ts b/server/src/services/native-runtime/paperclip-runner-tool-authority.test.ts index 5b1c58b5ad..6b5e6f0d33 100644 --- a/server/src/services/native-runtime/paperclip-runner-tool-authority.test.ts +++ b/server/src/services/native-runtime/paperclip-runner-tool-authority.test.ts @@ -99,7 +99,7 @@ describe("PaperclipRunnerToolAuthority", () => { issueId, runId, }); - expect(authority.definitions()).toHaveLength(30); + expect(authority.definitions()).toHaveLength(34); const questions = authority.definitions().find(tool => tool.name === "request_human_input")!; expect(questions.description).toContain("ask only the next unanswered question"); expect(questions.description).toContain("Never infer answers"); diff --git a/server/src/services/native-runtime/paperclip-runner-tool-authority.ts b/server/src/services/native-runtime/paperclip-runner-tool-authority.ts index 1b46dfe55d..27fd2f45e6 100644 --- a/server/src/services/native-runtime/paperclip-runner-tool-authority.ts +++ b/server/src/services/native-runtime/paperclip-runner-tool-authority.ts @@ -1,3 +1,4 @@ +import { executeAgentInstructionTool } from "./agent-instruction-tools.js"; import { createReadStream } from "node:fs"; import { publicChatTaskUrl } from "../chat-task-url.js"; import type { createAssignedMcpTools } from "./assigned-mcp-tools.js"; @@ -83,6 +84,7 @@ import { } from "./chat-attachment-read.js"; const IMPLEMENTED_OPERATIONS = new Set([ + "read_agent_instructions", "update_agent_instructions", "get_agent_instruction_history", "restore_agent_instructions", "search_api", "call_api", "hire_agent", "get_task_context", "get_task_history", "search_tasks", "report_progress", "request_human_input", @@ -368,6 +370,13 @@ export class PaperclipRunnerToolAuthority { throw new Error("paperclip_runner_tool_mode_denied"); } switch (call.tool) { + case "read_agent_instructions": + case "update_agent_instructions": + case "get_agent_instruction_history": + case "restore_agent_instructions": + return executeAgentInstructionTool({ db: this.db, binding: { + companyId: this.binding.companyId, agentId: this.binding.agentId, runId: this.binding.runId, + }, tool: call.tool, arguments: call.arguments }); case "create_skill": { const apiUrl = this.binding.apiUrl ?? process.env.PAPERCLIP_API_URL; const token = createLocalAgentJwt(this.binding.agentId, this.binding.companyId, context.actor.adapterType, this.binding.runId, context.run.responsibleUserId); diff --git a/server/src/services/native-runtime/runner-api-catalog.ts b/server/src/services/native-runtime/runner-api-catalog.ts index d8f8fd7d79..85da534726 100644 --- a/server/src/services/native-runtime/runner-api-catalog.ts +++ b/server/src/services/native-runtime/runner-api-catalog.ts @@ -42,6 +42,10 @@ function words(text: string): string[] { } function dedicatedTools(method: string, path: string): string[] { + if (/\/agents\/\{[^}]+\}\/instructions-bundle\/file$/.test(path)) return method === "GET" ? ["read_agent_instructions"] : method === "PUT" ? ["update_agent_instructions"] : []; + if (/\/agents\/\{[^}]+\}\/instructions-bundle\/history$/.test(path) && method === "GET") return ["get_agent_instruction_history"]; + if (/\/agents\/\{[^}]+\}\/instructions-bundle\/revision\/\{[^}]+\}$/.test(path) && method === "GET") return ["read_agent_instructions"]; + if (/\/agents\/\{[^}]+\}\/instructions-bundle\/restore$/.test(path) && method === "POST") return ["restore_agent_instructions"]; if (/\/companies\/\{[^}]+\}\/skills$/.test(path) && method === "POST") return ["create_skill"]; if (/\/companies\/\{[^}]+\}\/agent-hires$/.test(path) && method === "POST") return ["hire_agent"]; if (/\/projects$/.test(path)) return method === "GET" ? ["list_projects"] : method === "POST" ? ["create_project"] : []; diff --git a/server/src/services/native-runtime/runner-api.test.ts b/server/src/services/native-runtime/runner-api.test.ts index 5cc3175264..0746456f25 100644 --- a/server/src/services/native-runtime/runner-api.test.ts +++ b/server/src/services/native-runtime/runner-api.test.ts @@ -72,6 +72,10 @@ describe("runner API catalog", () => { expect(runnerApiOperation(createProject).dedicatedTools).toEqual(["create_project"]); expect(runnerApiOperation(projects).dedicatedTools).toEqual(["list_projects"]); expect(runnerApiOperation("GET /api/companies/{companyId}/project-repositories").dedicatedTools).toEqual(["list_project_repositories"]); + expect(runnerApiOperation("GET /api/agents/{id}/instructions-bundle/file").dedicatedTools).toEqual(["read_agent_instructions"]); + expect(runnerApiOperation("PUT /api/agents/{id}/instructions-bundle/file").dedicatedTools).toEqual(["update_agent_instructions"]); + expect(runnerApiOperation("GET /api/agents/{id}/instructions-bundle/history").dedicatedTools).toEqual(["get_agent_instruction_history"]); + expect(runnerApiOperation("POST /api/agents/{id}/instructions-bundle/restore").dedicatedTools).toEqual(["restore_agent_instructions"]); expect(runnerApiOperation("POST /api/companies/{companyId}/agent-hires").dedicatedTools).toEqual(["hire_agent"]); expect(runnerApiOperation("POST /api/companies/{companyId}/agent-hires").dedicatedToolGuidance).toContain("inherits the caller's native runtime"); }); diff --git a/server/src/services/native-runtime/runtime-context.test.ts b/server/src/services/native-runtime/runtime-context.test.ts index a2903c177a..f8996f1f7f 100644 --- a/server/src/services/native-runtime/runtime-context.test.ts +++ b/server/src/services/native-runtime/runtime-context.test.ts @@ -10,6 +10,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; const serviceMocks = vi.hoisted(() => ({ exportFiles: vi.fn(), + readCommittedForRuntime: vi.fn(), getEffectiveProfilesForAgent: vi.fn(), githubBotConnectionIdsForRun: vi.fn(), })); @@ -20,6 +21,11 @@ vi.mock("../chat-github-tools.js", () => ({ vi.mock("../agent-instructions.js", () => ({ agentInstructionsService: () => ({ exportFiles: serviceMocks.exportFiles }), + agentInstructionsBundleMode: (agent: { adapterConfig?: { instructionsBundleMode?: string } }) => agent.adapterConfig?.instructionsBundleMode ?? null, +})); + +vi.mock("../agent-instruction-revisions.js", () => ({ + agentInstructionRevisionService: () => ({ readCommittedForRuntime: serviceMocks.readCommittedForRuntime }), })); vi.mock("../tool-access.js", () => ({ @@ -50,6 +56,7 @@ async function makeTreeWritable(target: string): Promise { beforeEach(async () => { vi.clearAllMocks(); + serviceMocks.readCommittedForRuntime.mockResolvedValue(null); serviceMocks.githubBotConnectionIdsForRun.mockResolvedValue(new Set()); previousPaperclipHome = process.env.PAPERCLIP_HOME; previousInstanceId = process.env.PAPERCLIP_INSTANCE_ID; @@ -90,6 +97,60 @@ afterEach(async () => { }); describe("buildNativeRuntimeContext", () => { + it("uses committed instructions when the disk projection is stale", async () => { + serviceMocks.exportFiles.mockResolvedValue({ + entryFile: "AGENTS.md", + files: { "AGENTS.md": "Old disk projection\n", "reference.md": "Sibling reference\n" }, + }); + serviceMocks.readCommittedForRuntime.mockResolvedValue({ + revision: { entryFile: "AGENTS.md" }, content: "Saved canonical instructions\n", + }); + const context = await buildNativeRuntimeContext({ + db: {} as Db, + agent: { id: "agent-1", companyId: "company-1", name: "Reviewer", adapterType: "paperclip_runner", adapterConfig: { instructionsBundleMode: "managed" } }, + runId: "run-1", runtimeConfig: {}, runtimeSkillEntries: [], + }); + expect(await readFile(path.join(context.instructions.bundle.rootPath, context.instructions.entryPath), "utf8")) + .toBe("Saved canonical instructions\n"); + expect(await readFile(path.join(context.instructions.bundle.rootPath, "reference.md"), "utf8")) + .toBe("Sibling reference\n"); + expect(serviceMocks.readCommittedForRuntime).toHaveBeenCalledWith({ companyId: "company-1", agentId: "agent-1" }); + }); + + it("does not substitute canonical managed instructions into an external bundle", async () => { + serviceMocks.exportFiles.mockResolvedValue({ entryFile: "AGENTS.md", files: { "AGENTS.md": "External instructions\n" } }); + const context = await buildNativeRuntimeContext({ + db: {} as Db, + agent: { id: "agent-1", companyId: "company-1", name: "Reviewer", adapterConfig: { instructionsBundleMode: "external" } }, + runId: "run-1", runtimeConfig: {}, runtimeSkillEntries: [], + }); + expect(await readFile(path.join(context.instructions.bundle.rootPath, context.instructions.entryPath), "utf8")) + .toBe("External instructions\n"); + expect(serviceMocks.readCommittedForRuntime).not.toHaveBeenCalled(); + }); + + it("keeps the pinned prompt immutable while exposing the registered editable instruction copy", async () => { + serviceMocks.exportFiles.mockResolvedValue({ + entryFile: "instructions/CHARter.md", + files: { "instructions/CHARter.md": "Original instructions.\n" }, + }); + const workingRoot = path.join(temporaryRoots.at(-1)!, "private-run-copy"); + await mkdir(path.join(workingRoot, "instructions"), { recursive: true }); + await writeFile(path.join(workingRoot, "instructions/CHARter.md"), "Original instructions.\n"); + const context = await buildNativeRuntimeContext({ + db: {} as Db, + agent: { id: "agent-1", companyId: "company-1", name: "Reviewer", adapterType: "paperclip_runner", adapterConfig: {} }, + runId: "run-1", + runtimeConfig: {}, + runtimeSkillEntries: [], + instructionWorkingCopy: { rootPath: workingRoot, entryPath: "instructions/CHARter.md" }, + }); + expect(context.instructions.workingCopy).toEqual({ rootPath: workingRoot, entryPath: "instructions/CHARter.md" }); + await writeFile(path.join(workingRoot, "instructions/CHARter.md"), "Persist this next time.\n"); + expect(await readFile(path.join(context.instructions.bundle.rootPath, context.instructions.entryPath), "utf8")) + .toBe("Original instructions.\n"); + }); + it.each(["disabled", "degraded"] as const)( "omits an unavailable native MCP connection when it is %s without aborting runtime context creation", async (unavailableState) => { diff --git a/server/src/services/native-runtime/runtime-context.ts b/server/src/services/native-runtime/runtime-context.ts index 32bb32426e..5498f27998 100644 --- a/server/src/services/native-runtime/runtime-context.ts +++ b/server/src/services/native-runtime/runtime-context.ts @@ -21,7 +21,8 @@ import { type NativeRuntimeContextSnapshot, } from "../../vendor/paperclip-runner/index.js"; import { resolvePaperclipInstanceRoot } from "../../home-paths.js"; -import { agentInstructionsService } from "../agent-instructions.js"; +import { agentInstructionsService, agentInstructionsBundleMode } from "../agent-instructions.js"; +import { agentInstructionRevisionService } from "../agent-instruction-revisions.js"; import { toolAccessService } from "../tool-access.js"; import { filterResolvedGitHubConnectionsForRun } from "../git-credentials.js"; @@ -145,8 +146,24 @@ export async function materializeAsset(files: AssetFile[]): Promise ({ path: safeRelativePath(relativePath, "instruction path"), content: Buffer.from(content, "utf8"), mode: 0o444 })); @@ -222,12 +239,17 @@ export async function resolveNativeRuntimeMcpSnapshot(input: { db: Db; agent: Pi return { assignmentSetId: `sha256:${assignmentDigest}`, digest: assignmentDigest, bindingId: assignment.connections.length ? `native-mcp:${input.runId}` : null }; } -export async function buildNativeRuntimeContext(input: { db: Db; agent: RuntimeAgent; runId: string; runtimeConfig: Record; runtimeSkillEntries: PaperclipSkillEntry[] }): Promise { +export async function buildNativeRuntimeContext(input: { db: Db; agent: RuntimeAgent; runId: string; runtimeConfig: Record; runtimeSkillEntries: PaperclipSkillEntry[]; instructionWorkingCopy?: { rootPath: string; entryPath: string; kind?: "agent_files" } }): Promise { const [instructions, skills, mcp] = await Promise.all([ - materializeInstructionBundle(input.agent), + materializeInstructionBundle(input.db, input.agent, input.instructionWorkingCopy?.kind === "agent_files"), materializeSelectedSkills(input.runtimeConfig, input.runtimeSkillEntries, input.agent.adapterType === "paperclip_runner"), resolveNativeRuntimeMcpSnapshot({ db: input.db, agent: input.agent, runId: input.runId }), ]); - const snapshot = { prompt: { revision: PAPERCLIP_EXECUTION_PROMPT_REVISION, text: PAPERCLIP_EXECUTION_PROMPT, digest: nativeRuntimePromptDigest() }, instructions, skills, mcp }; + const snapshot = { + prompt: { revision: PAPERCLIP_EXECUTION_PROMPT_REVISION, text: PAPERCLIP_EXECUTION_PROMPT, digest: nativeRuntimePromptDigest() }, + instructions: { ...instructions, ...(input.instructionWorkingCopy ? { workingCopy: input.instructionWorkingCopy } : {}) }, + skills, + mcp, + }; return parseNativeRuntimeContext({ ...snapshot, aggregateDigest: canonicalNativeRuntimeContextDigest(snapshot) }); } diff --git a/server/src/services/plugin-managed-agents.ts b/server/src/services/plugin-managed-agents.ts index 13473ec8c0..f86f7a439d 100644 --- a/server/src/services/plugin-managed-agents.ts +++ b/server/src/services/plugin-managed-agents.ts @@ -5,6 +5,8 @@ import { companies, pluginEntities, pluginManagedResources, + plugins, + activityLog, } from "@paperclipai/db"; import type { Agent, @@ -12,11 +14,13 @@ import type { PluginManagedAgentDeclaration, PluginManagedAgentResolution, } from "@paperclipai/shared"; -import { notFound } from "../errors.js"; +import { isUuidLike } from "@paperclipai/shared"; +import { conflict, forbidden, notFound } from "../errors.js"; import { agentService } from "./agents.js"; import { approvalService } from "./approvals.js"; import { logActivity } from "./activity-log.js"; import { agentInstructionsBundleMode, agentInstructionsService } from "./agent-instructions.js"; +import { agentInstructionRevisionService } from "./agent-instruction-revisions.js"; const MANAGED_AGENT_ENTITY_TYPE = "managed_agent"; const DEFAULT_MANAGED_AGENT_ADAPTER_TYPE = "process"; @@ -180,7 +184,7 @@ export function pluginManagedAgentService( ) { const agentSvc = agentService(db); const approvalSvc = approvalService(db); - const instructions = agentInstructionsService(); + const instructions = agentInstructionsService(db); function declarationFor(agentKey: string) { const declaration = options.manifest?.agents?.find((agent) => agent.agentKey === agentKey); @@ -190,8 +194,8 @@ export function pluginManagedAgentService( return declaration; } - async function getBinding(companyId: string, agentKey: string) { - return db + async function getBinding(companyId: string, agentKey: string, client: Pick = db) { + return client .select() .from(pluginEntities) .where( @@ -210,6 +214,7 @@ export function pluginManagedAgentService( agentId: string, extraData: Record = {}, effectiveAdapterType?: string, + client: Pick = db, ) { const adapterType = effectiveAdapterType ?? (await resolveManagedAdapterType(companyId, declaration)); const defaultsJson = { @@ -227,7 +232,7 @@ export function pluginManagedAgentService( budgetMonthlyCents: declaration.budgetMonthlyCents ?? 0, instructions: declaration.instructions ?? null, }; - const managedResource = await db + const managedResource = await client .select({ id: pluginManagedResources.id }) .from(pluginManagedResources) .where(and( @@ -238,12 +243,12 @@ export function pluginManagedAgentService( )) .then((rows) => rows[0] ?? null); if (managedResource) { - await db + await client .update(pluginManagedResources) .set({ resourceId: agentId, defaultsJson, updatedAt: new Date() }) .where(eq(pluginManagedResources.id, managedResource.id)); } else { - await db.insert(pluginManagedResources).values({ + await client.insert(pluginManagedResources).values({ companyId, pluginId: options.pluginId, pluginKey: options.pluginKey, @@ -265,9 +270,9 @@ export function pluginManagedAgentService( lastReconciledAt: new Date().toISOString(), ...extraData, }; - const existing = await getBinding(companyId, declaration.agentKey); + const existing = await getBinding(companyId, declaration.agentKey, client); if (existing) { - return db + return client .update(pluginEntities) .set({ scopeKind: "company", @@ -281,7 +286,7 @@ export function pluginManagedAgentService( .returning() .then((rows) => rows[0]); } - return db + return client .insert(pluginEntities) .values({ pluginId: options.pluginId, @@ -305,6 +310,46 @@ export function pluginManagedAgentService( return rows.find((row) => rowIsManagedAgent(row, options.pluginKey, declaration.agentKey)) ?? null; } + async function relinkManagedAgent(companyId: string, declaration: PluginManagedAgentDeclaration, agentId: string) { + const adapterType = await resolveManagedAdapterType(companyId, declaration); + return db.transaction(async (tx) => { + const [agent] = await tx.select().from(agents).where(and( + eq(agents.id, agentId), eq(agents.companyId, companyId), ne(agents.status, "terminated"), + )).for("update"); + if (!agent || !rowIsManagedAgent(agent, options.pluginKey, declaration.agentKey)) { + throw conflict("Managed agent ownership changed before relink"); + } + const [plugin] = await tx.select().from(plugins).where(and( + eq(plugins.id, options.pluginId), eq(plugins.pluginKey, options.pluginKey), + )).for("share"); + if (!plugin || plugin.status !== "ready" || !plugin.manifestJson.capabilities.includes("agents.managed") + || !plugin.manifestJson.agents?.some((entry) => entry.agentKey === declaration.agentKey)) { + throw forbidden("Plugin relink is limited to its registered managed agent declaration"); + } + const marker = agent.metadata!.paperclipManagedResource as Record; + const previousPluginId = marker.pluginId; + if (typeof previousPluginId !== "string" || !isUuidLike(previousPluginId)) throw forbidden("Managed agent has no plugin owner"); + if (previousPluginId !== options.pluginId) { + const [previousPlugin] = await tx.select({ id: plugins.id }).from(plugins).where(eq(plugins.id, previousPluginId)); + if (previousPlugin) throw forbidden("Managed agent still belongs to another installed plugin"); + } + // Hard uninstall cascades the bindings, but deliberately leaves the agent + // and its canonical history. Transfer only its proven stable-key marker to + // the replacement installation, atomically with the new scoped bindings. + await upsertBinding(companyId, declaration, agentId, {}, adapterType, tx); + const [relinked] = await tx.update(agents).set({ + metadata: managedMetadata(options.pluginId, options.pluginKey, declaration, agent.metadata), + updatedAt: new Date(), + }).where(and(eq(agents.id, agentId), eq(agents.companyId, companyId))).returning(); + await tx.insert(activityLog).values({ + companyId, actorType: "plugin", actorId: options.pluginId, + action: "plugin.managed_agent.relinked", entityType: "agent", entityId: agentId, + details: { sourcePluginKey: options.pluginKey, managedResourceKey: declaration.agentKey, previousPluginId }, + }); + return relinked as Agent; + }); + } + async function companyAdapterUsage(companyId: string) { const rows = await db .select({ adapterType: agents.adapterType }) @@ -336,23 +381,39 @@ export function pluginManagedAgentService( const declared = declaredInstructionFiles(declaration, variables); if (!declared) return agent; - const materialized = await instructions.materializeManagedBundle( - agent, - declared.files, - { - entryFile: declared.entryFile, - replaceExisting: materializeOptions.replaceExisting, - clearLegacyPromptTemplate: true, - }, - ); + let adapterConfig: Record; + if (materializeOptions.replaceExisting) { + const revisions = agentInstructionRevisionService(db); + const target = { companyId, agentId: agent.id }; + const actor = { type: "plugin" as const, pluginId: options.pluginId, pluginKey: options.pluginKey, agentKey: declaration.agentKey }; + const baseline = await revisions.readForPluginReset(target, actor); + const receipt = await revisions.commitPluginReset({ ...target, entryFile: declared.entryFile, + baseRevisionId: baseline.snapshot?.revision.id ?? null, configuredEntryFile: baseline.configuredEntryFile, content: declared.files[declared.entryFile] ?? "" }, actor); + if (receipt.materialization === "pending") throw conflict("Instruction revision saved; retry reset to repair its disk copy", { revisionId: receipt.revision.id }); + const refreshed = await agentSvc.getById(agent.id); + if (!refreshed) throw notFound("Managed agent not found"); + for (const [file, content] of Object.entries(declared.files)) { + if (file !== declared.entryFile) await instructions.writeFile(refreshed, file, content); + } + // A stock reset owns declared instruction paths, not the agent's other + // persistent files (including a formerly configured entry). + adapterConfig = { ...refreshed.adapterConfig }; + delete adapterConfig.promptTemplate; + delete adapterConfig.bootstrapPromptTemplate; + } else { + const materialized = await instructions.materializeManagedBundle(agent, declared.files, { + entryFile: declared.entryFile, replaceExisting: false, clearLegacyPromptTemplate: true, + }); + adapterConfig = materialized.adapterConfig; + } const updated = await agentSvc.update(agent.id, { - adapterConfig: materialized.adapterConfig, + adapterConfig, }, { recordRevision: { source: `plugin:${optionsForRevisionSource()}:managed-agent-instructions`, }, }); - return (updated as Agent | null) ?? { ...agent, adapterConfig: materialized.adapterConfig }; + return (updated as Agent | null) ?? { ...agent, adapterConfig }; } async function managedInstructionDefaultDrift( @@ -375,8 +436,7 @@ export function pluginManagedAgentService( return { entryFile: declared.entryFile, changedFiles: [declared.entryFile] }; } - const paths = new Set([...Object.keys(declared.files), ...Object.keys(exported.files)]); - const changedFiles = [...paths] + const changedFiles = Object.keys(declared.files) .filter((filePath) => (exported.files[filePath] ?? null) !== (declared.files[filePath] ?? null)) .sort((left, right) => left.localeCompare(right)); if (exported.entryFile !== declared.entryFile && !changedFiles.includes(declared.entryFile)) { @@ -433,7 +493,7 @@ export function pluginManagedAgentService( spentMonthlyCents: 0, lastHeartbeatAt: null, }) as Agent; - created = await materializeDeclaredInstructions(companyId, created, declaration, { replaceExisting: true }); + created = await materializeDeclaredInstructions(companyId, created, declaration, { replaceExisting: false }); let approvalId: string | null = null; if (requiresApproval) { @@ -574,9 +634,7 @@ export function pluginManagedAgentService( const relinkCandidate = await findRelinkCandidate(companyId, declaration); if (relinkCandidate) { - await upsertBinding(companyId, declaration, relinkCandidate.id); - const relinkedAgent = await agentSvc.getById(relinkCandidate.id) as Agent | null; - if (!relinkedAgent) throw notFound("Managed agent not found"); + const relinkedAgent = await relinkManagedAgent(companyId, declaration, relinkCandidate.id); const agent = await backfillManagedPauseReason( companyId, declaration, @@ -596,8 +654,19 @@ export function pluginManagedAgentService( ? reconciled.agent.metadata : {}; const adapterType = await resolveManagedAdapterType(companyId, declaration); + // Reset content through the canonical CAS path before changing defaults. + // A conflict must leave the existing adapter configuration untouched. + const withInstructions = await materializeDeclaredInstructions(companyId, reconciled.agent, declaration, { replaceExisting: true }); + const defaults = declarationPatch(declaration, { adapterType }); + const adapterConfig = { ...defaults.adapterConfig } as Record; + if (declaration.instructions) { + for (const key of ["instructionsBundleMode", "instructionsRootPath", "instructionsEntryFile", "instructionsFilePath"]) { + if (withInstructions.adapterConfig[key] !== undefined) adapterConfig[key] = withInstructions.adapterConfig[key]; + } + } const updated = await agentSvc.update(reconciled.agent.id, { - ...declarationPatch(declaration, { adapterType }), + ...defaults, + adapterConfig, metadata: managedMetadata(options.pluginId, options.pluginKey, declaration, currentMetadata), }, { recordRevision: { @@ -605,7 +674,7 @@ export function pluginManagedAgentService( }, }); if (!updated) throw notFound("Managed agent not found"); - const updatedAgent = await materializeDeclaredInstructions(companyId, updated as Agent, declaration, { replaceExisting: true }); + const updatedAgent = updated as Agent; await upsertBinding(companyId, declaration, updatedAgent.id, {}, adapterType); await logActivity(db, { companyId, diff --git a/tests/runner-e2e/FIXTURES.md b/tests/runner-e2e/FIXTURES.md index 1b1041bd19..6aaf3b6f19 100644 --- a/tests/runner-e2e/FIXTURES.md +++ b/tests/runner-e2e/FIXTURES.md @@ -233,3 +233,19 @@ first provider turn intentionally omits task disposition, and their second turn must be an automatic, causally bound repair that records completion. They use public task comments/status APIs and run-detail evidence; no private runtime hooks or database mutations are used by the fixture. + +## Persistent agent files + +The `instruction_persistence` flow uses production managed storage and public file +APIs. The browser creates a supporting file, then a real agent edits its registered +AGENT_HOME with ordinary filesystem tools. Independent oracles verify instructions, +nested text, binary download bytes, and a stopped-run save receipt without new +revision history. The harness restarts the server and creates a fresh browser task +without disclosing the saved nonces. Its readback oracle downloads and verifies an +attachment's bytes and SHA-256, rather than accepting a filename or model claim. +A third task uploads a ready attachment and waits in an ordinary bounded shell +command while the board changes the current file through the public API. Stopped +cleanup must preserve the original candidate as a conflict. The browser reviews +current and incoming files and applies the run edits against the reviewed current +directory hash. All three tasks' runs count toward billing and teardown. The suite +is explicit-only. No private control-plane hooks or direct database writes are used. diff --git a/tests/runner-e2e/README.md b/tests/runner-e2e/README.md index 29287faf34..c1f73bb303 100644 --- a/tests/runner-e2e/README.md +++ b/tests/runner-e2e/README.md @@ -191,6 +191,30 @@ duplicating the final response. The second workflow restarts the isolated Paperclip server while the interaction is waiting, reloads that state, and then resumes it. The suite has no Daytona cells. +`instruction-persistence` is an explicit-only three-cell workflow: legacy and +native Codex locally, plus native Codex on Daytona. Each creates six browser tasks +for the same agent. The editor first creates a nested supporting file. The first +run edits its registered AGENT_HOME using ordinary filesystem tools: instructions, +nested text, editor-created content, and exact binary bytes. The oracle checks the +current files, a stopped-run save receipt, and absence of newly appended history. +The first task also publishes a small verification receipt for the normal +completion contract; the personal files stay in the agent directory. +After a Paperclip restart, a fresh task must upload a downloaded proof attachment +containing independent saved nonces absent from its prompt. A third task edits its +private copy while the browser edits the same current file. The later run sync +must win for that changed file, preserve an unrelated board-created file, and +produce no conflict candidate or manual review step. Exact bytes, downloads, +and receipts are independently checked; model claims alone cannot pass. +Three further tasks fill a sparse personal file to its 256 MiB limit, exceed +that limit, and clean it up. Every run must still succeed; the run UI must show +a warning while full and clear it after cleanup. Rejected bytes must not replace +the saved file. This adds at most one 256 MiB saved fixture per isolated agent. +The deadline is twenty minutes per cell, with six expected provider runs; +normal instance/Daytona cleanup, screenshots, evidence, and billing apply. Run with +`pnpm test:e2e:runner -- --suite instruction-persistence`. Managed agent directories +checkpoint and close the provider before collection while retaining conversation +state. The separate `daytona-warm-continuity` suite covers warm runtime behavior. + `daytona-warm-continuity` (**Daytona Warm Continuity**) is exactly two paid cells: legacy Codex and Runner Codex against one reusable warm Daytona configuration. Each cell creates a real project with a primary local-path @@ -424,6 +448,14 @@ content tags are never rebuilt or overwritten by the workflow. ### Match the local controller package to the Daytona image +When the controller runs on macOS or another platform different from the sandbox, +set `PAPERCLIP_RUNNER_REMOTE_BINARY_PATH` to a verified Linux amd64 +`paperclip-runnerd`, such as the binary copied from `/usr/local/bin/paperclip-runnerd` +in the pinned image. The controller must have these exact bytes for its artifact +identity check. A local macOS runner cannot substitute for the Linux binary, +even when the sandbox image contains a compatible runner. This also applies to +native Codex cells, which do not otherwise need the remote provider pack below. + Native ACPX (including Claude) and OpenCode Daytona cells also require `PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH` on the controller. The package and the image must come from the same verified build. Equal provider version numbers diff --git a/tests/runner-e2e/catalog.test.ts b/tests/runner-e2e/catalog.test.ts index ecfba3a556..749207e66c 100644 --- a/tests/runner-e2e/catalog.test.ts +++ b/tests/runner-e2e/catalog.test.ts @@ -91,10 +91,10 @@ describe("runner E2E catalog", () => { expect(localIntegrityTasks).toHaveLength(2); expect(openRouterBreadthTasks).toHaveLength(3); expect(runnerSuites.map((suite) => suite.expectedMatrixSize)).toEqual([ - 16, 16, 2, 8, 46, 23, 47, 20, 52, 28, 18, 6, 6, 4, 48, 16, 10, 2, + 3, 16, 16, 2, 8, 46, 23, 47, 20, 52, 28, 18, 6, 6, 4, 48, 16, 10, 2, ]); - expect(validateRunnerCatalog()).toHaveLength(368); - expect(new Set(runnerMatrix.map((entry) => entry.id)).size).toBe(368); + expect(validateRunnerCatalog()).toHaveLength(371); + expect(new Set(runnerMatrix.map((entry) => entry.id)).size).toBe(371); expect( runnerMatrix.filter((entry) => entry.suite.id === "core-compatibility"), ).toHaveLength(48); diff --git a/tests/runner-e2e/catalog.ts b/tests/runner-e2e/catalog.ts index 56da5bf4a8..662bc66175 100644 --- a/tests/runner-e2e/catalog.ts +++ b/tests/runner-e2e/catalog.ts @@ -1,3 +1,4 @@ +import { instructionPersistenceTask } from "./instruction-persistence.js"; import { apiResponseReadingTask } from "./api-response-reading.js"; import { accountingTasks } from "./accounting-cases.js"; import { continuationTasks } from "./continuation-cases.js"; @@ -985,6 +986,15 @@ const everydayProfiles = [ ].map(productionStoryProfile); export const runnerSuites: readonly RunnerSuiteFixture[] = [ + { + id: "instruction-persistence", label: "Instruction Persistence", + description: "Agent-owned text and binary files round trip through the editor, survive a server restart and fresh task, and synchronize concurrent edits per file with last-sync-wins.", + groups: [], profiles: codexContinuityProfiles, + environments: [localEnvironment, runnerEnvironments.find(environment => environment.id === "daytona")!], tasks: [instructionPersistenceTask], + excludedExecutionIds: ["instruction-persistence.legacy-codex.daytona.private-copy-persists"], + expectedMatrixSize: 3, manualOnly: true, + definitionMetadata: { version: 6, oracle: "current-directory-independent-nonce-binary-last-sync-wins-and-nonblocking-storage-quota", providerTurns: 6, restart: "between-tasks", instructions: "production" }, + }, { id: "grok-subscription-qualification", label: "Grok Build Subscription Qualification", manualOnly: true, description: "Explicit company subscription login across Grok browser workflows in local and Daytona environments.", @@ -1313,6 +1323,7 @@ export function validateRunnerCatalog(): MatrixExecution[] { ...localIntegrityTasks, ...openRouterBreadthTasks, daytonaWarmContinuityTask, + instructionPersistenceTask, ]; for (const [label, values] of [ ["suite", runnerSuites], diff --git a/tests/runner-e2e/instruction-persistence.test.ts b/tests/runner-e2e/instruction-persistence.test.ts new file mode 100644 index 0000000000..a0d547d983 --- /dev/null +++ b/tests/runner-e2e/instruction-persistence.test.ts @@ -0,0 +1,36 @@ +import { createHash } from "node:crypto"; +import { describe, expect, it } from "vitest"; +import { gradeInstructionPersistence, instructionPersistenceTask } from "./instruction-persistence.js"; +import { runnerMatrix } from "./catalog.js"; +const expectedContent = "original\nInstruction persistence nonce: fixture\n"; +const valid = () => ({ before: { revision: { id: "before" } }, + saveEvent: { runId: "first-run", state: "saved" }, fileProof: true, + after: { content: expectedContent, contentHash: createHash("sha256").update(expectedContent).digest("hex"), revision: { id: "after", source: "cleanup", sourceRunId: "first-run", contentHash: createHash("sha256").update(expectedContent).digest("hex") } }, + firstRunId: "first-run", expectedContent, + proof: { body: "Instruction persistence nonce: fixture\n", contentVerified: true }, + expectedProof: "Instruction persistence nonce: fixture\n" }); +describe("instruction persistence independent oracle", () => { + it("does not disclose the persisted nonce in the fresh-task marker", () => { + const nonce = "only-the-instruction-entry-reveals-this"; + expect(instructionPersistenceTask.buildVisibleMarker(nonce)).not.toContain(nonce); + }); + it("accepts exact cleanup content and a verified downloaded fresh-task proof", () => { + expect(gradeInstructionPersistence(valid()).every(row => row.passed)).toBe(true); + }); + it.each(["missing-save", "wrong-run", "missing-files", "wrong-bytes", "missing-proof", "unverified-proof", "wrong-proof"])("rejects %s evidence", kind => { + const value = valid(); + if (kind === "missing-save") value.saveEvent.state = "unavailable"; + if (kind === "wrong-run") value.saveEvent.runId = "another-run"; + if (kind === "missing-files") value.fileProof = false; + if (kind === "wrong-bytes") value.after.content = "claimed success"; + if (kind === "missing-proof") return expect(gradeInstructionPersistence({ ...value, proof: undefined }).some(row => !row.passed)).toBe(true); + if (kind === "unverified-proof") value.proof.contentVerified = false; + if (kind === "wrong-proof") value.proof.body = "claimed success"; + expect(gradeInstructionPersistence(value).some(row => !row.passed)).toBe(true); + }); + it("registers exactly three explicit cells without changing scheduled campaigns", () => { + const cells = runnerMatrix.filter(row => row.suite.id === "instruction-persistence"); + expect(cells.map(row => `${row.profile.id}.${row.environment.id}`)).toEqual(["legacy-codex.local", "runner-codex.local", "runner-codex.daytona"]); + expect(cells.every(row => row.suite.manualOnly && row.task.expectedRunCount === 6)).toBe(true); + }); +}); diff --git a/tests/runner-e2e/instruction-persistence.ts b/tests/runner-e2e/instruction-persistence.ts new file mode 100644 index 0000000000..6e60b304dd --- /dev/null +++ b/tests/runner-e2e/instruction-persistence.ts @@ -0,0 +1,232 @@ +import { createHash, randomBytes } from "node:crypto"; +import { expect, type Page } from "@playwright/test"; +import { pollUntil, type RunnerApi } from "./api.js"; +import { captureFirstTaskAttachments } from "./first-task-attachments.js"; +import { collectRunEvents } from "./run-observations.js"; +import { createTaskThroughUi } from "./user-actions.js"; +import type { LiveFixtureValues } from "./live-fixtures.js"; +import type { MatrixExecution, RunnerTaskFixture } from "./types.js"; + +type Row = Record; +export const instructionNonceLine = (nonce: string) => `Instruction persistence nonce: ${nonce}\n`; +export const instructionPersistenceTask: RunnerTaskFixture = { + id: "private-copy-persists", label: "Agent directory survives a fresh task", + groups: [], workMode: "standard", flow: "instruction_persistence", + expectedRunCount: 6, attemptTimeoutMs: { local: 20 * 60_000, daytona: 20 * 60_000 }, + expectedTerminalState: { issue: "done", run: "succeeded" }, + buildTitle: nonce => `Persist private instructions ${nonce}`, + buildVisibleMarker: () => "INSTRUCTIONS-VERIFIED", + buildPrompt: nonce => [ + "Edit your own registered writable agent instruction entry with ordinary filesystem tools. The runtime guidance gives its exact private path.", + "Use Node.js built-in fs for these byte-preserving edits. Apply each append exactly once: inspect the existing suffix before retrying any command, because a warning does not imply that its writes failed.", + `Preserve its existing bytes and append exactly this UTF-8 suffix, represented as a JSON string: ${JSON.stringify(`\n${instructionNonceLine(nonce)}`)}`, + "Decode the JSON string once and append those bytes. Do not trim or normalize the existing file and do not add another blank line or separator.", + `In AGENT_HOME, create notes/retained.txt containing exactly ${JSON.stringify(`Personal file nonce: ${nonce}\n`)}. Create notes/bytes.bin with exactly the bytes [0,255,17,128,9]. Read notes/from-editor.txt and append exactly a newline followed by Edited by agent. and a final newline.`, + "Do not use update_agent_instructions, restore_agent_instructions, or an instructions API to save it. Do not edit repository AGENTS.md or the read-only loaded bundle.", + "After verifying the edits, upload a small text/plain attachment named agent-file-check.txt containing only 'Private file edits verified'. Use this attachment as your task completion evidence; the personal files themselves stay in AGENT_HOME.", + "Reply only Instruction copy edited without printing filesystem paths, then complete this task after the file edit. Paperclip will collect it after the provider stops; do not claim it has already persisted. Do not create further tasks.", + ].join("\n"), + buildMatchers: () => [], // Independent current file and attachment oracle below. +}; + +export function gradeInstructionPersistence(input: { before: Row; after: Row; firstRunId: string; expectedContent: string; proof: Row | undefined; expectedProof: string; saveEvent?: Row; fileProof?: boolean }) { + return [ + { id: "directory-save-receipt", passed: input.saveEvent?.runId === input.firstRunId && input.saveEvent?.state === "saved" }, + { id: "nested-and-binary-files", passed: input.fileProof === true }, + { id: "exact-canonical-bytes", passed: input.after.content === input.expectedContent && input.after.contentHash === createHash("sha256").update(input.expectedContent).digest("hex") }, + { id: "fresh-task-downloaded-proof", passed: input.proof?.contentVerified === true && input.proof.body === input.expectedProof }, + ].map(check => ({ ...check, detail: check.passed ? `${check.id} verified independently` : `${check.id} missing or incorrect` })); +} + +export async function runInstructionPersistenceFlow(input: { + page: Page; api: RunnerApi; fixtures: LiveFixtureValues; execution: MatrixExecution; nonce: string; + secrets: readonly string[]; deadlineAt: number; + restart(): Promise; + observe(issue: Row, runs: Row[]): void; + capture(id: string, label: string, file: string): Promise; + evidence(name: string, data: unknown): Promise; +}) { + const { page, api, fixtures, execution, nonce } = input; + // Fixture names contain the campaign nonce. Use an unrelated value that the + // fresh task can obtain only from the saved entry (or forbidden task history). + const persistedNonce = randomBytes(16).toString("hex"); + const filePath = `/api/agents/${fixtures.agent.id}/instructions-bundle/file?path=AGENTS.md`; + const before = await api.get(filePath); + if (typeof before.content !== "string" || !before.contentHash) throw new Error("Managed instructions must expose a current file hash"); + const expectedContent = `${before.content}\n${instructionNonceLine(persistedNonce)}`; + const instructionsUrl = `/${fixtures.company.issuePrefix}/agents/${fixtures.agent.id}/instructions`; + const editorText = `Editor nonce: ${randomBytes(16).toString("hex")}`; + await page.goto(instructionsUrl); + await page.getByRole("button", { name: "Add agent file", exact: true }).click(); + await page.getByPlaceholder("TOOLS.md").fill("notes/from-editor.txt"); + await page.getByRole("button", { name: "Create", exact: true }).click(); + await page.getByRole("group", { name: "Instruction file view" }).getByRole("button", { name: "edit", exact: true }).click(); + await page.getByRole("textbox", { name: "Instruction file editor" }).fill(editorText); + await page.getByRole("button", { name: "Save changes", exact: true }).click(); + await expect(page.getByRole("button", { name: "Save changes", exact: true })).toBeDisabled(); + await expect(page.getByRole("button", { name: "History", exact: true })).toHaveCount(0); + let issue: Row = {}; + let runs: Row[] = []; + async function create(title: string, prompt: string) { + await createTaskThroughUi({ page, issuePrefix: fixtures.company.issuePrefix!, agentName: fixtures.agent.name, title, prompt, workMode: "standard", projectName: fixtures.project?.name }); + const found = await pollUntil({ label: `instruction task ${title}`, deadlineAt: input.deadlineAt, + load: async () => (await api.get(`/api/companies/${fixtures.company.id}/issues?limit=100`)).find(row => row.title === title), accept: row => Boolean(row) }); + if (!found) throw new Error("Browser-created instruction task missing"); + issue = found; + input.observe(issue, runs); + await page.goto(`/${fixtures.company.issuePrefix}/issues/${issue.identifier ?? issue.id}`); + } + async function settle(count: number) { + await pollUntil({ label: `instruction run ${count} completed`, deadlineAt: input.deadlineAt, + load: async () => { + issue = await api.get(`/api/issues/${issue.id}`); + const listed = await api.get(`/api/companies/${fixtures.company.id}/heartbeat-runs?limit=100`); + runs = await Promise.all(listed.map(row => api.get(`/api/heartbeat-runs/${row.id}`))); + runs.sort((a, b) => String(a.createdAt).localeCompare(String(b.createdAt))); + input.observe(issue, runs); + return { issue, runs }; + }, + accept: state => state.issue.status === "done" && state.runs.length === count && state.runs.every(row => row.status === "succeeded"), + reject: state => state.runs.some(row => ["failed", "cancelled", "timed_out"].includes(row.status)) ? "Instruction task provider run failed" : state.runs.length > count ? "Instruction task dispatched an extra run" : state.issue.status === "blocked" && state.runs.length === count && state.runs.every(row => row.status === "succeeded") ? `Instruction task reported a terminal blocker: ${JSON.stringify(state.issue.unblockDescriptor ?? {})}` : undefined, + }); + expect(runs.every(row => row.runtimeMode === execution.profile.expectedRuntimeMode)).toBe(true); + await page.reload(); + await expect(page.getByTestId("issue-detail-header").getByRole("button", { name: "Change status (current: Done)", exact: true })).toBeVisible(); + } + await create(execution.task.buildTitle(nonce), execution.task.buildPrompt(persistedNonce)); + await settle(1); + const firstRunId = runs[0]!.id; + const after = await pollUntil({ label: "stopped agent directory save", deadlineAt: Math.min(input.deadlineAt, Date.now() + 30_000), + load: () => api.get(filePath), accept: row => row.content === expectedContent }); + const events = await collectRunEvents((afterSeq, limit) => api.get(`/api/heartbeat-runs/${firstRunId}/events?afterSeq=${afterSeq}&limit=${limit}`)); + const saveEvent = events.find(row => row.eventType === "instruction_save" && row.payload?.state === "saved"); + expect(saveEvent).toBeTruthy(); + const readPersonal = (name: string) => api.get(`/api/agents/${fixtures.agent.id}/instructions-bundle/file?path=${encodeURIComponent(name)}`); + const note = await readPersonal("notes/retained.txt"); + const fromEditor = await readPersonal("notes/from-editor.txt"); + const binaryResponse = await api.request.get(`/api/agents/${fixtures.agent.id}/instructions-bundle/file?path=notes%2Fbytes.bin&download=true`); + expect(binaryResponse.ok()).toBe(true); + const binary = await binaryResponse.body(); + const fileProof = note.content === `Personal file nonce: ${persistedNonce}\n` && fromEditor.content === `${editorText}\nEdited by agent.\n` && binary.equals(Buffer.from([0,255,17,128,9])); + expect(fileProof).toBe(true); + const history = await api.get(`/api/agents/${fixtures.agent.id}/instructions-bundle/history?path=AGENTS.md`); + expect(history.revisions).toHaveLength(0); + await input.evidence("instruction-first-save.json", { before, after, run: runs[0], events }); + await input.capture("instruction-edited", "Private instructions saved after provider stop", "instruction-edited.png"); + // A new server and a new issue cannot pass by retaining model conversation. + await input.restart(); + expect((await api.get(filePath)).content).toBe(expectedContent); + await create("Read persisted instructions", [ + "Read your own loaded agent instruction entry (or its current registered private copy) using ordinary filesystem tools.", + "Find the line beginning 'Instruction persistence nonce: '. Copy that entire line plus one final newline into instruction-proof.txt. Then append the exact bytes of notes/retained.txt from AGENT_HOME. Verify notes/bytes.bin contains the bytes [0,255,17,128,9]. Do not infer the value from this task title or other task history. Do not change your instructions.", + "Upload instruction-proof.txt as a text/plain task attachment named instruction-proof.txt using the normal artifact workflow. A local file alone is insufficient.", + `Reply with exactly ${execution.task.buildVisibleMarker(nonce)} and complete the task.`, + ].join("\n")); + await settle(2); + const attachments = await captureFirstTaskAttachments(api, [{ ...issue, id: String(issue.id) }], input.secrets); + const proof = attachments.find(row => row.originalFilename === "instruction-proof.txt" || row.name === "instruction-proof.txt"); + const final = await api.get(filePath); + expect(final.revision.id).toBe(after.revision.id); + const checks = gradeInstructionPersistence({ before, after, firstRunId, expectedContent, proof, expectedProof: `${instructionNonceLine(persistedNonce)}Personal file nonce: ${persistedNonce}\n`, saveEvent: { runId: firstRunId, state: saveEvent?.payload?.state }, fileProof }); + await expect(page.getByTestId("task-chat-agent-bubble").filter({ hasText: execution.task.buildVisibleMarker(nonce) }).last()).toBeVisible(); + await input.capture("final-state", "Fresh task downloaded the persisted instruction nonce", "final-state.png"); + expect(checks.filter(check => !check.passed), "Independent instruction persistence checks").toEqual([]); + + checks.push({ id: "editor-round-trip-no-history", passed: true, detail: "A browser-created supporting file was edited by the agent; all current bytes persisted without revision rows" }); + const restored = final; + + // The provider publishes an ordinary attachment before a bounded command wait. + // A board edit during that run is superseded only for the same changed file. + const conflictSuffix = `\nLast completed synchronization: ${nonce}\n`; + const expectedCandidate = `${restored.content}${conflictSuffix}`; + await create("Sync a concurrent instruction edit", [ + "Use Node.js built-in fs. Apply the append exactly once, checking existing bytes before any retry.", + `Append exactly this UTF-8 suffix to your current registered writable instruction entry, represented as a JSON string: ${JSON.stringify(conflictSuffix)}`, + "Decode the JSON string once. Preserve all existing bytes. Do not use an instruction revision tool or instructions API.", + "After the file edit, upload a text/plain task attachment named instruction-candidate-ready.txt with the text ready. Use the ordinary artifact workflow.", + "Then execute the ordinary shell command sleep 45 and wait for it to finish. This gives the board time to edit the canonical instructions concurrently. Do not complete the task before that command finishes.", + "After the wait completes, reply Candidate edit ready and complete the task. Do not change the instructions again or claim that they saved.", + ].join("\n")); + await pollUntil({ label: "provider staged concurrent instruction edit", deadlineAt: input.deadlineAt, + load: () => api.get(`/api/issues/${issue.id}/attachments`), + accept: rows => rows.some(row => row.originalFilename === "instruction-candidate-ready.txt" || row.name === "instruction-candidate-ready.txt") }); + const active = await api.get(`/api/issues/${issue.id}/runs`); + expect(active.some(row => row.status === "running")).toBe(true); + const boardMarker = "Concurrent board instruction edit."; + await page.goto(instructionsUrl); + await page.getByText("AGENTS.md", { exact: true }).first().click(); + await page.getByRole("group", { name: "Instruction file view" }).getByRole("button", { name: "edit", exact: true }).click(); + const entryEditor = page.getByRole("textbox", { name: "editable markdown" }); + await entryEditor.click(); + await entryEditor.press("ControlOrMeta+End"); + await entryEditor.press("Enter"); + await entryEditor.pressSequentially(boardMarker); + await page.getByRole("button", { name: "Save changes", exact: true }).click(); + await expect(page.getByRole("button", { name: "Save changes", exact: true })).toBeDisabled(); + const board = await api.get(filePath); + expect(board.content).toContain(boardMarker); + expect(board.contentHash).not.toBe(restored.contentHash); + const unrelatedContent = `Concurrent independent file: ${nonce}`; + const unrelated = await api.request.put(`/api/agents/${fixtures.agent.id}/instructions-bundle/file`, { + data: { path: "notes/concurrent-editor.txt", content: unrelatedContent, baseHash: null }, + }); + expect(unrelated.ok()).toBe(true); + await page.goto(`/${fixtures.company.issuePrefix}/issues/${issue.identifier ?? issue.id}`); + await settle(3); + const syncRunId = runs[2]!.id; + const resolved = await pollUntil({ label: "last completed sync wins", deadlineAt: Math.min(input.deadlineAt, Date.now() + 30_000), + load: () => api.get(filePath), accept: row => row.content === expectedCandidate }); + const candidates = await api.get(`/api/agents/${fixtures.agent.id}/instructions-bundle/candidates`); + expect(candidates.some(row => row.runId === syncRunId)).toBe(false); + expect((await readPersonal("notes/concurrent-editor.txt")).content).toBe(unrelatedContent); + const syncEvents = await collectRunEvents((afterSeq, limit) => api.get(`/api/heartbeat-runs/${syncRunId}/events?afterSeq=${afterSeq}&limit=${limit}`)); + expect(syncEvents.some(row => row.eventType === "instruction_save" && row.payload?.state === "saved")).toBe(true); + await page.goto(instructionsUrl); + await expect(page.getByRole("button", { name: "Review preserved files", exact: true })).toHaveCount(0); + checks.push({ id: "per-file-last-sync-wins", passed: true, detail: "The later agent sync replaced the concurrent browser edit to its changed entry, preserved an unrelated new file, and created no conflict candidate" }); + await page.goto(`/${fixtures.company.issuePrefix}/issues/${issue.identifier ?? issue.id}`); + await input.capture("last-sync-wins", "Concurrent changes synchronized per file without a conflict-review step", "last-sync-wins.png"); + + const quotaTask = (action: string, receipt: string) => [ + "This is a controlled persistent-storage quota check. Use ordinary Node.js filesystem tools in your registered AGENT_HOME. Do not edit AGENTS.md.", + action, + `Upload a small text/plain task attachment named ${receipt}.txt containing the observed file size or cleanup result. This attachment is the primary task deliverable.`, + "Complete this task normally after uploading the receipt. A persistent-file storage warning is expected and must not prevent completion. Do not perform additional cleanup or change other personal files.", + ].join("\n"); + await create("Reach the agent file storage limit", quotaTask( + "Create quota-cache.bin using fs.openSync with flag w, fs.ftruncateSync(fd, 268435456), and fs.closeSync. This is a sparse fixture file, not a download. Verify its size using fs.statSync without reading the large contents.", "quota-full")); + await settle(4); + const fullRun = runs[3]!; + expect(fullRun.resultJson?.instructionSave).toMatchObject({ state: "saved", storageWarning: expect.stringContaining("Agent storage is full") }); + await page.goto(`/${fixtures.company.issuePrefix}/agents/${fixtures.agent.id}/runs/${fullRun.id}`); + await expect(page.getByRole("note").filter({ hasText: "Agent storage warning" })).toContainText("Runs can continue"); + // Public campaign screenshots are limited to sanitized task routes. Verify + // the warning in the real run UI, then capture its completed task outcome. + await page.goto(`/${fixtures.company.issuePrefix}/issues/${issue.identifier ?? issue.id}`); + await input.capture("storage-warning", "Task succeeded while agent storage reached its limit", "storage-warning.png"); + + await create("Keep running while agent storage is full", quotaTask( + "Verify quota-cache.bin already exists and its size is exactly 268435456. Grow only this file to 268435457 bytes with fs.truncateSync, then verify the new size. Leave it above the limit for this run's sync check.", "quota-exceeded")); + await settle(5); + const exceededRun = runs[4]!; + expect(exceededRun.resultJson?.instructionSave).toMatchObject({ state: "unavailable", errorCode: "AGENT_FILES_LIMIT_EXCEEDED", storageWarning: expect.stringContaining("Runs can continue") }); + const fullEvents = await collectRunEvents((afterSeq, limit) => api.get(`/api/heartbeat-runs/${exceededRun.id}/events?afterSeq=${afterSeq}&limit=${limit}`)); + expect(fullEvents.some(row => row.eventType === "instruction_save" && row.level === "warn" && row.payload?.state === "prepared" && row.payload?.storageWarning)).toBe(true); + + await create("Clean up agent storage during a normal task", quotaTask( + "Verify restored quota-cache.bin has size 268435456: the rejected oversized edit must not have replaced its saved bytes. Delete quota-cache.bin with fs.unlinkSync, then write notes/after-quota.txt containing exactly 'Runs still work after quota cleanup'.", "quota-cleaned")); + await settle(6); + const cleanedRun = runs[5]!; + expect(cleanedRun.resultJson?.instructionSave).toMatchObject({ state: "saved", storageWarning: null }); + expect((await readPersonal("notes/after-quota.txt")).content).toBe("Runs still work after quota cleanup"); + const bundle = await api.get(`/api/agents/${fixtures.agent.id}/instructions-bundle`); + expect(bundle.files.some((file: Row) => file.path === "quota-cache.bin")).toBe(false); + await page.goto(`/${fixtures.company.issuePrefix}/agents/${fixtures.agent.id}/runs/${cleanedRun.id}`); + await expect(page.getByRole("note").filter({ hasText: "Agent storage warning" })).toHaveCount(0); + checks.push({ id: "storage-full-does-not-block-runs", passed: true, detail: "A run saved a file at quota, a subsequent run succeeded despite an oversized save rejection, and the next run removed the full file and cleared its warning; all three tasks completed" }); + await input.evidence("api-state.json", { issue, runs, checks, canonicalInstructions: resolved, attachments }); + await input.evidence("instruction-persistence.json", { checks, before, after, final, restored, board, candidates, resolved, syncEvents, runs, attachments, fullEvents }); + await page.goto(`/${fixtures.company.issuePrefix}/issues/${issue.identifier ?? issue.id}`); + await input.capture("storage-recovered", "Agent completed a normal task and cleared storage warning after cleanup", "storage-recovered.png"); + return { issue, runs, checks }; +} diff --git a/tests/runner-e2e/runner.spec.ts b/tests/runner-e2e/runner.spec.ts index 6f372e4b09..4802d10f3b 100644 --- a/tests/runner-e2e/runner.spec.ts +++ b/tests/runner-e2e/runner.spec.ts @@ -1,3 +1,4 @@ +import { runInstructionPersistenceFlow } from "./instruction-persistence.js"; import { gradeApiResponsePaging, readResponseProof, responseEvidenceDescription } from "./api-response-reading.js"; import { observeBrowserBootstrap } from "./browser-bootstrap-diagnostics.js"; import { runAccountingFlow } from "./accounting-flow.js"; @@ -546,7 +547,7 @@ for (const execution of executions) { const credentials = credentialValues(); const secrets = normalizedSecrets(Object.values(credentials)); const api = new RunnerApi(request); - const companyRunFlow = ["continuation_accounting", "continuation", "context_integrity", "agent_chat", "everyday_workflow", "first_task"].includes(execution.task.flow); + const companyRunFlow = ["continuation_accounting", "continuation", "context_integrity", "agent_chat", "everyday_workflow", "first_task", "instruction_persistence"].includes(execution.task.flow); const consoleDiagnostics: Array> = []; const networkDiagnostics: Array> = []; const pageLifecycleDiagnostics: Array> = []; @@ -859,6 +860,16 @@ for (const execution of executions) { evidence: (name, data) => writeSanitizedJson(snapshotsDir, name, data, secrets), }); issue = accounting.issue as IssueRecord; selectedRuns = accounting.runs as RunRecord[]; + } else if (execution.task.flow === "instruction_persistence") { + const story = await runInstructionPersistenceFlow({ + page, api, fixtures, execution, nonce, secrets, deadlineAt: startedAtMs + deadlineMs, + restart: () => restartIsolatedPaperclipServer({ api, requestId: `instructions-${nonce}`, deadlineAt: startedAtMs + deadlineMs }), + observe: (currentIssue, currentRuns) => { issue = currentIssue as IssueRecord; selectedRuns = currentRuns as RunRecord[]; }, + capture: captureScreenshot, + evidence: (name, data) => writeSanitizedJson(snapshotsDir, name, data, secrets), + }); + issue = story.issue as IssueRecord; selectedRuns = story.runs as RunRecord[]; + matcherResults = story.checks.map(check => ({ matcher: { kind: "json_path" as const, path: `instructions.${check.id}`, expected: true }, passed: check.passed, detail: check.detail })); } else if (execution.task.flow === "continuation") { const continuation = await runContinuationFlow({ page, api, fixtures, execution, nonce, secrets, workspacePath, deadlineAt: startedAtMs + deadlineMs - 60_000, diff --git a/tests/runner-e2e/types.ts b/tests/runner-e2e/types.ts index df99ee2bf0..91b07a80d0 100644 --- a/tests/runner-e2e/types.ts +++ b/tests/runner-e2e/types.ts @@ -25,7 +25,8 @@ export type RunnerTaskFlow = | "plan_revision_acceptance" | "question_resume_completion" | "plan_approval_completion" - | "warm_three_turn"; + | "warm_three_turn" + | "instruction_persistence"; export interface SecretReference { type: "secret_ref"; diff --git a/ui/src/api/agents.ts b/ui/src/api/agents.ts index 5bde9d5e01..62de19bcfc 100644 --- a/ui/src/api/agents.ts +++ b/ui/src/api/agents.ts @@ -5,6 +5,11 @@ import type { AgentPermissions, AgentDetail, AgentInstructionsBundle, + AgentInstructionHistory, + AgentInstructionCandidate, + ResolveAgentInstructionCandidate, + AgentInstructionDiff, + AgentInstructionSnapshot, AgentInstructionsFileDetail, AgentSkillSnapshot, AdapterEnvironmentTestResult, @@ -167,12 +172,26 @@ export const agentsApi = { ), saveInstructionsFile: ( id: string, - data: { path: string; content: string; clearLegacyPromptTemplate?: boolean }, + data: { path: string; content: string; baseRevisionId?: string | null; baseHash?: string | null; clearLegacyPromptTemplate?: boolean }, companyId?: string, ) => api.put(agentPath(id, companyId, "/instructions-bundle/file"), data), - deleteInstructionsFile: (id: string, relativePath: string, companyId?: string) => + instructionCandidates: (id: string, companyId?: string) => + api.get(agentPath(id, companyId, "/instructions-bundle/candidates")), + resolveInstructionCandidate: (id: string, runId: string, data: ResolveAgentInstructionCandidate, companyId?: string) => + api.post(agentPath(id, companyId, `/instructions-bundle/candidates/${runId}/resolve`), data), + instructionHistory: (id: string, path: string, companyId?: string, cursor?: string) => + api.get(agentPath(id, companyId, `/instructions-bundle/history?path=${encodeURIComponent(path)}${cursor ? `&cursor=${encodeURIComponent(cursor)}` : ""}`)), + instructionRevision: (id: string, path: string, revisionId: string, companyId?: string) => + api.get(agentPath(id, companyId, `/instructions-bundle/revision/${revisionId}?path=${encodeURIComponent(path)}`)), + instructionDiff: (id: string, path: string, from: string, to: string, companyId?: string) => + api.get(agentPath(id, companyId, `/instructions-bundle/diff?path=${encodeURIComponent(path)}&from=${encodeURIComponent(from)}&to=${encodeURIComponent(to)}`)), + restoreInstructions: (id: string, data: { path: string; revisionId: string; baseRevisionId: string }, companyId?: string) => + api.post(agentPath(id, companyId, "/instructions-bundle/restore"), data), + downloadInstructionsFile: (id: string, relativePath: string, companyId?: string) => + `/api${agentPath(id, companyId, `/instructions-bundle/file?path=${encodeURIComponent(relativePath)}&download=true`)}`, + deleteInstructionsFile: (id: string, relativePath: string, companyId?: string, baseHash?: string) => api.delete( - agentPath(id, companyId, `/instructions-bundle/file?path=${encodeURIComponent(relativePath)}`), + agentPath(id, companyId, `/instructions-bundle/file?path=${encodeURIComponent(relativePath)}${baseHash ? `&baseHash=${baseHash}` : ""}`), ), pause: (id: string, companyId?: string) => api.post(agentPath(id, companyId, "/pause"), {}), resume: (id: string, companyId?: string) => api.post(agentPath(id, companyId, "/resume"), {}), diff --git a/ui/src/components/InstructionHistory.tsx b/ui/src/components/InstructionHistory.tsx new file mode 100644 index 0000000000..2d8dbecfdf --- /dev/null +++ b/ui/src/components/InstructionHistory.tsx @@ -0,0 +1,145 @@ +import { useState } from "react"; +import { useInfiniteQuery, useMutation, useQuery } from "@tanstack/react-query"; +import type { AgentInstructionsFileDetail } from "@paperclipai/shared"; +import { agentsApi } from "../api/agents"; +import { Button } from "./ui/button"; + +export function InstructionHistory({ + agentId, + companyId, + path, + currentRevisionId, + disabled, + onRestored, +}: { + agentId: string; + companyId?: string; + path: string; + currentRevisionId: string; + disabled: boolean; + onRestored: (file: AgentInstructionsFileDetail) => void; +}) { + const [open, setOpen] = useState(false); + const [selected, setSelected] = useState(null); + const history = useInfiniteQuery({ + queryKey: ["instruction-history", agentId, path, currentRevisionId], + queryFn: ({ pageParam }) => + agentsApi.instructionHistory(agentId, path, companyId, pageParam), + initialPageParam: undefined as string | undefined, + getNextPageParam: (page) => page.nextCursor ?? undefined, + enabled: open, + }); + const diff = useQuery({ + queryKey: ["instruction-diff", agentId, path, selected, currentRevisionId], + queryFn: () => + agentsApi.instructionDiff( + agentId, + path, + selected!, + currentRevisionId, + companyId, + ), + enabled: open && Boolean(selected), + }); + const restore = useMutation({ + mutationFn: () => + agentsApi.restoreInstructions( + agentId, + { path, revisionId: selected!, baseRevisionId: currentRevisionId }, + companyId, + ), + onSuccess: (file) => { + setSelected(null); + onRestored(file); + }, + }); + const error = history.error ?? diff.error ?? restore.error; + return ( +
+ + {open && ( +
+ {history.isLoading && ( +

Loading revisions…

+ )} + {error && ( +

+ {error.message} +

+ )} + {history.data?.pages + .flatMap((page) => page.revisions) + .map((revision) => ( +
+ + + {revision.source} ·{" "} + {new Date(revision.createdAt).toLocaleString()} + {revision.id === currentRevisionId ? " · Current" : ""} + +
+ ))} + {history.hasNextPage && ( + + )} + {diff.data && ( + <> +

Selected revision

+ 
+                {diff.data.from.content}
+
+

+ Changes from selected revision to current +

+ 
+                {diff.data.removed && `Removed:\n${diff.data.removed}\n`}
+                {diff.data.added && `Added:\n${diff.data.added}`}
+                {!diff.data.removed &&
+                  !diff.data.added &&
+                  "No content changes."}
+
+ + {disabled && ( +

+ Save or cancel your edits before restoring. +

+ )} + + )} +
+ )} +
+ ); +} diff --git a/ui/src/lib/queryKeys.ts b/ui/src/lib/queryKeys.ts index 56c4879b83..37bbd92812 100644 --- a/ui/src/lib/queryKeys.ts +++ b/ui/src/lib/queryKeys.ts @@ -215,6 +215,8 @@ export const queryKeys = { skills: (id: string) => ["agents", "skills", id] as const, instructionsBundle: (id: string) => ["agents", "instructions-bundle", id] as const, + instructionCandidates: (id: string) => + ["agents", "instruction-candidates", id] as const, instructionsFile: (id: string, relativePath: string) => ["agents", "instructions-bundle", id, "file", relativePath] as const, keys: (agentId: string) => ["agents", "keys", agentId] as const, diff --git a/ui/src/pages/AgentDetail.instructions.test.tsx b/ui/src/pages/AgentDetail.instructions.test.tsx index bd33d7c435..7fe9ba0daf 100644 --- a/ui/src/pages/AgentDetail.instructions.test.tsx +++ b/ui/src/pages/AgentDetail.instructions.test.tsx @@ -6,7 +6,8 @@ import { createRoot, type Root } from "react-dom/client"; import { QueryClient, QueryClientProvider } from "@tanstack/react-query"; import type { Agent, AgentInstructionsBundle, AgentInstructionsFileDetail, AgentInstructionsFileSummary } from "@paperclipai/shared"; import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; -import { PromptsTab } from "./AgentDetail"; +import { AgentFileRunNotice, PromptsTab } from "./AgentDetail"; +import { queryKeys } from "../lib/queryKeys"; const mockAgentsApi = vi.hoisted(() => ({ instructionsBundle: vi.fn(), @@ -14,9 +15,17 @@ const mockAgentsApi = vi.hoisted(() => ({ updateInstructionsBundle: vi.fn(), saveInstructionsFile: vi.fn(), deleteInstructionsFile: vi.fn(), + instructionHistory: vi.fn(), + instructionCandidates: vi.fn(), + resolveInstructionCandidate: vi.fn(), + instructionDiff: vi.fn(), + restoreInstructions: vi.fn(), })); const markdownEditorRenderMock = vi.hoisted(() => vi.fn()); +const copyTextToClipboardMock = vi.hoisted(() => vi.fn(async (_text: string) => {})); + +vi.mock("../lib/clipboard", () => ({ copyTextToClipboard: copyTextToClipboardMock })); vi.mock("../api/agents", () => ({ agentsApi: mockAgentsApi, @@ -226,6 +235,7 @@ describe("PromptsTab instruction editor", () => { saveAction = null; markdownEditorRenderMock.mockClear(); Object.values(mockAgentsApi).forEach((mock) => mock.mockReset()); + mockAgentsApi.instructionCandidates.mockResolvedValue([]); mockAgentsApi.updateInstructionsBundle.mockResolvedValue({}); mockAgentsApi.saveInstructionsFile.mockImplementation(async (_agentId, data) => ({ path: data.path, @@ -333,6 +343,7 @@ describe("PromptsTab instruction editor", () => { { path: "AGENTS", content: "# Updated", + baseRevisionId: null, clearLegacyPromptTemplate: false, }, "company-1", @@ -340,6 +351,151 @@ describe("PromptsTab instruction editor", () => { }); }); + it("loads preserved edits against the displayed head and retains their pinned base after a conflict", async () => { + const summary = makeSummary("AGENTS.md", "AGENTS.md"); + const revision = { id: "head-1", entryFile: "AGENTS.md" } as NonNullable; + mockAgentsApi.instructionCandidates.mockResolvedValue([{ runId: "preserved-run", entryFile: "AGENTS.md", baseRevisionId: "older-base", baseHash: "base-hash", state: "conflict", candidateHash: "candidate-hash", content: "preserved edit", errorCode: "INSTRUCTION_REVISION_CONFLICT", errorMessage: "Instructions changed", createdAt: "2026-01-01T00:00:00Z", updatedAt: "2026-01-01T00:00:00Z" }]); + await renderPromptsTab(makeBundle("AGENTS.md", [summary]), { "AGENTS.md": makeDetail(summary, "current content", { revision }) }); + await waitFor(() => expect(buttonByText(container, "Review preserved edits")).toBeDefined()); + await act(async () => { buttonByText(container, "Review preserved edits").click(); }); + const editor = await waitFor(() => { + const element = container.querySelector('[data-testid="markdown-editor"]'); + expect(element?.value).toBe("preserved edit"); return element!; + }); + mockAgentsApi.resolveInstructionCandidate.mockRejectedValue(new Error("Instructions changed since the base revision")); + await waitFor(() => expect(saveAction).toEqual(expect.any(Function))); + await act(async () => { saveAction?.(); }); + await waitFor(() => expect(mockAgentsApi.resolveInstructionCandidate).toHaveBeenCalledWith("agent-1", "preserved-run", { content: "preserved edit", baseRevisionId: "head-1" }, "company-1")); + expect(mockAgentsApi.saveInstructionsFile).not.toHaveBeenCalled(); + expect(editor.value).toBe("preserved edit"); + await act(async () => { queryClient.setQueryData(["agents", "instructions-bundle", "agent-1", "file", "AGENTS.md"], makeDetail(summary, "newer content", { revision: { ...revision, id: "head-2" } })); }); + await act(async () => { saveAction?.(); }); + await waitFor(() => expect(mockAgentsApi.resolveInstructionCandidate).toHaveBeenCalledTimes(2)); + expect(mockAgentsApi.resolveInstructionCandidate.mock.lastCall?.[2].baseRevisionId).toBe("head-1"); + expect(editor.value).toBe("preserved edit"); + mockAgentsApi.instructionsFile.mockResolvedValue(makeDetail(summary, "latest content", { revision: { ...revision, id: "head-3" } })); + await waitFor(() => expect(buttonByText(container, "Refresh current revision").disabled).toBe(false)); + await act(async () => { buttonByText(container, "Refresh current revision").click(); }); + await waitFor(() => expect(container.textContent).toContain("latest content")); + expect(editor.value).toBe("preserved edit"); + expect(mockAgentsApi.resolveInstructionCandidate).toHaveBeenCalledTimes(2); + await act(async () => { saveAction?.(); }); + await waitFor(() => expect(mockAgentsApi.resolveInstructionCandidate).toHaveBeenCalledTimes(3)); + expect(mockAgentsApi.resolveInstructionCandidate.mock.lastCall?.[2].baseRevisionId).toBe("head-3"); + }); + + it("keeps historical whole-folder failures out of the current editor while preserving legacy review", async () => { + const summary = makeSummary("AGENTS.md", "AGENTS.md"); + const failures = [1, 2, 3].map(attempt => ({ contract: "agent_files", runId: `failed-auth-${attempt}`, + entryFile: "AGENTS.md", state: "unavailable", content: null, + errorMessage: "The registered instruction copy could not be retrieved safely before environment release. No instruction save is claimed." })); + mockAgentsApi.instructionCandidates.mockResolvedValue(failures); + await renderPromptsTab(makeBundle("AGENTS.md", [summary], { persistence: "agent_files" }), { + "AGENTS.md": makeDetail(summary, "Successfully saved agent instructions"), + }); + await waitFor(() => expect(container.textContent).toContain("Successfully saved agent instructions")); + expect(container.textContent).not.toContain("could not be retrieved"); + expect(container.textContent).not.toContain("Preserved instruction edits"); + expect(container.querySelector('[role="alert"]')).toBeNull(); + await act(async () => { + queryClient.setQueryData(queryKeys.agents.instructionCandidates("agent-1"), [...failures, { + contract: "legacy", runId: "preserved-run", entryFile: "AGENTS.md", state: "conflict", + content: "Legacy edits to review", createdAt: "2026-01-01T00:00:00Z", + }]); + }); + await waitFor(() => expect(buttonByText(container, "Review preserved edits").disabled).toBe(false)); + expect(container.textContent).toContain("Preserved instruction edits"); + expect(container.textContent).not.toContain("could not be retrieved"); + }); + + it("scopes sync warnings to the affected run and displays a storage warning only once", async () => { + root = createRoot(container); + const render = async (instructionSave: Record) => act(async () => { + root?.render(); + }); + await render({ contract: "agent_files", state: "unavailable", errorMessage: "This run's files were not saved." }); + expect(container.textContent).toContain("Agent file sync failed for this run"); + expect(container.textContent).toContain("This run's files were not saved."); + await render({ contract: "agent_files", state: "unavailable", errorCode: "AGENT_FILES_LIMIT_EXCEEDED", errorMessage: "Save rejected", storageWarning: "Agent storage is full. Runs can continue." }); + expect(container.querySelectorAll('[role="note"]')).toHaveLength(1); + expect(container.textContent).toContain("Runs can continue"); + expect(container.textContent).not.toContain("Save rejected"); + await render({ contract: "agent_files", state: "unavailable", errorCode: "AGENT_FILES_SAVE_FAILED", errorMessage: "An I/O failure prevented saving this run's files.", storageWarning: "Agent storage is full. Runs can continue." }); + expect(container.querySelectorAll('[role="note"]')).toHaveLength(2); + expect(container.textContent).toContain("Runs can continue"); + expect(container.textContent).toContain("An I/O failure prevented saving this run's files."); + await render({ contract: "agent_files", state: "saved" }); + expect(container.textContent).toBe(""); + await render({ state: "conflict", errorMessage: "Legacy candidate needs review" }); + expect(container.textContent).toBe(""); + }); + + it("keeps changed-entry preserved edits readable and copyable without enabling a save", async () => { + const summary = makeSummary("CURRENT.md", "CURRENT.md"); + const revision = { id: "current-head", entryFile: "CURRENT.md" } as NonNullable; + const preserved = "# Original entry\nPreserve these exact edits.\n"; + mockAgentsApi.instructionCandidates.mockResolvedValue([{ runId: "old-entry-run", entryFile: "OLD.md", baseRevisionId: "old-head", baseHash: "base-hash", state: "conflict", candidateHash: "candidate-hash", content: preserved, errorCode: "INSTRUCTION_ENTRY_CHANGED", errorMessage: "The instruction entry changed", createdAt: "2026-01-01T00:00:00Z", updatedAt: "2026-01-01T00:00:00Z" }]); + await renderPromptsTab(makeBundle("CURRENT.md", [summary]), { "CURRENT.md": makeDetail(summary, "Current instructions", { revision }) }); + await waitFor(() => expect(buttonByText(container, "Review preserved edits").disabled).toBe(false)); + await act(async () => { buttonByText(container, "Review preserved edits").click(); }); + const review = await waitFor(() => { + const region = container.querySelector('[aria-label="Preserved edits for OLD.md"]'); + expect(region?.querySelector("pre")?.textContent).toBe(preserved); + return region!; + }); + expect(review.textContent).toContain("Read only"); + expect(review.textContent).toContain("CURRENT.md"); + expect(review.querySelector("textarea, input, [contenteditable=true]")).toBeNull(); + expect(saveAction).toBeNull(); + expect(mockAgentsApi.instructionsFile.mock.calls.every((call) => call[1] === "CURRENT.md")).toBe(true); + await act(async () => { review.querySelector('[aria-label="Copy preserved edits for OLD.md"]')!.click(); }); + expect(copyTextToClipboardMock).toHaveBeenCalledWith(preserved); + expect(saveAction).toBeNull(); + expect(mockAgentsApi.resolveInstructionCandidate).not.toHaveBeenCalled(); + expect(mockAgentsApi.saveInstructionsFile).not.toHaveBeenCalled(); + + // Choosing to edit and paste into the current entry is a separate ordinary save. + await selectInstructionMode("Edit"); + const editor = container.querySelector('[data-testid="markdown-editor"]')!; + expect(editor.value).toBe("Current instructions"); + await act(async () => { editor.dispatchEvent(new MouseEvent("pointerdown", { bubbles: true })); setNativeValue(editor, preserved); }); + await waitFor(() => expect(saveAction).toEqual(expect.any(Function))); + await act(async () => { saveAction?.(); }); + await waitFor(() => expect(mockAgentsApi.saveInstructionsFile).toHaveBeenCalledWith("agent-1", { path: "CURRENT.md", content: preserved, baseRevisionId: "current-head", clearLegacyPromptTemplate: false }, "company-1")); + expect(mockAgentsApi.resolveInstructionCandidate).not.toHaveBeenCalled(); + }); + + it("retains the draft and its base when a concurrent save conflicts", async () => { + const summary = makeSummary("AGENTS.md", "AGENTS.md"); + const revision = { id: "base-1", entryFile: "AGENTS.md" } as NonNullable; + await renderPromptsTab(makeBundle("AGENTS.md", [summary]), { "AGENTS.md": makeDetail(summary, "original", { revision }) }); + mockAgentsApi.saveInstructionsFile.mockRejectedValue(new Error("Instructions changed since the base revision")); + await selectInstructionMode("Edit"); + const editor = container.querySelector('[data-testid="markdown-editor"]')!; + await act(async () => { editor.dispatchEvent(new MouseEvent("pointerdown", { bubbles: true })); setNativeValue(editor, "my unsaved edit"); }); + await waitFor(() => expect(saveAction).toEqual(expect.any(Function))); + await act(async () => { saveAction?.(); }); + await waitFor(() => expect(container.querySelector('[role="alert"]')?.textContent).toContain("base revision")); + expect(editor.value).toBe("my unsaved edit"); + expect(mockAgentsApi.saveInstructionsFile).toHaveBeenCalledWith("agent-1", expect.objectContaining({ content: "my unsaved edit", baseRevisionId: "base-1" }), "company-1"); + }); + + it("shows revision content and restores against the displayed current head", async () => { + const summary = makeSummary("AGENTS.md", "AGENTS.md"); + const revision = { id: "current-1", entryFile: "AGENTS.md" } as NonNullable; + mockAgentsApi.instructionHistory.mockResolvedValue({ revisions: [{ id: "old-revision", source: "board", createdAt: "2026-01-01T00:00:00Z" }], nextCursor: null }); + mockAgentsApi.instructionDiff.mockResolvedValue({ from: { content: "old text" }, removed: "old", added: "new" }); + mockAgentsApi.restoreInstructions.mockResolvedValue(makeDetail(summary, "old text", { revision: { ...revision, id: "restored-1" } })); + await renderPromptsTab(makeBundle("AGENTS.md", [summary]), { "AGENTS.md": makeDetail(summary, "new text", { revision }) }); + await waitFor(() => expect(buttonByText(container, "History")).toBeDefined()); + await act(async () => { buttonByText(container, "History").click(); }); + await waitFor(() => expect(buttonByText(container, "old-revi")).toBeDefined()); + await act(async () => { buttonByText(container, "old-revi").click(); }); + await waitFor(() => expect(container.textContent).toContain("old text")); + await act(async () => { buttonByText(container, "Restore as new revision").click(); }); + await waitFor(() => expect(mockAgentsApi.restoreInstructions).toHaveBeenCalledWith("agent-1", { path: "AGENTS.md", revisionId: "old-revision", baseRevisionId: "current-1" }, "company-1")); + }); + it("ignores rich-editor mount normalization until the user interacts", async () => { const summary = makeSummary("AGENTS.md", "AGENTS.md"); const onDirtyChange = vi.fn(); diff --git a/ui/src/pages/AgentDetail.tsx b/ui/src/pages/AgentDetail.tsx index b6896c6f72..4beff8f0cb 100644 --- a/ui/src/pages/AgentDetail.tsx +++ b/ui/src/pages/AgentDetail.tsx @@ -1,3 +1,5 @@ +import type { AgentInstructionCandidate, AgentInstructionsBundle } from "@paperclipai/shared"; +import { InstructionHistory } from "../components/InstructionHistory"; import { AgentCharacter } from "../components/AgentCharacter"; import { characterStateForAgent } from "@paperclipai/shared"; import { mergeRunLogChunks, readChunkSeq } from "../lib/run-log-chunks"; @@ -2197,6 +2199,12 @@ export function PromptsTab({ const [instructionMode, setInstructionMode] = useState<"read" | "edit" | "raw">("read"); const [showFilePanel, setShowFilePanel] = useState(false); const [draft, setDraft] = useState(null); + const draftBaseRevisionRef = useRef(undefined); + const draftBaseHashRef = useRef(undefined); + const [candidateRunId, setCandidateRunId] = useState(null); + const [readOnlyCandidateRunId, setReadOnlyCandidateRunId] = useState(null); + const candidateAgentRef = useRef(agent.id); + candidateAgentRef.current = agent.id; const [bundleDraft, setBundleDraft] = useState<{ mode: "managed" | "external"; rootPath: string; @@ -2225,6 +2233,9 @@ export function PromptsTab({ }, []); const setSelectedFile = useCallback((filePath: string) => { editorInteractedRef.current = false; + draftBaseRevisionRef.current = undefined; + draftBaseHashRef.current = undefined; + setCandidateRunId(null); setSelectedFileState(filePath); }, []); @@ -2234,6 +2245,7 @@ export function PromptsTab({ setInstructionMode("read"); setShowFilePanel(false); setDraft(null); + setReadOnlyCandidateRunId(null); setBundleDraft(null); setNewFilePath(""); setShowNewFileInput(false); @@ -2251,6 +2263,7 @@ export function PromptsTab({ queryKey: queryKeys.agents.instructionsBundle(agent.id), queryFn: () => agentsApi.instructionsBundle(agent.id, companyId), enabled: Boolean(companyId && isLocal), + refetchInterval: draft === null ? 5000 : false, }); const persistedMode = bundle?.mode ?? "managed"; @@ -2284,10 +2297,68 @@ export function PromptsTab({ const selectedFileExists = bundleMatchesDraft && fileOptions.includes(selectedOrEntryFile); const selectedFileSummary = bundle?.files.find((file) => file.path === selectedOrEntryFile) ?? null; - const { data: selectedFileDetail, isLoading: fileLoading } = useQuery({ + const { data: selectedFileDetail, isLoading: fileLoading, error: fileError } = useQuery({ queryKey: queryKeys.agents.instructionsFile(agent.id, selectedOrEntryFile), queryFn: () => agentsApi.instructionsFile(agent.id, selectedOrEntryFile, companyId), enabled: Boolean(companyId && isLocal && selectedFileExists), + refetchInterval: draft === null ? 5000 : false, + }); + + const candidates = useQuery({ + queryKey: queryKeys.agents.instructionCandidates(agent.id), + queryFn: () => agentsApi.instructionCandidates(agent.id, companyId), + enabled: Boolean(companyId && isLocal && currentMode === "managed"), + }); + // Whole-folder failures belong to their run's diagnostics. They have no + // preserved edits to resolve and do not describe the current saved files. + const preservedCandidates = candidates.data?.filter((candidate) => candidate.contract !== "agent_files") ?? []; + const loadCandidate = useMutation({ + mutationFn: async (candidate: AgentInstructionCandidate) => { + if (candidate.content === null) throw new Error("These instruction edits have not been retrieved yet."); + const file = await agentsApi.instructionsFile(agent.id, candidate.entryFile, companyId).catch((error) => { + if (error instanceof ApiError && error.status === 404 && candidate.baseRevisionId === null) return null; + throw error; + }); + return { candidate, file, agentId: agent.id }; + }, + onSuccess: ({ candidate, file, agentId: requestedAgentId }) => { + if (candidateAgentRef.current !== requestedAgentId) return; + setSelectedFile(candidate.entryFile); + if (file) queryClient.setQueryData(queryKeys.agents.instructionsFile(agent.id, candidate.entryFile), file); + draftBaseRevisionRef.current = file?.revision?.id ?? null; + setCandidateRunId(candidate.runId); + setDraft(candidate.content); + setInstructionMode("edit"); + }, + }); + const resolveCandidate = useMutation({ + mutationFn: async (data: { runId: string; content: string; baseRevisionId: string | null }) => ({ + file: await agentsApi.resolveInstructionCandidate(agent.id, data.runId, { content: data.content, baseRevisionId: data.baseRevisionId }, companyId), + agentId: agent.id, + }), + onSuccess: ({ file, agentId: requestedAgentId }) => { + if (candidateAgentRef.current !== requestedAgentId) return; + setDraft(null); + setCandidateRunId(null); + draftBaseRevisionRef.current = undefined; + draftBaseHashRef.current = undefined; + queryClient.setQueryData(queryKeys.agents.instructionsFile(agent.id, file.path), file); + queryClient.invalidateQueries({ queryKey: queryKeys.agents.instructionsBundle(agent.id) }); + queryClient.invalidateQueries({ queryKey: queryKeys.agents.instructionCandidates(agent.id) }); + }, + }); + + const refreshCandidateBase = useMutation({ + mutationFn: async () => ({ + file: await agentsApi.instructionsFile(agent.id, selectedOrEntryFile, companyId), + agentId: agent.id, runId: candidateRunId, + }), + onSuccess: ({ file, agentId: requestedAgentId, runId }) => { + if (candidateAgentRef.current !== requestedAgentId || candidateRunId !== runId) return; + draftBaseRevisionRef.current = file.revision?.id ?? null; + queryClient.setQueryData(queryKeys.agents.instructionsFile(agent.id, file.path), file); + resolveCandidate.reset(); + }, }); const updateBundle = useMutation({ @@ -2299,9 +2370,9 @@ export function PromptsTab({ }) => agentsApi.updateInstructionsBundle(agent.id, data, companyId), onMutate: () => { editorInteractedRef.current = false; - setAwaitingRefresh(true); }, onSuccess: () => { + setAwaitingRefresh(true); queryClient.invalidateQueries({ queryKey: queryKeys.agents.instructionsBundle(agent.id) }); queryClient.invalidateQueries({ queryKey: queryKeys.agents.detail(agent.id) }); queryClient.invalidateQueries({ queryKey: queryKeys.agents.detail(agent.urlKey) }); @@ -2310,13 +2381,25 @@ export function PromptsTab({ }); const saveFile = useMutation({ - mutationFn: (data: { path: string; content: string; clearLegacyPromptTemplate?: boolean }) => + mutationFn: (data: { path: string; content: string; baseRevisionId?: string | null; baseHash?: string | null; clearLegacyPromptTemplate?: boolean }) => agentsApi.saveInstructionsFile(agent.id, data, companyId), onMutate: () => { editorInteractedRef.current = false; - setAwaitingRefresh(true); }, - onSuccess: (_, variables) => { + onSuccess: (file, variables) => { + setDraft(null); + draftBaseRevisionRef.current = undefined; + draftBaseHashRef.current = undefined; + queryClient.setQueryData(queryKeys.agents.instructionsFile(agent.id, variables.path), file); + // Keep the selected file present while the refreshed bundle is in flight. + // Otherwise removing its pending placeholder briefly selects AGENTS.md. + queryClient.setQueryData(queryKeys.agents.instructionsBundle(agent.id), previous => previous ? { + ...previous, files: [...previous.files.filter(item => item.path !== file.path), { + path: file.path, size: file.size, language: file.language, markdown: file.markdown, + isEntryFile: file.isEntryFile, editable: file.editable, deprecated: file.deprecated, + virtual: file.virtual, binary: file.binary, contentHash: file.contentHash, + }], + } : previous); setPendingFiles((prev) => prev.filter((f) => f !== variables.path)); queryClient.invalidateQueries({ queryKey: queryKeys.agents.instructionsBundle(agent.id) }); queryClient.invalidateQueries({ queryKey: queryKeys.agents.instructionsFile(agent.id, variables.path) }); @@ -2327,7 +2410,7 @@ export function PromptsTab({ }); const deleteFile = useMutation({ - mutationFn: (relativePath: string) => agentsApi.deleteInstructionsFile(agent.id, relativePath, companyId), + mutationFn: (relativePath: string) => agentsApi.deleteInstructionsFile(agent.id, relativePath, companyId, bundle?.files.find(file => file.path === relativePath)?.contentHash), onMutate: () => { editorInteractedRef.current = false; setAwaitingRefresh(true); @@ -2409,7 +2492,7 @@ export function PromptsTab({ return; } if (lastFileVersionRef.current !== versionKey) { - setDraft(null); + if (draftBaseRevisionRef.current === undefined) setDraft(null); lastFileVersionRef.current = versionKey; } }, [awaitingRefresh, currentMode, currentRootPath, selectedFileDetail, selectedFileExists, selectedOrEntryFile]); @@ -2452,8 +2535,8 @@ export function PromptsTab({ ), ); const fileDirty = draft !== null && draft !== currentContent; - const isDirty = bundleDirty || fileDirty; - const isSaving = updateBundle.isPending || saveFile.isPending || deleteFile.isPending || awaitingRefresh; + const isDirty = bundleDirty || fileDirty || candidateRunId !== null; + const isSaving = updateBundle.isPending || saveFile.isPending || resolveCandidate.isPending || loadCandidate.isPending || refreshCandidateBase.isPending || deleteFile.isPending || awaitingRefresh; useEffect(() => { onSavingChange(isSaving); }, [onSavingChange, isSaving]); useEffect(() => { onDirtyChange(isDirty); }, [onDirtyChange, isDirty]); @@ -2477,10 +2560,15 @@ export function PromptsTab({ entryFile: bundleDraft.entryFile, }); } - if (fileDirty) { + if (candidateRunId) { + await resolveCandidate.mutateAsync({ runId: candidateRunId, content: displayValue, + baseRevisionId: draftBaseRevisionRef.current ?? null }); + } else if (fileDirty) { await saveFile.mutateAsync({ path: selectedOrEntryFile, content: displayValue, + ...(bundle?.persistence === "agent_files" ? { baseHash: draftBaseHashRef.current !== undefined ? draftBaseHashRef.current : selectedFileDetail?.contentHash ?? null } : {}), + ...(selectedOrEntryFile === currentEntryFile && currentMode === "managed" ? { baseRevisionId: draftBaseRevisionRef.current !== undefined ? draftBaseRevisionRef.current : selectedFileDetail?.revision?.id ?? null } : {}), clearLegacyPromptTemplate: shouldClearLegacy, }); } @@ -2491,10 +2579,15 @@ export function PromptsTab({ bundle, bundleDirty, bundleDraft, + candidateRunId, + resolveCandidate, displayValue, fileDirty, isDirty, onSaveActionChange, + selectedFileDetail?.revision?.id, + currentEntryFile, + currentMode, saveFile, selectedOrEntryFile, updateBundle, @@ -2502,6 +2595,10 @@ export function PromptsTab({ useEffect(() => { onCancelActionChange(isDirty ? () => { + draftBaseRevisionRef.current = undefined; + draftBaseHashRef.current = undefined; + setCandidateRunId(null); + resolveCandidate.reset(); setDraft(null); if (bundle) { setBundleDraft({ @@ -2511,7 +2608,7 @@ export function PromptsTab({ }); } } : null); - }, [bundle, isDirty, onCancelActionChange, persistedMode, persistedRootPath]); + }, [bundle, isDirty, onCancelActionChange, persistedMode, persistedRootPath, resolveCandidate]); const handleSeparatorDrag = useCallback((event: React.MouseEvent) => { event.preventDefault(); @@ -2742,6 +2839,7 @@ export function PromptsTab({ size="icon" variant="outline" className="h-7 w-7" + aria-label="Add agent file" onClick={() => setShowNewFileInput(true)} > + @@ -2938,7 +3036,68 @@ export function PromptsTab({ - {selectedFileExists && fileLoading && !selectedFileDetail ? ( + {currentMode === "managed" && preservedCandidates.length > 0 && ( +
+

Preserved instruction edits

+

Older instruction-only sessions have edits to review.

+ {preservedCandidates.map((candidate) => ( +
+ {candidate.runId.slice(0, 8)} + {candidate.entryFile} · {formatDate(candidate.createdAt)} + + {candidate.errorMessage &&

{candidate.errorMessage}

} + {candidate.entryFile !== currentEntryFile &&

The instruction entry changed. These edits remain preserved for the original file.

} + {candidate.entryFile !== currentEntryFile && candidate.content !== null && readOnlyCandidateRunId === candidate.runId && ( +
+

Read only: {candidate.entryFile}. To keep any of these edits in {currentEntryFile}, copy them and edit the current entry explicitly.

+ + Copy preserved edits + + 
{candidate.content}
+
+ )} +
+ ))} +
+ )} + {candidateRunId &&
+

Reviewing preserved edits. Save to apply your resolved draft and close this preserved edit.

+
Compare current instructions + 
{currentContent}
+
+ {draftBaseRevisionRef.current !== (selectedFileDetail?.revision?.id ?? null) &&

The current instructions changed after this draft was loaded. Refresh the current revision, compare the instructions, and save your resolved draft again.

} + {(resolveCandidate.error || draftBaseRevisionRef.current !== (selectedFileDetail?.revision?.id ?? null)) && } +
} + {(candidates.error || loadCandidate.error || resolveCandidate.error || refreshCandidateBase.error) &&

{(candidates.error ?? loadCandidate.error ?? resolveCandidate.error ?? refreshCandidateBase.error)?.message} Your preserved edits remain available.

} + {(saveFile.error || fileError || updateBundle.error) &&

{(saveFile.error ?? fileError ?? updateBundle.error)?.message} Your unsaved edits are retained.

} + {selectedFileDetail?.receipt?.materialization === "pending" &&

Revision saved. The instruction file still needs to be rebuilt from the saved revision.

} + {selectedFileDetail?.revision && currentMode === "managed" && bundle?.persistence !== "agent_files" && { + setDraft(null); + draftBaseRevisionRef.current = undefined; + draftBaseHashRef.current = undefined; + queryClient.setQueryData(queryKeys.agents.instructionsFile(agent.id, selectedOrEntryFile), file); + queryClient.invalidateQueries({ queryKey: queryKeys.agents.instructionsBundle(agent.id) }); + }} + />} + {selectedFileDetail?.binary ? ( +
+

This file is preserved with the agent directory. Download it to view its contents.

+ Download {selectedOrEntryFile} +
+ ) : selectedFileExists && fileLoading && !selectedFileDetail ? ( ) : instructionMode === "read" ? (
@@ -2975,6 +3134,8 @@ export function PromptsTab({ value={displayValue} onChange={(value) => { if (!editorInteractedRef.current) return; + if (draftBaseRevisionRef.current === undefined) draftBaseRevisionRef.current = selectedFileDetail?.revision?.id ?? null; + if (draftBaseHashRef.current === undefined) draftBaseHashRef.current = selectedFileDetail?.contentHash ?? null; setDraft(value ?? ""); }} placeholder="# Agent instructions" @@ -2991,7 +3152,11 @@ export function PromptsTab({