Merge master agent-file persistence into rich ACP foundation

Preserve the extended harness and instruction-persistence suites, generated
contracts, and cleanup/collection ordering. Record the unqualified ACPX
agent-directory capability without broadening environment or path policy.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip committed 2026-09-29 08:36:37 -05:00
commit 8747917668
100 files changed
+7950 -262

No files matched your search

+11
View File
@@ -442,6 +442,17 @@ revokes the previous boot identity with a conditional update. Its own claim also
expires so another sweep can finish cleanup after a restart. Historical rows keep
null ownership fields and follow the previous recovery path.
## Agent file persistence and legacy revisions
Managed agent files are current filesystem contents, using the same persistent
instance storage as other workspaces. `agent_instruction_revisions` and
`agent_instruction_heads` are retained as read-only upgrade input. Their heads
are adopted once into the managed directory; new saves never append revisions.
`agent_instruction_working_copies` holds per-run baseline hashes, state, and
capture receipts. New receipts identify `paperclip.agent-files.v1`; historical
rows retain the instruction-only format. Completed directory runs discard their
baseline and private copies. See [Persistent agent files](agent-files.md).
## Large API response snapshots
`assets.byte_size` uses PostgreSQL `bigint` so saved responses and byte ranges can
+17
View File
@@ -1725,6 +1725,23 @@ read or change these preferences. The legacy instance general setting is retaine
for API compatibility but no longer controls shortcut behavior in the app;
users opt in individually after the upgrade.
### Persistent managed agent files (2026-09-28)
The Instructions Editor and agent execution share one current agent-owned
directory, scoped by company and agent. The configured instruction entry is one
file in this directory. Registered private copies synchronize supported files
across tasks and sessions, separately from task workspace persistence. Saves
require verified provider stop and current authorization, then synchronize changed
files using per-file last-sync-wins;
new content is not stored as revision history. Existing deployed revisions and
saved execution formats remain compatible during adoption. See
[Persistent agent files](agent-files.md).
Persistent-file storage limits are advisory for execution: a full folder cannot
pause the agent, fail its run, or prevent later runs. Show a warning on each run
while storage remains full, restore existing files so the agent can remove them,
and enforce the limits on saves. Cleanup clears the warning for future runs.
### Unsafe native workspace exports
An unsafe workspace link does not fail an accepted native task result. Retry
+12
View File
@@ -620,6 +620,18 @@ read or change these preferences. The legacy instance general setting is retaine
for API compatibility but no longer controls shortcut behavior in the app;
users opt in individually after the upgrade.
Managed agents own a persistent file directory across tasks and sessions. The
Instructions Editor and stopped agent execution synchronize the same current
files, including AGENTS.md and its supporting files. Task working directories and
provider home directories remain separate concepts. Concurrent runs synchronize only
the files they change, with the last sync winning for the same file. Temporary
copies are cleaned up; this storage does not add a revision-history system. See
[agent-files.md](agent-files.md) for lifecycle and upgrade compatibility.
Full agent storage produces a run warning without stopping current or future
work. Storage limits constrain saved file changes, not the agent's ability to run
and remove files to recover space.
### Unsafe native workspace exports
An unsafe workspace link does not fail an accepted native task result. Retry
+177
View File
@@ -0,0 +1,177 @@
# Persistent agent files
Each managed agent has one current directory, scoped by company and agent. The
Instructions Editor reads and writes this directory. `AGENTS.md` (or the
configured entry) is one file in it. Agents may create ordinary files and nested
folders for notes, memory, and other personal working material. Files in the
task working directory remain task files.
Agents can edit their own managed files under the responsible user’s current
target permissions. Access to another agent’s files additionally requires the
caller’s own target-scoped configuration permission; shared company membership
or a responsible user alone does not grant peer access.
## Layout
The canonical host directory keeps its existing physical location:
```
<instance>/companies/<company>/agents/<agent>/
instructions/ current agent files (editor)
AGENTS.md
notes/
any-supported-file
file-sync/ controller-only operational state
adopted.json
runs/<run>/live/ isolated writable copy while a run executes
```
The process starts in its existing task workspace. `AGENT_HOME` points to the
registered writable agent copy. Adapter `HOME` and `CODEX_HOME` keep their
existing meanings and are not personal-file storage. Local copies are outside
the task workspace. Remote providers currently confine file sync to their
workspace: their independent agent copy therefore lives under the excluded
`.paperclip-runtime/agent-files/<agent>/<run>/` area. It is not included in task
workspace sync, Git staging, or task deliverables.
Regular files (including binary bytes) and directories are supported, up to
100,000 entries (files and folders), 256 MiB per file and 2 GiB total. Symlinks,
hardlinks, and special
files are rejected, rather than followed or silently skipped. The instruction
entry remains valid UTF-8, at most 1 MiB, and cannot be deleted. The editor edits
text up to 1 MiB and offers downloads for binary or larger files. The reserved
`.paperclip-runtime` directory and the compatibility-only virtual file
`promptTemplate.legacy.md` are not user storage. Task cache and Git ignore
exclusions do not apply to this directory.
These storage limits are separate from the 1 MiB instruction/editor limit. Large
files are hashed and downloaded as streams; listings bound concurrent reads,
and only editor-sized text is buffered. Storage counts uncompressed file bytes,
not allocated disk
blocks. These are sync validation limits, not live filesystem quotas: an agent
can write beyond them while running. Storage limits never pause an agent, fail
a provider run, or block future task admission. A folder at or above a limit
produces a warning on each run until enough files have been removed or shrunk.
Existing saved files are restored even when already over quota, so the agent
can continue working and clean them up with ordinary filesystem tools. Unsafe
paths and links still fail validation; bypassing a storage quota does not
bypass those checks.
An API save above a storage limit returns 422 without changing the saved files.
If a stopped run exceeds a storage limit, none of its agent-folder changes are
saved. The run shows a nonblocking storage warning and its save receipt reports
`AGENT_FILES_LIMIT_EXCEEDED` with the specific limit
and, for an oversized file, its path. The previous saved folder is used on the
next run. The temporary run copy is discarded, including on a limit failure;
there is no retained recovery archive or partial-save option. Transient sync
failures get up to three attempts at the stop boundary before cleanup and an
explicit failure receipt. Individual file writes are atomic, but an I/O failure
partway through a sync can leave some files updated; a failed receipt does not
claim whole-folder success.
Sync failures are diagnostics for the affected run, not errors on the current
files in the Instructions Editor. Historical failures remain in the run log;
the run detail also shows warnings from its save receipt. The editor only shows
preserved instruction-only candidates that may need review, alongside errors
from the current browser edit. Later successful saves do not erase run history.
Larger folders take longer to hash, copy, and transfer on each run. There is one
canonical folder plus temporary working copies for currently active runs (and
remote staging when the transport needs it). No additional captured tree is
created. Terminal runs remove their private trees and baseline metadata, keeping
only a small receipt. Restart recovery retries interrupted cleanup without
removing a running provider's files. These are not aggregate disk quotas; the
operator still provisions storage for agents and the configured run concurrency.
## Run lifecycle
1. Under the agent lock, restore current files into a private run copy and save
a baseline of paths, kinds, modes, and hashes. This is sync metadata, not a
revision history.
2. Stage the copy through the existing workspace transport. Point `AGENT_HOME`
and instruction guidance at that registered root.
3. At the provider's verified checkpoint-and-stop boundary, retrieve the entire
directory into the existing working copy before releasing its environment.
4. Recheck the responsible user's current authorization. Under the same agent
lock used by editor writes, apply only files changed or deleted relative to
the starting baseline. For a competing edit or deletion of the same file,
the last synchronization to acquire the lock wins. Unchanged files do not
overwrite another run's changes; newly added unrelated files survive.
5. Record the outcome and remove temporary copies for successful and failed
runs. No per-run file versions, conflict copies, or review queue accumulate.
The next run starts with the current directory.
The whole-directory contract closes the provider process to establish a safe
collection boundary, including child processes. It preserves the provider's
resumable conversation. Only the loaded instruction entry participates in the
new runtime instruction digest; adding or editing another file does not change
that digest. Relative supporting files are read from `AGENT_HOME`, not from the
read-only prompt snapshot.
The editor supplies the hash of the file it read. A stale browser save returns
409 and retains the user's unsaved draft. Run synchronization itself uses
per-file last-sync-wins: a later run can overwrite a saved browser edit to the
same file. There is no text merge or historical copy to recover the overwritten
version. Ordinary task files continue using their existing workspace contract.
## Upgrade and recovery
Migration 0287 creates the preview tables idempotently after master’s 0285/0286.
Existing preview receipts, rows, constraints, and pending captures are retained.
On first use, while holding
the agent row lock, import any deployed revision heads into the existing managed
directory once. A controller-owned marker outside agent files prevents any
later replay of those heads. Existing revision rows remain readable for recovery;
new saves never append to them. Old UUID-based clients receive content tokens
and can still submit their previously recorded revision IDs, which are checked
against the corresponding bytes before a write.
Working-copy receipts and native runtime inputs record the new file contract.
A restored native session with no contract field keeps the old instruction-only
copy shape, prompt digest, paths, and collector. Its writes use the compatibility
bridge into current files, with the original baseline fence. Existing pending
legacy candidates remain resolvable. Neither old task workspaces nor arbitrary
external instruction roots are imported as agent directories.
Stock-agent and plugin resets update their declared files while retaining unrelated
personal files and formerly configured entries. Automatic stock upgrades first
record baseline hashes in the existing resource binding, then apply and finalize
under the agent lock. A failed file write or database commit retries against
those hashes and already-applied bytes. Removed, unchanged stock files are
removed; intervening personal edits stop the retry. This pending operation
metadata is cleared on success and does not retain file revisions.
External bundles retain their existing behavior. Their migration to managed
storage is an explicit configuration action. Historical task cwd, provider-home,
checkpoint, and workspace restoration formats are not rewritten.
Backups must include the persistent instance filesystem as well as the database.
New current-file bytes are not database revision rows. Old instruction-only
candidates are retained solely for upgrade compatibility.
Crash recovery can collect a stopped working copy without starting a model.
Missing stop proof or lost remote bytes produce a visible diagnostic, never a
save receipt. An interrupted apply can replay its changed files with the same
last-sync-wins rule. Cleanup resumes for terminal runs; no copy is retained as
an archive after cleanup succeeds.
## Verification
`agent-directory-working-copies.test.ts` exercises nested/binary files, directory
isolation, last-sync-wins edits and deletions, terminal cleanup, link rejection, old-head
adoption, and stable prompt digests. The legacy working-copy and native-tool
suites exercise compatibility. Workspace merge tests exercise preflight and
interrupted replay.
The explicit Product E2E `instruction-persistence` suite creates a file through
the browser editor, runs an agent that changes instructions and supporting files,
checks exact binary bytes via the public download route, restarts the server,
and asks a fresh task to prove restored contents using an independent nonce. A
third task edits its entry while the browser saves that same file; the later
run sync wins while a separate browser-created file survives, with no conflict
candidate or manual resolution. Three more tasks save a sparse file at its
256 MiB boundary, exceed that boundary with a nonfatal save rejection, then
remove it and save a new small file. All tasks must succeed, with warnings
visible in run details while full and cleared after cleanup.
Run results, including unavailable credentials, must be reported separately from
unit or matcher results; a passing matcher does not prove a live run.
+15
View File
@@ -0,0 +1,15 @@
# Legacy instruction revision compatibility
The revision-based design in the first version of PR #14325 has been superseded
by [persistent agent files](agent-files.md). Current managed files are authoritative;
new edits do not create revision or head rows.
Published migrations 0285 and 0286 and existing revision records are retained so
preview installations and saved sessions can upgrade safely. Old heads are
adopted once. Legacy collectors and UUID-based APIs compare their baseline bytes
against current files before writing. Read-only history and restore endpoints
remain for pre-upgrade records; they are not advertised as new agent capabilities
or shown as a history feature in the current file editor.
See the current document for directory layout, synchronization, conflicts,
authorization, recovery, and verification.
@@ -1,7 +1,7 @@
# Rich ACP integration and qualification report
Updated: 2026-09-29. Base: `c65fc9e3c81c41aafe421aa90a00514b84343285`.
Mainline integration: `d172197117a14b80a1eb2d2835a0e7cce2679656`.
Mainline integration: `3ca196b0a642aa21b8feba1fcd89edb53d1c622e`.
Status: implementation is split into a shared foundation and three provider PRs.
Provider review and qualification remain separate from foundation acceptance.
All three new profiles remain **pending qualification**. Cursor passed all five
@@ -196,6 +196,7 @@ qualification gates, not claims that a JavaScript path check confines a shell.
| P1 | Complete usage/billing provenance | Missing cache fields remain unknown. Pi price estimates are displayed separately. Budget qualification requires actual spend coverage, not an estimate presented as a bill. |
| P1 | Fork/history/model/mode controls not exposed by Paperclip | Research documents the native and ACP methods separately. Add governance-aware controls and durable lineage before enabling them. |
| P1 | Exact pending-request restoration after process death | Session transcript restoration does not restore callbacks. Expire unresolved requests unless a provider proves exact restoration. |
| P1 | Persistent agent-directory access through ACPX | Mainline `3ca196b0a` supplies `AGENT_HOME`, but its existing ACPX environment allowlist does not forward it. Pi also confines native writes to the task workspace, while the local persistent agent directory lives outside it. The disabled candidates do not claim this new capability. Bind and validate the company/agent/run-owned directory explicitly through launch and native-tool policy, including warm-run rebinding and cleanup-before-collection tests; do not widen ambient environment or filesystem access. |
| P2 | Remaining Copilot native diagnostic/config/account events | The provider inventory records every event and field, its projection or reason for omission. Preserve bounded useful context; avoid credentials, raw environment or unbounded blobs. |
| P2 | Pi native extension surfaces and unsupported slash commands | Arbitrary extensions/templates/themes may execute ambient code. Only reviewed runner-owned capabilities are admitted; structured native plan/goals are not fabricated. Native fork/clone/export are separate unmapped capabilities above. |
| P2 | Pi status/widget/title/editor and session/configuration notifications | `setStatus`, `setWidget`, `setTitle` and `set_editor_text` have no UI projection and are unused by the owned extension. Native session-name and thinking-level events also lack a dedicated projection. Add reviewed bounded notice schemas and governed configuration controls before exposing these fields; `notify` and interactive input already have separate bridges. |
+5
View File
@@ -276,6 +276,11 @@ results and follow-up coverage are recorded in that suite's guide.
Continuation accounting has an explicit-only eight-cell Product E2E [baseline suite](../tests/runner-e2e/CONTINUATION-ACCOUNTING.md), complementing the deterministic lifecycle inventory.
The explicit Product E2E `instruction-persistence` suite verifies private file
edits, nested and binary agent files, stopped-provider directory saves, server restart, and a fresh task's
downloaded proof on local native/legacy Codex and native Daytona. See the
[Product E2E runbook](../tests/runner-e2e/README.md).
The explicit-only Product E2E `api-response-reading` suite verifies retrieval of
large saved API responses on local and Daytona native Codex runs. See the
[Runner E2E guide](../tests/runner-e2e/README.md#bounded-api-response-reading).
@@ -2058,7 +2058,11 @@ async function buildRuntime(input: {
paperclipClaudeSettings = await writePaperclipClaudeSettings({
cwd,
stateDir,
agentHome,
// A run's writable AGENT_HOME moves, but the existing permission root
// must not invalidate its resumable conversation. Run copies are covered
// by the company root already granted in these settings; the process env
// still receives this run's actual AGENT_HOME.
agentHome: asString(workspaceContext.agentHomeForPermissions, agentHome),
companyId: agent.companyId,
});
skillCommandNotes.push(
@@ -0,0 +1,141 @@
import { createHash, randomUUID } from "node:crypto";
import { readFile } from "node:fs/promises";
import postgres from "postgres";
import { describe, expect, it } from "vitest";
import { applyPendingMigrations, inspectMigrations } from "./client.js";
import { EMBEDDED_POSTGRES_TEST_TIMEOUT_MS, getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./test-embedded-postgres.js";
// Exact SQL from PR #14325 before master assigned 0285/0286 to API response storage.
// Exercise real preview tables/receipts, not a second application of the new schema.
const previewMigrations = [
{ when: 1790474441205, sql: `CREATE TABLE "agent_instruction_heads" (
"company_id" uuid NOT NULL,
"agent_id" uuid NOT NULL,
"entry_file" text NOT NULL,
"revision_id" uuid NOT NULL,
"updated_at" timestamp with time zone DEFAULT now() NOT NULL,
CONSTRAINT "agent_instruction_heads_identity_uq" UNIQUE("company_id","agent_id","entry_file")
);
--> statement-breakpoint
CREATE TABLE "agent_instruction_revisions" (
"id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL,
"company_id" uuid NOT NULL,
"agent_id" uuid NOT NULL,
"entry_file" text NOT NULL,
"content_base64" text NOT NULL,
"content_hash" text NOT NULL,
"byte_length" integer NOT NULL,
"parent_revision_id" uuid,
"base_revision_id" uuid,
"restored_from_revision_id" uuid,
"actor_agent_id" uuid,
"actor_user_id" text,
"responsible_user_id" text,
"source_run_id" uuid,
"source" text NOT NULL,
"created_at" timestamp with time zone DEFAULT now() NOT NULL,
CONSTRAINT "agent_instruction_revisions_identity_uq" UNIQUE("company_id","agent_id","entry_file","id")
);
--> statement-breakpoint
ALTER TABLE "agent_instruction_heads" ADD CONSTRAINT "agent_instruction_heads_company_id_agent_id_entry_file_revision_id_agent_instruction_revisions_company_id_agent_id_entry_file_id_fk" FOREIGN KEY ("company_id","agent_id","entry_file","revision_id") REFERENCES "public"."agent_instruction_revisions"("company_id","agent_id","entry_file","id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
ALTER TABLE "agent_instruction_revisions" ADD CONSTRAINT "agent_instruction_revisions_company_id_agent_id_agents_company_id_id_fk" FOREIGN KEY ("company_id","agent_id") REFERENCES "public"."agents"("company_id","id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
CREATE INDEX "agent_instruction_revisions_history_idx" ON "agent_instruction_revisions" USING btree ("company_id","agent_id","entry_file","created_at","id");` },
{ when: 1790543348340, sql: `CREATE TABLE "agent_instruction_working_copies" (
"run_id" uuid PRIMARY KEY NOT NULL,
"company_id" uuid NOT NULL,
"agent_id" uuid NOT NULL,
"responsible_user_id" text NOT NULL,
"entry_file" text NOT NULL,
"base_revision_id" uuid,
"base_hash" text NOT NULL,
"local_root" text NOT NULL,
"execution_root" text NOT NULL,
"location" text NOT NULL,
"state" text DEFAULT 'prepared' NOT NULL,
"candidate_base64" text,
"candidate_hash" text,
"error_code" text,
"error_message" text,
"receipt" jsonb,
"process_stopped_at" timestamp with time zone,
"attempts" integer DEFAULT 0 NOT NULL,
"next_attempt_at" timestamp with time zone,
"created_at" timestamp with time zone DEFAULT now() NOT NULL,
"updated_at" timestamp with time zone DEFAULT now() NOT NULL
);
--> statement-breakpoint
ALTER TABLE "agent_instruction_working_copies" ADD CONSTRAINT "agent_instruction_working_copies_run_id_heartbeat_runs_id_fk" FOREIGN KEY ("run_id") REFERENCES "public"."heartbeat_runs"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
ALTER TABLE "agent_instruction_working_copies" ADD CONSTRAINT "agent_instruction_working_copies_company_id_agent_id_agents_company_id_id_fk" FOREIGN KEY ("company_id","agent_id") REFERENCES "public"."agents"("company_id","id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
CREATE INDEX "agent_instruction_copies_pending_idx" ON "agent_instruction_working_copies" USING btree ("state","next_attempt_at");--> statement-breakpoint
CREATE INDEX "agent_instruction_copies_agent_idx" ON "agent_instruction_working_copies" USING btree ("company_id","agent_id","created_at");` },
];
const migration = await readFile(new URL("./migrations/0287_serious_tinkerer.sql", import.meta.url), "utf8");
const migrationHash = createHash("sha256").update(migration).digest("hex");
const support = await getEmbeddedPostgresTestSupport();
const describePostgres = support.supported ? describe : describe.skip;
describePostgres("instruction revision migrations", () => {
it("keeps applied migration receipts, revisions, and pending copies intact on repeated upgrades", async () => {
const database = await startEmbeddedPostgresTestDatabase("instruction-migration-upgrade-");
const sql = postgres(database.connectionString, { max: 1, onnotice: () => {} });
try {
await sql`DROP TABLE agent_instruction_heads, agent_instruction_working_copies, agent_instruction_revisions`;
await sql`DELETE FROM drizzle.__drizzle_migrations WHERE hash = ${migrationHash}`;
for (const preview of previewMigrations) {
for (const statement of preview.sql.split("--> statement-breakpoint")) await sql.unsafe(statement);
await sql`INSERT INTO drizzle.__drizzle_migrations (hash, created_at)
VALUES (${createHash("sha256").update(preview.sql).digest("hex")}, ${preview.when})`;
}
const companyId = randomUUID(), otherCompanyId = randomUUID(), agentId = randomUUID();
const runId = randomUUID(), revisionId = randomUUID();
const bytes = Buffer.from("\uFEFF# Saved\r\nExact bytes ☃\n");
const contentBase64 = bytes.toString("base64");
const contentHash = createHash("sha256").update(bytes).digest("hex");
await sql`INSERT INTO companies (id, name, issue_prefix) VALUES
(${companyId}, 'Instructions', 'INS'), (${otherCompanyId}, 'Other', 'OTH')`;
await sql`INSERT INTO agents (id, company_id, name) VALUES (${agentId}, ${companyId}, 'Writer')`;
await sql`INSERT INTO heartbeat_runs (id, company_id, agent_id) VALUES (${runId}, ${companyId}, ${agentId})`;
await sql`INSERT INTO agent_instruction_revisions
(id, company_id, agent_id, entry_file, content_base64, content_hash, byte_length, source, source_run_id)
VALUES (${revisionId}, ${companyId}, ${agentId}, 'AGENTS.md', ${contentBase64}, ${contentHash}, ${bytes.length}, 'cleanup', ${runId})`;
await sql`INSERT INTO agent_instruction_heads (company_id, agent_id, entry_file, revision_id)
VALUES (${companyId}, ${agentId}, 'AGENTS.md', ${revisionId})`;
await sql`INSERT INTO agent_instruction_working_copies
(run_id, company_id, agent_id, responsible_user_id, entry_file, base_revision_id, base_hash, local_root, execution_root, location, state, candidate_base64, candidate_hash)
VALUES (${runId}, ${companyId}, ${agentId}, 'editor', 'AGENTS.md', ${revisionId}, ${contentHash}, '/private/copy', '/private/copy', 'local', 'pending_commit', ${contentBase64}, ${contentHash})`;
const snapshot = async () => ({
journal: await sql`SELECT * FROM drizzle.__drizzle_migrations ORDER BY id`,
relations: await sql`SELECT oid::text, relname FROM pg_class WHERE relname IN
('agent_instruction_revisions', 'agent_instruction_heads', 'agent_instruction_working_copies') ORDER BY relname`,
constraints: await sql`SELECT oid::text, conname, pg_get_constraintdef(oid) AS definition FROM pg_constraint
WHERE conrelid IN ('agent_instruction_revisions'::regclass, 'agent_instruction_heads'::regclass,
'agent_instruction_working_copies'::regclass) ORDER BY conname`,
revisions: await sql`SELECT * FROM agent_instruction_revisions`,
heads: await sql`SELECT * FROM agent_instruction_heads`,
copies: await sql`SELECT * FROM agent_instruction_working_copies`,
});
const before = await snapshot();
expect(before.relations).toHaveLength(3);
expect(await inspectMigrations(database.connectionString)).toMatchObject({
status: "needsMigrations", pendingMigrations: ["0287_serious_tinkerer.sql"],
});
await applyPendingMigrations(database.connectionString);
const upgraded = await snapshot();
expect(upgraded).toEqual({ ...before, journal: expect.arrayContaining(before.journal) });
expect(upgraded.journal).toHaveLength(before.journal.length + 1);
for (let attempt = 0; attempt < 2; attempt += 1) {
expect(await inspectMigrations(database.connectionString)).toMatchObject({ status: "upToDate" });
await applyPendingMigrations(database.connectionString);
expect(await snapshot()).toEqual(upgraded);
}
await expect(sql`UPDATE agent_instruction_heads SET company_id = ${otherCompanyId} WHERE revision_id = ${revisionId}`)
.rejects.toMatchObject({ code: "23503" });
await expect(sql`UPDATE agent_instruction_working_copies SET company_id = ${otherCompanyId} WHERE run_id = ${runId}`)
.rejects.toMatchObject({ code: "23503" });
} finally {
await sql.end();
await database.cleanup();
}
}, EMBEDDED_POSTGRES_TEST_TIMEOUT_MS);
});
@@ -0,0 +1,61 @@
-- Retain preview instruction rows and pending captures when upgrading PR #14325.
CREATE TABLE IF NOT EXISTS "agent_instruction_heads" (
"company_id" uuid NOT NULL,
"agent_id" uuid NOT NULL,
"entry_file" text NOT NULL,
"revision_id" uuid NOT NULL,
"updated_at" timestamp with time zone DEFAULT now() NOT NULL,
CONSTRAINT "agent_instruction_heads_identity_uq" UNIQUE("company_id","agent_id","entry_file")
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "agent_instruction_revisions" (
"id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL,
"company_id" uuid NOT NULL,
"agent_id" uuid NOT NULL,
"entry_file" text NOT NULL,
"content_base64" text NOT NULL,
"content_hash" text NOT NULL,
"byte_length" integer NOT NULL,
"parent_revision_id" uuid,
"base_revision_id" uuid,
"restored_from_revision_id" uuid,
"actor_agent_id" uuid,
"actor_user_id" text,
"responsible_user_id" text,
"source_run_id" uuid,
"source" text NOT NULL,
"created_at" timestamp with time zone DEFAULT now() NOT NULL,
CONSTRAINT "agent_instruction_revisions_identity_uq" UNIQUE("company_id","agent_id","entry_file","id")
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "agent_instruction_working_copies" (
"run_id" uuid PRIMARY KEY NOT NULL,
"company_id" uuid NOT NULL,
"agent_id" uuid NOT NULL,
"responsible_user_id" text NOT NULL,
"entry_file" text NOT NULL,
"base_revision_id" uuid,
"base_hash" text NOT NULL,
"local_root" text NOT NULL,
"execution_root" text NOT NULL,
"location" text NOT NULL,
"state" text DEFAULT 'prepared' NOT NULL,
"candidate_base64" text,
"candidate_hash" text,
"error_code" text,
"error_message" text,
"receipt" jsonb,
"process_stopped_at" timestamp with time zone,
"attempts" integer DEFAULT 0 NOT NULL,
"next_attempt_at" timestamp with time zone,
"created_at" timestamp with time zone DEFAULT now() NOT NULL,
"updated_at" timestamp with time zone DEFAULT now() NOT NULL
);
--> statement-breakpoint
DO $$ BEGIN ALTER TABLE "agent_instruction_heads" ADD CONSTRAINT "agent_instruction_heads_company_id_agent_id_entry_file_revision_id_agent_instruction_revisions_company_id_agent_id_entry_file_id_fk" FOREIGN KEY ("company_id","agent_id","entry_file","revision_id") REFERENCES "public"."agent_instruction_revisions"("company_id","agent_id","entry_file","id") ON DELETE cascade ON UPDATE no action; EXCEPTION WHEN duplicate_object THEN NULL; END $$;--> statement-breakpoint
DO $$ BEGIN ALTER TABLE "agent_instruction_revisions" ADD CONSTRAINT "agent_instruction_revisions_company_id_agent_id_agents_company_id_id_fk" FOREIGN KEY ("company_id","agent_id") REFERENCES "public"."agents"("company_id","id") ON DELETE cascade ON UPDATE no action; EXCEPTION WHEN duplicate_object THEN NULL; END $$;--> statement-breakpoint
DO $$ BEGIN ALTER TABLE "agent_instruction_working_copies" ADD CONSTRAINT "agent_instruction_working_copies_run_id_heartbeat_runs_id_fk" FOREIGN KEY ("run_id") REFERENCES "public"."heartbeat_runs"("id") ON DELETE cascade ON UPDATE no action; EXCEPTION WHEN duplicate_object THEN NULL; END $$;--> statement-breakpoint
DO $$ BEGIN ALTER TABLE "agent_instruction_working_copies" ADD CONSTRAINT "agent_instruction_working_copies_company_id_agent_id_agents_company_id_id_fk" FOREIGN KEY ("company_id","agent_id") REFERENCES "public"."agents"("company_id","id") ON DELETE cascade ON UPDATE no action; EXCEPTION WHEN duplicate_object THEN NULL; END $$;--> statement-breakpoint
CREATE INDEX IF NOT EXISTS "agent_instruction_revisions_history_idx" ON "agent_instruction_revisions" USING btree ("company_id","agent_id","entry_file","created_at","id");--> statement-breakpoint
CREATE INDEX IF NOT EXISTS "agent_instruction_copies_pending_idx" ON "agent_instruction_working_copies" USING btree ("state","next_attempt_at");--> statement-breakpoint
CREATE INDEX IF NOT EXISTS "agent_instruction_copies_agent_idx" ON "agent_instruction_working_copies" USING btree ("company_id","agent_id","created_at");
+7
View File
@@ -1996,6 +1996,13 @@
"when": 1790607525758,
"tag": "0286_complete_alice",
"breakpoints": true
},
{
"idx": 287,
"version": "7",
"when": 1790611116286,
"tag": "0287_serious_tinkerer",
"breakpoints": true
}
]
}
@@ -0,0 +1,119 @@
import {
pgTable,
uuid,
text,
integer,
timestamp,
index,
unique,
foreignKey,
jsonb,
} from "drizzle-orm/pg-core";
import { agents } from "./agents.js";
import { heartbeatRuns } from "./heartbeat_runs.js";
/** Append-only content. Base64 preserves every UTF-8 byte, including NUL and BOM. */
export const agentInstructionRevisions = pgTable(
"agent_instruction_revisions",
{
id: uuid("id").primaryKey().defaultRandom(),
companyId: uuid("company_id").notNull(),
agentId: uuid("agent_id").notNull(),
entryFile: text("entry_file").notNull(),
contentBase64: text("content_base64").notNull(),
contentHash: text("content_hash").notNull(),
byteLength: integer("byte_length").notNull(),
parentRevisionId: uuid("parent_revision_id"),
baseRevisionId: uuid("base_revision_id"),
restoredFromRevisionId: uuid("restored_from_revision_id"),
actorAgentId: uuid("actor_agent_id"),
actorUserId: text("actor_user_id"),
responsibleUserId: text("responsible_user_id"),
sourceRunId: uuid("source_run_id"),
source: text("source").notNull(),
createdAt: timestamp("created_at", { withTimezone: true })
.notNull()
.defaultNow(),
},
(t) => ({
owner: foreignKey({
columns: [t.companyId, t.agentId],
foreignColumns: [agents.companyId, agents.id],
}).onDelete("cascade"),
identity: unique("agent_instruction_revisions_identity_uq").on(
t.companyId,
t.agentId,
t.entryFile,
t.id,
),
history: index("agent_instruction_revisions_history_idx").on(
t.companyId,
t.agentId,
t.entryFile,
t.createdAt,
t.id,
),
}),
);
export const agentInstructionHeads = pgTable(
"agent_instruction_heads",
{
companyId: uuid("company_id").notNull(),
agentId: uuid("agent_id").notNull(),
entryFile: text("entry_file").notNull(),
revisionId: uuid("revision_id").notNull(),
updatedAt: timestamp("updated_at", { withTimezone: true })
.notNull()
.defaultNow(),
},
(t) => ({
identity: unique("agent_instruction_heads_identity_uq").on(
t.companyId,
t.agentId,
t.entryFile,
),
revision: foreignKey({
columns: [t.companyId, t.agentId, t.entryFile, t.revisionId],
foreignColumns: [
agentInstructionRevisions.companyId,
agentInstructionRevisions.agentId,
agentInstructionRevisions.entryFile,
agentInstructionRevisions.id,
],
}).onDelete("cascade"),
}),
);
/** A server-owned collection receipt, retained even when canonical CAS rejects an edit. */
export const agentInstructionWorkingCopies = pgTable(
"agent_instruction_working_copies",
{
runId: uuid("run_id").primaryKey().references(() => heartbeatRuns.id, { onDelete: "cascade" }),
companyId: uuid("company_id").notNull(),
agentId: uuid("agent_id").notNull(),
responsibleUserId: text("responsible_user_id").notNull(),
entryFile: text("entry_file").notNull(),
baseRevisionId: uuid("base_revision_id"),
baseHash: text("base_hash").notNull(),
localRoot: text("local_root").notNull(),
executionRoot: text("execution_root").notNull(),
location: text("location").notNull(),
state: text("state").notNull().default("prepared"),
candidateBase64: text("candidate_base64"),
candidateHash: text("candidate_hash"),
errorCode: text("error_code"),
errorMessage: text("error_message"),
receipt: jsonb("receipt").$type<Record<string, unknown>>(),
processStoppedAt: timestamp("process_stopped_at", { withTimezone: true }),
attempts: integer("attempts").notNull().default(0),
nextAttemptAt: timestamp("next_attempt_at", { withTimezone: true }),
createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
updatedAt: timestamp("updated_at", { withTimezone: true }).notNull().defaultNow(),
},
(t) => ({
owner: foreignKey({ columns: [t.companyId, t.agentId], foreignColumns: [agents.companyId, agents.id] }).onDelete("cascade"),
pending: index("agent_instruction_copies_pending_idx").on(t.state, t.nextAttemptAt),
history: index("agent_instruction_copies_agent_idx").on(t.companyId, t.agentId, t.createdAt),
}),
);
+2
View File
@@ -211,3 +211,5 @@ export { aiConnectionDefaults } from "./ai_connection_defaults.js";
export { aiProviderDefaults } from "./ai_provider_defaults.js";
export * from "./email.js";
export { announcementDismissals, announcementPublications } from "./announcement_dismissals.js";
export { agentInstructionRevisions, agentInstructionHeads, agentInstructionWorkingCopies } from "./agent_instruction_revisions.js";
@@ -1 +1 @@
[{"annotations":{"exposure":"always","operationId":"get_task_context","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read the active task and actor, including the exact approved Markdown revision when this issue has an accepted plan.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"get_task_context","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"get_task_history","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read bounded comments on the active task.","inputSchema":{"additionalProperties":false,"properties":{"limit":{"default":50,"maximum":200,"minimum":1,"type":"integer"}},"required":[],"type":"object"},"name":"get_task_history","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"list_documents","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"List revisioned documents on the active task.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"list_documents","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"read_document","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read the current revision of one active-task document.","inputSchema":{"additionalProperties":false,"properties":{"key":{"description":"Stable issue-document key.","maxLength":120,"minLength":1,"type":"string"}},"required":["key"],"type":"object"},"name":"read_document","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"list_document_revisions","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read bounded revision history for one active-task document.","inputSchema":{"additionalProperties":false,"properties":{"key":{"description":"Stable issue-document key.","maxLength":120,"minLength":1,"type":"string"},"limit":{"default":50,"maximum":200,"minimum":1,"type":"integer"}},"required":["key"],"type":"object"},"name":"list_document_revisions","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"report_progress","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Append a durable progress comment to the active task.","inputSchema":{"additionalProperties":false,"properties":{"body":{"description":"Multiline progress update.","maxLength":20000,"minLength":1,"type":"string"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"}},"required":["idempotencyKey","body"],"type":"object"},"name":"report_progress","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"answer_status_question","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Append the answer to a status-only wake without changing task disposition.","inputSchema":{"additionalProperties":false,"properties":{"body":{"description":"Concise status answer.","maxLength":20000,"minLength":1,"type":"string"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"}},"required":["idempotencyKey","body"],"type":"object"},"name":"answer_status_question","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"write_document","requiredClaimLine truncated
[{"annotations":{"exposure":"always","operationId":"get_task_context","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read the active task and actor, including the exact approved Markdown revision when this issue has an accepted plan.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"get_task_context","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"get_task_history","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read bounded comments on the active task.","inputSchema":{"additionalProperties":false,"properties":{"limit":{"default":50,"maximum":200,"minimum":1,"type":"integer"}},"required":[],"type":"object"},"name":"get_task_history","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"list_documents","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"List revisioned documents on the active task.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"list_documents","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"read_document","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read the current revision of one active-task document.","inputSchema":{"additionalProperties":false,"properties":{"key":{"description":"Stable issue-document key.","maxLength":120,"minLength":1,"type":"string"}},"required":["key"],"type":"object"},"name":"read_document","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"list_document_revisions","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read bounded revision history for one active-task document.","inputSchema":{"additionalProperties":false,"properties":{"key":{"description":"Stable issue-document key.","maxLength":120,"minLength":1,"type":"string"},"limit":{"default":50,"maximum":200,"minimum":1,"type":"integer"}},"required":["key"],"type":"object"},"name":"list_document_revisions","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"report_progress","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Append a durable progress comment to the active task.","inputSchema":{"additionalProperties":false,"properties":{"body":{"description":"Multiline progress update.","maxLength":20000,"minLength":1,"type":"string"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"}},"required":["idempotencyKey","body"],"type":"object"},"name":"report_progress","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"answer_status_question","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Append the answer to a status-only wake without changing task disposition.","inputSchema":{"additionalProperties":false,"properties":{"body":{"description":"Concise status answer.","maxLength":20000,"minLength":1,"type":"string"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"}},"required":["idempotencyKey","body"],"type":"object"},"name":"answer_status_question","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"write_document","requiredClaimLine truncated
@@ -24,7 +24,7 @@
"prpVersion": 1,
"nativeExecutionVersion": 1,
"catalogVersion": 1,
"catalogSha256": "sha256:a19534304480dfd3d357cc902e785af05158ce1496f4d324184112732ff94c9c",
"catalogSha256": "sha256:61c6ba99926c14285ca6a89a05fe316848b9e5bc38840d43dd19b0990dc482d0",
"driverContractVersion": 1,
"driverKind": "paperclip-deterministic",
"driverVersion": "1.0.0"
@@ -160,7 +160,7 @@
},
{
"path": "fixtures/evals/native-execution-seeded.json",
"sha256": "c6367c7c7e3dd01fed341b7a83b0008c9d5d1efdfa85bc70e1013bc304a008c7",
"sha256": "c9c56cc11b420302be2d0044aeee30302e43da74c076e13a31addfbcaacd0458",
"expectation": "accept",
"compatibilityCase": "canonical"
},
@@ -7,10 +7,86 @@
"src/scenarios/scenario-plan.ts"
],
"counts": {
"actions": 48,
"actions": 52,
"legacyRequirements": 106
},
"actions": [
{
"id": "read_agent_instructions",
"ownership": "always_agent_tool",
"surfaces": [
"live"
],
"legacyAliases": [],
"contractCase": "protocol-action:read_agent_instructions",
"contractOwner": "src/catalog/protocol-action-contracts.test.ts::read_agent_instructions has a schema-valid canonical example and every declared projection",
"legacyBehavioralCases": [],
"deterministicCases": [
"protocol-action:read_agent_instructions"
],
"legacyRequirementCases": [],
"deterministicOwners": [
"src/catalog/protocol-action-contracts.test.ts::read_agent_instructions has a schema-valid canonical example and every declared projection",
"src/scenarios/scenario-explorer.test.ts::renders every scenario with exposure, control plane, authorization, diff, and parity"
]
},
{
"id": "update_agent_instructions",
"ownership": "always_agent_tool",
"surfaces": [
"live"
],
"legacyAliases": [],
"contractCase": "protocol-action:update_agent_instructions",
"contractOwner": "src/catalog/protocol-action-contracts.test.ts::update_agent_instructions has a schema-valid canonical example and every declared projection",
"legacyBehavioralCases": [],
"deterministicCases": [
"protocol-action:update_agent_instructions"
],
"legacyRequirementCases": [],
"deterministicOwners": [
"src/catalog/protocol-action-contracts.test.ts::update_agent_instructions has a schema-valid canonical example and every declared projection",
"src/scenarios/scenario-explorer.test.ts::renders every scenario with exposure, control plane, authorization, diff, and parity"
]
},
{
"id": "get_agent_instruction_history",
"ownership": "always_agent_tool",
"surfaces": [
"live"
],
"legacyAliases": [],
"contractCase": "protocol-action:get_agent_instruction_history",
"contractOwner": "src/catalog/protocol-action-contracts.test.ts::get_agent_instruction_history has a schema-valid canonical example and every declared projection",
"legacyBehavioralCases": [],
"deterministicCases": [
"protocol-action:get_agent_instruction_history"
],
"legacyRequirementCases": [],
"deterministicOwners": [
"src/catalog/protocol-action-contracts.test.ts::get_agent_instruction_history has a schema-valid canonical example and every declared projection",
"src/scenarios/scenario-explorer.test.ts::renders every scenario with exposure, control plane, authorization, diff, and parity"
]
},
{
"id": "restore_agent_instructions",
"ownership": "always_agent_tool",
"surfaces": [
"live"
],
"legacyAliases": [],
"contractCase": "protocol-action:restore_agent_instructions",
"contractOwner": "src/catalog/protocol-action-contracts.test.ts::restore_agent_instructions has a schema-valid canonical example and every declared projection",
"legacyBehavioralCases": [],
"deterministicCases": [
"protocol-action:restore_agent_instructions"
],
"legacyRequirementCases": [],
"deterministicOwners": [
"src/catalog/protocol-action-contracts.test.ts::restore_agent_instructions has a schema-valid canonical example and every declared projection",
"src/scenarios/scenario-explorer.test.ts::renders every scenario with exposure, control plane, authorization, diff, and parity"
]
},
{
"id": "create_skill",
"ownership": "optional_agent_tool",
@@ -263,13 +263,13 @@
"id": "rf",
"legacyGroup": 13,
"name": "Reference files",
"owner": "optional domain tools + test-only escape hatch",
"operationIds": ["list_cases", "upsert_case", "list_routines", "manage_routine", "create_skill", "list_company_skills", "sync_company_skills", "list_secret_metadata", "read_secret_value", "export_company", "administer_company", "generic_api_request", "search_api", "call_api", "create_project"],
"owner": "always instruction tools + optional domain tools + test-only escape hatch",
"operationIds": ["list_cases", "upsert_case", "list_routines", "manage_routine", "create_skill", "list_company_skills", "sync_company_skills", "list_secret_metadata", "read_secret_value", "export_company", "administer_company", "generic_api_request", "search_api", "call_api", "create_project", "read_agent_instructions", "update_agent_instructions", "get_agent_instruction_history", "restore_agent_instructions"],
"controlPlaneOperationIds": ["append_audit_record"],
"realSurface": "project, case, routine, company-skill, secret, portability, and administration services",
"realSurface": "agent instruction revisions, project, case, routine, company-skill, secret, portability, and administration services",
"mockStateDomains": ["company", "cases", "routines", "skills", "secrets", "audit", "fault"],
"prpEvidence": "bounded domain projections, redacted broker receipts, company diffs, and audit references",
"gap": "Project and skill creation and API tools require the live server authority; generic_api_request is test-only and the remaining domain operations are scenario-only. Broad administer_company is deferred and cannot claim product coverage. Production API and paired dedicated-tool regressions are recorded separately in paperclip-evals/evals/runner-api-tools; the legacy scenario count is not evidence of that coverage."
"gap": "Instruction read/update/history/restore require the live canonical revision service, current responsible-user permissions, and pinned base revisions for writes. Their service, authority, and product E2E tests are separate from the legacy mock scenarios. Project and skill creation and API tools require the live server authority; generic_api_request is test-only and the remaining domain operations are scenario-only. Broad administer_company is deferred and cannot claim product coverage. Production API and paired dedicated-tool regressions are recorded separately in paperclip-evals/evals/runner-api-tools; the legacy scenario count is not evidence of that coverage."
},
{
"id": "mh",
@@ -6,7 +6,7 @@ Status: canonical explanatory contract for the Paperclip runner V1 surface.
This document keeps three independent meanings of **group** separate. PRP families describe wire evidence and controller commands; capability placement decides who owns an operation; behavioral eval groups organize the 106 scenario corpus. None of the three axes can be used as a substitute for another.
The generated totals are **105 PRP events in 31 event families**, **18 controller commands in 7 command families**, **10 control-plane operations**, **48 reconciled semantic operations** (14 always, 34 optional), and **106 scenarios in 16 behavior groups**.
The generated totals are **105 PRP events in 31 event families**, **18 controller commands in 7 command families**, **10 control-plane operations**, **52 reconciled semantic operations** (18 always, 34 optional), and **106 scenarios in 16 behavior groups**.
## Axis 1: PRP v1 event and command families
@@ -83,9 +83,9 @@ Placement has exactly three outcomes:
| `schedule_blocker_wake` | Dependency-resolution wake scheduling. |
| `reconcile_run` | Work assessment, terminal status arbitration, and recovery reconciliation. |
### Always-agent operations (14)
### Always-agent operations (18)
`answer_status_question`, `block_task`, `finish_task`, `get_task_context`, `get_task_history`, `inspect_operation_result`, `list_document_revisions`, `list_documents`, `read_document`, `register_deliverable`, `report_progress`, `request_human_input`, `request_review`, `write_document`.
`answer_status_question`, `block_task`, `finish_task`, `get_agent_instruction_history`, `get_task_context`, `get_task_history`, `inspect_operation_result`, `list_document_revisions`, `list_documents`, `read_agent_instructions`, `read_document`, `register_deliverable`, `report_progress`, `request_human_input`, `request_review`, `restore_agent_instructions`, `update_agent_instructions`, `write_document`.
### Optional operations (34) and grant groups (13)
@@ -127,6 +127,7 @@ Grant groups are documentation/exposure bundles, not additional authority. The o
| `finish_task` | `always_agent_tool` | none | `standard`<br>`skill_test` | `task_write` | `required` | no | `semantic_command:finish_task` | `scenario` + `live`<br>`live_codex` | `unbound`<br>semantic-operation item event plus active-task state diff, work-assessment, and issue-status-decision events<br>catalog PRP status: `audit_pending` |
| `generic_api_request` | `optional_agent_tool` | `test:generic_api_request` | `skill_test` | `test_escape_hatch` | `required` | yes | `mock_extension:test.generic_api` | `scenario` + `live`<br>`test_only` | `unbound`<br>test-only; excluded from product PRP evidence<br>catalog PRP status: `audit_pending` |
| `get_agent` | `optional_agent_tool` | `discovery:agents:read` | `standard`<br>`ask`<br>`planning`<br>`skill_test` | `read` | `none` | no | inline/no mapping | `live`<br>`live_codex` | `unbound`<br>read projection surfaced via a tool-result item event; no control-plane state diff<br>catalog PRP status: `audit_pending` |
| `get_agent_instruction_history` | `always_agent_tool` | none | `standard`<br>`ask`<br>`planning`<br>`skill_test` | `read` | `none` | no | inline/no mapping | `live`<br>`live_codex` | `agentInstructionRevisionService`<br>tool-result item event plus canonical revision receipt for writes; no scenario mock coverage<br>catalog PRP status: `bound` |
| `get_approval` | `optional_agent_tool` | `governance:approvals:read` | `standard`<br>`ask`<br>`planning`<br>`skill_test` | `read` | `none` | no | inline/no mapping | `live`<br>`live_codex` | `unbound`<br>read projection surfaced via a tool-result item event; no control-plane state diff<br>catalog PRP status: `audit_pending` |
| `get_approval_context` | `optional_agent_tool` | `governance:approvals:read` | `standard`<br>`ask`<br>`planning`<br>`skill_test` | `read` | `none` | no | inline/no mapping | `live`<br>`live_codex` | `unbound`<br>read projection surfaced via a tool-result item event; no control-plane state diff<br>catalog PRP status: `audit_pending` |
| `get_task_context` | `always_agent_tool` | none | `standard`<br>`ask`<br>`planning`<br>`skill_test` | `read` | `none` | no | `context_read:active_task` | `scenario` + `live`<br>`live_codex` | `PaperclipRunnerToolAuthority active issue/run + accepted plan revision`<br>bound company/assignment query plus exact accepted document revision projection<br>catalog PRP status: `bound` |
@@ -146,6 +147,7 @@ Grant groups are documentation/exposure bundles, not additional authority. The o
| `list_routines` | `optional_agent_tool` | `routines:read` | `standard`<br>`skill_test` | `read` | `none` | no | `mock_extension:routines.list` | `scenario`<br>`scenario_mock` | `unbound`<br>read projection surfaced via a tool-result item event; no control-plane state diff<br>catalog PRP status: `audit_pending` |
| `list_secret_metadata` | `optional_agent_tool` | `secrets:metadata:read` | `standard`<br>`skill_test` | `read` | `none` | no | `mock_extension:secrets.metadata` | `scenario`<br>`scenario_mock` | `unbound`<br>read projection surfaced via a tool-result item event; no control-plane state diff<br>catalog PRP status: `audit_pending` |
| `manage_routine` | `optional_agent_tool` | `routines:write` | `standard`<br>`skill_test` | `admin` | `required` | no | `mock_extension:routines.manage` | `scenario`<br>`scenario_mock` | `unbound`<br>company admin/portability item event plus audit record<br>catalog PRP status: `audit_pending` |
| `read_agent_instructions` | `always_agent_tool` | none | `standard`<br>`ask`<br>`planning`<br>`skill_test` | `read` | `none` | no | inline/no mapping | `live`<br>`live_codex` | `agentInstructionRevisionService`<br>tool-result item event plus canonical revision receipt for writes; no scenario mock coverage<br>catalog PRP status: `bound` |
| `read_document` | `always_agent_tool` | none | `standard`<br>`ask`<br>`planning`<br>`skill_test` | `read` | `none` | no | `snapshot_read:active_task_document` | `scenario` + `live`<br>`live_codex` | `unbound`<br>read projection surfaced via a tool-result item event; no control-plane state diff<br>catalog PRP status: `audit_pending` |
| `read_secret_value` | `optional_agent_tool` | `secrets:values:read` | `standard`<br>`skill_test` | `secret_read` | `none` | yes | `mock_extension:secrets.value` | `scenario`<br>`scenario_mock` | `unbound`<br>redacted tool-result item event; secret value never reaches the wire<br>catalog PRP status: `audit_pending` |
| `reassign_task` | `optional_agent_tool` | `delegation:tasks:assign` | `standard` | `company_write` | `required` | no | `semantic_command:reassign_task` | `scenario` + `live`<br>`live_codex` | `PaperclipRunnerToolAuthority.reassign_task`<br>semantic-operation item event plus company-entity state diff and audit record<br>catalog PRP status: `bound` |
@@ -154,11 +156,13 @@ Grant groups are documentation/exposure bundles, not additional authority. The o
| `request_approval` | `optional_agent_tool` | `governance:approvals:request` | `standard`<br>`skill_test` | `governance` | `required` | no | `semantic_command:request_approval` | `scenario` + `live`<br>`live_codex` | `unbound`<br>approval lifecycle plus governed-wait continuation and audit events<br>catalog PRP status: `audit_pending` |
| `request_human_input` | `always_agent_tool` | none | `standard`<br>`planning`<br>`ask`<br>`skill_test` | `task_write` | `required` | no | `semantic_command:request_human_input` | `scenario` + `live`<br>`live_codex` | `unbound`<br>semantic-operation item event plus active-task state diff, work-assessment, and issue-status-decision events<br>catalog PRP status: `audit_pending` |
| `request_review` | `always_agent_tool` | none | `standard`<br>`skill_test` | `task_write` | `required` | no | `semantic_command:request_review` | `scenario` + `live`<br>`live_codex` | `unbound`<br>semantic-operation item event plus active-task state diff, work-assessment, and issue-status-decision events<br>catalog PRP status: `audit_pending` |
| `restore_agent_instructions` | `always_agent_tool` | none | `standard`<br>`planning` | `company_write` | `recommended` | no | inline/no mapping | `live`<br>`live_codex` | `agentInstructionRevisionService`<br>tool-result item event plus canonical revision receipt for writes; no scenario mock coverage<br>catalog PRP status: `bound` |
| `schedule_wake` | `optional_agent_tool` | `control_plane:wakes` | `standard`<br>`skill_test` | `task_write` | `required` | no | inline/no mapping | `live`<br>`live_codex` | `unbound`<br>semantic-operation item event plus active-task state diff, work-assessment, and issue-status-decision events<br>catalog PRP status: `audit_pending` |
| `search_api` | `optional_agent_tool` | `api:discover` | `standard`<br>`ask`<br>`planning`<br>`skill_test` | `read` | `none` | no | inline/no mapping | `live`<br>`live_codex` | `PaperclipRunnerToolAuthority`<br>Authenticated PRP tool input/result and existing HTTP route authorization/activity records.<br>catalog PRP status: `bound` |
| `search_tasks` | `optional_agent_tool` | `discovery:tasks:read` | `standard`<br>`ask`<br>`planning`<br>`skill_test` | `read` | `none` | no | `snapshot_read:company_tasks` | `scenario` + `live`<br>`live_codex` | `unbound`<br>read projection surfaced via a tool-result item event; no control-plane state diff<br>catalog PRP status: `audit_pending` |
| `set_dependencies` | `optional_agent_tool` | `dependencies:write` | `standard`<br>`skill_test` | `company_write` | `required` | no | `semantic_command:set_dependencies` | `scenario` + `live`<br>`live_codex` | `issues.update.blockedByIssueIds`<br>semantic-operation item event plus company-entity state diff and audit record<br>catalog PRP status: `bound` |
| `sync_company_skills` | `optional_agent_tool` | `company_skills:write` | `standard`<br>`skill_test` | `admin` | `required` | no | `mock_extension:company_skills.sync` | `scenario`<br>`scenario_mock` | `unbound`<br>company admin/portability item event plus audit record<br>catalog PRP status: `audit_pending` |
| `update_agent_instructions` | `always_agent_tool` | none | `standard`<br>`planning` | `company_write` | `recommended` | no | inline/no mapping | `live`<br>`live_codex` | `agentInstructionRevisionService`<br>tool-result item event plus canonical revision receipt for writes; no scenario mock coverage<br>catalog PRP status: `bound` |
| `upsert_case` | `optional_agent_tool` | `cases:write` | `standard`<br>`skill_test` | `company_write` | `required` | no | `mock_extension:cases.upsert` | `scenario`<br>`scenario_mock` | `unbound`<br>semantic-operation item event plus company-entity state diff and audit record<br>catalog PRP status: `audit_pending` |
| `write_document` | `always_agent_tool` | none | `standard`<br>`planning`<br>`skill_test` | `task_write` | `required` | no | `semantic_command:write_document` | `scenario` + `live`<br>`live_codex` | `unbound`<br>semantic-operation item event plus active-task state diff, work-assessment, and issue-status-decision events<br>catalog PRP status: `audit_pending` |
@@ -182,7 +186,7 @@ Behavior groups describe expected outcomes and trajectories. They do not grant t
| [`ap` — Approvals](#behavior-group-ap-approvals) | optional governance tools + governed approver | `list_approvals`<br>`get_approval`<br>`get_approval_context`<br>`request_approval`<br>`decide_approval`<br>`comment_on_approval` | `route_wake`<br>`append_audit_record` | company approval, decision, issue-link, comment, and governed-action services | `company`<br>`task`<br>`approvals`<br>`actor`<br>`wake`<br>`audit`<br>`idempotency` | 6 | governed semantic receipts, audit references, and attention/continuation linkage | Production binding and additive governed-action receipts are unbound; board-only authority stays outside grants. |
| [`ar` — Artifacts](#behavior-group-ar-artifacts) | always tools + artifact/work-product services | `register_deliverable` | `append_audit_record` | attachment upload and issue work-product routes | `task`<br>`artifacts`<br>`workProducts`<br>`workspace`<br>`audit`<br>`idempotency` | 4 | artifact/work-product reference and durable inspectability receipt; never binary bytes | Production upload/register composite and additive durable-reference receipt are unbound. |
| [`er` — Errors and critical rules](#behavior-group-er-errors-and-critical-rules) | runner/control plane + optional workspace/wake tools | `get_workspace_runtime`<br>`control_workspace_service`<br>`schedule_wake`<br>`inspect_operation_result` | `release_task`<br>`enforce_budget`<br>`persist_run`<br>`replay_run`<br>`reconcile_run` | workspace runtime, monitor/recovery, budget, run persistence/replay, release, and terminal services | `workspace`<br>`budget`<br>`run`<br>`wake`<br>`audit`<br>`idempotency`<br>`fault` | 9 | runtime/workspace/attention/run lifecycle, typed denials, replay facts, and terminal causality | Budget stop reasons and semantic denial/conflict receipts require additive v1 envelopes; inspect_operation_result remains scenario-only. |
| [`rf` — Reference files](#behavior-group-rf-reference-files) | optional domain tools + test-only escape hatch | `list_cases`<br>`upsert_case`<br>`list_routines`<br>`manage_routine`<br>`create_skill`<br>`list_company_skills`<br>`sync_company_skills`<br>`list_secret_metadata`<br>`read_secret_value`<br>`export_company`<br>`administer_company`<br>`generic_api_request`<br>`search_api`<br>`call_api`<br>`create_project` | `append_audit_record` | project, case, routine, company-skill, secret, portability, and administration services | `company`<br>`cases`<br>`routines`<br>`skills`<br>`secrets`<br>`audit`<br>`fault` | 22 | bounded domain projections, redacted broker receipts, company diffs, and audit references | Project and skill creation and API tools require the live server authority; generic_api_request is test-only and the remaining domain operations are scenario-only. Broad administer_company is deferred and cannot claim product coverage. Production API and paired dedicated-tool regressions are recorded separately in paperclip-evals/evals/runner-api-tools; the legacy scenario count is not evidence of that coverage. |
| [`rf` — Reference files](#behavior-group-rf-reference-files) | always instruction tools + optional domain tools + test-only escape hatch | `list_cases`<br>`upsert_case`<br>`list_routines`<br>`manage_routine`<br>`create_skill`<br>`list_company_skills`<br>`sync_company_skills`<br>`list_secret_metadata`<br>`read_secret_value`<br>`export_company`<br>`administer_company`<br>`generic_api_request`<br>`search_api`<br>`call_api`<br>`create_project`<br>`read_agent_instructions`<br>`update_agent_instructions`<br>`get_agent_instruction_history`<br>`restore_agent_instructions` | `append_audit_record` | agent instruction revisions, project, case, routine, company-skill, secret, portability, and administration services | `company`<br>`cases`<br>`routines`<br>`skills`<br>`secrets`<br>`audit`<br>`fault` | 22 | bounded domain projections, redacted broker receipts, company diffs, and audit references | Instruction read/update/history/restore require the live canonical revision service, current responsible-user permissions, and pinned base revisions for writes. Their service, authority, and product E2E tests are separate from the legacy mock scenarios. Project and skill creation and API tools require the live server authority; generic_api_request is test-only and the remaining domain operations are scenario-only. Broad administer_company is deferred and cannot claim product coverage. Production API and paired dedicated-tool regressions are recorded separately in paperclip-evals/evals/runner-api-tools; the legacy scenario count is not evidence of that coverage. |
| [`mh` — Multi-hop](#behavior-group-mh-multi-hop) | composed semantic operations + control-plane continuation | `create_task`<br>`set_dependencies`<br>`request_human_input`<br>`request_approval`<br>`register_deliverable` | `route_wake`<br>`reconcile_run` | delegation, dependency, interaction, approval, artifact, and terminal orchestration services | `task`<br>`blockers`<br>`interactions`<br>`approvals`<br>`artifacts`<br>`wake`<br>`run`<br>`audit` | 4 | correlated operation receipts, state diffs, attention hops, work assessment, status decision, and terminal outcome | No generic transaction tool is allowed; shared mock/real conformance must prove each composed effect. |
| [`rs` — Restraint and no-call](#behavior-group-rs-restraint-and-no-call) | policy/exposure layer | `answer_status_question`<br>`read_secret_value`<br>`generic_api_request` | `enforce_budget` | task-mode, secret-broker, test-scope, pause, and budget policy checks | `actor`<br>`task`<br>`budget`<br>`secrets`<br>`audit`<br>`fault` | 3 | absence of forbidden effects plus typed policy denial/redaction receipts when a call is attempted | Typed redaction/authorization receipts need additive v1 evidence; generic_api_request is never a product fallback. |
| [`wk` — Wake situations](#behavior-group-wk-wake-situations) | control plane + always context/history tools | `get_task_context`<br>`get_task_history`<br>`schedule_wake` | `select_work`<br>`route_wake` | wakeup requests, heartbeat context, comment/interaction/approval/blocker wake routing, and scheduled wake services | `wake`<br>`task`<br>`comments`<br>`interactions`<br>`approvals`<br>`blockers`<br>`run` | 8 | attention request routing/resolution plus resumed session/run causality | Production scheduling binding is unbound; control-plane routing remains non-callable. |
@@ -459,10 +463,10 @@ Current responsibility-based paths are normative. Numbered `phase-*` or mileston
### Catalog split and deliberate replacement
- Scenario/eval catalog: **39** operations.
- Live dispatcher catalog: **36** operations.
- Shared: **27**; union/canonical authority: **48**.
- Live dispatcher catalog: **40** operations.
- Shared: **27**; union/canonical authority: **52**.
- Scenario-only: `administer_company`, `export_company`, `inspect_operation_result`, `list_cases`, `list_company_skills`, `list_goals`, `list_routines`, `list_secret_metadata`, `manage_routine`, `read_secret_value`, `sync_company_skills`, `upsert_case`.
- Live-only: `call_api`, `create_project`, `get_agent`, `get_approval`, `get_approval_context`, `hire_agent`, `list_project_repositories`, `schedule_wake`, `search_api`.
- Live-only: `call_api`, `create_project`, `get_agent`, `get_agent_instruction_history`, `get_approval`, `get_approval_context`, `hire_agent`, `list_project_repositories`, `read_agent_instructions`, `restore_agent_instructions`, `schedule_wake`, `search_api`, `update_agent_instructions`.
- The generated provider contract contains exactly the live catalog; the canonical union remains the migration authority until all scenario-only operations are either implemented, deferred, or removed by an explicit reconciliation decision.
- `generic_api_request` stays exported only for controlled tests and cannot be cited as real-surface, mock-parity, or PRP product coverage.
@@ -152,6 +152,7 @@ function createTransportBackedNativeSessionBackend(
? (input.provider.approvalPolicy ?? "never")
: "never",
baseInstructions: nativeSystemInstructions(input),
instructionWorkingCopyRoot: "runtimeContext" in input ? input.runtimeContext.instructions.workingCopy?.rootPath : undefined,
includeSkillInstructions: isCodex && "runtimeContext" in input,
skillInputs: isCodex
? nativeTaskSkillInputs(
@@ -104,6 +104,11 @@ export function nativeTaskConstraints(input: NativeExecutionInput): string[] {
}
return [
"Use only the assigned skills and provider-native tools.",
...(input.runtimeContext.instructions.workingCopy ? [
input.runtimeContext.instructions.workingCopy.kind === "agent_files"
? `For this turn, AGENT_HOME is ${input.runtimeContext.instructions.workingCopy.rootPath}. This replaces any prior turn's agent directory path. It contains your instructions and persistent personal files, separate from the task working directory. Changes save after the provider stops; check the save receipt.`
: `For this turn, the editable agent instruction file is ${input.runtimeContext.instructions.workingCopy.rootPath}/${input.runtimeContext.instructions.workingCopy.entryPath}. This replaces any private working-copy path from a previous turn. Ordinary edits save after the provider stops and only with a durable revision receipt. Use the agent instruction tools for immediate saves. Shared instruction assets and repository instructions are not collected.`,
] : []),
"Use Paperclip semantic tools for coordination and finalization.",
"Save requested plans and Paperclip documents directly with write_document. A saved Paperclip document is already a durable deliverable. Do not create a local file, compute file hashes, or call register_deliverable for it unless the user also requests a downloadable file. Cite the saved document in your completion evidence and final response.",
"When the requested result is a file, use register_deliverable before paperclip_finish. Compute its exact byte size and SHA-256, register the workspace-relative file, cite deliverable:<attachmentId> from the receipt as completion evidence, and include /api/attachments/<attachmentId>/content as the download link in your answer. A bare workspace filename is not a delivered result. For repository edits, cite an accessible PR or registered work product. Preserve existing work; do not upload unrelated files. If file publication fails, fix it or report the concrete blocker instead of claiming the file is delivered.",
@@ -20,21 +20,25 @@ describe("canonical semantic-catalog reconciliation authority", () => {
it("pins the reconciled op-set relationship between the two catalogs", () => {
const summary = capabilityCatalogReconciliation();
expect(summary.scenarioCount).toBe(39);
expect(summary.liveCount).toBe(36);
expect(summary.liveCount).toBe(40);
expect(summary.sharedCount).toBe(27);
expect(summary.unionCount).toBe(48);
expect(summary.unionCount).toBe(52);
// Any operation added to or removed from either catalog without a
// reconciliation decision changes these exact sets and fails the gate.
expect(summary.liveOnly).toEqual([
"call_api",
"create_project",
"get_agent",
"get_agent_instruction_history",
"get_approval",
"get_approval_context",
"hire_agent",
"list_project_repositories",
"read_agent_instructions",
"restore_agent_instructions",
"schedule_wake",
"search_api",
"update_agent_instructions",
]);
expect(summary.scenarioOnly).toEqual([
"administer_company",
@@ -53,7 +57,7 @@ describe("canonical semantic-catalog reconciliation authority", () => {
});
it("is the single source both catalogs derive their operation set from", () => {
expect(CAPABILITY_CANONICAL_OPERATIONS).toHaveLength(48);
expect(CAPABILITY_CANONICAL_OPERATIONS).toHaveLength(52);
const canonicalIds = new Set(CAPABILITY_CANONICAL_OPERATIONS.map((operation) => operation.operationId));
// Neither catalog may contain an operation absent from the canonical source.
for (const tool of SCENARIO_CATALOG) expect(canonicalIds.has(tool.operationId)).toBe(true);
@@ -87,7 +91,7 @@ describe("canonical semantic-catalog reconciliation authority", () => {
});
it("names placement, claims, task modes, side-effect class, idempotency, redaction, mock mapping, real binding status, and PRP evidence for every operation", () => {
expect(CAPABILITY_CANONICAL_CATALOG).toHaveLength(48);
expect(CAPABILITY_CANONICAL_CATALOG).toHaveLength(52);
for (const operation of CAPABILITY_CANONICAL_CATALOG) {
expect(operation.placement).toMatch(/^(always|optional)_agent_tool$/);
expect(Array.isArray(operation.requiredClaims)).toBe(true);
@@ -113,7 +117,7 @@ describe("canonical semantic-catalog reconciliation authority", () => {
it("classifies real binding status so generic_api_request is never product coverage", () => {
const summary = capabilityCatalogReconciliation();
expect(summary.byRealBindingStatus).toEqual({
live_codex: 35,
live_codex: 39,
scenario_mock: 12,
test_only: 1,
});
@@ -15,7 +15,12 @@ export interface NativeRuntimeAssetReference {
export interface NativeRuntimeContextSnapshot {
prompt: { revision: typeof PAPERCLIP_EXECUTION_PROMPT_REVISION; text: typeof PAPERCLIP_EXECUTION_PROMPT; digest: string };
instructions: { entryPath: string; bundle: NativeRuntimeAssetReference };
instructions: {
entryPath: string;
bundle: NativeRuntimeAssetReference;
/** Server-registered writable copy; excluded from the pinned prompt digest. */
workingCopy?: { rootPath: string; entryPath: string; kind?: "agent_files" };
};
skills: Array<{ key: string; runtimeName: string; versionId: string | null; bundle: NativeRuntimeAssetReference }>;
mcp: { assignmentSetId: string; digest: string; bindingId: string | null };
aggregateDigest: string;
@@ -103,7 +108,10 @@ export function parseNativeRuntimeContext(value: unknown): NativeRuntimeContextS
throw new NativeRuntimeContextError("input.runtimeContext.prompt.digest does not match prompt text");
}
const instructions = object(context.instructions, "input.runtimeContext.instructions");
exact(instructions, ["entryPath", "bundle"], "input.runtimeContext.instructions");
exact(instructions, ["entryPath", "bundle", "workingCopy"], "input.runtimeContext.instructions");
const workingCopy = instructions.workingCopy === undefined ? undefined : object(instructions.workingCopy, "input.runtimeContext.instructions.workingCopy");
if (workingCopy) exact(workingCopy, ["rootPath", "entryPath", "kind"], "input.runtimeContext.instructions.workingCopy");
if (workingCopy?.kind !== undefined && workingCopy.kind !== "agent_files") throw new NativeRuntimeContextError("Unknown agent file contract");
if (!Array.isArray(context.skills)) throw new NativeRuntimeContextError("input.runtimeContext.skills must be an array");
const skills = context.skills.map((value, index) => {
const skill = object(value, `input.runtimeContext.skills[${index}]`);
@@ -122,7 +130,15 @@ export function parseNativeRuntimeContext(value: unknown): NativeRuntimeContextS
exact(mcp, ["assignmentSetId", "digest", "bindingId"], "input.runtimeContext.mcp");
const parsed = {
prompt: { revision: PAPERCLIP_EXECUTION_PROMPT_REVISION, text: PAPERCLIP_EXECUTION_PROMPT, digest: nativeRuntimePromptDigest() },
instructions: { entryPath: safeRelativePath(instructions.entryPath, "input.runtimeContext.instructions.entryPath"), bundle: parseAsset(instructions.bundle, "input.runtimeContext.instructions.bundle") },
instructions: {
entryPath: safeRelativePath(instructions.entryPath, "input.runtimeContext.instructions.entryPath"),
bundle: parseAsset(instructions.bundle, "input.runtimeContext.instructions.bundle"),
...(workingCopy ? { workingCopy: {
...(workingCopy.kind === "agent_files" ? { kind: "agent_files" as const } : {}),
rootPath: text(workingCopy.rootPath, "input.runtimeContext.instructions.workingCopy.rootPath"),
entryPath: safeRelativePath(workingCopy.entryPath, "input.runtimeContext.instructions.workingCopy.entryPath"),
} } : {}),
},
skills,
mcp: {
assignmentSetId: text(mcp.assignmentSetId, "input.runtimeContext.mcp.assignmentSetId"),
@@ -141,6 +157,12 @@ export function composeNativeSystemInstructions(context: NativeRuntimeContextSna
return [
context.prompt.text,
entryContent.trim(),
context.instructions.workingCopy?.kind === "agent_files"
? `Your persistent agent directory (AGENT_HOME) is ${context.instructions.workingCopy.rootPath}. Your instruction entry is ${context.instructions.workingCopy.entryPath}, relative to that directory. All supported files and subfolders there are restored across tasks and sessions, and collected after this provider stops. Write task deliverables in the task working directory. Only changed or deleted files synchronize; the last sync wins for the same file. Temporary copies are cleaned up without retaining file history. Check the save receipt before claiming persistence.`
: context.instructions.workingCopy
? `Your editable agent instruction file is ${context.instructions.workingCopy.rootPath}/${context.instructions.workingCopy.entryPath}. Edit this registered private copy normally. After this run stops, Paperclip saves changed content as a persistent revision if your responsible user still has permission and the baseline has not changed. Check the run's instruction-save receipt before claiming persistence. Conflicts are preserved for explicit resolution. Repository instruction files, skills, and this run's loaded prompt are separate and are not collected.`
: null,
// Keep this canonical suffix intact for provider-specific asset remapping.
`Read-only instruction sibling root: ${context.instructions.bundle.rootPath}`,
].filter(Boolean).join("\n\n");
}
@@ -197,6 +197,7 @@ class BoundedLineDecoder {
}
const SAFE_ENVIRONMENT_KEYS = [
"AGENT_HOME",
"ALL_PROXY",
"CODEX_HOME",
"HOME",
@@ -660,7 +660,7 @@ export class CodexAppServerDriver implements HarnessDriver {
this.#options.transportFactory?.(context) ??
new ProcessCodexAppServerTransport({
workingDirectory,
args: createIsolatedCodexAppServerArgs(this.#options.environment, codexExecutableReadOnlyRoots(this.#options.environment ?? process.env)),
args: createIsolatedCodexAppServerArgs(this.#options.environment, codexExecutableReadOnlyRoots(this.#options.environment ?? process.env), this.#options.instructionWorkingCopyRoot),
environment: createSanitizedCodexEnvironment(this.#options.environment),
onDiagnostic: this.#options.onDiagnostic,
processGroup: true,
@@ -20,6 +20,8 @@ export interface CodexAppServerDriverOptions {
approvalPolicy?: "never" | "on-request" | "untrusted";
baseInstructions?: string;
includeSkillInstructions?: boolean;
/** Private instruction directory registered by the control plane for this run. */
instructionWorkingCopyRoot?: string;
/** Explicit selected skills, resolved from this task's assigned runtime assets. */
skillInputs?: readonly import("../../contracts/runtime-context.js").NativeSkillInput[];
conversationMode?: "task" | "direct" | "prepared";
@@ -12,6 +12,25 @@ import {
} from "./codex-security-config.js";
describe("Codex security configuration", () => {
it("allows only the registered private instruction directory while keeping shared context read-only", () => {
const args = createIsolatedCodexAppServerArgs({ HOME: "/host/home" }, ["/runtime/immutable-context"], "/runtime/instruction-edits/run-1").join("\n");
expect(args).toContain('"/runtime/instruction-edits/run-1"="write"');
expect(args).toContain('"/runtime/immutable-context"="read"');
expect(args).not.toContain('"/runtime"="write"');
expect(args).toContain('"/host/home"="none"');
expect(createIsolatedCodexAppServerArgs({ PAPERCLIP_INSTRUCTION_ROOT: "/host/home" }).join("\n"))
.not.toContain('"/host/home"="write"');
});
it("exposes AGENT_HOME only when it matches the controller-registered writable directory", () => {
const root = "/agent-files/run-1";
const registered = createIsolatedCodexAppServerArgs({ AGENT_HOME: root, HOME: "/provider" }, [], root).join("\n");
expect(registered).toContain('AGENT_HOME="/agent-files/run-1"');
expect(registered).toContain('"/provider"="none"');
expect(createIsolatedCodexAppServerArgs({ AGENT_HOME: "/arbitrary" }, [], root).join("\n")).not.toContain("AGENT_HOME");
expect(createIsolatedCodexAppServerArgs({ AGENT_HOME: root }).join("\n")).not.toContain("AGENT_HOME");
});
it("makes the installed npm Codex native sandbox executable readable without exposing its parent workspace", () => {
const command = evalProviderTransportOptions("codex").codexCommand!;
const manifest = createRequire(command).resolve(`@openai/codex-${process.platform}-${process.arch}/package.json`);
@@ -169,6 +169,8 @@ function tomlString(value: string): string {
export function createIsolatedCodexAppServerArgs(
source: NodeJS.ProcessEnv = process.env,
readOnlyRoots: string[] = [],
/** Server-registered run copy, never an environment/config-supplied root. */
instructionWorkingCopyRoot?: string,
): string[] {
const gitRoots = gitFilesystemRoots(source);
readOnlyRoots = [...new Set([...readOnlyRoots, ...codexNetworkReadOnlyRoots(source)])];
@@ -179,6 +181,7 @@ export function createIsolatedCodexAppServerArgs(
// Codex filters the configured `set` values through include_only as well.
// Retain the explicit command PATH/HOME/locale settings, not ambient secrets.
const commandEnvironment = codexCommandEnvironment(source);
if (instructionWorkingCopyRoot && source.AGENT_HOME === instructionWorkingCopyRoot) commandEnvironment.AGENT_HOME = instructionWorkingCopyRoot;
const shellEnvironmentKeys = [...new Set([...inheritedGitHubKeys, ...Object.keys(commandEnvironment)])].sort();
if (source.PAPERCLIP_GITHUB_LAUNCHER_DIR) readOnlyRoots = [...readOnlyRoots, source.PAPERCLIP_GITHUB_LAUNCHER_DIR];
const deniedHostRoots = [
@@ -199,6 +202,7 @@ export function createIsolatedCodexAppServerArgs(
...readOnlyRoots.map((path) => `${tomlString(resolve(path))}="read"`),
...gitRoots.read.map((path) => `${tomlString(path)}="read"`),
...gitRoots.write.map((path) => `${tomlString(path)}="write"`),
...(instructionWorkingCopyRoot ? [`${tomlString(resolve(instructionWorkingCopyRoot))}="write"`] : []),
...(source.PAPERCLIP_GITHUB_BROKER_TOKEN && source.GH_CONFIG_DIR
? [`${tomlString(resolve(source.GH_CONFIG_DIR))}="write"`] : []),
`":workspace_roots"={"."="write"}`,
@@ -210,6 +214,7 @@ export function createIsolatedCodexAppServerArgs(
...deniedHostRoots.map((path) => `${tomlString(path)}="none"`),
...readOnlyRoots.map((path) => `${tomlString(resolve(path))}="read"`),
...[...gitRoots.read, ...gitRoots.write].map((path) => `${tomlString(path)}="read"`),
...(instructionWorkingCopyRoot ? [`${tomlString(resolve(instructionWorkingCopyRoot))}="read"`] : []),
...(source.PAPERCLIP_GITHUB_BROKER_TOKEN && source.GH_CONFIG_DIR
? [`${tomlString(resolve(source.GH_CONFIG_DIR))}="write"`] : []),
`":workspace_roots"={"."="read"}`,
@@ -20,6 +20,7 @@ import {
PAPERCLIP_EXECUTION_PROMPT_REVISION,
canonicalNativeRuntimeContextDigest,
nativeRuntimePromptDigest,
parseNativeRuntimeContext,
type NativeRuntimeContextSnapshot,
} from "../contracts/runtime-context.js";
import { nativeMcpLaunchBinding } from "./native-mcp.js";
@@ -92,6 +93,17 @@ function context(
}
describe("runtime context materialization", () => {
it("restores the legacy working-copy contract without changing its digest or adding new fields", () => {
const old = context("/skills", "/instructions");
old.instructions.workingCopy = { rootPath: "/old-run/copy", entryPath: "AGENTS.md" };
old.aggregateDigest = canonicalNativeRuntimeContextDigest(old);
expect(parseNativeRuntimeContext(JSON.parse(JSON.stringify(old)))).toEqual(old);
expect(parseNativeRuntimeContext(old).instructions.workingCopy).not.toHaveProperty("kind");
const next = { ...old, instructions: { ...old.instructions, workingCopy: { ...old.instructions.workingCopy, kind: "agent_files" as const } } };
next.aggregateDigest = canonicalNativeRuntimeContextDigest(next);
expect(parseNativeRuntimeContext(next).instructions.workingCopy?.kind).toBe("agent_files");
});
it("validates native MCP launch bindings before they reach Codex", () => {
expect(nativeMcpLaunchBinding({})).toBeNull();
expect(() => nativeMcpLaunchBinding({
@@ -466,13 +466,13 @@ describe("workflow reports and stress traceability", () => {
candidateFailures: 36,
});
expect(report.coverage).toMatchObject({
canonicalOperations: 48,
canonicalOperations: 52,
capabilityCases: 106,
workflows: 12,
stressFindings: 44,
stressExclusions: 1,
});
expect(report.coverage.operations).toHaveLength(48);
expect(report.coverage.operations).toHaveLength(52);
expect(report.coverage.composedWorkflows).toHaveLength(12);
expect(
report.coverage.operations.find(
@@ -480,7 +480,7 @@ describe("workflow reports and stress traceability", () => {
)?.workflowIds.length,
).toBeGreaterThan(0);
expect(renderRunnerWorkflowMarkdown(report)).toContain(
"48 operations · 106 capability cases · 12 workflows",
"52 operations · 106 capability cases · 12 workflows",
);
expect(renderRunnerWorkflowJUnit(report)).toContain(
'tests="36" failures="36" skipped="0"',
@@ -3285,6 +3285,7 @@ export function createRunnerdCodexAppServerArgs(input: {
codexHome: string;
codexCommand?: string;
readOnlyRoots?: string[];
instructionWorkingCopyRoot?: string;
}): string[] {
// The filesystem policy denies HOME and CODEX_HOME to keep credentials and
// runner state outside provider reach. Always bind those names to the actual
@@ -3297,6 +3298,7 @@ export function createRunnerdCodexAppServerArgs(input: {
CODEX_HOME: input.codexHome,
},
[...(input.readOnlyRoots ?? []), ...codexExecutableReadOnlyRoots(input.environment ?? {}, input.codexCommand)],
input.instructionWorkingCopyRoot,
);
}
@@ -4669,6 +4671,7 @@ class DurablePrpCodexTransport implements CodexAppServerTransport {
environment: this.options.environment,
codexHome,
codexCommand: this.options.codexCommand,
instructionWorkingCopyRoot: runtimeContext?.instructions.workingCopy?.rootPath,
readOnlyRoots: [
...trustedRuntimeReadOnlyRoots(
this.options.environment,
@@ -5147,6 +5150,7 @@ class DurablePrpCodexTransport implements CodexAppServerTransport {
environment: this.options.environment,
codexHome,
codexCommand: this.options.codexCommand,
instructionWorkingCopyRoot: runtimeContext?.instructions.workingCopy?.rootPath,
readOnlyRoots: [
...trustedRuntimeReadOnlyRoots(this.options.environment),
...(runtimeContext
@@ -4014,6 +4014,45 @@ describe("executeNativeSession recovery", () => {
},
);
it("collects a failed run's private instructions only after its owned provider close joins", async () => {
const scopedIdentity = { ...identity, companyId: "instruction-close-company", runId: "instruction-close-run" };
let finishClose!: () => void;
const closed = new Promise<void>((resolve) => { finishClose = resolve; });
const order: string[] = [];
const capabilities = { resume: true, typedEvents: true, steering: false, interruption: false, structuredResult: true };
const session: NativeSession = {
identity: () => scopedIdentity,
capabilities: async () => capabilities,
async *events() { throw new Error("provider failed after editing instructions"); },
startTurn: async () => ({ turnId: "instruction-turn" }),
result: async () => null,
snapshot: async () => ({ backendKind: "local", sessionId: "instruction-session", identity: scopedIdentity,
providerSessionId: "instruction-provider", cursor: null, activeTurnId: null, pendingRuntimeRequests: [], lineage: [] }),
close: async () => { order.push("closing"); await closed; order.push("stopped"); },
};
const backend: NativeSessionBackend = {
descriptor: async () => ({ kind: "local", name: "instruction-close-test", version: "1", capabilities }),
openSession: async () => session,
};
const controlPlane: ControlPlanePort = {
openRun: async () => {}, checkpointSession: async () => {},
appendEvent: async () => ({ cursor: 0, highestContiguousSourceSeq: 0, disposition: "committed" }),
replayEvents: async () => ({ events: [], highestContiguousSourceSeq: 0 }), completeRun: async () => {},
};
const options = {
input: { ...input, binding: { ...input.binding, companyId: scopedIdentity.companyId, runId: scopedIdentity.runId } },
backend, controlPlane, runnerInstanceId: "instruction-runner", controlPlaneInstanceId: "control",
requireSessionCloseBeforeReturn: true,
onSessionClosed: async () => { order.push("collected"); },
};
const execution = executeNativeSession(options);
const failed = expect(execution).rejects.toThrow("provider failed after editing instructions");
await vi.waitFor(() => expect(order).toEqual(["closing"]));
finishClose();
await failed;
expect(order).toEqual(["closing", "stopped", "collected"]);
});
it("retires only the terminated remote resource, including two sandboxes for one run", async () => {
const scopedIdentity = { ...identity, companyId: "remote-stop-company", runId: "remote-stop-run" };
const binding = { ...scopedIdentity, remoteCleanupScope: "first-sandbox" };
@@ -4107,6 +4146,7 @@ describe("executeNativeSession recovery", () => {
vi.useFakeTimers();
try {
const closeFailure = new Error("required remote checkpoint close failed");
const onSessionClosed = vi.fn(async () => {});
const close = vi.fn(({ reason }: { reason: string }) =>
reason === "native session quarantined cleanup recovery"
? Promise.resolve()
@@ -4191,11 +4231,13 @@ describe("executeNativeSession recovery", () => {
runnerInstanceId: "runner-recovery",
controlPlaneInstanceId: "control-recovery",
requireSessionCloseBeforeReturn: true,
onSessionClosed,
}),
).rejects.toThrow(closeFailure);
await vi.advanceTimersByTimeAsync(3_000);
await execution;
expect(close).toHaveBeenCalledTimes(5);
expect(onSessionClosed).not.toHaveBeenCalled();
} finally {
vi.useRealTimers();
}
@@ -220,6 +220,9 @@ export interface ExecuteNativeSessionOptions {
* carries required persistence (for example, a remote runner checkpoint).
*/
requireSessionCloseBeforeReturn?: boolean;
/** Trusted cleanup observer, called only after the owned close completes.
* Requires requireSessionCloseBeforeReturn; never called for a live warm session. */
onSessionClosed?: () => Promise<void>;
onCheckpoint?: (
snapshot: PersistedNativeSession,
options?: CheckpointControlPlaneSessionOptions,
@@ -2756,6 +2759,7 @@ export async function executeNativeSession(
// generic retryable transport failure at the control-plane boundary.
throw protocolIntegrityFailure ?? closeError;
}
await options.onSessionClosed?.();
}
} else if (shouldClose && !failedCleanupDeferred) {
// A provider that ignores close must not keep execution pending forever.
@@ -1,3 +1,7 @@
import { readAgentInstructionsAction } from "./read-agent-instructions.js";
import { updateAgentInstructionsAction } from "./update-agent-instructions.js";
import { getAgentInstructionHistoryAction } from "./get-agent-instruction-history.js";
import { restoreAgentInstructionsAction } from "./restore-agent-instructions.js";
import { answerStatusQuestionAction } from "./answer-status-question.js";
import { blockTaskAction } from "./block-task.js";
import { finishTaskAction } from "./finish-task.js";
@@ -16,6 +20,11 @@ import { deepFreezeProtocolAction } from "./freeze.js";
/** Core actions that are always present in an authorized runner projection. */
export const PAPERCLIP_CORE_PROTOCOL_ACTIONS = deepFreezeProtocolAction([
readAgentInstructionsAction,
updateAgentInstructionsAction,
getAgentInstructionHistoryAction,
restoreAgentInstructionsAction,
answerStatusQuestionAction,
blockTaskAction,
finishTaskAction,
@@ -0,0 +1,99 @@
/** Canonical definition for `get_agent_instruction_history`. */
export const getAgentInstructionHistoryAction = {
"id": "get_agent_instruction_history",
"canonical": {
"operationId": "get_agent_instruction_history",
"surfaces": [
"live"
],
"placement": "always_agent_tool",
"optionalGroup": null,
"requiredClaims": [],
"taskModes": [
"standard",
"ask",
"planning",
"skill_test"
],
"sideEffectClass": "read",
"idempotency": "none",
"disabledByDefault": false,
"realBindingStatus": "live_codex",
"realServiceBinding": "agentInstructionRevisionService",
"prpEvidence": "tool-result item event plus canonical revision receipt for writes; no scenario mock coverage",
"prpBindingStatus": "bound",
"legacyAliases": [],
"note": "Live canonical instruction service; company scope and current responsible-user authorization are checked at invocation."
},
"documentation": {
"title": "Get agent instruction history",
"description": "List bounded canonical instruction revision metadata, including actor, source run, and restore origin. Use read_agent_instructions with entryFile and revisionId to inspect exact historical content.",
"note": null
},
"examples": {
"call": {
"operationId": "get_agent_instruction_history",
"input": {
"entryFile": "AGENTS.md"
}
},
"success": {
"ok": true,
"operationId": "get_agent_instruction_history",
"result": {}
}
},
"live": {
"order": 42,
"descriptor": {
"schema": "paperclip.semantic-tool.v1",
"operationId": "get_agent_instruction_history",
"version": 1,
"title": "Get agent instruction history",
"description": "List bounded canonical instruction revision metadata, including actor, source run, and restore origin. Use read_agent_instructions with entryFile and revisionId to inspect exact historical content.",
"exposure": "always",
"requiredClaims": [],
"allowedModes": [
"standard",
"ask",
"planning",
"skill_test"
],
"inputSchema": {
"type": "object",
"properties": {
"targetAgentId": {
"type": "string",
"pattern": "^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$",
"description": "Same-company target agent. Omit to use the calling agent."
},
"entryFile": {
"type": "string",
"minLength": 1,
"maxLength": 4096,
"description": "Configured relative entry filename returned by read_agent_instructions; retain it with the revision."
},
"cursor": {
"type": "string",
"minLength": 1,
"maxLength": 2048
},
"limit": {
"type": "integer",
"minimum": 1,
"maximum": 100
}
},
"required": [
"entryFile"
],
"additionalProperties": false
},
"outputSchema": {
"type": "object",
"additionalProperties": true
}
}
},
"scenario": null
} as const;
@@ -1,3 +1,7 @@
import { readAgentInstructionsAction } from "./read-agent-instructions.js";
import { updateAgentInstructionsAction } from "./update-agent-instructions.js";
import { getAgentInstructionHistoryAction } from "./get-agent-instruction-history.js";
import { restoreAgentInstructionsAction } from "./restore-agent-instructions.js";
import { reassignTaskAction } from "./reassign-task.js";
import { createSkillAction } from "./create-skill.js";
import { createProjectAction } from "./create-project.js";
@@ -49,6 +53,11 @@ import { writeDocumentAction } from "./write-document.js";
import { deepFreezeProtocolAction } from "./freeze.js";
export const PAPERCLIP_PROTOCOL_ACTIONS = deepFreezeProtocolAction([
readAgentInstructionsAction,
updateAgentInstructionsAction,
getAgentInstructionHistoryAction,
restoreAgentInstructionsAction,
createSkillAction,
createProjectAction,
listProjectRepositoriesAction,
@@ -0,0 +1,90 @@
/** Canonical definition for `read_agent_instructions`. */
export const readAgentInstructionsAction = {
"id": "read_agent_instructions",
"canonical": {
"operationId": "read_agent_instructions",
"surfaces": [
"live"
],
"placement": "always_agent_tool",
"optionalGroup": null,
"requiredClaims": [],
"taskModes": [
"standard",
"ask",
"planning",
"skill_test"
],
"sideEffectClass": "read",
"idempotency": "none",
"disabledByDefault": false,
"realBindingStatus": "live_codex",
"realServiceBinding": "agentInstructionRevisionService",
"prpEvidence": "tool-result item event plus canonical revision receipt for writes; no scenario mock coverage",
"prpBindingStatus": "bound",
"legacyAliases": [],
"note": "Live canonical instruction service; company scope and current responsible-user authorization are checked at invocation."
},
"documentation": {
"title": "Read canonical agent instructions",
"description": "Read the current canonical instruction entry and its revision, or inspect a historical revision by supplying both entryFile and revisionId. Read before updating; do not edit shared instruction caches.",
"note": null
},
"examples": {
"call": {
"operationId": "read_agent_instructions",
"input": {}
},
"success": {
"ok": true,
"operationId": "read_agent_instructions",
"result": {}
}
},
"live": {
"order": 40,
"descriptor": {
"schema": "paperclip.semantic-tool.v1",
"operationId": "read_agent_instructions",
"version": 1,
"title": "Read canonical agent instructions",
"description": "Read the current canonical instruction entry and its revision, or inspect a historical revision by supplying both entryFile and revisionId. Read before updating; do not edit shared instruction caches.",
"exposure": "always",
"requiredClaims": [],
"allowedModes": [
"standard",
"ask",
"planning",
"skill_test"
],
"inputSchema": {
"type": "object",
"properties": {
"targetAgentId": {
"type": "string",
"pattern": "^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$",
"description": "Same-company target agent. Omit to use the calling agent."
},
"entryFile": {
"type": "string",
"minLength": 1,
"maxLength": 4096,
"description": "Configured relative entry filename returned by read_agent_instructions; retain it with the revision."
},
"revisionId": {
"type": "string",
"pattern": "^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$",
"description": "Historical revision to inspect; also provide its entryFile."
}
},
"required": [],
"additionalProperties": false
},
"outputSchema": {
"type": "object",
"additionalProperties": true
}
}
},
"scenario": null
} as const;
@@ -0,0 +1,99 @@
/** Canonical definition for `restore_agent_instructions`. */
export const restoreAgentInstructionsAction = {
"id": "restore_agent_instructions",
"canonical": {
"operationId": "restore_agent_instructions",
"surfaces": [
"live"
],
"placement": "always_agent_tool",
"optionalGroup": null,
"requiredClaims": [],
"taskModes": [
"standard",
"planning"
],
"sideEffectClass": "company_write",
"idempotency": "recommended",
"disabledByDefault": false,
"realBindingStatus": "live_codex",
"realServiceBinding": "agentInstructionRevisionService",
"prpEvidence": "tool-result item event plus canonical revision receipt for writes; no scenario mock coverage",
"prpBindingStatus": "bound",
"legacyAliases": [],
"note": "Live canonical instruction service; company scope and current responsible-user authorization are checked at invocation."
},
"documentation": {
"title": "Restore canonical agent instructions",
"description": "Append a historical instruction revision as the current content using the current baseRevisionId. Requires the responsible user\u2019s current target edit permission. History remains intact; conflicts require an explicit resolution.",
"note": null
},
"examples": {
"call": {
"operationId": "restore_agent_instructions",
"input": {
"entryFile": "AGENTS.md",
"revisionId": "00000000-0000-4000-8000-000000000001",
"baseRevisionId": "00000000-0000-4000-8000-000000000002"
}
},
"success": {
"ok": true,
"operationId": "restore_agent_instructions",
"result": {}
}
},
"live": {
"order": 43,
"descriptor": {
"schema": "paperclip.semantic-tool.v1",
"operationId": "restore_agent_instructions",
"version": 1,
"title": "Restore canonical agent instructions",
"description": "Append a historical instruction revision as the current content using the current baseRevisionId. Requires the responsible user\u2019s current target edit permission. History remains intact; conflicts require an explicit resolution.",
"exposure": "always",
"requiredClaims": [],
"allowedModes": [
"standard",
"planning"
],
"inputSchema": {
"type": "object",
"properties": {
"targetAgentId": {
"type": "string",
"pattern": "^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$",
"description": "Same-company target agent. Omit to use the calling agent."
},
"entryFile": {
"type": "string",
"minLength": 1,
"maxLength": 4096,
"description": "Configured relative entry filename returned by read_agent_instructions; retain it with the revision."
},
"revisionId": {
"type": "string",
"pattern": "^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$",
"description": "Historical revision whose exact content should be restored."
},
"baseRevisionId": {
"type": "string",
"description": "Current revision read before restoring. Never replace a stale base silently.",
"pattern": "^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$"
}
},
"required": [
"entryFile",
"revisionId",
"baseRevisionId"
],
"additionalProperties": false
},
"outputSchema": {
"type": "object",
"additionalProperties": true
}
}
},
"scenario": null
} as const;
@@ -0,0 +1,102 @@
/** Canonical definition for `update_agent_instructions`. */
export const updateAgentInstructionsAction = {
"id": "update_agent_instructions",
"canonical": {
"operationId": "update_agent_instructions",
"surfaces": [
"live"
],
"placement": "always_agent_tool",
"optionalGroup": null,
"requiredClaims": [],
"taskModes": [
"standard",
"planning"
],
"sideEffectClass": "company_write",
"idempotency": "recommended",
"disabledByDefault": false,
"realBindingStatus": "live_codex",
"realServiceBinding": "agentInstructionRevisionService",
"prpEvidence": "tool-result item event plus canonical revision receipt for writes; no scenario mock coverage",
"prpBindingStatus": "bound",
"legacyAliases": [],
"note": "Live canonical instruction service; company scope and current responsible-user authorization are checked at invocation."
},
"documentation": {
"title": "Update canonical agent instructions",
"description": "Commit instruction content with the exact entryFile and baseRevisionId from a prior read. Requires the responsible user\u2019s current target edit permission. On conflict, preserve your candidate and explicitly resolve it; never overwrite the newer head.",
"note": null
},
"examples": {
"call": {
"operationId": "update_agent_instructions",
"input": {
"entryFile": "AGENTS.md",
"content": "# Instructions\n",
"baseRevisionId": null
}
},
"success": {
"ok": true,
"operationId": "update_agent_instructions",
"result": {}
}
},
"live": {
"order": 41,
"descriptor": {
"schema": "paperclip.semantic-tool.v1",
"operationId": "update_agent_instructions",
"version": 1,
"title": "Update canonical agent instructions",
"description": "Commit instruction content with the exact entryFile and baseRevisionId from a prior read. Requires the responsible user\u2019s current target edit permission. On conflict, preserve your candidate and explicitly resolve it; never overwrite the newer head.",
"exposure": "always",
"requiredClaims": [],
"allowedModes": [
"standard",
"planning"
],
"inputSchema": {
"type": "object",
"properties": {
"targetAgentId": {
"type": "string",
"pattern": "^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$",
"description": "Same-company target agent. Omit to use the calling agent."
},
"entryFile": {
"type": "string",
"minLength": 1,
"maxLength": 4096,
"description": "Configured relative entry filename returned by read_agent_instructions; retain it with the revision."
},
"content": {
"type": "string",
"maxLength": 1048576,
"description": "Complete UTF-8 instruction content, at most 1 MiB; empty content is valid."
},
"baseRevisionId": {
"type": [
"string",
"null"
],
"description": "Revision read before editing; null only when no canonical entry exists. Never replace a stale base silently.",
"pattern": "^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$"
}
},
"required": [
"entryFile",
"content",
"baseRevisionId"
],
"additionalProperties": false
},
"outputSchema": {
"type": "object",
"additionalProperties": true
}
}
},
"scenario": null
} as const;
@@ -27,7 +27,12 @@ const NAMESPACE: Readonly<Record<CapabilitySemanticOperationId, string>> = Objec
report_progress: "active_task", answer_status_question: "active_task", write_document: "documents",
request_human_input: "documents", register_deliverable: "documents", finish_task: "active_task",
block_task: "active_task", request_review: "active_task", search_tasks: "discovery",
list_agents: "discovery", get_agent: "discovery", create_skill: "skills", create_task: "delegation", create_project: "projects", list_project_repositories: "projects", list_projects: "projects",
list_agents: "discovery", get_agent: "discovery",
read_agent_instructions: "agent_instructions",
update_agent_instructions: "agent_instructions",
get_agent_instruction_history: "agent_instructions",
restore_agent_instructions: "agent_instructions",
create_skill: "skills", create_task: "delegation", create_project: "projects", list_project_repositories: "projects", list_projects: "projects",
set_dependencies: "delegation", reassign_task: "delegation", list_approvals: "governance", get_approval: "governance",
get_approval_context: "governance", request_approval: "governance",
decide_approval: "governance", comment_on_approval: "governance",
@@ -102,6 +107,9 @@ export function discoverCapabilityDefinitions(
const tokens = normalized.split(/[^a-z0-9]+/).filter((token) => token.length > 1);
const permitted = CAPABILITY_SEMANTIC_TOOL_CATALOG
.filter((descriptor) => descriptor.exposure === "optional")
// Retain dispatch support for pre-upgrade sessions, but do not advertise
// revision history/restore as a capability of current agent directories.
.filter((descriptor) => descriptor.operationId !== "get_agent_instruction_history" && descriptor.operationId !== "restore_agent_instructions")
.filter((descriptor) => options.namespace === undefined || capabilityToolNamespace(descriptor.operationId) === options.namespace)
.filter((descriptor) => decideCapabilitySemanticAuthorization(descriptor, context, "exposure").allowed)
.map((descriptor) => ({ descriptor, score: score(descriptor, tokens, normalized) }))
@@ -94,7 +94,7 @@ describe("Capability semantic catalog and authorization", () => {
it("publishes a stable narrow catalog without credentials or control-plane-owned tools", () => {
const names = CAPABILITY_SEMANTIC_TOOL_CATALOG.map((tool) => tool.operationId);
expect(new Set(names).size).toBe(names.length);
expect(names).toHaveLength(36);
expect(names).toHaveLength(40);
expect(names).toContain("get_task_context");
expect(names).toContain("finish_task");
expect(names).not.toContain("checkout_task");
@@ -33,6 +33,10 @@ export type CapabilitySemanticOperationId =
| "request_review"
| "list_agents"
| "get_agent"
| "read_agent_instructions"
| "update_agent_instructions"
| "get_agent_instruction_history"
| "restore_agent_instructions"
| "search_tasks"
| "list_approvals"
| "get_approval"
+7
View File
@@ -2789,3 +2789,10 @@ export { MEMORY_CONNECTOR_IDS, isMemoryConnectorId, type MemoryConnectorId } fro
export * from "./connection-routing.js";
export { WORKSPACE_RESTORE_FAILURE_CODES, hasWorkspaceRestoreFailure, safeWorkspaceRestorePath, isNativeWorkspaceExportRepairCause } from "./workspace-restore.js";
export type { AgentInstructionErrorCode, AgentInstructionErrorDetails, AgentInstructionSource, AgentInstructionRevision, AgentInstructionSnapshot, AgentInstructionCommitReceipt, AgentInstructionHistory, AgentInstructionDiff } from "./types/agent.js";
export { restoreAgentInstructionSchema } from "./validators/agent.js";
export type { AgentInstructionCandidate } from "./types/agent.js";
export { resolveAgentInstructionCandidateSchema, type ResolveAgentInstructionCandidate } from "./validators/agent.js";
+79
View File
@@ -32,6 +32,8 @@ export type AgentInstructionsBundleMode = "managed" | "external";
export interface AgentInstructionsFileSummary {
path: string;
contentHash?: string;
binary?: boolean;
size: number;
language: string;
markdown: boolean;
@@ -43,11 +45,14 @@ export interface AgentInstructionsFileSummary {
export interface AgentInstructionsFileDetail extends AgentInstructionsFileSummary {
content: string;
revision?: AgentInstructionRevision;
receipt?: AgentInstructionCommitReceipt;
}
export interface AgentInstructionsBundle {
agentId: string;
companyId: string;
persistence?: "agent_files";
mode: AgentInstructionsBundleMode | null;
rootPath: string | null;
managedRootPath: string;
@@ -336,3 +341,77 @@ export type AdapterAuthSignal = "present" | "absent" | "unknown";
export interface AdapterAuthSignalResponse {
status: AdapterAuthSignal;
}
export type AgentInstructionSource = "seed" | "board" | "api" | "tool" | "cleanup" | "restore";
export interface AgentInstructionRevision {
id: string;
companyId: string;
agentId: string;
entryFile: string;
contentHash: string;
byteLength: number;
parentRevisionId: string | null;
baseRevisionId: string | null;
restoredFromRevisionId: string | null;
actorAgentId: string | null;
actorUserId: string | null;
responsibleUserId: string | null;
sourceRunId: string | null;
source: AgentInstructionSource;
createdAt: string;
}
export interface AgentInstructionSnapshot {
revision: AgentInstructionRevision;
content: string;
}
export interface AgentInstructionCommitReceipt extends AgentInstructionSnapshot {
changed: boolean;
materialization: "current" | "pending";
}
export interface AgentInstructionHistory {
revisions: AgentInstructionRevision[];
nextCursor: string | null;
}
export interface AgentInstructionDiff {
from: AgentInstructionSnapshot;
to: AgentInstructionSnapshot;
prefix: string;
removed: string;
added: string;
suffix: string;
}
export type AgentInstructionErrorCode =
| "INSTRUCTION_BASE_REQUIRED"
| "INSTRUCTION_REVISION_REQUIRED"
| "INSTRUCTION_REVISION_CONFLICT"
| "INSTRUCTION_ENTRY_CHANGED"
| "INSTRUCTION_MANAGED_BUNDLE_REQUIRED"
| "INSTRUCTION_IDENTITY_INVALID"
| "INSTRUCTION_CONTENT_INVALID"
| "INSTRUCTION_PATH_INVALID"
| "RESPONSIBLE_USER_UNAVAILABLE"
| "RESPONSIBLE_USER_UNAUTHORIZED";
export interface AgentInstructionErrorDetails {
code: AgentInstructionErrorCode;
baseRevisionId?: string | null;
currentRevisionId?: string | null;
entryFile?: string;
reason?: string;
}
/** Preserved instruction edits; filesystem locations and responsible identity stay server-side. */
export interface AgentInstructionCandidate {
contract?: "agent_files" | "legacy";
runId: string;
entryFile: string;
baseRevisionId: string | null;
baseHash: string;
state: "conflict" | "pending_collection" | "pending_commit" | "unavailable";
candidateHash: string | null;
content: string | null;
errorCode: string | null;
errorMessage: string | null;
createdAt: string;
updatedAt: string;
}
+4
View File
@@ -1069,3 +1069,7 @@ export * from "./chat-channels.js";
export * from "./chat-github.js";
export * from "./email.js";
export type { AgentInstructionErrorCode, AgentInstructionErrorDetails, AgentInstructionSource, AgentInstructionRevision, AgentInstructionSnapshot, AgentInstructionCommitReceipt, AgentInstructionHistory, AgentInstructionDiff } from "./agent.js";
export type { AgentInstructionCandidate } from "./agent.js";
+17 -3
View File
@@ -34,10 +34,18 @@ export const updateAgentInstructionsBundleSchema = z.object({
export type UpdateAgentInstructionsBundle = z.infer<typeof updateAgentInstructionsBundleSchema>;
export const upsertAgentInstructionsFileSchema = z.object({
path: z.string().trim().min(1),
content: z.string(),
path: z.string().min(1).max(512),
content: z.string().max(1024 * 1024),
baseRevisionId: z.string().uuid().nullable().optional(),
baseHash: z.string().regex(/^[a-f0-9]{64}$/).nullable().optional(),
clearLegacyPromptTemplate: z.boolean().optional().default(false),
});
}).strict();
export const restoreAgentInstructionSchema = z.object({
path: z.string().min(1).max(512),
revisionId: z.string().uuid(),
baseRevisionId: z.string().uuid(),
}).strict();
export type UpsertAgentInstructionsFile = z.infer<typeof upsertAgentInstructionsFileSchema>;
@@ -289,3 +297,9 @@ export const updateAgentPermissionsSchema = z.object({
});
export type UpdateAgentPermissions = z.infer<typeof updateAgentPermissionsSchema>;
export const resolveAgentInstructionCandidateSchema = z.object({
baseRevisionId: z.string().uuid().nullable(),
content: z.string().max(1024 * 1024),
}).strict();
export type ResolveAgentInstructionCandidate = z.infer<typeof resolveAgentInstructionCandidateSchema>;
+2
View File
@@ -985,3 +985,5 @@ export * from "./chat-channels.js";
export * from "./chat-github.js";
export * from "./email.js";
export { restoreAgentInstructionSchema } from "./agent.js";
@@ -0,0 +1,539 @@
import fs from "node:fs/promises";
import { execFile as execFileCallback } from "node:child_process";
import { promisify } from "node:util";
import * as executionTargetTools from "@paperclipai/adapter-utils/execution-target";
import * as ssh from "@paperclipai/adapter-utils/ssh";
const execFile = promisify(execFileCallback);
import os from "node:os";
import path from "node:path";
import { createHash, randomUUID } from "node:crypto";
import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
import { eq } from "drizzle-orm";
import { agentFileStore, fileHash, inspectAgentFile, snapshotAgentFiles, MAX_AGENT_FILE_BYTES, MAX_AGENT_DIRECTORY_BYTES, MAX_AGENT_DIRECTORY_ENTRIES } from "../services/agent-file-store.js";
import { agents, companies, authUsers, companyMemberships, principalPermissionGrants, heartbeatRuns, environmentLeases, environments, agentInstructionWorkingCopies, agentInstructionRevisions, agentInstructionHeads, createDb } from "@paperclipai/db";
import { startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
import { agentInstructionRevisionService } from "../services/agent-instruction-revisions.js";
import { agentInstructionWorkingCopyService, instructionWorkingCopyGuidance } from "../services/agent-instruction-working-copies.js";
import { resolveManagedInstructionsRoot } from "../services/agent-instructions.js";
import { buildNativeRuntimeContext } from "../services/native-runtime/runtime-context.js";
import type { EnvironmentRuntimeService } from "../services/environment-runtime.js";
import { remoteTerminationReceipt } from "../services/remote-execution-termination.js";
describe("persistent agent directories", () => {
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
let db: ReturnType<typeof createDb>;
let copies: ReturnType<typeof agentInstructionWorkingCopyService>;
let revisions: ReturnType<typeof agentInstructionRevisionService>;
const previousHome = process.env.PAPERCLIP_HOME;
let home: string;
let companyId: string, agentId: string, userId: string, root: string;
const entryFile = "policy/INSTRUCTIONS.txt";
const initial = "\uFEFF# Original\r\n☃\n";
const target = () => ({ companyId, agentId });
const board = () => ({ type: "board" as const, userId, source: "session" as const });
async function run() {
const runId = randomUUID();
await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, invocationSource: "on_demand", responsibleUserId: userId });
return (await copies.prepare({ ...target(), runId, cwd: home }))!;
}
beforeAll(async () => {
home = await fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), "instruction-working-copies-")));
process.env.PAPERCLIP_HOME = home;
database = await startEmbeddedPostgresTestDatabase("instruction-copies-db-");
db = createDb(database.connectionString);
copies = agentInstructionWorkingCopyService(db);
revisions = agentInstructionRevisionService(db);
}, 90_000);
afterAll(async () => {
if (previousHome === undefined) delete process.env.PAPERCLIP_HOME; else process.env.PAPERCLIP_HOME = previousHome;
await database?.cleanup();
if (home) {
const writable = async (dir: string) => {
await fs.chmod(dir, 0o700);
for (const entry of await fs.readdir(dir, { withFileTypes: true })) if (entry.isDirectory()) await writable(path.join(dir, entry.name));
};
await writable(home);
await fs.rm(home, { recursive: true, force: true });
}
});
beforeEach(async () => {
companyId = randomUUID(); agentId = randomUUID(); userId = randomUUID();
await db.insert(companies).values({ id: companyId, name: "Instruction tests", issuePrefix: randomUUID().slice(0, 8) });
await db.insert(authUsers).values({ id: userId, name: "Editor", email: `${userId}@example.test`, createdAt: new Date(), updatedAt: new Date() });
root = resolveManagedInstructionsRoot({ ...target(), id: agentId, name: "Target", adapterConfig: {} });
await db.insert(agents).values({ id: agentId, companyId, name: "Target", adapterConfig: { instructionsBundleMode: "managed", instructionsRootPath: root, instructionsEntryFile: entryFile } });
await db.insert(companyMemberships).values([
{ companyId, principalType: "user", principalId: userId, membershipRole: "operator" },
{ companyId, principalType: "agent", principalId: agentId, membershipRole: "member" },
]);
await db.insert(principalPermissionGrants).values({ companyId, principalType: "user", principalId: userId, permissionKey: "agents:configure", scope: { agentIds: [agentId] } });
await fs.mkdir(path.dirname(path.join(root, entryFile)), { recursive: true });
await fs.writeFile(path.join(root, entryFile), initial);
});
it.each([".paperclip-runtime/state", "notes/.paperclip-runtime/state", "promptTemplate.legacy.md"])("rejects reserved board path %s before mutation", async (reserved) => {
await expect(agentFileStore(db).write({ ...target(), path: reserved, bytes: Buffer.from("reserved"), baseHash: null }, board())).rejects.toMatchObject({ status: 422 });
await expect(fs.stat(path.join(root, reserved))).rejects.toMatchObject({ code: "ENOENT" });
expect(await run()).toBeTruthy();
});
it("round trips nested text, empty directories and binary bytes independently of task files", async () => {
const first = await run();
expect(first.localRoot.startsWith(path.join(home, ".paperclip-runtime"))).toBe(false);
await fs.mkdir(path.join(first.localRoot, "notes", "empty"), { recursive: true });
await fs.writeFile(path.join(first.localRoot, "notes", "fact.txt"), "remember me");
const bytes = Buffer.from([0, 255, 17, 128, 9]);
await fs.writeFile(path.join(first.localRoot, "image.bin"), bytes);
await fs.writeFile(path.join(home, "task-only.txt"), "not personal");
expect((await copies.collectStopped({ companyId, runId: first.runId }))?.state).toBe("saved");
copies = agentInstructionWorkingCopyService(db);
const next = await run();
expect(await fs.readFile(path.join(next.localRoot, "notes", "fact.txt"), "utf8")).toBe("remember me");
expect(await fs.readFile(path.join(next.localRoot, "image.bin"))).toEqual(bytes);
expect((await fs.stat(path.join(next.localRoot, "notes", "empty"))).isDirectory()).toBe(true);
await expect(fs.stat(path.join(next.localRoot, "task-only.txt"))).rejects.toMatchObject({ code: "ENOENT" });
expect(await db.select().from(agentInstructionRevisions).where(eq(agentInstructionRevisions.agentId, agentId))).toHaveLength(0);
});
async function sparseFile(filename: string, size: number) {
const handle = await fs.open(filename, "w");
try { await handle.truncate(size); } finally { await handle.close(); }
}
it("saves and restores files beyond the former file and folder limits, with streaming inspection and download", async () => {
const first = await run();
const size = 66 * 1024 * 1024;
await sparseFile(path.join(first.localRoot, "large.bin"), size);
expect((await copies.collectStopped({ companyId, runId: first.runId }))?.state).toBe("saved");
await copies.release(companyId, first.runId);
const next = await run();
const restored = await inspectAgentFile(next.localRoot, "large.bin");
expect(restored).toMatchObject({ size, bytes: null });
const store = agentFileStore(db);
const download = (await store.download(companyId, agentId, "large.bin", board()))!;
expect(download.size).toBe(size);
// The lock is released before consuming the stream. An editor replacement
// must not change the already opened download's bytes.
await store.write({ ...target(), path: "large.bin", bytes: Buffer.from("replacement"), baseHash: restored!.hash }, board());
const hash = createHash("sha256");
let downloaded = 0;
for await (const chunk of download.stream) { downloaded += chunk.length; hash.update(chunk); }
expect(downloaded).toBe(size);
expect(hash.digest("hex")).toBe(restored!.hash);
expect(await fs.readFile(path.join(root, "large.bin"), "utf8")).toBe("replacement");
await store.write({ ...target(), path: "empty.bin", bytes: Buffer.alloc(0), baseHash: null }, board());
const empty = (await store.download(companyId, agentId, "empty.bin", board()))!;
expect(empty.size).toBe(0);
for await (const _chunk of empty.stream) throw new Error("Empty download must have no chunks");
}, 30_000);
it("accepts the exact 256 MiB file boundary without retaining a text buffer", async () => {
expect(MAX_AGENT_FILE_BYTES).toBe(256 * 1024 * 1024);
await sparseFile(path.join(root, "boundary.bin"), MAX_AGENT_FILE_BYTES);
expect(await inspectAgentFile(root, "boundary.bin")).toMatchObject({ size: MAX_AGENT_FILE_BYTES, bytes: null });
});
it("checks quota for a small editor save without reading unrelated file bytes", async () => {
const asset = path.join(root, "large.bin");
await sparseFile(asset, MAX_AGENT_FILE_BYTES);
const open = vi.spyOn(fs, "open");
try {
await agentFileStore(db).write({ ...target(), path: "note.txt", bytes: Buffer.from("small edit"), baseHash: null }, board());
expect(open.mock.calls.some(([filename]) => filename === asset)).toBe(false);
expect(await fs.readFile(path.join(root, "note.txt"), "utf8")).toBe("small edit");
} finally { open.mockRestore(); }
});
it("reports an oversized run file without a partial save and removes its temporary copy", async () => {
const copy = await run();
await sparseFile(path.join(copy.localRoot, "too-large.bin"), MAX_AGENT_FILE_BYTES + 1);
await fs.writeFile(path.join(copy.localRoot, entryFile), "changed instructions");
const failed = await copies.collectStopped({ companyId, runId: copy.runId });
expect(failed).toMatchObject({ state: "unavailable", errorCode: "AGENT_FILES_LIMIT_EXCEEDED", candidateHash: null, nextAttemptAt: null, attempts: 1 });
expect(failed?.receipt?.storageWarning).toContain("Runs can continue");
expect(failed?.errorMessage).toContain('"too-large.bin" exceeds the 256 MiB');
expect(await fs.readFile(path.join(root, entryFile), "utf8")).toBe(initial);
await expect(fs.stat(path.join(root, "too-large.bin"))).rejects.toMatchObject({ code: "ENOENT" });
expect((await copies.reportUnavailable(companyId, copy.runId))?.errorCode).toBe("AGENT_FILES_LIMIT_EXCEEDED");
await copies.release(companyId, copy.runId);
await expect(fs.stat(copy.localRoot)).rejects.toMatchObject({ code: "ENOENT" });
expect((await copies.get(companyId, copy.runId))?.receipt?.baseline).toBeUndefined();
const next = await run();
expect(await fs.readFile(path.join(next.localRoot, entryFile), "utf8")).toBe(initial);
await fs.writeFile(path.join(next.localRoot, "next-task.txt"), "still working");
expect((await copies.collectStopped({ companyId, runId: next.runId }))?.state).toBe("saved");
expect(await fs.readFile(path.join(root, "next-task.txt"), "utf8")).toBe("still working");
});
it("warns on each run at the file limit and clears the warning after ordinary agent cleanup", async () => {
await sparseFile(path.join(root, "full.bin"), MAX_AGENT_FILE_BYTES);
const first = await run();
expect(first.receipt?.storageWarning).toContain("256 MiB");
expect(instructionWorkingCopyGuidance(first)).toContain("Runs can continue");
const unchanged = await copies.collectStopped({ companyId, runId: first.runId });
expect(unchanged).toMatchObject({ state: "unchanged", errorCode: null });
expect(unchanged?.receipt?.storageWarning).toContain("Agent storage is full");
copies = agentInstructionWorkingCopyService(db);
const second = await run();
expect(second.receipt?.storageWarning).toContain("Agent storage is full");
await fs.unlink(path.join(second.localRoot, "full.bin"));
await fs.writeFile(path.join(second.localRoot, "task-output.txt"), "work continues");
expect(await copies.collectStopped({ companyId, runId: second.runId })).toMatchObject({ state: "saved", receipt: { storageWarning: null } });
const next = await run();
expect(next.receipt?.storageWarning).toBeNull();
expect(await fs.readFile(path.join(next.localRoot, "task-output.txt"), "utf8")).toBe("work continues");
}, 30_000);
it("starts successive runs with an already oversized saved file and lets the agent remove it", async () => {
await sparseFile(path.join(root, "old-large.bin"), MAX_AGENT_FILE_BYTES + 1);
const first = await run();
expect(first.state).toBe("prepared");
expect(first.receipt?.storageWarning).toContain("256 MiB");
expect(await fs.readFile(path.join(first.localRoot, entryFile), "utf8")).toBe(initial);
const stopped = await copies.collectStopped({ companyId, runId: first.runId });
expect(stopped).toMatchObject({ state: "unavailable", errorCode: "AGENT_FILES_LIMIT_EXCEEDED" });
await expect(fs.stat(first.localRoot)).rejects.toMatchObject({ code: "ENOENT" });
copies = agentInstructionWorkingCopyService(db);
const cleanup = await run();
expect(cleanup.state).toBe("prepared");
await fs.unlink(path.join(cleanup.localRoot, "old-large.bin"));
expect(await copies.collectStopped({ companyId, runId: cleanup.runId })).toMatchObject({ state: "saved", receipt: { storageWarning: null } });
expect((await run()).receipt?.storageWarning).toBeNull();
}, 30_000);
it("starts when the existing folder exceeds its total quota and accepts agent cleanup", async () => {
for (let index = 0; index < 8; index++) await fs.writeFile(path.join(root, `quota-part-${index}.bin`), "fixture");
const lstat = fs.lstat.bind(fs);
// Keep this regression small on disk while exercising the real filesystem,
// snapshots, database receipts, restore, and merge with 2 GiB of metadata.
const sizes = vi.spyOn(fs, "lstat").mockImplementation(async (...args: Parameters<typeof fs.lstat>) => {
const stat = await lstat(...args);
if (path.basename(String(args[0])).startsWith("quota-part-")) Object.assign(stat, { size: MAX_AGENT_FILE_BYTES });
return stat;
});
try {
const first = await run();
expect(first.receipt?.storageWarning).toContain("2 GiB");
expect((await copies.collectStopped({ companyId, runId: first.runId }))?.errorCode).toBe("AGENT_FILES_LIMIT_EXCEEDED");
const cleanup = await run();
for (let index = 0; index < 8; index++) await fs.unlink(path.join(cleanup.localRoot, `quota-part-${index}.bin`));
await fs.writeFile(path.join(cleanup.localRoot, "small.txt"), "recovered");
expect(await copies.collectStopped({ companyId, runId: cleanup.runId })).toMatchObject({ state: "saved", receipt: { storageWarning: null } });
expect((await run()).receipt?.storageWarning).toBeNull();
} finally { sizes.mockRestore(); }
});
it("rejects more than 2 GiB in aggregate before hashing or saving any file", async () => {
expect(MAX_AGENT_DIRECTORY_BYTES).toBe(2 * 1024 * 1024 * 1024);
const copy = await run();
for (let index = 0; index < 8; index++) await sparseFile(path.join(copy.localRoot, `part-${index}.bin`), MAX_AGENT_FILE_BYTES);
// The eight allowed files exactly fill the quota; the entry is additional.
const result = await copies.collectStopped({ companyId, runId: copy.runId });
expect(result).toMatchObject({ state: "unavailable", errorCode: "AGENT_FILES_LIMIT_EXCEEDED" });
expect(result?.errorMessage).toContain("2 GiB total storage limit");
expect(await fs.readdir(root)).toEqual(["policy"]);
});
it("rejects excessive entry count before snapshotting", async () => {
const directory = path.join(home, "many-entries");
await fs.mkdir(directory);
await fs.writeFile(path.join(directory, "empty"), "");
const entries = await fs.readdir(directory, { withFileTypes: true });
// Exercise the walker bound without creating 100,001 physical files.
const readdir = vi.spyOn(fs, "readdir").mockResolvedValue(Array(MAX_AGENT_DIRECTORY_ENTRIES + 1).fill(entries[0]));
try {
await expect(snapshotAgentFiles(directory)).rejects.toMatchObject({ status: 422, message: expect.stringContaining("100,000-entry limit") });
} finally { readdir.mockRestore(); }
}, 30_000);
it("keeps the entry's 1 MiB limit and cleans up failed synchronization", async () => {
const copy = await run();
await fs.writeFile(path.join(copy.localRoot, entryFile), "a".repeat(1024 * 1024 + 1));
const result = await copies.collectStopped({ companyId, runId: copy.runId });
expect(result?.state).toBe("unavailable");
await expect(fs.stat(copy.localRoot)).rejects.toMatchObject({ code: "ENOENT" });
expect(result?.errorMessage).toContain("at most 1 MiB");
expect(await fs.readFile(path.join(root, entryFile), "utf8")).toBe(initial);
});
it("merges independent changes and uses the last synchronization for same-file edits", async () => {
const a = await run(), b = await run();
await fs.writeFile(path.join(a.localRoot, "a.txt"), "A");
await fs.writeFile(path.join(b.localRoot, "b.txt"), "B");
await Promise.all([a, b].map(copy => copies.collectStopped({ companyId, runId: copy.runId })));
expect(await fs.readFile(path.join(root, "a.txt"), "utf8")).toBe("A");
expect(await fs.readFile(path.join(root, "b.txt"), "utf8")).toBe("B");
const c = await run(), d = await run();
await fs.writeFile(path.join(c.localRoot, "a.txt"), "C");
await fs.writeFile(path.join(d.localRoot, "a.txt"), "D");
await fs.writeFile(path.join(c.localRoot, "b.txt"), "new B");
expect((await copies.collectStopped({ companyId, runId: c.runId }))?.state).toBe("saved");
expect((await copies.collectStopped({ companyId, runId: d.runId }))?.state).toBe("saved");
expect(await fs.readFile(path.join(root, "a.txt"), "utf8")).toBe("D");
expect(await fs.readFile(path.join(root, "b.txt"), "utf8")).toBe("new B");
expect(await copies.list(companyId, agentId, board())).toEqual([]);
for (const copy of [a, b, c, d]) await expect(fs.stat(path.dirname(copy.localRoot))).rejects.toMatchObject({ code: "ENOENT" });
expect(await fs.readdir(path.join(path.dirname(root), "file-sync", "runs"))).toEqual([]);
});
it("makes concurrent edit/delete races last-sync-wins and preserves unrelated new files", async () => {
await fs.mkdir(path.join(root, "notes"));
await fs.writeFile(path.join(root, "notes", "old.txt"), "old");
const deleting = await run(), editing = await run();
await fs.rm(path.join(deleting.localRoot, "notes"), { recursive: true });
await fs.writeFile(path.join(editing.localRoot, "notes", "old.txt"), "edited");
await fs.writeFile(path.join(editing.localRoot, "notes", "new.txt"), "unrelated");
await copies.collectStopped({ companyId, runId: editing.runId });
expect((await copies.collectStopped({ companyId, runId: deleting.runId }))?.state).toBe("saved");
await expect(fs.stat(path.join(root, "notes", "old.txt"))).rejects.toMatchObject({ code: "ENOENT" });
expect(await fs.readFile(path.join(root, "notes", "new.txt"), "utf8")).toBe("unrelated");
const remove = await run(), change = await run();
await fs.unlink(path.join(remove.localRoot, "notes", "new.txt"));
await fs.writeFile(path.join(change.localRoot, "notes", "new.txt"), "restored by later edit");
await copies.collectStopped({ companyId, runId: remove.runId });
await copies.collectStopped({ companyId, runId: change.runId });
expect(await fs.readFile(path.join(root, "notes", "new.txt"), "utf8")).toBe("restored by later edit");
});
it("handles a concurrently replaced parent and keeps the later file change", async () => {
await fs.mkdir(path.join(root, "notes"));
await fs.writeFile(path.join(root, "notes", "fact.txt"), "old");
const replacing = await run(), editing = await run();
await fs.rm(path.join(replacing.localRoot, "notes"), { recursive: true });
await fs.writeFile(path.join(replacing.localRoot, "notes"), "now a file");
await fs.writeFile(path.join(editing.localRoot, "notes", "fact.txt"), "new fact");
await copies.collectStopped({ companyId, runId: replacing.runId });
expect(await fs.readFile(path.join(root, "notes"), "utf8")).toBe("now a file");
expect((await copies.collectStopped({ companyId, runId: editing.runId }))?.state).toBe("saved");
expect(await fs.readFile(path.join(root, "notes", "fact.txt"), "utf8")).toBe("new fact");
});
it("cleans stopped copies after a crash during cleanup without touching active copies", async () => {
const finished = await run(), active = await run();
await db.update(agentInstructionWorkingCopies).set({ state: "saved", processStoppedAt: new Date() }).where(eq(agentInstructionWorkingCopies.runId, finished.runId));
copies = agentInstructionWorkingCopyService(db);
await copies.recoverCaptured();
await expect(fs.stat(finished.localRoot)).rejects.toMatchObject({ code: "ENOENT" });
expect((await fs.stat(active.localRoot)).isDirectory()).toBe(true);
});
it("cleans up a staging failure before any provider starts", async () => {
const runId = randomUUID();
await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, invocationSource: "on_demand", responsibleUserId: userId });
const shell = vi.spyOn(executionTargetTools, "runAdapterExecutionTargetShellCommand").mockResolvedValue({ exitCode: 1, signal: null, timedOut: false, stdout: "", stderr: "fixture failure" });
try {
await expect(copies.prepare({ ...target(), runId, cwd: home, target: { kind: "remote", transport: "ssh", environmentId: randomUUID(), remoteCwd: "/fixture/task",
spec: { host: "unused.invalid", port: 22, username: "test", remoteCwd: "/fixture/task" } } })).rejects.toThrow("Could not exclude");
const row = (await copies.get(companyId, runId))!;
expect(row.state).toBe("unavailable");
expect(row.receipt?.baseline).toBeUndefined();
await expect(fs.stat(path.dirname(row.localRoot))).rejects.toMatchObject({ code: "ENOENT" });
} finally { shell.mockRestore(); }
});
it.each(["succeeded", "failed", "cancelled", "timed_out", "interrupted"])("registers staging ownership before copying and cleans preparation for a %s run after restart", async (status) => {
const original = fs.cp.bind(fs);
const copy = vi.spyOn(fs, "cp").mockImplementationOnce(async (source, destination, options) => {
const rows = await db.select().from(agentInstructionWorkingCopies).where(eq(agentInstructionWorkingCopies.agentId, agentId));
expect(rows).toHaveLength(1);
expect(rows[0]).toMatchObject({ state: "preparing", localRoot: destination });
return original(source, destination, options);
});
const interrupted = await run();
copy.mockRestore();
const active = await run();
for (const row of [interrupted, active]) {
await db.update(agentInstructionWorkingCopies).set({ state: "preparing", baseHash: "preparing", receipt: { schema: "paperclip.agent-files.v1" } }).where(eq(agentInstructionWorkingCopies.runId, row.runId));
}
await db.update(heartbeatRuns).set({ status }).where(eq(heartbeatRuns.id, interrupted.runId));
copies = agentInstructionWorkingCopyService(db);
await copies.recoverStopped();
expect((await copies.get(companyId, interrupted.runId))?.state).toBe("unavailable");
await expect(fs.stat(path.dirname(interrupted.localRoot))).rejects.toMatchObject({ code: "ENOENT" });
expect((await fs.stat(active.localRoot)).isDirectory()).toBe(true);
});
it("retains remote cleanup across restart and failed retries until the original lease is cleaned", async () => {
const copy = await run();
const environmentId = randomUUID(), leaseId = randomUUID(), remoteCwd = "/fixture/task";
const executionRoot = path.posix.join(remoteCwd, ".paperclip-runtime", "agent-files", agentId, copy.runId);
await db.insert(environments).values({ id: environmentId, name: environmentId, driver: "sandbox" });
await db.insert(environmentLeases).values({ id: leaseId, companyId, environmentId, heartbeatRunId: copy.runId, provider: "daytona", providerLeaseId: "original-sandbox" });
await db.update(agentInstructionWorkingCopies).set({ state: "saved", processStoppedAt: new Date(), location: `remote:${environmentId}`, executionRoot,
receipt: { ...copy.receipt, cleanup: { leaseId, remoteCwd } } }).where(eq(agentInstructionWorkingCopies.runId, copy.runId));
copies = agentInstructionWorkingCopyService(db);
await copies.recoverCaptured();
const pending = (await copies.get(companyId, copy.runId))!;
expect(pending.receipt).toMatchObject({ cleanupPending: true, cleanup: { leaseId, remoteCwd } });
expect(pending.receipt?.baseline).toBeUndefined();
await expect(fs.stat(copy.localRoot)).rejects.toMatchObject({ code: "ENOENT" });
const execute = vi.fn().mockRejectedValueOnce(new Error("provider temporarily unavailable")).mockResolvedValue({ exitCode: 0, stdout: "", stderr: "" });
copies = agentInstructionWorkingCopyService(db, { environmentRuntime: { execute } as unknown as EnvironmentRuntimeService });
for (const expectedPending of [true, false]) {
await db.update(agentInstructionWorkingCopies).set({ nextAttemptAt: null }).where(eq(agentInstructionWorkingCopies.runId, copy.runId));
await copies.recoverCaptured();
expect((await copies.get(companyId, copy.runId))?.receipt?.cleanupPending).toBe(expectedPending);
}
expect(execute).toHaveBeenCalledTimes(2);
expect(execute).toHaveBeenLastCalledWith(expect.objectContaining({ lease: expect.objectContaining({ id: leaseId, providerLeaseId: "original-sandbox" }),
command: "rm", args: ["-rf", "--", executionRoot], bypassSession: true }));
await copies.recoverCaptured();
expect(execute).toHaveBeenCalledTimes(2);
await copies.release(companyId, copy.runId);
expect((await copies.get(companyId, copy.runId))?.receipt?.cleanupPending).toBe(false);
expect(execute).toHaveBeenCalledTimes(2);
});
it("finishes remote cleanup from a destruction receipt after the environment is deleted", async () => {
const copy = await run();
const environmentId = randomUUID(), leaseId = randomUUID(), remoteCwd = "/fixture/task";
const lease = { id: leaseId, companyId, environmentId, heartbeatRunId: copy.runId, provider: "daytona", providerLeaseId: "destroyed-sandbox" };
await db.insert(environments).values({ id: environmentId, name: environmentId, driver: "sandbox" });
await db.insert(environmentLeases).values({ ...lease, status: "released", releasedAt: new Date(), cleanupStatus: "success",
metadata: { remoteExecutionTermination: remoteTerminationReceipt(lease, { providerLeaseId: lease.providerLeaseId, state: "destroyed" }) } });
await db.update(agentInstructionWorkingCopies).set({ state: "saved", processStoppedAt: new Date(), location: `remote:${environmentId}`,
executionRoot: path.posix.join(remoteCwd, ".paperclip-runtime", "agent-files", agentId, copy.runId),
receipt: { ...copy.receipt, cleanupPending: true, cleanup: { leaseId, remoteCwd } } }).where(eq(agentInstructionWorkingCopies.runId, copy.runId));
await db.delete(environments).where(eq(environments.id, environmentId));
copies = agentInstructionWorkingCopyService(db);
await copies.recoverCaptured();
expect((await copies.get(companyId, copy.runId))?.receipt?.cleanupPending).toBe(false);
await expect(fs.stat(copy.localRoot)).rejects.toMatchObject({ code: "ENOENT" });
});
it.each([false, true])("rejects symlinks without saving any part of the tree, preserving an existing full-storage warning: %s", async (full) => {
if (full) await sparseFile(path.join(root, "full.bin"), MAX_AGENT_FILE_BYTES);
const copy = await run();
await fs.writeFile(path.join(copy.localRoot, "innocent.txt"), "changed");
await fs.symlink(path.join(home, "outside"), path.join(copy.localRoot, "escape"));
const result = await copies.collectStopped({ companyId, runId: copy.runId });
expect(result?.state).toBe("unavailable");
expect(result?.errorCode).toBe("AGENT_FILES_SAVE_FAILED");
expect(result?.receipt?.storageWarning).toBe(copy.receipt?.storageWarning);
if (full) expect(result?.receipt?.storageWarning).toContain("Agent storage is full");
await expect(fs.stat(copy.localRoot)).rejects.toMatchObject({ code: "ENOENT" });
await expect(fs.stat(path.join(root, "innocent.txt"))).rejects.toMatchObject({ code: "ENOENT" });
}, 30_000);
it("adopts the last deployed head once and bridges an old collector without appending history", async () => {
const id = randomUUID();
const content = "last deployed revision";
await db.insert(agentInstructionRevisions).values({ id, ...target(), entryFile, contentBase64: Buffer.from(content).toString("base64"), contentHash: fileHash(Buffer.from(content)), byteLength: content.length, source: "board" });
await db.insert(agentInstructionHeads).values({ ...target(), entryFile, revisionId: id });
const current = await revisions.readCurrent(target(), board());
expect(current?.content).toBe(content);
await revisions.commit({ ...target(), entryFile, content: "new directory contents", baseRevisionId: id, source: "cleanup" }, board());
await revisions.materializeCurrent(target());
expect(await fs.readFile(path.join(root, entryFile), "utf8")).toBe("new directory contents");
await expect(revisions.commit({ ...target(), entryFile, content: "stale legacy run", baseRevisionId: id, source: "cleanup" }, board())).rejects.toMatchObject({ status: 409 });
expect(await db.select().from(agentInstructionRevisions).where(eq(agentInstructionRevisions.agentId, agentId))).toHaveLength(1);
});
it("ordinary file edits leave the loaded instruction digest unchanged", async () => {
const make = async () => {
const copy = await run();
const context = await buildNativeRuntimeContext({ db, agent: { id: agentId, companyId, name: "Target", adapterConfig: { instructionsBundleMode: "managed", instructionsRootPath: root, instructionsEntryFile: entryFile } },
runId: copy.runId, runtimeConfig: {}, runtimeSkillEntries: [], instructionWorkingCopy: { rootPath: copy.executionRoot, entryPath: entryFile, kind: "agent_files" } });
return { copy, context };
};
const before = await make();
await fs.writeFile(path.join(before.copy.localRoot, "notes.txt"), "new personal knowledge");
await copies.collectStopped({ companyId, runId: before.copy.runId });
const after = await make();
expect(after.context.aggregateDigest).toBe(before.context.aggregateDigest);
expect(after.context.instructions.bundle.fileCount).toBe(1);
});
it("uses the workspace transport to restore after destruction of the remote filesystem", async () => {
const remoteCwd = path.join(home, "remote-task");
await fs.mkdir(remoteCwd, { recursive: true });
await execFile("git", ["init", remoteCwd]);
const runner: import("@paperclipai/adapter-utils/command-managed-runtime").CommandManagedRuntimeRunner = {
execute: async input => {
const startedAt = new Date().toISOString();
const env = { ...process.env, ...input.env };
const args = [...(input.args ?? [])];
if (input.stdin != null && (args[0] === "-c" || args[0] === "-lc")) {
env.PAPERCLIP_TEST_STDIN = input.stdin;
args[1] = `printf '%s' "$PAPERCLIP_TEST_STDIN" | (${args[1]})`;
}
try {
const result = await execFile(input.command, args, { cwd: input.cwd, env, timeout: input.timeoutMs, maxBuffer: 32 * 1024 * 1024 });
return { exitCode: 0, signal: null, timedOut: false, stdout: result.stdout, stderr: result.stderr, pid: null, startedAt };
} catch (error) {
const e = error as { code?: number; signal?: NodeJS.Signals; stdout?: string; stderr?: string };
return { exitCode: typeof e.code === "number" ? e.code : 1, signal: e.signal ?? null, timedOut: false, stdout: e.stdout ?? "", stderr: e.stderr ?? "", pid: null, startedAt };
}
},
};
const executionTarget = { kind: "remote" as const, transport: "sandbox" as const, environmentId: randomUUID(), remoteCwd, runner };
const prepare = async () => {
const runId = randomUUID();
await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, invocationSource: "on_demand", responsibleUserId: userId });
return (await copies.prepare({ ...target(), runId, cwd: home, target: executionTarget }))!;
};
await fs.mkdir(path.join(root, "build"));
await fs.writeFile(path.join(root, "build", "personal.txt"), "cache-like names are still agent files");
const first = await prepare();
expect(await fs.readFile(path.join(first.executionRoot, "build", "personal.txt"), "utf8")).toBe("cache-like names are still agent files");
await fs.mkdir(path.join(first.executionRoot, "notes"));
await fs.writeFile(path.join(first.executionRoot, "notes", "bytes.bin"), Buffer.from([0, 128, 255]));
await fs.mkdir(path.join(first.executionRoot, "node_modules"));
await fs.writeFile(path.join(first.executionRoot, "node_modules", "personal.txt"), "retain this too");
await fs.writeFile(path.join(remoteCwd, "task-only.txt"), "task");
expect((await execFile("git", ["-C", remoteCwd, "status", "--porcelain", "--untracked-files=all"])).stdout).toBe("?? task-only.txt\n");
expect((await copies.collectStopped({ companyId, runId: first.runId, target: executionTarget }))?.state).toBe("saved");
await copies.release(companyId, first.runId);
expect((await copies.get(companyId, first.runId))?.receipt?.baseline).toBeUndefined();
await expect(fs.stat(first.localRoot)).rejects.toMatchObject({ code: "ENOENT" });
await fs.rm(remoteCwd, { recursive: true });
await fs.mkdir(remoteCwd);
copies = agentInstructionWorkingCopyService(db);
const second = await prepare();
expect(await fs.readFile(path.join(second.executionRoot, "notes", "bytes.bin"))).toEqual(Buffer.from([0,128,255]));
expect(await fs.readFile(path.join(second.executionRoot, "node_modules", "personal.txt"), "utf8")).toBe("retain this too");
await expect(fs.stat(path.join(second.executionRoot, "task-only.txt"))).rejects.toMatchObject({ code: "ENOENT" });
await copies.collectStopped({ companyId, runId: second.runId, target: executionTarget });
await copies.release(companyId, second.runId);
});
it("stages SSH agent files at the registered root without a nested task workspace", async () => {
const remoteCwd = path.join(home, "ssh-task");
const exclude = vi.spyOn(executionTargetTools, "runAdapterExecutionTargetShellCommand").mockResolvedValue({ exitCode: 0, signal: null, timedOut: false, stdout: "", stderr: "" });
const stage = vi.spyOn(ssh, "syncDirectoryToSsh").mockImplementation(async input => {
await fs.mkdir(path.dirname(input.remoteDir), { recursive: true });
await fs.cp(input.localDir, input.remoteDir, { recursive: true });
});
const restore = vi.spyOn(ssh, "restoreWorkspaceFromSshExecution").mockImplementation(async input => {
await fs.cp(input.remoteDir!, input.localDir, { recursive: true });
});
try {
const runId = randomUUID();
await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, invocationSource: "on_demand", responsibleUserId: userId });
const target = { kind: "remote" as const, transport: "ssh" as const, environmentId: randomUUID(), remoteCwd,
spec: { host: "unused.invalid", port: 22, username: "test", remoteCwd } };
const copy = (await copies.prepare({ companyId, agentId, runId, cwd: home, target }))!;
expect(stage).toHaveBeenCalledWith(expect.objectContaining({ remoteDir: copy.executionRoot }));
expect(await fs.readFile(path.join(copy.executionRoot, entryFile), "utf8")).toBe(initial);
await fs.writeFile(path.join(copy.executionRoot, "ssh-note.txt"), "persistent SSH file");
expect((await copies.collectStopped({ companyId, runId, target }))?.state).toBe("saved");
expect(restore).toHaveBeenCalledWith(expect.objectContaining({ remoteDir: copy.executionRoot, restoreGitHistory: false }));
expect(await fs.readFile(path.join(root, "ssh-note.txt"), "utf8")).toBe("persistent SSH file");
} finally { stage.mockRestore(); restore.mockRestore(); exclude.mockRestore(); }
});
it("deduplicates concurrent stopped callbacks and discards completed operational snapshots", async () => {
const copy = await run();
await fs.writeFile(path.join(copy.localRoot, "note.txt"), "one write");
const results = await Promise.all([copies.collectStopped({ companyId, runId: copy.runId }), copies.collectStopped({ companyId, runId: copy.runId })]);
expect(results.every(result => result?.state === "saved")).toBe(true);
// Simulate a crash after save but before release. The restart sweeper must
// reclaim operational baselines without touching canonical files.
copies = agentInstructionWorkingCopyService(db);
await copies.recoverCaptured();
const row = await copies.get(companyId, copy.runId);
expect(row?.receipt?.schema).toBe("paperclip.agent-files.v1");
expect(row?.receipt?.baseline).toBeUndefined();
expect(await fs.readFile(path.join(root, "note.txt"), "utf8")).toBe("one write");
});
});
@@ -0,0 +1,87 @@
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { execFile as execFileCallback } from "node:child_process";
import { promisify } from "node:util";
import { afterEach, beforeEach, expect, it } from "vitest";
import { instructionGitExcludeProgram } from "../services/agent-instruction-files.js";
const execFile = promisify(execFileCallback);
let root: string;
beforeEach(async () => { root = await fs.mkdtemp(path.join(os.tmpdir(), "agent-git-exclusion-")); });
afterEach(async () => { await fs.rm(root, { recursive: true, force: true }); });
const git = (cwd: string, ...args: string[]) => execFile("git", ["-C", cwd, ...args]);
const exclude = (cwd: string) => execFile(process.execPath, ["-e", instructionGitExcludeProgram, cwd]);
async function assertPrivateFilesIgnored(cwd: string) {
const runtime = path.join(cwd, ".paperclip-runtime");
await fs.writeFile(path.join(runtime, "private.md"), "agent instructions");
await git(cwd, "add", "-A");
expect((await git(cwd, "diff", "--cached", "--name-only")).stdout).not.toContain(".paperclip-runtime");
expect((await git(cwd, "check-ignore", "--", ".paperclip-runtime/private.md", ".paperclip-runtime/.gitignore")).stdout.trim().split("\n")).toHaveLength(2);
}
it("excludes runtime files without changing tracked ignore rules or Git metadata", async () => {
await git(root, "init");
await fs.writeFile(path.join(root, ".gitignore"), "keep-me\n");
const metadata = path.join(root, ".git", "info", "exclude");
const before = await fs.readFile(metadata, "utf8");
await exclude(root);
await exclude(root);
await assertPrivateFilesIgnored(root);
expect(await fs.readFile(path.join(root, ".gitignore"), "utf8")).toBe("keep-me\n");
expect(await fs.readFile(metadata, "utf8")).toBe(before);
});
it("never writes to an external gitdir selected by task-controlled metadata", async () => {
const workspace = path.join(root, "workspace"), outside = path.join(root, "outside.git");
await fs.mkdir(workspace);
await git(workspace, "init", `--separate-git-dir=${outside}`);
const metadata = path.join(outside, "info", "exclude");
const before = await fs.readFile(metadata, "utf8");
await exclude(workspace);
await assertPrivateFilesIgnored(workspace);
expect(await fs.readFile(metadata, "utf8")).toBe(before);
});
it("supports linked worktrees and nested task directories without changing the common gitdir", async () => {
const repository = path.join(root, "repository"), worktree = path.join(root, "worktree");
await fs.mkdir(repository);
await git(repository, "init");
await git(repository, "-c", "user.name=Test", "-c", "user.email=test@example.test", "commit", "--allow-empty", "-m", "initial");
await git(repository, "worktree", "add", "--detach", worktree);
const nested = path.join(worktree, "nested");
await fs.mkdir(nested);
const metadata = path.join(repository, ".git", "info", "exclude");
const before = await fs.readFile(metadata, "utf8");
await exclude(nested);
await assertPrivateFilesIgnored(nested);
expect(await fs.readFile(metadata, "utf8")).toBe(before);
});
it.each(["directory", "file"])("rejects a symlink at the runtime exclusion %s", async (kind) => {
const workspace = path.join(root, "workspace"), outside = path.join(root, "outside");
await fs.mkdir(workspace);
await fs.mkdir(outside);
const externalFile = path.join(outside, ".gitignore");
await fs.writeFile(externalFile, "untouched\n");
const runtime = path.join(workspace, ".paperclip-runtime");
if (kind === "directory") await fs.symlink(outside, runtime);
else {
await fs.mkdir(runtime);
await fs.symlink(externalFile, path.join(runtime, ".gitignore"));
}
await expect(exclude(workspace)).rejects.toThrow("Unsafe runtime exclusion");
expect(await fs.readFile(externalFile, "utf8")).toBe("untouched\n");
});
it("replaces a linked ignore file without writing through its external inode", async () => {
const externalFile = path.join(root, "outside-ignore"), workspace = path.join(root, "workspace");
const runtime = path.join(workspace, ".paperclip-runtime");
await fs.mkdir(runtime, { recursive: true });
await fs.writeFile(externalFile, "untouched\n");
await fs.link(externalFile, path.join(runtime, ".gitignore"));
await exclude(workspace);
expect(await fs.readFile(externalFile, "utf8")).toBe("untouched\n");
expect(await fs.readFile(path.join(runtime, ".gitignore"), "utf8")).toBe("*\n");
});
@@ -0,0 +1,247 @@
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { randomUUID } from "node:crypto";
import { beforeAll, afterAll, beforeEach, describe, expect, it, vi } from "vitest";
import { and, eq, sql } from "drizzle-orm";
import { agents, agentApiKeys, companies, authUsers, companyMemberships, principalPermissionGrants, heartbeatRuns,
agentInstructionRevisions, agentInstructionHeads, issueThreadInteractions, issues, createDb } from "@paperclipai/db";
import { startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
import { agentInstructionRevisionService } from "../services/agent-instruction-revisions.js";
import { agentInstructionWorkingCopyService } from "../services/agent-instruction-working-copies.js";
import { instructionBytes, instructionPath, materializeInstructionBytes, readInstructionBytes } from "../services/agent-instruction-files.js";
import { agentInstructionsService, resolveManagedInstructionsRoot } from "../services/agent-instructions.js";
import type { AuthorizationActor } from "../services/authorization.js";
import { upsertAgentInstructionsFileSchema } from "@paperclipai/shared";
describe("canonical instruction revisions", () => {
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
let db: ReturnType<typeof createDb>;
let service: ReturnType<typeof agentInstructionRevisionService>;
const previousHome = process.env.PAPERCLIP_HOME;
let home: string;
let companyId: string, agentId: string, userId: string, actorId: string, runId: string, root: string;
let actor: AuthorizationActor;
const entryFile = "policy/INSTRUCTIONS.txt";
const initial = "\uFEFF# Original\r\n☃\0\n";
beforeAll(async () => {
home = await fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), "instruction-revisions-")));
process.env.PAPERCLIP_HOME = home;
database = await startEmbeddedPostgresTestDatabase("instruction-revisions-db-");
db = createDb(database.connectionString);
service = agentInstructionRevisionService(db);
}, 90_000);
afterAll(async () => {
vi.restoreAllMocks();
if (previousHome === undefined) delete process.env.PAPERCLIP_HOME; else process.env.PAPERCLIP_HOME = previousHome;
await database?.cleanup();
if (home) await fs.rm(home, { recursive: true, force: true });
});
beforeEach(async () => {
companyId = randomUUID(); agentId = randomUUID(); actorId = randomUUID(); userId = randomUUID(); runId = randomUUID();
await db.insert(companies).values({ id: companyId, name: "Instruction tests", issuePrefix: randomUUID().slice(0, 8) });
await db.insert(authUsers).values({ id: userId, name: "Editor", email: `${userId}@example.test`, createdAt: new Date(), updatedAt: new Date() });
root = resolveManagedInstructionsRoot({ id: agentId, companyId, name: "Target", adapterConfig: {} });
await db.insert(agents).values([
{ id: agentId, companyId, name: "Target", adapterConfig: { instructionsBundleMode: "managed", instructionsRootPath: root, instructionsEntryFile: entryFile } },
{ id: actorId, companyId, name: "Writer" },
]);
await db.insert(companyMemberships).values([
{ companyId, principalType: "user", principalId: userId, membershipRole: "operator" },
{ companyId, principalType: "agent", principalId: actorId, membershipRole: "member" },
]);
await db.insert(principalPermissionGrants).values([
{ companyId, principalType: "user", principalId: userId, permissionKey: "agents:configure", scope: { agentIds: [agentId] } },
{ companyId, principalType: "agent", principalId: actorId, permissionKey: "agents:configure", scope: { agentIds: [agentId] } },
]);
await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId: actorId, invocationSource: "on_demand", responsibleUserId: userId });
actor = { type: "agent", companyId, agentId: actorId, runId, source: "agent_jwt" };
await fs.mkdir(path.dirname(path.join(root, entryFile)), { recursive: true });
await fs.writeFile(path.join(root, entryFile), initial);
});
const target = () => ({ companyId, agentId });
async function base() { return (await service.readCurrent(target(), actor))!; }
async function save(content: string, baseRevisionId: string | null) {
return service.commit({ ...target(), entryFile, content, baseRevisionId, source: "cleanup" }, actor);
}
it.each(["current", "history", "revision", "diff", "candidates"])("denies peer %s reads without a configuration grant or responsible-user target access", async (operation) => {
const first = await base();
await db.delete(principalPermissionGrants).where(eq(principalPermissionGrants.principalId, userId));
const read = () => {
if (operation === "current") return service.readCurrent(target(), actor);
if (operation === "history") return service.history({ ...target(), entryFile }, actor);
if (operation === "revision") return service.readRevision({ ...target(), entryFile, revisionId: first.revision.id }, actor);
if (operation === "diff") return service.diff({ ...target(), entryFile, fromRevisionId: first.revision.id, toRevisionId: first.revision.id }, actor);
return agentInstructionWorkingCopyService(db).list(companyId, agentId, actor);
};
await expect(read()).rejects.toMatchObject({ status: 403 });
await db.insert(principalPermissionGrants).values({ companyId, principalType: "user", principalId: userId, permissionKey: "agents:configure", scope: { agentIds: [actorId] } });
await expect(read()).rejects.toMatchObject({ status: 403 });
});
it.each(["current", "history", "revision", "diff", "candidates", "write"])("requires the caller's own target-scoped grant for peer %s access", async (operation) => {
const first = await base();
await db.delete(principalPermissionGrants).where(eq(principalPermissionGrants.principalId, actorId));
const access = () => {
if (operation === "current") return service.readCurrent(target(), actor);
if (operation === "history") return service.history({ ...target(), entryFile }, actor);
if (operation === "revision") return service.readRevision({ ...target(), entryFile, revisionId: first.revision.id }, actor);
if (operation === "diff") return service.diff({ ...target(), entryFile, fromRevisionId: first.revision.id, toRevisionId: first.revision.id }, actor);
if (operation === "write") return save("authorized peer edit", first.revision.id);
return agentInstructionWorkingCopyService(db).list(companyId, agentId, actor);
};
// The responsible user retains permission on the target throughout.
await expect(access()).rejects.toMatchObject({ status: 403 });
await db.insert(principalPermissionGrants).values({ companyId, principalType: "agent", principalId: actorId, permissionKey: "agents:configure", scope: { agentIds: [actorId] } });
await expect(access()).rejects.toMatchObject({ status: 403 });
await db.update(principalPermissionGrants).set({ scope: { agentIds: [agentId] } }).where(eq(principalPermissionGrants.principalId, actorId));
// New saves have no historical revision row; authorized history lookups
// reach the normal not-found response rather than an authorization denial.
if (operation === "revision" || operation === "diff") await expect(access()).rejects.toMatchObject({ status: 404 });
else await expect(access()).resolves.toBeDefined();
});
it("allows self and board reads without requiring instruction edit permission", async () => {
await db.delete(principalPermissionGrants).where(eq(principalPermissionGrants.principalId, userId));
const selfRoot = resolveManagedInstructionsRoot({ id: actorId, companyId, name: "Writer", adapterConfig: {} });
await fs.mkdir(selfRoot, { recursive: true });
await fs.writeFile(path.join(selfRoot, "AGENTS.md"), "Self instructions");
expect((await service.readCurrent({ companyId, agentId: actorId }, actor))?.content).toBe("Self instructions");
expect((await service.readCurrent(target(), { type: "board", userId, source: "session" }))?.content).toBe(initial);
});
it("reads and replaces current bytes using content tokens without storing revisions", async () => {
const first = await base();
expect(first.content).toBe(initial);
const saved = await save("# Changed\r\n", first.revision.id);
expect(saved).toMatchObject({ changed: true, materialization: "current" });
expect(await fs.readFile(path.join(root, entryFile), "utf8")).toBe(saved.content);
expect((await save(saved.content, first.revision.id)).changed).toBe(false);
expect((await service.history({ ...target(), entryFile }, actor)).revisions).toHaveLength(0);
expect(await db.select().from(agentInstructionRevisions).where(eq(agentInstructionRevisions.agentId, agentId))).toHaveLength(0);
});
it("requires a matching initial content token and serializes competing saves", async () => {
await expect(save("without reading", null)).rejects.toMatchObject({ status: 409 });
const first = await base();
const results = await Promise.allSettled([save("one", first.revision.id), save("two", first.revision.id)]);
expect(results.filter(result => result.status === "fulfilled")).toHaveLength(1);
expect(results.find(result => result.status === "rejected")).toMatchObject({ reason: { status: 409 } });
});
it("does not claim a save when atomic file replacement fails", async () => {
const first = await base();
const rename = vi.spyOn(fs, "rename").mockRejectedValueOnce(new Error("interrupted replacement"));
await expect(save("new content", first.revision.id)).rejects.toThrow("interrupted replacement");
rename.mockRestore();
expect(await fs.readFile(path.join(root, entryFile), "utf8")).toBe(initial);
service = agentInstructionRevisionService(db);
await service.materializeCurrent(target());
expect((await base()).content).toBe(initial);
expect((await save("new content", first.revision.id)).content).toBe("new content");
});
it("allows authorized peer edits but rejects missing, forged and revoked responsible identity", async () => {
const first = await base();
await save("peer save", first.revision.id);
await expect(service.commit({ ...target(), entryFile, content: "forged", baseRevisionId: first.revision.id, source: "tool" }, { ...actor, onBehalfOfUserId: "forged" })).rejects.toMatchObject({ status: 403 });
await db.update(heartbeatRuns).set({ responsibleUserId: null }).where(eq(heartbeatRuns.id, runId));
await expect(save("missing", first.revision.id)).rejects.toMatchObject({ status: 403 });
await db.update(heartbeatRuns).set({ responsibleUserId: userId }).where(eq(heartbeatRuns.id, runId));
await db.update(companyMemberships).set({ status: "suspended" }).where(and(eq(companyMemberships.principalId, userId), eq(companyMemberships.companyId, companyId)));
await expect(save("revoked", first.revision.id)).rejects.toMatchObject({ status: 403 });
});
it("rechecks user target scope and agent containment, even with shadow authorization enabled", async () => {
const first = await base();
const previousShadow = process.env.PAPERCLIP_RESPONSIBLE_USER_AUTHZ_SHADOW;
process.env.PAPERCLIP_RESPONSIBLE_USER_AUTHZ_SHADOW = "true";
await db.update(principalPermissionGrants).set({ scope: { agentIds: [actorId] } }).where(eq(principalPermissionGrants.principalId, userId));
await expect(save("denied", first.revision.id)).rejects.toMatchObject({ status: 403 });
await db.update(principalPermissionGrants).set({ scope: null }).where(eq(principalPermissionGrants.principalId, userId));
await db.update(agents).set({ permissions: { trustPreset: "low_trust_review" } }).where(eq(agents.id, actorId));
await expect(save("restricted", first.revision.id)).rejects.toMatchObject({ status: 403 });
if (previousShadow === undefined) delete process.env.PAPERCLIP_RESPONSIBLE_USER_AUTHZ_SHADOW; else process.env.PAPERCLIP_RESPONSIBLE_USER_AUTHZ_SHADOW = previousShadow;
});
it("keeps suggest-only protected-change consent and explicit configure scope restrictions", async () => {
const first = await base();
await db.delete(principalPermissionGrants).where(eq(principalPermissionGrants.principalId, actorId));
await db.insert(principalPermissionGrants).values({ companyId, principalType: "agent", principalId: actorId, permissionKey: "agents:suggest-changes" });
await expect(save("needs approval", first.revision.id)).rejects.toMatchObject({ status: 403 });
await db.delete(principalPermissionGrants).where(eq(principalPermissionGrants.principalId, actorId));
await db.insert(principalPermissionGrants).values({ companyId, principalType: "agent", principalId: actorId, permissionKey: "agents:configure", scope: { agentIds: [actorId] } });
await expect(save("out of scope", first.revision.id)).rejects.toMatchObject({ status: 403 });
});
it("consumes accepted protected-change approval only with a successful CAS commit", async () => {
const first = await base();
const sourceRunId = randomUUID(), issueId = randomUUID(), interactionId = randomUUID();
await db.insert(heartbeatRuns).values({ id: sourceRunId, companyId, agentId: actorId, status: "succeeded" });
await db.insert(issues).values({ id: issueId, companyId, title: "Review instructions" });
await db.delete(principalPermissionGrants).where(eq(principalPermissionGrants.principalId, actorId));
await db.insert(principalPermissionGrants).values({ companyId, principalType: "agent", principalId: actorId, permissionKey: "agents:suggest-changes" });
await db.insert(issueThreadInteractions).values({ id: interactionId, companyId, issueId, kind: "request_confirmation", status: "accepted", sourceRunId, createdByAgentId: actorId,
payload: { version: 1, prompt: "Apply the instruction change?", detailsMarkdown: "```diff\n+approved\n```", target: { type: "custom", key: `agent:${agentId}:instructions`, revisionId: "proposal" } },
result: { version: 1, outcome: "accepted" }, resolvedByUserId: userId, resolvedAt: new Date(),
});
expect((await service.readCurrent(target(), actor))?.revision.id).toBe(first.revision.id);
expect((await db.select().from(issueThreadInteractions).where(eq(issueThreadInteractions.id, interactionId)))[0].result).not.toHaveProperty("consumedAt");
await expect(save("approved", randomUUID())).rejects.toMatchObject({ status: 409 });
expect((await db.select().from(issueThreadInteractions).where(eq(issueThreadInteractions.id, interactionId)))[0].result).not.toHaveProperty("consumedAt");
const receipt = await save("approved", first.revision.id);
expect(receipt.changed).toBe(true);
expect((await db.select().from(issueThreadInteractions).where(eq(issueThreadInteractions.id, interactionId)))[0].result).toMatchObject({ consumedByRunId: runId });
await expect(save("another unapproved change", receipt.revision.id)).rejects.toMatchObject({ status: 403 });
});
it("shares head/CAS and attribution between board and agent writers", async () => {
const first = await base();
const board = { type: "board", userId, source: "session" } as const;
const saved = await service.commit({ ...target(), entryFile, content: "board edit", baseRevisionId: first.revision.id, source: "board" }, board);
expect(saved.revision).toMatchObject({ actorUserId: userId, responsibleUserId: userId, actorAgentId: null, sourceRunId: null, source: "api" });
await expect(save("stale agent edit", first.revision.id)).rejects.toMatchObject({ status: 409 });
});
it("seeds a nested legacy managed entry and never imports a different AGENTS.md", async () => {
await fs.writeFile(path.join(root, "AGENTS.md"), "unrelated");
await db.update(agents).set({ adapterConfig: { instructionsFilePath: path.join(root, entryFile) } }).where(eq(agents.id, agentId));
const first = await base();
expect(first.content).toBe(initial);
expect(first.revision.entryFile).toBe(entryFile);
await db.update(agents).set({ adapterConfig: { instructionsBundleMode: "managed", instructionsRootPath: root, instructionsEntryFile: "missing.md" } }).where(eq(agents.id, agentId));
expect(await service.readCurrent(target(), actor)).toBeNull();
await expect(save("wrong entry", first.revision.id)).rejects.toMatchObject({ status: 409, details: { code: "INSTRUCTION_ENTRY_CHANGED" } });
});
it("rejects cross-company targets, foreign revisions, and revoked API keys", async () => {
const first = await base();
await expect(service.readRevision({ ...target(), entryFile, revisionId: randomUUID() }, actor)).rejects.toMatchObject({ status: 404 });
await expect(service.readCurrent({ companyId: randomUUID(), agentId }, actor)).rejects.toMatchObject({ status: 404 });
const keyId = randomUUID();
await db.insert(agentApiKeys).values({ id: keyId, companyId, agentId: actorId, name: "test", keyHash: randomUUID(), responsibleUserId: userId, revokedAt: new Date() });
await expect(service.commit({ ...target(), entryFile, content: "key", baseRevisionId: first.revision.id, source: "api" }, { ...actor, keyId })).rejects.toMatchObject({ status: 403 });
await expect(db.insert(agentInstructionHeads).values({ companyId: randomUUID(), agentId, entryFile, revisionId: first.revision.id })).rejects.toThrow();
});
it("rejects invalid content, symlinks, traversal and external bundles without changing canonical bytes", async () => {
const first = await base();
expect(() => instructionBytes("\ud800")).toThrow("UTF-8");
expect(() => instructionBytes(new Uint8Array([0xff]))).toThrow("UTF-8");
expect(() => instructionBytes("a".repeat(1024 * 1024 + 1))).toThrow("1 MiB");
for (const value of ["../AGENTS.md", "/etc/passwd", "a/../AGENTS.md", "a\\b", "a//b"]) expect(() => instructionPath(value)).toThrow();
await fs.unlink(path.join(root, entryFile));
await fs.symlink(path.join(home, "outside"), path.join(root, entryFile));
await expect(save("symlink", first.revision.id)).rejects.toMatchObject({ status: 422 });
await fs.unlink(path.join(root, entryFile));
await fs.rm(path.dirname(path.join(root, entryFile)), { recursive: true });
await fs.symlink(home, path.dirname(path.join(root, entryFile)));
await expect(readInstructionBytes(root, entryFile)).rejects.toMatchObject({ status: 422 });
await db.update(agents).set({ adapterConfig: { instructionsBundleMode: "external", instructionsRootPath: home } }).where(eq(agents.id, agentId));
await expect(save("external", first.revision.id)).rejects.toMatchObject({ details: { code: "INSTRUCTION_MANAGED_BUNDLE_REQUIRED" } });
expect((await db.select().from(agentInstructionRevisions).where(eq(agentInstructionRevisions.agentId, agentId)))).toHaveLength(0);
});
it.skipIf(process.platform !== "darwin")("accepts macOS temporary directory aliases while rejecting links within storage", async () => {
const aliasRoot = home.replace(/^\/private\/(var|tmp)\//, "/$1/");
expect(aliasRoot).not.toBe(home);
await materializeInstructionBytes(aliasRoot, "alias/AGENTS.md", Buffer.from("persisted"));
expect(await readInstructionBytes(aliasRoot, "alias/AGENTS.md")).toEqual(Buffer.from("persisted"));
await fs.symlink(path.join(home, "alias"), path.join(home, "linked-agent"));
await expect(readInstructionBytes(path.join(aliasRoot, "linked-agent"), "AGENTS.md"))
.rejects.toMatchObject({ status: 422 });
});
it("requires the content API for the configured entry and rejects forged fields", async () => {
await base();
const [agent] = await db.select().from(agents).where(eq(agents.id, agentId));
await expect(agentInstructionsService(db).writeFile(agent, entryFile, "bypass")).rejects.toMatchObject({ details: { code: "INSTRUCTION_REVISION_REQUIRED" } });
await expect(agentInstructionsService(db).deleteFile(agent, entryFile)).rejects.toMatchObject({ status: 422 });
expect(upsertAgentInstructionsFileSchema.safeParse({ path: entryFile, content: "bad", responsibleUserId: userId }).success).toBe(false);
});
});
@@ -0,0 +1,345 @@
import fs from "node:fs/promises";
import { execFile as execFileCallback } from "node:child_process";
import { promisify } from "node:util";
import * as gitWorkspaceSync from "@paperclipai/adapter-utils/git-workspace-sync";
import { DatabaseSync } from "node:sqlite";
import { captureDirectorySnapshot } from "@paperclipai/adapter-utils/workspace-restore-merge";
const execFile = promisify(execFileCallback);
import os from "node:os";
import path from "node:path";
import { randomUUID } from "node:crypto";
import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
import { eq } from "drizzle-orm";
import { agents, companies, authUsers, companyMemberships, principalPermissionGrants, heartbeatRuns, agentInstructionWorkingCopies, createDb } from "@paperclipai/db";
import { startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
import { agentInstructionRevisionService } from "../services/agent-instruction-revisions.js";
import { agentInstructionWorkingCopyService } from "../services/agent-instruction-working-copies.js";
import { appendHeartbeatRunEvent } from "../services/heartbeat-run-events.js";
import { resolveManagedInstructionsRoot } from "../services/agent-instructions.js";
import { buildNativeRuntimeContext } from "../services/native-runtime/runtime-context.js";
describe("registered run instruction copies", () => {
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
let db: ReturnType<typeof createDb>;
let copies: ReturnType<typeof agentInstructionWorkingCopyService>;
let revisions: ReturnType<typeof agentInstructionRevisionService>;
const previousHome = process.env.PAPERCLIP_HOME;
let home: string;
let companyId: string, agentId: string, userId: string, root: string;
const entryFile = "policy/INSTRUCTIONS.txt";
const initial = "\uFEFF# Original\r\n☃\n";
const target = () => ({ companyId, agentId });
const board = () => ({ type: "board" as const, userId, source: "session" as const });
async function run() {
const runId = randomUUID();
await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, invocationSource: "on_demand", responsibleUserId: userId });
return (await copies.prepare({ legacy: true, ...target(), runId, cwd: home }))!;
}
beforeAll(async () => {
home = await fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), "instruction-working-copies-")));
process.env.PAPERCLIP_HOME = home;
database = await startEmbeddedPostgresTestDatabase("instruction-copies-db-");
db = createDb(database.connectionString);
copies = agentInstructionWorkingCopyService(db);
revisions = agentInstructionRevisionService(db);
}, 90_000);
afterAll(async () => {
if (previousHome === undefined) delete process.env.PAPERCLIP_HOME; else process.env.PAPERCLIP_HOME = previousHome;
await database?.cleanup();
if (home) {
const writable = async (dir: string) => {
await fs.chmod(dir, 0o700);
for (const entry of await fs.readdir(dir, { withFileTypes: true })) if (entry.isDirectory()) await writable(path.join(dir, entry.name));
};
await writable(home);
await fs.rm(home, { recursive: true, force: true });
}
});
beforeEach(async () => {
companyId = randomUUID(); agentId = randomUUID(); userId = randomUUID();
await db.insert(companies).values({ id: companyId, name: "Instruction tests", issuePrefix: randomUUID().slice(0, 8) });
await db.insert(authUsers).values({ id: userId, name: "Editor", email: `${userId}@example.test`, createdAt: new Date(), updatedAt: new Date() });
root = resolveManagedInstructionsRoot({ ...target(), id: agentId, name: "Target", adapterConfig: {} });
await db.insert(agents).values({ id: agentId, companyId, name: "Target", adapterConfig: { instructionsBundleMode: "managed", instructionsRootPath: root, instructionsEntryFile: entryFile } });
await db.insert(companyMemberships).values([
{ companyId, principalType: "user", principalId: userId, membershipRole: "operator" },
{ companyId, principalType: "agent", principalId: agentId, membershipRole: "member" },
]);
await db.insert(principalPermissionGrants).values({ companyId, principalType: "user", principalId: userId, permissionKey: "agents:configure", scope: { agentIds: [agentId] } });
await fs.mkdir(path.dirname(path.join(root, entryFile)), { recursive: true });
await fs.writeFile(path.join(root, entryFile), initial);
});
it("persists an atomic editor replacement, ignores repository instructions, and gives the next run exact bytes", async () => {
const copy = await run();
await fs.writeFile(path.join(home, "AGENTS.md"), "Repository text must not become agent instructions");
const edited = "\uFEFF# Updated\r\nKeep this persistent. ☃\n";
const replacement = path.join(copy.localRoot, "edited.tmp");
await fs.writeFile(replacement, edited);
await fs.rename(replacement, path.join(copy.localRoot, entryFile));
expect(await fs.readFile(path.join(root, entryFile), "utf8")).toBe(initial);
const saved = await copies.collectStopped({ companyId, runId: copy.runId });
expect(saved).toMatchObject({ state: "saved", candidateHash: expect.any(String), processStoppedAt: expect.any(Date) });
expect(await fs.readFile(path.join(root, entryFile), "utf8")).toBe(edited);
copies = agentInstructionWorkingCopyService(db);
expect((await copies.collectStopped({ companyId, runId: copy.runId }))?.receipt).toEqual(saved?.receipt);
expect((await revisions.history({ ...target(), entryFile }, board())).revisions).toHaveLength(0);
const next = await run();
expect(await fs.readFile(path.join(next.localRoot, entryFile), "utf8")).toBe(edited);
});
it.each(["failed", "cancelled", "timed_out"])("retains edits from stopped %s runs without restarting execution", async (status) => {
const copy = await run();
await fs.writeFile(path.join(copy.localRoot, entryFile), status);
await db.update(heartbeatRuns).set({ status, finishedAt: new Date() }).where(eq(heartbeatRuns.id, copy.runId));
expect((await copies.collectStopped({ companyId, runId: copy.runId }))?.state).toBe("saved");
expect((await revisions.readCurrent(target(), board()))?.content).toBe(status);
const [unchangedRun] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, copy.runId));
expect(unchangedRun.status).toBe(status);
});
it.each(["saved", "unchanged"])("refreshes a stopped %s copy from the current canonical revision before retry", async (state) => {
const copy = await run();
if (state === "saved") await fs.writeFile(path.join(copy.localRoot, entryFile), "previous saved turn");
expect((await copies.collectStopped({ companyId, runId: copy.runId }))?.state).toBe(state);
const before = (await revisions.readCurrent(target(), board()))!;
const current = await revisions.commit({ ...target(), entryFile, baseRevisionId: before.revision.id, content: "new board instructions", source: "api" }, board());
const retried = (await copies.prepare({ legacy: true, ...target(), runId: copy.runId, cwd: home }))!;
expect(retried).toMatchObject({ state: "prepared", baseRevisionId: current.revision.id, baseHash: current.revision.contentHash, processStoppedAt: null });
expect(await fs.readFile(path.join(retried.localRoot, entryFile), "utf8")).toBe("new board instructions");
await fs.writeFile(path.join(retried.localRoot, entryFile), "retry adds a change");
expect((await copies.collectStopped({ companyId, runId: copy.runId }))?.state).toBe("saved");
});
it("never rebases uncollected private bytes or a live warm copy onto a newer head", async () => {
const stopped = await run();
await copies.collectStopped({ companyId, runId: stopped.runId });
await fs.writeFile(path.join(stopped.localRoot, entryFile), "uncollected private edit");
const warm = await run();
expect(await copies.hasChanges({ companyId, runId: warm.runId })).toBe(false);
const prior = (await revisions.readCurrent(target(), board()))!;
await revisions.commit({ ...target(), entryFile, baseRevisionId: prior.revision.id, content: "board change", source: "api" }, board());
for (const copy of [stopped, warm]) {
const retried = (await copies.prepare({ legacy: true, ...target(), runId: copy.runId, cwd: home }))!;
expect(retried.baseRevisionId).toBe(copy.baseRevisionId);
}
expect(await fs.readFile(path.join(stopped.localRoot, entryFile), "utf8")).toBe("uncollected private edit");
expect(await fs.readFile(path.join(warm.localRoot, entryFile), "utf8")).toBe(initial);
expect((await copies.collectStopped({ companyId, runId: stopped.runId }))?.state).toBe("conflict");
expect((await revisions.readCurrent(target(), board()))?.content).toBe("board change");
});
it.each(["before replacement", "after replacement"])("preserves the completed receipt when retry staging fails %s", async (point) => {
const copy = await run();
await fs.writeFile(path.join(copy.localRoot, entryFile), "completed turn");
const saved = (await copies.collectStopped({ companyId, runId: copy.runId }))!;
expect(saved.state).toBe("saved");
const prior = (await revisions.readCurrent(target(), board()))!;
const current = await revisions.commit({ ...target(), entryFile, baseRevisionId: prior.revision.id, content: "new board instructions", source: "api" }, board());
const originalMkdir = fs.mkdir.bind(fs);
const originalChmod = fs.chmod.bind(fs);
const mkdir = vi.spyOn(fs, "mkdir").mockImplementation(async (...args: Parameters<typeof fs.mkdir>) => {
if (point === "before replacement" && args[0] === copy.localRoot) throw new Error("retry staging failed");
return originalMkdir(...args);
});
const chmod = vi.spyOn(fs, "chmod").mockImplementation(async (...args) => {
if (point === "after replacement" && args[0] === path.join(copy.localRoot, entryFile)) throw new Error("retry staging failed");
return originalChmod(...args);
});
try {
await expect(copies.prepare({ legacy: true, ...target(), runId: copy.runId, cwd: home })).rejects.toThrow("retry staging failed");
} finally { mkdir.mockRestore(); chmod.mockRestore(); }
expect(await copies.get(companyId, copy.runId)).toEqual(saved);
copies = agentInstructionWorkingCopyService(db);
await copies.recoverStopped();
expect(await copies.get(companyId, copy.runId)).toEqual(saved);
expect((await revisions.readCurrent(target(), board()))?.revision.id).toBe(current.revision.id);
const retried = (await copies.prepare({ legacy: true, ...target(), runId: copy.runId, cwd: home }))!;
expect(retried).toMatchObject({ state: "prepared", baseRevisionId: current.revision.id, processStoppedAt: null });
expect(await fs.readFile(path.join(retried.localRoot, entryFile), "utf8")).toBe(current.content);
});
it("reads an existing canonical runtime snapshot without seeding or changing instruction bytes", async () => {
expect((await revisions.readCommittedForRuntime(target()))?.content).toBe(initial);
const copy = await run();
await fs.writeFile(path.join(copy.localRoot, entryFile), "saved runtime content");
await copies.collectStopped({ companyId, runId: copy.runId });
const current = (await revisions.readCurrent(target(), board()))!;
await fs.writeFile(path.join(root, entryFile), "current filesystem contents");
expect((await revisions.readCommittedForRuntime(target()))?.content).toBe("current filesystem contents");
const [agent] = await db.select().from(agents).where(eq(agents.id, agentId));
const context = await buildNativeRuntimeContext({ db, agent, runId: copy.runId, runtimeConfig: {}, runtimeSkillEntries: [] });
try {
expect(context.instructions.entryPath).toBe(entryFile);
expect(await fs.readFile(path.join(context.instructions.bundle.rootPath, entryFile), "utf8"))
.toBe("current filesystem contents");
} finally {
// Runtime assets are immutable, so make only this fixture's directories
// removable before the ordinary temporary-home cleanup.
const makeRemovable = async (directory: string): Promise<void> => {
await fs.chmod(directory, 0o700);
for (const child of await fs.readdir(directory, { withFileTypes: true })) {
if (child.isDirectory()) await makeRemovable(path.join(directory, child.name));
}
};
await makeRemovable(context.instructions.bundle.rootPath);
}
expect(await fs.readFile(path.join(root, entryFile), "utf8")).toBe("current filesystem contents");
expect((await revisions.history({ ...target(), entryFile }, board())).revisions).toHaveLength(0);
await expect(revisions.readCommittedForRuntime({ companyId: randomUUID(), agentId })).rejects.toThrow("Agent not found");
});
it("preserves a competing stale candidate and requires explicit resolution against the new head", async () => {
const first = await run();
const second = await run();
await fs.writeFile(path.join(first.localRoot, entryFile), "first run");
await fs.writeFile(path.join(second.localRoot, entryFile), "second run");
await copies.collectStopped({ companyId, runId: first.runId });
const conflict = await copies.collectStopped({ companyId, runId: second.runId });
expect(conflict).toMatchObject({ state: "conflict", errorCode: "INSTRUCTION_REVISION_CONFLICT" });
expect(Buffer.from(conflict!.candidateBase64!, "base64").toString("utf8")).toBe("second run");
expect((await revisions.readCurrent(target(), board()))?.content).toBe("first run");
copies = agentInstructionWorkingCopyService(db);
expect(await copies.list(companyId, agentId, board())).toHaveLength(1);
const current = (await revisions.readCurrent(target(), board()))!;
await copies.resolve({ ...target(), runId: second.runId, baseRevisionId: current.revision.id, content: "explicitly combined" }, board());
expect((await revisions.readCurrent(target(), board()))?.content).toBe("explicitly combined");
expect((await copies.collectStopped({ companyId, runId: second.runId }))?.state).toBe("resolved");
const resumed = (await copies.prepare({ legacy: true, ...target(), runId: second.runId, cwd: home }))!;
const resolved = (await revisions.readCurrent(target(), board()))!;
expect(resumed.baseRevisionId).toBe(resolved.revision.id);
expect(await fs.readFile(path.join(resumed.localRoot, entryFile), "utf8")).toBe("explicitly combined");
});
it("rechecks responsible-user permission at collection and preserves denied edits", async () => {
const copy = await run();
await fs.writeFile(path.join(copy.localRoot, entryFile), "candidate after revocation");
await db.update(companyMemberships).set({ status: "suspended" }).where(eq(companyMemberships.principalId, userId));
const denied = await copies.collectStopped({ companyId, runId: copy.runId });
expect(denied).toMatchObject({ state: "conflict", errorCode: "RESPONSIBLE_USER_UNAVAILABLE" });
expect(denied?.candidateBase64).not.toBeNull();
expect(await fs.readFile(path.join(root, entryFile), "utf8")).toBe(initial);
});
it("does not import a symlink or confuse a removed entry with an empty file", async () => {
const linked = await run();
await fs.unlink(path.join(linked.localRoot, entryFile));
await fs.symlink(path.join(home, "AGENTS.md"), path.join(linked.localRoot, entryFile));
expect((await copies.collectStopped({ companyId, runId: linked.runId }))?.state).toBe("pending_collection");
const missing = await run();
await fs.unlink(path.join(missing.localRoot, entryFile));
expect((await copies.collectStopped({ companyId, runId: missing.runId }))?.state).toBe("unavailable");
const empty = await run();
await fs.writeFile(path.join(empty.localRoot, entryFile), "");
expect((await copies.collectStopped({ companyId, runId: empty.runId }))?.state).toBe("saved");
expect((await revisions.readCurrent(target(), board()))?.content).toBe("");
});
it("keeps unchanged warm copies reusable without a content revision", async () => {
const copy = await run();
expect(await copies.hasChanges({ companyId, runId: copy.runId })).toBe(false);
expect((await copies.get(companyId, copy.runId))?.processStoppedAt).toBeNull();
expect((await revisions.history({ ...target(), entryFile }, board())).revisions).toHaveLength(0);
await fs.writeFile(path.join(copy.localRoot, entryFile), "changed after turn");
expect(await copies.hasChanges({ companyId, runId: copy.runId })).toBe(true);
expect((await revisions.readCurrent(target(), board()))?.content).toBe(initial);
});
it("advances an explicit save baseline only when the private copy matches that revision", async () => {
const copy = await run();
const local = path.join(copy.localRoot, entryFile);
await fs.writeFile(local, "explicit self-save");
const first = await revisions.commit({ ...target(), entryFile, baseRevisionId: copy.baseRevisionId, content: "explicit self-save", source: "tool" }, board());
await copies.acknowledgeExplicitSave({ ...target(), runId: copy.runId, entryFile, revisionId: first.revision.id, contentHash: first.revision.contentHash });
await fs.writeFile(local, "local edit after explicit save");
expect((await copies.collectStopped({ companyId, runId: copy.runId }))?.state).toBe("saved");
expect((await revisions.readCurrent(target(), board()))?.content).toBe("local edit after explicit save");
const competing = await run();
await fs.writeFile(path.join(competing.localRoot, entryFile), "older local draft");
const separate = await revisions.commit({ ...target(), entryFile, baseRevisionId: competing.baseRevisionId, content: "different explicit content", source: "tool" }, board());
await copies.acknowledgeExplicitSave({ ...target(), runId: competing.runId, entryFile, revisionId: separate.revision.id, contentHash: separate.revision.contentHash });
expect((await copies.collectStopped({ companyId, runId: competing.runId }))?.state).toBe("conflict");
expect((await revisions.readCurrent(target(), board()))?.content).toBe("different explicit content");
expect((await copies.list(companyId, agentId, board()))[0]).not.toHaveProperty("localRoot");
});
it("recovers captured bytes after restart and deduplicates concurrent cleanup callbacks", async () => {
const copy = await run();
await db.update(agentInstructionWorkingCopies).set({ state: "pending_commit", candidateBase64: Buffer.from("captured before restart").toString("base64"), processStoppedAt: new Date() })
.where(eq(agentInstructionWorkingCopies.runId, copy.runId));
copies = agentInstructionWorkingCopyService(db);
await copies.recoverCaptured();
expect((await revisions.readCurrent(target(), board()))?.content).toBe("captured before restart");
const duplicate = await run();
await fs.writeFile(path.join(duplicate.localRoot, entryFile), "one final edit");
await Promise.all([copies.collectStopped({ companyId, runId: duplicate.runId }), copies.collectStopped({ companyId, runId: duplicate.runId })]);
expect((await copies.get(companyId, duplicate.runId))?.state).toBe("saved");
expect((await revisions.history({ ...target(), entryFile }, board())).revisions).toHaveLength(0);
});
it("requires durable stop evidence before recovering an uncaptured copy", async () => {
const copy = await run();
await fs.writeFile(path.join(copy.localRoot, entryFile), "edit before controller restart");
await db.update(heartbeatRuns).set({ runtimeMode: "native", status: "failed", finishedAt: new Date() }).where(eq(heartbeatRuns.id, copy.runId));
copies = agentInstructionWorkingCopyService(db);
await copies.recoverStopped();
expect((await revisions.readCurrent(target(), board()))?.content).toBe(initial);
expect((await copies.list(companyId, agentId, board()))[0]).toMatchObject({ state: "pending_collection", content: null });
await appendHeartbeatRunEvent(db, { ...target(), runId: copy.runId, eventType: "native.local_process_stopped", stream: "system" });
await copies.recoverStopped();
expect((await revisions.readCurrent(target(), board()))?.content).toBe("edit before controller restart");
});
it("never treats a superseded stop receipt or unavailable environment as a save", async () => {
const copy = await run();
await fs.writeFile(path.join(copy.localRoot, entryFile), "unverified live edit");
await db.update(heartbeatRuns).set({ runtimeMode: "native", status: "failed", finishedAt: new Date() }).where(eq(heartbeatRuns.id, copy.runId));
await appendHeartbeatRunEvent(db, { ...target(), runId: copy.runId, eventType: "native.local_process_stopped", stream: "system" });
await appendHeartbeatRunEvent(db, { ...target(), runId: copy.runId, eventType: "native.process_start_requested", stream: "system" });
await copies.recoverStopped();
expect((await revisions.readCurrent(target(), board()))?.content).toBe(initial);
const lost = await copies.reportUnavailable(companyId, copy.runId);
expect(lost).toMatchObject({ state: "unavailable", candidateBase64: null, processStoppedAt: null });
expect(lost?.errorMessage).not.toContain(copy.localRoot);
});
it("keeps private run copies out of normal Git staging and workspace snapshots", async () => {
const workspace = path.join(home, `workspace-${randomUUID()}`);
await fs.mkdir(workspace);
await execFile("git", ["init", workspace]);
await execFile("git", ["-C", workspace, "-c", "user.name=Fixture", "-c", "user.email=fixture@example.test", "commit", "--allow-empty", "-m", "fixture"]);
const runId = randomUUID();
await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, invocationSource: "on_demand", responsibleUserId: userId });
const copy = (await copies.prepare({ legacy: true, ...target(), runId, cwd: workspace }))!;
expect(copy.localRoot).toBe(path.join(workspace, ".paperclip-runtime", `instruction-edits-${runId}`, "instructions"));
await fs.writeFile(path.join(workspace, "deliverable.txt"), "public work");
await execFile("git", ["-C", workspace, "add", "."]);
const staged = await execFile("git", ["-C", workspace, "diff", "--cached", "--name-only"]);
expect(staged.stdout).toBe("deliverable.txt\n");
const snapshot = await gitWorkspaceSync.readGitWorkspaceSnapshot(workspace);
try {
const paths: unknown = snapshot?.overlayPaths;
if (Array.isArray(paths)) {
expect(paths).toEqual(["deliverable.txt"]);
} else {
// The streaming Git snapshot stores the same path set in a private
// manifest. Inspect its actual records, not only the reported count.
expect(paths).toMatchObject({ kind: "path_manifest", version: 1, category: "overlay", count: 1 });
const manifest = paths as { filePath: string; category: string };
const manifestDb = new DatabaseSync(manifest.filePath, { readOnly: true, allowExtension: false });
try {
const rows = manifestDb.prepare("SELECT path FROM records WHERE category = ? ORDER BY path").all(manifest.category);
expect(rows.map((row) => row.path)).toEqual(["deliverable.txt"]);
} finally { manifestDb.close(); }
}
} finally {
if ("disposeGitWorkspaceSnapshot" in gitWorkspaceSync && typeof gitWorkspaceSync.disposeGitWorkspaceSnapshot === "function") {
await gitWorkspaceSync.disposeGitWorkspaceSnapshot(snapshot);
}
}
const files = await captureDirectorySnapshot(workspace, { exclude: [".git", ".paperclip-runtime"] });
expect([...files.entries].map(([relative]) => relative)).toEqual(["deliverable.txt"]);
await expect(copies.prepare({ legacy: true, ...target(), runId, cwd: home })).rejects.toThrow("different run workspace");
});
});
@@ -1,5 +1,6 @@
import express from "express";
import request from "supertest";
import { Readable } from "node:stream";
import { beforeEach, describe, expect, it, vi } from "vitest";
const mockAgentService = vi.hoisted(() => ({
@@ -24,6 +25,21 @@ const mockAgentInstructionsService = vi.hoisted(() => ({
materializeManagedBundle: vi.fn(),
}));
const mockInstructionWorkingCopies = vi.hoisted(() => ({ list: vi.fn(), resolve: vi.fn(), acknowledgeExplicitSave: vi.fn() }));
vi.mock("../services/agent-instruction-working-copies.js", () => ({ agentInstructionWorkingCopyService: () => mockInstructionWorkingCopies }));
const mockDownloadAgentFile = vi.hoisted(() => vi.fn());
vi.mock("../services/agent-file-store.js", async (importOriginal) => {
const actual = await importOriginal<typeof import("../services/agent-file-store.js")>();
return { ...actual, agentFileStore: (...args: Parameters<typeof actual.agentFileStore>) => ({ ...actual.agentFileStore(...args), download: mockDownloadAgentFile }) };
});
const mockInstructionRevisions = vi.hoisted(() => ({ readCurrent: vi.fn(), commit: vi.fn(), restore: vi.fn(), history: vi.fn(), readRevision: vi.fn(), diff: vi.fn(), materializeCurrent: vi.fn() }));
vi.mock("../services/agent-instruction-revisions.js", () => ({ agentInstructionRevisionService: () => mockInstructionRevisions }));
const mockAuthorizeInstructionRead = vi.hoisted(() => vi.fn());
vi.mock("../services/agent-instruction-authorization.js", () => ({ authorizeInstructionRead: mockAuthorizeInstructionRead }));
const mockAccessService = vi.hoisted(() => ({
canUser: vi.fn(),
decide: vi.fn(),
@@ -202,6 +218,9 @@ describe("agent instructions bundle routes", () => {
vi.doUnmock("../middleware/index.js");
registerModuleMocks();
vi.clearAllMocks();
mockAuthorizeInstructionRead.mockImplementation(async (_db, actor) => actor);
mockInstructionRevisions.readCurrent.mockResolvedValue(null);
mockInstructionRevisions.commit.mockResolvedValue({ revision: { id: "33333333-3333-4333-8333-333333333333", entryFile: "AGENTS.md", byteLength: 18 }, content: "# Updated Agent\n", changed: true, materialization: "current" });
mockBuiltInAgentService.ensureCompanyDefaultAgentGrants.mockResolvedValue(0);
mockSyncInstructionsBundleConfigFromFilePath.mockImplementation((_agent, config) => config);
mockFindServerAdapter.mockImplementation((_type: string) => ({ type: _type }));
@@ -286,6 +305,8 @@ describe("agent instructions bundle routes", () => {
entryFile: "AGENTS.md",
});
expect(mockAgentInstructionsService.getBundle).toHaveBeenCalled();
expect(mockAuthorizeInstructionRead).toHaveBeenCalledWith(expect.anything(), expect.objectContaining({ type: "board" }),
expect.objectContaining({ companyId: "company-1", id: "11111111-1111-4111-8111-111111111111" }));
});
it("requires instance-admin access for every external instruction entry point", async () => {
@@ -331,6 +352,7 @@ describe("agent instructions bundle routes", () => {
expect(mockAgentInstructionsService.updateBundle).not.toHaveBeenCalled();
expect(mockAgentInstructionsService.writeFile).not.toHaveBeenCalled();
expect(mockAgentInstructionsService.deleteFile).not.toHaveBeenCalled();
expect(mockAuthorizeInstructionRead).not.toHaveBeenCalled();
});
it("treats a host root mislabeled as managed as external", async () => {
@@ -383,6 +405,7 @@ describe("agent instructions bundle routes", () => {
expect(res.status, JSON.stringify(res.body)).toBe(200);
expect(mockAgentInstructionsService.getBundle).toHaveBeenCalled();
expect(mockAuthorizeInstructionRead).not.toHaveBeenCalled();
});
it("rejects a company admin that requests a new external instruction root", async () => {
@@ -484,7 +507,8 @@ describe("agent instructions bundle routes", () => {
expect(mockAgentService.create).not.toHaveBeenCalled();
});
it("denies non-privileged agents from reading peer instructions bundles", async () => {
it.each(["", "/file?path=AGENTS.md", "/file?path=private-support.md"])(
"denies peer instruction reads before any data dispatch: %s", async (suffix) => {
mockAgentService.getById.mockImplementation(async (id: string) => {
if (id === "agent-reader") {
return {
@@ -496,11 +520,10 @@ describe("agent instructions bundle routes", () => {
}
return makeAgent();
});
mockAccessService.decide.mockResolvedValue({
allowed: false,
reason: "deny_no_grant",
explanation: "Missing permission: agents:configure or agents:suggest-changes.",
});
// General agent visibility is allowed. It must not authorize instruction
// metadata, canonical entry bytes, or supporting files from a peer.
const { forbidden } = await vi.importActual<typeof import("../errors.js")>("../errors.js");
mockAuthorizeInstructionRead.mockRejectedValue(forbidden("Missing permission to read peer instructions"));
const res = await requestApp(
await createApp({
@@ -510,20 +533,18 @@ describe("agent instructions bundle routes", () => {
source: "agent_key",
}),
(baseUrl) => request(baseUrl)
.get("/api/agents/11111111-1111-4111-8111-111111111111/instructions-bundle"),
.get(`/api/agents/11111111-1111-4111-8111-111111111111/instructions-bundle${suffix}`),
);
expect(res.status, JSON.stringify(res.body)).toBe(403);
expect(res.body.error).toContain("Missing permission");
expect(mockAccessService.decide).toHaveBeenCalledWith(expect.objectContaining({
action: "agent_config:read",
resource: {
type: "agent",
companyId: "company-1",
agentId: "11111111-1111-4111-8111-111111111111",
},
}));
expect(mockAuthorizeInstructionRead).toHaveBeenCalledWith(expect.anything(), expect.objectContaining({
type: "agent", agentId: "agent-reader",
}), { companyId: "company-1", id: "11111111-1111-4111-8111-111111111111" });
expect(mockAgentInstructionsService.getBundle).not.toHaveBeenCalled();
expect(mockAgentInstructionsService.readFile).not.toHaveBeenCalled();
expect(mockInstructionRevisions.readCurrent).not.toHaveBeenCalled();
expect(mockInstructionRevisions.materializeCurrent).not.toHaveBeenCalled();
});
it("allows agents to read their own instructions bundles", async () => {
@@ -540,9 +561,12 @@ describe("agent instructions bundle routes", () => {
expect(res.status, JSON.stringify(res.body)).toBe(200);
expect(mockAgentInstructionsService.getBundle).toHaveBeenCalled();
expect(mockAuthorizeInstructionRead).toHaveBeenCalledWith(expect.anything(), expect.objectContaining({
type: "agent", agentId: "11111111-1111-4111-8111-111111111111",
}), { companyId: "company-1", id: "11111111-1111-4111-8111-111111111111" });
});
it("allows agents with suggest grants to read peer instructions bundles", async () => {
it.each(["AGENTS.md", "private-support.md"])("allows an authorized peer to read instruction file %s", async (filePath) => {
mockAccessService.decide.mockResolvedValue({
allowed: true,
reason: "allow_explicit_grant",
@@ -570,52 +594,100 @@ describe("agent instructions bundle routes", () => {
}),
(baseUrl) => request(baseUrl)
.get("/api/agents/11111111-1111-4111-8111-111111111111/instructions-bundle/file")
.query({ path: "AGENTS.md" }),
.query({ path: filePath }),
);
expect(res.status, JSON.stringify(res.body)).toBe(200);
expect(mockAccessService.decide).toHaveBeenCalledWith(expect.objectContaining({
action: "agent_config:read",
resource: {
type: "agent",
companyId: "company-1",
agentId: "11111111-1111-4111-8111-111111111111",
},
}));
expect(mockAuthorizeInstructionRead).toHaveBeenCalledWith(expect.anything(), expect.objectContaining({
type: "agent", agentId: "coach-agent",
}), { companyId: "company-1", id: "11111111-1111-4111-8111-111111111111" });
expect(mockAgentInstructionsService.readFile).toHaveBeenCalledWith(
expect.objectContaining({ id: "11111111-1111-4111-8111-111111111111" }),
"AGENTS.md",
filePath,
);
});
it("writes a bundle file and persists compatibility config", async () => {
it("commits entry bytes through the canonical service with the authenticated actor", async () => {
const res = await requestApp(await createApp(), (baseUrl) => request(baseUrl)
.put("/api/agents/11111111-1111-4111-8111-111111111111/instructions-bundle/file?companyId=company-1")
.send({
path: "AGENTS.md",
content: "# Updated Agent\n",
clearLegacyPromptTemplate: true,
}));
.send({ path: "AGENTS.md", content: "# Updated Agent\n", baseRevisionId: null }));
expect(res.status, JSON.stringify(res.body)).toBe(200);
expect(mockAgentInstructionsService.writeFile).toHaveBeenCalledWith(
expect.objectContaining({ id: "11111111-1111-4111-8111-111111111111" }),
"AGENTS.md",
"# Updated Agent\n",
{ clearLegacyPromptTemplate: true },
);
expect(mockAgentService.update).toHaveBeenCalledWith(
"11111111-1111-4111-8111-111111111111",
expect.objectContaining({
adapterConfig: expect.objectContaining({
instructionsBundleMode: "managed",
instructionsRootPath: "/tmp/agent-1",
instructionsEntryFile: "AGENTS.md",
instructionsFilePath: "/tmp/agent-1/AGENTS.md",
}),
}),
expect.any(Object),
);
expect(mockInstructionRevisions.commit).toHaveBeenCalledWith(expect.objectContaining({
agentId: "11111111-1111-4111-8111-111111111111", entryFile: "AGENTS.md", content: "# Updated Agent\n", baseRevisionId: null, source: "board",
}), expect.objectContaining({ type: "board" }));
expect(mockAgentInstructionsService.writeFile).not.toHaveBeenCalled();
expect(res.body.receipt.revision.id).toBe("33333333-3333-4333-8333-333333333333");
});
it("returns committed entry content independently of its disk copy", async () => {
mockInstructionRevisions.readCurrent.mockResolvedValue({ revision: { id: "33333333-3333-4333-8333-333333333333", entryFile: "AGENTS.md", byteLength: 9 }, content: "committed" });
const res = await requestApp(await createApp(), (baseUrl) => request(baseUrl)
.get("/api/agents/11111111-1111-4111-8111-111111111111/instructions-bundle/file?path=AGENTS.md"));
expect(res.status).toBe(200);
expect(res.body.content).toBe("committed");
expect(res.body.revision.id).toBe("33333333-3333-4333-8333-333333333333");
expect(mockAgentInstructionsService.readFile).not.toHaveBeenCalled();
});
it.each([Buffer.from([0, 255, 128, 17]), Buffer.alloc(0)])("streams an authorized binary download without text conversion (%j)", async bytes => {
mockAgentService.getById.mockResolvedValue({ ...makeAgent(), adapterConfig: { instructionsBundleMode: "managed" } });
mockDownloadAgentFile.mockResolvedValue({ size: bytes.length, stream: Readable.from(bytes.length ? [bytes] : []) });
const res = await requestApp(await createApp(), url => request(url)
.get("/api/agents/11111111-1111-4111-8111-111111111111/instructions-bundle/file")
.query({ path: "notes/data.bin", download: "true" }));
expect(res.status, JSON.stringify(res.body)).toBe(200);
expect(res.body).toEqual(bytes);
expect(res.headers["content-length"]).toBe(String(bytes.length));
expect(res.headers["content-disposition"]).toContain('filename="data.bin"');
expect(mockDownloadAgentFile).toHaveBeenCalledWith("company-1", "11111111-1111-4111-8111-111111111111", "notes/data.bin", expect.objectContaining({ type: "board" }));
expect(mockAgentInstructionsService.readFile).not.toHaveBeenCalled();
});
it("lists and resolves preserved edits with server scope, actor, and explicit revision", async () => {
const app = await createApp();
const runId = "55555555-5555-4555-8555-555555555555";
const baseRevisionId = "44444444-4444-4444-8444-444444444444";
const prefix = "/api/agents/11111111-1111-4111-8111-111111111111/instructions-bundle/candidates";
mockInstructionWorkingCopies.list.mockResolvedValue([{ runId, entryFile: "AGENTS.md", state: "conflict", content: "preserved" }]);
mockInstructionWorkingCopies.resolve.mockResolvedValue({ revision: { id: baseRevisionId, entryFile: "AGENTS.md", byteLength: 8 }, content: "resolved", changed: true, materialization: "current" });
const listed = await requestApp(app, (url) => request(url).get(prefix));
expect(listed.status).toBe(200);
expect(listed.body).toEqual([{ runId, entryFile: "AGENTS.md", state: "conflict", content: "preserved" }]);
expect(mockInstructionWorkingCopies.list).toHaveBeenCalledWith("company-1", "11111111-1111-4111-8111-111111111111", expect.objectContaining({ type: "board", userId: "local-board" }));
const resolved = await requestApp(app, (url) => request(url).post(`${prefix}/${runId}/resolve`).send({ baseRevisionId, content: "resolved" }));
expect(resolved.status).toBe(200);
expect(resolved.body).toMatchObject({ content: "resolved", receipt: { changed: true } });
expect(mockInstructionWorkingCopies.resolve).toHaveBeenCalledWith({ companyId: "company-1", agentId: "11111111-1111-4111-8111-111111111111", runId, baseRevisionId, content: "resolved" }, expect.objectContaining({ type: "board", userId: "local-board" }));
for (const body of [{ content: "missing base" }, { baseRevisionId, content: "forged", responsibleUserId: "other" }, { baseRevisionId, content: "forged", entryFile: "other.md" }]) {
expect((await requestApp(app, (url) => request(url).post(`${prefix}/${runId}/resolve`).send(body))).status).toBe(400);
}
expect(mockInstructionWorkingCopies.resolve).toHaveBeenCalledOnce();
});
it("exposes scoped history, diff and restore with the server actor", async () => {
const app = await createApp();
const revisionId = "33333333-3333-4333-8333-333333333333";
const baseRevisionId = "44444444-4444-4444-8444-444444444444";
const prefix = "/api/agents/11111111-1111-4111-8111-111111111111/instructions-bundle";
mockInstructionRevisions.history.mockResolvedValue({ revisions: [], nextCursor: null });
mockInstructionRevisions.diff.mockResolvedValue({ removed: "old", added: "new" });
mockInstructionRevisions.restore.mockResolvedValue({ revision: { id: revisionId, entryFile: "AGENTS.md", byteLength: 3 }, content: "old", changed: true, materialization: "current" });
expect((await requestApp(app, (url) => request(url).get(`${prefix}/history?path=AGENTS.md`))).status).toBe(200);
expect((await requestApp(app, (url) => request(url).get(`${prefix}/diff?path=AGENTS.md&from=${revisionId}&to=${baseRevisionId}`))).status).toBe(200);
const restored = await requestApp(app, (url) => request(url).post(`${prefix}/restore`).send({ path: "AGENTS.md", revisionId, baseRevisionId }));
expect(restored.status).toBe(200);
expect(restored.body.receipt.changed).toBe(true);
expect(mockInstructionRevisions.restore).toHaveBeenCalledWith({ companyId: "company-1", agentId: "11111111-1111-4111-8111-111111111111", entryFile: "AGENTS.md", revisionId, baseRevisionId }, expect.objectContaining({ type: "board", userId: "local-board" }));
});
it("requires a base revision and rejects client-supplied responsible identity", async () => {
const app = await createApp();
for (const body of [ { path: "AGENTS.md", content: "stale" }, { path: "AGENTS.md", content: "forged", baseRevisionId: null, responsibleUserId: "forged" } ]) {
const res = await requestApp(app, (baseUrl) => request(baseUrl)
.put("/api/agents/11111111-1111-4111-8111-111111111111/instructions-bundle/file").send(body));
expect([400, 422]).toContain(res.status);
}
expect(mockInstructionRevisions.commit).not.toHaveBeenCalled();
});
it("preserves managed instructions config when switching adapters", async () => {
@@ -1,7 +1,7 @@
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import { agentInstructionsService } from "../services/agent-instructions.js";
type TestAgent = {
@@ -12,7 +12,7 @@ type TestAgent = {
};
async function makeTempDir(prefix: string) {
return fs.mkdtemp(path.join(os.tmpdir(), prefix));
return fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), prefix)));
}
function makeAgent(adapterConfig: Record<string, unknown>): TestAgent {
@@ -29,6 +29,13 @@ describe("agent instructions service", () => {
const originalPaperclipInstanceId = process.env.PAPERCLIP_INSTANCE_ID;
const cleanupDirs = new Set<string>();
beforeEach(async () => {
const home = await makeTempDir("agent-instructions-test-home-");
cleanupDirs.add(home);
process.env.PAPERCLIP_HOME = home;
process.env.PAPERCLIP_INSTANCE_ID = "instructions-service-test";
});
afterEach(async () => {
if (originalPaperclipHome === undefined) delete process.env.PAPERCLIP_HOME;
else process.env.PAPERCLIP_HOME = originalPaperclipHome;
@@ -41,6 +48,37 @@ describe("agent instructions service", () => {
}));
});
it.each(["./AGENTS.md", "notes/../AGENTS.md", "notes\\..\\AGENTS.md", "../invalid"])("reads legacy entry configuration %s without breaking the bundle", async (entry) => {
const root = await makeTempDir("legacy-entry-"); cleanupDirs.add(root);
await fs.writeFile(path.join(root, "AGENTS.md"), "legacy instructions");
const agent = makeAgent({ instructionsBundleMode: "external", instructionsRootPath: root, instructionsEntryFile: entry });
const svc = agentInstructionsService();
const bundle = await svc.getBundle(agent);
expect(bundle.entryFile).toBe("AGENTS.md");
expect((await svc.readFile(agent, bundle.entryFile)).content).toBe("legacy instructions");
if (entry === "../invalid") expect(bundle.warnings.length).toBeGreaterThan(0);
});
it("lists external bundles containing large binary assets", async () => {
const root = await makeTempDir("external-large-assets-"); cleanupDirs.add(root);
await fs.writeFile(path.join(root, "AGENTS.md"), "external instructions");
const asset = await fs.open(path.join(root, "large.bin"), "w");
try { await asset.truncate(17 * 1024 * 1024); } finally { await asset.close(); }
const agent = makeAgent({ instructionsBundleMode: "external", instructionsRootPath: root });
const svc = agentInstructionsService();
const bundle = await svc.getBundle(agent);
expect(bundle.files).toContainEqual(expect.objectContaining({ path: "large.bin", binary: true, editable: false }));
expect((await svc.readFile(agent, "AGENTS.md")).content).toBe("external instructions");
});
it("rejects reserved paths while initializing an unconfigured managed bundle", async () => {
const root = await makeTempDir("unconfigured-reserved-path-"); cleanupDirs.add(root);
process.env.PAPERCLIP_HOME = root;
const svc = agentInstructionsService();
await expect(svc.writeFile(makeAgent({}), ".paperclip-runtime/state", "invalid")).rejects.toMatchObject({ status: 422 });
expect(await fs.readdir(root)).toEqual([]);
});
it("copies the existing bundle into the managed root when switching to managed mode", async () => {
const paperclipHome = await makeTempDir("paperclip-agent-instructions-home-");
const externalRoot = await makeTempDir("paperclip-agent-instructions-external-");
@@ -53,7 +91,7 @@ describe("agent instructions service", () => {
await fs.mkdir(path.join(externalRoot, "docs"), { recursive: true });
await fs.writeFile(path.join(externalRoot, "docs", "TOOLS.md"), "## Tools\n", "utf8");
const svc = agentInstructionsService();
const svc = agentInstructionsService({ select: () => ({ from: () => ({ where: async () => [] }) }) } as never);
const agent = makeAgent({
instructionsBundleMode: "external",
instructionsRootPath: externalRoot,
@@ -250,11 +288,10 @@ describe("agent instructions service", () => {
expect(bundle.mode).toBe("managed");
expect(bundle.rootPath).toBe(managedRoot);
expect(bundle.entryFile).toBe("AGENTS.md");
expect(bundle.entryFile).toBe("docs/MISSING.md");
expect(bundle.files.map((file) => file.path)).toEqual(["AGENTS.md"]);
expect(bundle.warnings).toEqual([
`Recovered managed instructions from disk at ${managedRoot}; ignoring stale configured root ${staleRoot}.`,
"Recovered managed instructions entry file from disk as AGENTS.md; previous entry docs/MISSING.md was missing.",
]);
expect(exported.files).toEqual({ "AGENTS.md": "# Managed Agent\n" });
});
@@ -293,8 +330,8 @@ describe("agent instructions service", () => {
expect(result.adapterConfig).toMatchObject({
instructionsBundleMode: "managed",
instructionsRootPath: managedRoot,
instructionsEntryFile: "AGENTS.md",
instructionsFilePath: path.join(managedRoot, "AGENTS.md"),
instructionsEntryFile: "docs/MISSING.md",
instructionsFilePath: path.join(managedRoot, "docs/MISSING.md"),
});
await expect(fs.readFile(path.join(managedRoot, "docs", "TOOLS.md"), "utf8")).resolves.toBe("## Tools\n");
});
@@ -334,8 +371,8 @@ describe("agent instructions service", () => {
expect(result.adapterConfig).toMatchObject({
instructionsBundleMode: "managed",
instructionsRootPath: managedRoot,
instructionsEntryFile: "AGENTS.md",
instructionsFilePath: path.join(managedRoot, "AGENTS.md"),
instructionsEntryFile: "docs/MISSING.md",
instructionsFilePath: path.join(managedRoot, "docs/MISSING.md"),
});
await expect(fs.stat(path.join(managedRoot, "docs", "TOOLS.md"))).rejects.toThrow();
expect(result.bundle.files.map((file) => file.path)).toEqual(["AGENTS.md"]);
@@ -373,11 +410,10 @@ describe("agent instructions service", () => {
expect(bundle.mode).toBe("managed");
expect(bundle.rootPath).toBe(managedRoot);
expect(bundle.entryFile).toBe("AGENTS.md");
expect(bundle.entryFile).toBe("docs/MISSING.md");
expect(bundle.files.map((file) => file.path)).toEqual(["AGENTS.md"]);
expect(bundle.warnings).toEqual([
`Recovered managed instructions from disk at ${managedRoot}; ignoring stale configured root ${staleRoot}.`,
"Recovered managed instructions entry file from disk as AGENTS.md; previous entry docs/MISSING.md was missing.",
]);
expect(exported.files).toEqual({ "AGENTS.md": "# Managed Agent\n" });
});
@@ -412,7 +412,7 @@ describe("built-in agent routes", () => {
.send({});
expect(res.status, JSON.stringify(res.body)).toBe(200);
expect(mockBuiltInAgentService.reset).toHaveBeenCalledWith(companyId, "briefs", {});
expect(mockBuiltInAgentService.reset).toHaveBeenCalledWith(companyId, "briefs", {}, expect.objectContaining({ type: "agent", agentId: "manager-agent" }));
expect(mockLogActivity).toHaveBeenCalledWith(expect.anything(), expect.objectContaining({
companyId,
actorType: "agent",
+127 -9
View File
@@ -1,3 +1,6 @@
import fs from "node:fs/promises";
import { fileHash } from "../services/agent-file-store.js";
import { stockHash } from "../services/managed-resource-drift.js";
import { randomUUID } from "node:crypto";
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
@@ -27,6 +30,7 @@ import {
getEmbeddedPostgresTestSupport,
startEmbeddedPostgresTestDatabase,
} from "./helpers/embedded-postgres.js";
import { agentInstructionRevisionService } from "../services/agent-instruction-revisions.js";
import { agentInstructionsService } from "../services/agent-instructions.ts";
import { agentService } from "../services/agents.ts";
import { approvalService } from "../services/approvals.ts";
@@ -116,6 +120,16 @@ describe("built-in agent asset loading", () => {
describeEmbeddedPostgres("built-in agents", () => {
let db!: ReturnType<typeof createDb>;
let tempDb: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>> | null = null;
const instructionOperator = { type: "board", userId: "local-board", source: "local_implicit" } as const;
async function writeInstructionEntry(agent: { id: string; companyId: string }, entryFile: string, content: string) {
const service = agentInstructionRevisionService(db);
const target = { companyId: agent.companyId, agentId: agent.id };
const baseline = await service.readCurrent(target, instructionOperator);
await service.commit({ ...target, entryFile, content, baseRevisionId: baseline?.revision.id ?? null, source: "board" }, instructionOperator);
const refreshed = await agentService(db).getById(agent.id);
return { adapterConfig: refreshed!.adapterConfig };
}
beforeAll(async () => {
tempDb = await startEmbeddedPostgresTestDatabase("paperclip-built-in-agents-");
@@ -553,7 +567,7 @@ describeEmbeddedPostgres("built-in agents", () => {
capabilities: "Custom purpose",
});
const reset = await builtIns.reset(companyId, "briefs");
const reset = await builtIns.reset(companyId, "briefs", {}, instructionOperator);
expect(reset).toMatchObject({
status: "ready",
@@ -795,13 +809,111 @@ describeEmbeddedPostgres("built-in agents", () => {
expect(approvalRows).toHaveLength(1);
});
it("automatically upgrades untouched stock instructions while preserving personal files", async () => {
const companyId = await seedCompany({ requireApproval: false });
const svc = builtInAgentService(db);
const created = await svc.ensure(companyId, "reflection-coach");
const agent = created.agent!;
const oldFiles = { "AGENTS.md": "# Previous stock instructions\n", "obsolete.txt": "old stock support" };
await writeInstructionEntry(agent, "AGENTS.md", oldFiles["AGENTS.md"]);
await agentInstructionsService(db).writeFile(agent, "obsolete.txt", oldFiles["obsolete.txt"]);
await agentInstructionsService(db).writeFile(agent, "personal.txt", "keep my notes");
await db.update(builtInManagedResources).set({ stockHash: stockHash(oldFiles), stockVersion: "previous",
defaultsJson: { entryFile: "AGENTS.md", files: Object.keys(oldFiles) },
}).where(and(eq(builtInManagedResources.companyId, companyId), eq(builtInManagedResources.resourceKind, "instructions")));
const updated = await svc.ensure(companyId, "reflection-coach");
expect(updated.resources.find(resource => resource.resourceKind === "instructions")).toMatchObject({ stockStatus: "stock_current" });
expect((await agentInstructionsService().readFile(updated.agent!, "AGENTS.md")).content).toContain("You are Reflection Coach");
expect((await agentInstructionsService().readFile(updated.agent!, "personal.txt")).content).toBe("keep my notes");
await expect(agentInstructionsService().readFile(updated.agent!, "obsolete.txt")).rejects.toMatchObject({ status: 404 });
const [event] = await db.select().from(activityLog).where(and(eq(activityLog.companyId, companyId), eq(activityLog.actorId, "built-in-reconcile")));
expect(event).toMatchObject({ actorType: "system", action: "agent.files_updated" });
});
it.each(["file", "database"])("retries a stock update after a %s failure without overwriting intervening edits", async (failure) => {
const companyId = await seedCompany({ requireApproval: false });
const svc = builtInAgentService(db);
const created = await svc.ensure(companyId, "reflection-coach");
const agent = created.agent!;
const oldFiles = { "AGENTS.md": "# Old stock\n", "obsolete.txt": "old supporting file" };
await writeInstructionEntry(agent, "AGENTS.md", oldFiles["AGENTS.md"]);
await agentInstructionsService(db).writeFile(agent, "obsolete.txt", oldFiles["obsolete.txt"]);
await db.update(builtInManagedResources).set({ stockHash: stockHash(oldFiles), stockVersion: "old",
defaultsJson: { entryFile: "AGENTS.md", files: Object.keys(oldFiles) },
}).where(and(eq(builtInManagedResources.companyId, companyId), eq(builtInManagedResources.resourceKind, "instructions")));
const trigger = `stock_failure_${companyId.replaceAll("-", "")}`;
const remove = fs.rm.bind(fs);
const fault = failure === "file" ? vi.spyOn(fs, "rm").mockImplementation(async (file, options) => {
if (String(file).endsWith("/obsolete.txt")) throw new Error("injected stock file failure");
return remove(file, options);
}) : null;
if (failure === "database") {
await db.execute(sql.raw(`CREATE FUNCTION ${trigger}() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN
IF NEW.company_id = '${companyId}' AND NEW.resource_kind = 'instructions' AND NEW.stock_hash <> OLD.stock_hash THEN
RAISE EXCEPTION 'injected stock binding failure'; END IF; RETURN NEW; END $$`));
await db.execute(sql.raw(`CREATE TRIGGER ${trigger} BEFORE UPDATE ON built_in_managed_resources FOR EACH ROW EXECUTE FUNCTION ${trigger}()`));
}
try { await expect(svc.ensure(companyId, "reflection-coach")).rejects.toThrow(); }
finally {
fault?.mockRestore();
if (failure === "database") {
await db.execute(sql.raw(`DROP TRIGGER ${trigger} ON built_in_managed_resources`));
await db.execute(sql.raw(`DROP FUNCTION ${trigger}()`));
}
}
const [pending] = await db.select().from(builtInManagedResources).where(and(eq(builtInManagedResources.companyId, companyId), eq(builtInManagedResources.resourceKind, "instructions")));
expect(pending.defaultsJson.pendingInstructionsUpdate).toBeTruthy();
expect(pending.stockHash).toBe(stockHash(oldFiles));
const partial = (await agentInstructionsService().readFile(agent, "AGENTS.md")).content;
expect(partial).toContain("You are Reflection Coach");
// A real operator edit after the failed attempt must stop the retry.
await writeInstructionEntry(agent, "AGENTS.md", "operator edit after failure");
await svc.ensure(companyId, "reflection-coach");
expect((await agentInstructionsService().readFile(agent, "AGENTS.md")).content).toBe("operator edit after failure");
await writeInstructionEntry(agent, "AGENTS.md", partial);
const retried = await svc.ensure(companyId, "reflection-coach");
expect(retried.resources.find(resource => resource.resourceKind === "instructions")).toMatchObject({ stockStatus: "stock_current" });
await expect(agentInstructionsService().readFile(agent, "obsolete.txt")).rejects.toMatchObject({ status: 404 });
const [complete] = await db.select().from(builtInManagedResources).where(eq(builtInManagedResources.id, pending.id));
expect(complete.defaultsJson.pendingInstructionsUpdate).toBeUndefined();
});
it.each([false, true])("supersedes an interrupted stock version, including rollback=%s", async (rollback) => {
const companyId = await seedCompany({ requireApproval: false });
const svc = builtInAgentService(db);
const created = await svc.ensure(companyId, "reflection-coach");
const agent = created.agent!;
const currentStock = (await agentInstructionsService().readFile(agent, "AGENTS.md")).content;
const oldFiles = { "AGENTS.md": rollback ? currentStock : "# Original stock" };
const interruptedFiles = { "AGENTS.md": "# Intermediate stock", "intermediate-only.txt": "intermediate support" };
await writeInstructionEntry(agent, "AGENTS.md", interruptedFiles["AGENTS.md"]);
await agentInstructionsService(db).writeFile(agent, "intermediate-only.txt", interruptedFiles["intermediate-only.txt"]);
await agentInstructionsService(db).writeFile(agent, "personal.txt", "personal notes");
await db.update(builtInManagedResources).set({ stockHash: stockHash(oldFiles), stockVersion: "old",
defaultsJson: { entryFile: "AGENTS.md", files: Object.keys(oldFiles), pendingInstructionsUpdate: {
stockHash: stockHash(interruptedFiles),
baseHashes: { "AGENTS.md": fileHash(Buffer.from(oldFiles["AGENTS.md"])), "intermediate-only.txt": null },
nextHashes: Object.fromEntries(Object.entries(interruptedFiles).map(([file, text]) => [file, fileHash(Buffer.from(text))])),
} },
}).where(and(eq(builtInManagedResources.companyId, companyId), eq(builtInManagedResources.resourceKind, "instructions")));
await writeInstructionEntry(agent, "AGENTS.md", "intervening operator edit");
await svc.ensure(companyId, "reflection-coach");
expect((await agentInstructionsService().readFile(agent, "AGENTS.md")).content).toBe("intervening operator edit");
await writeInstructionEntry(agent, "AGENTS.md", interruptedFiles["AGENTS.md"]);
const updated = await svc.ensure(companyId, "reflection-coach");
expect(updated.resources.find(resource => resource.resourceKind === "instructions")).toMatchObject({ stockStatus: "stock_current" });
expect((await agentInstructionsService().readFile(agent, "AGENTS.md")).content).toBe(currentStock);
await expect(agentInstructionsService().readFile(agent, "intermediate-only.txt")).rejects.toMatchObject({ status: 404 });
expect((await agentInstructionsService().readFile(agent, "personal.txt")).content).toBe("personal notes");
});
it("preserves Reflection Coach instruction drift on reconcile and restores it on reset", async () => {
const companyId = await seedCompany();
const builtIns = builtInAgentService(db);
const created = await builtIns.ensure(companyId, "reflection-coach");
const instructions = agentInstructionsService();
await instructions.writeFile(created.agent!, "AGENTS.md", "# Custom Reflection Coach\n\nOperator edit.\n");
await writeInstructionEntry(created.agent!, "AGENTS.md", "# Custom Reflection Coach\n\nOperator edit.\n");
const reconciled = await builtIns.ensure(companyId, "reflection-coach");
const drift = reconciled.resources.find((resource) => resource.resourceKind === "instructions");
@@ -815,7 +927,7 @@ describeEmbeddedPostgres("built-in agents", () => {
content: "# Custom Reflection Coach\n\nOperator edit.\n",
});
const reset = await builtIns.reset(companyId, "reflection-coach");
const reset = await builtIns.reset(companyId, "reflection-coach", {}, instructionOperator);
expect(reset.resources.find((resource) => resource.resourceKind === "instructions")).toMatchObject({
stockStatus: "stock_current",
resetAvailable: false,
@@ -823,6 +935,12 @@ describeEmbeddedPostgres("built-in agents", () => {
const resetFile = await instructions.readFile(reset.agent!, "AGENTS.md");
expect(resetFile.content).toContain("Reflection Coach");
expect(resetFile.content).not.toContain("Operator edit.");
const revisions = agentInstructionRevisionService(db);
const target = { companyId, agentId: created.agent!.id, entryFile: "AGENTS.md" };
const history = await revisions.history(target, instructionOperator);
expect(history.revisions).toHaveLength(0);
const current = await revisions.readCurrent(target, instructionOperator);
expect(current?.content).toBe(resetFile.content);
});
it("blocks deleting a built-in agent", async () => {
@@ -1270,7 +1388,7 @@ describeEmbeddedPostgres("built-in agents", () => {
const created = await builtIns.ensure(companyId, "summarizer");
const instructions = agentInstructionsService();
await instructions.writeFile(created.agent!, "AGENTS.md", "# Custom Summarizer\n\nOperator edit.\n");
await writeInstructionEntry(created.agent!, "AGENTS.md", "# Custom Summarizer\n\nOperator edit.\n");
const reconciled = await builtIns.ensure(companyId, "summarizer");
expect(reconciled.resources.find((resource) => resource.resourceKind === "instructions")).toMatchObject({
@@ -1279,7 +1397,7 @@ describeEmbeddedPostgres("built-in agents", () => {
changedFiles: ["AGENTS.md"],
});
const reset = await builtIns.reset(companyId, "summarizer");
const reset = await builtIns.reset(companyId, "summarizer", {}, instructionOperator);
expect(reset.resources.find((resource) => resource.resourceKind === "instructions")).toMatchObject({
stockStatus: "stock_current",
resetAvailable: false,
@@ -1419,7 +1537,7 @@ describeEmbeddedPostgres("built-in agents", () => {
const coach = created.agent!;
const instructionsSvc = agentInstructionsService();
const originalInstructions = "# Target Coder\n\nWork from the assigned issue.\n";
const prepared = await instructionsSvc.writeFile(target, "AGENTS.md", originalInstructions);
const prepared = await writeInstructionEntry(target, "AGENTS.md", originalInstructions);
let persistedTarget = (await agentsSvc.update(target.id, { adapterConfig: prepared.adapterConfig }))!;
const interactionsSvc = issueThreadInteractionService(db);
@@ -1431,7 +1549,7 @@ describeEmbeddedPostgres("built-in agents", () => {
if (interaction?.kind !== "request_confirmation" || interaction.status !== "accepted") {
return false;
}
const written = await instructionsSvc.writeFile(persistedTarget, "AGENTS.md", input.nextInstructions);
const written = await writeInstructionEntry(persistedTarget, "AGENTS.md", input.nextInstructions);
persistedTarget = (await agentsSvc.update(persistedTarget.id, { adapterConfig: written.adapterConfig }))!;
return true;
};
@@ -1568,7 +1686,7 @@ describeEmbeddedPostgres("built-in agents", () => {
const agent = created.agent!;
const instructionsSvc = agentInstructionsService();
await instructionsSvc.writeFile(agent, "AGENTS.md", "# Custom Reflection Coach\n\nDo not overwrite me.\n");
await writeInstructionEntry(agent, "AGENTS.md", "# Custom Reflection Coach\n\nDo not overwrite me.\n");
await db
.update(companySkills)
.set({ markdown: "---\nname: reflection-coach\n---\n\n# Custom skill\n" })
@@ -1597,7 +1715,7 @@ describeEmbeddedPostgres("built-in agents", () => {
const reset = await builtInAgentService(db).reset(companyId, "reflection-coach", {
resources: ["instructions", "routine"],
});
}, instructionOperator);
expect(reset.resources.find((resource) => resource.resourceKind === "instructions")).toMatchObject({
stockStatus: "stock_current",
resetAvailable: false,
@@ -264,6 +264,12 @@ describeEmbeddedPostgres("heartbeat list", () => {
summary: "completed",
stdout: oversizedStdout,
nestedHuge: { payload: oversizedNestedPayload },
instructionSave: {
state: "unavailable", contract: "agent_files", entryFile: "AGENTS.md",
errorCode: "AGENT_FILES_LIMIT_EXCEEDED",
storageWarning: "Agent storage is full. Runs can continue.".repeat(50),
privateSyncMetadata: oversizedNestedPayload,
},
workspaceRestoreFailure: "restore_unsafe_archive",
finalResponseRecorded: true,
executionBeforeRestore: { errorCode: "model_error", exitCode: 2, timedOut: false },
@@ -278,6 +284,11 @@ describeEmbeddedPostgres("heartbeat list", () => {
truncated: true,
truncationReason: "oversized_result_json",
stdoutTruncated: true,
instructionSave: {
state: "unavailable", contract: "agent_files", entryFile: "AGENTS.md",
errorCode: "AGENT_FILES_LIMIT_EXCEEDED",
storageWarning: "Agent storage is full. Runs can continue.".repeat(50).slice(0, 1024),
},
workspaceRestoreFailure: "restore_unsafe_archive",
finalResponseRecorded: true,
executionBeforeRestore: { errorCode: "model_error", exitCode: 2, timedOut: false },
@@ -285,6 +296,7 @@ describeEmbeddedPostgres("heartbeat list", () => {
expect(typeof result?.stdout).toBe("string");
expect((result?.stdout as string).length).toBeLessThan(oversizedStdout.length);
expect(result).not.toHaveProperty("nestedHuge");
expect(result?.instructionSave).not.toHaveProperty("privateSyncMetadata");
});
});
@@ -7,6 +7,8 @@ import { afterAll, afterEach, beforeAll, describe, expect, it } from "vitest";
import {
activityLog,
agentConfigRevisions,
agentInstructionRevisions,
agentInstructionHeads,
agents,
approvals,
companies,
@@ -23,6 +25,9 @@ import {
} from "./helpers/embedded-postgres.js";
import { buildHostServices } from "../services/plugin-host-services.js";
import { agentService } from "../services/agents.js";
import { agentInstructionRevisionService } from "../services/agent-instruction-revisions.js";
import { agentInstructionsService } from "../services/agent-instructions.js";
import { pluginRegistryService } from "../services/plugin-registry.js";
const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport();
const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip;
@@ -312,7 +317,7 @@ describeEmbeddedPostgres("plugin-managed agents", () => {
it("materializes declared managed agent instructions with local folder paths", async () => {
const previousHome = process.env.PAPERCLIP_HOME;
const previousInstance = process.env.PAPERCLIP_INSTANCE_ID;
const tempHome = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-managed-agent-home-"));
const tempHome = await fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-managed-agent-home-")));
const wikiRoot = await fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-managed-agent-wiki-")));
process.env.PAPERCLIP_HOME = tempHome;
process.env.PAPERCLIP_INSTANCE_ID = "test";
@@ -369,6 +374,39 @@ describeEmbeddedPostgres("plugin-managed agents", () => {
expect(content).toContain("You are the LLM Wiki Maintainer.");
expect(content).toContain(`Wiki root: \`${wikiRoot}\``);
expect(content).toContain(`Wiki schema: \`${path.join(wikiRoot, "AGENTS.md")}\``);
// A managed plugin reset must preserve the previous entry in canonical
// history instead of deleting a shared bundle or bypassing its write guard.
const edited = "# User-customized wiki instructions\n";
await fs.writeFile(instructionsFilePath as string, edited);
const reset = await services.agents.managedReset({ companyId, agentKey: "wiki-maintainer" });
expect(reset.status).toBe("reset");
expect(await fs.readFile(instructionsFilePath as string, "utf8")).toBe(content);
const history = await db.select().from(agentInstructionRevisions)
.where(eq(agentInstructionRevisions.agentId, created.agentId!));
expect(history).toHaveLength(0);
expect(await db.select().from(agentInstructionHeads)
.where(eq(agentInstructionHeads.agentId, created.agentId!))).toHaveLength(0);
const repeated = await services.agents.managedReset({ companyId, agentKey: "wiki-maintainer" });
expect(repeated.status).toBe("reset");
expect(await db.select().from(agentInstructionRevisions)
.where(eq(agentInstructionRevisions.agentId, created.agentId!))).toHaveLength(0);
const audit = await db.select().from(activityLog).where(eq(activityLog.action, "agent.files_updated"));
expect(audit).toHaveLength(1);
expect(audit.every((row) => row.actorType === "plugin" && row.actorId === pluginId)).toBe(true);
const bundles = agentInstructionsService(db);
await bundles.writeFile(repeated.agent!, "CUSTOM.md", "# Alternate configured entry\n");
const switched = await bundles.updateBundle(repeated.agent!, { entryFile: "CUSTOM.md" });
await agentService(db).update(created.agentId!, { adapterConfig: switched.adapterConfig });
const resetEntry = await services.agents.managedReset({ companyId, agentKey: "wiki-maintainer" });
expect(resetEntry.agent?.adapterConfig.instructionsEntryFile).toBe("AGENTS.md");
expect(resetEntry.defaultDrift).toBeNull();
expect(await fs.readFile(instructionsFilePath as string, "utf8")).toBe(content);
const preserved = await db.select().from(agentInstructionRevisions).where(eq(agentInstructionRevisions.agentId, created.agentId!));
expect(preserved).toHaveLength(0);
expect(await fs.readFile(path.join(path.dirname(instructionsFilePath as string), "CUSTOM.md"), "utf8")).toBe("# Alternate configured entry\n");
} finally {
if (previousHome === undefined) delete process.env.PAPERCLIP_HOME;
else process.env.PAPERCLIP_HOME = previousHome;
@@ -379,6 +417,60 @@ describeEmbeddedPostgres("plugin-managed agents", () => {
}
});
it("fences plugin instruction reset by exact ownership, current capability and canonical CAS", async () => {
const previousHome = process.env.PAPERCLIP_HOME;
const tempHome = await fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), "plugin-reset-cas-")));
process.env.PAPERCLIP_HOME = tempHome;
try {
const pluginManifest = manifest();
pluginManifest.agents![0]!.instructions = { content: "# Default\n" };
const { companyId, pluginId, services } = await seedCompanyAndPlugin({ manifest: pluginManifest });
const created = await services.agents.managedReconcile({ companyId, agentKey: "wiki-maintainer" });
const target = { companyId, agentId: created.agentId! };
const actor = { type: "plugin" as const, pluginId, pluginKey: pluginManifest.id, agentKey: "wiki-maintainer" };
const revisions = agentInstructionRevisionService(db);
const baselineRead = await revisions.readForPluginReset(target, actor);
const baseline = baselineRead.snapshot!;
const input = { ...target, entryFile: "AGENTS.md", baseRevisionId: baseline.revision.id,
configuredEntryFile: baselineRead.configuredEntryFile, content: "# Reset\n" };
await expect(revisions.commitPluginReset(input, { ...actor, pluginId: randomUUID() })).rejects.toMatchObject({ status: 403 });
await expect(revisions.commitPluginReset({ ...input, companyId: randomUUID() }, actor)).rejects.toMatchObject({ status: 404 });
await expect(revisions.commitPluginReset(input, { ...actor, agentKey: "foreign-agent" })).rejects.toMatchObject({ status: 403 });
await db.update(plugins).set({ manifestJson: { ...pluginManifest, capabilities: [] } }).where(eq(plugins.id, pluginId));
await expect(revisions.commitPluginReset(input, actor)).rejects.toMatchObject({ status: 403 });
await db.update(plugins).set({ manifestJson: pluginManifest }).where(eq(plugins.id, pluginId));
const originalMetadata = created.agent!.metadata;
await db.update(agents).set({ metadata: {} }).where(eq(agents.id, target.agentId));
await expect(revisions.commitPluginReset(input, actor)).rejects.toMatchObject({ status: 403 });
await db.update(agents).set({ metadata: originalMetadata }).where(eq(agents.id, target.agentId));
const [binding] = await db.select().from(pluginManagedResources).where(eq(pluginManagedResources.resourceId, target.agentId));
await db.update(pluginManagedResources).set({ resourceId: randomUUID() }).where(eq(pluginManagedResources.id, binding.id));
await expect(revisions.commitPluginReset(input, actor)).rejects.toMatchObject({ status: 403 });
await db.update(pluginManagedResources).set({ resourceId: target.agentId }).where(eq(pluginManagedResources.id, binding.id));
await expect(revisions.commitPluginReset({ ...input, entryFile: "OTHER.md" }, actor)).rejects.toMatchObject({ status: 403 });
const bundles = agentInstructionsService(db);
await bundles.writeFile(created.agent!, "OTHER.md", "# Other entry\n");
const switched = await bundles.updateBundle(created.agent!, { entryFile: "OTHER.md" });
await agentService(db).update(target.agentId, { adapterConfig: switched.adapterConfig });
await expect(revisions.commitPluginReset(input, actor)).rejects.toMatchObject({ status: 409, details: { code: "INSTRUCTION_ENTRY_CHANGED" } });
await agentService(db).update(target.agentId, { adapterConfig: created.agent!.adapterConfig });
const results = await Promise.allSettled([
revisions.commitPluginReset(input, actor),
revisions.commitPluginReset({ ...input, content: "# Concurrent reset\n" }, actor),
]);
expect(results.filter((result) => result.status === "fulfilled")).toHaveLength(1);
const loser = results.find((result) => result.status === "rejected") as PromiseRejectedResult;
expect(loser.reason).toMatchObject({ status: 409, details: { code: "INSTRUCTION_REVISION_CONFLICT" } });
const winner = results.find((result) => result.status === "fulfilled") as PromiseFulfilledResult<Awaited<ReturnType<typeof revisions.commitPluginReset>>>;
expect(await fs.readFile(created.agent!.adapterConfig.instructionsFilePath as string, "utf8")).toBe(winner.value.content);
const history = await db.select().from(agentInstructionRevisions).where(eq(agentInstructionRevisions.agentId, target.agentId));
expect(history).toHaveLength(0);
} finally {
if (previousHome === undefined) delete process.env.PAPERCLIP_HOME; else process.env.PAPERCLIP_HOME = previousHome;
await fs.rm(tempHome, { recursive: true, force: true });
}
});
it("repairs a missing binding by relinking a same-company managed agent marker", async () => {
const { companyId, pluginId, pluginManifest, services } = await seedCompanyAndPlugin();
const agentId = randomUUID();
@@ -410,6 +502,76 @@ describeEmbeddedPostgres("plugin-managed agents", () => {
expect(binding?.data).toMatchObject({ agentId });
});
it("preserves current-file authority when relinking after a hard uninstall and reinstall", async () => {
const previousHome = process.env.PAPERCLIP_HOME;
const tempHome = await fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), "plugin-reinstall-reset-")));
process.env.PAPERCLIP_HOME = tempHome;
try {
const pluginManifest = manifest();
pluginManifest.agents![0]!.instructions = { content: "# Original stock\n" };
const { companyId, pluginId, services } = await seedCompanyAndPlugin({ manifest: pluginManifest });
const created = await services.agents.managedReconcile({ companyId, agentKey: "wiki-maintainer" });
const target = { companyId, agentId: created.agentId! };
const revisions = agentInstructionRevisionService(db);
const original = await revisions.readForPluginReset(target, {
type: "plugin", pluginId, pluginKey: pluginManifest.id, agentKey: "wiki-maintainer",
});
const registry = pluginRegistryService(db);
await registry.uninstall(pluginId, true);
expect(await db.select().from(pluginManagedResources)).toHaveLength(0);
expect(await db.select().from(pluginEntities)).toHaveLength(0);
const nextManifest = structuredClone(pluginManifest);
nextManifest.agents![0]!.instructions = { content: "# Reinstalled stock\n" };
const installed = await registry.install({ packageName: "@paperclipai/plugin-managed-agents-test" }, nextManifest);
expect(installed!.id).not.toBe(pluginId);
await registry.updateStatus(installed!.id, { status: "ready" });
const reinstalledServices = buildHostServices(db, installed!.id, nextManifest.id, createEventBusStub(), undefined, {
manifest: nextManifest,
});
const relinked = await reinstalledServices.agents.managedReconcile({ companyId, agentKey: "wiki-maintainer" });
expect(relinked.status).toBe("relinked");
expect(relinked.agentId).toBe(created.agentId);
const reset = await reinstalledServices.agents.managedReset({ companyId, agentKey: "wiki-maintainer" });
expect(reset.agentId).toBe(created.agentId);
expect(reset.agent!.metadata).toMatchObject({ paperclipManagedResource: { pluginId: installed!.id }, pluginManagedAgent: { pluginId: installed!.id } });
expect(await fs.readFile(reset.agent!.adapterConfig.instructionsFilePath as string, "utf8")).toBe("# Reinstalled stock\n");
const history = await db.select().from(agentInstructionRevisions).where(eq(agentInstructionRevisions.agentId, created.agentId!));
expect(history).toHaveLength(0);
} finally {
if (previousHome === undefined) delete process.env.PAPERCLIP_HOME; else process.env.PAPERCLIP_HOME = previousHome;
await fs.rm(tempHome, { recursive: true, force: true });
}
});
it.each(["not-ready", "capability-removed", "declaration-removed", "other-plugin-owner"])(
"does not relink a managed agent when its authority is %s", async (reason) => {
const { companyId, pluginId, pluginManifest, services } = await seedCompanyAndPlugin();
const created = await services.agents.managedReconcile({ companyId, agentKey: "wiki-maintainer" });
await db.delete(pluginEntities);
await db.delete(pluginManagedResources);
if (reason === "not-ready") {
await db.update(plugins).set({ status: "disabled" }).where(eq(plugins.id, pluginId));
} else if (reason === "capability-removed") {
await db.update(plugins).set({ manifestJson: { ...pluginManifest, capabilities: [] } }).where(eq(plugins.id, pluginId));
} else if (reason === "declaration-removed") {
await db.update(plugins).set({ manifestJson: { ...pluginManifest, agents: [] } }).where(eq(plugins.id, pluginId));
} else {
const otherId = randomUUID();
await db.insert(plugins).values({ id: otherId, pluginKey: "paperclip.other-owner", packageName: "other",
version: "0.1.0", apiVersion: 1, categories: [], manifestJson: { ...pluginManifest, id: "paperclip.other-owner" }, status: "ready", installOrder: 2 });
await db.update(agents).set({ metadata: { ...created.agent!.metadata,
paperclipManagedResource: { pluginId: otherId, pluginKey: pluginManifest.id, resourceKind: "agent", resourceKey: "wiki-maintainer" },
} }).where(eq(agents.id, created.agentId!));
}
const [before] = await db.select().from(agents).where(eq(agents.id, created.agentId!));
await expect(services.agents.managedReconcile({ companyId, agentKey: "wiki-maintainer" })).rejects.toMatchObject({ status: 403 });
expect(await db.select().from(pluginEntities)).toHaveLength(0);
expect(await db.select().from(pluginManagedResources)).toHaveLength(0);
const [after] = await db.select().from(agents).where(eq(agents.id, created.agentId!));
expect(after.metadata).toEqual(before.metadata);
},
);
it("respects board approval policy for new managed agents", async () => {
const { companyId, services } = await seedCompanyAndPlugin({ requireApproval: true });
@@ -9,7 +9,7 @@ import {
writeEnvFileAtomicallyIfChanged,
} from "@paperclipai/shared/env-file";
import { resourceStatus, stockHash } from "../services/managed-resource-drift.js";
import { agentInstructionsService } from "../services/agent-instructions.js";
import { agentInstructionsService, resolveManagedInstructionsRoot } from "../services/agent-instructions.js";
/**
* Cross-cutting setup/sync rerun survival test.
@@ -23,7 +23,7 @@ import { agentInstructionsService } from "../services/agent-instructions.js";
* - .env -> updateEnvFileContents + writeEnvFileAtomicallyIfChanged
* - managed sandbox env -> resourceStatus/stockHash drift gate (the `shouldWrite`
* classifier the boot reconciler and built-in sync share)
* - managed instructions -> agentInstructionsService materialize + the drift gate that
* - managed instructions -> agentInstructionsService export + the drift gate that
* makes built-in/plugin/portability rebuilds skip operator edits
*
* The DB-backed end-to-end variants of the sandbox-row, skills-assignment, and
@@ -31,7 +31,7 @@ import { agentInstructionsService } from "../services/agent-instructions.js";
* server/src/__tests__/company-skills-service.test.ts (skills add/remove/replace)
* server/src/services/managed-environments.test.ts (sandbox row boot reconcile)
* server/src/__tests__/environment-service.test.ts (sandbox skip + stock update)
* server/src/__tests__/agent-instructions-service.test.ts (managed instruction drift)
* server/src/__tests__/agent-instruction-revisions.test.ts (guarded bundle materialization)
*/
const cleanupDirs = new Set<string>();
@@ -147,7 +147,7 @@ describe("setup/sync rerun survival — cross-cutting", () => {
).toBe("stock_current");
});
it("keeps operator edits and additions in a managed instructions tree across a re-materialize", async () => {
it("keeps operator edits and additions when managed instructions reconciliation detects drift", async () => {
const home = await tmp("pap16587-instr-home-");
process.env.PAPERCLIP_HOME = home;
process.env.PAPERCLIP_INSTANCE_ID = "test-instance";
@@ -156,18 +156,25 @@ describe("setup/sync rerun survival — cross-cutting", () => {
const agent = { id: "agent-1", companyId: "company-1", name: "Agent 1", adapterConfig: {} };
const stockFiles = { "AGENTS.md": "# stock\n", "docs/TOOLS.md": "## stock tools\n" };
const first = await svc.materializeManagedBundle(agent, stockFiles, { entryFile: "AGENTS.md" });
const root = first.bundle.managedRootPath!;
// Initial files are fixture setup. Bundle materialization now requires a
// real database so that it cannot bypass a canonical instruction revision.
const root = resolveManagedInstructionsRoot(agent);
for (const [name, content] of Object.entries(stockFiles)) {
await fs.mkdir(path.dirname(path.join(root, name)), { recursive: true });
await fs.writeFile(path.join(root, name), content, "utf8");
}
const managedAgent = { ...agent, adapterConfig: {
instructionsBundleMode: "managed", instructionsRootPath: root,
instructionsEntryFile: "AGENTS.md", instructionsFilePath: path.join(root, "AGENTS.md"),
} };
// Operator hand-edits a managed file and adds a brand-new operator-only file.
await fs.writeFile(path.join(root, "AGENTS.md"), "# stock\n\noperator note\n", "utf8");
await fs.writeFile(path.join(root, "OPERATOR.md"), "operator-added\n", "utf8");
// The drift gate the built-in/plugin/portability rebuilds share: classify the tree.
const currentFiles = {
"AGENTS.md": await fs.readFile(path.join(root, "AGENTS.md"), "utf8"),
"docs/TOOLS.md": await fs.readFile(path.join(root, "docs", "TOOLS.md"), "utf8"),
};
const currentFiles = (await svc.exportFiles(managedAgent)).files;
expect(currentFiles["OPERATOR.md"]).toBe("operator-added\n");
const status = resourceStatus({
resourceId: agent.id,
currentHash: stockHash(currentFiles),
+154 -4
View File
@@ -1,3 +1,5 @@
import { agentFileStore, agentFileTokenFromHash } from "../services/agent-file-store.js";
import { pipeline } from "node:stream/promises";
import { resolveAgentAppearance, agentAvatarUrl } from "@paperclipai/shared";
import { listOpenRouterModels } from "../services/openrouter-models.js";
import { prepareManagedAiRuntime, assertManagedAiProjectAuth, stripAiAuthBindings } from "../services/ai-connection-runtime.js";
@@ -38,6 +40,8 @@ import {
type AgentSkillSnapshot,
type InstanceSchedulerHeartbeatAgent,
upsertAgentInstructionsFileSchema,
restoreAgentInstructionSchema,
resolveAgentInstructionCandidateSchema,
updateAgentInstructionsBundleSchema,
updateAgentPermissionsSchema,
updateAgentInstructionsPathSchema,
@@ -63,7 +67,11 @@ import {
import { trackAgentCreated } from "@paperclipai/shared/telemetry";
import { validate } from "../middleware/validate.js";
import { inheritNativeRunnerAdapterConfig } from "../services/native-runtime/native-agent-runtime-inheritance.js";
import { agentInstructionsBundleMode } from "../services/agent-instructions.js";
import { agentInstructionRevisionService } from "../services/agent-instruction-revisions.js";
import { agentInstructionWorkingCopyService } from "../services/agent-instruction-working-copies.js";
import { authorizeInstructionRead } from "../services/agent-instruction-authorization.js";
import { instructionPath } from "../services/agent-instruction-files.js";
import { agentInstructionsBundleMode, deriveBundleState } from "../services/agent-instructions.js";
import {
agentService,
agentInstructionsService,
@@ -724,7 +732,17 @@ export function agentRoutes(
const recovery = recoveryService(db, { enqueueWakeup: heartbeat.wakeup });
const issueApprovalsSvc = issueApprovalService(db);
const secretsSvc = secretService(db);
const instructions = agentInstructionsService();
const instructions = agentInstructionsService(db);
const agentFiles = agentFileStore(db);
const instructionRevisions = agentInstructionRevisionService(db);
const instructionWorkingCopies = agentInstructionWorkingCopyService(db);
function instructionFileDetail(snapshot: import("@paperclipai/shared").AgentInstructionSnapshot,
receipt?: import("@paperclipai/shared").AgentInstructionCommitReceipt) {
const path = snapshot.revision.entryFile;
return { path, content: snapshot.content, contentHash: snapshot.revision.contentHash, size: snapshot.revision.byteLength, revision: snapshot.revision, receipt,
language: path.toLowerCase().endsWith(".md") ? "markdown" : "text", markdown: path.toLowerCase().endsWith(".md"),
isEntryFile: true, editable: true, deprecated: false, virtual: false };
}
const companySkills = companySkillService(db);
const workspaceOperations = workspaceOperationService(db);
const instanceSettings = instanceSettingsService(db);
@@ -5067,8 +5085,24 @@ export function agentRoutes(
const id = req.params.id as string;
const existing = await getAccessibleResource(req, res, svc.getById(id), "Agent not found");
if (!existing) return;
await assertCanReadAgent(req, existing);
assertExternalInstructionsAdmin(req, existing);
if (agentInstructionsBundleMode(existing) === "external") {
await assertCanReadAgent(req, existing);
} else {
await authorizeInstructionRead(db, req.actor, { companyId: existing.companyId, id: existing.id });
}
if (agentInstructionsBundleMode(existing) !== "external") {
const target = { companyId: existing.companyId, agentId: existing.id };
const current = await instructionRevisions.readCurrent(target, req.actor);
if (current) {
try { await instructionRevisions.materializeCurrent(target); }
catch (error) {
const bundle = await instructions.getBundle(existing);
bundle.warnings.push(`Saved instruction revision needs materialization: ${error instanceof Error ? error.message : String(error)}`);
res.json(bundle); return;
}
}
}
res.json(await instructions.getBundle(existing));
});
@@ -5124,8 +5158,12 @@ export function agentRoutes(
const id = req.params.id as string;
const existing = await getAccessibleResource(req, res, svc.getById(id), "Agent not found");
if (!existing) return;
await assertCanReadAgent(req, existing);
assertExternalInstructionsAdmin(req, existing);
if (agentInstructionsBundleMode(existing) === "external") {
await assertCanReadAgent(req, existing);
} else {
await authorizeInstructionRead(db, req.actor, { companyId: existing.companyId, id: existing.id });
}
const relativePath = typeof req.query.path === "string" ? req.query.path : "";
if (!relativePath.trim()) {
@@ -5133,6 +5171,19 @@ export function agentRoutes(
return;
}
if (req.query.download === "true" && agentInstructionsBundleMode(existing) === "managed") {
const download = await agentFiles.download(existing.companyId, existing.id, relativePath, req.actor);
if (download === null) throw notFound("Agent file not found");
res.setHeader("Content-Type", "application/octet-stream");
res.setHeader("X-Content-Type-Options", "nosniff");
res.attachment(relativePath.split("/").at(-1)!);
res.setHeader("Content-Length", download.size);
await pipeline(download.stream, res); return;
}
if (agentInstructionsBundleMode(existing) !== "external" && instructionPath(relativePath) === deriveBundleState(existing).entryFile) {
const snapshot = await instructionRevisions.readCurrent({ companyId: existing.companyId, agentId: existing.id }, req.actor);
if (snapshot) { res.json(instructionFileDetail(snapshot)); return; }
}
res.json(await instructions.readFile(existing, relativePath));
});
@@ -5140,9 +5191,42 @@ export function agentRoutes(
const id = req.params.id as string;
const existing = await getAccessibleResource(req, res, svc.getById(id), "Agent not found");
if (!existing) return;
const entryFile = deriveBundleState(existing).entryFile;
if (instructionPath(req.body.path) === entryFile) {
assertExternalInstructionsAdmin(req, existing);
if (req.body.baseHash !== undefined) req.body.baseRevisionId = req.body.baseHash === null ? null : agentFileTokenFromHash(req.body.baseHash);
if (req.body.baseRevisionId === undefined) throw unprocessable("Read the entry and supply baseRevisionId (null for a new entry)", { code: "INSTRUCTION_BASE_REQUIRED" });
// Clearing legacy prompt configuration remains a protected config change.
if (req.body.clearLegacyPromptTemplate) await assertCanManageInstructionsPath(req, existing);
const receipt = await instructionRevisions.commit({ companyId: existing.companyId, agentId: existing.id,
entryFile, content: req.body.content, baseRevisionId: req.body.baseRevisionId,
source: req.actor.type === "board" ? "board" : "api" }, req.actor);
if (req.actor.type === "agent" && req.actor.runId) {
await instructionWorkingCopies.acknowledgeExplicitSave({ companyId: existing.companyId, agentId: existing.id,
runId: req.actor.runId, entryFile: receipt.revision.entryFile, revisionId: receipt.revision.id,
contentHash: receipt.revision.contentHash }).catch(() => undefined);
}
if (req.body.clearLegacyPromptTemplate) {
const fresh = await svc.getById(existing.id);
if (fresh) {
const adapterConfig = { ...asRecord(fresh.adapterConfig) };
delete adapterConfig.promptTemplate;
delete adapterConfig.bootstrapPromptTemplate;
await svc.update(existing.id, { adapterConfig });
}
}
res.json(instructionFileDetail(receipt, receipt));
return;
}
await assertCanManageInstructionsPath(req, existing);
assertExternalInstructionsAdmin(req, existing);
if (agentInstructionsBundleMode(existing) === "managed" && req.body.path !== "promptTemplate.legacy.md") {
if (req.body.baseHash === undefined) throw unprocessable("Read the file and supply baseHash (null for a new file)");
await agentFiles.write({ companyId: existing.companyId, agentId: existing.id, path: req.body.path,
bytes: Buffer.from(req.body.content, "utf8"), baseHash: req.body.baseHash }, req.actor);
res.json(await instructions.readFile(existing, req.body.path)); return;
}
const actor = getActorInfo(req);
const result = await instructions.writeFile(existing, req.body.path, req.body.content, {
clearLegacyPromptTemplate: req.body.clearLegacyPromptTemplate,
@@ -5184,6 +5268,66 @@ export function agentRoutes(
res.json(result.file);
});
router.get("/agents/:id/instructions-bundle/candidates", async (req, res) => {
const existing = await getAccessibleResource(req, res, svc.getById(req.params.id as string), "Agent not found");
if (!existing) return;
assertExternalInstructionsAdmin(req, existing);
res.json(await instructionWorkingCopies.list(existing.companyId, existing.id, req.actor));
});
router.post("/agents/:id/instructions-bundle/candidates/:runId/resolve", validate(resolveAgentInstructionCandidateSchema), async (req, res) => {
const existing = await getAccessibleResource(req, res, svc.getById(req.params.id as string), "Agent not found");
if (!existing) return;
assertExternalInstructionsAdmin(req, existing);
const runId = req.params.runId as string;
if (!isUuidLike(runId)) throw unprocessable("Invalid instruction candidate run id");
const receipt = await instructionWorkingCopies.resolve({ companyId: existing.companyId, agentId: existing.id,
runId, baseRevisionId: req.body.baseRevisionId, content: req.body.content }, req.actor);
res.json(instructionFileDetail(receipt, receipt));
});
router.get("/agents/:id/instructions-bundle/history", async (req, res) => {
const existing = await getAccessibleResource(req, res, svc.getById(req.params.id as string), "Agent not found");
if (!existing) return;
assertExternalInstructionsAdmin(req, existing);
const entryFile = typeof req.query.path === "string" ? req.query.path : deriveBundleState(existing).entryFile;
const cursor = typeof req.query.cursor === "string" ? req.query.cursor : undefined;
if (cursor && !isUuidLike(cursor)) throw unprocessable("Invalid history cursor");
res.json(await instructionRevisions.history({ companyId: existing.companyId, agentId: existing.id, entryFile, cursor }, req.actor));
});
router.get("/agents/:id/instructions-bundle/revision/:revisionId", async (req, res) => {
const existing = await getAccessibleResource(req, res, svc.getById(req.params.id as string), "Agent not found");
if (!existing) return;
assertExternalInstructionsAdmin(req, existing);
const revisionId = req.params.revisionId as string;
if (!isUuidLike(revisionId)) throw unprocessable("Invalid instruction revision id");
const entryFile = typeof req.query.path === "string" ? req.query.path : deriveBundleState(existing).entryFile;
res.json(await instructionRevisions.readRevision({ companyId: existing.companyId, agentId: existing.id, entryFile, revisionId }, req.actor));
});
router.get("/agents/:id/instructions-bundle/diff", async (req, res) => {
const existing = await getAccessibleResource(req, res, svc.getById(req.params.id as string), "Agent not found");
if (!existing) return;
assertExternalInstructionsAdmin(req, existing);
const fromRevisionId = typeof req.query.from === "string" ? req.query.from : "";
const toRevisionId = typeof req.query.to === "string" ? req.query.to : "";
if (!isUuidLike(fromRevisionId) || !isUuidLike(toRevisionId)) throw unprocessable("Provide valid from and to revision ids");
const entryFile = typeof req.query.path === "string" ? req.query.path : deriveBundleState(existing).entryFile;
res.json(await instructionRevisions.diff({ companyId: existing.companyId, agentId: existing.id, entryFile, fromRevisionId, toRevisionId }, req.actor));
});
router.post("/agents/:id/instructions-bundle/restore", validate(restoreAgentInstructionSchema), async (req, res) => {
const existing = await getAccessibleResource(req, res, svc.getById(req.params.id as string), "Agent not found");
if (!existing) return;
assertExternalInstructionsAdmin(req, existing);
const receipt = await instructionRevisions.restore({ companyId: existing.companyId, agentId: existing.id,
entryFile: req.body.path, baseRevisionId: req.body.baseRevisionId, revisionId: req.body.revisionId }, req.actor);
if (req.actor.type === "agent" && req.actor.runId) {
await instructionWorkingCopies.acknowledgeExplicitSave({ companyId: existing.companyId, agentId: existing.id,
runId: req.actor.runId, entryFile: receipt.revision.entryFile, revisionId: receipt.revision.id,
contentHash: receipt.revision.contentHash }).catch(() => undefined);
}
res.json(instructionFileDetail(receipt, receipt));
});
router.delete("/agents/:id/instructions-bundle/file", async (req, res) => {
const id = req.params.id as string;
const existing = await getAccessibleResource(req, res, svc.getById(id), "Agent not found");
@@ -5197,6 +5341,12 @@ export function agentRoutes(
return;
}
if (agentInstructionsBundleMode(existing) === "managed") {
const baseHash = typeof req.query.baseHash === "string" && /^[a-f0-9]{64}$/.test(req.query.baseHash) ? req.query.baseHash : null;
if (baseHash === null) throw unprocessable("Read the file and supply baseHash before deleting it");
await agentFiles.write({ companyId: existing.companyId, agentId: existing.id, path: relativePath, bytes: null, baseHash }, req.actor);
res.json(await instructions.getBundle(existing)); return;
}
const actor = getActorInfo(req);
const result = await instructions.deleteFile(existing, relativePath);
await logActivity(db, {
+1 -1
View File
@@ -217,7 +217,7 @@ export function builtInAgentRoutes(db: Db) {
const key = req.params.key as string;
await assertBuiltInAgentsEnabled();
await assertCanProvisionBuiltInAgents(req, companyId);
const state = await svc.reset(companyId, key, req.body);
const state = await svc.reset(companyId, key, req.body, req.actor);
await logBuiltInAgentMutation(req, {
companyId,
action: "built_in_agent.reset",
+40 -4
View File
@@ -27,6 +27,8 @@ import {
updateAgentInstructionsPathSchema,
updateAgentInstructionsBundleSchema,
upsertAgentInstructionsFileSchema,
restoreAgentInstructionSchema,
resolveAgentInstructionCandidateSchema,
createAgentKeySchema,
builtInAgentEmptyMutationSchema,
builtInAgentProvisionSchema,
@@ -3455,8 +3457,8 @@ registry.registerPath({
method: "get",
path: "/api/agents/{id}/instructions-bundle/file",
tags: ["agents"],
summary: "Get agent instructions file",
request: { params: z.object({ id: z.string() }) },
summary: "Get agent file content or download its original bytes",
request: { params: z.object({ id: z.string() }), query: z.object({ path: z.string(), download: z.enum(["true", "false"]).optional() }) },
responses: { 200: r.ok(), 401: r.unauthorized, 404: r.notFound },
});
@@ -3472,12 +3474,46 @@ registry.registerPath({
responses: { 200: r.ok(), 400: r.badRequest, 401: r.unauthorized },
});
for (const operation of [
{ suffix: "history", summary: "List immutable instruction revisions", query: z.object({ path: z.string().optional(), cursor: z.string().uuid().optional() }) },
{ suffix: "revision/{revisionId}", summary: "Read exact instruction content at a revision", query: z.object({ path: z.string().optional() }) },
{ suffix: "diff", summary: "Compare instruction revisions (exact common prefix, removed, added, suffix)", query: z.object({ path: z.string().optional(), from: z.string().uuid(), to: z.string().uuid() }) },
]) {
registry.registerPath({ method: "get", path: `/api/agents/{id}/instructions-bundle/${operation.suffix}`, tags: ["agents"], summary: operation.summary,
request: { params: operation.suffix.includes("revisionId") ? z.object({ id: z.string(), revisionId: z.string().uuid() }) : z.object({ id: z.string() }), query: operation.query },
responses: { 200: r.ok(), 401: r.unauthorized, 403: r.forbidden, 404: r.notFound } });
}
registry.registerPath({ method: "post", path: "/api/agents/{id}/instructions-bundle/restore", tags: ["agents"], summary: "Restore a pre-upgrade instruction snapshot into current files with compare-and-swap",
request: { params: z.object({ id: z.string() }), body: jsonBody(restoreAgentInstructionSchema) },
responses: { 200: r.ok(), 401: r.unauthorized, 403: r.forbidden, 404: r.notFound, 409: r.conflict } });
registry.registerPath({
method: "get",
path: "/api/agents/{id}/instructions-bundle/candidates",
tags: ["agents"],
summary: "List preserved instruction edits and collection diagnostics",
request: { params: z.object({ id: z.string() }) },
responses: { 200: r.ok(), 401: r.unauthorized, 403: r.forbidden, 404: r.notFound },
});
registry.registerPath({
method: "post",
path: "/api/agents/{id}/instructions-bundle/candidates/{runId}/resolve",
tags: ["agents"],
summary: "Explicitly save a preserved instruction edit with compare-and-swap",
request: {
params: z.object({ id: z.string(), runId: z.string().uuid() }),
body: jsonBody(resolveAgentInstructionCandidateSchema),
},
responses: { 200: r.ok(), 401: r.unauthorized, 403: r.forbidden, 404: r.notFound, 409: r.conflict, 422: r.unprocessable },
});
registry.registerPath({
method: "delete",
path: "/api/agents/{id}/instructions-bundle/file",
tags: ["agents"],
summary: "Delete agent instructions file",
request: { params: z.object({ id: z.string() }) },
summary: "Delete agent file with compare-and-swap for managed storage",
request: { params: z.object({ id: z.string() }), query: z.object({ path: z.string(), baseHash: z.string().regex(/^[a-f0-9]{64}$/).optional() }) },
responses: { 200: r.ok(), 401: r.unauthorized },
});
@@ -0,0 +1,258 @@
import fs from "node:fs/promises";
import path from "node:path";
import { and, eq } from "drizzle-orm";
import { agents, environmentLeases, environments, agentInstructionWorkingCopies as copies, type Db } from "@paperclipai/db";
import { syncDirectoryToSsh, restoreWorkspaceFromSshExecution } from "@paperclipai/adapter-utils/ssh";
import { prepareAdapterExecutionTargetRuntime, runAdapterExecutionTargetShellCommand, type AdapterExecutionTarget, type PreparedAdapterExecutionTargetRuntime } from "@paperclipai/adapter-utils/execution-target";
import { withDirectoryMergeLock, directorySnapshotSha256, parseDirectorySnapshot, serializeDirectorySnapshot } from "@paperclipai/adapter-utils/workspace-restore-merge";
import { AGENT_FILES_CONTRACT, AgentFileLimitError, agentFileStore, agentStorageWarning, inspectAgentDirectory } from "./agent-file-store.js";
import { agentInstructionsBundleMode, deriveBundleState, resolveManagedInstructionsRoot } from "./agent-instructions.js";
import { instructionGitExcludeProgram } from "./agent-instruction-files.js";
import { resolveInstructionActor } from "./agent-instruction-authorization.js";
import { HttpError, conflict, notFound } from "../errors.js";
import type { AuthorizationActor } from "./authorization.js";
import type { EnvironmentRuntimeService } from "./environment-runtime.js";
import type { Environment, EnvironmentLease } from "@paperclipai/shared";
import { hasRemoteTerminationReceipt } from "./remote-execution-termination.js";
type Copy = typeof copies.$inferSelect;
const completed = new Set(["saved", "unchanged", "resolved", "unavailable"]);
const transports = new Map<string, PreparedAdapterExecutionTargetRuntime>();
const key = (row: Pick<Copy, "companyId" | "runId">) => `${row.companyId}:${row.runId}`;
export function isAgentDirectoryCopy(row: Pick<Copy, "receipt"> | null): boolean {
return row?.receipt?.schema === AGENT_FILES_CONTRACT;
}
function baseline(row: Copy) {
const snapshot = parseDirectorySnapshot(row.receipt?.baseline);
if (!snapshot || directorySnapshotSha256(snapshot) !== row.baseHash) throw new Error("Agent directory baseline is invalid");
return snapshot;
}
function actor(row: Copy): AuthorizationActor {
return { type: "agent", companyId: row.companyId, agentId: row.agentId, runId: row.runId, onBehalfOfUserId: row.responsibleUserId };
}
export function agentDirectoryWorkingCopyService(db: Db, get: (companyId: string, runId: string) => Promise<Copy | null>, patch: (row: Copy, values: Partial<typeof copies.$inferInsert>) => Promise<Copy>, environmentRuntime?: EnvironmentRuntimeService) {
const store = agentFileStore(db);
async function transport(row: Copy, target: AdapterExecutionTarget, recovering: boolean) {
if (target.kind !== "remote" || row.location !== `remote:${target.environmentId ?? ""}`) throw new Error("Agent directory environment changed");
if (recovering && target.transport === "ssh") throw new Error("The original SSH collector is unavailable");
if (target.transport === "ssh") {
// Reuse SSH's plain directory transfer without its task-workspace suffix
// or Git-history discovery. The registered root is the exact writable root.
await syncDirectoryToSsh({ spec: target.spec, localDir: row.localRoot, remoteDir: row.executionRoot, exclude: [".paperclip-runtime"] });
return { target, workspaceRemoteDir: row.executionRoot, runtimeRootDir: null,
assetDirs: {}, additionalSourceDirs: {}, additionalSourceFailures: [], workspaceSyncSnapshot: null,
restoreWorkspace: () => restoreWorkspaceFromSshExecution({ spec: target.spec, localDir: row.localRoot,
remoteDir: row.executionRoot, baselineSnapshot: { ...baseline(row), exclude: [".paperclip-runtime"] }, restoreGitHistory: false }) };
}
return prepareAdapterExecutionTargetRuntime({ target, runId: row.runId, adapterKey: "agent-files",
workspaceLocalDir: row.localRoot, workspaceRemoteDir: row.executionRoot,
syncWorkspace: true, workspaceInboundMode: recovering ? "adopt_remote" : undefined,
workspaceBaseline: baseline(row), workspaceGitSnapshot: null, workspaceFileMode: "all",
workspaceExclude: [".paperclip-runtime", ".paperclip-runtime/**"] });
}
async function prepare(input: { companyId: string; agentId: string; runId: string; target?: AdapterExecutionTarget | null; cwd: string }) {
const [agent] = await db.select().from(agents).where(and(eq(agents.id, input.agentId), eq(agents.companyId, input.companyId)));
if (!agent) throw notFound("Agent not found");
if (agentInstructionsBundleMode(agent) !== "managed") return null;
const bound = await resolveInstructionActor(db, { type: "agent", companyId: input.companyId, agentId: input.agentId, runId: input.runId });
const root = resolveManagedInstructionsRoot(agent);
const localRoot = path.join(path.dirname(root), "file-sync", "runs", input.runId, "live");
// Local copies live outside the task cwd. Remote copies use the reserved,
// excluded runtime area inside the provider's confined workspace. They are
// synchronized independently; provider HOME is never reinterpreted.
const executionRoot = input.target?.kind === "remote"
? path.posix.join(input.target.remoteCwd, ".paperclip-runtime", "agent-files", input.agentId, input.runId)
: localRoot;
const location = input.target?.kind === "remote" ? `remote:${input.target.environmentId ?? ""}` : "local";
let row = await get(input.companyId, input.runId);
if (row && (row.localRoot !== localRoot || row.executionRoot !== executionRoot || row.agentId !== input.agentId || row.location !== location)) throw conflict("Agent directory belongs to a different execution environment");
if (row && !completed.has(row.state) && row.state !== "preparing") {
if (input.target?.kind === "remote" && !transports.has(key(row))) transports.set(key(row), await transport(row, input.target, true));
return row;
}
// Register ownership before creating any bytes, so a crash during the copy
// leaves a recoverable preparing receipt instead of an orphan directory.
const preparing = { entryFile: deriveBundleState(agent).entryFile, baseRevisionId: null, baseHash: "preparing",
localRoot, executionRoot, location, state: "preparing", candidateBase64: null, candidateHash: null,
receipt: { schema: AGENT_FILES_CONTRACT, ...(input.target?.kind === "remote"
? { cleanup: { leaseId: input.target.leaseId ?? null, remoteCwd: input.target.remoteCwd } } : {}) }, processStoppedAt: null, attempts: 0,
nextAttemptAt: null, errorCode: null, errorMessage: null };
if (row) row = await patch(row, preparing);
else {
await db.insert(copies).values({ runId: input.runId, companyId: input.companyId, agentId: input.agentId, responsibleUserId: bound.onBehalfOfUserId!, ...preparing });
row = (await get(input.companyId, input.runId))!;
}
const { snapshot, storageWarning } = await store.locked(input.companyId, input.agentId, bound, false, async (_tx, _agent, canonical) => {
// Storage quotas govern saves, never admission to a future run. Restore
// existing bytes so the agent can work normally and remove excess files.
// Path/link validation remains mandatory even for an over-quota folder.
const inspected = await inspectAgentDirectory(canonical, false);
// A stopped lifecycle has already accounted for all its bytes. No live
// or pending candidate ever enters this replacement path.
await fs.rm(localRoot, { recursive: true, force: true });
await fs.mkdir(path.dirname(localRoot), { recursive: true, mode: 0o700 });
try {
await fs.cp(canonical, localRoot, { recursive: true, preserveTimestamps: true });
return inspected;
} catch (error) {
await fs.rm(path.dirname(localRoot), { recursive: true, force: true });
throw error;
}
}).catch(async error => {
await fs.rm(path.dirname(localRoot), { recursive: true, force: true });
throw error;
});
const values = { entryFile: deriveBundleState(agent).entryFile, baseRevisionId: null, baseHash: directorySnapshotSha256(snapshot),
localRoot, executionRoot, location, state: "preparing", candidateBase64: null, candidateHash: null,
receipt: { ...preparing.receipt, baseline: serializeDirectorySnapshot(snapshot), storageWarning },
processStoppedAt: null, attempts: 0, nextAttemptAt: null, errorCode: null, errorMessage: null };
try {
row = await patch(row, values);
} catch (error) {
await fs.rm(path.dirname(localRoot), { recursive: true, force: true });
throw error;
}
try {
if (input.target?.kind === "remote") {
const quote = (value: string) => `'${value.replaceAll("'", `'"'"'`)}'`;
const excluded = await runAdapterExecutionTargetShellCommand(input.runId, input.target,
`node -e ${quote(instructionGitExcludeProgram)} ${quote(input.target.remoteCwd)}`,
{ cwd: input.target.remoteCwd, env: {}, timeoutSec: 15 });
if (excluded.exitCode !== 0 || excluded.timedOut) throw new Error("Could not exclude agent files from task Git staging");
transports.set(key(row), await transport(row, input.target, false));
}
return await patch(row, { state: "prepared" });
} catch (error) {
// Preparation failed before a provider could start using this copy.
row = await patch(row, { state: "unavailable", processStoppedAt: new Date(), errorCode: "AGENT_FILES_PREPARE_FAILED",
errorMessage: "Agent files could not be staged. The temporary copy was discarded.", nextAttemptAt: null });
await release(row, input.target);
throw error;
}
}
async function retrieve(row: Copy, target?: AdapterExecutionTarget | null) {
if (row.location !== "local") {
let runtime = transports.get(key(row));
if (!runtime) {
if (!target) throw new Error("Agent directory transport is unavailable");
runtime = await transport(row, target, true);
transports.set(key(row), runtime);
}
await runtime.restoreWorkspace();
}
return inspectAgentDirectory(row.localRoot);
}
async function hasChanges(_row: Copy, _target?: AdapterExecutionTarget | null) {
// A whole-directory collector needs a quiescent provider, including its
// child processes. Checkpoint and close at the turn boundary before reading
// either local or remote files. The provider conversation remains resumable;
// ordinary file edits do not change the session configuration digest.
return true;
}
async function collectStopped(row: Copy, target?: AdapterExecutionTarget | null) {
if (completed.has(row.state)) { await release(row); return (await get(row.companyId, row.runId))!; }
row = await patch(row, { processStoppedAt: row.processStoppedAt ?? new Date() });
// The stopped working copy is the only temporary tree. Retry transient I/O
// at this boundary, then clean it up; there are no preserved run snapshots.
let inspected: Awaited<ReturnType<typeof retrieve>> | undefined;
for (;;) {
row = await patch(row, { attempts: row.attempts + 1 });
try {
inspected ??= await retrieve(row, target);
const { snapshot } = inspected;
const candidateHash = directorySnapshotSha256(snapshot);
let storageWarning = inspected.storageWarning;
if (candidateHash === row.baseHash) {
row = await patch(row, { state: "unchanged", errorCode: null, errorMessage: null, nextAttemptAt: null });
} else {
({ storageWarning } = await store.apply({ companyId: row.companyId, agentId: row.agentId, sourceDir: row.localRoot, baseline: baseline(row) }, actor(row)));
row = await patch(row, { state: "saved", candidateHash, errorCode: null, errorMessage: null, nextAttemptAt: null });
}
row = await patch(row, { receipt: { ...row.receipt, storageWarning } });
break;
} catch (error) {
const retryable = !(error instanceof HttpError) || error.status >= 500;
if (retryable && row.attempts < 3) continue;
row = await patch(row, { state: "unavailable", nextAttemptAt: null,
receipt: { ...row.receipt, storageWarning: error instanceof AgentFileLimitError ? agentStorageWarning(error.message) : row.receipt?.storageWarning ?? null },
errorCode: error instanceof AgentFileLimitError ? "AGENT_FILES_LIMIT_EXCEEDED" : "AGENT_FILES_SAVE_FAILED",
errorMessage: error instanceof HttpError && error.status === 422
? `${error.message}. This run's agent-folder changes were not saved; the temporary copy is discarded.`
: "Agent-file synchronization failed. No successful save is claimed; the temporary copy is discarded.",
});
break;
}
}
await release(row);
return (await get(row.companyId, row.runId))!;
}
async function release(row: Copy, target?: AdapterExecutionTarget | null) {
// Collection and environment teardown can both release the same copy.
// A compact successful cleanup receipt is final, even after restart.
if (completed.has(row.state) && row.processStoppedAt && row.receipt?.cleanupPending === false) return;
const runtime = transports.get(key(row));
transports.delete(key(row));
let cleanupPending = false;
await runtime?.cleanupWorkspaceSnapshot?.().catch(() => { cleanupPending = true; });
const cleanupTarget = runtime?.target ?? target;
if (row.processStoppedAt && completed.has(row.state) && cleanupTarget?.kind === "remote") {
const expected = path.posix.join(cleanupTarget.remoteCwd, ".paperclip-runtime", "agent-files", row.agentId, row.runId);
if (row.executionRoot !== expected) throw new Error("Agent directory cleanup path changed");
const quoted = `'${expected.replaceAll("'", `'"'"'`)}'`;
const remoteCleanupFailed = await runAdapterExecutionTargetShellCommand(row.runId, cleanupTarget, `rm -rf -- ${quoted}`,
{ cwd: cleanupTarget.remoteCwd, env: {}, timeoutSec: 15 }).then(result => result.exitCode !== 0 || result.timedOut, () => true);
cleanupPending ||= remoteCleanupFailed;
} else if (row.processStoppedAt && completed.has(row.state) && row.location !== "local") {
// Rebind only the original host-owned lease. Never acquire a replacement
// sandbox or persist transport credentials in a working-copy receipt.
cleanupPending ||= !await cleanupRemoteAfterRestart(row).catch(() => false);
}
if (completed.has(row.state) && row.processStoppedAt) {
try { await fs.rm(path.dirname(row.localRoot), { recursive: true, force: true }); }
catch {
// Leave the baseline marker so restart recovery retries failed cleanup.
await patch(row, { receipt: { ...row.receipt, cleanupPending: true } });
return;
}
await patch(row, { receipt: { schema: AGENT_FILES_CONTRACT, state: row.state, appliedCandidateHash: row.candidateHash, storageWarning: row.receipt?.storageWarning ?? null, cleanupPending,
...(cleanupPending ? { cleanup: row.receipt?.cleanup } : {}) },
nextAttemptAt: cleanupPending ? new Date(Date.now() + 30_000) : null });
}
}
async function cleanupRemoteAfterRestart(row: Copy): Promise<boolean> {
const cleanup = row.receipt?.cleanup as { leaseId?: string; remoteCwd?: string } | undefined;
// Older preview receipts did not record the lease ID. Accept only a unique
// lease still bound to this run and environment in that compatibility case.
const leases = await db.select().from(environmentLeases).where(and(
eq(environmentLeases.companyId, row.companyId),
cleanup?.leaseId ? eq(environmentLeases.id, cleanup.leaseId) : and(
eq(environmentLeases.environmentId, row.location.slice("remote:".length)), eq(environmentLeases.heartbeatRunId, row.runId)),
));
if (leases.length !== 1) return false;
const lease = leases[0]!;
const termination = lease.metadata?.remoteExecutionTermination as { state?: string } | undefined;
if (hasRemoteTerminationReceipt(lease) && termination?.state === "destroyed") return true;
if (!environmentRuntime || !lease.environmentId || row.location !== `remote:${lease.environmentId}`) return false;
const [environment] = await db.select().from(environments).where(eq(environments.id, lease.environmentId));
if (!environment) return false;
const remoteCwd = cleanup?.remoteCwd ?? lease.metadata?.remoteCwd;
if (typeof remoteCwd !== "string" || row.executionRoot !== path.posix.join(remoteCwd, ".paperclip-runtime", "agent-files", row.agentId, row.runId)) return false;
const result = await environmentRuntime.execute({ environment: environment as Environment, lease: lease as EnvironmentLease, command: "rm", args: ["-rf", "--", row.executionRoot],
cwd: remoteCwd, env: {}, timeoutMs: 15_000, bypassSession: true });
return result.exitCode === 0 && !result.timedOut;
}
async function serial<T>(row: Copy, fn: (current: Copy) => Promise<T>): Promise<T> {
// Duplicate stop callbacks and restart recovery must not race while moving
// the stopped working copy. This lock is outside the writable tree.
return withDirectoryMergeLock(path.resolve(row.localRoot, "../../.."), async () => {
const current = await get(row.companyId, row.runId);
if (!current) throw notFound("Agent directory copy not found");
return fn(current);
});
}
return { prepare, hasChanges,
collectStopped: (row: Copy, target?: AdapterExecutionTarget | null) => serial(row, current => collectStopped(current, target)),
release: (row: Copy) => serial(row, release),
};
}
+282
View File
@@ -0,0 +1,282 @@
import { createHash } from "node:crypto";
import fs from "node:fs/promises";
import { constants } from "node:fs";
import { Readable } from "node:stream";
import path from "node:path";
import { and, eq } from "drizzle-orm";
import { agents, activityLog, agentInstructionHeads, agentInstructionRevisions, type Db } from "@paperclipai/db";
import { captureDirectorySnapshot, mergeDirectoryWithBaseline, type DirectorySnapshot } from "@paperclipai/adapter-utils/workspace-restore-merge";
import { HttpError, conflict, notFound, unprocessable } from "../errors.js";
import { authorizeInstructionCommit, authorizeInstructionRead } from "./agent-instruction-authorization.js";
import { assertInstructionPathSafe, instructionPath, instructionBytes, materializeInstructionBytes, readInstructionBytes, MAX_INSTRUCTION_BYTES } from "./agent-instruction-files.js";
import { agentInstructionsBundleMode, deriveBundleState, resolveManagedInstructionsRoot } from "./agent-instructions.js";
import type { AuthorizationActor } from "./authorization.js";
type Tx = Parameters<Parameters<Db["transaction"]>[0]>[0];
type Agent = typeof agents.$inferSelect;
export const AGENT_FILES_CONTRACT = "paperclip.agent-files.v1";
export const MAX_AGENT_FILE_BYTES = 256 * 1024 * 1024;
export const MAX_AGENT_DIRECTORY_BYTES = 2 * 1024 * 1024 * 1024;
export const MAX_AGENT_DIRECTORY_ENTRIES = 100_000;
export const fileHash = (bytes: Uint8Array) => createHash("sha256").update(bytes).digest("hex");
export class AgentFileLimitError extends HttpError {
constructor(message: string) {
super(422, message, { code: "AGENT_FILES_LIMIT_EXCEEDED" });
}
}
function assertFileSize(size: number, relative: string) {
if (size > MAX_AGENT_FILE_BYTES) throw new AgentFileLimitError(`Agent file "${relative}" exceeds the 256 MiB per-file limit`);
}
function assertDirectorySize(size: number, count: number) {
if (size > MAX_AGENT_DIRECTORY_BYTES) throw new AgentFileLimitError("Agent directory exceeds the 2 GiB total storage limit");
if (count > MAX_AGENT_DIRECTORY_ENTRIES) throw new AgentFileLimitError("Agent directory exceeds the 100,000-entry limit (files and folders)");
}
export function agentStorageWarning(detail: string) {
return `Agent storage is full. ${detail}. Runs can continue; remove or shrink files in AGENT_HOME to free space. Changes exceeding the storage limits will not be saved.`;
}
function storageWarning(size: number, count: number, fullFile?: string) {
if (size >= MAX_AGENT_DIRECTORY_BYTES) return agentStorageWarning("The agent folder has reached its 2 GiB limit");
if (count >= MAX_AGENT_DIRECTORY_ENTRIES) return agentStorageWarning("The agent folder has reached its 100,000-entry limit");
if (fullFile) return agentStorageWarning(`Agent file "${fullFile}" has reached its 256 MiB limit`);
return null;
}
export function agentFilePath(value: string): string {
const relative = instructionPath(value);
if (relative.split("/").includes(".paperclip-runtime") || relative === "promptTemplate.legacy.md") {
throw unprocessable(`${relative} is reserved and cannot be used for agent files`);
}
return relative;
}
/** Compatibility ETag for clients whose old schema requires a UUID. This is a
* content token, not a revision ID: no snapshot or history row is created. */
export function agentFileToken(bytes: Uint8Array): string {
return agentFileTokenFromHash(fileHash(bytes));
}
export function agentFileTokenFromHash(h: string): string {
return `${h.slice(0, 8)}-${h.slice(8, 12)}-8${h.slice(13, 16)}-a${h.slice(17, 20)}-${h.slice(20, 32)}`;
}
/** Call under the agent row lock. Import deployed revision heads once, then the
* directory is authoritative. The adoption marker lives OUTSIDE agent files. */
export async function adoptAgentFiles(tx: Tx, agent: Agent): Promise<string> {
if (agentInstructionsBundleMode(agent) === "external") throw unprocessable("External instructions must be migrated to managed storage first");
const root = resolveManagedInstructionsRoot(agent);
const markerRoot = path.join(path.dirname(root), "file-sync");
const marker = "adopted.json";
await assertInstructionPathSafe(root, deriveBundleState(agent).entryFile);
await fs.mkdir(root, { recursive: true, mode: 0o700 });
const filename = await assertInstructionPathSafe(markerRoot, marker);
const adopted = await fs.readFile(filename, "utf8").catch((error: NodeJS.ErrnoException) => {
if (error.code === "ENOENT") return null;
throw error;
});
if (adopted !== null) {
if (JSON.parse(adopted).schema !== AGENT_FILES_CONTRACT) throw new Error("Unsupported agent file contract");
return root;
}
const legacy = await tx.select({ entryFile: agentInstructionHeads.entryFile, contentBase64: agentInstructionRevisions.contentBase64 })
.from(agentInstructionHeads).innerJoin(agentInstructionRevisions, eq(agentInstructionHeads.revisionId, agentInstructionRevisions.id))
.where(and(eq(agentInstructionHeads.companyId, agent.companyId), eq(agentInstructionHeads.agentId, agent.id)));
for (const entry of legacy) await materializeInstructionBytes(root, entry.entryFile, Buffer.from(entry.contentBase64, "base64"));
await materializeInstructionBytes(markerRoot, marker, Buffer.from(JSON.stringify({ schema: AGENT_FILES_CONTRACT })));
return root;
}
async function openAgentFile(root: string, relative: string) {
const filename = await assertInstructionPathSafe(root, relative);
const handle = await fs.open(filename, constants.O_RDONLY | constants.O_NOFOLLOW).catch((error: NodeJS.ErrnoException) => {
if (error.code === "ENOENT") return null;
throw error;
});
if (!handle) return null;
try {
const stat = await handle.stat();
if (!stat.isFile() || stat.nlink !== 1) throw unprocessable("Agent files must be regular files without links");
assertFileSize(stat.size, relative);
return { handle, size: stat.size };
} catch (error) { await handle.close(); throw error; }
}
/** Hash large files incrementally. Only editor-sized content is retained in RAM. */
export async function inspectAgentFile(root: string, relative: string, bufferLimit = MAX_INSTRUCTION_BYTES) {
const opened = await openAgentFile(root, relative);
if (!opened) return null;
const hash = createHash("sha256");
let chunks: Buffer[] | null = opened.size <= bufferLimit ? [] : null;
let size = 0;
try {
for await (const chunk of opened.handle.createReadStream()) {
size += chunk.length;
assertFileSize(size, relative);
hash.update(chunk);
if (size > bufferLimit) chunks = null;
chunks?.push(chunk);
}
return { size, hash: hash.digest("hex"), bytes: chunks ? Buffer.concat(chunks) : null };
} finally { await opened.handle.close(); }
}
export async function readAgentFile(root: string, relative: string): Promise<Buffer | null> {
return (await inspectAgentFile(root, relative, MAX_AGENT_FILE_BYTES))?.bytes ?? null;
}
/** Validate before staging and again after provider stop; never follow links or
* silently skip an unsupported file. Bounds apply to bytes, including binaries. */
async function scanAgentFiles(root: string, enforceLimits = true) {
await assertInstructionPathSafe(root, ".path-check");
let size = 0, count = 0;
const entries = new Set<string>();
let fullFile: string | undefined;
async function walk(dir: string) {
for (const item of await fs.readdir(path.join(root, dir), { withFileTypes: true })) {
const relative = agentFilePath(dir ? `${dir}/${item.name}` : item.name);
const stat = await fs.lstat(path.join(root, relative));
count++;
if (enforceLimits) assertDirectorySize(size, count);
entries.add(relative);
if (stat.isDirectory()) await walk(relative);
else if (stat.isFile() && stat.nlink === 1) {
size += stat.size;
if (stat.size >= MAX_AGENT_FILE_BYTES) fullFile ??= relative;
if (enforceLimits) {
assertFileSize(stat.size, relative);
assertDirectorySize(size, count);
}
} else throw unprocessable("Agent directories support regular files and directories, without links or special files");
}
}
await walk("");
return { size, entries, storageWarning: storageWarning(size, count, fullFile) };
}
export async function inspectAgentDirectory(root: string, enforceLimits = true) {
const usage = await scanAgentFiles(root, enforceLimits);
return { snapshot: await captureDirectorySnapshot(root), storageWarning: usage.storageWarning };
}
export async function snapshotAgentFiles(root: string, enforceLimits = true): Promise<DirectorySnapshot> {
return (await inspectAgentDirectory(root, enforceLimits)).snapshot;
}
export function agentFileStore(db: Db) {
async function locked<T>(companyId: string, agentId: string, actor: AuthorizationActor, write: boolean,
fn: (tx: Tx, agent: Agent, root: string, bound: AuthorizationActor) => Promise<T>) {
return db.transaction(async tx => {
const [agent] = await tx.select().from(agents).where(and(eq(agents.companyId, companyId), eq(agents.id, agentId))).for("update");
if (!agent) throw notFound("Agent not found");
const bound = write ? await authorizeInstructionCommit(tx, actor, agent) : await authorizeInstructionRead(tx, actor, agent);
const root = await adoptAgentFiles(tx, agent);
return fn(tx, agent, root, bound);
});
}
async function audit(tx: Tx, agent: Agent, actor: AuthorizationActor, details: Record<string, unknown>) {
await tx.insert(activityLog).values({ companyId: agent.companyId, actorType: actor.type === "board" ? "user" : "agent",
actorId: (actor.type === "board" ? actor.userId : actor.agentId)!, agentId: actor.type === "agent" ? actor.agentId : null,
runId: actor.runId, responsibleUserId: actor.type === "board" ? actor.userId : actor.onBehalfOfUserId,
action: "agent.files_updated", entityType: "agent", entityId: agent.id, details });
}
return {
locked,
download: async (companyId: string, agentId: string, relative: string, actor: AuthorizationActor) => {
const opened: { file: Awaited<ReturnType<typeof openAgentFile>> } = { file: null };
try {
const file = await locked(companyId, agentId, actor, false, async (_tx, _agent, root) => {
opened.file = await openAgentFile(root, instructionPath(relative));
return opened.file;
});
// Transfer outside the database lock. The open descriptor pins the file
// across concurrent atomic replacements; end bounds concurrent growth.
if (!file) return null;
if (file.size === 0) { await file.handle.close(); return { size: 0, stream: Readable.from([]) }; }
return { size: file.size, stream: file.handle.createReadStream({ end: file.size - 1 }) };
} catch (error) { await opened.file?.handle.close(); throw error; }
},
read: (companyId: string, agentId: string, relative: string, actor: AuthorizationActor) =>
locked(companyId, agentId, actor, false, (_tx, _agent, root) => readAgentFile(root, instructionPath(relative))),
write: (input: { companyId: string; agentId: string; path: string; bytes: Buffer | null; baseHash: string | null }, actor: AuthorizationActor) =>
locked(input.companyId, input.agentId, actor, true, async (tx, agent, root, bound) => {
const relative = agentFilePath(input.path);
if (input.bytes) assertFileSize(input.bytes.length, relative);
const previous = await inspectAgentFile(root, relative);
const currentHash = previous?.hash ?? null;
const incomingHash = input.bytes === null ? null : fileHash(input.bytes);
if (currentHash === incomingHash) return { contentHash: currentHash, changed: false };
if (currentHash !== input.baseHash) throw conflict("This file changed since it was read. Reload before saving.", { code: "AGENT_FILE_CONFLICT", path: relative, currentHash });
if (input.bytes === null && relative === deriveBundleState(agent).entryFile) throw unprocessable("The configured instruction entry cannot be deleted");
if (input.bytes !== null) {
if (relative === deriveBundleState(agent).entryFile) instructionBytes(input.bytes);
// A quota check needs metadata only. Do not hash unrelated large
// files while holding the editor's row lock for a one-file save.
const snapshot = await scanAgentFiles(root);
const total = snapshot.size + input.bytes.length - (previous?.size ?? 0);
const newEntries = relative.split("/").map((_part, i, parts) => parts.slice(0, i + 1).join("/")).filter(name => !snapshot.entries.has(name)).length;
assertDirectorySize(total, snapshot.entries.size + newEntries);
}
if (input.bytes === null) await fs.unlink(await assertInstructionPathSafe(root, relative));
else await materializeInstructionBytes(root, relative, input.bytes);
await audit(tx, agent, bound, { path: relative, contentHash: incomingHash });
return { contentHash: incomingHash, changed: true };
}),
apply: (input: { companyId: string; agentId: string; sourceDir: string; baseline: DirectorySnapshot }, actor: AuthorizationActor) =>
locked(input.companyId, input.agentId, actor, true, async (tx, agent, root, bound) => {
const incoming = await snapshotAgentFiles(input.sourceDir);
const entry = await readInstructionBytes(input.sourceDir, deriveBundleState(agent).entryFile);
if (entry === null) throw unprocessable("The configured instruction entry cannot be deleted");
instructionBytes(entry);
// Previously saved/imported bytes must remain readable, including when
// over quota. Enforce limits on the incoming and resulting tree so a
// run can delete files to recover instead of being locked out forever.
const current = await snapshotAgentFiles(root, false);
// Rebase only the run's changed paths onto the current tree. This makes
// same-file edits/deletions last-sync-wins while untouched files retain
// changes from other runs. Ancestors may need recreating after a writer
// replaced a directory with a file.
const entries = new Map(input.baseline.entries);
const changed = [...new Set([...input.baseline.entries, ...incoming.entries].map(([name]) => name))]
.filter(name => JSON.stringify(input.baseline.entries.get(name)) !== JSON.stringify(incoming.entries.get(name)));
for (const name of changed) {
const present = current.entries.get(name);
if (present) entries.set(name, present); else entries.delete(name);
if (incoming.entries.has(name)) {
for (let parent = path.posix.dirname(name); parent !== "."; parent = path.posix.dirname(parent)) {
if (current.entries.get(parent)?.kind === "dir") continue;
const before = current.entries.get(parent);
if (before) entries.set(parent, before); else entries.delete(parent);
}
}
}
const applyBaseline = { ...input.baseline, entries };
const finalEntries = new Map([...current.entries].map(([name, value]) => [name, { value, root }]));
for (const name of changed) if (!incoming.entries.has(name)) finalEntries.delete(name);
for (const [name, value] of incoming.entries) {
if (JSON.stringify(input.baseline.entries.get(name)) === JSON.stringify(value)) continue;
if (value.kind !== "dir") for (const child of finalEntries.keys()) if (child.startsWith(`${name}/`)) finalEntries.delete(child);
finalEntries.set(name, { value, root: input.sourceDir });
for (let parent = path.posix.dirname(name); parent !== "."; parent = path.posix.dirname(parent)) {
if (finalEntries.get(parent)?.value.kind !== "dir") finalEntries.set(parent, { value: { kind: "dir" }, root: input.sourceDir });
}
}
// An unchanged file created by another run can keep a removed folder
// alive. Count those surviving parent directories in quota preflight.
for (const name of finalEntries.keys()) {
for (let parent = path.posix.dirname(name); parent !== "."; parent = path.posix.dirname(parent)) {
if (!finalEntries.has(parent)) finalEntries.set(parent, { value: { kind: "dir" }, root });
}
}
let total = 0;
let fullFile: string | undefined;
for (const [name, item] of finalEntries) if (item.value.kind === "file") {
const size = (await fs.stat(path.join(item.root, name))).size;
assertFileSize(size, name);
if (size >= MAX_AGENT_FILE_BYTES) fullFile ??= name;
total += size;
}
assertDirectorySize(total, finalEntries.size);
await mergeDirectoryWithBaseline({ ...input, baseline: applyBaseline, targetDir: root });
await audit(tx, agent, bound, { sourceRunId: actor.runId, contract: AGENT_FILES_CONTRACT });
return { storageWarning: storageWarning(total, finalEntries.size, fullFile) };
}),
};
}
@@ -0,0 +1,195 @@
import { and, eq, isNull } from "drizzle-orm";
import {
agentApiKeys,
authUsers,
companyMemberships,
heartbeatRuns,
runIdentityContexts,
type Db,
} from "@paperclipai/db";
import { forbidden } from "../errors.js";
import {
authorizationService,
authorizationDeniedDetails,
type AuthorizationActor,
} from "./authorization.js";
import {
agentInstructionsChangeTargetKey,
changeConsentGateService,
} from "./change-consent-gate.js";
type Connection = Db | Parameters<Parameters<Db["transaction"]>[0]>[0];
/** Accept ONLY a server-authenticated actor, never request/tool JSON. Rebind run/key provenance on every call. */
export async function resolveInstructionActor(
db: Connection,
actor: AuthorizationActor,
): Promise<AuthorizationActor> {
if (actor.type === "board") {
if (!actor.userId)
throw forbidden("Instruction edits require an authenticated user");
// Discard caller membership caches so revoked access is observed.
return {
type: "board",
userId: actor.userId,
source: actor.source,
ignoreInstanceAdmin: actor.ignoreInstanceAdmin,
};
}
if (actor.type !== "agent" || !actor.agentId || !actor.companyId)
throw forbidden("Instruction identity is missing");
let responsibleUserId: string | null = null;
let keyScope = actor.keyScope;
if (actor.keyId) {
const [key] = await db
.select()
.from(agentApiKeys)
.where(
and(
eq(agentApiKeys.id, actor.keyId),
eq(agentApiKeys.companyId, actor.companyId),
eq(agentApiKeys.agentId, actor.agentId),
isNull(agentApiKeys.revokedAt),
),
);
if (!key)
throw forbidden("Instruction API key is missing or revoked", {
code: "INSTRUCTION_IDENTITY_INVALID",
});
responsibleUserId = key.responsibleUserId;
keyScope = key.scopeConfig ?? undefined;
}
if (actor.runId) {
const [run] = await db
.select()
.from(heartbeatRuns)
.where(
and(
eq(heartbeatRuns.id, actor.runId),
eq(heartbeatRuns.companyId, actor.companyId),
eq(heartbeatRuns.agentId, actor.agentId),
),
);
if (!run)
throw forbidden("Instruction run identity does not match", {
code: "INSTRUCTION_IDENTITY_INVALID",
});
if (actor.keyId && responsibleUserId !== run.responsibleUserId)
throw forbidden("Key and run responsible identity do not match", {
code: "INSTRUCTION_IDENTITY_INVALID",
});
responsibleUserId = run.responsibleUserId;
if (run.activeIdentityContextId) {
const [context] = await db
.select()
.from(runIdentityContexts)
.where(
and(
eq(runIdentityContexts.id, run.activeIdentityContextId),
eq(runIdentityContexts.runId, run.id),
eq(runIdentityContexts.companyId, run.companyId),
eq(runIdentityContexts.status, "accepted"),
),
);
if (!context)
throw forbidden("Instruction run identity is unavailable", {
code: "INSTRUCTION_IDENTITY_INVALID",
});
responsibleUserId = context.responsibleUserId;
}
}
if (
!responsibleUserId ||
(actor.onBehalfOfUserId && actor.onBehalfOfUserId !== responsibleUserId)
) {
throw forbidden("A current server-bound responsible user is required", {
code: "INSTRUCTION_IDENTITY_INVALID",
});
}
const [membership] = await db
.select({ userId: authUsers.id })
.from(companyMemberships)
.innerJoin(authUsers, eq(authUsers.id, companyMemberships.principalId))
.where(
and(
eq(companyMemberships.companyId, actor.companyId),
eq(companyMemberships.principalType, "user"),
eq(companyMemberships.principalId, responsibleUserId),
eq(companyMemberships.status, "active"),
),
);
if (!membership)
throw forbidden("The responsible user is unavailable in this company", {
code: "RESPONSIBLE_USER_UNAVAILABLE",
});
return {
type: "agent",
agentId: actor.agentId,
companyId: actor.companyId,
runId: actor.runId,
keyId: actor.keyId,
keyScope,
source: actor.source,
onBehalfOfUserId: responsibleUserId,
};
}
/** Instruction content is narrower than general same-company agent visibility. */
export async function authorizeInstructionRead(
db: Connection,
actor: AuthorizationActor,
target: { companyId: string; id: string },
) {
const bound = await resolveInstructionActor(db, actor);
const access = authorizationService(db);
const resource = { type: "agent" as const, companyId: target.companyId, agentId: target.id };
const peer = bound.type === "agent" && bound.agentId !== target.id;
const decision = await access.decide({
actor: bound,
action: peer ? "agent_config:read" : "agent:read",
resource,
scope: { targetAgentId: target.id },
});
if (!decision.allowed) throw forbidden(decision.explanation, authorizationDeniedDetails(decision));
return bound;
}
/** Current target edit access + agent containment + protected-change consent, shared by all writers. */
export async function authorizeInstructionCommit(
db: Connection,
actor: AuthorizationActor,
target: { companyId: string; id: string },
) {
const bound = await resolveInstructionActor(db, actor);
const access = authorizationService(db);
const input = {
actor: bound,
action: "agent_instructions:update" as const,
resource: {
type: "agent" as const,
companyId: target.companyId,
agentId: target.id,
},
scope: { targetAgentId: target.id, requiresChangeGrant: true },
};
let decision = await access.decide(input);
if (
!decision.allowed &&
decision.reason === "deny_missing_consent" &&
bound.type === "agent"
) {
await changeConsentGateService(db).assertConsented({
companyId: target.companyId,
actorAgentId: bound.agentId,
actorRunId: bound.runId,
targetKeys: [agentInstructionsChangeTargetKey(target.id)],
});
decision = await access.decide({
...input,
scope: { ...input.scope, consentedChange: true },
});
}
if (!decision.allowed)
throw forbidden(decision.explanation, authorizationDeniedDetails(decision));
return bound;
}
@@ -0,0 +1,178 @@
import fs from "node:fs/promises";
import { constants } from "node:fs";
import path from "node:path";
import { randomUUID } from "node:crypto";
import { unprocessable } from "../errors.js";
export const MAX_INSTRUCTION_BYTES = 1024 * 1024;
export function instructionPath(value: string): string {
if (
!value ||
value.length > 512 ||
value.includes("\0") ||
value.includes("\\") ||
path.posix.isAbsolute(value) ||
/^[a-z]:/i.test(value) ||
value.split("/").some((part) => !part || part === "." || part === "..")
) {
throw unprocessable("Use a relative instructions path without traversal", {
code: "INSTRUCTION_PATH_INVALID",
});
}
return value;
}
export function instructionBytes(content: string | Uint8Array): Buffer {
const byteLength =
typeof content === "string"
? Buffer.byteLength(content, "utf8")
: content.byteLength;
if (byteLength > MAX_INSTRUCTION_BYTES) {
throw unprocessable("Instructions exceed the 1 MiB limit", {
code: "INSTRUCTION_CONTENT_INVALID",
});
}
const bytes =
typeof content === "string"
? Buffer.from(content, "utf8")
: Buffer.from(content);
try {
// Preserve BOM; reject invalid UTF-8 and unpaired JS surrogates instead of replacing bytes.
const decoded = new TextDecoder("utf-8", {
fatal: true,
ignoreBOM: true,
}).decode(bytes);
if (typeof content === "string" && decoded !== content)
throw new Error("invalid Unicode");
} catch {
throw unprocessable("Instructions must be valid UTF-8", {
code: "INSTRUCTION_CONTENT_INVALID",
});
}
return bytes;
}
export async function assertInstructionPathSafe(
root: string,
entryFile: string,
) {
instructionPath(entryFile);
let absolute = path.resolve(root, entryFile);
// macOS exposes its system temporary directories through root-owned aliases.
// Resolve only those fixed OS aliases, never a link in operator/agent storage.
if (process.platform === "darwin") {
const alias = absolute.split(path.sep)[1];
if (alias === "var" || alias === "tmp" || alias === "etc") {
const systemPath = `/${alias}`;
const stat = await fs.lstat(systemPath);
if (stat.isSymbolicLink() && stat.uid === 0 && await fs.realpath(systemPath) === `/private/${alias}`) {
absolute = `/private${absolute}`;
}
}
}
let current = path.parse(absolute).root;
const parts = absolute.slice(current.length).split(path.sep);
for (const [i, part] of parts.entries()) {
current = path.join(current, part);
const stat = await fs
.lstat(current)
.catch((error: NodeJS.ErrnoException) => {
if (error.code === "ENOENT") return null;
throw error;
});
if (!stat) continue;
if (
stat.isSymbolicLink() ||
(i < parts.length - 1 ? !stat.isDirectory() : !stat.isFile())
) {
throw unprocessable(
"Instructions require regular files and directories without symlinks",
{ code: "INSTRUCTION_PATH_INVALID" },
);
}
}
return absolute;
}
export async function readInstructionBytes(
root: string,
entryFile: string,
): Promise<Buffer | null> {
const absolute = await assertInstructionPathSafe(root, entryFile);
const file = await fs
.open(absolute, constants.O_RDONLY | constants.O_NOFOLLOW)
.catch((error: NodeJS.ErrnoException) => {
if (error.code === "ENOENT") return null;
throw error;
});
if (!file) return null;
try {
const stat = await file.stat();
if (!stat.isFile() || stat.size > MAX_INSTRUCTION_BYTES) {
throw unprocessable(
"Instructions must be a regular UTF-8 file of at most 1 MiB",
{ code: "INSTRUCTION_CONTENT_INVALID" },
);
}
const buffer = Buffer.alloc(MAX_INSTRUCTION_BYTES + 1);
let length = 0;
while (length < buffer.length) {
const result = await file.read(
buffer,
length,
buffer.length - length,
length,
);
if (result.bytesRead === 0) break;
length += result.bytesRead;
}
return instructionBytes(buffer.subarray(0, length));
} finally {
await file.close();
}
}
/** Call while holding the canonical agent lock. Never writes through a symlink. */
export async function materializeInstructionBytes(
root: string,
entryFile: string,
bytes: Buffer,
) {
const absolute = await assertInstructionPathSafe(root, entryFile);
await fs.mkdir(path.dirname(absolute), { recursive: true });
await assertInstructionPathSafe(root, entryFile);
const temporary = path.join(
path.dirname(absolute),
`.instruction-${randomUUID()}.tmp`,
);
try {
const file = await fs.open(temporary, "wx", 0o600);
try {
await file.writeFile(bytes);
await file.sync();
} finally {
await file.close();
}
await assertInstructionPathSafe(root, entryFile);
await fs.rename(temporary, absolute);
} finally {
await fs.rm(temporary, { force: true });
}
}
// Keep the reserved runtime directory out of Git using a self-ignoring file
// inside that directory. Never write to a gitdir or exclude path derived from
// repository-controlled metadata; legitimate worktrees can keep external gitdirs.
export const instructionGitExcludeProgram = String.raw`
const fs=require('node:fs'),path=require('node:path'),crypto=require('node:crypto');
const cwd=fs.realpathSync(process.argv[1]);
const root=path.join(cwd,'.paperclip-runtime'),filename=path.join(root,'.gitignore');
const stat=(name)=>{try{return fs.lstatSync(name)}catch(e){if(e.code==='ENOENT')return null;throw e}};
const directory=stat(root);
if(directory && (directory.isSymbolicLink() || !directory.isDirectory()))throw Error('Unsafe runtime exclusion directory');
if(!directory)fs.mkdirSync(root,{mode:0o700});
const existing=stat(filename);
if(existing && (existing.isSymbolicLink() || !existing.isFile()))throw Error('Unsafe runtime exclusion file');
const temporary=path.join(root,'.ignore-'+crypto.randomUUID()+'.tmp');
try{fs.writeFileSync(temporary,'*\n',{flag:'wx',mode:0o600});fs.renameSync(temporary,filename)}finally{fs.rmSync(temporary,{force:true})}
`;
@@ -0,0 +1,429 @@
import { createHash } from "node:crypto";
import { adoptAgentFiles, agentFileToken } from "./agent-file-store.js";
import { and, desc, eq, getTableColumns, lt, or, sql } from "drizzle-orm";
import {
activityLog,
agents,
plugins,
pluginManagedResources,
agentInstructionHeads as heads,
agentInstructionRevisions as revisions,
type Db,
} from "@paperclipai/db";
import type {
AgentInstructionCommitReceipt,
AgentInstructionDiff,
AgentInstructionHistory,
AgentInstructionSnapshot,
AgentInstructionSource,
} from "@paperclipai/shared";
import { conflict, forbidden, notFound, unprocessable } from "../errors.js";
import {
agentInstructionsBundleMode,
deriveBundleState,
resolveManagedInstructionsRoot,
} from "./agent-instructions.js";
import {
instructionBytes,
instructionPath,
assertInstructionPathSafe,
readInstructionBytes,
materializeInstructionBytes,
} from "./agent-instruction-files.js";
import {
authorizeInstructionCommit,
authorizeInstructionRead,
} from "./agent-instruction-authorization.js";
import type { AuthorizationActor } from "./authorization.js";
type Tx = Parameters<Parameters<Db["transaction"]>[0]>[0];
type Revision = typeof revisions.$inferSelect;
type PluginResetActor = { type: "plugin"; pluginId: string; pluginKey: string; agentKey: string };
type RevisionActor = AuthorizationActor | PluginResetActor;
const { contentBase64: _contentColumn, ...revisionMetadataColumns } =
getTableColumns(revisions);
export type InstructionTarget = { companyId: string; agentId: string };
export type InstructionCommitInput = InstructionTarget & {
entryFile: string;
content: string | Uint8Array;
baseRevisionId: string | null;
source: Exclude<AgentInstructionSource, "seed" | "restore">;
};
function metadata(row: Omit<Revision, "contentBase64">) {
const { createdAt, source, ...fields } = row;
return {
...fields,
source: source as AgentInstructionSource,
createdAt: createdAt.toISOString(),
};
}
function snapshot(row: Revision): AgentInstructionSnapshot {
const { contentBase64, ...fields } = row;
return {
revision: metadata(fields),
content: Buffer.from(contentBase64, "base64").toString("utf8"),
};
}
function owner(target: InstructionTarget, entryFile: string) {
return and(
eq(revisions.companyId, target.companyId),
eq(revisions.agentId, target.agentId),
eq(revisions.entryFile, entryFile),
);
}
function headOwner(target: InstructionTarget, entryFile: string) {
return and(
eq(heads.companyId, target.companyId),
eq(heads.agentId, target.agentId),
eq(heads.entryFile, entryFile),
);
}
export function agentInstructionRevisionService(db: Db) {
async function lockTarget(
tx: Tx,
target: InstructionTarget,
actor?: AuthorizationActor,
) {
if (actor?.type === "agent" && actor.companyId !== target.companyId)
throw notFound("Agent not found");
const [agent] = await tx
.select()
.from(agents)
.where(
and(
eq(agents.id, target.agentId),
eq(agents.companyId, target.companyId),
),
)
.for("update");
if (!agent) throw notFound("Agent not found");
if (agentInstructionsBundleMode(agent) === "external") {
throw unprocessable(
"External host instructions do not support revision write-back. Ask an instance administrator to migrate this bundle to managed storage.",
{ code: "INSTRUCTION_MANAGED_BUNDLE_REQUIRED" },
);
}
const state = deriveBundleState(agent);
const entryFile = instructionPath(state.entryFile);
return { agent, entryFile, root: resolveManagedInstructionsRoot(agent) };
}
async function authorizeRead(
tx: Tx,
actor: AuthorizationActor,
target: InstructionTarget,
) {
return authorizeInstructionRead(tx, actor, { companyId: target.companyId, id: target.agentId });
}
// Trusted host-only reset authority. No HTTP or agent tool accepts this actor.
async function authorizePluginReset(tx: Tx, actor: PluginResetActor, target: InstructionTarget, agent: typeof agents.$inferSelect) {
const [plugin] = await tx.select().from(plugins).where(and(eq(plugins.id, actor.pluginId), eq(plugins.pluginKey, actor.pluginKey)));
const marker = agent.metadata?.paperclipManagedResource as Record<string, unknown> | undefined;
const [binding] = await tx.select().from(pluginManagedResources).where(and(
eq(pluginManagedResources.pluginId, actor.pluginId), eq(pluginManagedResources.pluginKey, actor.pluginKey),
eq(pluginManagedResources.companyId, target.companyId), eq(pluginManagedResources.resourceKind, "agent"),
eq(pluginManagedResources.resourceKey, actor.agentKey), eq(pluginManagedResources.resourceId, target.agentId),
));
if (!plugin || plugin.status !== "ready" || !plugin.manifestJson.capabilities.includes("agents.managed")
|| !plugin.manifestJson.agents?.some((declaration) => declaration.agentKey === actor.agentKey)
|| !binding || marker?.pluginId !== actor.pluginId || marker.pluginKey !== actor.pluginKey
|| marker.resourceKind !== "agent" || marker.resourceKey !== actor.agentKey) {
throw forbidden("Plugin reset is limited to its declared, bound managed agent");
}
const declared = plugin.manifestJson.agents!.find((declaration) => declaration.agentKey === actor.agentKey)!;
return { ...actor, declaredEntryFile: instructionPath(declared.instructions?.entryFile ?? "AGENTS.md") };
}
async function readForPluginReset(target: InstructionTarget, actor: PluginResetActor) {
return db.transaction(async (tx) => {
const state = await lockTarget(tx, target);
const bound = await authorizePluginReset(tx, actor, target, state.agent);
// Preserve the formerly configured entry before an explicit stock reset.
await currentFile(tx, target, state, bound);
const row = await currentFile(tx, target, { ...state, entryFile: bound.declaredEntryFile }, bound);
return { snapshot: row ? snapshot(row) : null, configuredEntryFile: state.entryFile };
});
}
async function head(tx: Tx, target: InstructionTarget, entryFile: string) {
const [result] = await tx
.select({ revision: revisions })
.from(heads)
.innerJoin(revisions, eq(revisions.id, heads.revisionId))
.where(headOwner(target, entryFile));
return result?.revision ?? null;
}
// The deployed revision tables are read-only upgrade input. Current bytes
// live in the agent directory; old UUID clients receive a content ETag.
async function currentFile(tx: Tx, target: InstructionTarget, state: Awaited<ReturnType<typeof lockTarget>>, actor?: RevisionActor) {
await adoptAgentFiles(tx, state.agent);
const bytes = await readInstructionBytes(state.root, state.entryFile);
if (bytes === null) return null;
return currentRow(target, state.entryFile, bytes, actor);
}
function currentRow(target: InstructionTarget, entryFile: string, bytes: Buffer, actor?: RevisionActor): Revision {
return { ...target, id: agentFileToken(bytes), entryFile, contentBase64: bytes.toString("base64"),
contentHash: createHash("sha256").update(bytes).digest("hex"), byteLength: bytes.length,
parentRevisionId: null, baseRevisionId: null, restoredFromRevisionId: null, source: "api",
actorAgentId: actor?.type === "agent" ? actor.agentId ?? null : null,
actorUserId: actor?.type === "board" ? actor.userId ?? null : null,
responsibleUserId: actor?.type === "board" ? actor.userId ?? null : actor?.type === "agent" ? actor.onBehalfOfUserId ?? null : null,
sourceRunId: actor && actor.type !== "plugin" ? actor.runId ?? null : null, createdAt: new Date(0) };
}
async function getRevision(
tx: Tx,
target: InstructionTarget,
entryFile: string,
id: string,
) {
const [row] = await tx
.select()
.from(revisions)
.where(and(owner(target, entryFile), eq(revisions.id, id)));
if (!row) throw notFound("Instruction revision not found");
return row;
}
async function readCurrent(
target: InstructionTarget,
actor: AuthorizationActor,
): Promise<AgentInstructionSnapshot | null> {
return db.transaction(async (tx) => {
const state = await lockTarget(tx, target, actor);
const bound = await authorizeRead(tx, actor, target);
const row = await currentFile(tx, target, state, bound);
return row ? snapshot(row) : null;
});
}
/** Trusted orchestration read for the already authorized run target. This
* never seeds or writes, and is not exposed through HTTP or model tools. */
async function readCommittedForRuntime(target: InstructionTarget): Promise<AgentInstructionSnapshot | null> {
return db.transaction(async (tx) => {
const state = await lockTarget(tx, target);
const current = await currentFile(tx, target, state);
return current ? snapshot(current) : null;
});
}
/** Rebuild disk from the current committed head under the same lock as commits. Safe after restart. */
async function materializeCurrent(target: InstructionTarget): Promise<void> {
await db.transaction(async (tx) => {
const state = await lockTarget(tx, target);
await adoptAgentFiles(tx, state.agent);
});
}
async function commitInternal(
input:
| InstructionCommitInput
| (Omit<InstructionCommitInput, "content" | "source"> & {
restoreRevisionId: string;
}),
actor: RevisionActor,
configuredEntryFile?: string,
): Promise<AgentInstructionCommitReceipt> {
instructionPath(input.entryFile);
const bytes = "content" in input ? instructionBytes(input.content) : null;
const result = await db.transaction(async (tx) => {
const state = await lockTarget(tx, input, actor.type === "plugin" ? undefined : actor);
const bound = actor.type === "plugin"
? await authorizePluginReset(tx, actor, input, state.agent)
: await authorizeInstructionCommit(tx, actor, state.agent);
if (bound.type === "plugin") {
if (input.entryFile !== bound.declaredEntryFile) throw forbidden("Plugin reset must restore its declared entry file");
if (configuredEntryFile !== state.entryFile) throw conflict("Configured instruction entry changed; read the current entry and retry", { code: "INSTRUCTION_ENTRY_CHANGED", entryFile: state.entryFile });
state.entryFile = input.entryFile;
}
if (state.entryFile !== input.entryFile)
throw conflict(
"Configured instruction entry changed; read the current entry and retry",
{ code: "INSTRUCTION_ENTRY_CHANGED", entryFile: state.entryFile },
);
await assertInstructionPathSafe(state.root, state.entryFile);
const current = await currentFile(tx, input, state, bound);
const restored =
"restoreRevisionId" in input
? await getRevision(
tx,
input,
input.entryFile,
input.restoreRevisionId,
)
: null;
const candidate = bytes ?? Buffer.from(restored!.contentBase64, "base64");
const bindEntry = async () => {
if (
!state.agent.adapterConfig.instructionsRootPath ||
!state.agent.adapterConfig.instructionsBundleMode ||
(bound.type === "plugin" && configuredEntryFile !== input.entryFile)
) {
await tx
.update(agents)
.set({
adapterConfig: {
...state.agent.adapterConfig,
instructionsBundleMode: "managed",
instructionsRootPath: state.root,
instructionsEntryFile: state.entryFile,
instructionsFilePath: `${state.root}/${state.entryFile}`,
},
updatedAt: new Date(),
})
.where(eq(agents.id, state.agent.id));
}
};
// An exact replay can safely return the durable receipt even after its base
// advanced. Restore also deduplicates identical content, preserving history.
if (current && current.contentBase64 === candidate.toString("base64")) {
await bindEntry();
return { row: current, changed: false };
}
// A pre-upgrade run can still pin a historical UUID. Compare its bytes,
// never let the old head project over a newer directory.
const legacyBase = input.baseRevisionId && current?.id !== input.baseRevisionId
? await tx.select().from(revisions).where(and(owner(input, input.entryFile), eq(revisions.id, input.baseRevisionId))).then(rows => rows[0])
: null;
if ((current?.id ?? null) !== input.baseRevisionId && !(legacyBase && legacyBase.contentHash === current?.contentHash)) {
throw conflict(
"Instructions changed since the base revision. Read the current entry before saving.",
{
code: "INSTRUCTION_REVISION_CONFLICT",
baseRevisionId: input.baseRevisionId,
currentRevisionId: current?.id ?? null,
},
);
}
await materializeInstructionBytes(state.root, state.entryFile, candidate);
const row = currentRow(input, input.entryFile, candidate, bound);
await tx.insert(activityLog).values({ companyId: input.companyId,
actorType: bound.type === "plugin" ? "plugin" : bound.type === "board" ? "user" : "agent",
actorId: (bound.type === "plugin" ? bound.pluginId : bound.type === "board" ? bound.userId : bound.agentId)!,
agentId: bound.type === "agent" ? bound.agentId : null, runId: row.sourceRunId,
responsibleUserId: row.responsibleUserId, action: "agent.files_updated", entityType: "agent", entityId: input.agentId,
details: { path: input.entryFile, contentHash: row.contentHash, source: restored ? "legacy_restore" : (input as InstructionCommitInput).source } });
await bindEntry();
return { row, changed: true };
});
return { ...snapshot(result.row), changed: result.changed, materialization: "current" };
}
async function history(
target: InstructionTarget & {
entryFile: string;
cursor?: string;
limit?: number;
},
actor: AuthorizationActor,
): Promise<AgentInstructionHistory> {
const limit = Math.max(1, Math.min(100, target.limit ?? 50));
return db.transaction(async (tx) => {
const state = await lockTarget(tx, target, actor);
const bound = await authorizeRead(tx, actor, target);
await currentFile(tx, target, state, bound);
instructionPath(target.entryFile);
const cursor = target.cursor
? await getRevision(tx, target, target.entryFile, target.cursor)
: null;
// Keep PostgreSQL timestamp precision; JS Date truncates microseconds and
// would skip revisions in the cursor's fractional millisecond.
const cursorTime = cursor
? sql`(select created_at from agent_instruction_revisions where id = ${cursor.id})`
: null;
const rows = await tx
.select(revisionMetadataColumns)
.from(revisions)
.where(
and(
owner(target, target.entryFile),
cursor && cursorTime
? or(
lt(revisions.createdAt, cursorTime),
and(
eq(revisions.createdAt, cursorTime),
lt(revisions.id, cursor.id),
),
)
: undefined,
),
)
.orderBy(desc(revisions.createdAt), desc(revisions.id))
.limit(limit + 1);
return {
revisions: rows.slice(0, limit).map(metadata),
nextCursor: rows.length > limit ? rows[limit - 1]!.id : null,
};
});
}
async function readRevision(
target: InstructionTarget & { entryFile: string; revisionId: string },
actor: AuthorizationActor,
) {
return db.transaction(async (tx) => {
await lockTarget(tx, target, actor);
await authorizeRead(tx, actor, target);
return snapshot(
await getRevision(
tx,
target,
instructionPath(target.entryFile),
target.revisionId,
),
);
});
}
async function diff(
target: InstructionTarget & {
entryFile: string;
fromRevisionId: string;
toRevisionId: string;
},
actor: AuthorizationActor,
): Promise<AgentInstructionDiff> {
const from = await readRevision(
{ ...target, revisionId: target.fromRevisionId },
actor,
);
const to = await readRevision(
{ ...target, revisionId: target.toRevisionId },
actor,
);
// Linear, bounded exact replacement diff. Code points keep surrogate pairs intact.
const a = Array.from(from.content),
b = Array.from(to.content);
let start = 0,
end = 0;
while (start < a.length && start < b.length && a[start] === b[start])
start++;
while (
end < a.length - start &&
end < b.length - start &&
a[a.length - end - 1] === b[b.length - end - 1]
)
end++;
return {
from,
to,
prefix: a.slice(0, start).join(""),
removed: a.slice(start, a.length - end).join(""),
added: b.slice(start, b.length - end).join(""),
suffix: end ? a.slice(-end).join("") : "",
};
}
return {
readCommittedForRuntime,
readCurrent,
readForPluginReset,
commitPluginReset: (input: Omit<InstructionCommitInput, "source"> & { configuredEntryFile: string }, actor: PluginResetActor) =>
commitInternal({ ...input, source: "api" }, actor, input.configuredEntryFile),
readRevision,
history,
diff,
materializeCurrent,
commit: (input: InstructionCommitInput, actor: AuthorizationActor) =>
commitInternal(input, actor),
restore: (
input: InstructionTarget & {
entryFile: string;
baseRevisionId: string;
revisionId: string;
},
actor: AuthorizationActor,
) =>
commitInternal({ ...input, restoreRevisionId: input.revisionId }, actor),
};
}
@@ -0,0 +1,356 @@
import { agentDirectoryWorkingCopyService, isAgentDirectoryCopy } from "./agent-directory-working-copies.js";
import { createHash } from "node:crypto";
import { execFile as execFileCallback } from "node:child_process";
import { promisify } from "node:util";
import fs from "node:fs/promises";
import path from "node:path";
import { and, asc, desc, eq, inArray, lte, or, isNull, isNotNull, sql } from "drizzle-orm";
import { agents, heartbeatRuns, agentInstructionWorkingCopies as copies, type Db } from "@paperclipai/db";
import {
prepareAdapterExecutionTargetRuntime,
runAdapterExecutionTargetShellCommand,
type AdapterExecutionTarget,
} from "@paperclipai/adapter-utils/execution-target";
import { conflict, notFound } from "../errors.js";
import { agentInstructionsService, agentInstructionsBundleMode } from "./agent-instructions.js";
import { agentInstructionRevisionService } from "./agent-instruction-revisions.js";
import { resolveInstructionActor } from "./agent-instruction-authorization.js";
import { assertInstructionPathSafe, instructionBytes, instructionPath, materializeInstructionBytes, MAX_INSTRUCTION_BYTES, readInstructionBytes, instructionGitExcludeProgram } from "./agent-instruction-files.js";
import { hasNativeLocalProcessStop } from "./native-local-process-stop.js";
import { remoteExecutionHasStopped } from "./remote-execution-termination.js";
import type { AuthorizationActor } from "./authorization.js";
import type { EnvironmentRuntimeService } from "./environment-runtime.js";
const execFile = promisify(execFileCallback);
type Copy = typeof copies.$inferSelect;
const hash = (bytes: Uint8Array) => createHash("sha256").update(bytes).digest("hex");
const quote = (value: string) => `'${value.replaceAll("'", `'"'"'`)}'`;
const completed = new Set(["saved", "unchanged", "resolved"]);
const MAX_COLLECTION_ATTEMPTS = 3;
// Only orchestration can register a live target; tools and HTTP callers cannot
// supply one. These handles are optional: durable captured bytes survive restart.
const liveTargets = new Map<string, AdapterExecutionTarget>();
const targetKey = (companyId: string, runId: string) => `${companyId}:${runId}`;
export function instructionWorkingCopyGuidance(copy: Pick<Copy, "executionRoot" | "entryFile" | "receipt">) {
if (isAgentDirectoryCopy(copy)) return `Your persistent agent directory is ${copy.executionRoot} (AGENT_HOME). Your instruction entry is ${copy.executionRoot}/${copy.entryFile}. Read and write your own files and subfolders there. This directory belongs to this agent across tasks and sessions; task files belong in the task working directory. Paperclip restores this directory before execution and saves changes after the provider stops. Regular files, including binary files, persist; symlinks and special files are unsupported. Check the agent-files save receipt before claiming persistence; Only files you change or delete are synchronized. If another run changes the same file, the last completed synchronization wins. Temporary copies are removed after synchronization; there is no per-run file history. Storage allows 256 MiB per file, 2 GiB total, and 100,000 entries; the instruction entry must remain UTF-8 and at most 1 MiB. Reaching a storage limit never prevents this or future tasks from running. Remove or shrink files to free space; changes that exceed the limits will not be saved.${typeof copy.receipt?.storageWarning === "string" ? `\n\n${copy.receipt.storageWarning}` : ""}`;
return `Your editable agent instruction file is ${copy.executionRoot}/${copy.entryFile}. Edit this registered private copy normally. After this run stops, Paperclip saves changed content as a persistent revision if your responsible user still has permission and the baseline has not changed. Check the run's instruction-save receipt before claiming persistence. Use read_agent_instructions, update_agent_instructions, get_agent_instruction_history, and restore_agent_instructions for immediate saves and history. Read first and pin the returned revision. Preserve conflicts; never silently retry against a newer head. Repository instructions, skills, and the loaded prompt are separate and are not collected.`;
}
/** Remote reads use the registered entry only; never scan for files called AGENTS.md. */
export function instructionCollectionScript(root: string, entry: string) {
instructionPath(entry);
return `node -e ${quote([
"const fs=require('node:fs'),path=require('node:path')",
"const root=process.argv[1],entry=process.argv[2],max=Number(process.argv[3])",
"const absolute=path.resolve(root,entry),parts=absolute.split('/').filter(Boolean)",
"let current='/'",
"for(let i=0;i<parts.length;i++){current=path.join(current,parts[i]);let s;try{s=fs.lstatSync(current)}catch(e){if(e.code==='ENOENT'){process.stdout.write(JSON.stringify({missing:true}));process.exit(0)}throw e}if(s.isSymbolicLink()||(i<parts.length-1?!s.isDirectory():!s.isFile()))throw Error('INSTRUCTION_PATH_INVALID')}",
"const fd=fs.openSync(absolute,fs.constants.O_RDONLY|fs.constants.O_NOFOLLOW)",
"try{const s=fs.fstatSync(fd);if(!s.isFile()||s.size>max)throw Error('INSTRUCTION_CONTENT_INVALID');const bytes=Buffer.alloc(max+1);let n=0;while(n<bytes.length){const count=fs.readSync(fd,bytes,n,bytes.length-n,n);if(!count)break;n+=count}if(n>max)throw Error('INSTRUCTION_CONTENT_INVALID');process.stdout.write(JSON.stringify({contentBase64:bytes.subarray(0,n).toString('base64')}))}finally{fs.closeSync(fd)}",
].join(";"))} ${quote(root)} ${quote(entry)} ${MAX_INSTRUCTION_BYTES}`;
}
export function agentInstructionWorkingCopyService(db: Db, options: { environmentRuntime?: EnvironmentRuntimeService } = {}) {
const revisions = agentInstructionRevisionService(db);
const scope = (companyId: string, runId: string) => and(eq(copies.companyId, companyId), eq(copies.runId, runId));
async function get(companyId: string, runId: string) {
const [row] = await db.select().from(copies).where(scope(companyId, runId));
return row ?? null;
}
async function patch(row: Copy, values: Partial<typeof copies.$inferInsert>) {
const [updated] = await db.update(copies).set({ ...values, updatedAt: new Date() }).where(and(
scope(row.companyId, row.runId), eq(copies.state, row.state), eq(copies.attempts, row.attempts), eq(copies.baseHash, row.baseHash),
)).returning();
// A late cleanup must not overwrite an explicit resolution or a newer
// baseline. Canonical content has its own independent head CAS.
return updated ?? (await get(row.companyId, row.runId))!;
}
function actorFor(row: Copy): AuthorizationActor {
return { type: "agent", companyId: row.companyId, agentId: row.agentId, runId: row.runId, onBehalfOfUserId: row.responsibleUserId };
}
const directories = agentDirectoryWorkingCopyService(db, get, patch, options.environmentRuntime);
async function prepare(input: { companyId: string; agentId: string; runId: string; target?: AdapterExecutionTarget | null; cwd: string; legacy?: boolean }) {
const existing = await get(input.companyId, input.runId);
if (isAgentDirectoryCopy(existing) || (!existing && !input.legacy)) return directories.prepare(input);
let refreshStoppedCopy = false;
const workspace = await fs.realpath(input.cwd);
const expectedLocalRoot = path.join(workspace, ".paperclip-runtime", `instruction-edits-${input.runId}`, "instructions");
if (existing && existing.localRoot !== expectedLocalRoot) throw conflict("The registered instruction copy belongs to a different run workspace");
const location = input.target?.kind === "remote" ? `remote:${input.target.environmentId ?? ""}` : "local";
if (existing && (existing.agentId !== input.agentId || existing.location !== location)) {
throw conflict("The registered instruction copy belongs to a different run environment");
}
// Preserve in-flight bytes and their CAS baseline. A completed, stopped copy
// can start a fresh lifecycle only after its recorded bytes are accounted for.
if (existing && existing.state !== "preparing") {
if (["conflict", "pending_commit", "unavailable"].includes(existing.state)) {
throw conflict("Resolve this run's preserved instruction candidate before editing its working copy again");
}
if (input.target) liveTargets.set(targetKey(input.companyId, input.runId), input.target);
if (completed.has(existing.state) && existing.processStoppedAt) {
const privateBytes = await readCandidate(existing, input.target);
// Refresh only bytes whose previous lifecycle is complete. A changed
// private file must keep its old CAS fence; never silently rebase edits.
refreshStoppedCopy = privateBytes === null || hash(privateBytes) === (existing.candidateHash ?? existing.baseHash);
if (!refreshStoppedCopy && privateBytes !== null) {
// A failed staging attempt can already have replaced the file before
// the completed row advances. Exact current canonical bytes are also
// accounted for; any other private edit still keeps its old fence.
const current = await revisions.readCommittedForRuntime({ companyId: input.companyId, agentId: input.agentId });
refreshStoppedCopy = current?.revision.entryFile === existing.entryFile
&& current.revision.contentHash === hash(privateBytes);
}
}
if (!refreshStoppedCopy) {
if (completed.has(existing.state) || existing.state === "unchanged_turn") {
// unchanged_turn is a live warm-owner observation, not stop proof.
return patch(existing, { state: "prepared", candidateBase64: null, candidateHash: null,
receipt: null, processStoppedAt: null, attempts: 0, nextAttemptAt: null });
}
return existing;
}
}
const [agent] = await db.select().from(agents).where(and(eq(agents.id, input.agentId), eq(agents.companyId, input.companyId)));
if (!agent) throw notFound("Agent not found");
if (agentInstructionsBundleMode(agent) !== "managed") return null;
const bound = await resolveInstructionActor(db, { type: "agent", companyId: input.companyId, agentId: input.agentId, runId: input.runId });
const baseline = existing?.baseRevisionId && !refreshStoppedCopy
? await revisions.readRevision({ companyId: input.companyId, agentId: input.agentId, entryFile: existing.entryFile, revisionId: existing.baseRevisionId }, bound).catch(async (error) => {
if ((error as { status?: number }).status !== 404) throw error;
const current = await revisions.readCurrent({ companyId: input.companyId, agentId: input.agentId }, bound);
if (current?.revision.id !== existing.baseRevisionId) throw conflict("The instruction baseline changed during preparation");
return current;
})
: await revisions.readCurrent({ companyId: input.companyId, agentId: input.agentId }, bound);
if (!baseline) return null;
const exported = await agentInstructionsService().exportFiles(agent, { rejectSymlinks: true });
const entryFile = instructionPath(baseline?.revision.entryFile ?? exported.entryFile);
const content = instructionBytes(baseline?.content ?? exported.files[entryFile] ?? "");
const localRoot = existing?.localRoot ?? expectedLocalRoot;
const target = input.target?.kind === "remote" ? input.target : null;
const executionRoot = target
? path.posix.join(target.remoteCwd, ".paperclip-runtime", `instruction-edits-${input.runId}`, "instructions")
: localRoot;
if (!existing) {
await db.insert(copies).values({
runId: input.runId, companyId: input.companyId, agentId: input.agentId,
responsibleUserId: bound.onBehalfOfUserId!, entryFile,
baseRevisionId: baseline?.revision.id ?? null, baseHash: hash(content),
localRoot, executionRoot, location, state: "preparing",
});
}
const row = (await get(input.companyId, input.runId))!;
await assertInstructionPathSafe(localRoot, entryFile);
await execFile(process.execPath, ["-e", instructionGitExcludeProgram, workspace], { timeout: 15_000 });
await fs.mkdir(localRoot, { recursive: true, mode: 0o700 });
for (const [name, text] of Object.entries({ ...exported.files, [entryFile]: content.toString("utf8") })) {
const relative = instructionPath(name);
const filename = await assertInstructionPathSafe(localRoot, relative);
await fs.mkdir(path.dirname(filename), { recursive: true, mode: 0o700 });
// Atomic replacement also allows interrupted preparation to restage an
// already-read-only sibling without writing through its existing inode.
await materializeInstructionBytes(localRoot, relative, instructionBytes(text));
await fs.chmod(filename, name === entryFile ? 0o600 : 0o400);
}
if (target) {
const staged = await prepareAdapterExecutionTargetRuntime({
target, runId: input.runId, adapterKey: `instruction-edits-${input.runId}`,
workspaceLocalDir: input.cwd, syncWorkspace: false,
assets: [{ key: "instructions", localDir: localRoot, followSymlinks: false }],
});
if (staged.assetDirs.instructions !== executionRoot) throw new Error("Instruction copy staging path changed");
const excluded = await runAdapterExecutionTargetShellCommand(input.runId, target,
`node -e ${quote(instructionGitExcludeProgram)} ${quote(target.remoteCwd)}`, { cwd: target.remoteCwd, env: {}, timeoutSec: 15 });
if (excluded.exitCode !== 0 || excluded.timedOut) throw new Error("Could not exclude private instructions from Git staging");
}
// A refresh does not supersede the completed turn until every local and
// remote staging step succeeds. Keep its receipt and stop evidence intact
// if preparation throws or the controller stops midway through staging.
const prepared = await patch(row, { state: "prepared", ...(refreshStoppedCopy ? {
entryFile, baseRevisionId: baseline.revision.id, baseHash: hash(content),
candidateBase64: null, candidateHash: null, receipt: null, processStoppedAt: null,
attempts: 0, nextAttemptAt: null, errorCode: null, errorMessage: null,
} : {}) });
if (input.target) liveTargets.set(targetKey(input.companyId, input.runId), input.target);
return prepared;
}
async function readCandidate(row: Copy, target?: AdapterExecutionTarget | null) {
if (row.location === "local") return readInstructionBytes(row.localRoot, row.entryFile);
if (target?.kind !== "remote") throw new Error("The original execution environment is needed to retrieve this instruction copy");
if (row.location !== `remote:${target.environmentId ?? ""}`) throw new Error("Instruction copy execution environment changed");
const expected = path.posix.join(target.remoteCwd, ".paperclip-runtime", `instruction-edits-${row.runId}`, "instructions");
if (expected !== row.executionRoot) throw new Error("Instruction copy execution environment changed");
const result = await runAdapterExecutionTargetShellCommand(row.runId, target, instructionCollectionScript(row.executionRoot, row.entryFile), { cwd: target.remoteCwd, env: {}, timeoutSec: 30 });
if (result.exitCode !== 0 || result.timedOut) throw new Error("Could not safely retrieve the stopped run's instruction file");
const payload = JSON.parse(result.stdout) as { missing?: boolean; contentBase64?: string };
if (payload.missing) return null;
if (typeof payload.contentBase64 !== "string" || payload.contentBase64.length > Math.ceil(MAX_INSTRUCTION_BYTES / 3) * 4) throw new Error("Instruction response exceeds the collection bound");
return instructionBytes(Buffer.from(payload.contentBase64, "base64"));
}
/** A terminal-turn probe is not a save: changed/invalid copies require a
* stopped provider before capture. Unchanged warm providers remain reusable. */
async function hasChanges(input: { companyId: string; runId: string; target?: AdapterExecutionTarget | null }) {
const row = await get(input.companyId, input.runId);
if (!row || completed.has(row.state)) return false;
if (isAgentDirectoryCopy(row)) return directories.hasChanges(row, input.target);
try {
const content = await readCandidate(row, input.target);
if (content !== null && hash(content) === row.baseHash) {
// This is an unchanged turn receipt, not stopped-process collection.
// No candidate bytes or new revision are persisted from a live owner.
await patch(row, { state: "unchanged_turn", nextAttemptAt: null });
return false;
}
return true;
} catch { return true; }
}
async function acknowledgeExplicitSave(input: { companyId: string; agentId: string; runId: string; entryFile: string; revisionId: string; contentHash: string }) {
const row = await get(input.companyId, input.runId);
if (!row || isAgentDirectoryCopy(row) || row.agentId !== input.agentId || row.entryFile !== input.entryFile || row.state !== "prepared") return;
const target = liveTargets.get(targetKey(row.companyId, row.runId));
const content = await readCandidate(row, target).catch(() => null);
// Never advance a stale local edit past an unrelated explicit update.
if (content === null || hash(content) !== input.contentHash) return;
await db.update(copies).set({ baseRevisionId: input.revisionId, baseHash: input.contentHash, updatedAt: new Date() })
.where(and(scope(row.companyId, row.runId), eq(copies.state, "prepared"), eq(copies.baseHash, row.baseHash)));
}
async function commitCandidate(row: Copy) {
if (isAgentDirectoryCopy(row)) return directories.collectStopped(row);
if (row.candidateBase64 === null || completed.has(row.state) || row.state === "conflict") return row;
try {
const receipt = await revisions.commit({
companyId: row.companyId, agentId: row.agentId, entryFile: row.entryFile,
baseRevisionId: row.baseRevisionId, content: Buffer.from(row.candidateBase64, "base64"), source: "cleanup",
}, actorFor(row));
return patch(row, { state: "saved", receipt: { ...receipt }, errorCode: null, errorMessage: null, nextAttemptAt: null });
} catch (error) {
const detail = error as { status?: number; details?: { code?: string }; message?: string };
const code = detail.details?.code ?? "INSTRUCTION_SAVE_FAILED";
const retryable = !detail.status || detail.status >= 500;
const message = detail.status === 403
? "Current permissions do not allow this instruction save. The candidate was preserved."
: detail.status === 409
? "Instructions changed after this run started. Review the preserved candidate against the current revision."
: "Instruction edits were preserved but not saved. Review the candidate before retrying.";
return patch(row, { state: retryable ? "pending_commit" : "conflict", errorCode: code, errorMessage: message, nextAttemptAt: retryable && row.attempts < MAX_COLLECTION_ATTEMPTS ? new Date(Date.now() + 30_000) : null });
}
}
/** Caller must have joined the owned provider process before granting this proof. */
async function collectStopped(input: { companyId: string; runId: string; target?: AdapterExecutionTarget | null }) {
let row = await get(input.companyId, input.runId);
if (!row) return row;
if (isAgentDirectoryCopy(row)) return directories.collectStopped(row, input.target);
if (completed.has(row.state) || row.state === "conflict") return row;
row = await patch(row, { processStoppedAt: row.processStoppedAt ?? new Date(), attempts: row.attempts + 1 });
if (completed.has(row.state) || row.state === "conflict") return row;
if (row.candidateBase64 !== null) return commitCandidate(row);
try {
const candidate = await readCandidate(row, input.target);
if (candidate === null) {
return patch(row, { state: "unavailable", errorCode: "INSTRUCTION_FILE_MISSING", errorMessage: "The registered instruction file was removed. Canonical instructions were kept; no edits were saved.", nextAttemptAt: null });
}
if (hash(candidate) === row.baseHash) return patch(row, { state: "unchanged", nextAttemptAt: null });
// Save the candidate durably before attempting authorization/CAS or releasing the environment.
row = await patch(row, { state: "pending_commit", candidateBase64: candidate.toString("base64"), candidateHash: hash(candidate), nextAttemptAt: new Date() });
return commitCandidate(row);
} catch (error) {
return patch(row, { state: row.attempts < MAX_COLLECTION_ATTEMPTS ? "pending_collection" : "unavailable", errorCode: "INSTRUCTION_COLLECTION_FAILED", errorMessage: "The registered instruction copy could not be read safely. Canonical instructions were kept; no save receipt exists.", nextAttemptAt: row.attempts < MAX_COLLECTION_ATTEMPTS ? new Date(Date.now() + 30_000) : null });
}
}
/** Restarts can retry persisted bytes without touching a live filesystem or launching a model. */
async function recoverCaptured() {
const pending = await db.select().from(copies).where(and(
eq(copies.state, "pending_commit"),
or(isNull(copies.nextAttemptAt), lte(copies.nextAttemptAt, new Date())),
lte(copies.attempts, MAX_COLLECTION_ATTEMPTS - 1),
)).limit(20);
for (const row of pending) {
const result = await commitCandidate(await patch(row, { attempts: row.attempts + 1 }));
if (isAgentDirectoryCopy(result) && completed.has(result.state)) await directories.release(result);
}
// A crash between a durable save and release must not retain run snapshots
// forever. Only discard copies with recorded stop proof and a terminal receipt.
const cleanup = await db.select().from(copies).where(and(
inArray(copies.state, [...completed, "unavailable"]), isNotNull(copies.processStoppedAt),
sql`${copies.receipt}->>'schema' = 'paperclip.agent-files.v1'`,
or(sql`${copies.receipt} ? 'baseline'`, sql`${copies.receipt}->>'cleanupPending' = 'true'`),
or(isNull(copies.nextAttemptAt), lte(copies.nextAttemptAt, new Date())),
)).orderBy(asc(copies.updatedAt)).limit(20);
for (const row of cleanup) await directories.release(row);
return pending.length;
}
/** Recover only the registered copy after independently recorded process-stop
* proof. Terminal run status is never a substitute for that proof. Remote
* receipts mean the environment is stopped; do not restart it just to read. */
async function recoverStopped() {
const pending = await db.select({ copy: copies, runtimeMode: heartbeatRuns.runtimeMode }).from(copies)
.innerJoin(heartbeatRuns, and(eq(heartbeatRuns.companyId, copies.companyId), eq(heartbeatRuns.id, copies.runId)))
.where(and(or(inArray(copies.state, ["prepared", "pending_collection"]),
and(eq(copies.state, "preparing"), sql`${copies.receipt}->>'schema' = 'paperclip.agent-files.v1'`)),
inArray(heartbeatRuns.status, ["succeeded", "failed", "cancelled", "timed_out", "interrupted"]),
lte(copies.attempts, MAX_COLLECTION_ATTEMPTS - 1))).limit(20);
for (const { copy: row, runtimeMode } of pending) {
if (row.state === "preparing" && isAgentDirectoryCopy(row)) {
// A provider cannot launch until preparation records "prepared". With
// the owning run terminal, this is an interrupted staging copy only.
await directories.release(await patch(row, { state: "unavailable", processStoppedAt: new Date(),
errorCode: "AGENT_FILES_PREPARE_FAILED", errorMessage: "Agent-file preparation was interrupted. The temporary copy was discarded.", nextAttemptAt: null }));
} else if (row.location === "local" && (row.processStoppedAt || runtimeMode === "native" &&
await hasNativeLocalProcessStop(db, row.companyId, row.runId))) {
const result = await collectStopped({ companyId: row.companyId, runId: row.runId });
if (result && isAgentDirectoryCopy(result) && completed.has(result.state)) await directories.release(result);
} else if (row.location !== "local" && await remoteExecutionHasStopped(db, row.companyId, row.runId)) {
const unavailable = await reportUnavailable(row.companyId, row.runId);
if (unavailable && isAgentDirectoryCopy(unavailable)) {
await directories.release(await patch(unavailable, { processStoppedAt: unavailable.processStoppedAt ?? new Date() }));
}
} else if (row.state === "prepared") {
await patch(row, { state: "pending_collection", errorCode: "INSTRUCTION_STOP_UNCONFIRMED",
errorMessage: "The provider's stop has not been confirmed. Instruction collection is pending; no save is claimed.", nextAttemptAt: null });
}
}
return pending.length;
}
async function list(companyId: string, agentId: string, actor: AuthorizationActor) {
await revisions.readCurrent({ companyId, agentId }, actor);
const rows = await db.select().from(copies).where(and(eq(copies.companyId, companyId), eq(copies.agentId, agentId), inArray(copies.state, ["conflict", "pending_collection", "pending_commit", "unavailable"]))).orderBy(desc(copies.createdAt)).limit(50);
return rows.map(row => ({ runId: row.runId, entryFile: row.entryFile, baseRevisionId: row.baseRevisionId,
baseHash: row.baseHash, state: row.state as "conflict" | "pending_collection" | "pending_commit" | "unavailable",
candidateHash: row.candidateHash, contract: isAgentDirectoryCopy(row) ? "agent_files" : "legacy",
content: row.candidateBase64 === null ? null : Buffer.from(row.candidateBase64, "base64").toString("utf8"),
errorCode: row.errorCode, errorMessage: row.errorMessage, createdAt: row.createdAt.toISOString(), updatedAt: row.updatedAt.toISOString() }));
}
async function resolve(input: { companyId: string; agentId: string; runId: string; baseRevisionId: string | null; content: string }, actor: AuthorizationActor) {
const row = await get(input.companyId, input.runId);
if (!row || row.agentId !== input.agentId) throw notFound("Instruction candidate not found");
if (completed.has(row.state)) throw conflict("This instruction candidate has already been resolved");
if (isAgentDirectoryCopy(row)) throw conflict("Agent-file synchronization failures have no preserved candidate");
const receipt = await revisions.commit({ companyId: input.companyId, agentId: input.agentId, entryFile: row.entryFile, content: input.content, baseRevisionId: input.baseRevisionId, source: "api" }, actor);
await patch(row, { state: "resolved", receipt: { ...receipt }, nextAttemptAt: null });
return receipt;
}
/** A controller may report loss before disposal, never fabricate a save or
* read a possibly live provider. Already captured candidates remain intact. */
async function reportUnavailable(companyId: string, runId: string) {
const row = await get(companyId, runId);
if (!row || completed.has(row.state) || row.state === "unchanged_turn" || row.candidateBase64 !== null || (isAgentDirectoryCopy(row) && row.candidateHash !== null) || row.state === "conflict") return row;
if (isAgentDirectoryCopy(row) && row.state === "unavailable") return row;
return patch(row, { state: "unavailable", errorCode: "INSTRUCTION_COLLECTION_UNAVAILABLE",
errorMessage: "The registered instruction copy could not be retrieved safely before environment release. No instruction save is claimed.", nextAttemptAt: null });
}
async function release(companyId: string, runId: string) { liveTargets.delete(targetKey(companyId, runId)); const row = await get(companyId, runId); if (row && isAgentDirectoryCopy(row)) await directories.release(row); }
return { prepare, get, hasChanges, acknowledgeExplicitSave, collectStopped, recoverCaptured, recoverStopped, list, resolve, reportUnavailable, release };
}
+129 -56
View File
@@ -1,5 +1,9 @@
import { inspectAgentFile, fileHash, agentFilePath, MAX_AGENT_FILE_BYTES } from "./agent-file-store.js";
import fs from "node:fs/promises";
import path from "node:path";
import { and, eq } from "drizzle-orm";
import { agentInstructionHeads, agents, type Db } from "@paperclipai/db";
import { instructionPath, assertInstructionPathSafe, instructionBytes, readInstructionBytes } from "./agent-instruction-files.js";
import { notFound, unprocessable } from "../errors.js";
import { resolveHomeAwarePath, resolvePaperclipInstanceRoot } from "../home-paths.js";
@@ -36,6 +40,8 @@ type AgentLike = {
type AgentInstructionsFileSummary = {
path: string;
contentHash?: string;
binary?: boolean;
size: number;
language: string;
markdown: boolean;
@@ -53,6 +59,7 @@ type AgentInstructionsFileDetail = AgentInstructionsFileSummary & {
type AgentInstructionsBundle = {
agentId: string;
companyId: string;
persistence?: "agent_files";
mode: BundleMode | null;
rootPath: string | null;
managedRootPath: string;
@@ -112,11 +119,7 @@ function isMarkdown(relativePath: string) {
}
function normalizeRelativeFilePath(candidatePath: string): string {
const normalized = path.posix.normalize(candidatePath.replaceAll("\\", "/")).replace(/^\/+/, "");
if (!normalized || normalized === "." || normalized === ".." || normalized.startsWith("../")) {
throw unprocessable("Instructions file path must stay within the bundle root");
}
return normalized;
return instructionPath(candidatePath);
}
function resolvePathWithinRoot(rootPath: string, relativePath: string): string {
@@ -130,7 +133,7 @@ function resolvePathWithinRoot(rootPath: string, relativePath: string): string {
return absolutePath;
}
function resolveManagedInstructionsRoot(agent: AgentLike): string {
export function resolveManagedInstructionsRoot(agent: AgentLike): string {
return path.resolve(
resolvePaperclipInstanceRoot(),
"companies",
@@ -153,7 +156,7 @@ function resolveLegacyInstructionsPath(candidatePath: string, config: Record<str
}
async function statIfExists(targetPath: string) {
return fs.stat(targetPath).catch(() => null);
return fs.lstat(targetPath).catch(() => null);
}
function shouldIgnoreInstructionsEntry(entry: { name: string; isDirectory(): boolean; isFile(): boolean }) {
@@ -172,14 +175,14 @@ function shouldIgnoreInstructionsEntry(entry: { name: string; isDirectory(): boo
async function listFilesRecursive(
rootPath: string,
options?: { rejectSymlinks?: boolean },
options?: { rejectSymlinks?: boolean; legacyExcludes?: boolean },
): Promise<string[]> {
const output: string[] = [];
async function walk(currentPath: string, relativeDir: string) {
const entries = await fs.readdir(currentPath, { withFileTypes: true }).catch(() => []);
for (const entry of entries) {
if (shouldIgnoreInstructionsEntry(entry)) continue;
if (options?.legacyExcludes && shouldIgnoreInstructionsEntry(entry)) continue;
const absolutePath = path.join(currentPath, entry.name);
const relativePath = normalizeRelativeFilePath(
relativeDir ? path.posix.join(relativeDir, entry.name) : entry.name,
@@ -206,13 +209,21 @@ async function listFilesRecursive(
async function readFileSummary(rootPath: string, relativePath: string, entryFile: string): Promise<AgentInstructionsFileSummary> {
const absolutePath = resolvePathWithinRoot(rootPath, relativePath);
const stat = await fs.stat(absolutePath);
// External bundles may contain large assets; listing must not read them.
const file = stat.size > MAX_AGENT_FILE_BYTES ? null : await inspectAgentFile(rootPath, relativePath);
return summarizeFile(relativePath, entryFile, file?.size ?? stat.size, file?.bytes ?? null, file?.hash);
}
function summarizeFile(relativePath: string, entryFile: string, size: number, bytes: Buffer | null, hash?: string): AgentInstructionsFileSummary {
let binary = bytes === null;
try { if (bytes?.includes(0)) binary = true; new TextDecoder("utf-8", { fatal: true }).decode(bytes ?? undefined); } catch { binary = true; }
return {
path: relativePath,
size: stat.size,
size,
language: inferLanguage(relativePath),
markdown: isMarkdown(relativePath),
isEntryFile: relativePath === entryFile,
editable: true,
editable: !binary, binary, contentHash: hash,
deprecated: false,
virtual: false,
};
@@ -231,7 +242,7 @@ async function readLegacyInstructions(agent: AgentLike, config: Record<string, u
return asString(config[PROMPT_KEY]) ?? "";
}
function deriveBundleState(agent: AgentLike): BundleState {
export function deriveBundleState(agent: AgentLike): BundleState {
const config = asRecord(agent.adapterConfig);
const warnings: string[] = [];
const storedModeRaw = config[MODE_KEY];
@@ -245,21 +256,23 @@ function deriveBundleState(agent: AgentLike): BundleState {
const storedEntryRaw = asString(config[ENTRY_KEY]);
if (storedEntryRaw) {
try {
entryFile = normalizeRelativeFilePath(storedEntryRaw);
} catch {
warnings.push(`Ignored invalid instructions entry file "${storedEntryRaw}".`);
// Historical config accepted normalized relative paths. Keep that read
// compatibility; new API writes still use strict instructionPath validation.
entryFile = normalizeRelativeFilePath(path.posix.normalize(storedEntryRaw.replaceAll("\\", "/")).replace(/^\/+/, ""));
} catch (error) {
warnings.push(error instanceof Error ? error.message : String(error));
}
}
if (!rootPath && legacyInstructionsPath) {
try {
const resolvedLegacyPath = resolveLegacyInstructionsPath(legacyInstructionsPath, config);
rootPath = path.dirname(resolvedLegacyPath);
entryFile = path.basename(resolvedLegacyPath);
mode = resolvedLegacyPath.startsWith(`${resolveManagedInstructionsRoot(agent)}${path.sep}`)
|| resolvedLegacyPath === path.join(resolveManagedInstructionsRoot(agent), entryFile)
? "managed"
: "external";
const managedRoot = resolveManagedInstructionsRoot(agent);
const managedRelative = path.relative(managedRoot, resolvedLegacyPath);
const inManagedRoot = managedRelative !== ".." && !managedRelative.startsWith(`..${path.sep}`) && !path.isAbsolute(managedRelative);
rootPath = inManagedRoot ? managedRoot : path.dirname(resolvedLegacyPath);
entryFile = inManagedRoot ? instructionPath(managedRelative.split(path.sep).join("/")) : path.basename(resolvedLegacyPath);
mode = inManagedRoot ? "managed" : "external";
if (!path.isAbsolute(legacyInstructionsPath)) {
warnings.push("Using legacy relative instructionsFilePath; migrate this agent to a managed or absolute external bundle.");
}
@@ -303,7 +316,7 @@ async function recoverManagedBundleState(agent: AgentLike, state: BundleState):
const files = await listFilesRecursive(managedRootPath);
if (files.length === 0) return state;
const recoveredEntryFile = files.includes(state.entryFile)
const recoveredEntryFile = asString(state.config[ENTRY_KEY]) ? state.entryFile : files.includes(state.entryFile)
? state.entryFile
: files.includes(ENTRY_FILE_DEFAULT)
? ENTRY_FILE_DEFAULT
@@ -370,6 +383,7 @@ function toBundle(agent: AgentLike, state: BundleState, files: AgentInstructions
return {
agentId: agent.id,
companyId: agent.companyId,
...(state.mode === "managed" ? { persistence: "agent_files" as const } : {}),
mode: state.mode,
rootPath: state.rootPath,
managedRootPath: resolveManagedInstructionsRoot(agent),
@@ -443,7 +457,8 @@ async function writeBundleFiles(
) {
for (const [relativePath, content] of Object.entries(files)) {
const normalizedPath = normalizeRelativeFilePath(relativePath);
const absolutePath = resolvePathWithinRoot(rootPath, normalizedPath);
instructionBytes(content);
const absolutePath = await assertInstructionPathSafe(rootPath, normalizedPath);
const existingStat = await statIfExists(absolutePath);
if (existingStat?.isFile() && !options?.overwriteExisting) continue;
await fs.mkdir(path.dirname(absolutePath), { recursive: true });
@@ -464,19 +479,26 @@ export function syncInstructionsBundleConfigFromFilePath(
return next;
}
const resolvedPath = resolveLegacyInstructionsPath(instructionsFilePath, adapterConfig);
const rootPath = path.dirname(resolvedPath);
const entryFile = path.basename(resolvedPath);
const mode: BundleMode = resolvedPath.startsWith(`${resolveManagedInstructionsRoot(agent)}${path.sep}`)
|| resolvedPath === path.join(resolveManagedInstructionsRoot(agent), entryFile)
? "managed"
: "external";
const managedRoot = resolveManagedInstructionsRoot(agent);
const relative = path.relative(managedRoot, resolvedPath);
const managed = relative !== ".." && !relative.startsWith(`..${path.sep}`) && !path.isAbsolute(relative);
const rootPath = managed ? managedRoot : path.dirname(resolvedPath);
const entryFile = managed ? instructionPath(relative.split(path.sep).join("/")) : path.basename(resolvedPath);
const mode: BundleMode = managed ? "managed" : "external";
return applyBundleConfig(next, { mode, rootPath, entryFile });
}
export function agentInstructionsService() {
export function agentInstructionsService(db?: Db) {
async function assertUnversionedEntry(agent: AgentLike, entryFile: string, connection: Db | Parameters<Parameters<Db["transaction"]>[0]>[0] | undefined = db) {
if (!connection) throw unprocessable("Bundle initialization requires the database-backed instructions service");
const [head] = await connection.select().from(agentInstructionHeads).where(and(eq(agentInstructionHeads.companyId, agent.companyId),
eq(agentInstructionHeads.agentId, agent.id), eq(agentInstructionHeads.entryFile, entryFile)));
if (head) throw unprocessable("This entry has revision history. Use the instruction content API with its baseRevisionId to update it.", { code: "INSTRUCTION_REVISION_REQUIRED" });
}
async function getBundle(agent: AgentLike): Promise<AgentInstructionsBundle> {
const state = await recoverManagedBundleState(agent, deriveBundleState(agent));
if (!state.rootPath) return toBundle(agent, state, []);
await assertInstructionPathSafe(state.rootPath, state.entryFile);
const stat = await statIfExists(state.rootPath);
if (!stat?.isDirectory()) {
return toBundle(agent, {
@@ -484,8 +506,12 @@ export function agentInstructionsService() {
warnings: [...state.warnings, `Instructions root does not exist: ${state.rootPath}`],
}, []);
}
const files = await listFilesRecursive(state.rootPath);
const summaries = await Promise.all(files.map((relativePath) => readFileSummary(state.rootPath!, relativePath, state.entryFile)));
const files = await listFilesRecursive(state.rootPath, { legacyExcludes: state.mode === "external" });
const summaries: AgentInstructionsFileSummary[] = [];
// Bound open descriptors and text buffers even for a large personal folder.
for (let index = 0; index < files.length; index += 8) {
summaries.push(...await Promise.all(files.slice(index, index + 8).map(relativePath => readFileSummary(state.rootPath!, relativePath, state.entryFile))));
}
return toBundle(agent, state, summaries);
}
@@ -507,24 +533,11 @@ export function agentInstructionsService() {
};
}
if (!state.rootPath) throw notFound("Agent instructions bundle is not configured");
const absolutePath = resolvePathWithinRoot(state.rootPath, relativePath);
const [content, stat] = await Promise.all([
fs.readFile(absolutePath, "utf8").catch(() => null),
fs.stat(absolutePath).catch(() => null),
]);
if (content === null || !stat?.isFile()) throw notFound("Instructions file not found");
const normalizedPath = normalizeRelativeFilePath(relativePath);
return {
path: normalizedPath,
size: stat.size,
language: inferLanguage(normalizedPath),
markdown: isMarkdown(normalizedPath),
isEntryFile: normalizedPath === state.entryFile,
editable: true,
deprecated: false,
virtual: false,
content,
};
await assertInstructionPathSafe(state.rootPath, relativePath);
const file = await inspectAgentFile(state.rootPath, relativePath);
if (file === null) throw notFound("Instructions file not found");
const summary = summarizeFile(relativePath, state.entryFile, file.size, file.bytes, file.hash);
return { ...summary, content: summary.binary ? "" : file.bytes!.toString("utf8") };
}
async function ensureWritableBundle(
@@ -551,11 +564,12 @@ export function agentInstructionsService() {
});
await fs.mkdir(managedRoot, { recursive: true });
const entryPath = resolvePathWithinRoot(managedRoot, entryFile);
const entryPath = await assertInstructionPathSafe(managedRoot, entryFile);
const entryStat = await statIfExists(entryPath);
if (!entryStat?.isFile()) {
const legacyInstructions = await readLegacyInstructions(agent, current.config);
if (legacyInstructions.trim().length > 0) {
await assertUnversionedEntry(agent, entryFile);
await fs.mkdir(path.dirname(entryPath), { recursive: true });
await fs.writeFile(entryPath, legacyInstructions, "utf8");
}
@@ -595,9 +609,13 @@ export function agentInstructionsService() {
nextRootPath = resolvedRoot;
}
await assertInstructionPathSafe(nextRootPath, nextEntryFile);
await fs.mkdir(nextRootPath, { recursive: true });
const existingFiles = await listFilesRecursive(nextRootPath);
if (nextMode === "managed" && !existingFiles.includes(nextEntryFile)) {
await assertUnversionedEntry(agent, nextEntryFile);
}
const exported = await exportFiles(agent);
if (existingFiles.length === 0) {
await writeBundleFiles(nextRootPath, exported.files);
@@ -618,7 +636,7 @@ export function agentInstructionsService() {
return { bundle: nextBundle, adapterConfig: nextConfig };
}
async function writeFile(
async function writeFileUnversioned(
agent: AgentLike,
relativePath: string,
content: string,
@@ -642,7 +660,14 @@ export function agentInstructionsService() {
return { bundle, file, adapterConfig };
}
const configured = await recoverManagedBundleState(agent, current);
if (normalizeRelativeFilePath(relativePath) === configured.entryFile) {
throw unprocessable("Entry edits require the canonical instruction commit service and baseRevisionId", { code: "INSTRUCTION_REVISION_REQUIRED" });
}
if (configured.mode !== "external") agentFilePath(relativePath);
const prepared = await ensureWritableBundle(agent, options);
instructionBytes(content);
await assertInstructionPathSafe(prepared.state.rootPath!, relativePath);
const absolutePath = resolvePathWithinRoot(prepared.state.rootPath!, relativePath);
await fs.mkdir(path.dirname(absolutePath), { recursive: true });
await fs.writeFile(absolutePath, content, "utf8");
@@ -654,7 +679,7 @@ export function agentInstructionsService() {
return { bundle, file, adapterConfig: prepared.adapterConfig };
}
async function deleteFile(agent: AgentLike, relativePath: string): Promise<{
async function deleteFileUnversioned(agent: AgentLike, relativePath: string): Promise<{
bundle: AgentInstructionsBundle;
adapterConfig: Record<string, unknown>;
}> {
@@ -668,13 +693,35 @@ export function agentInstructionsService() {
if (normalizedPath === state.entryFile) {
throw unprocessable("Cannot delete the bundle entry file");
}
const absolutePath = resolvePathWithinRoot(state.rootPath, normalizedPath);
const absolutePath = await assertInstructionPathSafe(state.rootPath, normalizedPath);
await fs.rm(absolutePath, { force: true });
const adapterConfig = buildPersistedBundleConfig(derived, state);
const bundle = await getBundle({ ...agent, adapterConfig });
return { bundle, adapterConfig };
}
// Reload configuration under the commit lock: a previously supporting file
// may have become the configured entry since the route loaded its agent.
async function withCurrentAgent<T>(agent: AgentLike, relativePath: string, operation: (current: AgentLike) => Promise<T>) {
if (!db) return operation(agent);
return db.transaction(async (tx) => {
const [current] = await tx.select().from(agents)
.where(and(eq(agents.id, agent.id), eq(agents.companyId, agent.companyId))).for("update");
if (!current) throw notFound("Agent not found");
if (relativePath !== LEGACY_PROMPT_TEMPLATE_PATH) {
// Historical entries also remain projections if the configured entry changes.
await assertUnversionedEntry(current, normalizeRelativeFilePath(relativePath), tx);
}
return operation(current);
});
}
async function writeFile(...args: Parameters<typeof writeFileUnversioned>) {
return withCurrentAgent(args[0], args[1], (current) => writeFileUnversioned(current, args[1], args[2], args[3]));
}
async function deleteFile(...args: Parameters<typeof deleteFileUnversioned>) {
return withCurrentAgent(args[0], args[1], (current) => deleteFileUnversioned(current, args[1]));
}
async function exportFiles(agent: AgentLike, options?: { rejectSymlinks?: boolean }): Promise<{
files: Record<string, string>;
entryFile: string;
@@ -684,7 +731,7 @@ export function agentInstructionsService() {
if (state.rootPath) {
const stat = await statIfExists(state.rootPath);
if (stat?.isDirectory()) {
const relativePaths = await listFilesRecursive(state.rootPath, options);
const relativePaths = await listFilesRecursive(state.rootPath, { ...options, legacyExcludes: true });
const files = Object.fromEntries(await Promise.all(relativePaths.map(async (relativePath) => {
const absolutePath = resolvePathWithinRoot(state.rootPath!, relativePath);
const content = await fs.readFile(absolutePath, "utf8");
@@ -704,7 +751,7 @@ export function agentInstructionsService() {
};
}
async function materializeManagedBundle(
async function materializeManagedBundleUnversioned(
agent: AgentLike,
files: Record<string, string>,
options?: {
@@ -716,6 +763,14 @@ export function agentInstructionsService() {
const rootPath = resolveManagedInstructionsRoot(agent);
const entryFile = options?.entryFile ? normalizeRelativeFilePath(options.entryFile) : ENTRY_FILE_DEFAULT;
for (const [relativePath, content] of Object.entries(files)) {
instructionBytes(content);
await assertInstructionPathSafe(rootPath, agentFilePath(relativePath));
}
const previous = await readInstructionBytes(rootPath, entryFile);
if (previous && !previous.equals(instructionBytes(files[entryFile] ?? ""))) {
throw unprocessable("Existing entry content must be saved through the canonical revision API before replacing a bundle", { code: "INSTRUCTION_REVISION_REQUIRED" });
}
if (options?.replaceExisting) {
await fs.rm(rootPath, { recursive: true, force: true });
}
@@ -744,6 +799,24 @@ export function agentInstructionsService() {
return { bundle, adapterConfig };
}
async function materializeManagedBundle(
...args: Parameters<typeof materializeManagedBundleUnversioned>
) {
if (!db) throw unprocessable("Bundle initialization requires the database-backed instructions service");
return db.transaction(async (tx) => {
const agent = args[0];
const [owner] = await tx.select({ id: agents.id }).from(agents).where(and(eq(agents.id, agent.id), eq(agents.companyId, agent.companyId))).for("update");
if (!owner) throw notFound("Agent not found");
await assertUnversionedEntry(agent, args[2]?.entryFile ? normalizeRelativeFilePath(args[2].entryFile) : ENTRY_FILE_DEFAULT, tx);
if (args[2]?.replaceExisting) {
const [existingHead] = await tx.select({ id: agentInstructionHeads.revisionId }).from(agentInstructionHeads)
.where(and(eq(agentInstructionHeads.companyId, agent.companyId), eq(agentInstructionHeads.agentId, agent.id))).limit(1);
if (existingHead) throw unprocessable("Replacing this bundle would remove a versioned entry. Use canonical content commits.", { code: "INSTRUCTION_REVISION_REQUIRED" });
}
return materializeManagedBundleUnversioned(...args);
});
}
return {
getBundle,
readFile,
+18 -3
View File
@@ -62,6 +62,7 @@ export type AuthorizationAction =
| PermissionKey
| "agent_config:read"
| "agent_config:update"
| "agent_instructions:update"
| "skill_config:update"
| "agent:read"
| "agent:wake"
@@ -148,7 +149,7 @@ function companyIdForResource(resource: AuthorizationResource) {
}
function permissionForAction(action: AuthorizationAction): PermissionKey | null {
if (action === "agent_config:read" || action === "agent_config:update" || action === "skill_config:update") {
if (action === "agent_config:read" || action === "agent_config:update" || action === "agent_instructions:update" || action === "skill_config:update") {
return null;
}
if (
@@ -998,6 +999,7 @@ export function authorizationService(db: Db | DbTransaction) {
input.action === "decision_triage:manage" ||
input.action === "agent_config:read" ||
input.action === "agent_config:update" ||
input.action === "agent_instructions:update" ||
input.action === "skill_config:update" ||
input.action === "inbox:manage" ||
input.action === "runtime:manage" ||
@@ -1734,7 +1736,7 @@ export function authorizationService(db: Db | DbTransaction) {
if (input.action === "agent_config:read") {
return decideWithAgentConfigReadGrant("user", input.actor.userId);
}
if (input.action === "agent_config:update") {
if (input.action === "agent_config:update" || input.action === "agent_instructions:update") {
return decideWithProtectedChangeGrants("user", input.actor.userId, {
direct: "agents:configure",
suggest: "agents:suggest-changes",
@@ -2227,6 +2229,19 @@ export function authorizationService(db: Db | DbTransaction) {
return decideWithAgentConfigReadGrant("agent", actorAgentId);
}
if (input.action === "agent_instructions:update") {
if (!isSimpleAssignableAgentStatus(actorAgent.status) || !input.actor.onBehalfOfUserId) {
return deny({ action: input.action, reason: "deny_missing_membership", explanation: "Instruction edits require an active agent and a responsible user." });
}
// Explicit configure/suggest restrictions still govern content changes.
// Only self edits may fall back to the responsible user's target access.
const restricted = await decideWithProtectedChangeGrants("agent", actorAgentId, {
direct: "agents:configure", suggest: "agents:suggest-changes",
});
if (restricted.reason !== "deny_no_grant" || input.resource.type !== "agent" || input.resource.agentId !== actorAgentId) return restricted;
return allow({ action: input.action, reason: "allow_self", explanation: "Own instruction content edit, subject to the responsible user's target edit access." });
}
if (input.action === "agent_config:update") {
return decideWithProtectedChangeGrants("agent", actorAgentId, {
direct: "agents:configure",
@@ -2391,7 +2406,7 @@ export function authorizationService(db: Db | DbTransaction) {
responsibleUserId,
}, "responsible-user authorization intersection denied");
return responsibleUserAuthzShadowMode() ? agentDecision : denied;
return input.action !== "agent_instructions:update" && responsibleUserAuthzShadowMode() ? agentDecision : denied;
}
async function decide(input: {
+143 -14
View File
@@ -1,3 +1,4 @@
import fs from "node:fs/promises";
import { readFileSync } from "node:fs";
import { createRequire } from "node:module";
import path from "node:path";
@@ -5,12 +6,16 @@ import { fileURLToPath } from "node:url";
import { readPaperclipSkillSyncPreference, writePaperclipSkillSyncPreference } from "@paperclipai/adapter-utils/server-utils";
import { and, desc, eq, ne } from "drizzle-orm";
import type { Db } from "@paperclipai/db";
import { agents, builtInManagedResources, companies, issueThreadInteractions, issues, routines, routineTriggers } from "@paperclipai/db";
import { activityLog, agents, builtInManagedResources, companies, issueThreadInteractions, issues, routines, routineTriggers } from "@paperclipai/db";
import { syncRoutineVariablesWithTemplate } from "@paperclipai/shared";
import type { Agent, Approval, CompanySkill, PermissionKey, Routine, RoutineTrigger, RoutineVariable } from "@paperclipai/shared";
import { conflict, HttpError, notFound, unprocessable } from "../errors.js";
import { logActivity } from "./activity-log.js";
import { agentInstructionsService } from "./agent-instructions.js";
import { adoptAgentFiles, agentFilePath, fileHash, readAgentFile, snapshotAgentFiles } from "./agent-file-store.js";
import { assertInstructionPathSafe, instructionBytes, materializeInstructionBytes } from "./agent-instruction-files.js";
import { agentInstructionRevisionService } from "./agent-instruction-revisions.js";
import type { AuthorizationActor } from "./authorization.js";
import { agentInstructionsBundleMode, agentInstructionsService } from "./agent-instructions.js";
import { agentService } from "./agents.js";
import { approvalService } from "./approvals.js";
import {
@@ -802,7 +807,7 @@ export function builtInAgentService(db: Db) {
const agentSvc = agentService(db);
const accessSvc = accessService(db);
const approvalSvc = approvalService(db);
const instructionsSvc = agentInstructionsService();
const instructionsSvc = agentInstructionsService(db);
const skillSvc = companySkillService(db);
const routineSvc = routineService(db);
@@ -916,9 +921,9 @@ export function builtInAgentService(db: Db) {
stockVersion: string;
stockHash: string;
defaultsJson: Record<string, unknown>;
}) {
}, connection: Db | Parameters<Parameters<Db["transaction"]>[0]>[0] = db) {
const now = new Date();
return db
return connection
.insert(builtInManagedResources)
.values(input)
.onConflictDoUpdate({
@@ -952,10 +957,15 @@ export function builtInAgentService(db: Db) {
return currentFiles;
}
async function materializeInstructions(agent: Agent, definition: BuiltInAgentDefinition, mode: "reconcile" | "reset") {
async function materializeInstructions(agent: Agent, definition: BuiltInAgentDefinition, mode: "reconcile" | "reset", actor?: AuthorizationActor) {
const bundle = definition.bundle!;
const stock = stockHash(bundle.instructions.files);
const binding = await getManagedResourceBinding(agent.companyId, definition.key, "instructions", "AGENTS.md");
// Stock operations also read the committed projection after interrupted writes.
if (agentInstructionsBundleMode(agent) === "managed") {
try { await agentInstructionRevisionService(db).materializeCurrent({ companyId: agent.companyId, agentId: agent.id }); }
catch (error) { if (!(error instanceof HttpError && error.status === 404)) throw error; }
}
const currentFiles = await currentInstructionFiles(agent, bundle);
const currentHash = Object.values(currentFiles).some((value) => value === null) ? null : stockHash(currentFiles);
const currentState = stockState({
@@ -969,6 +979,100 @@ export function builtInAgentService(db: Db) {
changedFiles: changedFileList(currentFiles, bundle.instructions.files),
});
if (!actor && mode === "reconcile" && agentInstructionsBundleMode(agent) === "managed"
&& binding && (binding.stockHash !== stock || binding.defaultsJson.pendingInstructionsUpdate)) {
const bindingWhere = and(eq(builtInManagedResources.companyId, agent.companyId),
eq(builtInManagedResources.bundleKey, definition.key), eq(builtInManagedResources.resourceKind, "instructions"),
eq(builtInManagedResources.resourceKey, "AGENTS.md"));
const incoming = Object.fromEntries(Object.entries(bundle.instructions.files)
.map(([file, content]) => [agentFilePath(file), instructionBytes(content)]));
// Persist only the pending operation's baseline hashes before touching
// files. If a write or DB commit fails, the next reconciliation can accept
// already-applied bytes and retry. This is not instruction revision history.
const prepared = await db.transaction(async (tx) => {
const [current] = await tx.select().from(agents).where(and(eq(agents.companyId, agent.companyId), eq(agents.id, agent.id))).for("update");
if (!current) throw notFound("Built-in agent not found");
if (agentInstructionsBundleMode(current) !== "managed") return false;
const root = await adoptAgentFiles(tx, current);
const [latest] = await tx.select().from(builtInManagedResources).where(bindingWhere);
if (!latest) return false;
const pending = latest.defaultsJson.pendingInstructionsUpdate as {
stockHash?: string; baseHashes?: Record<string, string | null>; nextHashes?: Record<string, string | null>;
} | undefined;
if (pending?.stockHash === stock) return true;
if (!pending && latest.stockHash === stock) return false;
const previousPaths = latest.defaultsJson.files;
if (!Array.isArray(previousPaths) || !previousPaths.every((file): file is string => typeof file === "string")) return false;
const ownedPaths = new Set(previousPaths);
if (pending) {
// A newer release (or rollback) can supersede an interrupted update.
// Recognize only the previous operation's baseline or intended bytes;
// never mistake an intervening operator edit for partially applied stock.
if (!pending.baseHashes || !pending.nextHashes) return false;
for (const [file, baseHash] of Object.entries(pending.baseHashes)) {
if (!(file in pending.nextHashes)) return false;
const bytes = await readAgentFile(root, agentFilePath(file));
const hash = bytes === null ? null : fileHash(bytes);
if (hash !== baseHash && hash !== pending.nextHashes[file]) return false;
ownedPaths.add(file);
}
} else {
const previousFiles: Record<string, string | null> = {};
for (const file of previousPaths) {
try { previousFiles[file] = (await instructionsSvc.readFile(current, file)).content; }
catch { previousFiles[file] = null; }
}
if (stockHash(previousFiles) !== latest.stockHash) return false;
}
await snapshotAgentFiles(root);
const baseHashes: Record<string, string | null> = {};
const nextHashes: Record<string, string | null> = {};
for (const file of new Set([...ownedPaths, ...Object.keys(incoming)])) {
const bytes = await readAgentFile(root, agentFilePath(file));
// A newly declared stock path must not overwrite a personal file.
if (!ownedPaths.has(file) && bytes !== null && !bytes.equals(incoming[file]!)) return false;
baseHashes[file] = bytes === null ? null : fileHash(bytes);
nextHashes[file] = incoming[file] ? fileHash(incoming[file]) : null;
}
await tx.update(builtInManagedResources).set({ defaultsJson: { ...latest.defaultsJson,
pendingInstructionsUpdate: { stockHash: stock, baseHashes, nextHashes } }, updatedAt: new Date() }).where(bindingWhere);
return true;
});
if (!prepared) return currentState;
return db.transaction(async (tx) => {
const [current] = await tx.select().from(agents).where(and(eq(agents.companyId, agent.companyId), eq(agents.id, agent.id))).for("update");
if (!current) throw notFound("Built-in agent not found");
if (agentInstructionsBundleMode(current) !== "managed") return currentState;
const root = await adoptAgentFiles(tx, current);
const [latest] = await tx.select().from(builtInManagedResources).where(bindingWhere);
const pending = latest?.defaultsJson.pendingInstructionsUpdate as { stockHash?: string; baseHashes?: Record<string, string | null> } | undefined;
if (pending?.stockHash !== stock || !pending.baseHashes) return currentState;
await snapshotAgentFiles(root);
// Preflight every changed or removed path before mutation; preserve any
// operator edit made between preparation, failure, and this retry.
for (const [file, baseHash] of Object.entries(pending.baseHashes)) {
const bytes = await readAgentFile(root, agentFilePath(file));
const hash = bytes === null ? null : fileHash(bytes);
const nextHash = incoming[file] ? fileHash(incoming[file]) : null;
if (hash !== baseHash && hash !== nextHash) return currentState;
}
for (const [file, bytes] of Object.entries(incoming)) await materializeInstructionBytes(root, file, bytes);
for (const file of Object.keys(pending.baseHashes)) {
if (!(file in incoming)) await fs.rm(await assertInstructionPathSafe(root, file), { force: true });
}
await upsertManagedResourceBinding({ companyId: agent.companyId, bundleKey: definition.key,
resourceKind: "instructions", resourceKey: "AGENTS.md", resourceId: agent.id,
stockVersion: bundle.stockVersion, stockHash: stock,
defaultsJson: { entryFile: bundle.instructions.entryFile, files: Object.keys(bundle.instructions.files) },
}, tx);
await tx.insert(activityLog).values({ companyId: agent.companyId, actorType: "system", actorId: "built-in-reconcile",
action: "agent.files_updated", entityType: "agent", entityId: agent.id,
details: { source: "built-in-stock-update", bundleKey: definition.key, stockHash: stock } });
return stockState({ resourceKind: "instructions", resourceKey: "AGENTS.md", resourceId: agent.id,
stockVersion: bundle.stockVersion, latestStockHash: stock, currentHash: stock, bindingStockHash: stock });
});
}
const shouldWrite =
mode === "reset"
|| currentState.stockStatus === "missing"
@@ -992,13 +1096,36 @@ export function builtInAgentService(db: Db) {
return currentState;
}
const materialized = await instructionsSvc.materializeManagedBundle(agent, bundle.instructions.files, {
entryFile: bundle.instructions.entryFile,
replaceExisting: true,
clearLegacyPromptTemplate: true,
});
let adapterConfig: Record<string, unknown>;
if (currentFiles[bundle.instructions.entryFile] !== null && currentFiles[bundle.instructions.entryFile] !== undefined) {
if (!actor && mode === "reconcile") return currentState;
if (!actor) throw unprocessable("Resetting existing instructions requires an authenticated operator", { code: "INSTRUCTION_IDENTITY_INVALID" });
const revisions = agentInstructionRevisionService(db);
const target = { companyId: agent.companyId, agentId: agent.id };
const baseline = await revisions.readCurrent(target, actor);
const receipt = await revisions.commit({ ...target, entryFile: bundle.instructions.entryFile,
baseRevisionId: baseline?.revision.id ?? null, content: bundle.instructions.files[bundle.instructions.entryFile] ?? "",
source: actor.type === "board" ? "board" : "api" }, actor);
if (receipt.materialization === "pending") throw conflict("Instruction revision saved; retry reset to repair its disk copy", { revisionId: receipt.revision.id });
const refreshed = await agentSvc.getById(agent.id);
if (!refreshed) throw notFound("Built-in agent not found");
for (const [file, content] of Object.entries(bundle.instructions.files)) {
if (file !== bundle.instructions.entryFile) await instructionsSvc.writeFile(refreshed, file, content);
}
// Preserve personal files outside this built-in bundle's declared paths.
adapterConfig = { ...refreshed.adapterConfig };
delete adapterConfig.promptTemplate;
delete adapterConfig.bootstrapPromptTemplate;
} else {
const materialized = await instructionsSvc.materializeManagedBundle(agent, bundle.instructions.files, {
entryFile: bundle.instructions.entryFile,
replaceExisting: false,
clearLegacyPromptTemplate: true,
});
adapterConfig = materialized.adapterConfig;
}
const updated = await agentSvc.update(agent.id, {
adapterConfig: materialized.adapterConfig,
adapterConfig,
}, {
allowBuiltInAgentMetadata: true,
recordRevision: { source: `built-in-bundle:${mode}:instructions` },
@@ -1480,13 +1607,14 @@ export function builtInAgentService(db: Db) {
definition: BuiltInAgentDefinition,
mode: "reconcile" | "reset",
resources?: Array<"instructions" | "skill" | "routine">,
actor?: AuthorizationActor,
) {
if (!definition.bundle) return [];
const selected = new Set(resources ?? ["instructions", "skill", "routine"]);
const existingStates = await bundleResourceStates(agent.companyId, definition, agent);
const byKind = new Map(existingStates.map((state) => [state.resourceKind, state]));
const instruction = selected.has("instructions")
? await materializeInstructions(agent, definition, mode)
? await materializeInstructions(agent, definition, mode, actor)
: byKind.get("instructions")!;
const refreshedAgent = await agentSvc.getById(agent.id) as Agent | null;
if (!refreshedAgent) throw notFound("Built-in agent not found");
@@ -1847,7 +1975,7 @@ export function builtInAgentService(db: Db) {
return state(definition, updated as Agent);
}
async function reset(companyId: string, key: string, input: { resources?: Array<"agent" | "instructions" | "skill" | "routine"> } = {}) {
async function reset(companyId: string, key: string, input: { resources?: Array<"agent" | "instructions" | "skill" | "routine"> } = {}, actor?: AuthorizationActor) {
const definition = requireBuiltInAgentDefinition(key);
const resetAgentDefaults = !input.resources || input.resources.includes("agent");
const current = resetAgentDefaults
@@ -1862,6 +1990,7 @@ export function builtInAgentService(db: Db) {
definition,
"reset",
input.resources ? selectedBundleResources ?? [] : undefined,
actor,
);
return state(definition, await agentSvc.getById(current.agent.id) as Agent, resources);
}
+1 -1
View File
@@ -112,7 +112,7 @@ function expandTargetKeysForLegacyCompatibility(targetKeys: string[]) {
return [...expanded];
}
export function changeConsentGateService(db: Db) {
export function changeConsentGateService(db: Db | Parameters<Parameters<Db["transaction"]>[0]>[0]) {
return {
assertConsented: async (input: {
companyId: string;
+1 -1
View File
@@ -3549,7 +3549,7 @@ export function companyPortabilityService(db: Db, storage?: StorageService) {
const companies = companyService(db);
const agents = agentService(db);
const assetRecords = assetService(db);
const instructions = agentInstructionsService();
const instructions = agentInstructionsService(db);
const access = accessService(db);
const projects = projectService(db);
const issues = issueService(db);
+118
View File
@@ -1,3 +1,5 @@
import { isAgentDirectoryCopy } from "./agent-directory-working-copies.js";
import type { PaperclipTurnContext } from "@paperclipai/adapter-utils/server-utils";
import { restoreNativeWorkspaceBestEffort } from "./native-runtime/native-workspace-best-effort.js";
import {
@@ -67,6 +69,7 @@ import {
startAdapterExecutionTargetPaperclipBridge,
} from "@paperclipai/adapter-utils/execution-target";
import { agentService } from "./agents.js";
import { agentInstructionWorkingCopyService, instructionWorkingCopyGuidance } from "./agent-instruction-working-copies.js";
import { normalizeLegacyRunnerProvider } from "@paperclipai/adapter-utils";
import fs from "node:fs/promises";
import path from "node:path";
@@ -3442,6 +3445,15 @@ const heartbeatRunSafeResultJsonColumn = sql<Record<string, unknown> | null>`
'error', left(${heartbeatRuns.resultJson} ->> 'error', ${HEARTBEAT_RUN_RESULT_SUMMARY_MAX_CHARS}),
'stdout', left(${heartbeatRuns.resultJson} ->> 'stdout', ${HEARTBEAT_RUN_RESULT_OUTPUT_MAX_CHARS}),
'stderr', left(${heartbeatRuns.resultJson} ->> 'stderr', ${HEARTBEAT_RUN_RESULT_OUTPUT_MAX_CHARS}),
'instructionSave', case when jsonb_typeof(${heartbeatRuns.resultJson} -> 'instructionSave') = 'object'
then jsonb_strip_nulls(jsonb_build_object(
'state', left(${heartbeatRuns.resultJson} #>> '{instructionSave,state}', 32),
'contract', left(${heartbeatRuns.resultJson} #>> '{instructionSave,contract}', 32),
'entryFile', left(${heartbeatRuns.resultJson} #>> '{instructionSave,entryFile}', 512),
'errorCode', left(${heartbeatRuns.resultJson} #>> '{instructionSave,errorCode}', 128),
'errorMessage', left(${heartbeatRuns.resultJson} #>> '{instructionSave,errorMessage}', 1024),
'storageWarning', left(${heartbeatRuns.resultJson} #>> '{instructionSave,storageWarning}', 1024)
)) end,
'workspaceRestoreFailure', case when ${heartbeatRuns.resultJson} ->> 'workspaceRestoreFailure'
in ('restore_permission_denied', 'restore_lock_timeout', 'restore_unsafe_archive', 'restore_failed')
then ${heartbeatRuns.resultJson} -> 'workspaceRestoreFailure' end,
@@ -9502,6 +9514,7 @@ export function heartbeatService(
environmentRuntimeService(db, {
pluginWorkerManager: options.pluginWorkerManager,
});
const instructionCopies = agentInstructionWorkingCopyService(db, { environmentRuntime });
const envOrchestrator = environmentRunOrchestrator(db, {
pluginWorkerManager: options.pluginWorkerManager,
environmentRuntime,
@@ -18868,6 +18881,14 @@ export function heartbeatService(
async function reapOrphanedRuns(opts?: { staleThresholdMs?: number }) {
const staleThresholdMs = opts?.staleThresholdMs ?? 0;
const now = new Date();
// Recovery never launches a provider or infers stopped ownership from
// terminal status. Uncaptured local copies require durable stop evidence.
await instructionCopies.recoverStopped().catch(error => {
logger.warn({ err: error }, "failed to recover stopped instruction copies");
});
await instructionCopies.recoverCaptured().catch(error => {
logger.warn({ err: error }, "failed to retry captured instruction revisions");
});
// Complete persisted native results before generic orphan recovery. The
// reconciler reads the durable workspace barrier and persisted runtime
@@ -22350,6 +22371,29 @@ export function heartbeatService(
await bindIssueToPersistedExecutionWorkspace(persistedExecutionWorkspace);
const workspaceRealization = realizationResult.workspaceRealization;
const executionTarget = realizationResult.executionTarget;
let instructionCopy: Awaited<ReturnType<typeof instructionCopies.prepare>> = null;
let instructionSave: Record<string, unknown> | null = null;
const collectStoppedInstructions = async () => {
if (!instructionCopy) return;
let saved = await instructionCopies.collectStopped({ companyId: agent.companyId, runId: run.id, target: executionTarget });
// Capture before disposal. Exhausted bounded collection leaves a durable
// explicit loss report, never a claim that missing bytes were saved.
while (saved?.state === "pending_collection" && saved.attempts < 3) {
saved = await instructionCopies.collectStopped({ companyId: agent.companyId, runId: run.id, target: executionTarget });
}
if (!saved) return;
const receipt = parseObject(saved.receipt);
const storageWarning = readNonEmptyString(receipt.storageWarning);
instructionSave = { state: saved.state, entryFile: saved.entryFile,
...(isAgentDirectoryCopy(saved) ? { contract: "agent_files", appliedCandidateHash: saved.candidateHash }
: { revisionId: parseObject(receipt.revision).id ?? null }), storageWarning, errorCode: saved.errorCode, errorMessage: saved.errorMessage };
await appendRunEvent(run, { eventType: "instruction_save", stream: "system",
level: !storageWarning && ["saved", "unchanged", "resolved"].includes(saved.state) ? "info" : "warn",
message: storageWarning ?? (saved.state === "saved" ? "Agent files saved."
: saved.state === "unchanged" ? "Instruction working copy is unchanged."
: saved.errorMessage ?? "Instruction edits were not saved. Review the preserved candidate in the agent instruction editor."),
payload: instructionSave });
};
if (managedAiRuntime && aiBinding) {
try { await assertManagedAiProjectAuth({ ...resolvedConfig, cwd: executionWorkspace.cwd }, aiBinding.provider, executionTarget); }
catch { throw new ConfigurationIncompleteFailure("Project authentication conflicts with this agent’s managed AI connection", { configurationIncomplete: { reason: "ai_connection_incompatible", actionUrl: `/agents/${agent.id}/runtime` } }); }
@@ -23159,6 +23203,62 @@ export function heartbeatService(
target: executionTarget,
workspaceId: persistedExecutionWorkspace?.id ?? null,
});
const hasInstructionFilesystem = nativeRuntimeResolution.kind !== "native"
? adapter.supportsInstructionsBundle === true
: !["claude_managed_agents_api", "aws_agentcore_harness_api"].includes(nativeRuntimeResolution.profile.backend);
if (hasInstructionFilesystem) {
try {
// Missing contract fields on a restored session mean the deployed
// legacy format. New sessions opt into whole-directory persistence.
const priorFileRun = taskSessionForRun?.lastRunId
? await db.select({ profile: heartbeatRuns.runnerProfileJson }).from(heartbeatRuns).where(and(
eq(heartbeatRuns.id, taskSessionForRun.lastRunId), eq(heartbeatRuns.companyId, agent.companyId), eq(heartbeatRuns.agentId, agent.id))).then(rows => rows[0])
: null;
const savedFileInput = parseObject(parseObject(run.runnerProfileJson).nativeExecutionInput);
const priorFileInput = Object.keys(savedFileInput).length ? savedFileInput : parseObject(parseObject(priorFileRun?.profile).nativeExecutionInput);
const priorWorkingCopy = parseObject(parseObject(parseObject(priorFileInput.runtimeContext).instructions).workingCopy);
instructionCopy = await instructionCopies.prepare({
companyId: agent.companyId, agentId: agent.id, runId: run.id,
target: executionTarget, cwd: executionWorkspace.cwd,
legacy: Object.keys(priorFileInput).length > 0 && priorWorkingCopy.kind !== "agent_files",
});
} catch (error) {
if ((error as { status?: number }).status !== 403) throw error;
// Missing write identity must not break a background run's read-only
// prompt. It must also never imply that ordinary file edits will save.
await appendRunEvent(run, { eventType: "instruction_save", stream: "system", level: "warn",
message: "Persistent instruction editing is unavailable. Use an authenticated user with instruction edit access and a managed instruction bundle.",
payload: { state: "unavailable", code: "INSTRUCTION_COPY_UNAVAILABLE" } });
const guidance = "No editable agent instruction working copy is registered for this turn. Use authenticated agent file tools for persistent edits; do not edit a private copy named in an earlier turn or claim its changes will persist.";
for (const key of ["paperclipTaskMarkdown", "paperclipTaskMarkdownCompact"]) {
context[key] = [readNonEmptyString(context[key]), guidance].filter(Boolean).join("\n\n");
}
}
if (instructionCopy) {
const storageWarning = readNonEmptyString(instructionCopy.receipt?.storageWarning);
if (storageWarning) {
instructionSave = { state: "prepared", contract: "agent_files", storageWarning };
// This is an advisory on the run, never an agent pause, execution
// failure, or scheduling gate. Keep it visible while work runs.
await db.update(heartbeatRuns).set({ resultJson: sql`coalesce(${heartbeatRuns.resultJson}, '{}'::jsonb) || ${JSON.stringify({ instructionSave })}::jsonb` }).where(eq(heartbeatRuns.id, run.id));
await appendRunEvent(run, { eventType: "instruction_save", stream: "system", level: "warn",
message: storageWarning, payload: instructionSave });
}
runtimeConfig = { ...runtimeConfig, instructionsFilePath: path.join(instructionCopy.localRoot, instructionCopy.entryFile) };
if (isAgentDirectoryCopy(instructionCopy)) {
const workspace = parseObject(context.paperclipWorkspace);
context.paperclipWorkspace = { ...workspace, agentHome: instructionCopy.executionRoot,
// Keep the pre-existing permission root stable for ACP session
// identity. The per-run copy is already under the company root.
agentHomeForPermissions: workspace.agentHome,
};
}
const guidance = instructionWorkingCopyGuidance(instructionCopy);
for (const key of ["paperclipTaskMarkdown", "paperclipTaskMarkdownCompact"]) {
context[key] = [readNonEmptyString(context[key]), guidance].filter(Boolean).join("\n\n");
}
}
}
let nativeExecution: NativeExecutionInput | null = null;
let nativeRunnerInstanceId: string | null = null;
if (nativeRuntimeResolution.kind === "native") {
@@ -23588,6 +23688,7 @@ export function heartbeatService(
runId: run.id,
runtimeConfig,
runtimeSkillEntries,
instructionWorkingCopy: instructionCopy ? { rootPath: instructionCopy.executionRoot, entryPath: instructionCopy.entryFile, ...(isAgentDirectoryCopy(instructionCopy) ? { kind: "agent_files" as const } : {}) } : undefined,
});
const nativeExecutionWithCheckpoint =
buildNativeExecutionWithCheckpoint({
@@ -24295,6 +24396,10 @@ export function heartbeatService(
},
onLog,
onEvent: onAdapterEvent,
instructionWorkingCopy: instructionCopy ? {
hasChanges: () => instructionCopies.hasChanges({ companyId: agent.companyId, runId: run.id, target: executionTarget }),
collectStopped: collectStoppedInstructions,
} : undefined,
preparationSpans: nativeRunnerPreparationSpans,
// Bootstrap with executable/home discovery while keeping
// configured provider values and the server-selected
@@ -24308,6 +24413,7 @@ export function heartbeatService(
process.env,
executionWorkspace.cwd,
),
...(instructionCopy && isAgentDirectoryCopy(instructionCopy) ? { AGENT_HOME: instructionCopy.executionRoot } : {}),
...(nativeMcpServer
? {
PAPERCLIP_NATIVE_MCP_NAME: nativeMcpServer.name,
@@ -24488,6 +24594,7 @@ export function heartbeatService(
issueId,
);
},
onProviderStopped: collectStoppedInstructions,
onDispatch: markDispatchStarted,
signal: executionControl.controller.signal,
...(executionTarget?.kind === "remote" && executionTarget.transport === "sandbox" ? {
@@ -24536,6 +24643,7 @@ export function heartbeatService(
if (!guardedDispatch.dispatched) return;
adapterResult = await guardedDispatch.resultPromise;
}
if (instructionSave) adapterResult.resultJson = { ...adapterResult.resultJson, instructionSave };
adapterResult = applyWorkspaceRestoreFailure(adapterResult);
// A returned result can include a failed restore. Keep the workspace
// barrier closed until required files have been restored.
@@ -24771,6 +24879,7 @@ export function heartbeatService(
"failed to revoke heartbeat-run MCP gateway tokens",
);
}
await instructionCopies.release(agent.companyId, run.id);
}
// Reconcile the referenced-project set against the real remote staging outcome. A referenced
// project can pass authorization and clone locally at run prep, then fail to stage into the
@@ -26162,6 +26271,15 @@ export function heartbeatService(
},
);
}
// A retained or unverified process stays above this release boundary.
// If no stopped-copy capture occurred, preserve an explicit loss report.
const uncapturedInstructions = await instructionCopies.reportUnavailable(run.companyId, run.id);
if (uncapturedInstructions?.state === "unavailable") {
await appendRunEvent(run, { eventType: "instruction_save", stream: "system", level: "warn",
message: "Instruction edits could not be recovered before environment release. No instruction save is claimed.",
payload: { state: "unavailable", code: uncapturedInstructions.errorCode } });
}
await instructionCopies.release(run.companyId, run.id);
await releaseEnvironmentLeasesForRun({
runId: run.id,
companyId: run.companyId,
@@ -0,0 +1,89 @@
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { randomUUID } from "node:crypto";
import { eq } from "drizzle-orm";
import { afterAll, beforeAll, beforeEach, describe, expect, it } from "vitest";
import { agents, authUsers, companies, companyMemberships, principalPermissionGrants, heartbeatRuns, issues, agentInstructionRevisions, createDb } from "@paperclipai/db";
import { startEmbeddedPostgresTestDatabase } from "../../__tests__/helpers/embedded-postgres.js";
import { resolveManagedInstructionsRoot } from "../agent-instructions.js";
import { PaperclipRunnerToolAuthority } from "./paperclip-runner-tool-authority.js";
describe("canonical instruction tools through native authority", () => {
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
let db: ReturnType<typeof createDb>;
let home: string;
const previousHome = process.env.PAPERCLIP_HOME;
let companyId: string, agentId: string, targetAgentId: string, userId: string, runId: string, issueId: string, root: string;
let authority: PaperclipRunnerToolAuthority;
const entryFile = "policy/ENTRY.md";
const original = "\uFEFF# Original\r\n\0☃\n";
beforeAll(async () => {
home = await fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), "instruction-tool-home-")));
process.env.PAPERCLIP_HOME = home;
database = await startEmbeddedPostgresTestDatabase("instruction-tool-db-");
db = createDb(database.connectionString);
}, 90_000);
afterAll(async () => {
if (previousHome === undefined) delete process.env.PAPERCLIP_HOME; else process.env.PAPERCLIP_HOME = previousHome;
await database?.cleanup();
if (home) await fs.rm(home, { recursive: true, force: true });
});
beforeEach(async () => {
[companyId, agentId, targetAgentId, userId, runId, issueId] = Array.from({ length: 6 }, () => randomUUID());
await db.insert(companies).values({ id: companyId, name: "Tool scope", issuePrefix: randomUUID().slice(0, 8) });
await db.insert(authUsers).values({ id: userId, name: "Editor", email: `${userId}@example.test`, createdAt: new Date(), updatedAt: new Date() });
root = resolveManagedInstructionsRoot({ id: targetAgentId, companyId, name: "Target", adapterConfig: {} });
await db.insert(agents).values([
{ id: agentId, companyId, name: "Caller", status: "active" },
{ id: targetAgentId, companyId, name: "Target", adapterConfig: { instructionsBundleMode: "managed", instructionsRootPath: root, instructionsEntryFile: entryFile } },
]);
await db.insert(companyMemberships).values([
{ companyId, principalType: "user", principalId: userId, membershipRole: "operator" },
{ companyId, principalType: "agent", principalId: agentId, membershipRole: "member" },
]);
await db.insert(principalPermissionGrants).values([
{ companyId, principalType: "user", principalId: userId, permissionKey: "agents:configure", scope: { agentIds: [targetAgentId] } },
{ companyId, principalType: "agent", principalId: agentId, permissionKey: "agents:configure", scope: { agentIds: [targetAgentId] } },
]);
await db.insert(issues).values({ id: issueId, companyId, title: "Maintain instructions", status: "in_progress", workMode: "standard", assigneeAgentId: agentId });
await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, status: "running", runtimeMode: "native", nativeIssueId: issueId, invocationSource: "on_demand", responsibleUserId: userId });
await db.update(issues).set({ executionRunId: runId }).where(eq(issues.id, issueId));
authority = new PaperclipRunnerToolAuthority(db, { companyId, agentId, issueId, runId });
await fs.mkdir(path.dirname(path.join(root, entryFile)), { recursive: true });
await fs.writeFile(path.join(root, entryFile), original);
});
const call = (tool: string, args: Record<string, unknown> = {}) => authority.execute({ tool, callId: randomUUID(), arguments: { targetAgentId, ...args } }) as Promise<any>;
it("bridges old native instruction tools to current files without writing history", async () => {
const first = await call("read_agent_instructions");
expect(first).toMatchObject({ entryFile, content: original });
const changed = await call("update_agent_instructions", { entryFile, content: "changed\r\n", baseRevisionId: first.revision.id });
expect(changed.revision).toMatchObject({ responsibleUserId: userId, actorAgentId: agentId, sourceRunId: runId });
expect((await call("get_agent_instruction_history", { entryFile })).revisions).toHaveLength(0);
expect(await fs.readFile(path.join(root, entryFile), "utf8")).toBe("changed\r\n");
await expect(call("update_agent_instructions", { entryFile, content: "stale", baseRevisionId: first.revision.id })).rejects.toMatchObject({ status: 409 });
expect((await call("read_agent_instructions")).content).toBe("changed\r\n");
});
it("rejects supplied identity and missing CAS bases before changing content", async () => {
const first = await call("read_agent_instructions");
for (const identity of ["companyId", "agentId", "runId", "responsibleUserId", "onBehalfOfUserId", "actor"]) {
await expect(call("update_agent_instructions", { entryFile, content: "forged", baseRevisionId: first.revision.id, [identity]: randomUUID() })).rejects.toMatchObject({ status: 400 });
}
await expect(call("update_agent_instructions", { entryFile, content: "missing base" })).rejects.toMatchObject({ status: 400 });
expect((await call("read_agent_instructions")).content).toBe(original);
expect(await db.select().from(agentInstructionRevisions).where(eq(agentInstructionRevisions.agentId, targetAgentId))).toHaveLength(0);
});
it("rechecks responsible-user access and rejects a cross-company target", async () => {
const first = await call("read_agent_instructions");
await db.delete(principalPermissionGrants).where(eq(principalPermissionGrants.principalId, userId));
await expect(call("update_agent_instructions", { entryFile, content: "revoked", baseRevisionId: first.revision.id })).rejects.toMatchObject({ status: 403 });
const foreignCompanyId = randomUUID(), foreignAgentId = randomUUID();
await db.insert(companies).values({ id: foreignCompanyId, name: "Foreign", issuePrefix: randomUUID().slice(0, 8) });
await db.insert(agents).values({ id: foreignAgentId, companyId: foreignCompanyId, name: "Foreign" });
await expect(call("read_agent_instructions", { targetAgentId: foreignAgentId })).rejects.toMatchObject({ status: 404 });
await expect(call("update_agent_instructions", { targetAgentId: foreignAgentId, entryFile, content: "foreign", baseRevisionId: null })).rejects.toMatchObject({ status: 404 });
await db.update(heartbeatRuns).set({ status: "cancelled" }).where(eq(heartbeatRuns.id, runId));
await expect(call("read_agent_instructions")).rejects.toThrow("paperclip_runner_tool_binding_not_authorized");
expect(await fs.readFile(path.join(root, entryFile), "utf8")).toBe(original);
});
});
@@ -0,0 +1,20 @@
import { describe, expect, it } from "vitest";
import type { Db } from "@paperclipai/db";
import { PaperclipRunnerToolAuthority } from "./paperclip-runner-tool-authority.js";
const binding = { companyId: "00000000-0000-4000-8000-000000000001", agentId: "00000000-0000-4000-8000-000000000002", issueId: "00000000-0000-4000-8000-000000000003", runId: "00000000-0000-4000-8000-000000000004" };
describe("dedicated canonical instruction tools", () => {
it("advertises read, CAS update, history, and restore on ordinary native runs", () => {
const authority = new PaperclipRunnerToolAuthority({} as Db, binding);
const tools = authority.definitions();
expect(tools.map((tool) => tool.name)).toEqual(expect.arrayContaining([
"read_agent_instructions", "update_agent_instructions", "get_agent_instruction_history", "restore_agent_instructions",
]));
for (const name of ["update_agent_instructions", "restore_agent_instructions"]) {
const tool = tools.find((candidate) => candidate.name === name)!;
expect(tool.inputSchema).toMatchObject({ required: expect.arrayContaining(["baseRevisionId"]), additionalProperties: false });
expect(tool.inputSchema).not.toHaveProperty("properties.responsibleUserId");
}
});
});
@@ -0,0 +1,76 @@
import type { AgentInstructionCommitReceipt } from "@paperclipai/shared";
import { z } from "zod";
import type { Db } from "@paperclipai/db";
import { badRequest, notFound } from "../../errors.js";
import { agentInstructionRevisionService } from "../agent-instruction-revisions.js";
import { agentInstructionWorkingCopyService } from "../agent-instruction-working-copies.js";
import { deriveBundleState } from "../agent-instructions.js";
import { agentService } from "../agents.js";
import type { AuthorizationActor } from "../authorization.js";
const target = { targetAgentId: z.string().uuid().optional() };
const entryFile = z.string().min(1).max(4096);
const baseRevisionId = z.string().uuid().nullable();
const schemas = {
read_agent_instructions: z.object({ ...target, entryFile: entryFile.optional(), revisionId: z.string().uuid().optional() }).strict()
.refine((input) => Boolean(input.entryFile) === Boolean(input.revisionId), "Historical reads require both entryFile and revisionId"),
update_agent_instructions: z.object({ ...target, entryFile, content: z.string().max(1024 * 1024), baseRevisionId }).strict(),
get_agent_instruction_history: z.object({ ...target, entryFile, cursor: z.string().min(1).max(2048).optional(), limit: z.number().int().min(1).max(100).optional() }).strict(),
restore_agent_instructions: z.object({ ...target, entryFile, revisionId: z.string().uuid(), baseRevisionId: z.string().uuid() }).strict(),
};
export type AgentInstructionToolName = keyof typeof schemas;
/** Called only after the native authority validates its server-bound active run. */
export async function executeAgentInstructionTool(input: {
db: Db;
binding: { companyId: string; agentId: string; runId: string };
tool: AgentInstructionToolName;
arguments: unknown;
}) {
// Never accept an actor, responsible user, company, or source run from tool JSON.
const actor: AuthorizationActor = { type: "agent", source: "agent_jwt", companyId: input.binding.companyId, agentId: input.binding.agentId, runId: input.binding.runId };
const service = agentInstructionRevisionService(input.db);
const parse = <T>(schema: z.ZodType<T>): T => {
const result = schema.safeParse(input.arguments);
if (!result.success) throw badRequest("Invalid instruction tool arguments", result.error.flatten());
return result.data;
};
const scope = (targetAgentId?: string) => ({ companyId: input.binding.companyId, agentId: targetAgentId ?? input.binding.agentId });
const acknowledge = async (receipt: AgentInstructionCommitReceipt, targetAgentId?: string) => {
// The commit is already durable. A failed baseline refresh keeps the old CAS
// fence, so later cleanup preserves a conflict instead of overwriting it.
await agentInstructionWorkingCopyService(input.db).acknowledgeExplicitSave({
...scope(targetAgentId), runId: input.binding.runId, entryFile: receipt.revision.entryFile,
revisionId: receipt.revision.id, contentHash: receipt.revision.contentHash,
}).catch(() => undefined);
return receipt;
};
switch (input.tool) {
case "read_agent_instructions": {
const args = parse(schemas.read_agent_instructions);
const target = scope(args.targetAgentId);
if (args.entryFile && args.revisionId) return service.readRevision({ ...target, entryFile: args.entryFile, revisionId: args.revisionId }, actor);
const snapshot = await service.readCurrent(target, actor);
if (snapshot) return { ...snapshot, entryFile: snapshot.revision.entryFile };
// An unseeded empty entry still needs its configured filename for the first CAS.
const agent = await agentService(input.db).getById(target.agentId);
if (!agent || agent.companyId !== target.companyId) throw notFound("Agent not found");
return { entryFile: deriveBundleState(agent).entryFile, revision: null, content: null };
}
case "update_agent_instructions": {
const args = parse(schemas.update_agent_instructions);
const receipt = await service.commit({ ...scope(args.targetAgentId), entryFile: args.entryFile, content: args.content, baseRevisionId: args.baseRevisionId, source: "tool" }, actor);
return acknowledge(receipt, args.targetAgentId);
}
case "get_agent_instruction_history": {
const args = parse(schemas.get_agent_instruction_history);
return service.history({ ...scope(args.targetAgentId), entryFile: args.entryFile, cursor: args.cursor, limit: args.limit }, actor);
}
case "restore_agent_instructions": {
const args = parse(schemas.restore_agent_instructions);
const receipt = await service.restore({ ...scope(args.targetAgentId), entryFile: args.entryFile, revisionId: args.revisionId, baseRevisionId: args.baseRevisionId }, actor);
return acknowledge(receipt, args.targetAgentId);
}
}
}
@@ -5917,6 +5917,57 @@ describe("native session same-turn steering", () => {
});
describe("native warm session supervision", () => {
it.each([
{ changed: false, closeFails: false },
{ changed: true, closeFails: false },
{ changed: true, closeFails: true },
])("collects changed instructions only after the owned warm provider stops (changed=$changed, close fails=$closeFails)", async ({ changed, closeFails }) => {
const identity = `instruction-close-${changed}-${closeFails}`;
let releaseClose!: () => void;
const closed = new Promise<void>((resolve) => { releaseClose = resolve; });
const close = vi.fn(async () => { await closed; if (closeFails) throw new Error("instruction provider close failed"); });
const collectStopped = vi.fn(async () => {});
const hasChanges = vi.fn(async () => changed);
const warmExecution = { ...execution,
binding: { ...execution.binding, runId: identity, executionWorkspaceId: identity },
session: { ...execution.session, normalizedSessionId: identity, lifecyclePolicy: { mode: "warm" as const, idleTimeoutMs: 60_000 } },
} as NativeExecutionInputV1;
state.execute.mockReset().mockImplementationOnce(async (options) => {
expect(options.requireSessionCloseBeforeReturn).toBe(true);
expect(options.onSessionClosed).toBe(collectStopped);
await options.onSession?.({ close });
return { result: { summary: "completed" }, terminal: { runTerminalState: "succeeded" },
turnId: identity, normalizedSessionId: identity, providerSessionId: identity,
driverKind: "test", driverVersion: "1", nativeEventCount: 1, highestContiguousSourceSeq: 1, usage: null };
});
const running = executePaperclipNativeSession({ db: leaseDb(warmExecution), execution: warmExecution,
runnerInstanceId: identity, runnerExecutionTarget: { kind: "remote", transport: "sandbox", environmentId: identity, remoteCwd: `/tmp/${identity}` },
instructionWorkingCopy: { hasChanges, collectStopped } });
const observed = running.then(() => null, error => error);
try {
if (changed) {
await vi.waitFor(() => expect(close).toHaveBeenCalledOnce());
expect(collectStopped).not.toHaveBeenCalled();
releaseClose();
const error = await observed;
if (closeFails) {
expect(error?.message).toBe("instruction provider close failed");
expect(collectStopped).not.toHaveBeenCalled();
} else {
expect(error).toBeNull();
expect(collectStopped).toHaveBeenCalledOnce();
}
} else {
expect(await observed).toBeNull();
expect(close).not.toHaveBeenCalled();
expect(collectStopped).not.toHaveBeenCalled();
}
} finally {
releaseClose();
await closeWarmNativeSessionsForEnvironment({ environmentId: identity, reason: "test cleanup" });
}
});
describe("warm session identity transitions", () => {
let previousHome: string | undefined;
let isolatedHome: string;
@@ -7181,6 +7181,12 @@ export async function executePaperclipNativeSession(input: {
chatAttachmentReadScope?: NativeChatAttachmentReadScope;
onLog?: (stream: "stdout" | "stderr", chunk: string) => Promise<void>;
onEvent?: (event: AdapterRuntimeEvent) => Promise<void>;
/** Only this run's registered private instruction entry.
* Probe at terminal; persist only after owned shutdown. */
instructionWorkingCopy?: {
hasChanges: () => Promise<boolean>;
collectStopped: () => Promise<void>;
};
/** Persist task-level continuity before a durable goal can outlive this run. */
onGoalCheckpoint?: (snapshot: PersistedNativeSession) => Promise<void>;
sessionGoalControl?: NativeSessionGoalControl | null;
@@ -8320,7 +8326,8 @@ async function executePaperclipNativeSessionWithinScope(
resumeSessionGoalHeartbeat: input.resumeSessionGoalHeartbeat,
// Every durable runner must finish its bounded suspension before
// the next run verifies and rotates the saved authority.
requireSessionCloseBeforeReturn: runnerdBackend !== null,
requireSessionCloseBeforeReturn: runnerdBackend !== null || input.instructionWorkingCopy !== undefined,
onSessionClosed: input.instructionWorkingCopy?.collectStopped,
onCheckpoint: async (snapshot) => {
if (warmSessionId !== null && warmConfigDigest !== null) {
await persistWarmNativeCheckpoint(
@@ -9080,12 +9087,22 @@ async function executePaperclipNativeSessionWithinScope(
// A following run cannot attach until the prior run's durable finalization
// is committed. Provider completion alone is not an authority boundary.
if (warmSessionId !== null && lifecyclePolicy.mode === "warm") {
const instructionCopy = input.instructionWorkingCopy;
const ownedSession = warmNativeSessions.get(warmSessionId);
const collectInstructions = Boolean(instructionCopy && ownedSession?.ownerToken === warmSessionOwnerToken &&
await instructionCopy.hasChanges());
if (collectInstructions && ownedSession) {
// Keep the unchanged warm path intact. A changed private instruction copy
// requires the existing checkpoint-and-close boundary before collection.
ownedSession.closeOnReleaseReason = "registered instruction edits require stopped-provider collection";
}
await releaseWarmNativeSession(
warmSessionId,
warmSessionOwnerToken,
lifecyclePolicy.idleTimeoutMs,
false,
);
if (collectInstructions) await instructionCopy!.collectStopped();
}
const adapterResult: AdapterExecutionResult = {
exitCode: native.terminal.runTerminalState === "succeeded" ? 0 : 1,
@@ -99,7 +99,7 @@ describe("PaperclipRunnerToolAuthority", () => {
issueId,
runId,
});
expect(authority.definitions()).toHaveLength(30);
expect(authority.definitions()).toHaveLength(34);
const questions = authority.definitions().find(tool => tool.name === "request_human_input")!;
expect(questions.description).toContain("ask only the next unanswered question");
expect(questions.description).toContain("Never infer answers");
@@ -1,3 +1,4 @@
import { executeAgentInstructionTool } from "./agent-instruction-tools.js";
import { createReadStream } from "node:fs";
import { publicChatTaskUrl } from "../chat-task-url.js";
import type { createAssignedMcpTools } from "./assigned-mcp-tools.js";
@@ -83,6 +84,7 @@ import {
} from "./chat-attachment-read.js";
const IMPLEMENTED_OPERATIONS = new Set([
"read_agent_instructions", "update_agent_instructions", "get_agent_instruction_history", "restore_agent_instructions",
"search_api", "call_api", "hire_agent",
"get_task_context", "get_task_history", "search_tasks", "report_progress",
"request_human_input",
@@ -368,6 +370,13 @@ export class PaperclipRunnerToolAuthority {
throw new Error("paperclip_runner_tool_mode_denied");
}
switch (call.tool) {
case "read_agent_instructions":
case "update_agent_instructions":
case "get_agent_instruction_history":
case "restore_agent_instructions":
return executeAgentInstructionTool({ db: this.db, binding: {
companyId: this.binding.companyId, agentId: this.binding.agentId, runId: this.binding.runId,
}, tool: call.tool, arguments: call.arguments });
case "create_skill": {
const apiUrl = this.binding.apiUrl ?? process.env.PAPERCLIP_API_URL;
const token = createLocalAgentJwt(this.binding.agentId, this.binding.companyId, context.actor.adapterType, this.binding.runId, context.run.responsibleUserId);
@@ -42,6 +42,10 @@ function words(text: string): string[] {
}
function dedicatedTools(method: string, path: string): string[] {
if (/\/agents\/\{[^}]+\}\/instructions-bundle\/file$/.test(path)) return method === "GET" ? ["read_agent_instructions"] : method === "PUT" ? ["update_agent_instructions"] : [];
if (/\/agents\/\{[^}]+\}\/instructions-bundle\/history$/.test(path) && method === "GET") return ["get_agent_instruction_history"];
if (/\/agents\/\{[^}]+\}\/instructions-bundle\/revision\/\{[^}]+\}$/.test(path) && method === "GET") return ["read_agent_instructions"];
if (/\/agents\/\{[^}]+\}\/instructions-bundle\/restore$/.test(path) && method === "POST") return ["restore_agent_instructions"];
if (/\/companies\/\{[^}]+\}\/skills$/.test(path) && method === "POST") return ["create_skill"];
if (/\/companies\/\{[^}]+\}\/agent-hires$/.test(path) && method === "POST") return ["hire_agent"];
if (/\/projects$/.test(path)) return method === "GET" ? ["list_projects"] : method === "POST" ? ["create_project"] : [];
@@ -72,6 +72,10 @@ describe("runner API catalog", () => {
expect(runnerApiOperation(createProject).dedicatedTools).toEqual(["create_project"]);
expect(runnerApiOperation(projects).dedicatedTools).toEqual(["list_projects"]);
expect(runnerApiOperation("GET /api/companies/{companyId}/project-repositories").dedicatedTools).toEqual(["list_project_repositories"]);
expect(runnerApiOperation("GET /api/agents/{id}/instructions-bundle/file").dedicatedTools).toEqual(["read_agent_instructions"]);
expect(runnerApiOperation("PUT /api/agents/{id}/instructions-bundle/file").dedicatedTools).toEqual(["update_agent_instructions"]);
expect(runnerApiOperation("GET /api/agents/{id}/instructions-bundle/history").dedicatedTools).toEqual(["get_agent_instruction_history"]);
expect(runnerApiOperation("POST /api/agents/{id}/instructions-bundle/restore").dedicatedTools).toEqual(["restore_agent_instructions"]);
expect(runnerApiOperation("POST /api/companies/{companyId}/agent-hires").dedicatedTools).toEqual(["hire_agent"]);
expect(runnerApiOperation("POST /api/companies/{companyId}/agent-hires").dedicatedToolGuidance).toContain("inherits the caller's native runtime");
});
@@ -10,6 +10,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
const serviceMocks = vi.hoisted(() => ({
exportFiles: vi.fn(),
readCommittedForRuntime: vi.fn(),
getEffectiveProfilesForAgent: vi.fn(),
githubBotConnectionIdsForRun: vi.fn(),
}));
@@ -20,6 +21,11 @@ vi.mock("../chat-github-tools.js", () => ({
vi.mock("../agent-instructions.js", () => ({
agentInstructionsService: () => ({ exportFiles: serviceMocks.exportFiles }),
agentInstructionsBundleMode: (agent: { adapterConfig?: { instructionsBundleMode?: string } }) => agent.adapterConfig?.instructionsBundleMode ?? null,
}));
vi.mock("../agent-instruction-revisions.js", () => ({
agentInstructionRevisionService: () => ({ readCommittedForRuntime: serviceMocks.readCommittedForRuntime }),
}));
vi.mock("../tool-access.js", () => ({
@@ -50,6 +56,7 @@ async function makeTreeWritable(target: string): Promise<void> {
beforeEach(async () => {
vi.clearAllMocks();
serviceMocks.readCommittedForRuntime.mockResolvedValue(null);
serviceMocks.githubBotConnectionIdsForRun.mockResolvedValue(new Set());
previousPaperclipHome = process.env.PAPERCLIP_HOME;
previousInstanceId = process.env.PAPERCLIP_INSTANCE_ID;
@@ -90,6 +97,60 @@ afterEach(async () => {
});
describe("buildNativeRuntimeContext", () => {
it("uses committed instructions when the disk projection is stale", async () => {
serviceMocks.exportFiles.mockResolvedValue({
entryFile: "AGENTS.md",
files: { "AGENTS.md": "Old disk projection\n", "reference.md": "Sibling reference\n" },
});
serviceMocks.readCommittedForRuntime.mockResolvedValue({
revision: { entryFile: "AGENTS.md" }, content: "Saved canonical instructions\n",
});
const context = await buildNativeRuntimeContext({
db: {} as Db,
agent: { id: "agent-1", companyId: "company-1", name: "Reviewer", adapterType: "paperclip_runner", adapterConfig: { instructionsBundleMode: "managed" } },
runId: "run-1", runtimeConfig: {}, runtimeSkillEntries: [],
});
expect(await readFile(path.join(context.instructions.bundle.rootPath, context.instructions.entryPath), "utf8"))
.toBe("Saved canonical instructions\n");
expect(await readFile(path.join(context.instructions.bundle.rootPath, "reference.md"), "utf8"))
.toBe("Sibling reference\n");
expect(serviceMocks.readCommittedForRuntime).toHaveBeenCalledWith({ companyId: "company-1", agentId: "agent-1" });
});
it("does not substitute canonical managed instructions into an external bundle", async () => {
serviceMocks.exportFiles.mockResolvedValue({ entryFile: "AGENTS.md", files: { "AGENTS.md": "External instructions\n" } });
const context = await buildNativeRuntimeContext({
db: {} as Db,
agent: { id: "agent-1", companyId: "company-1", name: "Reviewer", adapterConfig: { instructionsBundleMode: "external" } },
runId: "run-1", runtimeConfig: {}, runtimeSkillEntries: [],
});
expect(await readFile(path.join(context.instructions.bundle.rootPath, context.instructions.entryPath), "utf8"))
.toBe("External instructions\n");
expect(serviceMocks.readCommittedForRuntime).not.toHaveBeenCalled();
});
it("keeps the pinned prompt immutable while exposing the registered editable instruction copy", async () => {
serviceMocks.exportFiles.mockResolvedValue({
entryFile: "instructions/CHARter.md",
files: { "instructions/CHARter.md": "Original instructions.\n" },
});
const workingRoot = path.join(temporaryRoots.at(-1)!, "private-run-copy");
await mkdir(path.join(workingRoot, "instructions"), { recursive: true });
await writeFile(path.join(workingRoot, "instructions/CHARter.md"), "Original instructions.\n");
const context = await buildNativeRuntimeContext({
db: {} as Db,
agent: { id: "agent-1", companyId: "company-1", name: "Reviewer", adapterType: "paperclip_runner", adapterConfig: {} },
runId: "run-1",
runtimeConfig: {},
runtimeSkillEntries: [],
instructionWorkingCopy: { rootPath: workingRoot, entryPath: "instructions/CHARter.md" },
});
expect(context.instructions.workingCopy).toEqual({ rootPath: workingRoot, entryPath: "instructions/CHARter.md" });
await writeFile(path.join(workingRoot, "instructions/CHARter.md"), "Persist this next time.\n");
expect(await readFile(path.join(context.instructions.bundle.rootPath, context.instructions.entryPath), "utf8"))
.toBe("Original instructions.\n");
});
it.each(["disabled", "degraded"] as const)(
"omits an unavailable native MCP connection when it is %s without aborting runtime context creation",
async (unavailableState) => {
@@ -21,7 +21,8 @@ import {
type NativeRuntimeContextSnapshot,
} from "../../vendor/paperclip-runner/index.js";
import { resolvePaperclipInstanceRoot } from "../../home-paths.js";
import { agentInstructionsService } from "../agent-instructions.js";
import { agentInstructionsService, agentInstructionsBundleMode } from "../agent-instructions.js";
import { agentInstructionRevisionService } from "../agent-instruction-revisions.js";
import { toolAccessService } from "../tool-access.js";
import { filterResolvedGitHubConnectionsForRun } from "../git-credentials.js";
@@ -145,8 +146,24 @@ export async function materializeAsset(files: AssetFile[]): Promise<NativeRuntim
return { schema: NATIVE_RUNTIME_ASSET_SCHEMA, digest: assetDigest, manifestDigest, rootPath, fileCount: manifestFiles.length, totalBytes };
}
async function materializeInstructionBundle(agent: RuntimeAgent) {
async function materializeInstructionBundle(db: Db, agent: RuntimeAgent, agentFiles = false) {
if (agentFiles) {
const current = await agentInstructionRevisionService(db).readCommittedForRuntime({ companyId: agent.companyId, agentId: agent.id });
if (!current) throw new Error("Configured instruction entry is missing");
const entryPath = safeRelativePath(current.revision.entryFile, "instruction entry path");
return { entryPath, bundle: await materializeAsset([{ path: entryPath, content: Buffer.from(current.content, "utf8"), mode: 0o444 }]) };
}
const exported = await agentInstructionsService().exportFiles(agent, { rejectSymlinks: true });
if (agentInstructionsBundleMode(agent) === "managed") {
// The database head remains authoritative when a saved revision could not
// update its disk projection. Overlay the exact committed snapshot directly;
// a repair followed by a separate export can race another pending commit.
const current = await agentInstructionRevisionService(db).readCommittedForRuntime({ companyId: agent.companyId, agentId: agent.id });
if (current) {
exported.entryFile = current.revision.entryFile;
exported.files[current.revision.entryFile] = current.content;
}
}
const entryPath = safeRelativePath(exported.entryFile, "instruction entry path");
if (!(entryPath in exported.files)) throw new Error(`configured instruction entry is missing: ${entryPath}`);
const files = Object.entries(exported.files).map(([relativePath, content]) => ({ path: safeRelativePath(relativePath, "instruction path"), content: Buffer.from(content, "utf8"), mode: 0o444 }));
@@ -222,12 +239,17 @@ export async function resolveNativeRuntimeMcpSnapshot(input: { db: Db; agent: Pi
return { assignmentSetId: `sha256:${assignmentDigest}`, digest: assignmentDigest, bindingId: assignment.connections.length ? `native-mcp:${input.runId}` : null };
}
export async function buildNativeRuntimeContext(input: { db: Db; agent: RuntimeAgent; runId: string; runtimeConfig: Record<string, unknown>; runtimeSkillEntries: PaperclipSkillEntry[] }): Promise<NativeRuntimeContextSnapshot> {
export async function buildNativeRuntimeContext(input: { db: Db; agent: RuntimeAgent; runId: string; runtimeConfig: Record<string, unknown>; runtimeSkillEntries: PaperclipSkillEntry[]; instructionWorkingCopy?: { rootPath: string; entryPath: string; kind?: "agent_files" } }): Promise<NativeRuntimeContextSnapshot> {
const [instructions, skills, mcp] = await Promise.all([
materializeInstructionBundle(input.agent),
materializeInstructionBundle(input.db, input.agent, input.instructionWorkingCopy?.kind === "agent_files"),
materializeSelectedSkills(input.runtimeConfig, input.runtimeSkillEntries, input.agent.adapterType === "paperclip_runner"),
resolveNativeRuntimeMcpSnapshot({ db: input.db, agent: input.agent, runId: input.runId }),
]);
const snapshot = { prompt: { revision: PAPERCLIP_EXECUTION_PROMPT_REVISION, text: PAPERCLIP_EXECUTION_PROMPT, digest: nativeRuntimePromptDigest() }, instructions, skills, mcp };
const snapshot = {
prompt: { revision: PAPERCLIP_EXECUTION_PROMPT_REVISION, text: PAPERCLIP_EXECUTION_PROMPT, digest: nativeRuntimePromptDigest() },
instructions: { ...instructions, ...(input.instructionWorkingCopy ? { workingCopy: input.instructionWorkingCopy } : {}) },
skills,
mcp,
};
return parseNativeRuntimeContext({ ...snapshot, aggregateDigest: canonicalNativeRuntimeContextDigest(snapshot) });
}
+98 -29
View File
@@ -5,6 +5,8 @@ import {
companies,
pluginEntities,
pluginManagedResources,
plugins,
activityLog,
} from "@paperclipai/db";
import type {
Agent,
@@ -12,11 +14,13 @@ import type {
PluginManagedAgentDeclaration,
PluginManagedAgentResolution,
} from "@paperclipai/shared";
import { notFound } from "../errors.js";
import { isUuidLike } from "@paperclipai/shared";
import { conflict, forbidden, notFound } from "../errors.js";
import { agentService } from "./agents.js";
import { approvalService } from "./approvals.js";
import { logActivity } from "./activity-log.js";
import { agentInstructionsBundleMode, agentInstructionsService } from "./agent-instructions.js";
import { agentInstructionRevisionService } from "./agent-instruction-revisions.js";
const MANAGED_AGENT_ENTITY_TYPE = "managed_agent";
const DEFAULT_MANAGED_AGENT_ADAPTER_TYPE = "process";
@@ -180,7 +184,7 @@ export function pluginManagedAgentService(
) {
const agentSvc = agentService(db);
const approvalSvc = approvalService(db);
const instructions = agentInstructionsService();
const instructions = agentInstructionsService(db);
function declarationFor(agentKey: string) {
const declaration = options.manifest?.agents?.find((agent) => agent.agentKey === agentKey);
@@ -190,8 +194,8 @@ export function pluginManagedAgentService(
return declaration;
}
async function getBinding(companyId: string, agentKey: string) {
return db
async function getBinding(companyId: string, agentKey: string, client: Pick<Db, "select"> = db) {
return client
.select()
.from(pluginEntities)
.where(
@@ -210,6 +214,7 @@ export function pluginManagedAgentService(
agentId: string,
extraData: Record<string, unknown> = {},
effectiveAdapterType?: string,
client: Pick<Db, "select" | "insert" | "update"> = db,
) {
const adapterType = effectiveAdapterType ?? (await resolveManagedAdapterType(companyId, declaration));
const defaultsJson = {
@@ -227,7 +232,7 @@ export function pluginManagedAgentService(
budgetMonthlyCents: declaration.budgetMonthlyCents ?? 0,
instructions: declaration.instructions ?? null,
};
const managedResource = await db
const managedResource = await client
.select({ id: pluginManagedResources.id })
.from(pluginManagedResources)
.where(and(
@@ -238,12 +243,12 @@ export function pluginManagedAgentService(
))
.then((rows) => rows[0] ?? null);
if (managedResource) {
await db
await client
.update(pluginManagedResources)
.set({ resourceId: agentId, defaultsJson, updatedAt: new Date() })
.where(eq(pluginManagedResources.id, managedResource.id));
} else {
await db.insert(pluginManagedResources).values({
await client.insert(pluginManagedResources).values({
companyId,
pluginId: options.pluginId,
pluginKey: options.pluginKey,
@@ -265,9 +270,9 @@ export function pluginManagedAgentService(
lastReconciledAt: new Date().toISOString(),
...extraData,
};
const existing = await getBinding(companyId, declaration.agentKey);
const existing = await getBinding(companyId, declaration.agentKey, client);
if (existing) {
return db
return client
.update(pluginEntities)
.set({
scopeKind: "company",
@@ -281,7 +286,7 @@ export function pluginManagedAgentService(
.returning()
.then((rows) => rows[0]);
}
return db
return client
.insert(pluginEntities)
.values({
pluginId: options.pluginId,
@@ -305,6 +310,46 @@ export function pluginManagedAgentService(
return rows.find((row) => rowIsManagedAgent(row, options.pluginKey, declaration.agentKey)) ?? null;
}
async function relinkManagedAgent(companyId: string, declaration: PluginManagedAgentDeclaration, agentId: string) {
const adapterType = await resolveManagedAdapterType(companyId, declaration);
return db.transaction(async (tx) => {
const [agent] = await tx.select().from(agents).where(and(
eq(agents.id, agentId), eq(agents.companyId, companyId), ne(agents.status, "terminated"),
)).for("update");
if (!agent || !rowIsManagedAgent(agent, options.pluginKey, declaration.agentKey)) {
throw conflict("Managed agent ownership changed before relink");
}
const [plugin] = await tx.select().from(plugins).where(and(
eq(plugins.id, options.pluginId), eq(plugins.pluginKey, options.pluginKey),
)).for("share");
if (!plugin || plugin.status !== "ready" || !plugin.manifestJson.capabilities.includes("agents.managed")
|| !plugin.manifestJson.agents?.some((entry) => entry.agentKey === declaration.agentKey)) {
throw forbidden("Plugin relink is limited to its registered managed agent declaration");
}
const marker = agent.metadata!.paperclipManagedResource as Record<string, unknown>;
const previousPluginId = marker.pluginId;
if (typeof previousPluginId !== "string" || !isUuidLike(previousPluginId)) throw forbidden("Managed agent has no plugin owner");
if (previousPluginId !== options.pluginId) {
const [previousPlugin] = await tx.select({ id: plugins.id }).from(plugins).where(eq(plugins.id, previousPluginId));
if (previousPlugin) throw forbidden("Managed agent still belongs to another installed plugin");
}
// Hard uninstall cascades the bindings, but deliberately leaves the agent
// and its canonical history. Transfer only its proven stable-key marker to
// the replacement installation, atomically with the new scoped bindings.
await upsertBinding(companyId, declaration, agentId, {}, adapterType, tx);
const [relinked] = await tx.update(agents).set({
metadata: managedMetadata(options.pluginId, options.pluginKey, declaration, agent.metadata),
updatedAt: new Date(),
}).where(and(eq(agents.id, agentId), eq(agents.companyId, companyId))).returning();
await tx.insert(activityLog).values({
companyId, actorType: "plugin", actorId: options.pluginId,
action: "plugin.managed_agent.relinked", entityType: "agent", entityId: agentId,
details: { sourcePluginKey: options.pluginKey, managedResourceKey: declaration.agentKey, previousPluginId },
});
return relinked as Agent;
});
}
async function companyAdapterUsage(companyId: string) {
const rows = await db
.select({ adapterType: agents.adapterType })
@@ -336,23 +381,39 @@ export function pluginManagedAgentService(
const declared = declaredInstructionFiles(declaration, variables);
if (!declared) return agent;
const materialized = await instructions.materializeManagedBundle(
agent,
declared.files,
{
entryFile: declared.entryFile,
replaceExisting: materializeOptions.replaceExisting,
clearLegacyPromptTemplate: true,
},
);
let adapterConfig: Record<string, unknown>;
if (materializeOptions.replaceExisting) {
const revisions = agentInstructionRevisionService(db);
const target = { companyId, agentId: agent.id };
const actor = { type: "plugin" as const, pluginId: options.pluginId, pluginKey: options.pluginKey, agentKey: declaration.agentKey };
const baseline = await revisions.readForPluginReset(target, actor);
const receipt = await revisions.commitPluginReset({ ...target, entryFile: declared.entryFile,
baseRevisionId: baseline.snapshot?.revision.id ?? null, configuredEntryFile: baseline.configuredEntryFile, content: declared.files[declared.entryFile] ?? "" }, actor);
if (receipt.materialization === "pending") throw conflict("Instruction revision saved; retry reset to repair its disk copy", { revisionId: receipt.revision.id });
const refreshed = await agentSvc.getById(agent.id);
if (!refreshed) throw notFound("Managed agent not found");
for (const [file, content] of Object.entries(declared.files)) {
if (file !== declared.entryFile) await instructions.writeFile(refreshed, file, content);
}
// A stock reset owns declared instruction paths, not the agent's other
// persistent files (including a formerly configured entry).
adapterConfig = { ...refreshed.adapterConfig };
delete adapterConfig.promptTemplate;
delete adapterConfig.bootstrapPromptTemplate;
} else {
const materialized = await instructions.materializeManagedBundle(agent, declared.files, {
entryFile: declared.entryFile, replaceExisting: false, clearLegacyPromptTemplate: true,
});
adapterConfig = materialized.adapterConfig;
}
const updated = await agentSvc.update(agent.id, {
adapterConfig: materialized.adapterConfig,
adapterConfig,
}, {
recordRevision: {
source: `plugin:${optionsForRevisionSource()}:managed-agent-instructions`,
},
});
return (updated as Agent | null) ?? { ...agent, adapterConfig: materialized.adapterConfig };
return (updated as Agent | null) ?? { ...agent, adapterConfig };
}
async function managedInstructionDefaultDrift(
@@ -375,8 +436,7 @@ export function pluginManagedAgentService(
return { entryFile: declared.entryFile, changedFiles: [declared.entryFile] };
}
const paths = new Set([...Object.keys(declared.files), ...Object.keys(exported.files)]);
const changedFiles = [...paths]
const changedFiles = Object.keys(declared.files)
.filter((filePath) => (exported.files[filePath] ?? null) !== (declared.files[filePath] ?? null))
.sort((left, right) => left.localeCompare(right));
if (exported.entryFile !== declared.entryFile && !changedFiles.includes(declared.entryFile)) {
@@ -433,7 +493,7 @@ export function pluginManagedAgentService(
spentMonthlyCents: 0,
lastHeartbeatAt: null,
}) as Agent;
created = await materializeDeclaredInstructions(companyId, created, declaration, { replaceExisting: true });
created = await materializeDeclaredInstructions(companyId, created, declaration, { replaceExisting: false });
let approvalId: string | null = null;
if (requiresApproval) {
@@ -574,9 +634,7 @@ export function pluginManagedAgentService(
const relinkCandidate = await findRelinkCandidate(companyId, declaration);
if (relinkCandidate) {
await upsertBinding(companyId, declaration, relinkCandidate.id);
const relinkedAgent = await agentSvc.getById(relinkCandidate.id) as Agent | null;
if (!relinkedAgent) throw notFound("Managed agent not found");
const relinkedAgent = await relinkManagedAgent(companyId, declaration, relinkCandidate.id);
const agent = await backfillManagedPauseReason(
companyId,
declaration,
@@ -596,8 +654,19 @@ export function pluginManagedAgentService(
? reconciled.agent.metadata
: {};
const adapterType = await resolveManagedAdapterType(companyId, declaration);
// Reset content through the canonical CAS path before changing defaults.
// A conflict must leave the existing adapter configuration untouched.
const withInstructions = await materializeDeclaredInstructions(companyId, reconciled.agent, declaration, { replaceExisting: true });
const defaults = declarationPatch(declaration, { adapterType });
const adapterConfig = { ...defaults.adapterConfig } as Record<string, unknown>;
if (declaration.instructions) {
for (const key of ["instructionsBundleMode", "instructionsRootPath", "instructionsEntryFile", "instructionsFilePath"]) {
if (withInstructions.adapterConfig[key] !== undefined) adapterConfig[key] = withInstructions.adapterConfig[key];
}
}
const updated = await agentSvc.update(reconciled.agent.id, {
...declarationPatch(declaration, { adapterType }),
...defaults,
adapterConfig,
metadata: managedMetadata(options.pluginId, options.pluginKey, declaration, currentMetadata),
}, {
recordRevision: {
@@ -605,7 +674,7 @@ export function pluginManagedAgentService(
},
});
if (!updated) throw notFound("Managed agent not found");
const updatedAgent = await materializeDeclaredInstructions(companyId, updated as Agent, declaration, { replaceExisting: true });
const updatedAgent = updated as Agent;
await upsertBinding(companyId, declaration, updatedAgent.id, {}, adapterType);
await logActivity(db, {
companyId,
+16
View File
@@ -241,3 +241,19 @@ default. Its file case anchors the task to a public project workspace, validates
the model's claimed result by reading the actual final bytes, and also exercises
remote copy-back. Keep candidate admission scoped to the selected model and the
isolated operator environment; ordinary agent configuration must not enable it.
## Persistent agent files
The `instruction_persistence` flow uses production managed storage and public file
APIs. The browser creates a supporting file, then a real agent edits its registered
AGENT_HOME with ordinary filesystem tools. Independent oracles verify instructions,
nested text, binary download bytes, and a stopped-run save receipt without new
revision history. The harness restarts the server and creates a fresh browser task
without disclosing the saved nonces. Its readback oracle downloads and verifies an
attachment's bytes and SHA-256, rather than accepting a filename or model claim.
A third task uploads a ready attachment and waits in an ordinary bounded shell
command while the board changes the current file through the public API. Stopped
cleanup must preserve the original candidate as a conflict. The browser reviews
current and incoming files and applies the run edits against the reviewed current
directory hash. All three tasks' runs count toward billing and teardown. The suite
is explicit-only. No private control-plane hooks or direct database writes are used.
+32
View File
@@ -193,6 +193,30 @@ duplicating the final response. The second workflow restarts the isolated
Paperclip server while the interaction is waiting, reloads that state, and
then resumes it. The suite has no Daytona cells.
`instruction-persistence` is an explicit-only three-cell workflow: legacy and
native Codex locally, plus native Codex on Daytona. Each creates six browser tasks
for the same agent. The editor first creates a nested supporting file. The first
run edits its registered AGENT_HOME using ordinary filesystem tools: instructions,
nested text, editor-created content, and exact binary bytes. The oracle checks the
current files, a stopped-run save receipt, and absence of newly appended history.
The first task also publishes a small verification receipt for the normal
completion contract; the personal files stay in the agent directory.
After a Paperclip restart, a fresh task must upload a downloaded proof attachment
containing independent saved nonces absent from its prompt. A third task edits its
private copy while the browser edits the same current file. The later run sync
must win for that changed file, preserve an unrelated board-created file, and
produce no conflict candidate or manual review step. Exact bytes, downloads,
and receipts are independently checked; model claims alone cannot pass.
Three further tasks fill a sparse personal file to its 256 MiB limit, exceed
that limit, and clean it up. Every run must still succeed; the run UI must show
a warning while full and clear it after cleanup. Rejected bytes must not replace
the saved file. This adds at most one 256 MiB saved fixture per isolated agent.
The deadline is twenty minutes per cell, with six expected provider runs;
normal instance/Daytona cleanup, screenshots, evidence, and billing apply. Run with
`pnpm test:e2e:runner -- --suite instruction-persistence`. Managed agent directories
checkpoint and close the provider before collection while retaining conversation
state. The separate `daytona-warm-continuity` suite covers warm runtime behavior.
`daytona-warm-continuity` (**Daytona Warm Continuity**) is exactly two paid
cells: legacy Codex and Runner Codex against one reusable warm Daytona
configuration. Each cell creates a real project with a primary local-path
@@ -426,6 +450,14 @@ content tags are never rebuilt or overwritten by the workflow.
### Match the local controller package to the Daytona image
When the controller runs on macOS or another platform different from the sandbox,
set `PAPERCLIP_RUNNER_REMOTE_BINARY_PATH` to a verified Linux amd64
`paperclip-runnerd`, such as the binary copied from `/usr/local/bin/paperclip-runnerd`
in the pinned image. The controller must have these exact bytes for its artifact
identity check. A local macOS runner cannot substitute for the Linux binary,
even when the sandbox image contains a compatible runner. This also applies to
native Codex cells, which do not otherwise need the remote provider pack below.
Native ACPX (including Claude) and OpenCode Daytona cells also require
`PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH` on the controller. The package and
the image must come from the same verified build. Equal provider version numbers
+3 -3
View File
@@ -91,10 +91,10 @@ describe("runner E2E catalog", () => {
expect(localIntegrityTasks).toHaveLength(2);
expect(openRouterBreadthTasks).toHaveLength(3);
expect(runnerSuites.map((suite) => suite.expectedMatrixSize)).toEqual([
30, 16, 16, 2, 8, 46, 23, 47, 20, 52, 28, 18, 6, 6, 4, 48, 16, 10, 2,
30, 3, 16, 16, 2, 8, 46, 23, 47, 20, 52, 28, 18, 6, 6, 4, 48, 16, 10, 2,
]);
expect(validateRunnerCatalog()).toHaveLength(398);
expect(new Set(runnerMatrix.map((entry) => entry.id)).size).toBe(398);
expect(validateRunnerCatalog()).toHaveLength(401);
expect(new Set(runnerMatrix.map((entry) => entry.id)).size).toBe(401);
expect(
runnerMatrix.filter((entry) => entry.suite.id === "core-compatibility"),
).toHaveLength(48);
+11
View File
@@ -1,3 +1,4 @@
import { instructionPersistenceTask } from "./instruction-persistence.js";
import { apiResponseReadingTask } from "./api-response-reading.js";
import { accountingTasks } from "./accounting-cases.js";
import { continuationTasks } from "./continuation-cases.js";
@@ -1035,6 +1036,15 @@ export const runnerSuites: readonly RunnerSuiteFixture[] = [
expectedMatrixSize: 30,
definitionMetadata: { version: 1, qualification: "pending", scheduling: "explicit-only", admission: "host-exact-candidate-and-model", authenticatedDiscoveryDate: "2026-09-28" },
},
{
id: "instruction-persistence", label: "Instruction Persistence",
description: "Agent-owned text and binary files round trip through the editor, survive a server restart and fresh task, and synchronize concurrent edits per file with last-sync-wins.",
groups: [], profiles: codexContinuityProfiles,
environments: [localEnvironment, runnerEnvironments.find(environment => environment.id === "daytona")!], tasks: [instructionPersistenceTask],
excludedExecutionIds: ["instruction-persistence.legacy-codex.daytona.private-copy-persists"],
expectedMatrixSize: 3, manualOnly: true,
definitionMetadata: { version: 6, oracle: "current-directory-independent-nonce-binary-last-sync-wins-and-nonblocking-storage-quota", providerTurns: 6, restart: "between-tasks", instructions: "production" },
},
{
id: "grok-subscription-qualification", label: "Grok Build Subscription Qualification", manualOnly: true,
description: "Explicit company subscription login across Grok browser workflows in local and Daytona environments.",
@@ -1364,6 +1374,7 @@ export function validateRunnerCatalog(): MatrixExecution[] {
...localIntegrityTasks,
...openRouterBreadthTasks,
daytonaWarmContinuityTask,
instructionPersistenceTask,
];
for (const [label, values] of [
["suite", runnerSuites],
@@ -0,0 +1,36 @@
import { createHash } from "node:crypto";
import { describe, expect, it } from "vitest";
import { gradeInstructionPersistence, instructionPersistenceTask } from "./instruction-persistence.js";
import { runnerMatrix } from "./catalog.js";
const expectedContent = "original\nInstruction persistence nonce: fixture\n";
const valid = () => ({ before: { revision: { id: "before" } },
saveEvent: { runId: "first-run", state: "saved" }, fileProof: true,
after: { content: expectedContent, contentHash: createHash("sha256").update(expectedContent).digest("hex"), revision: { id: "after", source: "cleanup", sourceRunId: "first-run", contentHash: createHash("sha256").update(expectedContent).digest("hex") } },
firstRunId: "first-run", expectedContent,
proof: { body: "Instruction persistence nonce: fixture\n", contentVerified: true },
expectedProof: "Instruction persistence nonce: fixture\n" });
describe("instruction persistence independent oracle", () => {
it("does not disclose the persisted nonce in the fresh-task marker", () => {
const nonce = "only-the-instruction-entry-reveals-this";
expect(instructionPersistenceTask.buildVisibleMarker(nonce)).not.toContain(nonce);
});
it("accepts exact cleanup content and a verified downloaded fresh-task proof", () => {
expect(gradeInstructionPersistence(valid()).every(row => row.passed)).toBe(true);
});
it.each(["missing-save", "wrong-run", "missing-files", "wrong-bytes", "missing-proof", "unverified-proof", "wrong-proof"])("rejects %s evidence", kind => {
const value = valid();
if (kind === "missing-save") value.saveEvent.state = "unavailable";
if (kind === "wrong-run") value.saveEvent.runId = "another-run";
if (kind === "missing-files") value.fileProof = false;
if (kind === "wrong-bytes") value.after.content = "claimed success";
if (kind === "missing-proof") return expect(gradeInstructionPersistence({ ...value, proof: undefined }).some(row => !row.passed)).toBe(true);
if (kind === "unverified-proof") value.proof.contentVerified = false;
if (kind === "wrong-proof") value.proof.body = "claimed success";
expect(gradeInstructionPersistence(value).some(row => !row.passed)).toBe(true);
});
it("registers exactly three explicit cells without changing scheduled campaigns", () => {
const cells = runnerMatrix.filter(row => row.suite.id === "instruction-persistence");
expect(cells.map(row => `${row.profile.id}.${row.environment.id}`)).toEqual(["legacy-codex.local", "runner-codex.local", "runner-codex.daytona"]);
expect(cells.every(row => row.suite.manualOnly && row.task.expectedRunCount === 6)).toBe(true);
});
});
+232
View File
@@ -0,0 +1,232 @@
import { createHash, randomBytes } from "node:crypto";
import { expect, type Page } from "@playwright/test";
import { pollUntil, type RunnerApi } from "./api.js";
import { captureFirstTaskAttachments } from "./first-task-attachments.js";
import { collectRunEvents } from "./run-observations.js";
import { createTaskThroughUi } from "./user-actions.js";
import type { LiveFixtureValues } from "./live-fixtures.js";
import type { MatrixExecution, RunnerTaskFixture } from "./types.js";
type Row = Record<string, any>;
export const instructionNonceLine = (nonce: string) => `Instruction persistence nonce: ${nonce}\n`;
export const instructionPersistenceTask: RunnerTaskFixture = {
id: "private-copy-persists", label: "Agent directory survives a fresh task",
groups: [], workMode: "standard", flow: "instruction_persistence",
expectedRunCount: 6, attemptTimeoutMs: { local: 20 * 60_000, daytona: 20 * 60_000 },
expectedTerminalState: { issue: "done", run: "succeeded" },
buildTitle: nonce => `Persist private instructions ${nonce}`,
buildVisibleMarker: () => "INSTRUCTIONS-VERIFIED",
buildPrompt: nonce => [
"Edit your own registered writable agent instruction entry with ordinary filesystem tools. The runtime guidance gives its exact private path.",
"Use Node.js built-in fs for these byte-preserving edits. Apply each append exactly once: inspect the existing suffix before retrying any command, because a warning does not imply that its writes failed.",
`Preserve its existing bytes and append exactly this UTF-8 suffix, represented as a JSON string: ${JSON.stringify(`\n${instructionNonceLine(nonce)}`)}`,
"Decode the JSON string once and append those bytes. Do not trim or normalize the existing file and do not add another blank line or separator.",
`In AGENT_HOME, create notes/retained.txt containing exactly ${JSON.stringify(`Personal file nonce: ${nonce}\n`)}. Create notes/bytes.bin with exactly the bytes [0,255,17,128,9]. Read notes/from-editor.txt and append exactly a newline followed by Edited by agent. and a final newline.`,
"Do not use update_agent_instructions, restore_agent_instructions, or an instructions API to save it. Do not edit repository AGENTS.md or the read-only loaded bundle.",
"After verifying the edits, upload a small text/plain attachment named agent-file-check.txt containing only 'Private file edits verified'. Use this attachment as your task completion evidence; the personal files themselves stay in AGENT_HOME.",
"Reply only Instruction copy edited without printing filesystem paths, then complete this task after the file edit. Paperclip will collect it after the provider stops; do not claim it has already persisted. Do not create further tasks.",
].join("\n"),
buildMatchers: () => [], // Independent current file and attachment oracle below.
};
export function gradeInstructionPersistence(input: { before: Row; after: Row; firstRunId: string; expectedContent: string; proof: Row | undefined; expectedProof: string; saveEvent?: Row; fileProof?: boolean }) {
return [
{ id: "directory-save-receipt", passed: input.saveEvent?.runId === input.firstRunId && input.saveEvent?.state === "saved" },
{ id: "nested-and-binary-files", passed: input.fileProof === true },
{ id: "exact-canonical-bytes", passed: input.after.content === input.expectedContent && input.after.contentHash === createHash("sha256").update(input.expectedContent).digest("hex") },
{ id: "fresh-task-downloaded-proof", passed: input.proof?.contentVerified === true && input.proof.body === input.expectedProof },
].map(check => ({ ...check, detail: check.passed ? `${check.id} verified independently` : `${check.id} missing or incorrect` }));
}
export async function runInstructionPersistenceFlow(input: {
page: Page; api: RunnerApi; fixtures: LiveFixtureValues; execution: MatrixExecution; nonce: string;
secrets: readonly string[]; deadlineAt: number;
restart(): Promise<void>;
observe(issue: Row, runs: Row[]): void;
capture(id: string, label: string, file: string): Promise<void>;
evidence(name: string, data: unknown): Promise<void>;
}) {
const { page, api, fixtures, execution, nonce } = input;
// Fixture names contain the campaign nonce. Use an unrelated value that the
// fresh task can obtain only from the saved entry (or forbidden task history).
const persistedNonce = randomBytes(16).toString("hex");
const filePath = `/api/agents/${fixtures.agent.id}/instructions-bundle/file?path=AGENTS.md`;
const before = await api.get<Row>(filePath);
if (typeof before.content !== "string" || !before.contentHash) throw new Error("Managed instructions must expose a current file hash");
const expectedContent = `${before.content}\n${instructionNonceLine(persistedNonce)}`;
const instructionsUrl = `/${fixtures.company.issuePrefix}/agents/${fixtures.agent.id}/instructions`;
const editorText = `Editor nonce: ${randomBytes(16).toString("hex")}`;
await page.goto(instructionsUrl);
await page.getByRole("button", { name: "Add agent file", exact: true }).click();
await page.getByPlaceholder("TOOLS.md").fill("notes/from-editor.txt");
await page.getByRole("button", { name: "Create", exact: true }).click();
await page.getByRole("group", { name: "Instruction file view" }).getByRole("button", { name: "edit", exact: true }).click();
await page.getByRole("textbox", { name: "Instruction file editor" }).fill(editorText);
await page.getByRole("button", { name: "Save changes", exact: true }).click();
await expect(page.getByRole("button", { name: "Save changes", exact: true })).toBeDisabled();
await expect(page.getByRole("button", { name: "History", exact: true })).toHaveCount(0);
let issue: Row = {};
let runs: Row[] = [];
async function create(title: string, prompt: string) {
await createTaskThroughUi({ page, issuePrefix: fixtures.company.issuePrefix!, agentName: fixtures.agent.name, title, prompt, workMode: "standard", projectName: fixtures.project?.name });
const found = await pollUntil({ label: `instruction task ${title}`, deadlineAt: input.deadlineAt,
load: async () => (await api.get<Row[]>(`/api/companies/${fixtures.company.id}/issues?limit=100`)).find(row => row.title === title), accept: row => Boolean(row) });
if (!found) throw new Error("Browser-created instruction task missing");
issue = found;
input.observe(issue, runs);
await page.goto(`/${fixtures.company.issuePrefix}/issues/${issue.identifier ?? issue.id}`);
}
async function settle(count: number) {
await pollUntil({ label: `instruction run ${count} completed`, deadlineAt: input.deadlineAt,
load: async () => {
issue = await api.get<Row>(`/api/issues/${issue.id}`);
const listed = await api.get<Row[]>(`/api/companies/${fixtures.company.id}/heartbeat-runs?limit=100`);
runs = await Promise.all(listed.map(row => api.get<Row>(`/api/heartbeat-runs/${row.id}`)));
runs.sort((a, b) => String(a.createdAt).localeCompare(String(b.createdAt)));
input.observe(issue, runs);
return { issue, runs };
},
accept: state => state.issue.status === "done" && state.runs.length === count && state.runs.every(row => row.status === "succeeded"),
reject: state => state.runs.some(row => ["failed", "cancelled", "timed_out"].includes(row.status)) ? "Instruction task provider run failed" : state.runs.length > count ? "Instruction task dispatched an extra run" : state.issue.status === "blocked" && state.runs.length === count && state.runs.every(row => row.status === "succeeded") ? `Instruction task reported a terminal blocker: ${JSON.stringify(state.issue.unblockDescriptor ?? {})}` : undefined,
});
expect(runs.every(row => row.runtimeMode === execution.profile.expectedRuntimeMode)).toBe(true);
await page.reload();
await expect(page.getByTestId("issue-detail-header").getByRole("button", { name: "Change status (current: Done)", exact: true })).toBeVisible();
}
await create(execution.task.buildTitle(nonce), execution.task.buildPrompt(persistedNonce));
await settle(1);
const firstRunId = runs[0]!.id;
const after = await pollUntil({ label: "stopped agent directory save", deadlineAt: Math.min(input.deadlineAt, Date.now() + 30_000),
load: () => api.get<Row>(filePath), accept: row => row.content === expectedContent });
const events = await collectRunEvents<Row>((afterSeq, limit) => api.get(`/api/heartbeat-runs/${firstRunId}/events?afterSeq=${afterSeq}&limit=${limit}`));
const saveEvent = events.find(row => row.eventType === "instruction_save" && row.payload?.state === "saved");
expect(saveEvent).toBeTruthy();
const readPersonal = (name: string) => api.get<Row>(`/api/agents/${fixtures.agent.id}/instructions-bundle/file?path=${encodeURIComponent(name)}`);
const note = await readPersonal("notes/retained.txt");
const fromEditor = await readPersonal("notes/from-editor.txt");
const binaryResponse = await api.request.get(`/api/agents/${fixtures.agent.id}/instructions-bundle/file?path=notes%2Fbytes.bin&download=true`);
expect(binaryResponse.ok()).toBe(true);
const binary = await binaryResponse.body();
const fileProof = note.content === `Personal file nonce: ${persistedNonce}\n` && fromEditor.content === `${editorText}\nEdited by agent.\n` && binary.equals(Buffer.from([0,255,17,128,9]));
expect(fileProof).toBe(true);
const history = await api.get<Row>(`/api/agents/${fixtures.agent.id}/instructions-bundle/history?path=AGENTS.md`);
expect(history.revisions).toHaveLength(0);
await input.evidence("instruction-first-save.json", { before, after, run: runs[0], events });
await input.capture("instruction-edited", "Private instructions saved after provider stop", "instruction-edited.png");
// A new server and a new issue cannot pass by retaining model conversation.
await input.restart();
expect((await api.get<Row>(filePath)).content).toBe(expectedContent);
await create("Read persisted instructions", [
"Read your own loaded agent instruction entry (or its current registered private copy) using ordinary filesystem tools.",
"Find the line beginning 'Instruction persistence nonce: '. Copy that entire line plus one final newline into instruction-proof.txt. Then append the exact bytes of notes/retained.txt from AGENT_HOME. Verify notes/bytes.bin contains the bytes [0,255,17,128,9]. Do not infer the value from this task title or other task history. Do not change your instructions.",
"Upload instruction-proof.txt as a text/plain task attachment named instruction-proof.txt using the normal artifact workflow. A local file alone is insufficient.",
`Reply with exactly ${execution.task.buildVisibleMarker(nonce)} and complete the task.`,
].join("\n"));
await settle(2);
const attachments = await captureFirstTaskAttachments(api, [{ ...issue, id: String(issue.id) }], input.secrets);
const proof = attachments.find(row => row.originalFilename === "instruction-proof.txt" || row.name === "instruction-proof.txt");
const final = await api.get<Row>(filePath);
expect(final.revision.id).toBe(after.revision.id);
const checks = gradeInstructionPersistence({ before, after, firstRunId, expectedContent, proof, expectedProof: `${instructionNonceLine(persistedNonce)}Personal file nonce: ${persistedNonce}\n`, saveEvent: { runId: firstRunId, state: saveEvent?.payload?.state }, fileProof });
await expect(page.getByTestId("task-chat-agent-bubble").filter({ hasText: execution.task.buildVisibleMarker(nonce) }).last()).toBeVisible();
await input.capture("final-state", "Fresh task downloaded the persisted instruction nonce", "final-state.png");
expect(checks.filter(check => !check.passed), "Independent instruction persistence checks").toEqual([]);
checks.push({ id: "editor-round-trip-no-history", passed: true, detail: "A browser-created supporting file was edited by the agent; all current bytes persisted without revision rows" });
const restored = final;
// The provider publishes an ordinary attachment before a bounded command wait.
// A board edit during that run is superseded only for the same changed file.
const conflictSuffix = `\nLast completed synchronization: ${nonce}\n`;
const expectedCandidate = `${restored.content}${conflictSuffix}`;
await create("Sync a concurrent instruction edit", [
"Use Node.js built-in fs. Apply the append exactly once, checking existing bytes before any retry.",
`Append exactly this UTF-8 suffix to your current registered writable instruction entry, represented as a JSON string: ${JSON.stringify(conflictSuffix)}`,
"Decode the JSON string once. Preserve all existing bytes. Do not use an instruction revision tool or instructions API.",
"After the file edit, upload a text/plain task attachment named instruction-candidate-ready.txt with the text ready. Use the ordinary artifact workflow.",
"Then execute the ordinary shell command sleep 45 and wait for it to finish. This gives the board time to edit the canonical instructions concurrently. Do not complete the task before that command finishes.",
"After the wait completes, reply Candidate edit ready and complete the task. Do not change the instructions again or claim that they saved.",
].join("\n"));
await pollUntil({ label: "provider staged concurrent instruction edit", deadlineAt: input.deadlineAt,
load: () => api.get<Row[]>(`/api/issues/${issue.id}/attachments`),
accept: rows => rows.some(row => row.originalFilename === "instruction-candidate-ready.txt" || row.name === "instruction-candidate-ready.txt") });
const active = await api.get<Row[]>(`/api/issues/${issue.id}/runs`);
expect(active.some(row => row.status === "running")).toBe(true);
const boardMarker = "Concurrent board instruction edit.";
await page.goto(instructionsUrl);
await page.getByText("AGENTS.md", { exact: true }).first().click();
await page.getByRole("group", { name: "Instruction file view" }).getByRole("button", { name: "edit", exact: true }).click();
const entryEditor = page.getByRole("textbox", { name: "editable markdown" });
await entryEditor.click();
await entryEditor.press("ControlOrMeta+End");
await entryEditor.press("Enter");
await entryEditor.pressSequentially(boardMarker);
await page.getByRole("button", { name: "Save changes", exact: true }).click();
await expect(page.getByRole("button", { name: "Save changes", exact: true })).toBeDisabled();
const board = await api.get<Row>(filePath);
expect(board.content).toContain(boardMarker);
expect(board.contentHash).not.toBe(restored.contentHash);
const unrelatedContent = `Concurrent independent file: ${nonce}`;
const unrelated = await api.request.put(`/api/agents/${fixtures.agent.id}/instructions-bundle/file`, {
data: { path: "notes/concurrent-editor.txt", content: unrelatedContent, baseHash: null },
});
expect(unrelated.ok()).toBe(true);
await page.goto(`/${fixtures.company.issuePrefix}/issues/${issue.identifier ?? issue.id}`);
await settle(3);
const syncRunId = runs[2]!.id;
const resolved = await pollUntil({ label: "last completed sync wins", deadlineAt: Math.min(input.deadlineAt, Date.now() + 30_000),
load: () => api.get<Row>(filePath), accept: row => row.content === expectedCandidate });
const candidates = await api.get<Row[]>(`/api/agents/${fixtures.agent.id}/instructions-bundle/candidates`);
expect(candidates.some(row => row.runId === syncRunId)).toBe(false);
expect((await readPersonal("notes/concurrent-editor.txt")).content).toBe(unrelatedContent);
const syncEvents = await collectRunEvents<Row>((afterSeq, limit) => api.get(`/api/heartbeat-runs/${syncRunId}/events?afterSeq=${afterSeq}&limit=${limit}`));
expect(syncEvents.some(row => row.eventType === "instruction_save" && row.payload?.state === "saved")).toBe(true);
await page.goto(instructionsUrl);
await expect(page.getByRole("button", { name: "Review preserved files", exact: true })).toHaveCount(0);
checks.push({ id: "per-file-last-sync-wins", passed: true, detail: "The later agent sync replaced the concurrent browser edit to its changed entry, preserved an unrelated new file, and created no conflict candidate" });
await page.goto(`/${fixtures.company.issuePrefix}/issues/${issue.identifier ?? issue.id}`);
await input.capture("last-sync-wins", "Concurrent changes synchronized per file without a conflict-review step", "last-sync-wins.png");
const quotaTask = (action: string, receipt: string) => [
"This is a controlled persistent-storage quota check. Use ordinary Node.js filesystem tools in your registered AGENT_HOME. Do not edit AGENTS.md.",
action,
`Upload a small text/plain task attachment named ${receipt}.txt containing the observed file size or cleanup result. This attachment is the primary task deliverable.`,
"Complete this task normally after uploading the receipt. A persistent-file storage warning is expected and must not prevent completion. Do not perform additional cleanup or change other personal files.",
].join("\n");
await create("Reach the agent file storage limit", quotaTask(
"Create quota-cache.bin using fs.openSync with flag w, fs.ftruncateSync(fd, 268435456), and fs.closeSync. This is a sparse fixture file, not a download. Verify its size using fs.statSync without reading the large contents.", "quota-full"));
await settle(4);
const fullRun = runs[3]!;
expect(fullRun.resultJson?.instructionSave).toMatchObject({ state: "saved", storageWarning: expect.stringContaining("Agent storage is full") });
await page.goto(`/${fixtures.company.issuePrefix}/agents/${fixtures.agent.id}/runs/${fullRun.id}`);
await expect(page.getByRole("note").filter({ hasText: "Agent storage warning" })).toContainText("Runs can continue");
// Public campaign screenshots are limited to sanitized task routes. Verify
// the warning in the real run UI, then capture its completed task outcome.
await page.goto(`/${fixtures.company.issuePrefix}/issues/${issue.identifier ?? issue.id}`);
await input.capture("storage-warning", "Task succeeded while agent storage reached its limit", "storage-warning.png");
await create("Keep running while agent storage is full", quotaTask(
"Verify quota-cache.bin already exists and its size is exactly 268435456. Grow only this file to 268435457 bytes with fs.truncateSync, then verify the new size. Leave it above the limit for this run's sync check.", "quota-exceeded"));
await settle(5);
const exceededRun = runs[4]!;
expect(exceededRun.resultJson?.instructionSave).toMatchObject({ state: "unavailable", errorCode: "AGENT_FILES_LIMIT_EXCEEDED", storageWarning: expect.stringContaining("Runs can continue") });
const fullEvents = await collectRunEvents<Row>((afterSeq, limit) => api.get(`/api/heartbeat-runs/${exceededRun.id}/events?afterSeq=${afterSeq}&limit=${limit}`));
expect(fullEvents.some(row => row.eventType === "instruction_save" && row.level === "warn" && row.payload?.state === "prepared" && row.payload?.storageWarning)).toBe(true);
await create("Clean up agent storage during a normal task", quotaTask(
"Verify restored quota-cache.bin has size 268435456: the rejected oversized edit must not have replaced its saved bytes. Delete quota-cache.bin with fs.unlinkSync, then write notes/after-quota.txt containing exactly 'Runs still work after quota cleanup'.", "quota-cleaned"));
await settle(6);
const cleanedRun = runs[5]!;
expect(cleanedRun.resultJson?.instructionSave).toMatchObject({ state: "saved", storageWarning: null });
expect((await readPersonal("notes/after-quota.txt")).content).toBe("Runs still work after quota cleanup");
const bundle = await api.get<Row>(`/api/agents/${fixtures.agent.id}/instructions-bundle`);
expect(bundle.files.some((file: Row) => file.path === "quota-cache.bin")).toBe(false);
await page.goto(`/${fixtures.company.issuePrefix}/agents/${fixtures.agent.id}/runs/${cleanedRun.id}`);
await expect(page.getByRole("note").filter({ hasText: "Agent storage warning" })).toHaveCount(0);
checks.push({ id: "storage-full-does-not-block-runs", passed: true, detail: "A run saved a file at quota, a subsequent run succeeded despite an oversized save rejection, and the next run removed the full file and cleared its warning; all three tasks completed" });
await input.evidence("api-state.json", { issue, runs, checks, canonicalInstructions: resolved, attachments });
await input.evidence("instruction-persistence.json", { checks, before, after, final, restored, board, candidates, resolved, syncEvents, runs, attachments, fullEvents });
await page.goto(`/${fixtures.company.issuePrefix}/issues/${issue.identifier ?? issue.id}`);
await input.capture("storage-recovered", "Agent completed a normal task and cleared storage warning after cleanup", "storage-recovered.png");
return { issue, runs, checks };
}
+12 -1
View File
@@ -1,3 +1,4 @@
import { runInstructionPersistenceFlow } from "./instruction-persistence.js";
import { gradeApiResponsePaging, readResponseProof, responseEvidenceDescription } from "./api-response-reading.js";
import { observeBrowserBootstrap } from "./browser-bootstrap-diagnostics.js";
import { runAccountingFlow } from "./accounting-flow.js";
@@ -546,7 +547,7 @@ for (const execution of executions) {
const credentials = credentialValues();
const secrets = normalizedSecrets(Object.values(credentials));
const api = new RunnerApi(request);
const companyRunFlow = ["continuation_accounting", "continuation", "context_integrity", "agent_chat", "everyday_workflow", "first_task"].includes(execution.task.flow);
const companyRunFlow = ["continuation_accounting", "continuation", "context_integrity", "agent_chat", "everyday_workflow", "first_task", "instruction_persistence"].includes(execution.task.flow);
const consoleDiagnostics: Array<Record<string, unknown>> = [];
const networkDiagnostics: Array<Record<string, unknown>> = [];
const pageLifecycleDiagnostics: Array<Record<string, unknown>> = [];
@@ -859,6 +860,16 @@ for (const execution of executions) {
evidence: (name, data) => writeSanitizedJson(snapshotsDir, name, data, secrets),
});
issue = accounting.issue as IssueRecord; selectedRuns = accounting.runs as RunRecord[];
} else if (execution.task.flow === "instruction_persistence") {
const story = await runInstructionPersistenceFlow({
page, api, fixtures, execution, nonce, secrets, deadlineAt: startedAtMs + deadlineMs,
restart: () => restartIsolatedPaperclipServer({ api, requestId: `instructions-${nonce}`, deadlineAt: startedAtMs + deadlineMs }),
observe: (currentIssue, currentRuns) => { issue = currentIssue as IssueRecord; selectedRuns = currentRuns as RunRecord[]; },
capture: captureScreenshot,
evidence: (name, data) => writeSanitizedJson(snapshotsDir, name, data, secrets),
});
issue = story.issue as IssueRecord; selectedRuns = story.runs as RunRecord[];
matcherResults = story.checks.map(check => ({ matcher: { kind: "json_path" as const, path: `instructions.${check.id}`, expected: true }, passed: check.passed, detail: check.detail }));
} else if (execution.task.flow === "continuation") {
const continuation = await runContinuationFlow({
page, api, fixtures, execution, nonce, secrets, workspacePath, deadlineAt: startedAtMs + deadlineMs - 60_000,
+2 -1
View File
@@ -27,7 +27,8 @@ export type RunnerTaskFlow =
| "plan_revision_acceptance"
| "question_resume_completion"
| "plan_approval_completion"
| "warm_three_turn";
| "warm_three_turn"
| "instruction_persistence";
export interface SecretReference {
type: "secret_ref";
+22 -3
View File
@@ -5,6 +5,11 @@ import type {
AgentPermissions,
AgentDetail,
AgentInstructionsBundle,
AgentInstructionHistory,
AgentInstructionCandidate,
ResolveAgentInstructionCandidate,
AgentInstructionDiff,
AgentInstructionSnapshot,
AgentInstructionsFileDetail,
AgentSkillSnapshot,
AdapterEnvironmentTestResult,
@@ -167,12 +172,26 @@ export const agentsApi = {
),
saveInstructionsFile: (
id: string,
data: { path: string; content: string; clearLegacyPromptTemplate?: boolean },
data: { path: string; content: string; baseRevisionId?: string | null; baseHash?: string | null; clearLegacyPromptTemplate?: boolean },
companyId?: string,
) => api.put<AgentInstructionsFileDetail>(agentPath(id, companyId, "/instructions-bundle/file"), data),
deleteInstructionsFile: (id: string, relativePath: string, companyId?: string) =>
instructionCandidates: (id: string, companyId?: string) =>
api.get<AgentInstructionCandidate[]>(agentPath(id, companyId, "/instructions-bundle/candidates")),
resolveInstructionCandidate: (id: string, runId: string, data: ResolveAgentInstructionCandidate, companyId?: string) =>
api.post<AgentInstructionsFileDetail>(agentPath(id, companyId, `/instructions-bundle/candidates/${runId}/resolve`), data),
instructionHistory: (id: string, path: string, companyId?: string, cursor?: string) =>
api.get<AgentInstructionHistory>(agentPath(id, companyId, `/instructions-bundle/history?path=${encodeURIComponent(path)}${cursor ? `&cursor=${encodeURIComponent(cursor)}` : ""}`)),
instructionRevision: (id: string, path: string, revisionId: string, companyId?: string) =>
api.get<AgentInstructionSnapshot>(agentPath(id, companyId, `/instructions-bundle/revision/${revisionId}?path=${encodeURIComponent(path)}`)),
instructionDiff: (id: string, path: string, from: string, to: string, companyId?: string) =>
api.get<AgentInstructionDiff>(agentPath(id, companyId, `/instructions-bundle/diff?path=${encodeURIComponent(path)}&from=${encodeURIComponent(from)}&to=${encodeURIComponent(to)}`)),
restoreInstructions: (id: string, data: { path: string; revisionId: string; baseRevisionId: string }, companyId?: string) =>
api.post<AgentInstructionsFileDetail>(agentPath(id, companyId, "/instructions-bundle/restore"), data),
downloadInstructionsFile: (id: string, relativePath: string, companyId?: string) =>
`/api${agentPath(id, companyId, `/instructions-bundle/file?path=${encodeURIComponent(relativePath)}&download=true`)}`,
deleteInstructionsFile: (id: string, relativePath: string, companyId?: string, baseHash?: string) =>
api.delete<AgentInstructionsBundle>(
agentPath(id, companyId, `/instructions-bundle/file?path=${encodeURIComponent(relativePath)}`),
agentPath(id, companyId, `/instructions-bundle/file?path=${encodeURIComponent(relativePath)}${baseHash ? `&baseHash=${baseHash}` : ""}`),
),
pause: (id: string, companyId?: string) => api.post<Agent>(agentPath(id, companyId, "/pause"), {}),
resume: (id: string, companyId?: string) => api.post<Agent>(agentPath(id, companyId, "/resume"), {}),
+145
View File
@@ -0,0 +1,145 @@
import { useState } from "react";
import { useInfiniteQuery, useMutation, useQuery } from "@tanstack/react-query";
import type { AgentInstructionsFileDetail } from "@paperclipai/shared";
import { agentsApi } from "../api/agents";
import { Button } from "./ui/button";
export function InstructionHistory({
agentId,
companyId,
path,
currentRevisionId,
disabled,
onRestored,
}: {
agentId: string;
companyId?: string;
path: string;
currentRevisionId: string;
disabled: boolean;
onRestored: (file: AgentInstructionsFileDetail) => void;
}) {
const [open, setOpen] = useState(false);
const [selected, setSelected] = useState<string | null>(null);
const history = useInfiniteQuery({
queryKey: ["instruction-history", agentId, path, currentRevisionId],
queryFn: ({ pageParam }) =>
agentsApi.instructionHistory(agentId, path, companyId, pageParam),
initialPageParam: undefined as string | undefined,
getNextPageParam: (page) => page.nextCursor ?? undefined,
enabled: open,
});
const diff = useQuery({
queryKey: ["instruction-diff", agentId, path, selected, currentRevisionId],
queryFn: () =>
agentsApi.instructionDiff(
agentId,
path,
selected!,
currentRevisionId,
companyId,
),
enabled: open && Boolean(selected),
});
const restore = useMutation({
mutationFn: () =>
agentsApi.restoreInstructions(
agentId,
{ path, revisionId: selected!, baseRevisionId: currentRevisionId },
companyId,
),
onSuccess: (file) => {
setSelected(null);
onRestored(file);
},
});
const error = history.error ?? diff.error ?? restore.error;
return (
<div className="space-y-3">
<Button
type="button"
size="sm"
variant="outline"
aria-expanded={open}
onClick={() => setOpen(!open)}
>
History
</Button>
{open && (
<div className="space-y-3">
{history.isLoading && (
<p className="text-sm text-muted-foreground">Loading revisions…</p>
)}
{error && (
<p role="alert" className="text-sm text-destructive">
{error.message}
</p>
)}
{history.data?.pages
.flatMap((page) => page.revisions)
.map((revision) => (
<div key={revision.id} className="flex items-center gap-3">
<Button
type="button"
size="sm"
variant={selected === revision.id ? "secondary" : "ghost"}
onClick={() => setSelected(revision.id)}
>
<span className="font-mono">{revision.id.slice(0, 8)}</span>
</Button>
<span className="text-sm text-muted-foreground">
{revision.source} ·{" "}
{new Date(revision.createdAt).toLocaleString()}
{revision.id === currentRevisionId ? " · Current" : ""}
</span>
</div>
))}
{history.hasNextPage && (
<Button
type="button"
variant="ghost"
disabled={history.isFetchingNextPage}
onClick={() => void history.fetchNextPage()}
>
Older revisions
</Button>
)}
{diff.data && (
<>
<p className="text-sm text-muted-foreground">Selected revision</p>
<pre className="whitespace-pre-wrap break-words rounded-md border border-border p-3 font-mono text-sm">
{diff.data.from.content}
</pre>
<p className="text-sm text-muted-foreground">
Changes from selected revision to current
</p>
<pre className="whitespace-pre-wrap break-words rounded-md border border-border p-3 font-mono text-sm">
{diff.data.removed && `Removed:\n${diff.data.removed}\n`}
{diff.data.added && `Added:\n${diff.data.added}`}
{!diff.data.removed &&
!diff.data.added &&
"No content changes."}
</pre>
<Button
type="button"
disabled={
disabled ||
restore.isPending ||
selected === currentRevisionId
}
onClick={() => restore.mutate()}
>
Restore as new revision
</Button>
{disabled && (
<p className="text-sm text-muted-foreground">
Save or cancel your edits before restoring.
</p>
)}
</>
)}
</div>
)}
</div>
);
}
+2
View File
@@ -215,6 +215,8 @@ export const queryKeys = {
skills: (id: string) => ["agents", "skills", id] as const,
instructionsBundle: (id: string) =>
["agents", "instructions-bundle", id] as const,
instructionCandidates: (id: string) =>
["agents", "instruction-candidates", id] as const,
instructionsFile: (id: string, relativePath: string) =>
["agents", "instructions-bundle", id, "file", relativePath] as const,
keys: (agentId: string) => ["agents", "keys", agentId] as const,
+157 -1
View File
@@ -6,7 +6,8 @@ import { createRoot, type Root } from "react-dom/client";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import type { Agent, AgentInstructionsBundle, AgentInstructionsFileDetail, AgentInstructionsFileSummary } from "@paperclipai/shared";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { PromptsTab } from "./AgentDetail";
import { AgentFileRunNotice, PromptsTab } from "./AgentDetail";
import { queryKeys } from "../lib/queryKeys";
const mockAgentsApi = vi.hoisted(() => ({
instructionsBundle: vi.fn(),
@@ -14,9 +15,17 @@ const mockAgentsApi = vi.hoisted(() => ({
updateInstructionsBundle: vi.fn(),
saveInstructionsFile: vi.fn(),
deleteInstructionsFile: vi.fn(),
instructionHistory: vi.fn(),
instructionCandidates: vi.fn(),
resolveInstructionCandidate: vi.fn(),
instructionDiff: vi.fn(),
restoreInstructions: vi.fn(),
}));
const markdownEditorRenderMock = vi.hoisted(() => vi.fn());
const copyTextToClipboardMock = vi.hoisted(() => vi.fn(async (_text: string) => {}));
vi.mock("../lib/clipboard", () => ({ copyTextToClipboard: copyTextToClipboardMock }));
vi.mock("../api/agents", () => ({
agentsApi: mockAgentsApi,
@@ -226,6 +235,7 @@ describe("PromptsTab instruction editor", () => {
saveAction = null;
markdownEditorRenderMock.mockClear();
Object.values(mockAgentsApi).forEach((mock) => mock.mockReset());
mockAgentsApi.instructionCandidates.mockResolvedValue([]);
mockAgentsApi.updateInstructionsBundle.mockResolvedValue({});
mockAgentsApi.saveInstructionsFile.mockImplementation(async (_agentId, data) => ({
path: data.path,
@@ -333,6 +343,7 @@ describe("PromptsTab instruction editor", () => {
{
path: "AGENTS",
content: "# Updated",
baseRevisionId: null,
clearLegacyPromptTemplate: false,
},
"company-1",
@@ -340,6 +351,151 @@ describe("PromptsTab instruction editor", () => {
});
});
it("loads preserved edits against the displayed head and retains their pinned base after a conflict", async () => {
const summary = makeSummary("AGENTS.md", "AGENTS.md");
const revision = { id: "head-1", entryFile: "AGENTS.md" } as NonNullable<AgentInstructionsFileDetail["revision"]>;
mockAgentsApi.instructionCandidates.mockResolvedValue([{ runId: "preserved-run", entryFile: "AGENTS.md", baseRevisionId: "older-base", baseHash: "base-hash", state: "conflict", candidateHash: "candidate-hash", content: "preserved edit", errorCode: "INSTRUCTION_REVISION_CONFLICT", errorMessage: "Instructions changed", createdAt: "2026-01-01T00:00:00Z", updatedAt: "2026-01-01T00:00:00Z" }]);
await renderPromptsTab(makeBundle("AGENTS.md", [summary]), { "AGENTS.md": makeDetail(summary, "current content", { revision }) });
await waitFor(() => expect(buttonByText(container, "Review preserved edits")).toBeDefined());
await act(async () => { buttonByText(container, "Review preserved edits").click(); });
const editor = await waitFor(() => {
const element = container.querySelector<HTMLTextAreaElement>('[data-testid="markdown-editor"]');
expect(element?.value).toBe("preserved edit"); return element!;
});
mockAgentsApi.resolveInstructionCandidate.mockRejectedValue(new Error("Instructions changed since the base revision"));
await waitFor(() => expect(saveAction).toEqual(expect.any(Function)));
await act(async () => { saveAction?.(); });
await waitFor(() => expect(mockAgentsApi.resolveInstructionCandidate).toHaveBeenCalledWith("agent-1", "preserved-run", { content: "preserved edit", baseRevisionId: "head-1" }, "company-1"));
expect(mockAgentsApi.saveInstructionsFile).not.toHaveBeenCalled();
expect(editor.value).toBe("preserved edit");
await act(async () => { queryClient.setQueryData(["agents", "instructions-bundle", "agent-1", "file", "AGENTS.md"], makeDetail(summary, "newer content", { revision: { ...revision, id: "head-2" } })); });
await act(async () => { saveAction?.(); });
await waitFor(() => expect(mockAgentsApi.resolveInstructionCandidate).toHaveBeenCalledTimes(2));
expect(mockAgentsApi.resolveInstructionCandidate.mock.lastCall?.[2].baseRevisionId).toBe("head-1");
expect(editor.value).toBe("preserved edit");
mockAgentsApi.instructionsFile.mockResolvedValue(makeDetail(summary, "latest content", { revision: { ...revision, id: "head-3" } }));
await waitFor(() => expect(buttonByText(container, "Refresh current revision").disabled).toBe(false));
await act(async () => { buttonByText(container, "Refresh current revision").click(); });
await waitFor(() => expect(container.textContent).toContain("latest content"));
expect(editor.value).toBe("preserved edit");
expect(mockAgentsApi.resolveInstructionCandidate).toHaveBeenCalledTimes(2);
await act(async () => { saveAction?.(); });
await waitFor(() => expect(mockAgentsApi.resolveInstructionCandidate).toHaveBeenCalledTimes(3));
expect(mockAgentsApi.resolveInstructionCandidate.mock.lastCall?.[2].baseRevisionId).toBe("head-3");
});
it("keeps historical whole-folder failures out of the current editor while preserving legacy review", async () => {
const summary = makeSummary("AGENTS.md", "AGENTS.md");
const failures = [1, 2, 3].map(attempt => ({ contract: "agent_files", runId: `failed-auth-${attempt}`,
entryFile: "AGENTS.md", state: "unavailable", content: null,
errorMessage: "The registered instruction copy could not be retrieved safely before environment release. No instruction save is claimed." }));
mockAgentsApi.instructionCandidates.mockResolvedValue(failures);
await renderPromptsTab(makeBundle("AGENTS.md", [summary], { persistence: "agent_files" }), {
"AGENTS.md": makeDetail(summary, "Successfully saved agent instructions"),
});
await waitFor(() => expect(container.textContent).toContain("Successfully saved agent instructions"));
expect(container.textContent).not.toContain("could not be retrieved");
expect(container.textContent).not.toContain("Preserved instruction edits");
expect(container.querySelector('[role="alert"]')).toBeNull();
await act(async () => {
queryClient.setQueryData(queryKeys.agents.instructionCandidates("agent-1"), [...failures, {
contract: "legacy", runId: "preserved-run", entryFile: "AGENTS.md", state: "conflict",
content: "Legacy edits to review", createdAt: "2026-01-01T00:00:00Z",
}]);
});
await waitFor(() => expect(buttonByText(container, "Review preserved edits").disabled).toBe(false));
expect(container.textContent).toContain("Preserved instruction edits");
expect(container.textContent).not.toContain("could not be retrieved");
});
it("scopes sync warnings to the affected run and displays a storage warning only once", async () => {
root = createRoot(container);
const render = async (instructionSave: Record<string, unknown>) => act(async () => {
root?.render(<AgentFileRunNotice resultJson={{ instructionSave }} />);
});
await render({ contract: "agent_files", state: "unavailable", errorMessage: "This run's files were not saved." });
expect(container.textContent).toContain("Agent file sync failed for this run");
expect(container.textContent).toContain("This run's files were not saved.");
await render({ contract: "agent_files", state: "unavailable", errorCode: "AGENT_FILES_LIMIT_EXCEEDED", errorMessage: "Save rejected", storageWarning: "Agent storage is full. Runs can continue." });
expect(container.querySelectorAll('[role="note"]')).toHaveLength(1);
expect(container.textContent).toContain("Runs can continue");
expect(container.textContent).not.toContain("Save rejected");
await render({ contract: "agent_files", state: "unavailable", errorCode: "AGENT_FILES_SAVE_FAILED", errorMessage: "An I/O failure prevented saving this run's files.", storageWarning: "Agent storage is full. Runs can continue." });
expect(container.querySelectorAll('[role="note"]')).toHaveLength(2);
expect(container.textContent).toContain("Runs can continue");
expect(container.textContent).toContain("An I/O failure prevented saving this run's files.");
await render({ contract: "agent_files", state: "saved" });
expect(container.textContent).toBe("");
await render({ state: "conflict", errorMessage: "Legacy candidate needs review" });
expect(container.textContent).toBe("");
});
it("keeps changed-entry preserved edits readable and copyable without enabling a save", async () => {
const summary = makeSummary("CURRENT.md", "CURRENT.md");
const revision = { id: "current-head", entryFile: "CURRENT.md" } as NonNullable<AgentInstructionsFileDetail["revision"]>;
const preserved = "# Original entry\nPreserve these exact edits.\n";
mockAgentsApi.instructionCandidates.mockResolvedValue([{ runId: "old-entry-run", entryFile: "OLD.md", baseRevisionId: "old-head", baseHash: "base-hash", state: "conflict", candidateHash: "candidate-hash", content: preserved, errorCode: "INSTRUCTION_ENTRY_CHANGED", errorMessage: "The instruction entry changed", createdAt: "2026-01-01T00:00:00Z", updatedAt: "2026-01-01T00:00:00Z" }]);
await renderPromptsTab(makeBundle("CURRENT.md", [summary]), { "CURRENT.md": makeDetail(summary, "Current instructions", { revision }) });
await waitFor(() => expect(buttonByText(container, "Review preserved edits").disabled).toBe(false));
await act(async () => { buttonByText(container, "Review preserved edits").click(); });
const review = await waitFor(() => {
const region = container.querySelector<HTMLElement>('[aria-label="Preserved edits for OLD.md"]');
expect(region?.querySelector("pre")?.textContent).toBe(preserved);
return region!;
});
expect(review.textContent).toContain("Read only");
expect(review.textContent).toContain("CURRENT.md");
expect(review.querySelector("textarea, input, [contenteditable=true]")).toBeNull();
expect(saveAction).toBeNull();
expect(mockAgentsApi.instructionsFile.mock.calls.every((call) => call[1] === "CURRENT.md")).toBe(true);
await act(async () => { review.querySelector<HTMLButtonElement>('[aria-label="Copy preserved edits for OLD.md"]')!.click(); });
expect(copyTextToClipboardMock).toHaveBeenCalledWith(preserved);
expect(saveAction).toBeNull();
expect(mockAgentsApi.resolveInstructionCandidate).not.toHaveBeenCalled();
expect(mockAgentsApi.saveInstructionsFile).not.toHaveBeenCalled();
// Choosing to edit and paste into the current entry is a separate ordinary save.
await selectInstructionMode("Edit");
const editor = container.querySelector<HTMLTextAreaElement>('[data-testid="markdown-editor"]')!;
expect(editor.value).toBe("Current instructions");
await act(async () => { editor.dispatchEvent(new MouseEvent("pointerdown", { bubbles: true })); setNativeValue(editor, preserved); });
await waitFor(() => expect(saveAction).toEqual(expect.any(Function)));
await act(async () => { saveAction?.(); });
await waitFor(() => expect(mockAgentsApi.saveInstructionsFile).toHaveBeenCalledWith("agent-1", { path: "CURRENT.md", content: preserved, baseRevisionId: "current-head", clearLegacyPromptTemplate: false }, "company-1"));
expect(mockAgentsApi.resolveInstructionCandidate).not.toHaveBeenCalled();
});
it("retains the draft and its base when a concurrent save conflicts", async () => {
const summary = makeSummary("AGENTS.md", "AGENTS.md");
const revision = { id: "base-1", entryFile: "AGENTS.md" } as NonNullable<AgentInstructionsFileDetail["revision"]>;
await renderPromptsTab(makeBundle("AGENTS.md", [summary]), { "AGENTS.md": makeDetail(summary, "original", { revision }) });
mockAgentsApi.saveInstructionsFile.mockRejectedValue(new Error("Instructions changed since the base revision"));
await selectInstructionMode("Edit");
const editor = container.querySelector<HTMLTextAreaElement>('[data-testid="markdown-editor"]')!;
await act(async () => { editor.dispatchEvent(new MouseEvent("pointerdown", { bubbles: true })); setNativeValue(editor, "my unsaved edit"); });
await waitFor(() => expect(saveAction).toEqual(expect.any(Function)));
await act(async () => { saveAction?.(); });
await waitFor(() => expect(container.querySelector('[role="alert"]')?.textContent).toContain("base revision"));
expect(editor.value).toBe("my unsaved edit");
expect(mockAgentsApi.saveInstructionsFile).toHaveBeenCalledWith("agent-1", expect.objectContaining({ content: "my unsaved edit", baseRevisionId: "base-1" }), "company-1");
});
it("shows revision content and restores against the displayed current head", async () => {
const summary = makeSummary("AGENTS.md", "AGENTS.md");
const revision = { id: "current-1", entryFile: "AGENTS.md" } as NonNullable<AgentInstructionsFileDetail["revision"]>;
mockAgentsApi.instructionHistory.mockResolvedValue({ revisions: [{ id: "old-revision", source: "board", createdAt: "2026-01-01T00:00:00Z" }], nextCursor: null });
mockAgentsApi.instructionDiff.mockResolvedValue({ from: { content: "old text" }, removed: "old", added: "new" });
mockAgentsApi.restoreInstructions.mockResolvedValue(makeDetail(summary, "old text", { revision: { ...revision, id: "restored-1" } }));
await renderPromptsTab(makeBundle("AGENTS.md", [summary]), { "AGENTS.md": makeDetail(summary, "new text", { revision }) });
await waitFor(() => expect(buttonByText(container, "History")).toBeDefined());
await act(async () => { buttonByText(container, "History").click(); });
await waitFor(() => expect(buttonByText(container, "old-revi")).toBeDefined());
await act(async () => { buttonByText(container, "old-revi").click(); });
await waitFor(() => expect(container.textContent).toContain("old text"));
await act(async () => { buttonByText(container, "Restore as new revision").click(); });
await waitFor(() => expect(mockAgentsApi.restoreInstructions).toHaveBeenCalledWith("agent-1", { path: "AGENTS.md", revisionId: "old-revision", baseRevisionId: "current-1" }, "company-1"));
});
it("ignores rich-editor mount normalization until the user interacts", async () => {
const summary = makeSummary("AGENTS.md", "AGENTS.md");
const onDirtyChange = vi.fn();
+193 -13
View File
@@ -1,3 +1,5 @@
import type { AgentInstructionCandidate, AgentInstructionsBundle } from "@paperclipai/shared";
import { InstructionHistory } from "../components/InstructionHistory";
import { AgentCharacter } from "../components/AgentCharacter";
import { characterStateForAgent } from "@paperclipai/shared";
import { mergeRunLogChunks, readChunkSeq } from "../lib/run-log-chunks";
@@ -2197,6 +2199,12 @@ export function PromptsTab({
const [instructionMode, setInstructionMode] = useState<"read" | "edit" | "raw">("read");
const [showFilePanel, setShowFilePanel] = useState(false);
const [draft, setDraft] = useState<string | null>(null);
const draftBaseRevisionRef = useRef<string | null | undefined>(undefined);
const draftBaseHashRef = useRef<string | null | undefined>(undefined);
const [candidateRunId, setCandidateRunId] = useState<string | null>(null);
const [readOnlyCandidateRunId, setReadOnlyCandidateRunId] = useState<string | null>(null);
const candidateAgentRef = useRef(agent.id);
candidateAgentRef.current = agent.id;
const [bundleDraft, setBundleDraft] = useState<{
mode: "managed" | "external";
rootPath: string;
@@ -2225,6 +2233,9 @@ export function PromptsTab({
}, []);
const setSelectedFile = useCallback((filePath: string) => {
editorInteractedRef.current = false;
draftBaseRevisionRef.current = undefined;
draftBaseHashRef.current = undefined;
setCandidateRunId(null);
setSelectedFileState(filePath);
}, []);
@@ -2234,6 +2245,7 @@ export function PromptsTab({
setInstructionMode("read");
setShowFilePanel(false);
setDraft(null);
setReadOnlyCandidateRunId(null);
setBundleDraft(null);
setNewFilePath("");
setShowNewFileInput(false);
@@ -2251,6 +2263,7 @@ export function PromptsTab({
queryKey: queryKeys.agents.instructionsBundle(agent.id),
queryFn: () => agentsApi.instructionsBundle(agent.id, companyId),
enabled: Boolean(companyId && isLocal),
refetchInterval: draft === null ? 5000 : false,
});
const persistedMode = bundle?.mode ?? "managed";
@@ -2284,10 +2297,68 @@ export function PromptsTab({
const selectedFileExists = bundleMatchesDraft && fileOptions.includes(selectedOrEntryFile);
const selectedFileSummary = bundle?.files.find((file) => file.path === selectedOrEntryFile) ?? null;
const { data: selectedFileDetail, isLoading: fileLoading } = useQuery({
const { data: selectedFileDetail, isLoading: fileLoading, error: fileError } = useQuery({
queryKey: queryKeys.agents.instructionsFile(agent.id, selectedOrEntryFile),
queryFn: () => agentsApi.instructionsFile(agent.id, selectedOrEntryFile, companyId),
enabled: Boolean(companyId && isLocal && selectedFileExists),
refetchInterval: draft === null ? 5000 : false,
});
const candidates = useQuery({
queryKey: queryKeys.agents.instructionCandidates(agent.id),
queryFn: () => agentsApi.instructionCandidates(agent.id, companyId),
enabled: Boolean(companyId && isLocal && currentMode === "managed"),
});
// Whole-folder failures belong to their run's diagnostics. They have no
// preserved edits to resolve and do not describe the current saved files.
const preservedCandidates = candidates.data?.filter((candidate) => candidate.contract !== "agent_files") ?? [];
const loadCandidate = useMutation({
mutationFn: async (candidate: AgentInstructionCandidate) => {
if (candidate.content === null) throw new Error("These instruction edits have not been retrieved yet.");
const file = await agentsApi.instructionsFile(agent.id, candidate.entryFile, companyId).catch((error) => {
if (error instanceof ApiError && error.status === 404 && candidate.baseRevisionId === null) return null;
throw error;
});
return { candidate, file, agentId: agent.id };
},
onSuccess: ({ candidate, file, agentId: requestedAgentId }) => {
if (candidateAgentRef.current !== requestedAgentId) return;
setSelectedFile(candidate.entryFile);
if (file) queryClient.setQueryData(queryKeys.agents.instructionsFile(agent.id, candidate.entryFile), file);
draftBaseRevisionRef.current = file?.revision?.id ?? null;
setCandidateRunId(candidate.runId);
setDraft(candidate.content);
setInstructionMode("edit");
},
});
const resolveCandidate = useMutation({
mutationFn: async (data: { runId: string; content: string; baseRevisionId: string | null }) => ({
file: await agentsApi.resolveInstructionCandidate(agent.id, data.runId, { content: data.content, baseRevisionId: data.baseRevisionId }, companyId),
agentId: agent.id,
}),
onSuccess: ({ file, agentId: requestedAgentId }) => {
if (candidateAgentRef.current !== requestedAgentId) return;
setDraft(null);
setCandidateRunId(null);
draftBaseRevisionRef.current = undefined;
draftBaseHashRef.current = undefined;
queryClient.setQueryData(queryKeys.agents.instructionsFile(agent.id, file.path), file);
queryClient.invalidateQueries({ queryKey: queryKeys.agents.instructionsBundle(agent.id) });
queryClient.invalidateQueries({ queryKey: queryKeys.agents.instructionCandidates(agent.id) });
},
});
const refreshCandidateBase = useMutation({
mutationFn: async () => ({
file: await agentsApi.instructionsFile(agent.id, selectedOrEntryFile, companyId),
agentId: agent.id, runId: candidateRunId,
}),
onSuccess: ({ file, agentId: requestedAgentId, runId }) => {
if (candidateAgentRef.current !== requestedAgentId || candidateRunId !== runId) return;
draftBaseRevisionRef.current = file.revision?.id ?? null;
queryClient.setQueryData(queryKeys.agents.instructionsFile(agent.id, file.path), file);
resolveCandidate.reset();
},
});
const updateBundle = useMutation({
@@ -2299,9 +2370,9 @@ export function PromptsTab({
}) => agentsApi.updateInstructionsBundle(agent.id, data, companyId),
onMutate: () => {
editorInteractedRef.current = false;
setAwaitingRefresh(true);
},
onSuccess: () => {
setAwaitingRefresh(true);
queryClient.invalidateQueries({ queryKey: queryKeys.agents.instructionsBundle(agent.id) });
queryClient.invalidateQueries({ queryKey: queryKeys.agents.detail(agent.id) });
queryClient.invalidateQueries({ queryKey: queryKeys.agents.detail(agent.urlKey) });
@@ -2310,13 +2381,25 @@ export function PromptsTab({
});
const saveFile = useMutation({
mutationFn: (data: { path: string; content: string; clearLegacyPromptTemplate?: boolean }) =>
mutationFn: (data: { path: string; content: string; baseRevisionId?: string | null; baseHash?: string | null; clearLegacyPromptTemplate?: boolean }) =>
agentsApi.saveInstructionsFile(agent.id, data, companyId),
onMutate: () => {
editorInteractedRef.current = false;
setAwaitingRefresh(true);
},
onSuccess: (_, variables) => {
onSuccess: (file, variables) => {
setDraft(null);
draftBaseRevisionRef.current = undefined;
draftBaseHashRef.current = undefined;
queryClient.setQueryData(queryKeys.agents.instructionsFile(agent.id, variables.path), file);
// Keep the selected file present while the refreshed bundle is in flight.
// Otherwise removing its pending placeholder briefly selects AGENTS.md.
queryClient.setQueryData<AgentInstructionsBundle>(queryKeys.agents.instructionsBundle(agent.id), previous => previous ? {
...previous, files: [...previous.files.filter(item => item.path !== file.path), {
path: file.path, size: file.size, language: file.language, markdown: file.markdown,
isEntryFile: file.isEntryFile, editable: file.editable, deprecated: file.deprecated,
virtual: file.virtual, binary: file.binary, contentHash: file.contentHash,
}],
} : previous);
setPendingFiles((prev) => prev.filter((f) => f !== variables.path));
queryClient.invalidateQueries({ queryKey: queryKeys.agents.instructionsBundle(agent.id) });
queryClient.invalidateQueries({ queryKey: queryKeys.agents.instructionsFile(agent.id, variables.path) });
@@ -2327,7 +2410,7 @@ export function PromptsTab({
});
const deleteFile = useMutation({
mutationFn: (relativePath: string) => agentsApi.deleteInstructionsFile(agent.id, relativePath, companyId),
mutationFn: (relativePath: string) => agentsApi.deleteInstructionsFile(agent.id, relativePath, companyId, bundle?.files.find(file => file.path === relativePath)?.contentHash),
onMutate: () => {
editorInteractedRef.current = false;
setAwaitingRefresh(true);
@@ -2409,7 +2492,7 @@ export function PromptsTab({
return;
}
if (lastFileVersionRef.current !== versionKey) {
setDraft(null);
if (draftBaseRevisionRef.current === undefined) setDraft(null);
lastFileVersionRef.current = versionKey;
}
}, [awaitingRefresh, currentMode, currentRootPath, selectedFileDetail, selectedFileExists, selectedOrEntryFile]);
@@ -2452,8 +2535,8 @@ export function PromptsTab({
),
);
const fileDirty = draft !== null && draft !== currentContent;
const isDirty = bundleDirty || fileDirty;
const isSaving = updateBundle.isPending || saveFile.isPending || deleteFile.isPending || awaitingRefresh;
const isDirty = bundleDirty || fileDirty || candidateRunId !== null;
const isSaving = updateBundle.isPending || saveFile.isPending || resolveCandidate.isPending || loadCandidate.isPending || refreshCandidateBase.isPending || deleteFile.isPending || awaitingRefresh;
useEffect(() => { onSavingChange(isSaving); }, [onSavingChange, isSaving]);
useEffect(() => { onDirtyChange(isDirty); }, [onDirtyChange, isDirty]);
@@ -2477,10 +2560,15 @@ export function PromptsTab({
entryFile: bundleDraft.entryFile,
});
}
if (fileDirty) {
if (candidateRunId) {
await resolveCandidate.mutateAsync({ runId: candidateRunId, content: displayValue,
baseRevisionId: draftBaseRevisionRef.current ?? null });
} else if (fileDirty) {
await saveFile.mutateAsync({
path: selectedOrEntryFile,
content: displayValue,
...(bundle?.persistence === "agent_files" ? { baseHash: draftBaseHashRef.current !== undefined ? draftBaseHashRef.current : selectedFileDetail?.contentHash ?? null } : {}),
...(selectedOrEntryFile === currentEntryFile && currentMode === "managed" ? { baseRevisionId: draftBaseRevisionRef.current !== undefined ? draftBaseRevisionRef.current : selectedFileDetail?.revision?.id ?? null } : {}),
clearLegacyPromptTemplate: shouldClearLegacy,
});
}
@@ -2491,10 +2579,15 @@ export function PromptsTab({
bundle,
bundleDirty,
bundleDraft,
candidateRunId,
resolveCandidate,
displayValue,
fileDirty,
isDirty,
onSaveActionChange,
selectedFileDetail?.revision?.id,
currentEntryFile,
currentMode,
saveFile,
selectedOrEntryFile,
updateBundle,
@@ -2502,6 +2595,10 @@ export function PromptsTab({
useEffect(() => {
onCancelActionChange(isDirty ? () => {
draftBaseRevisionRef.current = undefined;
draftBaseHashRef.current = undefined;
setCandidateRunId(null);
resolveCandidate.reset();
setDraft(null);
if (bundle) {
setBundleDraft({
@@ -2511,7 +2608,7 @@ export function PromptsTab({
});
}
} : null);
}, [bundle, isDirty, onCancelActionChange, persistedMode, persistedRootPath]);
}, [bundle, isDirty, onCancelActionChange, persistedMode, persistedRootPath, resolveCandidate]);
const handleSeparatorDrag = useCallback((event: React.MouseEvent) => {
event.preventDefault();
@@ -2742,6 +2839,7 @@ export function PromptsTab({
size="icon"
variant="outline"
className="h-7 w-7"
aria-label="Add agent file"
onClick={() => setShowNewFileInput(true)}
>
+
@@ -2938,7 +3036,68 @@ export function PromptsTab({
</div>
</div>
{selectedFileExists && fileLoading && !selectedFileDetail ? (
{currentMode === "managed" && preservedCandidates.length > 0 && (
<div className="space-y-3">
<p className="text-sm font-medium">Preserved instruction edits</p>
<p className="text-sm text-muted-foreground">Older instruction-only sessions have edits to review.</p>
{preservedCandidates.map((candidate) => (
<div key={candidate.runId} className="flex flex-wrap items-center gap-3">
<span className="font-mono text-xs text-muted-foreground">{candidate.runId.slice(0, 8)}</span>
<span className="text-sm text-muted-foreground">{candidate.entryFile} · {formatDate(candidate.createdAt)}</span>
<Button type="button" variant="outline" size="sm"
disabled={candidate.content === null || (candidate.entryFile === currentEntryFile && (isDirty || isSaving))}
aria-expanded={candidate.entryFile !== currentEntryFile ? readOnlyCandidateRunId === candidate.runId : undefined}
onClick={() => {
if (candidate.entryFile !== currentEntryFile) {
setReadOnlyCandidateRunId((current) => current === candidate.runId ? null : candidate.runId);
} else {
loadCandidate.mutate(candidate);
}
}}>Review preserved edits</Button>
{candidate.errorMessage && <p className="text-sm text-muted-foreground">{candidate.errorMessage}</p>}
{candidate.entryFile !== currentEntryFile && <p className="text-sm text-muted-foreground">The instruction entry changed. These edits remain preserved for the original file.</p>}
{candidate.entryFile !== currentEntryFile && candidate.content !== null && readOnlyCandidateRunId === candidate.runId && (
<div role="region" aria-label={`Preserved edits for ${candidate.entryFile}`} className="w-full space-y-3">
<p className="text-sm text-muted-foreground">Read only: {candidate.entryFile}. To keep any of these edits in {currentEntryFile}, copy them and edit the current entry explicitly.</p>
<CopyText text={candidate.content} ariaLabel={`Copy preserved edits for ${candidate.entryFile}`}
className="inline-flex items-center gap-2 rounded-md border border-border px-3 py-2 text-sm text-muted-foreground hover:bg-accent hover:text-foreground">
<Copy className="h-3.5 w-3.5" />Copy preserved edits
</CopyText>
<pre className="overflow-x-auto whitespace-pre-wrap break-words rounded-md border border-border bg-muted p-3 font-mono text-sm">{candidate.content}</pre>
</div>
)}
</div>
))}
</div>
)}
{candidateRunId && <div role="status" className="space-y-3">
<p className="text-sm text-muted-foreground">Reviewing preserved edits. Save to apply your resolved draft and close this preserved edit.</p>
<details><summary className="cursor-pointer text-sm text-muted-foreground">Compare current instructions</summary>
<pre className="whitespace-pre-wrap break-words rounded-md border border-border p-3 font-mono text-sm">{currentContent}</pre>
</details>
{draftBaseRevisionRef.current !== (selectedFileDetail?.revision?.id ?? null) && <p className="text-sm text-destructive">The current instructions changed after this draft was loaded. Refresh the current revision, compare the instructions, and save your resolved draft again.</p>}
{(resolveCandidate.error || draftBaseRevisionRef.current !== (selectedFileDetail?.revision?.id ?? null)) && <Button type="button" variant="outline" size="sm" disabled={isSaving} onClick={() => refreshCandidateBase.mutate()}>Refresh current revision</Button>}
</div>}
{(candidates.error || loadCandidate.error || resolveCandidate.error || refreshCandidateBase.error) && <p role="alert" className="text-sm text-destructive">{(candidates.error ?? loadCandidate.error ?? resolveCandidate.error ?? refreshCandidateBase.error)?.message} Your preserved edits remain available.</p>}
{(saveFile.error || fileError || updateBundle.error) && <p role="alert" className="text-sm text-destructive">{(saveFile.error ?? fileError ?? updateBundle.error)?.message} Your unsaved edits are retained.</p>}
{selectedFileDetail?.receipt?.materialization === "pending" && <p role="status" className="text-sm text-muted-foreground">Revision saved. The instruction file still needs to be rebuilt from the saved revision.</p>}
{selectedFileDetail?.revision && currentMode === "managed" && bundle?.persistence !== "agent_files" && <InstructionHistory
key={selectedOrEntryFile} agentId={agent.id} companyId={companyId} path={selectedOrEntryFile}
currentRevisionId={selectedFileDetail.revision.id} disabled={isDirty || isSaving}
onRestored={(file) => {
setDraft(null);
draftBaseRevisionRef.current = undefined;
draftBaseHashRef.current = undefined;
queryClient.setQueryData(queryKeys.agents.instructionsFile(agent.id, selectedOrEntryFile), file);
queryClient.invalidateQueries({ queryKey: queryKeys.agents.instructionsBundle(agent.id) });
}}
/>}
{selectedFileDetail?.binary ? (
<div className="space-y-3 rounded-md border border-border p-4">
<p className="text-sm text-muted-foreground">This file is preserved with the agent directory. Download it to view its contents.</p>
<a className="text-sm text-primary underline" href={agentsApi.downloadInstructionsFile(agent.id, selectedOrEntryFile, companyId)} download>Download {selectedOrEntryFile}</a>
</div>
) : selectedFileExists && fileLoading && !selectedFileDetail ? (
<PromptEditorSkeleton />
) : instructionMode === "read" ? (
<div className="min-h-(--sz-420px) rounded-md border border-border bg-background p-4">
@@ -2975,6 +3134,8 @@ export function PromptsTab({
value={displayValue}
onChange={(value) => {
if (!editorInteractedRef.current) return;
if (draftBaseRevisionRef.current === undefined) draftBaseRevisionRef.current = selectedFileDetail?.revision?.id ?? null;
if (draftBaseHashRef.current === undefined) draftBaseHashRef.current = selectedFileDetail?.contentHash ?? null;
setDraft(value ?? "");
}}
placeholder="# Agent instructions"
@@ -2991,7 +3152,11 @@ export function PromptsTab({
<textarea
aria-label="Instruction file editor"
value={displayValue}
onChange={(event) => setDraft(event.target.value)}
onChange={(event) => {
if (draftBaseRevisionRef.current === undefined) draftBaseRevisionRef.current = selectedFileDetail?.revision?.id ?? null;
if (draftBaseHashRef.current === undefined) draftBaseHashRef.current = selectedFileDetail?.contentHash ?? null;
setDraft(event.target.value);
}}
className="min-h-(--sz-420px) w-full min-w-0 rounded-md border border-border bg-transparent px-3 py-2 font-mono text-sm outline-none"
placeholder="File contents"
/>
@@ -3194,6 +3359,20 @@ function RunsTab({
/* ---- Run Detail (expanded) ---- */
export function AgentFileRunNotice({ resultJson }: { resultJson: HeartbeatRun["resultJson"] }) {
const save = asRecord(resultJson?.instructionSave);
const storageWarning = asNonEmptyString(save?.storageWarning);
const error = asNonEmptyString(save?.errorMessage);
const syncFailure = save?.contract === "agent_files" && save.state === "unavailable" && error;
// A quota rejection is already explained by the storage warning. A separate
// I/O failure must stay visible even when storage was full at run start.
const showSyncFailure = syncFailure && (!storageWarning || save?.errorCode !== "AGENT_FILES_LIMIT_EXCEEDED");
return <>
{storageWarning && <InlineBanner tone="warning" title="Agent storage warning" compact>{storageWarning}</InlineBanner>}
{showSyncFailure && <InlineBanner tone="warning" title="Agent file sync failed for this run" compact>{error}</InlineBanner>}
</>;
}
function RunDetail({ run: initialRun, agentRouteId, adapterType, adapterConfig }: { run: HeartbeatRun; agentRouteId: string; adapterType: string; adapterConfig: Record<string, unknown> }) {
const queryClient = useQueryClient();
const navigate = useNavigate();
@@ -3562,6 +3741,7 @@ function RunDetail({ run: initialRun, agentRouteId, adapterType, adapterConfig }
)}
</div>
)}
<AgentFileRunNotice resultJson={run.resultJson} />
{run.error && (
<div className="text-xs">
<span className="text-red-600 dark:text-red-400">{run.error}</span>
@@ -0,0 +1,182 @@
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
import type { Meta, StoryObj } from "@storybook/react-vite";
import { expect, userEvent, waitFor, within } from "storybook/test";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import type { AgentInstructionsFileDetail } from "@paperclipai/shared";
import { AgentFileRunNotice, PromptsTab } from "@/pages/AgentDetail";
import { Button } from "@/components/ui/button";
import { queryKeys } from "@/lib/queryKeys";
import { storybookAgents } from "../fixtures/paperclipData";
const agent = { ...storybookAgents[0]!, id: "agent-files-story", adapterType: "codex_local" as const };
const root = "/instance/agents/agent-files-story/instructions";
const originalInstructions = "# Agent instructions\n\nRead your notes in `notes/context.txt` before starting a task.\n\nKeep useful files in your agent directory for future tasks.\n";
const incomingInstructions = "# Agent instructions\n\nRead your notes before starting a task. Verify changes before handing them off.\n";
const noop = () => {};
const syncFailure = 'Agent file "large.bin" exceeds the 256 MiB per-file limit. This run\'s agent-folder changes were not saved; the temporary copy is discarded.';
const storageWarning = "Agent storage is full. The agent folder has reached its 2 GiB limit. Runs can continue; remove or shrink files in AGENT_HOME to free space. Changes exceeding the storage limits will not be saved.";
function AgentFilesStory({ failedSync = false, fullStorageRun = false, historicalFailures = false }: { failedSync?: boolean; fullStorageRun?: boolean; historicalFailures?: boolean }) {
const client = useMemo(() => new QueryClient({ defaultOptions: { queries: { retry: false, staleTime: Infinity }, mutations: { retry: false } } }), []);
const [ready, setReady] = useState(false);
const [dirty, setDirty] = useState(false);
const [saving, setSaving] = useState(false);
const [saved, setSaved] = useState(0);
const save = useRef<(() => void) | null>(null);
const cancel = useRef<(() => void) | null>(null);
const simulateAgent = useRef(noop);
const onSave = useCallback((action: (() => void) | null) => { save.current = action; }, []);
const onCancel = useCallback((action: (() => void) | null) => { cancel.current = action; }, []);
useEffect(() => {
let version = 0;
const file = (path: string, content: string, binary = false): AgentInstructionsFileDetail => ({
path, content, contentHash: `fixture-${++version}`, binary,
size: binary ? 4 : new TextEncoder().encode(content).length,
language: path.endsWith(".md") ? "markdown" : "text", markdown: path.endsWith(".md"),
isEntryFile: path === "AGENTS.md", editable: !binary, deprecated: false, virtual: false,
});
const files = new Map([
["AGENTS.md", file("AGENTS.md", originalInstructions)],
["notes/context.txt", file("notes/context.txt", "Use short, concrete updates.\n")],
["cache.bin", file("cache.bin", "", true)],
]);
const bundle = () => ({ agentId: agent.id, companyId: agent.companyId, persistence: "agent_files", mode: "managed",
rootPath: root, managedRootPath: root, entryFile: "AGENTS.md", resolvedEntryPath: `${root}/AGENTS.md`, editable: true,
warnings: [], legacyPromptTemplateActive: false, legacyBootstrapPromptTemplateActive: false,
files: [...files.values()].map(({ content: _content, ...summary }) => summary),
});
simulateAgent.current = () => {
files.set("AGENTS.md", file("AGENTS.md", incomingInstructions));
client.invalidateQueries({ queryKey: queryKeys.agents.instructionsBundle(agent.id) });
client.invalidateQueries({ queryKey: queryKeys.agents.instructionsFile(agent.id, "AGENTS.md") });
};
const originalFetch = window.fetch;
const fixture: typeof fetch = async (input, init) => {
const url = new URL(typeof input === "string" ? input : input instanceof URL ? input.href : input.url, window.location.origin);
const base = `/api/agents/${agent.id}/instructions-bundle`;
if (!url.pathname.startsWith(base)) return originalFetch(input, init);
const method = init?.method ?? (input instanceof Request ? input.method : "GET");
const suffix = url.pathname.slice(base.length);
const data = JSON.parse(String(init?.body ?? "{}"));
if (suffix === "" && method === "GET") return Response.json(bundle());
if (suffix === "/file") {
const path = method === "PUT" ? data.path : url.searchParams.get("path")!;
const current = files.get(path);
if (method === "PUT" || method === "DELETE") {
const baseHash = method === "PUT" ? data.baseHash : url.searchParams.get("baseHash");
if (baseHash !== (current?.contentHash ?? null)) return Response.json({ error: "This file changed after you started editing. Compare the current file before saving again." }, { status: 409 });
if (method === "DELETE") { files.delete(path); return Response.json({ ok: true }); }
const next = file(path, data.content);
files.set(path, next); setSaved(count => count + 1);
return Response.json(next);
}
return current ? Response.json(current) : Response.json({ error: "File not found" }, { status: 404 });
}
if (suffix === "/candidates") return Response.json(historicalFailures ? [1, 2, 3].map(attempt => ({
contract: "agent_files", runId: `failed-auth-${attempt}`, entryFile: "AGENTS.md", state: "unavailable", content: null, candidateHash: null,
errorMessage: "The registered instruction copy could not be retrieved safely before environment release. No instruction save is claimed.",
})) : []);
return Response.json({ error: "This action is not included in this story." }, { status: 400 });
};
window.fetch = fixture; setReady(true);
return () => { if (window.fetch === fixture) window.fetch = originalFetch; client.clear(); };
}, [client, historicalFailures]);
return <QueryClientProvider client={client}>
<div className="space-y-6 p-6">
<div className="space-y-3 rounded-md border border-border bg-muted p-4">
<p className="text-sm">Storybook simulation · Real agent instructions editor with in-memory file responses. No provider runs or files on disk. Binary download is shown as a link only.</p>
<Button variant="outline" onClick={() => simulateAgent.current()}>Simulate agent edit</Button>
<p role="status" className="text-sm text-muted-foreground">{saved ? `${saved} file save completed in this story.` : "Saved agent files are ready for the next task."}</p>
</div>
{(failedSync || fullStorageRun) && <section aria-label="Affected run">
<p className="text-sm font-medium">Affected run</p>
<AgentFileRunNotice resultJson={{ instructionSave: { contract: "agent_files", state: failedSync ? "unavailable" : "saved",
errorCode: failedSync ? "AGENT_FILES_LIMIT_EXCEEDED" : null,
errorMessage: failedSync ? syncFailure : null, storageWarning: fullStorageRun ? storageWarning : null } }} />
</section>}
{ready && <PromptsTab agent={agent} companyId={agent.companyId} onDirtyChange={setDirty} onSavingChange={setSaving}
onSaveActionChange={onSave} onCancelActionChange={onCancel} />}
<div className="flex items-center justify-between border-t border-border pt-4">
<Button variant="outline" disabled={!dirty || saving} onClick={() => cancel.current?.()}>Cancel changes</Button>
<Button disabled={!dirty || saving} onClick={() => save.current?.()}>Save changes</Button>
</div>
</div>
</QueryClientProvider>;
}
const meta = { title: "Agents/Persistent files", component: AgentFilesStory, parameters: { layout: "fullscreen" } } satisfies Meta<typeof AgentFilesStory>;
export default meta;
type Story = StoryObj<typeof meta>;
export const InstructionsAndFiles: Story = {};
export const BrowserEdit: Story = {
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await userEvent.click(await canvas.findByRole("button", { name: "edit" }));
const editor = await canvas.findByRole("textbox");
await userEvent.click(editor);
await userEvent.type(editor, "Browser edits persist for future tasks. ");
await userEvent.click(canvas.getByRole("button", { name: "Save changes" }));
await expect(await canvas.findByText("1 file save completed in this story.")).toBeVisible();
await waitFor(() => expect(canvas.getByRole("button", { name: "Save changes" })).toBeDisabled());
},
};
export const AgentEditArrives: Story = {
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await canvas.findByRole("button", { name: "edit" });
await userEvent.click(canvas.getByRole("button", { name: "Simulate agent edit" }));
await expect(await canvas.findByText(/Verify changes before handing them off/)).toBeVisible();
},
};
export const BinaryFile: Story = {
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await userEvent.click(await canvas.findByText("cache.bin"));
await expect(await canvas.findByRole("link", { name: "Download cache.bin" })).toBeVisible();
},
};
export const LastSyncWins: Story = { ...AgentEditArrives };
export const StorageLimit: Story = {
args: { failedSync: true },
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await expect(await canvas.findByRole("note")).toHaveTextContent("256 MiB per-file limit");
await expect(canvas.queryByRole("button", { name: "Review preserved files" })).not.toBeInTheDocument();
},
};
export const HistoricalFailuresAfterSuccessfulSave: Story = {
args: { historicalFailures: true },
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await canvas.findByRole("button", { name: "edit" });
await userEvent.click(canvas.getByRole("button", { name: "Simulate agent edit" }));
await expect(await canvas.findByText(/Verify changes before handing them off/)).toBeVisible();
await expect(canvas.queryByText("Agent file sync")).not.toBeInTheDocument();
await expect(canvas.queryByText(/could not be retrieved safely/)).not.toBeInTheDocument();
await expect(canvas.queryByRole("alert")).not.toBeInTheDocument();
},
};
export const StaleEditor: Story = {
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await userEvent.click(await canvas.findByRole("button", { name: "edit" }));
await userEvent.type(await canvas.findByRole("textbox"), "Keep this unsaved draft. ");
await userEvent.click(canvas.getByRole("button", { name: "Simulate agent edit" }));
await userEvent.click(canvas.getByRole("button", { name: "Save changes" }));
await expect(await canvas.findByRole("alert")).toHaveTextContent("Your unsaved edits are retained.");
await expect(canvas.getByRole("textbox")).toHaveTextContent("Keep this unsaved draft.");
},
};
export const FullStorageRunWarning: Story = {
args: { fullStorageRun: true },
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await expect(await canvas.findByRole("note")).toHaveTextContent("Runs can continue");
await expect(canvas.getByRole("note")).toHaveTextContent("2 GiB");
await expect(await canvas.findByRole("button", { name: "edit" })).toBeEnabled();
},
};