From 4ff493fb40fa731eb506b5e18250d3e8a733a912 Mon Sep 17 00:00:00 2001 From: Nicky Leach Date: Mon, 26 Oct 2026 14:11:35 -0700 Subject: [PATCH] feat(plugins): include managed skills for selected environments (#17774) ## Thinking Path > - Paperclip manages agents and their execution environments. > - Plugins can vendor company skills and provide environments. > - Environment tools need their instructions before the provider starts. > - Persistent computers also need those skills before any run begins. > - This change adds environment requirements to normal skill selection and lifecycle synchronization. > - Plugin names prevent ambiguity, and required skills take delivery precedence without deleting company data. ## Linked Issues or Issue Description **Problem or motivation** An environment can provide desktop tools while its agent receives no instructions for those tools. A first-run-only import also leaves idle computers without required skills. Same-name company skills can make provider discovery ambiguous. **Proposed solution** Let plugin environment drivers declare their managed `skillKeys`. Install missing requirements during lifecycle reconciliation or new-run preparation. Deliver updates through the durable runtime event journal. Give managed imports stable plugin names and report shadowed assignments. **Alternatives considered** ROM links do not make a skill a normal Paperclip selection. Polling duplicates the existing lifecycle delivery queue. Replacing a same-name company record would destroy independent data and assignments. **Roadmap alignment** This extends the shipped Plugin system and Skills Manager. The existing related-PR search found no duplicate contract. ## What Changed - Validate `environmentDrivers[].skillKeys` as unique references to the same plugin's managed skills. Require a ready, enabled plugin with `skills.managed` authority. - Resolve company-scoped skills before version capture. Preserve explicit pins, company edits, and saved recovery selections. Serialize concurrent first installs. - Allow lifecycle callers to request missing-skill installation through `agents.getRuntime`. Ordinary task and recovery reads do not install new declarations. Pass task-selected environment IDs through the SDK. - Add SQL migration `0378_environment_skill_runtime_hooks`. Required skill edits, effective default changes, environment changes, and plugin requirement changes enqueue durable runtime events. Reuse lifecycle delivery and retries; add no polling. - Import managed defaults as `plugin--`, including the SKILL.md name. Preserve canonical keys. Keep same-name company records separate without counter suffixes. - Required environment skills win delivery over same-name company assignments. Preserve saved assignments and library content. Record shadowed and winning keys in run warnings, run snapshots, and plugin runtime state. Removing the environment requirement restores the ordinary assignment. - Document the contract for `driver: "plugin"`. Other drivers and plugins without requirements keep existing behavior. Existing managed copies retain edits and names until reset. ## Verification - Latest targeted local checks: environment skills (5), managed skills (9), heartbeat execution configuration (27), and SDK worker RPC host (23): **65 tests passed** (including the targeted schema-drift check). Disposable PostgreSQL exercised the migration, first install before any run, concurrent lifecycle reads, implicit edit events, inherited defaults, rollback, plugin requirement changes, shadowing, preserved records, version capture, and recovery. - Review follow-up also verifies case-insensitive/runtime-name aliases, recovery of a task-mentioned shadowed skill, preserved quoted/multiline YAML, and zero runtime events for repeated local-folder settings writes. - Migration numbering and safety checks passed. Whitespace and secret/PII scans passed. - Earlier targeted manifest validation passed (43 tests). No validator behavior changed in this follow-up. - Broad tests, typecheck, build and end-to-end verification passed in [GitHub CI](https://github.com/paperclipai/paperclip/actions/runs/40387647219) for current head `ce57f2a4d`. Greptile is 5/5 with no unresolved threads. The [commitperclip review check](https://github.com/paperclipai/paperclip/actions/runs/40387646208) also passed after restoring its required duplicate-search checkbox. - Earlier attempts found a SQL alias collision, an unparenthesized CASE expression, and fixture assumptions about bundled catalog rows and YAML quoting. These were corrected before the passing checks. The first broad CI run also exposed a schema-drift check that assumed every journal entry had a schema snapshot; it now reads the latest actual snapshot, so SQL-only migrations need no invented snapshot. A test callback replacement also affected an unrelated fixture; it was restored before the final passing execution-configuration suite. Sandbox runs could not start PostgreSQL; the reported database tests used local socket access. ## Risks - Apply migration 0378 before using the updated lifecycle contract. It adds trigger functions and event producers; it does not rewrite agent assignments or skill content. - Required environment skills take precedence over same-name assignments, including task-selected skills. Both library records remain. Run warnings identify omissions. - Company edits are preserved. Plugin upgrades do not reset installed content or rename existing copies. Explicit reset adopts current defaults. - Namespaced library slugs can be shared by distinct canonical records. Callers must use IDs or canonical keys when a slug is ambiguous. - No live provider request or deployment was performed. The downstream plugin must be built against this Core revision. Persistent-home synchronization retains its existing edit-conflict and retry behavior. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, and code execution. The exact model identifier and context-window size are not exposed in this task context. ## Checklist - [x] Thinking path and issue description included - [x] Model and available capabilities recorded - [x] Roadmap and related PRs checked - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] Public references only; no internal issue IDs or instance URLs - [x] Targeted local tests pass; broad verification delegated to GitHub CI - [x] New behavior covered and documentation updated - [x] Current-head CI green and Greptile 5/5 with no unresolved comments --------- Co-authored-by: Paperclip --- doc/DEVELOPING.md | 28 +++ .../db/src/migration-snapshot-drift.test.ts | 18 +- .../0378_environment_skill_runtime_hooks.sql | 111 ++++++++++ packages/db/src/migrations/meta/_journal.json | 9 +- packages/plugins/sdk/src/protocol.ts | 2 +- packages/plugins/sdk/src/types.ts | 5 +- packages/plugins/sdk/src/worker-rpc-host.ts | 4 +- .../plugins/sdk/tests/worker-rpc-host.test.ts | 4 +- packages/shared/src/types/plugin.ts | 2 + packages/shared/src/validators/plugin.test.ts | 11 + packages/shared/src/validators/plugin.ts | 8 + .../src/__tests__/environment-skills.test.ts | 196 ++++++++++++++++++ .../heartbeat-execution-configuration.test.ts | 57 ++++- .../__tests__/plugin-managed-skills.test.ts | 51 ++++- server/src/services/company-skills.ts | 6 +- server/src/services/environment-skills.ts | 64 ++++++ .../heartbeat/execution-configuration.ts | 29 ++- .../services/plugin-agent-runtime-state.ts | 24 ++- server/src/services/plugin-host-services.ts | 2 +- server/src/services/plugin-managed-skills.ts | 53 ++--- 20 files changed, 615 insertions(+), 69 deletions(-) create mode 100644 packages/db/src/migrations/0378_environment_skill_runtime_hooks.sql create mode 100644 server/src/__tests__/environment-skills.test.ts create mode 100644 server/src/services/environment-skills.ts diff --git a/doc/DEVELOPING.md b/doc/DEVELOPING.md index 3715e0983da..e22fa6b0e46 100644 --- a/doc/DEVELOPING.md +++ b/doc/DEVELOPING.md @@ -2363,6 +2363,34 @@ identity mismatches. ## App-Shipped Skills Catalog +Plugin environment drivers can declare `skillKeys` that reference their own +manifest-managed skills. Selecting that environment adds those skills to the +normal authorized skill selection, including an explicitly empty optional +selection. Core installs a missing managed skill in the current company's +library during lifecycle reconciliation or new-run admission and preserves existing +skill edits and explicit version pins. New managed skills use +`plugin--` for both the library slug and SKILL.md name. +The short ID is the final dot-separated plugin ID component, without a leading +`plugin-`. An unrelated skill can keep the same slug in a separate canonical +record. The required environment skill wins during delivery: a selected company +skill with the same slug or name is omitted and reported as shadowed. Stored agent +assignments and both library records remain unchanged. Run warnings and snapshots, +and the plugin runtime response, identify the omitted key and the winning key. Existing company copies keep their edits and names +until reset. Lifecycle callers request installation with +`agents.getRuntime(agentId, companyId, undefined, true)`; ordinary runtime reads and +recovery do not install newly declared skills. Run +preparation captures the same immutable versions as ordinary assignments; +recovery uses the saved selection. The lifecycle runtime reader includes the +effective default environment's skills before synchronizing the persistent home. +A task provider can pass its selected environment ID to `agents.getRuntime` when +a task overrides that default. This +contract currently applies to `driver: "plugin"` environments. Other drivers +and plugin drivers without `skillKeys` keep their existing behavior. No agent +configuration is rewritten, and no background scan is added. Changes to implicitly +selected skills, the effective default environment, or plugin requirements enqueue +normal durable runtime events, so an idle ready computer receives updates without +a new run. Failed synchronization retains the event for the existing retry path. + The Paperclip app ships a curated catalog of company skills out of the box. The catalog is a workspace package at `packages/skills-catalog`: diff --git a/packages/db/src/migration-snapshot-drift.test.ts b/packages/db/src/migration-snapshot-drift.test.ts index b67c985698e..918e3435ef9 100644 --- a/packages/db/src/migration-snapshot-drift.test.ts +++ b/packages/db/src/migration-snapshot-drift.test.ts @@ -15,19 +15,13 @@ import { generateDrizzleJson, generateMigration } from "drizzle-kit/api"; const migrationsDir = fileURLToPath(new URL("./migrations", import.meta.url)); const schemaDir = fileURLToPath(new URL("./schema", import.meta.url)); -type JournalEntry = { idx: number; tag: string }; - async function readNewestSnapshot(): Promise<{ file: string; snapshot: Record }> { - const journal = JSON.parse( - await readFile(path.join(migrationsDir, "meta", "_journal.json"), "utf8"), - ) as { entries: JournalEntry[] }; - const newest = journal.entries.at(-1); - if (!newest) throw new Error("migration journal has no entries"); - const file = `${String(newest.idx).padStart(4, "0")}_snapshot.json`; - const snapshot = JSON.parse(await readFile(path.join(migrationsDir, "meta", file), "utf8")) as Record< - string, - unknown - >; + // SQL-only migrations can advance the journal without changing the schema snapshot. + const files = (await readdir(path.join(migrationsDir, "meta"))) + .filter(file => /^\d+_snapshot\.json$/.test(file)).sort(); + const file = files.at(-1); + if (!file) throw new Error("migration metadata has no schema snapshots"); + const snapshot = JSON.parse(await readFile(path.join(migrationsDir, "meta", file), "utf8")) as Record; return { file, snapshot }; } diff --git a/packages/db/src/migrations/0378_environment_skill_runtime_hooks.sql b/packages/db/src/migrations/0378_environment_skill_runtime_hooks.sql new file mode 100644 index 00000000000..5c9b7f31ae1 --- /dev/null +++ b/packages/db/src/migrations/0378_environment_skill_runtime_hooks.sql @@ -0,0 +1,111 @@ +-- Environment requirements use the same durable runtime journal as explicit skill assignments. +CREATE OR REPLACE FUNCTION paperclip_environment_selects_runtime_skill(p_company uuid, p_environment uuid, p_skill uuid) RETURNS boolean +LANGUAGE sql STABLE SET search_path = public, pg_temp AS $$ + SELECT EXISTS ( + SELECT 1 FROM environments e + JOIN plugins p ON p.plugin_key = e.config->>'pluginKey' + JOIN plugin_managed_resources r ON r.plugin_id = p.id AND r.company_id = p_company + AND r.resource_kind = 'skill' AND r.resource_id = p_skill + CROSS JOIN LATERAL jsonb_array_elements(coalesce(p.manifest_json->'environmentDrivers', '[]'::jsonb)) AS declared_driver(value) + WHERE e.id = coalesce(p_environment, (SELECT default_environment_id FROM instance_settings LIMIT 1)) + AND e.driver = 'plugin' AND declared_driver.value->>'driverKey' = e.config->>'driverKey' + AND coalesce(declared_driver.value->'skillKeys', '[]'::jsonb) ? r.resource_key); +$$; +--> statement-breakpoint +CREATE OR REPLACE FUNCTION paperclip_invalidate_skill_source_runtimes(p_company uuid, sources uuid[]) RETURNS void +LANGUAGE plpgsql SET search_path = public, pg_temp AS $$ +DECLARE candidate uuid; +BEGIN + FOR candidate IN SELECT a.id FROM agents a WHERE a.company_id = p_company + AND a.status NOT IN ('pending_approval', 'terminated') AND EXISTS ( + SELECT 1 FROM company_skills s WHERE s.company_id = p_company AND s.access_source_id = ANY(sources) + AND (paperclip_agent_selects_runtime_skill(a.adapter_config, a.adapter_type, s.key, s.slug) + OR paperclip_environment_selects_runtime_skill(p_company, a.default_environment_id, s.id))) + ORDER BY a.id FOR NO KEY UPDATE LOOP + PERFORM paperclip_record_agent_runtime(p_company, candidate); + END LOOP; +END; +$$; +--> statement-breakpoint +CREATE OR REPLACE FUNCTION paperclip_skill_runtime_changed() RETURNS trigger +LANGUAGE plpgsql SET search_path = public, pg_temp AS $$ +DECLARE previous jsonb; current jsonb; keys text[]; company uuid; candidate uuid; +BEGIN + IF TG_OP <> 'INSERT' THEN previous := to_jsonb(OLD); END IF; + IF TG_OP <> 'DELETE' THEN current := to_jsonb(NEW); END IF; + IF TG_OP = 'UPDATE' AND + jsonb_build_array(OLD.key, OLD.slug, OLD.markdown, OLD.source_type, OLD.source_locator, OLD.source_ref, + OLD.file_inventory, OLD.current_version_id, OLD.access_source_id, OLD.compatibility, OLD.trust_level, OLD.metadata) + IS NOT DISTINCT FROM + jsonb_build_array(NEW.key, NEW.slug, NEW.markdown, NEW.source_type, NEW.source_locator, NEW.source_ref, + NEW.file_inventory, NEW.current_version_id, NEW.access_source_id, NEW.compatibility, NEW.trust_level, NEW.metadata) + THEN RETURN NEW; END IF; + company := coalesce((current->>'company_id')::uuid, (previous->>'company_id')::uuid); + SELECT array_agg(DISTINCT lower(reference)) INTO keys FROM ( + SELECT row->>'key' AS reference FROM unnest(ARRAY[previous, current]) row + UNION ALL SELECT regexp_replace(row->>'key', '.*/', '') FROM unnest(ARRAY[previous, current]) row + UNION ALL SELECT CASE WHEN row->>'key' LIKE 'paperclipai/paperclip/%' THEN row->>'slug' + ELSE (row->>'slug') || '--' || left(encode(sha256(convert_to(row->>'key', 'UTF8')), 'hex'), 10) END + FROM unnest(ARRAY[previous, current]) row + ) refs WHERE reference IS NOT NULL; + FOR candidate IN SELECT a.id FROM agents a WHERE a.company_id = company + AND a.status NOT IN ('pending_approval', 'terminated') AND ( + (a.adapter_type <> 'paperclip_runner' AND keys && ARRAY['paperclipai/paperclip/paperclip', 'paperclipai/paperclip/complain', 'paperclipai/paperclip/suggestion-box']) OR + EXISTS (SELECT 1 FROM jsonb_array_elements(CASE WHEN jsonb_typeof(a.adapter_config#>'{paperclipSkillSync,desiredSkills}') = 'array' + THEN a.adapter_config#>'{paperclipSkillSync,desiredSkills}' ELSE '[]'::jsonb END) choice + WHERE lower(trim(CASE WHEN jsonb_typeof(choice) = 'string' THEN choice#>>'{}' ELSE choice->>'key' END)) = ANY(keys)) + OR paperclip_environment_selects_runtime_skill(company, a.default_environment_id, coalesce((current->>'id')::uuid, (previous->>'id')::uuid))) + ORDER BY a.id FOR NO KEY UPDATE LOOP + PERFORM paperclip_record_agent_runtime(company, candidate); + END LOOP; + IF TG_OP = 'DELETE' THEN RETURN OLD; END IF; + RETURN NEW; +END; +$$; +--> statement-breakpoint +CREATE OR REPLACE FUNCTION paperclip_environment_runtimes_changed() RETURNS trigger +LANGUAGE plpgsql SET search_path = public, pg_temp AS $$ +DECLARE previous jsonb; current jsonb; candidate record; +BEGIN + IF TG_OP <> 'INSERT' THEN previous := to_jsonb(OLD); END IF; + IF TG_OP <> 'DELETE' THEN current := to_jsonb(NEW); END IF; + IF TG_TABLE_NAME = 'plugin_company_settings' THEN + IF coalesce((previous->>'enabled')::boolean, true) IS NOT DISTINCT FROM coalesce((current->>'enabled')::boolean, true) THEN + IF TG_OP = 'DELETE' THEN RETURN OLD; END IF; + RETURN NEW; + END IF; + ELSIF TG_OP = 'UPDATE' AND (CASE TG_TABLE_NAME + WHEN 'environments' THEN previous->'config' IS NOT DISTINCT FROM current->'config' AND previous->'driver' IS NOT DISTINCT FROM current->'driver' + WHEN 'plugins' THEN previous->'manifest_json' IS NOT DISTINCT FROM current->'manifest_json' AND previous->'status' IS NOT DISTINCT FROM current->'status' + ELSE previous->'default_environment_id' IS NOT DISTINCT FROM current->'default_environment_id' END) THEN RETURN NEW; END IF; + FOR candidate IN SELECT a.id, a.company_id FROM agents a + WHERE a.status NOT IN ('pending_approval', 'terminated') AND (CASE TG_TABLE_NAME + WHEN 'environments' THEN coalesce(a.default_environment_id, (SELECT default_environment_id FROM instance_settings LIMIT 1)) + = coalesce((current->>'id')::uuid, (previous->>'id')::uuid) + WHEN 'plugins' THEN EXISTS (SELECT 1 FROM environments e + WHERE e.id = coalesce(a.default_environment_id, (SELECT default_environment_id FROM instance_settings LIMIT 1)) + AND e.driver = 'plugin' AND e.config->>'pluginKey' = coalesce(current->>'plugin_key', previous->>'plugin_key')) + WHEN 'plugin_company_settings' THEN a.company_id = coalesce((current->>'company_id')::uuid, (previous->>'company_id')::uuid) + AND EXISTS (SELECT 1 FROM environments e JOIN plugins p ON p.plugin_key = e.config->>'pluginKey' + WHERE e.id = coalesce(a.default_environment_id, (SELECT default_environment_id FROM instance_settings LIMIT 1)) + AND e.driver = 'plugin' AND p.id = coalesce((current->>'plugin_id')::uuid, (previous->>'plugin_id')::uuid)) + ELSE a.default_environment_id IS NULL END) + ORDER BY a.company_id, a.id FOR NO KEY UPDATE OF a LOOP + PERFORM paperclip_record_agent_runtime(candidate.company_id, candidate.id); + END LOOP; + IF TG_OP = 'DELETE' THEN RETURN OLD; END IF; + RETURN NEW; +END; +$$; +--> statement-breakpoint +CREATE TRIGGER paperclip_environment_runtime_changed AFTER INSERT OR UPDATE ON environments + FOR EACH ROW EXECUTE FUNCTION paperclip_environment_runtimes_changed(); +--> statement-breakpoint +CREATE TRIGGER paperclip_plugin_environment_runtime_changed AFTER UPDATE ON plugins + FOR EACH ROW EXECUTE FUNCTION paperclip_environment_runtimes_changed(); +--> statement-breakpoint +CREATE TRIGGER paperclip_plugin_company_environment_runtime_changed AFTER INSERT OR UPDATE OR DELETE ON plugin_company_settings + FOR EACH ROW EXECUTE FUNCTION paperclip_environment_runtimes_changed(); +--> statement-breakpoint +CREATE TRIGGER paperclip_default_environment_runtime_changed AFTER UPDATE ON instance_settings + FOR EACH ROW EXECUTE FUNCTION paperclip_environment_runtimes_changed(); diff --git a/packages/db/src/migrations/meta/_journal.json b/packages/db/src/migrations/meta/_journal.json index c69031562fa..e4fe28f439e 100644 --- a/packages/db/src/migrations/meta/_journal.json +++ b/packages/db/src/migrations/meta/_journal.json @@ -2633,6 +2633,13 @@ "when": 1792790134086, "tag": "0377_clear_settled_native_run_process_identity", "breakpoints": true + }, + { + "idx": 378, + "version": "7", + "when": 1793041667443, + "tag": "0378_environment_skill_runtime_hooks", + "breakpoints": true } ] -} \ No newline at end of file +} diff --git a/packages/plugins/sdk/src/protocol.ts b/packages/plugins/sdk/src/protocol.ts index 6fa7d47e944..e60187e1945 100644 --- a/packages/plugins/sdk/src/protocol.ts +++ b/packages/plugins/sdk/src/protocol.ts @@ -2153,7 +2153,7 @@ export interface WorkerToHostMethods { result: PluginAgentRuntimeIdentity | null, ]; "agents.getRuntime": [ - params: { agentId: string; companyId: string }, + params: { agentId: string; companyId: string; environmentId?: string; installEnvironmentSkills?: boolean }, result: PluginAgentRuntimeState | null, ]; "agents.get": [ diff --git a/packages/plugins/sdk/src/types.ts b/packages/plugins/sdk/src/types.ts index 641da2864b1..2e5431846c0 100644 --- a/packages/plugins/sdk/src/types.ts +++ b/packages/plugins/sdk/src/types.ts @@ -1716,6 +1716,8 @@ export interface PluginAgentConnectionAssignments { export interface PluginAgentRuntimeState { connectionAssignments: PluginAgentConnectionAssignments; skills: PluginAgentRuntimeSkill[]; + /** Selected company skills omitted because the environment requires the same skill name. */ + shadowedSkills?: Array<{ key: string; shadowedBy: string }>; revision: string; agent: Agent; instructionsRootPath: string | null; @@ -1731,7 +1733,8 @@ export type PluginAgentRuntimeIdentity = Pick; - getRuntime(agentId: string, companyId: string): Promise; + /** Lifecycle reconciliation can install missing environment skills; task and recovery reads leave this false. */ + getRuntime(agentId: string, companyId: string, environmentId?: string, installEnvironmentSkills?: boolean): Promise; list(input: { companyId: string; status?: Agent["status"]; limit?: number; offset?: number }): Promise; get(agentId: string, companyId: string): Promise; /** Pause an agent. Throws if agent is terminated or not found. Requires `agents.pause`. */ diff --git a/packages/plugins/sdk/src/worker-rpc-host.ts b/packages/plugins/sdk/src/worker-rpc-host.ts index 9a4472ea918..21a30742ca6 100644 --- a/packages/plugins/sdk/src/worker-rpc-host.ts +++ b/packages/plugins/sdk/src/worker-rpc-host.ts @@ -1166,8 +1166,8 @@ export function startWorkerRpcHost(options: WorkerRpcHostOptions): WorkerRpcHost async getRuntimeIdentity(agentId, companyId) { return callHost("agents.getRuntimeIdentity", { agentId, companyId }); }, - async getRuntime(agentId, companyId) { - return callHost("agents.getRuntime", { agentId, companyId }); + async getRuntime(agentId, companyId, environmentId, installEnvironmentSkills) { + return callHost("agents.getRuntime", { agentId, companyId, ...(environmentId ? { environmentId } : {}), ...(installEnvironmentSkills ? { installEnvironmentSkills: true } : {}) }); }, async list(input) { return callHost("agents.list", { diff --git a/packages/plugins/sdk/tests/worker-rpc-host.test.ts b/packages/plugins/sdk/tests/worker-rpc-host.test.ts index 61f523d84ad..67adcba34b0 100644 --- a/packages/plugins/sdk/tests/worker-rpc-host.test.ts +++ b/packages/plugins/sdk/tests/worker-rpc-host.test.ts @@ -1198,6 +1198,7 @@ describe("Durable lifecycle inbox RPC", () => { ctx.data.register("drain", async ({ companyId }) => { const events = await ctx.events.listLifecycle(companyId!, 10, "6"); await ctx.events.acknowledgeLifecycle(companyId!, events[0].id); + await ctx.agents.getRuntime("agent-a", companyId!, "environment-a", true); return events; }); } }); @@ -1220,11 +1221,12 @@ describe("Durable lifecycle inbox RPC", () => { input.write(serializeMessage(createRequest(method, params, id))); }); try { - await call("initialize", { manifest: { id: "fixture.lifecycle", apiVersion: 1, version: "1.0.0", displayName: "Lifecycle", description: "Fixture", author: "Tests", categories: ["automation"], capabilities: ["events.subscribe"], entrypoints: {} }, config: {}, databaseNamespace: null }); + await call("initialize", { manifest: { id: "fixture.lifecycle", apiVersion: 1, version: "1.0.0", displayName: "Lifecycle", description: "Fixture", author: "Tests", categories: ["automation"], capabilities: ["events.subscribe", "agents.lifecycle.manage"], entrypoints: {} }, config: {}, databaseNamespace: null }); expect((await call("getData", { key: "drain", companyId: "company-a", params: {} }) as { result: unknown }).result).toEqual([event]); expect(calls).toEqual([ { method: "events.listLifecycle", params: { companyId: "company-a", limit: 10, afterId: "6" } }, { method: "events.acknowledgeLifecycle", params: { companyId: "company-a", eventId: "7" } }, + { method: "agents.getRuntime", params: { companyId: "company-a", agentId: "agent-a", environmentId: "environment-a", installEnvironmentSkills: true } }, ]); } finally { worker.stop(); reader.close(); input.destroy(); output.destroy(); } }); diff --git a/packages/shared/src/types/plugin.ts b/packages/shared/src/types/plugin.ts index e4b437f77ea..efe36ab31b3 100644 --- a/packages/shared/src/types/plugin.ts +++ b/packages/shared/src/types/plugin.ts @@ -191,6 +191,8 @@ export interface SandboxProviderCapabilities { } export interface PluginEnvironmentDriverDeclaration { + /** Plugin-managed skill keys automatically included when this environment is selected. */ + skillKeys?: string[]; /** Provides Runner execution, resource preparation, and PRP connections via environmentTask. */ supportsTasks?: boolean; freshWorkspaceVersion?: 1; diff --git a/packages/shared/src/validators/plugin.test.ts b/packages/shared/src/validators/plugin.test.ts index 43e789f0cf9..5876c61d074 100644 --- a/packages/shared/src/validators/plugin.test.ts +++ b/packages/shared/src/validators/plugin.test.ts @@ -33,6 +33,17 @@ describe("plugin capability constants", () => { }); describe("plugin manifest validators", () => { + it("limits environment skills to unique declarations owned by the plugin", () => { + const manifest = { ...buildSandboxProviderManifest({ skillKeys: ["desktop"] }), + capabilities: ["environment.drivers.register", "skills.managed"], + skills: [{ skillKey: "desktop", displayName: "Desktop" }] }; + expect(pluginManifestV1Schema.parse(manifest).environmentDrivers![0].skillKeys).toEqual(["desktop"]); + for (const keys of [["missing"], ["desktop", "desktop"]]) { + expect(pluginManifestV1Schema.safeParse({ ...manifest, + environmentDrivers: [{ ...manifest.environmentDrivers[0], skillKeys: keys }] }).success).toBe(false); + } + expect(pluginManifestV1Schema.safeParse({ ...manifest, capabilities: ["environment.drivers.register"] }).success).toBe(false); + }); it("requires authority for required agent lifecycle participants", () => { const manifest = { id: "example.lifecycle", apiVersion: 1, version: "0.1.0", displayName: "Lifecycle", description: "Lifecycle tests", author: "Tests", categories: ["automation"], entrypoints: { worker: "worker.js" }, agentLifecycle: true }; expect(pluginManifestV1Schema.safeParse({ ...manifest, capabilities: ["agents.read"] }).success).toBe(false); diff --git a/packages/shared/src/validators/plugin.ts b/packages/shared/src/validators/plugin.ts index 4663958df23..92a827afcb3 100644 --- a/packages/shared/src/validators/plugin.ts +++ b/packages/shared/src/validators/plugin.ts @@ -177,6 +177,7 @@ export type SandboxProviderCapabilitiesInput = z.infer !manifest.skills?.some(skill => skill.skillKey === key))) { + ctx.addIssue({ code: z.ZodIssueCode.custom, message: "Environment skill keys must be unique references to this plugin's managed skills", path: ["environmentDrivers", index, "skillKeys"] }); + } + } + if (manifest.agents && manifest.agents.length > 0) { if (!manifest.capabilities.includes("agents.managed")) { ctx.addIssue({ diff --git a/server/src/__tests__/environment-skills.test.ts b/server/src/__tests__/environment-skills.test.ts new file mode 100644 index 00000000000..c47ad5acb07 --- /dev/null +++ b/server/src/__tests__/environment-skills.test.ts @@ -0,0 +1,196 @@ +import { createHash, randomUUID } from "node:crypto"; +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { and, eq } from "drizzle-orm"; +import { agents, companies, companySkills, createDb, environments, pluginCompanySettings, plugins, resourceLifecycleEvents, type Db } from "@paperclipai/db"; +import type { PaperclipPluginManifestV1 } from "@paperclipai/shared"; +import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest"; +import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js"; +import { environmentSkillConfig } from "../services/environment-skills.js"; +import { pluginAgentRuntimeState } from "../services/plugin-agent-runtime-state.js"; +import { companySkillService } from "../services/company-skills.js"; +import { instanceSettingsService } from "../services/instance-settings.js"; +import { withSkillActor } from "../services/skill-access.js"; +import { buildNativeRuntimeContext } from "../services/native-runtime/runtime-context.js"; + +const support = await getEmbeddedPostgresTestSupport(); +const describeDatabase = support.supported ? describe : describe.skip; +describeDatabase("environment managed skills", () => { + let database: Awaited>; + let db: Db; + let home: string; + let companyId: string; + let pluginId: string; + let manifest: PaperclipPluginManifestV1; + let environment: { driver: string; config: Record }; + beforeAll(async () => { + database = await startEmbeddedPostgresTestDatabase("paperclip-environment-skills-"); db = createDb(database.connectionString); + home = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-environment-skills-")); + }, 90_000); + afterAll(async () => { await database?.cleanup(); if (home) { + const release = async (directory: string): Promise => { + await fs.chmod(directory, 0o700); + for (const entry of await fs.readdir(directory, { withFileTypes: true })) if (entry.isDirectory()) await release(path.join(directory, entry.name)); + }; + await release(home); await fs.rm(home, { recursive: true, force: true }); + } }); + afterEach(() => vi.unstubAllEnvs()); + beforeEach(async () => { + vi.stubEnv("PAPERCLIP_HOME", home); + companyId = randomUUID(); pluginId = randomUUID(); + manifest = { id: `fixture.${pluginId}`, apiVersion: 1, version: "1.0.0", displayName: "Desktop", description: "Fixture", author: "Tests", categories: ["automation"], + capabilities: ["environment.drivers.register", "skills.managed", "agents.lifecycle.manage", "events.subscribe"], + resourceLifecycle: ["agent_runtime"], entrypoints: { worker: "worker.js" }, + environmentDrivers: [{ driverKey: "desktop", displayName: "Desktop", skillKeys: ["computer-use"], configSchema: {} }], + skills: [{ skillKey: "computer-use", displayName: "Computer use", slug: "computer-use", + markdown: "---\nname: computer-use\ndescription: Use the installed desktop tools.\n---\n\nRead references/linux.md.", + files: [{ path: "references/linux.md", content: "Use the installed cua-driver." }] }] }; + await db.insert(companies).values({ id: companyId, name: "Environment skills", issuePrefix: `E${companyId.slice(0, 6)}` }); + await db.insert(plugins).values({ id: pluginId, pluginKey: manifest.id, packageName: "fixture", version: "1.0.0", status: "ready", manifestJson: manifest }); + environment = { driver: "plugin", config: { pluginKey: manifest.id, driverKey: "desktop", driverConfig: {} } }; + }); + it("installs the company skill and delivers its versioned files through runtime sync and the native snapshot", async () => { + const config = { paperclipSkillSync: { desiredSkills: [] } }; + const [selection, concurrent] = await Promise.all([environmentSkillConfig(db, companyId, environment, config), environmentSkillConfig(db, companyId, environment, config)]); + expect(concurrent).toEqual(selection); + const selected = selection.config; + const [skill] = await db.select().from(companySkills).where(and(eq(companySkills.companyId, companyId), eq(companySkills.key, `plugin/fixture-${pluginId}/computer-use`))); + expect(selected).toMatchObject({ paperclipSkillSync: { desiredSkills: [skill.key] } }); + expect(config.paperclipSkillSync.desiredSkills).toEqual([]); + const [env] = await db.insert(environments).values({ name: "Desktop", ...environment }).returning(); + const [agent] = await db.insert(agents).values({ companyId, name: "Worker", adapterType: "paperclip_runner", defaultEnvironmentId: env.id, + lifecycleRequiredPluginIds: [pluginId], adapterConfig: config }).returning(); + const state = await pluginAgentRuntimeState(db, pluginId, companyId, agent.id); + expect(state!.skills).toEqual([expect.objectContaining({ key: skill.key, runtimeName: expect.stringMatching(/^plugin-.*-computer-use--/), versionId: expect.any(String) })]); + await withSkillActor({ type: "agent", agentId: agent.id, companyId, source: "agent_key" }, async () => { + await companySkillService(db).withRuntimeSkillEntries(companyId, { selectionConfig: selected, snapshotLatest: true }, async entries => { + const context = await buildNativeRuntimeContext({ db, agent, runId: randomUUID(), runtimeConfig: selected, runtimeSkillEntries: entries }); + expect(context.skills).toEqual([expect.objectContaining({ key: skill.key, runtimeName: state!.skills[0].runtimeName, versionId: state!.skills[0].versionId })]); + expect(await fs.readFile(path.join(context.skills[0].bundle.rootPath, "references/linux.md"), "utf8")).toContain("cua-driver"); + }); + }); + await db.update(agents).set({ defaultEnvironmentId: null }).where(eq(agents.id, agent.id)); + await instanceSettingsService(db).update({ defaultEnvironmentId: env.id }); + expect((await pluginAgentRuntimeState(db, pluginId, companyId, agent.id))!.skills).toEqual(state!.skills); + await instanceSettingsService(db).update({ defaultEnvironmentId: null }); + expect((await pluginAgentRuntimeState(db, pluginId, companyId, agent.id))!.skills).toEqual([]); + expect((await pluginAgentRuntimeState(db, pluginId, companyId, agent.id, env.id))!.skills).toEqual(state!.skills); + await expect(pluginAgentRuntimeState(db, pluginId, companyId, agent.id, randomUUID())).rejects.toMatchObject({ status: 422 }); + const invalidSkill = { skillKey: "new-invalid", displayName: "New invalid skill", markdown: "---\nname: new-invalid\nmetadata:\n sources:\n - kind: github-dir\n repo: fixture/skills\n---\nUnpinned." }; + const changedManifest = { ...manifest, skills: [...manifest.skills!, invalidSkill], environmentDrivers: manifest.environmentDrivers!.map(driver => ({ ...driver, skillKeys: [...driver.skillKeys!, "new-invalid"] })) }; + await db.update(plugins).set({ manifestJson: changedManifest }).where(eq(plugins.id, pluginId)); + expect((await pluginAgentRuntimeState(db, pluginId, companyId, agent.id, env.id))!.skills).toEqual(state!.skills); + await expect(environmentSkillConfig(db, companyId, environment, config)).rejects.toThrow(/pinned Git commit/); + await db.update(plugins).set({ manifestJson: manifest }).where(eq(plugins.id, pluginId)); + const pinned = { paperclipSkillSync: { desiredSkills: [{ key: skill.key, versionId: state!.skills[0].versionId }] } }; + expect((await environmentSkillConfig(db, companyId, environment, pinned)).config).toEqual(pinned); + await db.update(companySkills).set({ markdown: "Agent customization" }).where(eq(companySkills.id, skill.id)); + await environmentSkillConfig(db, companyId, environment, config); + expect((await db.select().from(companySkills).where(eq(companySkills.id, skill.id)))[0].markdown).toBe("Agent customization"); + }); + it("installs and refreshes environment skills through lifecycle reconciliation before any run", async () => { + const [env] = await db.insert(environments).values({ name: randomUUID(), ...environment }).returning(); + const [agent] = await db.insert(agents).values({ companyId, name: "Background worker", adapterType: "paperclip_runner", defaultEnvironmentId: env.id, + lifecycleRequiredPluginIds: [pluginId], adapterConfig: { paperclipSkillSync: { desiredSkills: [] } } }).returning(); + expect((await pluginAgentRuntimeState(db, pluginId, companyId, agent.id))!.skills).toEqual([]); + const states = await Promise.all([1, 2].map(() => pluginAgentRuntimeState(db, pluginId, companyId, agent.id, undefined, true))); + expect(states[0]!.skills).toEqual(states[1]!.skills); + const [skill] = await db.select().from(companySkills).where(and(eq(companySkills.companyId, companyId), eq(companySkills.key, `plugin/fixture-${pluginId}/computer-use`))); + expect(skill.slug).toMatch(/^plugin-.*-computer-use$/); + const runtime = states[0]!.skills[0]; + expect(await fs.readFile(path.join(runtime.source, "SKILL.md"), "utf8")).toContain(`name: "${skill.slug}"`); + const events = () => db.select().from(resourceLifecycleEvents).where(and(eq(resourceLifecycleEvents.companyId, companyId), eq(resourceLifecycleEvents.resourceType, "agent_runtime"))); + const before = (await events()).length; + const edited = skill.markdown + "\nCompany edit."; + await fs.chmod(path.join(skill.sourceLocator!, "SKILL.md"), 0o600); + await fs.writeFile(path.join(skill.sourceLocator!, "SKILL.md"), edited); + await db.update(companySkills).set({ markdown: edited }).where(eq(companySkills.id, skill.id)); + expect((await events()).slice(before).map(event => event.resourceId)).toEqual([agent.id]); + const refreshed = (await pluginAgentRuntimeState(db, pluginId, companyId, agent.id, undefined, true))!.skills[0]; + expect(refreshed.versionId).not.toBe(runtime.versionId); + expect(await fs.readFile(path.join(refreshed.source, "SKILL.md"), "utf8")).toContain("Company edit."); + const settled = (await events()).length; + await pluginAgentRuntimeState(db, pluginId, companyId, agent.id, undefined, true); + expect(await events()).toHaveLength(settled); + const changedManifest = { ...manifest, skills: [...manifest.skills!, { skillKey: "extra", displayName: "Extra" }], + environmentDrivers: manifest.environmentDrivers!.map(driver => ({ ...driver, skillKeys: [...driver.skillKeys!, "extra"] })) }; + await db.update(plugins).set({ manifestJson: changedManifest }).where(eq(plugins.id, pluginId)); + expect((await events()).slice(settled).map(event => event.resourceId)).toEqual([agent.id]); + expect((await pluginAgentRuntimeState(db, pluginId, companyId, agent.id, undefined, true))!.skills).toHaveLength(2); + const saved = await db.select().from(agents).where(eq(agents.id, agent.id)); + expect(saved[0].adapterConfig).toEqual(agent.adapterConfig); + await db.delete(companySkills).where(eq(companySkills.id, skill.id)); + expect((await pluginAgentRuntimeState(db, pluginId, companyId, agent.id, undefined, true))!.skills).toHaveLength(2); + }); + it("queues background delivery for inherited defaults and changed environment requirements", async () => { + const [env] = await db.insert(environments).values({ name: randomUUID(), ...environment }).returning(); + const [agent] = await db.insert(agents).values({ companyId, name: "Inherited worker", adapterType: "paperclip_runner", lifecycleRequiredPluginIds: [pluginId] }).returning(); + const events = () => db.select().from(resourceLifecycleEvents).where(and(eq(resourceLifecycleEvents.companyId, companyId), eq(resourceLifecycleEvents.resourceType, "agent_runtime"))); + let before = (await events()).length; + await instanceSettingsService(db).update({ defaultEnvironmentId: env.id }); + expect((await events()).slice(before).map(event => event.resourceId)).toEqual([agent.id]); + await pluginAgentRuntimeState(db, pluginId, companyId, agent.id, undefined, true); + before = (await events()).length; + const [settings] = await db.insert(pluginCompanySettings).values({ companyId, pluginId, enabled: true }).returning(); + for (let change = 0; change < 3; change++) { + await db.update(pluginCompanySettings).set({ settingsJson: { localFolders: { path: `fixture-${change}` } } }).where(eq(pluginCompanySettings.id, settings.id)); + } + expect(await events()).toHaveLength(before); + await db.update(pluginCompanySettings).set({ enabled: false }).where(eq(pluginCompanySettings.id, settings.id)); + expect((await events()).slice(before).map(event => event.resourceId)).toEqual([agent.id]); + await db.update(pluginCompanySettings).set({ enabled: true }).where(eq(pluginCompanySettings.id, settings.id)); + expect((await events()).slice(before).map(event => event.resourceId)).toEqual([agent.id, agent.id]); + before = (await events()).length; + await db.delete(pluginCompanySettings).where(eq(pluginCompanySettings.id, settings.id)); + expect(await events()).toHaveLength(before); + await expect(db.transaction(async tx => { + await tx.update(environments).set({ config: { ...environment.config, driverKey: "unrelated" } }).where(eq(environments.id, env.id)); + throw new Error("rollback"); + })).rejects.toThrow("rollback"); + expect(await events()).toHaveLength(before); + await db.update(environments).set({ config: { ...environment.config, driverKey: "unrelated" } }).where(eq(environments.id, env.id)); + expect((await events()).slice(before).map(event => event.resourceId)).toEqual([agent.id]); + expect((await pluginAgentRuntimeState(db, pluginId, companyId, agent.id, undefined, true))!.skills).toEqual([]); + before = (await events()).length; + await instanceSettingsService(db).update({ defaultEnvironmentId: null }); + expect((await events()).slice(before).map(event => event.resourceId)).toEqual([agent.id]); + }); + it("delivers the required plugin skill and reports a same-name company assignment as shadowed", async () => { + const name = `plugin-${pluginId}-computer-use`; + const original = await companySkillService(db).createLocalSkill(companyId, { name, markdown: `---\nname: ${name}\n---\nCompany instructions.\n` }); + const config = { paperclipSkillSync: { desiredSkills: [original.key] } }; + const [env] = await db.insert(environments).values({ name: randomUUID(), ...environment }).returning(); + const [agent] = await db.insert(agents).values({ companyId, name: "Conflicting worker", adapterType: "paperclip_runner", defaultEnvironmentId: env.id, + lifecycleRequiredPluginIds: [pluginId], adapterConfig: config }).returning(); + const state = await pluginAgentRuntimeState(db, pluginId, companyId, agent.id, undefined, true); + const key = `plugin/fixture-${pluginId}/computer-use`; + expect(state!.skills.map(skill => skill.key)).toEqual([key]); + expect(state!.shadowedSkills).toEqual([{ key: original.key, shadowedBy: key }]); + for (const reference of [name, original.key.toUpperCase(), `${name}--${createHash("sha256").update(original.key).digest("hex").slice(0, 10)}`]) { + const aliased = await environmentSkillConfig(db, companyId, environment, { paperclipSkillSync: { desiredSkills: [reference] } }); + expect(aliased.config).toMatchObject({ paperclipSkillSync: { desiredSkills: [key] } }); + expect(aliased.shadowedSkills).toEqual(state!.shadowedSkills); + } + expect((await db.select().from(companySkills).where(eq(companySkills.id, original.id)))[0]).toMatchObject({ markdown: original.markdown, slug: original.slug }); + expect((await db.select().from(agents).where(eq(agents.id, agent.id)))[0].adapterConfig).toEqual(config); + await db.update(agents).set({ defaultEnvironmentId: null }).where(eq(agents.id, agent.id)); + const restored = await pluginAgentRuntimeState(db, pluginId, companyId, agent.id, undefined, true); + expect(restored!.skills.map(skill => skill.key)).toEqual([original.key]); + expect(restored!.shadowedSkills).toBeUndefined(); + }); + it("keeps installations company-scoped and rejects disabled providers", async () => { + const peerId = randomUUID(); + await db.insert(companies).values({ id: peerId, name: "Peer", issuePrefix: `P${peerId.slice(0, 6)}` }); + await environmentSkillConfig(db, companyId, environment, {}); + await environmentSkillConfig(db, peerId, environment, {}); + const peerSkills = await db.select().from(companySkills).where(and(eq(companySkills.companyId, peerId), eq(companySkills.key, `plugin/fixture-${pluginId}/computer-use`))); + expect(peerSkills).toHaveLength(1); + const skills = await db.select().from(companySkills).where(and(eq(companySkills.companyId, companyId), eq(companySkills.key, `plugin/fixture-${pluginId}/computer-use`))); + expect(peerSkills[0].id).not.toBe(skills[0].id); + await db.insert(pluginCompanySettings).values({ companyId, pluginId, enabled: false }); + await expect(environmentSkillConfig(db, companyId, environment, {})).rejects.toMatchObject({ status: 422 }); + expect(await environmentSkillConfig(db, companyId, { driver: "local", config: null }, {})).toEqual({ config: {}, skillKeys: [] }); + expect(await environmentSkillConfig(db, companyId, { ...environment, config: { ...environment.config, driverKey: "unrelated" } }, {})).toEqual({ config: {}, skillKeys: [] }); + }); +}); diff --git a/server/src/__tests__/heartbeat-execution-configuration.test.ts b/server/src/__tests__/heartbeat-execution-configuration.test.ts index bcfee2323b8..0e89a77fae7 100644 --- a/server/src/__tests__/heartbeat-execution-configuration.test.ts +++ b/server/src/__tests__/heartbeat-execution-configuration.test.ts @@ -7,7 +7,7 @@ import { tmpdir } from "node:os"; import path from "node:path"; import { eq } from "drizzle-orm"; import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest"; -import { agents, authUsers, companies, companyMemberships, companySkills, createDb, heartbeatRunEvents, heartbeatRuns, issues, projects } from "@paperclipai/db"; +import { agents, authUsers, companies, companyMemberships, companySkills, createDb, heartbeatRunEvents, heartbeatRuns, issues, projects, plugins, environments } from "@paperclipai/db"; import { companySkillService } from "../services/company-skills.js"; import { withSkillVersionCacheLock } from "../services/skill-version-cache.js"; import { registerServerAdapter, unregisterServerAdapter, type AdapterExecutionContext } from "../adapters/index.js"; @@ -220,24 +220,71 @@ describePostgres("heartbeat execution configuration through adapter dispatch", ( } }); - async function configuration(f: Awaited>, config: Record, snapshot: Record = {}, type = "paperclip_runner", explicitTaskDirectory = false, requestedExecutionWorkspaceMode: HeartbeatExecutionConfigurationInput["policy"]["requestedExecutionWorkspaceMode"] = "shared_workspace") { + async function configuration(f: Awaited>, config: Record, snapshot: Record = {}, type = "paperclip_runner", explicitTaskDirectory = false, requestedExecutionWorkspaceMode: HeartbeatExecutionConfigurationInput["policy"]["requestedExecutionWorkspaceMode"] = "shared_workspace", selectedEnvironmentForConfig: HeartbeatExecutionConfigurationInput["environment"]["selectedEnvironmentForConfig"] = null) { const [agent] = await db.update(agents).set({ adapterType: type, adapterConfig: config }).where(eq(agents.id, f.agentId)).returning(); const [run] = await db.insert(heartbeatRuns).values({ companyId: f.companyId, agentId: f.agentId, issueId: f.issueId, scopeKind: "issue", status: "running", contextSnapshot: snapshot }).returning(); const resolvedInstanceSettings = await instanceSettingsService(db).get(); - return { run, prepare: () => prepareHeartbeatExecutionConfiguration(db, { + const appendRunEvent = vi.fn(); + return { run, appendRunEvent, prepare: () => prepareHeartbeatExecutionConfiguration(db, { run, agent, task: { issueId: f.issueId, context: { ...snapshot, issueId: f.issueId }, responsibleUserId: null, acceptedPlanContinuationWake: false, acceptedPlanWakeRoutingDecision: null, taskProjectContext: null, routineEnvContext: { routineId: null, env: null, responsibleUserId: null } }, config: { sourceWorkspaceConfig: config, aiBinding: undefined }, policy: { trustPreset: { kind: "standard", preset: "standard", boundary: null, sourcePresets: {} }, useIsolatedTaskDirectory: false, explicitTaskDirectory, requestedExecutionWorkspaceMode }, - environment: { selectedEnvironmentId: null, selectedEnvironmentForConfig: null, + environment: { selectedEnvironmentId: selectedEnvironmentForConfig?.id ?? null, selectedEnvironmentForConfig, resolvedInstanceSettings, persistedNativeExecutionInput: null }, - services: { secretsSvc: secretService(db), companySkills: companySkillService(db), appendRunEvent: vi.fn(), finalizeAiConnectionBusyDeferral: vi.fn() }, + services: { secretsSvc: secretService(db), companySkills: companySkillService(db), appendRunEvent, finalizeAiConnectionBusyDeferral: vi.fn() }, onFailureSecretsResolved: () => {}, onManagedAiRuntimeAcquired: () => {}, onIdentityRedactorPrepared: () => {}, }) }; } + it("does not install new environment declarations when recovering a saved skill selection", async () => { + const f = await seed(), pluginId = randomUUID(), pluginKey = `fixture.${randomUUID()}`; + await db.insert(plugins).values({ id: pluginId, pluginKey, packageName: "fixture", version: "1.0.0", status: "ready", + manifestJson: { id: pluginKey, apiVersion: 1, version: "1.0.0", displayName: "Fixture", description: "Fixture", author: "Tests", categories: ["automation"], + capabilities: ["environment.drivers.register", "skills.managed"], entrypoints: { worker: "worker.js" }, + environmentDrivers: [{ driverKey: "tasks", displayName: "Tasks", configSchema: {}, skillKeys: ["new-invalid"] }], + skills: [{ skillKey: "new-invalid", displayName: "New invalid skill", markdown: "---\nname: new-invalid\nmetadata:\n sources:\n - kind: github-dir\n repo: fixture/skills\n---\nUnpinned." }] } }); + const [environment] = await db.insert(environments).values({ name: "Fixture", driver: "plugin", config: { pluginKey, driverKey: "tasks", driverConfig: {} } }).returning(); + const config = { paperclipSkillSync: { desiredSkills: [] } }; + const restoring = await configuration(f, config, { skillVersions: {}, skillSelectionSnapshot: true }, "paperclip_runner", false, "shared_workspace", environment); + expect(await restoring.prepare()).toMatchObject({ prepared: true, runtimeSkillEntries: [] }); + const fresh = await configuration(f, config, {}, "paperclip_runner", false, "shared_workspace", environment); + const rejected = await fresh.prepare().then(() => false, error => { expect(error.message).toMatch(/pinned Git commit/); return true; }); + expect(rejected).toBe(true); + }); + + it("reports a shadowed assignment and captures only the required plugin skill", async () => { + const f = await seed(), pluginId = randomUUID(), pluginKey = `fixture.${randomUUID()}`; + const name = `plugin-${pluginKey.split(".").at(-1)}-desktop`; + const existing = await companySkillService(db).createLocalSkill(f.companyId, { name, markdown: `---\nname: ${name}\n---\nCompany skill.` }); + await db.insert(plugins).values({ id: pluginId, pluginKey, packageName: "fixture", version: "1.0.0", status: "ready", + manifestJson: { id: pluginKey, apiVersion: 1, version: "1.0.0", displayName: "Fixture", description: "Fixture", author: "Tests", categories: ["automation"], + capabilities: ["environment.drivers.register", "skills.managed"], entrypoints: { worker: "worker.js" }, + environmentDrivers: [{ driverKey: "tasks", displayName: "Tasks", configSchema: {}, skillKeys: ["desktop"] }], + skills: [{ skillKey: "desktop", displayName: "Desktop" }] } }); + const [environment] = await db.insert(environments).values({ name: randomUUID(), driver: "plugin", config: { pluginKey, driverKey: "tasks" } }).returning(); + await db.update(issues).set({ description: `Use [company skill](skill://${existing.id}).` }).where(eq(issues.id, f.issueId)); + const config = { paperclipSkillSync: { desiredSkills: [existing.key] } }; + const run = await configuration(f, config, {}, "paperclip_runner", false, "shared_workspace", environment); + const result = await run.prepare(); + expect(result.prepared).toBe(true); + if (!result.prepared) throw new Error("Configuration did not finish"); + const key = `plugin/${pluginKey.replaceAll(".", "-")}/desktop`; + expect(result.runtimeSkillEntries.map(skill => skill.key)).toEqual([key]); + expect(run.appendRunEvent).toHaveBeenCalledWith(expect.anything(), expect.objectContaining({ eventType: "skills.shadowed", level: "warn", + message: expect.stringContaining(existing.key), payload: { shadowedSkills: [{ key: existing.key, shadowedBy: key }] } })); + const [saved] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, run.run.id)); + expect(saved.contextSnapshot!.shadowedSkills).toEqual([{ key: existing.key, shadowedBy: key }]); + expect(Object.keys(saved.contextSnapshot!.skillVersions as object)).toEqual([key]); + const recovery = await configuration(f, config, saved.contextSnapshot!, "paperclip_runner", false, "shared_workspace", environment); + const resumed = await recovery.prepare(); + expect(resumed.prepared).toBe(true); + if (!resumed.prepared) throw new Error("Recovery did not finish"); + expect(resumed.runtimeSkillEntries.map(skill => skill.key)).toEqual([key]); + expect(recovery.appendRunEvent.mock.calls.some(([, event]) => event.eventType === "skills.shadowed")).toBe(false); + }); it("keeps default task-directory configuration stable when the directory is reused", async () => { const f = await seed(); const initial = await (await configuration(f, {}, {}, "paperclip_runner", true)).prepare(); diff --git a/server/src/__tests__/plugin-managed-skills.test.ts b/server/src/__tests__/plugin-managed-skills.test.ts index deb82643ed2..954d6cb182f 100644 --- a/server/src/__tests__/plugin-managed-skills.test.ts +++ b/server/src/__tests__/plugin-managed-skills.test.ts @@ -14,6 +14,7 @@ import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase, } from "./helpers/embedded-postgres.js"; +import { companySkillService } from "../services/company-skills.js"; import { buildHostServices } from "../services/plugin-host-services.js"; const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport(); @@ -119,7 +120,8 @@ describeEmbeddedPostgres("plugin-managed skills", () => { expect(created.status).toBe("created"); expect(created.skill).toMatchObject({ - name: "Wiki Maintainer Skill", + name: "plugin-managed-skills-test-wiki-maintainer", + slug: "plugin-managed-skills-test-wiki-maintainer", key: "plugin/paperclip-managed-skills-test/wiki-maintainer", sourceType: "catalog", fileInventory: expect.arrayContaining([ @@ -144,6 +146,35 @@ describeEmbeddedPostgres("plugin-managed skills", () => { }); }); + it("keeps a same-slug user skill separate through automatic install, reset, and relinking", async () => { + const { companyId, services } = await seedCompanyAndPlugin(); + const original = await companySkillService(db).createLocalSkill(companyId, { name: "Wiki Maintainer Skill", markdown: "# User instructions\n" }); + const created = await services.skills.managedReconcile({ companyId, skillKey: "wiki-maintainer" }); + expect(created.skillId).not.toBe(original.id); + expect(created.skill!.key).toBe("plugin/paperclip-managed-skills-test/wiki-maintainer"); + expect(created.skill!.slug).toBe("plugin-managed-skills-test-wiki-maintainer"); + expect(created.skill!.markdown).toContain('name: "plugin-managed-skills-test-wiki-maintainer"'); + await services.skills.managedReset({ companyId, skillKey: "wiki-maintainer" }); + await db.delete(pluginManagedResources).where(eq(pluginManagedResources.resourceId, created.skillId!)); + const relinked = await services.skills.managedReconcile({ companyId, skillKey: "wiki-maintainer" }); + expect(relinked.skillId).toBe(created.skillId); + expect((await db.select().from(companySkills).where(eq(companySkills.id, original.id)))[0]).toMatchObject({ key: original.key, slug: original.slug, markdown: original.markdown }); + }); + + it("claims the plugin name while preserving a conflicting company skill as a separate record", async () => { + const { companyId, services } = await seedCompanyAndPlugin(); + const original = await companySkillService(db).createLocalSkill(companyId, { + name: "plugin-managed-skills-test-wiki-maintainer", markdown: "# User instructions\n", + }); + const installed = await services.skills.managedReconcile({ companyId, skillKey: "wiki-maintainer" }); + expect(installed.skill!.slug).toBe(original.slug); + expect(installed.skill!.id).not.toBe(original.id); + await services.skills.managedReset({ companyId, skillKey: "wiki-maintainer" }); + const records = (await db.select().from(companySkills)).filter(skill => [original.key, installed.skill!.key].includes(skill.key)); + expect(records).toHaveLength(2); + expect(records.find(skill => skill.id === original.id)).toMatchObject({ key: original.key, markdown: original.markdown }); + expect(await db.select().from(pluginManagedResources)).toEqual([expect.objectContaining({ resourceId: installed.skill!.id })]); + }); it("preserves operator edits during reconcile and restores manifest defaults on reset", async () => { const { companyId, services } = await seedCompanyAndPlugin(); const created = await services.skills.managedReconcile({ companyId, skillKey: "wiki-maintainer" }); @@ -168,7 +199,7 @@ describeEmbeddedPostgres("plugin-managed skills", () => { const reset = await services.skills.managedReset({ companyId, skillKey: "wiki-maintainer" }); expect(reset.status).toBe("reset"); expect(reset.skill).toMatchObject({ - name: "Wiki Maintainer Skill", + name: "plugin-managed-skills-test-wiki-maintainer", description: "Use LLM Wiki tools to maintain company knowledge.", }); expect(reset.skill?.markdown).toContain("key: \"plugin/paperclip-managed-skills-test/wiki-maintainer\""); @@ -248,6 +279,18 @@ describeEmbeddedPostgres("plugin-managed skills", () => { expect(reset.defaultDrift).toBeNull(); }); + it("preserves unrelated YAML while replacing quoted and multiline identity fields", async () => { + const declaration = manifest(); + declaration.skills = [{ skillKey: "yaml-skill", displayName: "YAML skill", markdown: + '---\n"name": |\n Previous name\nslug: old-slug\ndescription: >\n Keep this folded description.\nmetadata:\n "category": desktop\n# Keep this comment.\n---\nBody stays unchanged.\n' }]; + const { companyId, services } = await seedCompanyAndPlugin(declaration); + const created = await services.skills.managedReconcile({ companyId, skillKey: "yaml-skill" }); + expect(created.skill!.markdown).toContain('name: "plugin-managed-skills-test-yaml-skill"'); + expect(created.skill!.markdown).not.toContain("Previous name"); + expect(created.skill!.markdown).toContain('description: >\n Keep this folded description.\nmetadata:\n "category": desktop\n# Keep this comment.'); + expect(created.skill!.markdown).toMatch(/Body stays unchanged\.\n$/); + await services.skills.managedReset({ companyId, skillKey: "yaml-skill" }); + }); it("adds the canonical managed key to manifest-provided markdown skills", async () => { const pluginManifest = manifest(); pluginManifest.skills = [ @@ -274,8 +317,8 @@ describeEmbeddedPostgres("plugin-managed skills", () => { expect(created.status).toBe("created"); expect(created.skill).toMatchObject({ key: "plugin/paperclip-managed-skills-test/markdown-skill", - name: "markdown-skill", + name: "plugin-managed-skills-test-markdown-skill", }); - expect(created.skill?.markdown).toContain("key: \"plugin/paperclip-managed-skills-test/markdown-skill\""); + expect(created.skill?.markdown).toContain('key: "plugin/paperclip-managed-skills-test/markdown-skill"'); }); }); diff --git a/server/src/services/company-skills.ts b/server/src/services/company-skills.ts index f29f4ece222..af1b4e063af 100644 --- a/server/src/services/company-skills.ts +++ b/server/src/services/company-skills.ts @@ -601,7 +601,7 @@ function uniqueImportedSkillKey(companyId: string, baseSlug: string, usedKeys: S return candidate; } -function buildSkillRuntimeName(key: string, slug: string) { +export function buildSkillRuntimeName(key: string, slug: string) { if (key.startsWith("paperclipai/paperclip/")) return slug; return `${slug}--${hashSkillValue(key)}`; } @@ -6238,6 +6238,8 @@ export function companySkillService(db: Db, options: { actor?: AuthorizationActo files: Record, options?: { onConflict?: PackageSkillConflictStrategy; + /** Managed plugin namespaces may share a slug while retaining separate canonical records. */ + allowSlugCollision?: boolean; }, ): Promise { await ensureSkillInventoryCurrent(companyId); @@ -6286,7 +6288,7 @@ export function companySkillService(db: Db, options: { actor?: AuthorizationActo const normalizedSlug = normalizeSkillSlug(importedSkill.slug) ?? importedSkill.slug; const existingByIncomingKey = existingByKey.get(importedSkill.key) ?? null; const existingByIncomingSlug = existingBySlug.get(normalizedSlug) ?? null; - const conflict = existingByIncomingKey ?? existingByIncomingSlug; + const conflict = existingByIncomingKey ?? (options?.allowSlugCollision ? null : existingByIncomingSlug); if (!conflict || conflictStrategy === "replace") { const skillToPersist = conflict && conflict.key !== importedSkill.key diff --git a/server/src/services/environment-skills.ts b/server/src/services/environment-skills.ts new file mode 100644 index 00000000000..1f922c8a5c1 --- /dev/null +++ b/server/src/services/environment-skills.ts @@ -0,0 +1,64 @@ +import { and, eq, inArray, or, sql } from "drizzle-orm"; +import { companySkills, pluginCompanySettings, pluginManagedResources, plugins, type Db } from "@paperclipai/db"; +import { readPaperclipSkillSyncPreference, resolvePaperclipDesiredSkillNames, writePaperclipSkillSyncPreference } from "@paperclipai/adapter-utils/server-utils"; +import { unprocessable } from "../errors.js"; +import { buildSkillRuntimeName } from "./company-skills.js"; +import { pluginManagedSkillService } from "./plugin-managed-skills.js"; + +/** Environment capabilities join the normal authorized, versioned skill selection. */ +export async function environmentSkillConfig(db: Db, companyId: string, + environment: { driver: string; config: Record | null } | null | undefined, + config: Record, installMissing = true): Promise<{ config: Record; skillKeys: string[]; shadowedSkills?: Array<{ key: string; shadowedBy: string }> }> { + if (environment?.driver !== "plugin") return { config, skillKeys: [] }; + const { pluginKey, driverKey } = environment.config ?? {}; + if (typeof pluginKey !== "string" || typeof driverKey !== "string") return { config, skillKeys: [] }; + const [plugin] = await db.select().from(plugins).where(eq(plugins.pluginKey, pluginKey)); + const keys = plugin?.manifestJson.environmentDrivers?.find(driver => driver.driverKey === driverKey)?.skillKeys ?? []; + if (!plugin || !keys.length) return { config, skillKeys: [] }; + const [settings] = await db.select().from(pluginCompanySettings).where(and( + eq(pluginCompanySettings.companyId, companyId), eq(pluginCompanySettings.pluginId, plugin.id))); + if (plugin.status !== "ready" || settings?.enabled === false || !plugin.manifestJson.capabilities.includes("skills.managed")) { + throw unprocessable("Environment skills require an enabled plugin with skills.managed capability"); + } + // Existing bindings need only a bounded metadata query, not a file-by-file drift scan. + const installed = await db.select({ resourceKey: pluginManagedResources.resourceKey, key: companySkills.key }) + .from(pluginManagedResources).innerJoin(companySkills, and( + eq(companySkills.id, pluginManagedResources.resourceId), eq(companySkills.companyId, companyId))) + .where(and(eq(pluginManagedResources.companyId, companyId), eq(pluginManagedResources.pluginId, plugin.id), + eq(pluginManagedResources.resourceKind, "skill"), inArray(pluginManagedResources.resourceKey, keys))); + const selected = new Map(installed.map(skill => [skill.resourceKey, skill.key])); + for (const key of keys) { + if (selected.has(key) || !installMissing) continue; + const result = await db.transaction(async tx => { + // Serialize first installation across concurrent agents in this company. + await tx.execute(sql`select pg_advisory_xact_lock(hashtextextended(${`environment-skill:${companyId}:${plugin.id}:${key}`}, 0))`); + return pluginManagedSkillService(tx as unknown as Db, { pluginId: plugin.id, pluginKey, manifest: plugin.manifestJson }).reconcile(key, companyId); + }); + if (!result.skill) throw unprocessable("Environment skill is unavailable"); + selected.set(key, result.skill.key); + } + const preference = readPaperclipSkillSyncPreference(config); + const explicit = new Set(preference.desiredSkillEntries.map(entry => entry.key)); + const skillKeys = keys.flatMap(key => selected.has(key) ? [selected.get(key)!] : []); + const candidates = await db.select({ key: companySkills.key, slug: companySkills.slug, name: companySkills.name }) + .from(companySkills).where(and(eq(companySkills.companyId, companyId), + or(inArray(sql`lower(${companySkills.key})`, [...new Set([...explicit, ...skillKeys])].map(key => key.toLowerCase())), + inArray(sql`lower(${companySkills.slug})`, [...explicit].map(key => key.toLowerCase().split("/").at(-1)!.replace(/--[a-f0-9]{10}$/, "")))))); + const names = candidates.map(skill => ({ key: skill.key, runtimeName: buildSkillRuntimeName(skill.key, skill.slug) })); + const entries = preference.desiredSkillEntries.map(entry => ({ ...entry, + key: resolvePaperclipDesiredSkillNames(writePaperclipSkillSyncPreference({}, [entry]), names)[0] ?? entry.key, + })); + const selectedKeys = new Set(entries.map(entry => entry.key)); + const required = candidates.filter(skill => skillKeys.includes(skill.key)); + const shadowedSkills = candidates.flatMap(skill => { + if (skillKeys.includes(skill.key) || !selectedKeys.has(skill.key)) return []; + const winner = required.find(candidate => candidate.slug === skill.slug || candidate.name === skill.name); + return winner ? [{ key: skill.key, shadowedBy: winner.key }] : []; + }); + const shadowed = new Set(shadowedSkills.map(skill => skill.key)); + return { config: writePaperclipSkillSyncPreference(config, [ + ...entries.filter(entry => !shadowed.has(entry.key)), + ...skillKeys.filter(key => !selectedKeys.has(key))]), skillKeys, + ...(shadowedSkills.length ? { shadowedSkills } : {}), + }; +} diff --git a/server/src/services/heartbeat/execution-configuration.ts b/server/src/services/heartbeat/execution-configuration.ts index 2fb2a0ef832..ec531455fed 100644 --- a/server/src/services/heartbeat/execution-configuration.ts +++ b/server/src/services/heartbeat/execution-configuration.ts @@ -26,6 +26,7 @@ import { resolveAssignedConnectionInstructionsForRun } from "../native-runtime/a import { recordFailedSkillPreparation, type NativeRunHistoricalSpan } from "../native-runtime/native-run-trace.js"; import { collectRunFailureSecretValues } from "../run-failure-diagnostics.js"; import { skillVersionSelectionMap } from "../runtime-skill-selections.js"; +import { environmentSkillConfig } from "../environment-skills.js"; import type { selectHeartbeatEnvironment } from "./environment-selection.js"; import type { createHeartbeatRetries } from "./retries.js"; import type { createHeartbeatLifecycle } from "./run-lifecycle.js"; @@ -231,18 +232,30 @@ export async function prepareHeartbeatExecutionConfiguration(db: Db, input: Hear } else { delete context.paperclipSecrets; } - const effectiveResolvedConfig = applyRunScopedMentionedSkillKeys( - resolvedConfig, - runScopedSkillKeys, - ); + const storedVersions = parseObject(run.contextSnapshot?.skillVersions); + const hasStoredVersions = Object.prototype.hasOwnProperty.call(run.contextSnapshot ?? {}, "skillVersions"); + const savedShadowedSkills = Array.isArray(run.contextSnapshot?.shadowedSkills) + ? run.contextSnapshot.shadowedSkills.flatMap(value => { + const saved = parseObject(value); + return typeof saved.key === "string" && typeof saved.shadowedBy === "string" && Object.hasOwn(storedVersions, saved.shadowedBy) + ? [{ key: saved.key, shadowedBy: saved.shadowedBy }] : []; + }) : []; + const selectedConfig = applyRunScopedMentionedSkillKeys(resolvedConfig, runScopedSkillKeys); + const { config: effectiveResolvedConfig, skillKeys: environmentSkills, shadowedSkills } = hasStoredVersions + ? { config: selectedConfig, skillKeys: [], shadowedSkills: savedShadowedSkills } + : await environmentSkillConfig(db, agent.companyId, selectedEnvironmentForConfig, selectedConfig); + if (shadowedSkills?.length) { + context.shadowedSkills = shadowedSkills; + if (!hasStoredVersions) await appendRunEvent(run, { eventType: "skills.shadowed", stream: "system", level: "warn", + message: shadowedSkills.map(({ key, shadowedBy }) => `Skill "${key}" was skipped because required plugin skill "${shadowedBy}" uses the same name.`).join("\n"), + payload: { shadowedSkills } }); + } const runtimeSkillPreference = readPaperclipSkillSyncPreference( effectiveResolvedConfig, ); const nativeRunnerPreparationSpans: NativeRunHistoricalSpan[] = []; const skillsPrepareStartedAtMs = Date.now(); let skillDeliveryConfig = effectiveResolvedConfig; - const storedVersions = parseObject(run.contextSnapshot?.skillVersions); - const hasStoredVersions = Object.prototype.hasOwnProperty.call(run.contextSnapshot ?? {}, "skillVersions"); const prepareSkills = async () => { try { @@ -270,7 +283,7 @@ export async function prepareHeartbeatExecutionConfiguration(db: Db, input: Hear // assignments from delivery; explicit task mentions remain errors below. skillDeliveryConfig = writePaperclipSkillSyncPreference(effectiveResolvedConfig, resolved.map(entry => ({ key: entry.key, versionId: entry.versionId ?? null }))); - for (const key of runScopedMentionedSkillKeys) { + for (const key of [...runScopedMentionedSkillKeys.filter(key => !shadowedSkills?.some(skill => skill.key === key)), ...(hasStoredVersions ? [] : environmentSkills)]) { const entry = resolved.find(candidate => candidate.key === key); if (!entry || entry.sourceStatus === "missing") { throw new ConfigurationIncompleteFailure("A referenced skill is unavailable. Restore its installed version or update the task's skill reference.", { @@ -282,7 +295,7 @@ export async function prepareHeartbeatExecutionConfiguration(db: Db, input: Hear const skillVersions = Object.fromEntries(resolved.map(entry => [entry.key, entry.versionId ?? entry.currentVersionId ?? null])); context.skillVersions = skillVersions; context.skillSelectionSnapshot = true; - await db.update(heartbeatRuns).set({ contextSnapshot: sql`coalesce(${heartbeatRuns.contextSnapshot}, '{}'::jsonb) || ${JSON.stringify({ skillVersions, skillSelectionSnapshot: true })}::jsonb` }).where(eq(heartbeatRuns.id, run.id)); + await db.update(heartbeatRuns).set({ contextSnapshot: sql`coalesce(${heartbeatRuns.contextSnapshot}, '{}'::jsonb) || ${JSON.stringify({ skillVersions, skillSelectionSnapshot: true, ...(shadowedSkills?.length ? { shadowedSkills } : {}) })}::jsonb` }).where(eq(heartbeatRuns.id, run.id)); } return resolved; })); diff --git a/server/src/services/plugin-agent-runtime-state.ts b/server/src/services/plugin-agent-runtime-state.ts index 0770246ef14..a92efeb4697 100644 --- a/server/src/services/plugin-agent-runtime-state.ts +++ b/server/src/services/plugin-agent-runtime-state.ts @@ -1,8 +1,8 @@ import { and, desc, eq } from "drizzle-orm"; -import { agents, plugins, pluginCompanySettings, resourceLifecycleEvents, type Db } from "@paperclipai/db"; +import { agents, environments, plugins, pluginCompanySettings, resourceLifecycleEvents, type Db } from "@paperclipai/db"; import type { PluginAgentRuntimeState, PluginAgentRuntimeIdentity } from "@paperclipai/plugin-sdk"; import { agentRecordQueries } from "../lib/agent-records.js"; -import { conflict, forbidden } from "../errors.js"; +import { conflict, forbidden, unprocessable } from "../errors.js"; import { agentInstructionsService } from "./agent-instructions.js"; import { pluginAgentConnectionAssignments } from "./plugin-agent-connections.js"; import { companySkillService } from "./company-skills.js"; @@ -11,6 +11,7 @@ import { readPaperclipSkillSyncPreference } from "@paperclipai/adapter-utils/ser import { skillVersionSelectionMap } from "./runtime-skill-selections.js"; import { instanceSettingsService } from "./instance-settings.js"; import { agentInstanceSecretService } from "./agent-instance-secrets.js"; +import { environmentSkillConfig } from "./environment-skills.js"; async function readState(db: Db, pluginId: string, companyId: string, agentId: string, identityOnly: true): Promise; async function readState(db: Db, pluginId: string, companyId: string, agentId: string, identityOnly?: false): Promise | null>; @@ -50,24 +51,33 @@ export async function pluginAgentRuntimeIdentity(db: Db, pluginId: string, compa return readState(db, pluginId, companyId, agentId, true); } -export async function pluginAgentRuntimeState(db: Db, pluginId: string, companyId: string, agentId: string): Promise { +export async function pluginAgentRuntimeState(db: Db, pluginId: string, companyId: string, agentId: string, environmentId?: string, installEnvironmentSkills = false): Promise { for (let attempt = 0; attempt < 3; attempt++) { const state = await readState(db, pluginId, companyId, agentId); if (!state) return null; - const preference = readPaperclipSkillSyncPreference(state.agent.adapterConfig); - const settings = await instanceSettingsService(db).getExperimental(); + const instance = await instanceSettingsService(db).get(); + const effectiveEnvironmentId = environmentId ?? state.agent.defaultEnvironmentId ?? instance.defaultEnvironmentId; + const [environment] = effectiveEnvironmentId + ? await db.select().from(environments).where(eq(environments.id, effectiveEnvironmentId)) : []; + if (environmentId && !environment) throw unprocessable("The selected environment is unavailable"); + const { config: selectionConfig, skillKeys: environmentSkills, shadowedSkills } = await environmentSkillConfig(db, companyId, environment, state.agent.adapterConfig, installEnvironmentSkills); + const preference = readPaperclipSkillSyncPreference(selectionConfig); + const settings = instance.experimental; const skills = await withSkillActor({ type: "agent", agentId, companyId, source: "agent_key" }, () => companySkillService(db).listRuntimeSkillEntries(companyId, { - selectionConfig: state.agent.adapterConfig, includeLegacy: state.agent.adapterType !== "paperclip_runner", snapshotLatest: true, + selectionConfig, includeLegacy: state.agent.adapterType !== "paperclip_runner", snapshotLatest: true, versionSelections: skillVersionSelectionMap(preference.desiredSkillEntries, { versionPinsEnabled: settings.enableBetaSkills }), })); + if (environmentSkills.some(key => !skills.some(skill => skill.key === key && skill.sourceStatus !== "missing"))) { + throw unprocessable("An environment skill is unavailable to this agent"); + } const connectionAssignments = await pluginAgentConnectionAssignments(db, companyId, agentId, { publicMcp: settings.enablePublicMcp, chatConnectors: settings.enableChatConnectors }); const fallback = !state.instructionsRootPath ? await agentInstructionsService().exportFiles(state.agent, { rejectSymlinks: true }) : null; const entryFile = fallback?.entryFile ?? state.instructionsEntryFile; const entryContent = fallback?.files[entryFile]; const current = await readState(db, pluginId, companyId, agentId); if (!current) return null; - if (current.revision === state.revision) return { ...current, instructionsEntryFile: entryFile, ...(entryContent === undefined ? {} : { instructionsEntryContent: entryContent }), skills, connectionAssignments }; + if (current.revision === state.revision) return { ...current, instructionsEntryFile: entryFile, ...(entryContent === undefined ? {} : { instructionsEntryContent: entryContent }), skills, connectionAssignments, ...(shadowedSkills?.length ? { shadowedSkills } : {}) }; } throw conflict("Agent runtime changed while resolving synchronization inputs; retry"); } diff --git a/server/src/services/plugin-host-services.ts b/server/src/services/plugin-host-services.ts index bab31883a7a..a670eaa6d22 100644 --- a/server/src/services/plugin-host-services.ts +++ b/server/src/services/plugin-host-services.ts @@ -3010,7 +3010,7 @@ export function buildHostServices( async getRuntime(params) { const companyId = ensureCompanyId(params.companyId); await ensurePluginAvailableForCompany(companyId); - return pluginAgentRuntimeState(db, pluginId, companyId, params.agentId); + return pluginAgentRuntimeState(db, pluginId, companyId, params.agentId, params.environmentId, params.installEnvironmentSkills); }, async list(params) { const companyId = ensureCompanyId(params.companyId); diff --git a/server/src/services/plugin-managed-skills.ts b/server/src/services/plugin-managed-skills.ts index 0d6ca9c3549..021446e737a 100644 --- a/server/src/services/plugin-managed-skills.ts +++ b/server/src/services/plugin-managed-skills.ts @@ -3,7 +3,7 @@ import type { Db } from "@paperclipai/db"; import { pluginManagedResources, } from "@paperclipai/db"; -import { normalizeAgentUrlKey } from "@paperclipai/shared"; +import { joinFrontmatterBlock, normalizeAgentUrlKey, splitFrontmatterBlock } from "@paperclipai/shared"; import type { CompanySkill, PaperclipPluginManifestV1, @@ -30,6 +30,11 @@ function canonicalSkillKey(pluginKey: string, skillKey: string) { return `plugin/${pluginKeySlug(pluginKey)}/${skillKey}`; } +function managedSkillName(pluginKey: string, declaration: PluginManagedSkillDeclaration) { + const shortPlugin = pluginKeySlug(pluginKey.split(".").at(-1)!).replace(/^plugin-/, ""); + return `plugin-${shortPlugin}-${normalizeAgentUrlKey(declaration.slug ?? declaration.skillKey) ?? declaration.skillKey}`; +} + function yamlString(value: string) { return JSON.stringify(value); } @@ -41,7 +46,8 @@ function buildDefaultMarkdown( const description = declaration.description?.trim() || `${declaration.displayName} plugin skill.`; return [ "---", - `name: ${yamlString(declaration.displayName)}`, + `name: ${yamlString(managedSkillName(pluginKey, declaration))}`, + `slug: ${yamlString(managedSkillName(pluginKey, declaration))}`, `description: ${yamlString(description)}`, `key: ${yamlString(canonicalSkillKey(pluginKey, declaration.skillKey))}`, "---", @@ -53,25 +59,24 @@ function buildDefaultMarkdown( ].join("\n"); } -function withManagedSkillKey(markdown: string, canonicalKey: string) { - const keyLine = `key: ${yamlString(canonicalKey)}`; - const normalized = markdown.replace(/\r\n/g, "\n"); - const frontmatter = /^---\n([\s\S]*?)\n---(\n?)/.exec(normalized); - if (!frontmatter) { - return [ - "---", - keyLine, - "---", - "", - normalized, - ].join("\n"); - } - - const currentBody = frontmatter[1] ?? ""; - const nextBody = /^key\s*:/m.test(currentBody) - ? currentBody.replace(/^key\s*:.*$/m, keyLine) - : [currentBody, keyLine].filter(Boolean).join("\n"); - return `---\n${nextBody}\n---${frontmatter[2] ?? ""}${normalized.slice(frontmatter[0].length)}`; +function withManagedSkillKey(markdown: string, canonicalKey: string, name: string) { + const block = splitFrontmatterBlock(markdown.replace(/\r\n/g, "\n")); + const fields: Record = { key: canonicalKey, name, slug: name }; + const seen = new Set(); + let continuation = false; + const lines = block.frontmatterText.split("\n").flatMap(line => { + const field = /^(key|name|slug|"(?:key|name|slug)"|'(?:key|name|slug)')\s*:/.exec(line); + if (field) { + const key = field[1].replace(/["']/g, ""); + seen.add(key); continuation = true; + return [`${key}: ${yamlString(fields[key])}`]; + } + if (continuation && /^(?:\s|$)/.test(line)) return []; + continuation = false; + return [line]; + }); + for (const [key, value] of Object.entries(fields)) if (!seen.has(key)) lines.push(`${key}: ${yamlString(value)}`); + return joinFrontmatterBlock({ ...block, hasFrontmatter: true, frontmatterText: lines.join("\n") }); } function buildPackageFiles( @@ -82,7 +87,7 @@ function buildPackageFiles( const canonicalKey = canonicalSkillKey(pluginKey, declaration.skillKey); const files: Record = { [`${root}/SKILL.md`]: declaration.markdown?.trim() - ? withManagedSkillKey(declaration.markdown, canonicalKey) + ? withManagedSkillKey(declaration.markdown, canonicalKey, managedSkillName(pluginKey, declaration)) : buildDefaultMarkdown(pluginKey, declaration), }; for (const file of declaration.files ?? []) { @@ -112,7 +117,7 @@ function buildSkillDefaults( return { skillKey: declaration.skillKey, displayName: declaration.displayName, - slug: declaration.slug ?? declaration.skillKey, + slug: managedSkillName(pluginKey, declaration), description: declaration.description ?? null, canonicalKey: canonicalSkillKey(pluginKey, declaration.skillKey), files: [ @@ -286,7 +291,7 @@ export function pluginManagedSkillService( const results = await skills.importPackageFiles( companyId, buildPackageFiles(options.pluginKey, declaration), - { onConflict: "replace" }, + { onConflict: "replace", allowSlugCollision: true }, ); const imported = results.find((result) => result.skill.key === canonicalSkillKey(options.pluginKey, declaration.skillKey)