Synchronize assigned skills through agent runtime hooks (#16501)

## Thinking Path

> - Paperclip manages shared skills and agent assignments.
> - Runtime plugins need installed skills before a task starts.
> - Skill content can change without changing the agent row.
> - Source access and version pins also affect delivered files.
> - This pull request feeds those changes into the durable runtime
stream.
> - The runtime reader resolves the same immutable skill sources used
for execution.

## Linked Issues or Issue Description

Builds on merged PR #16499. This pull request now targets `master` and
contains only the five skill synchronization commits.

**Problem**

Plugins receive no durable notification when an assigned skill changes.
A saved assignment can also become inaccessible after private-source
access changes.

**Proposed solution**

Invalidate referencing agents at installed skill mutations. Invalidate
inherited access when source bindings, project privacy, or policy
change. Return authorized skill sources and version IDs through the
owning runtime reader.

**Alternatives considered**

Activity-only subscriptions miss service writers. Per-task
synchronization adds preparation work. Copying current files would
ignore version pins.

**Roadmap alignment**

This extends existing skills, source synchronization, and plugin runtime
support.

## What Changed

- Defer multi-skill source runtime hooks until commit, after publication
holds all skill rows. Keep file-save journaling immediate to preserve
filesystem rollback.

- Add transactional invalidations for skill content, companion files,
versions, rename, deletion, policy, source access, and pin-setting
changes.
- Recognize canonical keys and supported legacy aliases.
- Reuse the execution skill resolver, current agent access, and instance
version-pin rules.
- Resolve and materialize sources outside the agent row lock. Recheck
ownership and runtime revision afterward.
- Keep task-only mentions and running-task snapshots in the execution
path.
- Document conservative invalidations for pinned assignments and
inherited access.

## Verification

- Rebased onto origin/master `9f4a956b90`, which contains merged PR
#16499. All five commits are unchanged according to `git range-diff`; no
conflict resolutions or source changes were needed.
- On rebased head `58b065941`, all 237 focused tests passed: runtime
skill hooks/source publication (41), plus content service, assignment
routes, policy routes, and project source access (196).
- Server typecheck passed after rebuilding Runner with the installed
Rust toolchain. Migration numbering/safety and map checks passed;
snapshot pruning made no changes. Migration CLI checks required local
IPC permission.
- Real PostgreSQL coverage includes concurrent source
publication/version saves, policy-only tightening/restoration,
referencing-agent fanout, company isolation, no-op counters, supporting
files, rename/deletion, rollback, private-source access, pinned/latest
bytes, and unassignment.
- Earlier stack validation passed workspace typecheck/build. Fresh
hosted CI and review must verify the rebased head before handoff.
- The broad local suite was previously limited by an unchanged
diagnostic test that rejects "private" in an error stack under this
/private/tmp worktree; that unchanged suite passed all 108 tests in the
original checkout. Hosted full-suite checks provide exact-head
verification.
- Diff whitespace and full PR secret/PII scan passed.
- All hosted CI/security checks are green on `58b065941`; fresh standard
Greptile and Apex both returned 5/5 with no findings or unresolved
threads. The PR targets master and is mergeable without conflicts.

## Risks

- Apply migration 0348 after the runtime-stream migration in #16499.
- Content invalidations include pinned assignments. The resolved version
stays pinned; consumers can avoid writes when its version/hash is
unchanged.
- Project access and source changes invalidate only agents assigned
skills from the affected sources. Company policy still invalidates the
company. Source polling bookkeeping and popularity counters do not
create work.
- The reader rebuilds installed caches without upstream requests.
Concurrent changes cause a bounded retry and then a conflict.
- Private transport integration and live remote computer qualification
remain separate.

## Model Used

OpenAI Codex, GPT-6, with reasoning, repository inspection, code
execution, and testing. The exact deployment model ID and context window
are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub references)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id
- [x] I have run tests locally and they pass (focused suites; broad
local limitation documented above)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Nicky LeachandPaperclip authored and GitHub committed 2026-10-16 08:28:47 -07:00
1 parent 9f4a956b90
commit 5fbece9dcc
12 files changed
+5518 -4479

No files matched your search

+29
View File
@@ -65,3 +65,32 @@ revision, so repeated identical saves against that revision create no audit rows
or events. Later runtime changes invalidate that deduplication. The file API still
returns `changed: false` for identical content and keeps conditional-write checks.
Consumers can skip rewriting content whose hash already matches.
## Skills
Migration 0348 invalidates referencing agents when installed skill content,
companion files, versions, source bindings, or identities change. Rename checks
both old and new keys. String selections, structured selections, and supported
legacy aliases are recognized. Unrelated agents and other companies receive no
content invalidation. Private-source bindings, membership, and privacy changes
invalidate only agents assigned skills from the affected sources. Project workspace
authorization-policy changes use the same boundary. Company policy
changes conservatively invalidate the company.
Source retry/lease bookkeeping and skill popularity counters do not emit changes.
Multi-skill source publication defers its skill invalidation until transaction
commit, after it acquires all skill rows. This avoids an agent-lock/skill-lock
cycle with concurrent version saves. Individual file edits retain immediate
journaling and their existing filesystem rollback behavior.
Assignment edits already use the configuration boundary. The runtime reader
returns selected skill entries with host source paths and immutable version IDs.
It uses the execution resolver, current agent access, the instance pin flag, and
legacy operational-skill rules. It performs no upstream refresh. The resolver can
rebuild an expired version cache from saved bytes. Slow materialization happens
outside the agent row lock. The reader rechecks ownership and revision afterward;
it retries concurrent changes three times and then returns a conflict.
Pinned assignments receive conservative content invalidations too. Their resolved
version and bytes stay pinned when pin support is enabled. Consumers can compare
versions or destination hashes and avoid rewriting unchanged trees. Task-only
skill mentions and a running task's saved version selection remain task-owned.
+8
View File
@@ -202,3 +202,11 @@ responsible person's other grants for an agent-created source.
Initial automatic imports reserve company work credits before provider reads.
Failures retain the reservation; completed jobs cannot replenish the queue's
work allowance. Jobs remain durable and resume as company credits refill.
Plugin runtime synchronization uses the agent's current skill access and immutable
version resolver. A change to an installed skill invalidates its referencing
agents. Private-source access changes also request reconciliation. This is a
host-to-plugin hook, not a new agent-facing endpoint. See
[contract](../../doc/plugins/AGENT_RUNTIME_SYNC.md#skills) and
[database coverage](../../server/src/__tests__/agent-runtime-skills.test.ts).
+6
View File
@@ -121,3 +121,9 @@ Sources: Source: [SkillStudio.tsx](../../ui/src/pages/SkillStudio.tsx), [company
### Documentation scope: Skills, Skill Studio, and teams
Main library/Studio/source/policy and CLI team-install paths are mapped. Full source-provider conflicts, harness sync, and remote package variants remain; catalog UI is not a current top-level route.
Installed skill updates and assignments can drive opt-in plugin runtime
synchronization. Version pins remain effective. See
[contract](../../doc/plugins/AGENT_RUNTIME_SYNC.md#skills) and
[database coverage](../../server/src/__tests__/agent-runtime-skills.test.ts).
@@ -0,0 +1,166 @@
CREATE OR REPLACE FUNCTION paperclip_invalidate_company_agent_runtimes(p_company uuid) RETURNS void
LANGUAGE plpgsql SET search_path = public, pg_temp AS $$
DECLARE candidate uuid;
BEGIN
FOR candidate IN SELECT id FROM agents WHERE company_id = p_company
AND status NOT IN ('pending_approval', 'terminated') ORDER BY id FOR UPDATE LOOP
PERFORM paperclip_record_agent_runtime(p_company, candidate);
END LOOP;
END;
$$;
--> statement-breakpoint
CREATE OR REPLACE FUNCTION paperclip_company_agent_runtimes_changed() RETURNS trigger
LANGUAGE plpgsql SET search_path = public, pg_temp AS $$
DECLARE column_name text; changed boolean := false;
BEGIN
IF TG_OP = 'UPDATE' AND TG_NARGS > 0 THEN
FOREACH column_name IN ARRAY TG_ARGV LOOP
IF to_jsonb(OLD)->column_name IS DISTINCT FROM to_jsonb(NEW)->column_name THEN changed := true; END IF;
END LOOP;
IF NOT changed THEN RETURN NEW; END IF;
END IF;
IF TG_OP = 'UPDATE' AND (to_jsonb(OLD) - ARRAY['updated_at', 'created_at']) IS NOT DISTINCT FROM
(to_jsonb(NEW) - ARRAY['updated_at', 'created_at']) THEN RETURN NEW; END IF;
IF TG_OP <> 'INSERT' THEN PERFORM paperclip_invalidate_company_agent_runtimes(OLD.company_id); END IF;
IF TG_OP = 'INSERT' OR (TG_OP = 'UPDATE' AND NEW.company_id IS DISTINCT FROM OLD.company_id) THEN
PERFORM paperclip_invalidate_company_agent_runtimes(NEW.company_id);
END IF;
IF TG_OP = 'DELETE' THEN RETURN OLD; END IF;
RETURN NEW;
END;
$$;
--> statement-breakpoint
CREATE OR REPLACE FUNCTION paperclip_agent_selects_runtime_skill(config jsonb, adapter text, skill_key text, skill_slug text) RETURNS boolean
LANGUAGE sql IMMUTABLE SET search_path = public, pg_temp AS $$
SELECT (adapter <> 'paperclip_runner' AND skill_key IN
('paperclipai/paperclip/paperclip', 'paperclipai/paperclip/complain', 'paperclipai/paperclip/suggestion-box')) OR
EXISTS (SELECT 1 FROM jsonb_array_elements(CASE WHEN jsonb_typeof(config#>'{paperclipSkillSync,desiredSkills}') = 'array'
THEN config#>'{paperclipSkillSync,desiredSkills}' ELSE '[]'::jsonb END) choice
WHERE lower(trim(CASE WHEN jsonb_typeof(choice) = 'string' THEN choice#>>'{}' ELSE choice->>'key' END)) = ANY(ARRAY[
lower(skill_key), lower(regexp_replace(skill_key, '.*/', '')),
lower(CASE WHEN skill_key LIKE 'paperclipai/paperclip/%' THEN skill_slug
ELSE skill_slug || '--' || left(encode(sha256(convert_to(skill_key, 'UTF8')), 'hex'), 10) END)]));
$$;
--> statement-breakpoint
CREATE OR REPLACE FUNCTION paperclip_invalidate_skill_source_runtimes(p_company uuid, sources uuid[]) RETURNS void
LANGUAGE plpgsql SET search_path = public, pg_temp AS $$
DECLARE candidate uuid;
BEGIN
FOR candidate IN SELECT a.id FROM agents a WHERE a.company_id = p_company
AND a.status NOT IN ('pending_approval', 'terminated') AND EXISTS (
SELECT 1 FROM company_skills s WHERE s.company_id = p_company AND s.access_source_id = ANY(sources)
AND paperclip_agent_selects_runtime_skill(a.adapter_config, a.adapter_type, s.key, s.slug))
ORDER BY a.id FOR UPDATE LOOP
PERFORM paperclip_record_agent_runtime(p_company, candidate);
END LOOP;
END;
$$;
--> statement-breakpoint
CREATE OR REPLACE FUNCTION paperclip_skill_access_runtime_changed() RETURNS trigger
LANGUAGE plpgsql SET search_path = public, pg_temp AS $$
DECLARE previous jsonb; current jsonb; column_name text; changed boolean := false; company uuid; sources uuid[];
BEGIN
IF TG_OP <> 'INSERT' THEN previous := to_jsonb(OLD); END IF;
IF TG_OP <> 'DELETE' THEN current := to_jsonb(NEW); END IF;
IF TG_OP = 'UPDATE' THEN
FOREACH column_name IN ARRAY TG_ARGV LOOP
IF previous->column_name IS DISTINCT FROM current->column_name THEN changed := true; END IF;
END LOOP;
IF TG_NARGS > 0 AND NOT changed THEN RETURN NEW; END IF;
IF (previous - ARRAY['updated_at', 'created_at']) IS NOT DISTINCT FROM
(current - ARRAY['updated_at', 'created_at']) THEN RETURN NEW; END IF;
END IF;
company := coalesce((current->>'company_id')::uuid, (previous->>'company_id')::uuid);
IF TG_TABLE_NAME = 'company_skill_sources' THEN
sources := ARRAY[(previous->>'id')::uuid, (current->>'id')::uuid];
ELSIF TG_TABLE_NAME = 'project_skill_sources' THEN
sources := ARRAY[(previous->>'source_id')::uuid, (current->>'source_id')::uuid];
ELSE
SELECT array_agg(DISTINCT source_id) INTO sources FROM project_skill_sources WHERE company_id = company
AND project_id = ANY(CASE WHEN TG_TABLE_NAME = 'projects'
THEN ARRAY[(previous->>'id')::uuid, (current->>'id')::uuid]
ELSE ARRAY[(previous->>'project_id')::uuid, (current->>'project_id')::uuid] END);
END IF;
PERFORM paperclip_invalidate_skill_source_runtimes(company, sources);
IF TG_OP = 'DELETE' THEN RETURN OLD; END IF;
RETURN NEW;
END;
$$;
--> statement-breakpoint
CREATE OR REPLACE FUNCTION paperclip_skill_runtime_changed() RETURNS trigger
LANGUAGE plpgsql SET search_path = public, pg_temp AS $$
DECLARE previous jsonb; current jsonb; keys text[]; company uuid; candidate uuid;
BEGIN
IF TG_OP <> 'INSERT' THEN previous := to_jsonb(OLD); END IF;
IF TG_OP <> 'DELETE' THEN current := to_jsonb(NEW); END IF;
IF TG_OP = 'UPDATE' AND
jsonb_build_array(OLD.key, OLD.slug, OLD.markdown, OLD.source_type, OLD.source_locator, OLD.source_ref,
OLD.file_inventory, OLD.current_version_id, OLD.access_source_id, OLD.compatibility, OLD.trust_level, OLD.metadata)
IS NOT DISTINCT FROM
jsonb_build_array(NEW.key, NEW.slug, NEW.markdown, NEW.source_type, NEW.source_locator, NEW.source_ref,
NEW.file_inventory, NEW.current_version_id, NEW.access_source_id, NEW.compatibility, NEW.trust_level, NEW.metadata)
THEN RETURN NEW; END IF;
company := coalesce((current->>'company_id')::uuid, (previous->>'company_id')::uuid);
SELECT array_agg(DISTINCT lower(reference)) INTO keys FROM (
SELECT row->>'key' AS reference FROM unnest(ARRAY[previous, current]) row
UNION ALL SELECT regexp_replace(row->>'key', '.*/', '') FROM unnest(ARRAY[previous, current]) row
UNION ALL SELECT CASE WHEN row->>'key' LIKE 'paperclipai/paperclip/%' THEN row->>'slug'
ELSE (row->>'slug') || '--' || left(encode(sha256(convert_to(row->>'key', 'UTF8')), 'hex'), 10) END
FROM unnest(ARRAY[previous, current]) row
) refs WHERE reference IS NOT NULL;
FOR candidate IN SELECT a.id FROM agents a WHERE a.company_id = company
AND a.status NOT IN ('pending_approval', 'terminated') AND (
(a.adapter_type <> 'paperclip_runner' AND keys && ARRAY['paperclipai/paperclip/paperclip', 'paperclipai/paperclip/complain', 'paperclipai/paperclip/suggestion-box']) OR
EXISTS (SELECT 1 FROM jsonb_array_elements(CASE WHEN jsonb_typeof(a.adapter_config#>'{paperclipSkillSync,desiredSkills}') = 'array'
THEN a.adapter_config#>'{paperclipSkillSync,desiredSkills}' ELSE '[]'::jsonb END) choice
WHERE lower(trim(CASE WHEN jsonb_typeof(choice) = 'string' THEN choice#>>'{}' ELSE choice->>'key' END)) = ANY(keys)))
ORDER BY a.id FOR UPDATE LOOP
PERFORM paperclip_record_agent_runtime(company, candidate);
END LOOP;
IF TG_OP = 'DELETE' THEN RETURN OLD; END IF;
RETURN NEW;
END;
$$;
--> statement-breakpoint
DROP TRIGGER IF EXISTS paperclip_skill_runtime_changed ON company_skills;
CREATE CONSTRAINT TRIGGER paperclip_skill_runtime_changed AFTER INSERT OR UPDATE OR DELETE ON company_skills
DEFERRABLE INITIALLY IMMEDIATE
FOR EACH ROW EXECUTE FUNCTION paperclip_skill_runtime_changed();
--> statement-breakpoint
DROP TRIGGER IF EXISTS paperclip_skill_policy_runtime_changed ON company_skill_policies;
CREATE TRIGGER paperclip_skill_policy_runtime_changed AFTER INSERT OR UPDATE OR DELETE ON company_skill_policies
FOR EACH ROW EXECUTE FUNCTION paperclip_company_agent_runtimes_changed();
--> statement-breakpoint
DROP TRIGGER IF EXISTS paperclip_skill_source_runtime_changed ON project_skill_sources;
CREATE TRIGGER paperclip_skill_source_runtime_changed AFTER INSERT OR UPDATE OR DELETE ON project_skill_sources
FOR EACH ROW EXECUTE FUNCTION paperclip_skill_access_runtime_changed('project_id', 'source_id', 'active');
--> statement-breakpoint
DROP TRIGGER IF EXISTS paperclip_skill_source_access_runtime_changed ON company_skill_sources;
CREATE TRIGGER paperclip_skill_source_access_runtime_changed AFTER INSERT OR UPDATE OR DELETE ON company_skill_sources
FOR EACH ROW EXECUTE FUNCTION paperclip_skill_access_runtime_changed('access_scope', 'removed_at');
--> statement-breakpoint
DROP TRIGGER IF EXISTS paperclip_project_access_runtime_changed ON project_access_members;
CREATE TRIGGER paperclip_project_access_runtime_changed AFTER INSERT OR UPDATE OR DELETE ON project_access_members
FOR EACH ROW EXECUTE FUNCTION paperclip_skill_access_runtime_changed();
--> statement-breakpoint
DROP TRIGGER IF EXISTS paperclip_project_privacy_runtime_changed ON projects;
CREATE TRIGGER paperclip_project_privacy_runtime_changed AFTER UPDATE ON projects
FOR EACH ROW EXECUTE FUNCTION paperclip_skill_access_runtime_changed('visibility', 'privacy_owner_user_id', 'personal_owner_user_id', 'execution_workspace_policy');
--> statement-breakpoint
CREATE OR REPLACE FUNCTION paperclip_skill_pins_runtime_changed() RETURNS trigger
LANGUAGE plpgsql SET search_path = public, pg_temp AS $$
DECLARE company uuid;
BEGIN
IF coalesce(OLD.experimental->'enableBetaSkills', 'false'::jsonb) IS DISTINCT FROM coalesce(NEW.experimental->'enableBetaSkills', 'false'::jsonb) THEN
FOR company IN SELECT DISTINCT company_id FROM agents ORDER BY company_id LOOP
PERFORM paperclip_invalidate_company_agent_runtimes(company);
END LOOP;
END IF;
RETURN NEW;
END;
$$;
--> statement-breakpoint
DROP TRIGGER IF EXISTS paperclip_skill_pins_runtime_changed ON instance_settings;
CREATE TRIGGER paperclip_skill_pins_runtime_changed AFTER UPDATE ON instance_settings
FOR EACH ROW EXECUTE FUNCTION paperclip_skill_pins_runtime_changed();
File diff suppressed because it is too large. Load diff
+7
View File
@@ -2423,6 +2423,13 @@
"when": 1792100545989,
"tag": "0347_flippant_imperial_guard",
"breakpoints": true
},
{
"idx": 348,
"version": "7",
"when": 1792101163248,
"tag": "0348_agent_skill_runtime_hooks",
"breakpoints": true
}
]
}
+1
View File
@@ -271,6 +271,7 @@ export type {
PluginIssueSummariesClient,
PluginAgentsClient,
PluginAgentRuntimeState,
PluginAgentRuntimeSkill,
PluginAccessClient,
PluginAccessMembersClient,
PluginAccessInvitesClient,
+11
View File
@@ -1687,7 +1687,18 @@ export interface PluginApprovalsClient {
* Requires `agents.read` for reads; `agents.pause` / `agents.resume` /
* `agents.invoke` for write operations.
*/
export interface PluginAgentRuntimeSkill {
key: string;
runtimeName: string;
source: string;
versionId?: string | null;
currentVersionId?: string | null;
sourceStatus?: "available" | "missing";
missingDetail?: string | null;
}
export interface PluginAgentRuntimeState {
skills: PluginAgentRuntimeSkill[];
revision: string;
agent: Agent;
instructionsRootPath: string | null;
@@ -0,0 +1,159 @@
import { randomUUID } from "node:crypto";
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { and, eq } from "drizzle-orm";
import { agents, companies, companyMemberships, companySkills, companySkillVersions, companySkillSources, projectSkillSources, projects, projectAccessMembers, createDb, instanceSettings, pluginConfig, plugins, resourceLifecycleEvents, type Db } from "@paperclipai/db";
import { LOW_TRUST_REVIEW_PRESET } from "@paperclipai/shared";
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
import { pluginAgentRuntimeState } from "../services/plugin-agent-runtime-state.js";
import { instanceSettingsService } from "../services/instance-settings.js";
const support = await getEmbeddedPostgresTestSupport();
const describeDatabase = support.supported ? describe : describe.skip;
describeDatabase("skill runtime invalidation", () => {
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
let db: Db;
let home: string;
let companyId: string;
let pluginId: string;
let skillId: string;
let key: string;
beforeAll(async () => {
database = await startEmbeddedPostgresTestDatabase("paperclip-runtime-skills-"); db = createDb(database.connectionString);
home = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-runtime-skills-home-"));
}, 90_000);
afterAll(async () => { await database?.cleanup(); await fs.rm(home, { recursive: true, force: true }); });
afterEach(() => vi.unstubAllEnvs());
beforeEach(async () => {
vi.stubEnv("PAPERCLIP_HOME", home);
companyId = randomUUID(); pluginId = randomUUID(); skillId = randomUUID(); key = `company/${companyId}/fixture`;
await db.insert(companies).values({ id: companyId, name: "Skill runtime fixture", issuePrefix: `S${companyId.slice(0, 6)}` });
await db.insert(plugins).values({ id: pluginId, pluginKey: pluginId, packageName: "runtime-fixture", version: "1.0.0", status: "ready", manifestJson: { resourceLifecycle: ["agent_runtime"], capabilities: ["events.subscribe", "agents.lifecycle.manage"] } as never });
await db.insert(pluginConfig).values({ companyId, pluginId, configJson: {} });
await db.insert(companySkills).values({ id: skillId, companyId, key, slug: "fixture", name: "Fixture", markdown: "old", sourceType: "github", sourceLocator: "https://github.com/fixture/skills" });
});
async function agent(selection: unknown[] = [key], company = companyId) {
const [created] = await db.insert(agents).values({ companyId: company, name: randomUUID(), adapterType: "paperclip_runner", lifecycleRequiredPluginIds: [pluginId], adapterConfig: { paperclipSkillSync: { desiredSkills: selection } } }).returning();
return created;
}
const events = () => db.select().from(resourceLifecycleEvents).where(and(eq(resourceLifecycleEvents.companyId, companyId), eq(resourceLifecycleEvents.resourceType, "agent_runtime"))).orderBy(resourceLifecycleEvents.id);
async function publish(content: string, revision: number) {
return db.transaction(async tx => {
const [version] = await tx.insert(companySkillVersions).values({ companyId, companySkillId: skillId, revisionNumber: revision, fileInventory: [{ path: "SKILL.md", kind: "skill", content }] }).returning();
await tx.update(companySkills).set({ markdown: content, currentVersionId: version.id }).where(eq(companySkills.id, skillId));
return version;
});
}
it("invalidates string, structured, and legacy alias assignments while leaving other agents untouched", async () => {
const first = await agent(); const second = await agent([{ key, versionId: randomUUID() }]); const alias = await agent(["fixture"]);
await agent([]); await agent(["unrelated"]);
const otherCompany = randomUUID();
await db.insert(companies).values({ id: otherCompany, name: "Other company", issuePrefix: `X${otherCompany.slice(0, 6)}` });
await agent([key], otherCompany);
const before = (await events()).length;
await db.update(companySkills).set({ markdown: "changed" }).where(eq(companySkills.id, skillId));
expect((await events()).slice(before).map(row => row.resourceId).sort()).toEqual([first.id, second.id, alias.id].sort());
const after = (await events()).length;
await db.update(companySkills).set({ starCount: 9, updatedAt: new Date() }).where(eq(companySkills.id, skillId));
expect(await events()).toHaveLength(after);
});
it.each(["paperclip", "complain", "suggestion-box"])("invalidates implicit legacy %s skills with an empty selection", async slug => {
const legacy = await agent([]);
const native = await agent([]);
await db.update(agents).set({ adapterType: "process" }).where(eq(agents.id, legacy.id));
const [skill] = await db.insert(companySkills).values({ companyId, key: `paperclipai/paperclip/${slug}`, slug, name: slug, markdown: "implicit" }).returning();
const before = (await events()).length;
await db.update(companySkills).set({ markdown: "changed implicit" }).where(eq(companySkills.id, skill.id));
expect((await events()).slice(before)).toEqual([expect.objectContaining({ resourceId: legacy.id })]);
await db.delete(companySkills).where(eq(companySkills.id, skill.id));
expect((await events()).slice(before).map(row => row.resourceId)).toEqual([legacy.id, legacy.id]);
expect((await events()).slice(before).some(row => row.resourceId === native.id)).toBe(false);
});
it("captures companion-file changes, rename, and deletion using both old and new references", async () => {
const old = await agent(); const next = await agent([`${key}-next`]);
const before = (await events()).length;
await db.update(companySkills).set({ fileInventory: [{ path: "references/guide.md", kind: "reference", size: 5 }] }).where(eq(companySkills.id, skillId));
expect((await events()).slice(before)).toEqual([expect.objectContaining({ resourceId: old.id })]);
const renamed = (await events()).length;
await db.update(companySkills).set({ key: `${key}-next`, slug: "next" }).where(eq(companySkills.id, skillId));
expect((await events()).slice(renamed).map(row => row.resourceId).sort()).toEqual([old.id, next.id].sort());
const deleted = (await events()).length;
await db.delete(companySkills).where(eq(companySkills.id, skillId));
expect((await events()).slice(deleted)).toEqual([expect.objectContaining({ resourceId: next.id })]);
});
it("invalidates private-source access changes while ignoring source polling bookkeeping", async () => {
const assigned = await agent();
const [project] = await db.insert(projects).values({ companyId, name: "Private source", visibility: "private" }).returning();
const [source] = await db.insert(companySkillSources).values({ companyId, repositoryUrl: "https://github.com/fixture/private", fullName: "fixture/private", trackingRef: "HEAD", accessScope: "projects" }).returning();
await db.update(companySkills).set({ accessSourceId: source.id }).where(eq(companySkills.id, skillId));
const [binding] = await db.insert(projectSkillSources).values({ companyId, projectId: project.id, sourceId: source.id, repositoryUrl: source.repositoryUrl, trackingRef: "HEAD" }).returning();
await db.insert(agents).values(Array.from({ length: 250 }, () => ({ companyId, name: randomUUID(), adapterType: "paperclip_runner", adapterConfig: { paperclipSkillSync: { desiredSkills: ["unrelated"] } } })));
const before = (await events()).length;
await db.update(projectSkillSources).set({ nextSyncAt: new Date(), failureCount: 1 }).where(eq(projectSkillSources.id, binding.id));
expect(await events()).toHaveLength(before);
const [access] = await db.insert(projectAccessMembers).values({ companyId, projectId: project.id, subjectType: "agent", subjectId: assigned.id }).returning();
expect(await events()).toHaveLength(before + 1);
await db.delete(projectAccessMembers).where(eq(projectAccessMembers.id, access.id));
expect(await events()).toHaveLength(before + 2);
await db.update(projects).set({ visibility: "open" }).where(eq(projects.id, project.id));
expect(await events()).toHaveLength(before + 3);
expect((await events()).slice(before).map(row => row.resourceId)).toEqual([assigned.id, assigned.id, assigned.id]);
const [unrelatedProject] = await db.insert(projects).values({ companyId, name: "Unrelated", visibility: "private" }).returning();
await db.insert(projectAccessMembers).values({ companyId, projectId: unrelatedProject.id, subjectType: "agent", subjectId: assigned.id });
await db.update(projects).set({ visibility: "open" }).where(eq(projects.id, unrelatedProject.id));
expect(await events()).toHaveLength(before + 3);
await db.update(projectSkillSources).set({ sourceId: null }).where(eq(projectSkillSources.id, binding.id));
expect((await events()).slice(before + 3)).toEqual([expect.objectContaining({ resourceId: assigned.id })]);
});
it("keeps publication and invalidation in the same transaction", async () => {
await agent(); const before = await events();
await expect(db.transaction(async tx => {
await tx.update(companySkills).set({ markdown: "rolled back" }).where(eq(companySkills.id, skillId));
throw new Error("rollback");
})).rejects.toThrow("rollback");
expect(await events()).toEqual(before);
expect((await db.select().from(companySkills).where(eq(companySkills.id, skillId)))[0].markdown).toBe("old");
});
it("invalidates policy-only project access changes and removes or restores the selected skill", async () => {
const assigned = await agent();
await agent([]);
await db.insert(companyMemberships).values({ companyId, principalType: "agent", principalId: assigned.id, status: "active" });
const [project] = await db.insert(projects).values({ companyId, name: "Policy-bound source", visibility: "open" }).returning();
const [source] = await db.insert(companySkillSources).values({ companyId, repositoryUrl: "https://github.com/fixture/policy", fullName: "fixture/policy", trackingRef: "HEAD", accessScope: "projects" }).returning();
await db.insert(projectSkillSources).values({ companyId, projectId: project.id, sourceId: source.id, repositoryUrl: source.repositoryUrl, trackingRef: "HEAD" });
await db.update(companySkills).set({ accessSourceId: source.id }).where(eq(companySkills.id, skillId));
await publish("policy-scoped content", 1);
const state = () => pluginAgentRuntimeState(db, pluginId, companyId, assigned.id);
expect((await state())!.skills.map(skill => skill.key)).toEqual([key]);
const before = (await events()).length;
const restricted = { authorizationPolicy: { trustBoundary: { mode: LOW_TRUST_REVIEW_PRESET, projectIds: [] } } };
await db.update(projects).set({ executionWorkspacePolicy: restricted }).where(eq(projects.id, project.id));
expect((await events()).slice(before).map(event => event.resourceId)).toEqual([assigned.id]);
expect((await state())!.skills).toEqual([]);
await db.update(projects).set({ executionWorkspacePolicy: null }).where(eq(projects.id, project.id));
expect((await events()).slice(before).map(event => event.resourceId)).toEqual([assigned.id, assigned.id]);
expect((await state())!.skills.map(skill => skill.key)).toEqual([key]);
const restored = (await events()).length;
await db.update(projects).set({ executionWorkspacePolicy: null }).where(eq(projects.id, project.id));
expect(await events()).toHaveLength(restored);
});
it("resolves immutable pinned and latest skill sources and removes unassigned skills", async () => {
await instanceSettingsService(db).getExperimental();
await db.update(instanceSettings).set({ experimental: { enableBetaSkills: true } });
const first = await publish("first version", 1);
const pinned = await agent([{ key, versionId: first.id }]);
const latest = await agent();
const second = await publish("second version", 2);
const pinnedState = await pluginAgentRuntimeState(db, pluginId, companyId, pinned.id);
const latestState = await pluginAgentRuntimeState(db, pluginId, companyId, latest.id);
expect(pinnedState!.skills).toEqual([expect.objectContaining({ key, versionId: first.id })]);
expect(latestState!.skills).toEqual([expect.objectContaining({ key, versionId: second.id })]);
expect(await fs.readFile(path.join(pinnedState!.skills[0].source, "SKILL.md"), "utf8")).toBe("first version");
expect(await fs.readFile(path.join(latestState!.skills[0].source, "SKILL.md"), "utf8")).toBe("second version");
await db.update(agents).set({ adapterConfig: { paperclipSkillSync: { desiredSkills: [] } } }).where(eq(agents.id, latest.id));
expect((await pluginAgentRuntimeState(db, pluginId, companyId, latest.id))!.skills).toEqual([]);
});
});
+41 -2
View File
@@ -5,8 +5,8 @@ import { promises as fs } from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { agents, folders, companySkillSources, companySkillSourceEntries, companies, companySkills, companySkillVersions, companySkillRepositorySnapshots, createDb } from '@paperclipai/db';
import { eq, sql } from 'drizzle-orm';
import { agents, folders, companySkillSources, companySkillSourceEntries, companies, companySkills, companySkillVersions, companySkillRepositorySnapshots, resourceLifecycleEvents, createDb } from '@paperclipai/db';
import { and, eq, sql } from 'drizzle-orm';
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from './helpers/embedded-postgres.js';
import { unprocessable } from '../errors.js';
import { githubFixture } from './helpers/github-skills.js';
@@ -37,6 +37,45 @@ describe.skipIf(!support.supported)('skill source persistence', () => {
if (originalHome === undefined) delete process.env.PAPERCLIP_HOME; else process.env.PAPERCLIP_HOME = originalHome;
await testDb?.cleanup(); if (home) await fs.rm(home, { recursive: true, force: true });
});
it('publishes two assigned skills while a concurrent version save holds the second skill', async () => {
const companyId = randomUUID();
await db.insert(companies).values({ id: companyId, name: 'Concurrent skill publication', issuePrefix: 'CSP' });
const originals = { 'a/SKILL.md': md('a'), 'b/SKILL.md': md('b') };
const sourceContext = { ...context, actor: { type: 'system' as const }, read: () => githubFixture(originals, {}, sha) };
const created = await skillSourceService(db).create(companyId, { repositoryUrl: 'https://github.com/acme/skills', trackingRef: 'main', commitSha: sha, selectedPaths: Object.keys(originals) }, sourceContext);
const selected = await Promise.all(created.source.entries.map(entry => companySkillService(db).getById(companyId, entry.skillId!)));
const second = selected.find(skill => skill!.name === 'b')!;
const [agent] = await db.insert(agents).values({ companyId, name: 'Uses both skills', adapterType: 'paperclip_runner', adapterConfig: { paperclipSkillSync: { desiredSkills: selected.map(skill => skill!.key) } } }).returning();
const sourceDb = createDb(testDb.connectionString, { maxConnections: 1 });
const versionDb = createDb(testDb.connectionString, { maxConnections: 1 });
const [backend] = await sourceDb.execute(sql`SELECT pg_backend_pid() AS pid`);
let locked!: () => void; let resume!: () => void;
const secondLocked = new Promise<void>(resolve => { locked = resolve; });
const maySave = new Promise<void>(resolve => { resume = resolve; });
const version = versionDb.transaction(async tx => {
await tx.execute(sql`SET LOCAL lock_timeout = '8s'`);
await tx.select().from(companySkills).where(eq(companySkills.id, second.id)).for('update');
locked(); await maySave;
return companySkillService(versionDb).createVersion(companyId, second.id, {}, null, { database: tx, skipInventoryRefresh: true, skill: second, fileInventory: [{ path: 'SKILL.md', kind: 'skill', content: md('manual') }] });
});
await secondLocked;
const changed = { 'a/SKILL.md': `${md('a')}Changed A.`, 'b/SKILL.md': `${md('b')}Changed B.` };
const refresh = skillSourceService(sourceDb).refresh(companyId, created.source.id, { ...sourceContext, read: () => githubFixture(changed, {}, 'b'.repeat(40)) });
try {
await vi.waitFor(async () => {
const [row] = await db.execute(sql`SELECT wait_event_type FROM pg_stat_activity WHERE pid = ${Number(backend.pid)}`);
expect(row.wait_event_type).toBe('Lock');
}, { timeout: 5000 });
} finally {
resume();
try { await Promise.all([version, refresh]); }
finally { await Promise.all([sourceDb.$client.end(), versionDb.$client.end()]); }
}
expect((await refresh).updated).toHaveLength(2);
const runtimeEvents = await db.select().from(resourceLifecycleEvents).where(and(eq(resourceLifecycleEvents.resourceId, agent.id), eq(resourceLifecycleEvents.resourceType, 'agent_runtime')));
expect(runtimeEvents.length).toBeGreaterThan(1);
}, 20000);
it('explicitly repairs a missing branch while retaining connection, selection and installed versions', async () => {
const companyId = randomUUID();
await db.insert(companies).values({ id: companyId, name: 'Branch repair', issuePrefix: 'BRP' });
@@ -2,11 +2,16 @@ import { and, desc, eq } from "drizzle-orm";
import { agents, plugins, pluginCompanySettings, resourceLifecycleEvents, type Db } from "@paperclipai/db";
import type { PluginAgentRuntimeState } from "@paperclipai/plugin-sdk";
import { agentRecordQueries } from "../lib/agent-records.js";
import { forbidden } from "../errors.js";
import { conflict, forbidden } from "../errors.js";
import { agentInstructionsService } from "./agent-instructions.js";
import { companySkillService } from "./company-skills.js";
import { withSkillActor } from "./skill-access.js";
import { readPaperclipSkillSyncPreference } from "@paperclipai/adapter-utils/server-utils";
import { skillVersionSelectionMap } from "./runtime-skill-selections.js";
import { instanceSettingsService } from "./instance-settings.js";
import { agentInstanceSecretService } from "./agent-instance-secrets.js";
export async function pluginAgentRuntimeState(db: Db, pluginId: string, companyId: string, agentId: string): Promise<PluginAgentRuntimeState | null> {
async function readState(db: Db, pluginId: string, companyId: string, agentId: string): Promise<Omit<PluginAgentRuntimeState, "skills"> | null> {
return db.transaction(async tx => {
const [agent] = await tx.select().from(agents)
.where(and(eq(agents.companyId, companyId), eq(agents.id, agentId))).for("share");
@@ -30,3 +35,22 @@ export async function pluginAgentRuntimeState(db: Db, pluginId: string, companyI
};
});
}
export async function pluginAgentRuntimeState(db: Db, pluginId: string, companyId: string, agentId: string): Promise<PluginAgentRuntimeState | null> {
for (let attempt = 0; attempt < 3; attempt++) {
const state = await readState(db, pluginId, companyId, agentId);
if (!state) return null;
const preference = readPaperclipSkillSyncPreference(state.agent.adapterConfig);
const settings = await instanceSettingsService(db).getExperimental();
const skills = await withSkillActor({ type: "agent", agentId, companyId, source: "agent_key" }, () =>
companySkillService(db).listRuntimeSkillEntries(companyId, {
selectionConfig: state.agent.adapterConfig, includeLegacy: state.agent.adapterType !== "paperclip_runner", snapshotLatest: true,
versionSelections: skillVersionSelectionMap(preference.desiredSkillEntries, { versionPinsEnabled: settings.enableBetaSkills }),
}));
const current = await readState(db, pluginId, companyId, agentId);
if (!current) return null;
if (current.revision === state.revision) return { ...current, skills };
}
throw conflict("Agent runtime changed while resolving synchronization inputs; retry");
}
+2
View File
@@ -98,6 +98,7 @@ export function skillSourceService(db: Db) {
return `github/${scan.repositoryId}/${identity}/${normalizeAgentUrlKey(name) || 'skill'}`;
}
async function publish(tx: Tx, source: SourceRow, scan: ScannedSkillSource, selectedPaths: string[], excludedFolders: string[], context: SkillSourceContext, selectionReviewed: boolean): Promise<Omit<SkillSourceRefreshResult, 'source'>> {
await tx.execute(sql`SET CONSTRAINTS paperclip_skill_runtime_changed DEFERRED`);
const mode = scan.packageMode ?? 'skills';
if (mode === 'repository' && scan.skills.length > 0 && (scan.repositoryPackage?.error || !scan.repositoryFiles?.length)) {
throw unprocessable(scan.repositoryPackage?.error ?? 'Repository package has no files. Scan it again before importing.');
@@ -396,6 +397,7 @@ export function skillSourceService(db: Db) {
const directories: string[] = [];
try {
return await db.transaction(async tx => {
await tx.execute(sql`SET CONSTRAINTS paperclip_skill_runtime_changed DEFERRED`);
await lock(tx);
for (const skillId of skillIds) await skills.makeImportedSkillEditable(companyId, skillId, context.actor, folderId, tx, directories);
await remove(tx, skillIds.length, 0);