Files
PaperClipAI/packages/db/src/migrations/0348_agent_skill_runtime_hooks.sql
T
Nicky LeachandPaperclip 5fbece9dcc Synchronize assigned skills through agent runtime hooks (#16501)
## Thinking Path

> - Paperclip manages shared skills and agent assignments.
> - Runtime plugins need installed skills before a task starts.
> - Skill content can change without changing the agent row.
> - Source access and version pins also affect delivered files.
> - This pull request feeds those changes into the durable runtime
stream.
> - The runtime reader resolves the same immutable skill sources used
for execution.

## Linked Issues or Issue Description

Builds on merged PR #16499. This pull request now targets `master` and
contains only the five skill synchronization commits.

**Problem**

Plugins receive no durable notification when an assigned skill changes.
A saved assignment can also become inaccessible after private-source
access changes.

**Proposed solution**

Invalidate referencing agents at installed skill mutations. Invalidate
inherited access when source bindings, project privacy, or policy
change. Return authorized skill sources and version IDs through the
owning runtime reader.

**Alternatives considered**

Activity-only subscriptions miss service writers. Per-task
synchronization adds preparation work. Copying current files would
ignore version pins.

**Roadmap alignment**

This extends existing skills, source synchronization, and plugin runtime
support.

## What Changed

- Defer multi-skill source runtime hooks until commit, after publication
holds all skill rows. Keep file-save journaling immediate to preserve
filesystem rollback.

- Add transactional invalidations for skill content, companion files,
versions, rename, deletion, policy, source access, and pin-setting
changes.
- Recognize canonical keys and supported legacy aliases.
- Reuse the execution skill resolver, current agent access, and instance
version-pin rules.
- Resolve and materialize sources outside the agent row lock. Recheck
ownership and runtime revision afterward.
- Keep task-only mentions and running-task snapshots in the execution
path.
- Document conservative invalidations for pinned assignments and
inherited access.

## Verification

- Rebased onto origin/master `9f4a956b90`, which contains merged PR
#16499. All five commits are unchanged according to `git range-diff`; no
conflict resolutions or source changes were needed.
- On rebased head `58b065941`, all 237 focused tests passed: runtime
skill hooks/source publication (41), plus content service, assignment
routes, policy routes, and project source access (196).
- Server typecheck passed after rebuilding Runner with the installed
Rust toolchain. Migration numbering/safety and map checks passed;
snapshot pruning made no changes. Migration CLI checks required local
IPC permission.
- Real PostgreSQL coverage includes concurrent source
publication/version saves, policy-only tightening/restoration,
referencing-agent fanout, company isolation, no-op counters, supporting
files, rename/deletion, rollback, private-source access, pinned/latest
bytes, and unassignment.
- Earlier stack validation passed workspace typecheck/build. Fresh
hosted CI and review must verify the rebased head before handoff.
- The broad local suite was previously limited by an unchanged
diagnostic test that rejects "private" in an error stack under this
/private/tmp worktree; that unchanged suite passed all 108 tests in the
original checkout. Hosted full-suite checks provide exact-head
verification.
- Diff whitespace and full PR secret/PII scan passed.
- All hosted CI/security checks are green on `58b065941`; fresh standard
Greptile and Apex both returned 5/5 with no findings or unresolved
threads. The PR targets master and is mergeable without conflicts.

## Risks

- Apply migration 0348 after the runtime-stream migration in #16499.
- Content invalidations include pinned assignments. The resolved version
stays pinned; consumers can avoid writes when its version/hash is
unchanged.
- Project access and source changes invalidate only agents assigned
skills from the affected sources. Company policy still invalidates the
company. Source polling bookkeeping and popularity counters do not
create work.
- The reader rebuilds installed caches without upstream requests.
Concurrent changes cause a bounded retry and then a conflict.
- Private transport integration and live remote computer qualification
remain separate.

## Model Used

OpenAI Codex, GPT-6, with reasoning, repository inspection, code
execution, and testing. The exact deployment model ID and context window
are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub references)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id
- [x] I have run tests locally and they pass (focused suites; broad
local limitation documented above)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-16 08:28:47 -07:00

167 lines
9.7 KiB
PL/PgSQL

CREATE OR REPLACE FUNCTION paperclip_invalidate_company_agent_runtimes(p_company uuid) RETURNS void
LANGUAGE plpgsql SET search_path = public, pg_temp AS $$
DECLARE candidate uuid;
BEGIN
FOR candidate IN SELECT id FROM agents WHERE company_id = p_company
AND status NOT IN ('pending_approval', 'terminated') ORDER BY id FOR UPDATE LOOP
PERFORM paperclip_record_agent_runtime(p_company, candidate);
END LOOP;
END;
$$;
--> statement-breakpoint
CREATE OR REPLACE FUNCTION paperclip_company_agent_runtimes_changed() RETURNS trigger
LANGUAGE plpgsql SET search_path = public, pg_temp AS $$
DECLARE column_name text; changed boolean := false;
BEGIN
IF TG_OP = 'UPDATE' AND TG_NARGS > 0 THEN
FOREACH column_name IN ARRAY TG_ARGV LOOP
IF to_jsonb(OLD)->column_name IS DISTINCT FROM to_jsonb(NEW)->column_name THEN changed := true; END IF;
END LOOP;
IF NOT changed THEN RETURN NEW; END IF;
END IF;
IF TG_OP = 'UPDATE' AND (to_jsonb(OLD) - ARRAY['updated_at', 'created_at']) IS NOT DISTINCT FROM
(to_jsonb(NEW) - ARRAY['updated_at', 'created_at']) THEN RETURN NEW; END IF;
IF TG_OP <> 'INSERT' THEN PERFORM paperclip_invalidate_company_agent_runtimes(OLD.company_id); END IF;
IF TG_OP = 'INSERT' OR (TG_OP = 'UPDATE' AND NEW.company_id IS DISTINCT FROM OLD.company_id) THEN
PERFORM paperclip_invalidate_company_agent_runtimes(NEW.company_id);
END IF;
IF TG_OP = 'DELETE' THEN RETURN OLD; END IF;
RETURN NEW;
END;
$$;
--> statement-breakpoint
CREATE OR REPLACE FUNCTION paperclip_agent_selects_runtime_skill(config jsonb, adapter text, skill_key text, skill_slug text) RETURNS boolean
LANGUAGE sql IMMUTABLE SET search_path = public, pg_temp AS $$
SELECT (adapter <> 'paperclip_runner' AND skill_key IN
('paperclipai/paperclip/paperclip', 'paperclipai/paperclip/complain', 'paperclipai/paperclip/suggestion-box')) OR
EXISTS (SELECT 1 FROM jsonb_array_elements(CASE WHEN jsonb_typeof(config#>'{paperclipSkillSync,desiredSkills}') = 'array'
THEN config#>'{paperclipSkillSync,desiredSkills}' ELSE '[]'::jsonb END) choice
WHERE lower(trim(CASE WHEN jsonb_typeof(choice) = 'string' THEN choice#>>'{}' ELSE choice->>'key' END)) = ANY(ARRAY[
lower(skill_key), lower(regexp_replace(skill_key, '.*/', '')),
lower(CASE WHEN skill_key LIKE 'paperclipai/paperclip/%' THEN skill_slug
ELSE skill_slug || '--' || left(encode(sha256(convert_to(skill_key, 'UTF8')), 'hex'), 10) END)]));
$$;
--> statement-breakpoint
CREATE OR REPLACE FUNCTION paperclip_invalidate_skill_source_runtimes(p_company uuid, sources uuid[]) RETURNS void
LANGUAGE plpgsql SET search_path = public, pg_temp AS $$
DECLARE candidate uuid;
BEGIN
FOR candidate IN SELECT a.id FROM agents a WHERE a.company_id = p_company
AND a.status NOT IN ('pending_approval', 'terminated') AND EXISTS (
SELECT 1 FROM company_skills s WHERE s.company_id = p_company AND s.access_source_id = ANY(sources)
AND paperclip_agent_selects_runtime_skill(a.adapter_config, a.adapter_type, s.key, s.slug))
ORDER BY a.id FOR UPDATE LOOP
PERFORM paperclip_record_agent_runtime(p_company, candidate);
END LOOP;
END;
$$;
--> statement-breakpoint
CREATE OR REPLACE FUNCTION paperclip_skill_access_runtime_changed() RETURNS trigger
LANGUAGE plpgsql SET search_path = public, pg_temp AS $$
DECLARE previous jsonb; current jsonb; column_name text; changed boolean := false; company uuid; sources uuid[];
BEGIN
IF TG_OP <> 'INSERT' THEN previous := to_jsonb(OLD); END IF;
IF TG_OP <> 'DELETE' THEN current := to_jsonb(NEW); END IF;
IF TG_OP = 'UPDATE' THEN
FOREACH column_name IN ARRAY TG_ARGV LOOP
IF previous->column_name IS DISTINCT FROM current->column_name THEN changed := true; END IF;
END LOOP;
IF TG_NARGS > 0 AND NOT changed THEN RETURN NEW; END IF;
IF (previous - ARRAY['updated_at', 'created_at']) IS NOT DISTINCT FROM
(current - ARRAY['updated_at', 'created_at']) THEN RETURN NEW; END IF;
END IF;
company := coalesce((current->>'company_id')::uuid, (previous->>'company_id')::uuid);
IF TG_TABLE_NAME = 'company_skill_sources' THEN
sources := ARRAY[(previous->>'id')::uuid, (current->>'id')::uuid];
ELSIF TG_TABLE_NAME = 'project_skill_sources' THEN
sources := ARRAY[(previous->>'source_id')::uuid, (current->>'source_id')::uuid];
ELSE
SELECT array_agg(DISTINCT source_id) INTO sources FROM project_skill_sources WHERE company_id = company
AND project_id = ANY(CASE WHEN TG_TABLE_NAME = 'projects'
THEN ARRAY[(previous->>'id')::uuid, (current->>'id')::uuid]
ELSE ARRAY[(previous->>'project_id')::uuid, (current->>'project_id')::uuid] END);
END IF;
PERFORM paperclip_invalidate_skill_source_runtimes(company, sources);
IF TG_OP = 'DELETE' THEN RETURN OLD; END IF;
RETURN NEW;
END;
$$;
--> statement-breakpoint
CREATE OR REPLACE FUNCTION paperclip_skill_runtime_changed() RETURNS trigger
LANGUAGE plpgsql SET search_path = public, pg_temp AS $$
DECLARE previous jsonb; current jsonb; keys text[]; company uuid; candidate uuid;
BEGIN
IF TG_OP <> 'INSERT' THEN previous := to_jsonb(OLD); END IF;
IF TG_OP <> 'DELETE' THEN current := to_jsonb(NEW); END IF;
IF TG_OP = 'UPDATE' AND
jsonb_build_array(OLD.key, OLD.slug, OLD.markdown, OLD.source_type, OLD.source_locator, OLD.source_ref,
OLD.file_inventory, OLD.current_version_id, OLD.access_source_id, OLD.compatibility, OLD.trust_level, OLD.metadata)
IS NOT DISTINCT FROM
jsonb_build_array(NEW.key, NEW.slug, NEW.markdown, NEW.source_type, NEW.source_locator, NEW.source_ref,
NEW.file_inventory, NEW.current_version_id, NEW.access_source_id, NEW.compatibility, NEW.trust_level, NEW.metadata)
THEN RETURN NEW; END IF;
company := coalesce((current->>'company_id')::uuid, (previous->>'company_id')::uuid);
SELECT array_agg(DISTINCT lower(reference)) INTO keys FROM (
SELECT row->>'key' AS reference FROM unnest(ARRAY[previous, current]) row
UNION ALL SELECT regexp_replace(row->>'key', '.*/', '') FROM unnest(ARRAY[previous, current]) row
UNION ALL SELECT CASE WHEN row->>'key' LIKE 'paperclipai/paperclip/%' THEN row->>'slug'
ELSE (row->>'slug') || '--' || left(encode(sha256(convert_to(row->>'key', 'UTF8')), 'hex'), 10) END
FROM unnest(ARRAY[previous, current]) row
) refs WHERE reference IS NOT NULL;
FOR candidate IN SELECT a.id FROM agents a WHERE a.company_id = company
AND a.status NOT IN ('pending_approval', 'terminated') AND (
(a.adapter_type <> 'paperclip_runner' AND keys && ARRAY['paperclipai/paperclip/paperclip', 'paperclipai/paperclip/complain', 'paperclipai/paperclip/suggestion-box']) OR
EXISTS (SELECT 1 FROM jsonb_array_elements(CASE WHEN jsonb_typeof(a.adapter_config#>'{paperclipSkillSync,desiredSkills}') = 'array'
THEN a.adapter_config#>'{paperclipSkillSync,desiredSkills}' ELSE '[]'::jsonb END) choice
WHERE lower(trim(CASE WHEN jsonb_typeof(choice) = 'string' THEN choice#>>'{}' ELSE choice->>'key' END)) = ANY(keys)))
ORDER BY a.id FOR UPDATE LOOP
PERFORM paperclip_record_agent_runtime(company, candidate);
END LOOP;
IF TG_OP = 'DELETE' THEN RETURN OLD; END IF;
RETURN NEW;
END;
$$;
--> statement-breakpoint
DROP TRIGGER IF EXISTS paperclip_skill_runtime_changed ON company_skills;
CREATE CONSTRAINT TRIGGER paperclip_skill_runtime_changed AFTER INSERT OR UPDATE OR DELETE ON company_skills
DEFERRABLE INITIALLY IMMEDIATE
FOR EACH ROW EXECUTE FUNCTION paperclip_skill_runtime_changed();
--> statement-breakpoint
DROP TRIGGER IF EXISTS paperclip_skill_policy_runtime_changed ON company_skill_policies;
CREATE TRIGGER paperclip_skill_policy_runtime_changed AFTER INSERT OR UPDATE OR DELETE ON company_skill_policies
FOR EACH ROW EXECUTE FUNCTION paperclip_company_agent_runtimes_changed();
--> statement-breakpoint
DROP TRIGGER IF EXISTS paperclip_skill_source_runtime_changed ON project_skill_sources;
CREATE TRIGGER paperclip_skill_source_runtime_changed AFTER INSERT OR UPDATE OR DELETE ON project_skill_sources
FOR EACH ROW EXECUTE FUNCTION paperclip_skill_access_runtime_changed('project_id', 'source_id', 'active');
--> statement-breakpoint
DROP TRIGGER IF EXISTS paperclip_skill_source_access_runtime_changed ON company_skill_sources;
CREATE TRIGGER paperclip_skill_source_access_runtime_changed AFTER INSERT OR UPDATE OR DELETE ON company_skill_sources
FOR EACH ROW EXECUTE FUNCTION paperclip_skill_access_runtime_changed('access_scope', 'removed_at');
--> statement-breakpoint
DROP TRIGGER IF EXISTS paperclip_project_access_runtime_changed ON project_access_members;
CREATE TRIGGER paperclip_project_access_runtime_changed AFTER INSERT OR UPDATE OR DELETE ON project_access_members
FOR EACH ROW EXECUTE FUNCTION paperclip_skill_access_runtime_changed();
--> statement-breakpoint
DROP TRIGGER IF EXISTS paperclip_project_privacy_runtime_changed ON projects;
CREATE TRIGGER paperclip_project_privacy_runtime_changed AFTER UPDATE ON projects
FOR EACH ROW EXECUTE FUNCTION paperclip_skill_access_runtime_changed('visibility', 'privacy_owner_user_id', 'personal_owner_user_id', 'execution_workspace_policy');
--> statement-breakpoint
CREATE OR REPLACE FUNCTION paperclip_skill_pins_runtime_changed() RETURNS trigger
LANGUAGE plpgsql SET search_path = public, pg_temp AS $$
DECLARE company uuid;
BEGIN
IF coalesce(OLD.experimental->'enableBetaSkills', 'false'::jsonb) IS DISTINCT FROM coalesce(NEW.experimental->'enableBetaSkills', 'false'::jsonb) THEN
FOR company IN SELECT DISTINCT company_id FROM agents ORDER BY company_id LOOP
PERFORM paperclip_invalidate_company_agent_runtimes(company);
END LOOP;
END IF;
RETURN NEW;
END;
$$;
--> statement-breakpoint
DROP TRIGGER IF EXISTS paperclip_skill_pins_runtime_changed ON instance_settings;
CREATE TRIGGER paperclip_skill_pins_runtime_changed AFTER UPDATE ON instance_settings
FOR EACH ROW EXECUTE FUNCTION paperclip_skill_pins_runtime_changed();