Files
PaperClipAI/packages/db/src/migrations/0354_agent_connection_assignment_hooks.sql
T
Nicky LeachandPaperclip 5f5602bba7 feat(plugins): synchronize standing connection assignments (#16603)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Lifecycle plugins prepare agent runtimes outside the control plane.
> - They need committed connection assignment changes without polling.
> - A reader and its revision hooks must cover the same fields.
> - This pull request adds a bounded standing-assignment snapshot and
its matching hooks.
> - Plugins can react to assignment changes while execution keeps its
existing access checks.

## Linked Issues or Issue Description

**What happened?**

The agent runtime reader does not expose standing connection
assignments. Changes to connection installs, active profiles, and
connector feature switches do not request synchronization. Adding
company-wide invalidation also exposes lock inversions in assignment
writers and AI adoption.

**Expected behavior**

The owning lifecycle plugin can read a bounded assignment snapshot.
Every committed change to its stored fields advances the runtime
revision. Rollback removes both the change and its event. Assignment
writers finish without a lock-order cycle.

**Steps to reproduce**

1. Install a connection or bind an active profile to an agent.
2. Read its runtime inputs through the owning lifecycle plugin.
3. Rename or disable the connection, revoke the assignment, or switch
connector features.
4. Expect a committed runtime event and an updated snapshot without
periodic polling.

**Additional context**

Extracted from #16503. Catalog publication #16602 and the separate test
maintenance PRs #16601 and #16610 are merged. This PR now targets master
and is the active review. Tool-definition hooks #16604 and
access/credential hooks #16605 remain drafts without review requests.

## What Changed

- Add SDK types and an owning-plugin runtime snapshot for installed
connections, active profile IDs, and effective connector feature
switches.
- Reject more than 512 connections or 128 active profiles instead of
returning partial data.
- Add migration 0354 for the exact stored fields used by the snapshot,
with transaction-coalesced invalidation. Agent-targeted bindings and
installs notify only their old/new target agents;
project/routine/issue/gateway bindings are outside the snapshot and
produce no notification.
- Use ordered agent locks that are compatible with foreign-key
references and exclude runtime readers. Prelock complete old/new target
sets before replacing assignments or restoring managed profiles.
- Record one company baseline at delivery startup for process-managed
feature overlays.
- Lock assignment inputs before install saves, task completion, and
email assignment. Setup finalization reads the locked connection state
and prelocks company agents before activating, re-enabling, or reviving
a profile.
- Make profile binding and reassignment writes transactional and lock
parent profiles first. Keep target validation and profile/detail reads
on the reserved transaction connection, including with a one-connection
pool.
- Lock company agents in order before validated AI adoption. Keep
adoption and shared configuration writes at `FOR NO KEY UPDATE`,
avoiding an upgrade that can deadlock foreign-key inserts during gateway
creation. Configuration fields cannot change agent identity; lifecycle
transitions and deletion retain their existing locks.
- Document supported coverage and the inputs reserved for later PRs.

## Verification

- Thirteen affected suites passed: 757 tests covering runtime
hooks/assignments/skills, concurrent skill publication, tool
access/catalog, task connection intents, lifecycle delivery, email, AI
accounts, migration histories, snapshot drift, and the agent lifecycle
module. This includes the merged catalog PR's OAuth and
overlapping-refresh regressions.
- The assignment suite passes all 24 cases, including bounded snapshots,
company/agent isolation, rollback, health-field no-ops, process-start
feature overlays, and real lock contention.
- Four one-connection PostgreSQL regressions reproduce the earlier
helper-read hangs before the correction; bind, unbind, delete, and
reassign complete with it. The earlier adoption/gateway probe exposed
both lock upgrades: changing only validation still deadlocks in shared
configuration. Current scoped coverage verifies adoption with
agent-targeted binding writes; gateway-only bindings no longer
invalidate this snapshot.
- The original ordered-adoption regression reproduces PostgreSQL `40P01`
without ordered company-agent locking. Scoped-recipient tests reproduce
unrelated-peer journaling before the trigger correction and verify
repeated bind/unbind operations leave peers untouched, ignored binding
types produce no events, both reassignment recipients are reached, and
coalescing remains intact.
- Six replacement contention cases cover installs, existing setup,
activation, re-enabling, profile revival, and profile reassignment.
Negative controls reproduce `40P01` without the corresponding
target/company prelocks; all cases pass with the guards. Activation
decisions use the connection row reread under lock.
- An initial contention fixture failed because its settings connection
lacked an MCP URL. Supplying the fixture URL and controlled lookup
corrected that test setup error.
- Server typecheck, database migration numbering/safety, five-snapshot
retention, feature-map structural validation, and `git diff --check`
passed. Migration metadata preserves historical journal entries and adds
no schema change. The complete diff was reviewed for secrets and
personal data before push.
- Earlier hosted PR CI runs 38983440861 and 38985530069 passed; the
superseded promotion run 38983440470 was cancelled. Superagent
acknowledged the recipient fix and accepted the documented company-wide
cost at `359b5b2f30a8a0248177367e458663d5379810d1`. Its two standard
Greptile summaries were conditional on a later setup correction and were
not counted as independent acceptance; the setup correction is now
included here.
- At `7da246aab46274db1a120a0546abbb404c5427c7`, hosted PR run
38986958988 passed: all 55 checks succeeded, with two optional Storybook
checks skipped. Superagent passed, all review threads are resolved, and
standard Greptile independently returned 5/5 with no blocking findings
(review completed 2026-10-17 00:40 UTC). Apex independently returned 5/5
with no actionable findings (completed 2026-10-17 00:50 UTC). CI was
rechecked after Apex and remains green; the branch is mergeable against
master. Full local build and repository-wide tests were not run.

## Risks

Migration 0354 adds database functions and triggers without changing
table schema. Apply it through the normal migration flow. Agent-targeted
bindings and installs only notify their affected agents. Explicit
company-wide assignments, connection metadata, profile
activation/removal, and connector feature switches retain synchronous,
transaction-atomic company fanout. Each such transaction takes ordered
row locks and inserts up to one connector runtime event per eligible
company agent: O(number of eligible agents) work and journal storage,
with no company-size cap or cross-transaction mutation quota. This
remains an intentional scope tradeoff for this PR; deferred fanout is
not implemented. Many of these conservative changes can request
preparation even if an individual snapshot stays unchanged. Snapshot
limits fail closed. This PR provides desired assignments, not
credentials or task authorization. Later PRs must add mutation coverage
before consumers can rely on immediate synchronization of tool
definitions or credentials.

## Model Used

OpenAI Codex, GPT-6, with reasoning, repository inspection, code
execution, and testing. The exact deployment model ID and context window
are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-16 17:55:47 -07:00

185 lines
11 KiB
PL/PgSQL

CREATE OR REPLACE FUNCTION paperclip_record_agent_runtime(p_company uuid, p_agent uuid) RETURNS bigint
LANGUAGE plpgsql SET search_path = public, pg_temp AS $$
DECLARE revision bigint;
BEGIN
PERFORM 1 FROM agents WHERE company_id = p_company AND id = p_agent
AND status NOT IN ('pending_approval', 'terminated') FOR NO KEY UPDATE;
IF FOUND THEN
INSERT INTO resource_lifecycle_events(company_id, resource_type, resource_id, action)
VALUES (p_company, 'agent_runtime', p_agent, 'update') RETURNING id INTO revision;
END IF;
RETURN revision;
END;
$$;
--> statement-breakpoint
CREATE OR REPLACE FUNCTION paperclip_invalidate_company_agent_runtimes(p_company uuid) RETURNS void
LANGUAGE plpgsql SET search_path = public, pg_temp AS $$
DECLARE candidate uuid;
BEGIN
FOR candidate IN SELECT id FROM agents WHERE company_id = p_company
AND status NOT IN ('pending_approval', 'terminated') ORDER BY id FOR NO KEY UPDATE LOOP
PERFORM paperclip_record_agent_runtime(p_company, candidate);
END LOOP;
END;
$$;
--> statement-breakpoint
CREATE OR REPLACE FUNCTION paperclip_invalidate_skill_source_runtimes(p_company uuid, sources uuid[]) RETURNS void
LANGUAGE plpgsql SET search_path = public, pg_temp AS $$
DECLARE candidate uuid;
BEGIN
FOR candidate IN SELECT a.id FROM agents a WHERE a.company_id = p_company
AND a.status NOT IN ('pending_approval', 'terminated') AND EXISTS (
SELECT 1 FROM company_skills s WHERE s.company_id = p_company AND s.access_source_id = ANY(sources)
AND paperclip_agent_selects_runtime_skill(a.adapter_config, a.adapter_type, s.key, s.slug))
ORDER BY a.id FOR NO KEY UPDATE LOOP
PERFORM paperclip_record_agent_runtime(p_company, candidate);
END LOOP;
END;
$$;
--> statement-breakpoint
CREATE OR REPLACE FUNCTION paperclip_skill_runtime_changed() RETURNS trigger
LANGUAGE plpgsql SET search_path = public, pg_temp AS $$
DECLARE previous jsonb; current jsonb; keys text[]; company uuid; candidate uuid;
BEGIN
IF TG_OP <> 'INSERT' THEN previous := to_jsonb(OLD); END IF;
IF TG_OP <> 'DELETE' THEN current := to_jsonb(NEW); END IF;
IF TG_OP = 'UPDATE' AND
jsonb_build_array(OLD.key, OLD.slug, OLD.markdown, OLD.source_type, OLD.source_locator, OLD.source_ref,
OLD.file_inventory, OLD.current_version_id, OLD.access_source_id, OLD.compatibility, OLD.trust_level, OLD.metadata)
IS NOT DISTINCT FROM
jsonb_build_array(NEW.key, NEW.slug, NEW.markdown, NEW.source_type, NEW.source_locator, NEW.source_ref,
NEW.file_inventory, NEW.current_version_id, NEW.access_source_id, NEW.compatibility, NEW.trust_level, NEW.metadata)
THEN RETURN NEW; END IF;
company := coalesce((current->>'company_id')::uuid, (previous->>'company_id')::uuid);
SELECT array_agg(DISTINCT lower(reference)) INTO keys FROM (
SELECT row->>'key' AS reference FROM unnest(ARRAY[previous, current]) row
UNION ALL SELECT regexp_replace(row->>'key', '.*/', '') FROM unnest(ARRAY[previous, current]) row
UNION ALL SELECT CASE WHEN row->>'key' LIKE 'paperclipai/paperclip/%' THEN row->>'slug'
ELSE (row->>'slug') || '--' || left(encode(sha256(convert_to(row->>'key', 'UTF8')), 'hex'), 10) END
FROM unnest(ARRAY[previous, current]) row
) refs WHERE reference IS NOT NULL;
FOR candidate IN SELECT a.id FROM agents a WHERE a.company_id = company
AND a.status NOT IN ('pending_approval', 'terminated') AND (
(a.adapter_type <> 'paperclip_runner' AND keys && ARRAY['paperclipai/paperclip/paperclip', 'paperclipai/paperclip/complain', 'paperclipai/paperclip/suggestion-box']) OR
EXISTS (SELECT 1 FROM jsonb_array_elements(CASE WHEN jsonb_typeof(a.adapter_config#>'{paperclipSkillSync,desiredSkills}') = 'array'
THEN a.adapter_config#>'{paperclipSkillSync,desiredSkills}' ELSE '[]'::jsonb END) choice
WHERE lower(trim(CASE WHEN jsonb_typeof(choice) = 'string' THEN choice#>>'{}' ELSE choice->>'key' END)) = ANY(keys)))
ORDER BY a.id FOR NO KEY UPDATE LOOP
PERFORM paperclip_record_agent_runtime(company, candidate);
END LOOP;
IF TG_OP = 'DELETE' THEN RETURN OLD; END IF;
RETURN NEW;
END;
$$;
--> statement-breakpoint
CREATE OR REPLACE FUNCTION paperclip_invalidate_connector_agent(p_company uuid, p_agent uuid) RETURNS void
LANGUAGE plpgsql SET search_path = public, pg_temp AS $$
DECLARE setting text := 'paperclip.runtime_agent_' || replace(p_agent::text, '-', '_'); xid text := pg_current_xact_id()::text;
BEGIN
IF current_setting('paperclip.runtime_' || replace(p_company::text, '-', '_'), true) IS NOT DISTINCT FROM xid
OR current_setting(setting, true) IS NOT DISTINCT FROM xid THEN RETURN; END IF;
PERFORM paperclip_record_agent_runtime(p_company, p_agent);
PERFORM set_config(setting, xid, true);
END;
$$;
--> statement-breakpoint
CREATE OR REPLACE FUNCTION paperclip_invalidate_connector_company(p_company uuid) RETURNS void
LANGUAGE plpgsql SET search_path = public, pg_temp AS $$
DECLARE setting text := 'paperclip.runtime_' || replace(p_company::text, '-', '_'); xid text := pg_current_xact_id()::text; candidate uuid;
BEGIN
IF current_setting(setting, true) IS NOT DISTINCT FROM xid THEN RETURN; END IF;
FOR candidate IN SELECT id FROM agents WHERE company_id = p_company
AND status NOT IN ('pending_approval', 'terminated') ORDER BY id FOR NO KEY UPDATE LOOP
PERFORM paperclip_invalidate_connector_agent(p_company, candidate);
END LOOP;
PERFORM set_config(setting, xid, true);
END;
$$;
--> statement-breakpoint
CREATE OR REPLACE FUNCTION paperclip_connector_runtime_changed() RETURNS trigger
LANGUAGE plpgsql SET search_path = public, pg_temp AS $$
DECLARE company uuid; query text; old_projection text; new_projection text; target record; candidate uuid;
BEGIN
IF TG_TABLE_NAME = 'tool_profiles' AND TG_OP = 'INSERT' THEN RETURN NULL; END IF;
IF TG_OP = 'UPDATE' THEN
SELECT 'jsonb_build_array(' || string_agg(format('to_jsonb(o)->%L', field), ',') || ')',
'jsonb_build_array(' || string_agg(format('to_jsonb(n)->%L', field), ',') || ')'
INTO old_projection, new_projection FROM unnest(TG_ARGV[1:TG_NARGS - 1]) field;
END IF;
IF TG_TABLE_NAME IN ('tool_profile_bindings', 'tool_connection_installs') THEN
IF TG_OP = 'INSERT' THEN query := 'SELECT company_id, target_type, target_id FROM new_rows';
ELSIF TG_OP = 'DELETE' THEN query := 'SELECT company_id, target_type, target_id FROM old_rows';
ELSE query := format('WITH changed AS (
SELECT o.company_id AS old_company, o.target_type AS old_type, o.target_id AS old_target,
n.company_id AS new_company, n.target_type AS new_type, n.target_id AS new_target
FROM old_rows o FULL JOIN new_rows n ON o.%I = n.%I WHERE %s IS DISTINCT FROM %s)
SELECT old_company AS company_id, old_type AS target_type, old_target AS target_id FROM changed WHERE old_company IS NOT NULL UNION
SELECT new_company AS company_id, new_type AS target_type, new_target AS target_id FROM changed WHERE new_company IS NOT NULL',
TG_ARGV[0], TG_ARGV[0], old_projection, new_projection);
END IF;
FOR target IN EXECUTE 'SELECT DISTINCT company_id, target_type, target_id FROM (' || query || ') targets
WHERE target_type IN (''agent'', ''company'')
ORDER BY company_id, target_type DESC, target_id' LOOP
IF target.target_type = 'company' AND target.target_id = target.company_id::text THEN
PERFORM paperclip_invalidate_connector_company(target.company_id);
ELSIF target.target_type = 'agent' AND target.target_id ~ '^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$' THEN
SELECT id INTO candidate FROM agents WHERE company_id = target.company_id AND id = target.target_id::uuid;
IF FOUND THEN PERFORM paperclip_invalidate_connector_agent(target.company_id, candidate); END IF;
END IF;
END LOOP;
RETURN NULL;
END IF;
IF TG_OP = 'INSERT' THEN query := 'SELECT DISTINCT company_id FROM new_rows ORDER BY company_id';
ELSIF TG_OP = 'DELETE' THEN query := 'SELECT DISTINCT company_id FROM old_rows ORDER BY company_id';
ELSE
query := format('WITH changed AS (
SELECT o.company_id AS old_company, n.company_id AS new_company FROM old_rows o FULL JOIN new_rows n ON o.%I = n.%I
WHERE %s IS DISTINCT FROM %s)
SELECT old_company AS company_id FROM changed WHERE old_company IS NOT NULL UNION
SELECT new_company AS company_id FROM changed WHERE new_company IS NOT NULL ORDER BY company_id', TG_ARGV[0], TG_ARGV[0], old_projection, new_projection);
END IF;
FOR company IN EXECUTE query LOOP
PERFORM paperclip_invalidate_connector_company(company);
END LOOP;
RETURN NULL;
END;
$$;
--> statement-breakpoint
DO $$
DECLARE relation text; arguments text;
BEGIN
FOREACH relation IN ARRAY ARRAY['tool_connections', 'tool_connection_installs', 'tool_profiles', 'tool_profile_bindings'] LOOP
arguments := CASE relation
WHEN 'tool_connections' THEN '''id'', ''company_id'', ''name'', ''transport'', ''enabled'', ''status'''
WHEN 'tool_connection_installs' THEN '''id'', ''company_id'', ''connection_id'', ''target_type'', ''target_id'''
WHEN 'tool_profiles' THEN '''id'', ''company_id'', ''status'''
ELSE '''id'', ''company_id'', ''profile_id'', ''target_type'', ''target_id''' END;
EXECUTE format('DROP TRIGGER IF EXISTS paperclip_connector_runtime_insert ON %I', relation);
EXECUTE format('DROP TRIGGER IF EXISTS paperclip_connector_runtime_update ON %I', relation);
EXECUTE format('DROP TRIGGER IF EXISTS paperclip_connector_runtime_delete ON %I', relation);
EXECUTE format('CREATE TRIGGER paperclip_connector_runtime_insert AFTER INSERT ON %I REFERENCING NEW TABLE AS new_rows FOR EACH STATEMENT EXECUTE FUNCTION paperclip_connector_runtime_changed(%s)', relation, arguments);
EXECUTE format('CREATE TRIGGER paperclip_connector_runtime_update AFTER UPDATE ON %I REFERENCING OLD TABLE AS old_rows NEW TABLE AS new_rows FOR EACH STATEMENT EXECUTE FUNCTION paperclip_connector_runtime_changed(%s)', relation, arguments);
EXECUTE format('CREATE TRIGGER paperclip_connector_runtime_delete AFTER DELETE ON %I REFERENCING OLD TABLE AS old_rows FOR EACH STATEMENT EXECUTE FUNCTION paperclip_connector_runtime_changed(%s)', relation, arguments);
END LOOP;
END;
$$;
--> statement-breakpoint
CREATE OR REPLACE FUNCTION paperclip_connector_flags_runtime_changed() RETURNS trigger
LANGUAGE plpgsql SET search_path = public, pg_temp AS $$
DECLARE company uuid; previous jsonb := coalesce(to_jsonb(OLD)->'experimental', '{}'::jsonb);
current jsonb := coalesce(to_jsonb(NEW)->'experimental', '{}'::jsonb);
BEGIN
IF jsonb_build_array(coalesce(previous->'enablePublicMcp', 'false'::jsonb), coalesce(previous->'enableChatConnectors', 'false'::jsonb))
IS DISTINCT FROM jsonb_build_array(coalesce(current->'enablePublicMcp', 'false'::jsonb), coalesce(current->'enableChatConnectors', 'false'::jsonb)) THEN
FOR company IN SELECT DISTINCT company_id FROM agents ORDER BY company_id LOOP
PERFORM paperclip_invalidate_connector_company(company);
END LOOP;
END IF;
RETURN NULL;
END;
$$;
--> statement-breakpoint
DROP TRIGGER IF EXISTS paperclip_connector_flags_runtime_changed ON instance_settings;
CREATE TRIGGER paperclip_connector_flags_runtime_changed AFTER INSERT OR UPDATE OR DELETE ON instance_settings
FOR EACH ROW EXECUTE FUNCTION paperclip_connector_flags_runtime_changed();