mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-30 21:49:18 +01:00
## Thinking Path > - Paperclip manages agents and their execution environments. > - Plugins can vendor company skills and provide environments. > - Environment tools need their instructions before the provider starts. > - Persistent computers also need those skills before any run begins. > - This change adds environment requirements to normal skill selection and lifecycle synchronization. > - Plugin names prevent ambiguity, and required skills take delivery precedence without deleting company data. ## Linked Issues or Issue Description **Problem or motivation** An environment can provide desktop tools while its agent receives no instructions for those tools. A first-run-only import also leaves idle computers without required skills. Same-name company skills can make provider discovery ambiguous. **Proposed solution** Let plugin environment drivers declare their managed `skillKeys`. Install missing requirements during lifecycle reconciliation or new-run preparation. Deliver updates through the durable runtime event journal. Give managed imports stable plugin names and report shadowed assignments. **Alternatives considered** ROM links do not make a skill a normal Paperclip selection. Polling duplicates the existing lifecycle delivery queue. Replacing a same-name company record would destroy independent data and assignments. **Roadmap alignment** This extends the shipped Plugin system and Skills Manager. The existing related-PR search found no duplicate contract. ## What Changed - Validate `environmentDrivers[].skillKeys` as unique references to the same plugin's managed skills. Require a ready, enabled plugin with `skills.managed` authority. - Resolve company-scoped skills before version capture. Preserve explicit pins, company edits, and saved recovery selections. Serialize concurrent first installs. - Allow lifecycle callers to request missing-skill installation through `agents.getRuntime`. Ordinary task and recovery reads do not install new declarations. Pass task-selected environment IDs through the SDK. - Add SQL migration `0378_environment_skill_runtime_hooks`. Required skill edits, effective default changes, environment changes, and plugin requirement changes enqueue durable runtime events. Reuse lifecycle delivery and retries; add no polling. - Import managed defaults as `plugin-<short-plugin-id>-<skill-slug>`, including the SKILL.md name. Preserve canonical keys. Keep same-name company records separate without counter suffixes. - Required environment skills win delivery over same-name company assignments. Preserve saved assignments and library content. Record shadowed and winning keys in run warnings, run snapshots, and plugin runtime state. Removing the environment requirement restores the ordinary assignment. - Document the contract for `driver: "plugin"`. Other drivers and plugins without requirements keep existing behavior. Existing managed copies retain edits and names until reset. ## Verification - Latest targeted local checks: environment skills (5), managed skills (9), heartbeat execution configuration (27), and SDK worker RPC host (23): **65 tests passed** (including the targeted schema-drift check). Disposable PostgreSQL exercised the migration, first install before any run, concurrent lifecycle reads, implicit edit events, inherited defaults, rollback, plugin requirement changes, shadowing, preserved records, version capture, and recovery. - Review follow-up also verifies case-insensitive/runtime-name aliases, recovery of a task-mentioned shadowed skill, preserved quoted/multiline YAML, and zero runtime events for repeated local-folder settings writes. - Migration numbering and safety checks passed. Whitespace and secret/PII scans passed. - Earlier targeted manifest validation passed (43 tests). No validator behavior changed in this follow-up. - Broad tests, typecheck, build and end-to-end verification passed in [GitHub CI](https://github.com/paperclipai/paperclip/actions/runs/40387647219) for current head `ce57f2a4d`. Greptile is 5/5 with no unresolved threads. The [commitperclip review check](https://github.com/paperclipai/paperclip/actions/runs/40387646208) also passed after restoring its required duplicate-search checkbox. - Earlier attempts found a SQL alias collision, an unparenthesized CASE expression, and fixture assumptions about bundled catalog rows and YAML quoting. These were corrected before the passing checks. The first broad CI run also exposed a schema-drift check that assumed every journal entry had a schema snapshot; it now reads the latest actual snapshot, so SQL-only migrations need no invented snapshot. A test callback replacement also affected an unrelated fixture; it was restored before the final passing execution-configuration suite. Sandbox runs could not start PostgreSQL; the reported database tests used local socket access. ## Risks - Apply migration 0378 before using the updated lifecycle contract. It adds trigger functions and event producers; it does not rewrite agent assignments or skill content. - Required environment skills take precedence over same-name assignments, including task-selected skills. Both library records remain. Run warnings identify omissions. - Company edits are preserved. Plugin upgrades do not reset installed content or rename existing copies. Explicit reset adopts current defaults. - Namespaced library slugs can be shared by distinct canonical records. Callers must use IDs or canonical keys when a slug is ambiguous. - No live provider request or deployment was performed. The downstream plugin must be built against this Core revision. Persistent-home synchronization retains its existing edit-conflict and retry behavior. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, and code execution. The exact model identifier and context-window size are not exposed in this task context. ## Checklist - [x] Thinking path and issue description included - [x] Model and available capabilities recorded - [x] Roadmap and related PRs checked - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] Public references only; no internal issue IDs or instance URLs - [x] Targeted local tests pass; broad verification delegated to GitHub CI - [x] New behavior covered and documentation updated - [x] Current-head CI green and Greptile 5/5 with no unresolved comments --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
112 lines
7.4 KiB
PL/PgSQL
112 lines
7.4 KiB
PL/PgSQL
-- Environment requirements use the same durable runtime journal as explicit skill assignments.
|
|
CREATE OR REPLACE FUNCTION paperclip_environment_selects_runtime_skill(p_company uuid, p_environment uuid, p_skill uuid) RETURNS boolean
|
|
LANGUAGE sql STABLE SET search_path = public, pg_temp AS $$
|
|
SELECT EXISTS (
|
|
SELECT 1 FROM environments e
|
|
JOIN plugins p ON p.plugin_key = e.config->>'pluginKey'
|
|
JOIN plugin_managed_resources r ON r.plugin_id = p.id AND r.company_id = p_company
|
|
AND r.resource_kind = 'skill' AND r.resource_id = p_skill
|
|
CROSS JOIN LATERAL jsonb_array_elements(coalesce(p.manifest_json->'environmentDrivers', '[]'::jsonb)) AS declared_driver(value)
|
|
WHERE e.id = coalesce(p_environment, (SELECT default_environment_id FROM instance_settings LIMIT 1))
|
|
AND e.driver = 'plugin' AND declared_driver.value->>'driverKey' = e.config->>'driverKey'
|
|
AND coalesce(declared_driver.value->'skillKeys', '[]'::jsonb) ? r.resource_key);
|
|
$$;
|
|
--> statement-breakpoint
|
|
CREATE OR REPLACE FUNCTION paperclip_invalidate_skill_source_runtimes(p_company uuid, sources uuid[]) RETURNS void
|
|
LANGUAGE plpgsql SET search_path = public, pg_temp AS $$
|
|
DECLARE candidate uuid;
|
|
BEGIN
|
|
FOR candidate IN SELECT a.id FROM agents a WHERE a.company_id = p_company
|
|
AND a.status NOT IN ('pending_approval', 'terminated') AND EXISTS (
|
|
SELECT 1 FROM company_skills s WHERE s.company_id = p_company AND s.access_source_id = ANY(sources)
|
|
AND (paperclip_agent_selects_runtime_skill(a.adapter_config, a.adapter_type, s.key, s.slug)
|
|
OR paperclip_environment_selects_runtime_skill(p_company, a.default_environment_id, s.id)))
|
|
ORDER BY a.id FOR NO KEY UPDATE LOOP
|
|
PERFORM paperclip_record_agent_runtime(p_company, candidate);
|
|
END LOOP;
|
|
END;
|
|
$$;
|
|
--> statement-breakpoint
|
|
CREATE OR REPLACE FUNCTION paperclip_skill_runtime_changed() RETURNS trigger
|
|
LANGUAGE plpgsql SET search_path = public, pg_temp AS $$
|
|
DECLARE previous jsonb; current jsonb; keys text[]; company uuid; candidate uuid;
|
|
BEGIN
|
|
IF TG_OP <> 'INSERT' THEN previous := to_jsonb(OLD); END IF;
|
|
IF TG_OP <> 'DELETE' THEN current := to_jsonb(NEW); END IF;
|
|
IF TG_OP = 'UPDATE' AND
|
|
jsonb_build_array(OLD.key, OLD.slug, OLD.markdown, OLD.source_type, OLD.source_locator, OLD.source_ref,
|
|
OLD.file_inventory, OLD.current_version_id, OLD.access_source_id, OLD.compatibility, OLD.trust_level, OLD.metadata)
|
|
IS NOT DISTINCT FROM
|
|
jsonb_build_array(NEW.key, NEW.slug, NEW.markdown, NEW.source_type, NEW.source_locator, NEW.source_ref,
|
|
NEW.file_inventory, NEW.current_version_id, NEW.access_source_id, NEW.compatibility, NEW.trust_level, NEW.metadata)
|
|
THEN RETURN NEW; END IF;
|
|
company := coalesce((current->>'company_id')::uuid, (previous->>'company_id')::uuid);
|
|
SELECT array_agg(DISTINCT lower(reference)) INTO keys FROM (
|
|
SELECT row->>'key' AS reference FROM unnest(ARRAY[previous, current]) row
|
|
UNION ALL SELECT regexp_replace(row->>'key', '.*/', '') FROM unnest(ARRAY[previous, current]) row
|
|
UNION ALL SELECT CASE WHEN row->>'key' LIKE 'paperclipai/paperclip/%' THEN row->>'slug'
|
|
ELSE (row->>'slug') || '--' || left(encode(sha256(convert_to(row->>'key', 'UTF8')), 'hex'), 10) END
|
|
FROM unnest(ARRAY[previous, current]) row
|
|
) refs WHERE reference IS NOT NULL;
|
|
FOR candidate IN SELECT a.id FROM agents a WHERE a.company_id = company
|
|
AND a.status NOT IN ('pending_approval', 'terminated') AND (
|
|
(a.adapter_type <> 'paperclip_runner' AND keys && ARRAY['paperclipai/paperclip/paperclip', 'paperclipai/paperclip/complain', 'paperclipai/paperclip/suggestion-box']) OR
|
|
EXISTS (SELECT 1 FROM jsonb_array_elements(CASE WHEN jsonb_typeof(a.adapter_config#>'{paperclipSkillSync,desiredSkills}') = 'array'
|
|
THEN a.adapter_config#>'{paperclipSkillSync,desiredSkills}' ELSE '[]'::jsonb END) choice
|
|
WHERE lower(trim(CASE WHEN jsonb_typeof(choice) = 'string' THEN choice#>>'{}' ELSE choice->>'key' END)) = ANY(keys))
|
|
OR paperclip_environment_selects_runtime_skill(company, a.default_environment_id, coalesce((current->>'id')::uuid, (previous->>'id')::uuid)))
|
|
ORDER BY a.id FOR NO KEY UPDATE LOOP
|
|
PERFORM paperclip_record_agent_runtime(company, candidate);
|
|
END LOOP;
|
|
IF TG_OP = 'DELETE' THEN RETURN OLD; END IF;
|
|
RETURN NEW;
|
|
END;
|
|
$$;
|
|
--> statement-breakpoint
|
|
CREATE OR REPLACE FUNCTION paperclip_environment_runtimes_changed() RETURNS trigger
|
|
LANGUAGE plpgsql SET search_path = public, pg_temp AS $$
|
|
DECLARE previous jsonb; current jsonb; candidate record;
|
|
BEGIN
|
|
IF TG_OP <> 'INSERT' THEN previous := to_jsonb(OLD); END IF;
|
|
IF TG_OP <> 'DELETE' THEN current := to_jsonb(NEW); END IF;
|
|
IF TG_TABLE_NAME = 'plugin_company_settings' THEN
|
|
IF coalesce((previous->>'enabled')::boolean, true) IS NOT DISTINCT FROM coalesce((current->>'enabled')::boolean, true) THEN
|
|
IF TG_OP = 'DELETE' THEN RETURN OLD; END IF;
|
|
RETURN NEW;
|
|
END IF;
|
|
ELSIF TG_OP = 'UPDATE' AND (CASE TG_TABLE_NAME
|
|
WHEN 'environments' THEN previous->'config' IS NOT DISTINCT FROM current->'config' AND previous->'driver' IS NOT DISTINCT FROM current->'driver'
|
|
WHEN 'plugins' THEN previous->'manifest_json' IS NOT DISTINCT FROM current->'manifest_json' AND previous->'status' IS NOT DISTINCT FROM current->'status'
|
|
ELSE previous->'default_environment_id' IS NOT DISTINCT FROM current->'default_environment_id' END) THEN RETURN NEW; END IF;
|
|
FOR candidate IN SELECT a.id, a.company_id FROM agents a
|
|
WHERE a.status NOT IN ('pending_approval', 'terminated') AND (CASE TG_TABLE_NAME
|
|
WHEN 'environments' THEN coalesce(a.default_environment_id, (SELECT default_environment_id FROM instance_settings LIMIT 1))
|
|
= coalesce((current->>'id')::uuid, (previous->>'id')::uuid)
|
|
WHEN 'plugins' THEN EXISTS (SELECT 1 FROM environments e
|
|
WHERE e.id = coalesce(a.default_environment_id, (SELECT default_environment_id FROM instance_settings LIMIT 1))
|
|
AND e.driver = 'plugin' AND e.config->>'pluginKey' = coalesce(current->>'plugin_key', previous->>'plugin_key'))
|
|
WHEN 'plugin_company_settings' THEN a.company_id = coalesce((current->>'company_id')::uuid, (previous->>'company_id')::uuid)
|
|
AND EXISTS (SELECT 1 FROM environments e JOIN plugins p ON p.plugin_key = e.config->>'pluginKey'
|
|
WHERE e.id = coalesce(a.default_environment_id, (SELECT default_environment_id FROM instance_settings LIMIT 1))
|
|
AND e.driver = 'plugin' AND p.id = coalesce((current->>'plugin_id')::uuid, (previous->>'plugin_id')::uuid))
|
|
ELSE a.default_environment_id IS NULL END)
|
|
ORDER BY a.company_id, a.id FOR NO KEY UPDATE OF a LOOP
|
|
PERFORM paperclip_record_agent_runtime(candidate.company_id, candidate.id);
|
|
END LOOP;
|
|
IF TG_OP = 'DELETE' THEN RETURN OLD; END IF;
|
|
RETURN NEW;
|
|
END;
|
|
$$;
|
|
--> statement-breakpoint
|
|
CREATE TRIGGER paperclip_environment_runtime_changed AFTER INSERT OR UPDATE ON environments
|
|
FOR EACH ROW EXECUTE FUNCTION paperclip_environment_runtimes_changed();
|
|
--> statement-breakpoint
|
|
CREATE TRIGGER paperclip_plugin_environment_runtime_changed AFTER UPDATE ON plugins
|
|
FOR EACH ROW EXECUTE FUNCTION paperclip_environment_runtimes_changed();
|
|
--> statement-breakpoint
|
|
CREATE TRIGGER paperclip_plugin_company_environment_runtime_changed AFTER INSERT OR UPDATE OR DELETE ON plugin_company_settings
|
|
FOR EACH ROW EXECUTE FUNCTION paperclip_environment_runtimes_changed();
|
|
--> statement-breakpoint
|
|
CREATE TRIGGER paperclip_default_environment_runtime_changed AFTER UPDATE ON instance_settings
|
|
FOR EACH ROW EXECUTE FUNCTION paperclip_environment_runtimes_changed();
|