Nicky LeachandPaperclip
194fc11406
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Lifecycle plugins must refresh runtime preparation when tool definitions change. > - Standing assignments alone do not cover catalogs, profile entries, or policies. > - New journal hooks must preserve the order used by setup finalization. > - This pull request adds tool-definition invalidation and its required writer guards. > - It leaves credentials and provider access changes for the next independent capability. ## Linked Issues or Issue Description **What happened?** Catalog, profile-entry, policy, gateway, and connection-configuration changes do not request agent runtime synchronization. A new profile-entry trigger also exposes a deletion/finalization deadlock if the child row is changed before its parent is locked. **Expected behavior** Committed tool-definition changes advance runtime revisions without polling. Bookkeeping does not. Entry changes and profile replacement use a consistent parent/reference lock order and commit atomically. **Steps to reproduce** 1. Change a catalog descriptor, profile entry, policy, or connection configuration. 2. Expect committed runtime events for the affected agents. 3. Run profile-entry deletion concurrently with setup finalization while both wait for an agent row. 4. Expect both operations to complete without PostgreSQL deadlock 40P01. **Additional context** Extracted from #16503. Catalog publication #16602 and standing assignments #16603 are merged. This PR is rebased onto master and is the active review. It addresses the original Apex profile-entry deletion finding. Credential/access hooks #16605 remain a draft without review requests. ## What Changed - Add migration 0355 for application/connection configuration, catalog, profile-entry, policy, and MCP gateway invalidation. - Preserve transaction coalescing and ignore health bookkeeping and trust-rule usage counters. - Keep credential-only fields outside this migration's coverage. - Preserve merged recipient scoping for bindings and installations. Profile-entry edits resolve old/new standing profile bindings. Policy edits resolve old/new agent selectors with their intersection/list semantics. - Lock old and new profile-entry references before parent profiles and child mutations. Recheck the reference set under the parent lock, retry changed snapshots up to three times, and merge unspecified fields from the locked profile. - Make entry add/edit/delete and profile replacement transactional, including validation and profile reads on the reserved connection. - Lock policy rows in ID order and change priorities in one statement so scoped agent locks are ordered across the full set. - Lock the managed profile before connection removal deletes its entries in the same transaction. - Update assignment tests whose old absence assertions are superseded by expanded configuration coverage. - Document the supported capability and the remaining credential/access boundary. ## Verification - Final head: `809fcb11794bfb12c545af0a2f96ab9bc2b570a2`. It is rebased onto master with merged #16623. GitHub reports no merge conflicts. [CI run 39293925136](https://github.com/paperclipai/paperclip/actions/runs/39293925136) has 55 successful checks and two optional Storybook skips. - Standard Greptile returned an independent 5/5. The final Apex review returned 5/5 on this exact head at 17:47 UTC on 2026-10-19. Its summary explicitly clears this PR for merge independently of draft #16605. The security scan passed. All review threads are resolved. - Two real-PostgreSQL negative controls reproduce `40P01` without the old-reference guard or the concurrent-snapshot check. Both regression tests pass with the final correction. Server typecheck, migration safety and numbering, five-snapshot retention, journal history, feature maps, and diff checks pass. The actual PR diff was scanned for secrets and personal data before push. - The final 15-suite local run passed 821/822 tests. The one failure was in the unchanged runtime-delivery test: a startup callback changed its global callback count during an independent transaction. The test-only correction is isolated in #16871. That PR passes all 17 runtime-hook tests, server types, hosted CI, and standard Greptile 5/5. Its notification-disabled control fails at the independent event-delivery assertion. No changes from #16871 are included here; this feature head passes hosted CI independently. - Earlier affected-suite runs passed 814 tests after rebase and 820 tests after recipient and whitespace corrections. Four affected suites passed 514 tests before the second reference-contention regression was added. Single-connection tests cover entry add, edit, delete, and replacement. Validation and profile reads use the reserved transaction connection. - Review fixes have negative controls: the old code hangs with a one-connection pool; parent-lock omissions deadlock entry deletion or app removal against setup finalization; per-row policy reorder deadlocks under contention; unscoped triggers journal unrelated peers; default SQL trimming misses JavaScript whitespace. Corrected cases pass. Selector tests cover all 25 ECMAScript whitespace characters. Recipient tests cover repeated edits, old/new targets, ignored bindings, rollback, coalescing, and company scope. The definition suite now has 28 cases. - Earlier standard scores that depended on #16605 were not accepted as independent clearance. The first Apex review returned 4/5 for replacement versus application deletion. The final correction locks old and new references, checks the reference set again under the parent lock, and retries before any mutation. The final independent reviews clear that correction. - Preserve prior failures for acceptance: draft run 38975432017 had failures whose causes were not re-investigated. Run 38990400765 exposed the unrelated durable-resume watchdog race, fixed in merged #16623. In run 39287657148, two jobs reported cancellation without a failed-test summary; the cause remains unknown. Both passed on retry without source changes. Initial local fixtures and SQL bindings also needed correction: an internal helper name was replaced with public `archiveConnection`, unchanged priorities were changed to exercise reorder, a driver timestamp was bound as ISO text, and a superseded company-fanout assertion was updated. These recovered checks do not establish a cause for the unexplained CI failures. - Local checks target the affected services and real database boundaries. A full local build and repository-wide test run were not performed. Hosted CI supplies the broader checks. Provider responses in tests are controlled; live provider behavior was not qualified. ## Risks Migration 0355 adds statement triggers without changing table schema. Apply it through the normal migration flow. A tool-definition event can advance a runtime revision without changing the bounded assignment snapshot. Consumers still need authorized readers for definitions. Credential-only changes, grants, secrets, and endpoints remain for the access-hook migration. Profile replacement now rolls back as a unit if any entry fails. Binding and install notifications retain scoped recipients. Profile-entry changes use standing bindings, and policies use old/new agent selectors. Unscoped policies, company-bound profiles, and other conservative definition changes retain synchronous transaction-atomic company-wide fanout: O(eligible company agents) locks and journal storage, without a fanout ceiling or mutation quota. This retains the accepted tradeoff from #16603 for company-wide changes. ## Model Used OpenAI Codex, GPT-6, with reasoning, repository inspection, code execution, and testing. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run local checks; results and the isolated test failure are recorded above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
Paperclip feature map
Start here when reproducing a user-facing bug, verifying a change, or deciding which surfaces a fix must cover. Each recipe describes what a person can do, where they can do it, the expected result, and the evidence needed to verify it. Product behavior is still governed by the implementation spec.
The sibling agent access map records APIs, agent entry points and permission boundaries for the same capability IDs. Follow the shared map contract when maintaining either map.
Browse features
Open the published Storybook guide for the assembled feature and capability index. The Markdown recipes in this directory are the authored source. Each has a same-named agent-access companion.
Run node scripts/storybook-catalog.mjs to generate local navigation in
maps/generated/feature-map.md, maps/generated/agent-access-map.md, and the
Storybook guide. A new recipe is discovered automatically; do not register it in
this README or maintain counts here.
Before driving a journey
- Use this checkout's isolated test drive for manual product checks. It creates a temporary instance and prints its URL and data directory. Do not guess that a server on port 3100 belongs to you.
- Record the commit, URL, company, login role, relevant experimental settings, adapter, runtime mode, and live versus simulated dependencies. Use disposable tasks and provider resources. A fresh test drive creates a company and CEO, but no task or first run.
- Use the current navigation and company prefix. Paths in recipes omit that prefix. Record whether the streamlined or production shell is selected; Agent Chat, chat connectors, and the combined Inbox/Tasks view have separate gates. A hidden surface is an unmet prerequisite, not a successful test.
- Run the smallest relevant test first. Vitest commands below run from the repository root after installing dependencies. Playwright recipes use their named configuration and its isolated environment, not an unrelated running instance. Reserve expensive runner/provider tests for the behavior at issue.
- Exercise the actual user action, inspect its result, reload, and verify the persisted state or continuation. Use read-only API checks to corroborate UI evidence; creating state through the API does not prove the creation UI.
For agent-driven acceptance work, start with the Paperclip verification skill for an isolated launch, ownership/readiness checks, browser driving, evidence, and cleanup. It uses this directory as its feature map. The existing dev-workspace run/verify skill and evaluation skill describe runtime ownership and evidence handling. Reuse them; this map introduces no environment launcher, credentials, scheduled job, or second test framework.
Evidence contract
Report each entry point as passed, failed, blocked (with its missing prerequisite), or not run. Include the feature filename and entry-point ID, commit/environment, user action, expected and observed result, command/exit code, and evidence links. Screenshots or traces should show both the action and the discriminating result. Include task/run/request IDs when relevant, without secrets.
A fix is verified across its affected surfaces only when each has evidence or an explicit reason it does not apply. Shared code alone does not prove parity. Keep product regressions visible; do not change the recipe to bless a failure. For Paperclip-assigned work, attach evidence through the artifact workflow.
Inventory and remaining depth
The generated maps/generated/coverage.json lists the current non-test TSX page
modules and the recipes that directly reference each one. An unreferenced module
is a documentation candidate, not proof that its feature is absent or untested.
Remaining workflow limits live in each recipe. Inventory and example associations
describe documentation scope, not runtime health.
Maintaining the reference
The map documents user journeys. It does not execute them automatically; generated inventories record documentation scope rather than live test health.
When maintaining a recipe, compare its entry points and expected results with
the current source. Check that linked tests still exercise the stated behavior
and that local references still exist. Keep the same-named agent access companion
and stable sub-feature IDs aligned when behavior or access changes. Source-only
access inventory remains distinct from live verification. Run
node scripts/check-maps.mjs after map edits for structural/reference checks;
follow the shared contract for the source and behavioral checks it cannot perform.
Each recipe starts with a small frontmatter navigation record, then an H1:
---
navigation: {"section":"Tasks and conversations","sectionOrder":20,"order":10}
---
The H1 supplies the feature title, and the first paragraph supplies its summary. Orders are local to the section; leave gaps for additions. Equal orders use the stable filename as a tie-breaker. Keep a section's name/order consistent across its recipes. A new feature or capability needs only its recipe and companion.
Optional ### Storybook verification guidance lives inside the recipe's Gotchas
section. Each level-four heading contains a backtick-quoted capability key and
has Scope (ui, cli, or browser),
Storybook, Production, and Sources fields. These are verification instructions
and intentional scope limits, not run results or a required story for every API.
The generated gap report combines this guidance with current story associations.
After frontmatter, each recipe starts with an H1 and a user-visible description, then exactly:
- Sub-features — stable backticked IDs and observable behavior/states.
- How to get to it (user POV) — one H3 backticked entry-point ID per surface.
- Driving it with — starts with
Preconditions:; repeat each entry-point H3, withAutomated:andManual:paragraphs. Name test scope and gaps honestly. - Gotchas — misleading look-alikes, feature gates, and invalid evidence.
New or updated entries follow
Poteto's feature-map examples.
Existing entries with a Driving it heading can adopt the named harness when
their recipes are updated. The original format was inspired by
Omnigent's feature map.
Paperclip's recipes and checks follow its own task model and test infrastructure.