4 Commits

Include renames

Author	SHA1	Message	Date
DottaandPaperclip	9ef434fe79	docs: expose feature map IDs and add a recipe index (#16987) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its feature map describes user journeys and their verification paths. > - The map already defines feature IDs and full capability IDs. > - The README and child pages did not show those IDs consistently. > - Readers need those IDs to connect issues, tests, stories, and evidence. > - This pull request adds a recipe index and displays the existing IDs. > - The benefit is clear references across the product and agent access maps. ## Linked Issues or Issue Description **Issue type** Missing documentation. **Where is the issue?** `maps/feature-map/README.md`, the child recipes, and their agent access companions. **What's wrong?** The README sends readers to Storybook to browse features. Child pages show local keys, such as `installation`, without the full `plugins/installation` ID. Readers must inspect the generator to find the full reference. **Suggested fix** List recipe links and existing IDs in the README. Show each feature ID and full capability ID in its child pages. Explain capability IDs and entry-point references. Keep the current keys and anchors. Related work: Refs #15288. Refs #16052. Refs #15983. No duplicate open PR or issue was found by title search. ## What Changed - Add an index grouped by feature area. It lists every recipe, feature ID, and full capability ID. - Explain how IDs are formed and used in issues, tests, Storybook, and evidence. - Add feature IDs to all product recipes and agent access companions. - Show full capability IDs beside recipe keys and below access and Storybook guidance headings. - Preserve existing keys, heading anchors, and generated ID values. Keep the latest upstream company capability in the index. ## Verification - `node scripts/check-maps.mjs` passes for 34 feature families and 175 capability IDs. - `node --test scripts/check-maps.test.mjs scripts/storybook-feature-index.test.mjs` passes all 17 tests. - `node scripts/storybook-feature-index.mjs` passes on this checkout and generates the feature indexes and overview inputs. - `git diff --check` passes. - A source comparison confirms that the README matches the recipe index. After removal of the added ID labels, all 68 child pages match the upstream prose and headings. - Full workspace checks were attempted: `pnpm -r typecheck`, `pnpm test:run`, and `pnpm build`. They could not start because the local pnpm wrapper could not fetch and verify the pinned pnpm 9.15.4 release. This checkout also has no workspace dependencies installed. - `node scripts/storybook-catalog.mjs` could not run because `storybook/internal/csf-tools` is not installed. The direct map and index checks above pass without that dependency. - Remote typecheck, release-registry, and full build checks passed. The first attempts of [server shard 3/12](https://github.com/paperclipai/paperclip/actions/runs/39460473910/job/118758430448) and [server shard 12/12](https://github.com/paperclipai/paperclip/actions/runs/39460473910/job/118758429932) stopped after their runners received a shutdown signal. The test steps were canceled. GitHub rejects the rerun request with HTTP 403: "The workflow run containing this job is already running". Five other CI jobs remain active. The next action is to rerun the two interrupted shards after workflow 39460473910 ends. No application changes were made to address these runner interruptions. - To inspect the result, open the feature-map README, follow a recipe link, and compare its full capability IDs with its access companion. ## Risks - Low risk. The change updates map documentation only. It changes no application code or ID schema. - The readable README index and child labels repeat source IDs. Keep them aligned when a capability changes. - Generated capability descriptions now include the visible ID prefix. Generated IDs, labels, navigation order, and verification fields remain the same. - Full local workspace checks and catalog generation remain unverified for the environment reasons above. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, model `gpt-6.1-sol`, with a 475,000-token context window and `xhigh` reasoning effort. The model used local file tools, shell commands, and GitHub CLI access. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-20 12:00:36 -05:00
Nicky LeachandPaperclip	5f5602bba7	feat(plugins): synchronize standing connection assignments (#16603) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Lifecycle plugins prepare agent runtimes outside the control plane. > - They need committed connection assignment changes without polling. > - A reader and its revision hooks must cover the same fields. > - This pull request adds a bounded standing-assignment snapshot and its matching hooks. > - Plugins can react to assignment changes while execution keeps its existing access checks. ## Linked Issues or Issue Description **What happened?** The agent runtime reader does not expose standing connection assignments. Changes to connection installs, active profiles, and connector feature switches do not request synchronization. Adding company-wide invalidation also exposes lock inversions in assignment writers and AI adoption. **Expected behavior** The owning lifecycle plugin can read a bounded assignment snapshot. Every committed change to its stored fields advances the runtime revision. Rollback removes both the change and its event. Assignment writers finish without a lock-order cycle. **Steps to reproduce** 1. Install a connection or bind an active profile to an agent. 2. Read its runtime inputs through the owning lifecycle plugin. 3. Rename or disable the connection, revoke the assignment, or switch connector features. 4. Expect a committed runtime event and an updated snapshot without periodic polling. **Additional context** Extracted from #16503. Catalog publication #16602 and the separate test maintenance PRs #16601 and #16610 are merged. This PR now targets master and is the active review. Tool-definition hooks #16604 and access/credential hooks #16605 remain drafts without review requests. ## What Changed - Add SDK types and an owning-plugin runtime snapshot for installed connections, active profile IDs, and effective connector feature switches. - Reject more than 512 connections or 128 active profiles instead of returning partial data. - Add migration 0354 for the exact stored fields used by the snapshot, with transaction-coalesced invalidation. Agent-targeted bindings and installs notify only their old/new target agents; project/routine/issue/gateway bindings are outside the snapshot and produce no notification. - Use ordered agent locks that are compatible with foreign-key references and exclude runtime readers. Prelock complete old/new target sets before replacing assignments or restoring managed profiles. - Record one company baseline at delivery startup for process-managed feature overlays. - Lock assignment inputs before install saves, task completion, and email assignment. Setup finalization reads the locked connection state and prelocks company agents before activating, re-enabling, or reviving a profile. - Make profile binding and reassignment writes transactional and lock parent profiles first. Keep target validation and profile/detail reads on the reserved transaction connection, including with a one-connection pool. - Lock company agents in order before validated AI adoption. Keep adoption and shared configuration writes at `FOR NO KEY UPDATE`, avoiding an upgrade that can deadlock foreign-key inserts during gateway creation. Configuration fields cannot change agent identity; lifecycle transitions and deletion retain their existing locks. - Document supported coverage and the inputs reserved for later PRs. ## Verification - Thirteen affected suites passed: 757 tests covering runtime hooks/assignments/skills, concurrent skill publication, tool access/catalog, task connection intents, lifecycle delivery, email, AI accounts, migration histories, snapshot drift, and the agent lifecycle module. This includes the merged catalog PR's OAuth and overlapping-refresh regressions. - The assignment suite passes all 24 cases, including bounded snapshots, company/agent isolation, rollback, health-field no-ops, process-start feature overlays, and real lock contention. - Four one-connection PostgreSQL regressions reproduce the earlier helper-read hangs before the correction; bind, unbind, delete, and reassign complete with it. The earlier adoption/gateway probe exposed both lock upgrades: changing only validation still deadlocks in shared configuration. Current scoped coverage verifies adoption with agent-targeted binding writes; gateway-only bindings no longer invalidate this snapshot. - The original ordered-adoption regression reproduces PostgreSQL `40P01` without ordered company-agent locking. Scoped-recipient tests reproduce unrelated-peer journaling before the trigger correction and verify repeated bind/unbind operations leave peers untouched, ignored binding types produce no events, both reassignment recipients are reached, and coalescing remains intact. - Six replacement contention cases cover installs, existing setup, activation, re-enabling, profile revival, and profile reassignment. Negative controls reproduce `40P01` without the corresponding target/company prelocks; all cases pass with the guards. Activation decisions use the connection row reread under lock. - An initial contention fixture failed because its settings connection lacked an MCP URL. Supplying the fixture URL and controlled lookup corrected that test setup error. - Server typecheck, database migration numbering/safety, five-snapshot retention, feature-map structural validation, and `git diff --check` passed. Migration metadata preserves historical journal entries and adds no schema change. The complete diff was reviewed for secrets and personal data before push. - Earlier hosted PR CI runs 38983440861 and 38985530069 passed; the superseded promotion run 38983440470 was cancelled. Superagent acknowledged the recipient fix and accepted the documented company-wide cost at `359b5b2f30a8a0248177367e458663d5379810d1`. Its two standard Greptile summaries were conditional on a later setup correction and were not counted as independent acceptance; the setup correction is now included here. - At `7da246aab46274db1a120a0546abbb404c5427c7`, hosted PR run 38986958988 passed: all 55 checks succeeded, with two optional Storybook checks skipped. Superagent passed, all review threads are resolved, and standard Greptile independently returned 5/5 with no blocking findings (review completed 2026-10-17 00:40 UTC). Apex independently returned 5/5 with no actionable findings (completed 2026-10-17 00:50 UTC). CI was rechecked after Apex and remains green; the branch is mergeable against master. Full local build and repository-wide tests were not run. ## Risks Migration 0354 adds database functions and triggers without changing table schema. Apply it through the normal migration flow. Agent-targeted bindings and installs only notify their affected agents. Explicit company-wide assignments, connection metadata, profile activation/removal, and connector feature switches retain synchronous, transaction-atomic company fanout. Each such transaction takes ordered row locks and inserts up to one connector runtime event per eligible company agent: O(number of eligible agents) work and journal storage, with no company-size cap or cross-transaction mutation quota. This remains an intentional scope tradeoff for this PR; deferred fanout is not implemented. Many of these conservative changes can request preparation even if an individual snapshot stays unchanged. Snapshot limits fail closed. This PR provides desired assignments, not credentials or task authorization. Later PRs must add mutation coverage before consumers can rely on immediate synchronization of tool definitions or credentials. ## Model Used OpenAI Codex, GPT-6, with reasoning, repository inspection, code execution, and testing. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-16 17:55:47 -07:00
Devin FoleyandPaperclip	0b62c736a0	feat(connections): add generic Daytona API and Toolbox access (#16483) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Apps connections give agents controlled access to external services. > - Daytona provides sandbox, file, command, Git, and account APIs. > - Operators need to assign this access without exposing a server credential. > - A small task-specific tool list cannot serve general Daytona work. > - This pull request adds the published HTTP actions through the existing connection gateway. > - Operators keep control of identity, agent access, and each action. ## Linked Issues or Issue Description **Problem or motivation** Paperclip has a Daytona sandbox provider for its own execution infrastructure. It lacks a normal Apps connection for agents to manage an operator's Daytona account. Reusing the server's provider credential would bypass normal connection ownership and action permissions. **Proposed solution** Add a Daytona connection with personal or shared vault credentials. Generate its action catalog from the official platform and Toolbox API specifications. Use the existing run-bound gateway, grants, action policies, approvals, revocation, and audit paths. Sandbox IDs come from Daytona; tools need no Paperclip task or lease ID. **Alternatives considered** A four-action diagnostic subset cannot support general provider work. Launching the provider CLI would expose host filesystem behavior. Native OAuth is not included because a supported third-party registration path was not verified. The native method uses an organization API key. Existing Composio setup is a separate possible OAuth route and is not qualified by this change. **Roadmap alignment** This extends the existing Apps and Connections provider coverage in `ROADMAP.md`. It uses that system's credential and permission model. GitHub searches found no existing Daytona Apps connector PR or issue. ## What Changed - Add the Daytona catalog entry, provider source override, official theme logos, permission review, and setup documentation. - Generate 274 HTTP actions: 169 platform and 105 Toolbox operations. Record two WebSocket upgrades separately; do not advertise them as HTTP tools. Record source URLs and hashes. - Add provider-focused execution for sandboxes, snapshots, volumes, files, Git, commands, computer use, and account resources. Provider authority still limits which endpoints accept an organization key. - Validate action arguments. Resolve Toolbox targets through the fixed platform API. Check sandbox identity and organization before using an approved Daytona proxy. Keep normal guarded network requests and vault resolution. - Transfer file bytes without reading or writing host files. Share one caller deadline across prerequisite reads, dispatch, and response reading. Bound requests and responses. Do not automatically retry writes with unknown outcomes. Redact the selected credential from results and errors. - Classify credential access, access changes, interactive input, command execution, file permissions, and deletion (including TTL and auto-delete) as destructive actions. Keep new and changed actions under catalog review. - Upgrade saved catalogs and retire the earlier diagnostic names. Hide retired actions from the UI and agent discovery even when provider refresh fails. Preserve existing action choices and historical records. Reject stale same-name contracts during agent discovery and dispatch before a board refresh. - Add database-backed gateway and catalog-upgrade regressions, schema generator tests, a browser setup check, and both capability map updates. ## Verification - Passed on the review-fix head: 57 focused Vitest cases (18 Daytona database/client cases, five HTTP boundary cases, and 34 shared catalog cases), four generator tests, `pnpm -r typecheck`, and `pnpm build`. The five HTTP cases use a real local server through the pinned production transport. The normal browser setup check and 17 generator/artwork cases passed on the initial PR head; those UI and artwork paths are unchanged. - Passed on this branch: brand asset validation, token gates, map structure checks, and `git diff --check`. All latest-head GitHub checks pass on `ea979ccb3a64dc9508807823540f935c4316ea35`. Apex is 5/5, with all five findings addressed and zero unresolved threads. The branch has no merge conflicts. - The initial catalog-generation command could not find the external research corpus. The supported `--definitions-only` mode regenerated the definitions successfully. Unrelated generated drift was excluded. - An earlier full local Vitest attempt on the preview branch failed when bundled PostgreSQL could not start on macOS. After replay onto current master, all three unchanged event-sequencing cases pass with bundled PostgreSQL; the earlier startup cause remains uncertain. - The first full-suite command on this branch used my external test database override. Chat fixtures require a current migrated schema and failed because `agents.lifecycle_state` was absent in that disposable database. A focused comparison reproduces failure with the override and passes with normal setup. That attempt was stopped, and the full suite restarted without the override. Focused Daytona cases use their own migrated PostgreSQL 16 databases. - The clean local full-suite attempt recorded embedded PostgreSQL startup failures in two unchanged native-runtime cases and two failures in unchanged Git fixtures. It was stopped after these failures were isolated; no full local suite pass is claimed. The 15 matching native-runtime cases pass separately. The Git cases fail with the local `init.defaultBranch=main` setting and pass with `GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=init.defaultBranch GIT_CONFIG_VALUE_0=master` limited to the test command. No global Git setting was changed. Full Linux CI is green. - Fixtures use real database persistence and run-bound gateway sessions with provider responses replaced by fixtures. They cover personal/shared vault ownership, reads and writes, file bytes, Off policies, revocation, organization mismatch, untrusted proxy rejection, output bounds, saved catalog upgrades, failed refresh, and permission preservation. - The first Apex review found five valid issues: multipart transport compatibility, destructive classification, caller deadlines, audit filters, and stale saved contracts. All five were fixed with regressions before the second review. A typecheck then found the nullable saved schema hash; admission now accepts that stored type and rejects null as stale. - The browser check covers setup controls without submitting a provider key. No real Daytona account, provider write, native OAuth flow, or agent model run is qualified. Live provider read/write proof remains pending a test account. ## Risks - Daytona keys have broad Toolbox authority, including keys with no selected resource scopes. Only assign this connection to trusted agents and review its action permissions. - Remote writes or commands can continue after a timeout. Calls do not automatically retry an uncertain outcome. - The bridge targets Daytona Cloud. Proxy hosts are restricted; an unrecognized future regional host fails closed. - Files and responses have an 8 MiB bound. Each action has one caller deadline, capped at 30 seconds across all provider requests and response reading. Interactive WebSocket sessions are not exposed as HTTP actions. - The generated JSON snapshot exceeds 1,000 lines because it holds the provider input contracts. Generator logic and execution code remain in separate small modules. Updates require explicit regeneration and catalog review. - No database migration or lockfile change is required. ## Model Used OpenAI Codex, GPT-6. The runtime identifies the model family but does not expose the exact backend model ID or context window. Used reasoning, repository inspection, code editing, shell tools, database verification, and browser tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-15 14:05:26 -07:00
DottaandPaperclip	7d9b93aee2	docs: establish paired feature and agent access maps (#16025) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Contributors use the feature map to understand and verify product capabilities. > - Agents need APIs and usable entry points within their permission boundaries. > - Product recipes and agent access need separate records with shared capability IDs. > - This pull request establishes that contract and gives maintenance ownership of both maps. > - The benefit is repeatable access review without changing product behavior or permissions. ## Linked Issues or Issue Description Refs: #15983, which added the portable verification skills and updated the feature map. This is its contract-only follow-up. Related: #15288 added the original feature map. **Issue type** Missing documentation and maintenance contract. **Where is the issue?** The product map, its missing agent access companions, and the verification skills. **What's wrong?** There is no shared convention for API presence, agent entry points, permission boundaries, access gaps, or evidence. An API declaration does not prove that a permitted native or remote agent can use it. **Suggested fix** Keep the product and agent access maps as siblings under `maps/`. Reuse capability IDs. Record representative source inventory separately from operation review and live proof. Keep both maps aligned through the maintenance skill. ## What Changed - Move the existing product map to `maps/feature-map/`. Preserve its recipes and Poteto format. Update relative links and add companion links. - Add `maps/agent-access-map/` with 35 area files and 171 matching capability IDs. Seed representative API operations, agent paths, policies and known gaps from the reviewed source inventory. Mark entries as `baseline`, `source-only`, and live verification `not-run`. - Define the shared contract in `maps/README.md`. Track API presence, agent availability, policy, evidence and review depth separately. Require operation enumeration before marking a broad capability operation-reviewed. - Extend `maintain-verification-skill` to reconcile both maps and inspect actual route policy and production tool admission. Update creation, verification, notices and contributor references to use the shared contract. - Add `scripts/check-maps.mjs` and ten regression tests. Check matching areas/IDs, indexes, references, contract fields, API method/path examples, justified exceptions and evidence-state consistency. - Keep the coverage inventory unchanged. Preserve skill invocation policies and both MIT license blocks. No product API, permission, UI, runtime helper, migration, dependency or workflow changes. ## Verification - Passed: `node scripts/check-maps.mjs` reports 35 paired areas and 171 matching capability IDs. All local map references resolve. - Passed: `node --test scripts/check-maps.test.mjs` runs ten tests. Fixtures cover missing companions, renamed/orphan/duplicate IDs, broken references, missing operation fields, empty exceptions, missing API examples, empty policy fields, and verified evidence without passed verification. - Passed: metadata validation for all three verification skills. Skill and contributor links resolve. Invocation policies and both license blocks are unchanged. - Compared relocated recipes with current master. Contents differ only in relative links and companion links. `coverage.json` is byte-for-byte unchanged. Master's updated read-only interaction test reference is retained. - Passed: `git diff --check`. The branch starts from current master and contains only this contract change. - Repository-wide typecheck, tests and build were not run locally for this documentation and Node structural-check scope. Live agent access was not exercised. Those gaps are explicit in the baseline entries; the structural check does not certify behavior. CI and automated review are pending. ## Risks - The companions are representative source baselines. They are not complete operation inventories or passed agent journeys. - The checker cannot establish actual permissions, native tool admission, operation completeness or truthful live evidence. Maintenance must inspect source and verify with the actual permitted agent context. - External links to the old product-map directory may need updates. Repository references have been updated. - Existing access gaps and board/user restrictions remain recorded. This contract does not change authority or implement missing paths. ## Model Used OpenAI GPT-6 through Codex assisted with this change. The session does not expose the exact serving model ID, configured context-window size, or reasoning setting. Capabilities used: source analysis, file edits, shell commands, and structural regression tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-12 14:52:28 -05:00